CN117065342A - Cloud game online fight method and device and electronic equipment - Google Patents

Cloud game online fight method and device and electronic equipment Download PDF

Info

Publication number
CN117065342A
CN117065342A CN202311154919.1A CN202311154919A CN117065342A CN 117065342 A CN117065342 A CN 117065342A CN 202311154919 A CN202311154919 A CN 202311154919A CN 117065342 A CN117065342 A CN 117065342A
Authority
CN
China
Prior art keywords
game
participants
cloud game
instruction frame
cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311154919.1A
Other languages
Chinese (zh)
Inventor
李文星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Migu Cultural Technology Co Ltd
China Mobile Communications Group Co Ltd
MIGU Interactive Entertainment Co Ltd
Original Assignee
Migu Cultural Technology Co Ltd
China Mobile Communications Group Co Ltd
MIGU Interactive Entertainment Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Migu Cultural Technology Co Ltd, China Mobile Communications Group Co Ltd, MIGU Interactive Entertainment Co Ltd filed Critical Migu Cultural Technology Co Ltd
Priority to CN202311154919.1A priority Critical patent/CN117065342A/en
Publication of CN117065342A publication Critical patent/CN117065342A/en
Pending legal-status Critical Current

Links

Classifications

    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50Controlling the output signals based on the game progress
    • A63F13/53Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
    • A63F13/533Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game for prompting the player, e.g. by displaying a game menu
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55Controlling game characters or game objects based on the game progress
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Optics & Photonics (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The present disclosure relates to a cloud game online fight method, device and electronic equipment, wherein the method comprises: acquiring positions corresponding to user terminals of all participants in online fight respectively; respectively distributing cloud game instances of the target game to the user terminals of all the participants based on the position information; acquiring an instruction frame message through an intermediate server; the instruction frame message comprises instruction frame messages sent by the user terminals of all the participants; the intermediate server determines the corresponding positions of the user terminals of all the participants; and carrying out game synchronous control on the cloud game instance of the target game based on the instruction frame message. According to the method, cloud game examples can be respectively distributed to all the participants, the distributed cloud game examples output game video streams, the problem of video stream delay is solved, the instruction frame information of all the participants is received through the intermediate server, and the cloud game examples are synchronously controlled based on the instruction frame information, so that the cloud game examples of all the participants can be synchronous, and the fairness of game fight is improved.

Description

Cloud game online fight method and device and electronic equipment
Technical Field
The disclosure relates to the field of cloud games, and in particular relates to an online fight method and device for cloud games and electronic equipment.
Background
Currently, cloud games have realized a network fight function of a stand-alone game. In the related art, by starting a cloud game instance in the cloud and connecting terminals of a plurality of users to the cloud game instance, online fight of the plurality of users can be achieved.
However, in the manner of online fight of multiple people provided in the related art, because the cloud game instance generally allocates an instance closer to the party initiating the game, if the distance between two users is longer, the delay of the other party or parties participating in the game is larger, so that the game experience of some users is worse, and the online fight of multiple people cannot be guaranteed to be performed fairly.
Disclosure of Invention
The present disclosure provides a cloud game online fight method, a cloud game online fight device, an electronic device and a storage medium.
According to a first aspect of the present disclosure, there is provided a cloud game online fight method, the method comprising: acquiring position information corresponding to user terminals of all participants in online fight respectively; respectively distributing cloud game instances of the target game to the user terminals of all the participants based on the position information; acquiring an instruction frame message through an intermediate server; the instruction frame message comprises instruction frame messages sent by user terminals of all the participants; the intermediate server determines based on the position information corresponding to the user terminals of the participants respectively; and carrying out game synchronous control on the cloud game instance of the target game based on the instruction frame message.
According to a second aspect of the present disclosure, there is provided a cloud game online fight device, the device comprising: the acquisition module is used for acquiring the position information corresponding to the user terminals of all the participants in the online fight respectively; the distribution module is used for respectively distributing cloud game instances of the target games to the user terminals of all the participants based on the position information; the acquisition module is also used for acquiring the instruction frame message through the intermediate server; the instruction frame message comprises instruction frame messages sent by user terminals of all the participants; the intermediate server determines based on the position information corresponding to the user terminals of the participants respectively; and the synchronization module is used for synchronously controlling the game of the cloud game instance of the target game based on the instruction frame message.
According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: a memory and a processor, the memory having stored thereon a computer program, the processor implementing the method as described above when executing the program.
According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the above-described method of the present disclosure.
According to the cloud game online fight method, device, electronic equipment and storage medium, through the acquisition of the position information respectively corresponding to the user terminals of all the online fight participants, cloud game instances of target games are respectively distributed to the user terminals of all the participants based on the position information; and acquiring the instruction frame message through the intermediate server, and performing game synchronous control on the cloud game instance of the target game based on the instruction frame message. The cloud game instances can be respectively distributed to all the participants, the game video streams of all the participants are output by the distributed cloud game instances, the problem of video stream delay is solved, meanwhile, the instruction frame message of all the participants is received through the intermediate server, and the cloud game instances of the target game are synchronously controlled based on the instruction frame message, so that the cloud game instances of all the participants can be synchronous, and the fairness of the fight game is improved.
Drawings
Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments, with reference to the following drawings, wherein:
FIG. 1 is a system architecture diagram of cloud gaming online combat provided in an exemplary embodiment of the present disclosure;
FIG. 2 is a clock synchronization diagram of online engagement of cloud games provided in an exemplary embodiment of the present disclosure;
FIG. 3 is a diagram of an instruction frame synchronization for online engagement of a cloud game provided in an exemplary embodiment of the present disclosure;
FIG. 4 is a flow chart of a method for online engagement of cloud games provided in an exemplary embodiment of the present disclosure;
FIG. 5 is a schematic block diagram of functional modules of a cloud gaming online combat device provided in an exemplary embodiment of the present disclosure;
FIG. 6 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure;
fig. 7 is a block diagram of a computer system according to an exemplary embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below. It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.
For example, in response to receiving an active request from a user, a prompt is sent to the user to explicitly prompt the user that the operation it is requesting to perform will require personal information to be obtained and used with the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium for executing the operation of the technical scheme of the present disclosure according to the prompt information.
As an alternative but non-limiting implementation, in response to receiving an active request from a user, the manner in which the prompt information is sent to the user may be, for example, a popup, in which the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window. It will be appreciated that the above-described notification and user authorization process is merely illustrative and not limiting of the implementations of the present disclosure, and that other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.
Currently, cloud games have realized a network fight function of a stand-alone game. In the related art, by starting a cloud game instance in the cloud and connecting terminals of a plurality of users to the cloud game instance, online fight of the plurality of users can be achieved.
However, in the manner of online fight of multiple people provided in the related art, because the cloud game instance generally allocates an instance closer to the party initiating the game, if the distance between two users is longer, the delay of the other party or parties participating in the game is larger, so that the game experience of some users is worse, and the online fight of multiple people cannot be guaranteed to be performed fairly.
Based on this, the embodiment of the disclosure provides a cloud game multi-instance combat method, as shown in fig. 1, firstly, user matching is completed at a user matching server, game instances nearby to two combat parties are respectively pulled up after the matching is completed, a user client and the game instances establish point-to-point audio and video stream connection, video stream quality and time delay are guaranteed, meanwhile, two cloud instances need to be subjected to progress calibration, when the progress of the two calibrated cloud instances is consistent, an instruction transfer server starts to receive a client instruction, and the received instruction is simultaneously forwarded to the two instances, so that the two instances are guaranteed to receive the same instruction at the same game progress, game performance consistency in the two user instances is guaranteed, and the aim of networked combat of users in different places is achieved.
The method provided by the present disclosure is described in detail in the following steps:
step 1, user matching.
The user selects an online fight game mode at a cloud game client to start the cloud game, the client establishes long connection with a matching server, the incoming user and game information start user matching, and after the matching user succeeds, the client of the two successful sides of the matching is notified to start the cloud game. Meanwhile, the cloud game instance is started, so that the cloud game progress of both parties can be kept consistent as much as possible.
And 2, starting an example.
The two online users are respectively scheduled to respective optimal cloud game instances according to own scheduling strategies, and start according to a cloud game normal mode. In the process of scheduling the cloud game instance, the distance between the cloud server and the client is nearest, and the performance of the cloud game instance is optimal.
And step 3, performing progress calibration.
Because of the influence of various factors, different cloud game instance starting time periods may have differences, in order to ensure that the picture progress of two cloud game instance and the game performance of both parties are consistent after an operation instruction of any party is received, before the user operation is allowed, synchronous calibration of the progress of the cloud game instance needs to be performed once after the game is started.
Step 3.1: the waiting picture with the identification of each game is pre-stored to the intermediate server as a progress calibration page.
After the FTG (fighting) game is started successfully, a page waits for a user operation (the game progress is irrelevant to the user operation before the picture appears, the picture is a waiting picture, after the operation is received, the game progress is relevant to the user operation), and the page can be stored as a waiting picture with identification.
In one possible implementation, when the game reference frame is identified, the image comparison scheme can be modified to intelligent identification of characters on the pattern, for example, the identification picture has fields such as "continue to press any key" and "continuous".
Step 3.2: when the cloud game is started, the picture is requested to the intermediate server and is compared with the game picture acquired by the picture acquisition service on the cloud game instance, and the intermediate server does not receive the operation request of the client before the comparison is successful.
Step 3.3: and when the picture comparison similarity is higher than a preset threshold (such as 98%), the cloud server sends a calibration success message notification to the intermediate service.
And 3.4, after receiving the notification of the successful calibration message sent by the cloud servers of the two parties, the intermediate server sends a synchronous notification to the cloud servers of the two parties and the client.
And 4, performing clock calibration.
Step 4.1: as shown in fig. 2, the client sends a request packet to the server, which contains the timestamp T1 of the packet leaving the client.
Step 4.2: when the server receives the packet, the time stamp T2 of arrival of the packet, the time stamp T3 of departure of the packet are sequentially filled, and then the packet is immediately returned to the client.
Step 4.3: when the client receives the response packet, the timestamp T4 returned by the packet is recorded.
Step 4.4: the client calculates the client and server clock bias t.
Assuming equal request and reply packet transfer delays (error in the order of milliseconds is negligible), i.e. d 1 =d 2 Then can be solved to
Step 4.5: the client adjusts the local clock with clock bias t so that its time coincides with the server time.
And 5, performing image encoding and decoding.
After the cloud image acquisition service acquires images, the time stamp of the calibrated reference clock in the step 4 is encoded together with the image information to identify the acquisition time of each frame of images, and the time stamp is decoded when the client decodes and then is used for identifying the acquisition time of the images;
and 6, carrying out instruction forwarding and synchronization.
Step 6.1, after clock synchronization is completed, taking the progress calibration message received by the client in step 3 as a starting time point, sending a message in a command frame mode through a long connection between the client and the transit server, and sending a command frame message to the intermediate server at a rate of 30 frames per second or 60 frames per second, wherein the message is exemplified as follows:
{clientTimestamp:1668767756029,
imagTimestamp:1668767756000,
ctrl action [ instruction A1] }
clientTimestamp after calibration
imagTimestamp-client decodes the timestamp information carried in the rendered last frame of image
ctrl action-all the manipulation instruction information of the user before the current frame and the previous frame.
And 6.2, when the intermediate server receives command frames of the two parties of the fight, carrying out frame synchronization according to the client time stamps of the two parties, if the client time stamps of the received frames of the two parties differ by less than 1000/frame number (ms), copying the commands of the two operation frames and then forwarding the copied commands to the cloud servers of the two parties, otherwise, indicating that one party of the frame information has delay, and waiting for the arrival of the command frame at one end of the slower network. And as shown in the middle part of fig. 3, the instruction frames of the two parties are aligned and combined, and then the instruction is copied and forwarded to the cloud game instance of the two parties. Meanwhile, the transfer server end is required to store all instruction frame information in a lasting manner in order to realize functions such as cloud instance crash restarting or game picture replay and the like.
And 7, synchronizing the client side pictures.
When the two game frames are not synchronized, as in step 6, the command frame message will carry the timestamp of the current image frame of the client, if the interval between the image timestamps carried in the command frames after the two game frames are synchronized exceeds the preset threshold (for example, 125 ms), it is indicated that the image delay of the client at one end is larger, the control command received at the time point when the frames are not synchronized is discarded, the transmission is not performed, the command transmission is performed after the frames are synchronized, and the fairness of the game is ensured. If the long-time picture is asynchronous, if the asynchronous time length exceeds 1S, the user can be prompted to wait for network abnormality of the opposite side.
And 8, synchronizing the cloud game.
Due to the performance difference of the cloud server, the possibility of blocking in the game running process and the like, the problem that cloud game pictures are asynchronous can also occur. And synchronizing the game picture frames acquired by the cloud to an intermediate server, comparing front and rear pictures one by one after synchronizing the intermediate server according to the image time stamps, and if the comparison similarity of N pictures is smaller than a preset threshold (such as 98%), considering that the cloud game is not synchronized, suspending command forwarding at the moment, and continuously receiving command input after synchronizing the cloud pictures.
And 9, reconnecting the dropped client.
If the intermediate server continuously does not receive 30 frames (1 s) of instruction frame information, the client is prompted, the other party is disconnected, waiting is performed, and meanwhile instruction receiving operation is forbidden.
And 10, restarting the cloud game crash.
In step 6, all instruction frame information is stored to an intermediate server in a lasting mode, if a cloud instance is restarted after running, after the restarting instance finishes the progress calibration of step 3, the maintained instruction is played back quickly according to 10 times of the original frame interval time, so that the game picture of the restarting instance is synchronous with the game of the opponent, and after the instruction playback is finished, the two-party instruction is started and accepted.
Based on the above embodiments, the present disclosure further provides a method for online fight of a cloud game, as shown in fig. 4, including the following steps:
in step S100, position information corresponding to each user terminal of each party in online fight is acquired.
Wherein, each party of online fight comprises at least two parties. And obtaining the geographic position information of the user terminal under the condition that the user knows and agrees. Alternatively, the location information of the user terminal may be obtained through a beidou satellite system, a global positioning system, or the like, which is not limited in the embodiments of the present disclosure.
In step S200, cloud game instances of the target game are respectively allocated to the user terminals of the participants based on the location information.
The target game is a game started by the user terminals of all the participants. The cloud game instance is an instance of a cloud operation target game.
In step S300, the instruction frame message is acquired through the intermediate server.
The instruction frame message comprises instruction frame messages sent by user terminals of all the participants. The intermediate server determines based on the location information respectively corresponding to the user terminals of the participants
In one possible implementation, the instruction frame message includes all of the manipulation instruction information of the user before the current frame and the previous frame by the user terminal.
In step S400, game synchronization control is performed on the cloud game instance of the target game based on the instruction frame message.
In one possible implementation manner, the intermediate server synchronously transmits the instruction frame message transmitted by the user terminal of each participant to the cloud game instance of each participant, so as to realize synchronous control of the cloud game instance of the target game.
According to the method, cloud game instances can be respectively distributed to all the participants, the game video streams of all the participants are output by the distributed cloud game instances, the problem of video stream delay is solved, meanwhile, the instruction frame message of all the participants is received through the intermediate server, and the cloud game instances of the target game are synchronously controlled based on the instruction frame message, so that the cloud game instances of all the participants can be synchronous, and the fairness of the fight game is improved.
In order to more clearly describe the present disclosure, based on the above embodiment, in step S200, further includes:
in step S210, game instances are respectively allocated to the respective participants respectively corresponding to the user terminals.
In one possible implementation, cloud game instances are allocated to user terminals of each participant nearby, for example, a first cloud game instance closest to the first user terminal is allocated to the first user terminal, and a second cloud game instance closest to the second user terminal is allocated to the second user terminal.
The distance between the server of each cloud game instance and the user terminal of the corresponding participant is smaller than a first preset distance.
By the method, game examples can be distributed nearby for all the participants, and game video streams of all the participants are output by the nearest cloud game examples, so that the problem of video stream delay caused by the fact that a plurality of users are connected with the same game example is solved, and the experience of the users is improved.
In order to more clearly describe the present disclosure, based on the above embodiment, in step S300, further includes:
in step S310, the intermediate server is determined based on the location information corresponding to the user terminals of the respective participants.
In one possible implementation manner, in order to ensure that instructions of two parties of the fight reach the transfer server as simultaneously as possible, the transfer server can be distributed and deployed in machine rooms in all places of the country, and after the matching of the users is completed, the instructions are dispatched to the machine room close to the middle of the geographic positions of the parties of the users. The distance between the intermediate server and the user terminals of all the participants is smaller than a second preset distance.
In step S320, the instruction frame messages respectively sent by the user terminals of the respective participants are acquired through the intermediate server.
The first instruction frame information sent by the first user terminal and the second instruction frame information sent by the second user terminal are obtained through an intermediate server.
To more clearly describe the present disclosure, the above-mentioned party includes a first user terminal and a second user terminal, and the instruction frame message carries a timestamp of a frame corresponding to an instruction of the user terminal, and in step S400, the method includes:
in step S410, a first timestamp corresponding to the first user terminal and a second timestamp corresponding to the second user terminal are obtained based on the instruction frame message, respectively.
The first timestamp comprises a timestamp of a picture frame corresponding to the first user terminal operation instruction, and the second timestamp comprises a timestamp of a picture frame corresponding to the first user terminal operation instruction.
In step S420, if the time interval between the first timestamp and the second timestamp is smaller than the preset time interval, the intermediate server sends the first instruction frame message and the second instruction frame message to the first cloud game instance corresponding to the first user terminal, and simultaneously sends the first instruction frame message and the second instruction frame message to the second cloud game instance corresponding to the second user terminal.
In one possible implementation manner, before receiving or sending the instruction frame message, the intermediate server performs picture synchronization and clock synchronization on the first cloud game instance, the second cloud game instance, the first user terminal and the second user terminal, after synchronization, the cloud server starts to encode a time stamp into a video frame according to a clock after synchronization, the transfer server starts to receive an operation instruction from the user terminal and forwards the operation instruction, and the cloud game instance receives the same operation instruction at the same time, so that the game progress of both parties is ensured to be consistent.
The following describes the manner of picture synchronization and clock synchronization.
In step S500, in the process of waiting for the game, a current first game frame of the first cloud game instance and a current second game frame of the second cloud game instance are obtained.
In step S510, in the case that the similarity between the first game frame and the second game frame and the pre-stored game waiting frame is greater than the preset threshold, notification information of game start is sent to the first user terminal, the second user terminal, the first cloud game instance and the second cloud game instance through the intermediate server.
Typically, after the FTG game is started successfully, there is a page waiting for the user to operate (the game progress is irrelevant to the user operation before the picture appears, the picture is a waiting picture, and after the operation is received, the game progress is relevant to the user operation), and in one possible implementation, the page may be stored as an identifiable waiting picture.
To more clearly describe the present disclosure, based on the above embodiment, before sending, by the intermediate server, notification information of game start to the first user terminal, the second user terminal, the first cloud game instance, and the second cloud game instance at the same time, the method further includes:
in step S600, clock information of the first user terminal, the second user terminal, the first cloud game instance, and the second cloud game instance is obtained respectively.
In one possible implementation, the clock information includes timestamp information. Illustratively, the first user terminal sends a request packet to the server of the first cloud game instance, where the request packet includes a timestamp T1 of the departure of the request packet from the first user terminal. When the server of the first cloud game instance receives the packet, sequentially filling the arrival time stamp T2 of the packet and the departure time stamp T3 of the packet, immediately returning the packet to the first user terminal, and recording the time stamp T4 of the packet return.
In step S610, clock synchronization is performed between the first user terminal, the second user terminal, the first cloud game instance, and the second cloud game instance based on the clock information.
The clock synchronization between the user terminal and the cloud game instance may refer to the related description of fig. 2 in the embodiments of the present disclosure, which is not repeated here.
In one possible implementation manner, clock synchronization may be performed on each user terminal first, and then clock synchronization between the user terminal and the cloud game instance may be implemented in the above manner. Alternatively, the clock deviation between the user terminals may be determined by simultaneously acquiring the clocks of the user terminals, and notifying the client terminal to adjust the clocks based on the clock deviation.
By the method, the clock calibration is performed before the intermediate server sends the successful calibration notice to the cloud game instance and the user terminal, so that clock synchronization of four ends is ensured, and clock basis is provided for subsequent picture and instruction synchronization.
To more clearly illustrate the disclosure, based on the above embodiments, the method further includes:
in step S700, a time stamp of a current image frame of a user terminal of each participant in an online fight is acquired during the game fight.
In step S710, in case that the time stamp interval of the current image frame of the user terminal where any two participants exist is greater than the preset interval, the intermediate server stops receiving and/or transmitting the instruction frame message.
In one possible implementation, as shown in fig. 5, the user a network situation is better, there is normal network delay, but the frame delay and the frame dropping situation do not occur in the picture, and the user B network has fluctuations, resulting in the frame delay and the frame dropping situation. When the user A sees the 1 st frame of picture, the B user is blocked, the 0 th frame of picture is still displayed, and at the moment, the user A performs A2 operation to be applied to the 1 st frame of picture, but at the moment, the user B does not see the 1 st frame of picture, so that the B cannot react, and the B user is unfair.
When two game pictures are not synchronized, the command frame message carries the time stamp of the current image frame of the client, if the time stamp interval of the image carried in the command frame after the synchronization of the two parties is too large, when the time interval exceeds 125ms, the large blocking occurs once, the image delay of the user side is larger at one end, as shown in fig. 5, the pictures of the two parties at the time point 2 and 3 are not synchronized until the time point 4, the picture is re-synchronized, the command A2 is applied to the 1 st frame picture at the time point 2, at this time, the control command received at the time points 2 and 3 of the picture non-synchronization can be discarded (namely, the command A2) and not forwarded, and the command is forwarded after the picture synchronization, so that the fairness of the game is ensured. If the long-time picture is asynchronous, for example, the asynchronous time length exceeds 1S, the user can be prompted to wait for network abnormality of the opposite side.
To more clearly illustrate the disclosure, based on the above embodiments, the method further includes:
in step S720, in the case where the intermediate server does not receive the instruction frame message transmitted by the user terminal of any one of the parties for a preset time, the reception and/or transmission of the instruction frame message is stopped.
For example, if the intermediate server does not receive the instruction frame message of the first user terminal for 1s (30 frames) continuously, the second user terminal is prompted to drop the line of the other party, please wait, and at the same time, the instruction accepting and/or transmitting operation is prohibited.
To more clearly illustrate the disclosure, based on the above embodiments, the method further includes:
in step S800, the acquired instruction frame message is stored to the intermediate server.
In step S810, if any one of the participants restarts the cloud game instance, the stored instruction frame message is played back at a preset speed on the restarted cloud game instance.
In one possible implementation manner, all instruction frame messages are stored in a persistence mode to an intermediate server, if a cloud instance is restarted after running, after the restart instance finishes progress calibration, the stored instructions are played back quickly according to the preset multiple speed of the original frame interval time, for example, 10 times of the speed, so that the game picture of the restart instance is synchronous with the games of other participants, and after the instruction playback is finished, the instructions of all the participants are started and accepted.
To more clearly illustrate the disclosure, based on the above embodiments, the method further includes:
in step S900, a current image of a cloud game instance corresponding to each participant is obtained.
In one possible implementation, images of the I-frame cloud game instance may be periodically transmitted to an intermediate server during the game.
In step S910, if the similarity between the images of the cloud game instances corresponding to any two participants is less than the preset threshold, the intermediate server stops receiving and/or sending the instruction frame message.
In one possible implementation manner, after the synchronization of the image time stamps, the intermediate server performs the comparison of the front and rear frames one by one, for example, performs the image comparison of the front and rear frames 5 frames, if the comparison similarity of 10 frames is smaller than 98%, the cloud game is considered to be asynchronous, the instruction forwarding can be suspended at this time, and since the cloud game behavior is only related to the operation, after stopping the control, the final game frames will be kept consistent, and after the cloud frames are synchronized, that is, the similarity of the cloud frames collected by the similar time stamps is greater than 98%, the instruction input is continuously accepted.
By the method, in the game process, when the game picture of the user terminal is not synchronous, the picture of the cloud game instance is not synchronous or the user is disconnected and restarted, the intermediate server does not receive and/or send the instruction frame message, the problem of the instruction non-synchronization among the participants is avoided, the fairness of game fight is improved, and the experience of the user is further improved.
According to the method provided by the embodiment of the disclosure, cloud game examples are respectively distributed to the user terminals of all the participants, picture synchronization and clock synchronization of all the participants and all the cloud game examples are carried out before the game starts, a basis is provided for subsequent instruction synchronization, in the game process, instruction frame information of all the user terminals of all the participants is received and forwarded through the intermediate server, so that the instruction frame information can reach the cloud game examples corresponding to all the user terminals at the same time, and in the case that pictures of the user terminals or the cloud game examples are not synchronous, the receiving and/or sending of the instruction frame information is stopped, the fairness of game fight is improved, and the experience of users is further improved.
In the case of dividing each functional module by adopting corresponding each function, the embodiment of the disclosure provides an online cloud game fight device, which can be a server or a chip applied to the server. Fig. 5 is a schematic block diagram of functional modules of a cloud game online fight device according to an exemplary embodiment of the present disclosure. As shown in fig. 5, the apparatus includes:
an obtaining module 201, configured to obtain location information corresponding to user terminals of all parties in online fight;
the allocation module 202 is configured to allocate cloud game instances of the target game to the user terminals of the participants respectively based on the location information; the acquisition module is also used for acquiring the instruction frame message through the intermediate server; the instruction frame message comprises instruction frame messages sent by user terminals of all the participants; the intermediate server determines based on the position information corresponding to the user terminals of the participants respectively;
and the synchronization module 203 is configured to perform game synchronization control on the cloud game instance of the target game based on the instruction frame message.
In one possible implementation manner, the allocation module is further configured to allocate game instances to the respective user terminals respectively corresponding to the participants; the distance between the server of each game instance and the user terminal of the corresponding party is smaller than a first preset distance.
In one possible implementation manner, the apparatus further includes: the determining module is used for determining the intermediate server based on the position information corresponding to the user terminals of all the participants respectively; the distance between the intermediate server and the user terminals of all the participants is smaller than a second preset distance; the acquisition module is also used for acquiring instruction frame messages respectively sent by the user terminals of all the participants through the intermediate server.
In one possible implementation manner, the participant includes a first user terminal and a second user terminal, and the instruction frame message carries a timestamp of a picture frame corresponding to an instruction of the user terminal; the acquisition module is further used for respectively acquiring a first timestamp corresponding to the first user terminal and a second timestamp corresponding to the second user terminal based on the instruction frame message;
the synchronization module is further configured to send, through the intermediate server, a first instruction frame message and a second instruction frame message to a first cloud game instance corresponding to the first user terminal, and send, at the same time, the first instruction frame message and the second instruction frame message to a second cloud game instance corresponding to the second user terminal, if a time interval between the first timestamp and the second timestamp is less than a preset time interval.
In one possible implementation manner, the obtaining module is further configured to obtain, during a game waiting process, a current first game frame of the first cloud game instance and a current second game frame of the second cloud game instance;
the synchronization module is further configured to send notification information of game start to the first user terminal, the second user terminal, the first cloud game instance, and the second cloud game instance through the intermediate server when the similarity between the first game frame and the second game frame and the pre-stored game waiting frame is greater than a preset threshold.
In one possible implementation manner, the obtaining module is further configured to obtain clock information of the first user terminal, the second user terminal, the first cloud game instance, and the second cloud game instance, respectively;
the synchronization module is further used for performing clock synchronization among the first user terminal, the second user terminal, the first cloud game instance and the second cloud game instance based on the clock information.
In one possible implementation, the obtaining module is further configured to obtain, during the game play, a timestamp of a current image frame of a user terminal of each participant in the online play,
the synchronization module is further used for stopping receiving and/or sending the instruction frame message by the intermediate server under the condition that the time stamp interval of the current image frames of the user terminals of any two participants is larger than the preset interval.
In one possible implementation, the synchronization module is further configured to: and stopping receiving and/or sending the instruction frame message under the condition that the intermediate server does not receive the instruction frame message sent by the user terminal of any one of the participants for a preset time.
In one possible implementation manner, the apparatus further includes: the storage module is used for storing the acquired instruction frame information to the intermediate server;
the synchronization module is further configured to, when any one of the participants restarts the cloud game instance, play back the stored instruction frame message on the restarted cloud game instance at a preset speed.
In one possible implementation manner, the obtaining module is further configured to obtain current images of cloud game instances corresponding to the participants respectively;
the synchronization module is further configured to stop receiving and/or sending the instruction frame message by the intermediate server when the similarity between the images of the cloud game instances corresponding to any two participants is less than a preset threshold.
The embodiment of the disclosure also provides an electronic device, including: at least one processor; a memory for storing the at least one processor-executable instruction; wherein the at least one processor is configured to execute the instructions to implement the above-described methods disclosed by embodiments of the present disclosure.
Fig. 6 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. As shown in fig. 6, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801, the processor 1801 may perform corresponding steps in the above-described methods disclosed by embodiments of the present disclosure.
The processor 1801 may also be referred to as a central processing unit (central processing unit, CPU), which may be an integrated circuit chip with signal processing capabilities. The steps of the above-described methods disclosed in the embodiments of the present disclosure may be accomplished by instructions in the form of integrated logic circuits or software in hardware in the processor 1801. The processor 1801 may be a general purpose processor, a digital signal processor (digital signal processing, DSP), an ASIC, an off-the-shelf programmable gate array (field-programmable gate array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present disclosure may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may reside in a memory 1802 such as random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as is well known in the art. The processor 1801 reads the information in the memory 1802 and, in combination with its hardware, performs the steps of the method described above.
In addition, various operations/processes according to the present disclosure, in the case of being implemented by software and/or firmware, may be installed from a storage medium or network to a computer system having a dedicated hardware structure, such as the computer system 1900 shown in fig. 7, which is capable of performing various functions including functions such as those described above, and the like, when various programs are installed. Fig. 7 is a block diagram of a computer system according to an exemplary embodiment of the present disclosure.
Computer system 1900 is intended to represent various forms of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the computer system 1900 includes a computing unit 1901, and the computing unit 1901 can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a Random Access Memory (RAM) 1903. In the RAM1903, various programs and data required for the operation of the computer system 1900 may also be stored. The computing unit 1901, ROM1902, and RAM1903 are connected to each other via a bus 1904. An input/output (I/O) interface 1905 is also connected to bus 1904.
Various components in computer system 1900 are connected to I/O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 may be any type of device capable of inputting information to the computer system 1900, and the input unit 1906 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device. The output unit 1907 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 1908 may include, but is not limited to, magnetic disks, optical disks. The communication unit 1909 allows the computer system 1900 to exchange information/data with other devices over a network, such as the internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.
The computing unit 1901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the above-described methods disclosed by embodiments of the present disclosure may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1908. In some embodiments, some or all of the computer programs may be loaded and/or installed onto electronic device 1900 via ROM1902 and/or communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above-described methods of the disclosed embodiments by any other suitable means (e.g., by means of firmware).
The disclosed embodiments also provide a computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the above-described method disclosed by the disclosed embodiments.
A computer readable storage medium in embodiments of the present disclosure may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium described above can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specifically, the computer-readable storage medium described above may include one or more wire-based electrical connections, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The disclosed embodiments also provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the above-described methods of the disclosed embodiments.
In an embodiment of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computers may be connected to the user computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computers.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules, components or units referred to in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a module, component or unit does not in some cases constitute a limitation of the module, component or unit itself.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
The above description is merely illustrative of some embodiments of the present disclosure and of the principles of the technology applied. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It will be appreciated by those skilled in the art that modifications may be made to the above embodiments without departing from the scope and spirit of the disclosure. The scope of the present disclosure is defined by the appended claims.

Claims (13)

1. A method of online combat of a cloud game, the method comprising:
acquiring position information corresponding to user terminals of all participants in online fight respectively;
respectively distributing cloud game instances of a target game to the user terminals of all the participants based on the position information;
acquiring an instruction frame message through an intermediate server; the instruction frame message comprises instruction frame messages sent by user terminals of all the participants; the intermediate server determines based on the position information respectively corresponding to the user terminals of the participants;
and performing game synchronous control on the cloud game instance of the target game based on the instruction frame message.
2. The method according to claim 1, wherein the assigning cloud game instances of the target game to the user terminals of the respective participants based on the location information, respectively, comprises:
respectively distributing game examples to the user terminals corresponding to the participants; the distance between the server of each game instance and the user terminal of the corresponding party is smaller than a first preset distance.
3. The method of claim 1, wherein the obtaining, by the intermediate server, the instruction frame message comprises:
Determining an intermediate server based on the position information corresponding to the user terminals of the participants respectively; the distance between the intermediate server and the user terminals of the participants is smaller than a second preset distance;
and acquiring the instruction frame messages respectively sent by the user terminals of all the participants through the intermediate server.
4. The method of claim 1, wherein the participant comprises a first user terminal and a second user terminal, and wherein the instruction frame message carries a timestamp of a picture frame corresponding to an instruction of the user terminal; the game synchronization control for the cloud game instance of the target game based on the instruction frame message comprises the following steps:
based on the instruction frame message, respectively acquiring a first time stamp corresponding to the first user terminal and a second time stamp corresponding to the second user terminal;
and under the condition that the time interval between the first time stamp and the second time stamp is smaller than a preset time interval, sending the first instruction frame message and the second instruction frame message to a first cloud game instance corresponding to the first user terminal through the intermediate server, and simultaneously sending the first instruction frame message and the second instruction frame message to a second cloud game instance corresponding to the second user terminal.
5. The method according to claim 4, wherein the method further comprises:
in the game waiting process, acquiring a current first game picture of a first cloud game instance and a current second game picture of a second cloud game instance;
and under the condition that the similarity between the first game picture and the second game picture and the pre-stored game waiting picture is larger than a preset threshold value, sending notification information of game start to the first user terminal, the second user terminal, the first cloud game instance and the second cloud game instance through the intermediate server.
6. The method of claim 5, wherein the method further comprises:
respectively acquiring clock information of the first user terminal, the second user terminal, the first cloud game instance and the second cloud game instance;
and carrying out clock synchronization among the first user terminal, the second user terminal, the first cloud game instance and the second cloud game instance based on the clock information.
7. The method according to claim 1, wherein the method further comprises:
In the game fight process, the time stamp of the current image frame of the user terminal of each participant in the online fight is obtained,
and stopping receiving and/or sending the instruction frame message by the intermediate server under the condition that the time stamp interval of the current image frames of the user terminals with any two participants is larger than the preset interval.
8. The method according to claim 1, wherein the method further comprises:
and stopping receiving and/or sending the instruction frame message under the condition that the intermediate server does not receive the instruction frame message sent by the user terminal of any one of the participants for a preset time.
9. The method according to claim 1, wherein the method further comprises:
storing the acquired instruction frame information to an intermediate server;
and under the condition that any one of the participants restarts the cloud game instance, playing back the stored instruction frame message on the restarted cloud game instance according to a preset speed.
10. The method according to claim 1, wherein the method further comprises:
acquiring current images of cloud game instances corresponding to the participants respectively;
and stopping receiving and/or sending the instruction frame message by the intermediate server under the condition that the similarity between the images of the cloud game instances corresponding to any two participants is smaller than a preset threshold value.
11. An online combat device for cloud games, the device comprising:
the acquisition module is used for acquiring the position information corresponding to the user terminals of all the participants in the online fight respectively;
the distribution module is used for respectively distributing cloud game instances of the target games to the user terminals of all the participants based on the position information;
the acquisition module is also used for acquiring the instruction frame message through the intermediate server; the instruction frame message comprises instruction frame messages sent by user terminals of all the participants; the intermediate server determines based on the position information respectively corresponding to the user terminals of the participants;
and the synchronization module is used for performing game synchronization control on the cloud game instance of the target game based on the instruction frame message.
12. An electronic device, comprising:
at least one processor;
a memory for storing the at least one processor-executable instruction;
wherein the at least one processor is configured to execute the instructions to implement the memory failure handling method of any of claims 1-10.
13. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the memory failure handling method according to any of claims 1-10.
CN202311154919.1A 2023-09-07 2023-09-07 Cloud game online fight method and device and electronic equipment Pending CN117065342A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311154919.1A CN117065342A (en) 2023-09-07 2023-09-07 Cloud game online fight method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311154919.1A CN117065342A (en) 2023-09-07 2023-09-07 Cloud game online fight method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN117065342A true CN117065342A (en) 2023-11-17

Family

ID=88708006

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311154919.1A Pending CN117065342A (en) 2023-09-07 2023-09-07 Cloud game online fight method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN117065342A (en)

Similar Documents

Publication Publication Date Title
US9282232B2 (en) Collaborative image control
CN111837399B (en) Playback method, server and medium for synchronizing media content
JP6469279B1 (en) Content distribution server, content distribution system, content distribution method and program
US9485467B2 (en) Information processing system and information processing devices
CN113457123B (en) Interaction method and device based on cloud game, electronic equipment and readable storage medium
CN113423018A (en) Game data processing method, device and storage medium
CN107484011B (en) Video resource decoding method and device
JP2008152625A (en) Image display device, image data processing system and image data processing method
CN111819858B (en) Methods, systems, and media for synchronized media content playback on multiple devices
CN112057846B (en) Interaction method, device, equipment and storage medium for cloud game service scheduling
US11523146B2 (en) Live broadcast method and apparatus, electronic device, and storage medium
WO2022017007A1 (en) Audio data processing method, server, and storage medium
CN112702557A (en) Screen sharing method, device, equipment and storage medium based on call
US9549006B2 (en) Self-adaptive sample period for content sharing in communication sessions
KR20160021449A (en) Control method, device and system for video playing
US20220021715A1 (en) Live streaming method and apparatus, device and computer readable storage medium
CN117065342A (en) Cloud game online fight method and device and electronic equipment
KR101647541B1 (en) Method, apparatus and system for providing interaction to game play
EP3934257A1 (en) Livestreaming method, apparatus and device, and computer-readable storage medium
CN114144758B (en) Display method, display control method, display terminal, server and display system
CN113726534A (en) Conference control method, conference control device, electronic equipment and storage medium
CN108667795B (en) Virtual reality video multi-person sharing system and virtual reality equipment
US20230379529A1 (en) Distributed media stream playback suspension and synchronization
KR20200080041A (en) Method and apparatus for generating multi channel images using mobile terminal
CN113747237B (en) Data processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination