CN105721885B - Method, device and system for processing transcoding data - Google Patents
Method, device and system for processing transcoding data Download PDFInfo
- Publication number
- CN105721885B CN105721885B CN201610178154.9A CN201610178154A CN105721885B CN 105721885 B CN105721885 B CN 105721885B CN 201610178154 A CN201610178154 A CN 201610178154A CN 105721885 B CN105721885 B CN 105721885B
- Authority
- CN
- China
- Prior art keywords
- transcoding
- server
- video data
- audio
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 142
- 238000012545 processing Methods 0.000 title claims abstract description 22
- 230000008569 process Effects 0.000 claims description 96
- 230000004044 response Effects 0.000 claims description 30
- 238000010586 diagram Methods 0.000 description 11
- 230000003993 interaction Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
- H04N21/2335—Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a method, a device and a system for processing transcoding data, which comprise the following steps: receiving audio and video data to be transcoded, wherein the audio and video data to be transcoded carries a matching identifier, and matching the audio data with corresponding video data according to the matching identifier; and sending the audio data and the video data which are matched with each other to the same transcoding server so that the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file, thereby improving the transcoding efficiency of the video.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, and a system for processing transcoded data.
Background
With the development of computer technology, people share and watch videos through a computer network, and due to the fact that the collection formats of videos are various, the videos are often required to be converted from one format to another format through a video transcoding technology so as to meet the requirement of playing.
The existing transcoding method is that after audio and video data are collected, videos are pushed to a Content Delivery Network (CDN) proxy server, audio is relayed nationwide, when a user requests a video stream, the video stream is issued by the CDN, the audio stream is obtained from a relay service and is transcoded and watched by a client, transcoding is carried out by the client, and transcoding efficiency is low.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus, and a system for processing transcoded data, which improve the transcoding efficiency of video.
A method of transcoding data processing, the method comprising:
receiving audio and video data to be transcoded, wherein the audio and video data to be transcoded carries a matching identifier, and matching the audio data with corresponding video data according to the matching identifier;
and sending the audio data and the video data which are matched with each other to the same transcoding server so that the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file.
An apparatus for transcoding data processing, the apparatus comprising:
the matching module is used for receiving audio and video data to be transcoded, carrying a matching identifier and matching the audio data and the corresponding video data according to the matching identifier;
and the distribution module is used for sending the audio data and the video data which are matched with each other to the same transcoding server so that the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file.
According to the method and the device for processing the transcoding data, the audio and video data to be transcoded are received, the audio data and the corresponding video data are matched according to the matching identification, the audio data and the video data which are matched with each other are sent to the same transcoding server, so that the transcoding server merges and transcodes the audio data and the video data which are matched with each other to generate the target transcoding file, the audio data and the video data which are matched with each other are sent to the same transcoding server, the audio data corresponding to the video data can be quickly found, data interaction among a plurality of servers is not needed, the speed of merging the audio data and the video data is increased, and the transcoding efficiency of the video is improved.
A method of transcoding data processing, the method comprising:
the method comprises the steps that a distribution server receives audio and video data to be transcoded, the audio and video data to be transcoded carry matching marks, and the audio data and the corresponding video data are matched according to the matching marks;
the distribution server sends the audio data and the video data which are matched with each other to the same transcoding server;
the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file;
and the transcoding server generates transcoding state information corresponding to the audio data and the video data which are matched with each other according to the generation result of the target transcoding file.
A system of transcoding data processing, the system comprising:
the distribution server is used for receiving audio and video data to be transcoded, matching identification is carried by the audio and video data to be transcoded, the audio data and the corresponding video data are matched according to the matching identification, and the audio data and the video data which are matched with each other are sent to the same transcoding server;
and the transcoding server is used for merging and transcoding the audio data and the video data which are matched with each other to generate a target transcoding file, and generating transcoding state information corresponding to the audio data and the video data which are matched with each other according to a generation result of the target transcoding file.
The method and the system for processing the transcoding data receive audio and video data to be transcoded through the distribution server, the audio and video data to be transcoded carries the matching identification, the audio data and the corresponding video data are matched according to the matching identification, the distribution server sends the audio data and the video data which are matched with each other to the same transcoding server, the transcoding server merges and transcodes the audio data and the video data which are matched with each other to generate a target transcoding file, the transcoding server generates transcoding state information corresponding to the audio data and the video data which are matched with each other according to a generation result of the target transcoding file, the audio data and the video data which are matched with each other are sent to the same transcoding server, the audio data corresponding to the video data can be quickly found, data interaction among a plurality of servers is not needed, and the speed of the transcoding server for merging the audio data and the video data is accelerated, the transcoding efficiency of the video is improved.
Drawings
FIG. 1 is a diagram of an application environment of a method of transcoding data in one embodiment;
FIG. 2 is a diagram illustrating an internal structure of the distribution server of FIG. 1 according to one embodiment;
FIG. 3 is a flow diagram of a method of transcoding data processing in one embodiment;
fig. 4 is a flowchart of sending audio data and video data that match each other to the same transcoding server in one embodiment;
fig. 5 is a flowchart of sending audio data and video data matched with each other to the same transcoding server in another embodiment;
FIG. 6 is a flow diagram of another method of transcoding data processing in one embodiment;
fig. 7 is a flowchart of a distribution server sending matched audio data and video data to the same transcoding server in one embodiment;
fig. 8 is a flowchart of sending audio data and video data matched with each other to the same transcoding server by the distribution server in another embodiment;
FIG. 9 is a block diagram of an apparatus for transcoding data in one embodiment;
FIG. 10 is a block diagram of the distribution module in one embodiment;
FIG. 11 is a block diagram of the distribution module in one embodiment;
fig. 12 is a block diagram showing the structure of an apparatus for transcoding data in another embodiment;
FIG. 13 is a block diagram of a system that transcodes data processing in one embodiment;
fig. 14 is a block diagram showing a configuration of a system for transcoding data in another embodiment.
Detailed Description
Fig. 1 is a diagram of an application environment in which a method for transcoding data operates according to an embodiment. As shown in fig. 1, the application environment includes an audio/video backend server 110, a distribution server 120, a transcoding server 130, a cloud server 140, a state server 150, and a terminal 160. Wherein each server and terminal communicate over a network.
The terminal 160 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, and the like. The terminal 160 can receive audio and video data from the transcoding server 130 through a network, the state server 150 is used for searching whether audio and video data with successful transcoding exists according to the received video searching request, the distribution server 120 is used for receiving the audio and video data to be transcoded from the audio and video background server 110, matching the audio and video data to be transcoded, managing and distributing the audio and video data to the same transcoding server 130 in a unified mode, reselecting a target transcoding server when transcoding fails according to the response of the transcoding server 130, and automatically completing restoration and modifying transcoding information stored in the cloud server 140. The transcoding server 130 merges the audio and video data and transcodes the data, and the response data packet is returned to the distribution server 120 after successful transcoding.
In one embodiment, the internal structure of the distribution server 120 in fig. 1 is as shown in fig. 2, and the distribution server 120 includes a processor, a storage medium, a memory, and a network interface connected by a system bus. The storage medium of the distribution server 120 stores an operating system, a database and a device for transcoding data, where the database is used to store data, such as audio and video data, and the device for transcoding data is used to implement a method for transcoding data processing suitable for the distribution server 120. The processor of the distribution server 120 is used to provide computing and control capabilities to support the operation of the entire distribution server 120. The memory of the distribution server 120 provides an environment for the operation of the means for transcoding data in the storage medium. The network interface of the distribution server 120 is used for communicating with external terminals and servers through network connection, such as sending data to be transcoded to the transcoding server 130.
In one embodiment, as shown in fig. 3, a method for transcoding data processing is provided, which is exemplified by being applied to a distribution server in the application environment, and includes the following steps:
step S210, receiving audio and video data to be transcoded, wherein the audio and video data to be transcoded carries a matching identifier, and matching the audio data with corresponding video data according to the matching identifier.
Specifically, the audio/video data to be transcoded can be received from the audio/video background server, the audio/video data to be transcoded can be contained in the transcoding request, and the matching identifier is used for identifying the source of the audio/video data, as if the audio data and the video data of one program carry the same matching identifier, the audio data and the video data of the same chat room carry the same matching identifier. And matching the audio data and the video data with the same matching identification so as to be conveniently transmitted to the same transcoding server.
And step S220, sending the audio data and the video data which are matched with each other to the same transcoding server so that the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file.
Specifically, the distribution server may select an appropriate target transcoding server according to working state parameters of the transcoding server, where the working state parameters include a process idle number, server hardware configuration, and the like. The audio data and the video data which are matched with each other can be sent to the same process or different processes of the same transcoding server. The audio data corresponding to the video data can be quickly found out by sending the audio data to the same transcoding server, out-of-order data packets can be sequenced in the same server, data interaction among a plurality of servers is not needed, and the speed of converging the audio data and the video data is increased. After the target transcoding server and the process are determined, corresponding transcoding information can be recorded, which is used for determining specific information corresponding to a sender of the audio data and the video data which are matched with each other, such as an IP (Internet Protocol) address of the target transcoding server, a port of the target transcoding server, a process number of the target transcoding server, and the like. The transcoding server merges the audio data and the VIDEO data which are matched with each other, and then performs transcoding, for example, the audio data packet and the VIDEO data packet are combined and converted into a data packet in a flv (FLASH VIDEO, streaming media format) format.
In the embodiment, through receiving audio and video data to be transcoded, the audio and video data to be transcoded carries the matching identification, the audio data and the corresponding video data are matched according to the matching identification, the audio data and the video data which are matched with each other are sent to the same transcoding server, so that the transcoding server merges the audio data and the video data which are matched with each other and transcodes the audio data and the video data to generate the target transcoding file, the audio data and the video data which are matched with each other are sent to the same transcoding server, the audio data corresponding to the video data can be found quickly, data interaction does not need to be carried out among a plurality of servers, the speed of merging the audio data and the video data is accelerated, and the transcoding efficiency of the video.
In one embodiment, as shown in fig. 4, step S220 includes:
and step S221, acquiring working state parameters of the transcoding server, selecting a target transcoding server according to the working state parameters, and sending the audio data and the video data which are matched with each other to the same process of the target transcoding server.
Specifically, the idle ratio of the transcoding server is judged according to the working state parameters of the transcoding server, and the transcoding server with the large idle ratio is used as the selected target transcoding server. If the working state parameters are the number of the unused processes and the total number of the processes to be distributed, calculating the proportion of the number of the unused processes and the total number of the processes to be distributed corresponding to different transcoding servers to obtain the idle proportion corresponding to different transcoding servers, and taking the transcoding server with the large idle proportion as the selected target transcoding server. The operating state parameters may also include hardware configuration, memory remaining, etc. And after the target transcoding server is determined, acquiring an idle process in the target transcoding server, and sending the audio data and the video data which are matched with each other to the same process of the target transcoding server. The data packets are more conveniently sorted and processed in the same process, and the communication among the processes is not needed, so that the transcoding efficiency is further improved.
Step S222, transcoding information corresponding to the target transcoding server is obtained, and the transcoding information is sent to the cloud server to be stored.
Specifically, the transcoding information is used to determine an object for actually transcoding the audio data and the video data that are matched with each other, such as an IP address of a target transcoding server, a port of the target transcoding server, a process number of the target transcoding server, and the like. The transcoding information can be stored corresponding to the matching identification of the audio data and the video data, so that different matching identifications correspond to different transcoding information, when the audio data and the video data need to be transcoded next time, the corresponding transcoding information can be obtained according to the matching identification carried by the audio data and the video data, and a target transcoding server for transcoding is determined according to the transcoding information. The transcoding information is sent to the cloud server to be stored, so that a plurality of different distribution servers can conveniently and quickly obtain the transcoding information from the cloud server when the transcoding data is distributed next time. And the servers are clearly separated from one another, and the working efficiency is high.
In one embodiment, as shown in fig. 5, step S220 includes:
step S223, obtaining the transcoding information, where the transcoding information includes the matching identifier.
Specifically, the transcoding information includes matching identifiers so as to establish a correspondence between the actual transcoding object and the audio and video source.
And step S224, searching whether the matching identification carried by the audio and video data to be transcoded exists in the transcoding information, if so, entering step S225, and otherwise, entering step S226.
Specifically, if the matching identifier carried by the audio and video data to be transcoded exists in the transcoding information, it is indicated that the audio and video data with the same source as the audio and video data to be transcoded is transcoded, and a first transcoding server and a first process used for transcoding before the audio and video data with the same source are obtained from the transcoding information. Therefore, the method can enter step S225, and audio and video data to be transcoded are sent to the first transcoding server, so that the audio and video data with the same source are transcoded by the same transcoding server, and operations such as sequencing of data packets are facilitated. If the matching identification carried by the audio and video data to be transcoded does not exist in the transcoding information, the audio and video data to be transcoded is sent from a new source, and a new transcoding server can be determined to transcode according to the current working state of the transcoding server.
Step S225, obtaining a first transcoding server corresponding to the matching identifier, and sending the audio data and the video data that are matched to each other to the first transcoding server.
Specifically, the audio and video data with the same matching identification are sent to the same transcoding server, if the audio and video data are transcoded intermittently due to network errors, it is guaranteed that audio and video data packets distributed from the same source and audio and video data packets distributed before can reach the same transcoding server, and the audio and video data sent by different users in sequence like the same video call room number can reach the same transcoding server for transcoding. And sending the audio data and the video data which are matched with each other to the same target process of the same transcoding server according to the process number of the target transcoding server in the transcoding information.
Step S226, the process proceeds to the step of obtaining the operating state parameters of the transcoding server.
In this embodiment, the same audio/video stream with the matching identifier is sent to the same transcoding machine or the same process of the same transcoding server by querying the transcoding information.
In one embodiment, as shown, after step S220, the method further includes: and if no response data packet returned by the transcoding server according to the audio data and the video data which are matched with each other is received, modifying the original target transcoding server in the transcoding information into a second transcoding server, and sending the audio data and the video data which are matched with each other to the second transcoding server next time, or modifying the original target process in the transcoding information into a second process, and sending the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time.
Specifically, each time the transcoding server receives audio data and video data which are matched with each other, if transcoding is successful, a corresponding response data packet is returned, and the response data packet carries a matching identifier for identifying which source data packet is successfully transcoded. If no response data packet returned by the transcoding server according to the audio data and the video data which are matched with each other is received, the transcoding server which performs transcoding or the target of the transcoding server goes wrong, and the transcoding server or the process in the transcoding server needs to be replaced. In one embodiment, the number of audio and video data to be transcoded, which do not receive a response data packet, is counted, if the number exceeds a preset threshold value, an original target transcoding server in transcoding information is modified into a second transcoding server, audio data and video data which are matched with each other are sent to the second transcoding server next time, or an original target process in transcoding information is modified into a second process, and audio data and video data which are matched with each other are sent to the second process of the original target transcoding server next time. If 4000 continuous packets do not return response data packets after finding that the audio/video data packets to be transcoded corresponding to a certain matching identifier are distributed to the original target transcoding server or the original target process distributed to the original target transcoding server, the original target transcoding server or the original target process is considered to have problems, and a new transcoding server or process can be switched. And the information is modified in the transcoding information to ensure that the information is synchronized at each distribution server. When the original target transcoding server or the original target process is found to have problems, warning information can be sent to a mailbox of a user or displayed on a terminal, so that the reminding effect is achieved. In this embodiment, when it is detected that a process of the transcoding server or the transcoding server has an error by whether the response packet is received, the process can be self-repaired, and another transcoding server or process is used for transcoding.
In one embodiment, as shown in fig. 6, there is provided a method of transcoding data processing, including:
and S310, the distribution server receives audio and video data to be transcoded, the audio and video data to be transcoded carries a matching identifier, and the audio data and the corresponding video data are matched according to the matching identifier.
Specifically, the distribution server may receive audio and video data to be transcoded from the audio and video backend server, where the audio and video data to be transcoded may be included in the transcoding request, and the matching identifier is used to identify a source of the audio and video data, as if the audio data and the video data of one program carry the same matching identifier, and the audio data and the video data of the same chat room carry the same matching identifier. And matching the audio data and the video data with the same matching identification so as to be conveniently transmitted to the same transcoding server.
And step S320, the distribution server sends the audio data and the video data which are matched with each other to the same transcoding server.
Specifically, the distribution server may select an appropriate target transcoding server according to working state parameters of the transcoding server, where the working state parameters include a process idle number, server hardware configuration, and the like. The audio data and the video data which are matched with each other can be sent to the same process or different processes of the same transcoding server. The audio data corresponding to the video data can be quickly found out by sending the audio data to the same transcoding server, out-of-order data packets can be sequenced in the same server, data interaction among a plurality of servers is not needed, and the speed of converging the audio data and the video data is increased. After the target transcoding server and the process are determined, corresponding transcoding information can be recorded, and the corresponding transcoding information is used for determining specific information corresponding to a sender of the audio data and the video data which are matched with each other, such as an IP address of the target transcoding server, a port of the target transcoding server, a process number of the target transcoding server and the like.
And step S330, the transcoding server performs confluence and transcoding on the audio data and the video data which are matched with each other to generate a target transcoding file.
Specifically, the transcoding server merges the audio data and the video data which are matched with each other, and then performs transcoding, for example, the audio data packet and the video data packet are combined and converted into a data packet in an flv format.
Step S340, the transcoding server generates transcoding state information corresponding to the audio data and the video data which are matched with each other according to the generation result of the target transcoding file.
Specifically, the transcoding state information is used to record whether transcoding is successful, and may include a matching identifier, a video identifier, and the like, and is used to record whether transcoding of the data to be transcoded corresponding to the matching identifier is successful. And the target transcoding file which is transcoded successfully can be stored, and the transcoding file which is transcoded successfully can be searched for playing according to the transcoding state information when the target transcoding file needs to be played next time.
In the embodiment, audio and video data to be transcoded are received through a distribution server, the audio data and the corresponding video data are matched according to the matching identification, the distribution server sends the audio data and the video data which are matched with each other to the same transcoding server, the transcoding server merges and transcodes the audio data and the video data which are matched with each other to generate a target transcoding file, the transcoding server generates transcoding state information corresponding to the audio data and the video data which are matched with each other according to a generation result of the target transcoding file, the audio data and the video data which are matched with each other are sent to the same transcoding server, the audio data corresponding to the video data can be quickly found, data interaction among a plurality of servers is not needed, and the speed of merging the audio data and the video data by the transcoding server is accelerated, the transcoding efficiency of the video is improved.
In one embodiment, as shown in fig. 7, step S320 includes:
step S321, the distribution server obtains the working state parameters of the transcoding server, selects a target transcoding server according to the working state parameters, and sends the audio data and the video data which are matched with each other to the same process of the target transcoding server.
Specifically, the distribution server judges the idle ratio of the transcoding server according to the working state parameters of the transcoding server, and takes the transcoding server with a large idle ratio as the selected target transcoding server. If the working state parameters are the number of the unused processes and the total number of the processes to be distributed, calculating the proportion of the number of the unused processes and the total number of the processes to be distributed corresponding to different transcoding servers to obtain the idle proportion corresponding to different transcoding servers, and taking the transcoding server with the large idle proportion as the selected target transcoding server. The operating state parameters may also include hardware configuration, memory remaining, etc. And after the target transcoding server is determined, acquiring an idle process in the target transcoding server, and sending the audio data and the video data which are matched with each other to the same process of the target transcoding server. The data packets are more conveniently sorted and processed in the same process, and the communication among the processes is not needed, so that the transcoding efficiency is further improved.
Step S322, the distribution server obtains the transcoding information corresponding to the target transcoding server, and sends the transcoding information to the cloud server for storage.
Specifically, the transcoding information is used to determine an object for actually transcoding the audio data and the video data that are matched with each other, such as an IP address of a target transcoding server, a port of the target transcoding server, a process number of the target transcoding server, and the like. The transcoding information can be stored corresponding to the matching identification of the audio data and the video data, so that different matching identifications correspond to different transcoding information, when the audio data and the video data need to be transcoded next time, the corresponding transcoding information can be obtained according to the matching identification carried by the audio data and the video data, and a target transcoding server for transcoding is determined according to the transcoding information. The distribution server sends the transcoding information to the cloud server for storage, and a plurality of different distribution servers can conveniently and quickly obtain the transcoding information from the cloud server when the transcoding data is distributed next time. And the servers are clearly separated from one another, and the working efficiency is high.
In one embodiment, as shown in fig. 8, step S320 includes:
step S323, the distribution server obtains the transcoding information, where the transcoding information includes the matching identifier.
Specifically, the transcoding information includes matching identifiers so as to establish a correspondence between the actual transcoding object and the audio and video source.
Step S324, the distribution server searches whether a matching identifier carried by the audio and video data to be transcoded exists in the transcoding information, if yes, step S325 is carried out, and if not, step S326 is carried out.
Specifically, if the matching identifier carried by the audio and video data to be transcoded exists in the transcoding information, it is indicated that the audio and video data with the same source as the audio and video data to be transcoded is transcoded, and a first transcoding server and a first process used for transcoding before the audio and video data with the same source are obtained from the transcoding information. Therefore, the method can enter step S225, and audio and video data to be transcoded are sent to the first transcoding server, so that the audio and video data with the same source are transcoded by the same transcoding server, and operations such as sequencing of data packets are facilitated. If the matching identification carried by the audio and video data to be transcoded does not exist in the transcoding information, the audio and video data to be transcoded is sent from a new source, and a new transcoding server can be determined to transcode according to the current working state of the transcoding server.
Step S325, obtain the first transcoding server corresponding to the matching identifier, and the distribution server sends the audio data and the video data that are matched to each other to the first transcoding server.
Specifically, the audio and video data with the same matching identification are sent to the same transcoding server, if the audio and video data are transcoded intermittently due to network errors, it is guaranteed that audio and video data packets distributed from the same source and audio and video data packets distributed before can reach the same transcoding server, and the audio and video data sent by different users in sequence like the same video call room number can reach the same transcoding server for transcoding. And sending the audio data and the video data which are matched with each other to the same target process of the same transcoding server according to the process number of the target transcoding server in the transcoding information.
In step S326, the distribution server obtains the operating state parameters of the transcoding server.
In this embodiment, the distribution server sends the same audio/video stream with the matching identifier to the same transcoding machine or the same process of the same transcoding server by querying the transcoding information.
In one embodiment, after step S330, the method further includes: and the transcoding server generates a response data packet of the target transcoding file and sends the response data packet to the distribution server.
Specifically, each time the transcoding server receives audio data and video data which are matched with each other, if transcoding is successful, a corresponding response data packet is returned to the distribution server, and the response data packet carries a matching identifier for identifying which source data packet is successfully transcoded.
After the step of sending the audio data and the video data which are matched with each other to the same transcoding server by the distribution server, the method further comprises the following steps: and if the distribution server does not receive the response data packet, modifying the original target transcoding server in the transcoding information into a second transcoding server, and sending the audio data and the video data which are matched with each other to the second transcoding server next time, or modifying the original target process in the transcoding information into a second process by the distribution server, and sending the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time.
Specifically, if a response data packet returned by the transcoding server according to the audio data and the video data which are matched with each other is not received, it is indicated that a problem occurs in the transcoding server for transcoding or the target of the transcoding server, and the transcoding server needs to be replaced or a process in the transcoding server needs to be replaced. In one embodiment, the number of audio and video data to be transcoded, which do not receive a response data packet, is counted, if the number exceeds a preset threshold value, an original target transcoding server in transcoding information is modified into a second transcoding server, audio data and video data which are matched with each other are sent to the second transcoding server next time, or an original target process in transcoding information is modified into a second process, and audio data and video data which are matched with each other are sent to the second process of the original target transcoding server next time. If 4000 continuous packets do not return response data packets after finding that the audio/video data packets to be transcoded corresponding to a certain matching identifier are distributed to the original target transcoding server or the original target process distributed to the original target transcoding server, the original target transcoding server or the original target process is considered to have problems, and a new transcoding server or process can be switched. And the information is modified in the transcoding information to ensure that the information is synchronized at each distribution server. When the original target transcoding server or the original target process is found to have problems, warning information can be sent to a mailbox of a user or displayed on a terminal, so that the reminding effect is achieved. In this embodiment, when it is detected that a process of the transcoding server or the transcoding server has an error by whether the response packet is received, the process can be self-repaired, and another transcoding server or process is used for transcoding.
In one embodiment, after step S340, the method further includes: and the transcoding server sends transcoding state information to the state server, and the state server receives a video searching request and searches whether a corresponding video transcoding file with successful transcoding exists according to the transcoding state information.
Specifically, the state server stores transcoding state information corresponding to each video, and the transcoding state is divided into successful transcoding and unsuccessful transcoding. And when the video needs to be played next time, the state server receives a video searching request, searches whether a corresponding video transcoding file with successful transcoding exists according to the transcoding state information, and if so, the corresponding video transcoding file can be directly sent to the terminal for playing without secondary transcoding. If the user needs to watch the video, the Content Delivery Network (CDN) can be used for returning the source, and the CDN queries a corresponding video transcoding file with successful transcoding through a state server, acquires a flv video stream from the transcoding server, and issues the flv video stream to the user.
In one embodiment, as shown in fig. 9, there is provided an apparatus for transcoding data, including:
the matching module 410 is configured to receive audio and video data to be transcoded, where the audio and video data to be transcoded carries a matching identifier, and match the audio data with corresponding video data according to the matching identifier.
And the distribution module 420 is configured to send the audio data and the video data that are matched with each other to the same transcoding server, so that the transcoding server performs merging and transcoding on the audio data and the video data that are matched with each other to generate a target transcoding file.
In one embodiment, as shown in FIG. 10, the distribution module 420 includes:
the first distribution unit 421 is configured to obtain working state parameters of the transcoding server, select a target transcoding server according to the working state parameters, and send audio data and video data that are matched with each other to the same process of the target transcoding server;
and the transcoding information storage unit 422 is configured to acquire transcoding information corresponding to the target transcoding server and send the transcoding information to the cloud server for storage.
In one embodiment, as shown in fig. 11, the distribution module 420 further includes:
the searching and judging unit 423 is used for acquiring transcoding information, wherein the transcoding information comprises a matching identifier, searching whether the matching identifier carried by the audio and video data to be transcoded exists in the transcoding information, if so, entering the second distribution unit, and if not, entering the first distribution unit 421;
and the second distribution unit 424 is configured to obtain the first transcoding server corresponding to the matching identifier, and send the audio data and the video data that are matched with each other to the first transcoding server.
In one embodiment, as shown in fig. 12, the apparatus further comprises:
and the repair module 430 is configured to modify the original target transcoding server in the transcoding information into a second transcoding server if a response data packet returned by the transcoding server according to the mutually matched audio data and video data is not received, and send the mutually matched audio data and video data to the second transcoding server next time, or modify the original target process in the transcoding information into a second process and send the mutually matched audio data and video data to the second process of the original target transcoding server next time.
In one embodiment, as shown in fig. 13, there is provided a system for transcoding data processing, comprising:
and the distribution server 510 is configured to receive audio and video data to be transcoded, where the audio and video data to be transcoded carries a matching identifier, match the audio data and the corresponding video data according to the matching identifier, and send the audio data and the video data that are matched with each other to the same transcoding server.
And the transcoding server 520 is configured to merge and transcode the audio data and the video data that are matched with each other to generate a target transcoding file, and generate transcoding state information corresponding to the audio data and the video data that are matched with each other according to a generation result of the target transcoding file.
In an embodiment, the distribution server 510 is further configured to obtain working state parameters of the transcoding server, select a target transcoding server according to the working state parameters, send audio data and video data that are matched with each other to the same process of the target transcoding server, obtain transcoding information corresponding to the target transcoding server, and send the transcoding information to the cloud server for storage.
In an embodiment, the distribution server 510 is further configured to obtain transcoding information, where the transcoding information includes a matching identifier, find whether a matching identifier carried by the audio/video data to be transcoded exists in the transcoding information, if so, obtain a first transcoding server corresponding to the matching identifier, send the audio data and the video data that are matched with each other to the first transcoding server, and if not, obtain a working state parameter of the transcoding server.
In one embodiment, the transcoding server 520 is further configured to generate a response packet indicating that the target transcoded file is successful and send the response packet to the distribution server 510.
The distribution server 510 is further configured to modify the original target transcoding server in the transcoding information into a second transcoding server if the response data packet is not received, and send the audio data and the video data that are matched with each other to the second transcoding server next time, or modify the original target process in the transcoding information into a second process, and send the audio data and the video data that are matched with each other to the second process of the original target transcoding server next time.
In one embodiment, as shown in fig. 14, the system further comprises: the state server 530, the transcoding server 520 is further configured to send transcoding state information to the state server 530, and the state server 530 is configured to receive a video search request, and search whether a corresponding video transcoding file with successful transcoding exists according to the transcoding state information.
It will be understood by those skilled in the art that all or part of the processes in the methods of the embodiments described above may be implemented by hardware related to instructions of a computer program, which may be stored in a computer readable storage medium, for example, in the storage medium of a computer system, and executed by at least one processor in the computer system, so as to implement the processes of the embodiments including the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (18)
1. A method of transcoding data processing, the method comprising:
receiving audio and video data to be transcoded, wherein the audio and video data to be transcoded carries a matching identifier, matching the audio data with corresponding video data according to the matching identifier, and the matching identifier is used for identifying the source of the audio and video data;
the method comprises the steps of obtaining working state parameters of a transcoding server, selecting a target transcoding server according to the working state parameters, and sending audio data and video data which are matched with each other to the same process of the target transcoding server, so that the process merges and transcodes the audio data and the video data which are matched with each other to generate a target transcoding file, wherein the audio data and the video data which are matched with each other come from the same source.
2. The method of claim 1, further comprising:
and acquiring transcoding information corresponding to the target transcoding server, and sending the transcoding information to the cloud server for storage.
3. The method of claim 2, wherein the step of sending the audio data and the video data that match each other to the same transcoding server further comprises:
acquiring transcoding information, wherein the transcoding information comprises a matching identifier;
searching whether a matching identifier carried by audio and video data to be transcoded exists in the transcoding information, if so, acquiring a first transcoding server corresponding to the matching identifier, and sending the audio data and the video data which are matched with each other to the first transcoding server;
and if the working state parameter does not exist, the step of obtaining the working state parameter of the transcoding server is carried out.
4. The method of claim 2, wherein after the step of sending the audio data and the video data that match each other to the same transcoding server, further comprising:
if a response data packet returned by the transcoding server according to the audio data and the video data which are matched with each other is not received, modifying the original target transcoding server in the transcoding information into a second transcoding server, and sending the audio data and the video data which are matched with each other to the second transcoding server next time;
or modifying the original target process in the transcoding information into a second process, and sending the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time.
5. A method of transcoding data processing, the method comprising:
the method comprises the steps that a distribution server receives audio and video data to be transcoded, the audio and video data to be transcoded carry matching marks, the audio data and the corresponding video data are matched according to the matching marks, and the matching marks are used for marking the source of the audio and video data;
the method comprises the steps that a distribution server obtains working state parameters of a transcoding server, selects a target transcoding server according to the working state parameters, and sends audio data and video data which are matched with each other to the same process of the target transcoding server, wherein the audio data and the video data which are matched with each other come from the same source;
converging and transcoding the audio data and the video data which are matched with each other by the same process in the target transcoding server to generate a target transcoding file;
and the transcoding server generates transcoding state information corresponding to the audio data and the video data which are matched with each other according to the generation result of the target transcoding file.
6. The method of claim 5, further comprising:
the distribution server acquires transcoding information corresponding to the target transcoding server and sends the transcoding information to the cloud server for storage.
7. The method of claim 6, wherein the step of sending the audio data and the video data matched with each other to the same transcoding server by the distribution server further comprises:
the method comprises the steps that a distribution server obtains transcoding information, wherein the transcoding information comprises matching identification;
the distribution server searches whether a matching identifier carried by the audio and video data to be transcoded exists in the transcoding information, and if the matching identifier exists, a first transcoding server corresponding to the matching identifier is obtained;
the distribution server sends the audio data and the video data which are matched with each other to the first transcoding server;
and if the working state parameters do not exist, the step of acquiring the working state parameters of the transcoding server by the distribution server is entered.
8. The method of claim 6, wherein after the step of merging and transcoding the audio data and the video data matched with each other by the transcoding server to generate the target transcoded file, the method further comprises:
the transcoding server generates a response data packet of the target transcoding file success and sends the response data packet to the distribution server;
after the step of sending the audio data and the video data which are matched with each other to the same transcoding server by the distribution server, the method further comprises the following steps:
if the distribution server does not receive the response data packet, the original target transcoding server in the transcoding information is modified into a second transcoding server, and audio data and video data which are matched with each other are sent to the second transcoding server next time;
or the distribution server modifies the original target process in the transcoding information into a second process, and sends the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time.
9. The method of claim 5, wherein after the step of generating the transcoding status information corresponding to the audio data and the video data that match each other by the transcoding server according to the generation result of the target transcoding file, the method further comprises:
and the transcoding server sends transcoding state information to the state server, and when the state server receives a video searching request, the state server searches whether a corresponding video transcoding file with successful transcoding exists according to the transcoding state information.
10. An apparatus for transcoding data processing, the apparatus comprising:
the matching module is used for receiving audio and video data to be transcoded, matching the audio data and the corresponding video data according to the matching identification, wherein the matching identification is used for identifying the source of the audio and video data;
the distribution module is used for acquiring working state parameters of the transcoding server, selecting a target transcoding server according to the working state parameters, and sending the audio data and the video data which are matched with each other to the same process of the target transcoding server so that the process can merge and transcode the audio data and the video data which are matched with each other to generate a target transcoding file, wherein the audio data and the video data which are matched with each other come from the same source.
11. The apparatus of claim 10, wherein the distribution module comprises:
and the transcoding information storage unit is used for acquiring transcoding information corresponding to the target transcoding server and sending the transcoding information to the cloud server for storage.
12. The apparatus of claim 11, wherein the distribution module further comprises:
the searching and judging unit is used for acquiring transcoding information, the transcoding information comprises a matching identifier, whether the matching identifier carried by audio and video data to be transcoded exists in the transcoding information is searched, if yes, the second distribution unit is started, and if not, the first distribution unit is started;
and the second distribution unit is used for acquiring the first transcoding server corresponding to the matching identifier and sending the audio data and the video data which are matched with each other to the first transcoding server.
13. The apparatus of claim 11, further comprising:
and the restoration module is used for modifying the original target transcoding server in the transcoding information into a second transcoding server if a response data packet returned by the transcoding server according to the audio data and the video data which are matched with each other is not received, and sending the audio data and the video data which are matched with each other to the second transcoding server next time, or modifying the original target process in the transcoding information into a second process, and sending the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time.
14. A system for transcoding data processing, the system comprising:
the distribution server is used for receiving audio and video data to be transcoded, the audio and video data to be transcoded carries a matching identifier, the working state parameters of the transcoding server are obtained according to the matching identifier matching the audio data and the corresponding video data, a target transcoding server is selected according to the working state parameters, the audio data and the video data which are matched with each other are sent to the same process of the target transcoding server, and the matching identifier is used for identifying the source of the audio and video data;
and the transcoding server is used for merging and transcoding the audio data and the video data which are matched with each other to generate a target transcoding file, and generating transcoding state information corresponding to the audio data and the video data which are matched with each other according to a generation result of the target transcoding file, wherein the audio data and the video data which are matched with each other come from the same source.
15. The system of claim 14, wherein the distribution server is further configured to obtain transcoding information corresponding to a target transcoding server, and send the transcoding information to the cloud server for storage.
16. The system of claim 15, wherein the distribution server is further configured to obtain transcoding information, where the transcoding information includes a matching identifier, find whether a matching identifier carried by audio/video data to be transcoded exists in the transcoding information, if so, obtain a first transcoding server corresponding to the matching identifier, send the audio data and the video data that are matched with each other to the first transcoding server, and if not, obtain a working state parameter of the transcoding server.
17. The system of claim 15, wherein the transcoding server is further configured to generate a response packet indicating that the target transcoded file is successful and send the response packet to the distribution server;
and the distribution server is also used for modifying the original target transcoding server in the transcoding information into a second transcoding server and sending the audio data and the video data which are matched with each other to the second transcoding server next time or modifying the original target process in the transcoding information into a second process and sending the audio data and the video data which are matched with each other to the second process of the original target transcoding server next time if the response data packet is not received.
18. The system of claim 15, further comprising: the transcoding server is also used for sending transcoding state information to the state server;
the state server is used for receiving the video searching request and searching whether the corresponding video transcoding file with successful transcoding exists or not according to the transcoding state information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178154.9A CN105721885B (en) | 2016-03-25 | 2016-03-25 | Method, device and system for processing transcoding data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178154.9A CN105721885B (en) | 2016-03-25 | 2016-03-25 | Method, device and system for processing transcoding data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105721885A CN105721885A (en) | 2016-06-29 |
| CN105721885B true CN105721885B (en) | 2019-12-20 |
Family
ID=56158172
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610178154.9A Active CN105721885B (en) | 2016-03-25 | 2016-03-25 | Method, device and system for processing transcoding data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105721885B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107124622A (en) * | 2017-04-14 | 2017-09-01 | 武汉鲨鱼网络直播技术有限公司 | A kind of audio frequency and video interflow compact system and method |
| CN106941613A (en) * | 2017-04-14 | 2017-07-11 | 武汉鲨鱼网络直播技术有限公司 | A kind of compacting of audio frequency and video interflow and supplying system and method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686447A (en) * | 2013-12-20 | 2014-03-26 | 广东威创视讯科技股份有限公司 | Video transmission method and system |
| CN104243998A (en) * | 2014-09-29 | 2014-12-24 | 广州华多网络科技有限公司 | Data processing method, data processing device and related servers |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7155475B2 (en) * | 2002-02-15 | 2006-12-26 | Sony Corporation | System, method, and computer program product for media publishing request processing |
| US7266611B2 (en) * | 2002-03-12 | 2007-09-04 | Dilithium Networks Pty Limited | Method and system for improved transcoding of information through a telecommunication network |
| WO2010062761A1 (en) * | 2008-11-03 | 2010-06-03 | Novarra, Inc. | Method and system for transforming and delivering video file content for mobile devices |
| CN103179430A (en) * | 2011-12-20 | 2013-06-26 | 中国电信股份有限公司 | Method, device and server for audio and video content transcoding on basis of cloud computing |
| CN104252394A (en) * | 2014-09-05 | 2014-12-31 | 江苏维罗可思信息科技有限公司 | Online cloud transcoding service system software |
| CN104506881A (en) * | 2014-12-31 | 2015-04-08 | 成都东方盛行电子有限责任公司 | Scheduling method for audio/video fragment transcoding |
-
2016
- 2016-03-25 CN CN201610178154.9A patent/CN105721885B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686447A (en) * | 2013-12-20 | 2014-03-26 | 广东威创视讯科技股份有限公司 | Video transmission method and system |
| CN104243998A (en) * | 2014-09-29 | 2014-12-24 | 广州华多网络科技有限公司 | Data processing method, data processing device and related servers |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105721885A (en) | 2016-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160239540A1 (en) | Data Query Method and Apparatus, Server, and System | |
| CN109788020B (en) | Agent distribution method and related equipment | |
| WO2018121619A1 (en) | Multimedia data processing method and device for service, server, and storage medium | |
| US20150227496A1 (en) | Method and system for microblog resource sharing | |
| CN108810657B (en) | Method and system for setting video cover | |
| CN109636514B (en) | Business data processing method and device, computing equipment and storage medium | |
| WO2019019645A1 (en) | Method and apparatus for executing information pushing task, and computer device and storage medium | |
| WO2008119266A1 (en) | Prompt method and apparatus of short message splitting | |
| WO2023061060A1 (en) | Audio and video code stream scheduling method, system, medium and electronic apparatus | |
| CN112769636B (en) | Video short message link monitoring method and device, electronic equipment and storage medium | |
| CN105721885B (en) | Method, device and system for processing transcoding data | |
| CN114064275A (en) | Data processing method and device | |
| US11163827B2 (en) | Video processing method, device, terminal and storage medium | |
| US20220385969A1 (en) | Method and apparatus for displaying live clip | |
| US20150026274A1 (en) | Method and apparatus for routing a message | |
| CN108415908B (en) | Multimedia data processing method and server | |
| CN111859127A (en) | Subscription method and device of consumption data and storage medium | |
| CN115119011A (en) | Live broadcast room content rebroadcasting method, background server and electronic equipment | |
| CN113132745B (en) | Live broadcast service system, method and server | |
| CN109379704B (en) | Method, device and equipment for correcting regional information of short message and storage medium | |
| CN110855785B (en) | Data processing method and device | |
| CN114270389A (en) | Information acquisition method, device, system, electronic equipment and storage medium | |
| CN106331270B (en) | Method and device for managing numbers of contact persons | |
| CN113438489B (en) | Push method and push system of cloud video | |
| CN112769776B (en) | Distributed service response method, system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |