CN114024946B - Method and device for adjusting streaming media code rate and computer readable storage medium - Google Patents
Method and device for adjusting streaming media code rate and computer readable storage medium Download PDFInfo
- Publication number
- CN114024946B CN114024946B CN202111288393.7A CN202111288393A CN114024946B CN 114024946 B CN114024946 B CN 114024946B CN 202111288393 A CN202111288393 A CN 202111288393A CN 114024946 B CN114024946 B CN 114024946B
- Authority
- CN
- China
- Prior art keywords
- mobile communication
- communication device
- code rate
- rsrp
- sinr
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000010295 mobile communication Methods 0.000 claims abstract description 148
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 238000010801 machine learning Methods 0.000 claims abstract description 21
- 230000000449 premovement Effects 0.000 claims description 28
- 238000003062 neural network model Methods 0.000 claims description 14
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000012706 support-vector machine Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 5
- 230000000903 blocking effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/309—Measuring or estimating channel quality parameters
- H04B17/318—Received signal strength
- H04B17/327—Received signal code power [RSCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/309—Measuring or estimating channel quality parameters
- H04B17/336—Signal-to-interference ratio [SIR] or carrier-to-interference ratio [CIR]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Electromagnetism (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The application provides a method and a device for adjusting a streaming media code rate and a computer readable storage medium. According to the technical scheme, first information and second information are acquired, the first information comprises a first position of mobile communication equipment, a moving speed and a moving direction of the mobile communication equipment at the first position and SINR and RSRP at the first position, the second information comprises distances between the positions of each of N base stations and the first position, then a machine learning model is used for acquiring SINR and RSRP of the mobile communication equipment at the current position based on the first information and the second information, a target streaming media code rate of the mobile communication equipment at the current position is determined based on a mapping relation between the SINR and RSRP stored in the mobile communication equipment and the streaming media code rate, and finally the streaming media code stream of the mobile communication equipment is adjusted to the target streaming media code rate. The definition and real-time performance of the images and the sounds sent by the plug-flow end can be effectively ensured.
Description
Technical Field
The present disclosure relates to the field of streaming media technologies, and in particular, to a method and an apparatus for adjusting a streaming media code rate, and a computer readable storage medium.
Background
The streaming media method (e.g., live broadcast) is a method in which one end (push end) transmitting video and audio transmits the video and audio to a streaming server according to a code rate, and then a terminal downloads the video and audio from the streaming server and plays the video and audio. Wherein, the code rate refers to the number of data bits transmitted in unit time.
The higher the code rate used by the push end, the higher the definition of the transmitted image and sound, and thus the higher the definition of the image and sound received by the terminal. However, the code rate of the push end is limited by the network condition of the push end, and in particular, the push end may use a high code rate under the condition that the network condition is good. If the network condition of the push end is poor, the higher the code rate still used by the push end, the phenomenon of blocking of the images and the sounds sent by the push end can be caused, so that the phenomenon of blocking of the images and the sounds received by the terminal can be caused. Therefore, the code rate of the push end should be matched with the network condition of the push end.
The network state of the push end in the moving state is usually changed, so that the code rate of the push end needs to be dynamically adjusted based on the network condition of the push end to ensure the definition and real-time performance (i.e. no jamming) of the images and sounds sent by the push end in the moving state.
A method for dynamically adjusting the code rate of a push end based on the network condition of the push end comprises the following steps: the plug-flow end adjusts the code rate according to the network condition of timing measurement. Specifically, the push end sends at least one additional data packet to the streaming media server at regular time, then determines the network condition by detecting the round trip delay and the packet loss rate of the at least one additional data packet, finally adjusts the code rate according to the obtained network condition, and when the network condition is good, increases the code rate to ensure the definition of the sent image and sound, and when the network condition is poor, decreases the code rate to ensure that the sent image and sound do not get stuck.
However, the effect and sound sent by the push end after the code rate of the push end is dynamically adjusted still have a clamping phenomenon, so that the image and sound received by the terminal also have the clamping phenomenon.
Disclosure of Invention
The embodiment of the application provides a method and a device for adjusting a streaming media code rate and a computer readable storage medium, which can effectively ensure definition and instantaneity of images and sounds sent by a streaming end.
In a first aspect, the present application provides a method for adjusting a streaming media code rate, which is applied to a mobile communication device, where a mapping relationship between a signal to interference plus noise ratio SINR and a reference signal received power RSRP and the streaming media code rate is stored in the mobile communication device, and the method includes: acquiring first information, wherein the first information comprises a first position of mobile communication equipment, a moving speed, a moving direction, a first SINR and a first RSRP when the mobile communication equipment is positioned at the first position; acquiring second information, wherein the second information indicates the distance between the position of each base station in N base stations and the first position, and N is a positive integer; acquiring a second SINR and a second RSRP of the mobile communication device at the current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at a post-movement location based on a pre-movement location of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement location, SINR and RSRP of the mobile communication device at the pre-movement location, and a distance between each of the at least one base station and the pre-movement location of the mobile communication device; obtaining a stream media code rate mapped with a second SINR and a second RSRP in the mapping relation to obtain a target stream media code rate of the mobile communication equipment at the current position; and adjusting the stream media code stream of the mobile communication equipment to the target stream media code rate.
The method provided in this embodiment may obtain the second SINR and the second RSRP corresponding to the current location of the mobile communication device based on the moving speed, the moving direction, the first SINR and the first RSRP of the mobile communication device at the first location, and the distance between the location of each of the N base stations and the first location using a machine learning model. Wherein the second SINR and the second RSRP may be used to reflect the network conditions of the mobile communication device at the current location.
It will be appreciated that in the prior art, in order to detect the current network condition, the mobile communication device needs to send at least one additional data packet to the streaming server, and then determine the network condition by detecting the round trip delay and the packet loss rate of the at least one additional data packet, which takes time, and during the detection, the network condition of the mobile communication device may change, so that a stuck phenomenon may still occur. In this embodiment, the network condition of the mobile communication device at the current location can be directly predicted based on the information of the mobile communication device at the first location through the machine learning model, and the operations such as packet sending and the like are not required to be performed in order to detect the network condition of the current location, so that the instantaneity and the stability are better ensured.
With reference to the first aspect, in one possible implementation manner, a distance between a position of each of the N base stations and the first position is less than or equal to a preset distance.
In this implementation, since the base station having a smaller distance from the first location has a larger influence on the network condition of the mobile communication device when the first location is located, in contrast, the base station having a larger distance from the first location has a substantially smaller influence on the network condition of the mobile communication device when the first location is located, in this implementation, when N base stations are acquired, each of the acquired N base stations satisfies that the distance from the first location is smaller than or equal to a preset distance.
With reference to the first aspect, in one possible implementation manner, the N base stations are N base stations closest to the first location from all base stations with a distance between the N base stations and the first location being less than or equal to a preset distance.
With reference to the first aspect, in a possible implementation manner, the method further includes: and receiving third information, wherein the third information comprises the mapping relation between the signal to interference plus noise ratio SINR and reference signal received power RSRP and stream media code rate.
With reference to the first aspect, in one possible implementation manner, the machine model includes any one of the following: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model.
In a second aspect, the present application provides an adjustment apparatus for a streaming media code rate, which is applied to a mobile communication device, where a mapping relationship between a signal to interference plus noise ratio SINR and a reference signal received power RSRP and the streaming media code rate is stored in the mobile communication device, and the apparatus includes: the mobile communication device comprises an acquisition module, a first information processing module and a second information processing module, wherein the acquisition module is used for acquiring first information, and the first information comprises a first position of the mobile communication device, a moving speed, a moving direction, a first SINR and a first RSRP when the mobile communication device is located at the first position; the acquisition module is further used for acquiring second information, the second information indicates the distance between the position of each base station in the N base stations and the first position, and N is a positive integer; a prediction module for acquiring a second SINR and a second RSRP of the mobile communication device at the current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at the post-movement location based on a pre-movement location of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement location, SINR and RSRP of the mobile communication device at the pre-movement location, and a distance between each of the at least one base station and the pre-movement location of the mobile communication device; the determining module is used for obtaining the stream media code rate mapped with the second SINR and the second RSRP in the mapping relation to obtain the target stream media code rate of the mobile communication equipment at the current position; and the adjusting module is used for adjusting the stream media code stream of the mobile communication equipment to the target stream media code rate.
With reference to the second aspect, in one possible implementation manner, a distance between a location of each of the N base stations and the first location is less than or equal to a preset distance.
With reference to the second aspect, in one possible implementation manner, the N base stations are N base stations closest to the first location from all base stations with a distance from the first location less than or equal to a preset distance.
With reference to the second aspect, in a possible implementation manner, the apparatus further includes: and the receiving module is used for receiving third information, wherein the third information comprises the signal-to-interference-plus-noise ratio SINR and the mapping relation between the reference signal received power RSRP and the streaming media code rate.
With reference to the second aspect, in one possible implementation manner, the machine model includes any one of the following: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model.
In a third aspect, the present application provides an apparatus for adjusting a streaming media code rate, including: a processor coupled to a memory for storing a computer program which, when invoked by the processor, causes the mobile communication device to perform the method as described in the first aspect and any one of the possible implementations thereof.
In a fourth aspect, the present application provides a mobile communication device comprising the apparatus of the second or third aspect or any one of the possible implementations thereof.
In a fifth aspect, the present application provides a chip comprising at least one processor and a communication interface, the communication interface and the at least one processor being interconnected by wires, the at least one processor being adapted to run a computer program or instructions to perform a method as described in the first aspect and any one of the possible implementations.
In a sixth aspect, the present application provides a computer readable storage medium storing program code for computer execution, the program code comprising instructions for performing the method as described in the first aspect and any one of the possible implementations thereof.
In a seventh aspect, the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method according to the first aspect or any one of the possible implementations thereof.
Drawings
Fig. 1 is a schematic structural diagram of a streaming media system according to an embodiment of the present application;
Fig. 2 is a schematic flow chart of a method for adjusting a streaming media code rate according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a relationship between a mobile communication device and a base station position during a moving process according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a neural network model according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an apparatus for adjusting a streaming media code rate according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an apparatus for adjusting a streaming media code rate according to another embodiment of the present application.
Detailed Description
The streaming media method (e.g., live broadcast) is a method in which one end (push end) transmitting video and audio transmits the video and audio to a streaming server according to a code rate, and then a terminal downloads the video and audio from the streaming server and plays the video and audio. Wherein, the code rate refers to the number of data bits transmitted in unit time.
Illustratively, fig. 1 is a schematic structural diagram of a streaming media system according to an embodiment of the present application. As shown in fig. 1, the streaming media system includes a push end 101, a streaming media server 102, and a terminal 103.
The push end 101 is an end that sends a streaming media file, for example, the streaming media file includes but is not limited to: video streams or audio streams. The terminal 103 is one end that receives and plays the streaming media file. For the streaming system shown in fig. 1, the push end 101 sends images and sounds to the streaming server 102 according to the code rate, and then the terminal 103 downloads and plays the images and sounds from the streaming server 102.
It is noted that the specific form of the terminal 103 is not limited in the embodiment of the present application. For example, the terminal may be a device that provides voice and/or data connectivity to a user, e.g., a handheld device with wireless connectivity, an in-vehicle device, etc. A terminal may also be referred to as a User Equipment (UE), an access terminal (access terminal), a user unit (user unit), a user station (user station), a mobile station (mobile), a remote station (remote station), a remote terminal (remote terminal), a mobile device (mobile device), a user terminal (user terminal), a wireless communication device (wireless telecom equipment), a user agent (user agent), a user equipment (user equipment), or a user device. The terminal may be a Station (STA) in a wireless local area network (wireless local Area networks, WLAN), may be a cellular telephone, a cordless telephone, a session initiation protocol (session initiation protocol, SIP) phone, a wireless local loop (wireless local loop, WLL) station, a personal digital assistant (personal digital assistant, PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, a terminal in a next generation communication system (e.g., a fifth generation (5G) communication network) or a terminal in a future evolved public land mobile network (public land mobile network, PLMN) network, etc. Wherein 5G may also be referred to as a New Radio (NR). In one possible application scenario, the terminal is a terminal device that often works on the ground, such as a vehicle-mounted device or a mobile phone.
For the streaming media system shown in fig. 1, the higher the code rate used by the push end 101, the higher the definition of the transmitted image and sound, and thus the higher the definition of the image and sound received by the terminal 103. However, the code rate of the push end 101 is limited by the network condition of the push end 101, and in particular, the push end 101 may use a high code rate when the network condition is good. If the network condition of the push end 101 is poor, the higher the code rate still used by the push end 101, the phenomenon of blocking the images and sounds sent by the push end 101 will occur, so that the phenomenon of blocking the images and sounds received by the terminal 103 will occur. Therefore, the code rate of the push end 101 should be matched to the network condition of the push end 101.
The network state of the moving push end 101 is generally changed, so it is necessary to dynamically adjust the code rate of the push end 101 based on the network condition of the push end 101 to ensure the definition and real-time (i.e. no jamming) of the images and sounds sent by the push end 101 in the moving state.
At present, a method for dynamically adjusting the code rate of the push end 101 based on the network condition of the push end 101 is as follows: the push end 101 adjusts the code rate according to the network conditions measured at the timing. Specifically, the push end 101 periodically sends at least one additional data packet to the streaming server 102, then determines the network condition by detecting the round trip delay and the packet loss rate of the at least one additional data packet, and finally adjusts the code rate according to the obtained network condition, and when the network condition is good, increases the code rate to ensure the definition of the sent image and sound, and when the network condition is poor, decreases the code rate to ensure that the sent image and sound do not get stuck.
However, in the above method, the push end 101 still transmits the video and the sound using the code rate corresponding to the last network condition during the measurement of the network condition. It will be appreciated that during this measurement of the network conditions, changes in the network conditions may also occur. For example, the code rate corresponding to the previous network condition is higher, but the network condition is poor during the measurement of the network condition, and at this time, due to the higher code rate corresponding to the previous network condition, the image and the sound sent during the measurement of the network condition are blocked, and further, the image and the sound received by the terminal are blocked.
In view of this, the present application provides a method for adjusting a streaming media code rate. The method comprises the steps of firstly obtaining first information and second information, wherein the first information comprises a first position of a mobile communication device, a moving speed and a moving direction of the mobile communication device when the mobile communication device is located at the first position, a signal-to-interference-plus-noise ratio (signal to interference plus noise ratio, SINR) when the mobile communication device is located at the first position and reference signal received power (reference signal receiving power, RSRP) when the mobile communication device is located at the first position, the second information comprises a distance between the position of each base station in N base stations and the first position, then obtaining the SINR and the RSRP of the mobile communication device at the current position based on the first information and the second information by using a machine learning model, determining a target streaming media code rate of the mobile communication device at the current position based on a mapping relation between the SINR and the RSRP stored in the mobile communication device and the streaming media code rate, and finally adjusting the streaming media code stream of the mobile communication device to the target streaming media code rate.
Fig. 2 is a schematic flow chart of a method for adjusting a streaming media code rate according to an embodiment of the present application. As shown in fig. 2, the method of the present embodiment may include S201, S202, S203, S204, and S205. The implemented method may be performed by a mobile communication device.
In this embodiment, the mapping relationship between SINR and RSRP and the streaming media code rate is stored in the mobile communication device. It will be appreciated that SINR and RSRP may be used to reflect the network conditions of the mobile communication device and, therefore, the mapping relationship between network conditions and streaming media code rates may also be considered to be stored in the mobile communication device.
Illustratively, table 1 is a schematic structural diagram of the mapping relationship provided in the embodiments of the present application. As shown in table 1, the RSRP is expressed in decibel milliwatts (decibel relative to one milliwatt, dbm), the SINR is expressed in decibels (db), and the streaming media rate is expressed in kilobits per second (kilo bit per second, kbps).
TABLE 1
| SINR/db | RSRP/dbm | Stream media code rate/Kbps |
| Greater than 10 | Greater than-80 | 4096 |
| 5～10 | -80～-90 | 1024 |
| 0～5 | -90～-100 | 640 |
| -5～0 | -100～-130 | 320 |
| Less than-5 | Less than-130 | 96 |
| … | … | … |
It is noted herein that the numbers in the above tables are only examples and do not limit the present application.
The steps in the method shown in fig. 2 are described in detail below.
S201, acquiring first information, where the first information includes a first location of the mobile communication device, a moving speed, a moving direction, a first SINR, and a first RSRP when the mobile communication device is located at the first location.
Wherein the first location may be considered a location of the mobile communication device before moving to the current location, i.e. may be considered a historical location. Accordingly, the first SINR and the first RSRP in the present embodiment correspond to the SINR and the RSRP of the first location.
It is noted that SINR and RSRP may be used to describe network conditions of the mobile communication device, and specific description thereof may refer to description in the related art, which is not repeated herein.
In this embodiment, the movement speed and the movement direction are used to describe the movement condition of the mobile communication device, and it is understood that the movement condition of the mobile communication device has an influence on the network condition of the mobile communication device. For example, when a mobile communication device moves rapidly, there is a possibility that the network condition changes relatively much, i.e., the network condition may be unstable. In view of this, the moving speed and moving direction of the mobile communication device are also acquired in the present embodiment.
Here, the present embodiment is not limited to a specific implementation manner how to acquire the first location of the mobile communication device, the moving speed, the moving direction, the first SINR, and the first RSRP when the mobile communication device is located at the first location. For example, the first location, the movement speed and the movement direction when the mobile communication device is located at the first location may be obtained by the sensor, and the first SINR and the first RSRP may be obtained by the gateway.
S202, acquiring second information, wherein the second information indicates the distance between the position of each base station in N base stations and the first position, and N is a positive integer.
It will be appreciated that the network condition of a mobile communication device is also typically related to the distance between the mobile communication device and a base station. Fig. 3 is a schematic structural diagram illustrating a relationship between a mobile communication device and a base station during a mobile procedure according to an embodiment of the present application. As shown in fig. 3, when the mobile communication device 103 is at the location 1, the network condition may be greatly affected by the base station 1 or the base station 2, and when the mobile communication device moves to the location 2, the network condition may be greatly affected by the base station 4 or the base station 5. It will be further appreciated that, regardless of the location to which the mobile communication device is moved, generally, there are typically a plurality of base stations in the vicinity of that location, and therefore, in order to more fully obtain the network condition of the mobile communication device, the present implementation obtains N distances corresponding to the N base stations, respectively. For example, take N equal to 5.
S203, obtaining a second SINR and a second RSRP of the mobile communication device at the current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at the post-movement location based on the pre-movement location of the mobile communication device, the movement speed and movement direction of the mobile communication device at the pre-movement location, the SINR and RSRP of the mobile communication device at the pre-movement location, and the distance between each of the at least one base station and the pre-movement location of the mobile communication device.
For example, the machine model includes any one of the following: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model. The detailed description of the machine learning model may refer to the description in the related art, and will not be repeated here.
In this embodiment, the machine learning model may be regarded as a prediction model capable of predicting SINR and RSRP at a post-movement position of the mobile communication device by a pre-movement position of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement position, SINR and RSRP of the mobile communication device at the pre-movement position, and a distance between each of the at least one base stations and the pre-movement position of the mobile communication device.
In this embodiment, the second SINR and the second RSRP may be used to describe the network condition of the mobile communication device at the current location.
Taking fig. 3 as an example, when the mobile communication device is located at the location 2, in order to obtain the network condition of the mobile communication device at the location 2, the moving speed, the first SINR and the first RSRP and the moving direction at the location 1 and the distance between each of the at least one base stations and the location of the mobile communication device before moving can be input into the machine learning model by inputting the location information of the mobile communication device at the location 1, and then obtaining the second SINR and the second RSRP.
It is to be understood that, in the prediction using the machine learning model in the present embodiment, the machine learning model should be trained in advance, that is, the machine learning model is trained by acquiring a large amount of data so that the trained machine learning model has a prediction function. For the obtained data, reference may be made to the description in the related art for how to train the machine learning model on the obtained data, which is not described herein.
S204, obtaining the stream media code rate mapped with the second SINR and the second RSRP in the mapping relation, and obtaining the target stream media code rate of the mobile communication equipment at the current position.
In this embodiment, since the mapping relationship between the SINR and the RSRP and the streaming media code rate is already stored in the mobile communication device, after the second SINR and the second RSRP of the mobile communication device at the current position are obtained, the target streaming media code rate of the mobile communication device at the current position can be determined based on the stored mapping relationship.
Still referring to table 1, in one example, it is assumed that the value of the obtained second SINR is 8db, and the second RSRP is-85 dbm, and at this time, it may be determined that the target streaming media code rate of the mobile communication device at the current location should be 1024Kbps according to the mapping relationship of table 1.
Still referring to table 1, in another example, it is assumed that the value of the obtained second SINR is 8db, and the second RSRP is-74 dbm, and at this time, it may be determined that the second SINR corresponds to 1024Kbps and the second RSRP corresponds to 4096Kbps according to the mapping relationship of table 1. It will be appreciated that for streaming services it is more important to ensure that the streaming file does not clip, and therefore in this case it is determined that the target streaming rate for the mobile communication device at the current location should be 1024Kbps.
S205, the stream media code stream of the mobile communication equipment is adjusted to the target stream media code rate.
In this embodiment, after determining the target streaming media code rate based on the mapping relationship, if the target streaming media code rate changes compared with the previous streaming media code rate, the streaming media code stream of the mobile communication device is adjusted to the target streaming media code rate.
According to the method for adjusting the streaming media code rate, a machine learning model can be used to obtain a second SINR and a second RSRP corresponding to the current position of the mobile communication device based on the moving speed, the moving direction, the first SINR and the first RSRP of the mobile communication device at the first position and the distance between the position of each of the N base stations and the first position. Wherein the second SINR and the second RSRP may be used to reflect the network conditions of the mobile communication device at the current location.
As an alternative embodiment, the distance between the position of each of the N base stations and the first position is less than or equal to a preset distance.
It will be appreciated that when the mobile communication device is in the first location, base stations further from the first location have little effect on the network conditions of the mobile communication device, and therefore, base stations of this type are less likely to be required as factors affecting the network conditions of the mobile communication device. In contrast, when the mobile communication device is in the first position, the base station closer to the first position is more likely to affect the network condition of the mobile communication device, and therefore, in this embodiment, the base station having a distance from the first position that is smaller than or equal to the preset distance is taken as a factor that may affect the network condition of the mobile communication device, that is, when the distance between the position of each of the N base stations and the first position is acquired, the distance between the position of each of the N base stations and the first position is smaller than or equal to the preset distance.
A method of determining a distance between a base station and a first location is described below.
Assuming that the longitude and latitude of the first location of the mobile communication device are expressed as (LonA, latA), the longitude and latitude of a certain base station B near the first location are (LonB, latB), the east longitude takes the positive value of longitude according to the 0 degree longitude, the west longitude takes the negative value, the north latitude takes the 90 minus the latitude value, and the south latitude takes the 90 plus the latitude value, after the above processing, it is assumed that the first location is expressed as (MLonA, MLatA), and the location of the base station B is expressed as (MLonB, MLatB). The distance between the first location and the location of base station B can be expressed by the following formula:
Distance(A，B)＝R*Arccos(C)*Pi/180
C＝sin(MLatA)*sin(MLatB)*cos(MLonA-MLonB)+cos(MLatA)*cos(MLatB)
Where R is the earth radius, 6371.004 km, distance (A, B) represents the Distance between the first location and base station B in the same units as the earth radius.
Still further, the N base stations may be N base stations closest to the first location among all base stations having a distance from the first location less than or equal to a preset distance.
In one implementation, the distances between all the base stations of the mobile communication device within a certain geographic range of the first location and the first location may be calculated based on the above formula, and then the first N base stations are taken after being arranged in increasing order. Illustratively, N is equal to 5.
More specifically, in this embodiment, when N base stations are N base stations closest to the first location among all base stations whose distances from the first location are less than or equal to a preset distance, the second information is the distances between the closest N base stations and the first location, respectively, when the second SINR and the second RSRP of the mobile communication device at the current location are acquired based on the first information and the second information using the machine learning model. Illustratively, taking the machine learning model as the neural network model, N is equal to 5, at this time, as shown in fig. 4, the second information includes the closest point distance, the second closest point distance, the third closest point distance, the fourth closest point distance, and the fifth closest point distance.
As an alternative embodiment, the method further comprises: and receiving third information, wherein the third information comprises a mapping relation between the stored signal to interference plus noise ratio SINR and reference signal received power RSRP and stream media code rate in the mobile communication equipment.
Fig. 5 is a device for adjusting a streaming media code rate according to an embodiment of the present application, which is applied to a mobile communication device, where a mapping relationship between a signal to interference plus noise ratio SINR and a reference signal received power RSRP and the streaming media code rate is stored in the mobile communication device, and the device 500 includes: an acquisition module 501, a prediction module 502, a determination module 503, and an adjustment module 504.
An obtaining module 501, configured to obtain first information, where the first information includes a first location of a mobile communication device, a moving speed, a moving direction, a first SINR, and a first RSRP when the mobile communication device is located at the first location; the obtaining module 501 is further configured to obtain second information, where the second information indicates a distance between a position of each of the N base stations and the first position, and N is a positive integer; a prediction module 502 for obtaining a second SINR and a second RSRP of the mobile communication device at a current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at a post-movement location based on a pre-movement location of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement location, SINR and RSRP of the mobile communication device at the pre-movement location, and a distance between each of at least one base station and the pre-movement location of the mobile communication device; a determining module 503, configured to obtain a stream media code rate mapped with the second SINR and the second RSRP in the mapping relationship, to obtain a target stream media code rate of the mobile communication device at the current position; and the adjusting module 504 is configured to adjust the streaming media code stream of the mobile communication device to the target streaming media code rate.
In one possible implementation, the distance between the location of each of the N base stations and the first location is less than or equal to a preset distance.
In one possible implementation, the N base stations are N base stations closest to the first location among all base stations having a distance from the first location less than or equal to a preset distance.
In one possible implementation, the apparatus 500 further includes: a receiving module 505, configured to receive third information, where the third information includes the signal-to-interference-plus-noise ratio SINR and a mapping relationship between a reference signal received power RSRP and a streaming media code rate.
In one possible implementation, the machine model includes any one of the following: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model.
Fig. 6 is a schematic structural diagram of an apparatus for adjusting a streaming media code rate according to another embodiment of the present application. The apparatus shown in fig. 6 may be used to perform the method described in any of the previous embodiments.
As shown in fig. 6, the apparatus 600 of the present embodiment includes: memory 601, processor 602, communication interface 603 and bus 604. The memory 601, the processor 602, and the communication interface 603 are connected to each other by a bus 604.
The memory 601 may be a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a random access memory (random access memory, RAM). The memory 601 may store a program, and the processor 602 is configured to perform the steps of the method shown in fig. 2 when the program stored in the memory 601 is executed by the processor 602.
The processor 602 may employ a general-purpose central processing unit (central processing unit, CPU), microprocessor, application specific integrated circuit (application specific integrated circuit, ASIC), or one or more integrated circuits for executing associated programs to implement the method illustrated in fig. 2 of the present application.
The processor 602 may also be an integrated circuit chip with signal processing capabilities. In implementation, various steps of the method of fig. 2 of the embodiments of the present application may be accomplished by instructions in the form of integrated logic circuits or software of hardware in the processor 602.
The processor 602 may also be a general purpose processor, a digital signal processor (digital signal processing, DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 601, and the processor 602 reads information in the memory 601 and in combination with its hardware performs the functions necessary to be performed by the units comprised in the device, for example, the steps/functions of the embodiment shown in fig. 2 can be performed.
The communication interface 603 may enable communication between the apparatus 600 and other devices or communication networks using, but is not limited to, a transceiver-like transceiver.
A bus 604 may include a path to transfer information between elements of the apparatus 600 (e.g., the memory 601, the processor 602, the communication interface 603).
It should be understood that the apparatus 600 shown in the embodiments of the present application may be an electronic device, or may be a chip configured in an electronic device.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present application are all or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. In addition, the character "/" herein generally indicates that the associated object is an "or" relationship, but may also indicate an "and/or" relationship, and may be understood by referring to the context.
In the present application, "at least one" means one or more, and "a plurality" means two or more. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (8)
1. The method for adjusting the stream media code rate is characterized by being applied to mobile communication equipment, wherein the mapping relation between the signal to interference plus noise ratio SINR and reference signal received power RSRP and the stream media code rate is stored in the mobile communication equipment, and the method comprises the following steps:
acquiring first information, wherein the first information comprises a first position of the mobile communication equipment, a moving speed, a moving direction, a first SINR and a first RSRP when the mobile communication equipment is positioned at the first position;
acquiring second information, wherein the second information indicates the distance between the position of each base station in N base stations and the first position, and N is a positive integer; the N base stations are N nearest base stations from the first position in all base stations with the distance between the N base stations and the first position being smaller than or equal to a preset distance;
Acquiring a second SINR and a second RSRP of the mobile communication device at a current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at a post-movement location based on a pre-movement location of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement location, SINR and RSRP of the mobile communication device at the pre-movement location, and a distance between each of at least one base station and the mobile communication device at the pre-movement location;
obtaining a stream media code rate mapped with the second SINR and the second RSRP in the mapping relation to obtain a target stream media code rate of the mobile communication equipment at the current position;
and adjusting the stream media code stream of the mobile communication equipment to the target stream media code rate.
2. The method according to claim 1, wherein the method further comprises:
and receiving third information, wherein the third information comprises the mapping relation between the signal to interference plus noise ratio SINR and reference signal received power RSRP and stream media code rate.
3. The method according to any one of claims 1 to 2, wherein the machine model comprises any one of: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model.
4. An apparatus for adjusting a streaming media code rate, which is applied to a mobile communication device, wherein a mapping relationship between a signal-to-interference-plus-noise ratio SINR and a reference signal received power RSRP and the streaming media code rate is stored in the mobile communication device, the apparatus comprises:
an acquisition module, configured to acquire first information, where the first information includes a first location of the mobile communication device, a movement speed, a movement direction, a first SINR, and a first RSRP when the mobile communication device is located at the first location;
the acquisition module is further configured to acquire second information, where the second information indicates a distance between a position of each of N base stations and the first position, and N is a positive integer; the N base stations are N nearest base stations from the first position in all base stations with the distance between the N base stations and the first position being smaller than or equal to a preset distance;
a prediction module for obtaining a second SINR and a second RSRP of the mobile communication device at a current location based on the first information and the second information using a machine learning model for predicting SINR and RSRP of the mobile communication device at a post-movement location based on a pre-movement location of the mobile communication device, a movement speed and a movement direction of the mobile communication device at the pre-movement location, SINR and RSRP of the mobile communication device at the pre-movement location, and a distance between each of at least one base station and the pre-movement location of the mobile communication device;
A determining module, configured to obtain a streaming media code rate mapped with the second SINR and the second RSRP in the mapping relationship, to obtain a target streaming media code rate of the mobile communication device at the current position;
and the adjusting module is used for adjusting the stream media code stream of the mobile communication equipment to the target stream media code rate.
5. The apparatus of claim 4, wherein the apparatus further comprises:
and the receiving module is used for receiving third information, wherein the third information comprises the signal-to-interference-plus-noise ratio SINR and the mapping relation between the reference signal received power RSRP and the streaming media code rate.
6. The apparatus of any one of claims 4 to 5, wherein the machine model comprises any one of: a fully connected neural network model, a convolutional neural network model, a long and short memory neural network model and a support vector machine model.
7. An apparatus for adjusting a streaming media code rate, comprising: memory, processor, and transceiver;
the memory is used for storing program instructions;
the processor is configured to invoke program instructions in the memory to perform the method of any of claims 1 to 3.
8. A computer readable storage medium storing program code for computer execution, the program code comprising instructions for performing the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111288393.7A CN114024946B (en) | 2021-11-02 | 2021-11-02 | Method and device for adjusting streaming media code rate and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111288393.7A CN114024946B (en) | 2021-11-02 | 2021-11-02 | Method and device for adjusting streaming media code rate and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114024946A CN114024946A (en) | 2022-02-08 |
| CN114024946B true CN114024946B (en) | 2024-02-23 |
Family
ID=80059875
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111288393.7A Active CN114024946B (en) | 2021-11-02 | 2021-11-02 | Method and device for adjusting streaming media code rate and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114024946B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105898800A (en) * | 2015-10-30 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Multi-link aggregated link bandwidth distribution method and apparatus applied to mobile network |
| CN106656629A (en) * | 2017-01-13 | 2017-05-10 | 南京理工大学 | Prediction method for stream media playing quality |
| CN106817721A (en) * | 2015-11-30 | 2017-06-09 | ***通信集团公司 | A kind of method of streaming media service bandwidth estimation, device, terminal and server |
| CN109040801A (en) * | 2018-07-19 | 2018-12-18 | 北京达佳互联信息技术有限公司 | Media code rate by utilizing adaptive approach, device, computer equipment and storage medium |
| CN109756911A (en) * | 2019-01-31 | 2019-05-14 | 腾讯科技（深圳）有限公司 | Network quality prediction technique, business reorganization method, relevant device and storage medium |
| CN111491262A (en) * | 2020-04-26 | 2020-08-04 | 中国信息通信研究院 | Method and device for determining signal strength of mobile broadband network |
| CN112272299A (en) * | 2020-11-27 | 2021-01-26 | 广州市网星信息技术有限公司 | Video coding method, device, equipment and storage medium |
| CN112383823A (en) * | 2020-10-16 | 2021-02-19 | 北京奇艺世纪科技有限公司 | Code rate self-adaption method and device, electronic equipment and storage medium |
| CN112953922A (en) * | 2021-02-03 | 2021-06-11 | 西安电子科技大学 | Self-adaptive streaming media control method, system, computer equipment and application |
| CN113518250A (en) * | 2020-08-07 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Multimedia data processing method, device, equipment and readable storage medium |
-
2021
- 2021-11-02 CN CN202111288393.7A patent/CN114024946B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105898800A (en) * | 2015-10-30 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Multi-link aggregated link bandwidth distribution method and apparatus applied to mobile network |
| CN106817721A (en) * | 2015-11-30 | 2017-06-09 | ***通信集团公司 | A kind of method of streaming media service bandwidth estimation, device, terminal and server |
| CN106656629A (en) * | 2017-01-13 | 2017-05-10 | 南京理工大学 | Prediction method for stream media playing quality |
| CN109040801A (en) * | 2018-07-19 | 2018-12-18 | 北京达佳互联信息技术有限公司 | Media code rate by utilizing adaptive approach, device, computer equipment and storage medium |
| WO2020015678A1 (en) * | 2018-07-19 | 2020-01-23 | 北京达佳互联信息技术有限公司 | Media code rate self-adaptation method and apparatus, computer device, and storage medium |
| CN109756911A (en) * | 2019-01-31 | 2019-05-14 | 腾讯科技（深圳）有限公司 | Network quality prediction technique, business reorganization method, relevant device and storage medium |
| CN111491262A (en) * | 2020-04-26 | 2020-08-04 | 中国信息通信研究院 | Method and device for determining signal strength of mobile broadband network |
| CN113518250A (en) * | 2020-08-07 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Multimedia data processing method, device, equipment and readable storage medium |
| CN112383823A (en) * | 2020-10-16 | 2021-02-19 | 北京奇艺世纪科技有限公司 | Code rate self-adaption method and device, electronic equipment and storage medium |
| CN112272299A (en) * | 2020-11-27 | 2021-01-26 | 广州市网星信息技术有限公司 | Video coding method, device, equipment and storage medium |
| CN112953922A (en) * | 2021-02-03 | 2021-06-11 | 西安电子科技大学 | Self-adaptive streaming media control method, system, computer equipment and application |
Non-Patent Citations (5)
| Title |
|---|
| Improved support for dynamic adaptive streaming over HTTP in 3GPP (3GPP TR 26.938 version 14.0.0 Release 14).ETSI TR 126 938.2017,(第V14.0.0期),全文. * |
| LTE ; Packet-switched Streaming Service (PSS) * |
| RTR/TSGS-0426938ve00.TECHNICAL REPORT Universal Mobile Telecommunications System (UMTS) * |
| 基于丢包率统计的无线流媒体码率动态适配技术的研究;李婵文;;中国新通信(第04期);全文 * |
| 移动流媒体动态自适应速率选择算法研究;李陶深;蒋波;葛志辉;;广西大学学报(自然科学版)(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114024946A (en) | 2022-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9237476B2 (en) | Methods and apparatus for providing information indicative of traffic delay of a wireless link | |
| US20100020776A1 (en) | Wireless network-based location approximation | |
| US7328021B1 (en) | Method and system for using a measure of spectral interference to determine whether to output signals to a wireless link | |
| US20140143629A1 (en) | Wireless Access Point Mapping | |
| US8447327B2 (en) | Method and apparatus for a buffering scheme for OTDOA based location positioning | |
| US20040214565A1 (en) | Wireless communications apparatus and wireless communications system | |
| KR20180058744A (en) | Broadcast of distance measurement messages for WLAN RTT measurements | |
| KR20110113783A (en) | Methods and apparatus for providing a handover control system associated with a wireless communication network | |
| US20130195089A1 (en) | Super long range (slr) devices and low rate enabler (lre) devices | |
| WO2015077987A1 (en) | Method, base station and user equipment for selecting and configuring transmission mode | |
| US20050213601A1 (en) | Method and apparatus to provide hidden node protection | |
| CN109981214B (en) | Transmission control method and device | |
| CN114679733A (en) | Signal reporting method, terminal equipment and network equipment | |
| US20070183390A1 (en) | Determination and use of adaptive thresholds for received messages | |
| EP3475721A1 (en) | Combined fine timing measurement (ftm) and non-ftm messaging for position determination | |
| US9860084B2 (en) | Techniques for managing sounding intervals of a wireless communications device | |
| CN114024946B (en) | Method and device for adjusting streaming media code rate and computer readable storage medium | |
| JP2008153821A (en) | Radio communication system, terminal, base station, and control program | |
| US10439785B2 (en) | Distance measurement adjustment at a mobile device | |
| CN111886905A (en) | Method and apparatus for power control for in-device coexistence | |
| CN109644059B (en) | Method for measurement, terminal equipment and network equipment | |
| CN113453145B (en) | Method for predicting air interface state and wireless coverage equipment | |
| WO2022147799A1 (en) | Method for determining congestion level, terminal device, and computer storage medium | |
| US7602755B2 (en) | Method and apparatus of scanning control signal | |
| US8565784B2 (en) | Signal level measurement for mobile positioning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |