CN106230839B

CN106230839B - Receiving control method and device for real-time streaming transmission

Info

Publication number: CN106230839B
Application number: CN201610633869.9A
Authority: CN
Inventors: 王雄才
Original assignee: Hisense Broadband Multimedia Technology Co Ltd
Current assignee: Hisense Broadband Multimedia Technology Co Ltd
Priority date: 2016-08-03
Filing date: 2016-08-03
Publication date: 2020-02-07
Anticipated expiration: 2036-08-03
Also published as: CN106230839A

Abstract

The disclosure discloses a receiving control method and device for real-time streaming transmission. The method comprises the following steps: the control command receiving and sending thread is triggered to read data and then monopolizes a data receiving channel in real-time streaming transmission; receiving data through an exclusive data receiving channel, wherein media data exist in the data; in the data receiving process, the control command receiving and sending thread reads the control command and the media data, and the media data is cached to wait for the data reading thread to read; and releasing the exclusive data receiving channel when the reading of the control command is completed. The control command receiving and sending thread reads the media data and the control command, and caches the media data to wait for the data reading thread to read, so that the situation that data disorder occurs in data reading when the same data receiving channel receives the media data and the control command under a multithreading mechanism can be effectively avoided, and the data reading thread and the control command receiving and sending thread can read the required data respectively.

Description

Receiving control method and device for real-time streaming transmission

Technical Field

The present disclosure relates to the field of transmission control technologies, and in particular, to a receiving control method and apparatus for real-time streaming transmission.

Background

Real-Time Streaming, i.e. data transmission implemented by using the RTSP Protocol (Real Time Streaming Protocol), is mainly implemented by controlling media data transmission through the RTSP Protocol, which is implemented by an application layer in media data transmission. In the network layer of media data Transmission, the Transmission Control Protocol (TCP), User Datagram Protocol (UDP) or other protocols are mostly used.

In the transmission of media data, the transmitted data includes the transmission of control commands in addition to the media data itself. In the media data transmission based on the TCP protocol, for the receiving end, the control command reading and the media data reading are performed on the same port, that is, the control command and the media data are received by the same port, so that the control command and the media data are read at the same port, which causes a mixed transmission of the control command and the media data, that is, data interleaving in the RTSP standard.

The existing RTSP protocol is implemented in the transmission of media data based on a single thread mechanism. Under the single-thread mechanism, the execution process is simple, and only data reading is needed to be carried out in sequence. For example, when media data is read, only the data length needs to be analyzed from a specific character according to the specification of RTSP standard on the interleaved data, and corresponding data is read according to the analyzed data length.

However, in the case of multiple threads, the implementation process will fail, resulting in a data disorder.

Specifically, since the control command and the media data are transmitted on the same port, that is, the same data receiving channel, in the process of executing media data reading by the thread responsible for reading the media data, if the server returns the control command, the media data cannot be read, but the control command is read.

At this time, the thread responsible for reading the control command cannot read the control command returned by the server, which causes the data in the system to be disturbed.

Disclosure of Invention

In order to solve the technical problem of data disorder occurring when data reading is performed in multiple threads in the related art, the present disclosure provides a reception control method and apparatus for real-time streaming.

A reception control method of real-time streaming, comprising:

the control command receiving and sending thread is triggered to request a data receiving channel in real-time streaming transmission to be monopolized before sending data;

receiving data through the exclusive data receiving channel, wherein media data exist in the data;

in the data receiving process, the control command receiving and sending thread reads the control command and the media data, and caches the media data to wait for the data reading thread to read;

and releasing the exclusive data receiving channel when the reading of the control command is completed.

A reception control apparatus for real-time streaming, comprising:

the command receiving and dispatching thread monopolizing module is used for controlling the command receiving and dispatching thread to monopolize a data receiving channel in the real-time streaming transmission before being triggered to request data transmission;

the data receiving module is used for receiving data through the exclusive data receiving channel, and media data exist in the data;

the read cache module is used for reading the control command and the media data by the control command receiving and sending thread in the data receiving process, and caching the media data to wait for the reading of the data reading thread;

and the exclusive releasing module is used for releasing the exclusive data receiving channel when the control command is read.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the data receiving end of real-time streaming transmission, after the control command transceiving thread is triggered to read data, the control command transceiving thread monopolizes the data receiving channel in real-time streaming transmission, receives data through the monopolized data receiving channel, inevitably receives media data in the data receiving channel along with the progress of data reception, and controls commands required by the control command transceiving thread, so that the control command transceiving thread reads the media data and the control commands on one hand, and caches the media data to wait for the data reading thread to read, thereby effectively avoiding the situation that the data reading of the control command transceiving thread and the data reading thread is disordered when the same data receiving channel receives the media data and the control commands under a multi-thread mechanism, and further, when the same data receiving channel receives the media data and the control command, the data reading thread and the control command receiving and sending thread can read the required data respectively.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic illustration of an implementation environment according to the present disclosure;

FIG. 2 is a block diagram illustrating an apparatus in accordance with an exemplary embodiment;

FIG. 3 is a flow chart illustrating a method of reception control for real-time streaming according to an exemplary embodiment;

FIG. 4 is a flow chart of the step of receiving data through an exclusive data receiving channel in the corresponding embodiment of FIG. 3;

fig. 5 is a flowchart of a step of performing a read of a control command and media data by a control command transceiving thread, buffering the media data, and waiting for a read of a data read thread in data reception performed in the corresponding embodiment of fig. 3;

FIG. 6 is a diagram illustrating the existence of data during data transmission in accordance with an exemplary embodiment;

FIG. 7 is a flow chart illustrating a method of reception control for real-time streaming according to an exemplary embodiment;

FIG. 8 is a diagram illustrating a format of media data in accordance with an exemplary embodiment;

FIG. 9 is a schematic diagram illustrating sequential execution in a data receiving end in accordance with an illustrative embodiment;

FIG. 10 is an idealized model diagram illustrating data reading under a multi-threaded scheme in accordance with an illustrative embodiment;

FIG. 11 is a schematic diagram illustrating data reads that do not meet an ideal model under a multi-threaded scheme in accordance with an illustrative embodiment;

FIG. 12 is a data read schematic under a multi-threaded scheme, shown in accordance with an illustrative embodiment;

FIG. 13 is a diagram illustrating the operation of a control command transceiving thread and a media data read thread in accordance with an illustrative embodiment;

FIG. 14 is a timing diagram illustrating the operation of a control command transceiving thread and a media data read thread in accordance with an illustrative embodiment;

FIG. 15 is a timing diagram illustrating the operation of a control command transceiving thread and a media data reading thread in accordance with another illustrative embodiment;

FIG. 16 illustrates a receive control apparatus for real-time streaming according to an exemplary embodiment;

FIG. 17 is a block diagram of a data receiving module of the corresponding embodiment of FIG. 16;

FIG. 18 is a block diagram of a read cache module of the corresponding embodiment of FIG. 16;

fig. 19 is a block diagram illustrating a reception control apparatus for real-time streaming according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

FIG. 1 is a schematic illustration of an implementation environment according to the present disclosure. The implementation environment includes: at least one data receiver 110 and a server 130.

The association between the data receiver 110 and the server 130 includes a network association and/or a protocol of hardware, and a data association therebetween.

Fig. 2 is a block diagram illustrating an apparatus 200 according to an example embodiment. For example, the apparatus 200 may be the data receiving end 110 in the implementation environment shown in FIG. 1. The data receiving end 110 may be, for example, a set-top box and a player of a smart television.

Referring to fig. 2, the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, a sensor component 214, and a communication component 216.

The processing component 202 generally controls overall operation of the device 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations, among others. The processing components 202 may include one or more processors 218 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 202 can include one or more modules that facilitate interaction between the processing component 202 and other components. For example, the processing component 202 can include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

The memory 204 is configured to store various types of data to support operations at the apparatus 200. Examples of such data include instructions for any application or method operating on the apparatus 200. The memory 204 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. Also stored in memory 204 are one or more modules configured to be executed by the one or more processors 218 to perform all or a portion of the steps of performing any of the methods illustrated in fig. 3, 4, 5, and 7, as described below.

The power supply component 206 provides power to the various components of the device 200. The power components 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 200.

The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

The audio component 210 is configured to output and/or input audio signals. For example, audio component 210 includes a Microphone (MIC) configured to receive external audio signals when apparatus 200 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 204 or transmitted via the communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

The sensor component 214 includes one or more sensors for providing various aspects of status assessment for the device 200. For example, the sensor assembly 214 may detect an open/closed state of the device 200, the relative positioning of the components, the sensor assembly 214 may also detect a change in position of the device 200 or a component of the device 200, and a change in temperature of the device 200. In some embodiments, the sensor assembly 214 may also include a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 216 is configured to facilitate wired or wireless communication between the apparatus 200 and other devices. The device 200 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the methods described below.

Fig. 3 is a flow chart illustrating a method of reception control for real-time streaming according to an exemplary embodiment. The reception control method of real-time streaming is applied to the data receiving terminal 110 shown in fig. 1. As shown in fig. 3, the receiving control method of real-time streaming may be performed by the data receiving end 110 and may include the following steps.

In step 310, the control command transceiving thread itself is triggered to monopolize the data receiving channel in the real-time streaming before requesting data transmission.

The control command receiving and sending thread may be any thread that needs to read a control command at a data receiving end. The data receiving channel in the real-time streaming transmission is used for realizing media data transmission in the real-time streaming transmission and control command transmission alternately performed with the media data transmission. That is, the data transmitted in the data receiving channel is the interleaved data.

After a control command receiving and sending thread is triggered to execute the data reading process, the control command receiving and sending thread applies for the control right of the data receiving channel to monopolize the data receiving channel.

Specifically, after the control command transceiving thread is triggered to execute the data reading process, the control right of the data receiving channel is firstly applied to request for monopolizing the data receiving channel, whether the monopolizing of the data receiving channel is successful or not is judged, and if the monopolizing of the data receiving channel is not successful, the application of the control right of the data receiving channel is continued until the data receiving channel is monopolized.

For the control command transceiving thread which monopolizes the data receiving channel, the control command transceiving thread can read data transmitted in the subsequent data receiving channel; and for other threads, the data reading operation is prohibited, thereby providing protection for the execution of the thread data reading.

In an exemplary embodiment, in the data receiving end, there are a plurality of threads that can read data in the data receiving channel, and the threads can be roughly divided into two categories, namely a control command transceiving thread and a data reading thread. The control command transceiving thread is a thread for obtaining a control command in the data receiving end, and the media data reading thread is a thread responsible for obtaining media data.

It should be noted that, in the real-time streaming transmission performed by the data receiving end, the running threads include two major types, namely a control command transceiving thread and a media data reading thread, and the number of each type of threads can be flexibly adjusted according to actual operation needs.

In step 330, data reception is performed through the exclusive data reception channel, and the media data exists in the data.

And receiving the data transmitted by the server at a network layer where the exclusive data receiving channel is positioned. For the control command transceiving thread of the exclusive data receiving channel, it is to obtain the control command from the data receiving channel.

However, since media data is inevitably received before a control command is received in a data reception channel exclusively owned by a control command transmission/reception thread, there is reception of media data in data reception performed by a data reception channel exclusively owned by a control command transmission/reception channel.

In step 350, during the data receiving, the control command transceiving thread reads the control command and the media data, and buffers the media data to wait for the data reading thread to read.

The control command transceiving thread reads the data received by the network layer, namely the exclusive data receiving channel. It is understood that in the real-time streaming transmission performed during the period that the control command transceiving thread monopolizes the data receiving channel, the received data includes, on one hand, the media data and the control command to be transmitted subsequently; on the other hand, the received data is media data, and the received data may also be a control command.

Therefore, during the period that the control command receiving and sending thread monopolizes the data receiving channel, a scene that the same data receiving channel sequentially receives the media data and the control command occurs, and in this scene, the control command receiving thread reads the control command and the media data and caches the read media data.

In step 370, the exclusive data reception channel is released upon completion of the reading of the control command.

After the thread performing data reading, that is, the control command transceiving thread completes the control command reading performed by the thread, the exclusive data receiving channel is released immediately, so that other threads can read data conveniently.

In the receiving control of the real-time streaming transmission, each thread alternatively monopolizes the data receiving channel, and then reads the received data in the data receiving channel, so that for the media data and the control command which alternately appear in the data receiving channel, the ordered reading of various data can be realized, and a protection mechanism formed during the data reading process also prevents the data which is originally required to be read by the thread from being read by other threads, and controls the normal operation of data receiving under a multithreading mechanism.

FIG. 4 is a depiction of the details of step 330, shown in accordance with an exemplary embodiment. The thread is a control command transceiving thread, and as shown in fig. 4, step 330 may include the following steps.

In step 331, the control command transmitting/receiving thread transmits a request, and waits for receiving data in the data receiving channel when the transmission of the request is completed.

The control command receiving and sending thread is used for reading the control command in the data receiving channel. The control command is not sent by the server actively, but after a request is sent to the server, the party can receive the control command returned by the server in response to the request. In an exemplary embodiment, the request sent to the server carries a sending command method, so that the server can respond to the request.

Specifically, the control command transceiving thread sends the request through the data sending channel, and the request from the data receiving end to the server is sent in the data sending channel.

In step 333, data reception is performed on the data reception channel, and the received data includes the requested control command.

When the server sends data, the data receiving channel receives the data. In this data reception, since the control command thread has already sent a request to the server, the data received by the data reception channel includes the control command sent by the server in response to the request.

It can be clearly seen that, for the control command transceiving thread, when the control command transceiving thread is in the exclusive data receiving channel, the sending of the request and the receiving of the control command are completed, and the whole process of sending the request until receiving the control command is protected, and is not limited to the stage of receiving the control command.

This is because if the control right of the data receiving channel is applied after the request is sent, the control command responded by the server is likely to be read by other threads in case of failure of the application, which may result in an error.

Therefore, when the control command transceiving thread needs to read the control command, the exclusive use of the data receiving channel should be realized before the request is sent, so as to avoid the occurrence of data reading errors.

In addition, the proportion of the data volume occupied by the request in the whole real-time streaming transmission is extremely low, most of the data volume is media data, and the response speed of the server to the request is extremely high, so that the short monopolization of the data receiving channel in the whole process from the request sending to the control command receiving does not influence the program performance.

In the embodiment corresponding to fig. 4, in combination with the embodiment corresponding to fig. 3, a process for implementing control command reception by a control command transceiving thread in a data receiving end is provided. One or more control command receiving and sending threads exist in the data receiving end, an implementation flow for implementing control command receiving by the control command receiving and sending threads is provided in the embodiment corresponding to fig. 3, and a more detailed specific implementation is provided for implementing data receiving after monopolizing the data receiving channel in fig. 3 in the embodiment corresponding to fig. 4, thereby providing a guarantee for data reading implemented by the control command receiving and sending threads.

FIG. 5 is a depiction of the details of step 350, shown in accordance with an exemplary embodiment. The transmission data is a control command, and as shown in fig. 5, step 350 may include the following steps.

In step 351, it is determined whether the received data is media data or not in response to the reception of the data in the data reception channel, and if not, step 353 is executed, and if yes, step 355 is executed.

As described above, the data received by the data receiving channel exclusive to the control command transceiving thread includes the control command returned by the server in response to the request.

The sending of the control command by the server is controlled by the control command thread, namely the server responds to the request sent by the control command thread to the server so as to return the control command; for media data, it is sent by the server to the data receiver on its own initiative.

Therefore, even if the control command transceiving thread monopolizes the data receiving channel, the data receiving channel still can receive the media data actively sent by the server. That is, the data received by the data receiving channel includes control commands and, in addition, media data.

When the control command receiving and sending thread reads the control command, the existence of the media data in the received data needs to be identified firstly, and then the media data existing in the data reading thread is cached on one hand, and the control command is read on the other hand.

In an exemplary embodiment, determining whether the received data is media data may be performed according to a start character of the media data.

In the media data, the first character, i.e. the header identifier is $ (0x24), so that the first character is found to be $ (0x24) in the received data, and the data where the character is located is the media data.

In step 353, a read of the control command is performed.

After the control command receiving and sending thread exclusive of the data receiving channel sends a request to the server, the control command is received in the data receiving channel, and the control command receiving and sending thread directly reads the control command.

In an exemplary embodiment, the start character of the control command is a specific character, for example, a character of R switch, such as RTSP, and the control command is terminated by a carriage return line change, that is, CRLF is an end mark, so that the data corresponding to the control command can be found in the received data according to the start character and the end mark of the control command, and then read.

In step 355, the media data is buffered to await reading by the data read thread.

And reading the media data and caching the media data for the existence of the media data in the data receiving channel exclusive to the control command transceiving thread.

That is, for the media data that arrives at the data receiving channel before the control command, the control command transceiving thread reads and buffers across threads, so as to avoid the situation of data reading error, and ensure the subsequent data reading of the media data reading thread.

In an exemplary embodiment, when the control command transceiving thread reads the media data, the control command transceiving thread caches the media data in a First Input First Output (FIFO) queue to realize pre-caching of the media data, and improves the efficiency of a subsequent media data reading thread for searching the media data under the action of the FIFO queue.

In an exemplary embodiment, the reading of the media data by the control command transceiving thread is performed according to the first character in the media data, i.e., all subsequent data beginning with the character $ (0x24) is read.

It should be further noted that, for the data received in the data receiving channel, although the control command and the media data exist in an interleaving manner, the control command and the media data are not mixed, that is, the control command and the media data are crossed but exist independently and completely.

Fig. 6 is a diagram illustrating existence of data in data transmission according to an exemplary embodiment. Such as

As shown in fig. 6, the media data 410 and the control command 430 are complete data segments that exist independently of each other. In the media data 410, the character $ (0x24) is its header 411. In the control command 430, the start character is a character starting with R, i.e., RTSP/1.0200 OK.

Therefore, the control command transceiving thread can read the media data 410 according to the header mark 411 of the data in the data receiving channel, and can also read the control command 430 according to the characters beginning with R.

In the embodiments implemented in fig. 3 and 4, in combination with the embodiment corresponding to fig. 5, in the implementation of exclusive use of the data receiving channel and reading of data in the data receiving channel performed by the control command transceiving thread, a specific implementation is provided for reading of the control command performed by the control command transceiving thread, that is, an implementation process of reading the control command and processing existing media data are provided for existence of interleaving of media data and a control command in received data, and further, control command reading of the control command transceiving thread is implemented under a multithreading mechanism.

Fig. 7 is a flow chart illustrating a method of reception control for real-time streaming according to an example embodiment. The thread triggered to read data is a media data reading thread, and as shown in fig. 7, the method for controlling reception of real-time streaming may include the following steps.

In step 510, after the data reading thread itself is triggered to read data, it is determined whether there is cached media data, if yes, step 530 is executed, and if no, step 550 is executed.

As described above, when the control command transceiving thread monopolizes the data receiving channel, the control command transceiving thread performs reading and caching processing on the media data existing in the data, so that other threads can read the cached media data in subsequent data reading.

The media data reading thread reads the media data in the data receiving terminal so as to provide the media data for the corresponding application. Therefore, when the media data reading thread is triggered to read data, whether the media data needed by the media data exists in the cached media data is judged firstly, if yes, the media data does not need to be read from the data receiving channel, and the cached media data is directly read.

In step 530, the buffered media data is read.

After the cached media data is judged to exist, the media data reading thread can directly read the media data, and at the moment, the data receiving channel does not need to be monopolized, and the data reading in the data receiving channel is not needed.

In step 550, the data read thread monopolizes the data receive channel in real-time streaming.

If the media data corresponding to the media data reading thread does not exist in the cache, the application of the control right of the data receiving channel needs to be carried out so as to monopolize the data receiving channel, and further, the media data in the data receiving channel can be read.

In the manner described above, a media data reading implementation process is provided for a media data reading thread in a data receiving end under a multithreading mechanism. In the implementation process, the media data reading thread and the control command transceiving thread are mutually matched, so that the receiving and reading of the media data and the control command are not disordered, and the condition of data reading errors is not caused.

Here, through the embodiment corresponding to fig. 7, in the embodiment that the media data reading thread constituted by fig. 1 and 5 reads the media data, a specific implementation is provided for the media data reading step, and further through the embodiment corresponding to fig. 7, the problem that the media data is read before reaching the data receiving channel when the media data is read with respect to the media data reading thread under the multi-thread mechanism is solved, so that the media data reading by the media data thread can normally operate under the multi-thread mechanism.

In addition, in combination with the embodiments respectively corresponding to fig. 4 and fig. 5, under the multithreading mechanism of the data receiving end, a specific implementation that the media data reading thread and the control command transceiving thread are mutually matched is provided, so that reading of interleaved data under the multithreading mechanism is ensured, data crosstalk is prevented, and system crash or media decoder abnormality is avoided.

In a reception control method for real-time streaming according to an exemplary embodiment, after step 370, the reception control method for real-time streaming may further include the following steps.

And judging whether the data reading performed by the media data reading thread is finished, if not, returning to the step 510, and if so, finishing.

If the data reading performed by the media data reading thread needs to be continued, that is, if there is still unread media data, then the process returns to the media data reading thread to perform the data reading implementation process, that is, returns to step 510.

As can be seen from the embodiments corresponding to fig. 1 and fig. 7, when the media data reading thread still needs to read the media data, the process returns to step 510 to determine the cached media data first, and if the cached media data does not exist for the media data reading thread, the media data needs to be read through the exclusive data receiving channel.

Through the embodiments respectively corresponding to fig. 3, fig. 4, fig. 5 and fig. 7, the interleaved data reading by the control command transceiving thread and the media data reading thread in cooperation with each other under the multi-thread mechanism is realized for the data receiving terminal, that is, the control command and the media data which are interleaved for transmission are accurately read between the control command transceiving thread and the media data reading thread, so that the reliability and efficiency of interleaved data reading under the multi-thread mechanism are improved.

The following describes the receiving control method of real-time streaming transmission by taking a data receiving end as a set top box and a server as an RTSP server as an example and combining a specific application scenario.

The transmission of media data is based on protocols such as TCP, UDP or RTP/RTCP, and the transmission control is implemented by a real-time streaming protocol, i.e. RTSP protocol.

The corresponding transmission modes are specifically exemplified as shown in the following table:

transmission mode	Description of the invention
		MP2T/TCP	Direct MPEG2-TS bearer using TCP without RTP encapsulation
MP2T/UDP	Direct MPEG2-TS over UDP without RTP encapsulation
		MP2T/RTP/TCP	MPEG2-TS bearer using TCP-based RTP
MP2T/RTP/UDP	Using UDP-based RTP to carry MPEG2-TS

MPEG2 represents audio/video data stream in TS stream format, and MPEG2-TS is a transmitted data stream, i.e. media data.

Here, the TCP protocol will be described as follows.

In the transmission of media data based on the TCP protocol, the control command is also implemented by using the TCP protocol, that is, the control command and the media data use the same port, that is, for the data receiving end, the control command and the media data exist in the same TCP data receiving channel, and there is a mixed transmission of the control command data and the media data.

In the transmission implementation, the transmitting and receiving process of the control command is controlled by sending a request carrying a command sending method to an RTSP server; and for media data transmission, the server sends actively,

in the RTSP standard, for the way of carrying media data based on TCP, the control command and the media data are allowed to interleave, i.e. both are transmitted through TCP, to achieve firewall penetration, and fig. 8 is a format of the media data shown according to an exemplary embodiment, in which a header identification 411 of 0x24 is included.

The transmission of control commands and media data will be interspersed and therefore the processes performed by the data receiving end, i.e. the set-top box, will also be performed sequentially. For example, fig. 9 is a schematic diagram illustrating sequential execution in a data sink according to an exemplary embodiment, i.e., read media data 610, send request 630, and read control command 650 are executed sequentially.

According to the implementation in the embodiment shown in fig. 9, in a multi-thread scheme, which is described here by taking two threads as an example, an ideal model as shown in fig. 10 will be obtained.

In fig. 10, the vertical axis represents the executed task, the horizontal axis represents the time axis, and the CPU time slice operation axis is represented. As can be seen from fig. 10, if the CPU time slice is divided exactly according to the complete cycle of one action, the division is consistent with the single thread scenario, and no problem occurs.

However, in practice, the result shown in FIG. 11 may occur because the allocation of CPU slots is determined entirely by the operating system's scheduling algorithm and does not satisfy the ideal model.

In fig. 11, media data 1 and media data 2 are a complete set of data, but are divided into 2 CPU slots for execution, and the transmission of requests and the reading of control commands are also divided into 2 CPU slots for execution.

On this basis, referring further to fig. 12, if the CPU time slice is changed as shown in fig. 12, a control command returned by the server is inserted into the CPU time slice between the read media data 1 and the read media data 2, and the media data 1, the control command and the media data 2 exist in an interleaved manner in the same TCP data receiving channel.

Because the control command is between the media data 1 and the media data 2, if the server returns the control command after the media data reading thread executes the media data 1, at this time, the possibility read by the media data reading thread is not the media data, but the control command; for the control command receiving and sending thread, the media data reading thread reads the control command, so the control command cannot be read, system disorder is caused, and further later command character analysis abnormity, even crash or media decoder abnormity is caused.

Therefore, it is necessary to implement the receiving control in the media data transmission based on the embodiments corresponding to fig. 3, fig. 4, fig. 5, and fig. 7 of the present disclosure, so that the accurate reading of the media data and the control command can be implemented in multiple mechanisms.

Fig. 13 is a flow diagram illustrating a receive control process for real-time streaming under a multi-threaded mechanism in accordance with an exemplary embodiment.

The control command transceiving thread shown in fig. 13 represents a type of thread that performs control command reading in the set-top box, and the media data reading thread represents a type of thread that performs media data reading in the set-top box.

According to the embodiments corresponding to fig. 3 and 4, the control command transceiving thread executes the processes shown in steps 810 to 840, and sends the request and receives the control command when the TCP data receiving channel is exclusive.

According to the embodiment shown in fig. 5, the control command transceiver thread performs the determination of whether media data exists, i.e., whether the data starts with 0x24, and the processes of steps 850 to 870 are performed.

When media data is read, it is stored in a FIFO queue.

For the media data reading thread, when it is triggered to perform data reading, the embodiments corresponding to fig. 1 and fig. 7 will be implemented.

Specifically, step 910 is executed to first determine whether the FIFO queue has media data that needs to be read, and if yes, the media data is directly read to the application, that is, step 920.

If not, the TCP data reception channel is monopolized, and step 930 and step 940 are performed. In this case, the media data may be received in the exclusive TCP data receiving channel, and the media data reading thread may read the media data directly from the TCP data receiving channel.

Here, through the process shown in fig. 13, the control command transceiving thread and the media data reading thread cooperate with each other to realize interleaved data reading under the multi-thread mechanism.

Further, the timing sequence corresponding to the embodiment shown in fig. 13 is shown in fig. 14 and fig. 15.

In the embodiment shown in fig. 13, the order in which the control command transceiving thread and the media data reading thread perform data reading is not set, but the data reading is performed separately according to the actual operation requirement.

Thus, fig. 14 shows that the control command transceiver thread performs data read prior to the media data read thread and performs cross-thread pre-buffering of the media data for the media data read thread.

At this time, the media data reading thread does not need to read the media data from the TCP data receiving channel, and only reads the media data from the FIFO queue.

Fig. 15 shows a timing procedure of performing data reading by the media data reading thread before the control command transceiving thread, in this procedure, since the control command transceiving thread does not perform pre-buffering of the media data, the media data reading thread needs to read the media data in an exclusive TCP data receiving channel.

The following is an embodiment of the apparatus of the present disclosure, which may be used to execute an embodiment of a reception control method for real-time streaming transmission executed by the data receiving terminal 110 of the present disclosure. For details not disclosed in the embodiments of the apparatus of the present disclosure, please refer to the embodiments of the receiving control method of real-time streaming transmission of the present disclosure.

Fig. 16 is a block diagram illustrating a reception control apparatus for real-time streaming according to an exemplary embodiment. The reception control apparatus for real-time streaming can be used in the data receiving end 110 in the implementation environment shown in fig. 1, and executes all the steps of the reception control method for real-time streaming shown in fig. 3. As shown in fig. 16, the receiving control device for real-time streaming includes but is not limited to: a command transceiving thread exclusive module 1210, a data receiving module 1230, a read cache module 1250, and an exclusive release module 1270.

The command receiving and dispatching thread exclusive module 1210 is configured to control the command receiving and dispatching thread to be triggered to monopolize a data receiving channel in the real-time streaming before requesting data transmission.

The data receiving module 1230 is configured to receive data through an exclusive data receiving channel, where media data exists in the data.

The read buffer module 1250 is configured to, during data receiving, control the command transceiving thread to read the control command and the media data, and buffer the media data to wait for the data reading thread to read the media data.

An exclusive releasing module 1270, configured to release an exclusive data receiving channel when the control command is read.

Fig. 17 is a depiction of details of a data receiving module, shown in accordance with an example embodiment. The data receiving module 1230, as shown in fig. 17, includes but is not limited to: a request transmitting unit 1231 and a reception executing unit 1233.

A request sending unit 1231, configured to send a request through the control command transceiving thread, and when the request sending is completed, the receiving execution unit 1233 waits for receiving the data in the data receiving channel.

And a receiving execution unit 1233, configured to receive data in the data receiving channel, where the received data includes the requested control command and the media data.

FIG. 18 is a depiction of details of a read cache module in accordance with an illustrative embodiment. The read cache module 1250, as shown in FIG. 18, includes but is not limited to: a media data judgment unit 1251, a buffer unit 1253, and a command reading unit 1255.

The media data determining unit 1251 is configured to determine whether the received data is media data along with the reception of the data in the data receiving channel, and if so, notify the buffer unit 1253, and if not, notify the command reading unit 1255.

The buffer unit 1253 is configured to buffer the media data to wait for the data reading thread to read.

A command reading unit 1255 for reading the control command.

Fig. 19 is a block diagram illustrating a reception control apparatus for real-time streaming according to an exemplary embodiment. The receiving control device for real-time streaming transmission, as shown in fig. 19, includes but is not limited to: a data read thread trigger module 1310, a cache read module 1330, and a data read thread exclusive module 1350.

The data reading thread triggering module 1310 is configured to determine whether cached media data exists after the data reading thread is triggered to perform data reading, notify the cache reading module 1330 if the cached media data exists, and notify the data reading thread to monopolize the module if the cached media data does not exist.

The buffer reading module 1330 is configured to read the buffered media data.

A data read thread exclusive module 1350, configured to enable the data read thread to exclusive the data receiving channel in the real-time streaming.

Optionally, the present disclosure further provides a reception control apparatus for real-time streaming, where the reception control apparatus for real-time streaming may be used in the data receiving terminal 110 in the implementation environment shown in fig. 1, and execute all or part of the steps of the reception control method for real-time streaming shown in any one of fig. 3, 4, 5, and 7. The device comprises:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform:

The specific manner in which the processor of the apparatus in this embodiment performs the operation has been described in detail in the embodiment related to the reception control method of the real-time streaming, and will not be elaborated upon here.

In an exemplary embodiment, a storage medium is also provided that is a computer-readable storage medium, such as may be transitory and non-transitory computer-readable storage media, including instructions. The storage medium refers to, for example, the memory 204 including instructions executable by the processor 218 of the apparatus 200 to perform the reception control method of the real-time streaming.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A reception control method of real-time streaming, comprising:

2. The method of claim 1, wherein the step of receiving data through an exclusive data reception channel comprises:

sending a request through the control command receiving and sending thread, and waiting for receiving the data in the data receiving channel when the request is sent;

and receiving data in the data receiving channel, wherein the received data comprises the requested control command and the media data.

3. The method according to claim 1, wherein the step of the control command transceiving thread performing reading of the control command and the media data during the ongoing data reception, and buffering the media data to wait for reading of the data reading thread comprises:

with the receiving of the data in the data receiving channel, judging whether the received data is media data, if so, caching the media data to wait for the reading of a data reading thread, and if not, then, caching the media data

And reading the control command.

4. The method of claim 1, further comprising:

and after the data reading thread is triggered to read data, judging whether cached media data exist or not, and if so, reading the cached media data.

5. The method of claim 4, further comprising:

and when the cached media data does not exist, the data reading thread monopolizes a data receiving channel in the real-time streaming transmission.

6. A reception control apparatus for real-time streaming, comprising:

7. The apparatus of claim 6, wherein the data receiving module comprises:

a request sending unit, configured to send a request through the control command transceiving thread, where the receiving and executing unit waits to receive data in the data receiving channel when the request sending is completed;

the receiving execution unit is used for receiving data in the data receiving channel, and the received data comprises requested control commands and media data.

8. The apparatus of claim 6, wherein the read cache module comprises:

the media data judging unit is used for judging whether the received data is media data or not along with the reception of the data in the data receiving channel, if so, the cache unit is informed, and if not, the command reading unit is informed;

the cache unit is used for caching the media data so as to wait for the reading of a data reading thread;

and the command reading unit is used for reading the control command.

9. The apparatus of claim 6, further comprising:

the data reading thread triggering module is used for judging whether cached media data exist or not after the data reading thread is triggered to read data, and if so, informing the cache reading module;

the cache reading module is used for reading the cached media data.

10. The apparatus of claim 9, further comprising:

and the data reading thread monopolizing module is used for judging that the data reading thread monopolizes the data receiving channel in the real-time streaming transmission when the cached media data does not exist.