CN112437356B

CN112437356B - Streaming media data processing method and device

Info

Publication number: CN112437356B
Application number: CN202011271272.7A
Authority: CN
Inventors: 邓练兵; 欧阳可佩; 文少杰
Original assignee: Zhuhai Dahengqin Technology Development Co Ltd
Current assignee: Zhuhai Dahengqin Technology Development Co Ltd
Priority date: 2020-11-13
Filing date: 2020-11-13
Publication date: 2021-09-28
Anticipated expiration: 2040-11-13
Also published as: CN112437356A

Abstract

The embodiment of the invention provides a streaming media data processing method and a device, which are used for receiving a client request, wherein the client request comprises Uniform Resource Locator (URL) information and a URL suffix parameter; when the URL information is matched with the mark information and the URL suffix parameter is empty, sending the video frame corresponding to the mark information to the client; when the URL information is matched with the mark information and the URL suffix parameter is not empty, sending the video frame corresponding to the mark information and the first video file corresponding to the URL suffix parameter to the client; and when the URL information is not matched with the mark information, sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client. The embodiment of the invention uniformly controls the streaming media through the server, reasonably applies the broadband and reduces the phenomena of mosaic and pause of the video.

Description

Streaming media data processing method and device

Technical Field

The present invention relates to the field of streaming media technologies, and in particular, to a streaming media data processing method and a streaming media data processing apparatus.

Background

The streaming media technology is a technology in which multimedia data is transmitted from a server to a client via a network in a streaming manner, and the client can receive the multimedia data stream and play the multimedia data stream. At present, in a conventional video monitoring system, an independent streaming media server in the system or a streaming media function module in a certain device is used to copy and distribute video streams, so that a video client is decoded and played, and the video is decoded and put on the wall, while a storage server or a storage function module in the system acquires videos forwarded by the streaming media server to realize video storage.

In the prior art, a Real-Time Transport Protocol (RTP)/Real-Time Streaming Protocol (RTSP) Streaming technology is often adopted to realize Streaming media transmission, the RTP/RTSP Streaming technology is usually based on RTP/Datagram RTSP Protocol UDP (User Datagram Protocol) transmission, and a server pushes a code stream according to a coding rate. However, the RTP/RTSP streaming technology has poor adaptability to the network, and the loss of video packets is easily caused by the packet loss and jitter of the network, which further causes the phenomenon of mosaic and pause of the video, and is not beneficial to practical application.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are proposed to provide a streaming media data processing method and a corresponding streaming media data processing apparatus that overcome or at least partially solve the above problems.

In order to solve the above problems, an embodiment of the present invention discloses a streaming media data processing method, which is applied to a streaming media server, wherein the streaming media server is connected to a client, and the server stores a video frame, a first video file and a second video file, and the video frame has mark information; the method comprises the following steps:

receiving a client request, wherein the client request comprises Uniform Resource Location (URL) information and a URL suffix parameter;

when the URL information is matched with the mark information and the URL suffix parameter is empty, sending the video frame corresponding to the mark information to the client;

when the URL information is matched with the mark information and the URL suffix parameter is not empty, sending the video frame corresponding to the mark information and the first video file corresponding to the URL suffix parameter to the client;

and when the URL information is not matched with the mark information, sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client.

Optionally, the streaming media server stores a video recording composition command subfunction and a composition linked list, and the method further includes:

receiving a video synthesis command;

adding the video synthesis command to the synthesis linked list according to the video synthesis command subfunction;

and acquiring a synthesis task from the synthesis linked list, and synthesizing a video file according to the synthesis task.

Optionally, the streaming media server stores a frame rate parameter, and the step of sending the video frame corresponding to the tag information and the first video file corresponding to the URL suffix parameter to the client includes:

according to the client request, increasing a frame rate parameter on the first video file;

and adjusting the transmission rate of the first video according to the frame rate parameter so as to change the playing rate of the client.

Optionally, the method further comprises:

and performing packet processing on the video frame.

Optionally, the streaming media server is deployed in a cascade manner or in a distributed manner.

Optionally, a distributor is disposed between the streaming media server and the client; the step of receiving the client request includes:

and receiving the client request forwarded by the distributor according to the preset load balancing rule.

Optionally, the first video file, the second video file, and the video frame are forwarded to the client by the distributor.

The embodiment of the invention also discloses a streaming media data processing device which is applied to a streaming media server, wherein the streaming media server is connected with a client, the server stores video frames, a first video file and a second video file, and the video frames have mark information; the device comprises:

the system comprises a first receiving module, a second receiving module and a third receiving module, wherein the first receiving module is used for receiving a client request which comprises Uniform Resource Location (URL) information and a URL suffix parameter;

the first sending module is used for sending the video frame corresponding to the mark information to the client when the URL information is matched with the mark information and the URL suffix parameter is empty;

the second sending module is used for sending the video frame corresponding to the marking information and the first video file corresponding to the URL suffix parameter to the client when the URL information is matched with the marking information and the URL suffix parameter is not empty;

and the third sending module is used for sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client when the URL information is not matched with the mark information.

The embodiment of the invention also discloses an electronic device, which comprises a processor, a memory and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, the steps of the streaming media data processing method are realized.

The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the streaming media data processing method are realized.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, a client request is received, wherein the client request comprises Uniform Resource Location (URL) information and a URL suffix parameter; when the URL information is matched with the mark information and the URL suffix parameter is empty, sending the video frame corresponding to the mark information to the client; when the URL information is matched with the mark information and the URL suffix parameter is not empty, sending the video frame corresponding to the mark information and the first video file corresponding to the URL suffix parameter to the client; and when the URL information is not matched with the mark information, sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client. The streaming media is stored and forwarded by the streaming media server, so that the secondary distribution of the video content is facilitated. The streaming media server performs unified control on streaming media, reasonably applies broadband, and reduces the phenomena of mosaic and pause of videos. The streaming media server supports a load balancing function, dynamically and uniformly loads the client requests to each streaming media server, and the streaming media processing efficiency is improved.

Drawings

FIG. 1 is a schematic diagram of a streaming media server according to an embodiment of the invention;

FIG. 2 is a flow chart illustrating the steps of a method for processing streaming media data according to an embodiment of the present invention;

fig. 3 is a block diagram of a streaming media data processing apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

Referring to fig. 1, a schematic diagram of a streaming media server according to an embodiment of the present invention is shown.

The streaming media server 101 has a plurality of threads, which at least include: a configuration management thread 102, a collection thread 103, a video recording thread 104 and an rtsp service thread 105.

The configuration management thread 102 can communicate with the outside, and is the only thread for the streaming media server to communicate with the outside. For example: receiving an external configuration export command, a configuration import command, a video synthesis command, and the like, and setting by a person skilled in the art according to actual requirements, which is not limited in the embodiment of the present invention; in addition, the configuration management thread may also receive configuration information sent by each internal thread, for example, the registration camera configuration item sent by the acquisition thread, the registration video recording parameter and the video recording composition parameter sent by the video recording thread, and those skilled in the art may set the configuration information according to actual requirements, which is not limited in the embodiment of the present invention.

The collection thread 103 may collect videos shot by each camera, generate an information file of the camera, and send the collected videos to the video recording module.

The video recording thread 104 may receive the video sent by the acquisition thread and synthesize the video, so that the server supports the video synthesizing function.

The rtsp service thread 105 can forward the real-time synthesized video of the video recording thread, so that the server supports the real-time preview function of a plurality of clients; videos stored in the server can also be acquired, so that the server supports the playback preview function of a plurality of clients.

The streaming media server also has a disk management process 106, which can monitor the update status of the camera information file generated by the acquisition thread and update the configuration of the server.

The streaming server also supports an FTP unit 107(File Transfer Protocol) so that a user downloads a plurality of videos in the streaming server through a client.

The streaming media service designs and configures the management thread for external communication, and other threads do not participate in the external communication, so that the streaming media server framework is simplified. And the single-thread design is fully used, so that the resource overhead and response delay caused by the creation of the thread are avoided, and the complicated design of a thread pool is also omitted. The setting of all parameters of the streaming media server supports standard json (JavaScript Object Notation) or xml (Extensible Markup Language), and facilitates the integration and development of a third party.

Referring to fig. 2, a flowchart illustrating steps of an embodiment of a streaming media data processing method according to the present invention is shown, where the embodiment of the present invention is applied to a streaming media server, the streaming media server is connected to a client, and the server stores video frames, a first video file and a second video file, where the video frames have mark information; the method may specifically comprise the steps of:

step 201, receiving a client request, wherein the client request comprises Uniform Resource Locator (URL) information and a URL suffix parameter;

and receiving a client request sent by a client by an rtsp service thread in the streaming media server, wherein the client request comprises Uniform Resource Location (URL) information and URL suffix information.

The uniform resource locator URL information is a representation method for locating information positions, and consists of three parts: resource type, host domain name storing resource, resource file name.

The URL suffix information refers to a positioning parameter attached after URL information is uniformly resource-positioned, and may be null.

For example, the client request includes "http:// abc123. com/pfp/ask/jk.avi", where "http:// abc123. com" is URL location information and "/pfp/ask/jk.avi" is URL suffix information.

and step S11, receiving the client request forwarded by the distributor according to the preset load balancing rule.

The streaming server may employ a distributor-based load balancing mechanism. The load balancing mechanism based on the distributor is redirection distribution of IP (Internet Protocol)/TCP (Transmission Control Protocol)/HTTP (HyperText Transfer Protocol). A special front-end node, called dispatcher (dispatcher), is required. All client requests pass through the distributor and are distributed by it to the back-end streaming server for processing. Such distributor-based request distribution mechanisms are typically transparent to the client. The embodiment of the invention adopts a relay mechanism or a TCP (transmission control protocol) transmission mechanism, after a client request reaches a distributor, the distributor transmits the request to the selected streaming media server according to a determined load distribution algorithm, and the streaming media server receives the client request forwarded by the distributor according to a preset load balancing rule and processes the client request.

Step 202, when the URL information is matched with the tag information and the URL suffix parameter is empty, sending the video frame corresponding to the tag information to the client;

after receiving the client request, the rtsp service thread matches the URL information in the client request with the video frame mark information transmitted by the video recording thread, and when the URL information is matched with the video frame mark information and the URL suffix parameter is null, the video frame corresponding to the video frame mark information is directly transmitted to the client, so that the real-time forwarding function is realized.

Step 203, when the URL information is matched with the tag information and the URL suffix parameter is not empty, sending the video frame corresponding to the tag information and the first video file corresponding to the URL suffix parameter to the client;

after receiving the client request, the rtsp service thread matches the URL information in the client request with the video frame mark information transmitted by the video recording thread, when the URL information is matched with the video frame mark information and the URL suffix parameter is not empty, the client request is considered as a request for on-time on-demand, at the moment, the most matched video file is searched in a corresponding directory according to the URL suffix parameter, the most matched video file is read from a hard disk to serve as a first video file, and then the video frame corresponding to the first video file and the mark information is transmitted to the client, so that the function of on-time on-demand is realized.

Optionally, the streaming media server stores a frame rate parameter, and the step 203 includes:

step S21, adding a frame rate parameter to the first video file according to the client request;

step S22, adjusting the transmission rate of the first video according to the frame rate parameter, so as to change the playing rate of the client.

The frame rate parameter (Frames Per second, number of transmission Frames Per second) refers to the number of Frames in which an image is filled Per second.

When the client requests a command of variable-speed playing, a frame rate parameter is added to the first video file for sending, the maximum speed for sending the first video is adjusted to the speed transmitted by the client, and the sending can be suspended when the client cannot receive the first video, so that slow playing and single-step playing can be realized.

And 204, when the URL information is not matched with the mark information, sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client.

After receiving the client request, the rtsp service thread matches the URL information in the client request with the video frame mark information transmitted by the video recording thread, and when the URL information is not matched with the video frame mark information, the client request is considered to be a file on-demand request, a video is read from the hard disk to serve as a second video file, and the second video file and the video frame corresponding to the mark information are transmitted to the client, so that the file on-demand function is realized.

step S31, receiving a video synthesis command;

step S32, adding the video synthesis command to the synthesis linked list according to the video synthesis command subfunction;

and step S33, acquiring a synthesis task from the synthesis linked list, and synthesizing a video file according to the synthesis task.

When the configuration management thread receives the video synthesis command, the video synthesis command subfunction registered in advance by the video thread is called, the video synthesis command is added into the synthesis linked list of the video thread, and when the existing video on the hard disk meets the synthesis requirement, the video thread can take out the synthesis task from the synthesis linked list and synthesize the video.

Optionally, the method further comprises:

step S41, performing a packetization process on the video frame.

The streaming media server can perform packet processing on the video frames which need to be sent to the client, that is, the video frames are encapsulated according to the IP protocol, so that the video frames can be transmitted in the network.

The plurality of streaming media servers support cascade deployment and distributed deployment. Based on RTSP protocol and DSS (real-time Streaming Server), the method is realized and has equipment access and control and cascade or distributed Streaming media Server deployment.

When the embodiment of the present invention adopts the relay mechanism, the streaming media server will process the result, for example: the first video file, the second video file and the video frame are transmitted back to the distributor and then forwarded to the client side by the distributor.

According to the embodiment of the invention, a client request is received, wherein the client request comprises Uniform Resource Location (URL) information and a URL suffix parameter; when the URL information is matched with the mark information and the URL suffix parameter is empty, sending the video frame corresponding to the mark information to the client; when the URL information is matched with the mark information and the URL suffix parameter is not empty, sending the video frame corresponding to the mark information and the first video file corresponding to the URL suffix parameter to the client; and when the URL information is not matched with the mark information, sending the video frame corresponding to the mark information and the second video file corresponding to the URL information to the client. The streaming media is stored and forwarded by the streaming media server, so that the secondary distribution of the video content is facilitated. The streaming media server performs unified control on streaming media, reasonably applies broadband, and reduces the phenomena of mosaic and pause of videos. The streaming media server supports a load balancing function, dynamically and uniformly loads the client requests to each streaming media server, and the streaming media processing efficiency is improved.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 3, a block diagram of a streaming media data processing apparatus according to an embodiment of the present invention is shown, where the embodiment of the present invention is applied to a streaming media server, the streaming media server is connected to a client, and the server stores video frames, a first video file and a second video file, and the video frames have mark information; the device may specifically include the following modules:

a first receiving module 301, configured to receive a client request, where the client request includes URL information and URL suffix parameters;

a first sending module 302, configured to send, when the URL information matches the tag information and the URL suffix parameter is empty, a video frame corresponding to the tag information to the client;

a second sending module 303, configured to send, when the URL information matches the tag information and the URL suffix parameter is not empty, a video frame corresponding to the tag information and a first video file corresponding to the URL suffix parameter to the client;

a third sending module 304, configured to send, to the client, the video frame corresponding to the tag information and the second video file corresponding to the URL information when the URL information is not matched with the tag information.

the second receiving module is used for receiving the video synthesis command;

an adding module, configured to add the video composition command to the composition linked list according to the video composition command subfunction;

and the synthesis module is used for acquiring a synthesis task from the synthesis linked list and synthesizing a video file according to the synthesis task.

Optionally, the second sending module includes:

a frame rate sub-module, configured to add a frame rate parameter to the first video file according to the client request;

and the adjusting module is used for adjusting the transmission rate of the first video according to the frame rate parameter so as to change the playing rate of the client.

Optionally, the apparatus further comprises:

and the packaging module is used for packaging the video frame.

Optionally, a distributor is disposed between the streaming media server and the client; the first receiving module comprises:

and the distribution receiving submodule is used for receiving the client request forwarded by the distributor according to the preset load balancing rule.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

An embodiment of the present invention further provides an electronic device, which includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and when executed by the processor, the computer program implements the steps of the streaming media data processing method described above.

An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the streaming media data processing method as described above.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The foregoing describes in detail a streaming media data processing method and a streaming media data processing apparatus provided by the present invention, and specific examples are applied herein to explain the principles and embodiments of the present invention, and the descriptions of the foregoing examples are only used to help understand the method and the core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A streaming media data processing method is characterized in that the streaming media data processing method is applied to a streaming media server, the streaming media server is connected with a client, a video frame, a first video file and a second video file are stored in the streaming media server, and the video frame is a real-time video file; the video frame is provided with mark information, and the mark information is a storage address of the video frame on the streaming media server; the method comprises the following steps:

2. The method of claim 1, wherein the streaming media server stores a video cassette composition command subfunction and a composition linked list, the method further comprising:

receiving a video synthesis command;

3. The method according to claim 1, wherein the streaming media server stores a frame rate parameter, and the step of sending the video frame corresponding to the tag information and the first video file corresponding to the URL suffix parameter to the client comprises:

4. The method of claim 1, further comprising:

and performing packet processing on the video frame.

5. The method according to any one of claims 1 to 4, wherein the streaming media server is deployed in a cascade or distributed manner.

6. The method according to claim 5, wherein a distributor is arranged between the streaming media server and the client; the step of receiving the client request includes:

7. The method of claim 6,

the first video file, the second video file and the video frame are forwarded to the client by the distributor.

8. The streaming media data processing device is applied to a streaming media server, the streaming media server is connected with a client, a video frame, a first video file and a second video file are stored in the streaming media server, and the video frame is a real-time video file; the video frame is provided with mark information, and the mark information is a storage address of the video frame on the streaming media server; the device comprises:

9. An electronic device, comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program, when executed by the processor, implementing the steps of the streaming media data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the streaming media data processing method according to any one of claims 1 to 7.