CN111181965B - Audio processing method and device - Google Patents
Audio processing method and device Download PDFInfo
- Publication number
- CN111181965B CN111181965B CN201911400650.4A CN201911400650A CN111181965B CN 111181965 B CN111181965 B CN 111181965B CN 201911400650 A CN201911400650 A CN 201911400650A CN 111181965 B CN111181965 B CN 111181965B
- Authority
- CN
- China
- Prior art keywords
- audio
- audio streams
- channel
- decoding
- streams
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides an audio processing method and an audio processing device, wherein the method is applied to a video network and comprises the following steps: in the process of performing the cross-domain audio/video service, receiving multiple audio streams sent by a cross-domain audio/video server, marking decoding addresses corresponding to the multiple audio streams according to channel numbers carried in the multiple audio streams to ensure that the audio streams in the received multiple audio streams all have their own channel numbers and decoding addresses, decoding the multiple audio streams according to the decoding addresses to obtain decoded multiple audio streams, and finally playing the decoded multiple audio streams, so that the situation that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different can be avoided, that is, the audio streams can be normally played, dynamic processing of the multiple audio streams in the cross-domain audio/video service is realized, and the cross-domain audio/video service can be normally performed.
Description
Technical Field
The present invention relates to the field of video networking technologies, and in particular, to an audio processing method and apparatus.
Background
Because the video networking technology can realize real-time transmission of full-network high-definition audio and video, the video networking technology is increasingly applied to cross-domain (cross-server) audio and video services.
Currently, in a cross-domain audio/video service, such as a multi-party audio/video conference, a terminal needs to process audio streams across multiple domains, and when terminals in different domains are switched to serve as speakers, terminals corresponding to the received audio streams may have various different situations.
The possible situations mentioned above include: the audio stream addresses are the same but the audio stream channels are different, or the audio stream channels are the same but the audio stream addresses are different, both of which may cause the terminal to be unable to play audio normally.
Disclosure of Invention
In view of the above, embodiments of the present invention are proposed to provide an audio processing method and apparatus that overcome or at least partially solve the above-mentioned problems.
In order to solve the above problem, an embodiment of the present invention provides an audio processing method, where the method is applied to a video network, and the method includes:
receiving a plurality of paths of audio streams sent by a cross-domain audio and video server in the process of carrying out cross-domain audio and video service;
marking a decoding address corresponding to the multi-channel audio stream according to a channel number carried in the multi-channel audio stream;
decoding the multi-channel audio stream according to the decoding address to obtain a decoded multi-channel audio stream;
and playing the decoded multi-channel audio stream.
Optionally, the marking, according to the channel number carried in the multiple audio streams, a decoding address corresponding to the multiple audio streams includes:
when the number of the audio flow paths of the multi-path audio flow is less than or equal to the threshold number, marking the decoding address corresponding to the multi-path audio flow according to the channel number;
and when the number of the audio streams is greater than the threshold number, screening the multiple audio streams to obtain the audio streams with the threshold number, and marking the decoding addresses corresponding to the audio streams with the threshold number.
Optionally, the obtaining, by filtering the multiple audio streams, the threshold number of audio streams includes:
acquiring the receiving time of each audio stream in the multiple audio streams;
and screening the multiple audio streams according to the receiving time to obtain the audio streams with the threshold number.
Optionally, the obtaining, by filtering the multiple audio streams according to each of the receiving times, the threshold number of audio streams includes:
according to each receiving time, sequencing the multi-channel audio stream according to the sequence of the receiving time from the back to the first to obtain a sequencing result;
and acquiring the audio streams with the threshold number of the audio streams with the top sorting order from the multi-path audio streams based on the sorting result.
Optionally, the marking, according to the channel number carried in the multiple audio streams, a decoding address corresponding to the multiple audio streams includes:
saving the multi-channel audio stream to a cross-domain audio processor;
in the cross-domain audio processor, marking a decoding address corresponding to the multi-path audio stream.
Optionally, in the cross-domain audio processor, after marking a decoding address corresponding to the multiple audio streams, the method further includes:
and when a next multi-channel audio stream is received, emptying the multi-channel audio stream in the cross-domain audio processor and clearing the mark.
In order to solve the above problem, an embodiment of the present invention provides an audio processing apparatus, where the apparatus is applied to a video network, and the apparatus includes:
the receiving module is used for receiving a plurality of paths of audio streams sent by a cross-domain audio and video server in the process of performing cross-domain audio and video service;
the marking module is used for marking the decoding addresses corresponding to the multi-channel audio stream according to the channel numbers carried in the multi-channel audio stream;
the decoding module is used for decoding the multi-channel audio stream according to the decoding address to obtain a decoded multi-channel audio stream;
and the playing module is used for playing the decoding multi-channel audio stream.
Optionally, the marking module comprises:
the first marking submodule is used for marking the decoding addresses corresponding to the multi-channel audio streams according to the channel numbers when the number of the audio streams of the multi-channel audio streams is less than or equal to the threshold number;
and the screening submodule is used for screening the multi-channel audio stream to obtain the audio stream with the threshold number and marking the decoding address corresponding to the audio stream with the threshold number when the number of the audio stream is greater than the threshold number.
Optionally, the apparatus further comprises:
and the clearing submodule is used for clearing the multi-channel audio stream in the cross-domain audio processor and clearing the mark when the next multi-channel audio stream is received.
In order to solve the above problem, an embodiment of the present invention provides an electronic device, including:
one or more processors; and
one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the audio processing methods described above.
In order to solve the above-described problems, an embodiment of the present invention provides a computer-readable storage medium storing a computer program that causes a processor to execute the above-described audio processing method.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, in the process of performing the cross-domain audio/video service, multiple audio streams sent by the cross-domain audio/video server are received, decoding addresses corresponding to the multiple audio streams are marked according to channel numbers carried in the multiple audio streams, so as to ensure that the audio streams in the received multiple audio streams all have their own channel numbers and decoding addresses, the multiple audio streams are decoded according to the decoding addresses to obtain decoded multiple audio streams, and finally the decoded multiple audio streams are played, so that the situation that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different can be avoided, that is, the audio streams can be normally played, dynamic processing of the multiple audio streams in the cross-domain audio/video service is realized, and the cross-domain audio/video service can be normally performed.
Drawings
Fig. 1 is a flowchart illustrating steps of an audio processing method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of an audio processing method according to a second embodiment of the present invention;
fig. 3 shows a block diagram of an audio processing apparatus according to a third embodiment of the present invention;
FIG. 4 illustrates a networking diagram of a video network of the present invention;
FIG. 5 is a schematic diagram of a hardware architecture of a node server according to the present invention;
fig. 6 shows a hardware architecture diagram of an access switch of the present invention;
fig. 7 shows a schematic hardware structure diagram of an ethernet protocol conversion gateway according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is described in detail with reference to the accompanying drawings and the detailed description thereof.
Referring to fig. 1, a flowchart illustrating steps of an audio processing method according to an embodiment of the present invention is provided, where the method may be applied to a video network, and specifically includes the following steps:
In this embodiment, the cross-domain audio/video service refers to an audio/video service performed across servers, and the cross-domain audio/video service may include a video networking multi-party conference.
A cross-domain audio-video server refers to a server that handles cross-domain audio streams.
In the process of carrying out the cross-domain audio and video service, the cross-domain audio and video server can send a plurality of paths of audio streams to the terminal, and the plurality of paths of audio streams can be played at the terminal side, so that the cross-domain audio and video service can be smoothly carried out. The terminal here is a terminal accessing the video network, and may be a mobile terminal accessing the video network, such as a mobile phone, a PAD (Portable Android Device, tablet computer), and other mobile electronic devices. The present invention may also be a PC (Personal Computer) end accessing to an internet of view, such as an electronic device like a desktop Computer and a notebook Computer, and specifically, the present invention may be determined according to business requirements, which is not limited in this embodiment of the present invention.
After receiving the multiple audio streams sent by the cross-domain audio and video server, step 502 is executed.
In the multi-channel audio stream, each channel of audio stream carries its own channel number, the cross-domain audio/video server may store the multi-channel audio stream to the cross-domain audio processor, and in the cross-domain audio processor, a decoding address corresponding to the multi-channel audio stream may be marked, when a next multi-channel audio stream is received, the multi-channel audio stream in the cross-domain audio processor is cleared, and the mark is cleared, further, the marking process of step 502 may be repeated to ensure that the received multi-channel audio streams all have their own channel numbers and decoding addresses, thereby avoiding that the terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different, that is, ensuring that the audio streams can be normally played.
When a speaker is switched in the cross-domain audio and video service, the next multi-channel audio stream can be received.
The threshold number of channels refers to a number of channels threshold preset by service personnel for comparison with the number of audio streams, the threshold number of channels may be four channels, five channels, or the like, and may be adjusted accordingly according to actual requirements.
When the number of the audio frequency flow paths of the multi-path audio frequency flow is less than or equal to the threshold number, marking the decoding address corresponding to the multi-path audio frequency flow according to the channel number; and when the number of the audio streams is greater than the threshold number, screening the multiple audio streams to obtain the threshold number of audio streams, and marking the decoding addresses corresponding to the threshold number of audio streams.
Illustratively, when the threshold number of channels is 4 and the number of audio channels of the multi-channel audio stream is less than or equal to 4, the decoding address corresponding to the multi-channel audio stream is marked according to the channel number. And when the number of the audio streams of the multi-channel audio stream is more than 4, screening the multi-channel audio stream to obtain the audio streams with the threshold number, and marking the decoding addresses corresponding to the audio streams with the threshold number.
In a specific implementation manner, the obtaining of the audio stream with the threshold number by filtering the multiple audio streams may specifically include:
the cross-domain audio processor can acquire the receiving time of each audio stream in the multiple audio streams, further can obtain threshold paths of audio streams from the multiple audio streams by screening according to the receiving times, sorts the multiple audio streams according to the receiving times from the back to the first to obtain a sorting result, and finally acquires the threshold paths of audio streams with the front sorting sequence from the multiple audio streams based on the sorting result.
After marking the decoding addresses corresponding to the multiple audio streams according to the channel numbers carried in the multiple audio streams, step 503 is performed.
And step 503, decoding the multiple audio streams according to the decoding addresses to obtain decoded multiple audio streams.
After the decoding addresses corresponding to the multiple audio streams are marked, the multiple audio streams in the cross-domain audio processor can be sent to an audio mixing processing module, and the multiple audio streams are decoded and mixed in the audio mixing processing module according to the decoding addresses to obtain decoded multiple audio streams.
After the multiple decoded audio streams are obtained, step 504 is performed.
After the decoded multi-channel audio stream subjected to the decoding process and the audio mixing process is obtained, the audio mixing processing module may send the decoded multi-channel audio stream to the audio playing module, and the audio playing module plays the decoded multi-channel audio stream.
In the embodiment of the invention, in the process of performing cross-domain audio/video service, multiple paths of audio streams sent by a cross-domain audio/video server are received, decoding addresses corresponding to the multiple paths of audio streams are marked according to channel numbers carried in the multiple paths of audio streams, so as to ensure that the received audio streams in the multiple paths of audio streams all have own channel numbers and decoding addresses, the multiple paths of audio streams are decoded according to the decoding addresses to obtain decoded multiple paths of audio streams, and finally the decoded multiple paths of audio streams are played, so that the situation that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different can be avoided, that is, the audio streams can be normally played, dynamic processing of the multiple paths of audio streams in the cross-domain audio/video service is realized, and the cross-domain audio/video service can be normally performed.
Referring to fig. 2, a flowchart of steps of an audio processing method provided in the second embodiment of the present invention is shown, where the method may be applied to a video network, and specifically includes the following steps:
In this embodiment, the cross-domain audio/video service refers to an audio/video service performed across servers, and the cross-domain audio/video service may include a video networking multi-party conference.
A cross-domain audio-video server refers to a server that handles cross-domain audio streams.
In the process of performing the cross-domain audio and video service, the cross-domain audio and video server can send a plurality of paths of audio streams to the terminal, and the plurality of paths of audio streams can be played at the terminal side, so that the cross-domain audio and video service can be smoothly performed. The terminal here is a terminal accessing to the video network, and may be a mobile terminal accessing to the video network, such as a mobile phone, a PAD (Portable Android Device, tablet computer), and other mobile electronic devices. The present invention may also be a PC (Personal Computer) end accessing to an internet of view, such as an electronic device like a desktop Computer and a notebook Computer, and specifically, the present invention may be determined according to business requirements, which is not limited in this embodiment of the present invention.
Illustratively, a heuristic 3 terminal of the video network may turn on a cross-domain audio-video service, and a cross-domain audio-video server may send multiple audio streams to the heuristic 3 terminal.
After receiving the multiple audio streams sent by the cross-domain audio and video server, step 602 is executed.
In the multi-channel audio stream, each channel of audio stream carries its own channel number, the cross-domain audio/video server may store the multi-channel audio stream to the cross-domain audio processor, and in the cross-domain audio processor, a decoding address corresponding to the multi-channel audio stream may be marked, when a next multi-channel audio stream is received, the multi-channel audio stream in the cross-domain audio processor is cleared, and the mark is cleared, further, the marking process of step 502 may be repeated to ensure that the received multi-channel audio streams all have their own channel numbers and decoding addresses, thereby avoiding that the terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different, that is, ensuring that the audio streams can be normally played.
The threshold number of channels is a number of channels threshold preset by a service person for comparison with the number of audio streams, where the threshold number of channels may be four channels or five channels, and the like, and may be adjusted accordingly according to actual requirements.
Illustratively, when the threshold number of channels is 4, then when the number of audio channels of the multiple audio streams is less than or equal to 4, the decoding addresses corresponding to the multiple audio streams are marked according to the channel numbers.
Each path of audio stream carries its own channel number, the cross-domain audio/video server can store multiple paths of audio streams to the cross-domain audio processor, in the cross-domain audio processor, a decoding address corresponding to the multiple paths of audio streams is marked, when a next multiple paths of audio streams are received, the multiple paths of audio streams in the cross-domain audio processor are cleared, and the mark is cleared, further, the marking process of step 502 needs to be repeated to ensure that the received multiple paths of audio streams all have their own channel numbers and decoding addresses, so as to avoid the situation that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the decoding addresses of the audio streams are the same but the decoding addresses of the audio streams are different, that is, to ensure that the audio streams can be normally played.
In a specific implementation manner of the present invention, the step 603 may include:
and a substep S01, acquiring the receiving time of each audio stream in the multi-channel audio streams.
In this embodiment, the receiving time refers to a time corresponding to each received audio stream in the multiple audio streams.
In the process of receiving multiple audio streams by the terminal, the time corresponding to each audio stream can be recorded, so that the receiving sequence of each audio stream can be obtained.
After the receiving time of each audio stream in the multiple audio streams is obtained, substep S02 is performed.
And a substep S02 of screening the multiple audio streams to obtain audio streams with threshold channels according to each receiving time.
After obtaining each receiving time, the audio streams with the threshold number may be obtained by less selecting from the multiple audio streams according to each receiving time, and the screening process may be performed according to the sequence of the receiving times, and specifically, the following specific implementation manner may be described in detail.
In a specific implementation manner of the present invention, the sub-step S02 may include:
and a substep M01 of sequencing the multi-channel audio streams according to the receiving time from the back to the first to obtain a sequencing result.
After the receiving time of each audio stream in the multiple audio streams is obtained, the multiple audio streams may be sorted according to the receiving time corresponding to each audio stream and according to an order from the receiving time to the receiving time, so as to obtain a sorting result, for example, the multiple audio streams include: A. b, C, D, E, after sequencing according to the order from the receiving time to the first, can get the sequencing result: B. c, a, D, E, it can also be known that the terminal receives the audio stream E first and the audio stream B last.
After the multi-channel audio streams are sorted according to the receiving time from the back to the first to obtain a sorting result, the substep M02 is executed.
And a substep M02 of obtaining the audio streams with the threshold number of paths in the front sorting order from the multiple audio streams based on the sorting result.
After the multiple audio streams are sorted according to the sequence from the receiving time to the receiving time, and the sorting result is obtained, the threshold number of audio streams with the top sorting order may be obtained from the multiple audio streams based on the sorting result, for example, when the threshold number is 4, the sorting result of the multiple audio streams is: B. c, A, D, E, the audio stream of threshold number that obtain from it is: B. c, A and D.
Of course, the terminal may delete audio streams that are not acquired because of the late ranking.
And step 604, decoding the multiple audio streams according to the decoding addresses to obtain decoded multiple audio streams.
The terminal can send the multiple audio streams in the cross-domain audio processor to the audio mixing processing module, and the audio mixing processing module performs decoding processing and mixing processing on the multiple audio streams according to the decoding addresses to obtain decoded multiple audio streams.
After the decoded multiple audio streams are obtained, step 605 is performed.
After obtaining the decoded multiple audio streams subjected to the decoding process and the audio mixing process, the audio mixing processing module may send the decoded multiple audio streams to the audio playing module, and the audio playing module plays the decoded multiple audio streams.
In the embodiment of the invention, in the process of performing the cross-domain audio/video service, multiple audio streams sent by a cross-domain audio/video server are received, when the number of audio streams of the multiple audio streams is less than or equal to the threshold number, decoding addresses corresponding to the multiple audio streams are marked according to channel numbers, when the number of audio streams is greater than the threshold number, the audio streams of the threshold number are obtained by screening the multiple audio streams, decoding addresses corresponding to the audio streams of the threshold number are marked, so that the received audio streams of the multiple audio streams all have own channel numbers and decoding addresses, decoding processing is performed on the multiple audio streams according to the decoding addresses, the decoded multiple audio streams are obtained, and the decoded multiple audio streams are played, so that the situation that the audio cannot be normally played by a terminal due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different, is avoided, the audio streams can be normally played, and the cross-domain audio/video service can be dynamically processed, and the normal performance of the cross-domain audio/video service is ensured.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 3, a block diagram of an audio processing apparatus according to a third embodiment of the present invention is shown, where the audio processing apparatus 700 may be applied to a video network, and the apparatus specifically includes:
the receiving module 701 is used for receiving a plurality of paths of audio streams sent by a cross-domain audio and video server in the process of performing a cross-domain audio and video service;
a marking module 702, configured to mark, according to a channel number carried in the multiple audio streams, a decoding address corresponding to the multiple audio streams;
a decoding module 703, configured to perform decoding processing on the multiple audio streams according to the decoding address to obtain decoded multiple audio streams;
and a playing module 704, configured to play the decoded multiple audio streams.
Optionally, the marking module comprises:
the first marking submodule is used for marking the decoding address corresponding to the multi-channel audio stream according to the channel number when the number of the audio streams of the multi-channel audio stream is less than or equal to the threshold number;
and the screening submodule is used for screening the multiple audio streams to obtain the audio streams with the threshold number and marking the decoding addresses corresponding to the audio streams with the threshold number when the number of the audio streams is greater than the threshold number.
Optionally, the screening submodule comprises:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring the receiving time of each audio stream in a plurality of audio streams;
and the screening unit is used for screening the multiple audio streams to obtain the audio streams with threshold channels according to the receiving time.
Optionally, the screening unit comprises:
the sequencing subunit is used for sequencing the multi-channel audio stream according to the receiving time from the back to the first to obtain a sequencing result;
and the acquisition subunit is used for acquiring the audio streams with the threshold number in the front sorting order from the multiple audio streams based on the sorting result.
Optionally, the marking module comprises:
the saving submodule is used for saving the multi-channel audio stream to the cross-domain audio processor;
and the second marking submodule is used for marking the decoding address corresponding to the multi-path audio stream in the cross-domain audio processor.
Optionally, the apparatus further comprises:
and the clearing submodule is used for clearing the multi-channel audio stream in the cross-domain audio processor and clearing the mark when the next multi-channel audio stream is received.
In the embodiment of the invention, in the process of performing the cross-domain audio/video service, a receiving module can be used for receiving multiple audio streams sent by a cross-domain audio/video server, a marking module is used for marking decoding addresses corresponding to the multiple audio streams according to channel numbers carried in the multiple audio streams so as to ensure that the audio streams in the received multiple audio streams all have own channel numbers and decoding addresses, the decoding module is used for decoding the multiple audio streams according to the decoding addresses to obtain the decoded multiple audio streams, and finally the playing module is used for playing the decoded multiple audio streams, so that the situation that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different can be avoided, that the audio streams can be normally played, dynamic processing of the multiple audio streams in the cross-domain audio/video service is realized, and the cross-domain audio/video service can be normally performed.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an electronic device, including:
one or more processors; and
one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the audio processing method as described above.
An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program for causing a processor to execute the audio processing method.
The video networking is an important milestone for network development, is a real-time network, can realize high-definition video real-time transmission, and pushes a plurality of internet applications to high-definition video, and high-definition faces each other.
The video networking adopts a real-time high-definition video exchange technology, can integrate required services such as dozens of services of video, voice, pictures, characters, communication, data and the like on a system platform on a network platform, such as high-definition video conference, video monitoring, intelligent monitoring analysis, emergency command, digital broadcast television, delayed television, network teaching, live broadcast, VOD on demand, television mail, personal Video Recorder (PVR), intranet (self-office) channels, intelligent video broadcast control, information distribution and the like, and realizes high-definition quality video broadcast through a television or a computer.
To better understand the embodiments of the present invention, the following description refers to the internet of view:
some of the technologies applied in the video networking are as follows:
network Technology (Network Technology)
Network technology innovation in video networking has improved over traditional Ethernet (Ethernet) to face the potentially enormous video traffic on the network. Unlike pure network Packet Switching (Packet Switching) or network Circuit Switching (Circuit Switching), the Packet Switching is adopted by the technology of the video networking to meet the Streaming requirement. The video networking technology has the advantages of flexibility, simplicity and low price of packet switching, and simultaneously has the quality and safety guarantee of circuit switching, and realizes the seamless connection of a whole network switching type virtual circuit and a data format.
Switching Technology (Switching Technology)
The video network adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the defects of the Ethernet on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, and directly bears an IP data packet. The user data does not require any format conversion across the entire network. The video network is a higher-level form of the Ethernet, is a real-time exchange platform, can realize the large-scale high-definition video real-time transmission of the whole network which can not be realized by the current Internet, and pushes a plurality of network video applications to high-definition and unification.
Server Technology (Server Technology)
The server technology on the video networking and unified video platform is different from the traditional server, the streaming media transmission of the video networking and unified video platform is established on the basis of connection orientation, the data processing capacity of the video networking and unified video platform is independent of flow and communication time, and a single network layer can contain signaling and data transmission. For voice and video services, the complexity of video networking and unified video platform streaming media processing is much simpler than that of data processing, and the efficiency is greatly improved by more than one hundred times compared with that of a traditional server.
Storage Technology (Storage Technology)
The super-high speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the media content with super-large capacity and super-large flow, the program information in the server instruction is mapped to the specific hard disk space, the media content is not passed through the server any more, and is directly sent to the user terminal instantly, and the general waiting time of the user is less than 0.2 second. The optimized sector distribution greatly reduces the mechanical motion of the magnetic head track seeking of the hard disk, the resource consumption only accounts for 20% of that of the IP internet of the same grade, but concurrent flow which is 3 times larger than that of the traditional hard disk array is generated, and the comprehensive efficiency is improved by more than 10 times.
Network Security Technology (Network Security Technology)
The structural design of the video network completely eliminates the network security problem troubling the internet structurally by the modes of independent service permission control each time, complete isolation of equipment and user data and the like, generally does not need antivirus programs and firewalls, avoids the attack of hackers and viruses, and provides a structural carefree security network for users.
Service Innovation Technology (Service Innovation Technology)
The unified video platform integrates services and transmission, and is not only automatically connected once whether a single user, a private network user or a network aggregate. The user terminal, the set-top box or the PC are directly connected to the unified video platform to obtain various multimedia video services in various forms. The unified video platform adopts a menu type configuration table mode to replace the traditional complex application programming, can realize complex application by using very few codes, and realizes infinite new service innovation.
Networking of the video network is as follows:
the video network is a centralized control network structure, and the network can be a tree network, a star network, a ring network and the like, but on the basis of the centralized control node, the whole network is controlled by the centralized control node in the network.
As shown in fig. 4, the view network is divided into two parts, an access network and a metropolitan network.
The devices of the access network part can be mainly classified into 3 types: node server, access switch, terminal (including various set-top boxes, coding boards, memories, etc.). The node server is connected to an access switch, which may be connected to a plurality of terminals and may be connected to an ethernet network.
The node server is a node which plays a centralized control function in the access network and can control the access switch and the terminal. The node server can be directly connected with the access switch or directly connected with the terminal.
Similarly, devices of the metropolitan network portion may also be classified into 3 types: a metropolitan area server, a node switch and a node server. The metro server is connected to a node switch, which may be connected to a plurality of node servers.
The node server is a node server of the access network part, namely the node server belongs to both the access network part and the metropolitan area network part.
The metropolitan area server is a node which plays a central control function in the metropolitan area network and can control the node switch and the node server. The metropolitan area server can be directly connected with the node switch or directly connected with the node server.
Therefore, the whole video network is a network structure controlled by a hierarchical centralized way, and the network controlled by the node server and the metropolitan area server can be in various structures such as a tree, a star, a ring and the like.
The access network part can form a unified video platform (the part in the dotted circle), and a plurality of unified video platforms can form a video network; each unified video platform may be interconnected via metropolitan area and wide area video networking.
Video networking device classification
1.1 devices in the video network of the embodiment of the present invention can be mainly classified into 3 types: servers, switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.). The video network as a whole can be divided into a metropolitan area network (or national network, global network, etc.) and an access network.
1.2 wherein the devices of the access network part can be mainly classified into 3 types: node servers, access switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.).
The specific hardware structure of each access network device is as follows:
a node server:
as shown in fig. 5, the system mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;
the packets coming from the network interface module 201, the cpu module 203 and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of looking up the address table 205 on the incoming packet, thereby obtaining the direction information of the packet; and stores the packet in a queue of the corresponding packet buffer 206 according to the packet's steering information; if the queue of the packet buffer 206 is nearly full, it is discarded; the switching engine module 202 polls all packet buffer queues for forwarding if the following conditions are met: 1) The port send buffer is not full; 2) The queue packet counter is greater than zero. The disk array module 204 mainly implements control over the hard disk, including initialization, reading, writing, and other operations on the hard disk; the CPU module 203 is mainly responsible for protocol processing with an access switch and a terminal (not shown in the figure), configuring an address table 205 (including a downlink protocol packet address table, an uplink protocol packet address table, and a data packet address table), and configuring the disk array module 204.
The access switch:
as shown in fig. 6, the network interface module (downlink network interface module 301, uplink network interface module 302), switching engine module 303 and CPU module 304 are mainly included;
wherein, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the Destination Address (DA), the Source Address (SA), the packet type, and the packet length of the packet meet the requirements, if so, allocates a corresponding stream identifier (stream-id) and enters the switch engine module 303, otherwise, discards the stream identifier; the packet (downstream data) coming from the upstream network interface module 302 enters the switching engine module 303; the incoming data packet of the CPU module 304 enters the switching engine module 303; the switching engine module 303 performs an operation of looking up the address table 306 on the incoming packet, thereby obtaining the direction information of the packet; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in association with the stream-id; if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downlink network interface to the uplink network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the guiding information of the packet; if the queue of the packet buffer 307 is nearly full, it is discarded.
The switching engine module 303 polls all packet buffer queues, which in this embodiment of the present invention is divided into two cases:
if the queue is from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queued packet counter is greater than zero; 3) Obtaining a token generated by a code rate control module;
if the queue is not from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queue packet counter is greater than zero.
The rate control module 308 is configured by the CPU module 304, and generates tokens for packet buffer queues from all downstream network interfaces to upstream network interfaces at programmable intervals to control the rate of upstream forwarding.
The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306, and configuration of the code rate control module 308.
Ethernet protocol conversion gateway：
As shown in fig. 7, the apparatus mainly includes a network interface module (a downlink network interface module 401 and an uplink network interface module 402), a switch engine module 403, a CPU module 404, a packet detection module 405, a rate control module 408, an address table 406, a packet buffer 407, a MAC adding module 409, and a MAC deleting module 410.
Wherein, the data packet coming from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the ethernet MAC DA, the ethernet MAC SA, the ethernet length or frame type, the video network destination address DA, the video network source address SA, the video network packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id); then, the MAC deletion module 410 subtracts MAC DA, MAC SA, length or frame type (2 byte) and enters the corresponding receiving buffer, otherwise, discards it;
the downlink network interface module 401 detects the sending buffer of the port, if there is a packet, the ethernet MAC DA of the corresponding terminal is known according to the destination address DA of the packet in the video network, the ethernet MAC DA of the terminal, the MAC SA of the ethernet protocol gateway, and the ethernet length or frame type are added, and the packet is sent.
The other modules in the ethernet protocol gateway function similarly to the access switch.
A terminal:
the system mainly comprises a network interface module, a service processing module and a CPU module; for example, the set-top box mainly comprises a network interface module, a video and audio coding and decoding engine module and a CPU module; the coding board mainly comprises a network interface module, a video and audio coding engine module and a CPU module; the memory mainly comprises a network interface module, a CPU module and a disk array module.
1.3 devices of the metropolitan area network part can be largely classified into 2 types: node server, node exchanger, metropolitan area server. The node switch mainly comprises a network interface module, a switching engine module and a CPU module; the metropolitan area server mainly comprises a network interface module, a switching engine module and a CPU module.
2. Video networking packet definition
2.1 Access network packet definition
The data packet of the access network mainly comprises the following parts: destination Address (DA), source Address (SA), reserved byte, payload (PDU), CRC.
As shown in the following table, the data packet of the access network mainly includes the following parts:
| DA | SA | Reserved | Payload | CRC |
wherein:
the Destination Address (DA) is composed of 8 bytes (byte), the first byte represents the type of the data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are 256 possibilities at most, the second byte to the sixth byte are metropolitan area network addresses, and the seventh byte and the eighth byte are access network addresses;
the Source Address (SA) is also composed of 8 bytes (byte), and is defined to be the same as the Destination Address (DA);
the reserved byte consists of 2 bytes;
the payload part has different lengths according to the types of different datagrams, 64 bytes if various protocol packets, 32+1024=1056 bytes if single-multicast data packets, and certainly not limited to the above 2 types;
the CRC consists of 4 bytes and is calculated in accordance with the standard ethernet CRC algorithm.
2.2 packet definition for metropolitan area networks
The topology of a metropolitan area network is a graph and there may be 2, or even more than 2, connections between two devices, i.e., there may be more than 2 connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the address of the metro network device is unique, and in order to accurately describe the connection relationship between the metro network devices, parameters are introduced in the embodiment of the present invention: a label to uniquely describe a metropolitan area network device.
In the present specification, the definition of the Label is similar to that of the Label of MPLS (Multi-Protocol Label Switch), and assuming that there are two connections between the device a and the device B, there are 2 labels for the packet from the device a to the device B, and there are 2 labels for the packet from the device B to the device a. The label is classified into an incoming label and an outgoing label, and assuming that the label (incoming label) of the packet entering the device a is 0x0000, the label (outgoing label) of the packet leaving the device a may become 0x0001. The network access process of the metro network is a network access process under centralized control, that is, address allocation and label allocation of the metro network are both dominated by the metro server, and the node switch and the node server are both passively executed, which is different from label allocation of MPLS, and label allocation of MPLS is a result of mutual negotiation between the switch and the server.
As shown in the following table, the data packet of the metro network mainly includes the following parts:
| DA | SA | Reserved | label (R) | Payload | CRC |
Namely Destination Address (DA), source Address (SA), reserved byte (Reserved), tag, payload (PDU), CRC. The format of the tag may be defined as follows: the tag is 32 bits with the upper 16 bits reserved and only the lower 16 bits used, which is located between the reserved bytes and the payload of the packet.
Based on the above characteristics of the video networking, one of the core concepts of the embodiments of the present invention is provided, during the process of performing the cross-domain audio/video service, receiving multiple audio streams sent by the cross-domain audio/video server, marking decoding addresses corresponding to the multiple audio streams according to channel numbers carried in the multiple audio streams, so as to ensure that the audio streams in the received multiple audio streams all have their own channel numbers and decoding addresses, performing decoding processing on the multiple audio streams according to the decoding addresses, obtaining decoded multiple audio streams, and finally playing the decoded multiple audio streams, so that it is possible to avoid that a terminal cannot normally play audio due to the fact that the decoding addresses of the audio streams are the same but the channel numbers are different, or the channel numbers are the same but the decoding addresses of the audio streams are different, that the audio streams can be normally played, thereby dynamically processing the multiple audio streams in the cross-domain audio/video service, and ensuring that the cross-domain audio/video service can be normally performed.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one of skill in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the true scope of the embodiments of the present invention.
Finally, it should also be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "include", "including" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or terminal device including a series of elements includes not only those elements but also other elements not explicitly listed or inherent to such process, method, article, or terminal device. Without further limitation, an element defined by the phrases "comprising one of \ 8230; \8230;" does not exclude the presence of additional like elements in a process, method, article, or terminal device that comprises the element.
The foregoing describes in detail an audio processing method and apparatus provided by the present invention, and the principles and embodiments of the present invention are explained by applying specific examples, and the descriptions of the above examples are only used to help understand the method and core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. An audio processing method, applied to a video network, the method comprising:
receiving a plurality of paths of audio streams sent by a cross-domain audio and video server in the process of carrying out cross-domain audio and video service;
marking decoding addresses corresponding to the multi-channel audio streams according to channel numbers carried in the multi-channel audio streams, wherein the channel numbers correspond to the decoding addresses one to one;
decoding the multi-channel audio stream according to the decoding address to obtain a decoded multi-channel audio stream;
and playing the decoded multi-channel audio stream.
2. The method according to claim 1, wherein said marking the decoding address corresponding to the multiple audio streams according to the channel number carried in the multiple audio streams comprises:
when the number of the audio flow paths of the multi-path audio flow is less than or equal to the threshold number, marking the decoding address corresponding to the multi-path audio flow according to the channel number;
and when the number of the audio streams is greater than the threshold number, screening the multiple audio streams to obtain the audio streams with the threshold number, and marking the decoding addresses corresponding to the audio streams with the threshold number.
3. The method of claim 2, wherein the filtering the plurality of audio streams to obtain the threshold number of audio streams comprises:
acquiring the receiving time of each audio stream in the multiple audio streams;
and screening the multi-channel audio stream according to each receiving time to obtain the audio stream with the threshold channel number.
4. The method of claim 3, wherein the filtering the plurality of audio streams to obtain the threshold number of audio streams according to each of the receiving times comprises:
according to each receiving time, sequencing the multi-channel audio stream according to the sequence of the receiving time from the back to the first to obtain a sequencing result;
and acquiring the audio streams with the threshold number of the audio streams with the top sorting order from the multi-path audio streams based on the sorting result.
5. The method according to claim 1, wherein said marking the decoding address corresponding to the multiple audio streams according to the channel number carried in the multiple audio streams comprises:
saving the multi-channel audio stream to a cross-domain audio processor;
and marking the decoding addresses corresponding to the multi-path audio streams in the cross-domain audio processor.
6. The method of claim 5, wherein after marking, in the cross-domain audio processor, the decoding address corresponding to the multiple audio streams, the method further comprises:
and when a next multi-channel audio stream is received, emptying the multi-channel audio stream in the cross-domain audio processor and clearing the mark.
7. An audio processing apparatus, wherein the apparatus is applied to a video network, the apparatus comprising:
the receiving module is used for receiving a plurality of paths of audio streams sent by a cross-domain audio and video server in the process of carrying out cross-domain audio and video services;
a marking module, configured to mark, according to a channel number carried in the multiple audio streams, a decoding address corresponding to the multiple audio streams, where the channel number corresponds to the decoding address one to one;
the decoding module is used for decoding the multi-channel audio stream according to the decoding address to obtain a decoded multi-channel audio stream;
and the playing module is used for playing the decoding multi-channel audio stream.
8. The apparatus of claim 7, wherein the marking module comprises:
the first marking submodule is used for marking the decoding addresses corresponding to the multi-channel audio streams according to the channel numbers when the number of the audio streams of the multi-channel audio streams is less than or equal to the threshold number;
and the screening submodule is used for screening the multi-channel audio stream to obtain the audio stream with the threshold number and marking the decoding address corresponding to the audio stream with the threshold number when the number of the audio stream is greater than the threshold number.
9. An electronic device, comprising:
one or more processors; and
one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the electronic device to perform the audio processing method of any of claims 1-6.
10. A computer-readable storage medium storing a computer program for causing a processor to execute the audio processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911400650.4A CN111181965B (en) | 2019-12-30 | 2019-12-30 | Audio processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911400650.4A CN111181965B (en) | 2019-12-30 | 2019-12-30 | Audio processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111181965A CN111181965A (en) | 2020-05-19 |
| CN111181965B true CN111181965B (en) | 2023-04-14 |
Family
ID=70621541
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911400650.4A Active CN111181965B (en) | 2019-12-30 | 2019-12-30 | Audio processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111181965B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114520687B (en) * | 2022-02-17 | 2023-11-03 | 深圳震有科技股份有限公司 | Audio data processing method, device and equipment applied to satellite system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937517A (en) * | 2006-10-08 | 2007-03-28 | 华为技术有限公司 | Video conference system and its data transmission method and device |
| CN107979591A (en) * | 2017-11-15 | 2018-05-01 | 深圳大希创新科技有限公司 | A kind of audio-video terminal multichannel on-line control system and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104661085A (en) * | 2013-11-22 | 2015-05-27 | 中兴通讯股份有限公司 | Multi-way wireless display method and device |
| CN108965993B (en) * | 2017-07-20 | 2022-01-14 | 视联动力信息技术股份有限公司 | Method and device for decoding multi-channel video stream |
| CN110070878B (en) * | 2019-03-26 | 2021-05-04 | 苏州科达科技股份有限公司 | Decoding method of audio code stream and electronic equipment |
-
2019
- 2019-12-30 CN CN201911400650.4A patent/CN111181965B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937517A (en) * | 2006-10-08 | 2007-03-28 | 华为技术有限公司 | Video conference system and its data transmission method and device |
| CN107979591A (en) * | 2017-11-15 | 2018-05-01 | 深圳大希创新科技有限公司 | A kind of audio-video terminal multichannel on-line control system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111181965A (en) | 2020-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111193788A (en) | Audio and video stream load balancing method and device | |
| CN109120879B (en) | Video conference processing method and system | |
| CN109474715B (en) | Resource configuration method and device based on video network | |
| CN110475090B (en) | Conference control method and system | |
| CN110460804B (en) | Conference data transmitting method, system, device and computer readable storage medium | |
| CN109788235B (en) | Video networking-based conference recording information processing method and system | |
| CN111478884B (en) | Method, device and storage medium for data transmission among different video networks | |
| CN110113557B (en) | Multistage conference implementation method and video networking system | |
| CN109005378B (en) | Video conference processing method and system | |
| CN111327868A (en) | Method, terminal, server, device and medium for setting conference speaking party role | |
| CN109743284B (en) | Video processing method and system based on video network | |
| CN111654659B (en) | Conference control method and device | |
| CN110830826A (en) | Video transcoding equipment scheduling method and system | |
| CN110446058B (en) | Video acquisition method, system, device and computer readable storage medium | |
| CN109889516B (en) | Method and device for establishing session channel | |
| CN110022500B (en) | Packet loss processing method and device | |
| CN111181965B (en) | Audio processing method and device | |
| CN110049069B (en) | Data acquisition method and device | |
| CN110557411A (en) | video stream processing method and device based on video network | |
| CN110233872B (en) | Data transmission method based on video network and video network terminal | |
| CN110401807B (en) | Communication method and device of video telephone system | |
| CN109495709B (en) | Video network management system and method | |
| CN110493311B (en) | Service processing method and device | |
| CN109788231B (en) | Video telephone service processing method and device | |
| CN110166840B (en) | Service processing method, system and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |