CN110166185A - Processing method, device, storage medium and the processor of audio data - Google Patents

Processing method, device, storage medium and the processor of audio data Download PDF

Info

Publication number
CN110166185A
CN110166185A CN201910408187.1A CN201910408187A CN110166185A CN 110166185 A CN110166185 A CN 110166185A CN 201910408187 A CN201910408187 A CN 201910408187A CN 110166185 A CN110166185 A CN 110166185A
Authority
CN
China
Prior art keywords
client
audio data
destination
audio
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910408187.1A
Other languages
Chinese (zh)
Other versions
CN110166185B (en
Inventor
田建超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Guangzhou Shizhen Information Technology Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Guangzhou Shizhen Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd, Guangzhou Shizhen Information Technology Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority to CN201910408187.1A priority Critical patent/CN110166185B/en
Publication of CN110166185A publication Critical patent/CN110166185A/en
Application granted granted Critical
Publication of CN110166185B publication Critical patent/CN110166185B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/1607—Details of the supervisory signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1809—Selective-repeat protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L5/00—Arrangements affording multiple use of the transmission path
    • H04L5/003—Arrangements for allocating sub-channels of the transmission path
    • H04L5/0053—Allocation of signaling, i.e. of overhead other than pilot signals
    • H04L5/0055—Physical resource allocation for ACK/NACK

Abstract

The invention discloses a kind of processing method of audio data, device, storage medium and processors.This method comprises: receiving the first audio data of the first client of one or more of multiple client respectively, target audio data are obtained；In the case where having lacked the first object audio data of destination client in target audio data, obtain the dbjective state of destination client, wherein, multiple client includes destination client, dbjective state, which is used to indicate, to be forbidden being forwarded first object audio data, or allows for first object audio data to be forwarded to client associated with destination client；Dbjective state indicate forbid being forwarded first object audio data in the case where, forbid sending destination request to destination client, wherein destination request retransmits first object audio data for request target client.Through the invention, achieved the effect that improve the efficiency for handling audio data.

Description

Processing method, device, storage medium and the processor of audio data
Technical field
The present invention relates to data processing fields, in particular to a kind of processing method of audio data, device, storage Medium and processor.
Background technique
Currently, server can receive the audio data that all clients send over when handling audio data, Therefrom determine the audio data for needing to be forwarded to other clients.And the audio data of some clients is not need to forward To other clients, for example, the volume that some client issues is smaller, then do not need to relay to its audio data other Client.
If the audio data that some client is sent has missing, server just directly can send packet loss weight to the client Pass (NACK) request.But if the audio data of client does not need to be transmitted to other clients, in the audio data packet loss When, server still can send NACK request to the client, and the audio data actually retransmitted back is not forwarded to In other clients, so that useless NACK be caused to request and retransmit, there is asking for the low efficiency handled audio data Topic.
Aiming at the problem that low efficiency in the prior art handled audio data, effective solution is not yet proposed at present Certainly scheme.
Summary of the invention
The main purpose of the present invention is to provide a kind of processing method of audio data, device, storage medium and processor, The technical issues of at least to solve the low efficiency handled audio data.
To achieve the goals above, according to an aspect of an embodiment of the present invention, a kind of processing of audio data is provided Method.This method comprises: receiving the first audio data of the first client of one or more of multiple client respectively, obtain Target audio data；In the case where having lacked the first object audio data of destination client in target audio data, obtain The dbjective state of destination client, wherein multiple client includes destination client, and dbjective state, which is used to indicate, forbids to first Target audio data are forwarded, or allow for first object audio data to be forwarded to client associated with destination client End；Dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to destination client send out Send destination request, wherein destination request retransmits first object audio data for request target client.
Optionally, after the dbjective state for obtaining destination client, this method further include: indicated in dbjective state fair Perhaps it in the case where first object audio data being forwarded to client associated with destination client, is sent to destination client Destination request；Receive the first object audio data that destination client response destination request is sent；By first object audio data It is forwarded to client associated with destination client.
Optionally, the first audio data for receiving the first client of one or more of multiple client respectively it Before, this method further include: receive the second audio data of multiple client；According to the second audio data of each client, really The dbjective state of fixed each client.
Optionally, according to the second audio data of each client, determine that the dbjective state of each client includes: from every The first volume data is extracted in the second audio data of a client, wherein the first volume data is used to indicate corresponding visitor The volume for the audio that family end issues；According to the first volume data of each client, determined from multiple client with each The first object client of the associated first object quantity of client, wherein each client is for receiving and each client The second audio data of associated each first object client is held, the first volume data of each first object client refers to The volume of the audio shown is greater than the volume of audio indicated by the first volume data of the second client, and the second client is more Any client in a client in addition to the first object client of each client and first object quantity；Respectively by The state of the first object client of one destination number is determined as first object state, wherein first object state is used to indicate The second audio data by first object client is allowed to be forwarded to client associated with first object client；Respectively will The state of second destination client of the second destination number in multiple client is determined as the second dbjective state, wherein second Second destination client of destination number is the visitor in multiple client in addition to the first object client of first object quantity Family end, the second dbjective state, which is used to indicate, forbids the second audio data to the second destination client to be forwarded.
Optionally, after forbidding sending destination request to destination client, this method further include: receive multiple client Third audio data；According to the third audio data of each client, the dbjective state of each client is updated.
Optionally, according to the third audio data of each client, packet is updated to the dbjective state of each client It includes: extracting the second volume data from the third audio data of each client, wherein the second volume data is used to indicate pair The volume for the audio that the client answered issues；According to the second volume data of each client, determined from multiple client The third destination client of third destination number associated with each client, wherein each client is for receiving and often The third audio data of a associated each third destination client of client, the second volume of each third destination client The volume for the audio that data indicate is greater than the volume of audio indicated by the second volume data of third client, third client End is any client in multiple client in addition to the third destination client of each client and third destination number；Point The state of the third destination client of third destination number first object state is not updated to, wherein first object state is used Allow the third audio data of third destination client being forwarded to client associated with third destination client in instruction； The state of the 4th destination client of the 4th destination number in multiple client is updated to the second dbjective state respectively, In, the 4th destination client of the 4th destination number be in multiple client except the third destination client of third destination number it Outer client, the second dbjective state, which is used to indicate, forbids the third audio data to the 4th destination client to be forwarded.
Optionally, the third audio data for receiving multiple client includes: receiving multiple client every target time section Third audio data.
Optionally, according to the third audio data of each client, packet is updated to the dbjective state of each client Include: the third audio data every target time section according to each client is updated the dbjective state of each client.
Optionally, before the dbjective state for obtaining destination client, this method further include: obtain object listing, wherein The identification information of the first audio data in object listing including each first client；It does not include the first mesh in object listing In the case where the identification information for marking audio data, determines and lacked first object audio data in target audio data.
Optionally, the first audio data for receiving the first client of one or more of multiple client respectively, obtains Target audio data include: the first audio data for obtaining the first client of one or more under conference scenario respectively, Obtain target audio data.
To achieve the goals above, according to another aspect of an embodiment of the present invention, it additionally provides a kind of to server transmission Target audio data, wherein target audio data include the first sound of the first client of one or more of multiple client Frequency evidence；The first object audio data of destination client, and the mesh in destination client have been lacked in target audio data In the case that mark state instruction forbids server to be forwarded the first object audio data of destination client out, forbid target Client receives the destination request that server is sent, wherein multiple client includes destination client, and destination request is for requesting Destination client retransmits first object audio data.
To achieve the goals above, according to another aspect of an embodiment of the present invention, a kind of place of audio data is additionally provided Manage device.The device includes: receiving unit, for receiving the of the first client of one or more of multiple client respectively One audio data obtains target audio data；Acquiring unit, for having lacked the of destination client in target audio data In the case where one target audio data, the dbjective state of destination client is obtained, wherein multiple client includes target customer End, dbjective state, which is used to indicate, to be forbidden being forwarded first object audio data, or is allowed first object audio data It is forwarded to client associated with destination client；Forbid unit, forbids for being indicated in dbjective state to first object In the case that audio data is forwarded, forbid sending destination request to destination client, wherein destination request is for requesting mesh Mark client retransmits first object audio data.
To achieve the goals above, according to another aspect of an embodiment of the present invention, a kind of place of audio data is additionally provided Manage device.The device transmission unit, for sending target audio data to server, wherein target audio data include multiple First audio data of the first client of one or more of client；Forbid unit, for lacking in target audio data The first object audio data of destination client is lost, and the dbjective state in destination client indicates and forbids server to mesh In the case that the first object audio data of mark client is forwarded, destination client is forbidden to receive the target that server is sent Request, wherein multiple client includes destination client, and destination request is used for request target client for first object audio number According to re-transmission.
To achieve the goals above, according to another aspect of the present invention, a kind of storage medium is additionally provided.The storage medium Program including storage, wherein the method that equipment where control storage medium executes the embodiment of the present invention in program operation.
To achieve the goals above, according to another aspect of the present invention, a kind of processor is additionally provided.The processor is used for Run program, wherein the method for the embodiment of the present invention is executed when program is run.
In embodiments of the present invention, the first audio of the first client of one or more of multiple client is received respectively Data obtain target audio data；The feelings of the first object audio data of destination client have been lacked in target audio data Under condition, the dbjective state of destination client is obtained, wherein multiple client includes destination client, and dbjective state is used to indicate Forbid being forwarded first object audio data, or allows to be forwarded to first object audio data and destination client phase Associated client；Dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to mesh It marks client and sends destination request, wherein destination request retransmits first object audio data for request target client.? That is in the case where having lacked the first object audio data of destination client in the target audio data got, really Determining first object audio data is the audio data (enlivening audio) for whether needing to be forwarded, if it is not, then forbidding to mesh The first object audio data that client request obtains missing is marked, can reduce and weight is made requests to the audio data of client Pass, solve the technical issues of low efficiency handled to audio data, so reached raising to audio data at The technical effect of the efficiency of reason.
Detailed description of the invention
The attached drawing constituted part of this application is used to provide further understanding of the present invention, schematic reality of the invention It applies example and its explanation is used to explain the present invention, do not constitute improper limitations of the present invention.In the accompanying drawings:
Figure 1A is a kind of flow chart of the processing method of audio data according to an embodiment of the present invention；
Figure 1B is the flow chart of the processing method of another audio data according to an embodiment of the present invention；
Fig. 2 is the signal that the audio data between a kind of server and client side according to an embodiment of the present invention is transmitted Figure；
Fig. 3 is the flow chart of another audio data processing method according to an embodiment of the present invention；
Fig. 4 is the flow chart of another audio data processing method according to an embodiment of the present invention；
Fig. 5 is a kind of schematic diagram of the processing unit of audio data according to an embodiment of the present invention；And
Fig. 6 is the schematic diagram of the processing unit of another audio data according to an embodiment of the present invention.
Specific embodiment
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase Mutually combination.The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only The embodiment of the application a part, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people Member's every other embodiment obtained without making creative work, all should belong to the model of the application protection It encloses.
It should be noted that the description and claims of this application and term " first " in above-mentioned attached drawing, " Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way Data be interchangeable under appropriate circumstances, so as to embodiments herein described herein.In addition, term " includes " and " tool Have " and their any deformation, it is intended that cover it is non-exclusive include, for example, containing a series of steps or units Process, method, system, product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include without clear Other step or units listing to Chu or intrinsic for these process, methods, product or equipment.
Embodiment 1
The embodiment of the invention provides a kind of processing methods of audio data.Below from server side to the embodiment of the present invention Audio data processing method be introduced.
Figure 1A is a kind of flow chart of the processing method of audio data according to an embodiment of the present invention.As shown in Figure 1A, should Method the following steps are included:
Step S102 receives the first audio data of the first client of one or more of multiple client respectively, obtains To target audio data.
In the technical solution that above-mentioned steps S102 of the present invention is provided, server receives one in multiple client respectively Or the first audio data of multiple first clients, obtain target audio data.Wherein, client (Client) is to generate audio The source of data can be audio client, for example, for audio client used in multiterminal voice conferencing scene.It is optional Ground, the client of the embodiment may include audio receiving module (AudioRecvStream) and audio sending module (AudioSendStream), wherein the audio receiving module of client can be used for receiving other clients of server forwarding The audio data of generation, the audio sending module of client can be used for sending the audio data of client generation to server. Wherein, audio data is used to indicate client generated audio in the process of running, and can be includes a variety of audio datas Packets of audio data, for example, including the volume data (audioLevel) of audio, time that duration, the audio of audio start, audio The time etc. of end.Wherein, volume data is used to indicate the size of the sound of audio caused by client, can be by audio number It is carried according to the packet header of packet.
Optionally, the server of the embodiment receives the audio data that multiple client sends over, wherein server It also include audio receiving module and audio sending module, the audio receiving module of server can receive the audio of client transmission The audio data of the client received can be forwarded in other clients by the audio sending module of data, server.It should The server of embodiment can from the audio data of the multiple client received selected section client audio data, will It is forwarded in other clients, that is, the audio data that server is not all clients received is all forwarded to Client, but selectively forward the audio data of the part client in the audio data of all clients received. Optionally, server selects the biggish road the N audio data of volume in the audio data of all clients received, wherein N It can be the natural number more than or equal to 3, by the road N, audio data is determined as enlivening audio data, and multiple client is transmitted across The audio data that client in the audio data come in addition to the audio data of the road N generates is determined as inactive audio data.Its In, enliven audio data be the audio data of other clients can be forwarded to by server, and enliven audio data be not by Server is forwarded to the audio data of other clients, and client each in this way can receive in addition to the audio data of oneself The biggish road the N audio data of volume, thus when multiple users are talked simultaneously by client, it is ensured that each user is logical It crosses client and receives the big road the N audio data of volume ratio oneself, to achieve the purpose that reduce bandwidth, and may also reach up Filter the effect of the environmental disturbances sound of surrounding.
Step S104, in the case where the first object audio data of destination client has been lacked in target audio data, Obtain the dbjective state of destination client.
In the technical solution that above-mentioned steps S104 of the present invention is provided, server is receiving one in multiple client respectively First audio data of a or multiple first clients after obtaining target audio data, lacks in target audio data In the case where the first object audio data of destination client, the dbjective state of destination client is obtained, wherein multiple client Including destination client, dbjective state, which is used to indicate, to be forbidden being forwarded first object audio data, or is allowed first Target audio data forwarding extremely client associated with destination client.
In this embodiment, server can receive under normal circumstances all clients transmission audio data, but by In the audio data that the factors such as network, line fault may prevent client to generate from normally being sent to server, to go out Existing packet drop, so that server receives incomplete target audio data.The multiple client of the embodiment includes target Client, the destination client can be any client in multiple client, can produce first object audio data, such as The first object audio data is lacked in fruit target audio data, then the first object audio data is target audio data Missing packet, at this moment server obtains the dbjective state (isActived) of destination client, which can serve to indicate that taboo The audio data only generated to destination client is forwarded, that is, the first object audio data that destination client generates is not Belong to and enliven audio data (Active Audio), do not need to forward it in multiple client in addition to destination client Other clients；The dbjective state, which may be used to indicate that, allows the audio data for generating destination client to be forwarded to and mesh The associated client of client is marked, that is, the first object audio data that destination client generates, which belongs to, enlivens audio data, Need to forward it to other clients in addition to destination client in multiple client.
Optionally, the server of the embodiment includes endpoint (endpoint) corresponding with each client, which can With the dbjective state for recording corresponding client, which can be updated every the object time.
It should be noted that the embodiment does not limit the audio data lacked in target audio data, two can be lacked The audio data of a or more than two clients.
Step S106, dbjective state indicate forbid being forwarded first object audio data in the case where, forbid Destination request is sent to destination client.
In the technical solution that above-mentioned steps S106 of the present invention is provided, after the dbjective state for obtaining destination client, Dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to destination client send mesh Mark request, wherein destination request retransmits first object audio data for request target client.
In this embodiment, the dbjective state in destination client, which indicates, forbids turning first object audio data In the case where hair, then first object audio data is in current time output and inactive audio data, for example, lesser for volume Audio data is also not required to forward it in other clients even if server gets the first object audio data, because And dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to destination client send out Destination request is sent, which can be retransmission request NACK, that is, recipient is not connecing in the case where enabling NACK In the case where the data for receiving sender's transmission, the request retransmitted to data can be sent to sender.
The embodiment dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to Destination client sends destination request, can be avoided NACK request excessive under packet drop, to reduce server end The parsing at end is sent and received, the re-transmission of the audio data of client can be reduced, to reach the uplink band for reducing client The purpose of wide and server downlink bandwidth.
In this embodiment, the first audio number of the first client of one or more of multiple client is received respectively According to obtaining target audio data；The case where first object audio data of destination client has been lacked in target audio data Under, obtain the dbjective state of destination client, wherein multiple client includes destination client, and dbjective state is used to indicate taboo Only first object audio data is forwarded, or allows for first object audio data to be forwarded to related to destination client The client of connection；Dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to target Client sends destination request, wherein destination request retransmits first object audio data for request target client.Also It is to say, in the case where the first object audio data of destination client has been lacked in the target audio data got, determines First object audio data is the audio data (enlivening audio) for whether needing to be forwarded, if it is not, then forbidding to target Client request obtains the first object audio data of missing, can reduce and make requests re-transmission to the audio data of client, It solves the technical issues of low efficiency handled to audio data, and then has reached to improve and audio data is handled The technical effect of efficiency.
As an alternative embodiment, in step S104, after the dbjective state for obtaining destination client, this method Further include: indicating in dbjective state allows for first object audio data to be forwarded to client associated with destination client In the case where, destination request is sent to destination client；Receive the first object sound that destination client response destination request is sent Frequency evidence；First object audio data is forwarded to client associated with destination client.
In this embodiment, dbjective state can indicate to allow the audio data for generating destination client to be forwarded to and mesh The associated client of client is marked, that is, the first object audio data that the destination client generates is to enliven audio data, For example, first object audio data is audio data caused by the maximum client of volume in multiple client, in this way in mesh In the case that mark state instruction goes out to allow for first object audio data to be forwarded to client associated with destination client, clothes Device be engaged in destination client transmission destination request, request target client retransmits first object audio data, and destination client exists After receiving the destination request, the destination request is responded, first object audio data is forwarded to related to destination client The client of connection.
For example, multiple client is client 1, client 2, client 3 and client 4, wherein client 1 produces Volume indicated by raw audio data is maximum, is determined as enlivening audio data for its audio data, dbjective state allow by It is forwarded to client 2 associated with client 1, client 3 and client 4, so that client 2, client 3 and visitor Family end 4 can only receive the audio data except oneself outer maximum client 1 of sound.It should be noted that associated finger here It is associated between destination client and the client that can receive its audio data.
As an alternative embodiment, in step S102, one or more of multiple client the is received respectively Before first audio data of one client, this method further include: receive the second audio data of multiple client；According to each The second audio data of client determines the dbjective state of each client.
In this embodiment, in the first audio number for receiving the first client of one or more of multiple client respectively According to before, it is thus necessary to determine that the dbjective state of each client can receive the second audio data of multiple client, more by this The second audio data of a client determines the dbjective state of each client, for example, determining forbids producing each client The state that raw audio data is forwarded, or determining allows the audio data for generating each client to be forwarded to and target The state of the associated client of client.
Below to the second audio data according to each client of the embodiment, the dbjective state of each client is determined Method be introduced.
As an alternative embodiment, determining each client according to the second audio data of each client Dbjective state includes: that the first volume data is extracted from the second audio data of each client, wherein the first volume data It is used to indicate the volume for the audio that corresponding client issues；According to the first volume data of each client, from multiple clients The first object client of first object quantity associated with each client is determined in end, wherein each client is used In the second audio data for receiving each first object client associated with each client, each first object client The volume of audio that indicates of the first volume data be greater than the sound of audio indicated by the first volume data of the second client Amount, the second client are appointing in addition to the first object client of each client and first object quantity in multiple client One client；The state of the first object client of first object quantity is determined as first object state respectively, wherein first Dbjective state is used to indicate that allow the second audio data by first object client to be forwarded to related to first object client The client of connection；The state of the second destination client of the second destination number in multiple client is determined as the second mesh respectively Mark state, wherein the second destination client of the second destination number is the first mesh that first object quantity is removed in multiple client Mark the client except client, the second dbjective state be used to indicate forbid to the second audio data of the second destination client into Row forwarding.
In this embodiment, the second audio data of each client includes the first volume data, first volume data It can be carried by the packet header of second audio data, be used to indicate and generate the sound of audio that the client of second audio data is issued Amount.It, can be according to each client after extracting the first volume data in the second audio data from each client First volume data determines the first object visitor of first object quantity associated with each client from multiple client Family end, each client are used to receive the second audio data of each first object client associated with each client, The volume for the audio that first volume data of each first object client indicates is greater than the first volume of the second client The volume of audio indicated by data, and the second client is in multiple client except each client and first object quantity Any client except first object client, that is, each client can receive the first object number in addition to oneself Second audio data caused by the client of the larger volume of amount.It can be in multiple client, in addition to each client Client the first volume data indicated by volume be ranked up, therefrom determine volume biggish N (N≤3) a client End, determines it as client associated with each client, which can receive related to each client The second audio data that the client of connection generates.
For example, multiple client is client 1, client 2, client 3 and client 4, to client 1, client Volume indicated by second audio data caused by end 2, client 3 and client 4 is ranked up, caused by client 1 The sound of volume > client 3 indicated by second audio data caused by volume > client 2 indicated by second audio data The volume of amount > client 4.Destination number is set as 1, then for client 1, the second audio number caused by client 2 According in volume indicated by indicated volume second audio data caused by client 2, client 3 and client 4 most Greatly, then client 2 is client associated with client 1, and client 1 can receive the of the client 2 of server forwarding Two audio datas；For client 2, volume indicated by second audio data caused by client 1 client 1, Maximum in volume indicated by second audio data caused by client 3 and client 4, then client 1 is and 2 phase of client Associated client, client 2 can receive the second audio data of the client 1 of server forwarding；For client 3 Speech, volume indicated by second audio data caused by client 1 is caused by client 1, client 2 and the client 4 Maximum in volume indicated by second audio data, then client 1 is client associated with client 3, and client 3 can be with Receive the second audio data of the client 1 of server forwarding；For client 4, the second audio caused by client 1 In volume indicated by the second audio data caused by client 1, client 2 and client 3 of volume indicated by data Maximum, then client 1 is client associated with client 4, and client 4 can receive the client 1 of server forwarding Second audio data.
In the first object client for determining first object quantity associated with each client from multiple client After end, the state of the first object client of first object quantity can be determined as first object state respectively, this first Dbjective state is used to indicate that allow the second audio data by first object client to be forwarded to related to first object client The client of connection, that is, the second audio data of the first object client is to enliven audio data, for example, multiple client For client 1, client 2, client 3 and client 4, volume > visitor indicated by second audio data caused by client 1 The volume of volume > client 4 of volume > client 3 indicated by second audio data caused by family end 2, by client 1 Dbjective state, which is determined as being used to indicate, allows the second audio data by client 1 to be forwarded to client associated with client 1 2, client 3, the first object state of client 4 are held, second audio data caused by the client 1 is to enliven audio number According to.And for client 1, the dbjective state of client 2 is to be used to indicate to allow to forward the second audio data of client 2 To the first object state of client 1 associated with client 2.
In this embodiment, first object quantity associated with each client is being determined from multiple client After first object client, the state of the second destination client of the second destination number in multiple client is determined respectively For the second dbjective state, which is used to indicate second forbidden to the second destination client of the second destination number Audio data is forwarded, and the second destination client of second destination number is in multiple client except first object quantity Client except first object client, that is, the second audio number that the second destination client of the second destination number generates According to for inactive audio data.For example, multiple client is client 1, client 2, client 3 and client 4, client 1 Volume > client indicated by second audio data caused by volume > client 2 indicated by generated second audio data The volume of the volume > client 4 at end 3, the dbjective state of client 1, which is determined as being used to indicate, to be allowed the second of client 1 Audio data is forwarded to the first object state of client 2 associated with client 1, client 3, client 4, by client 2, client 3, the dbjective state of client 4 are determined as the second dbjective state for forbidding being forwarded its second audio data, The second audio data that the client 2, client 3, client 4 generate is inactive audio data.And to client 1 and Speech, the dbjective state of client 2 are to be used to indicate that allow the second audio data by client 2 to be forwarded to related to client 2 The first object state of the client 1 of connection, the dbjective state of client 1, client 3 and client 4 are to forbid to its second sound For frequency according to the second dbjective state being forwarded, the second audio data that client 1, client 3 and client 4 generate is non-live The audio data of jump, to realize the determination to the dbjective state of multiple client, to reduce the audio data of client Retransmit, thus achieve the purpose that reduce client upstream bandwidth and server downlink bandwidth, improve to audio data into The efficiency of row processing
As an alternative embodiment, forbid after sending destination request to destination client in step S106, it should Method further include: receive the third audio data of multiple client；According to the third audio data of each client, to each visitor The dbjective state at family end is updated.
In this embodiment, after forbidding sending destination request to destination client, over time, client Dbjective state can be changed with the audio data of generation, which can be in the time later to each client The dbjective state at end is updated.The second audio data that can receive multiple client passes through the second of multiple client Audio data determines the dbjective state of each client, for example, determine the audio data forbidding generating each client into The state of row forwarding, or determine that allow the audio data for generating each client to be forwarded to associated with destination client The state of client.The dbjective state of each client is updated, updated dbjective state can be with the mesh before update Mark state is identical or different.The dbjective state of client can be updated into the endpoint corresponding with client into server (endpoint) in.
As an alternative embodiment, according to the third audio data of each client, to the mesh of each client It includes: to extract the second volume data from the third audio data of each client that mark state, which is updated, wherein the second sound Amount data are used to indicate the volume for the audio that corresponding client issues；According to the second volume data of each client, from more The third destination client of third destination number associated with each client is determined in a client, wherein Mei Geke Family end is used to receive the third audio data of each third destination client associated with each client, each third target The volume for the audio that second volume data of client indicates is greater than sound indicated by the second volume data of third client The volume of frequency, third client be in multiple client except each client and third destination number third destination client it Outer any client；The state of the third destination client of third destination number is updated to first object state respectively, In, first object state, which is used to indicate, allows the audio data by third destination client to be forwarded to and third destination client phase Associated client；The state of the 4th destination client of the 4th destination number in multiple client is updated to second respectively Dbjective state, wherein the 4th destination client of the 4th destination number is the third that third destination number is removed in multiple client Client except destination client, the second dbjective state, which is used to indicate, to be forbidden carrying out the audio data of the 4th destination client Forwarding.
In this embodiment, the third audio data of each client includes the second volume data, second volume data It can be carried by the packet header of third audio data, be used to indicate and generate the sound of audio that the client of third audio data is issued Amount.It, can be according to each client after extracting the second volume data in the third audio data from each client Second volume data determines the third target visitor of third destination number associated with each client from multiple client Family end, each client are used to receive the third audio data of each third destination client associated with each client, The volume for the audio that second volume data of each third destination client indicates is greater than the second volume of third client The volume of audio indicated by data, and third client is in multiple client except each client and third destination number Any client except third destination client, that is, each client can receive the third number of targets in addition to oneself Third audio data caused by the client of the larger volume of amount.It can be in multiple client, in addition to each client Client the second volume data indicated by volume be ranked up, therefrom determine volume biggish N (N≤3) a client End, determines it as client associated with each client, which can receive related to each client The third audio data that the client of connection generates.
In the third target customer for determining third destination number associated with each client from multiple client After end, the state of the third destination client of third destination number can be updated to first object state respectively, this first Dbjective state be used to indicate allow for the third audio data of third destination client to be forwarded to it is related to third destination client The client of connection, that is, the second audio data of the third destination client is to enliven audio data.
In this embodiment, third destination number associated with each client is being determined from multiple client After third destination client, the state of the 4th destination client of the 4th destination number in multiple client is updated respectively For the second dbjective state, which is used to indicate the third for forbidding the 4th destination client to the 4th destination number Audio data is forwarded, and the 4th destination client of the 4th destination number is in multiple client except third destination number Client except first object client, that is, the second audio number that the 4th destination client of the 4th destination number generates According to for inactive audio data, so that the update to the dbjective state of multiple client is realized, to reduce the audio of client The re-transmission of data improves to achieve the purpose that reduce the upstream bandwidth of client and the downlink bandwidth of server to audio The efficiency that data are handled.
As an alternative embodiment, the third audio data for receiving multiple client includes: every the object time Section receives the third audio data of multiple client；According to the third audio data of each client, to the mesh of each client It includes: third audio data every target time section according to each client that mark state, which is updated, to each client Dbjective state is updated.
In this embodiment, it when the dbjective state to multiple client is updated, can be connect every target time section The third audio data of multiple client is received, for example, the third audio data that multiple client generates was received every 15 minutes, this Place is with no restrictions.It, can be according to each client received after receiving the third audio data of multiple client Third audio data is according to the method described above updated the dbjective state of each client, so that pressing every target time section According to the third audio data of each client, the dbjective state of each client is updated, to reduce the audio of client The re-transmission of data improves to achieve the purpose that reduce the upstream bandwidth of client and the downlink bandwidth of server to audio The efficiency that data are handled.
As an alternative embodiment, in step S104, before the dbjective state for obtaining destination client, this method Further include: obtain object listing, wherein the mark letter of the first audio data in object listing including each first client Breath；In object listing do not include first object audio data identification information in the case where, determine in target audio data lack First object audio data is lost.
In this embodiment, multiple client sends packets of audio data to server, the case where object listing is non-empty Under, which includes the identification information that server each of receives the first audio data that the first client is sent, Since audio data is all to send in chronological order to server, sequence number is carried, the identification information of the embodiment can be with For sequence number, for example, multiple client is 10 clients, then multiple client transmits Sequence Number from 1 to 10 to server Audio data.After obtaining object listing, in object listing including first object audio data identification information feelings It under condition, determines and has lacked first object audio data in target audio data, which is missing packet, the reality First object audio data can be determined as any audio data lacked in object listing by applying example, for example, when server does not have There is the audio data for receiving Serial No. 5, and packets of audio data after Serial No. 5 and before has received, then services Device can determine that the audio data of Serial No. 5 is to lack packet.
After having lacked first object audio data in determining target audio data, indicates and forbid pair in dbjective state In the case that first object audio data is forwarded, forbid sending destination request to destination client, be indicated in dbjective state In the case where allowing for first object audio data to be forwarded to client associated with destination client out, to destination client Destination request is sent, the first object audio data that destination client response destination request is sent is received；By first object audio Data forwarding is to client associated with destination client, so that the re-transmission of the audio data of client is reduced, to reach The purpose of the upstream bandwidth of client and the downlink bandwidth of server is reduced, the efficiency handled audio data is improved.
As an alternative embodiment, step S102, receives one or more of multiple client first respectively First audio data of client, obtaining target audio data includes: the one or more obtained under conference scenario respectively First audio data of the first client, obtains target audio data.
The audio data processing method of the embodiment is suitable for conference scenario, for example, the conference scenario is the meeting of multiterminal voice Discuss scene.Server obtains the first audio data of the first client of one or more under conference scenario respectively, obtains Target audio data, in the case where the first object audio data of destination client has been lacked in target audio data, service Device obtains the dbjective state of destination client, and the feelings for forbidding being forwarded to first object audio data are indicated in dbjective state Under condition, server, which is forbidden sending to the destination client under conference scenario, is used for request target client for first object sound Frequency according to re-transmission destination request.
For example, under conference scenario, client 1, client 2, client 3, client are used respectively there are four user Meeting is added in end 4.Server obtains the audio data that client 1, client 3, client 4 are sent, the audio data of client 2 There is packet drop, and the dbjective state of client 2 indicates and server is forbidden to turn the audio data of client 2 Hair, then server, which is forbidden sending to the client 2 under conference scenario, is used for request target client for first object audio The destination request that data retransmit can reduce and make requests re-transmission to the audio data of the client under conference scenario, into And achieve the effect that improve the efficiency for handling the audio data under conference scenario.
From terminal side, the processing method to the audio data of the embodiment of the present invention is introduced below.
Figure 1B is the flow chart of the processing method of another audio data according to an embodiment of the present invention.As shown in Figure 1B, Method includes the following steps:
Step S202 sends target audio data to server.
In the technical solution that above-mentioned steps S202 of the present invention is provided, terminal to server sends target audio data, In, target audio data include the first audio data of the first client of one or more of multiple client.
In this embodiment, terminal to server sends target audio data, and server receives the target sound frequency respectively According to.Wherein, client is to generate the source of audio data, can be audio client, for example, in multiterminal voice conferencing scene Used audio client.Optionally, the client of the embodiment may include audio receiving module and audio sending module, Wherein, the audio receiving module of client can be used for receiving the audio data that other clients of server forwarding generate, visitor The audio sending module at family end can be used for sending the audio data of client generation to server.
Step S204 has lacked the first object audio data of destination client in target audio data, and in target The dbjective state of client indicates the case where forbidding server to be forwarded the first object audio data of destination client Under, forbid destination client to receive the destination request that server is sent.
In the technical solution that above-mentioned steps S204 of the present invention is provided, multiple client includes destination client, and target is asked It asks and retransmits first object audio data for request target client.
In this embodiment, terminal can to server send all clients generate audio data, but due to network, The audio data that the factors such as line fault may prevent client to generate to server from normally sending, to packet loss feelings occur Condition.The multiple client of the embodiment includes destination client, which can be any visitor in multiple client Family end can produce first object audio data, should if having lacked the first object audio data in target audio data First object audio data is the missing packet of target audio data, and the dbjective state in destination client, which indicates, forbids server In the case where being forwarded to the first object audio data of destination client, terminal forbids destination client to receive server hair The destination request sent, that is, the first object audio data that destination client generates is not belonging to enliven audio data (Active Audio), other clients in addition to destination client in multiple client are not needed to forward it to.
It should be noted that the embodiment does not limit the audio data lacked in target audio data, two can be lacked The audio data of a or more than two clients.
In this embodiment, the dbjective state in destination client, which indicates, forbids turning first object audio data In the case where hair, then first object audio data is in current time output and inactive audio data, for example, lesser for volume Audio data is also not required to forward it in other clients even if server gets the first object audio data, because And dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to destination client send out Destination request is sent, which can be retransmission request NACK, that is, recipient is not connecing in the case where enabling NACK In the case where the data for receiving sender's transmission, the request retransmitted to data can be sent to sender.
The embodiment dbjective state indicate forbid being forwarded first object audio data in the case where, forbid to Destination client sends destination request, can be avoided NACK request excessive under packet drop, to reduce server end The parsing at end is sent and received, the re-transmission of the audio data of client can be reduced, to reach the uplink band for reducing client The purpose of wide and server downlink bandwidth.
As a kind of optional example, which simply requests in the case where there is the audio data packet loss of client The audio data of the client of the re-transmission biggish fixed quantity of sound, and the audio data of the lesser client of volume, that is, use Showed packet loss, also not request retransmission when.
The embodiment sends target audio data to server, in target audio shortage of data the first of destination client In the case where target audio data, determine that first object audio data is the audio number for whether needing to be forwarded by server According to (enlivening audio), if it is not, then destination client is forbidden to receive the first object audio data for request missing Request, the audio data so as to reduce to client makes requests re-transmission, reduces server end and send and receive end The data processing amount of parsing, and the re-transmission of the packets of audio data of client can be reduced, to reduce the uplink band of client Wide and server downlink bandwidth solves the technical issues of low efficiency handled to audio data, and then reaches Improve the technical effect of the efficiency handled audio data.
Embodiment 2
It is illustrated below with reference to scheme of the preferred embodiment to the embodiment of the present invention.Specifically with multiterminal semanteme meeting Audio data transmission method in view scene is illustrated.
In this embodiment, under multiterminal voice conferencing scene, there is multiple client, each client is for generating sound Frequency evidence, and the audio data is sent to server, Server Default can receive the audio number that all clients send over According to, wherein audio data is at least used to indicate the volume of the audio of client sending.In the audio data that client sends over In, server can select the wherein biggish road the N audio data of volume, and by the road N, audio data is determined as enlivening audio data, and Audio data will be enlivened to send to other clients, so that each client can only receive in addition to the audio data of oneself, The maximum road the N audio data of volume, thus when multiple users are talked simultaneously by client, it is ensured that each user passes through Client receives the big road the N audio data of volume ratio oneself, reaches reduction bandwidth, while filtering out the environmental disturbances sound of surrounding Effect.
In this embodiment, it is server when not receiving data that packet loss, which retransmits (NACK), sends and retransmits to sender Request, that is, NACK is requested.It notifies sender to retransmit data by the retransmission request, message can be sent to notice side, For example, " I does not receive message ".Optionally, the packets of audio data of the embodiment is all sent in chronological order, can be with Serial number is all had, which, which can be, carries out ascending order arrangement and continuous serial number by sending time.The NACK of the embodiment is requested In carry loss packets of audio data sequence number, thus allow sender retransmit lose packets of audio data.
Fig. 2 is the signal that the audio data between a kind of server and client side according to an embodiment of the present invention is transmitted Figure.As shown in Fig. 2, server 20, client (Client) 11, client 12, client 13, client 14, wherein client 11 include audio sending module (AudioSendStream) 111 and audio receiving module (AudioRecvStream) 112, client End 12 includes audio sending module 121 and audio receiving module 122, and client 13 includes that audio sending module 131 and audio connect Module 132 is received, client 14 includes audio sending module 141 and audio receiving module 142；Server includes endpoint (endpoint) 21, endpoint 22, endpoint 23 and endpoint 24, wherein endpoint 21 includes audio receiving module 211, audio transmission mould Block 212 and endpoint input pin 213, endpoint 22 include that audio receiving module 221, audio sending module 222 and endpoint input are drawn Foot 223, endpoint 23 include audio receiving module 231, audio sending module 232 and endpoint input pin 233, and endpoint 24 includes sound Frequency receiving module 241, audio sending module 242 and endpoint input pin 243, server further include filter (exceptSelfFilter)25。
In this embodiment, client 11 is corresponding with the endpoint 21 of server 20, and endpoint 21 is for server 20 and visitor The transmission of audio data between family end 11；Client 12 is corresponding with the endpoint 22 of server 20, and endpoint 22 is used for server The transmission of audio data between 20 and client 12；Client 13 is corresponding with the endpoint 13 of server, and endpoint 13 is for taking The transmission for the audio data being engaged between device 20 and client 13；Client 14 is corresponding with the endpoint 14 of server, and endpoint 14 is used The transmission of audio data between server 20 and client 14.
Client 11, client 12, client 13, client 14 and the corresponding endpoint 21 of the embodiment, endpoint 22, end Point 23 and endpoint 24 have audio sending module (audio sending module 121, audio sending module 131, audio sending module 141, Audio sending module 212, audio sending module 222, audio sending module 232 and audio sending module 242) and audio reception mould (audio receiving module 112, audio receiving module 122, audio receiving module 132, audio receiving module 142, audio receive mould to block Block 211, audio receiving module 231 and audio receiving module 241), wherein audio sending module namely audio streams module, For the transmission of audio data, audio receiving module namely audio stream receiving module, the reception for audio data.
In the conference scenario of the embodiment, meeting, clothes are added in client 11, client 12, client 13, client 14 Business device receives the audio data that client 11, client 12, client 13, client 14 generate respectively in the session.Wherein, The volume for the audio data that client 11 is currently generated is greater than the audio data that client 12, client 13, client 14 generate Volume.When multiple users, which pass through corresponding client, to talk simultaneously, the attention of those who answer is limited, and server can be with Only the audio data of the selection maximum client 11 of volume is forwarded (N=1).That is, although server has received client The audio data that end 12, client 13, client 14 generate respectively in the session, but there is no by it to other clients End is forwarded, and has been only forwarding the audio data of the generation of client 11, client 12, client 13, client 14 receive Audio data indicated by audio volume be all larger than oneself generation audio data indicated by audio volume, that is, Other clients can only receive in addition to oneself, and the maximum audio data all the way of the volume of audio may be used also to reduce bandwidth To filter out the environmental disturbances sound of surrounding.
Optionally, in this embodiment, although the volume of audio indicated by the audio data of client 11 is in client 12, maximum in volume indicated by the audio data of client 13, client 14, but client 11 cannot receive the sound of oneself Frequency evidence.Server can select the audio number of the maximum client of volume from client 12, client 13, client 14 According to will be stored by input pin 213 to filter 25, and client 11 be forwarded it to by filter 25, to make client By in addition to oneself, the maximum audio data all the way of the volume of audio may filter out to reduce bandwidth at 11, end The environmental disturbances sound of surrounding.
Fig. 3 is the flow chart of another audio data processing method according to an embodiment of the present invention.As shown in figure 3, the party Method the following steps are included:
Step S301, server obtain the maximum road the N audio data of volume in active conference.
In this embodiment, server obtains the multi-path audio-frequency data that multiple client sends over, wherein every road audio Data and each client correspond.Selected from the multi-path audio-frequency data that multiple client sends over wherein volume it is larger N the natural number of N≤3 (can be) audio data, the road the N audio data can be determined as to enliven audio data (active audio)。
Optionally, in this embodiment, the audio data that client is sent can be packets of audio data, packets of audio data Packet header can carry the information volume (audioLevel) for the audio that client currently generates, which indicates the sound of audio Measure size.The packets of audio data that server can be sent every target time section for each client in conference scenario, mentions The information volume in each packets of audio data is taken, the information volume in each packets of audio data is ranked up, information volume is taken The corresponding client of the maximum road the N packets of audio data of indicated volume, as can produce the client for enlivening audio. Wherein, above-mentioned target time section namely update cycle of audio data is enlivened.
Step S302, server by every audio data of each client whether be enliven audio data state update arrive Each endpoint of server.
The maximum road the N audio data of volume in obtaining active conference, and be determined as the road N audio data to enliven audio number It whether is to enliven the state of audio data to update to each end of server by every audio data of each client according to later Point updates the state to corresponding with client 11 for example, the audio data that above-mentioned client 11 is sent is to enliven audio data Endpoint 21 in, and client 12, client 13, client 14 be inactive audio data, by the state respectively updates arrive pair In endpoint 22, endpoint 23 and the endpoint 24 answered.
In this embodiment, in the audio data for determining multiple client enliven audio data after, update clothes Connection status between the filter (exceptSelfFilter) and the endpoint input pin in different endpoints of business device can incite somebody to action Endpoint input pin in endpoint corresponding with inactive audio data removes, for example, above-mentioned client 12, client 13, visitor The audio data at family end 14 be inactive audio data, will endpoint input pin 223 corresponding with client 12, with client 13 Corresponding endpoint input pin 233, the corresponding endpoint input pin 243 of client 14 removes, thus not by client 12, client The audio data of 13, client 14 is held to be forwarded in other clients；New is enlivened the corresponding endpoint of audio number by the embodiment Input pin is connected with filter, for example, the audio data of above-mentioned client 11 is to enliven audio data, client 11 is right The endpoint input pin 213 answered is connected with filter 25, and such server can will enliven audio data and be transmitted to other visitors Family end.
Step S303, the audio receiving module of each endpoint of server, according to the audio data of corresponding client whether For the packet drop for enlivening audio data and packets of audio data, it is determined whether request retransmission packets of audio data.
It optionally, in this embodiment, whether is to enliven audio number by every audio data of each client in server According to state update to after each endpoint of server, the audio receiving module of each endpoint of server is subsequent to be can receive The audio data that corresponding client sends over.Optionally, the audio receiving module of each endpoint of server determines whether The audio data that corresponding client sends over is received, if not receiving the audio number that corresponding client sends over According to then judging whether the state of audio data of the client of endpoint record is to enliven audio data, in corresponding client Audio data be enliven audio data in the case where, to corresponding client request retransmission of audio data packet, in corresponding visitor In the case that the audio data at family end is inactive audio data, not to corresponding client request retransmission of audio data packet, from And avoid in inactive audio data there are in the case where packet loss, server still can give generation the inactive audio data Client sends NACK request, causes useless NACK to request and retransmit, so as to avoid causing bandwidth resources to waste.
Optionally, in this embodiment, sending NACK request is carried out in the audio receiving module of each endpoint, It is the case where judging whether there is audio data packet loss, optional that is, after server receives the audio data of multiple client Ground, the packets of audio data of the embodiment are all sent in chronological order, can all have a serial number, the serial number can be by Sending time carries out ascending order arrangement and continuous serial number, for example, multiple client has sent audio data of the sequence number from 1 to 10 Packet, when server does not receive the packets of audio data of Serial No. 5, and the packets of audio data before and after Serial No. 5 is all It has received, then server can determine that the packets of audio data of Serial No. 5 is lost, if generating the audio number of Serial No. 5 According to client, be confirmed as generating in the nearest update cycle for enlivening audio data and enlivened the client of audio data End, then server needs to send NACK request to the client, and the sequence of the packets of audio data of loss is carried in NACK request Row number, for example, being 5, so that client be allowed to retransmit the packets of audio data lost.Client receive NACK request after, The packets of audio data of loss can be retransmitted to server by the audio sending module of the client, so that server receives work Jump packets of audio data.
Fig. 4 is the flow chart of another audio data processing method according to an embodiment of the present invention.As shown in figure 4, the party Method the following steps are included:
Step S401 obtains the packets of audio data that multiple client is sent.
Step S402, from for sending the missing packet number for obtaining packets of audio data in the module that NACK is requested.
The missing packet number (nackList) of the embodiment is calculated according to the sequence number of the packets of audio data received Missing packet number out, for example, multiple client has sent packets of audio data of the sequence number from 1 to 10, if server does not have There is a packets of audio data for receiving Serial No. 5, and the later packets of audio data of Serial No. 5 has received, then server can To determine that the packets of audio data of Serial No. 5 is lost.
Step S403 judges to lack whether packet number is empty.
Step S404 judges whether the state of client corresponding with missing packet number is to generate to enliven audio data State.
In the case where lacking packet number is not sky, whether the state of judgement client corresponding with missing packet number To generate the state for enlivening audio data.
Step S405, the audio data of request retransmission client corresponding with missing packet number.
If it is judged that the state of client corresponding with missing packet number is to generate the state for enlivening audio data, then The audio data of request retransmission client corresponding with missing packet number.
It is sky in missing packet number, or the state of client corresponding with missing packet number is not to generate to enliven sound The state of frequency evidence then terminates the process of the method handled audio data.
The embodiment can simply request the method for retransmitting and enlivening audio with a kind of server end under conference scenario, in server There are in the case where dropped audio data, determine whether the audio data lost is to need to be forwarded in the audio data got Other clients enliven audio data, are retransmitted with the audio data for determining the need for loss, if it is not required, then Forbid the audio data lost to destination client request, can be avoided to client and be transmitted across more NACK requests, thus The data processing amount that server end sends and receives the parsing at end is reduced, and the weight of the packets of audio data of client can be reduced It passes, to reduce the upstream bandwidth of client and the downlink bandwidth of server.
It should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions It is executed in computer system, although also, logical order is shown in flow charts, and it in some cases, can be with not The sequence being same as herein executes shown or described step.
Embodiment 3
The embodiment of the invention also provides a kind of processing units of audio data.It should be noted that the sound of the embodiment The processing unit of frequency evidence can be used for executing the processing method of the audio data shown in figure 1A of the embodiment of the present invention.
Fig. 5 is a kind of schematic diagram of the processing unit of audio data according to an embodiment of the present invention.As shown in figure 5, the sound The processing unit 500 of frequency evidence includes: receiving unit 10, acquiring unit 20 and forbids unit 30.
Receiving unit 10, for receiving the first audio number of the first client of one or more of multiple client respectively According to obtaining target audio data.
Acquiring unit 20, the feelings of the first object audio data for having lacked destination client in target audio data Under condition, the dbjective state of destination client is obtained, wherein multiple client includes destination client, and dbjective state is used to indicate Forbid being forwarded first object audio data, or allows to be forwarded to first object audio data and destination client phase Associated client.
Forbid unit 30, forbids the case where being forwarded to first object audio data for indicating in dbjective state Under, forbid sending destination request to destination client, wherein destination request is used for request target client for first object audio Data retransmit.
It should be noted that the receiving unit 10 in the embodiment can be used for executing the step in the embodiment of the present application S102, the acquiring unit 20 in the embodiment can be used for executing the step S104 in the embodiment of the present application, in the embodiment Unit 30 is forbidden to can be used for executing the step S106 in the embodiment of the present application.
The embodiment has lacked the first object audio data of destination client in the target audio data got In the case of, determine that first object audio data is the audio data (enlivening audio) for whether needing to be forwarded, if it is not, then Forbid the first object audio data lacked to destination client request, the audio data progress to client can be reduced Request retransmission solves the technical issues of low efficiency handled to audio data, and then has reached raising to audio data The technical effect of the efficiency handled.
The embodiment of the invention also provides a kind of processing units of audio data.It should be noted that the sound of the embodiment The processing unit of frequency evidence can be used for executing the processing method of audio data shown in Figure 1B of the embodiment of the present invention.
Fig. 6 is the schematic diagram of the processing unit of another audio data according to an embodiment of the present invention.As shown in fig. 6, should The processing unit 600 of audio data includes: transmission unit 40 and forbids unit 50.
Transmission unit 40, for sending target audio data to server, wherein target audio data include multiple clients First audio data of the first client of one or more of end.
Forbid unit 50, for having lacked the first object audio data of destination client in target audio data, and Dbjective state in destination client, which indicates, forbids server to be forwarded the first object audio data of destination client In the case where, forbid destination client to receive the destination request that server is sent, wherein multiple client includes target customer End, destination request retransmit first object audio data for request target client.
It should be noted that the transmission unit 40 in the embodiment can be used for executing the step in the embodiment of the present application S202 forbids unit 50 to can be used for executing the step S204 in the embodiment of the present application in the embodiment.
The embodiment sends target audio data to server, in target audio shortage of data the first of destination client In the case where target audio data, determine that first object audio data is the audio number for whether needing to be forwarded by server According to (enlivening audio), if it is not, then destination client is forbidden to receive the first object audio data for request missing Request, the audio data so as to reduce to client makes requests re-transmission, reduces server end and send and receive end The data processing amount of parsing, and the re-transmission of the packets of audio data of client can be reduced, to reduce the uplink band of client Wide and server downlink bandwidth solves the technical issues of low efficiency handled to audio data, and then reaches Improve the technical effect of the efficiency handled audio data.
Embodiment 4
The embodiment of the invention also provides a kind of storage mediums.The storage medium includes the program of storage, wherein in program Equipment executes the audio data processing method in the embodiment of the present invention where controlling storage medium when operation.
Embodiment 5
The embodiment of the invention also provides a kind of processors.The processor is for running program, wherein program is held when running Audio data processing method in the row embodiment of the present invention.
Obviously, those skilled in the art should be understood that each module of the above invention or each step can be with general Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored Be performed by computing device in the storage device, perhaps they are fabricated to each integrated circuit modules or by they In multiple modules or step be fabricated to single integrated circuit module to realize.In this way, the present invention is not limited to any specific Hardware and software combines.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.

Claims (15)

1. a kind of processing method of audio data characterized by comprising
The first audio data for receiving the first client of one or more of multiple client respectively, obtains target sound frequency According to；
In the case where having lacked the first object audio data of destination client in the target audio data, the mesh is obtained Mark the dbjective state of client, wherein the multiple client includes the destination client, and the dbjective state is used to indicate Forbid being forwarded the first object audio data, or allow for the first object audio data to be forwarded to it is described The associated client in destination client；
The dbjective state indicate forbid being forwarded the first object audio data in the case where, forbid to described Destination client sends destination request, wherein the destination request is for requesting the destination client by the first object Audio data retransmits.
2. the method according to claim 1, wherein after the dbjective state for obtaining the destination client, The method also includes:
The dbjective state indicate allow for the first object audio data to be forwarded to it is related to the destination client In the case where the client of connection, the destination client Xiang Suoshu sends the destination request；
It receives the destination client and responds the first object audio data that the destination request is sent；
The first object audio data is forwarded to the client associated with the destination client.
3. the method according to claim 1, wherein receiving one or more of multiple client the respectively Before first audio data of one client, the method also includes:
Receive the second audio data of the multiple client；
According to the second audio data of each client, the dbjective state of each client is determined.
4. according to the method described in claim 3, it is characterized in that, according to each client the second audio number According to determining that the dbjective state of each client includes:
The first volume data is extracted from the second audio data of each client, wherein first volume Data are used to indicate the volume for the audio that the corresponding client issues；
According to first volume data of each client, determined from the multiple client and each visitor The first object client of the associated first object quantity in family end, wherein each client is for receiving and each institute State the second audio data of the associated each first object client of client, each first object client The volume for the audio that first volume data at end indicates is greater than indicated by first volume data of the second client Audio volume, second client be the multiple client in remove each client and the first object number Any client except the first object client of amount；
The state of the first object client of the first object quantity is determined as first object state respectively, wherein The first object state, which is used to indicate, allows the second audio data by the first object client to be forwarded to and institute State the associated client of first object client；
The state of the second destination client of the second destination number in the multiple client is determined as the second target respectively State, wherein second destination client of second destination number is that first mesh is removed in the multiple client The client except the first object client of quantity is marked, second dbjective state, which is used to indicate, forbids to described second The second audio data of destination client is forwarded.
5. the method according to claim 1, wherein forbid to the destination client send destination request it Afterwards, the method also includes:
Receive the third audio data of the multiple client；
According to the third audio data of each client, the dbjective state of each client is carried out more Newly.
6. according to the method described in claim 5, it is characterized in that, according to each client the third audio number According to being updated to the dbjective state of each client and include:
The second volume data is extracted from the third audio data of each client, wherein second volume Data are used to indicate the volume for the audio that the corresponding client issues；
According to second volume data of each client, determined from the multiple client and each visitor The third destination client of the associated third destination number in family end, wherein each client is for receiving and each institute State the third audio data of the associated each third destination client of client, each third target customer The volume for the audio that second volume data at end indicates is greater than indicated by second volume data of third client Audio volume, the third client be the multiple client in remove each client and the third number of targets Any client except the third destination client of amount；
The state of the third destination client of the third destination number is updated to first object state respectively, wherein The first object state, which is used to indicate, to be allowed for the third audio data of the third destination client to be forwarded to and institute State the associated client in third destination client；
The state of the 4th destination client of the 4th destination number in the multiple client is updated to the second target respectively State, wherein the 4th destination client of the 4th destination number is that the third mesh is removed in the multiple client The client except the third destination client of quantity is marked, second dbjective state, which is used to indicate, forbids to the described 4th The third audio data of destination client is forwarded.
7. according to the method described in claim 5, it is characterized in that, obtaining the third packets of audio data of the multiple client It includes:
The third audio data of the multiple client is obtained every target time section.
8. the method according to the description of claim 7 is characterized in that according to the third audio number of each client According to being updated to the dbjective state of each client and include:
Every the target time section according to the third audio data of each client, to each client The dbjective state is updated.
9. method as claimed in any of claims 1 to 8, which is characterized in that obtaining the destination client Before dbjective state, the method also includes:
Obtain object listing, wherein include first audio data of each first client in the object listing Identification information；
In the object listing do not include the first object audio data identification information in the case where, determine the target The first object audio data has been lacked in audio data.
10. method as claimed in any of claims 1 to 8, which is characterized in that receive in multiple client respectively First audio data of one or more first clients, obtaining target audio data includes:
First audio data for obtaining one or more of first clients under conference scenario respectively, obtains institute State target audio data.
11. a kind of processing method of audio data characterized by comprising
Target audio data are sent to server, wherein the target audio data include one or more in multiple client First audio data of a first client；
The first object audio data of destination client is lacked in the target audio data, and in the destination client Dbjective state indicate the feelings for forbidding the server to be forwarded the first object audio data of the destination client Under condition, the destination client is forbidden to receive the destination request that the server is sent, wherein the multiple client includes institute Destination client is stated, the destination request is for requesting the destination client to retransmit the first object audio data.
12. a kind of processing unit of audio data characterized by comprising
Receiving unit is obtained for receiving the first audio data of the first client of one or more of multiple client respectively To target audio data；
Acquiring unit, the case where for having lacked the first object audio data of destination client in the target audio data Under, obtain the dbjective state of the destination client, wherein the multiple client includes the destination client, the mesh Mark state, which is used to indicate, to be forbidden being forwarded the first object audio data, or is allowed the first object audio number According to being forwarded to the client associated with the destination client；
Forbid unit, forbids the case where being forwarded to the first object audio data for indicating in the dbjective state Under, forbid sending destination request to the destination client, wherein the destination request is for requesting the destination client will The first object audio data retransmits.
13. a kind of processing unit of audio data characterized by comprising
Transmission unit, for sending target audio data to server, wherein the target audio data include multiple client One or more of the first client the first audio data；
Forbid unit, for having lacked the first object audio data of destination client in the target audio data, and The dbjective state of the destination client indicates the first object audio number for forbidding the server to the destination client In the case where being forwarded, the destination client is forbidden to receive the destination request that the server is sent, wherein described more A client includes the destination client, and the destination request is for requesting the destination client by the first object sound Frequency is according to re-transmission.
14. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program When control the storage medium where equipment perform claim require any one of 1 to 11 described in method.
15. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run Benefit require any one of 1 to 11 described in method.
CN201910408187.1A 2019-05-15 2019-05-15 Audio data processing method and device, storage medium and processor Active CN110166185B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910408187.1A CN110166185B (en) 2019-05-15 2019-05-15 Audio data processing method and device, storage medium and processor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910408187.1A CN110166185B (en) 2019-05-15 2019-05-15 Audio data processing method and device, storage medium and processor

Publications (2)

Publication Number Publication Date
CN110166185A true CN110166185A (en) 2019-08-23
CN110166185B CN110166185B (en) 2021-07-09

Family

ID=67631008

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910408187.1A Active CN110166185B (en) 2019-05-15 2019-05-15 Audio data processing method and device, storage medium and processor

Country Status (1)

Country Link
CN (1) CN110166185B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111049792A (en) * 2019-10-08 2020-04-21 广州视源电子科技股份有限公司 Audio transmission method and device, terminal equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1199832A1 (en) * 2000-10-20 2002-04-24 Alcatel Transmission procedure to avoid useless retransmissions
US20040148396A1 (en) * 2001-06-01 2004-07-29 Michael Meyer Method and transmitter for an efficient packet data transfer in a transmission protocol with repeat requests
CN1972223A (en) * 2006-12-13 2007-05-30 中兴通讯股份有限公司 A method and apparatus for monitoring multicast data flow
CN105376516A (en) * 2015-10-19 2016-03-02 深圳市航安网络科技有限公司 Mobile video conference system and implementing method thereof
US20160219100A1 (en) * 2015-01-22 2016-07-28 Alibaba Group Holding Limited Method and Apparatus of Processing Retransmission Request in Distributed Computing
CN109510905A (en) * 2018-12-06 2019-03-22 中通天鸿（北京）通信科技股份有限公司 The sound mixing method and system of multi-path voice

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1199832A1 (en) * 2000-10-20 2002-04-24 Alcatel Transmission procedure to avoid useless retransmissions
US20040148396A1 (en) * 2001-06-01 2004-07-29 Michael Meyer Method and transmitter for an efficient packet data transfer in a transmission protocol with repeat requests
CN1972223A (en) * 2006-12-13 2007-05-30 中兴通讯股份有限公司 A method and apparatus for monitoring multicast data flow
US20160219100A1 (en) * 2015-01-22 2016-07-28 Alibaba Group Holding Limited Method and Apparatus of Processing Retransmission Request in Distributed Computing
CN105376516A (en) * 2015-10-19 2016-03-02 深圳市航安网络科技有限公司 Mobile video conference system and implementing method thereof
CN109510905A (en) * 2018-12-06 2019-03-22 中通天鸿（北京）通信科技股份有限公司 The sound mixing method and system of multi-path voice

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111049792A (en) * 2019-10-08 2020-04-21 广州视源电子科技股份有限公司 Audio transmission method and device, terminal equipment and storage medium
CN111049792B (en) * 2019-10-08 2022-03-22 广州视源电子科技股份有限公司 Audio transmission method and device, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN110166185B (en) 2021-07-09

Similar Documents

Publication Publication Date Title
CN107864084B (en) The transmission method and device of data packet
CN102685689B (en) Pt service method
CN109151913A (en) A kind of control method and relevant apparatus of service quality
CN106488169B (en) A kind of method, system and the relevant device of data transmission
KR100741463B1 (en) Method and apparatus in a communication network
CN104363407B (en) A kind of video conferencing system means of communication and related device
CN107135216B (en) Method for enhancing streaming media transmission in weak network environment
CN103534978B (en) A kind of packet information packet transmission method and equipment
CN107819833A (en) A kind of method and device for accessing live even wheat
CN104023006A (en) Multi-path transmission system and method based on application layer relaying
CN100391212C (en) Method for realizing interactive multimedia data transmission on internet
CN108667573A (en) A kind of data processing method, device and relevant device
CN105450969B (en) A kind of real time video data transmission method and electronic equipment
CN105992167A (en) Method and device for realizing long term evolution (LTE) cluster multicast
CN109286854A (en) The detection of network flow-medium Caton and optimization system and method based on WiFi network
CN108923890A (en) A kind of data transmission method, user equipment, base station and system
JP2007274372A (en) Viewer terminal and communication restriction method thereof
CN104756447B (en) A kind of method and Video Record Processor for recording session information
CN110460801A (en) The method and device of data forwarding between a kind of media server
CN108012342A (en) Multicast service handling method and device in a kind of LTE-M networks
CN110166185A (en) Processing method, device, storage medium and the processor of audio data
CN101296110B (en) Real-time monitoring apparatus and method
CN115660791A (en) Order processing method and system for business trip service
CN115277649A (en) Method and device for collaboratively editing document in multimedia conference scene
CN104703152B (en) A kind of SMS processing and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant