CN111966448A

CN111966448A - Audio transmission method and system

Info

Publication number: CN111966448A
Application number: CN202010673627.9A
Authority: CN
Inventors: 张志勇; 范志刚; 卢涛
Original assignee: Xian Wanxiang Electronics Technology Co Ltd
Current assignee: Xian Wanxiang Electronics Technology Co Ltd
Priority date: 2020-07-14
Filing date: 2020-07-14
Publication date: 2020-11-20

Abstract

The present disclosure provides an audio transmission method and system, relates to the field of audio processing, and can solve the problems of low audio transmission efficiency and prolonged audio time in the prior art. The specific technical scheme is as follows: receiving a music playing instruction sent by a client; controlling the virtual machine operating system to open the audio software; controlling the front end of the Virtio virtual sound card to acquire audio data from audio software; controlling the front end of the Virtio virtual sound card to send audio data to the rear end of the Virtio virtual sound card; and controlling the rear end of the Virtio virtual sound card to send the audio data to the zero client through the network. The audio data can be directly sent through the front end of the Virtio virtual sound card and the rear end of the Virtio virtual sound card, so that the transmission path of the audio data is reduced, the audio transmission efficiency is improved, the audio delay is reduced, the real-time performance is improved, and the user experience is effectively improved.

Description

Audio transmission method and system

Technical Field

The present disclosure relates to the field of audio processing, and in particular, to an audio transmission method and system.

Background

In an existing KVM (Kernel-based Virtual Machine, abbreviated as Kernel-based Virtual Machine) based cloud desktop system, for the design of a Virtual sound card, the design is mainly realized by a pure software simulation mode of a Virtual operating system simulator (e.g., qemu), and this implementation scheme is called as a full virtualization implementation scheme. The virtual sound card is located in qemu, and data communication between the virtual sound card and a zero client (zero client, abbreviated as ZC) needs to be relayed through KVM. Because a long path is needed for input/output (I/O) operation of voice data each time, the virtual sound card implemented in a full virtualization manner has low voice transmission efficiency, large voice delay, poor real-time performance, and poor user experience.

Disclosure of Invention

The embodiment of the disclosure provides an audio transmission method and system, which can solve the problems of low audio transmission efficiency and prolonged audio time in the prior art. The technical scheme is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided an audio transmission method, which is applied to a server, where the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps:

receiving a music playing instruction sent by a client;

controlling the virtual machine operating system to open audio software;

controlling the front end of the Virtio virtual sound card to acquire audio data from the audio software;

controlling the front end of the Virtio virtual sound card to send the audio data to the rear end of the Virtio virtual sound card;

and controlling the rear end of the Virtio virtual sound card to send the audio data to the zero client through a network.

The audio transmission method provided by the embodiment of the disclosure is applied to a server, and the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps: receiving a music playing instruction sent by a client; controlling the virtual machine operating system to open the audio software; controlling the front end of the Virtio virtual sound card to acquire audio data from audio software; controlling the front end of the Virtio virtual sound card to send audio data to the rear end of the Virtio virtual sound card; and controlling the rear end of the Virtio virtual sound card to send the audio data to the zero client through the network. The audio data can be directly sent through the front end of the Virtio virtual sound card and the rear end of the Virtio virtual sound card, so that the transmission path of the audio data is reduced, the audio transmission efficiency is improved, the audio delay is reduced, the real-time performance is improved, and the user experience is effectively improved.

In one embodiment, the controlling the front end of the Virtio virtual sound card to send the audio data to the back end of the Virtio virtual sound card includes:

controlling the front end of the Virtio virtual sound card to store the audio data in a cache;

and when the fact that the complete audio data are stored in the cache is detected, the complete audio data are sent to the rear end of the Virtio virtual sound card.

In one embodiment, the controlling the virtual sound card front end of the Virtio to store the audio data in a cache includes:

and controlling the front end of the Virtio virtual sound card to store the audio data in a buffer of a circular queue.

According to a second aspect of the embodiments of the present disclosure, there is provided an audio transmission method, which is applied to a zero client, the method including:

sending a music playing instruction to a server;

receiving audio data sent by the server;

and playing the audio data.

According to a third aspect of the embodiments of the present disclosure, there is provided an audio transmission method, which is applied to a server, where the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps:

receiving a control instruction sent by a zero client, wherein the control instruction indicates that the zero client needs to use a voice function of a microphone;

controlling the virtual machine operating system to open voice software;

controlling the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client through a network;

controlling the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card;

and controlling the front end of the Virtio virtual sound card to send the microphone data to the voice software.

In one embodiment, the controlling the virtual sound card back end of the Virtio to send the microphone data to the virtual sound card front end of the Virtio includes:

controlling the rear end of the Virtio virtual sound card to store the microphone data in a cache;

and when detecting that the complete microphone data are stored in the cache, sending the complete microphone data to the front end of the Virtio virtual sound card.

In one embodiment, the controlling the virtual sound card backend of the Virtio to store the microphone data in a cache includes:

and controlling the rear end of the Virtio virtual sound card to store the microphone data in a buffer memory of an annular queue.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an audio transmission method, which is applied to a zero client, the method including:

sending a control instruction to a server, wherein the control instruction indicates that the zero client needs to use the voice function of a microphone;

receiving microphone data;

and sending the microphone data to the server.

According to a fifth aspect of embodiments of the present disclosure, there is provided an audio transmission system, the system comprising: zero client and server; the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client is used for sending a music playing instruction to the server;

the server is used for receiving a music playing instruction sent by the client;

the server is used for controlling the virtual machine operating system to open audio software;

the server is used for controlling the front end of the Virtio virtual sound card to acquire audio data from the audio software;

the server is used for controlling the front end of the Virtio virtual sound card to send the audio data to the rear end of the Virtio virtual sound card;

the server is used for controlling the rear end of the Virtio virtual sound card to send the audio data to the zero client through a network;

the zero client is used for receiving the audio data sent by the server;

and the zero client is used for playing the audio data.

In one embodiment, the server is configured to control the Virtio virtual sound card front end to store the audio data in a cache;

and the server is used for sending the complete audio data to the rear end of the Virtio virtual sound card after detecting that the complete audio data is stored in the cache.

In an embodiment, the server is configured to control the Virtio virtual sound card front end to store the audio data in a buffer of a circular queue.

According to a sixth aspect of embodiments of the present disclosure, there is provided an audio transmission system, the system comprising: zero client and server; the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client is used for sending a control instruction to the server, and the control instruction indicates that the zero client needs to use the voice function of the microphone;

the server is used for receiving a control instruction sent by a zero client, wherein the control instruction indicates that the zero client needs to use a voice function of a microphone;

the server is used for controlling the virtual machine operating system to open voice software;

the zero client is used for receiving microphone data;

the zero client is used for sending the microphone data to the server;

the server is used for controlling the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client through a network;

the server is used for controlling the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card;

and the server is used for controlling the Virtio virtual sound card front end to send the microphone data to the voice software.

In one embodiment, the server is configured to control the virtual sound card backend of the Virtio to store the microphone data in a cache;

and the server is used for sending the complete microphone data to the front end of the Virtio virtual sound card after detecting that the complete microphone data is stored in the cache.

In an embodiment, the server is configured to control the virtual sound card backend of the Virtio to store the microphone data in a buffer of a circular queue.

According to a seventh aspect of the embodiments of the present disclosure, there is provided an audio transmission apparatus applied to a server including: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the device comprises:

the first receiving module is used for receiving a music playing instruction sent by the client;

the first control module is used for controlling the virtual machine operating system to open audio software;

the second control module is used for controlling the front end of the Virtio virtual sound card to acquire audio data from the audio software;

the third control module is used for controlling the front end of the Virtio virtual sound card to send the audio data to the rear end of the Virtio virtual sound card;

and the fourth control module is used for controlling the Virtio virtual sound card rear end to send the audio data to the zero client through a network.

In one embodiment, the third control module includes:

the first control submodule is used for controlling the front end of the Virtio virtual sound card to store the audio data in a cache;

and the first sending submodule is used for sending the complete audio data to the rear end of the Virtio virtual sound card after detecting that the complete audio data is stored in the cache.

In one embodiment, the first control sub-module includes:

and the first control subunit is used for controlling the front end of the Virtio virtual sound card to store the audio data in a buffer of an annular queue.

According to an eighth aspect of the embodiments of the present disclosure, there is provided an audio transmission apparatus, which is applied to a zero client, the apparatus including:

the first sending module is used for sending a music playing instruction to the server;

the second receiving module is used for receiving the audio data sent by the server;

and the playing module is used for playing the audio data.

According to a ninth aspect of the embodiments of the present disclosure, there is provided an audio transmission apparatus, the apparatus being applied to a server, the server including: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the device comprises:

the third receiving module is used for receiving a control instruction sent by a zero client, wherein the control instruction indicates that the zero client needs to use the voice function of a microphone;

the fifth control module is used for controlling the virtual machine operating system to open the voice software;

the sixth control module is used for controlling the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client through a network;

the seventh control module is used for controlling the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card;

and the eighth control module is used for controlling the Virtio virtual sound card front end to send the microphone data to the voice software.

In one embodiment, a seventh control module includes:

the second control submodule is used for controlling the rear end of the Virtio virtual sound card to store the microphone data in a cache;

and the second sending submodule is used for sending the complete microphone data to the front end of the Virtio virtual sound card after detecting that the complete microphone data are stored in the cache.

In one embodiment, the second control sub-module includes:

and the second control subunit is used for controlling the rear end of the Virtio virtual sound card to store the microphone data in a buffer of an annular queue.

According to a tenth aspect of the embodiments of the present disclosure, there is provided an audio transmission apparatus, which is applied to a zero client, the apparatus including:

the second sending module is used for sending a control instruction to the server, wherein the control instruction indicates that the zero client needs to use the voice function of the microphone;

the fourth receiving module is used for receiving the microphone data;

and the third sending module is used for sending the microphone data to the server.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a schematic structural diagram of a full virtualization implementation in the related art according to an embodiment of the present disclosure;

fig. 2 is a flowchart of an audio transmission method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic structural diagram of a virtio-based virtualization framework provided by an embodiment of the present disclosure;

fig. 4 is a flowchart of an audio transmission method provided by an embodiment of the present disclosure;

fig. 5 is a flowchart of an audio transmission method provided by an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a virtio-based virtualization framework provided by an embodiment of the present disclosure;

fig. 7 is a flowchart of an audio transmission method provided by an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a virtio-based virtualization framework provided by an embodiment of the present disclosure;

fig. 9 is a block diagram of an audio transmission system provided by an embodiment of the present disclosure;

fig. 10 is a block diagram of an audio transmission system provided by an embodiment of the present disclosure;

fig. 11 is a block diagram of an audio transmission device provided in an embodiment of the present disclosure;

fig. 12 is a block diagram of an audio transmission device provided in an embodiment of the present disclosure;

fig. 13 is a block diagram of an audio transmission device provided in an embodiment of the present disclosure;

fig. 14 is a structural diagram of an audio transmission device according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Desktop virtualization, also commonly referred to as desktop cloud or virtual desktop, has been developed rapidly in recent years and is widely used in some office scenarios. The desktop virtualization can provide a flexible and safe desktop environment for a user, and the server is deployed in a desktop system in a virtualization manner, so that resources such as user data, an operating system and application are abstracted and virtualized, a great deal of convenience such as dynamic resource scheduling, user authentication and desktop management is brought to the user, and the user experience is almost consistent with that of the traditional physical PC desktop.

However, currently, the support for audio and especially voice functionality in desktop virtualization is not mature enough. In an existing KVM (Kernel-based Virtual Machine) -based cloud desktop system, for the design of a Virtual sound card, the design is mainly realized by a pure software simulation mode of a Virtual operating system simulator (e.g., qemu), and this implementation scheme is called a full virtualization implementation scheme. The basic structure diagram of the implementation scheme is shown in fig. 1, and referring to fig. 1, a virtual sound card is located in qemu, and data communication between the virtual sound card and a ZC needs to be relayed through a KVM.

The biggest disadvantages of the above implementation are: on one hand, the fully virtualized sound card must communicate with the ZC terminal through the KVM, but I/O data operation between the fully virtualized sound card and the KVM is very slow, and each transmission needs to be switched over once and needs to copy once data, which makes data transmission itself between the fully virtualized sound card and the KVM very slow; on the other hand, each I/O operation of the sound data also needs to pass through a long path, and specifically, the bidirectional audio stream path of the fully virtualized sound card is as follows:

for Phone data, i.e. audio streams generated by Guest client systems. Taking the audio data stream for listening to music by using earphones as an example, the flow path is as follows:

Guest Audio APP--＞Guest Sound Card Driver--＞Sound Card--＞kvm--＞--＞Internet--＞ZC--＞Phone；

specifically, the audio data stream is input to a sound card driver in Guest from audio software in Guest, then is input to a sound card driver in Guest by the sound card driver in Guest, then is input to kvm by the sound card driver in Guest, and then is sent to ZC by kvm through a network, and then is sent to Phone by ZC for playing.

For Mic (microphone) data, i.e. the audio data stream acquired by the physical Mic inserted on ZC, the flow path is as follows:

Mic--＞ZC-＞Internet-＞kvm--＞Sound Card-＞Guest Sound Card Driver-＞Guest Audio APP；

specifically, after receiving microphone data, a microphone in the ZC sends the microphone data to the ZC, the ZC sends the microphone data to kvm through a network, the kvm sends the microphone data to a sound card in Guest, the sound card in Guest sends the microphone data to a sound card driver in Guest, and the sound card driver in Guest sends the microphone data to an application in Guest.

It can be known from the above analysis that each I/O operation of the voice data needs to pass through a long path, so that the virtual sound card implemented in a full virtualization manner has low voice transmission efficiency, large voice delay, poor real-time performance, and poor user experience.

An embodiment of the present disclosure provides an audio transmission method, as shown in fig. 2, where the method is applied to a server, and the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the audio transmission method comprises the following steps:

101. and receiving a music playing instruction sent by the client.

102. And controlling the virtual machine operating system to open the audio software.

103. And controlling the front end of the Virtio virtual sound card to acquire audio data from the audio software.

104. And controlling the front end of the Virtio virtual sound card to send the audio data to the rear end of the Virtio virtual sound card.

105. And controlling the rear end of the Virtio virtual sound card to send the audio data to the zero client through the network.

The scheme is based on a virtio virtualization framework, and realizes an audio redirection system, and the overall architecture and signal connection of the audio redirection system can be referred to fig. 3. The audio redirection system comprises: the virtual sound card assembly and a sound processing module are arranged in the ZC; wherein, virtual sound card subassembly includes: the virtual sound card simulator comprises a virtual sound card front end and a virtual sound card rear end, wherein the virtual sound card front end is located in a Guest operating system, and the virtual sound card rear end is located in the QEMU virtual operating system simulator.

Specifically, the scheme provides a virtio-based paravirtualized framework, a paravirtualized sound card is designed, abnormal capture and hardware simulation can be effectively reduced through the design of the paravirtualized scheme, and performance and user experience of the virtual sound card are improved. By means of the design of the method, I/O paths of audio data can be effectively reduced, instruction capture and hardware simulation are reduced, state switching between a root mode and a non-root mode of a Virtual Machine (VM) is reduced, and I/O efficiency of the VM can be greatly improved. Virtio is a general solution for fast communication between Guest and Host. The scheme can be compatible with not only a virtual machine based on the KVM, but also various virtualization platforms such as Xen, VMWARE and the like. The communication flow of the audio data is as follows:

for Phone data, i.e. audio streams generated by Guest client systems, the data stream is:

guest Audio APP- > virtio virtual sound card front end- > virtio virtual sound card rear end- > Internet- > ZC- > Phone;

specifically, the audio data stream is input to the front end of the virtio virtual sound card from the audio software in Guest, then input to the rear end of the virtio virtual sound card from the front end of the virtio virtual sound card, then sent to the ZC through the network from the rear end of the virtio virtual sound card, and then sent to the Phone by the ZC for playing.

Compared with the prior art, Guest Audio APP is more than Guest Sound Card Driver, more than Sound Card, more than kvm, more than Internet, more than ZC, more than Phone; the scheme of the present disclosure significantly reduces the flow path of the audio data stream.

Further, in the related art, the fully virtualized sound card must communicate with the ZC terminal through the KVM, I/O data operation between the fully virtualized sound card and the KVM is very slow, and each transmission needs to be switched once context, and also needs to copy once data, so that data transmission itself between the fully virtualized sound card and the KVM is very slow.

In one embodiment, the step 104 includes the following sub-steps:

a1, controlling the Virtio virtual sound card front end to store the audio data in the buffer.

And A2, when detecting that the complete audio data are stored in the cache, sending the complete audio data to the rear end of the Virtio virtual sound card.

In order to send the complete audio data to the zero client at one time, a cache is further arranged in the method, the front end of the Virtio virtual sound card stores the acquired audio data fragments in the cache, and after the complete audio data are acquired, the complete audio data are sent to the rear end of the Virtio virtual sound card.

In one embodiment, the step a1 includes the following sub-steps:

and controlling the front end of the Virtio virtual sound card to store the audio data in a buffer of the circular queue.

An embodiment of the present disclosure provides an audio transmission method, as shown in fig. 4, which is applied to a zero client, and the method includes the following steps:

201. and sending a music playing instruction to the server.

202. And receiving the audio data sent by the server.

203. And playing the audio data.

In the related art, when sending audio data between a zero client and a server, the zero client needs to forward the audio data through an audio service, for example: the server firstly sends the audio data to the Push Video audio service, and then the Push Video audio service forwards the audio data to the zero client, so that the delay of the zero client in playing the audio data is large.

In the present disclosure, the zero client itself may perform transceiving of audio data, for example: the zero client can be internally provided with a sound processing module which can directly receive the audio data, thereby reducing the delay of the zero client in playing the audio data.

Specifically, the sound processing module in the ZC is configured to forward the acquired audio data to a physical sound card that really exists on the ZC, and the real physical sound card controls an audio playing module (an earphone jack) to play sound.

The audio transmission method provided by the embodiment of the disclosure comprises the following steps: and sending a music playing instruction to the server. Receiving audio data sent by a server; and playing the audio data. The zero client can transmit and receive the audio data, so that the delay of the zero client in playing the audio data is reduced.

Further, in the embodiment of the present invention, virtual sound cards based on Virtio (front end of Virtio virtual sound card and back end of Virtio virtual sound card) can process bidirectional audio streams, that is, from a remote server to an audio playing module (earphone or speaker) on the ZC or from a voice collecting device on the ZC to a remote server, for example, voice function at ZC.

The embodiments described above describe the technical solutions from the remote server to the audio playing module (earphone or speaker) on the ZC, and the embodiments described below describe the audio playing function and the audio function from the voice collecting device on the ZC to the remote server, for example, at the ZC.

An embodiment of the present disclosure provides an audio transmission method, as shown in fig. 5, where the method is applied to a server, and the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the audio transmission method comprises the following steps:

301. and receiving a control instruction sent by the zero client, wherein the control instruction indicates that the zero client needs to use the voice function of the microphone.

302. And controlling the virtual machine operating system to open the voice software.

303. And controlling the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client through the network.

304. And controlling the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card.

305. And controlling the front end of the Virtio virtual sound card to send the microphone data to the voice software.

The scheme is based on a virtio virtualization framework, and realizes an audio redirection system, and the overall architecture and signal connection of the audio redirection system can refer to fig. 6. The audio redirection system comprises: the virtual sound card assembly and a sound processing module are arranged in the ZC; wherein, virtual sound card subassembly includes: the virtual sound card simulator comprises a virtual sound card front end and a virtual sound card rear end, wherein the virtual sound card front end is located in a Guest operating system, and the virtual sound card rear end is located in the QEMU virtual operating system simulator.

for Mic (microphone) data, that is, audio data stream collected by physical Mic inserted on ZC, taking voice chat software as an example, the data flow direction is as follows:

mic- > ZC- > Internet- > virtio virtual sound card back end- > virtio virtual sound card front end- > Guest Audio APP.

Specifically, after receiving the microphone data, a microphone in the ZC sends the microphone data to the ZC, the ZC sends the microphone data to the virtual sound card rear end of virtio through the network, the virtual sound card rear end of virtio sends the microphone data to the virtual sound card front end of virtio, and the virtual sound card front end of virtio sends the microphone data to an application in Guest.

Compared with the related technology, Mic- - > ZC- > Internet- > kvm- > Sound Card- > Guest Sound Card Driver- > Guest Audio APP;

the scheme of the present disclosure significantly reduces the flow path of the microphone data.

Furthermore, in the related art, the fully virtualized sound card must communicate with the ZC terminal through the KVM, I/O data operation between the fully virtualized sound card and the KVM is very slow, and each transmission needs to be switched once context, and also needs to copy data once, so that data transmission itself between the fully virtualized sound card and the KVM is very slow.

The audio transmission method provided by the embodiment of the disclosure is applied to a server, and the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps: receiving a control instruction sent by a zero client, wherein the control instruction indicates that the zero client needs to use the voice function of a microphone; controlling a virtual machine operating system to open voice software; controlling the rear end of the Virtio virtual sound card to receive microphone data sent by a zero client through a network; controlling the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card; and controlling the front end of the Virtio virtual sound card to send the microphone data to the voice software. The microphone data can be directly sent through the Virtio virtual sound card rear end and the Virtio virtual sound card front end, so that the path of microphone data transmission is reduced, the voice transmission efficiency is improved, the voice delay is reduced, the real-time performance is improved, and the user experience is effectively improved.

In one embodiment, the above step 304 comprises the following sub-steps:

c1, controlling the rear end of the Virtio virtual sound card to store the microphone data in a cache;

and C2, when detecting that the complete microphone data are stored in the cache, sending the complete microphone data to the front end of the Virtio virtual sound card.

In order to send the complete microphone data to the front end of the Virtio virtual sound card at one time, a cache is further arranged in the method, the rear end of the Virtio virtual sound card stores the acquired fragments of the microphone data in the cache, and after the complete microphone data are acquired, the complete microphone data are sent to the front end of the Virtio virtual sound card.

In one embodiment, the step C1 includes the following sub-steps:

and controlling the rear end of the Virtio virtual sound card to store the microphone data in a buffer memory of a circular queue.

An embodiment of the present disclosure provides an audio transmission method, as shown in fig. 7, which is applied to a zero client, and includes the following steps:

401. and sending a control instruction to the server, wherein the control instruction indicates that the zero client needs to use the voice function of the microphone.

402. Microphone data is received.

403. The microphone data is sent to a server.

In the related art, when the client sends microphone data to the server, the client needs to forward the data through an audio service, for example: the null client sends the microphone data to the Push Video audio service and then forwards the microphone data to the server by the Push Video audio service, which causes the voice delay to be large.

In the present disclosure, the zero client itself may perform transceiving of audio data, for example: a sound processing module can be arranged in the zero client side and can directly receive microphone data, so that voice delay is reduced.

Specifically, the sound processing module in the ZC is configured to control the Mic to acquire an audio.

The audio transmission method provided by the embodiment of the disclosure comprises the following steps: sending a control instruction to a server, wherein the control instruction indicates that a zero client needs to use the voice function of a microphone; receiving microphone data; the microphone data is sent to a server. The zero client can collect microphone data, so that voice delay is reduced.

The technical scheme of the disclosure is described by specific embodiments.

The semi-virtual sound card is designed based on the virtio semi-virtual framework, abnormal capture and hardware simulation can be effectively reduced through the design of the semi-virtual scheme, and performance and user experience of the virtual sound card are improved. By means of the design of the method, I/O paths of audio data can be effectively reduced, instruction capture and hardware simulation are reduced, state switching between a root mode and a non-root mode of a Virtual Machine (VM) is reduced, and I/O efficiency of the VM can be greatly improved. Virtio is a general solution for fast communication between Guest and Host. The scheme can be compatible with not only a virtual machine based on the KVM, but also various virtualization platforms such as Xen, VMWARE and the like. The communication flow of the audio data is as follows:

The invention provides a scheme for realizing audio redirection in desktop virtualization, which is based on a virtio virtualization framework and realizes an audio redirection system, and the overall architecture and signal connection of the system can refer to FIG. 8. The audio redirection system comprises: the virtual sound card component of virtio and a sound processing module arranged in the ZC are arranged; wherein the virtio virtual sound card component comprises: the virtual sound card system comprises a virtio virtual sound card front end and a virtio virtual sound card rear end, wherein the virtio virtual sound card front end is positioned in a Guest operating system, and the virtio virtual sound card rear end is positioned in a QEMU virtual operating system simulator.

The following describes the functions of the components of the virtio virtual sound card assembly:

(1) virtio virtual sound card front end

The assembly is a set of sound card driving program based on a Guest operating system and based on a PCIE interface, the sound card driving program is a set of filter driving, and the main purposes are as follows: full operation of the Guest operating system on the virtual sound card is captured and redirected. The main operations of the Guest system on the sound card include audio data, Mic data, and the like.

(2) virtio virtual sound card back end

The component is a virtual sound card device based on a PCI interface of QEMU, is realized in the QEMU, exists in an application layer space of a Host system, and is mainly used for carrying out quick communication and data interaction with the front end of the virtual sound card in a Guest operating system through a ring queue. The virtual sound card back end component transmits the data and the instruction acquired from the virtual sound card front end component to the sound processing module in the ZC through the network.

(3) Sound processing module in ZC

And the sound processing module in the ZC is used for forwarding the acquired sound data and instructions to a physical sound card which really exists on the ZC, controlling an audio playing module (an earphone socket) to play sound by the real physical sound card, and controlling Mic to acquire audio and the like.

In the embodiment of the invention, a virtual sound card based on virtio can process bidirectional audio streams, namely, the audio streams are transmitted from a remote server to an audio playing module (earphone or sound box) on a ZC or from a voice acquisition device on the ZC to the remote server, for example, the music playing function and the voice function of the ZC.

Taking music playing as an example, a process flow from a remote server to an audio playing device at a ZC end is described as follows:

the ZC sends a music playing instruction to a Guest operating system;

the Guest operating system opens a corresponding music player according to a music playing instruction sent by the ZC end;

the Guest operating system calls the front end of the virtual sound card to play music and sends the acquired audio data to the ZC;

the method specifically comprises the following steps:

the front end of the virtual sound card of virtio sends the audio data acquired from the audio software to the back end of the virtual sound card of virtio in the Host system through a circular queue;

and after the back end of the virtio virtual sound card receives the audio data sent by the front end of the virtio virtual sound card, sending the received audio data to the ZC through the network.

The ZC sends the received audio data to a local sound processing module for processing;

and the sound processing module of the ZC injects the acquired audio data into a local physical sound card of the ZC and carries out local playing through the physical sound card.

Taking the voice call function using nails on the ZC as an example, the processing flow of the microphone (Mic) data from the ZC to the remote server will be described as follows:

the ZC sends a control instruction for dialing a voice call to the Guest operating system;

the Guest operating system opens the voice call function of the nail, and at the moment, the nail calls the front end of the virtio virtual sound card in the Guest operating system to monitor the arrival of Mic data;

when the front end of the virtio virtual sound card monitors Mic data, sending the Mic data to nailing software, and accordingly acquiring the Mic data;

and the nailing software sends the received MIC data to the opposite end for voice communication through the network.

On the other hand, the processing on the ZC side includes:

a voice processing module in the ZC acquires the voice input of the Mic on the ZC and sends Mic data to the rear end of a virtual sound card through a network;

and the rear end of the virtio virtual sound card sends the received Mic data to the front end of the virtual sound card through a circular queue.

An embodiment of the present disclosure provides an audio transmission system, as shown in fig. 9, the system includes: zero client 10 and server 20; the server 20 includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client 10 is configured to send a music playing instruction to the server 20;

the server 20 is configured to receive a music playing instruction sent by the client 10;

the server 20 is configured to control the virtual machine operating system to open audio software;

the server 20 is configured to control the front end of the Virtio virtual sound card to obtain audio data from the audio software;

the server 20 is configured to control the front end of the Virtio virtual sound card to send the audio data to the back end of the Virtio virtual sound card;

the server 20 is configured to control the virtual sound card backend of the Virtio to send the audio data to the zero client 10 through a network;

the zero client 10 is configured to receive the audio data sent by the server 20;

the zero client 10 is configured to play the audio data.

In one embodiment, the server 20 is configured to control the Virtio virtual sound card front end to store the audio data in a cache;

the server 20 is configured to send the complete audio data to the rear end of the Virtio virtual sound card after detecting that the complete audio data is stored in the cache.

In an embodiment, the server 20 is configured to control the Virtio virtual sound card front end to store the audio data in a buffer of a circular queue.

An embodiment of the present disclosure provides an audio transmission system, as shown in fig. 10, the system includes: zero client 30 and server 40; the server 40 includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client 30 is configured to send a control instruction to the server 40, where the control instruction indicates that the zero client 30 needs to use a voice function of a microphone;

the server 40 is configured to receive a control instruction sent by the zero client 30, where the control instruction indicates that the zero client 30 needs to use a voice function of a microphone;

the server 40 is configured to control the virtual machine operating system to open the voice software;

the zero client 30 is configured to receive microphone data;

the zero client 30 is configured to send the microphone data to the server 40;

the server 40 is configured to control the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client 30 through a network;

the server 40 is configured to control the virtual sound card back end of the Virtio to send the microphone data to the virtual sound card front end of the Virtio;

and the server 40 is configured to control the virtual sound card front end of the Virtio to send the microphone data to the voice software.

In one embodiment, the server 40 is configured to control the virtual sound card backend of the Virtio to store the microphone data in a cache;

the server 40 is configured to send the complete microphone data to the front end of the Virtio virtual sound card after detecting that the complete microphone data is stored in the cache.

In one embodiment, the server 40 is configured to control the virtual sound card backend of the Virtio to store the microphone data in a buffer of a circular queue.

Based on the audio transmission method described in the embodiment corresponding to fig. 2, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute the embodiment of the method of the present disclosure.

An embodiment of the present disclosure provides an audio transmission apparatus, as shown in fig. 11, where the audio transmission apparatus is applied to a server, and the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the device comprises:

the first receiving module 11 is configured to receive a music playing instruction sent by a client;

the first control module 12 is configured to control the virtual machine operating system to open the audio software;

the second control module 13 is configured to control the front end of the Virtio virtual sound card to obtain audio data from the audio software;

the third control module 14 is configured to control the front end of the Virtio virtual sound card to send the audio data to the rear end of the Virtio virtual sound card;

and the fourth control module 15 is configured to control the virtual sound card backend of the Virtio to send the audio data to the zero client through a network.

In one embodiment, the third control module 14 includes:

In one embodiment, the first control sub-module includes:

Based on the audio transmission method described in the embodiment corresponding to fig. 4, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.

An embodiment of the present disclosure provides an audio transmission apparatus, as shown in fig. 12, where the apparatus is applied to a zero client, and the apparatus includes:

a first sending module 21, configured to send a music playing instruction to a server;

a second receiving module 22, configured to receive the audio data sent by the server;

and the playing module 23 is configured to play the audio data.

Based on the audio transmission method described in the embodiment corresponding to fig. 5, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.

An embodiment of the present disclosure provides an audio transmission apparatus, as shown in fig. 13, where the apparatus is applied to a server, and the server includes: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the device comprises:

a third receiving module 31, configured to receive a control instruction sent by a zero client, where the control instruction indicates that the zero client needs to use a voice function of a microphone;

a fifth control module 32, configured to control the virtual machine operating system to open the voice software;

a sixth control module 33, configured to control the rear end of the Virtio virtual sound card to receive microphone data sent by the zero client through a network;

a seventh control module 34, configured to control the rear end of the Virtio virtual sound card to send the microphone data to the front end of the Virtio virtual sound card;

an eighth control module 35, configured to control the virtual sound card front end of Virtio to send the microphone data to the voice software.

In one embodiment, the seventh control module 34 includes:

In one embodiment, the second control sub-module includes:

Based on the audio transmission method described in the embodiment corresponding to fig. 7, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.

An embodiment of the present disclosure provides an audio transmission apparatus, as shown in fig. 14, where the apparatus is applied to a zero client, and the apparatus includes:

a second sending module 41, configured to send a control instruction to a server, where the control instruction indicates that the zero client needs to use a voice function of a microphone;

a fourth receiving module 42, configured to receive microphone data;

a third sending module 43, configured to send the microphone data to the server.

Based on the audio transmission method described in the embodiment corresponding to fig. 2, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to fig. 2, which is not described herein again.

Based on the audio transmission method described in the embodiment corresponding to fig. 4, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to fig. 4, which is not described herein again.

Based on the audio transmission method described in the embodiment corresponding to fig. 5, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to fig. 5, which is not described herein again.

Based on the audio transmission method described in the embodiment corresponding to fig. 7, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to fig. 7, which is not described herein again.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio transmission method is applied to a server, and the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps:

receiving a music playing instruction sent by a client;

controlling the virtual machine operating system to open audio software;

2. The method according to claim 1, wherein said controlling the Virtio virtual sound card front end to send the audio data to the Virtio virtual sound card back end comprises:

3. The method of claim 2, wherein the controlling the Virtio virtual sound card front end to store the audio data in a cache comprises:

4. An audio transmission method, applied to a zero client, the method comprising:

sending a music playing instruction to a server;

receiving audio data sent by the server;

and playing the audio data.

5. An audio transmission method is applied to a server, and the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end; the method comprises the following steps:

controlling the virtual machine operating system to open voice software;

6. The method of claim 5, wherein said controlling the Virtio virtual sound card back end to send the microphone data to the Virtio virtual sound card front end comprises:

7. The method of claim 6, wherein the controlling the Virtio virtual sound card backend to store the microphone data in a cache comprises:

8. An audio transmission method, applied to a zero client, the method comprising:

receiving microphone data;

and sending the microphone data to the server.

9. An audio transmission system, the system comprising: zero client and server; the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client is used for sending a music playing instruction to the server;

the zero client is used for receiving the audio data sent by the server;

and the zero client is used for playing the audio data.

10. An audio transmission system, the system comprising: zero client and server; the server comprises: the virtual machine operation system is internally provided with a Virtio virtual sound card front end, and the virtual operation system simulator is internally provided with a Virtio virtual sound card rear end;

the zero client is used for receiving microphone data;

the zero client is used for sending the microphone data to the server;