CN116540973A

CN116540973A - Cross-domain audio processing method and device, electronic equipment and storage medium

Info

Publication number: CN116540973A
Application number: CN202310517462.XA
Authority: CN
Inventors: 康克勤
Original assignee: Ecarx Hubei Tech Co Ltd
Current assignee: Ecarx Hubei Tech Co Ltd
Priority date: 2023-05-06
Filing date: 2023-05-06
Publication date: 2023-08-04

Abstract

The invention discloses a cross-domain audio processing method, a device, an electronic device and a storage medium, wherein the method is applied to a system on a chip, and the system on a chip at least comprises two operating system domains, and comprises the following steps: the method comprises the steps of obtaining audio data according to a first real sound card device, writing the audio data into a first virtual sound card device, wherein the first real sound card device and the first virtual sound card device are located in a first operating system domain, transmitting the audio data of the first virtual sound card device to a second virtual sound card device based on a shared memory, and processing the audio data received by the second virtual sound card device based on the second real sound card device, wherein the second real sound card device and the second virtual sound card device are located in a second operating system domain. The transmission of the audio data between the first operating system domain and the second operating system domain is realized through the shared memory, so that the cross-domain processing of the audio data is realized, the conflict problem existing in the processing of the audio data between different operating system domains is solved, and the user experience is improved.

Description

Cross-domain audio processing method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a cross-domain audio processing method, apparatus, electronic device, and storage medium.

Background

The automobile audio function is used as an indispensable requirement of a user in the driving process, can provide entertainment experience for the user, and can provide important auxiliary functions for driving safety, for example, lane departure voice prompts, vehicle fault voice prompts, fatigue driving voice prompts and other driving safety prompts are processed and played through an automobile audio system.

Currently, a System On Chip (SOC) On a vehicle is usually a multi-core heterogeneous processor, each core runs a different operating System, but only one of the SOC is used as a hardware controller for controlling audio processing, and in some cases, there may be problems of conflict in audio data processing of different operating System domains, for example: when the audio data playing requirements exist in two different operating system domains, the audio data playing requirement of one operating system domain can be met at the same time, and the audio data playing requirement of the other operating system domain can be processed only after the former is finished, so that the user experience is reduced.

Disclosure of Invention

The invention provides a cross-domain audio processing method, a device, electronic equipment and a storage medium, which are used for solving the conflict problem existing in audio data processing between different operating system domains and improving the user experience.

According to an aspect of the present invention, there is provided a cross-domain audio processing method applied to a system on a chip, the system on a chip including at least two operating system domains, the method comprising:

acquiring audio data according to a first real sound card device, and writing the audio data into a first virtual sound card device, wherein the first real sound card device and the first virtual sound card device are positioned in a first operating system domain;

transmitting audio data of the first virtual sound card device to the second virtual sound card device based on the shared memory;

and processing the audio data received by the second virtual sound card device based on the second real sound card device, wherein the second real sound card device and the second virtual sound card device are positioned in a second operating system domain.

According to another aspect of the present invention, there is provided a cross-domain audio processing apparatus applied to a system on a chip including at least two operating system domains, the apparatus comprising:

the data acquisition module is used for acquiring audio data according to the first real sound card equipment and writing the audio data into the first virtual sound card equipment, wherein the first real sound card equipment and the first virtual sound card equipment are positioned in a first operating system domain;

The cross-domain transmission module is used for transmitting the audio data of the first virtual sound card device to the second virtual sound card device based on the shared memory;

and the data processing module is used for processing the audio data received by the second virtual sound card device based on the second real sound card device, wherein the second real sound card device and the second virtual sound card device are positioned in a second operating system domain.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least two chips, at least one of the chips configuring a first operating system domain and at least one of the chips configuring a second operating system domain; and

a memory communicatively coupled to the at least two chips; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the cross-domain audio processing method according to any one of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a cross-domain audio processing method according to any embodiment of the present invention.

According to the technical scheme, the audio data are acquired according to the first real sound card device, the audio data are written into the first virtual sound card device, the first real sound card device and the first virtual sound card device are located in a first operating system domain, the audio data of the first virtual sound card device are transmitted to the second virtual sound card device based on the shared memory, the audio data received by the second virtual sound card device are processed based on the second real sound card device, and the second real sound card device and the second virtual sound card device are located in a second operating system domain. According to the embodiment of the invention, the shared memory is used for transmitting the audio data from the first virtual sound card device of the first operating system domain to the second virtual sound card device of the second operating system domain, and the second real sound card device reads and processes the audio data in the second virtual sound card device, so that the cross-domain processing of the audio data is realized, the conflict problem existing in the processing of the audio data among different operating system domains is solved, and the user experience is improved.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a cross-domain audio processing method according to a first embodiment of the present invention;

fig. 2 is a flowchart of a cross-domain audio processing method according to a second embodiment of the present invention;

fig. 3 is a flowchart of a dynamic cross-domain switching method of a controller according to a second embodiment of the present invention;

FIG. 4 is a schematic diagram of a cross-domain audio processing framework provided in accordance with a third embodiment of the present invention;

FIG. 5 is a flow chart of a cross-domain audio processing method according to a third embodiment of the present invention;

FIG. 6 is a flow chart of a cross-domain audio processing method according to a fourth embodiment of the present invention;

FIG. 7 is a flow chart of a cross-domain audio processing method according to a fifth embodiment of the present invention;

fig. 8 is a schematic structural diagram of a cross-domain audio processing apparatus according to a sixth embodiment of the present invention;

Fig. 9 is a schematic structural diagram of an electronic device implementing a cross-domain audio processing method according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

Fig. 1 is a flowchart of a cross-domain audio processing method according to an embodiment of the present invention, where the method may be applied to a case of performing cross-domain processing on audio data, and the method may be performed by a cross-domain audio processing apparatus, where the cross-domain audio processing apparatus may be implemented in a form of hardware and/or software, and the cross-domain audio processing apparatus may be configured in an electronic device, for example, may not be limited to a vehicle-mounted device or the like. As shown in fig. 1, the first embodiment provides a cross-domain audio processing method, which is applied to a system on a chip, where the system on a chip includes at least two operating system domains, and specifically includes the following steps:

s110, acquiring audio data according to the first real sound card equipment, and writing the audio data into the first virtual sound card equipment, wherein the first real sound card equipment and the first virtual sound card equipment are positioned in a first operating system domain.

The first real sound card device may refer to a real sound card device located in a first operating system domain, where the first real sound card device has a control capability of audio hardware, but does not have a cross-domain audio data transmission function, and can only be used in the first operating system domain. The first virtual sound card device may refer to a virtual sound card device located in a first operating system domain, where the first virtual sound card device does not have control capability of audio hardware, but has a cross-domain audio data transmission function. The first operating system domain may be understood as one of the operating systems needing cross-domain audio processing, and the first operating system domain may include an Android domain, a Linux domain, a QNX (Quick UNIX) domain, and the like. The audio data may refer to audio data to be cross-domain processed, and may include different types of information related to the audio to be cross-domain processed, such as the size of the audio and pulse code modulated (Pulse Code Modulation, PCM) audio data or files to be cross-domain processed, and the like.

In the embodiment of the present invention, a System On Chip (SOC) may call a first real sound card device in a first operating System domain to collect audio data according to an audio processing process or an audio processing instruction, and then transfer the audio data to a first virtual sound card device by calling a preset interface function or an application programming interface (Application Programming Interface, API), where the audio data may include, but is not limited to, the following: the method comprises the steps that audio data input by a user, audio data generated by a first operating system domain according to audio playing requirements, recorded audio data collected by a first real sound card device are called by the first operating system domain, and the like. It can be understood that the first real sound card device in the first operating system domain has the control capability of audio hardware, but does not have the cross-domain audio data transmission function; the first virtual sound card device is only used for transmission of cross-domain audio data.

S120, transmitting the audio data of the first virtual sound card device to the second virtual sound card device based on the shared memory.

The shared Memory (SHM) may be understood as a block of pre-allocated physical addresses, which may be used for transmission of audio data between the first operating system domain and the second operating system domain, that is, the shared Memory may be accessed by both the first operating system domain and the second operating system domain. The second virtual sound card device may refer to a virtual sound card device located in a second operating system domain, where the second virtual sound card device does not have control capability of audio hardware, but has a cross-domain audio data transmission function.

In the embodiment of the invention, the SOC may control the first operating system domain to write the audio data in the first virtual sound card device into the shared memory, and then control the second virtual sound card device in the second operating system domain to read the corresponding audio data from the shared memory.

S130, processing the audio data received by the second virtual sound card device based on the second real sound card device, wherein the second real sound card device and the second virtual sound card device are located in a second operating system domain.

The second real sound card device may be a real sound card device located in a second operating system domain, where the second real sound card device has a control capability of audio hardware, but does not have a cross-domain audio data transmission function, and can only be used in the second operating system domain. The second operating system domain may be understood as another operating system that needs to perform cross-domain audio processing, and the second operating system domain may include an Android domain, a Linux domain, a QNX (Quick UNIX) domain, and the like.

In the embodiment of the present invention, after the audio data is written into the second virtual sound card device, the SOC may control the second real sound card device to read and process the audio data from the second virtual sound card device, for example, may be: the second real sound card device transmits the read audio data to the audio playing hardware player for playing, or: the second real sound card device stores the read audio data in a database for storage, which is not limited in the embodiment of the present invention.

Example two

Fig. 2 is a flowchart of a cross-domain audio processing method according to a second embodiment of the present invention, which is further optimized and expanded based on the foregoing embodiments, and may be combined with each of the optional technical solutions in the foregoing embodiments. As shown in fig. 2, the cross-domain audio processing method provided in the second embodiment specifically includes the following steps:

s210, calling the first real sound card device to collect audio data.

In the embodiment of the invention, the SOC can call the first real sound card device in the first operating system domain to collect the audio data to be cross-domain processed according to the audio processing process or the audio processing instruction and the like. In some embodiments, the SOC of the vehicle may first control the audio direct memory access driver (Audio Direct Memory Access, ADMA) controller to obtain audio data collected from the vehicle-mounted speaker, and then send the audio data to the first real sound card device in the first operating system domain as audio data to be cross-domain processed.

S220, controlling the first virtual sound card device to acquire the audio data of the first real sound card device.

In the embodiment of the present invention, the SOC may call the standard interface function in the LIB library of the advanced Linux sound architecture (Advanced Linux Sound Architecture, ALSA) according to the audio processing process or the audio processing instruction, etc., to control the first virtual sound card device to obtain the audio data of the first real sound card device, where the manner of controlling the first virtual sound card device to obtain the audio data of the first real sound card device may include, but is not limited to, at least one of the following: controlling a first real sound card device to call a write audio interface function to write audio data into a first virtual sound card device; and controlling the first virtual sound card device to call a reading audio interface function to read audio data from the first real sound card device.

S230, controlling the first virtual sound card device to write the audio data into the shared memory.

In the embodiment of the invention, the SOC can control the kernel driver of the first operating system domain to call the standard interface function or the custom interface function in the ALSA LIB library, and write the audio data into the pre-allocated shared memory.

S240, creating a read-write subscript value of the audio data written in the first data management thread management shared memory in the first operating system domain.

The first data management thread may be a thread created in the first operating system domain and used for managing audio data, where the first management thread may manage a read-write subscript value corresponding to the audio data in the shared memory, and when the read-write subscript value changes, it indicates that the audio data in the shared memory is being read or written.

In the embodiment of the present invention, a thread creation function of a first operating system domain may be called to create a first data management thread, and a read-write subscript value corresponding to audio data in a shared memory is managed by using the thread. In some embodiments, if the first operating system domain is a Linux domain, a pthread_create function may be invoked to create the first data management thread.

S250, controlling the second virtual sound card device to read the audio data in the shared memory.

In the embodiment of the invention, the SOC may call the standard interface function in the ALSA LIB library to control the second virtual sound card device in the second operating system domain to read the audio data sent from the first operating system domain in the shared memory according to the audio processing process or the audio processing instruction and the like.

S260, creating a read-write subscript value of the read audio data in the shared memory by the second data management thread in the second operating system domain.

The second management thread may be a thread created in the second operating system domain and used for managing audio data, where the second management thread may manage a read-write subscript value corresponding to the audio data in the shared memory, and when the read-write subscript value changes, it indicates that the audio data in the shared memory is being read or written.

In the embodiment of the present invention, a thread creation function of the second operating system domain may be called to create a second data management thread, and the thread is used to manage a read-write subscript value corresponding to audio data in the shared memory. The transmission process of the audio data in the shared memory can be as follows: 1) The first operating system domain calls an interface function in the ALSA LIB library to write the audio data into the shared memory, calls a first data management thread to update a write index value (write_idx) of the audio data in the shared memory, and waits for a read index value (read_idx) of the audio data in the shared memory to change; 2) The second operating system domain calls an interface function in the ALSA LIB library to read the audio data from the shared memory, and calls a second data management thread to update a read_idx (index_idx) of the audio data in the shared memory; 3) When the first data management thread of the first operating system domain detects that the read index value (read_idx) of the audio data in the shared memory changes, the first operating system domain calls the interface function in the ALSA LIB library again to write the audio data of the next frame into the shared memory, and the process is repeatedly executed until all the audio data are transmitted to the second virtual sound card device through the shared memory.

S270, calling the second real sound card device to read the audio data received by the second virtual sound card device.

In the embodiment of the invention, the SOC can call the standard interface function in the ALSA LIB library to control the second real sound card device to read the audio data received by the second virtual sound card device according to the audio processing process or the audio processing instruction and the like.

S280, transmitting the audio data to a controller with control right of the second real sound card device, and processing the audio data at the controller.

The controller may be understood as a controller having control right and capable of transmitting audio data to the audio playing device, and the different operating system domains need the controller when the audio data needs to be played, it may be understood that only one controller having control right may configure the control right of the controller to one of the first operating system domain and the second operating system domain, and the controller may not be limited to an ADMA controller or the like.

In the embodiment of the present invention, after the audio data is written into the second real Sound card device, the SOC may control the second real Sound card device to transmit the audio data to a controller having control authority, such as an ADMA controller, and the like, and the controller processes the audio data, for example, the controller may transmit the audio data to an Inter-IC Sound bus (I2S) controller, and then the I2S controller controls the audio playing device to play the audio data.

Further, on the basis of the above embodiment of the present invention, a cross-domain audio processing method provided in the second embodiment further includes:

registering a first real sound card device and a first virtual sound card device in a first operating system domain, and registering a second real sound card device and a second virtual sound card device in a second operating system domain.

In the embodiment of the invention, in order to realize cross-domain processing of audio data between the first operating system domain and the second operating system domain, corresponding real sound card equipment and virtual sound card equipment can be respectively registered in the first operating system domain and the second operating system domain, wherein the real sound card equipment has the control capability of audio hardware, and the virtual sound card equipment is only used for transmitting the cross-domain audio data.

Further, on the basis of the above embodiment of the present invention, the method for processing cross-domain audio provided in the second embodiment further includes a method for dynamically switching between cross-domain of the controller. As shown in fig. 3, the method for dynamically switching the controller across domains provided in the second embodiment specifically includes the following steps:

s310, transmitting a controller demand instruction to a second operating system domain through a remote inter-core communication bus in the first operating system domain.

The remote inter-core communication bus (Remote Processer Message, RPMSG) can be understood as a message bus, a set of communication interfaces are arranged in the kernel driver, and the RPMSG interfaces can be called in the multi-core heterogeneous processor to realize communication among different CPU cores. The controller demand instruction may refer to a demand instruction sent by the first operating system domain for obtaining control rights of the controller.

In the embodiment of the invention, when the first operating system domain needs the control right of the controller, the SOC can control the application process of the first operating system domain to call the send interface function of the RPMSG mechanism, and a controller demand instruction for acquiring the control right of the controller is transmitted to the second operating system domain.

S320, removing the control right of the corresponding controller in the second operating system domain, and feeding back right removing information to the first operating system domain.

The right removing message may be understood as a message fed back to the first operating system domain after the second operating system removes the control right of the corresponding controller.

In the embodiment of the invention, after the second operating system domain calls the receive interface function of the RPMSG mechanism to receive the controller demand instruction, the SOC can control the application process of the second operating system domain to call the write interface function, write the corresponding register of the controller in the kernel driver to remove the control right of the (Disable) corresponding controller, and transmit the right removal message feedback of the controller to the first operating system domain by calling the send interface function of the RPMSG mechanism.

S330, after the first operating system domain receives the right removing message, the control right of the controller is obtained.

In the embodiment of the invention, the SOC can control the application process of the first operating system domain to call the receiver interface function of the RPMSG mechanism to acquire the right removal message fed back by the second operating system domain, and then control the application process to call the write interface function to write the register corresponding to the controller in the kernel driver to acquire the control right of the (Enable) corresponding controller. Further, after the first operating system domain obtains the control right of the controller, the first real sound card device can be called by the control application process to realize the playing and recording operation of the audio data. By the dynamic cross-domain switching of the controller, the control right of the controller can be obtained by different operation system domains flexibly, so that each operation system domain can realize the playing and recording operation of audio data through the real sound card equipment.

According to the technical scheme, the first real sound card device is called to collect audio data, the first virtual sound card device is controlled to obtain the audio data of the first real sound card device, the first virtual sound card device is controlled to write the audio data into the shared memory, a first data management thread is created in a first operating system domain to manage a read-write index value of the audio data written into the shared memory, a second virtual sound card device is controlled to read the audio data in the shared memory, a second data management thread is created in a second operating system domain to manage the read-write index value of the audio data read in the shared memory, the second real sound card device is called to read the audio data received by the second virtual sound card device, the audio data is transmitted to a controller with control right of the second real sound card device, and the audio data is processed in the controller. According to the embodiment of the invention, the audio data is written into the shared memory from the first virtual sound card equipment of the first operating system domain, then the read-write index value of the read audio data in the shared memory is managed by the first data management thread and the second data management thread, so that the audio data is transmitted to the second virtual sound card equipment of the second operating system domain, finally the audio data in the second virtual sound card equipment is read by the second real sound card equipment and is transmitted to the controller with control right for processing, cross-domain processing of the audio data is realized, the conflict problem existing in the audio data processing between different operating system domains is solved, and the user experience is improved.

Example III

Fig. 4 is a schematic diagram of a cross-domain audio processing framework according to a third embodiment of the present invention. As shown in fig. 4, the frame includes: the multi-core heterogeneous SOC comprises an A76 core CPU running an Android system and an A55 core CPU running a Linux system; the Android domain consists of a user space and an Android kernel driver, wherein the user space comprises a main application program AP_PROCESS for processing audio data of the user space, and the Android kernel driver realizes a standard ALSA audio framework for controlling bottom audio hardware; the Linux domain consists of a user space and a Linux kernel driver, wherein the user space comprises a main application program CP_PROCESS for processing audio data of the user space, and the Linux kernel driver realizes a standard ALSA audio framework for controlling bottom audio hardware; the hardware module belongs to a hardware part shared by an Android domain and a Linux domain and comprises a shared memory, an ADMA controller and an I2S controller, wherein the shared memory (SHM) is an allocated physical memory used for transmitting data between the Android domain and the Linux domain, the two domains can access the memory, the ADMA controller is a hardware module used for transmitting and receiving data, the ADMA controller transmits audio data to the I2S controller, and finally the I2S controller transmits processed audio data to a loudspeaker for playing, or the I2S controller transmits collected recording audio data to the ADMA controller; and the RPMSG message is used for realizing communication among different CPU cores.

Based on the above-mentioned cross-domain audio processing framework, fig. 5 is a flowchart of a cross-domain audio processing method according to the third embodiment of the present invention. Based on the above embodiment, the present embodiment provides an implementation manner of a cross-domain audio processing method, which takes the first operating system domain as the Android domain and the second operating system domain as the Linux domain as an example, so as to implement cross-domain playing of audio data. As shown in fig. 5, a cross-domain audio processing method provided in the third embodiment of the present invention specifically includes the following steps:

s410, controlling an AP_PROCESS PROCESS of the Android domain user space to open the first virtual sound card device.

In the embodiment of the invention, the AP_PROCESS PROCESS of the Android domain user space can call the standard function pcm_open of the ALSA LIB library to open the first virtual sound card device, and obtain a file descriptor, which is a handle, and the handle is used by subsequent read-write operations. The sound card device is a file operation node provided by the kernel driver, and the user space can access the file operation node. The ALSA LIB library is a library that the ALSA driver provides to the application upper layer access, and the open/read/write function of the sound card apparatus can be realized through the interface function in the library.

S420, controlling the Android kernel driver to create a read-write subscript value of the read audio data in the first data management thread management shared memory, and informing the ALSA driver to read and write the audio data.

In the embodiment of the invention, a first data management thread, namely an ap_audio_thread thread, is created in the Android kernel driver, and manages the lower index values of audio data read (read_idx) and write (write_idx) in the SHM shared memory, and the lower index values are currently corresponding to the write_idx and counted, and meanwhile, when data is written, the ALSA driver is informed to carry out read and write processing

S430, controlling the CP_PROCESS PROCESS of the Linux domain user space to open the second virtual sound card device.

In the embodiment of the invention, the cp_process PROCESS of the Linux domain user space can call the standard function pcm_open of the ALSA LIB library to open the second virtual sound card device, and obtain a file descriptor.

S440, controlling the Linux kernel driver to create a read-write subscript value of the read audio data in the second data management thread management shared memory, and informing the ALSA driver to read and write the audio data.

In the embodiment of the invention, a second data management thread, namely a cp_audio_thread, is created in the Linux kernel driver, and the thread manages the subscript values of audio data read (read_idx) and write (write_idx) in the SHM shared memory, and currently corresponds to the read_idx and counts the values, and meanwhile, when data is written, the ALSA driver is informed to perform read and write processing.

S450, controlling an AP_PROCESS PROCESS of the Android domain user space to write the audio data to be played into the first virtual sound card device.

In the embodiment of the invention, the AP_PROCESS PROCESS of the Android domain user space can call the pcm_write function of the ALSA LIB library, audio data to be played is written into the first virtual sound card device, and the Android kernel driver can PROCESS the audio data in the first virtual sound card device next.

S460, controlling the Android kernel driver to write the audio data into the SHM, updating the written subscript value, and waiting for the change of the read subscript value in the SHM.

In the embodiment of the invention, a self-defined function copy_user in an ALSA driving frame is called in an Android kernel driving to write audio data into a SHM shared memory, count and update is carried out on a write index value write_idx, the write index value write_idx counts the size of data of one frame at a time, and meanwhile, the read index value read_idx in the SHM is waited to change (the read index value read_idx is updated into the shared memory by a Linux domain). When the read index value read_idx changes in the SHM, it is indicated that the Linux domain has started the process of reading audio data from the SHM.

S470, the CP_PROCESS PROCESS of the Linux domain user space is controlled to read the audio data from the second virtual sound card device.

In the embodiment of the invention, the cp_process may call the standard function pcm_read of the ALSA LIB library to read audio data from the second virtual sound card device.

S480, controlling the Linux kernel driver to read the audio data from the SHM, and updating the reading index value.

In the embodiment of the invention, a self-defined function copy_user in an ALSA driving frame can be called in a Linux kernel driving to read audio data from a SHM shared memory, and a second data management thread is called to count and update a read index value read_idx, wherein the read index value read_idx counts one frame of data size at a time.

And S490, after the first data management thread is controlled to detect that the read index value of the audio data in the SHM changes, the next frame of audio data is continuously processed.

In the embodiment of the invention, the first data management thread ap_audio_thread of the Android domain periodically detects whether the read_idx of the audio data in the SHM is updated by counting, and when detecting that the read_idx is changed, the first data management thread can inform the copy_user function in the ALSA driver to continuously write the next frame of audio data into the SHM, and the operations are repeatedly performed until all the audio data are transmitted to the second virtual sound card device.

S4100, controlling a CP_PROCESS PROCESS of the Linux domain user space to write the audio data in the second virtual sound card device into the second real sound card device.

In the embodiment of the invention, the cp_process PROCESS may call an ALSA LIB library function to write the read audio data in the second virtual sound card device into the second real sound card device.

S4110, the second real sound card device transmits the audio data to the I2S controller through the ADMA controller, and the I2S controller controls the vehicle-mounted loudspeaker to play the audio data.

It can be understood that the embodiment of the present invention is also applicable to the case of playing by sending audio data from the Linux domain to the Android domain, and the specific process is similar to the above, so that the description is omitted here.

According to the technical scheme, the AP_PROCESS PROCESS of the Android domain user space is controlled to open the first virtual sound card device, the Android kernel driver is controlled to create a read-write lower standard value of the audio data read in the shared memory, the ALSA driver is informed to read and write the audio data, the CP_PROCESS PROCESS of the Linux domain user space is controlled to open the second virtual sound card device, the Linux kernel driver is controlled to create a read-write lower standard value of the audio data read in the second data management thread management shared memory, the ALSA driver is informed to read and write the audio data, the AP_PROCESS PROCESS of the Android domain user space is controlled to write the audio data to be played into the first virtual sound card device, the Android kernel driver is controlled to write the audio data into the SHM, the lower standard value is updated, the change of the audio data is waited for being read in the SHM, the CP_PROCESS PROCESS of the Linux domain user space is controlled to read the audio data from the second virtual sound card device, the Linux kernel driver is controlled to read the audio data from the second sound card device, the audio data is controlled to be read by the Linux domain user space, and the audio data is controlled to be continuously read into the audio card device by the audio card 2, and the audio device is controlled to be read to be played by the audio card 2. According to the embodiment of the invention, the audio data is transmitted from the first virtual sound card device in the Android domain to the second virtual sound card device in the Linux domain through the shared memory, and then the second real sound card device controls the audio data to be played through the ADMA controller, so that the cross-domain playing of the audio data is realized, the conflict problem existing in the audio processing between different operating system domains is solved, and the user experience is improved.

Example IV

Fig. 6 is a flowchart of a cross-domain audio processing method according to a fourth embodiment of the present invention. Based on the above embodiment, the embodiment provides an implementation manner of a cross-domain audio processing method, which takes a first operating system domain as a Linux domain and a second operating system domain as an Android domain as an example, and can implement cross-domain processing of recorded audio data. As shown in fig. 6, a cross-domain audio processing method provided in a fourth embodiment of the present invention specifically includes the following steps:

s510, controlling a CP_PROCESS PROCESS of a Linux domain user space to open the first real sound card device.

In the embodiment of the invention, the cp_process PROCESS of the Linux domain user space can call the standard function pcm_open of the ALSA LIB library to open the first real sound card device, and obtain a file descriptor.

S520, controlling the Linux kernel driver to create a read-write subscript value of the read audio data in the first data management thread management shared memory, and informing the ALSA driver to read and write the audio data.

S530, controlling an AP_PROCESS PROCESS of the Android domain user space to open the second virtual sound card device.

S540, controlling the Android kernel driver to create a read-write subscript value of the read audio data in the shared memory managed by the second data management thread, and informing the ALSA driver to read and write the audio data.

S550, the CP_PROCESS PROCESS of the Linux domain user space is controlled to read the audio data from the first real sound card device.

In the embodiment of the invention, the cp_process PROCESS can call the standard function pcm_read of the ALSA LIB library to read audio data from the first real sound card device, wherein the audio data is read from the hardware device, reaches the ADMA controller through the I2S controller, and finally obtains effective audio data.

S560, controlling a CP_PROCESS PROCESS of the Linux domain user space to write the audio data read in the first real sound card device into the first virtual sound card device.

In the embodiment of the invention, the cp_process may call the standard function pcm_write of the ALSA LIB library to write the data read from the first real sound card device into the first virtual sound card device, and then transmit the audio data to the SHM through the first virtual sound card device.

S570, controlling the Linux kernel driver to write the audio data into the SHM, updating the written index value, and waiting for conversion of the read index value in the SHM.

S580, the AP_PROCESS PROCESS of the Android domain user space is controlled to read the audio data from the second virtual sound card device.

S590, controlling the Android kernel driver to read the audio data from the SHM, and updating the reading index value.

S5100, after the first data management thread is controlled to detect that the read index value of the audio data in the SHM changes, the next frame of audio data is continuously processed.

In the embodiment of the present invention, after the first data management thread, i.e., the cp_audio_thread thread detects that the read_idx value of the audio data in the SHM changes, the copy_user function in the ALSA driver is notified to continue writing the next frame of audio data into the SHM. And repeatedly executing the steps until all the audio data are transmitted to the second virtual sound card equipment.

It can be appreciated that the embodiment of the present invention is also applicable to the case of transmitting the recorded audio data from the Android domain to the Linux domain, and the specific process is similar to the above, so that the description thereof will not be repeated here.

According to the technical scheme, a first real sound card device is opened by controlling a CP_PROCESS PROCESS of a Linux domain user space, a Linux kernel driver is controlled to create a read-write lower standard value of the read audio data in a first data management thread management shared memory, the ALSA driver is informed to read and write the audio data, an AP_PROCESS PROCESS of an Android domain user space is controlled to open a second virtual sound card device, the Android kernel driver is controlled to create a read-write lower standard value of the read audio data in the second data management thread management shared memory, the ALSA driver is informed to read and write the audio data, the CP_PROCESS PROCESS of the Linux domain user space is controlled to read the audio data from the first real sound card device, the CP_PROCESS PROCESS of the Linux domain user space is controlled to write the audio data read from the first real sound card device into the first virtual sound card device, the kernel driver is controlled to write the audio data into an SHM, the write lower standard value is updated, the change of the read lower standard value in the SHM is waited, the AP_PROCESS of the Android domain user space is controlled to read the audio data from the second sound card device, and the audio data is read from the second sound card device is controlled to be read and the audio data is continuously after the audio data is read from the second sound card device is controlled to read. According to the embodiment of the invention, the recorded audio data acquired from the first real sound card device in the Linux domain is transmitted to the second virtual sound card device in the Android domain through the shared memory, so that the cross-domain processing of the recorded audio data is realized, the conflict problem existing in the audio processing between different operating system domains is solved, and the user experience is improved.

Example five

Fig. 7 is a flowchart of a cross-domain audio processing method according to a fifth embodiment of the present invention. Based on the above embodiment, the embodiment provides an implementation manner of a cross-domain audio processing method, which takes the first operating system domain as the Android domain and the second operating system domain as the Linux domain as an example, so as to realize dynamic cross-domain switching of the ADMA controller. As shown in fig. 7, a cross-domain audio processing method provided in a fifth embodiment of the present invention specifically includes the following steps:

s610, an AP_PROCESS PROCESS controlling the Android domain user space transmits a controller demand instruction to the Linux domain through an RPMSG message.

In the embodiment of the invention, the AP_PROCESS PROCESS of the Android domain can send a controller demand instruction, namely an AP_NEED_ADMA instruction, through a send function in an RPMSG mechanism, so that the Android domain can indicate that the Android domain NEEDs the control right of an ADMA controller.

S620, a CP_PROCESS PROCESS controlling the Linux domain user space receives a controller demand instruction.

In the embodiment of the invention, the Linux domain cp_process PROCESS can receive an ap_new_adma instruction through a receive function in the RPMSG mechanism, and prepares to release the control right of the ADMA controller.

S630, controlling the Linux domain to remove the control right of the ADMA controller, and feeding back right removing information to the Android domain.

In the embodiment of the invention, the CP_PROCESS PROCESS informs the Linux kernel driver through a write function, removes the ADMA controller by writing a register of the ADMA controller in the kernel driver, and sends a permission removal message, namely a CP_DISABLE_ADMA instruction, to the Android domain through a send function of an RPMSG mechanism.

S640, controlling the Android domain to acquire the control right of the ADMA controller after receiving the right removal message.

In the embodiment of the invention, the AP_PROCESS PROCESS can receive the authority removal message sent from the Linux domain through a receiver function in the RPMSG mechanism, and inform the Android kernel driver through a write function, and acquire the control authority of the ADMA controller through writing a register of the ADMA controller in the kernel driver. Further, after the Android domain obtains the control right of the ADMA controller, the ap_process PROCESS may perform the audio data playing and recording operation through the first real sound card device in the Android domain.

It can be understood that the embodiment of the present invention is also applicable to the case of switching the control right of the ADMA controller from the Android domain to the Linux domain, and the specific process is similar to the above, so that the description is omitted here.

According to the technical scheme, the AP_PROCESS PROCESS controlling the Android domain user space transmits the controller demand instruction to the Linux domain through the RPMSG message, the CP_PROCESS PROCESS controlling the Linux domain user space receives the controller demand instruction, the Linux domain is controlled to remove the control right of the ADMA controller, the permission removal message is fed back to the Android domain, and the Android domain is controlled to acquire the control right of the ADMA controller after receiving the permission removal message. According to the embodiment of the invention, the controller demand instruction and the permission removal message between the Android domain and the Linux domain are sent and received through the RPMSG message, the control right of the controller between different operating system domains is flexibly switched, each operating system domain can realize the playing and recording operation of audio data through the real sound card device, and the user experience is improved.

Example six

Fig. 8 is a schematic structural diagram of a cross-domain audio processing device according to a sixth embodiment of the present invention. As shown in fig. 8, the apparatus is applied to a system on chip, the system on chip including at least two operating system domains, the apparatus comprising:

the data acquisition module 71 is configured to acquire audio data according to a first real sound card device and write the audio data into a first virtual sound card device, where the first real sound card device and the first virtual sound card device are located in a first operating system domain.

The cross-domain transmission module 72 is configured to transmit audio data of the first virtual sound card device to the second virtual sound card device based on the shared memory.

The data processing module 73 is configured to process the audio data received by the second virtual sound card device based on the second real sound card device, where the second real sound card device and the second virtual sound card device are located in a second operating system domain.

According to the technical scheme, the data acquisition module acquires audio data according to the first real sound card device and writes the audio data into the first virtual sound card device, wherein the first real sound card device and the first virtual sound card device are located in a first operating system domain, the cross-domain transmission module transmits the audio data of the first virtual sound card device to the second virtual sound card device based on the shared memory, and the data processing module processes the audio data received by the second virtual sound card device based on the second real sound card device, wherein the second real sound card device and the second virtual sound card device are located in a second operating system domain. According to the embodiment of the invention, the shared memory is used for transmitting the audio data from the first virtual sound card device of the first operating system domain to the second virtual sound card device of the second operating system domain, and the second real sound card device reads and processes the audio data in the second virtual sound card device, so that the cross-domain processing of the audio data is realized, the conflict problem existing in the processing of the audio data among different operating system domains is solved, and the user experience is improved.

Further, on the basis of the above embodiment of the invention, the data acquisition module 71 includes:

and the data acquisition unit is used for calling the first real sound card equipment to acquire audio data.

And the data acquisition unit is used for controlling the first virtual sound card device to acquire the audio data of the first real sound card device.

Further, on the basis of the above embodiment of the present invention, the data acquisition unit is specifically configured to at least one of:

controlling a first real sound card device to call a write audio interface function to write audio data into a first virtual sound card device;

and controlling the first virtual sound card device to call a reading audio interface function to read audio data from the first real sound card device.

Further, on the basis of the above embodiment of the present invention, the cross-domain transmission module 72 includes:

and the data writing unit is used for controlling the first virtual sound card equipment to write the audio data into the shared memory.

The first data management unit is used for creating a read-write index value of the audio data written in the shared memory by the first data management thread in the first operating system domain.

The first data reading unit is used for controlling the second virtual sound card device to read the audio data in the shared memory.

And the second data management unit is used for creating a read-write index value of the read audio data in the shared memory by the second data management thread in the second operating system domain.

Further, on the basis of the above embodiment of the invention, the data processing module 73 includes:

and the second data reading unit is used for calling the second real sound card device to read the audio data received by the second virtual sound card device.

And the data transmission and processing unit is used for transmitting the audio data to a controller with control right of the second real sound card device and processing the audio data in the controller.

Further, on the basis of the above embodiment of the present invention, the apparatus further includes:

and the sound card registration module is used for registering the first real sound card device and the first virtual sound card device in the first operating system domain and registering the second real sound card device and the second virtual sound card device in the second operating system domain.

and the demand instruction transmission module is used for transmitting the controller demand instruction to the second operating system domain through the remote inter-core communication bus in the first operating system domain.

And the control right removing module is used for removing the control right of the corresponding controller in the second operating system domain and feeding back a right removing message to the first operating system domain.

And the control right acquisition module is used for acquiring the control right of the controller after the first operating system domain receives the right removal message.

The cross-domain audio processing device provided by the embodiment of the invention can execute the cross-domain audio processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example seven

Fig. 9 shows a schematic diagram of an electronic device 80 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 9, the electronic device 80 includes at least one processor 81, and a memory, such as a Read Only Memory (ROM) 82, a Random Access Memory (RAM) 83, etc., communicatively connected to the at least one processor 81, in which the memory stores a computer program executable by the at least one processor, and the processor 81 can perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 82 or the computer program loaded from the storage unit 88 into the Random Access Memory (RAM) 83. In the RAM 83, various programs and data required for the operation of the electronic device 80 can also be stored. The processor 81, the ROM 82 and the RAM 83 are connected to each other via a bus 84. An input/output (I/O) interface 85 is also connected to bus 84.

Various components in the electronic device 80 are connected to the I/O interface 85, including: an input unit 86 such as a keyboard, mouse, etc.; an output unit 87 such as various types of displays, speakers, and the like; a storage unit 88 such as a magnetic disk, an optical disk, or the like; and a communication unit 89, such as a network card, modem, wireless communication transceiver, etc. The communication unit 89 allows the electronic device 80 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.

Processor 81 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 81 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 81 performs the various methods and processes described above, such as cross-domain audio processing methods.

In some embodiments, the cross-domain audio processing method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 88. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 80 via the ROM 82 and/or the communication unit 89. When the computer program is loaded into RAM 83 and executed by processor 81, one or more steps of the cross-domain audio processing method described above may be performed. Alternatively, in other embodiments, the processor 81 may be configured to perform the cross-domain audio processing method in any other suitable way (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A method of cross-domain audio processing applied to a system-on-chip, the system-on-chip comprising at least two operating system domains, the method comprising:

acquiring audio data according to a first real sound card device and writing the audio data into a first virtual sound card device, wherein the first real sound card device and the first virtual sound card device are positioned in a first operating system domain;

Transmitting the audio data of the first virtual sound card device to a second virtual sound card device based on a shared memory;

and processing the audio data received by the second virtual sound card device based on a second real sound card device, wherein the second real sound card device and the second virtual sound card device are positioned in a second operating system domain.

2. The method as recited in claim 1, further comprising:

registering the first real sound card device and the first virtual sound card device in the first operating system domain, and registering the second real sound card device and the second virtual sound card device in the second operating system domain.

3. The method of claim 1, wherein the obtaining audio data from the first real sound card apparatus and writing the audio data to the first virtual sound card apparatus comprises:

calling the first real sound card device to collect the audio data;

and controlling the first virtual sound card device to acquire the audio data of the first real sound card device.

4. A method according to claim 3, wherein said controlling said first virtual sound card apparatus to obtain said audio data of said first real sound card apparatus comprises at least one of:

Controlling the first real sound card device to call a write audio interface function to write the audio data into the first virtual sound card device;

and controlling the first virtual sound card device to call an audio reading interface function to read the audio data from the first real sound card device.

5. The method of claim 1, wherein the transmitting the audio data of the first virtual sound card device to a second virtual sound card device based on a shared memory comprises:

controlling the first virtual sound card device to write the audio data into the shared memory;

creating a first data management thread in the first operating system domain to manage the read-write subscript value of the audio data written in the shared memory;

controlling the second virtual sound card device to read the audio data in the shared memory;

creating a second data management thread in the second operating system domain to manage the read-write subscript value of the audio data read in the shared memory.

6. The method of claim 1, wherein the processing the audio data received by the second virtual sound card device based on the second real sound card device comprises:

Calling the second real sound card device to read the audio data received by the second virtual sound card device;

transmitting the audio data to a controller with control right of the second real sound card device, and processing the audio data at the controller.

7. The method as recited in claim 1, further comprising:

transmitting a controller demand instruction at the first operating system domain to the second operating system domain over a remote inter-core communication bus;

removing control rights of the corresponding controllers in the second operating system domain, and feeding back rights removal messages to the first operating system domain;

and acquiring the control right of the controller after the first operating system domain receives the right removing message.

8. A cross-domain audio processing apparatus for use with a system-on-chip, the system-on-chip comprising at least two operating system domains, the apparatus comprising:

9. An electronic device, the electronic device comprising:

at least two chips, at least one of the chips configuring a first operating system domain and at least one of the chips configuring a second operating system domain;

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the cross-domain audio processing method of any one of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to perform the cross-domain audio processing method of any one of claims 1-7.