CN115209439A

CN115209439A - Wireless terminal and method for sending and receiving audio and video data

Info

Publication number: CN115209439A
Application number: CN202110384836.6A
Authority: CN
Inventors: 许超杰
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-04-09
Filing date: 2021-04-09
Publication date: 2022-10-18
Also published as: WO2022213711A1

Abstract

The application discloses a wireless terminal, a method for sending audio and video data, a method for receiving audio and video data and an audio and video data transmission system. The first wireless terminal comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring first data which is audio and video original data; the first WiFi module comprises a first coding unit and a sending unit, and the first coding unit is used for coding the first data and generating second data; the sending unit is configured to send the second data to the second wireless terminal.

Description

Wireless terminal and method for sending and receiving audio and video data

Technical Field

The present application relates to the field of audio and video data transmission, and in particular, to a wireless terminal, a method for transmitting audio and video data, a method for receiving audio and video data, and an audio and video communication system.

Background

In the related art, in the process of encoding of the audio and video data by the sending device, transmitting the audio and video data by the wireless air interface, and decoding by the receiving device, transmission delay is generated due to factors such as hardware of the audio and video data sending device or the audio data receiving device, and a link state of the wireless air interface, so that the problem of audio and video playing is blocked.

Disclosure of Invention

In view of this, embodiments of the present application provide a wireless terminal, a method for sending audio/video data, a method for receiving audio/video data, and an audio/video data transmission system, so as to at least solve the problem that audio/video playing is jammed due to transmission delay generated during transmission of video data or audio data in the related art.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a method for sending audio and video data in a first wireless terminal, wherein the first wireless terminal comprises an acquisition module and a first WiFi module, and the method comprises the following steps:

collecting first data based on the collecting module; the first data is audio and video original data;

encoding the first data based on a first encoding unit in the first WiFi module to generate second data;

and transmitting the second data to a second wireless terminal based on the transmitting unit in the first WiFi module.

In the above solution, the encoding the first data based on the first encoding unit in the first WiFi module and generating the second data includes:

and if the coding capability of the first coding unit meets the coding requirement of the first data, starting the first coding unit to code the first data to generate the second data.

In the foregoing solution, the sending the second data to the second wireless terminal based on the sending unit in the first WiFi module includes:

and monitoring a wireless air interface state, and feeding back the air interface state to the first coding unit so that the first coding unit determines a coding rate according to the air interface state.

In the foregoing solution, the encoding the first data based on the first encoding unit in the first WiFi module and generating second data further includes:

dividing each frame of data in the first data into at least two pieces of slice data according to the air interface state; and encoding the first data to generate the second data in units of the single slice data.

The embodiment of the present application further provides a method for receiving audio and video data in a second wireless terminal, where the second wireless terminal includes a second WiFi module and a playing module, and the method includes:

receiving the second data based on a wireless link established between a receiving unit of a second WiFi module and a transmitting unit of a first WiFi module;

decoding the second data based on a decoding unit of a second WiFi module to obtain third data;

and playing the corresponding audio and video by the playing module according to the third data.

In the foregoing solution, the decoding the second data based on the decoding unit in the second WiFi module to obtain third data includes: and if the second data are detected to be encoding normal data, controlling the decoding unit to decode the second data.

In the foregoing solution, the decoding unit in the second WiFi module decodes the second data to obtain third data, further including: determining a resolution corresponding to the second data;

the playing of the corresponding audio and video by the playing module comprises the following steps: and playing the audio data corresponding to the third data according to the resolution.

In the foregoing solution, the decoding unit based on the second WiFi module decodes the second data to obtain third data, further including: and if the second data is detected to be coding abnormal data, controlling the receiving unit to send data abnormal feedback to the sending unit, so that the first WiFi module codes the first data again and sends the second data.

The embodiment of the present application further provides a first wireless terminal, configured to send audio and video data, including:

the acquisition module is used for acquiring first data, and the first data is audio and video original data; and

the first WiFi module comprises a first coding unit and a sending unit, and the first coding unit is used for coding the first data and generating second data; the sending unit is configured to send the second data to the second wireless terminal.

In the foregoing solution, the first wireless terminal further includes:

and the processor is used for starting the first coding unit to code the first data to generate the second data if the processor determines that the coding capability of the first coding unit meets the coding requirement of the first data.

In the foregoing scheme, the sending unit is configured to establish a wireless link with a receiving unit in the second wireless terminal to transmit the second data, and the sending unit is further configured to monitor an air interface state of the wireless link and feed back the air interface state to the first encoding unit, so that the first encoding unit determines an encoding code rate according to the air interface state.

In the foregoing scheme, the first encoding unit is further configured to divide each frame of data in the first data into at least two pieces of slice data according to the air interface state; and encoding the first data to generate the second data in units of the single slice data.

In the above scheme, the processor and the acquisition module are electrically connected to the first WiFi module respectively.

In the above scheme, the acquisition module, the processor and the first WiFi module are electrically connected in sequence; the processor includes a second encoding unit; the second encoding unit is configured to encode the first data if the processor determines that the second encoding unit encodes the first data.

The embodiment of the present application further provides a second wireless terminal, configured to receive audio and video data, and applied to the first wireless terminal, including:

the second WiFi module comprises a receiving unit and a decoding unit, and the receiving unit is used for receiving the second data; the decoding unit is used for decoding the second data to obtain third data; and

and the playing module is used for playing the audio and video corresponding to the third data.

In the above solution, the second WiFi module is further configured to control the decoding unit to decode the second data if it is detected that the second data is encoded normal data.

In the above solution, the decoding unit is further configured to determine a resolution corresponding to the second data; the playing module is further used for playing the audio and video corresponding to the third data according to the resolution.

In the foregoing scheme, the second WiFi module is further configured to control the receiving unit to send data exception feedback to the sending unit if it is detected that the second data is coding exception data, so that the first WiFi module codes the first data again and sends the second data.

Embodiments of the present application also provide a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of any one of the above methods.

The embodiment of the application also provides an audio and video data transmission system, which comprises a first wireless terminal and a second wireless terminal,

wherein, first wireless terminal is used for sending audio and video data, first wireless terminal includes: the acquisition module is used for acquiring first data, and the first data is audio and video original data; the first WiFi module comprises a first coding unit and a sending unit, and the first coding unit is used for coding the first data and generating second data; the sending unit is used for sending the second data to a second wireless terminal;

the second wireless terminal is used for receiving audio and video data, and comprises: the second WiFi module comprises a receiving unit and a decoding unit, and the receiving unit is used for receiving the second data; the decoding unit is used for decoding the second data to obtain third data; and the playing module is used for playing the audio and video corresponding to the third data.

In the embodiment of the application, the first wireless terminal can complete the coding of the audio and video data in the first coding unit of the first WiFi module, and can directly send the audio and video data through the sending unit of the first WiFi module, so that the transmission link of the audio and video data at the first wireless terminal is reduced, the hardware transmission path of the audio and video data at the first wireless terminal is shortened, the transmission delay of the audio and video data is reduced, the sending efficiency of the audio and video data is improved, and the phenomenon of blockage in the audio and video playing process is avoided.

Drawings

Fig. 1 is a schematic structural diagram of a first wireless terminal according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a first wireless terminal according to yet another embodiment of the present application;

fig. 3 is a schematic diagram of a first wireless terminal sending audio/video data according to an embodiment of the present application;

fig. 4 is a schematic view illustrating feedback of an air interface state of a wireless link according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a first wireless terminal according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a first wireless terminal according to yet another embodiment of the present application;

fig. 7 is a schematic structural diagram of a second wireless terminal according to an embodiment of the present application;

fig. 8 is a schematic flow chart of an implementation of a method for transmitting audio and video data in an audio and video data transmitting device according to an embodiment of the present application;

fig. 9 is a schematic implementation flowchart of a method for receiving audio and video data in a second wireless terminal according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an audio and video data transmission system according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an audio-video communication system according to still another embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and specific embodiments.

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.

In addition, in the embodiments of the present application, "first", "second", and the like are used for distinguishing similar objects, and are not necessarily used for describing a specific order or a sequential order.

Before the technical solution of the embodiment of the present application is explained in detail, first, a simple explanation is made on an audio and video data transmission method in the related art.

In the related art, the delay of audio and video transmission can be divided into three types, the first type is the delay on the transmitting terminal, and the delay on the transmitting terminal can be subdivided into the delays caused by audio and video data acquisition, preprocessing and encoding, and the delays generated under the three conditions are mainly related to hardware capability and an adopted encoding algorithm, for example, the signal conversion delay of an audio sound card is about 1-30ms, and the encoding delay is about 2.5-60 ms. In addition, the delay on the first wireless terminal further includes transmission delay of audio and video data transmitted to the wireless transmission module through a Central Processing Unit (CPU) after being processed, and delay inside the wireless transmission module. The second type of delay is the wireless air interface transmission delay, which depends on the link state of the wireless air interface. The third type of delay is the delay at the second wireless terminal, which is determined by the internal delay, decoding delay and display delay of the wireless receiving module. In addition, no matter the first wireless terminal or the second wireless terminal is in the running process, the CPU and the cache can simultaneously process the requests of multiple applications and external devices, and when the CPU is occupied by other requests, the processing request of the audio and video can also be delayed.

Based on this, in the embodiment of the application, the transmission delay of the audio and video data in the transmission on the hardware level is reduced by adjusting the hardware path of the audio and video data transmission.

An embodiment of the present application provides a first wireless terminal 10, configured to send audio and video data, and fig. 1 is a schematic structural diagram of the first wireless terminal 10 provided in the embodiment of the present application. As shown in fig. 1, the first wireless terminal 10 includes:

the acquisition module 101 is used for acquiring first data, wherein the first data is audio and video original data; and

a first WiFi module 102, where the first WiFi module 102 includes a first encoding unit 1021 and a sending unit 1022, where the first encoding unit 1021 is configured to perform encoding processing on the first data and generate second data; the sending unit 1022 is configured to send the second data to the second wireless terminal 70.

In this embodiment of the application, the acquisition module 101 is configured to acquire raw audio/video data, and it is understood that the acquisition module 101 may include an image acquisition unit and/or a sound acquisition unit. The image acquisition unit is used for sensing ambient light and acquiring original image data; the image acquisition unit may be an image sensor, and may be, for example, a CCD image sensor or a CMOS image sensor. The sound collection unit is used for collecting audio data, and the sound collection unit can be a microphone, for example. The acquisition module 101 can acquire a large amount of audio/video original data (i.e., first data), and the data amount of the audio/video data to be transmitted can be reduced by encoding and compressing the large amount of first data, so that the transmission speed of the audio/video data is increased. In practical application, the first WiFi module 102 may be electrically connected to the acquisition module 101, after the acquisition module 101 acquires the first data, the first data may be transmitted to the first WiFi module 102 through an electrical connection between the acquisition module 101 and the first WiFi module 102, for example, a Mobile Industry Processor Interface (MIPI), and the like, and the first encoding unit 1021 in the first WiFi module 102 may encode the first data acquired by the acquisition module 101 to generate second data, and send the second data to the second wireless terminal through the sending unit 1022. The first data acquired by the acquisition module 101 is directly transmitted to the first WiFi module 102, the first encoding unit 1021 in the first WiFi module 102 encodes the first data, and the sending unit 1022 in the first WiFi module 102 sends the second data generated by encoding, without transmitting the first data to other hardware units, so that transmission paths of the audio and video data in different hardware units are greatly reduced, and transmission delay on a hardware level can be reduced.

Referring to fig. 2, in some embodiments of the present application, the first wireless terminal 10 further includes a processor 201, and the processor 201 may be electrically connected to the first WiFi module 102, and configured to determine whether the encoding capability of the first encoding unit 1021 satisfies an encoding requirement of first data, and if the processor 201 determines that the encoding capability of the first encoding unit 1021 satisfies the encoding requirement of the first data, the processor 201 starts the first encoding unit 1021 to perform encoding processing on the first data to generate the second data.

It is understood that different data puts different encoding requirements on the encoding capability of the first encoding unit 1021, and therefore before encoding the first data, the processor 201 needs to determine whether the first encoding unit 1021 has the capability of encoding the first data. The processor 201 may determine the encoding capability of the first encoding unit 1021 through the command parameter, and if the processor 201 determines that the encoding capability of the first encoding unit 1021 can meet the encoding requirement of the first data, the first encoding unit 1021 is started to encode the first data, and the acquisition module 101 may directly transmit the first data to the first WiFi module 102, so that the first data does not need to be transmitted to other hardware units. In practical applications, the processor 201 may detect the encoding capability of the first encoding unit 1021 when the first wireless terminal 10 is powered on, where the encoding capability of the first encoding unit 1021 may be determined according to indexes such as a maximum data amount that the first encoding unit 1021 can process, a compression amount, and whether the first encoding unit 1021 conforms to an audio/video encoding format.

In an embodiment, as shown in fig. 3, the sending unit 1022 is configured to establish a wireless link with a receiving unit 7011 in the second wireless terminal 70 to transmit the second data, and the sending unit 1022 is further configured to monitor an air interface state of the wireless link, and feed back the air interface state to the first encoding unit 1021, so that the first encoding unit 1021 determines an encoding code rate according to the air interface state.

Here, when transmitting second data to second wireless terminal 70, transmitting section 1022 needs to establish a wireless link with receiving section 7011 of second wireless terminal 70, and transmits the second data to second wireless terminal 70 based on the established wireless link. In practical applications, an air interface state of a wireless link may affect transmission efficiency of second data, for example, when interference of the air interface is large, the transmission state of the wireless link is poor, so that a second data packet is stacked, and a large transmission delay is caused, for this reason, the air interface state of the wireless link needs to be monitored by the sending unit 1022, and a monitored result is fed back to the first encoding unit 1021, and the first encoding unit 1021 can adjust an encoding code rate of first data according to the air interface state of the wireless link, as shown in fig. 4, where fig. 4 shows a feedback schematic diagram of the air interface state of the wireless link. If the air interface state of the wireless link is in a good state, the first encoding unit 1021 can increase the encoding code rate of the first data, so that the sending efficiency of the audio and video data can be improved. In practical applications, the feedback time of the air interface state of the wireless link of the sending unit 1022 needs to determine the corresponding feedback time according to the current transmission rate, for example, the current transmission rate is 30fps, and the corresponding feedback time is 100-200ms. In practical application, the resolution in the encoding process of the first data may also be adjusted in real time according to the air interface state of the wireless link, for example, if the air interface state of the wireless link is not good, the resolution in the encoding process of the first data may be adjusted to be low, the data amount of the correspondingly generated second data is reduced, which is beneficial to improving the sending efficiency of the second data under the condition that the air interface state of the wireless link is not good, so that the cache accumulation in the sending process of the audio and video data may be reduced, and the transmission delay of the audio and video data may be reduced.

In an embodiment, the first encoding unit 1021 is further configured to divide each frame of data in the first data into at least two pieces of slice data according to the air interface state; and encoding the first data to generate the second data in units of the single slice data.

Here, in the process of encoding the first data by the first encoding unit 1021, the first encoding unit 1021 may divide each frame of the first data into a plurality of slice data, the first encoding unit 1021 performs encoding processing on the first data of a single slice data in units of a single slice data, which reduces the data amount that the first encoding unit 1021 needs to process for a single encoding, and after the first encoding unit 1021 performs encoding processing on the first data of a single slice data, generates second data of a single slice data, and the communication transmitting unit 1022 transmits the second data of a single slice data to the second wireless terminal 70, so that the transmission efficiency of the audio/video data can be improved, and particularly, in the case where the state of the empty port is in a bad state, the transmission delay of the audio/video data can be reduced. In practical applications, when the state of the air interface is in a good state, the data amount of the audio/video data transmission should be increased, so that the first encoding unit 1021 can directly encode the first data in the process of encoding the first data without further dividing each frame of the first data into multiple pieces of slice data, thereby ensuring that the audio/video data is successfully transmitted and increasing the single data processing amount of the first encoding unit 1021.

In one embodiment, as shown in fig. 5, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102 respectively.

Here, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102, so that on a hardware connection, the first data acquired by the acquisition module 101 may be directly transmitted to the first WiFi module 102, the first data is encoded by the first encoding unit 1021 in the first WiFi module 102, and the second data is transmitted by the transmitting unit 1022 in the first WiFi module 102, thereby implementing a shortest hardware transmission path between the acquisition and transmission of the audio and video data, and reducing a transmission delay of the audio and video data on a hardware level.

In an embodiment, as shown in fig. 6, the acquisition module 101, the processor 201 and the first WiFi module 102 are electrically connected in sequence; the processor 201 comprises a second encoding unit 2011; the second encoding unit 2011 is configured to encode the first data if the processor 201 determines that the second encoding unit 2011 encodes the first data.

Here, the acquisition module 101 is electrically connected to the processor 201, the processor 201 is further electrically connected to the first WiFi module 102, the processor 201 further includes a second encoding unit 2011, when the acquisition module 101 acquires first data, the processor 201 may determine whether the first data is encoded by the first encoding unit 1021 in the first WiFi module 102, for example, may determine whether the encoding capability of the first encoding unit 1021 meets the encoding requirement of the first data, if the processor 201 determines that the first data is encoded by the first encoding unit 1021, the first data is transmitted from the processor 201 to the first encoding unit 1021 and the encoding processing of the first data is completed by the first encoding unit, if the processor 201 determines that the first data is encoded by the second encoding unit 2011, the acquisition module 101 may transmit the acquired first data to the second encoding unit 2011, the second encoding unit 2011 encodes the first data to generate second data, and transmit the second data to the transmission unit 1022 of the first data to the first WiFi module 102, and the transmission of the second data to the transmission unit 1022 is controlled by the second WiFi module 102, and the transmission level of the data is reduced, and the transmission of the data can be controlled by the hardware chip 201.

In the above embodiment, the acquisition module 101 is electrically connected to the processor 201; the processor 201 comprises a second encoding unit 2011; the second encoding unit 2011 is configured to encode the first data if the processor 201 determines that the second encoding unit 2011 encodes the first data, so that transmission of audio and video data on a hardware layer can be dynamically controlled, hardware change is reduced, and time delay of data transmission on the hardware layer is reduced to a certain extent.

The embodiment of the present application further provides a second wireless terminal 70, as shown in fig. 7, where the second wireless terminal 70 includes:

a second WiFi module 701; the second WiFi module 701 includes a receiving unit 7011 and a decoding unit 7012; the receiving unit 7011 is configured to establish a wireless link with the transmitting unit 1022 of the first WiFi module 102, and receive second data transmitted by the wireless link; the decoding unit 7012 is configured to decode the second data to obtain third data; and a playing module 702, which may be electrically connected to the second WiFi module 701, and is configured to play the audio and video corresponding to the third data.

Here, the second wireless terminal 70 includes a second WiFi module 701 and a playing module 702, where the second WiFi module 701 can establish a wireless link with the first WiFi module 102 of the first wireless terminal 10, a receiving unit 7011 of the second WiFi module 701 can receive second data on the wireless link, the receiving unit 7011 can transmit the received second data to a decoding unit 7012 of the second WiFi module 701, the decoding unit 7012 can decode the second data to obtain third data, the second WiFi module 701 is electrically connected to the playing module 702, so that the decoding unit 7012 transmits the third data obtained by decoding the second data to the playing module 702, and the playing module 702 can play an audio and video corresponding to the third data. In the second wireless terminal 70 provided in the embodiment of the present application, after receiving the audio and video data sent by the first wireless terminal 10, the decoding unit 7012 directly decodes the audio and video data, which reduces a hardware transmission path of the audio and video data in the second wireless terminal, and reduces an internal time delay of the audio and video data in the second wireless terminal, thereby improving transmission efficiency of the audio and video data.

In an embodiment, the second WiFi module 701 is further configured to control the decoding unit 7012 to decode the second data if it is detected that the second data is encoded normal data.

Here, the second WiFi module 701 may further detect whether the second data is normal encoded data in the process of receiving the second data, and when the second data is normal encoded data, the second WiFi module 701 indicates that no data error, frame missing, or frame error occurs in the process of generating the second data in an encoding process, the second WiFi module 701 controls the decoding unit 7012 to decode the second data, and the third data decoded by the decoding unit 7012 is complete, so that the playing module 702 is ensured to be capable of normally playing corresponding audio and video data, and the display delay of the audio and video data receiving end is reduced.

In an embodiment, the decoding unit 7012 is further configured to determine a resolution corresponding to the second data; the playing module 702 is further configured to play the audio and video data corresponding to the third data according to the resolution.

Here, in the process of decoding the second data by the decoding unit 7012, it may be determined to generate a resolution corresponding to the second data, where the resolution corresponding to the second data is a coding parameter used when the first wireless terminal 10 codes the first data, and in the process of playing, by the playing module 702, the audio and video corresponding to the third data is played according to the resolution corresponding to the second data, and when the resolution corresponding to the second data changes, the resolution of the played audio and video data also changes correspondingly, so that the resolution may be adjusted in real time in the process of playing the audio and video data. In practical application, when the resolution of some frames is reduced, the visual experience of a user is not greatly influenced, the audio and video is displayed timely and smoothly, the audio and video display time delay is reduced, and the audio and video playing effect is improved.

In an embodiment, the second WiFi module 701 is further configured to, if it is detected that the second data is abnormal data, control the receiving unit 7011 to send data abnormality feedback to the sending unit 1022, so that the first WiFi module 102 encodes the first data again and sends the second data.

Here, in the process of detecting the second data by the second WiFi module 701, if the second data is abnormal data, for example, the second data has a frame drop or a frame error, a data error may occur in the third data obtained by decoding, so that a situation that the playing cannot be performed or the playing is stuck may occur in the playing process, the receiving unit 7011 may feed back the situation that the data of the second data is abnormal to the sending unit 1022 of the first wireless terminal 10, so that the first WiFi module 102 of the first wireless terminal 10 recodes the corresponding first data and sends the second data to the second wireless terminal 70 again, and it is ensured that the second data received by the second wireless terminal 70 is normal data, and the phenomenon that the playing cannot be performed or the playing is stuck is avoided in the audio and video playing process.

In the above embodiment, the second WiFi module 701 is further configured to control the communication receiving unit 7011 to send data exception feedback to the communication sending unit if it is detected that the second data is coding exception data, so that the first WiFi module 102 codes the first data again and sends the second data, a real-time feedback of frame error or frame loss can be implemented, and it is ensured that the second wireless terminal 70 can receive an audio/video data packet with normal coded data, thereby reducing a display delay of the audio/video data in a playing process.

The embodiment of the present application further provides a method for sending audio and video data, which is applied to a first wireless terminal 10, where the first wireless terminal 10 includes an acquisition module 101 and a first WiFi module 102, and the method is shown in fig. 8 and includes:

s801: acquiring first data based on the acquisition module 101; the first data are audio and video original data.

In this embodiment of the application, a schematic structural diagram of the first wireless terminal 10 is shown in fig. 1, and a large amount of audio/video raw data (first data) can be acquired by using an acquisition module 101, where the first data is the audio/video raw data, it can be understood that the acquisition module 101 may include an image acquisition unit and/or a sound acquisition unit, where the image acquisition unit is used to sense ambient light and acquire the image raw data, and the image acquisition unit may be an image sensor, for example, a CCD image sensor or a CMOS image. The sound collection unit is used for collecting audio data, and the sound collection unit can be a microphone, for example.

S802: the first data is encoded based on the first encoding unit 1021 in the first WiFi module 102, and the second data is generated.

Here, a large amount of first data acquired by the acquisition module 101 needs to be encoded and compressed, so that the data amount of the audio and video data that needs to be transmitted can be reduced, and the transmission speed of the audio and video data is further improved, and the first encoding unit 1021 in the first WiFi module 102 is responsible for encoding the first data to generate second data. In practical applications, the first WiFi module 102 may be electrically connected to the collection module 101, and after the collection module 101 collects the first data, the first data may be transmitted to the first WiFi module 102 through an electrical connection, for example, an MIPI interface, between the collection module 101 and the first WiFi module 102.

S803: the second data is transmitted to the second wireless terminal based on the transmitting unit 1022 in the first WiFi module 102.

Here, the generated second data needs to be transmitted to the second wireless terminal 70, and the second data is transmitted to the second wireless terminal 70 through the transmitting unit 1022 in the first WiFi module 102, in practical applications, a wireless link may be established between the first WiFi module 102 of the first wireless terminal 10 and the second WiFi module 701 of the second wireless terminal 70, so that the transmission and reception of the second data can be realized through the wireless link. In practical applications, the wireless communication method between the first wireless terminal 10 and the second wireless terminal 70 may be any one of WiFi, bluetooth, zigbee, and UWB. The first data acquired by the acquisition module 101 is directly transmitted to the first WiFi module 102, the first encoding unit 1021 in the first WiFi module 102 encodes the first data, and the sending unit 1022 in the first WiFi module 102 sends the second data generated by encoding, without transmitting the first data to other hardware units, so that transmission paths of the audio and video data in different hardware units are greatly reduced, and transmission delay on a hardware level can be reduced.

In the above embodiment, the encoding of the audio and video data can be completed based on the first encoding unit 1021 in the first WiFi module 102, and the audio and video data can be sent out directly based on the sending unit 1022 in the first WiFi module 102, which reduces the transmission link of the audio and video data at the first wireless terminal 10, thereby shortening the hardware transmission path of the audio and video data at the first wireless terminal 10, reducing the transmission delay of the audio and video data, improving the sending efficiency of the audio and video data, and avoiding the stutter phenomenon in the audio and video playing process.

In an embodiment, the encoding the first data based on the first encoding unit 1021 in the first WiFi module 102 and generating second data includes:

if the encoding capability of the first encoding unit 1021 meets the encoding requirement of the first data, the first encoding unit 1021 is started to perform encoding processing on the first data to generate the second data.

Here, it can be understood that different data put forth different encoding requirements on the encoding capability of the first encoding unit 1021, and therefore before encoding the first data, it is necessary to determine whether the encoding capability of the first encoding unit 1021 satisfies the encoding requirement of the first data, and if the encoding capability of the first encoding unit 1021 satisfies the encoding data of the first data, the first encoding unit 1021 may be started to perform encoding processing on the first data to generate the second data, so that the second data matching the encoding requirement can be obtained. Referring to fig. 2, in some embodiments of the present application, the first wireless terminal 10 further includes a processor 201, and the processor 201 may determine the encoding capability of the first encoding unit 1021 through a command parameter, in practical applications, detection of the encoding capability of the first encoding unit 1021 by the processor 201 may be detected when the first wireless terminal 10 is powered on, where the encoding capability of the first encoding unit 1021 may be determined according to indexes such as a maximum data amount that the first encoding unit 1021 can process, a compression amount, and whether it conforms to an audio-video encoding format.

In one embodiment, the transmitting the second data to the second wireless terminal 70 based on the transmitting unit 1022 in the first WiFi module 102 includes:

monitoring a wireless air interface state, and feeding back the air interface state to the first encoding unit 1021, so that the first encoding unit 1021 determines an encoding code rate according to the air interface state.

Here, based on the wireless link established between the sending unit 1022 in the first WiFi module 102 and the receiving unit 7011 of the second wireless terminal 70, sending of the second data can be achieved, and an air interface state of the wireless link can affect transmission efficiency of the second data, for example, when the air interface interference is large, the transmission state of the wireless link is poor, so that the second data packet is stacked, and a large transmission delay is caused. If the air interface state of the wireless link is in a good state, the first encoding unit 1021 can increase the encoding code rate of the first data, so that the sending efficiency of the audio and video data can be improved. In practical applications, the feedback time of the air interface state of the wireless link of the sending unit 1022 needs to determine the corresponding feedback time according to the current transmission rate, for example, the current transmission rate is 30fps, and the corresponding feedback time is 100-200ms. In practical applications, the resolution in the first data encoding process may also be adjusted in real time according to the air interface state of the wireless link, for example, if the air interface state of the wireless link is not good, the resolution in the first data encoding process may be adjusted to be low, the data amount of the correspondingly generated second data is reduced, and it is beneficial to improve the sending efficiency of the second data under the condition that the air interface state of the wireless link is not good.

In an embodiment, the encoding the first data based on the first encoding unit 1021 in the first WiFi module 102 and generating second data further includes:

Here, in order to further improve the transmission efficiency and the processing efficiency of the audio/video data, each frame of data in the first data may be divided into at least two slice data, and the first encoding unit 1021 may perform encoding processing on the first data of a single slice data in units of a single slice data, so as to reduce the data amount that the first encoding unit 1021 needs to process for a single encoding. In practical applications, after the first encoding unit 1021 generates the second data of the single slice data, the transmitting unit 1022 can transmit the second data of the single slice data to the second wireless terminal 70, so that the transmitting efficiency of the audio/video data can be improved, and particularly, the transmitting delay of the audio/video data can be reduced when the state of the air interface is in a bad state. In practical application, whether each frame of data in the first data is divided into at least two slice data or not can be determined according to the air interface state, and when the air interface state is in a good condition, the data volume of audio/video data transmission should be increased, so that the first data can be directly encoded by the first encoding unit 1021 in the process of encoding the first data, and the first data of each frame does not need to be further divided into a plurality of slice data, so that the single data processing volume of the first encoding unit 1021 is increased while the audio/video data is ensured to be successfully transmitted.

An embodiment of the present application further provides a method for receiving audio and video data in a second wireless terminal, where the second wireless terminal 70 includes a second WiFi module 701 and a playing module 702, and the method is shown in fig. 9 and includes:

s901: the second data is received based on the wireless link established between the receiving unit 7011 of the second WiFi module 701 and the transmitting unit 1022 of the first WiFi module 102.

In this embodiment of the application, as shown in fig. 7, in the process of transmitting audio and video data, a receiving unit 7011 of the second WiFi module 701 establishes a wireless link with a transmitting unit 1022 of the first WiFi module 102, the second data is transmitted on the established wireless link, the second data is transmitted from the transmitting unit 1022, and is received by the receiving unit 7011, so that the second data is received.

S902: the decoding unit 7012 based on the second WiFi module 701 decodes the second data to obtain third data.

Here, the received second data is encoded by the original audio/video data, and therefore the corresponding audio/video playing can be realized only by decoding the second data, where the decoding of the second data is completed by the decoding unit 7012 on the second WiFi module 701, and in practical application, through electrical connection, the receiving unit 7011 can transmit the received second data to the decoding unit 7012 of the second WiFi module 701, so that the second WiFi module 701 can be responsible for receiving and decoding the second data.

In an embodiment, the decoding unit 7012 in the second WiFi module 701 decodes the second data to obtain third data, including: if it is detected that the second data is encoded normal data, the decoding unit 7012 is controlled to decode the second data.

Here, the second data may have data errors, missing frames, and erroneous frames during the encoding generation process, and before decoding the second data, it may be further determined whether the second data is encoded normal data, which can ensure that the third data decoded by the decoding unit 7012 is complete data, and does not have a pause during the playing process, thereby reducing the display delay at the receiving end of the audio/video data. In practical applications, the second WiFi module 701 may detect whether the second data is encoded normal data during the process of receiving the second data.

In an embodiment, the decoding unit 7012 based on the second WiFi module 701 decodes the second data to obtain third data, which further includes: if it is detected that the second data is abnormal data, the receiving unit 7011 is controlled to send abnormal data feedback to the sending unit 1022, so that the first WiFi module 102 encodes the first data again and sends the second data.

Here, if it is detected that the second data is abnormal data, for example, the second data has a frame drop or an error frame, and a data error occurs in the decoded third data, which may cause a situation that the playing cannot be performed or the playing is blocked during the playing process, in order to ensure the normal playing of the video data, the second data of the normal data needs to be obtained, and therefore, the data abnormality needs to be fed back to the first wireless terminal 10, because the communication is established between the first wireless terminal 10 and the second wireless terminal 70 through the sending unit 1022 and the receiving unit 7011, the receiving unit 7011 may be controlled to feed back the situation that the data of the second data is abnormal to the sending unit 1022 of the first wireless terminal 10, so that the first WiFi module 102 of the first wireless terminal 10 recodes the corresponding first data and sends the second data to the second wireless terminal 70 again, and the second data received by the second wireless terminal 70 is kept as the coded data, thereby avoiding a situation that the playing cannot be performed or the playing is blocked during the normal audio/video playing.

S903: according to the third data, the playing module 702 plays the corresponding audio and video.

Here, the second WiFi module 701 is electrically connected to the playing module 702, so that the decoding unit 7012 transmits the third data obtained by decoding the second data to the playing module 702, and the playing module 702 can play the audio and video corresponding to the third data. In the embodiment of the present application, the second WiFi module 701 of the second wireless terminal 70 can implement receiving and encoding of the second data, and the playing module 702 can implement playing of the audio/video data, so that a hardware transmission path of the audio/video data in the second wireless terminal 70 is reduced, an internal time delay of the audio/video data in the second wireless terminal 70 is reduced, and thus transmission efficiency of the audio/video data is improved.

In an embodiment, the decoding unit 7012 in the second WiFi module 701 decodes the second data to obtain third data, further including:

and determining the resolution corresponding to the second data.

Here, in the process of decoding the second data by decoding section 7012 of second WiFi module 701, the resolution corresponding to the second data can be generated, where the resolution corresponding to the second data is an encoding parameter used when first wireless terminal 10 encodes the first data.

The playing of the corresponding audio and video by the playing module 702 includes: and playing the audio data corresponding to the third data according to the resolution.

Here, in the process of playing the corresponding audio and video according to the third data, the playing module 702 plays the corresponding audio and video according to the resolution corresponding to the second data, and when the resolution corresponding to the second data changes, the resolution of the played audio and video data also changes correspondingly, so that the resolution can be adjusted in real time in the process of playing the audio and video data. In practical application, when the resolution of some frames is reduced, the visual experience of a user is not greatly influenced, and the audio and video is displayed timely and smoothly, so that the display time delay of the audio and video is reduced, and the playing effect of the audio and video is improved.

The embodiment of the present application further provides an audio/video data transmission system 100, as shown in fig. 10, which includes a first wireless terminal 10 and a second wireless terminal 70, wherein,

the first wireless terminal 10 comprises an acquisition module 101 and a first WiFi module 102, wherein the acquisition module 101 is used for acquiring first data, and the first data is audio and video original data; the first WiFi module 102 includes a first encoding unit 1021 and a sending unit 1022, where the first encoding unit 1021 is configured to perform encoding processing on the first data and generate second data; the sending unit 1022 is configured to send the second data to the second wireless terminal 70; and (c) a second step of,

the second wireless terminal 70 is configured to receive audio and video data, and includes a second WiFi module 701 and a playing module 702; the second WiFi module 701 includes a receiving unit 7011 and a decoding unit 7012, where the receiving unit 7011 is configured to receive the second data; the decoding unit 7012 is configured to decode the second data to obtain third data, and the playing module 702 is configured to play an audio/video corresponding to the third data.

In an embodiment, as shown in fig. 11, the first wireless terminal 10 in the audio/video data transmission system 100 further includes:

a processor 201, configured to, if the processor 201 determines that the encoding capability of the first encoding unit 1021 meets the encoding requirement of the first data, start the first encoding unit 1021 to perform encoding processing on the first data, and generate the second data.

In an example, the sending unit 1022 is configured to establish a radio link with the receiving unit 7011 in the second wireless terminal 70 to transmit the second data, and the sending unit 1022 is further configured to monitor an air interface state of the radio link, and feed the air interface state back to the first encoding unit 1021, so that the first encoding unit 1021 determines an encoding code rate according to the air interface state.

In an example, the first encoding unit 1021 is further configured to divide each frame of data in the first data into at least two pieces of slice data according to the air interface state; and encoding the first data to generate the second data in units of the single slice data.

In an example, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102 respectively.

In an example, the acquisition module 101, the processor 201 and the first WiFi module 102 are electrically connected in sequence; the processor 201 comprises a second encoding unit 2011; the second encoding unit 2011 is configured to encode the first data if the processor 201 determines that the second encoding unit 2011 encodes the first data.

In an example, the second WiFi module 701 is further configured to control the decoding unit 7012 to decode the second data if it is detected that the second data is encoded normal data.

In an example, the decoding unit 7012 is also configured to determine a resolution to which the second data corresponds; the playing module 702 is further configured to play the audio and video corresponding to the third data according to the resolution.

In an example, the second WiFi module 701 is further configured to control the receiving unit 7011 to send data exception feedback to the sending unit 1022, so that the first WiFi module 102 encodes the first data again and sends the second data, if it is detected that the second data is encoding exception data.

In an exemplary embodiment, the present application further provides a storage medium, that is, a computer storage medium, specifically, a computer readable storage medium, for example, including the first memory 1003 storing a computer program, where the computer program is executable by the processor 1002 of the terminal to complete the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, or CD-ROM.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, terminal and method may be implemented in other manners. The above-described device embodiments are merely illustrative, for example, the division of the unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or in other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method in a first wireless terminal for transmitting audio-video data, the first wireless terminal comprising an acquisition module and a first WiFi module, wherein the method comprises:

acquiring first data based on the acquisition module; the first data is audio and video original data;

2. The method of claim 1, wherein the encoding the first data based on the first encoding unit in the first WIFI module and generating second data comprises:

3. The method of claim 2, wherein the transmitting the second data to a second wireless terminal based on a transmitting unit in the first WiFi module comprises:

4. The method of claim 3, wherein the encoding the first data based on the first encoding unit in the first WiFi module and generating second data, further comprises:

5. A method for receiving audio and video data in a second wireless terminal, the second wireless terminal comprising a second WiFi module and a play module, the method comprising:

6. The method of claim 5, wherein decoding the second data based on a decoding unit in the second WiFi module to obtain third data comprises: and if the second data are detected to be encoding normal data, controlling the decoding unit to decode the second data.

7. The method of claim 6, wherein decoding the second data based on a decoding unit in the second WiFi module to obtain third data, further comprises: determining a resolution corresponding to the second data;

8. The method of claim 5, wherein the second WiFi module-based decoding unit decodes the second data to obtain third data, further comprising: and if the second data are detected to be encoding abnormal data, controlling the receiving unit to send data abnormal feedback to the sending unit so that the first WiFi module encodes the first data again and sends the second data.

9. A first wireless terminal for transmitting audio/video data, comprising:

10. The first wireless terminal of claim 9, further comprising:

11. The first wireless terminal according to claim 9, wherein the sending unit is configured to establish a wireless link with a receiving unit in the second wireless terminal to transmit the second data, and the sending unit is further configured to monitor an air interface state of the wireless link, and feed back the air interface state to the first encoding unit, so that the first encoding unit determines an encoding code rate according to the air interface state.

12. The first wireless terminal of claim 10, wherein the first encoding unit is further configured to divide each frame of data in the first data into at least two pieces of slice data according to the air interface status; and encoding the first data in units of the single slice data to generate the second data.

13. The first wireless terminal of claim 10, wherein:

the processor and the acquisition module are respectively electrically connected with the first WiFi module.

14. The first wireless terminal of claim 10, comprising:

the acquisition module, the processor and the first WiFi module are electrically connected in sequence; the processor includes a second encoding unit; the second encoding unit is configured to encode the first data if the processor determines that the second encoding unit encodes the first data.

15. A second wireless terminal for receiving audio and video data, comprising:

16. The second wireless terminal of claim 15, wherein the second WiFi module is further configured to control the decoding unit to decode the second data if the second data is detected to be encoded normal data.

17. The second wireless terminal of claim 16, wherein the decoding unit is further configured to determine a resolution corresponding to the second data; the playing module is further used for playing the audio and video corresponding to the third data according to the resolution.

18. The second wireless terminal of claim 15, wherein the second WiFi module is further configured to control the receiving unit to send data anomaly feedback to the sending unit if it is detected that the second data is coding anomaly data, so that the first WiFi module re-codes the first data and sends the second data.

19. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data transmission method according to any one of claims 1 to 8.

20. An audio and video data transmission system is characterized by comprising a first wireless terminal and a second wireless terminal,