CN111865766A

CN111865766A - Interactive method, medium, equipment and system based on audio-video transmission

Info

Publication number: CN111865766A
Application number: CN202010700187.1A
Authority: CN
Inventors: 应臻恺; 徐婷婷
Original assignee: Shanghai Pateo Electronic Equipment Manufacturing Co Ltd
Current assignee: Pateo Connect and Technology Shanghai Corp
Priority date: 2020-07-20
Filing date: 2020-07-20
Publication date: 2020-10-30
Anticipated expiration: 2040-07-20
Also published as: CN111865766B

Abstract

The invention provides an interaction method, medium, equipment and system based on audio-video transmission, wherein the interaction method based on the audio-video transmission comprises the following steps: acquiring interaction information corresponding to the audio and video data, and identifying; determining a comment category to which the interactive information belongs according to the recognition result; searching special effect information consistent with the comment types in a preset effect material library; and overlapping the interaction information and the special effect information to generate feedback information corresponding to the audio and video data. The invention provides a method for carrying out visualization interaction on shared audio and video data by two parties at different places, which effectively realizes emotional communication between the two parties at different places.

Description

Interactive method, medium, equipment and system based on audio-video transmission

Technical Field

The invention belongs to the technical field of audio-video identification, relates to an interaction method based on audio-video identification, and particularly relates to an interaction method, medium, equipment and system based on audio-video transmission.

Background

Along with the development of new economy, the internet gradually becomes the world leading factor, and in the short decades, the network becomes more and more perfect, which draws the distance between people, changes the earth into a real village and leads more people to realize the convenience of 'long distance between people and one line of situation'. The development of networks has also promoted the change of communication means, and the traditional communication mode can not meet the requirements of people. From E-mai to smart phones, the network has brought the promotion of communication speed, has reduced communication cost, has more changed the quality and the form of traditional network communication, makes the exchange no longer just restrict ordinary language characters, utilizes the video to make each other's friend of every side can meet each other.

At present, voice interaction and video interaction can be performed between users through various social software, and most of the social software depends on common equipment such as mobile phones and computers, so that the equipment on which the interaction between the users depends is single, and further, the communication form between the users is single.

In the prior art, voice interaction and video interaction between users can only simply restore feedback information of the other party, but can not process the feedback information, so that the feedback effect of the other party is enhanced.

Therefore, how to provide an interaction method, medium, device and system based on audio-video transmission to solve the defects that the prior art cannot further identify the interaction information of both users and enhance the interaction effect, and the like, becomes a technical problem to be solved by those skilled in the art.

Disclosure of Invention

In view of the above drawbacks of the prior art, an object of the present invention is to provide an interactive method, medium, device and system based on audio-video transmission, which are used to solve the problem that the prior art cannot further identify and enhance the interactive effect of the interactive information between the two users.

In order to achieve the above and other related objects, an aspect of the present invention provides an interactive method based on audio-video transmission, including: acquiring interaction information corresponding to the audio and video data, and identifying; determining a comment category to which the interactive information belongs according to the recognition result; searching special effect information consistent with the comment types in a preset effect material library; and overlapping the interaction information and the special effect information to generate feedback information corresponding to the audio and video data.

In an embodiment of the present invention, the interactive information includes voice interactive information and/or image interactive information; the step of obtaining the interaction information corresponding to the audio and video data and identifying comprises the following steps: performing semantic recognition on the voice interaction information, and determining comment mood and comment mood of a user; and/or performing image recognition on the image interaction information, and determining the comment expression and comment action of the user.

In an embodiment of the present invention, the step of determining the comment type to which the interactive information belongs according to the recognition result includes: dividing comment attributes of one or more of the comment tone, the comment mood, the comment expression and the comment action; and performing deduplication optimization processing on the comment attributes to take different comment attributes as the comment categories.

In an embodiment of the present invention, the step of searching for special effect information consistent with the comment category in a preset effect material library includes: calling sound effect information consistent with the comment types from the preset effect material library; and/or calling animation information consistent with the comment types from the preset effect material library; and generating the special effect information according to at least one of the sound effect information and the animation information.

In an embodiment of the present invention, after the step of superimposing the interaction information and the special effect information to generate the feedback information corresponding to the audio-video data, the interaction method based on audio-video transmission further includes: and sending one of the interaction information, the special effect information and the feedback information corresponding to the audio-video data.

In an embodiment of the present invention, the interactive method based on audio/video transmission further includes: generating a plurality of feedback information according to the interaction information and the special effect information from different sources; and generating bullet screen information according to the plurality of feedback information in a time sequence and sending the bullet screen information so that an initiator of the audio-video data can browse feedback conditions of a plurality of people.

Another aspect of the present invention provides a medium, on which a computer program is stored, which, when being executed by a processor, implements the interactive method based on audio-video transmission.

Yet another aspect of the invention provides an apparatus comprising: a processor and a memory; the memory is used for storing computer programs, and the processor is used for executing the computer programs stored by the memory so as to enable the equipment to execute the interactive method based on audio-video transmission.

In a final aspect of the present invention, an interactive system based on audio-video transmission is provided, where the interactive system based on audio-video transmission includes: the vehicle-mounted terminal is used for sending audio-video data, receiving information corresponding to the audio-video data or receiving the audio-video data and sending the information corresponding to the audio-video data; the information is one of interaction information, special effect information and feedback information; the client is used for receiving audio-video data, sending the information corresponding to the audio-video data or sending the audio-video data, and receiving the information corresponding to the audio-video data; the server is in communication connection with the vehicle end and the mobile client respectively and is used for transmitting audio-video data, acquiring the interaction information corresponding to the audio-video data and identifying the interaction information; determining a comment category to which the interactive information belongs according to the recognition result; searching the special effect information consistent with the comment type in a preset effect material library; and overlapping the interaction information and the special effect information to generate the feedback information corresponding to the audio and video data.

In an embodiment of the present invention, the client is an intelligent device in a home: the intelligent equipment is integrated equipment combining a television, an external camera and an intelligent sound box and is used for receiving audio and video data, sending the information corresponding to the audio and video data or sending the audio and video data and receiving the information corresponding to the audio and video data.

As described above, the interactive method, medium, device and system based on audio-video transmission according to the present invention have the following advantages:

the invention breaks through the limitation of single equipment such as mobile phones, computers and the like, and can more conveniently carry out voice interaction and video interaction through various network equipment which can be applied by both parties at different places. The invention also provides a method for carrying out visualization interaction on the shared audio and video data by the two parties at different places, further identifies the interaction information of the two parties of the user and enhances the interaction effect, effectively realizes the emotional communication between the two parties at different places, and improves the equipment experience during the interaction between the users.

Drawings

Fig. 1 is a schematic flow chart illustrating an interactive method based on audio/video transmission according to an embodiment of the present invention.

Fig. 2 is a flowchart illustrating an interactive information recognition method based on audio/video transmission according to an embodiment of the present invention.

Fig. 3 is a schematic view illustrating a comment category flow of the interaction method based on audio-video transmission according to an embodiment of the present invention.

Fig. 4 is a schematic diagram illustrating feedback information generation of an interaction method based on audio-video transmission according to an embodiment of the invention.

Fig. 5 is a schematic structural diagram of a vehicle end of the present invention in an embodiment.

Fig. 6 is a schematic structural diagram of a client according to an embodiment of the invention.

Fig. 7 is a schematic structural diagram of a server according to an embodiment of the invention.

FIG. 8 is a schematic diagram of audio/video data transmission of a vehicle owner in an embodiment of the interactive system based on audio/video transmission according to the present invention.

Fig. 9 is a schematic diagram illustrating audio-video data transmission in a home according to an embodiment of the interactive system based on audio-video transmission of the present invention.

FIG. 10 is a schematic diagram of an interactive system based on audio/video transmission according to an embodiment of the present invention.

Description of the element reference numerals

5 vehicle end

6 client

7 service end

71 processor

72 memory

73 communication interface

74 system bus

S11-S14

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

The interaction method based on audio-video transmission provides a method for carrying out visualization interaction on shared audio-video data by both sides in different places, and effectively realizes emotional communication between both sides in different places.

The principle and implementation of an interactive method, medium, device and system based on audio-video transmission according to the present embodiment will be described in detail below with reference to fig. 1 to 10, so that those skilled in the art can understand the interactive method, medium, device and system based on audio-video transmission without creative work.

Please refer to fig. 1, which is a schematic flowchart illustrating an interactive method based on audio/video transmission according to an embodiment of the present invention. As shown in fig. 1, the interaction method based on audio-video transmission specifically includes the following steps:

And S11, acquiring the interaction information corresponding to the audio-video data and identifying.

In the present embodiment, the audio/video data includes at least one of voice data, picture data, and video data. The interactive information comprises voice interactive information and/or image interactive information.

Specifically, the voice interaction information is "Wow, Tai excellent! ", the image interaction information includes gesture information and face information. The gesture information includes: an OK gesture where the thumb bends with the index finger, a thumbs up gesture, a barycentric gesture where the thumb crosses with the index finger, etc. The facial information includes facial expression information and facial action information of face, wherein, facial expression information includes expression such as joy, anger, sadi etc. facial action information includes puckered mouth parent's action.

Please refer to fig. 2, which is a flowchart illustrating an interactive information recognition method according to an embodiment of the present invention. As shown in fig. 2, S11 includes:

(1) and performing semantic recognition on the voice interaction information, and determining the comment mood and comment mood of the user. And/or

Specifically, for the voice interaction information "Wow, Taiwan! And performing semantic recognition, judging that the comment mood of the user is happy, and the comment mood is praise.

(2) And carrying out image recognition on the image interaction information, and determining the comment expression and comment action of the user.

Specifically, image recognition is carried out on the action of puckering the mouth parent of the face action information, and then the comment expression of the user is judged to be liked; and (4) carrying out image recognition on the heart-comparing gesture of the intersection of the thumb and the index finger, judging the comment action of the user as the heart-comparing gesture, and expressing the preference of the user.

And S12, determining the comment type to which the interactive information belongs according to the recognition result.

Please refer to fig. 3, which is a schematic view illustrating a comment category flow of the interaction method based on audio/video transmission according to an embodiment of the present invention. As shown in fig. 3, S12 includes:

(1) and dividing comment attributes of one or more of the comment tone, the comment mood, the comment expression and the comment action.

Specifically, the comment attribute is like and likes in combination with the user's comment mood being happy, the comment mood being like, and the user's comment expression being like and the comment action being more than heart.

(2) And performing deduplication optimization processing on the comment attributes to take different comment attributes as the comment categories.

Specifically, two comment attributes of like and like are taken as comment categories. The different comment attributes are judged by taking different special effect information contents as reference, and the different comment attributes can be expressed by different special effect information contents, namely, the comment attributes are different from each other, for example, special effect information which can be used for love is liked, special effect information which can be liked for love is liked, and two comment attributes are different from each other.

And S13, searching special effect information consistent with the comment types in a preset effect material library.

In the present embodiment, S13 includes:

(1) and calling sound effect information consistent with the comment types from the preset effect material library. The sound effect information includes drumbeats, whistles, melody or background sounds, and sound effect information included in programs similar to KTV singing. And/or

Specifically, sound effect information (clapper sound) in accordance with the comment category "like" is called in the preset effect material library.

(2) And calling animation information consistent with the comment types from the preset effect material library. The animation information comprises animation information such as flower sprinklers, stars, love hearts and emoticons similar to emoticons contained in various social software.

Specifically, animation information (love heart) in accordance with the comment category "like" is called in the preset effects material library.

(3) And generating the special effect information according to at least one of the sound effect information and the animation information.

And S14, overlapping the interaction information and the special effect information to generate feedback information corresponding to the audio and video data.

Please refer to fig. 4, which is a schematic diagram illustrating feedback information generation of an interaction method based on audio/video transmission according to an embodiment of the present invention. As shown in fig. 4, sound effect information and animation information are determined in the effect material library, and are combined to generate special effect information, and finally, the interaction information and the special effect information are superimposed to generate feedback information.

In this embodiment, the interactive method based on audio-video transmission further includes: and sending one of the interaction information, the special effect information and the feedback information corresponding to the audio-video data.

In this embodiment, the interactive method based on audio-video transmission further includes: generating a plurality of feedback information according to the interaction information and the special effect information from different sources; and generating bullet screen information according to the plurality of feedback information in a time sequence and sending the bullet screen information so that an initiator of the audio-video data can browse feedback conditions of a plurality of people.

The protection scope of the interactive method based on audio and video transmission of the present invention is not limited to the execution sequence of the steps listed in this embodiment, and all the schemes of adding, subtracting, and replacing steps in the prior art according to the principle of the present invention are included in the protection scope of the present invention.

The present embodiment provides a computer storage medium having a computer program stored thereon, which when executed by a processor implements the audio/video transmission-based interactive method.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned computer-readable storage media comprise: various computer storage media that can store program codes, such as ROM, RAM, magnetic or optical disks.

The device of the invention comprises: a processor and a memory; the memory is used for storing computer programs, and the processor is used for executing the computer programs stored by the memory so as to enable the equipment to execute the interactive method based on audio-video transmission. Specifically, an interactive system based on audio-video transmission at least comprises a first device and a second device which are respectively used by two users. The apparatus has voice input and output means and video input and output means. The device is preloaded with a social software application program capable of voice interaction or video interaction or an application program capable of recognizing voice interaction information or video interaction information and processing the interaction information. The device can be a vehicle end, a client end, a server end or other electronic devices which can be used for executing the interactive method based on audio-video transmission and a combination of various electronic devices for executing the interactive method based on audio-video transmission.

Further, the special effect information may be automatically generated after the device analyzes and identifies the interaction information, or may be selected by a user through manual clicking. The equipment presents a voice and video interactive interface based on audio-video data browsing to a user, and a touch key for the user to select special effect information is arranged in the interactive interface, so that the user selects the special effect information to send after manually clicking the corresponding special effect information key.

Please refer to fig. 5, which is a schematic structural diagram of a vehicle end of the present invention in an embodiment. As shown in fig. 5, the apparatus is a vehicle end, including: a processor and a memory. The memory is used for storing computer programs, and the processor is used for executing the computer programs stored in the memory, so that the vehicle end executes the interaction method based on audio-video transmission.

Please refer to fig. 6, which is a schematic structural diagram of a client according to an embodiment of the present invention. As shown in fig. 6, the device is a client, and includes: a processor and a memory. The memory is used for storing computer programs, and the processor is used for executing the computer programs stored by the memory so as to enable the client to execute the interactive method based on audio-video transmission. The client comprises a desktop computer, a notebook computer, a tablet computer, a smart phone, a smart television, a Personal Digital Assistant (PDA for short) and the like, and further comprises a smart sound box or other internet of things equipment with voice and video functions for smart homes.

Please refer to fig. 7, which is a schematic structural diagram of a server according to an embodiment of the present invention. As shown in fig. 7, the present embodiment provides a server 7, where the server 7 includes: a processor 71, memory 72, communication interface 73, or/and system bus 74; the memory 72 and the communication interface 73 are connected to the processor 71 through the system bus 74 and perform communication with each other, the memory 72 is used for storing computer programs, the communication interface 73 is used for communicating with other devices, and the processor 71 is used for running the computer programs, so that the server 7 executes the steps of the interactive method based on audio-video transmission. It should be noted that the server may be arranged on one or more entity servers according to various factors such as functions, loads, and the like, or may be formed by a distributed or centralized server cluster.

The above-mentioned system bus 74 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. The communication interface 73 is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). The memory 72 may include a Random Access Memory (RAM), and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory.

The processor 71 may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the integrated circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

In an embodiment, the interactive system based on audio-video transmission of the present invention includes: the system comprises a vehicle end, a client and a server.

The vehicle terminal is used for sending audio-video data, receiving information corresponding to the audio-video data or receiving audio-video data and sending the information corresponding to the audio-video data; the information is one of interaction information, special effect information and feedback information.

The client is used for receiving audio-video data, sending the information corresponding to the audio-video data or sending the audio-video data, and receiving the information corresponding to the audio-video data.

The server is in communication connection with the vehicle end and the mobile client respectively and is used for transmitting audio-video data, acquiring the interaction information corresponding to the audio-video data and identifying the interaction information; determining a comment category to which the interactive information belongs according to the recognition result; searching the special effect information consistent with the comment type in a preset effect material library; and overlapping the interaction information and the special effect information to generate the feedback information corresponding to the audio and video data.

In this embodiment, the client is an intelligent device in a home.

The intelligent equipment is integrated equipment combining a television, an external camera and an intelligent sound box and is used for receiving audio and video data, sending the information corresponding to the audio and video data or sending the audio and video data and receiving the information corresponding to the audio and video data.

In embodiment 1, please refer to fig. 8, which shows a schematic diagram of audio/video data transmission of a car owner in an embodiment of the interactive system based on audio/video transmission according to the present invention. As shown in fig. 8, taking the car owner receiving the inquiry information of the family in the car as an example, the specific implementation process of the audio-video interaction includes:

(1) the family of the owner sends out the voice information of home through the internet of things equipment at home? When to get home? ", the voice information in home is transmitted to the vehicle end through the service end and is listened by the vehicle owner.

(2) The car owner takes the picture or short video in the car or takes the picture or short video of the environment outside the car as the audio and video data of the car owner through the car terminal or the mobile phone terminal of the car owner. The pictures or short videos in the automobile comprise self-shooting of the automobile owner or shooting information of the automobile owner and other people in the automobile, and the environment outside the automobile comprises beautiful scenery along the way, driving road conditions and the like. The specific implementation mode can be that the car owner terminal combines an external camera to acquire photos or short videos, or the car owner mobile phone acquires the photos or the short videos and then synchronizes the photos or the short videos to the car owner terminal, then the car owner audio-video data are sent to the server terminal through the car owner mobile phone, and the car owner audio-video data are forwarded to the home Internet of things equipment through the server terminal.

(3) And the family of the vehicle owner receives the audio and video data of the vehicle owner through the home Internet of things equipment. The household Internet of things equipment comprises an intelligent television and an intelligent sound box with a screen display.

(4) The receiving situation is divided into a situation that the home smart television is turned on and a situation that the home smart television is not turned on. When the intelligent television is opened, the intelligent television automatically pops up a reminding message, and the family of the vehicle owner checks the audio and video data of the vehicle owner through the television remote controller; when the intelligent sound box is not opened, the intelligent sound box reminds the family of the car owner to check the audio and video data of the car owner through voice. For example, the speech message is "the audio-video clip sharing from Shanghai F HU820 (license plate number) or owner, now view? ".

(5) After the family member confirms by opening the television, the family member browses the audio and video data of the car owner or directly browses the audio and video data through the intelligent sound box with the screen display and feeds back the audio and video data. Specifically, the family member presses a certain key on the remote controller for interactive feedback or presses a screen picture of the smart sound box for interactive feedback.

For beautiful scenery along the way, family members feed back voice interaction information' WoW! Am o! "family member feeds back image interaction information" like a praise and heart; aiming at the unobstructed road condition, the family feeds back the voice interactive information' Wa! The road condition is really good! "family feeds back the image interaction information" OK gesture "; for group photo of the owner and other people in the car, the family member feeds back the voice interaction information "WoW! And (4) the user can feel happy, and the family feeds back image interaction information of praise and love.

(6) Family members may also add voice interaction information in reply, such as "know la! Fun of play! ".

(7) And after the server identifies the interactive information, generating feedback information by combining the superposition of the special effect information, and feeding the feedback information back to the vehicle end. The vehicle terminal reminds the vehicle owner to check the feedback information through voice. For example, "do you echo at home, see now? ".

(8) The owner attaches special effect information when viewing the feedback information.

In embodiment 2, please refer to fig. 9, which is a schematic diagram illustrating audio-video data transmission in a home in an embodiment of an interactive system based on audio-video transmission according to the present invention. As shown in fig. 9, taking the example that the owner of the vehicle receives the inquiry information sent by the vehicle owner at home, the specific implementation process of the audio-video interaction includes:

(1) when a car owner drives a car, a voice message is sent to the internet of things equipment at home, which is what a mother does and eats today? Is the baby mani? ".

(2) The family owner shoots the family audio-video data of the corresponding scene that the owner wants to see through the external camera of smart television, the intelligent sound box camera or the cell-phone WeChat, and the family audio-video data of this family sends to the car end through the server. For example, a photograph of a food at home, a video of a baby sleeping or playing, a photograph of a parent teasing a baby.

(3) And in consideration of safety, if the car owner is reminded to check the car after stopping in the driving process, or the driving state of the car machine is automatically detected, and the home audio-video data is automatically locked in the driving process.

Specifically, after the car owner parks, the car owner inputs a viewing instruction by long pressing a steering wheel key or directly sends a voice viewing instruction. Aiming at the food photo at home, the car owner sends out voice interactive information' WoW! The food is really delicious! "mom's skill is simply too attractive", "good want to eat one bite at once", the owner sends out the image interaction information "the gesture of like a praise and heart"; aiming at the sound sleeping and playing video of the baby, the car owner sends out voice interactive information' Haoxian TA! "really want to be parent", the car owner sends the image interaction information "gesture of heart comparing" and "action of puckering the face and making the mouth parent".

(4) The service end analyzes the interaction information of the vehicle owner, superposes the special effect information and feeds back the interaction information to home Internet of things equipment, and the home receives the feedback information of the vehicle owner through home intelligent televisions, intelligent sound boxes and other Internet of things equipment.

In embodiment 3, please refer to fig. 10, which is a schematic structural diagram of an interactive system based on audio/video transmission according to an embodiment of the present invention. As shown in fig. 10, the interactive system based on audio-video transmission of the present invention may be a one-to-one interaction or a one-to-many interaction. In fig. 10, one vehicle-mounted terminal shares audio/video data and distributes the audio/video data to a plurality of clients through a server. Taking a concrete application scenario of marriage as an example, the vehicle end is a wedding car, and the client is a device used by relatives and friends of men. When a man goes to a woman and connects a bride at home, everyone is compelled to see the appearance of the bride, the bride connects the bride to a wedding car, the voice, the picture or the video of the bride and the bride are shot through the car end of the bride and are sent to devices used by relatives and friends of a plurality of brides as audio and video data, and comment information of each relatives and friends is displayed in a bullet screen mode through a screen at the car end, for example, special effects of 'the bride is beautiful and beautiful', 'the bride is happy', a flower animation, a drumbeat sound, a love heart and the like are displayed.

In combination with the interactive system based on audio-video transmission shown in fig. 10, other embodiments further include that, when the user is driving to go home or is far away from home and cannot go home for a year, the user sends a new year blessing to multiple families and shares audio-video data of the location of the user through the vehicle terminal or other devices of the user, and receives feedback information of the multiple families.

In summary, the interactive method, medium, device and system based on audio-video transmission of the present invention break through the limitation of single devices such as mobile phones and computers, and perform audio interaction and video interaction more conveniently through multiple network devices applicable by both parties at different places. The invention also provides a method for carrying out visualization interaction on the shared audio and video data by the two parties at different places, further identifies the interaction information of the two parties of the user and enhances the interaction effect, effectively realizes the emotional communication between the two parties at different places, and improves the equipment experience during the interaction between the users. The invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. An interactive method based on audio-video transmission is characterized in that the interactive method based on audio-video transmission comprises the following steps:

acquiring interaction information corresponding to the audio and video data, and identifying;

determining a comment category to which the interactive information belongs according to the recognition result;

searching special effect information consistent with the comment types in a preset effect material library;

and overlapping the interaction information and the special effect information to generate feedback information corresponding to the audio and video data.

2. The audio-visual transmission-based interaction method according to claim 1, wherein the interaction information includes voice interaction information and/or image interaction information; the step of obtaining the interaction information corresponding to the audio and video data and identifying comprises the following steps:

performing semantic recognition on the voice interaction information, and determining comment mood and comment mood of a user; and/or

And carrying out image recognition on the image interaction information, and determining the comment expression and comment action of the user.

3. The audio-visual transmission-based interaction method according to claim 2, wherein the step of determining the comment category to which the interaction information belongs according to the recognition result includes:

dividing comment attributes of one or more of the comment tone, the comment mood, the comment expression and the comment action;

And performing deduplication optimization processing on the comment attributes to take different comment attributes as the comment categories.

4. The audio-visual transmission-based interaction method according to claim 1, wherein the step of searching for special effect information consistent with the comment category in a preset effect material library comprises:

calling sound effect information consistent with the comment types from the preset effect material library; and/or

Calling animation information consistent with the comment type from the preset effect material library;

and generating the special effect information according to at least one of the sound effect information and the animation information.

5. The audio-visual transmission-based interaction method according to claim 1, wherein after the step of superimposing the interaction information and the special effect information to generate the feedback information corresponding to the audio-visual data, the audio-visual transmission-based interaction method further comprises:

and sending one of the interaction information, the special effect information and the feedback information corresponding to the audio-video data.

6. The audio-visual transmission-based interaction method according to claim 1, further comprising:

Generating a plurality of feedback information according to the interaction information and the special effect information from different sources;

and generating bullet screen information according to the plurality of feedback information in a time sequence and sending the bullet screen information so that an initiator of the audio-video data can browse feedback conditions of a plurality of people.

7. A medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the audio/video transmission-based interaction method according to any one of claims 1 to 6.

8. An apparatus, comprising: a processor and a memory;

the memory is used for storing a computer program, and the processor is used for executing the computer program stored by the memory to make the device execute the interactive method based on audio-video transmission according to any one of claims 1 to 6.

9. An interactive system based on audio-video transmission, which is characterized in that the interactive system based on audio-video transmission comprises:

the vehicle-mounted terminal is used for sending audio-video data, receiving information corresponding to the audio-video data or receiving the audio-video data and sending the information corresponding to the audio-video data; the information is one of interaction information, special effect information and feedback information;

The client is used for receiving audio-video data, sending the information corresponding to the audio-video data or sending the audio-video data, and receiving the information corresponding to the audio-video data;

10. The interactive system based on audio-visual transmission as claimed in claim 9, wherein the client is a home intelligent device: