CN116437137A

CN116437137A - Live broadcast processing method and device, electronic equipment and storage medium

Info

Publication number: CN116437137A
Application number: CN202310686538.1A
Authority: CN
Inventors: 徐智伟; 简伟华; 王胜慧
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2023-06-09
Filing date: 2023-06-09
Publication date: 2023-07-14
Anticipated expiration: 2043-06-09
Also published as: CN116437137B

Abstract

The disclosure relates to a live broadcast processing method, a live broadcast processing device, electronic equipment and a storage medium, wherein the method comprises the steps of displaying a first live broadcast picture corresponding to at least one target object in a first window; the target object comprises any one of a host player corresponding to the first direct play account and a link object interacted with the host player link; acquiring target virtual images corresponding to at least one target object respectively; rendering in the second window to generate a second live broadcast picture; the second live broadcast picture comprises mixed virtual images, the mixed virtual images are obtained by replacing the second face area of the target virtual image with the first face area of the corresponding target object, the facial expression of each mixed virtual image is synchronous with the facial expression of the corresponding target object, and the second live broadcast picture is used for displaying a live broadcast interface between preset live broadcast rooms corresponding to the first live broadcast account. Therefore, the deviation between the expression of the live-broadcast virtual image and the expression of the host is reduced, and the accuracy of expression transfer is improved.

Description

Live broadcast processing method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of computers, and in particular relates to a live broadcast processing method, a live broadcast processing device, electronic equipment and a storage medium.

Background

With the rapid development of the live broadcasting industry, live broadcasting forms of virtual image live broadcasting are presented. The live broadcasting of the virtual image is mainly realized by replacing the real image of the host by the self-selected virtual image of the host.

However, under the conditions of faster expression change, larger expression amplitude and the like of the host, the expression change of the avatar still cannot be driven well through the expression of the host based on the conventional expression driving model, so that a certain deviation exists between the expression of the avatar presented by live broadcast and the expression of the host, and the accuracy of expression transmission is reduced.

Disclosure of Invention

The disclosure provides a live broadcast processing method, a live broadcast processing device, electronic equipment and a storage medium, so as to at least solve at least one technical problem in the related art. The technical scheme of the present disclosure is as follows:

according to a first aspect of an embodiment of the present disclosure, a live broadcast processing method is provided, applied to a first client, including:

responding to an image operation instruction triggered by a first direct-broadcasting account, and displaying a first direct-broadcasting picture corresponding to at least one target object in a first window; the target object comprises any one of a host player corresponding to the first direct-play account and a link object interacted with the host player link, and the first direct-play picture comprises first face areas respectively corresponding to the at least one target object;

Responding to a selection operation instruction of at least one virtual image, and acquiring target virtual images respectively corresponding to at least one target object;

rendering in the second window to generate a second live broadcast picture; the second live broadcast picture comprises mixed virtual images with the same number of objects as the at least one target object, the mixed virtual images are obtained by replacing second face areas of the target virtual images with first face areas of corresponding target objects, the face expressions of the mixed virtual images are synchronous with the face expressions of the target objects, and the second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account.

In an optional implementation manner, the target gesture actions of the mixed virtual images displayed in the second live broadcast picture are respectively matched with the target gesture actions of the target objects of the face areas in the mixed virtual images;

wherein the target gesture motion comprises one or more of a head gesture motion and a limb gesture motion.

In an optional embodiment, the rendering in the second window to generate the second live view includes:

Performing key point detection of a target part on the first direct broadcast picture based on a preset detection frequency, and extracting a target detection area and part key point data corresponding to the target detection area from the first direct broadcast picture in real time according to a key point detection result, wherein the target part comprises one or more of a face and a limb part, and the target detection area comprises one or more of a first face area of the target object and a first limb part area of the target object;

based on each extracted target detection area, generating a detection image sequence frame corresponding to the first direct broadcast picture; the detection image sequence frame is used for being displayed in the first window;

determining gesture motion data of the target object in each frame of the detection image sequence frame based on the position key point data; the gesture motion data is used for synchronously adjusting the gesture motion of the target virtual image;

and rendering and generating a second live broadcast picture in the second window based on the target detection area, the gesture action data and the image model data of the target virtual image.

In an optional embodiment, in a case where the target detection area includes a first face area of the target object and the gesture motion data includes head motion data, the rendering in the second window to generate a second live view based on the target detection area, the gesture motion data, and the avatar model data of the target avatar includes:

Replacing a second face area in the target avatar with a first face area of a corresponding target object to obtain an adjusted face area of the target avatar;

determining a head pose parameter of the target avatar based on the head motion data;

based on the adjusted face area and the image model data of the target virtual image, obtaining the mixed virtual image, and based on the head gesture parameter, rendering and generating a second live broadcast picture containing the mixed virtual image in the second window, wherein the frame rate of the second live broadcast picture is related to the numerical value of the preset detection frequency;

the head motion data comprise one or more of head rotation angle, head transverse offset and head longitudinal offset.

In an optional embodiment, the method for obtaining the first direct broadcast picture includes any one of the following:

the method comprises the steps of calling an image acquisition module through a third client corresponding to the target object to obtain, capturing a live broadcast picture of the target object to obtain, recording a live broadcast picture of the target object to obtain, and calling the image acquisition module through the first client to acquire.

In an alternative embodiment, the method further comprises:

acquiring the second live broadcast picture displayed in the second window through a second client; the second client is used for pushing the live broadcast content comprising the second live broadcast picture to a third client, and the third client is used for displaying the live broadcast content in the preset live broadcast room; or alternatively

And pushing the live broadcast content containing the second live broadcast picture to a third client, wherein the third client is used for displaying the live broadcast content in the preset live broadcast room.

In an optional embodiment, in a case that the preset live broadcasting room is a multi-user live broadcasting room, the preset live broadcasting room further displays the first direct broadcasting picture, and the first direct broadcasting picture and the second direct broadcasting picture are respectively displayed in different live broadcasting windows of the preset live broadcasting room.

In an optional embodiment, the display position of the second live screen in the live interface includes any one of the following:

the live broadcast system comprises a live broadcast interface, a live broadcast system and a live broadcast system, wherein the live broadcast interface comprises a live broadcast window position of a first live broadcast account in the live broadcast interface, a live broadcast window position of a second live broadcast account in the live broadcast interface and positions except for all live broadcast windows in the live broadcast interface, and the second live broadcast account is any one of the live broadcast accounts which participate in online interaction between the preset live broadcast rooms except for the first live broadcast account.

In an alternative embodiment, the target gesture of the mixed avatar is matched with the target gesture of a gesture reference object, and the gesture reference object comprises a connecting line object which participates in the connecting line interaction of the preset live broadcasting room and is different from the target object of the face area in each mixed avatar;

In an optional embodiment, the obtaining, in response to the selection operation instruction for the at least one avatar, the target avatars respectively corresponding to the at least one target object includes:

responding to the selection operation of at least one virtual image, and acquiring initial virtual images corresponding to at least one target object respectively;

if the selection operation of the special effect element is detected, acquiring the selected target special effect element;

and adding the target special effect element into the corresponding initial avatar to obtain the target avatar.

According to a second aspect of the embodiments of the present disclosure, there is provided a live broadcast processing apparatus, applied to a first client, including:

the first display module is configured to execute an image operation instruction triggered by responding to the first direct-broadcasting account, and display a first direct-broadcasting picture corresponding to at least one target object in a first window; the target object comprises any one of a host player corresponding to the first direct-play account and a link object interacted with the host player link, and the first direct-play picture comprises first face areas respectively corresponding to the at least one target object;

A first acquisition module configured to perform acquisition of target avatars respectively corresponding to at least one target object in response to a selection operation instruction for the at least one avatar;

a rendering module configured to perform rendering in a second window to generate a second live view; the second live broadcast picture comprises mixed virtual images with the same number of objects as the at least one target object, the mixed virtual images are obtained by replacing second face areas of the target virtual images with first face areas of corresponding target objects, the face expressions of the mixed virtual images are synchronous with the face expressions of the target objects, and the second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account.

In an alternative embodiment, the rendering module includes:

a detection sub-module configured to perform key point detection of a target part on the first direct broadcast picture based on a preset detection frequency, and extract a target detection area and part key point data corresponding to the target detection area in real time from the first direct broadcast picture according to a key point detection result, wherein the target part comprises one or more of a face and a limb part, and the target detection area comprises one or more of a first face area of the target object and a first limb part area of the target object;

a sequence frame generation sub-module configured to execute detection image sequence frames corresponding to the first direct broadcast picture based on the extracted target detection areas; the detection image sequence frame is used for being displayed in the first window;

a gesture motion determination sub-module configured to perform determining gesture motion data of the target object in each of the detected image sequence frames based on the location keypoint data; the gesture motion data is used for synchronously adjusting the gesture motion of the target virtual image;

and a rendering sub-module configured to perform rendering in the second window to generate a second live view based on the target detection area, the gesture motion data, and the avatar model data of the target avatar.

In an alternative embodiment, in the case that the target detection area includes a first face area of the target object, and the gesture motion data includes head motion data, the rendering sub-module is specifically configured to perform:

In an alternative embodiment, the apparatus further comprises:

the first processing module is configured to acquire the second live broadcast picture displayed in the second window through a second client; the second client is used for pushing the live broadcast content comprising the second live broadcast picture to a third client, and the third client is used for displaying the live broadcast content in the preset live broadcast room; or alternatively

And the second processing module is configured to execute live content push flow containing the second live broadcast picture to a third client, and the third client is used for displaying the live content in the preset live broadcast room.

In an alternative embodiment, the first acquisition module is specifically configured to perform:

According to a third aspect of the disclosed embodiments, there is provided a computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the live broadcast processing method as described in any of the above embodiments.

According to a fourth aspect of embodiments of the present disclosure, there is provided an electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the live processing method according to any of the above embodiments.

According to a fifth aspect of the disclosed embodiments, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the live processing method provided in any of the embodiments described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

in the embodiment of the disclosure, a first direct-broadcasting picture corresponding to at least one target object is displayed in a first window in response to an image operation instruction triggered by a first direct-broadcasting account; the target object comprises any one of a host player corresponding to the first direct play account and a link object interacted with the host player link, and the first direct play picture comprises at least one first face area respectively corresponding to the target object; responding to a selection operation instruction of at least one virtual image, and acquiring target virtual images corresponding to at least one target object respectively; rendering in the second window to generate a second live broadcast picture; the second live broadcast picture comprises mixed virtual images with the same number as that of at least one target object, the mixed virtual images are obtained by replacing second face areas of the target virtual images with first face areas of corresponding target objects, the face expressions of the mixed virtual images are synchronous with the face expressions of the target objects, and the second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account. In this way, the mixed virtual image is obtained by replacing the second face area of the target virtual image with the first face area of the corresponding target object, and the facial expression of each mixed virtual image is synchronous with the facial expression of the corresponding target object, namely, the real human face is combined with the body of the target virtual image, so that the expression change of the virtual image is not required to be driven by an expression driving model, the deviation between the expression of the virtual image and the expression of the host broadcast is reduced, and the expression transfer accuracy of the generated mixed virtual image is improved. In addition, the first direct-broadcasting account has control right for the image operation of the target object, and the target object can be a connecting object different from the first direct-broadcasting account, so that the image control of the connecting object can be realized, the interaction form between the main broadcasting is enriched, the single limitation of the interaction form of the direct broadcasting room is improved, the diversity and the interestingness of the direct broadcasting interaction are improved, and the attraction of the direct broadcasting room after the direct broadcasting processing is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.

Fig. 1 is an architecture diagram illustrating a system for applying a live processing method according to an exemplary embodiment.

Fig. 2 is a flow chart illustrating an application live processing method according to an exemplary embodiment.

Fig. 3 is a partial flow chart illustrating a live processing method according to an exemplary embodiment.

Fig. 4 is a schematic diagram illustrating an interface before and after adjustment of a live process, according to an example embodiment.

FIG. 5 is a multi-terminal interaction process diagram illustrating a live interface processing method according to an example embodiment.

Fig. 6 is a block diagram of a live processing device, according to an example embodiment.

Fig. 7 is a block diagram of an electronic device for live processing, according to an example embodiment.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

Fig. 1 is an architecture diagram of a system for applying a live processing method, which may include a terminal 10, a live server 20, and a terminal 30, according to an exemplary embodiment, referring to fig. 1.

The

terminals

10 and 30 may be, but are not limited to, physical devices such as notebook computers, desktop computers, tablet computers, smart phones, etc. The number of

terminals

10 or 30 may be one or more. The

terminals

10 and 30 are provided with target applications, such as clients and applets, for performing live broadcast processing with the live broadcast server 20, and an operating system for supporting running of media resource applications. The terminal 10 may be a main broadcasting terminal, the terminal 30 may be a viewer terminal, and optionally, the target application program carried on the terminal 10 may include a first client, a second client and a third client, where the first client is used for executing processing on the live broadcast picture, the second client is used for capturing the live broadcast picture processed by the first client and pushing the live broadcast picture to the third client, and the live broadcast picture is displayed by a live broadcast interface of the third client. Alternatively, the live broadcast processing function of the first client may be integrated on the second client or the third client, so as to reduce the number of clients carried on the terminal and related to live broadcast. The target application program carried on the terminal 30 may include at least a client for viewing live broadcast. Of course, in some scenarios, the anchor, audience identities of the

terminals

10 and 30 may be interchanged.

The live server 20 may be, but not limited to, an independent server, a server cluster or a distributed system formed by a plurality of physical servers, or one or more cloud servers providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and artificial intelligence platforms, and the like. The terminal 10 and the live broadcast server 20, and the terminal 30 and the live broadcast server 20 may be directly or indirectly connected through wired or wireless communication, which is not limited in this embodiment of the present disclosure. Through communication among the terminal 10, the terminal 30 and the live broadcast server 20, the live broadcast server 20 can provide background support for live broadcast processing for the terminal 10 and the terminal 30 to realize a required live broadcast processing procedure.

The live broadcast processing method provided by the embodiment of the disclosure may be executed by a live broadcast processing device, where the live broadcast processing device may be integrated in electronic equipment such as a terminal in a hardware form or in a software form, or may be implemented by the terminal alone or by the terminal and the server cooperatively executing.

Of course, the method provided by the embodiments of the present disclosure is not limited to the system architecture diagram shown in fig. 1, but may be used in other possible system architecture diagrams, and the embodiments of the present disclosure are not limited thereto. The functions that can be implemented by the corresponding devices of the system architecture diagram shown in fig. 1 will be described together in the following method embodiments, which are not described in detail herein.

It should be noted that, when the following embodiments of the present disclosure are applied to specific products or technologies, user license authorization or consent needs to be obtained, and the collection, use and processing of the relevant data need to comply with relevant laws and regulations and standards of relevant countries and regions.

Fig. 2 is a flowchart illustrating a live broadcast processing method according to an exemplary embodiment, and as shown in fig. 2, the live broadcast processing method is applied to a first client to illustrate, including the following steps.

In step S201, a first direct-broadcasting picture corresponding to at least one target object is displayed in a first window in response to an image operation instruction triggered by a first direct-broadcasting account; the target object comprises any one of a host player corresponding to the first direct play account and a link object interacted with the host player link, and the first direct play picture comprises at least one first face area respectively corresponding to the target object.

The first direct broadcast account may be a anchor account (e.g., an account of anchor a) participating in the preset live room. The character operation instruction is an instruction for triggering adjustment of an object character shown in a current live-broadcast picture, which may be a play-out picture or a picture currently being live-broadcast. The persona operation instructions may be generated by a trigger operation for a functionality control in the first client or may be obtained from other clients in communication with the first client.

The first window is used for displaying a first direct-play picture of at least one target object needing to be subjected to image adjustment, and the first window can be a part of interface area in a user operation interface corresponding to the first client, for example, can include, but is not limited to, a left side area, a right side area, an upper side area, a lower side area or the like in the user operation interface.

The number of target objects may be one or more, and the target objects include any one of a host corresponding to the first direct broadcast account and a link object interacting with the host link. Each target object can be displayed in a preset live broadcasting room corresponding to the first live broadcasting account number in an image mode, namely, each target object displays a personal image or a representative image in the preset live broadcasting room. Wherein the link object refers to another anchor or viewer object participating in the link interaction in the preset live room, and the link interaction may include, but is not limited to, link PK, etc.

The first live view may be at least a portion of the original live window view corresponding before the target object is not personally adjusted. The first direct broadcast picture comprises at least one first face area corresponding to the target objects respectively, and the first face area is used for reflecting real person face information of the target objects in real time. For example, a common anchor picture formed by the live broadcast window 1 corresponding to the anchor a and the live broadcast window 2 corresponding to the anchor b is displayed in the preset live broadcast room, if the target object is the anchor b, the first anchor picture may only include all pictures or part of pictures of the live broadcast window 2 corresponding to the anchor b, and the first anchor picture includes a face area of a real person of the anchor b.

Optionally, the first client may obtain a first direct-broadcasting picture corresponding to the target object in response to an image operation instruction triggered by the first direct-broadcasting account, and display the first direct-broadcasting picture including the first face area corresponding to the at least one target object in a first window in a user operation interface corresponding to the first client. For example, the number of live windows corresponding to the first live frame may match the number of target objects.

In an alternative embodiment, the acquisition path of the first direct broadcast picture may include any one of the following:

The method comprises the steps of calling an image acquisition module through a third client corresponding to a target object to acquire, capturing a live broadcast picture of the target object to acquire, recording a live broadcast picture of the target object to acquire, and calling the image acquisition module through a first client to acquire any one of the live broadcast picture of the target object. Wherein the third client may be an application or applet providing live services. The image acquisition module can be a camera, a video recorder and the like.

In an exemplary, in a main broadcasting scene, a live broadcast picture of a main broadcasting own real human face can be obtained through a camera and used as the first live broadcast picture. The camera may be invoked by the third client or by the first client itself.

In the PK link wheat scene, if the target object is the opponent PK anchor, the picture of the opponent PK anchor at least comprising the face can be obtained by carrying out screen capturing or screen recording on the live picture of the opponent PK anchor, and the obtained picture is used as the first direct broadcasting picture.

According to the embodiment, the acquisition ways of the first direct-broadcasting pictures are various, so that the required direct-broadcasting pictures can be flexibly acquired according to actual conditions to perform virtual image adjustment, particularly, a screen capturing or screen recording mode is introduced, the required first direct-broadcasting pictures can be quickly acquired without waiting for the authorization of the terminal equipment of the target object, and the direct-broadcasting processing safety of the terminal equipment of the target object and the direct-broadcasting processing efficiency are improved.

Alternatively, the first client may obtain an object account identifier of the at least one target object in response to an image operation instruction triggered by the first live account, and obtain object image frame data of the at least one target object from the live server based on the object account identifier; and then, generating a corresponding first direct broadcast picture based on the acquired object image frame data, and displaying the first direct broadcast picture in a first window.

In step S203, in response to a selection operation instruction for at least one avatar, target avatars respectively corresponding to at least one target object are acquired.

Wherein the avatar is a special effect avatar distinguishing from a real person. The avatar may include, but is not limited to, a virtual person (e.g., cartoon character, game character, etc.), food (e.g., fruit, vegetables, etc.), animals and plants (e.g., cats, dogs, green plants, etc.), buildings, living goods, digital people, etc. The target avatar is an operation object selected and used for performing an avatar adjustment operation.

Alternatively, the first client may present at least one avatar to the user object, and in response to a selection operation instruction for the at least one avatar, the selected avatar may be used as the target avatar, and the number of the target avatars may be one or more.

In an alternative embodiment, when the number of target avatars is a plurality, different target avatars may correspond to different target objects, respectively. For example, taking a multi-person live broadcast room created by a host a and a host b as an example, if the first direct broadcast account is an account corresponding to the host a, the target objects may be host a and host b, the host a may select the target avatar a for the host, and the host a may also select the target avatar b, that is, two different target avatars, for the host b, which respectively correspond to different target objects.

In an alternative embodiment, the number of target avatars may be determined based on the number of target objects, each of which may correspond to one target avatar, respectively, e.g., n target objects corresponding to n target avatars.

The kinds of the target avatars may be the same as or different from the number of target objects, for example, m kinds of the n target avatars, where m is less than or equal to n. If the two objects are different, the multiple target objects can share one target virtual image for image adjustment. For example, in case that there are a plurality of target objects, at least two of the target objects may correspond to the same target avatar. For example, taking a multi-person live broadcast room created by a host a, a host c and a host d as an example, if the first direct-cast account is an account corresponding to the host a, the target objects are host c and host d, the host a may select the target avatar c for the host c, and the host a may also select the same target avatar c as the host c for the host d, that is, the unused target objects (host c and host d) share the same target avatar c.

In an alternative embodiment, the obtaining the target avatars respectively corresponding to the at least one target object in response to the selection operation instruction for the at least one avatar includes:

and adding the target special effect element into the corresponding initial virtual image to obtain the target virtual image.

Optionally, the first client may display a special effect window displaying at least one of at least one candidate avatar, a special effect element, which may include, but is not limited to, at least one of a special effect, apparel, hairstyle, props, virtual background. And responding to the selection operation of at least one avatar displayed in the special effect window, and taking the selected avatar as an initial avatar corresponding to at least one target object respectively, wherein the initial avatar is a default original avatar which is not subjected to special effect decoration. If the original dressing selection operation is detected to be executed for the initial virtual image, a selected target special effect element is obtained, wherein the target special effect element can include at least one of special effects, clothes, hairstyles, props and virtual backgrounds. Then, a target avatar is added to the original avatar to perform an avatar composition, and the composed avatar is used as the target avatar. According to the embodiment, the display effect of the selected virtual image is adjusted by introducing the special effect element, so that the diversity and individuation requirements of the target virtual image are increased, and the live interaction effect is improved.

In step S205, rendering and generating a second live broadcast picture in a second window; the second live broadcast picture comprises mixed virtual images with the same number as that of at least one target object, the mixed virtual images are obtained by replacing second face areas of the target virtual images with first face areas of corresponding target objects, the face expressions of the mixed virtual images are synchronous with the face expressions of the target objects, and the second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account.

The second window is used for displaying a second live broadcast picture of at least one target object after image adjustment, the second window can be any other interface area except the first window in the user operation interface corresponding to the first client, and the arrangement mode of the first window and the second window in the user operation interface can include but is not limited to left-right arrangement, up-down arrangement, diagonal arrangement and the like.

The second live view screen includes a mixed avatar having the same number of objects as at least one target object, the mixed avatar being obtained by replacing a second face area of the target avatar with a first face area of a corresponding target object, that is, the mixed avatar being obtained by performing area fusion based on the first face area of the target object and target avatar areas of the target avatars respectively corresponding to the target objects, the target avatar areas being areas of the target avatars other than the second face area, thereby rendering the generated mixed avatar combining the real face of the target object and the body of the corresponding target avatar.

The second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account number, and the second live broadcast picture in the live broadcast interface can be displayed to a live broadcast audience so as to externally present the mixed virtual image with the adjusted image.

According to the embodiment, the mixed virtual images are obtained by replacing the second face area of the target virtual image with the first face area of the corresponding target object, and the facial expression of each mixed virtual image is synchronous with the facial expression of the corresponding target object, namely, the real human face is combined with the body of the target virtual image, the expression change of the virtual image is not required to be driven through the expression driving model, the deviation between the expression of the virtual image and the expression of the host broadcast is reduced, and the expression transfer accuracy of the generated mixed virtual image is improved. In addition, the first direct-broadcasting account has control right for the image operation of the target object, and the target object can be a connecting object different from the first direct-broadcasting account, so that the image control of the connecting object can be realized, the interaction form between the main broadcasting is enriched, the interaction limitation of the direct broadcasting room due to single interaction form is improved, the diversity and the interestingness of the direct broadcasting interaction are improved, and the attraction of the direct broadcasting room after the direct broadcasting processing is improved.

In addition, unlike the way of driving the avatar expression of the target avatar by the expression of the target object, the method of replacing the corresponding second face area of the target avatar by the first face area of the target object is not needed to calculate the real-time expression data of the target object, and the avatar expression is regulated in real time based on the real-time expression data, so that the overall calculation amount of the avatar regulation is reduced, the dependence on high-quality communication network and computer resources is reduced, and the live broadcast processing threshold is also reduced.

In an alternative embodiment, the target gesture motion of each mixed avatar displayed in the second live broadcast picture is matched with the target gesture motion of the target object to which the face region in each mixed avatar belongs.

Wherein the target gesture motion comprises one or more of a head gesture motion and a limb gesture motion. The head posture action includes, but is not limited to, at least one of a head rotation action, a head movement action, and the like, which refers to a movement action in a certain movement direction of the target object in the spatial environment, and for example, the movement direction may be at least one of left/right, front/rear, up/down, and other directions. Limb gesture movements include, but are not limited to, gesture movements of limbs, gestures, and the like.

Alternatively, taking the example that the target gesture motion includes a head gesture motion, the head of the corresponding mixed avatar may be driven in real time by the head motion of the target object, so that the head gesture motion of the mixed avatar displayed in the second live view is matched in real time with the head gesture motion of the target object to which the face region in the mixed avatar belongs. Therefore, the mixed virtual image displayed in the live interface has the real face of the target object of the face area in the mixed virtual image, and the head of the mixed virtual image is synchronous with the action of the head of the target object, so that the uncoordinated movement of the face and the movement of the head of the mixed virtual image are avoided, the appearance of a user is influenced, and the display effect of the mixed virtual image in the live interface is improved.

Optionally, in the case that the first direct broadcast picture may include a first limb portion area of the target object, the target gesture motion may further include a limb gesture motion, and the limb gesture motion of the corresponding hybrid avatar is driven in real time by the limb gesture motion of the target object, so that the limb gesture motion of the second limb portion area in the hybrid avatar matches with the limb gesture motion of the corresponding portion of the first limb portion area, and the limb gesture motion may include any one of a limb motion, a gesture motion, and the like. Therefore, by considering the limb gesture motion of the driving mixed virtual image, the gesture motion of the corresponding target object is matched with the gesture motion of the corresponding position of the corresponding target object, the interactive information quantity transmitted by the mixed virtual image is improved, and the live broadcast effect is improved.

In another alternative embodiment, the target gesture motion of the hybrid avatar matches the target gesture motion of the gesture reference object. Wherein the target gesture motion comprises one or more of a head gesture motion and a limb gesture motion. The gesture reference object may include, in addition to the target object to which the face area in each mixed avatar belongs, a connecting object that participates in the connecting interaction of the preset living room and is different from the target object to which the face area in each mixed avatar belongs, that is, the target gesture actions of the face area and the corresponding target part area of the mixed avatar originate from the same object.

In practical application, taking a preset live broadcast room created by a host a, a host b and a host c as an example, the host a performs image adjustment for the host b to generate a mixed virtual image 1, the face of the mixed virtual image 1 is the face of the live broadcast b, and the target gesture action of the target part area in the mixed virtual image 1 can also be matched with the target gesture action of the host b. At this time, the gesture motion of the mixed avatar 1 may follow the gesture motion of the anchor b itself, so that the mixed avatar 2 delivers more interactive information of the anchor b.

For another example, the anchor a performs the character adjustment for the anchor b to generate the mixed avatar 2, and the face of the mixed avatar 2 belongs to the anchor b, but the target gesture of the target part area in the mixed avatar 2 does not match the target gesture of the anchor b, but may match the part actions of other anchors (for example, anchor a or anchor c), that is, the face area of the mixed avatar and the target gesture of the corresponding target part area originate from different objects. At this time, the gesture of the mixed avatar 2 may follow the gesture of the other anchor, and the gesture control of the anchor b by the other anchor can be realized.

According to the embodiment, the expression and the target gesture of the face area in the mixed virtual image are split, and are regulated and controlled according to the account object to which the mixed virtual image belongs, so that gesture motion control of other live account numbers on the mixed virtual image can be realized, a new live interaction mode is unlocked, and the interaction mode and effect of live interaction are further enriched.

In an alternative embodiment, as shown in fig. 3, rendering the second live view in the second window includes:

in step S301, based on a preset detection frequency, performing key point detection on a target part of the first direct broadcast picture, and extracting, in real time, a target detection area and position key point data corresponding to the target detection area from the first direct broadcast picture according to a key point detection result, wherein the target part comprises one or more of a face and a limb part, and the target detection area comprises one or more of a first face area of a target object and a first limb part area of the target object;

In step S303, a detected image sequence frame corresponding to the first direct broadcast picture is generated based on each extracted target detection area; detecting an image sequence frame for display in a first window;

in step S305, pose motion data of the target object in each frame of the detection image sequence frame is determined based on the position key point data; the gesture motion data is used for synchronously adjusting the gesture motion of the target virtual image;

in step S307, a second live view is rendered in the second window based on the target detection area, the gesture motion data, and the avatar model data of the target avatar.

The preset detection frequency may be determined based on a frame rate of the first direct broadcast picture, and if the frame rate of the first direct broadcast picture is 30 frames/second, the preset detection frequency may be less than or equal to 30 times/second, for example, the preset detection frequency is 30 times/second. If the frame rate of the first on-stream picture is 60 frames/second, the preset detection frequency may be less than or equal to 60 times/second, for example, the preset detection frequency is 30 times/second, 60 times/second, etc.

The target site includes one or more of a face and a limb site. The location key point data corresponds to the target location, including one or more of facial key point data and limb key point data. Taking a face as an example, facial key points include eyebrows, eyes, nose, mouth, facial contours, and the like. Accordingly, the face key point data may be position data, coordinate data, or the like of the detected face key point. Taking the limb position as an example, the limb key points comprise the joint points of the limbs, the finger joint points and the like. Accordingly, the limb key point data may be position data, coordinate data, etc. of the detected limb key point.

The detected image sequence frames are formed based on the target detection areas corresponding to the corresponding frames of the first direct broadcast picture. The target detection area includes one or more of a first face area of the target object, a first limb portion area of the target object. The detection image sequence frame is used for displaying the position key points corresponding to the position key points in the first window, for example, at least one detection information such as the face key points, the limb key points and the like can be displayed in the displayed detection image sequence frame.

Optionally, taking the detection of the key points of the face and the key points of the limb part as an example, on the first client, the real person picture in the first direct broadcast picture is subjected to face key point detection and key point detection of the limb part, for example, the main broadcast detection picture is obtained frame by frame at a preset detection frequency of 30 times per second, the detection picture comprises a face picture and a limb part picture, and a detection image sequence frame is formed based on a target detection area formed by each first face area corresponding to the face picture and each first limb part area corresponding to the limb part picture. Then, calculating gesture action data of a target object in each frame of the detection image sequence frame based on the position key point data; the gesture motion data is used to synchronously adjust gesture motions of the target avatar, such as steering motions of the head of the avatar, moving motions of limbs, gestures of fingers, and the like. And then, rendering and generating a second live broadcast picture in a second window through a rendering module based on the target detection area, the gesture action data and the acquired image model data of the target virtual image.

According to the embodiment, the first face area of the target object is extracted in real time, the first face area reflects real-time face data of a live person, the real face and the virtual image are fused in real time, the real face and the virtual image are displayed in the second live broadcast picture, and the problem of head action matching is considered in the fusion process, so that the reality of the mixed virtual image is further improved. In addition, compared with the traditional expression driving mode, the embodiment of the application only considers the mode of driving the face and the head of the real person, ensures the authenticity of the virtual image, and simultaneously reduces the overall calculation amount in the live broadcast processing process.

In an optional embodiment, in a case where the target detection area includes a first face area of the target object and the gesture motion data includes head motion data, the step S307, rendering the second live view in the second window based on the target detection area, the gesture motion data, and the avatar model data of the target avatar includes: replacing the second face area in the target virtual image with the first face area of the corresponding target object to obtain an adjusted face area of the target virtual image; determining a head pose parameter of the target avatar based on the head motion data; and based on the image model data of the face area, the head gesture parameters and the target virtual image, obtaining a mixed virtual image, and rendering and generating a second live broadcast picture containing the mixed virtual image in a second window, wherein the frame rate of the second live broadcast picture is related to the numerical value of the preset detection frequency. Optionally, the head motion data includes one or more of a head rotation angle, a head lateral offset, a head longitudinal offset.

Optionally, in the rendering process, the first client may replace the first face area to the second face area corresponding to the target avatar frame by frame first, so as to obtain the adjusted face area of the target avatar. Then, based on the head rotation angle, the head lateral offset, the head longitudinal offset included in the head motion data, head posture parameters of the target avatar, such as the head rotation parameter, the head offset amplitude parameter, etc., are determined, and based on the adjusted face area, the head posture parameters, avatar model data of the target avatar are reconstructed, resulting in a mixed avatar. And rendering the second live broadcast picture containing the mixed avatar in the second window through a rendering module. The frame rate of the second live broadcast picture is related to the value of the preset detection frequency, for example, the preset detection frequency is 30 times/second, and the second live broadcast picture generated by rendering is a real-time picture of 30 frames/second or a real-time picture of less than 30 frames/second.

In the above embodiment, the mixed avatar is generated by the head pose parameter of the target avatar determined based on the head motion data, and since the head pose parameter includes one or more of the head rotation angle, the head lateral offset, and the head longitudinal offset, that is, the head motion data is quantized into the head pose parameter, the calculation amount of the head matching process is reduced, and the live broadcast processing efficiency is accelerated.

In an optional embodiment, in a case that the preset live broadcasting room is a single live broadcasting room, the target object includes at least one of a main broadcasting corresponding to the first direct broadcasting account and audience objects participating in live broadcasting online interaction;

under the condition that the preset live broadcasting room is a multi-user live broadcasting room, the target object comprises at least one of a host player corresponding to the first direct broadcasting account, at least one opposite side host player in the multi-user live broadcasting room and audience objects participating in live broadcasting online interaction.

In practical application, taking a single live broadcast room created by a host a as an example, a first direct broadcast account is an account corresponding to the host a, if the host a needs to adjust its display image during the process of opening or live broadcasting, at this time, the target object is the host a, and the host a can select a target virtual image for itself to adjust the image. In the live PK communication process between the anchor a and the audience m, if the audience m is input in the PK communication process, the anchor a can adjust the image of the audience m as punishment, at the moment, the target object is the audience m, and the anchor a can select one target virtual image for the audience m to perform image punishment adjustment. Of course, if the audience m wins in the current PK link, the anchor a may adjust the image of the audience m as a reward, and at this time, the target object is the audience m, and the anchor a may select a target avatar m for the audience m to adjust the image reward.

Taking a multi-person live broadcasting room created by a host a, a host b and a host c as an example, a first direct broadcasting account is an account corresponding to the host a, an own host is the host a, the other host is the host b and the host c, and if the host a needs to adjust the display image of the host during the starting or live broadcasting process, the target object is the host a, and the host a can select one target virtual image for the host to adjust the image. If the anchor a needs to adjust the display image of the opposite anchor, the target object is anchor b or anchor c, the anchor a can select one target virtual image 1 for anchor b to adjust the image, and the anchor a can select one target virtual image 2 for anchor c to adjust the image. If the audience n is invited to perform live broadcast online interaction, the target object may be the audience n, and the anchor a may select a target avatar n for the anchor b to perform avatar adjustment.

In the above embodiment, the target object may not be limited to the host, but may also be a connection object participating in live broadcast connection interaction, so that the first direct broadcast account number not only can control image adjustment of the host, but also can control the image of the connection object, enrich interaction forms between the host and the host, and between the host and the audience, improve single interaction forms between the live broadcast room, improve diversity and interest of live broadcast interaction, and facilitate increasing attraction between live broadcast rooms after live broadcast processing.

In an alternative embodiment, the method further comprises: acquiring a second live broadcast picture displayed in a second window through a second client; the second client is used for pushing the live broadcast content comprising the second live broadcast picture to the third client, and the third client is used for displaying the live broadcast content in the preset live broadcast room. The second client may be an application program for assisting live broadcast, and the third client may be an application program with live broadcast function. Here, the second live broadcast picture generated by the first client rendering can be captured through the second client, and the second live broadcast picture is pushed to the third client for live broadcast display. Thus, by transplanting the live broadcast processing function to the special first client, live broadcast blocking problems caused by large calculation amount of the second client and the third client are reduced.

In an alternative embodiment, the method further comprises: and pushing the live broadcast content containing the second live broadcast picture to a third client, wherein the third client is used for displaying the live broadcast content in a preset live broadcast room. Here, by embedding the live broadcast push function of the second client to the first client, the live broadcast processing operation path is simplified, and the live broadcast processing efficiency is improved.

In an alternative embodiment, in the case that the preset live broadcasting room is a multi-user live broadcasting room, the preset live broadcasting room further displays a first direct broadcasting picture, and the first direct broadcasting picture and the second direct broadcasting picture are respectively displayed in different live broadcasting windows of the preset live broadcasting room.

In practical application, taking a multi-person live broadcast room created by a host a and a host b as an example, before image adjustment, as shown in (a) in fig. 4, a window screen 401 of a live broadcast window of the host a and a window screen 402 of a live broadcast window of the host b are preset in the live broadcast room. After the character adjustment, as shown in (b) of fig. 4, the preset live room displays a first live view 404 of the live window of the anchor b in addition to a second live view 403 adjusted by the anchor a operation, and the first live view 404 may be identical to the window view 402. In the second live view 403, a mixed avatar 4032 of the anchor a's own avatar 4031 and the anchor a's avatar adjustment to the anchor b is shown, the mixed avatar 4032 combining the face of the anchor b and the body of the target avatar.

According to the embodiment, the live broadcast picture before the target object is adjusted is displayed besides the mixed virtual image after the target object is adjusted, so that the vivid contrast of the live broadcast display image is formed, and the live broadcast interaction rate is increased and the interestingness of the live broadcast space is further improved.

the method comprises the steps that a first direct-broadcasting account number is located at a position of a direct-broadcasting window in a direct-broadcasting interface, a second direct-broadcasting account number is located at a position of a direct-broadcasting window in the direct-broadcasting interface and a position of the direct-broadcasting account number except for each direct-broadcasting window in the direct-broadcasting interface, wherein the second direct-broadcasting account number is any one of the direct-broadcasting account numbers except for the first direct-broadcasting account number in the direct-broadcasting participation account number.

In practical application, the second live broadcast picture can be displayed at the position of the live broadcast window in the first live broadcast picture, and as shown in fig. 4, the second live broadcast picture controlled and adjusted by the anchor a is displayed at the position of the anchor room where the anchor a is located. Or the second live broadcast picture can be displayed at the position of a live broadcast window of the second live broadcast account in the live broadcast interface, wherein the second live broadcast account is any one of the live broadcast participation accounts except the first live broadcast account. For example, the second live view that the anchor a only regulates on the anchor b replaces the anchor room where the anchor b is located. Or, the second live view may be displayed at a position in the live view interface other than each live view window, for example, other blank positions of the live view interface, positions of live comment areas, and the like. Therefore, through flexible adjustment of the display position of the second live broadcast picture, live broadcast interactivity and diversity are further enriched, and live broadcast effect is improved.

For ease of understanding, consider the interaction between a first client, a second client, and a third client as an example, and fig. 5 is a multi-terminal interaction process diagram illustrating a live interface processing method according to an exemplary embodiment, where the live processing process includes the following steps, as shown in fig. 5.

1) The third client obtains real face data of the anchor 1 and sound of the anchor 1 through an image obtaining module (such as a camera) and an audio collecting module (a microphone), streams the real face data and the sound of the anchor 1, and sends the streamed data to the first client.

2) When the anchor is opened, the first client can acquire a live broadcast picture of the real human face of the anchor 1 through the camera. If the PK is connected with the wheat, the first client can acquire the face picture of the opposite PK anchor (such as anchor 2) in a screen capturing mode.

3) On the first client, face key point detection is performed on live broadcasting pictures of real faces, for example, detection is performed 30 times in 1 second, face pictures of a main broadcasting are obtained frame by frame, and face image sequence frames are formed.

4) And on the first client, calculating the steering angle of each frame of the anchor face through the face key point data, and simultaneously matching the steering action of the head of the virtual image.

5) On the first client, the face picture is combined with the face of the target avatar frame by frame to obtain a mixed avatar, and a real-time picture of 30 frames in 1 second, namely a second live picture, is rendered and generated. As shown in fig. 5, the anchor 1 performs visualization adjustment on itself through the first client to obtain the mixed avatar 1. In addition, the first client list may synchronize facial image data of other anchors (e.g., anchor 2, anchor 3, and anchor 4) at the same time, and perform face model mapping on the selected different target avatars to obtain other mixed avatars, such as mixed avatar 2, mixed avatar 3, and mixed avatar 4.

6) And capturing a picture window of the second live broadcast picture rendered in real time by using the second client to carry out push live broadcast. For example, if the anchor 1 performs visualization processing only on itself, only the visualization adjustment content of the mixed avatar 1 is displayed in the second live window. If the anchor 1 performs visualization processing on itself and other anchors at the same time, the visualization adjustment contents of the mixed avatar 1, the mixed avatar 2, the mixed avatar 3 and the mixed avatar 4 are displayed in the second live window.

7) The third live broadcast end and the audience end can see live broadcast content with combination of the real expression and the virtual image.

According to the embodiment, the mixed virtual images are obtained by replacing the second face area of the target virtual image with the first face area of the corresponding target object, and the facial expression of each mixed virtual image is synchronous with the facial expression of the corresponding target object, namely, the real human face is combined with the body of the target virtual image, the expression change of the virtual image is not required to be driven through the expression driving model, the deviation between the expression of the virtual image and the expression of the host broadcast is reduced, and the expression transfer accuracy of the generated mixed virtual image is improved. In addition, the first direct-broadcasting account has control right for the image operation of the target object, and the target object can be a connecting object of different direct-broadcasting accounts, so that the image control of the connecting object can be realized, the interaction form between the main broadcasting is enriched, the single limitation of the interaction form of the direct broadcasting room is improved, the diversity and the interestingness of the direct broadcasting interaction are improved, and the attraction of the direct broadcasting room after the direct broadcasting processing is improved.

Fig. 6 is a block diagram of a live processing device, according to an example embodiment. Referring to fig. 6, the apparatus may be applied to a first client, including:

The first display module 610 is configured to execute an image operation instruction triggered by responding to the first direct-broadcasting account, and display a first direct-broadcasting picture corresponding to at least one target object in a first window; the target object comprises any one of a host player corresponding to the first direct play account and a link object interacted with the host player link, and the first direct play picture comprises at least one first face area respectively corresponding to the target object;

a first obtaining module 620 configured to perform obtaining target avatars respectively corresponding to at least one target object in response to a selection operation instruction for the at least one avatar;

a rendering module 630 configured to perform rendering in the second window to generate a second live view; the second live broadcast picture comprises mixed virtual images with the same number as that of at least one target object, the mixed virtual images are obtained by replacing second face areas of the target virtual images with first face areas of corresponding target objects, the face expressions of the mixed virtual images are synchronous with the face expressions of the target objects, and the second live broadcast picture is used for being displayed in a live broadcast interface of a preset live broadcast room corresponding to the first live broadcast account.

In an alternative embodiment, the rendering module includes:

the detection sub-module is configured to perform key point detection of a target part on the first direct broadcast picture based on a preset detection frequency, and extract a target detection area and part key point data corresponding to the target detection area in real time from the first direct broadcast picture according to a key point detection result, wherein the target part comprises one or more of a face and a limb part, and the target detection area comprises one or more of a first face area of a target object and a first limb part area of the target object;

a sequence frame generation sub-module configured to execute detection image sequence frames corresponding to the first direct broadcast picture based on the extracted target detection areas; detecting an image sequence frame for display in a first window;

And a rendering sub-module configured to perform rendering in a second window to generate a second live view based on the target detection area, the gesture motion data, and the avatar model data of the target avatar.

In an alternative embodiment, in the case that the target detection area includes a first face area of the target object and the gesture motion data includes head motion data, the rendering sub-module is specifically configured to perform:

replacing the second face area in the target virtual image with the first face area of the corresponding target object to obtain an adjusted face area of the target virtual image;

based on the image model data of the adjusted face area and the target virtual image, obtaining a mixed virtual image, and based on the head gesture parameter, rendering and generating a second live broadcast picture containing the mixed virtual image in a second window, wherein the frame rate of the second live broadcast picture is related to the numerical value of the preset detection frequency;

the head motion data comprises one or more of head rotation angle, head transverse offset and head longitudinal offset.

In an alternative embodiment, the first direct-play picture acquisition path includes any one of the following:

The method comprises the steps of calling an image acquisition module through a third client corresponding to a target object to obtain, capturing a live broadcast picture of the target object, recording a live broadcast picture of the target object, and calling the image acquisition module through a first client to acquire.

In an alternative embodiment, the apparatus further comprises:

the first processing module is configured to execute the acquisition of a second live broadcast picture displayed in a second window through a second client; the second client is used for pushing the live broadcast content comprising the second live broadcast picture to the third client, and the third client is used for displaying the live broadcast content in a preset live broadcast room; or alternatively

And the second processing module is configured to execute live broadcast content push flow containing the second live broadcast picture to a third client, and the third client is used for displaying the live broadcast content in a preset live broadcast room.

The method comprises the steps that a first direct-broadcasting account is located at a position of a live broadcast window in a live broadcast interface, a second direct-broadcasting account is located at a position of a live broadcast window in the live broadcast interface and a position except for each live broadcast window in the live broadcast interface, wherein the second direct-broadcasting account is any one of the direct-broadcasting accounts which participate in online interaction between preset live broadcasting rooms except for the first direct-broadcasting account.

In an alternative embodiment, the target gesture actions of the mixed virtual images are matched with the target gesture actions of gesture reference objects, and the gesture reference objects comprise connecting line objects which participate in the connecting line interaction of a preset live broadcasting room and are different from the target objects of the face areas in the mixed virtual images;

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 7 is a block diagram illustrating an electronic device for live broadcast processing, which may be a terminal, according to an exemplary embodiment, and an internal structure diagram thereof may be as shown in fig. 7. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the electronic device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of live processing. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the electronic equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 7 is merely a block diagram of a portion of the structure associated with the disclosed aspects and is not limiting of the electronic device to which the disclosed aspects apply, and that a particular electronic device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In an exemplary embodiment, there is also provided an electronic device including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement a live processing method as in the embodiments of the present disclosure.

In an exemplary embodiment, a computer readable storage medium is also provided, which when executed by a processor of an electronic device, causes the electronic device to perform the live broadcast processing method in the embodiments of the present disclosure. The computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In an exemplary embodiment, a computer program product containing instructions is also provided which, when run on a computer, cause the computer to perform the method of live processing in the embodiments of the present disclosure.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. The live broadcast processing method is characterized by being applied to a first client and comprising the following steps of:

2. The method of claim 1, wherein the target gesture actions of the mixed avatars shown in the second live view are respectively matched with the target gesture actions of the target objects to which the face regions in the mixed avatars belong;

3. The method of claim 2, wherein the rendering in the second window to generate the second live view comprises:

4. The method of claim 3, wherein, in the case where the target detection area includes a first face area of the target object and the gesture motion data includes head motion data, the rendering in the second window to generate a second live view based on the target detection area, the gesture motion data, and the avatar model data of the target avatar comprises:

5. The method of claim 1, wherein the first direct-play picture acquisition path includes any one of:

6. The method according to any one of claims 1-5, further comprising:

7. The method of claim 6, wherein the preset live room further displays the first direct-play picture in case the preset live room is a multi-person live room, and the first direct-play picture and the second direct-play picture are respectively displayed in different live windows of the preset live room.

8. The method of claim 6, wherein the presentation location of the second live view in the live interface comprises any of:

9. The method according to claim 1, wherein the target gesture actions of the mixed avatars are matched with the target gesture actions of gesture reference objects, the gesture reference objects comprising link objects participating in the link interaction of the preset living room and being different from the target objects to which the face regions in the respective mixed avatars belong;

10. The method according to any one of claims 1 to 5, 9, wherein the acquiring the target avatars respectively corresponding to the at least one target object in response to a selection operation instruction for the at least one avatar comprises:

11. A live broadcast processing apparatus, for application to a first client, comprising:

12. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the live processing method of any of claims 1 to 10.

13. A computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the live broadcast processing method of any of claims 1 to 10.