WO2022105862A1 - 视频生成及显示方法、装置、设备、介质 - Google Patents

视频生成及显示方法、装置、设备、介质 Download PDF

Info

Publication number
WO2022105862A1
WO2022105862A1 PCT/CN2021/131705 CN2021131705W WO2022105862A1 WO 2022105862 A1 WO2022105862 A1 WO 2022105862A1 CN 2021131705 W CN2021131705 W CN 2021131705W WO 2022105862 A1 WO2022105862 A1 WO 2022105862A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
user
image
virtual gift
avatar
Prior art date
Application number
PCT/CN2021/131705
Other languages
English (en)
French (fr)
Inventor
杨光
徐珊
吴迪
李灵
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2022105862A1 publication Critical patent/WO2022105862A1/zh
Priority to US18/320,508 priority Critical patent/US20230291980A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/816Monomedia components thereof involving special video data, e.g 3D video
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/10Office automation; Time management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00Animation
    • G06T13/203D [Three Dimensional] animation
    • G06T13/403D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4784Supplemental services, e.g. displaying phone caller identification, shopping application receiving rewards
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8146Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Definitions

  • the present disclosure relates to the technical field of video processing, and in particular, to a video generation and display method, apparatus, device, and medium.
  • video live broadcast platforms After logging in to the live video platform, users can customize an exclusive virtual gift in the virtual gift column, so that the live video platform can design a virtual gift that can express the user's personalized emotions according to its ideas and requirements.
  • video live broadcast platforms generally design virtual gift videos manually, with a long production cycle and high cost.
  • the present disclosure provides a video generation and display method, apparatus, device, and medium.
  • the present disclosure provides a video generation method, including:
  • the personalized data is fused with the basic avatar model to obtain a fused avatar image
  • the virtual gift video is sent to the first target device, and the virtual gift video is used for display in the target gift tray page of the first target device.
  • the present disclosure provides a video display method, including:
  • a virtual gift video is displayed.
  • the present disclosure provides a video generation device, comprising:
  • a first receiving unit configured to receive a generation request carrying personalized data sent by the first target device
  • a first fusion unit configured to fuse the personalized data with the basic avatar model in response to the generation request to obtain a fused avatar image
  • a video generation unit configured to generate a virtual gift video according to the avatar image
  • the first sending unit is configured to send the virtual gift video to the first target device, where the virtual gift video is used for displaying in the target gift tray page of the first target device.
  • the present disclosure provides a video display device, comprising:
  • a first obtaining unit configured to obtain personalized data when a generation operation input by a user is detected
  • the second sending unit is configured to send a generation request carrying the personalized data to the second target device, and the generation request is used to make the second target device feed back a virtual gift video, where the virtual gift video is fused by the personalized data and the basic avatar model generate;
  • a second receiving unit configured to receive the virtual gift video fed back by the second target device
  • the video display unit is configured to display the virtual gift video within the target gift tray page.
  • the present disclosure provides a computing device, comprising:
  • the processor is configured to read executable instructions from the memory and execute the executable instructions to implement the video generation method described in the first aspect or the video display method described in the second aspect.
  • the present disclosure provides a computer-readable storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, enables the processor to implement the video generation method described in the first aspect or the second aspect The video display method.
  • the video generation and display method, device, device, and medium according to the embodiments of the present disclosure can fuse personalized data with a basic avatar model to obtain a fused avatar image, and generate a avatar image for use in the target gift according to the avatar image.
  • the virtual gift video displayed in the tray page is used to realize the automatic generation of the virtual gift video, so that the production cycle of the virtual gift video can be shortened, and the production cost of the virtual gift video can be saved.
  • FIG. 1 is an application environment architecture diagram of a video generation method provided by an embodiment of the present disclosure
  • FIG. 2 is an application environment architecture diagram of another video generation method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of a video generation method according to an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of a method for sending a virtual gift video according to an embodiment of the present disclosure
  • FIG. 5 is a schematic flowchart of another method for sending a virtual gift video according to an embodiment of the present disclosure
  • FIG. 6 is a schematic flowchart of a video display method according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of a gift tray interface according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a data upload interface provided by an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a video generation apparatus according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a video display device according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of a computing device according to an embodiment of the present disclosure.
  • the term “including” and variations thereof are open-ended inclusions, ie, "including but not limited to”.
  • the term “based on” is “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • the video generation and display method provided by the present disclosure can be applied to the architectures shown in FIG. 1 and FIG. 2 , which will be described in detail with reference to FIG. 1 and FIG. 2 .
  • FIG. 1 shows an application environment architecture diagram of a video generation method provided by an embodiment of the present disclosure.
  • the architecture diagram may include at least one electronic device 110 on the client side and at least one server 120 on the server side.
  • the electronic device 110 may establish a connection with the server 120 and perform information exchange through a network protocol such as Hyper Text Transfer Protocol over Secure Socket Layer (HTTPS).
  • HTTPS Hyper Text Transfer Protocol over Secure Socket Layer
  • the electronic device 110 may be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable device, an all-in-one computer, a smart home device, or other device with communication functions, or a device simulated by a virtual machine or a simulator .
  • the server 120 may be a device with storage and computing functions, such as a cloud server or a server cluster.
  • the user can apply for customizing the virtual gift video in the video live broadcast platform on the electronic device 110 .
  • the live video platform can make virtual gift videos for users based on their customized applications.
  • the virtual gift video can be generated in the following manner.
  • the video live broadcast platform is used as the video live broadcast application for description.
  • the electronic device 110 detects a generation operation for generating a virtual gift video input by the user, it can acquire the user's personalized data, and send a generation request carrying the personalized data to the server 120 .
  • the server 120 may fuse the personalized data with the basic avatar model to obtain a fused avatar image, and generate a virtual gift video according to the avatar image , and then send the virtual gift video to the electronic device 110 .
  • the electronic device 110 may receive the virtual gift video fed back by the server 120, and display the virtual gift video in the target gift tray page.
  • the architecture shown in FIG. 1 can realize the automatic generation of the virtual gift video, thereby shortening the production cycle of the virtual gift video and saving the production cost of the virtual gift video.
  • the video generation and display method provided by the embodiments of the present disclosure can be applied not only to the above-mentioned architecture composed of an electronic device and a server, but also to an architecture composed of multiple electronic devices, which will be specifically described with reference to FIG. 2 .
  • FIG. 2 shows an application environment architecture diagram of another video generation method provided by an embodiment of the present disclosure.
  • the architecture diagram may include at least one first electronic device 210 and at least one second electronic device 220 .
  • the first electronic device 210 may establish a connection with the second electronic device 220 through a network protocol such as HTTPS and perform information exchange.
  • the first electronic device 210 and the second electronic device 220 may be devices with communication functions, such as mobile phones, tablet computers, desktop computers, notebook computers, in-vehicle terminals, wearable devices, all-in-one computers, smart home devices, etc. It is a device simulated by a virtual machine or an emulator.
  • the user using the first electronic device 210 can issue a customized application for a virtual gift video to the user using the second electronic device 220 through the first electronic device 210 .
  • the user using the second electronic device 220 may make a virtual gift video for the user using the first electronic device 210 based on the received customization application.
  • the virtual gift video can be generated in the following manner.
  • the first electronic device 210 detects a generation operation input by the user for generating a virtual gift video, it can acquire the user's personalized data, and send a generation request carrying the personalized data to the second electronic device 220 .
  • the second electronic device 220 After receiving the generation request carrying the personalized data sent by the first electronic device 210, the second electronic device 220 can fuse the personalized data with the basic avatar model to obtain a fused avatar image, and according to the avatar image , generate a virtual gift video, and then send the virtual gift video to the first electronic device 210 .
  • the first electronic device 210 may receive the virtual gift video fed back by the second electronic device 220, and display the virtual gift video in the target gift tray page.
  • the architecture shown in FIG. 2 can realize the automatic generation of the virtual gift video.
  • the second electronic device 220 After the second electronic device 220 receives the generation request carrying the personalized data sent by the first electronic device 210, the second electronic device 220 does not need to be used. of users manually make a virtual gift video, thereby shortening the production cycle of the virtual gift video and saving the production cost of the virtual gift video.
  • the video display method may be performed by a first target device, which may be the electronic device 110 in the client shown in FIG. 1 or the first electronic device 210 shown in FIG. 2 .
  • the video generation method may be performed by a second target device, and the second target device may be the server 120 in the server shown in FIG. 1 or the second electronic device 220 shown in FIG. 2 .
  • the electronic device 110, the first electronic device 210 and the second electronic device 220 may be mobile phones, tablet computers, desktop computers, notebook computers, in-vehicle terminals, wearable devices, all-in-one computers, smart home devices, etc. with communication functions, respectively.
  • the device can also be a device simulated by a virtual machine or an emulator.
  • the server 120 may be a device with storage and computing functions, such as a cloud server or a server cluster.
  • FIG. 3 shows a schematic flowchart of a video generation method provided by an embodiment of the present disclosure.
  • the video generation method may include the following steps.
  • S310 Receive a generation request that carries personalized data and is sent by the first target device.
  • the first target device can be used to send a generation request carrying personalized data to the second target device, and the generation request can be used to make the second target device automatically generate for the user according to the personalized data.
  • Customized virtual gift videos can be used to send a generation request carrying personalized data to the second target device, and the generation request can be used to make the second target device automatically generate for the user according to the personalized data.
  • the personalization data may include at least one of user information, a user gesture image, and a user face image.
  • the user information may include at least one of a user ID, a user nickname, and a user avatar.
  • the user identification may be a unique identification code of the user such as a user account, a user identification (Identity Document, ID).
  • the user gesture image may include at least one of a user body gesture image and a user facial gesture image.
  • the personalization data may include at least one body gesture image of the user.
  • the body gestures in the two or more body gesture images may form a continuous body motion, and the two or more body gesture images may also respectively contain independent body gestures.
  • the user body gesture image may include gesture actions of the user's head, neck, limbs, and torso.
  • the two or more body posture images can be consecutive image frames in the body movement video uploaded by the user, and the body movement video can be used as personalized data
  • the second target device can acquire more than two body gesture images of the user from the body action video.
  • the personalization data may include at least one facial gesture image of the user.
  • the facial gestures in the two or more facial gesture images may form a continuous facial action, and the two or more facial gesture images may also respectively contain independent facial gestures.
  • the user's facial gesture image may include gestures of the user's eyebrows, eyes, nose, mouth, and facial muscles.
  • the two or more facial gesture images can form continuous facial actions
  • the two or more facial gesture images can be consecutive image frames in the facial action video uploaded by the user, and the facial action video can be used as personalized data
  • the second target device can obtain more than two facial gesture images of the user from the facial action video.
  • the user's full-body action video can also be used as personalized data, so that the second target device can acquire more than two full-body posture images of the user from the full-body action video, and the full-body posture images can include facial posture and body posture.
  • the second target device may first obtain a preset basic avatar model, and then fuse the personalized data carried in the generation request with the basic avatar model to obtain a fusion
  • the fused avatar model is then used to generate a fused avatar image.
  • S320 when the personalized data includes a user gesture image, and the user gesture image includes at least one of a user body gesture image and a user facial gesture image, S320 may specifically include:
  • the user pose image is used to transfer the pose of the basic avatar model to generate a fused avatar image.
  • a deformation transfer technology such as a triangle mesh deformation transfer technology, etc.
  • a deformation transfer technology can be used to transfer the body poses in each body pose image to the base.
  • virtual image model obtain the fused virtual image model corresponding to each body posture image, that is, the virtual image model with the body posture in each body posture image, and then use the fused virtual image model corresponding to each body posture image A fused avatar image corresponding to the body posture image is generated.
  • the basic avatar model may be preset with a plurality of preset body poses.
  • the second target device may first establish a network model of the user by using each body gesture image to obtain a source model corresponding to each body gesture image.
  • a basic avatar model with a preset body posture similar to the body posture in any body posture image can be selected, and a grid model of the basic avatar model can be constructed to obtain a target model.
  • the deformation relationship between the source model and the target model with similar body poses can be determined.
  • the posture of each source model can be transferred to the target model by using the deformation transfer technology, and the fused virtual image model can be obtained.
  • the body posture of the avatar in the virtual gift video generated by using the avatar model can be associated with the user who applies for customizing the virtual gift video, so as to improve the user's experience.
  • a facial expression animation generation technology such as a facial expression animation generation technology based on Blendshape, etc.
  • the facial postures are respectively mapped to the faces of the basic avatar model, and the fused avatar model corresponding to each facial posture image is obtained, that is, the avatar model with the facial posture in each facial posture image, and then each facial posture image is used.
  • the fused avatar model corresponding to the posture image generates a fused avatar image corresponding to the facial posture image.
  • the second target device may first collect images of different facial gestures of different users, then use the facial gestures in the facial gesture images of each user as the expression base of the user, and extract each face Face landmarks in pose images. Then, mesh deformation fitting is performed on the face feature points in each facial pose image for the face of the basic avatar model, so that the face feature points in each image obtained by fitting can be aligned with the basic avatar model.
  • the coordinates of the facial feature points of the face can also be aligned with the depth position, and starting from the face grids in these fitted images, a set of face grids with personalized expression mixture shapes is constructed for each user.
  • the constructed grids are combined into a rank-three tensor to build a bilinear face model with two attributes of identity and expression.
  • the second target device After the second target device receives the generation request, it can obtain the facial gesture image carried in the generation request, and then use the bilinear face model to fuse the face mesh of the facial gesture image carried in the generation request to construct the generated
  • the user's face database in the facial gesture image carried in the request is further searched for a face model with the smallest error from the face grid in the facial gesture image in the face database.
  • the expression base and the expression base control coefficient of the face in the facial pose image carried in the generation request can be determined through the found face model, and the expression base control coefficient is applied to the expression base of the basic avatar model,
  • the facial postures in each facial posture image carried in the generation request are respectively mapped to the faces of the basic avatar model.
  • the facial posture of the avatar in the virtual gift video generated by using the avatar model can be associated with the user who applies for customizing the virtual gift video, so as to improve the user's experience.
  • S320 when the personalized data includes the user's face image, S320 may specifically include:
  • a three-dimensional (3Dimensions, 3D) face reconstruction technology (such as a 3D face reconstruction technology based on weakly supervised learning, etc.) can be used to automatically replace the face in the user's face image on the basic virtual image model to obtain a fused avatar image, that is, an avatar model with the face in the user's face image, and then use the avatar model with the face in the user's face image to generate the fused avatar image.
  • the basic avatar model in the FaceBuilder plug-in, can be re-created by using the user's face image, so that the created face model can achieve a movie-level effect.
  • the face of the avatar in the virtual gift video generated by using the avatar model can be associated with the user who applies for customizing the virtual gift video, so as to improve the user's experience.
  • S320 when the personalized data includes user information, and the user information includes at least one of a user ID, a user nickname, and a user avatar, S320 may specifically include:
  • the user information is added to the preset position of the avatar image, and the fused avatar image is generated.
  • the basic avatar model can be preset with multiple preset posture combinations, the preset posture combinations can include multiple preset postures, and the multiple preset postures can form continuous actions such as a like action and a greeting action. At least one of a preset body pose and a preset facial pose may be included.
  • the second target device may first use the basic avatar model to generate multiple avatar images under different preset posture combinations, and then, according to the preset information adding method, add user information to the avatar image The preset position of the fused avatar image is generated.
  • the preset information adding manner may include at least one of a preset font color, a preset font background, and a preset display effect.
  • the preset position can be any position in the avatar image, and can be set as required.
  • the user information of the avatar in the virtual gift video generated by using the avatar model can be associated with the user information of the user who applies for customizing the virtual gift video, the user information is exposed, the user's sense of participation is increased, and the user's experience is further improved.
  • the fusion of each personalized data and the basic avatar model may be performed separately.
  • the method respectively fuses it into the basic avatar model, and obtains the fused avatar image.
  • the basic virtual image model may be at least one of a cartoon image model and a virtual character model.
  • the following descriptions are given by taking the basic virtual image model as a cartoon image model and the basic virtual image model as a virtual character model as an example. .
  • the customized virtual gift video can be a small gift video.
  • the user can send a generation request carrying a body gesture image to the second target device.
  • the body posture can form continuous body movements, and the second target device can automatically transfer the body posture in the body posture image to the cartoon image model in response to the generation request, and generate a fused virtual image image.
  • the customized virtual gift video can be a large-scale gift video.
  • the user can send a body pose image, a face pose image and a user face image to the second target device.
  • the generation request, the body gestures in the body gesture image can form continuous body movements
  • the facial gestures in the facial gesture image can form continuous facial movements
  • the second target device can respond to the generation request, automatically replace the face of the virtual character model with The face in the user's face image, and then the body pose in the body pose image and the facial pose in the face pose image are transferred to the cartoon image model, and finally the fused virtual image image is generated.
  • the second target device may render an avatar image, and combine and play the rendered avatar images continuously to generate a virtual gift video in which the avatar performs coherent actions.
  • audio data may also be acquired, and a virtual gift video may be generated by using the avatar image and the audio data.
  • the audio data may be audio data carried through the generation request specified by the user, or may be preset audio data, which is not limited herein.
  • the second target device can acquire audio data, then render the avatar image, combine the rendered avatar images and play them continuously to generate a virtual gift video in which the avatar performs coherent actions, and finally, according to the frame images in the virtual gift video and audio data timing, add audio data to the virtual gift video.
  • the second target device after generating the virtual gift video, can send the virtual gift video to the first target device, so that the first target device can display the received virtual gift video in the target gift tray page.
  • the personalized data and the basic avatar model can be fused to obtain a fused avatar image, and according to the avatar image, a virtual gift video for displaying in the target gift tray page can be generated to The automatic generation of the virtual gift video is realized, so that the production cycle of the virtual gift video can be shortened, and the production cost of the virtual gift video can be saved.
  • the second target device can display the target live broadcast room
  • the electronic device of the live video displays the virtual gift video.
  • FIG. 4 shows a schematic flowchart of a method for sending a virtual gift video provided by an embodiment of the present disclosure.
  • the virtual gift video sending method may further include the following steps.
  • S410 Receive a generation request that carries personalized data and is sent by the first target device.
  • S410-S440 are similar to S310-S340 shown in FIG. 3 , and details are not described here.
  • S450 Receive interactive data for the target live broadcast room sent by the first target device, where the interactive data includes video information corresponding to the virtual gift video.
  • the first target device can be used to send the video information corresponding to the virtual gift video to the second target device, and the video information can carry the live broadcast room ID of the target live broadcast room,
  • the second target device is made aware of the sending object of the virtual gift video.
  • the video information may include at least one of a video ID of the virtual gift video and a video name of the virtual gift video.
  • the interaction data may further include the presentation time of the virtual gift video.
  • the interaction data may further include user comment information, user operation information, and the like.
  • the second target device may query the virtual gift video corresponding to the video information among the pre-stored multiple virtual gift videos.
  • the second target device may add the virtual gift video to the live video according to the time sequence of the frame images in the live video and the frame images in the virtual gift video.
  • the first frame image for adding the virtual gift video in the live video can be determined according to the presentation time of the virtual gift video, and then according to the time sequence of the frame images in the live video and the frame images in the virtual gift video, each The frame image in the virtual gift video is added to the target position in the frame image in the corresponding live video to add the virtual gift video to the live video.
  • the target position may be the middle of the image, the bottom of the image, the left side of the image, or the right side of the image, which is not limited herein.
  • the second target device may send the merged live video to the electronic device associated with the target live room.
  • the electronic device associated with the target live room may be an electronic device that is acquiring live video resources of the target live room, or it may be understood that the electronic device associated with the target live room is an electronic device that is displaying the live video of the target live room.
  • the electronic device, that is, the electronic device associated with the target live broadcast room is the electronic device used by the audience of the target live broadcast room.
  • the second target device when the second target device receives an application for presenting a customized virtual gift video to the host in the target live room, the virtual gift video can be distributed along with the live stream of the host, saving the traffic of the content distribution network.
  • FIG. 5 shows a schematic flowchart of another virtual gift video sending method provided by an embodiment of the present disclosure.
  • the virtual gift video sending method may further include the following steps.
  • S510 Receive a generation request that carries personalized data and is sent by the first target device.
  • S540 Send the virtual gift video to the first target device, where the virtual gift video is used for display in the target gift tray page of the first target device.
  • S510-S540 are similar to S310-S340 shown in FIG. 3 , and details are not described here.
  • S550 is similar to S450 shown in FIG. 4 , and details are not described here.
  • S560 Send gift data to the electronic device associated with the target live broadcast room, where the gift data includes user information corresponding to the first target device and video information corresponding to the virtual gift video.
  • the second target device after the second target device receives the video information, it can directly send gift data to the electronic device associated with the target live broadcast room, so that the electronic device associated with the target live broadcast room can
  • the live video display interface of the target live room displays the virtual gift video and user information of the user who presents the virtual gift video.
  • the electronic device associated with the target live broadcast room can query the virtual gift video corresponding to the video information from the pre-stored multiple virtual gift videos, and perform a live broadcast on the target live broadcast room.
  • the queried virtual gift video is played superimposed on the live video content in the room.
  • the gift data may further include the presentation time of the virtual gift video, so that the electronic device associated with the target live broadcast room can, according to the gift data, in the live video display interface of the target live broadcast room, according to the virtual gift
  • the presentation time of the video displays the virtual gift video and user information of the user who presented the virtual gift video.
  • the electronic device associated with the target live room when the electronic device associated with the target live room receives multiple pieces of gift data, it can sort the display order of the virtual gift videos corresponding to the multiple pieces of gift data according to the presentation time of the virtual gift videos, and then according to the sorted display Each virtual gift video is displayed in sequence.
  • the video generation method may further include: The electronic device associated with the target live room sends the virtual gift video, so that the electronic device associated with the target live room stores the virtual gift video.
  • the method for sending a virtual gift video may further include: sending video information corresponding to the virtual gift video to an electronic device associated with the target live broadcast room, so that the electronic device associated with the target live broadcast room stores the video information corresponding to the virtual gift video. video information.
  • the second target device when receiving an application for presenting a customized virtual gift video to the host of the target live broadcast room, can distribute the virtual gift video separately, so as to improve the display level of the virtual gift video, so that the virtual gift video is not displayed by other devices. The content is blocked to ensure the display effect of the virtual gift video.
  • FIG. 6 shows a schematic flowchart of a video display method provided by an embodiment of the present disclosure.
  • the video display method may include the following steps.
  • S610 Obtain personalized data when a generation operation input by the user is detected.
  • a generation operation for generating the virtual gift video can be input in the first target device.
  • the generating operation may include a triggering operation on a control for triggering the generation of the virtual gift video.
  • the trigger operation may be a click operation, a double-click operation, a long-press operation, or a voice operation.
  • control for triggering the generation of the virtual gift video may be a generate button within the target gift tray page for triggering the generation of the virtual gift video.
  • the target gift tray page can be a customized gift tray page on a live video platform (such as in a live video application), or can be a customized gift tray page in a live video display interface of any live room of the live video platform, or it can be a customized gift tray page.
  • the customized gift tray page in any interface of other platforms with the virtual gift video customization function is not limited here.
  • the personalized data uploaded by the user may be acquired.
  • the personalized data may include at least one of user information, a user gesture image, and a user face image.
  • the user information may include at least one of a user ID, a user nickname, and a user avatar.
  • the user gesture image may include at least one of a user body gesture image and a user facial gesture image.
  • the first target device may use the personalized data to generate a generation request that carries the personalized data, and sends the generation request that carries the personalized data to the second target device.
  • the generation request is used to make the second target device feed back the virtual gift video, which is generated by fusion of the personalized data and the basic virtual image model, which has been described in the embodiment shown in FIG. 3 and will not be repeated here.
  • the first target device may receive a virtual gift video fed back by the second target device.
  • the second target device can also feed back video information corresponding to the virtual gift video to the first target device, and the first target device can receive the video information corresponding to the virtual gift video fed back by the second target device, and store the corresponding video information of the virtual gift video. video information.
  • the second target device may also feed back a virtual gift preview image corresponding to the virtual gift video to the first target device.
  • the first target device may display the virtual gift preview image of the received virtual gift video in the target gift tray page.
  • FIG. 7 shows a schematic diagram of a gift tray interface provided by an embodiment of the present disclosure.
  • the first target device displays a live video display interface 701 of the target live room of the live video platform
  • the live video display interface 701 can display a customized gift tray page 702
  • the customized gift tray page 702 can display the received The virtual gift preview image 703 of the virtual gift video for the customized virtual gift video to distinguish the customized virtual gift video from other virtual gift videos.
  • the personalized data and the basic avatar model can be fused to obtain a fused avatar image, and according to the avatar image, a virtual gift video for displaying in the target gift tray page can be generated to The automatic generation of the virtual gift video is realized, so that the production cycle of the virtual gift video can be shortened, and the production cost of the virtual gift video can be saved.
  • the S610 may specifically include:
  • the user video or user image input by the user and containing the personalized data is obtained.
  • a data uploading interface can be displayed, so that the user can shoot or select the user video or user image containing personalized data in the data uploading interface , and then input a user video or a user image containing personalized data to the first target device.
  • FIG. 8 shows a schematic diagram of a data upload interface provided by an embodiment of the present disclosure.
  • the first target device displays a data upload interface 801 .
  • the data upload interface 801 may display a preview window 802 for capturing user videos or user images and a target button 803 for triggering capturing user videos or user images.
  • the user can click the target button 803 to capture the user image and preview the captured user image in the preview window 802 ; the user can also long press the target button 803 to capture the user video and preview the captured user video in the preview window 802 .
  • the user gesture image may be an image frame or a user image in the user video
  • the user face image may also be an image frame or a user image in the user video. That is, the user video and the user image may respectively include at least one of a user gesture image and a user face image.
  • a data uploading interface can be displayed, so that the user can shoot or select a user video or user image containing personalized data in the data uploading interface, and then input the personalized data to the first target device.
  • the user video or user image so that when the first target device detects the generation operation input by the user, the personalized data that has been input by the user can be obtained.
  • the user can upload personalized data more conveniently and quickly.
  • a sending operation may be input for the displayed virtual gift video in the target gift tray page, so that the first target device detects the virtual gift During the sending operation of the gift video, video information corresponding to the virtual gift video is sent to the second target device.
  • the sending operation may be a click operation, a double-click operation, a long-press operation, or a voice operation on the virtual gift video.
  • the first target device may also play the virtual gift video in the current interface.
  • the first target device can obtain the video information of the virtual gift video, and query the virtual gift video corresponding to the video information from the pre-stored multiple virtual gift videos, Then, the queried virtual gift video is played.
  • the user can click the virtual gift video on the target gift tray page of the live video display interface, so that the first target device sends the The video information corresponding to the virtual gift video, and the virtual gift video corresponding to the video information is queried among the pre-stored virtual gift videos, and then the queried virtual gift video is superimposed and played on the live video content of the target live broadcast room.
  • the virtual gift video can be played on the electronic device used by the user.
  • FIG. 9 shows a schematic structural diagram of a video generation apparatus provided by an embodiment of the present disclosure.
  • the video generating apparatus 900 may be set in the above-mentioned second target device, and the second target device may be the server 120 in the server shown in FIG. 1 or the second electronic device shown in FIG. 2 . 220.
  • the second electronic device 220 may be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle-mounted terminal, a wearable device, an all-in-one computer, a smart home device, or other device with a communication function, or a virtual machine or a simulator to simulate device of.
  • the server 120 may be a device with storage and computing functions, such as a cloud server or a server cluster.
  • the video generating apparatus 900 may include a first receiving unit 910 , a first fusion unit 920 , a video generating unit 930 and a first sending unit 940 .
  • the first receiving unit 910 may be configured to receive a generation request carrying personalized data sent by the first target device.
  • the first fusion unit 920 may be configured to, in response to the generation request, fuse the personalized data with the basic avatar model to obtain a fused avatar image.
  • the video generating unit 930 may be configured to generate a virtual gift video according to the avatar image.
  • the first sending unit 940 may be configured to send the virtual gift video to the first target device, where the virtual gift video is to be displayed in the target gift tray page of the first target device.
  • the personalized data and the basic avatar model can be fused to obtain a fused avatar image, and according to the avatar image, a virtual gift video for displaying in the target gift tray page can be generated to The automatic generation of the virtual gift video is realized, so that the production cycle of the virtual gift video can be shortened, and the production cost of the virtual gift video can be saved.
  • the personalization data may include at least one of user information, a user gesture image, and a user face image.
  • the user information may include at least one of a user ID, a user nickname, and a user avatar.
  • the user gesture image may include at least one of a user body gesture image and a user facial gesture image.
  • the personalization data may include a user gesture image, which may include at least one of a user body gesture image and a user facial gesture image.
  • the first fusion unit 920 may be further configured as:
  • the user pose image is used to transfer the pose of the basic avatar model to generate a fused avatar image.
  • the personalization data may include an image of the user's face.
  • the first fusion unit 920 may be further configured as:
  • the personalization data may include user information, and the user information may include at least one of a user identification, a user nickname, and a user avatar.
  • the first fusion unit 920 may be further configured as:
  • the user information is added to the preset position of the avatar image, and the fused avatar image is generated.
  • the video generating apparatus 900 may further include a third receiving unit, a second obtaining unit, a second fusion unit, and a third sending unit.
  • the third receiving unit may be configured to receive interactive data for the target live broadcast room sent by the first target device, where the interactive data includes video information corresponding to the virtual gift video.
  • the second obtaining unit may be configured to obtain a virtual gift video corresponding to the video information.
  • the second fusion unit may be configured to fuse the virtual gift video with the live video of the target live room to form a merged live video.
  • the third sending unit may be configured to send the merged live video to the electronic device associated with the target live room.
  • the video generating apparatus 900 may further include a fourth receiving unit and a fourth sending unit.
  • the fourth receiving unit may be configured to receive interactive data for the target live broadcast room sent by the first target device, where the interactive data includes video information corresponding to the virtual gift video.
  • the fourth sending unit may be configured to send gift data to the electronic device associated with the target live room, where the gift data includes user information corresponding to the first target device and video information corresponding to the virtual gift video.
  • the video generating apparatus 900 shown in FIG. 9 may execute various steps in the method embodiments shown in FIG. 3 to FIG. 5 , and implement each process and The effect will not be repeated here.
  • FIG. 10 shows a schematic structural diagram of a video display device provided by an embodiment of the present disclosure.
  • the video display apparatus 1000 may be set in a first target device, and the first target device may be the electronic device 110 in the client shown in FIG. 1 or the first electronic device shown in FIG. 2 .
  • the electronic device 110 and the first electronic device 210 may be devices with communication functions, such as mobile phones, tablet computers, desktop computers, notebook computers, in-vehicle terminals, wearable devices, all-in-one computers, and smart home devices, and may also be virtual machines. Or a device emulated by an emulator.
  • the video display apparatus 1000 may include a first obtaining unit 1010 , a second sending unit 1020 , a second receiving unit 1030 and a video display unit 1040 .
  • the first obtaining unit 1010 may be configured to obtain personalized data when a generation operation input by a user is detected.
  • the second sending unit 1020 can be configured to send a generation request carrying the personalized data to the second target device, and the generation request is used to make the second target device feed back a virtual gift video, the virtual gift video is composed of the personalized data and the basic avatar Model fusion generation.
  • the second receiving unit 1030 may be configured to receive the virtual gift video fed back by the second target device.
  • the video display unit 1040 may be configured to display a virtual gift video within the target gift tray page.
  • the personalized data and the basic avatar model can be fused to obtain a fused avatar image, and according to the avatar image, a virtual gift video for displaying in the target gift tray page can be generated to The automatic generation of the virtual gift video is realized, so that the production cycle of the virtual gift video can be shortened, and the production cost of the virtual gift video can be saved.
  • the personalization data may include at least one of user information, a user gesture image, and a user face image.
  • the user information may include at least one of a user ID, a user nickname, and a user avatar.
  • the user gesture image may include at least one of a user body gesture image and a user facial gesture image.
  • the personalization data may include at least one of a user gesture image and a user face image
  • the user gesture image may include at least one of a user body gesture image and a user face gesture image.
  • the first obtaining unit 1010 is further configured to:
  • a generation operation input by a user When a generation operation input by a user is detected, a user video input by the user including personalized data is acquired.
  • the video display device 1000 shown in FIG. 10 can perform various steps in the method embodiments shown in FIGS. 6 to 8 , and implement various processes and processes in the method embodiments shown in FIGS. The effect will not be repeated here.
  • Embodiments of the present disclosure also provide a computing device.
  • the computing device may include a processor and a memory, and the memory may be used to store executable instructions.
  • the processor may be configured to read executable instructions from the memory and execute the executable instructions to implement the video generation method or the video display method in the above-mentioned embodiments.
  • FIG. 11 shows a schematic structural diagram of a computing device provided by an embodiment of the present disclosure. Referring specifically to FIG. 11 below, it shows a schematic structural diagram of a computing device 1100 suitable for implementing an embodiment of the present disclosure.
  • the computing device 1100 in the embodiment of the present disclosure may be the first target device that executes the video display method, and may also be the second target device that executes the video generation method.
  • the first target device may be an electronic device, such as the electronic device 110 in the client shown in FIG. 1 or the first electronic device 210 shown in FIG. 2 .
  • the second target device may be an electronic device or a server, such as the server 120 in the server shown in FIG. 1 or the second electronic device 220 shown in FIG. 2 .
  • Electronic devices may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (eg, in-vehicle navigation terminals), Mobile terminals such as wearable devices, etc., and stationary terminals such as digital TVs, desktop computers, smart home devices, and the like.
  • the server may be a device with storage and computing functions, such as a cloud server or a server cluster.
  • computing device 1100 shown in FIG. 11 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the computing device 1100 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 1101 that may be loaded into a random access device according to a program stored in a read only memory (ROM) 1102 or from a storage device 1108 Various appropriate actions and processes are executed by accessing the program in the memory (RAM) 1103 .
  • ROM read only memory
  • RAM memory
  • various programs and data required for the operation of the computing device 1100 are also stored.
  • the processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104.
  • An input/output (I/O) interface 1105 is also connected to the bus 1104 .
  • the following devices can be connected to the I/O interface 1105: input devices 1106 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 1107 such as a computer; a storage device 1108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1109. Communication means 1109 may allow computing device 1100 to communicate wirelessly or by wire with other devices to exchange data.
  • FIG. 11 shows a computing device 1100 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by the processor, the processor enables the processor to implement the video generation method or the video display method in the foregoing embodiments.
  • embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via the communication device 1109, or from the storage device 1108, or from the ROM 1102.
  • the processing device 1101 When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the video generation method or the video display method of the embodiment of the present disclosure are executed.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.
  • clients, servers can communicate using any currently known or future developed network protocol, such as HTTP, and can be interconnected with any form or medium of digital data communication (eg, a communication network).
  • a communication network examples include local area networks (“LAN”), wide area networks (“WAN”), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned computing device; or may exist alone without being assembled into the computing device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the computing device, the computing device is made to execute:
  • the personalized data is obtained; the generation request carrying the personalized data is sent to the second target device, and the generation request is used to make the second target device feedback the virtual gift video, the virtual gift video Generated by the fusion of personalized data and basic virtual image model; receive the virtual gift video fed back by the second target device; display the virtual gift video on the target gift tray page.
  • computer program code for performing operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional procedural programming languages - such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).
  • LAN local area network
  • WAN wide area network
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure may be implemented in a software manner, and may also be implemented in a hardware manner. Among them, the name of the unit does not constitute a limitation of the unit itself under certain circumstances.
  • exemplary types of hardware logic components include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical Devices (CPLDs) and more.
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLDs Complex Programmable Logical Devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Economics (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Accounting & Taxation (AREA)
  • Human Resources & Organizations (AREA)
  • General Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Tourism & Hospitality (AREA)
  • Data Mining & Analysis (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Primary Health Care (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Game Theory and Decision Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本公开涉及一种视频生成及显示方法、装置、设备、介质。其中,视频生成方法包括:接收第一目标设备发送的携带有个性化数据的生成请求;响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;根据虚拟形象图像,生成虚拟礼物视频;将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。根据本公开实施例,能够缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。

Description

视频生成及显示方法、装置、设备、介质
本公开要求于2020年11月20日提交中国专利局、申请号为202011309792.2、发明名称为“视频生成及显示方法、装置、设备、介质”的中国专利申请的优先权,其全部内容通过引用结合在本公开中。
技术领域
本公开涉及视频处理技术领域,尤其涉及一种视频生成及显示方法、装置、设备、介质。
背景技术
随着计算机技术和移动通信技术的迅速发展,基于电子设备的各种视频直播平台得到了普遍应用,极大地丰富了人们的日常生活。用户可以通过各种视频直播平台方便地观看视频直播,并且与视频直播的主播进行交互,例如向主播赠送虚拟礼物。
用户登录视频直播平台后,可以在虚拟礼物栏中定制专属的虚拟礼物,使视频直播平台能够按其想法和要求设计一个可以表达用户个性化情感的虚拟礼物。目前,视频直播平台一般通过人工方式设计虚拟礼物视频,制作周期较长、成本较高。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题,本公开提供了一种视频生成及显示方法、装置、设备、介质。
第一方面,本公开提供了一种视频生成方法,包括:
接收第一目标设备发送的携带有个性化数据的生成请求;
响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;
根据虚拟形象图像,生成虚拟礼物视频;
将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
第二方面,本公开提供了一种视频显示方法,包括:
当检测到用户输入的生成操作时,获取个性化数据;
将携带有个性化数据的生成请求发送至第二目标设备,生成请求用于使第二目标设备反馈虚拟礼物视频,虚拟礼物视频由个性化数据与基础虚拟形象模型融合生成;
接收第二目标设备反馈的虚拟礼物视频;
在目标礼物托盘页面内,显示虚拟礼物视频。
第三方面,本公开提供了一种视频生成装置,包括:
第一接收单元,配置为接收第一目标设备发送的携带有个性化数据的生成请求;
第一融合单元,配置为响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;
视频生成单元,配置为根据虚拟形象图像,生成虚拟礼物视频;
第一发送单元,配置为将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
第四方面,本公开提供了一种视频显示装置,包括:
第一获取单元,配置为当检测到用户输入的生成操作时,获取个性化数据;
第二发送单元,配置为将携带有个性化数据的生成请求发送至第二目标设备,生成请求用于使第二目标设备反馈虚拟礼物视频,虚拟礼物视频由个性化数据与基础虚拟形象模型融合生成;
第二接收单元,配置为接收第二目标设备反馈的虚拟礼物视频;
视频显示单元,配置为在目标礼物托盘页面内,显示虚拟礼物视频。
第五方面,本公开提供了一种计算设备,包括:
处理器;
存储器,用于存储可执行指令;
其中,处理器用于从存储器中读取可执行指令,并执行可执行指令以实现第一方面所述的视频生成方法或者第二方面所述的视频显示方法。
第六方面,本公开提供了一种计算机可读存储介质,该存储介质存储有计算机程序,当计算机程序被处理器执行时,使得处理器实现第一方面所述的视频生成方法或者第二方面所述的视频显示方法。
本公开实施例提供的技术方案与现有技术相比具有如下优点:
本公开实施例的视频生成及显示方法、装置、设备、介质,能够将个性化数据与基础虚拟形象模型进行融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成用于在目标礼物托盘页面内显示的虚拟礼物视频,以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
附图说明
结合附图并参考以下具体实施方式,本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中,相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的,原件和元素不一定按照比例绘制。
图1为本公开实施例提供的一种视频生成方法的应用环境架构图;
图2为本公开实施例提供的另一种视频生成方法的应用环境架构图;
图3为本公开实施例提供的一种视频生成方法的流程示意图;
图4为本公开实施例提供的一种虚拟礼物视频发送方法的流程示意图;
图5为本公开实施例提供的另一种虚拟礼物视频发送方法的流程示意图;
图6为本公开实施例提供的一种视频显示方法的流程示意图;
图7为本公开实施例提供的一种礼物托盘界面的示意图;
图8为本公开实施例提供的一种数据上传界面的示意图;
图9为本公开实施例提供的一种视频生成装置的结构示意图;
图10为本公开实施例提供的一种视频显示装置的结构示意图;
图11为本公开实施例提供的一种计算设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例,然而应当理解的是,本公开可以通过各种形式来实现,而且不应该被解释为限于这里阐述的实施例,相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是,本公开的附图及实施例仅用于示例性作用,并非用于限制本公开的保护范围。
应当理解,本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行,和/或并行执行。此外,方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括,即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”;术语“另一实施例”表示“至少一个另外的实施例”;术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意,本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分,并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意,本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的,本领域技术人员应当理解,除非在上下文另有明确指出,否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的,而并不是用于对这些消息或信息的范围进行限制。
本公开所提供的视频生成及显示方法可以应用于图1和图2所示的架构中,具体结合图1和图2进行详细说明。
图1示出了本公开实施例提供的一种视频生成方法的应用环境架构图。
如图1所示,该架构图中可以包括客户端的至少一个电子设备110和服务端的至少一个服务器120。电子设备110可以通过网络协议如超文本传输安全协议(Hyper Text Transfer Protocol over Secure Socket Layer,HTTPS)与服务器120建立连接并进行信息交互。其中,电子设备110可以是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备,也可以是虚拟机或者模拟器模拟的设备。服务器120可以是云服务器或者服务器集群等具有存储及计算功能的设备。
基于上述架构,用户可以在电子设备110上的视频直播平台内申请定制虚拟礼物视频。视频直播平台可以基于用户的定制申请,为其制作虚拟礼物视频。
由此,为了提高虚拟礼物视频的制作效率以及节约虚拟礼物视频的制作成本,可以通过如下方式对虚拟礼物视频进行生成。这里,以视频直播平台为视频直播应用程序进行说明。当电子设备110检测到用户输入的用于生成虚拟礼物视频的生成操作时,可以获取用户的个性化数据,并且将携带有个性化数据的生成请求发送至服务器120。服务器120在接收到电子设备110发送的携带有个性化数据的生成请求之后,可以将个性化数据与基础 虚拟形象模型融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成虚拟礼物视频,然后将虚拟礼物视频发送至电子设备110。电子设备110可以接收服务器120反馈的虚拟礼物视频,并且在目标礼物托盘页面内,显示虚拟礼物视频。
因此,图1所示的架构可以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
另外,本公开实施例提供的视频生成及显示方法除了可以应用在上述电子设备和服务器组成的架构中,还可以应用在多个电子设备组成的架构中,具体结合图2进行说明。
图2示出了本公开实施例提供的另一种视频生成方法的应用环境架构图。
如图2所示,该架构图中可以包括至少一个第一电子设备210和至少一个第二电子设备220。第一电子设备210可以通过网络协议如HTTPS与第二电子设备220建立连接并进行信息交互。其中,第一电子设备210和第二电子设备220可以分别是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备,也可以是虚拟机或者模拟器模拟的设备。
基于上述架构,使用第一电子设备210的用户可以通过第一电子设备210向使用第二电子设备220的用户发出虚拟礼物视频的定制申请。使用第二电子设备220的用户可以基于接收到的定制申请,为使用第一电子设备210的用户制作虚拟礼物视频。
由此,为了提高虚拟礼物视频的制作效率以及节约虚拟礼物视频的制作成本,可以通过如下方式对虚拟礼物视频进行生成。当第一电子设备210检测到用户输入的用于生成虚拟礼物视频的生成操作时,可以获取用户的个性化数据,并且将携带有个性化数据的生成请求发送至第二电子设备220。第二电子设备220在接收到第一电子设备210发送的携带有个性化数据的生成请求之后,可以将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成虚拟礼物视频,然后将虚拟礼物视频发送至第一电子设备210。第一电子设备210可以接收第二电子设备220反馈的虚拟礼物视频,并且在目标礼物托盘页面内,显示虚拟礼物视频。
因此,图2所示的架构可以实现对虚拟礼物视频的自动生成,在第二电子设备220接收到第一电子设备210发送的携带有个性化数据的生成请求之后,无需使用第二电子设备220的用户手动制作虚拟礼物视频,进而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
根据上述架构,下面结合图3-图8对本公开实施例提供的视频生成及显示方法进行说明。在一些实施例中,该视频显示方法可以由第一目标设备执行,第一目标设备可以为图1中所示的客户端中的电子设备110或者图2中所示的第一电子设备210。在另一些实施例中,该视频生成方法可以由第二目标设备执行,第二目标设备可以为图1中所示的服务端中的服务器120或者图2中所示的第二电子设备220。其中,电子设备110、第一电子设备210和第二电子设备220可以分别是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备,也可以是虚拟机或者模拟器模拟的设备。服务器120可以是云服务器或者服务器集群等具有存储及计算功能的设备。
图3示出了本公开实施例提供的一种视频生成方法的流程示意图。
如图3所示,该视频生成方法可以包括如下步骤。
S310、接收第一目标设备发送的携带有个性化数据的生成请求。
在用户想要定制虚拟礼物视频时,可以使用第一目标设备向第二目标设备发送携带有个性化数据的生成请求,该生成请求可以用于使第二目标设备自动根据个性化数据为用户生成定制的虚拟礼物视频。
在本公开一些实施例中,个性化数据可以包括用户信息、用户姿态图像和用户人脸图像中的至少一种。
可选地,用户信息可以包括用户标识、用户昵称和用户头像中的至少一种。
用户标识可以为用户账号、用户身份标识(Identity Document,ID)等用户的唯一标识码。
可选地,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
个性化数据可以包括用户的至少一个身体姿态图像。当个性化数据包括用户的两个以上身体姿态图像时,两个以上身体姿态图像中的身体姿态可以形成连续身体动作,两个以上身体姿态图像也可以分别包含独立的身体姿态。具体地,用户身体姿态图像可以包括用户的头部、颈部、四肢和躯干的姿态动作。
在两个以上身体姿态图像中的身体姿态可以形成连续身体动作的情况下,两个以上身体姿态图像可以为用户上传的身体动作视频中的连续图像帧,该身体动作视频可以作为个性化数据,使得第二目标设备可以从身体动作视频中获取用户的两个以上身体姿态图像。
个性化数据可以包括用户的至少一个面部姿态图像。当个性化数据包括用户的两个以上面部姿态图像时,两个以上面部姿态图像中的面部姿态可以形成连续面部动作,两个以上面部姿态图像也可以分别包含独立的面部姿态。具体地,用户面部姿态图像可以包括用户的眉毛、眼睛、鼻子、嘴巴和面部肌肉的姿态。
在两个以上面部姿态图像中的面部姿态可以形成连续面部动作的情况下,两个以上面部姿态图像可以为用户上传的面部动作视频中的连续图像帧,该面部动作视频可以作为个性化数据,使得第二目标设备可以从面部动作视频中获取用户的两个以上面部姿态图像。
另外,还可以将用户的全身动作视频作为个性化数据,使得第二目标设备可以从全身动作视频中获取用户的两个以上全身姿态图像,全身姿态图像可以包括面部姿态和身体姿态。
S320、响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像。
例如,在第二目标设备接收到生成请求之后,可以响应于生成请求,首先获取预先设置的基础虚拟形象模型,然后,将生成请求所携带的个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象模型,再利用融合后的虚拟形象模型生成融合后的虚拟形象图像。
在本公开一些实施例中,在个性化数据包括用户姿态图像,并且用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种的情况下,S320可以具体包括:
利用用户姿态图像对基础虚拟形象模型进行姿态迁移,生成融合后的虚拟形象图像。
在一些实施例中,在用户姿态图像包括用户身体姿态图像的情况下,可以利用形变迁移技术(如三角网格的形变迁移技术等),将每个身体姿态图像中的身体姿态分别迁移至基础虚拟形象模型,得到每个身体姿态图像对应的融合后的虚拟形象模型,即具有每个身体姿态图像中的身体姿态的虚拟形象模型,进而利用每个身体姿态图像对应的融合后的虚拟形象模型生成该身体姿态图像对应的融合后的虚拟形象图像。
具体地,基础虚拟形象模型可以预先设置有多个预设身体姿态。第二目标设备在接收到生成请求之后,可以首先利用每个身体姿态图像分别建立用户的网络模型,得到每个身体姿态图像对应的源模型。然后,可以选取具有与任一个身体姿态图像中的身体姿态相近的预设身体姿态的基础虚拟形象模型,并构建该基础虚拟形象模型的网格模型,得到目标模型。接着,可以确定身体姿态相近的源模型与目标模型之间的形变关系。最后,可以基于所确定的形变关系,利用形变迁移技术,将每个源模型的姿态分别迁移到目标模型上,即可得到融合后的虚拟形象模型。
由此,可以将利用虚拟形象模型生成的虚拟礼物视频中的虚拟形象的身体姿态与申请定制虚拟礼物视频的用户相关联,提升用户的体验。
在另一些实施例中,在用户姿态图像包括用户面部姿态图像的情况下,可以利用人脸表情动画生成技术(如基于Blendshape的人脸表情动画生成技术等),将每个面部姿态图像中的面部姿态分别映射至基础虚拟形象模型的人脸上,得到每个面部姿态图像对应的融合后的虚拟形象模型,即具有每个面部姿态图像中的面部姿态的虚拟形象模型,进而利用每个面部姿态图像对应的融合后的虚拟形象模型生成该面部姿态图像对应的融合后的虚拟形象图像。
具体地,在第二目标设备接收到生成请求之前,可以首先采集不同用户的不同面部姿态图像,然后将每个用户的面部姿态图像中的面部姿态作为该用户的表情基,并且提取每个面部姿态图像中的人脸特征点。然后,针对基础虚拟形象模型的人脸对每个面部姿态图像中的人脸特征点进行网格变形拟合,使得拟合得到的每个图像中的人脸特征点既能够对齐基础虚拟形象模型的人脸特征点坐标,又能够对齐深度位置,并且从这些拟合得到的图像中的人脸网格出发,为每个用户构造一组具有个性化的表情混合形状的人脸网格,将所构造的网格组合成一个秩三张量,建立了一个具有身份和表情两种属性的双线性人脸模型。在第二目标设备接收到生成请求之后,可以获取生成请求中携带的面部姿态图像,然后通过双线性人脸模型,对生成请求中携带的面部姿态图像的人脸网格进行融合,构建生成请求中携带的面部姿态图像中的用户的人脸数据库,进而在人脸数据库寻找与该面部姿态图像中的人脸网格误差最小的人脸模型。最后,可以通过寻找到的人脸模型,确定生成请求中携带的面部姿态图像中的人脸的表情基和表情基控制系数,并将表情基控制系数作用在基础虚拟形象模型的表情基上,从而将生成请求中携带的每个面部姿态图像中的面部姿态分别映射至基础虚拟形象模型的人脸上。
由此,可以将利用虚拟形象模型生成的虚拟礼物视频中的虚拟形象的面部姿态与申请定制虚拟礼物视频的用户相关联,提升用户的体验。
在本公开另一些实施例中,在个性化数据包括用户人脸图像的情况下,S320可以具体 包括:
利用用户人脸图像对基础虚拟形象模型进行换脸,生成融合后的虚拟形象图像。
在一些实施例中,可以利用三维(3Dimensions,3D)人脸重建技术(如基于弱监督学习的3D人脸重建技术等),自动将用户人脸图像中的人脸替换至基础虚拟形象模型上,得到融合后的虚拟形象图像,即具有用户人脸图像中的人脸的虚拟形象模型,进而利用具有用户人脸图像中的人脸的虚拟形象模型生成融合后的虚拟形象图像。
在另一些实施例中,可以在FaceBuilder插件中,利用用户人脸图像对基础虚拟形象模型进行人脸模型的重新创建,使得创建后的人脸模型可以达到影视级效果。
由此,可以将利用虚拟形象模型生成的虚拟礼物视频中的虚拟形象的人脸与申请定制虚拟礼物视频的用户相关联,提升用户的体验。
在本公开又一些实施例中,在个性化数据包括用户信息,并且用户信息包括用户标识、用户昵称和用户头像中的至少一种的情况下,S320可以具体包括:
生成基础虚拟形象模型对应的虚拟形象图像;
按照预设的信息添加方式,将用户信息添加至虚拟形象图像的预设位置,生成融合后的虚拟形象图像。
其中,基础虚拟形象模型可以预先设置有多个预设姿态组合,预设姿态组合可以包括多个预设姿态,多个预设姿态可以形成连续动作如点赞动作和打招呼动作等,预设姿态可以包括预设身体姿态和预设面部姿态中的至少一种。第二目标设备在接收到生成请求之后,可以首先利用基础虚拟形象模型生成不同预设姿态组合下的多个虚拟形象图像,然后,按照预设的信息添加方式,将用户信息添加至虚拟形象图像的预设位置,生成融合后的虚拟形象图像。
具体地,预设的信息添加方式可以包括预设的字体颜色、预设的字体背景和预设的显示效果中的至少一种。预设位置可以为虚拟形象图像中的任意位置,可以根据需要进行设置。
由此,可以将利用虚拟形象模型生成的虚拟礼物视频中的虚拟形象的用户信息与申请定制虚拟礼物视频的用户信息相关联,将用户信息外露,增加用户的参与感,进而进一步提升用户的体验。
在本公开再一些实施例中,在个性化数据可以包括用户信息、用户姿态图像和用户人脸图像中的两个以上的情况下,可以分别按照每种个性化数据与基础虚拟形象模型的融合方法分别对将其融合至基础虚拟形象模型中,得到融合后的虚拟形象图像。
在本公开实施例中,基础虚拟形象模型可以为卡通形象模型和虚拟人物模型中的至少一种,下面分别以基础虚拟形象模型为卡通形象模型和基础虚拟形象模型为虚拟人物模型为例进行说明。
在基础虚拟形象模型为卡通形象模型的情况下,所定制的虚拟礼物视频可以为小型礼物视频,此时,用户可以向第二目标设备发送携带有身体姿态图像的生成请求,身体姿态图像中的身体姿态可以形成连续身体动作,第二目标设备可以响应于生成请求,自动将身体姿态图像中的身体姿态迁移至卡通形象模型,生成融合后的虚拟形象图像。
在基础虚拟形象模型为虚拟人物模型的情况下,所定制的虚拟礼物视频可以为大型礼物视频,此时,用户可以向第二目标设备发送携带有身体姿态图像、面部姿态图像和用户人脸图像的生成请求,身体姿态图像中的身体姿态可以形成连续身体动作,面部姿态图像中的面部姿态可以形成连续面部动作,第二目标设备可以响应于生成请求,自动将虚拟人物模型的人脸替换为用户人脸图像中的人脸,然后将身体姿态图像中的身体姿态和面部姿态图像中的面部姿态迁移至卡通形象模型,最后生成融合后的虚拟形象图像。
S330、根据虚拟形象图像,生成虚拟礼物视频。
在本公开一些实施例中,第二目标设备可以对虚拟形象图像进行渲染,并将渲染后的虚拟形象图像组合并连续播放,生成虚拟形象做出连贯动作的虚拟礼物视频。
在本公开另一些实施例中,还可以获取音频数据,并利用虚拟形象图像和音频数据,生成虚拟礼物视频。
具体地,音频数据可以为用户指定的通过生成请求携带的音频数据,也可以为预先设置的音频数据,在此不作限制。
第二目标设备可以获取音频数据,然后对虚拟形象图像进行渲染,并将渲染后的虚拟形象图像组合并连续播放,生成虚拟形象做出连贯动作的虚拟礼物视频,最后根据虚拟礼物视频中帧图像和音频数据的时序,将音频数据加入虚拟礼物视频。
S340、将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
在本公开实施例中,第二目标设备在生成虚拟礼物视频之后,可以将虚拟礼物视频发送至第一目标设备,使得第一目标设备可以在目标礼物托盘页面内显示接收到的虚拟礼物视频。
在本公开实施例中,能够将个性化数据与基础虚拟形象模型进行融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成用于在目标礼物托盘页面内显示的虚拟礼物视频,以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
在本公开另一种实施方式中,在S340之后,当使用第一目标设备的用户申请向目标直播间的主播赠送定制好的虚拟礼物视频时,第二目标设备可以使正在显示该目标直播间的直播视频的电子设备显示该虚拟礼物视频。
图4示出了本公开实施例提供的一种虚拟礼物视频发送方法的流程示意图。
在本公开一些实施例中,该虚拟礼物视频发送方法还可以包括如下步骤。
S410、接收第一目标设备发送的携带有个性化数据的生成请求。
S420、响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像。
S430、根据虚拟形象图像,生成虚拟礼物视频。
S440、将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
其中,S410-S440与图3所示的S310-S340相似,在此不做赘述。
S450、接收第一目标设备发送的针对目标直播间的互动数据,互动数据包括虚拟礼物视频对应的视频信息。
在用户想要在目标直播间内发送该虚拟礼物视频时,可以使用第一目标设备向第二目标设备发送虚拟礼物视频对应的视频信息,该视频信息可以携带有目标直播间的直播间ID,使得第二目标设备知晓虚拟礼物视频的发送对象。
在一些实施例中,视频信息可以包括虚拟礼物视频的视频ID和虚拟礼物视频的视频名称中的至少一种。
在另一些实施例中,互动数据还可以包括虚拟礼物视频的赠送时间。
在又一些实施例中,互动数据还可以包括用户评论信息、用户操作信息等。
S460、获取视频信息对应的虚拟礼物视频。
在本公开实施例中,在第二目标设备接收到视频信息之后,可以在预先存储的多个虚拟礼物视频中,查询该视频信息对应的虚拟礼物视频。
S470、将虚拟礼物视频与目标直播间的直播视频融合,形成融合后的直播视频。
在本公开实施例中,第二目标设备可以根据直播视频中帧图像和虚拟礼物视频中帧图像的时序,将虚拟礼物视频加入直播视频。
在一些实施例中,可以根据虚拟礼物视频的赠送时间确定直播视频中用于加入虚拟礼物视频的首个帧图像,然后根据直播视频中帧图像和虚拟礼物视频中帧图像的时序,将每个虚拟礼物视频中的帧图像添加至对应的直播视频中的帧图像中的目标位置,以将虚拟礼物视频加入直播视频。
其中,目标位置可以为图像中部、图像底部、图像左侧或图像右侧,在此不作限制。
S480、向与目标直播间相关联的电子设备发送融合后的直播视频。
在本公开实施例中,第二目标设备可以在形成融合后的直播视频之后,向与目标直播间相关联的电子设备发送融合后的直播视频。
其中,与目标直播间相关联的电子设备可以为正在获取目标直播间的直播视频资源的电子设备,也可以理解为与目标直播间相关联的电子设备为正在显示该目标直播间的直播视频的电子设备,即与目标直播间相关联的电子设备为目标直播间的观众所使用的电子设备。
由此,第二目标设备在接收到向目标直播间的主播赠送定制好的虚拟礼物视频的申请时,可以将虚拟礼物视频跟随主播直播流分发,节省内容分发网络的流量。
图5示出了本公开实施例提供的另一种虚拟礼物视频发送方法的流程示意图。
在本公开另一些实施例中,该虚拟礼物视频发送方法还可以包括如下步骤。
S510、接收第一目标设备发送的携带有个性化数据的生成请求。
S520、响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像。
S530、根据虚拟形象图像,生成虚拟礼物视频。
S540、将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
其中,S510-S540与图3所示的S310-S340相似,在此不做赘述。
S550、接收第一目标设备发送的针对目标直播间的互动数据,互动数据包括虚拟礼物视频对应的视频信息。
其中,S550与图4所示的S450相似,在此不做赘述。
S560、向与目标直播间相关联的电子设备发送礼物数据,礼物数据包括第一目标设备对应的用户信息和虚拟礼物视频对应的视频信息。
在本公开实施例中,在第二目标设备接收到视频信息之后,可以直接向与目标直播间相关联的电子设备发送礼物数据,使得与目标直播间相关联的电子设备可以根据礼物数据,在目标直播间的直播视频显示界面内显示虚拟礼物视频和赠送虚拟礼物视频的用户的用户信息。
具体地,与目标直播间相关联的电子设备可以在接收到虚拟礼物视频对应的视频信息之后,在预先存储的多个虚拟礼物视频中,查询该视频信息对应的虚拟礼物视频,并且在目标直播间的直播视频内容上叠加播放查询到的虚拟礼物视频。
在本公开另一些实施例中,礼物数据还可以包括虚拟礼物视频的赠送时间,使得与目标直播间相关联的电子设备可以根据礼物数据,在目标直播间的直播视频显示界面内,按照虚拟礼物视频的赠送时间显示虚拟礼物视频和赠送虚拟礼物视频的用户的用户信息。
例如,当与目标直播间相关联的电子设备接收到多个礼物数据时,可以根据虚拟礼物视频的赠送时间对多个礼物数据对应的虚拟礼物视频的显示顺序进行排序,然后按照排序后的显示顺序依次显示各个虚拟礼物视频。
在本公开实施例中,为了使得与目标直播间相关联的电子设备可以基于虚拟礼物视频对应的视频信息显示虚拟礼物视频,在S540之后、在S560之前,该视频生成方法还可以包括:向与目标直播间相关联的电子设备发送虚拟礼物视频,使与目标直播间相关联的电子设备存储该虚拟礼物视频。
可选地,该虚拟礼物视频发送方法还可以包括:向与目标直播间相关联的电子设备发送虚拟礼物视频对应的视频信息,使与目标直播间相关联的电子设备存储该虚拟礼物视频对应的视频信息。
由此,第二目标设备在接收到向目标直播间的主播赠送定制好的虚拟礼物视频的申请时,可以单独分发虚拟礼物视频,以提高虚拟礼物视频的显示层级,使得虚拟礼物视频不被其它内容所遮挡,保证虚拟礼物视频的显示效果。
图6示出了本公开实施例提供的一种视频显示方法的流程示意图。
如图6所示,该视频显示方法可以包括如下步骤。
S610、当检测到用户输入的生成操作时,获取个性化数据。
在用户想要定制虚拟礼物视频时,可以在第一目标设备内输入用于生成虚拟礼物视频的生成操作。
其中,生成操作可以包括对用于触发生成虚拟礼物视频的控件的触发操作。具体地,触发操作可以为点击操作、双击操作、长按操作或者语音操作等。
例如,用于触发生成虚拟礼物视频的控件可以为目标礼物托盘页面内的用于触发生成 虚拟礼物视频的生成按钮。
其中,目标礼物托盘页面可以为视频直播平台(如视频直播应用程序内)的定制礼物托盘页面,也可以为视频直播平台的任意直播间的直播视频显示界面内的定制礼物托盘页面,还可以为其他具有虚拟礼物视频定制功能的平台的任意界面内的定制礼物托盘页面,在此不作限制。
在第一目标设备检测到用户输入的生成操作之后,可以获取用户上传的个性化数据。
其中,个性化数据可以包括用户信息、用户姿态图像和用户人脸图像中的至少一种。
可选地,用户信息可以包括用户标识、用户昵称和用户头像中的至少一种。
可选地,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
需要说明的是,个性化数据已在图3所示的实施例中说明,在此不做赘述。
S620、将携带有个性化数据的生成请求发送至第二目标设备。
在本公开实施例中,第一目标设备可以利用个性化数据生成携带有个性化数据的生成请求,并且将携带有个性化数据的生成请求发送至第二目标设备。
其中,生成请求用于使第二目标设备反馈虚拟礼物视频,虚拟礼物视频由个性化数据与基础虚拟形象模型融合生成,已在图3所示的实施例中进行说明,在此不做赘述。
S630、接收第二目标设备反馈的虚拟礼物视频。
在本公开实施例中,第一目标设备可以接收第二目标设备反馈的虚拟礼物视频。
可选地,第二目标设备还可以向第一目标设备反馈虚拟礼物视频对应的视频信息,第一目标设备可以接收第二目标设备反馈虚拟礼物视频对应的视频信息,并且存储该虚拟礼物视频对应的视频信息。
可选地,第二目标设备还可以向第一目标设备反馈虚拟礼物视频对应的虚拟礼物预览图像。
S640、在目标礼物托盘页面内,显示虚拟礼物视频。
在本公开实施例中,可选地,第一目标设备可以将接收到的虚拟礼物视频的虚拟礼物预览图像显示于目标礼物托盘页面内。
图7示出了本公开实施例提供的一种礼物托盘界面的示意图。如图7所示,第一目标设备显示有视频直播平台的目标直播间的直播视频显示界面701,直播视频显示界面701内可以显示定制礼物托盘页面702,定制礼物托盘页面702中可以显示接收到的虚拟礼物视频的虚拟礼物预览图像703,以将定制的虚拟礼物视频与其他虚拟礼物视频进行区分。
在本公开实施例中,能够将个性化数据与基础虚拟形象模型进行融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成用于在目标礼物托盘页面内显示的虚拟礼物视频,以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
在本公开另一种实现方式中,在个性化数据包括用户姿态图像和用户人脸图像中的至少一种,并且用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种的情况下,S610可以具体包括:
当检测到用户输入的生成操作时,获取该用户输入的包含个性化数据的用户视频或用 户图像。
在一些实施例中,在获取用户输入的包含个性化数据的用户视频或用户图像之前,可以显示数据上传界面,使用户可以在数据上传界面内拍摄或者选取包含个性化数据的用户视频或用户图像,进而向第一目标设备输入包含个性化数据的用户视频或用户图像。
图8示出了本公开实施例提供的一种数据上传界面的示意图。如图8所示,第一目标设备显示有数据上传界面801,数据上传界面801内可以显示有拍摄用户视频或用户图像的预览窗口802和用于触发拍摄用户视频或用户图像的目标按钮803。用户可以点击目标按钮803拍摄用户图像,并在预览窗口802内预览拍摄的用户图像;用户也可以长按目标按钮803拍摄用户视频,并在预览窗口802内预览拍摄的用户视频。
可选地,用户姿态图像可以为用户视频中的图像帧或者用户图像,用户人脸图像也可以为用户视频中的图像帧或者用户图像。即用户视频和用户图像可以分别包括用户姿态图像和用户人脸图像中的至少一种。
在另一些实施例中,在S610之前,可以显示数据上传界面,使用户可以在数据上传界面内拍摄或者选取包含个性化数据的用户视频或用户图像,进而向第一目标设备输入包含个性化数据的用户视频或用户图像,以使第一目标设备在检测到用户输入的生成操作时,可以获取用户已经输入的个性化数据。
其中,用户输入包含个性化数据的用户视频或用户图像的方法已在上文说明,在此不做赘述。
由此,在本公开实施例中,可以使用户更方便、快捷地上传个性化数据。
在本公开实施例中,在S640之后,在用户想要定制虚拟礼物视频时,可以在目标礼物托盘页面内,针对所显示的虚拟礼物视频输入发送操作,使得第一目标设备在检测到针对虚拟礼物视频的发送操作时,向第二目标设备发送虚拟礼物视频对应的视频信息。
具体地,发送操作可以为对虚拟礼物视频的点击操作、双击操作、长按操作或者语音操作等。
可选地,第一目标设备在检测到针对虚拟礼物视频的发送操作之后,还可以在当前界面内播放虚拟礼物视频。
具体地,第一目标设备在检测到针对虚拟礼物视频的发送操作之后,可以获取虚拟礼物视频的视频信息,并在预先存储的多个虚拟礼物视频中,查询该视频信息对应的虚拟礼物视频,进而播放查询到的虚拟礼物视频。
继续以第一目标设备显示有目标直播间的直播视频显示界面为例,用户可以在直播视频显示界面的目标礼物托盘页面内点击虚拟礼物视频,以使第一目标设备向第二目标设备发送该虚拟礼物视频对应的视频信息,并且在预先存储的多个虚拟礼物视频中,查询该视频信息对应的虚拟礼物视频,然后在目标直播间的直播视频内容上叠加播放查询到的虚拟礼物视频。
由此,可以在用户向主播赠送虚拟礼物视频后,在用户使用的电子设备上实现对该虚拟礼物视频的播放。
图9示出了本公开实施例提供的一种视频生成装置的结构示意图。
在一些实施例中,该视频生成装置900可以设置于上述第二目标设备中,第二目标设备可以为图1中所示的服务端中的服务器120或者图2中所示的第二电子设备220。其中,第二电子设备220可以是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备,也可以是虚拟机或者模拟器模拟的设备。服务器120可以是云服务器或者服务器集群等具有存储及计算功能的设备。
如图9所示,该视频生成装置900可以包括第一接收单元910、第一融合单元920、视频生成单元930和第一发送单元940。
该第一接收单元910可以配置为接收第一目标设备发送的携带有个性化数据的生成请求。
该第一融合单元920可以配置为响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像。
该视频生成单元930可以配置为根据虚拟形象图像,生成虚拟礼物视频。
该第一发送单元940可以配置为将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
在本公开实施例中,能够将个性化数据与基础虚拟形象模型进行融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成用于在目标礼物托盘页面内显示的虚拟礼物视频,以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
在本公开一些实施例中,个性化数据可以包括用户信息、用户姿态图像和用户人脸图像中的至少一种。
可选地,用户信息可以包括用户标识、用户昵称和用户头像中的至少一种。
可选地,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
在一些实施例中,个性化数据可以包括用户姿态图像,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
相应地,该第一融合单元920可以进一步配置为:
利用用户姿态图像对基础虚拟形象模型进行姿态迁移,生成融合后的虚拟形象图像。
在一些实施例中,个性化数据可以包括用户人脸图像。
相应地,该第一融合单元920可以进一步配置为:
利用用户人脸图像对基础虚拟形象模型进行换脸,生成融合后的虚拟形象图像。
在一些实施例中,个性化数据可以包括用户信息,用户信息可以包括用户标识、用户昵称和用户头像中的至少一种。
相应地,该第一融合单元920可以进一步配置为:
生成基础虚拟形象模型对应的虚拟形象图像;
按照预设的信息添加方式,将用户信息添加至虚拟形象图像的预设位置,生成融合后的虚拟形象图像。
在本公开一些实施例中,该视频生成装置900还可以包括第三接收单元、第二获取单元、第二融合单元和第三发送单元。
该第三接收单元可以配置为接收第一目标设备发送的针对目标直播间的互动数据,互动数据包括虚拟礼物视频对应的视频信息。
该第二获取单元可以配置为获取视频信息对应的虚拟礼物视频。
该第二融合单元可以配置为将虚拟礼物视频与目标直播间的直播视频融合,形成融合后的直播视频。
该第三发送单元可以配置为向与目标直播间相关联的电子设备发送融合后的直播视频。
在本公开另一些实施例中,该视频生成装置900还可以包括第四接收单元和第四发送单元。
该第四接收单元可以配置为接收第一目标设备发送的针对目标直播间的互动数据,互动数据包括虚拟礼物视频对应的视频信息。
该第四发送单元可以配置为向与目标直播间相关联的电子设备发送礼物数据,礼物数据包括第一目标设备对应的用户信息和虚拟礼物视频对应的视频信息。
需要说明的是,图9所示的视频生成装置900可以执行图3至图5所示的方法实施例中的各个步骤,并且实现图3至图5所示的方法实施例中的各个过程和效果,在此不做赘述。
图10示出了本公开实施例提供的一种视频显示装置的结构示意图。
在一些实施例中,该视频显示装置1000可以设置于第一目标设备中,第一目标设备可以为图1中所示的客户端中的电子设备110或者图2中所示的第一电子设备210。其中,电子设备110和第一电子设备210可以是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备,也可以是虚拟机或者模拟器模拟的设备。
如图10所示,该视频显示装置1000可以包括第一获取单元1010、第二发送单元1020、第二接收单元1030和视频显示单元1040。
该第一获取单元1010可以配置为当检测到用户输入的生成操作时,获取个性化数据。
该第二发送单元1020可以配置为将携带有个性化数据的生成请求发送至第二目标设备,生成请求用于使第二目标设备反馈虚拟礼物视频,虚拟礼物视频由个性化数据与基础虚拟形象模型融合生成。
该第二接收单元1030可以配置为接收第二目标设备反馈的虚拟礼物视频。
该视频显示单元1040可以配置为在目标礼物托盘页面内,显示虚拟礼物视频。
在本公开实施例中,能够将个性化数据与基础虚拟形象模型进行融合,得到融合后的虚拟形象图像,并且根据虚拟形象图像,生成用于在目标礼物托盘页面内显示的虚拟礼物视频,以实现对虚拟礼物视频的自动生成,从而可以缩短虚拟礼物视频的制作周期,并且节约虚拟礼物视频的制作成本。
在本公开一些实施例中,个性化数据可以包括用户信息、用户姿态图像和用户人脸图像中的至少一种。
可选地,用户信息可以包括用户标识、用户昵称和用户头像中的至少一种。
可选地,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
在本公开一些实施例中,个性化数据可以包括用户姿态图像和用户人脸图像中的至少一种,用户姿态图像可以包括用户身体姿态图像和用户面部姿态图像中的至少一种。
相应地,该第一获取单元1010进一步配置为:
当检测到用户输入的生成操作时,获取所述用户输入的包含个性化数据的用户视频。
需要说明的是,图10所示的视频显示装置1000可以执行图6至图8所示的方法实施例中的各个步骤,并且实现图6至图8所示的方法实施例中的各个过程和效果,在此不做赘述。
本公开实施例还提供了一种计算设备,该计算设备可以包括处理器和存储器,存储器可以用于存储可执行指令。其中,处理器可以用于从存储器中读取可执行指令,并执行可执行指令以实现上述实施例中的视频生成方法或视频显示方法。
图11示出了本公开实施例提供的一种计算设备的结构示意图。下面具体参考图11,其示出了适于用来实现本公开实施例中的计算设备1100的结构示意图。
本公开实施例中的计算设备1100可以为执行视频显示方法的第一目标设备,也可以为执行视频生成方法的第二目标设备。其中,第一目标设备可以为电子设备,如图1中所示的客户端中的电子设备110或者图2中所示的第一电子设备210。第二目标设备可以为电子设备或服务器,如图1中所示的服务端中的服务器120或者图2中所示的第二电子设备220。电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)、可穿戴设备、等等的移动终端以及诸如数字TV、台式计算机、智能家居设备等等的固定终端。服务器可以是云服务器或者服务器集群等具有存储及计算功能的设备。
需要说明的是,图11示出的计算设备1100仅仅是一个示例,不应对本公开实施例的功能和使用范围带来任何限制。
如图11所示,该计算设备1100可以包括处理装置(例如中央处理器、图形处理器等)1101,其可以根据存储在只读存储器(ROM)1102中的程序或者从存储装置1108加载到随机访问存储器(RAM)1103中的程序而执行各种适当的动作和处理。在RAM 1103中,还存储有计算设备1100操作所需的各种程序和数据。处理装置1101、ROM 1102以及RAM1103通过总线1104彼此相连。输入/输出(I/O)接口1105也连接至总线1104。
通常,以下装置可以连接至I/O接口1105:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1106;包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置1107;包括例如磁带、硬盘等的存储装置1108;以及通信装置1109。通信装置1109可以允许计算设备1100与其他设备进行无线或有线通信以交换数据。虽然图11示出了具有各种装置的计算设备1100,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
本公开实施例还提供了一种计算机可读存储介质,该存储介质存储有计算机程序,当计算机程序被处理器执行时,使得处理器实现用上述实施例中的视频生成方法或视频显示方法。
特别地,根据本公开的实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的实施例包括一种计算机程序产品,其包括承载在非暂态计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中,该计算机程序可以通过通信装置1109从网络上被下载和安装,或者从存储装置1108被安装,或者从ROM 1102被安装。在该计算机程序被处理装置1101执行时,执行本公开实施例的视频生成方法或视频显示方法中限定的上述功能。
需要说明的是,本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的***、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行***、装置或者器件使用或者与其结合使用。而在本公开中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令执行***、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。
在一些实施方式中,客户端、服务器可以利用诸如HTTP之类的任何当前已知或未来研发的网络协议进行通信,并且可以与任意形式或介质的数字数据通信(例如,通信网络)互连。通信网络的示例包括局域网(“LAN”),广域网(“WAN”),网际网(例如,互联网)以及端对端网络(例如,ad hoc端对端网络),以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述计算设备中所包含的;也可以是单独存在,而未装配入该计算设备中。
上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该计算设备执行时,使得该计算设备执行:
接收第一目标设备发送的携带有个性化数据的生成请求;响应于生成请求,将个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;根据虚拟形象图像,生成虚拟礼物视频;将虚拟礼物视频发送至第一目标设备,虚拟礼物视频用于在第一目标设备的目标礼物托盘页面内显示。
或者,当检测到用户输入的生成操作时,获取个性化数据;将携带有个性化数据的生成请求发送至第二目标设备,生成请求用于使第二目标设备反馈虚拟礼物视频,虚拟礼物视频由个性化数据与基础虚拟形象模型融合生成;接收第二目标设备反馈的虚拟礼物视频;在目标礼物托盘页面内,显示虚拟礼物视频。
在本公开实施例中,可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码,上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图,图示了按照本公开各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的***来实现,或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。其中,单元的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如,非限制性地,可以使用的示范类型的硬件逻辑部件包括:现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上***(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中,机器可读介质可以是有形的介质,其可以包含或存储以供指令执行***、装置或设备使用或与指令执行***、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体***、装置或设备,或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开中所涉及的公开范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述公开构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外,虽然采用特定次序描绘了各操作,但是这不应当理解为要求这些操作以所示出 的特定次序或以顺序次序执行来执行。在一定环境下,多任务和并行处理可能是有利的。同样地,虽然在上面论述中包含了若干具体实现细节,但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地,在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题,但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反,上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (14)

  1. 一种视频生成方法,其特征在于,包括:
    接收第一目标设备发送的携带有个性化数据的生成请求;
    响应于所述生成请求,将所述个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;
    根据所述虚拟形象图像,生成虚拟礼物视频;
    将所述虚拟礼物视频发送至所述第一目标设备,所述虚拟礼物视频用于在所述第一目标设备的目标礼物托盘页面内显示。
  2. 根据权利要求1所述的方法,其特征在于,所述个性化数据包括用户信息、用户姿态图像和用户人脸图像中的至少一种;
    其中,所述用户信息包括用户标识、用户昵称和用户头像中的至少一种,所述用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种。
  3. 根据权利要求1所述的方法,其特征在于,所述个性化数据包括用户姿态图像,所述用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种;
    所述将所述个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像,包括:
    利用所述用户姿态图像对所述基础虚拟形象模型进行姿态迁移,生成所述融合后的虚拟形象图像。
  4. 根据权利要求1所述的方法,其特征在于,所述个性化数据包括用户人脸图像;
    所述将所述个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像,包括:
    利用所述用户人脸图像对所述基础虚拟形象模型进行换脸,生成所述融合后的虚拟形象图像。
  5. 根据权利要求1所述的方法,其特征在于,所述个性化数据包括用户信息,所述用户信息包括用户标识、用户昵称和用户头像中的至少一种;
    所述将所述个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像,包括:
    生成所述基础虚拟形象模型对应的虚拟形象图像;
    按照预设的信息添加方式,将所述用户信息添加至所述虚拟形象图像的预设位置,生成所述融合后的虚拟形象图像。
  6. 根据权利要求1所述的方法,其特征在于,在所述将所述虚拟礼物视频发送至所述第一目标设备之后,所述方法还包括:
    接收所述第一目标设备发送的针对目标直播间的互动数据,所述互动数据包括所述虚拟礼物视频对应的视频信息;
    获取所述视频信息对应的所述虚拟礼物视频;
    将所述虚拟礼物视频与所述目标直播间的直播视频融合,形成融合后的直播视频;
    向与所述目标直播间相关联的电子设备发送所述融合后的直播视频。
  7. 根据权利要求1所述的方法,其特征在于,在所述将所述虚拟礼物视频发送至所述第一目标设备之后,所述方法还包括:
    接收所述第一目标设备发送的针对目标直播间的互动数据,所述互动数据包括所述虚 拟礼物视频对应的视频信息;
    向与所述目标直播间相关联的电子设备发送礼物数据,所述礼物数据包括所述第一目标设备对应的用户信息和所述虚拟礼物视频对应的视频信息。
  8. 一种视频显示方法,其特征在于,包括:
    当检测到用户输入的生成操作时,获取个性化数据;
    将携带有所述个性化数据的生成请求发送至第二目标设备,所述生成请求用于使所述第二目标设备反馈虚拟礼物视频,所述虚拟礼物视频由所述个性化数据与基础虚拟形象模型融合生成;
    接收所述第二目标设备反馈的所述虚拟礼物视频;
    在目标礼物托盘页面内,显示所述虚拟礼物视频。
  9. 根据权利要求8所述的方法,其特征在于,所述个性化数据包括用户信息、用户姿态图像和用户人脸图像中的至少一种;
    其中,所述用户信息包括用户标识、用户昵称和用户头像中的至少一种,所述用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种。
  10. 根据权利要求8所述的方法,其特征在于,所述个性化数据包括用户姿态图像和用户人脸图像中的至少一种,所述用户姿态图像包括用户身体姿态图像和用户面部姿态图像中的至少一种;
    所述当检测到用户输入的生成操作时,获取个性化数据,包括:
    当检测到用户输入的生成操作时,获取所述用户输入的包含所述个性化数据的用户视频。
  11. 一种视频生成装置,其特征在于,包括:
    第一接收单元,配置为接收第一目标设备发送的携带有个性化数据的生成请求;
    第一融合单元,配置为响应于所述生成请求,将所述个性化数据与基础虚拟形象模型融合,得到融合后的虚拟形象图像;
    视频生成单元,配置为根据所述虚拟形象图像,生成虚拟礼物视频;
    第一发送单元,配置为将所述虚拟礼物视频发送至所述第一目标设备,所述虚拟礼物视频用于在所述第一目标设备的目标礼物托盘页面内显示。
  12. 一种视频显示装置,其特征在于,包括:
    第一获取单元,配置为当检测到用户输入的生成操作时,获取个性化数据;
    第二发送单元,配置为将携带有所述个性化数据的生成请求发送至第二目标设备,所述生成请求用于使所述第二目标设备反馈虚拟礼物视频,所述虚拟礼物视频由所述个性化数据与基础虚拟形象模型融合生成;
    第二接收单元,配置为接收所述第二目标设备反馈的所述虚拟礼物视频;
    视频显示单元,配置为在目标礼物托盘页面内,显示所述虚拟礼物视频。
  13. 一种计算设备,其特征在于,包括:
    处理器;
    存储器,用于存储可执行指令;
    其中,所述处理器用于从所述存储器中读取所述可执行指令,并执行所述可执行指令以实现上述权利要求1-7中任一项所述的视频生成方法或者上述权利要求8-10中任一项所述的视频显示方法。
  14. 一种计算机可读存储介质,其特征在于,所述存储介质存储有计算机程序,当所述计算机程序被处理器执行时,使得处理器实现上述权利要求1-7中任一项所述的视频生成方法或者上述权利要求8-10中任一项所述的视频显示方法。
PCT/CN2021/131705 2020-11-20 2021-11-19 视频生成及显示方法、装置、设备、介质 WO2022105862A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/320,508 US20230291980A1 (en) 2020-11-20 2023-05-19 Method and apparatus for video generation and displaying, device, and medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011309792.2A CN112423022A (zh) 2020-11-20 2020-11-20 视频生成及显示方法、装置、设备、介质
CN202011309792.2 2020-11-20

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/320,508 Continuation US20230291980A1 (en) 2020-11-20 2023-05-19 Method and apparatus for video generation and displaying, device, and medium

Publications (1)

Publication Number Publication Date
WO2022105862A1 true WO2022105862A1 (zh) 2022-05-27

Family

ID=74774257

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131705 WO2022105862A1 (zh) 2020-11-20 2021-11-19 视频生成及显示方法、装置、设备、介质

Country Status (3)

Country Link
US (1) US20230291980A1 (zh)
CN (1) CN112423022A (zh)
WO (1) WO2022105862A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112423022A (zh) * 2020-11-20 2021-02-26 北京字节跳动网络技术有限公司 视频生成及显示方法、装置、设备、介质
CN113761281B (zh) * 2021-04-26 2024-05-14 腾讯科技(深圳)有限公司 虚拟资源处理方法、装置、介质及电子设备
CN113542789A (zh) * 2021-07-19 2021-10-22 广州虎牙科技有限公司 直播礼物特效的展示方法、装置、设备及存储介质
CN114187392B (zh) * 2021-10-29 2024-04-19 北京百度网讯科技有限公司 虚拟偶像的生成方法、装置和电子设备
CN114239241B (zh) * 2021-11-30 2023-02-28 北京百度网讯科技有限公司 卡片的生成方法、装置及电子设备
CN116630488A (zh) * 2022-02-10 2023-08-22 北京字跳网络技术有限公司 视频图像处理方法、装置、电子设备及存储介质
CN115065835A (zh) * 2022-05-20 2022-09-16 广州方硅信息技术有限公司 直播的表情显示处理方法、服务器、电子设备及存储介质
CN116781965B (zh) * 2023-08-25 2023-11-24 深圳有咖互动科技有限公司 虚拟物品合成方法、装置、电子设备和计算机可读介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106375775A (zh) * 2016-09-26 2017-02-01 广州华多网络科技有限公司 虚拟礼物展示方法及装置
CN110766777A (zh) * 2019-10-31 2020-02-07 北京字节跳动网络技术有限公司 虚拟形象的生成方法、装置、电子设备及存储介质
US20200077157A1 (en) * 2018-08-28 2020-03-05 Gree, Inc. Video distribution system for live distributing video containing animation of character object generated based on motion of distributor user, distribution method, and storage medium storing video distribution program
US20200143447A1 (en) * 2016-12-26 2020-05-07 Hong Kong Liveme Corporation Limited Method and device for recommending gift and mobile terminal
CN111491174A (zh) * 2020-05-29 2020-08-04 广州华多网络科技有限公司 虚拟礼物获取及展示方法、装置、设备及存储介质
CN111541950A (zh) * 2020-05-07 2020-08-14 腾讯科技(深圳)有限公司 表情的生成方法、装置、电子设备及存储介质
CN111935491A (zh) * 2020-06-28 2020-11-13 百度在线网络技术(北京)有限公司 直播的特效处理方法、装置以及服务器
CN112423022A (zh) * 2020-11-20 2021-02-26 北京字节跳动网络技术有限公司 视频生成及显示方法、装置、设备、介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106375775A (zh) * 2016-09-26 2017-02-01 广州华多网络科技有限公司 虚拟礼物展示方法及装置
US20200143447A1 (en) * 2016-12-26 2020-05-07 Hong Kong Liveme Corporation Limited Method and device for recommending gift and mobile terminal
US20200077157A1 (en) * 2018-08-28 2020-03-05 Gree, Inc. Video distribution system for live distributing video containing animation of character object generated based on motion of distributor user, distribution method, and storage medium storing video distribution program
CN110766777A (zh) * 2019-10-31 2020-02-07 北京字节跳动网络技术有限公司 虚拟形象的生成方法、装置、电子设备及存储介质
CN111541950A (zh) * 2020-05-07 2020-08-14 腾讯科技(深圳)有限公司 表情的生成方法、装置、电子设备及存储介质
CN111491174A (zh) * 2020-05-29 2020-08-04 广州华多网络科技有限公司 虚拟礼物获取及展示方法、装置、设备及存储介质
CN111935491A (zh) * 2020-06-28 2020-11-13 百度在线网络技术(北京)有限公司 直播的特效处理方法、装置以及服务器
CN112423022A (zh) * 2020-11-20 2021-02-26 北京字节跳动网络技术有限公司 视频生成及显示方法、装置、设备、介质

Also Published As

Publication number Publication date
US20230291980A1 (en) 2023-09-14
CN112423022A (zh) 2021-02-26

Similar Documents

Publication Publication Date Title
WO2022105862A1 (zh) 视频生成及显示方法、装置、设备、介质
US11158102B2 (en) Method and apparatus for processing information
CN109313812B (zh) 具有上下文增强的共享体验
WO2022121601A1 (zh) 一种直播互动方法、装置、设备及介质
WO2022083383A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
US20220200938A1 (en) Methods and systems for providing virtual collaboration via network
WO2018010682A1 (zh) 直播方法、直播数据流展示方法和终端
US20210312671A1 (en) Method and apparatus for generating video
US20200410732A1 (en) Method and apparatus for generating information
WO2022105846A1 (zh) 虚拟对象显示方法及装置、电子设备、介质
WO2021098537A1 (zh) 用于目标设备的视图调整方法、装置、电子设备和介质
WO2022089192A1 (zh) 一种互动处理方法、装置、电子设备和存储介质
WO2022171024A1 (zh) 图像显示方法、装置、设备及介质
WO2022148242A1 (zh) 二维码显示方法、装置、设备及介质
CN113806306B (zh) 媒体文件处理方法、装置、设备、可读存储介质及产品
WO2023103897A1 (zh) 图像处理方法、装置、设备及存储介质
WO2023051244A1 (zh) 图像生成方法、装置、设备及存储介质
WO2023169305A1 (zh) 特效视频生成方法、装置、电子设备及存储介质
CN112055252A (zh) 多屏互动方法、装置、计算机可读介质及电子设备
WO2024027819A1 (zh) 图像处理方法、装置、设备及存储介质
WO2024088144A1 (zh) 增强现实画面的处理方法、装置、电子设备及存储介质
JP2023538825A (ja) ピクチャのビデオへの変換の方法、装置、機器および記憶媒体
WO2024051540A1 (zh) 特效处理方法、装置、电子设备及存储介质
WO2023241427A1 (zh) 图像处理方法、装置、设备及存储介质
WO2024027611A1 (zh) 视频直播方法、装置、电子设备以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21894009

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04-09-2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21894009

Country of ref document: EP

Kind code of ref document: A1