CN113727024A

CN113727024A - Multimedia information generation method, apparatus, electronic device, storage medium, and program product

Info

Publication number: CN113727024A
Application number: CN202111005750.4A
Authority: CN
Inventors: 徐悦然; 龚烨菲; 闫鑫
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-08-30
Filing date: 2021-08-30
Publication date: 2021-11-30
Anticipated expiration: 2041-08-30
Also published as: CN113727024B

Abstract

The present disclosure provides a multimedia information generation method, apparatus, electronic device, storage medium, and program product. The multimedia information generation method comprises the following steps: acquiring first multimedia information, the first multimedia information being acquired based on a photographing template having a plurality of predetermined display areas, and the photographing template being generated by adding an effect mark in each of the plurality of predetermined display areas, a first object being displayed in a part of the predetermined display areas of the plurality of predetermined display areas, and at least one of the remaining predetermined display areas being used for photographing and displaying a second object; second multimedia information is generated, the first object is displayed in a part of the predetermined display area, and the second object is displayed in at least one predetermined display area. The close-shot mode disclosed by the invention increases the interaction among the objects, so that the creation desire of the objects participating in close-shot is improved, the interaction is enhanced, and the user experience is optimized.

Description

Multimedia information generation method, apparatus, electronic device, storage medium, and program product

Technical Field

The present disclosure relates to the field of multimedia information processing, and in particular, to a multimedia information generating method, apparatus, electronic device, storage medium, and program product.

Background

In the related art, taking a video included in multimedia information as an example, in a case where a first object wants to be captured together with a second object in a video of a second object that has already been released, a currently existing shooting manner is to display a video of the first object to be captured and a video of the second object side by side (for example, a left side is the first object to be captured, and a right side is a video of the second object), the first object can be captured together with the second object by capturing its own video within a video capturing duration of the second object, and a video effect after the capture is: a video shot by a first object and a video of a second object are displayed side by side in one display frame. However, such a close-up template in the prior art (for example, a layout template of a close-up position (for example, the above-mentioned template displayed side-by-side) is fixed, and as a result of completing the close-up through such a close-up template, an object (for example, a user) unilaterally closes up with a video shot of another object, and there is a lack of interaction between the objects, resulting in low desire for object creation and weak interactivity.

Disclosure of Invention

The present disclosure provides a multimedia information generating method and apparatus, so as to solve at least the problems of lack of interaction between objects, low object creation desire, and weak interactivity in the related art, and also not solve any of the problems. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a multimedia information generating method, including: acquiring first multimedia information, wherein the first multimedia information is acquired based on a shooting template having a plurality of predetermined display areas, and the shooting template is generated by adding an effect mark in each of the plurality of predetermined display areas, wherein a first object is displayed in a part of the predetermined display areas, and at least one of the remaining predetermined display areas of the plurality of predetermined display areas is used for shooting and displaying a second object; and generating second multimedia information, wherein the first object is displayed in the part of the predetermined display area in the second multimedia information, and the second object is displayed in the at least one predetermined display area in the second multimedia information.

Optionally, the step of acquiring the first multimedia information may include: generating first multimedia information based on the photographing template, wherein the part of the predetermined display areas is used for photographing and displaying the first object.

Optionally, the generating of the shooting template may include: performing the following operations in each of the plurality of predetermined display areas: acquiring a virtual display object, wherein the virtual display object is a virtual object generated based on the characteristics of an object; determining a characteristic position of the virtual display object, wherein the characteristic position is position information corresponding to a characteristic of the object; and adding an effect mark corresponding to the characteristic position in each preset display area.

Optionally, the step of generating the multimedia information may include: determining the characteristic position of an object in a preset display area, wherein the characteristic position of the object is position information corresponding to the characteristic of a first object or a second object obtained by shooting; and synthesizing the effect mark in the preset display area for shooting the object with the object according to the characteristic position of the object and the characteristic position corresponding to the effect mark in the preset display area for shooting the object, thereby generating the multimedia information comprising the object.

Alternatively, the operation of synthesizing the effect mark in the predetermined display area for photographing the object with the object may include: determining characteristic points of effect marks meeting preset conditions; determining object feature matching points corresponding to the feature points of the effect marks based on the feature points of the effect marks; and when the distance between the position of the feature matching point of the object and the position of the feature point of the corresponding effect mark is smaller than or equal to a preset threshold value, synthesizing the effect mark in the preset display area for shooting the object onto the object in a layer superposition mode.

Alternatively, the determining the feature points of the effect markers satisfying the preset condition may include: and sampling the characteristic points of the effect marks or determining the characteristic points representing the effect marks based on the calculation processing of the characteristic points of the effect marks.

Optionally, when the distance between the feature matching point of the object and the feature point of the corresponding effect mark is less than or equal to a predetermined threshold, the method may further include: the distance between a predetermined number of object matching feature points and the feature point of the corresponding effect mark is less than or equal to a predetermined threshold.

Optionally, the first object and the second object may be located in different display layers, and a predefined sequential relationship exists between the display layer where the first object is located and the display layer where the second object is located, where the predefined sequential relationship is changed according to different object requirements.

Optionally, the total shooting time of the second object may be less than or equal to the total shooting time of the first multimedia information.

Optionally, the effect indicia may include at least one of graffiti indicia, magic expression indicia, sticker indicia.

According to a second aspect of the embodiments of the present disclosure, there is provided a multimedia information generating apparatus including: an acquisition module configured to: acquiring first multimedia information, wherein the first multimedia information is acquired based on a shooting template having a plurality of predetermined display areas, and the shooting template is generated by adding an effect mark in each of the plurality of predetermined display areas, wherein a first object is displayed in a part of the plurality of predetermined display areas, and at least one of the remaining predetermined display areas of the plurality of predetermined display areas is used for shooting and displaying a second object; a generation module configured to: and generating second multimedia information, wherein the first object is displayed in the part of the predetermined display area in the second multimedia information, and the second object is displayed in the at least one predetermined display area in the second multimedia information.

Optionally, the operation of acquiring the first multimedia information by the acquisition module may include: generating first multimedia information based on the photographing template, wherein the part of the predetermined display areas is used for photographing and displaying the first object.

Alternatively, the capture template may be generated by: performing the following operations in each of the plurality of predetermined display areas: acquiring a virtual display object, wherein the virtual display object is a virtual object generated based on the characteristics of an object; determining a characteristic position of the virtual display object, wherein the characteristic position is position information corresponding to a characteristic of the object; and adding an effect mark corresponding to the characteristic position in each preset display area.

Optionally, the generating operation of the multimedia information may include: determining the characteristic position of an object in a preset display area, wherein the characteristic position of the object is position information corresponding to the characteristic of a first object or a second object obtained by shooting; and synthesizing the effect mark in the preset display area for shooting the object with the object according to the characteristic position of the object and the characteristic position corresponding to the effect mark in the preset display area for shooting the object, thereby generating the multimedia information comprising the object.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions, wherein the processor is configured to execute the instructions to implement the multimedia information generation method as described above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of an electronic device/server, enable the electronic device/server to perform the multimedia information generating method as described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the multimedia information generating method as described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: through the multimedia information generation method and device disclosed by the invention, the interaction between the objects is increased by the close-shot mode, so that the creation desire of the objects participating in close-shot is improved, the interactivity is enhanced, and the user experience is optimized.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is an exemplary system architecture diagram in which exemplary embodiments of the present disclosure may be applied;

fig. 2 is a flowchart illustrating a multimedia information generating method according to an exemplary embodiment of the present disclosure;

fig. 3 is a schematic diagram illustrating an example of a multimedia information generation method according to an exemplary embodiment of the present disclosure;

FIG. 4 is a schematic diagram illustrating editable content according to an exemplary embodiment of the disclosure;

fig. 5 is a schematic diagram illustrating an example of a multimedia information generation method according to an exemplary embodiment of the present disclosure;

fig. 6 is a block diagram illustrating a multimedia information generating apparatus according to an exemplary embodiment of the present disclosure;

fig. 7 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The embodiments described in the following examples do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

In this case, the expression "at least one of the items" in the present disclosure means a case where three types of parallel expressions "any one of the items", "a combination of any plural ones of the items", and "the entirety of the items" are included. For example, "include at least one of a and B" includes the following three cases in parallel: (1) comprises A; (2) comprises B; (3) including a and B. For another example, "at least one of the first step and the second step is performed", which means that the following three cases are juxtaposed: (1) executing the step one; (2) executing the step two; (3) and executing the step one and the step two.

As mentioned in the background of the present disclosure, in the related art, taking a video included in multimedia information as an example, when an object participating in a close-up wants to close up with an object in an existing video, the object participating in the close-up may enter a shooting window through a predetermined input (e.g., clicking on an access icon of a close-up function), in which a left portion of the window is a shooting frame and a right portion of the window is the existing video, and the object participating in the close-up may shoot the video through another predetermined input (e.g., clicking on a shooting button) under the above-mentioned shooting window, and at the same time, the existing video may also start playing in response to another predetermined input to complete the close-up. However, the conventional close shot (which may be referred to as a closed shot frame) as described above has a weak interactive feeling, and the object participating in the close shot has poor connectivity with the original video object, and has no significant effect on the revival and retention of the original video work.

In view of the above, according to an exemplary embodiment of the present disclosure, the present disclosure provides a multimedia information generating method and apparatus, taking a video included in multimedia information as an example, by setting a pit bit number and a position display area for taking a photo when an original video is shot, arranging and combining into different display effects, and enabling an object participating in the photo to take a photo with an object in the original video under a shooting environment of the original video (instead of a shooting environment in which the object is located), which may improve an interest level of the object participating in the photo, improve interactivity and connectivity of the object participating in the photo with the object in the original video, and may allow a plurality of objects participating in the photo to join in the photo (i.e., re-create the original video), so that extensibility is enhanced.

Hereinafter, a multimedia information generation method and apparatus according to an exemplary embodiment of the present disclosure will be described in detail with reference to fig. 1 to 7.

Fig. 1 is an exemplary system architecture diagram to which exemplary embodiments of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few. A user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages (e.g., multimedia information (e.g., video) data upload requests, multimedia information data acquisition requests), etc. Various communication client applications, such as a video recording application, a video compression application, a video and audio editing application, an instant messaging tool, a mailbox client, social platform software, and the like, may be installed on the

terminal devices

101, 102, 103. The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices having a display screen and capable of playing, recording and editing video, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the

terminal device

101, 102, 103 is software, it may be installed in the electronic devices listed above, it may be implemented as a plurality of software or software modules (for example, to provide distributed services), or it may be implemented as a single software or software module. And is not particularly limited herein.

The

terminal apparatuses

101, 102, 103 may be mounted with an image capturing device (e.g., a camera) to capture multimedia information (e.g., video) data, and further, the

terminal apparatuses

101, 102, 103 may also be mounted with a component (e.g., a speaker) for converting an electric signal into sound to play sound, and may also be mounted with a device (e.g., a microphone) for converting an analog audio signal into a digital audio signal to capture sound.

The

terminal devices

101, 102, 103 may perform acquisition of multimedia information (e.g., video) data using an image acquisition apparatus mounted thereon and acquisition of audio data using an audio acquisition apparatus mounted thereon, and the

terminal devices

101, 102, 103 may encode, store, and transmit the acquired video data and audio data and may decode and play encoded video and audio received from another terminal device or from the server 105.

The server 105 may be a server that provides various services, such as a background server that provides support for a video recording-type application, a video compression-type application, a video editing-type application, and the like installed on the

terminal apparatuses

101, 102, 103, or a storage server that stores encoded video and audio uploaded by the

terminal apparatuses

101, 102, 103 and can transmit the stored encoded video and audio to the

terminal apparatuses

101, 102, 103 in response to a request of the

terminal apparatuses

101, 102, 103.

The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the multimedia information generating method provided in the embodiment of the present application is generally executed by the

terminal devices

101, 102, and 103, and accordingly, the multimedia information generating apparatus is generally disposed in the

terminal devices

101, 102, and 103.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation, and the disclosure is not limited thereto.

Fig. 2 is a flowchart illustrating a multimedia information generating method according to an exemplary embodiment of the present disclosure. The multimedia information generation method 200 according to the exemplary embodiment of the present disclosure may be applied to a client (e.g., the

terminal devices

101, 102, 103 shown in fig. 1), but is not limited thereto, and the multimedia information generation method 200 may also be applied to a server (e.g., the server 105 shown in fig. 1).

Referring to fig. 2, according to an exemplary embodiment of the present disclosure, in step S201, first multimedia information is acquired. Specifically, the first multimedia information is acquired based on a photographing template having a plurality of predetermined display areas, and the photographing template is generated by adding an effect mark in each of the plurality of predetermined display areas. The first object is displayed in a part of the predetermined display areas of the plurality of predetermined display areas, and at least one of the remaining predetermined display areas of the plurality of predetermined display areas is used to photograph and display the second object.

According to an exemplary embodiment of the present disclosure, the step of acquiring the first multimedia information may include: and generating first multimedia information based on the shooting template. Further, a portion of the plurality of predetermined display areas is used to capture and display the first object.

In this way, by reserving the predetermined display area when the original video is shot, the possibility of taking a close shot in the shooting environment of the original video can be provided for the subsequent subjects participating in the close shot, thereby optimizing the user experience.

According to an exemplary embodiment of the present disclosure, the generating of the photographing template may include: performing the following operations in each of a plurality of predetermined display areas: acquiring a virtual display object, wherein the virtual display object is a virtual object generated based on the characteristics of the object; determining a characteristic position of the virtual display object, wherein the characteristic position is position information corresponding to the characteristic of the object; and adding an effect mark corresponding to the characteristic position in each preset display area.

Alternatively, the operation related to generating the photographing template may be performed on the client, but is not limited thereto. Alternatively, the related operation of generating the shooting template may also be performed in the server, in which case the multimedia information generating method 200 is performed on the client and the shooting template is received from the server to achieve the acquisition of the shooting template.

According to an exemplary embodiment of the present disclosure, the generating of the multimedia information may include: determining the characteristic position of an object in a preset display area, wherein the characteristic position of the object is position information corresponding to the characteristic of a first object or a second object obtained by shooting; and synthesizing the effect mark in the preset display area for shooting the object with the object according to the characteristic position of the object and the characteristic position corresponding to the effect mark in the preset display area for shooting the object, thereby generating the multimedia information comprising the object.

According to an exemplary embodiment of the present disclosure, the operation of synthesizing an effect mark in a predetermined display area for photographing a subject with the subject may include: determining characteristic points of effect marks meeting preset conditions; determining object feature matching points corresponding to the feature points of the effect marks based on the feature points of the effect marks; when the distance between the position of the feature matching point of the object and the position of the feature point of the corresponding effect mark is less than or equal to a predetermined threshold value, the effect mark in a predetermined display area for photographing the object is synthesized on the object in a layer-by-layer manner.

According to an exemplary embodiment of the present disclosure, determining the feature point of the effect mark satisfying the preset condition may include the following ways: and sampling the characteristic points of the effect marks or determining the characteristic points representing the effect marks based on the calculation processing of the characteristic points of the effect marks.

Specifically, in the process of determining the feature points of the effect marker based on the preset condition, the preset condition may be interval sampling of all the feature points, or may be a process based on all the feature points, and finally a preset number of feature points capable of representing the effect marker are determined.

According to an exemplary embodiment of the present disclosure, when a distance between a feature matching point of an object and a feature point of a corresponding effect marker is less than or equal to a predetermined threshold, it may further include: the distance between a predetermined number of object matching feature points and the feature point of the corresponding effect mark is less than or equal to a predetermined threshold.

As an example, when the feature of the object is a facial feature of a person, the distance of a plurality of features (e.g., at least three features) (e.g., a pair of eyes, a nose) in the facial feature is less than a threshold value, the entire face can be matched (e.g., adsorbed). Alternatively, for each feature (e.g., nose), the nose can adsorb only if the number of nose feature points that meet the threshold is less than the threshold.

According to an exemplary embodiment of the present disclosure, the first object and the second object may be located in different display layers, and the display layer in which the first object is located and the display layer in which the second object is located have a predefined sequential relationship. Alternatively, the predefined sequential relationship may vary according to different object requirements.

Here, regarding the setting of the display layer, assuming that the first object is a user a, the second object is a user B, and the video taken by the user a is referred to as an original video, there may be the following three exemplary cases:

case one, setting about a single map layer, that is, user a is a person, user B is a person, and user B who participates in the snap wishes to be positioned in front of user a (with the relative position of user a and user B as a reference). In this case, when the user B performs shooting, the user B subjected to portrait segmentation is normally shot to obtain a co-shooting video of the user B and the user a, so that the user B joins in the shooting environment of the user a to complete the co-shooting. Here, the predetermined display areas in the original video are at least two, and in the snap video, the user a and the user B are displayed in the two predetermined display areas, respectively. In addition, in this case, there is no need to set the layer order that user B has at the time of shooting, because the layer in which user B is located in the related process of layer overlay display is located above the layer in which the original video is located (i.e., the layer in which user a is located) by default (i.e., user a is located at the bottom of all layers).

Case two, regarding the setting of the single map layer, that is, user a is a person, user B is a person, and user B who participates in the snap wishes to be positioned behind user a (with the relative position of user a and user B as a reference). In this case, when the user B performs shooting, firstly, the user a and the user B are subjected to portrait segmentation, the sequence of the layers where the user a and the user B are located is set to be that the layers of the user B are below the layers of the user a (that is, the user B is located at the bottom of all the layers), and the user B after the portrait segmentation and the sequence of the layers are subjected to shooting to obtain a co-shooting video of the user B and the user a, so that the user B is added into the shooting environment of the user a to complete the co-shooting. Here, the predetermined display areas in the original video are at least two, and in the snap video, the user a and the user B are displayed in the two predetermined display areas, respectively.

Case three, regarding the setting of the multi-person map layer, that is, user a is at least two persons and user B is one person, or user a is one person and user B is at least two persons, or both user a and user B are at least two persons. In this case, before the user B formally shoots the video, firstly, the person image segmentation is performed on all people, the sequence of the layer where each person is located is set, and then the user B after the person image segmentation and the layer sequence setting is shot to obtain the co-shooting video of the user B and the user a, so that the user B is added into the shooting environment of the user a to complete the co-shooting.

Further, as an example of case three, when the user a is two persons (i.e., the users a1 and a2) and the user B is one person, before the user B formally photographs a video, portrait segmentation is performed on the users a1 and a2 and the user B, respectively, and the order of the layers where the users a1 and a2 are located (here, the users a1 and a2 are located in the same layer) and the layer where the user B is located is set, and then the user B after the portrait segmentation and the layer order setting is photographed to obtain a co-photographed video of the user B and the user a. In this case, user B may be located in front of both users a1, a2 similar to the situation, or may be located behind both users a1, a2 similar to the situation.

Here, it should be understood that the above cases are exemplary descriptions, and any combination may be made to complete the setting of the layer order.

According to an exemplary embodiment of the present disclosure, the total photographing time period of the second object may be less than or equal to the total time period of the first multimedia information.

According to an exemplary embodiment of the present disclosure, the effect mark may include at least one of a graffiti mark, a magic expression mark, and a sticker mark.

Here, the effect indicia in the present disclosure may be provided by an application on the client, may be received from any other device, or may be input by the user (e.g., user-made effect indicia).

By way of example, the multimedia information may include video, images, and any other multimedia form of information. For convenience of description, the related description is made with video as an example in the description of the examples.

According to an exemplary embodiment of the present disclosure, in step S202, second multimedia information is generated. Specifically, the first object is displayed in a part of predetermined display areas in the second multimedia information, and the second object is displayed in at least one predetermined display area in the second multimedia information.

According to an exemplary embodiment of the present disclosure, the generating step of the second multimedia information is similar to the generating step of the multimedia information described in step S201, and is not described herein again.

Alternatively, the "subject" referred to in the present disclosure may be any subject that can be used for photographing, such as a human, an animal, a still, and the like. In the embodiments of the present disclosure, the object is exemplified by a human.

The multimedia information generating method 200 is described below with reference to fig. 3, taking the example that the multimedia information is a video. Fig. 3 is a schematic diagram illustrating an example of a multimedia information generation method according to an exemplary embodiment of the present disclosure. In the example of fig. 3, the photographing template includes 6 predetermined display areas, and the effect mark in each display area is a graffiti mark.

Referring to fig. 3, in a first step, a photographing operation is entered, indicating that a photographing window is currently displayed. And secondly, starting a doodling operation, namely setting the 1 st display area in the 6 preset display areas and setting a corresponding doodling mark before formally shooting the video. And thirdly, setting the remaining 5 preset display areas and corresponding graffiti marks. Fourth, a formal photographing operation is started, in this example, the subject of photographing is three persons, and the three persons of photographing are displayed in 3 different predetermined display areas, that is, each person is located in a different one of the predetermined display areas. In this example, the remaining 3 predetermined display areas are available for subsequent shots and display of others. Fifth, the original video shot is completed and the video is further edited, for example, adding other graffiti, magic expressions, stickers, music, text (e.g., editable content shown in fig. 4), etc. Sixthly, seventhly, editing the video related information and performing personalized setting. And eighthly, distributing the video, wherein the video can be selectively distributed to double columns or single columns. It should be understood that the above-described fifth to seventh steps may be omitted, and one of the eighth and ninth steps may be performed.

The process of generating the shooting template is specifically described below by way of example. The graffiti pattern in the above example is specifically described. The specific process for generating the shooting template comprises the following steps: performing the following operations in each of a plurality of predetermined display areas: acquiring a virtual display object (for example, a character model diagram, which will be described below as an example); capturing feature points of the character model graph, and identifying the feature point positions of the character model graph (hereinafter, the feature points are the middle points between two eyes as an example); and adding a graffiti mark corresponding to the position of the characteristic point in each preset display area. Alternatively, the feature points may be points corresponding to a certain position of the body of the character model diagram, or any other points capable of determining the approximate position of the character.

After generating the shooting template, when the real person needs to be shot by using the shooting template, the specific process of shooting can be as follows: in the shooting process, feature points of a real person are captured, the position of the midpoint between two eyes is identified, and when the distance between the feature point position of the real person and the feature point position of a character model graph is smaller than or equal to a preset threshold value, a graffiti mark in the preset display area is synthesized on the real person in a mode of overlaying a graph layer. Alternatively, taking the example of fig. 3 as an example, there are three real persons and 6 graffiti marks, and in this way, three real persons can be synthetically displayed respectively at 3 different graffiti marks.

Alternatively, the graffiti markings that have been synthesized may change as the corresponding person's expression changes.

Through the mode, the preset display area is reserved when the original video is shot, the possibility of taking a photo in the shooting environment of the original video can be provided for the follow-up object taking part in the photo, and therefore user experience is optimized.

According to the embodiment of the disclosure, a series of expression templates (for example, templates after doodling) are set, the expressions are customized by a user or are doodled online or are customized expression templates, and the close-shot object (for example, close-shot user) completes close-shot through expression matching with the corresponding position in the expression templates, so that close-shot experience of the close-shot user is improved.

The multimedia information generating method 200 is described below with reference to fig. 5, taking the example that the multimedia information is a video. Fig. 5 is a schematic diagram illustrating an example of a multimedia information generation method according to an exemplary embodiment of the present disclosure. The example of fig. 5 is an example continued on the basis of the example of fig. 3, that is, in the example of fig. 5, the shooting template also includes 6 predetermined display areas, the effect mark in each display area is a graffiti mark, and the shooting object in the original video is three persons, and the shooting object joining in a co-shooting (hereinafter, simply referred to as a joining in co-shooting user) is one person.

Referring to fig. 5, in a first step, a shooting window for joining a snapshot is entered. And secondly, starting to shoot the video, namely shooting the joining user in a 4 th preset display area different from three preset display areas in which three persons are displayed in the original video according to the requirement of the joining user before the video is shot formally. Here, it should be understood that the display content added to the snap user has already undergone the object segmentation processing and the layer sequence setting processing, and the specific processing procedure refers to the foregoing description and will not be described herein again. And thirdly, secondarily creating the shot video, namely further editing the video, wherein the editing process is similar to the fifth step described with reference to fig. 3, and the details are not repeated here. The fourth step and the fifth step are similar to the sixth to ninth steps described with reference to fig. 3, and will not be described again here. It should be understood that the third step and the fourth step are described above.

According to an exemplary embodiment of the present disclosure, after the second multimedia information (e.g., video) is generated in step S202, when the remaining predetermined display area still exists in the second multimedia information, all related operations of joining in a auction may be continuously implemented, and so on until the remaining predetermined display area does not exist in the current multimedia information.

The above process is illustrated in tabular form below.

TABLE 1

Human being	Works and works	Human being	Works and works
				Original author	User a	Production of work a1	User aa	Generating work aa1
User b	Generating work b1	User ba	Product ba1
					User c	Generating work c1	User ca	To generate work ca1
…	…	…	…
					User n +1	Generating work (n +1)1	User (n +1) a	Generate work (n +1) a1

Referring to table 1, the original author issues the original work, and user a watches and joins the beat to generate work a1, completing one-time forwarding. After the work is added to the auction, the original author may be notified in various possible ways, such as, but not limited to, private letter (support masking function) ways. User aa views user a's creative work and joins the beat, resulting in work aa1, and so on. Therefore, the method can be shared by n individuals to generate n different works and develop different subscriber lines.

Here, it should be noted that the video of the originator is not changeable, what the current photographer can change is editable content related to the current photographer, and what the current photographer cannot change related content of all previous photographers, which can be achieved by setting the layer order of the layer where each photographer is located as described above. With this arrangement, the integrity of the video captured by each photographer can be maintained (i.e., not modified by other photographers).

Through the mode shown in table 1, for example, one video of one user can be expanded to more videos of more people by adding a close-shot mode, so that the interest of the user in producing content is promoted, the originality of the user can be effectively played, and the expandability is promoted.

According to the exemplary embodiment of the disclosure, the interaction between the objects is increased through the close-shot mode in the multimedia information generation method of the disclosure, so that the creation desire of the objects participating in close-shot is improved, the interaction is enhanced, and the user experience is optimized.

Further, according to an exemplary embodiment of the present disclosure, in photographing for a snap, effect marks (e.g., doodle expressions, stickers, magic expressions, etc.) automatically adsorb a predetermined region of an object (e.g., a person's face), and the effect marks (e.g., various expressions, etc.) interact with the object, so that the snap video has interest. By continuously adding new objects, virtual interaction between the objects is realized, creation inspiration of the objects is stimulated, and interestingness of videos is increased.

Fig. 6 is a block diagram illustrating a multimedia information generating apparatus according to an exemplary embodiment of the present disclosure.

Referring to fig. 6, the multimedia information generating apparatus 600 includes an obtaining module 601 and a generating module 602. Specifically, the obtaining module 601 is configured to: acquiring first multimedia information, wherein the first multimedia information is acquired based on a shooting template having a plurality of predetermined display areas, and the shooting template is generated by adding an effect mark in each of the plurality of predetermined display areas, wherein a first object is displayed in a part of the plurality of predetermined display areas, and at least one of remaining predetermined display areas of the plurality of predetermined display areas is used for shooting and displaying a second object. The generation module 602 is configured to: and generating second multimedia information, wherein the first object is displayed in a part of the predetermined display area in the second multimedia information, and the second object is displayed in at least one predetermined display area in the second multimedia information.

According to an exemplary embodiment of the present disclosure, the operation of the obtaining module obtaining the first multimedia information may include: first multimedia information is generated based on the photographing template, wherein a part of predetermined display areas of the plurality of predetermined display areas are used to photograph and display the first object.

According to an exemplary embodiment of the present disclosure, a photographing template may be generated by: performing the following operations in each of a plurality of predetermined display areas: acquiring a virtual display object, wherein the virtual display object is a virtual object generated based on the characteristics of the object; determining a characteristic position of the virtual display object, wherein the characteristic position is position information corresponding to the characteristic of the object; and adding an effect mark corresponding to the characteristic position in each preset display area.

According to an exemplary embodiment of the present disclosure, the generating operation of the multimedia information may include: determining the characteristic position of an object in a preset display area, wherein the characteristic position of the object is position information corresponding to the characteristic of a first object or a second object obtained by shooting; and synthesizing the effect mark in the preset display area for shooting the object with the object according to the characteristic position of the object and the characteristic position corresponding to the effect mark in the preset display area for shooting the object, thereby generating the multimedia information comprising the object.

The processing specifically performed by each of the above-described modules and the related processing information have been described above with reference to fig. 2, and the description is not repeated here.

Through the above manner, the interaction between the objects is increased according to the close-shot manner of the multimedia information generation device of the exemplary embodiment of the present disclosure, so that the creation desire of the objects participating in close-shot is improved, the interactivity is enhanced, and the user experience is optimized.

In addition, when the device provided in the above embodiment implements the functions thereof, only the division of the above functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to implement all or part of the functions described above.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 7 is a block diagram illustrating an electronic device 700 according to an example embodiment of the present disclosure. The electronic device 700 may be, for example: a smartphone, a tablet, a laptop, or a desktop computer. The electronic device 700 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and so forth.

In general, the electronic device 700 includes: a processor 701 and a memory 702.

The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 701 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 701 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 702 may include one or more computer-readable storage media, which may be non-transitory. Memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 702 is used to store at least one instruction for execution by processor 701 to implement the method of the present disclosure as shown in fig. 2.

In some embodiments, the electronic device 700 may further optionally include: a peripheral interface 703 and at least one peripheral. The processor 701, the memory 702, and the peripheral interface 703 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 704, touch screen display 705, camera 706, audio circuitry 707, positioning components 708, and power source 709.

The peripheral interface 703 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 701 and the memory 702. In some embodiments, processor 701, memory 702, and peripheral interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 704 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 704 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 704 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 704 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 704 may also include NFC (Near Field Communication) related circuits, which are not limited by this disclosure.

The display screen 705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to capture touch signals on or over the surface of the display screen 705. The touch signal may be input to the processor 701 as a control signal for processing. At this point, the display 705 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 705 may be one, disposed on the front panel of the electronic device 700; in other embodiments, the number of the display screens 705 may be at least two, and the at least two display screens are respectively disposed on different surfaces of the electronic device 700 or are in a folding design; in still other embodiments, the display 705 may be a flexible display disposed on a curved surface or on a folded surface of the electronic device 700. Even more, the display 705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display 705 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or the like.

The camera assembly 706 is used to capture images or video. Optionally, camera assembly 706 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 706 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuitry 707 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 701 for processing or inputting the electric signals to the radio frequency circuit 704 to realize voice communication. For stereo capture or noise reduction purposes, the microphones may be multiple and disposed at different locations of the electronic device 700. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 707 may also include a headphone jack.

The positioning component 708 is operable to locate a current geographic Location of the electronic device 700 to implement a navigation or LBS (Location Based Service). The Positioning component 708 can be a Positioning component based on the GPS (Global Positioning System) in the united states, the beidou System in china, the graves System in russia, or the galileo System in the european union.

The power supply 709 is used to supply power to various components in the electronic device 700. The power source 709 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When power source 709 includes a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the electronic device 700 also includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: acceleration sensor 711, gyro sensor 712, pressure sensor 713, fingerprint sensor 714, optical sensor 715, and proximity sensor 716.

The acceleration sensor 711 may detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the electronic device 700. For example, the acceleration sensor 711 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 701 may control the touch screen 705 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 712 may detect a body direction and a rotation angle of the electronic device 700, and the gyro sensor 712 may cooperate with the acceleration sensor 711 to acquire a 3D motion of the user with respect to the electronic device 700. From the data collected by the gyro sensor 712, the processor 701 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 713 may be disposed on a side bezel of electronic device 700 and/or an underlying layer of touch display 705. When the pressure sensor 713 is disposed on a side frame of the electronic device 700, a user holding signal of the electronic device 700 may be detected, and the processor 701 may perform left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed at a lower layer of the touch display 705, the processor 701 controls the operability control on the UI according to the pressure operation of the user on the touch display 705. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 714 is used for collecting a fingerprint of a user, and the processor 701 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the identity of the user according to the collected fingerprint. When the user identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking a screen, viewing encrypted information, downloading software, paying, changing settings, and the like. The fingerprint sensor 714 may be disposed on the front, back, or side of the electronic device 700. When a physical button or vendor Logo is provided on the electronic device 700, the fingerprint sensor 714 may be integrated with the physical button or vendor Logo.

The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 may control the display brightness of the touch display 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 705 is increased; when the ambient light intensity is low, the display brightness of the touch display 705 is turned down. In another embodiment, processor 701 may also dynamically adjust the shooting parameters of camera assembly 706 based on the ambient light intensity collected by optical sensor 715.

A proximity sensor 716, also referred to as a distance sensor, is typically disposed on the front panel of the electronic device 700. The proximity sensor 716 is used to capture the distance between the user and the front of the electronic device 700. In one embodiment, the processor 701 controls the touch display screen 705 to switch from the bright screen state to the dark screen state when the proximity sensor 716 detects that the distance between the user and the front surface of the electronic device 700 is gradually decreased; when the proximity sensor 716 detects that the distance between the user and the front surface of the electronic device 700 gradually becomes larger, the processor 701 controls the touch display screen 705 to switch from the breath screen state to the bright screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 7 does not constitute a limitation of the electronic device 700 and may include more or fewer components than those shown, or combine certain components, or employ a different arrangement of components.

According to an embodiment of the present disclosure, there may also be provided a computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a multimedia information generation method according to the present disclosure. Examples of the computer-readable storage medium herein include: read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD + R, CD-RW, CD + RW, DVD-ROM, DVD-R, DVD + R, DVD-RW, DVD + RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or compact disc memory, Hard Disk Drive (HDD), solid-state drive (SSD), card-type memory (such as a multimedia card, a Secure Digital (SD) card or a extreme digital (XD) card), magnetic tape, a floppy disk, a magneto-optical data storage device, an optical data storage device, a hard disk, a magnetic tape, a magneto-optical data storage device, a hard disk, a magnetic tape, a magnetic data storage device, a magnetic tape, a magnetic data storage device, a magnetic tape, a magnetic data storage device, a magnetic tape, a magnetic data storage device, a magnetic tape, a magnetic data storage device, A solid state disk, and any other device configured to store and provide a computer program and any associated data, data files, and data structures to a processor or computer in a non-transitory manner such that the processor or computer can execute the computer program. The computer program in the computer-readable storage medium described above can be run in an environment deployed in a computer apparatus, such as a client, a host, a proxy device, a server, and the like, and further, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed fashion by one or more processors or computers.

According to an embodiment of the present disclosure, there may also be provided a computer program product, in which instructions are executable by a processor of a computer device to perform the multimedia information generating method described above.

According to the multimedia information generation method disclosed by the embodiment of the disclosure, the interaction between the objects can be increased, the creation desire of the objects participating in the co-shooting is improved, the interaction is enhanced, and the user experience is optimized.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating multimedia information, comprising:

acquiring first multimedia information, wherein the first multimedia information is acquired based on a shooting template having a plurality of predetermined display areas, and the shooting template is generated by adding an effect mark in each of the plurality of predetermined display areas, wherein a first object is displayed in a part of the predetermined display areas, and at least one of the remaining predetermined display areas of the plurality of predetermined display areas is used for shooting and displaying a second object;

and generating second multimedia information, wherein the first object is displayed in the part of the predetermined display area in the second multimedia information, and the second object is displayed in the at least one predetermined display area in the second multimedia information.

2. The multimedia information generating method of claim 1, wherein the step of acquiring the first multimedia information comprises:

generating first multimedia information based on the photographing template, wherein the part of the predetermined display areas is used for photographing and displaying the first object.

3. The multimedia information generating method as claimed in claim 1, wherein the generating of the photographing template comprises:

performing the following operations in each of the plurality of predetermined display areas:

acquiring a virtual display object, wherein the virtual display object is a virtual object generated based on the characteristics of an object;

determining a characteristic position of the virtual display object, wherein the characteristic position is position information corresponding to a characteristic of the object;

and adding an effect mark corresponding to the characteristic position in each preset display area.

4. The multimedia information generating method according to claim 1 or 2, wherein the generating of the multimedia information comprises:

determining the characteristic position of an object in a preset display area, wherein the characteristic position of the object is position information corresponding to the characteristic of a first object or a second object obtained by shooting;

and synthesizing the effect mark in the preset display area for shooting the object with the object according to the characteristic position of the object and the characteristic position corresponding to the effect mark in the preset display area for shooting the object, thereby generating the multimedia information comprising the object.

5. The multimedia information generating method according to claim 4, wherein the operation of synthesizing the effect mark in the predetermined display area for photographing the object with the object comprises:

determining characteristic points of effect marks meeting preset conditions;

determining object feature matching points corresponding to the feature points of the effect marks based on the feature points of the effect marks;

and when the distance between the position of the feature matching point of the object and the position of the feature point of the corresponding effect mark is smaller than or equal to a preset threshold value, synthesizing the effect mark in the preset display area for shooting the object onto the object in a layer superposition mode.

6. The multimedia information generating method as claimed in claim 5, wherein the determining the feature point of the effect mark satisfying the preset condition comprises: and sampling the characteristic points of the effect marks or determining the characteristic points representing the effect marks based on the calculation processing of the characteristic points of the effect marks.

7. A multimedia information generating apparatus, characterized by comprising:

an acquisition module configured to: acquiring first multimedia information, wherein the first multimedia information is acquired based on a shooting template having a plurality of predetermined display areas, and the shooting template is generated by adding an effect mark in each of the plurality of predetermined display areas, wherein a first object is displayed in a part of the plurality of predetermined display areas, and at least one of the remaining predetermined display areas of the plurality of predetermined display areas is used for shooting and displaying a second object;

a generation module configured to: and generating second multimedia information, wherein the first object is displayed in the part of the predetermined display area in the second multimedia information, and the second object is displayed in the at least one predetermined display area in the second multimedia information.

8. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions,

wherein the processor is configured to execute the instructions to implement the multimedia information generating method of any of claims 1 to 6.

9. A computer-readable storage medium in which instructions, when executed by a processor of an electronic device/server, enable the electronic device/server to perform the multimedia information generating method of any one of claims 1 to 6.

10. A computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the multimedia information generating method according to any of claims 1 to 6.