CN111343499A

CN111343499A - Video synthesis method and device

Info

Publication number: CN111343499A
Application number: CN201811550638.7A
Authority: CN
Inventors: 李越; 严佳
Original assignee: Beijing Qihoo Technology Co Ltd
Current assignee: Beijing Qihoo Technology Co Ltd
Priority date: 2018-12-18
Filing date: 2018-12-18
Publication date: 2020-06-26

Abstract

The invention discloses a video synthesis method and a video synthesis device. The video synthesis method of the invention, through confirming the material that the video needs of the synthesis, carry out the processing of multiple data frame according to the material; wherein, the processing of each item has dependency relationship and is executed asynchronously; and synthesizing the video according to the final processing result of the data frame. The material sub-data frames and the sub-processing items are processed, and the processing items are mutually dependent, so that the data frames can be processed orderly. And each processing item is asynchronously executed, so that the processing time of the material can be effectively saved, and the processing efficiency is improved.

Description

Video synthesis method and device

Technical Field

The invention relates to the technical field of video processing, in particular to a video synthesis method and a video synthesis device.

Background

At present, short videos are more and more popular, and people also more and more like shooting and sharing short videos. With the development of technology, people no longer only satisfy the video shot by shooting, but hope to add other contents in the video, such as some picture special effects or sound special effects. The existing video processing technology is relatively troublesome, the processing time is relatively long, the instant requirements of users cannot be met, and the user experience is influenced to a certain extent.

Disclosure of Invention

In view of the above, the present invention has been made to provide a video composition method, apparatus, electronic device and computer-readable storage medium that overcome or at least partially solve the above problems.

According to an aspect of the present invention, there is provided a video composition method including:

determining materials required by the synthesized video;

executing a plurality of items of processing related to the data frame according to the materials; wherein, the processing of each item has dependency relationship and is executed asynchronously;

and synthesizing the video according to the final processing result of the data frame.

Preferably, the material comprises video material and/or picture material.

Preferably, the treatment comprises one or more of:

extracting data frames from the material;

decoding the data frame;

modifying the decoded data frame;

encoding the modified data frame;

the data frame is written to a file.

Preferably, the performing a plurality of processes on the data frame according to the material includes: a plurality of threads are initiated, each thread performing a respective process on the data frame.

Preferably, the threads respectively perform a process on the data frame, including: each thread puts the processing result of the data frame into the cache region corresponding to the thread, and reads the processing result from the cache region of the thread depending on the thread, and the processing result is taken as the processing object of the thread.

Preferably, each processing result contains a time stamp.

Preferably, the executing of the data frame by the threads further comprises: and when the cache region corresponding to each thread is full, the corresponding thread suspends the execution processing.

Preferably, the method is applied to a mobile device, and the method further comprises: and monitoring the resources of the mobile equipment, and adjusting the frequency of each item of processing when the resource consumption reaches a preset threshold value.

According to another aspect of the present invention, there is provided a video compositing apparatus comprising:

a material determination unit adapted to determine a material required for synthesizing a video;

a data frame processing unit adapted to perform a plurality of items of processing on the data frame based on the material; wherein, the processing of each item has dependency relationship and is executed asynchronously;

and the video synthesis unit is suitable for synthesizing the video according to the final processing result of the data frame.

Preferably, the material comprises video material and/or picture material.

Preferably, the treatment comprises one or more of:

extracting data frames from the material;

decoding the data frame;

modifying the decoded data frame;

encoding the modified data frame;

the data frame is written to a file.

Preferably, the data frame processing unit is further adapted to start a plurality of threads, each thread performing a respective processing on a data frame.

Preferably, the data frame processing unit is further adapted to place the processing result of the data frame into the buffer corresponding to the thread by each thread, and read the processing result from the buffer of the thread depending on each thread as the processing object of the thread.

Preferably, each processing result contains a time stamp.

Preferably, the data frame processing unit is further adapted to suspend execution of the corresponding thread when the buffer corresponding to each thread is full.

Preferably, the apparatus is applied to a mobile device, and the apparatus further includes:

and the resource monitoring unit is suitable for monitoring the resources of the mobile equipment, and when the resource consumption reaches a preset threshold value, the frequency of each item of processing is adjusted.

In accordance with still another aspect of the present invention, there is provided an electronic apparatus including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a method as any one of the above.

According to a further aspect of the invention, there is provided a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement a method as any one of the above.

From the above, the technical solution of the present invention is to determine the material required for synthesizing the video, and execute a plurality of processing on the data frame according to the material; wherein, the processing of each item has dependency relationship and is executed asynchronously; and synthesizing the video according to the final processing result of the data frame. The material sub-data frames and the sub-processing items are processed, and the processing items are mutually dependent, so that the data frames can be processed orderly. And each processing item is asynchronously executed, so that the processing time of the material can be effectively saved, and the processing efficiency is improved.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 shows a schematic flow diagram of a video compositing method according to an embodiment of the invention;

fig. 2 is a schematic structural diagram of a video compositing apparatus according to an embodiment of the invention;

FIG. 3 shows a schematic structural diagram of an electronic device according to one embodiment of the invention;

fig. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Fig. 1 shows a flow diagram of a video composition method according to an embodiment of the invention. As shown in fig. 1, the method includes:

in step S110, materials required for synthesizing a video are determined.

The material required for synthesizing the video is actually the material which needs to be processed to a certain extent and is output as a video file. For example, an original video shot by a user needs to be played after adding a special picture effect, and the original video is the required material.

Step S120, executing a plurality of items of processing related to the data frame according to the material; wherein, the processing items have dependency relationship and are executed asynchronously.

A plurality of processing on data frames is performed on the basis of the material, i.e. the material is processed, or rendered, so to speak, to obtain a more effective video file. The processing of the material needs to be performed by data frames, and the processing of each data frame needs to be performed by different processing items. After the execution of the previous processing item is completed, the latter processing item is processed again on its processing result. Here, different processing items are handled by different units, for example, a first unit performs a first processing item and a second unit performs a second processing item. After the first unit executes the first processing item on the first data frame, the processing result is output to the second unit, and the second unit executes the second processing item on the processing result. At the same time, the first unit starts to perform the first processing item on the second data frame. Therefore, each processing item is asynchronously executed, the processing time of the material can be effectively saved, and the processing efficiency is improved.

Step S130, synthesizing a video according to the final processing result of the data frame.

And synthesizing the final processing result of each data frame in the steps into a video to finish the processing of the material. For example, a first data frame added with a firework special effect and a second data frame added with a snowflake special effect are combined into a video, and a new video file with a better picture effect is obtained.

According to the technical scheme, materials required by video synthesis are determined, and a plurality of items of processing related to data frames are executed according to the materials; wherein, the processing of each item has dependency relationship and is executed asynchronously; and synthesizing the video according to the final processing result of the data frame. The material sub-data frames and the sub-processing items are processed, and the processing items are mutually dependent, so that the data frames can be processed orderly. And each processing item is asynchronously executed, so that the processing time of the material can be effectively saved, and the processing efficiency is improved.

In a specific embodiment, as shown in the method of fig. 1, the material in step S110 includes video material and/or picture material.

The material of the composite video may be a video or a picture. For example, when a user takes a video, a certain special effect of a picture needs to be added to the video or characters and scenes appearing in the video need to be processed, so that a new video file with a better picture effect is obtained.

If the user needs to play several independent pictures in one video, the above video synthesis method can also be used to process the pictures. For example, a picture is processed as a data frame. If only one picture is available, different scene special effects can be added to the same picture, and one scene special effect can be treated as one data frame. For example, the first data frame is a scene for adding fireworks special effect into a picture, and the second data frame is a scene for adding snowflake special effect into the picture.

In a specific embodiment, in the method shown in fig. 1, the processing in step S120 includes one or more of the following: extracting data frames from the material; decoding the data frame; modifying the decoded data frame; encoding the modified data frame; the data frame is written to a file.

The present embodiment provides specific processing items performed on a data frame. The first processing item extracts data frames from the material. If the material is a video, different frames in the video can be directly acquired as a data frame. If the material is a picture, one picture can be used as a data frame.

And a second processing item for decoding the data frame. Data frames extracted from material are encoded files, which generally cannot be edited and modified directly. Decoding is to convert a data frame into a file that can be edited, or into a file that can be rendered, for subsequent processing.

And a third processing item, modifying the decoded data frame. The decoded data frame is an editable file, on which modifications can be made directly. For example, fireworks or snowflake effects, etc. are added thereto.

And a fourth process for encoding the modified data frame. The modified data frame needs to be re-encoded and converted into a non-editable file for output.

And a fifth processing item writing the data frame to a file. And writing all the data frames which finish the processing items into the same video file to finish video synthesis.

The five processing items can be adaptively selected to be processed according to the actual situation of the material. For example, when the material is a complete video material, the above five processing items need to be completely executed; when the material has finished the modified data frame, only need to encode and synthesize it, then correspondingly process the fourth processing item and the fifth processing item. The corresponding processing items can be selected for processing according to specific situations so as to further improve the video synthesis efficiency.

In a specific embodiment, in the method shown in fig. 1, the performing of the plurality of processing on the data frame according to the material in step S120 includes: a plurality of threads are initiated, each thread performing a respective process on the data frame.

The processing operations of the processing items are performed by threads. In order to improve processing efficiency and perform asynchronous execution, each processing item is processed by one thread. That is, when processing a data frame, a plurality of threads need to be started, and each thread executes a processing operation of one processing item.

Taking the above five processing items as an example, when processing a data frame, five threads need to be correspondingly started, a first thread executes a first processing item, a second thread executes a second processing item, and so on. The first thread completes the first processing item of the first data frame, outputs the processing result to the second thread, and the second thread executes the second processing item on the processing result. At the same time, the first thread begins executing the first processing item on the second data frame, and so on. The threads work independently, the latter thread executes processing depending on the processing result of the former thread, and the processing and the synthesis of the data frame are completed by mutual cooperation, so that the working efficiency of video synthesis is effectively improved.

In a specific embodiment, the method, wherein the executing the processing on the data frame by each thread respectively includes: each thread puts the processing result of the data frame into the cache region corresponding to the thread, and reads the processing result from the cache region of the thread depending on the thread, and the processing result is taken as the processing object of the thread.

The processing rate of each thread may be different due to the execution of different processing items, which may result in a situation where the threads cannot be tightly connected. For example, the processing rate of the first thread is higher than that of the second thread, when the first thread completes processing the second data frame, the second thread does not complete processing the first data frame, so that the first thread cannot output the processing result of the second data frame, and the second thread needs to wait for completing the current processing. The latency directly affects the processing of subsequent threads, thereby reducing the processing efficiency of video compositing.

In order to avoid the above situation, the processing efficiency of video composition is further improved. Each thread is provided with a corresponding cache region, the processing result is stored in the cache region, and the subsequent thread reads the processing result from the cache region of the previous thread for processing. For example, the first thread finishes processing the first data frame, puts the processing result into the first buffer area, and continues to read the second data frame to execute the first processing item. The second thread reads the processing result of the first data frame from the first cache region to execute the second processing item. When the first thread finishes processing the second data frame, the second thread does not finish processing the first data frame, and cannot process the second data frame. At this time, the first thread puts the processing result of the second data frame into the first buffer area, and then continues to read the third data frame to execute the first processing item. And after the second thread finishes the current operation, reading the processing result of the second data frame from the buffer area for processing. That is to say, as long as the capacity of the buffer area is enough to store the processing result, the thread can always execute the processing operation, the thread waiting is avoided, the working efficiency of the thread is effectively improved, and the efficiency of the whole video synthesis is improved.

Of course, to ensure that threads can read processing results from the correct cache, the dependencies between threads are set. Between two adjacent threads, the latter thread depends on the former thread, and reads the processing result from the cache region of the former thread. For example, a second thread depends on a first thread, a third thread depends on a second thread, and so on. The second thread reads the processing result from the cache region of the first thread, while the third thread can only read the processing result from the cache region of the second thread and cannot read the processing result from the cache region of the first thread, so that each data frame can be processed by each thread in sequence, omission is avoided, and the processing accuracy is ensured.

In a specific embodiment, in the method, each processing result includes a time stamp.

When a large number of processing results are stored in the cache region, how to determine the processing result read by the thread is also a problem to be solved.

As can be seen from the above embodiments, the thread executes the processing items for each data frame in turn. For example, a video material is sequentially divided into a first data frame, a second data frame, and a third data frame, and when a thread is processed, the thread also needs to sequentially process the first data frame, the second data frame, and the third data frame in the order. Therefore, when the data frame is written into the video file finally, the data frame does not need to be reordered, and the data frame is directly written into the video file to be output. Therefore, in order to ensure that the thread reads the processing result of the contended data frame from the buffer, a time stamp is added to the processing result. The time stamp serves as a tag that can mark each processing result for differentiation.

In a specific example, the time stamps of the processing results in the buffer of the first thread are 9:05, 9:10 and 9:15, respectively. Because the first thread executes the processing items according to the sequence of the first data frame, the second data frame and the third data frame, according to the sequence of the timestamps, it can be determined that the processing result with the timestamp of 9:05 corresponds to the first data frame, the processing result with the timestamp of 9:10 corresponds to the second data frame, and the processing result with the timestamp of 9:15 corresponds to the third data frame. And after the second thread judges the sequence of the time stamps, sequentially reading the processing result of 9:05, the processing result of 9:10 and the processing result of 9:15, and thus accurately processing according to the sequence of the data frames.

Of course, in addition to using the time stamp to mark and distinguish the processing result, other marking methods may be used to distinguish the processing result, for example, adding a digital label to the processing result, and the first data frame corresponds to the number 1, and the second data frame corresponds to the number 2. The setting can be specifically carried out according to the requirements, and the embodiment does not make specific requirements.

In a specific embodiment, in the method, each thread performs a processing on a data frame, and the processing further includes: and when the cache region corresponding to each thread is full, the corresponding thread suspends the execution processing.

Since the capacity of the buffer is limited, if the processing rate of the previous thread is much higher than that of the next thread, the buffer may be full and cannot store more processing results. If the thread continues to execute the processing and inputs the processing result into the buffer, the data in the buffer may overflow, resulting in program crash. Therefore, when the cache is full, the corresponding thread needs to be suspended, and the execution process needs to be resumed after the cache is idle, so as to ensure that the program is executed correctly and smoothly.

In a specific embodiment, as in the method shown in fig. 1, the video composition method is applied in a mobile device, and the method further includes: and monitoring the resources of the mobile equipment, and adjusting the frequency of each item of processing when the resource consumption reaches a preset threshold value.

The video synthesis method of the above embodiment is applied to a mobile device. Due to the fact that resources of the initiating device are limited, the video synthesis method needs to start a plurality of threads to execute processing, and the requirements for resources are large. In order to ensure that the mobile device has enough resources for normal operation of other programs or successful completion of video synthesis, the resources of the mobile device need to be monitored so as to respond in time and adjust accordingly when the resources are insufficient, and release certain resources.

And setting a resource consumption threshold, and adjusting the processing frequency of each processing item when the resource consumption of the mobile equipment reaches the preset threshold, so as to release the resources of the mobile equipment and ensure that other programs can run normally or video synthesis can be completed.

In a specific example, the memory resource of the mobile device is 4G, and when the memory resource consumption reaches 3.5G, the remaining resources cannot support each thread to continue executing each processing item of the composite video at the current frequency. Therefore, the processing frequency of each thread is adjusted, the processing speed of each thread is reduced or some non-urgent threads are suspended, so that certain memory resources are released, and each thread is ensured to smoothly complete video synthesis.

Fig. 2 shows a video compositing apparatus according to an embodiment of the invention. As shown in fig. 2, the apparatus 200 includes:

a material determination unit 210 adapted to determine the material required for synthesizing the video.

A data frame processing unit 220 adapted to perform a plurality of processes with respect to the data frame according to the material; wherein, the processing items have dependency relationship and are executed asynchronously.

A video composition unit 230 adapted to compose a video according to the final processing result of the data frame.

In a specific embodiment, as in the apparatus 200 shown in fig. 2, the material determined by the material determination unit 210 includes video material and/or picture material.

In a specific embodiment, as shown in the apparatus 200 of fig. 2, the processing of the data frame processing unit 220 includes one or more of the following: extracting data frames from the material; decoding the data frame; modifying the decoded data frame; encoding the modified data frame; the data frame is written to a file.

In a specific embodiment, such as the apparatus 200 shown in fig. 2, the data frame processing unit 220 is further adapted to start a plurality of threads, each of which performs a processing related to the data frame.

The processing operation of the processing item is performed by a thread, which is initiated by the data frame processing unit 220. In order to improve processing efficiency and perform asynchronous execution, each processing item is processed by one thread. That is, when processing a data frame, a plurality of threads need to be started, and each thread executes a processing operation of one processing item.

In a specific embodiment, in the above apparatus, the data frame processing unit 220 is further adapted to place the processing result of the data frame into the buffer corresponding to the thread by each thread, and read the processing result from the buffer of the thread depending on each thread as the processing object of the thread.

In a specific embodiment, such as the apparatus 200 shown in FIG. 2, each processing result contains a timestamp.

In a specific embodiment, as shown in the apparatus 200 shown in fig. 2, the data frame processing unit 220 is further adapted to suspend the execution of the processing by the corresponding thread when the buffer corresponding to each thread is full.

In a specific embodiment, the apparatus 200 is applied to a mobile device, such as the apparatus 200 shown in fig. 2. The apparatus 200 further comprises: and the resource monitoring unit is suitable for monitoring the resources of the mobile equipment, and when the resource consumption reaches a preset threshold value, the frequency of each item of processing is adjusted.

The video synthesis apparatus of the above embodiment is applied to a mobile device. Due to the fact that resources of the initiating device are limited, the video synthesis method needs to start a plurality of threads to execute processing, and the requirements for resources are large. In order to ensure that the mobile device has enough resources for normal operation of other programs or successful completion of video synthesis, the resources of the mobile device need to be monitored so as to respond in time and adjust accordingly when the resources are insufficient, and release certain resources.

In summary, according to the technical solution of the present invention, a plurality of data frame processing items are executed according to a material determined to be required for synthesizing a video; wherein, the processing of each item has dependency relationship and is executed asynchronously; and synthesizing the video according to the final processing result of the data frame. The material sub-data frames and the sub-processing items are processed, and the processing items are mutually dependent, so that the data frames can be processed orderly. And each processing item is asynchronously executed, so that the processing time of the material can be effectively saved, and the processing efficiency is improved.

It should be noted that:

the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of a video compositing apparatus, electronic device, and computer-readable storage medium according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

For example, fig. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the invention. The electronic device comprises a processor 310 and a memory 320 arranged to store computer executable instructions (computer readable program code). The memory 320 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 320 has a storage space 330 storing computer readable program code 331 for performing any of the method steps described above. For example, the storage space 330 for storing the computer readable program code may comprise respective computer readable program codes 331 for respectively implementing various steps in the above method. The computer readable program code 331 may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a computer readable storage medium such as described in fig. 4. Fig. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer readable storage medium 400 has stored thereon a computer readable program code 331 for performing the steps of the method according to the invention, readable by a processor 310 of the electronic device 300, which computer readable program code 331, when executed by the electronic device 300, causes the electronic device 300 to perform the steps of the method described above, in particular the computer readable program code 331 stored on the computer readable storage medium may perform the method shown in any of the embodiments described above. The computer readable program code 331 may be compressed in a suitable form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The invention provides a1, a video synthesis method, comprising:

determining materials required by the synthesized video;

A2, the method of a1, wherein the material includes video material and/or picture material.

A3, the method of A1, wherein the treating comprises one or more of:

extracting data frames from the material;

decoding the data frame;

modifying the decoded data frame;

encoding the modified data frame;

the data frame is written to a file.

A4, the method of A1, wherein the performing a plurality of processing on data frames from the material comprises:

a plurality of threads are initiated, each thread performing a respective process on the data frame.

A5, the method as in A4, wherein the executing the processing about the data frame by each thread comprises:

each thread puts the processing result of the data frame into the cache region corresponding to the thread, and reads the processing result from the cache region of the thread depending on the thread, and the processing result is taken as the processing object of the thread.

A6, the method of a5, wherein each processing result includes a timestamp.

A7, the method as in A5, wherein the executing the processing about the data frame by each thread further comprises:

and when the cache region corresponding to each thread is full, the corresponding thread suspends the execution processing.

A8, the method as in a1, wherein the method is applied in a mobile device, the method further comprising:

and monitoring the resources of the mobile equipment, and adjusting the frequency of each item of processing when the resource consumption reaches a preset threshold value.

The present invention also provides B9, a video compositing apparatus, comprising:

B10, the apparatus as in B9, wherein the material comprises video material and/or picture material.

B11, the apparatus of B9, wherein the processing comprises one or more of:

extracting data frames from the material;

decoding the data frame;

modifying the decoded data frame;

encoding the modified data frame;

the data frame is written to a file.

B12, the apparatus according to B9, wherein the data frame processing unit is further adapted to start a plurality of threads, each thread performing a process on a data frame.

The apparatus according to B13 or B12, wherein the data frame processing unit is further adapted to put the processing result of the data frame into the buffer corresponding to the thread by each thread, and read the processing result from the buffer of the thread depending on each thread as the processing target of the thread.

B14, the apparatus as in B13, wherein each processing result contains a time stamp.

B15, the apparatus according to B13, wherein the data frame processing unit is further adapted to suspend the execution of the corresponding thread when the buffer corresponding to each thread is full.

B16, the apparatus according to B9, wherein the apparatus is applied in a mobile device, the apparatus further comprises:

The invention also provides C17, an electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any one of a 1-A8.

The invention also provides D18, a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of a1-a 8.

Claims

1. A video compositing method, comprising:

determining materials required by the synthesized video;

2. The method of claim 1, wherein the material comprises video material and/or picture material.

3. The method of claim 1, wherein the processing comprises one or more of:

extracting data frames from the material;

decoding the data frame;

modifying the decoded data frame;

encoding the modified data frame;

the data frame is written to a file.

4. The method of claim 1, wherein said performing a plurality of processes on a data frame based on said material comprises:

5. The method of claim 4, wherein the threads performing a respective process on the data frame comprises:

6. The method of claim 5, wherein each processing result includes a timestamp.

7. The method of claim 5, wherein the threads performing a respective process on the data frame further comprise:

8. A video compositing apparatus, comprising:

9. An electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-7.

10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-7.