CN113099107A

CN113099107A - Video shooting method, device and medium based on terminal and computer equipment

Info

Publication number: CN113099107A
Application number: CN202110220592.8A
Authority: CN
Inventors: 邵珠超
Original assignee: Wuxi Wentai Information Technology Co ltd
Current assignee: Wuxi Wentai Information Technology Co ltd
Priority date: 2021-02-26
Filing date: 2021-02-26
Publication date: 2021-07-09
Anticipated expiration: 2041-02-26
Also published as: CN113099107B

Abstract

The embodiment of the disclosure relates to the technical field of terminal video recording, in particular to a video shooting method and device based on a terminal. The method comprises the following steps: receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images. By adopting the scheme, the motion blur caused by slow motion shooting is reduced, and the quality of the slow motion video is improved.

Description

Video shooting method, device and medium based on terminal and computer equipment

Technical Field

The embodiment of the disclosure relates to the technical field of terminal video recording, in particular to a terminal-based video shooting method, a terminal-based video shooting device, computer equipment for realizing the terminal-based video shooting method and a computer-readable storage medium.

Background

With the development of electronic technology, various intelligent electronic devices have come along, and mobile phones, tablet computers and the like have become indispensable tools in the life of people. People's requirement to each terminal equipment is also higher and higher, especially the demand that records to the camera is more and more diversified, in order to satisfy the user and utilize the object that terminal equipment can be better catch quick travel, for example: running, jumping, etc. of humans, animals, etc. are optionally recorded in slow motion to capture some details of the subject being photographed to improve the quality of the video being photographed.

The slow motion shooting is to shoot at a higher frame rate speed in the shooting process and play at a normal frame rate speed in the later period, so that the video shows short-time content in a longer time to form a slow motion effect, and the details of a target object can be clearly seen.

However, motion blur (motion blur) is caused by slow motion shooting, and a user can reduce the motion blur by using a small shutter angle and a large aperture and increasing the sensitivity, but the picture definition is reduced by the motion blur, the picture noise is increased, and the video quality of the slow motion shooting is reduced.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a terminal-based video shooting method, apparatus, computer device and computer-readable storage medium, which aim to solve the technical problem of motion blur caused by slow motion shooting in the prior art to some extent.

The embodiment of the application provides a video shooting method based on a terminal, which comprises the following steps: receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction;

acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of the first images and each frame of the second images correspond to each other in sequence;

respectively carrying out image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to the first image and a second target image corresponding to the second image;

performing fusion processing on each frame of the first target image and the corresponding second target image to obtain multiple frames of third target images;

and forming a slow motion video based on the plurality of frames of the third target image.

In an embodiment, the performing image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image respectively to obtain a first target image corresponding to each frame of the first image and a corresponding second target image of the second image includes: extracting first depth information of each pixel point in each frame of the first image and second depth information of each pixel point in each frame of the second image;

obtaining a foreground image and a background image of the corresponding first image based on the first depth information of each pixel point;

obtaining a foreground image and a background image of the corresponding second image based on the second depth information of each pixel point;

respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of the first image to obtain a plurality of corresponding frames of the first target image;

and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of the second image to obtain a plurality of corresponding frames of the second target image. In one embodiment, the method further comprises: sequentially acquiring each frame of first image and each frame of second image in the first image set and the second image set;

respectively carrying out image matching processing on each frame of first image and each frame of corresponding second image so as to extract the same element parts in each frame of first image and each frame of corresponding second image and obtain each frame of first sub-image and each frame of corresponding second sub-image;

respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first sub-image to obtain a plurality of corresponding frames of the first target image;

and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second sub-image to obtain a plurality of corresponding frames of second target images.

In one embodiment, the method further comprises: and before the image matching processing is respectively carried out on each frame of first image and each frame of corresponding second image, image correction processing is respectively carried out on each frame of first image and each frame of corresponding second image.

In one embodiment, the forming a slow motion video based on a plurality of frames of the third target image includes: acquiring intermediate frame images between two adjacent third target images in the plurality of frames of third target images based on an optical flow interpolation algorithm;

and performing frame interpolation processing on a plurality of frames of the third target image based on the intermediate frame image to obtain a slow motion video.

In one embodiment, the method further comprises: previewing the slow motion video in response to a preset operation;

when the slow motion video is previewed, detecting whether a rephoto instruction for the slow motion video is received;

and when the rephotography instruction is received, starting a slow motion shooting mode of the terminal camera.

The embodiment of the application provides a video shooting device based on terminal, the device includes:

the slow motion shooting mode starting unit is used for receiving a slow motion shooting instruction and starting a slow motion shooting mode based on the slow motion shooting instruction;

the image acquisition unit is used for acquiring a first image set acquired by the main camera and a second image set acquired by the auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of the first images and each frame of the second images correspond to each other in sequence;

the image processing unit is used for respectively carrying out image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of the first image and a second target image corresponding to the second image;

the image fusion unit is used for carrying out fusion processing on each frame of the first target image and the corresponding second target image to obtain a plurality of frames of third target images;

and a slow-motion video forming unit configured to form a slow-motion video based on the plurality of frames of the third target image.

In one embodiment, the image processing unit is configured to extract first depth information of each pixel point in each frame of the first image and second depth information of each pixel point in each frame of the second image;

and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of the second image to obtain a plurality of corresponding frames of the second target image.

The embodiment of the application provides computer equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the steps of the video shooting method based on the terminal provided by any embodiment of the application.

The embodiment of the application provides a computer readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to realize the steps of the terminal-based video shooting method provided by any embodiment of the application.

The embodiment of the application provides a terminal-based video shooting method, a terminal-based video shooting device, computer equipment and a computer-readable storage medium, wherein a slow motion shooting mode is started based on a slow motion shooting instruction by receiving the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of the first images and each frame of the second images correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to the first image and a second target image corresponding to the second image; performing fusion processing on each frame of the first target image and the corresponding second target image to obtain multiple frames of third target images; and forming a slow motion video based on the plurality of frames of the third target image. In this way, according to the scheme of the embodiment, when the slow motion mode is started for video shooting, the first image set and the second image set within the preset time of the main camera and the auxiliary camera are obtained, image compensation processing and image blurring processing are respectively performed on the foreground image and the background image of each frame of the first image in the first image set and the second image set and the foreground image and the background image of each frame of the second image in the first image set and the second image set to obtain the corresponding first target image and second target image, image fusion is performed on the first target image and the corresponding second target image to obtain the third target image, so that the foreground image of the image is protruded, the background image is blurred, the slow motion video is formed according to the third target image, the motion blur caused by slow motion shooting is reduced to a certain extent, and the quality of the slow motion video is improved.

Drawings

FIG. 1 is a diagram of an application scenario of a terminal-based video capture method in one embodiment;

FIG. 2 is a flow diagram of a method for terminal-based video capture in one embodiment;

FIG. 3 is a schematic diagram of an interface for slow motion capture in one embodiment;

FIG. 4 is a diagram illustrating a preview of a slow motion video, in one embodiment;

FIG. 5 is a schematic diagram of a terminal-based video capture device in one embodiment;

FIG. 6 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The video shooting method based on the terminal can be applied to the application environment shown in fig. 1. The application environment comprises a terminal 101, wherein a slow motion shooting instruction is received through the terminal 101, and a slow motion shooting mode is started based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images. The terminal 101 may be, but is not limited to, various electronic devices such as a personal computer, a notebook computer, a smart phone, and a tablet computer.

In one embodiment, as shown in fig. 2, a terminal-based video capture method is provided. The present embodiment is mainly illustrated by applying the method to the terminal 101 in fig. 1. In this embodiment, the method includes the steps of:

step S201: and receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction.

The slow motion is a technical method for changing motion realization forms, the basic effect is to prolong the actual motion process, so that the transient change is delayed and amplified, and the main motion is highlighted. The slow motion shooting is to shoot more frames than the normal playing speed in the normal time and then play the frames at the normal frame rate speed, so that the video shows the content in a short time in a longer time to increase the visual experience. In an example, when the terminal 101 captures a video by using a mobile phone, a picture is obtained at 50fps, that is, when the video is captured in one second, 50 frames of images can be obtained, but when the captured video is played, the video is played at 25fps, that is, when the video is played in one second, 25 frames of images can be played, which visually causes an experience that the video is played slowly, it should be noted that, in this embodiment, a frame rate of capturing the video, for example, 50fps, and a frame rate of playing the video, for example, 25fps, may be specifically set as required, and are not limited thereto.

Specifically, the user starts a camera shooting function using the terminal 101, such as a mobile phone, and enters a camera shooting interface, as shown in fig. 3, for the user to enter the camera shooting interface, different shooting modes are included at the bottom of the camera shooting interface, and the shooting modes may be, for example, a slow motion 301, a video 302, a photo 303, and the like, but the disclosure is not limited thereto. In an example, a user starts a camera shooting function at a terminal 101 such as a mobile phone, enters a camera shooting interface, clicks a slow motion 301 virtual key of the camera shooting interface, namely, an instruction for triggering a slow motion shooting mode, the terminal 101 such as the mobile phone receives the slow motion shooting instruction, and responds to the slow motion shooting instruction, and the terminal 101 such as the mobile phone starts the slow motion shooting mode to enter shooting of a slow motion video.

Step S202: acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence.

The preset time refers to a time period for acquiring the first image and the corresponding second image by the main camera and the auxiliary camera in the same time period. The main camera and the auxiliary camera are the same camera, and respectively acquire a corresponding first image set and a corresponding second image set in the process of shooting videos, photos and the like by the main camera and the auxiliary camera, wherein the first image set and the second image set comprise multiple frames of first images and multiple frames of second images which are arranged in sequence, and the first images and the second images of the frames correspond in sequence. It should be noted that, the main camera and the auxiliary camera are arranged adjacent to each other, and when the first image set and the second image set are acquired and acquired, due to a certain distance between the main camera and the auxiliary camera, in a shooting process, angles are different, so that different image factors exist between each frame of the first image set acquired and each frame of the second image set corresponding to each frame of the first image set and the second image set.

It should be noted that, as shown in fig. 1, the two cameras may be vertically adjacent to each other, and are disposed at the rear of the terminal 101, such as a mobile phone. In yet another example, the two cameras may also be disposed laterally adjacent to each other, as a front-facing two camera of the terminal 101 such as a mobile phone, but is not limited thereto, and the disclosure is not particularly limited. Wherein, the two cameras can be a color camera and the other is a black and white camera. Cameras with different focal lengths can be adopted, and of course, the same camera can be adopted for the two cameras, but the disclosure is not limited thereto.

Specifically, after the terminal 101, such as a mobile phone, enters a camera shooting interface and starts a slow motion shooting mode, the camera enters the slow motion shooting mode to perform video shooting, at this time, a main camera and an auxiliary camera in the two cameras respectively acquire and acquire multi-frame images of a shot object, and in the same shooting time period, multi-frame first images in a first image set acquired by the main camera and corresponding multi-frame second images in a second image set acquired by the auxiliary camera are arranged in sequence according to the video shooting time. That is, in the same time period, the number of frames of the first images in the first image set acquired by the main camera is the same as the number of frames of the second images in the second image set acquired by the auxiliary camera, for example, in 1s, the number of frames of the first images in the first image set acquired by the main camera and the number of frames of the second images in the second image set acquired by the auxiliary camera are both 50 frames of images, and each frame of the first images acquired by the main camera and each frame of the second images acquired by the auxiliary camera in the video shooting time period are in one-to-one correspondence with each other according to time, that is, by setting the fixed data stream frame rates of the main camera and the auxiliary camera, when the main camera and the auxiliary camera perform slow motion shooting, the current frame rates are the same, a frame synchronization image is output, so as to ensure that the number of frames of the first images in the first image set acquired by the main camera and the second images acquired by the auxiliary camera are the same, thereby ensuring that the number of frames of the first images in the first image set by the main camera and the two images are in one-to-one correspondence with each other according to the video capturing time.

Step S203: and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first image and the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image.

The foreground image refers to a subject factor in a captured image, that is, a portion that can be interested in one image. The background image refers to a portion of the captured image that is not of interest to a person. For example, in an image, the image includes a lion running on a grassland, the lion running at this time is a foreground image of the image, and other parts such as the grassland are background images. The image compensation processing means that in the image acquisition process, external factors such as the direction of a light source, brightness intensity, color and the like, or external shielding and other factors affect the acquired image, so that the pixel quality of the image is reduced, and therefore compensation processing such as illumination compensation and the like needs to be performed on the image, so that the image quality of the image is improved, and a main factor of the image, namely a foreground image, is highlighted. The image blurring processing is processing such as blurring of a captured image, so that a background image in the image is blurred, and even if the depth of field in the image becomes shallow, a focus is focused on a subject, and a subject factor of the image, that is, a foreground image, is emphasized.

Specifically, according to the first image acquired by the main camera of the terminal 101, such as a mobile phone, and the second image acquired by the auxiliary camera, for example, the first image and the second image of the lion running on the grassland in the above example, the foreground image in the first image and the foreground image of the corresponding second image, such as the lion running on the grassland, are respectively subjected to image compensation processing, so that the foreground image of the first image and the foreground image of the second image, such as the lion running on the grassland, can be more prominent to some extent, and simultaneously the background image in the first image and the background image of the second image, such as the grassland, are respectively subjected to image blurring processing, so that the background image in the first image and the background image in the corresponding second image, such as the grassland, are blurred, and in one example, the background image in the first image and the background image of the corresponding second image are subjected to image blurring processing by using a gaussian blurring algorithm Blurring processing to focus on the foreground image of the first image and the corresponding foreground image of the second image, i.e., the lion running in the figure, but is not limited thereto, and the disclosure is not particularly limited. It should be noted that, in the slow motion video shooting process, image compensation processing is performed on foreground images of other frames of first images in the first image set and foreground images of corresponding other frames of second images in the second image set acquired and acquired by the main camera, and image blurring processing is performed on background images of other frames of first images in the first image set and background images of corresponding other frames of second images in the second image set, so as to obtain first target images corresponding to the first images of each frame and corresponding second target images of the second images.

Step S204: and carrying out fusion processing on each frame of the first target image and the corresponding second target image to obtain a plurality of frames of third target images.

Step S205: and forming a slow motion video based on the plurality of frames of the third target images.

The fusion processing refers to image fusion processing, and beneficial information in respective channels is extracted to the maximum extent by processing image data about the same target collected by a multi-source channel through image processing, computer technology and the like, and finally, a high-quality image is synthesized, so that the utilization rate of image information is improved, the computer interpretation precision and reliability are improved, and the spatial resolution and the spectral resolution of an original image are improved. The slow motion video refers to a video with a playing speed reduced from a normal playing speed in a video playing process, for example, when a user takes a video and records the video at 50fps, the video is played through a normal video, for example, 25fps, and the video is 2 times full of the normal playing speed.

Specifically, image compensation processing and image blurring processing are respectively performed on a foreground image and a background image of each frame of first image and each frame of corresponding second image acquired by a main camera and an auxiliary camera respectively, so that a foreground image of the image, such as a running lion, is projected, and a background image of the image, such as a grassland and other parts, is blurred to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image respectively, and the acquired first target image and the acquired second target image are subjected to fusion processing, so that multiple frames of third target images are obtained, and the quality of the images is improved. The image fusion processing algorithm may, for example, adopt a pixel-level image fusion algorithm, a feature-level image fusion algorithm, and the like to perform image fusion processing on each frame of the acquired first target image and each frame of the corresponding second target image, so as to acquire multiple frames of third target images. And forming a slow motion video according to the acquired multiple frames of third target images.

In the embodiment, a slow motion shooting instruction is received through the terminal 101, and a slow motion shooting mode is started based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images. In this way, when the slow motion mode is turned on to perform video shooting, the first image set and the second image set within the preset time of the main camera and the auxiliary camera are obtained, image compensation processing and image blurring processing are respectively performed on the foreground image and the background image of each frame of the first image in the first image set and the second image set and the foreground image and the background image of each frame of the second image in the first image set and the second image set to obtain the corresponding first target image and second target image, image fusion is performed on the first target image and the corresponding second target image to obtain the third target image, so that the foreground image of the image is protruded, the background image is blurred, a slow motion video is formed according to the third target image, motion blur caused by slow motion shooting is reduced to a certain extent, and the quality of the slow motion video is improved.

On the basis of the foregoing embodiments, in some embodiments of the present disclosure, performing image compensation processing and image blurring processing on the foreground image and the background image of each frame of the first image and the corresponding second image, respectively, to obtain a first target image corresponding to each frame of the first image and a second target image corresponding to the second image, includes extracting first depth information of each pixel point in each frame of the first image and second depth information of each pixel point in each frame of the second image; obtaining a foreground image and a background image of a corresponding first image based on the first depth information of each pixel point; and obtaining a foreground image and a background image of the corresponding second image based on the second depth information of each pixel point.

The depth information refers to the number of bits required to actually store the gray scale or color of the image in the pixel depth.

Specifically, a certain distance exists between the main camera and the auxiliary camera in the setting, images of the same scene formed by the two lenses have certain difference, namely parallax, pixel points corresponding to each frame of second image in each frame of first image set acquired by the main camera and the auxiliary camera and in each frame of second image set in the second image set are acquired by detection, the pixel points corresponding to each frame of first image and each frame of second image are matched, parallax data are obtained according to matching, and first depth information of each pixel point in each frame of first image and second depth information of each pixel point in each frame of second image are acquired according to the acquired parallax data and the distance and the focal length of the double cameras. Specifically, a real-time depth background model is initialized through the acquired first depth information of each pixel point in each frame of first image, when the main camera collects each frame of first image in the first image set, the real-time depth background model is updated, and a foreground image and a background image of the current first image are extracted according to the depth background model and the currently collected real-time first image. Meanwhile, initializing a real-time depth background model through the acquired second depth information of each pixel point in each frame of second image, updating the real-time depth background model when the auxiliary camera acquires each frame of second image in the second image set, and extracting a foreground image and a background image of the current second image according to the depth background model and the currently acquired real-time second image.

Respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second image to obtain corresponding multi-frame second target images. For a detailed description, please refer to the description of step S203 in the above embodiment, which is not described herein.

In this way, in this embodiment, the first depth information of each pixel point in each frame of first image in the first image set and the second depth information of each pixel point in each frame of second image in the second image set, which are collected and extracted by the main camera, are obtained, the foreground image and the background image of each frame of first image and the foreground image and the background image of each frame of second image are respectively obtained according to the first depth information of each pixel point in each frame of first image and the second depth information of each pixel point in each frame of second image, the foreground image and the background image of each frame of first image are respectively subjected to image compensation processing and image blurring processing to obtain corresponding first target images, and the foreground image and the background image of each frame of second image are respectively subjected to image compensation processing and image blurring processing to obtain corresponding second target images to protrude the foreground image of the first image, and the effect of image blurring processing is improved, and the motion blur caused by slow motion shooting is reduced to a certain extent, so that the quality of a slow motion video is improved.

On the basis of the foregoing embodiments, in some embodiments of the present disclosure, the method for terminal-based video shooting further includes sequentially acquiring each frame of the first image and the corresponding frame of the second image in the first image set and the second image set. And respectively carrying out image matching processing on each frame of first image and each frame of corresponding second image so as to extract the same element parts in each frame of first image and each frame of corresponding second image, and obtaining each frame of first sub-image and each frame of corresponding second sub-image.

Specifically, because a certain distance may exist between the two cameras in the setting, when the user starts the slow motion shooting mode, the terminal 101 enters a shooting interface of the camera, for example, a mobile phone, and there is a shooting angle difference when shooting the same object due to the distance between the two cameras in the shooting process, there may be a situation that a partial image is not simultaneously shot by the main camera and the auxiliary camera in each frame of the acquired first image and each frame of the corresponding second image. Feature point matching is carried out on each frame of first image and the corresponding second image, so that the same element parts in each frame of first image and the corresponding second image are extracted, and each frame of first sub-image and the corresponding second sub-image are obtained, namely the image content of each frame of first sub-image and the image content of the corresponding second sub-image are the same.

On the basis of the above embodiments, the foreground image and the background image of each frame of the first sub-image are respectively subjected to image compensation processing and image blurring processing, so as to obtain a corresponding plurality of frames of the first target image. And respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second sub-image to obtain a plurality of corresponding frames of second target images. For a detailed description, please refer to the description of step S203 in the above embodiment, which is not described herein.

In this way, in the embodiment, the same element parts in each frame of the first image and the corresponding second image in the acquired first image set and second image set are extracted by performing image matching processing on each frame of the first image and the corresponding second image to obtain each frame of the first sub-image and the corresponding second sub-image, and the foreground image and the background image of each frame of the first sub-image and the corresponding second sub-image are respectively subjected to image compensation processing and image blurring processing to acquire a plurality of frames of first target images and a plurality of frames of second target images.

On the basis of the foregoing embodiments, in some embodiments of the present disclosure, the method for terminal-based video shooting further includes performing image correction processing on each frame of the first image and each frame of the second image before performing image matching processing on each frame of the first image and each frame of the second image.

The image correction is to restore the original image of the distorted image by establishing a mathematical model according to the distortion cause of the image, extracting the required information from the contaminated or distorted image signal, and restoring the original image of the image along the inverse process of the distortion of the image.

Specifically, when the terminal 101, such as a mobile phone, performs slow motion shooting, the main camera and the auxiliary camera acquire multiple frames of first images and multiple frames of second images, and perform Image correction Processing on the first images and the second images corresponding to the respective frames of the multiple frames of first images and the multiple frames of second images, respectively, where the Image correction Processing may be, for example, post-Processing on the first images and the second images corresponding to the respective frames by an Image Signal Processing (ISP), such as performing linear correction, noise removal, dead pixel removal, interpolation, white balance, automatic exposure control, and the like on the first images and the second images corresponding to the respective frames, so that the embodiment performs Image correction Processing on the first images and the second images corresponding to the respective frames of the multiple frames of first images and multiple frames of second images acquired by the main camera and the auxiliary camera in advance, and restores details of a shot object to a certain extent well, the quality of the image is improved, and the motion blur caused by slow motion shooting is reduced, so that the quality of the slow motion video is improved.

On the basis of the above embodiments, in some embodiments of the present disclosure, an intermediate frame image between two adjacent third target images in the plurality of frames of third target images is acquired based on an optical flow interpolation algorithm. And performing frame interpolation processing on the multiple frames of third target images based on the intermediate frame images to obtain the slow motion video. The optical flow interpolation is a method for finding the corresponding relation between the previous frame and the current frame by using the change of pixels in an image sequence in a time domain and the correlation between adjacent frames so as to calculate the motion information of an object between the adjacent frames, namely, the optical flow is generated by the self movement of a foreground object in a scene, the movement of a camera or the common movement of the foreground object and the camera.

Specifically, the slow motion algorithm mainly obtains intermediate frame images between adjacent ones of the multiple frames of third target images by an optical flow interpolation method, and is implemented by frame interpolation according to the obtained intermediate frame images, namely, the generated intermediate frames are inserted into two adjacent frames of third target images in the multiple frames of third target images. In an example, a common video of 30fps can be subjected to frame interpolation calculation through a frame interpolation algorithm, and a slow motion video of 240fps is further obtained, so that the embodiment maintains that a first image and a second image of multiple frames respectively acquired by a main camera and an auxiliary camera are not distorted to a certain extent through frame interpolation processing of the video, and reduces motion blur caused by slow motion shooting to a certain extent, thereby improving the quality of the slow motion video.

On the basis of the above embodiments, in some embodiments of the present disclosure, the method for terminal-based video capture further includes previewing the slow-motion video in response to a preset operation. When previewing the slow motion video, whether a rephoto instruction for the slow motion video is received is detected. And when a rephotography command is received, starting a slow motion shooting mode of the terminal camera.

Specifically, a user enters a camera shooting interface through the terminal 101 such as a mobile phone to trigger a preview slow motion video instruction, when the terminal 101 such as the mobile phone receives the preview slow motion video instruction, the terminal 101 such as the mobile phone plays a video recorded in a slow motion shooting mode, as shown in fig. 4, the interface diagram is an interface diagram of a video recorded by the user in the terminal 101 such as the mobile phone in the slow motion shooting mode, and the playing interface diagram includes a virtual rephotograph key 401, a virtual editing key 402, a virtual forwarding key 403, and the like, but is not limited thereto. When a user watches the slow-motion video, the user thinks that the video recording visual effect is not good, and at the moment, the user clicks the virtual rephotograph key 401, the terminal 101 such as a mobile phone detects that a rephotograph instruction for the slow-motion video is received, responds to the rephotograph instruction for the slow-motion video, and the camera of the terminal 101 such as the mobile phone reenters a camera shooting interface and starts a slow-motion shooting mode to shoot the slow-motion video. Therefore, the slow motion video is watched by the user, and the slow motion shooting mode can be directly started on the playing interface when the user feels bad, so that the operation of shooting again is reduced, and the user experience is improved.

It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 5, a terminal-based video shooting device is provided, and the device includes a slow motion shooting mode starting unit 501, an image obtaining unit 502, an image processing unit 503, an image fusion unit 504, and a slow motion video forming unit 505, where the slow motion shooting mode starting unit 501 is configured to receive a slow motion shooting instruction and start a slow motion shooting mode based on the slow motion shooting instruction. The image acquisition unit 502 is configured to acquire a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within a preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; the image processing unit 503 is configured to perform image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image, respectively, to obtain a first target image corresponding to each frame of the first image and a corresponding second target image of the second image; the image fusion unit 504 is configured to perform fusion processing on each frame of the first target image and the corresponding second target image to obtain multiple frames of the third target image. The slow motion video forming unit 505 is configured to form a slow motion video based on the plurality of frames of the third target image.

In an embodiment of the present invention, the image processing unit 503 is further configured to extract first depth information of each pixel point in each frame of the first image and second depth information of each pixel point in each frame of the second image; obtaining a foreground image and a background image of a corresponding first image based on the first depth information of each pixel point; obtaining a foreground image and a background image of a corresponding second image based on the second depth information of each pixel point; respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second image to obtain corresponding multi-frame second target images.

In an embodiment of the present invention, the image obtaining unit 502 is further configured to sequentially obtain each frame of the first image in the first image set and each frame of the second image; respectively carrying out image matching processing on each frame of first image and each frame of corresponding second image so as to extract the same element parts in each frame of first image and each frame of corresponding second image and obtain each frame of first sub-image and each frame of corresponding second sub-image; the image processing unit 503 is further configured to perform image compensation processing and image blurring processing on the foreground image and the background image of each frame of the first sub-image, respectively, to obtain multiple corresponding frames of the first target image; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second sub-image to obtain a plurality of corresponding frames of second target images. In an embodiment of the present invention, the image obtaining unit 502 further includes an image correction unit, configured to perform image correction processing on each frame of the first image and each frame of the second image before performing image matching processing on each frame of the first image and each frame of the second image. In an embodiment of the present invention, the slow-motion video forming unit 505 is further configured to obtain an intermediate frame image between two adjacent third target images in the plurality of frames of third target images based on an optical flow interpolation algorithm; and performing frame interpolation processing on the multiple frames of third target images based on the intermediate frame images to obtain the slow motion video.

In an embodiment of the present invention, the slow motion video forming unit 505 further includes a slow motion video previewing unit, configured to preview a slow motion video in response to a preset operation; when a slow motion video is previewed, detecting whether a rephoto instruction for the slow motion video is received or not; and when a rephoto command is received, starting a slow motion shooting mode of the terminal camera.

In the above embodiment, the slow motion shooting mode turning-on unit 501 is configured to receive a slow motion shooting instruction, and turn on the slow motion shooting mode based on the slow motion shooting instruction. The image acquisition unit 502 is configured to acquire a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within a preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; the image processing unit 503 is configured to perform image compensation processing and image blurring processing on each frame of the first image and the foreground image and the background image of the corresponding second image, respectively, to obtain a first target image corresponding to each frame of the first image and a corresponding second target image of the second image; the image fusion unit 504 is configured to perform fusion processing on each frame of the first target image and the corresponding second target image to obtain multiple frames of the third target image. The slow motion video forming unit 505 is configured to form a slow motion video based on the plurality of frames of the third target image. In this way, when the slow motion mode is turned on to perform video shooting, the first image set and the second image set within the preset time of the main camera and the auxiliary camera are obtained, image compensation processing and image blurring processing are respectively performed on the foreground image and the background image of each frame of the first image in the first image set and the second image set and the foreground image and the background image of each frame of the second image in the first image set and the second image set to obtain the corresponding first target image and second target image, image fusion is performed on the first target image and the corresponding second target image to obtain the third target image, so that the foreground image of the image is protruded, the background image is blurred, a slow motion video is formed according to the third target image, motion blur caused by slow motion shooting is reduced to a certain extent, and the quality of the slow motion video is improved.

For specific limitations of the terminal-based video capture device, reference may be made to the above limitations of the terminal-based video capture method, which are not described herein again. The respective modules in the above-described terminal-based video photographing apparatus may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 6. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized by WiFi, operator network, Near Field Communication (NFC) or other technologies. The computer program is executed by a processor to implement a method of terminal-based video capture. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the terminal-based video capture device provided herein may be implemented in the form of a computer program that is executable on a computer device such as that shown in fig. 6. The memory of the computer device may store various program modules constituting the apparatus for terminal-based video capture, such as a slow motion capture mode activation unit 501, an image acquisition unit 502, an image processing unit 503, an image fusion unit 504, and a slow motion video formation unit 505 shown in fig. 5, and a computer program constituted by the various program modules causes a processor to execute the steps in the terminal-based video capture method according to the embodiments of the present application described in the present specification.

For example, the computer apparatus shown in fig. 5 may execute step S201, the image acquisition unit 502 execute step S202, the image processing unit 503 execute step S203, the image fusion unit 504 execute step S204, and the slow-motion video forming unit 505 execute step S205 by the slow-motion shooting mode activation unit 501 in the terminal-based video shooting apparatus shown in fig. 5.

In one embodiment, there is provided a computer device comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program: receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images.

In one embodiment, the processor, when executing the computer program, further performs the steps of: respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image, wherein the image compensation processing and the image blurring processing comprise the following steps: extracting first depth information of each pixel point in each frame of first image and second depth information of each pixel point in each frame of second image; obtaining a foreground image and a background image of a corresponding first image based on the first depth information of each pixel point; obtaining a foreground image and a background image of a corresponding second image based on the second depth information of each pixel point; respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second image to obtain corresponding multi-frame second target images.

In one embodiment, the processor, when executing the computer program, further performs the steps of: sequentially acquiring each frame of first image in the first image set and each frame of second image in the second image set and corresponding each frame of second image; respectively carrying out image matching processing on each frame of first image and each frame of corresponding second image so as to extract the same element parts in each frame of first image and each frame of corresponding second image and obtain each frame of first sub-image and each frame of corresponding second sub-image; respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first sub-image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second sub-image to obtain a plurality of corresponding frames of second target images.

In one embodiment, the processor, when executing the computer program, further performs the steps of: before image matching processing is carried out on each frame of first image and each frame of corresponding second image, image correction processing is carried out on each frame of first image and each frame of corresponding second image. In one embodiment, the processor, when executing the computer program, further performs the steps of: forming a slow-motion video based on the plurality of frames of the third target image includes: acquiring an intermediate frame image between two adjacent third target images in the multiple frames of third target images based on an optical flow interpolation algorithm; and performing frame interpolation processing on the multiple frames of third target images based on the intermediate frame images to obtain the slow motion video.

In one embodiment, the processor, when executing the computer program, further performs the steps of: previewing the slow motion video in response to a preset operation; when a slow motion video is previewed, detecting whether a rephoto instruction for the slow motion video is received; and when a rephotography command is received, starting a slow motion shooting mode of the terminal camera.

In the above embodiment, by receiving a slow motion shooting instruction, the slow motion shooting mode is turned on based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images. In this way, when the slow motion mode is turned on to perform video shooting, the first image set and the second image set within the preset time of the main camera and the auxiliary camera are obtained, image compensation processing and image blurring processing are respectively performed on the foreground image and the background image of each frame of the first image in the first image set and the second image set and the foreground image and the background image of each frame of the second image in the first image set and the second image set to obtain the corresponding first target image and second target image, image fusion is performed on the first target image and the corresponding second target image to obtain the third target image, so that the foreground image of the image is protruded, the background image is blurred, a slow motion video is formed according to the third target image, motion blur caused by slow motion shooting is reduced to a certain extent, and the quality of the slow motion video is improved.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction; acquiring a first image set acquired by a main camera and a second image set acquired by an auxiliary camera within preset time; the first image set and the second image set respectively comprise a plurality of frames of first images and a plurality of frames of second images which are arranged in sequence, and each frame of first image and each frame of second image correspond to each other in sequence; respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image; performing fusion processing on each frame of first target image and corresponding second target image to obtain a plurality of frames of third target images; and forming a slow motion video based on the plurality of frames of the third target images.

In one embodiment, the computer program when executed by the processor implements the steps of: respectively carrying out image compensation processing and image blurring processing on each frame of first image and the foreground image and the background image of the corresponding second image to obtain a first target image corresponding to each frame of first image and a second target image corresponding to the second image, wherein the image compensation processing and the image blurring processing comprise the following steps: extracting first depth information of each pixel point in each frame of first image and second depth information of each pixel point in each frame of second image; obtaining a foreground image and a background image of a corresponding first image based on the first depth information of each pixel point; obtaining a foreground image and a background image of a corresponding second image based on the second depth information of each pixel point; respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second image to obtain corresponding multi-frame second target images. In one embodiment, the computer program when executed by the processor implements the steps of: sequentially acquiring each frame of first image in the first image set and each frame of second image in the second image set and corresponding each frame of second image; respectively carrying out image matching processing on each frame of first image and each frame of corresponding second image so as to extract the same element parts in each frame of first image and each frame of corresponding second image and obtain each frame of first sub-image and each frame of corresponding second sub-image; respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of first sub-image to obtain corresponding multi-frame first target images; and respectively carrying out image compensation processing and image blurring processing on the foreground image and the background image of each frame of second sub-image to obtain a plurality of corresponding frames of second target images. In one embodiment, the computer program when executed by the processor implements the steps of: before image matching processing is carried out on each frame of first image and each frame of corresponding second image, image correction processing is carried out on each frame of first image and each frame of corresponding second image. In one embodiment, the computer program when executed by the processor implements the steps of: forming a slow-motion video based on the plurality of frames of the third target image includes: acquiring an intermediate frame image between two adjacent third target images in the multiple frames of third target images based on an optical flow interpolation algorithm; and performing frame interpolation processing on the multiple frames of third target images based on the intermediate frame images to obtain the slow motion video.

In one embodiment, the computer program when executed by the processor implements the steps of: previewing the slow motion video in response to a preset operation; when a slow motion video is previewed, detecting whether a rephoto instruction for the slow motion video is received; and when a rephotography command is received, starting a slow motion shooting mode of the terminal camera.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM is available in many forms, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), and the like.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A video shooting method based on a terminal is provided with a main camera and an auxiliary camera, and is characterized by comprising the following steps:

receiving a slow motion shooting instruction, and starting a slow motion shooting mode based on the slow motion shooting instruction;

2. The method according to claim 1, wherein the performing image compensation processing and image blurring processing on each frame of the foreground image and the background image of the first image and the corresponding second image respectively to obtain a first target image corresponding to the first image and a corresponding second target image of the second image for each frame comprises:

extracting first depth information of each pixel point in each frame of the first image and second depth information of each pixel point in each frame of the second image;

3. The method according to any one of claims 1 to 2, further comprising:

sequentially acquiring each frame of first image and each frame of second image in the first image set and the second image set;

4. The method of claim 3, further comprising:

and before the image matching processing is respectively carried out on each frame of first image and each frame of corresponding second image, image correction processing is respectively carried out on each frame of first image and each frame of corresponding second image.

5. The method according to any one of claims 1 to 2, wherein the forming of the slow motion video based on the plurality of frames of the third target image comprises:

acquiring intermediate frame images between two adjacent third target images in the plurality of frames of third target images based on an optical flow interpolation algorithm;

6. The method of claim 5, further comprising:

previewing the slow motion video in response to a preset operation;

7. A terminal-based video capture device, comprising:

8. The apparatus of claim 7, wherein the image processing unit is configured to:

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.