WO2022206312A1 - 全景视频的自动剪辑方法、装置、终端及存储介质 - Google Patents
全景视频的自动剪辑方法、装置、终端及存储介质 Download PDFInfo
- Publication number
- WO2022206312A1 WO2022206312A1 PCT/CN2022/079779 CN2022079779W WO2022206312A1 WO 2022206312 A1 WO2022206312 A1 WO 2022206312A1 CN 2022079779 W CN2022079779 W CN 2022079779W WO 2022206312 A1 WO2022206312 A1 WO 2022206312A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video frame
- time point
- video
- point pair
- timestamp
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000009877 rendering Methods 0.000 claims abstract description 100
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 58
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 240000007643 Phytolacca americana Species 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Definitions
- the invention belongs to the technical field of video processing, and in particular relates to an automatic editing method, device, terminal and storage medium for panoramic video.
- One shot is widely used in video post-processing.
- unnecessary video clips for example, clips containing obstacles
- multiple video clips are cut out, and then multiple video clips are connected using software technology. Together, the video makes people feel that multiple clips were shot in one shot, without obvious frame skipping.
- an embodiment of the present invention provides an automatic editing method for panoramic video, the method includes the following steps:
- the method before the step of obtaining the second time point pair by using a preset feature matching algorithm based on the first time point pair, the method further includes:
- the step of obtaining a second time point pair using a preset feature matching algorithm based on the first time point pair includes:
- the second time point pair is obtained by using a preset feature matching algorithm based on the panoramic picture of each video frame and the first time point pair.
- the first time point pair includes a first initial timestamp and a first termination timestamp
- the step of obtaining the second time point pair by using a preset feature matching algorithm based on the first time point pair includes the following steps: :
- the timestamp of the current first video frame and the found timestamp of the second video frame are set as the second time point pair, if the first video frame is not found two video frames, then jump to the step of determining the current first video frame according to the first initial timestamp.
- the step of using the feature matching algorithm to find a second video frame matching the first video frame includes:
- the first gaze point area picture is determined according to the shooting angle of the current first video frame, and the feature matching algorithm is used to find a second video frame matching the first gaze point area image.
- the first time point pair includes a first initial timestamp and a first termination timestamp
- the step of using a preset feature matching algorithm based on the first time point pair to obtain the second time point pair further include:
- the timestamp of the current third video frame and the found timestamp of the fourth video frame are set as the second time point pair; if the fourth video frame is not found four video frames, then jump to the step of determining the current third video frame according to the second initial timestamp.
- the step of using the feature matching algorithm to find the fourth video frame matching the current third video frame includes:
- the second gaze point area picture is determined according to the shooting angle of the current third video frame, and the feature matching algorithm is used to find a fourth video frame matching the second gaze point area image.
- the step of calculating the rendering parameters of the rendering model for the corresponding animation interpolation type according to the shooting angle of view of the current video frame or the second time point includes:
- the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame
- the current decoding timestamp is within the time period formed by the second time point pair, the current decoding timestamp, the shooting angle of the video frame corresponding to the second time point pair, and using the animation interpolation type Interpolation calculation, calculate the rendering parameters of the rendering model according to the interpolation calculation result.
- the animation interpolation type is one or more of linear, slow in and fast out, fast in and slow out, or first slow, then fast and then slow.
- the second time point pair includes a second initial timestamp and a second termination timestamp, and before the step of rendering the panorama video picture fitted on the rendering model according to the rendering parameter, further includes :
- the step of rendering the panoramic video picture fitted on the rendering model according to the rendering parameters includes:
- the panoramic video picture fitted on the rendering model is rendered according to the rendering parameter and the gradient fusion parameter.
- an embodiment of the present invention provides an automatic editing device for panoramic video, wherein the device includes:
- a mark obtaining unit configured to obtain the first time point pair marked in the panoramic video file, the first time point pair representing the video segment that the user desires to cut;
- a cropping area determination unit configured to use a preset feature matching algorithm based on the first time point pair to obtain a second time point pair, the second time point pair representing the actual video segment to be cropped;
- a video generation unit configured to calculate the rendering parameters of the rendering model for the corresponding animation interpolation type according to the shooting angle of the current video frame or the second time point, and according to the rendering parameters, pair the panoramic video picture fitted on the rendering model. Render until the current video frame is the last frame, resulting in a clipped flat video.
- an embodiment of the present invention also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program When implementing the steps of the method described above.
- an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
- the present invention obtains the first time point pair marked in the panoramic video file, the first time point pair represents the video segment that the user expects to cut out, and uses a preset feature matching algorithm to obtain the second time point pair based on the first time point pair, and the third time point pair is obtained.
- the two time-point pairs represent the actual video clips to be cropped, the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame or the corresponding animation interpolation type of the second time-point pair, and the panoramic video images fitted to the rendering model are calculated according to the rendering parameters. Rendering is performed, thereby realizing automatic editing of one-shot video to the end, reducing the editing complexity and improving the editing efficiency.
- Fig. 1 is the realization flow chart of the automatic editing method of panoramic video provided by the first embodiment of the present invention
- Embodiment 2 is a schematic structural diagram of an automatic editing device for panoramic video provided by Embodiment 2 of the present invention.
- FIG. 3 is a schematic structural diagram of a terminal according to Embodiment 3 of the present invention.
- FIG. 1 shows the implementation process of the automatic editing method for panoramic video provided by the first embodiment of the present invention. For the convenience of description, only the part related to the embodiment of the present invention is shown, and the details are as follows:
- step S101 a first time point pair marked in the panoramic video file is acquired.
- the embodiments of the present invention are applicable to terminal equipment, and the terminal equipment includes a computer, a camera, a smart phone, a tablet, etc., and the terminal equipment can implement the method by installing corresponding panoramic video processing software or plug-ins.
- the first time point pair includes a first initial time stamp and a first termination time stamp
- the first time point pair represents a video segment that the user desires to cut, that is, the first initial time stamp and the first end time stamp.
- the video clips between the termination time stamps are the video clips that the user desires to cut out, and the first time point pair can be obtained according to the user's point editing operation during the shooting process.
- step S102 a preset feature matching algorithm is used to obtain a second time point pair based on the first time point pair.
- a preset feature matching algorithm is used to obtain a second time point pair based on the first time point pair, where the second time point pair represents the actual video segment to be trimmed, and the second time point pair includes the second initial timestamp and
- the second termination timestamp that is, the video segment between the second initial timestamp and the second termination timestamp is the actual video segment to be cut obtained by using the feature matching algorithm.
- the feature matching algorithm can be SIFT, SURF, ORB, Algorithms like BRISK or FREAK.
- the current first video frame is determined according to the first initial timestamp, and the feature matching algorithm is used to find the second video frame matching the current first video frame, If the second video frame is found, the timestamp of the current first video frame and the found timestamp of the second video frame are set as the second time point pair, that is, the second initial timestamp of the second time point pair is The timestamp of the current first video frame, and the second termination timestamp of the second time point pair is the timestamp of the found second video frame.
- the difference between the timestamp of the current first video frame and the first initial timestamp is within a first difference range
- the first difference range may be determined according to the first fixed value preset by the user and the first initial timestamp
- the maximum value of the first difference range can be set to zero, that is, the current first video frame is the video frame before the first initial time stamp; the time stamp of the second video frame and the first termination time stamp
- the difference is within a second difference range
- the second difference range can also be determined according to the second fixed value preset by the user and the first initial timestamp, and further, the minimum value of the second difference range can be set is zero, that is, the second video frame found is the video frame after the first termination timestamp.
- the found video frame whose matching degree with the current first video frame reaches the matching degree threshold can be used as the second video frame, preferably , according to the shooting angle of the current first video frame, determine the first gaze point area picture, and use the feature matching algorithm to find the second video frame matching the first gaze point area picture, thereby reducing the amount of calculation in the feature matching process.
- an initial time stamp can be set.
- the set initial time stamp is represented by the third initial time stamp, and the video frame corresponding to the third initial time stamp is used as the first first video.
- the video frame corresponding to the first initial time stamp is taken as the first first video frame
- the obtained video frame corresponding to the previous timestamp or the next timestamp of the third initial timestamp is taken as the current first video frame in the order from near to far, until the second video frame or the first video frame is found.
- Feature matching is performed on all video frames corresponding to a difference range.
- the video frame corresponding to the third initial timestamp may be a video frame with a preset duration before the first initial timestamp , for example, a video frame 4 seconds before the first initial timestamp, and accordingly, the first difference range is also adjusted accordingly according to the third initial timestamp.
- the third initial time stamp can also be obtained by a feature matching algorithm, and specifically, the third initial time stamp can be determined according to the changes of video pictures of N video frames before the first initial time stamp.
- an initial time stamp can also be set.
- the set initial time stamp is represented by the fourth initial time stamp.
- the video frame corresponding to the fourth initial time stamp is used to perform feature matching, for example , if the fourth time stamp is equal to the first termination time stamp, first use the video frame corresponding to the first termination time stamp to perform feature matching, and if the second video frame is not found, obtain the The feature matching is performed on the video frames corresponding to the previous timestamp or the next timestamp of the four initial timestamps until the second video frame or all the video frames corresponding to the second difference value range are found to have feature matching.
- the video frame corresponding to the fourth initial timestamp may be a preset duration after the first termination timestamp.
- the first difference range is also adjusted according to the fourth initial timestamp.
- the fourth initial timestamp can also be obtained by a feature matching algorithm. Specifically, the fourth initial timestamp can be determined according to the changes of video images of N video frames after the first termination timestamp.
- the current third video frame is determined according to the first termination timestamp, and the feature matching algorithm is used to find the second video that matches the current first video frame.
- the timestamp of the current third video frame and the timestamp of the fourth video frame are set as the second time point pair, if the fourth video frame is not found, then jump to The step of determining the current third video frame by the second initial time stamp, so as to search for the fourth video frame according to the re-determined current third video frame.
- the difference between the timestamp of the current third video frame and the first termination timestamp is within the third difference range
- the difference between the timestamp of the fourth video frame and the first initial timestamp is within the fourth difference range .
- the specific implementation manner of searching for the fourth video frame according to the current third video frame is similar to the specific implementation manner of searching for the second video frame according to the first video frame, and details are not described herein.
- the second gazing point area picture is determined according to the shooting angle of the current third video frame, and the feature matching algorithm is used to find the second gazing point.
- the fourth video frame of regional picture matching thereby reducing the amount of calculation in the feature matching process.
- the preset feature matching algorithm when using the preset feature matching algorithm to obtain the second time point pair, it is further judged whether the vertical direction satisfies the constraints, only when the constraints are met.
- the feature matching algorithm is used to further acquire the second time point pair, so as to improve the effectiveness of acquiring the second time point pair and ensure the video editing effect.
- the gaze point area of the video frame corresponding to the first initial time stamp may be used as the reference image
- the video frame corresponding to the first termination time stamp may be used as the image to be detected
- corner detection is performed on the image to be detected and the reference image. The corner detection results determine whether the vertical constraints are satisfied.
- each video frame is obtained from the panoramic video file, and each obtained video frame is rendered into a panoramic image
- the second time point pair is obtained by using the preset feature matching algorithm based on the first time point pair
- the second time point pair is obtained by using the preset feature matching algorithm based on the panoramic image of each video frame and the first time point pair, in other words , when the feature matching algorithm is used to obtain the second time point pair, it is all obtained based on the panoramic image.
- step S103 the rendering parameters of the rendering model are calculated for the corresponding animation interpolation type according to the shooting angle of view of the current video frame or the second time point, and the panoramic video picture fitted on the rendering model is rendered according to the rendering parameters.
- the rendering parameters may include pitch (pitch angle), yaw (yaw angle), roll (roll angle), fov (field of view), distance (distance), etc. of the virtual camera.
- Each video The shooting angle of the frame can be obtained from the data of the gyroscope when shooting the panoramic video file.
- the rendering parameters are calculated differently when the current decoding timestamp is within or outside the time period formed by the second time point pair, so preferably, if the current decoding timestamp is not within the time period formed by the second time point pair , the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame. If the current decoding time stamp is within the time period formed by the second time point pair, then the video frame corresponding to the current decoding time stamp and the second time point pair is calculated according to the current decoding time stamp. and use the animation interpolation type to perform interpolation calculation, and calculate the rendering parameters of the rendering model according to the interpolation calculation results, so as to determine the calculation method of rendering parameters according to the time period of the current decoding timestamp.
- the shooting angle of view of the video frame corresponding to the second time point pair specifically refers to the shooting angle of view of the video frame corresponding to the second initial timestamp of the second time point pair, and the second end time of the second time point pair. Poke the shooting angle of the corresponding video frame.
- the animation interpolation type is one or more of linear, slow in and fast out, fast in and slow out, or first slow, then fast and then slow, so as to enrich the animation production effect, wherein the interpolation type may be pre-specified by the user One type, or multiple combinations.
- the animation interpolation type is linear first, then fast in and slow out.
- the video frame corresponding to the second initial timestamp before rendering the panoramic video image fitted to the rendering model according to the rendering parameters.
- the exposure difference of the video frame corresponding to the second termination timestamp is used to obtain the gradient fusion parameters of each video frame to be fused according to the exposure difference.
- the fusion parameter renders the panoramic video picture fitted on the rendering model, so as to further improve the visual effect of the corresponding transition video animation at the second time point.
- gradient fusion parameters usually include transparency.
- the difference Before obtaining the gradient fusion parameters of each video frame to be fused according to the exposure difference, further, determine whether the exposure difference is greater than the preset exposure difference threshold, if not, then according to the exposure difference The difference obtains the gradient fusion parameters of each video frame to be fused. If the exposure difference threshold is exceeded, a corresponding reminder will be sent to the user to remind the user whether to continue to automatically edit the video when the exposure difference is large, thereby improving the Effectiveness of automatic clipping. It should be noted here that if there are multiple sets of second time point pairs, the above-described method can be used to obtain the gradient fusion parameters of the video frames to be fused corresponding to each group of second time point pairs, and based on the rendering parameters and gradient Fusion parameters for rendering.
- step S104 it is determined whether the current video frame is the last frame, if not, jump to step S103, if yes, execute step S105.
- step S105 a clipped video is generated.
- the clipped video can be exported, so that the user can play the exported video on a video playback device.
- the first time point pair marked in the panoramic video file is obtained, the first time point pair represents the video segment that the user expects to cut out, and the second time point pair is obtained by using a preset feature matching algorithm based on the first time point pair.
- the time point pair, the second time point pair represents the actual video clip to be cropped, the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame or the animation interpolation type corresponding to the second time point pair, and the rendering parameters are fitted to the rendering according to the rendering parameter pair.
- the panoramic video picture of the model is rendered until the current video frame is the last frame, and the edited flat video is generated, so as to realize the automatic editing of the one-shot video to the end, reducing the editing complexity and improving the editing efficiency.
- Embodiment 2 is a diagrammatic representation of Embodiment 1:
- FIG. 2 shows the structure of the automatic editing device for panoramic video provided by the second embodiment of the present invention.
- the parts related to the embodiment of the present invention are shown, including:
- the mark obtaining unit 21 is used to obtain the first time point pair marked in the panoramic video file, and the first time point pair represents the video segment that the user desires to cut out;
- a cropping region determination unit 22 configured to obtain a second time point pair using a preset feature matching algorithm based on the first time point pair, the second time point pair representing the actual video segment to be cropped;
- the video generation unit 23 is used for calculating the rendering parameters of the rendering model for the corresponding animation interpolation type according to the shooting angle of view of the current video frame or the second time point, and rendering the panoramic video picture fitted on the rendering model according to the rendering parameters.
- the video frame is the last frame, resulting in a clipped flat video.
- the device further includes:
- the panoramic video generation unit is used to obtain each video frame from the panoramic video file, and render each obtained video frame into a panoramic image;
- the cropping region determination unit further includes:
- the region determination subunit is configured to use a preset feature matching algorithm to obtain the second time point pair based on the panoramic picture of each video frame and the first time point pair.
- the first time point pair includes a first initial timestamp and a first termination timestamp
- the cropping region determining unit includes:
- a first determining unit configured to determine the current first video frame according to the first initial time stamp, wherein the difference between the time stamp of the current first video frame and the first initial time stamp is within the first difference range;
- a first search unit configured to use a feature matching algorithm to search for a second video frame matching the current first video frame, wherein the difference between the timestamp of the second video frame and the first termination timestamp is within a second difference range ;
- the first acquisition unit is configured to set the timestamp of the current first video frame and the timestamp of the found second video frame as the second time point pair if the second video frame is found, and if the second video frame is not found. video frame, trigger the first determination unit to determine the current first video frame according to the first initial timestamp.
- the first search unit further includes:
- the first search subunit is configured to determine the first gaze point area picture according to the shooting angle of the current first video frame, and use a feature matching algorithm to search for the second video frame matching the first gaze point area image.
- the first time point pair includes a first initial timestamp and a first termination timestamp
- the cropping region determining unit includes:
- a second determining unit configured to determine the current third video frame according to the first termination timestamp, wherein the difference between the timestamp of the current third video frame and the first termination timestamp is within a third difference range;
- the second search unit is configured to use a feature matching algorithm to search for a fourth video frame matching the current third video frame, wherein the difference between the timestamp of the fourth video frame and the first initial timestamp is within the fourth difference range ;as well as
- the second acquiring unit is configured to set the timestamp of the current third video frame and the timestamp of the found fourth video frame as the second time point pair if the fourth video frame is found, and if the fourth video frame is not found video frame, trigger the second determination unit to determine the current third video frame according to the second initial timestamp.
- the second search unit includes:
- the second search subunit is used to determine the second gaze point area picture according to the shooting angle of the current third video frame, and use the feature matching algorithm to search for the fourth video frame that matches the second gaze point area image.
- the video generation unit further includes:
- a first parameter calculation unit configured to calculate rendering parameters of the rendering model according to the shooting angle of the current video frame if the current decoding timestamp is not within the time period formed by the second time point pair;
- the second parameter calculation unit is configured to use animation interpolation according to the current decoding timestamp, the shooting angle of the video frame corresponding to the second time point pair, if the current decoding time stamp is within the time period formed by the second time point pair.
- Type to perform interpolation calculation, and calculate the rendering parameters of the rendering model according to the interpolation calculation result.
- the animation interpolation type is one or more of linear, slow in and fast out, fast in and slow out, or first slow then fast and then slow.
- the second time point pair includes a second initial timestamp and a second termination timestamp
- the apparatus further includes:
- an exposure degree acquiring unit configured to acquire an exposure degree difference between the video frame corresponding to the second initial timestamp and the video frame corresponding to the second termination timestamp;
- a fusion parameter acquisition unit used for acquiring gradient fusion parameters of each video frame to be fused according to the exposure difference
- the video generation unit also includes:
- the rendering fusion unit is used to render the panoramic video image fitted to the rendering model according to the rendering parameters and the gradient fusion parameters.
- each unit of the automatic editing device for panoramic video may be implemented by corresponding hardware or software units, and each unit may be an independent software and hardware unit, or may be integrated into a software and hardware unit.
- each unit of the automatic editing device for panoramic video may be implemented by corresponding hardware or software units, and each unit may be an independent software and hardware unit, or may be integrated into a software and hardware unit.
- FIG. 3 shows the structure of the terminal provided by Embodiment 3 of the present invention. For convenience of description, only the part related to the embodiment of the present invention is shown.
- the terminal 3 in the embodiment of the present invention includes a processor 30 , a memory 31 , and a computer program 32 stored in the memory 31 and running on the processor 30 .
- the processor 30 executes the computer program 32
- the steps in the above-mentioned method embodiments are implemented, for example, steps S101 to S105 shown in FIG. 1 .
- the processor 30 executes the computer program 32
- the functions of the units in the above-mentioned apparatus embodiments for example, the functions of the units 21 to 23 shown in FIG. 2, are implemented.
- the first time point pair marked in the panoramic video file is obtained, the first time point pair represents the video segment that the user expects to cut out, and the second time point pair is obtained by using a preset feature matching algorithm based on the first time point pair.
- the time point pair, the second time point pair represents the actual video clip to be cropped, the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame or the animation interpolation type corresponding to the second time point pair, and the rendering parameters are fitted to the rendering according to the rendering parameter pair.
- the panoramic video picture of the model is rendered until the current video frame is the last frame, and the edited flat video is generated, so as to realize the automatic editing of the one-shot video to the end, reducing the editing complexity and improving the editing efficiency.
- Embodiment 4 is a diagrammatic representation of Embodiment 4:
- a computer-readable storage medium where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented, for example, as shown in FIG. 1 . Steps S101 to S105 shown. Or, when the computer program is executed by the processor, the functions of the units in the above-mentioned apparatus embodiments, for example, the functions of the units 21 to 23 shown in FIG. 2 , are implemented.
- the first time point pair marked in the panoramic video file is obtained, the first time point pair represents the video segment that the user expects to cut out, and the second time point pair is obtained by using a preset feature matching algorithm based on the first time point pair.
- the time point pair, the second time point pair represents the actual video clip to be cropped, the rendering parameters of the rendering model are calculated according to the shooting angle of the current video frame or the animation interpolation type corresponding to the second time point pair, and the rendering parameters are fitted to the rendering according to the rendering parameter pair.
- the panoramic video picture of the model is rendered until the current video frame is the last frame, and the edited flat video is generated, so as to realize the automatic editing of the one-shot video to the end, reducing the editing complexity and improving the editing efficiency.
- the computer-readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program codes, recording medium, for example, memory such as ROM/RAM, magnetic disk, optical disk, flash memory, and the like.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
本发明适用于视频处理技术领域,提供了一种全景视频的自动剪辑方法、装置、终端及存储介质,该方法包括:获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段,基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频,从而实现了一镜到底视频的自动剪辑,降低了剪辑复杂度,并提高了剪辑效率。
Description
本发明属于视频处理技术领域,尤其涉及一种全景视频的自动剪辑方法、装置、终端及存储介质。
一镜到底在视频后期处理中的应用极其广泛,在一段拍摄视频中,去掉中间不需要的视频片段(例如,包含障碍物的片段),裁剪出多段视频片段,然后利用软件技术将多段视频连接到一起,在视频上让人觉得多段片段是一镜到底拍摄的,而无明显跳帧痕迹。
但基于现有技术都是通过手动完成的,剪辑复杂度高、效率低。
基于此,有必要针对上述技术问题,提供一种全景视频的自动剪辑方法、装置、终端及存储介质。
一方面,本发明一实施例提供一种全景视频的自动剪辑方法,所述方法包括下述步骤:
获取全景视频文件中标记的第一时间点对,所述第一时间点对表征用户期望裁剪掉的视频片段;
基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对,所述第二时间点对表征实际待裁剪的视频片段;
根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染,重复该步骤直至当前视频帧为最后一帧,生成剪辑后的平面视频。
优选地,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤之前,还包括:
从所述全景视频文件中获取每个视频帧,将获取到的每个视频帧渲染成全景画面;
所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,包括:
基于所述每个视频帧的全景画面以及所述第一时间点对使用预设的特征匹配算法获取第二时间点对。
优选地,所述第一时间点对包括第一初始时间戳和第一终止时间戳,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,包括:
使用所述特征匹配算法查找与所述第一视频帧匹配的第二视频帧,其中,所述第二视频帧的时间戳与所述第一终止时间戳的差值在第二差值范围内;
若查找到所述第二视频帧,则将当前第一视频帧的时间戳和查找到的所述第二视频帧的时间戳设置为所述第二时间点对,若未查找到所述第二视频帧,则跳转至根据所述第一初始时间戳确定当前第一视频帧的步骤。
优选地,所述使用所述特征匹配算法查找与所述第一视频帧匹配的第二视频帧的步骤,包括:
根据当前第一视频帧的拍摄视角确定第一注视点区域画面,使用所述特征匹配算法查找与所述第一注视点区域画面匹配的第二视频帧。
优选地,所述第一时间点对包括第一初始时间戳和第一终止时间戳,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,还包括:
根据所述第一终止时间戳确定当前第三视频帧,其中,当前第三视频帧的时间戳与所述第一终止时间戳的差值在第三差值范围内;
使用所述特征匹配算法查找与当前第三视频帧匹配的第四视频帧,其中,所述第四视频帧的时间戳与所述第一初始时间戳的差值在第四差值范围内;
若查找到所述第四视频帧,则将当前第三视频帧的时间戳和查找到的所述第四视频帧的时间戳设置为所述第二时间点对,若未查找到所述第四视频帧,则跳转至根据所述第二初始时间戳确定当前第三视频帧的步骤。
优选地,所述使用所述特征匹配算法查找与当前第三视频帧匹配的第四视频帧的步骤,包括:
根据当前第三视频帧的拍摄视角确定第二注视点区域画面,使用所述特征匹配算法查找与所述第二注视点区域画面匹配的第四视频帧。
优选地,所述根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数的步骤,包括:
若当前解码时间戳不处于由所述第二时间点对构成的时间段内,则根据当前视频帧的拍摄视角计算渲染模型的渲染参数;
若当前解码时间戳处于由所述第二时间点对构成的时间段内,则根据当前解码时间戳、所述第二时间点对对应的视频帧的拍摄视角、并使用所述动画插值类型进行插值计算,根据插值计算结果计算渲染模型的渲染参数。
优选地,所述动画插值类型为线性、慢进快出、快进慢出或先慢后快再慢中的一种或多种。
优选地,所述第二时间点对包括第二初始时间戳和第二终止时间戳,所述根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染的步骤之前,还包括:
获取所述第二初始时间戳对应的视频帧与所述第二终止时间戳对应的视频帧的曝光度差;
根据所述曝光度差获取每个待融合处理的视频帧的渐变融合参数;
根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染的步骤,包括:
根据所述渲染参数和所述渐变融合参数对贴合在所述渲染模型的全景视频画面进行渲染。
另一方面,本发明一实施例提供了一种全景视频的自动剪辑装置,其特征在于,所述装置包括:
标记获取单元,用于获取全景视频文件中标记的第一时间点对,所述第一时间点对表征用户期望裁剪掉的视频片段;
裁剪区域确定单元,用于基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对,所述第二时间点对表征实际待裁剪的视频片段;以及
视频生成单元,用于根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频。
另一方面,本发明一实施例还提供了一种终端,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如上所述方法的步骤。
另一方面,本发明一实施例还提供了一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行时实现如上所述方法的步骤。
技术效果
本发明获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段,基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,从而实现了一镜到底视频的自动剪辑,降低了剪辑复杂度,并提高了剪辑效率。
图1是本发明实施例一提供的全景视频的自动剪辑方法的实现流程图;
图2是本发明实施例二提供的全景视频的自动剪辑装置的结构示意图;以及
图3是本发明实施例三提供的终端的结构示意图。
为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。
以下结合具体实施例对本发明的具体实现进行详细描述:
实施例一:
图1示出了本发明实施例一提供的全景视频的自动剪辑方法的实现流程,为了便于说明,仅示出了与本发明实施例相关的部分,详述如下:
在步骤S101中,获取全景视频文件中标记的第一时间点对。
本发明实施例适用于终端设备,该终端设备包括电脑、相机、智能手机和平板等,该终端设备可通过安装相应的全景视频处理软件或插件实现本方法。
在本发明实施例中,该第一时间点对包括第一初始时间戳和第一终止时间戳,该第一时间点对表征用户期望裁剪掉的视频片段,即第一初始时间戳和第一终止时间戳之间的视频片段即为用户期望裁剪掉的视频片段,该第一时间点对可根据用户在拍摄过程中的打点编辑操作得到。
在步骤S102中,基于第一时间点对使用预设的特征匹配算法获取第二时间点对。
在本发明实施例中,基于用户打点编辑过程中标记的第一时间点对并不十分精确,在实际播放过程中可能出现跳帧的情况,因此,需要重新确定实际需要裁减掉的视频片段,本方法中基于第一时间点对使用预设的特征匹配算法获取第二时间点对,该第二时间点对表征实际待裁剪的视频片段,该第二时间点对包括第二初始时间戳和第二终止时间戳,即第二初始时间戳和第二终止时间戳之间的视频片段即为使用特征匹配算法获取到的实际待裁剪的视频片段,特征匹配算法可以是SIFT、SURF、ORB、BRISK或FREAK等算法。
在使用预设的特征匹配算法获取第二时间点对时,优选地,根据第一初始时间戳确定当前第一视频帧,使用特征匹配算法查找与当前第一视频帧匹配的第二视频帧,若查找到第二视频帧,则将当前第一视频帧的时间戳和查找到的第二视频帧的时间戳设置为第二时间点对,即第二时间点对的第二初始时间戳为当前第一视频帧的时间戳,第二时间点对的第二终止时间戳为查找到的第二视频帧的时间戳。进一步地,若未查找到第二视频帧,则跳转至根据第一初始时间戳确定当前第一视频帧的步骤,以根据重新确定的当前第一视频帧进行第二视频帧的查找。其中,当前第一视频帧的时间戳与第一初始时间戳的差值在第一差值范围内,该第一差值范围可以根据用户预设的第一固定值及第一初始时间戳确定,进一步地,可将该第一差值范围的最大值设置为零,即当前第一视频帧为第一初始时间戳之前的视频帧;第二视频帧的时间戳与第一终止时间戳的差值在第二差值范围内,该第二差值范围同样可以根据用户预设的第二固定值及第一初始时间戳确定,进一步地,可将该第二差值范围的最小值设置为零,即查找到的第二视频帧为第一终止时间戳之后的视频帧。
在使用特征匹配算法查找与当前第一视频帧匹配的第二视频帧时,可以将查找到的、与当前第一视频帧的匹配度达到匹配度阈值的视频帧作为第二视频帧,优选地,根据当前第一视频帧的拍摄视角确定第一注视点区域画面,使用特征匹配算法查找与第一注视点区域画面匹配的第二视频帧,从而减小了特征匹配过程中的计算量。
在确定当前第一视频帧时,可以设置一初始时间戳,为便于描述,用第三初始时间戳表示该设置的初始时间戳,将第三初始时间戳对应的视频帧作为首个第一视频帧,例如,若该第三初始时间戳等于第一初始时间戳,则将第一初始时间戳对应的视频帧作为首个第一视频帧,若根据该第一视频帧未查找到第二视频帧,则依照由近到远的顺序将获取到的该第三初始时间戳的前一时间戳或后一时间戳对应的视频帧作为当前第一视频帧,直至查找到第二视频帧或第一差值范围内对应的所有视频帧都进行了特征匹配。此外,考虑到实际拍摄过程中当碰到障碍物等情况时通常会有一个短时间的逗留,因此,第三初始时间戳对应的视频帧可以是第一初始时间戳之前预设时长的视频帧,例如第一初始时间戳之前4秒的视频帧,相应地,第一差值范围也根据该第三初始时间戳进行相应地调整。当然,第三初始时间戳也可以通过特征匹配算法得到,具体地,可以根据第一初始时间戳之前N个视频帧的视频画面的变化情况确定第三初始时间戳。
在查找第二视频帧时,可以同样设置一初始时间戳,为便于描述,用第四初始时间戳表示该设置的初始时间戳,首先使用第四初始时间戳对应的视频帧进行特征匹配,例如,若该第四时间戳等于第一终止时间戳,则首先使用第一终止时间戳对应的视频帧进行特征匹配,若未查找到第二视频帧,则依照由近到远的顺序获取该第四初始时间戳的前一时间戳或后一时间戳对应的视频帧进行特征匹配,直至查找到第二视频帧或第二差值范围内对应的所有视频帧都进行了特征匹配。此外,考虑到实际拍摄过程中在刚绕过障碍物等情况时同样可能会有一个短时间的逗留,因此,第四初始时间戳对应的视频帧可以是第一终止时间戳之后预设时长的视频帧,例如第一终止时间戳之后4秒的视频帧,相应地,第一差值范围也根据该第四初始时间戳进行调整。当然,第四初始时间戳也可以通过特征匹配算法得到,具体地,可以根据第一终止时间戳之后N个视频帧的视频画面的变化情况确定第四初始时间戳。
在使用预设的特征匹配算法获取第二时间点对时,又一优选地,根据第一终止时间戳确定当前第三视频帧,使用特征匹配算法查找与当前第一视频帧匹配的第二视频帧,若查找到第四视频帧,则将当前第三视频帧的时间戳和第四视频帧的时间戳设置为第二时间点对,若未查找到第四视频帧,则跳转至根据第二初始时间戳确定当前第三视频帧的步骤,以根据重新确定的当前第三视频帧进行第四视频帧的查找。其中,当前第三视频帧的时间戳与第一终止时间戳的差值在第三差值范围内,第四视频帧的时间戳与第一初始时间戳的差值在第四差值范围内。根据当前第三视频帧查找第四视频帧的具体实现方式与根据第一视频帧查找第二视频帧的具体实现方式类似,在此不作赘述。
在使用特征匹配算法查找与当前第三视频帧匹配的第四视频帧时,优选地,根据当前第三视频帧的拍摄视角确定第二注视点区域画面,使用特征匹配算法查找与第二注视点区域画面匹配的第四视频帧,从而减小了特征匹配过程中的计算量。
考虑到拍摄过程中相机抖动幅度较大时会影响剪辑效果,因此,在使用预设的特征匹配算法获取第二时间点对时,进一步地,判断垂直方向是否满足约束条件,只有满足约束条件时才使用特征匹配算法进一步获取第二时间点对,以提高第二时间点对获取的有效性,并保证视频剪辑效果。具体地,可以将第一初始时间戳对应的视频帧的注视点区域作为参考图像,将第一终止时间戳对应的视频帧作为待检测图像,然后对待检测图像和参考图像进行角点检测,基于角点检测结果确定是否满足垂直方向的约束。
优选地,在基于第一时间点对使用预设的特征匹配算法获取第二时间点对之前,从全景视频文件中获取每个视频帧,将获取到的每个视频帧渲染成全景画面,在基于第一时间点对使用预设的特征匹配算法获取第二时间点对时,基于每个视频帧的全景画面以及第一时间点对使用预设的特征匹配算法获取第二时间点对,换言之,在使用特征匹配算法获取第二时间点对时全部基于全景画面进行获取。
在这里需要指出的是,用户实际期望裁剪掉的视频片段可能为多个,即,全景视频文件中可能标记有多组第一时间点对,对于每组第一时间点对,均可采用本步骤描述的方法查找对应的第二时间点对。
在步骤S103中,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染。
在本发明实施例中,渲染参数可以包括虚拟相机的pitch(俯仰角)、yaw(偏航角)、roll(横滚角)、fov(视场角)和distance(距离)等,每个视频帧的拍摄视角可通过拍摄全景视频文件时的陀螺仪的数据得到。
基于当前解码时间戳处于第二时间点对构成的时间段之内或之外时渲染参数的计算方式不同,从而优选地,若当前解码时间戳不处于由第二时间点对构成的时间段内,则根据当前视频帧的拍摄视角计算渲染模型的渲染参数,若当前解码时间戳处于由第二时间点对构成的时间段内,则根据当前解码时间戳、第二时间点对对应的视频帧的拍摄视角并使用动画插值类型进行插值计算,根据插值计算结果计算渲染模型的渲染参数,从而根据当前解码时间戳所处的时间段确定渲染参数的计算方式。其中,该第二时间点对对应的视频帧的拍摄视角具体是指该第二时间点对的第二初始时间戳对应的视频帧的拍摄视角,以及该第二时间点对的第二终止时间戳对应的视频帧的拍摄视角。
在这里需要指出的是,若存在多组第二时间点对,且当前解码时间戳处于由某一第二时间点对构成的时间段内,则根据构成该时间段的第二时间点对对应的视频帧的拍摄视角并使用动画插值类型进行插值计算,根据插值计算结果计算渲染模型的渲染参数。
优选地,动画插值类型为线性、慢进快出、快进慢出或先慢后快再慢中的一种或多种,以丰富动画制作效果,其中,该插值类型可以是用户预先指定的一种类型,也可以是多种组合,例如,动画插值类型为先线性再快进慢出。
考虑到待裁剪掉的视频片段拍摄前后可能存在曝光度的差异,从而优选地,在根据渲染参数对贴合在渲染模型的全景视频画面进行渲染之前,获取第二初始时间戳对应的视频帧与第二终止时间戳对应的视频帧的曝光度差,根据曝光度差获取每个待融合处理的视频帧的渐变融合参数,若当前视频帧为待融合处理的视频帧,则根据渲染参数和渐变融合参数对贴合在渲染模型的全景视频画面进行渲染,以进一步提高第二时间点对对应的过渡视频动画的视觉效果。其中,渐变融合参数通常包括透明度。在根据曝光度差获取每个待融合处理的视频帧的渐变融合参数之前,进一步地,判断曝光度差是否大于预设的曝光度差阈值,若未超过该曝光度差阈值,则根据曝光度差获取每个待融合处理的视频帧的渐变融合参数,若超过该曝光度差阈值,则向用户发出相应的提醒,以在曝光度差较大时提醒用户是否继续自动剪辑视频,从而提高了自动剪辑的有效性。在这里需要说明的是,若存在多组第二时间点对,则可采用以上描述方法获取每组第二时间点对对应的待融合处理的视频帧的渐变融合参数,并基于渲染参数和渐变融合参数进行渲染。
在步骤S104中,判断当前视频帧是否为最后一帧,若否,则跳转至步骤S103,若是,则执行步骤S105。
在步骤S105中,生成剪辑后的视频。
在本发明实施例中,在生成剪辑后的平面视频之后,可以导出该剪辑后的视频,以便于用户将导出后的该视频在视频播放设备上进行播放。
在本发明实施例中,获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段,基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频,从而实现了一镜到底视频的自动剪辑,降低了剪辑复杂度,并提高了剪辑效率。
实施例二:
图2示出了本发明实施例二提供的全景视频的自动剪辑装置的结构,为了便于说明,仅示出了与本发明实施例相关的部分,其中包括:
标记获取单元21,用于获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段;
裁剪区域确定单元22,用于基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段;以及
视频生成单元23,用于根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频。
优选地,该装置还包括:
全景视频生成单元,用于从全景视频文件中获取每个视频帧,将获取到的每个视频帧渲染成全景画面;
该裁剪区域确定单元还包括:
区域确定子单元,用于基于每个视频帧的全景画面以及第一时间点对使用预设的特征匹配算法获取第二时间点对。
优选地,第一时间点对包括第一初始时间戳和第一终止时间戳,该裁剪区域确定单元包括:
第一确定单元,用于根据第一初始时间戳确定当前第一视频帧,其中,当前第一视频帧的时间戳与第一初始时间戳的差值在第一差值范围内;
第一查找单元,用于使用特征匹配算法查找与当前第一视频帧匹配的第二视频帧,其中,第二视频帧的时间戳与第一终止时间戳的差值在第二差值范围内;以及
第一获取单元,用于若查找到第二视频帧,则将当前第一视频帧的时间戳和查找到的第二视频帧的时间戳设置为第二时间点对,若未查找到第二视频帧,则触发第一确定单元根据第一初始时间戳确定当前第一视频帧。
优选地,第一查找单元还包括:
第一查找子单元,用于根据当前第一视频帧的拍摄视角确定第一注视点区域画面,使用特征匹配算法查找与第一注视点区域画面匹配的第二视频帧。
优选地,第一时间点对包括第一初始时间戳和第一终止时间戳,该裁剪区域确定单元包括:
第二确定单元,用于根据第一终止时间戳确定当前第三视频帧,其中,当前第三视频帧的时间戳与第一终止时间戳的差值在第三差值范围内;
第二查找单元,用于使用特征匹配算法查找与当前第三视频帧匹配的第四视频帧,其中,第四视频帧的时间戳与第一初始时间戳的差值在第四差值范围内;以及
第二获取单元,用于若查找到第四视频帧,则将当前第三视频帧的时间戳和查找到的第四视频帧的时间戳设置为第二时间点对,若未查找到第四视频帧,则触发第二确定单元根据第二初始时间戳确定当前第三视频帧。
优选地,第二查找单元包括:
第二查找子单元,用于根据当前第三视频帧的拍摄视角确定第二注视点区域画面,使用特征匹配算法查找与第二注视点区域画面匹配的第四视频帧
优选地,视频生成单元还包括:
第一参数计算单元,用于若当前解码时间戳不处于由第二时间点对构成的时间段内,则根据当前视频帧的拍摄视角计算渲染模型的渲染参数;以及
第二参数计算单元,用于若当前解码时间戳处于由第二时间点对构成的时间段内,则根据当前解码时间戳、第二时间点对对应的视频帧的拍摄视角、并使用动画插值类型进行插值计算,根据插值计算结果计算渲染模型的渲染参数。
优选地,动画插值类型为线性、慢进快出、快进慢出或先慢后快再慢中的一种或多种。
优选地,第二时间点对包括第二初始时间戳和第二终止时间戳,该装置还包括:
曝光度获取单元,用于获取第二初始时间戳对应的视频帧与第二终止时间戳对应的视频帧的曝光度差;以及
融合参数获取单元,用于根据曝光度差获取每个待融合处理的视频帧的渐变融合参数;
视频生成单元还包括:
渲染融合单元,用于根据渲染参数和渐变融合参数对贴合在渲染模型的全景视频画面进行渲染。
在本发明实施例中,全景视频的自动剪辑装置的各单元可由相应的硬件或软件单元实现,各单元可以为独立的软、硬件单元,也可以集成为一个软、硬件单元,在此不用以限制本发明。全景视频的自动剪辑装置的各单元的具体实施方式可参考前述方法实施例的描述,在此不再赘述。
实施例三:
图3示出了本发明实施例三提供的终端的结构,为了便于说明,仅示出了与本发明实施例相关的部分。
本发明实施例的终端3包括处理器30、存储器31以及存储在存储器31中并可在处理器30上运行的计算机程序32。该处理器30执行计算机程序32时实现上述各方法实施例中的步骤,例如图1所示的步骤S101至S105。或者,处理器30执行计算机程序32时实现上述各装置实施例中各单元的功能,例如图2所示单元21至23的功能。
在本发明实施例中,获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段,基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频,从而实现了一镜到底视频的自动剪辑,降低了剪辑复杂度,并提高了剪辑效率。
实施例四:
在本发明实施例中,提供了一种计算机可读存储介质,该计算机可读存储介质存储有计算机程序,该计算机程序被处理器执行时实现上述方法实施例中的步骤,例如,图1所示的步骤S101至S105。或者,该计算机程序被处理器执行时实现上述各装置实施例中各单元的功能,例如图2所示单元21至23的功能。
在本发明实施例中,获取全景视频文件中标记的第一时间点对,第一时间点对表征用户期望裁剪掉的视频片段,基于第一时间点对使用预设的特征匹配算法获取第二时间点对,第二时间点对表征实际待裁剪的视频片段,根据当前视频帧的拍摄视角或第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据渲染参数对贴合在渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频,从而实现了一镜到底视频的自动剪辑,降低了剪辑复杂度,并提高了剪辑效率。
本发明实施例的计算机可读存储介质可以包括能够携带计算机程序代码的任何实体或装置、记录介质,例如,ROM/RAM、磁盘、光盘、闪存等存储器。
以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。
Claims (10)
- 一种全景视频的自动剪辑方法,其特征在于,所述方法包括下述步骤:获取全景视频文件中标记的第一时间点对,所述第一时间点对表征用户期望裁剪掉的视频片段;基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对,所述第二时间点对表征实际待裁剪的视频片段;根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染,重复该步骤直至当前视频帧为最后一帧,生成剪辑后的平面视频。
- 如权利要求1所述的方法,其特征在于,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤之前,还包括:从所述全景视频文件中获取每个视频帧,将获取到的每个视频帧渲染成全景画面;所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,包括:基于所述每个视频帧的全景画面以及所述第一时间点对使用预设的特征匹配算法获取第二时间点对。
- 如权利要求1所述的方法,其特征在于,所述第一时间点对包括第一初始时间戳和第一终止时间戳,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,包括:根据所述第一初始时间戳确定当前第一视频帧,其中,当前第一视频帧的时间戳与所述第一初始时间戳的差值在第一差值范围内;使用所述特征匹配算法查找与当前第一视频帧匹配的第二视频帧,其中,所述第二视频帧的时间戳与所述第一终止时间戳的差值在第二差值范围内;若查找到所述第二视频帧,则将当前第一视频帧的时间戳和查找到的所述第二视频帧的时间戳设置为所述第二时间点对,若未查找到所述第二视频帧,则跳转至根据所述第一初始时间戳确定当前第一视频帧的步骤;所述使用所述特征匹配算法查找与当前第一视频帧匹配的第二视频帧的步骤,包括:根据当前第一视频帧的拍摄视角确定第一注视点区域画面,使用所述特征匹配算法查找与所述第一注视点区域画面匹配的第二视频帧。
- 如权利要求1所述的方法,其特征在于,所述第一时间点对包括第一初始时间戳和第一终止时间戳,所述基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对的步骤,还包括:根据所述第一终止时间戳确定当前第三视频帧,其中,当前第三视频帧的时间戳与所述第一终止时间戳的差值在第三差值范围内;使用所述特征匹配算法查找与当前第三视频帧匹配的第四视频帧,其中,所述第四视频帧的时间戳与所述第一初始时间戳的差值在第四差值范围内;若查找到所述第四视频帧,则将当前第三视频帧的时间戳和查找到的所述第四视频帧的时间戳设置为所述第二时间点对,若未查找到所述第四视频帧,则跳转至根据所述第二初始时间戳确定当前第三视频帧的步骤;所述使用所述特征匹配算法查找与当前第三视频帧匹配的第四视频帧的步骤,包括:根据当前第三视频帧的拍摄视角确定第二注视点区域画面,使用所述特征匹配算法查找与所述第二注视点区域画面匹配的第四视频帧。
- 如权利要求1所述的方法,其特征在于,所述根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数的步骤,包括:若当前解码时间戳不处于由所述第二时间点对构成的时间段内,则根据当前视频帧的拍摄视角计算渲染模型的渲染参数;若当前解码时间戳处于由所述第二时间点对构成的时间段内,则根据当前解码时间戳、所述第二时间点对对应的视频帧的拍摄视角、并使用所述动画插值类型进行插值计算,根据插值计算结果计算渲染模型的渲染参数。
- 如权利要求1所述的方法,其特征在于,所述动画插值类型为线性、慢进快出、快进慢出或先慢后快再慢中的一种或多种。
- 如权利要求1所述的方法,其特征在于,所述第二时间点对包括第二初始时间戳和第二终止时间戳,所述根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染的步骤之前,还包括:获取所述第二初始时间戳对应的视频帧与所述第二终止时间戳对应的视频帧的曝光度差;根据所述曝光度差获取每个待融合处理的视频帧的渐变融合参数;根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染的步骤,包括:根据所述渲染参数和所述渐变融合参数对贴合在所述渲染模型的全景视频画面进行渲染。
- 一种全景视频的自动剪辑装置,其特征在于,所述装置包括:标记获取单元,用于获取全景视频文件中标记的第一时间点对,所述第一时间点对表征用户期望裁剪掉的视频片段;裁剪区域确定单元,用于基于所述第一时间点对使用预设的特征匹配算法获取第二时间点对,所述第二时间点对表征实际待裁剪的视频片段;以及视频生成单元,用于根据当前视频帧的拍摄视角或所述第二时间点对对应的动画插值类型计算渲染模型的渲染参数,根据所述渲染参数对贴合在所述渲染模型的全景视频画面进行渲染,直至当前视频帧为最后一帧,生成剪辑后的平面视频。
- 一种终端,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述方法的步骤。
- 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现如权利要求1至7任一项所述方法的步骤。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110351843.6 | 2021-03-31 | ||
CN202110351843.6A CN113115106B (zh) | 2021-03-31 | 2021-03-31 | 全景视频的自动剪辑方法、装置、终端及存储介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022206312A1 true WO2022206312A1 (zh) | 2022-10-06 |
Family
ID=76713516
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/079779 WO2022206312A1 (zh) | 2021-03-31 | 2022-03-08 | 全景视频的自动剪辑方法、装置、终端及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113115106B (zh) |
WO (1) | WO2022206312A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113115106B (zh) * | 2021-03-31 | 2023-05-05 | 影石创新科技股份有限公司 | 全景视频的自动剪辑方法、装置、终端及存储介质 |
CN115002335B (zh) * | 2021-11-26 | 2024-04-09 | 荣耀终端有限公司 | 视频处理方法、装置、电子设备和计算机可读存储介质 |
CN114866837B (zh) * | 2022-05-26 | 2023-10-13 | 影石创新科技股份有限公司 | 视频处理方法、装置、计算机设备和存储介质 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108322831A (zh) * | 2018-02-28 | 2018-07-24 | 广东美晨通讯有限公司 | 视频播放控制方法、移动终端及计算机可读存储介质 |
CN109688463A (zh) * | 2018-12-27 | 2019-04-26 | 北京字节跳动网络技术有限公司 | 一种剪辑视频生成方法、装置、终端设备及存储介质 |
CN110087123A (zh) * | 2019-05-15 | 2019-08-02 | 腾讯科技(深圳)有限公司 | 视频文件制作方法、装置、设备及可读存储介质 |
CN110691202A (zh) * | 2019-08-28 | 2020-01-14 | 咪咕文化科技有限公司 | 视频剪辑方法、装置及计算机存储介质 |
CN110855904A (zh) * | 2019-11-26 | 2020-02-28 | Oppo广东移动通信有限公司 | 视频处理方法、电子装置和存储介质 |
CN113115106A (zh) * | 2021-03-31 | 2021-07-13 | 影石创新科技股份有限公司 | 全景视频的自动剪辑方法、装置、终端及存储介质 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10154228B1 (en) * | 2015-12-23 | 2018-12-11 | Amazon Technologies, Inc. | Smoothing video panning |
CN108282694B (zh) * | 2017-01-05 | 2020-08-18 | 阿里巴巴集团控股有限公司 | 全景视频渲染方法、装置及电子设备 |
CN107529091B (zh) * | 2017-09-08 | 2020-08-04 | 广州华多网络科技有限公司 | 视频剪辑方法及装置 |
CN107888988A (zh) * | 2017-11-17 | 2018-04-06 | 广东小天才科技有限公司 | 一种视频剪辑方法及电子设备 |
CN107968922A (zh) * | 2017-11-23 | 2018-04-27 | 深圳岚锋创视网络科技有限公司 | 一种全景视频录制为平面视频的方法、装置及便携式终端 |
CN108366294A (zh) * | 2018-03-06 | 2018-08-03 | 广州市千钧网络科技有限公司 | 一种视频裁剪方法及装置 |
WO2020103040A1 (en) * | 2018-11-21 | 2020-05-28 | Boe Technology Group Co., Ltd. | A method for generating and displaying panorama images based on rendering engine and a display apparatus |
CN109618093A (zh) * | 2018-12-14 | 2019-04-12 | 深圳市云宙多媒体技术有限公司 | 一种全景视频直播方法及*** |
CN110703976B (zh) * | 2019-08-28 | 2021-04-13 | 咪咕文化科技有限公司 | 剪辑方法、电子设备和计算机可读存储介质 |
CN110971929B (zh) * | 2019-10-31 | 2022-07-29 | 咪咕互动娱乐有限公司 | 云游戏视频处理方法、电子设备及存储介质 |
-
2021
- 2021-03-31 CN CN202110351843.6A patent/CN113115106B/zh active Active
-
2022
- 2022-03-08 WO PCT/CN2022/079779 patent/WO2022206312A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108322831A (zh) * | 2018-02-28 | 2018-07-24 | 广东美晨通讯有限公司 | 视频播放控制方法、移动终端及计算机可读存储介质 |
CN109688463A (zh) * | 2018-12-27 | 2019-04-26 | 北京字节跳动网络技术有限公司 | 一种剪辑视频生成方法、装置、终端设备及存储介质 |
CN110087123A (zh) * | 2019-05-15 | 2019-08-02 | 腾讯科技(深圳)有限公司 | 视频文件制作方法、装置、设备及可读存储介质 |
CN110691202A (zh) * | 2019-08-28 | 2020-01-14 | 咪咕文化科技有限公司 | 视频剪辑方法、装置及计算机存储介质 |
CN110855904A (zh) * | 2019-11-26 | 2020-02-28 | Oppo广东移动通信有限公司 | 视频处理方法、电子装置和存储介质 |
CN113115106A (zh) * | 2021-03-31 | 2021-07-13 | 影石创新科技股份有限公司 | 全景视频的自动剪辑方法、装置、终端及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN113115106B (zh) | 2023-05-05 |
CN113115106A (zh) | 2021-07-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022206312A1 (zh) | 全景视频的自动剪辑方法、装置、终端及存储介质 | |
CN106375674B (zh) | 寻找和使用与相邻静态图像相关的视频部分的方法和装置 | |
WO2020252910A1 (zh) | 图像畸变修正方法、装置、电子设备及可读存储介质 | |
WO2016155377A1 (zh) | 图片展示方法和装置 | |
US20230040548A1 (en) | Panorama video editing method,apparatus,device and storage medium | |
US20090162042A1 (en) | Guided photography based on image capturing device rendered user recommendations | |
US20100321505A1 (en) | Target tracking apparatus, image tracking apparatus, methods of controlling operation of same, and digital camera | |
KR101655078B1 (ko) | 움직이는 사진 생성 방법 및 장치 | |
JP6899002B2 (ja) | 画像処理方法、装置、コンピュータ読み取り可能記憶媒体及び電子機器 | |
JP2014506026A (ja) | 携帯型装置で捕捉される多重画像からの肖像画像合成 | |
JP5949331B2 (ja) | 画像生成装置、画像生成方法及びプログラム | |
JP2004181233A5 (zh) | ||
JP2010011289A (ja) | 撮像装置及びプログラム | |
CN113973190A (zh) | 视频虚拟背景图像处理方法、装置及计算机设备 | |
US10491804B2 (en) | Focus window determining method, apparatus, and device | |
JP7253622B2 (ja) | パノラマ映像の手ぶれ補正方法及び携帯端末 | |
JPWO2018062538A1 (ja) | 表示装置およびプログラム | |
CN112036311A (zh) | 基于眼部状态检测的图像处理方法、装置及存储介质 | |
WO2022121963A1 (zh) | 一种图像遮挡检测方法、装置、拍摄设备及介质 | |
WO2022028407A1 (zh) | 一种全景视频剪辑方法、装置、存储介质及设备 | |
CN110809797B (zh) | 微视频***、格式和生成方法 | |
CN107547939A (zh) | 全景视频文件剪辑的方法、***及便携式终端 | |
WO2019000715A1 (zh) | 图像处理方法及其*** | |
TWI284851B (en) | Photographing apparatus and photographing method | |
JP7444604B2 (ja) | 画像処理装置及び方法、及び撮像装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22778496 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22778496 Country of ref document: EP Kind code of ref document: A1 |