CN116095484B - Video anti-shake method and device and electronic equipment - Google Patents
Video anti-shake method and device and electronic equipment Download PDFInfo
- Publication number
- CN116095484B CN116095484B CN202111285418.8A CN202111285418A CN116095484B CN 116095484 B CN116095484 B CN 116095484B CN 202111285418 A CN202111285418 A CN 202111285418A CN 116095484 B CN116095484 B CN 116095484B
- Authority
- CN
- China
- Prior art keywords
- frame image
- path
- frame
- optimal path
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000005457 optimization Methods 0.000 claims abstract description 82
- 238000012937 correction Methods 0.000 claims description 57
- 238000012545 processing Methods 0.000 claims description 56
- 230000008569 process Effects 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 9
- 238000009825 accumulation Methods 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 4
- 239000011159 matrix material Substances 0.000 description 13
- 230000009466 transformation Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000005259 measurement Methods 0.000 description 6
- 238000001914 filtration Methods 0.000 description 5
- 230000006641 stabilisation Effects 0.000 description 5
- 238000011105 stabilization Methods 0.000 description 5
- 230000008602 contraction Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 206010044565 Tremor Diseases 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The application provides a video anti-shake method, a video anti-shake device and electronic equipment, and relates to the technical field of artificial intelligence. The video anti-shake method can self-adapt to correct the result obtained by the first path optimization when the EIS is used for anti-shake, so that error accumulation generated by the first path optimization is relieved, stable images can be output, and the anti-shake effect is improved. In addition, in the method, the position of the next frame image can be predicted by using the offset of the OIS and the position of the frame images with preset quantity before the first path optimization is carried out, so that the prediction accuracy is improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a video anti-shake method, a video anti-shake device and electronic equipment.
Background
Video capture is a fundamental function of camera-equipped electronic devices (e.g., cell phones, tablets, motion cameras, etc.). Shooting video with high definition and strong picture stability is one of the core competitiveness of the product. Currently, electronic anti-shake technology (electric image stabilization, EIS) is mainly used when capturing video, but in some situations, such as: the single EIS technology has poor anti-shake effect on a picture tremble scene caused by hand tremble blurring under long exposure.
Disclosure of Invention
The application provides a video anti-shake method, a video anti-shake device and electronic equipment, which can ensure stable output of images and improve anti-shake effect.
In a first aspect, the present application provides a video anti-shake method, including: acquiring a t frame image and a first position corresponding to the t frame image, wherein the first position is represented by motion data of equipment for shooting the t frame image when shooting the t frame image, and t is a positive integer greater than or equal to 1; inputting the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image; according to the first position, the position of the (n-1) frame image before the t frame image and the first prediction position, carrying out first path optimization to obtain a first optimal path, wherein the first optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image, the t frame image and the (t+1) frame image are stable images, and n is a positive integer greater than or equal to 1; determining that a target deviation value between the position of a target frame image and a predicted position corresponding to the target frame image is larger than a preset deviation threshold value, correcting a first ideal position corresponding to a (t+1) th frame image in a first optimal path by utilizing a target error correction amount, wherein the target frame image comprises a t-th frame image and n-th frame images before the t-th frame image; according to the first position and the position of the n frame images before the t frame image, carrying out path optimization for the second time to obtain a second optimal path, wherein the second optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image and the t frame image are stable images; and processing the t frame image according to the first position and a second ideal position corresponding to the t frame image in the second optimal path to obtain a stable image. Therefore, when the EIS is used for anti-shake, the result obtained by the first path optimization can be corrected in a self-adaptive manner, so that error accumulation generated by the first path optimization is relieved, the influence of the result of the first path optimization on the second path optimization when the next frame of image is processed is reduced, stable images can be output, and the anti-shake effect is improved. Wherein the result of the first path optimization may be used to determine the boundary for cropping the next frame image when processing the next frame image and performing the second path optimization.
In one possible implementation manner, the correcting the first ideal position corresponding to the (t+1) th frame image in the first optimal path by using the target error correction amount specifically includes: and correcting the first ideal position corresponding to the (t+1) th frame image in the first optimal path by utilizing the target error correction quantity and a preset correction quantity scaling ratio.
In one possible implementation, the target error correction is a preset correction; or, the target error correction amount is a difference between an ideal position of the (t-1) th frame image in the optimal path obtained by performing the second path optimization on the (t-1) th frame image and an ideal position of the (t-1) th frame image in the optimal path obtained by performing the first path optimization on the (t-1) th frame image.
In one possible implementation manner, the method includes inputting the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first predicted position corresponding to the (t+1) th frame image, and specifically includes: acquiring the offset of an optical anti-shake module on equipment for shooting a t-th frame image; and inputting the first position, the position and the offset of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image. In this way, when two-stage EIS anti-shake is performed, the offset of an optical anti-shake module (i.e. OIS) and the historical real position of each frame image are used as inputs, and the motion state of the future 1 frame is predicted before the first path optimization, so that the prediction accuracy is improved.
In a second aspect, the present application provides a video anti-shake apparatus, the apparatus comprising: the acquisition module is used for acquiring a t frame image and a first position corresponding to the t frame image, wherein the first position is represented by motion data of equipment for shooting the t frame image when shooting the t frame image, and t is a positive integer greater than or equal to 1; the processing module inputs the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image; the processing module is further used for carrying out first path optimization according to the first position, the position of the (n-1) frame image before the t frame image and the first prediction position to obtain a first optimal path, wherein the first optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image, the t frame image and the (t+1) frame image are stable images, and n is a positive integer greater than or equal to 1; the processing module is further used for determining that a target deviation value between the position of the target frame image and a predicted position corresponding to the target frame image is larger than a preset deviation threshold value, correcting a first ideal position corresponding to a (t+1) th frame image in a first optimal path by utilizing a target error correction amount, wherein the target frame image comprises a t frame image and n frame images before the t frame image; the processing module is further used for performing path optimization for the second time according to the first position and the position of the n frame images before the t frame image to obtain a second optimal path, wherein the second optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image and the t frame image are stable images; and the processing module is also used for processing the t frame image according to the first position and a second ideal position corresponding to the t frame image in the second optimal path to obtain a stable image.
In one possible implementation, the processing module is further configured to: and correcting the first ideal position corresponding to the (t+1) th frame image in the first optimal path by utilizing the target error correction quantity and a preset correction quantity scaling ratio.
In one possible implementation, the target error correction is a preset correction; or, the target error correction amount is a difference between an ideal position of the (t-1) th frame image in the optimal path obtained by performing the second path optimization on the (t-1) th frame image and an ideal position of the (t-1) th frame image in the optimal path obtained by performing the first path optimization on the (t-1) th frame image.
In one possible implementation, the obtaining module is further configured to: acquiring the offset of an optical anti-shake module on equipment for shooting a t-th frame image; the processing module is further used for: and inputting the first position, the position and the offset of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image.
In a third aspect, the present application provides an electronic device, including:
at least one memory for storing a program;
at least one processor for executing the memory-stored program, the processor being for performing the method as provided in the first aspect when the memory-stored program is executed.
In a fourth aspect, the present application provides a computer readable storage medium storing a computer program which, when run on an electronic device, causes the electronic device to perform the method as provided in the first aspect.
In a fifth aspect, the present application provides a computer program product, characterized in that the computer program product, when run on an electronic device, causes the electronic device to perform the method as provided in the first aspect.
It will be appreciated that the advantages of the second to fifth aspects may be found in the relevant description of the first aspect, and are not described here again.
Drawings
Fig. 1 is a schematic hardware structure of an electronic device according to an embodiment of the present application;
fig. 2 is a flow chart of a video anti-shake method according to an embodiment of the present application;
fig. 3 is a schematic diagram of steps of a video anti-shake method according to an embodiment of the present application;
FIG. 4 is a schematic diagram illustrating steps of another video anti-shake method according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a video anti-shake device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The term "and/or" herein is an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. The symbol "/" herein indicates that the associated object is or is a relationship, e.g., A/B indicates A or B.
The terms "first" and "second" and the like in the description and in the claims are used for distinguishing between different objects and not for describing a particular sequential order of objects. For example, the first response message and the second response message, etc. are used to distinguish between different response messages, and are not used to describe a particular order of response messages.
In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units and the like; the plurality of elements means two or more elements and the like.
For example, the electronic anti-shake technology (electric image stabilization, EIS) mainly analyzes and collects images on an image sensor through a program, and when distortion and stretching occur in the content of a single frame image picture or a large jump exists between continuous multi-frame image pictures, namely, the shake phenomenon of video exists, the shake phenomenon can be eliminated by utilizing image transformation, moving pixel positions and the like, so that the anti-shake is realized, and the implementation principle is more like that of carrying out post-processing on the images.
Generally, the electronic anti-shake technology adopts a Two-Stage electronic anti-shake technology (Two-Stage electric image stabilization, two-Stage EIS). The two-stage electronic anti-shake technology can cut images in two steps on the premise of ensuring the anti-shake effect, so that the calculated amount and the storage amount of an image processing module in front of an EIS module on a passage are reduced. The two-stage electronic anti-shake technology mainly obtains an optimal path with stable video pictures through path optimization twice, and then adjusts the output video picture track to the optimal path, so that the video with optimal picture stability is obtained. It can be understood that, during the movement of the electronic device, the inertial measurement unit (inertial measurement unit, IMU) thereon can record the motion state (such as axial angular velocity, acceleration, etc.) of the electronic device at a certain moment, and the IMU records the real motion process of the body of the electronic device, that is, the real path of the body of the electronic device. When video is shot, the motion state of the body of the electronic equipment at the corresponding moment of each frame of picture can be expressed as a vector, namely a point on the path. The multiple continuous vectors corresponding to the multiple continuous picture frames form a motion path of the body of the electronic equipment. The optimal path refers to a continuous, smooth, virtual and ideal path obtained by processing a real path of a body of the electronic device by using algorithms such as filtering or optimization. In addition, the process of calculating the optimal path is path optimization.
In the two-stage electronic anti-shake technology, the position of a future frame of image is generally predicted by historical data of body motion of electronic equipment. And then carrying out first path optimization by using the position of the current frame image, the predicted position of the future frame image and the positions of the preset number of images before the current frame image to obtain an optimal path. And then, according to the predicted position of the future one-frame image and the position corresponding to the future one-frame image after the first path optimization, obtaining a clipping boundary when the second path optimization is carried out on the future one-frame image. And then, carrying out path optimization for the second time by using the position of the current frame image and the positions of the images of the preset number before the current frame image to obtain an optimal path. And finally, outputting an image which is the image after the anti-shake processing from the real position of the current frame image and the ideal position of the current frame contained in the optimal path obtained by optimizing the second path.
Since there is an error in predicting the position of the future image, and the position of the future image obtained by prediction is needed in the first path optimization, there is an error in the optimal path obtained by the first path optimization. The accumulation of errors in the position of a future frame of image obtained by subsequent prediction will cause the optimal path obtained by subsequent path optimization to become worse and worse, so that the EIS image stabilizing effect is poor, i.e. the anti-shake effect is poor.
In order to solve the problem that the anti-shake effect is poor when EIS is used for anti-shake, the embodiment of the application provides a video anti-shake method. The video anti-shake method can self-adapt to correct the result obtained by the first path optimization when the EIS is used for anti-shake, so that error accumulation generated by the first path optimization is relieved, stable images can be output, and the anti-shake effect is improved.
By way of example, fig. 1 shows a schematic diagram of a hardware architecture of an electronic device. As shown in fig. 1, the electronic device 100 includes: a camera module 110, an inertial measurement unit (inertial measurement unit, IMU) module 120, a processor 130, and a memory 140.
Wherein the camera module 110 may be used to capture images. For example, the camera module 110 may include an image sensor 111. The image sensor 111 may be used to generate an image at a certain moment, which may be composed of a plurality of pixels. In addition, an optical anti-shake (optical image stabilization, OIS) module 112 may also be included in the camera module 110. The optical anti-shake module 112 may dynamically adjust the relative position between the image sensor 111 and a lens (not shown) of the camera module 110 during the process of generating an image by the image sensor 111 to implement mechanical anti-shake, and may output a moving distance of the body of the electronic device 100.
The inertial measurement unit module 120 may be configured to record rotation and translation of the body of the electronic device 100 in real time, and output a pose (i.e., a representation of rotation and translation data) of the body at a certain moment, where a pose for a period of time is a motion path of the body during the period of time. For example, the inertial measurement unit module 120 may include a gyroscope and/or an accelerometer, or the like.
Processor 130 is the computing core and control core of electronic device 100. Processor 130 may include one or more processing units. For example, the processor 130 may include one or more of an application processor (application processor, AP), a modem (modem), a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural-Network Processor (NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors. Illustratively, the processor 130 may process the video image by an image processing algorithm, such as processing the brightness, color, pixels, etc. of the video image. The processor 130 may calculate various data by an anti-shake algorithm to obtain a stable image.
The memory 140 may store a program that is executable by the processor 130 to cause the processor 130 to perform the methods provided herein. Memory 140 may also store data. The processor 130 may read the data stored in the memory 140. The memory 140 and the processor 130 may be separately provided. Optionally, the memory 140 may also be integrated in the processor 130.
It should be understood that the structure illustrated in fig. 1 of this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments of the present solution, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
It is appreciated that in the embodiments of the present application, the electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR) device, a Virtual Reality (VR) device, an artificial intelligence (artificial intelligence, AI) device, a wearable device, a vehicle-mounted device, an intelligent home device, and/or a smart city device, and the specific type of the electronic device is not particularly limited in the embodiments of the present application.
Next, a video anti-shake method provided in the embodiments of the present application is described in detail based on what has been described above.
Exemplary, fig. 2 illustrates a video anti-shake method provided in an embodiment of the present application. The video anti-shake method may be performed by the electronic device 100 in fig. 1 described above, or may be performed by any apparatus, device, platform, or cluster of devices having computing and processing capabilities. For convenience of description, the electronic device 100 in fig. 1 is described as an execution body. Specifically, as shown in fig. 2, the method includes the steps of:
s201, acquiring a t frame image and a first position corresponding to the t frame image, wherein the first position is represented by motion data of the electronic device 100 when shooting the t frame image, and t is a positive integer greater than or equal to 1.
Specifically, the electronic device 100 may acquire the image of the t frame through the image sensor 111 thereon, and may acquire the motion data (such as acceleration data, angular velocity data, etc.) of the electronic device 100 when the image of the t frame is captured through the IMU module 120 thereon, so as to obtain the first position corresponding to the image of the t frame.
S202, inputting the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first predicted position of the path of the (t+1) th frame image.
Specifically, after the first position is acquired, the first position and the position of the n-th frame image before the t-th frame image may be input to a future path prediction model, to obtain a first predicted position of the path of the (t+1) -th frame image.
As a possible implementation manner, since the offset detected by the optical anti-shake module 112 on the electronic device 100 may also reflect the motion state of the electronic device 100, the offset detected by the optical anti-shake module 112 on the electronic device 100 may also be input to the future path prediction model at the same time, so as to obtain the first predicted position, thereby improving the accuracy of the path prediction.
S203, performing first path optimization according to the first position, the position of the (n-1) frame image before the t frame image and the first prediction position to obtain a first optimal path, wherein the first optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image, the t frame image and the (t+1) frame image are stable images, and n is a positive integer greater than or equal to 1.
Specifically, after the first predicted path of the (t+1) -th frame image is predicted, the first path optimization may be performed according to the first position, the position of the n-frame image before the t-th frame image, and the first predicted position, to obtain the first optimal path. For example, the first position, the position of the n-frame image before the t-th frame image, and the first predicted position may be input to the path optimization model, the first path optimization may be performed, and the first optimal path may be obtained. In addition, a filtering algorithm (such as mean filtering) may also be used to process the first position, the position of the n-frame image before the t-frame image, and the first predicted position to obtain the first optimal path.
S204, determining whether a target deviation value between the position of the target frame image and a predicted position corresponding to the target frame image is larger than a preset deviation threshold value, wherein the target frame image comprises a t frame image and n frame images before the t frame image.
Specifically, after the first path optimization is performed, it may be determined whether a target deviation value between the position of the target frame image and the predicted position corresponding to the target frame image is greater than a preset deviation threshold. Wherein the target frame image includes a t-th frame image and an n-frame image preceding the t-th frame image. For example, the deviation value between the position of each frame image and the predicted position corresponding to the position may be determined, and then the average value, the variance value, and the like of all the deviation values may be used as the target deviation value.
When the target deviation value is greater than the preset deviation threshold, it indicates that the deviation between the ideal position corresponding to the (t+1) th frame image and the real path of the (t+1) th frame image in the first optimal path obtained by the first path optimization is greater, and at this time, the ideal position corresponding to the (t+1) th frame image needs to be corrected, that is, S205 is executed. Otherwise, S206 is performed.
S205, correcting a first ideal position corresponding to the (t+1) th frame image in the first optimal path by using the target error correction quantity.
Specifically, the first ideal position corresponding to the (t+1) th frame image in the first optimal path may be corrected using a preset error correction amount.
In addition, the difference between the ideal position of the (t-1) th frame image in the optimal path obtained by performing the second path optimization on the (t-1) th frame image and the ideal position of the (t-1) th frame image in the optimal path obtained by performing the first path optimization may be used for correction. That is, the target error correction amount may be a preset value, or may be a difference between an ideal position of the (t-1) th frame image in the optimal path obtained by performing the second path optimization on the (t-1) th frame image and an ideal position of the (t-1) th frame image in the optimal path obtained by performing the first path optimization.
In one example, the shrinkage boundary corresponding to the (t+1) th frame image may also be determined according to the first ideal position and the first predicted position. Specifically, a shrinkage boundary corresponding to the (t+1) th frame image may be determined according to the first ideal position and the first predicted position. For example, the contraction boundary may be data obtained by subtracting an absolute value between the first ideal position and the first predicted position from a preset boundary threshold. In addition, the contraction boundary corresponding to each frame image can be used for clipping the frame image when the frame image is processed (for example, when the second path optimization is performed), so that the storage capacity and the calculated amount of the image are reduced, and the processing efficiency is improved. For example, the preset boundary threshold may be a boundary for clipping a frame image to be processed when performing path optimization for the first time on the frame image.
S206, performing second path optimization according to the first position and the position of the n frame images before the t frame image to obtain a second optimal path, wherein the second optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image and the t frame image are stable images.
Specifically, the second path optimization may be performed according to the first position and the position of the n frame images before the t frame image, to obtain the second optimal path. The second optimal path includes ideal positions of each frame image when the (n-1) frame image before the t frame image and the t frame image are both stable images. For example, the first position and the position of the n frame image before the t frame image may be input to the path optimization model, the first path optimization may be performed, and the second optimal path may be obtained. In addition, a filtering algorithm (such as mean filtering) may also be used to process the first position and the position of the n-frame image before the t-frame image to obtain the second optimal path.
S207, processing the t frame image according to the first position and a second ideal position corresponding to the t frame image in the second optimal path to obtain a stable image.
Specifically, after the second optimal path is obtained, the t frame image can be processed according to the first position and the second ideal position corresponding to the t frame image in the second optimal path, so as to obtain a stable image. The first position and the second ideal position can be input into a rode rotation matrix formula to obtain a coordinate transformation matrix which is a relation before and after the pixel coordinate on the t frame image changes, so that the pixels of the t frame image can be moved, and a stable image can be output, thereby achieving the aim of EIS anti-shake. It can be understood that, because the EIS adopts a software algorithm for obtaining the pixel shift transformation of the picture based on the motion state transformation from the position to the optimal path, and the formula for calculating the pixel shift transformation from the motion state transformation is the rodgers rotation matrix formula, a stable image can be obtained through the rodgers rotation matrix formula.
Therefore, in the video anti-shake method provided by the embodiment of the application, in the process of using the EIS anti-shake technology, the self-adaptive error correction technology is adopted to dynamically adjust the first path optimization result, so that the problem of error accumulation in the first path optimization is solved, and the two-stage anti-shake image stabilization effect is improved.
The foregoing describes the video anti-shake method provided by the embodiments of the present application, so as to facilitate understanding of the following examples.
Exemplary, fig. 3 shows a process diagram of a video anti-shake method. As shown in fig. 3, the method specifically comprises the following steps:
s301, initializing.
Specifically, the initializing may include: error correction Δs, correction on threshold e, correction scaling parameter α, camera internal matrix K, optimization boundary threshold σ, rodrich formula R, etc.
Wherein, the correction amount scaling parameter alpha can be understood as the force of correction; when the correction amount scaling parameter α is large, the correction force is large, and when the correction amount scaling parameter α is small, the correction force is small.
The camera internal parameter matrix K is a parameter determined by the physical characteristics of the camera itself on the electronic device, and can be obtained by a classical camera calibration algorithm (such as Zhang Zhengyou calibration method).
The optimized boundary threshold σ is the clipping ratio that needs clipping in using the EIS algorithm. Since the essence of the EIS algorithm is to move the pixels of the input image, after moving the pixels, the whole image is not a regular rectangle, but a rectangle with a specified resolution needs to be output, so that the boundary needs to be cut, and the cutting proportion is the boundary threshold parameter.
In one example, S301 need not be executed each time, and may be selected according to actual situations, which is not limited herein.
S302, acquiring a t-th frame imageAnd the corresponding position->
S303, predicting the position of the future 1 frame to obtain the position of the future 1 frame
Specifically, the t-th frame imageAnd corresponding position->The positions of n images before the t frame image are used as input and are input into a pre-trained future path prediction model to obtain the predicted position of the future 1 frame ∈>For example, taking n=29 as an example, then +.>Input to a pre-trained future path prediction model to obtain the predicted position +.1-frame of the future>
S304, optimizing the first path.
Specifically, the position corresponding to the t-th frame imagePosition of (n-1) images before t-th frame image, position of future 1 frame +.>As input, performing a first path optimization to obtain a first optimized optimal path, wherein the optimal path comprises an ideal position of a t frame image and an ideal position of a future 1 frame image. For example, taking n=29 as an example, the position as input is +.>When the first path optimization is performed, the threshold value of each frame boundary is sigma, and the output optimal path for the first optimization is +. >
In one example, when performing the first path optimization, the t-th frame image may be performed using the optimization boundary threshold sigma at the time of initializationClipping is carried out to reduce the calculated amount of subsequent processing and improve the processing efficiency.
S305, judging whether correction is started or not.
Specifically, a difference between the position of the (n+1) th frame image including the t-th frame and the corresponding predicted position may be calculated, and then it is determined whether to start correction or not from the difference. Illustratively, taking n=29 as an example, the average error is predicted If the error is greater than the on threshold e, then step S306 is performed, otherwise step S306 is skipped.
S306, correcting the ideal position of the future 1 frame in the first optimal path.
Specifically, the ideal position of the future 1 frame in the optimal path obtained for the first time can be corrected using the initialization error correction amount Δs and the correction amount scaling parameter α. Exemplary, ideal locations of future frames in the corrected first optimal path
S307, the shrinkage boundary of the future 1-frame image is obtained.
Specifically, since the EIS algorithm essentially moves pixels of an input image, after moving the pixels, the entire image is not a regular rectangle, but a rectangle of a prescribed resolution needs to be output, and therefore, a clipping of the boundary is required. In addition, when the first path optimization is performed in the two-stage anti-shake, and a part of the trimmable boundary is used, the trimmable boundary range is reduced when the second path optimization is performed, and thus the boundary needs to be contracted.
Wherein the position of the (t-1) th frame can be predicted based on the position of the (t-1) th frame predicted when the frame image is processedAnd (t-1) the first optimal path corrected in processing the frame image +.>The boundary threshold for the t-th frame is shrunk. New boundary threshold In addition, the boundary threshold of the (t+1) -th frame image may be shrunk at this time so as to facilitate the subsequent processing of the (t+1) -th frame image. Wherein the boundary threshold of the (t+1) th frame image is +.>Wherein sigma t+1 The boundary threshold sigma may be optimized at initialization.
S308, path optimization for the second time.
Specifically, the position corresponding to the t-th frame imageAnd the positions of n images before the t frame image are used as input, and the second path optimization is carried out to obtain a second optimized optimal path, wherein the optimal path comprises the ideal position of the t frame image. For example, taking n=29 as an example, the position as input is +.>In doing the second path optimization, the frame boundary thresholds are respectively (∈) t-29 ,…,∈ t-1 ,∈ t ) Optimal path for second optimization of output
In one example, in performing the second path optimization, the contraction boundary ε of the (t-1) th frame image obtained in processing the (t-1) th frame image may be employed t For the t frame image cut after the first path optimization Cutting is carried out again to reduce the calculated amount of subsequent processing and improve the processing efficiency.
S309, calculating error correction amount.
Specifically, the difference of the positions corresponding to the t frame image in the optimal path optimized by the two paths can be taken as the error correction amountThe error correction amount calculated here is an error correction amount when the next frame image is processed.
S310, outputting an anti-shake image.
Specifically, the position in the anti-shake optimal path corresponding to the current t-th frame image is obtainedThen, the optimal anti-shake path of the t frame image can be used for the R/L-th frame image by using the Rodrigues rotation matrix formula R, the camera internal reference matrix and the like>And its true position->Processing to obtain transformation matrix->Then, the obtained transformation matrix H is applied to the t-th frame image +.>The stable image can be obtained>
Finally, after outputting the anti-shake image, it may jump to S302 to process the next frame in a loop. Specifically, after the processing of the t-th frame image is completed, the processing of the (t+1) -th frame image may be started. When the process goes to S302, t may be updated to (t+1).
By way of example, fig. 4 shows a process schematic of another video anti-shake method. Fig. 4 differs from fig. 3 mainly in that in fig. 4 the offset detected by the OIS module on the electronic device is added when predicting the path of a future frame of image.
As shown in fig. 4, the method specifically comprises the following steps:
s401, initializing.
Specifically, the initializing may include: initializing error correction quantity delta s, correcting an opening threshold value e, correcting quantity scaling parameter alpha, camera internal reference matrix K, optimizing boundary threshold value sigma, a Rodrigas formula R and the like.
In one example, S401 does not have to be executed every time, and may be selected according to actual situations, which is not limited herein.
S402, acquiring a t-th frame imageAnd the corresponding position->
S403, guiding future 1 frame path prediction by OIS offset to obtain the position of the future 1 frame
Specifically, the t-th frame imageAnd corresponding position->The positions of n images before the t frame image and the offset f detected by the OIS module on the electronic equipment are used as inputs and are input into a pre-trained future path prediction model to obtain the predicted position of the future 1 frame->For example, taking n=29 as an example, then +.>Input to a pre-trained future path prediction model to obtain the predicted position +.1-frame of the future>
S404, optimizing the first path.
Specifically, the position corresponding to the t-th frame imagePosition of (n-1) images before t-th frame image, position of future 1 frame +.>As input, performing a first path optimization to obtain a first optimized optimal path, wherein the optimal path comprises an ideal position of a t frame image and an ideal position of a future 1 frame image. For example, taking n=29 as an example, the position as input is +. >When the first path optimization is performed, the threshold value of each frame boundary is sigma, and the output optimal path for the first optimization is +.>
In one example, the first path optimization is performedThe t-th frame image can be performed by adopting the optimized boundary threshold sigma pair during initializationClipping is carried out to reduce the calculated amount of subsequent processing and improve the processing efficiency.
S405, judging whether correction is started.
Specifically, a difference between the position of the (n+1) th frame image including the t-th frame and the corresponding predicted position may be calculated, and then it is determined whether to start correction or not from the difference. Illustratively, taking n=29 as an example, the average error is predicted If the error is greater than the on threshold e, then S406 is performed, otherwise step S406 is skipped.
S406, correcting the first optimal path
Specifically, the ideal position of the future 1 frame in the optimal path obtained for the first time can be corrected using the initialization error correction amount Δs and the correction amount scaling parameter α. Exemplary, ideal locations of future frames in the corrected first optimal path
S407, obtaining the shrinkage boundary of the future 1-frame image.
Specifically, since the essence of the EIS algorithm is to move the pixels of the input image, after moving the pixels, the whole image is not a regular rectangle, but we need to output a rectangle of a prescribed resolution, so we need to crop the boundary. In addition, when the first path optimization is performed in the two-stage anti-shake, and a part of the trimmable boundary is used, the trimmable boundary range is reduced when the second path optimization is performed, and thus the boundary needs to be contracted.
Wherein the position of the (t-1) th frame can be predicted based on the position of the (t-1) th frame predicted when the frame image is processedAnd (t-1) the first optimal path corrected in processing the frame image +.>The boundary threshold for the t-th frame is shrunk. New boundary threshold In addition, the boundary threshold of the (t+1) -th frame image may be shrunk at this time so as to facilitate the subsequent processing of the (t+1) -th frame image. Wherein the boundary threshold of the (t+1) th frame image is +.>Wherein sigma t+1 The boundary threshold sigma may be optimized at initialization.
S408, path optimization for the second time.
Specifically, the position corresponding to the t-th frame imageAnd the positions of n images before the t frame image are used as input, and the second path optimization is carried out to obtain a second optimized optimal path, wherein the optimal path comprises the ideal position of the t frame image. For example, taking n=29 as an example, the position as input is +.>In doing the second path optimization, the frame boundary thresholds are respectively (∈) t-29 ,…,∈ t-1 ,∈ t ) Optimal path for second optimization of outputWherein, each value in the optimal path is the ideal position corresponding to each frame of image.
In one ofIn an example, in performing the second path optimization, the contraction boundary ε of the (t-1) th frame image obtained in processing the (t-1) th frame image may be employed t For the t frame image cut after the first path optimizationCutting is carried out again to reduce the calculated amount of subsequent processing and improve the processing efficiency.
S409, calculating error correction amount.
Specifically, the difference of the positions corresponding to the t frame image in the optimal path optimized by the two paths can be taken as the error correction amountThe error correction amount calculated here is an error correction amount when the next frame image is processed.
S410, outputting an anti-shake image.
Specifically, the position in the anti-shake optimal path corresponding to the current t-th frame image is obtainedThen, the optimal anti-shake path of the t frame image can be used for the R/L-th frame image by using the Rodrigues rotation matrix formula R, the camera internal reference matrix and the like>And its true position->Processing to obtain transformation matrix->Then, the obtained transformation matrix H is applied to the t-th frame image +.>The stable image can be obtained>
Finally, after outputting the anti-shake image, it may jump to S402 to process the next frame in a loop. Specifically, after the processing of the t-th frame image is completed, the processing of the (t+1) -th frame image may be started. When jumping to S402, t may be updated to (t+1).
It is understood that the method provided in any embodiment of the present application may be performed by any apparatus, device, platform, or device cluster having computing and processing capabilities, other than the electronic device 100, where a manner of performing the method by an executing body other than the electronic device 100 is also within the scope of protection of the present application. In addition, the execution sequence of each step in any embodiment of the application can be adjusted according to the actual situation on the premise of no contradiction, and the adjusted technical scheme is also within the scope of the application. In addition, each step in any embodiment of the present application may also be selectively performed, which is not limited herein. In addition, all or part of any features of any embodiment of the present application may be freely and arbitrarily combined without contradiction, and the combined technical solution is also within the protection scope of the present application.
Based on the method in the above embodiment, the embodiment of the present application further provides a video anti-shake device. Referring to fig. 5, fig. 5 is a schematic structural diagram of a video anti-shake device according to an embodiment of the present application. As shown in fig. 5, the video anti-shake apparatus 500 includes one or more processors 501 and an interface circuit 502. Optionally, the video anti-shake apparatus 500 may further include a bus 503.
Wherein:
the processor 501 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in the processor 501. The processor 501 may be a general purpose processor, a digital communicator (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The methods and steps disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The interface circuit 502 may be used for transmitting or receiving data, instructions, or information, and the processor 501 may process using the data, instructions, or other information received by the interface circuit 502 and may transmit processing completion information through the interface circuit 502.
Optionally, the video anti-shake device further includes a memory, which may include a read-only memory and a random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM). Optionally, the memory stores executable software modules or data structures and the processor may perform corresponding operations by invoking operational instructions stored in the memory (which may be stored in an operating system).
Alternatively, the interface circuit 502 may be used to output the execution result of the processor 501.
It should be noted that, the functions corresponding to the processor 501 and the interface circuit 502 may be implemented by a hardware design, a software design, or a combination of hardware and software, which is not limited herein.
It will be appreciated that the steps of the method embodiments described above may be performed by logic circuitry in the form of hardware in a processor or instructions in the form of software. The video anti-shake device can be applied to the electronic device 100 in fig. 2 to implement the method provided in the embodiment of the present application.
It is to be appreciated that the processor in embodiments of the present application may be a central processing unit (central processing unit, CPU), but may also be other general purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate arrays (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor.
The method steps in the embodiments of the present application may be implemented by hardware, or may be implemented by a processor executing software instructions. The software instructions may be comprised of corresponding software modules that may be stored in random access memory (random access memory, RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable PROM (EPROM), electrically erasable programmable EPROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
It will be appreciated that the various numerical numbers referred to in the embodiments of the present application are merely for ease of description and are not intended to limit the scope of the embodiments of the present application.
Claims (10)
1. A method for video anti-shake, the method comprising:
acquiring a t frame image and a first position corresponding to the t frame image, wherein the first position is represented by motion data of equipment for shooting the t frame image when shooting the t frame image, and t is a positive integer greater than or equal to 1;
inputting the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image;
performing a first path optimization according to the first position, the position of the (n-1) frame image before the t frame image and the first prediction position to obtain a first optimal path, wherein the first optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image, the t frame image and the (t+1) frame image are stable images, and n is a positive integer greater than or equal to 1;
determining that a target deviation value between a position of a target frame image and a predicted position corresponding to the target frame image is larger than a preset deviation threshold value, and correcting a first ideal position corresponding to the (t+1) th frame image in the first optimal path by using a target error correction amount, wherein the target frame image comprises the t frame image and n frame images before the t frame image;
Performing a second path optimization according to the first position and the position of the n frame image before the t frame image to obtain a second optimal path, wherein the second optimal path comprises ideal positions of all frame images when the (n-1) frame image before the t frame image and the t frame image are stable images;
and processing the t frame image according to the first position and a second ideal position corresponding to the t frame image in the second optimal path to obtain a stable image.
2. The method according to claim 1, wherein the correcting the first ideal position corresponding to the (t+1) th frame image in the first optimal path by using the target error correction amount specifically includes:
and correcting a first ideal position corresponding to the (t+1) th frame image in the first optimal path by utilizing the target error correction amount and a preset correction amount scaling ratio.
3. The method according to claim 1 or 2, wherein the target error correction amount is a preset correction amount;
or the target error correction amount is the difference between the ideal position of the (t-1) th frame image in the optimal path obtained by carrying out the second path optimization on the (t-1) th frame image and the ideal position of the (t-1) th frame image in the optimal path obtained by carrying out the first path optimization on the (t-1) th frame image.
4. The method according to claim 1 or 2, wherein the step of inputting the first position and the position of the n-frame image before the t-frame image into a future path prediction model to obtain a first predicted position corresponding to the (t+1) -th frame image specifically includes:
acquiring the offset of an optical anti-shake module on equipment for shooting the t-th frame image;
and inputting the first position, the position of the n frame images before the t frame image and the offset into a future path prediction model to obtain a first prediction position corresponding to the (t+1t+1) th frame image.
5. A video anti-shake apparatus, the apparatus comprising:
the acquisition module is used for acquiring a t frame image and a first position corresponding to the t frame image, wherein the first position is represented by motion data of equipment for shooting the t frame image when shooting the t frame image, and t is a positive integer greater than or equal to 1;
the processing module inputs the first position and the position of the n frame images before the t frame image into a future path prediction model to obtain a first prediction position corresponding to the (t+1t+1) th frame image;
the processing module is further configured to perform a first path optimization according to the first position, the position of the (n-1) frame image before the t frame image, and the first prediction position, so as to obtain a first optimal path, where the first optimal path includes ideal positions of each frame image when the (n-1) frame image before the t frame image, and the (t+1t+1) th frame image are all stable images, and n is a positive integer greater than or equal to 1;
The processing module is further configured to determine that a target deviation value between a position of a target frame image and a predicted position corresponding to the target frame image is greater than a preset deviation threshold, and correct a first ideal position corresponding to the (t+1t+1) th frame image in the first optimal path by using a target error correction amount, where the target frame image includes the t-th frame image and an n-frame image before the t-th frame image;
the processing module is further configured to perform a second path optimization according to the first position and the position of the n frame image before the t frame image, to obtain a second optimal path, where the second optimal path includes ideal positions of each frame image when the (n-1) frame image before the t frame image and the t frame image are both stable images;
the processing module is further configured to process the t frame image according to the first position and a second ideal position corresponding to the t frame image in the second optimal path, so as to obtain a stable image.
6. The apparatus of claim 5, wherein the processing module is further configured to:
and correcting a first ideal position corresponding to the (t+1) th frame image in the first optimal path by utilizing the target error correction amount and a preset correction amount scaling ratio.
7. The apparatus according to claim 5 or 6, wherein the target error correction amount is a preset correction amount;
or the target error correction amount is the difference between the ideal position of the (t-1) th frame image in the optimal path obtained by carrying out the second path optimization on the (t-1) th frame image and the ideal position of the (t-1) th frame image in the optimal path obtained by carrying out the first path optimization on the (t-1) th frame image.
8. The apparatus of claim 5 or 6, wherein the device comprises a plurality of sensors,
the acquisition module is further configured to: acquiring the offset of an optical anti-shake module on equipment for shooting the t-th frame image;
the processing module is further configured to: and inputting the first position, the position of the n frame images before the t frame image and the offset into a future path prediction model to obtain a first prediction position corresponding to the (t+1) th frame image.
9. An electronic device, comprising:
at least one memory for storing a program;
at least one processor for executing a memory-stored program, which processor is adapted to perform the method according to any of claims 1-4 when the memory-stored program is executed.
10. A computer readable storage medium storing a computer program which, when run on an electronic device, causes the electronic device to perform the method of any one of claims 1-4.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111285418.8A CN116095484B (en) | 2021-10-29 | 2021-10-29 | Video anti-shake method and device and electronic equipment |
| PCT/CN2022/112824 WO2023071428A1 (en) | 2021-10-29 | 2022-08-16 | Video anti-shake method and apparatus, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111285418.8A CN116095484B (en) | 2021-10-29 | 2021-10-29 | Video anti-shake method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116095484A CN116095484A (en) | 2023-05-09 |
| CN116095484B true CN116095484B (en) | 2024-04-09 |
Family
ID=86159025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111285418.8A Active CN116095484B (en) | 2021-10-29 | 2021-10-29 | Video anti-shake method and device and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116095484B (en) |
| WO (1) | WO2023071428A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1744674A (en) * | 2005-10-09 | 2006-03-08 | 北京中星微电子有限公司 | Video electronic flutter-proof method |
| JP2008104223A (en) * | 2007-12-03 | 2008-05-01 | Sharp Corp | Hand shaking correction device |
| CN110062165A (en) * | 2019-04-22 | 2019-07-26 | 联想(北京)有限公司 | Method for processing video frequency, device and the electronic equipment of electronic equipment |
| JP2020120362A (en) * | 2019-01-28 | 2020-08-06 | キヤノン株式会社 | Image processing device, image processing method, and program |
| CN113436113A (en) * | 2021-07-22 | 2021-09-24 | 黑芝麻智能科技有限公司 | Anti-shake image processing method, device, electronic equipment and storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4079375B2 (en) * | 2004-10-28 | 2008-04-23 | シャープ株式会社 | Image stabilizer |
| US7755667B2 (en) * | 2005-05-17 | 2010-07-13 | Eastman Kodak Company | Image sequence stabilization method and camera having dual path image sequence stabilization |
| JP6448218B2 (en) * | 2014-05-12 | 2019-01-09 | キヤノン株式会社 | IMAGING DEVICE, ITS CONTROL METHOD, AND INFORMATION PROCESSING SYSTEM |
| JP6736362B2 (en) * | 2016-06-03 | 2020-08-05 | キヤノン株式会社 | Image processing device, image processing method, and program |
| US10545215B2 (en) * | 2017-09-13 | 2020-01-28 | Google Llc | 4D camera tracking and optical stabilization |
| US11006044B1 (en) * | 2020-03-03 | 2021-05-11 | Qualcomm Incorporated | Power-efficient dynamic electronic image stabilization |
| CN113286076B (en) * | 2021-04-09 | 2022-12-06 | 华为技术有限公司 | Shooting method and related equipment |
-
2021
- 2021-10-29 CN CN202111285418.8A patent/CN116095484B/en active Active
-
2022
- 2022-08-16 WO PCT/CN2022/112824 patent/WO2023071428A1/en unknown
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1744674A (en) * | 2005-10-09 | 2006-03-08 | 北京中星微电子有限公司 | Video electronic flutter-proof method |
| JP2008104223A (en) * | 2007-12-03 | 2008-05-01 | Sharp Corp | Hand shaking correction device |
| JP2020120362A (en) * | 2019-01-28 | 2020-08-06 | キヤノン株式会社 | Image processing device, image processing method, and program |
| CN110062165A (en) * | 2019-04-22 | 2019-07-26 | 联想(北京)有限公司 | Method for processing video frequency, device and the electronic equipment of electronic equipment |
| CN113436113A (en) * | 2021-07-22 | 2021-09-24 | 黑芝麻智能科技有限公司 | Anti-shake image processing method, device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Optimum Motion Estimation Algorithm for Fast and Robust Digital Image Stabilization;Haruhisa Okuda;《EEE Transactions on Consumer Electronics》;第52卷(第1期);全文 * |
| 视频稳像优化算法研究;瞿晖;《优秀硕士论文》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116095484A (en) | 2023-05-09 |
| WO2023071428A1 (en) | 2023-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10937169B2 (en) | Motion-assisted image segmentation and object detection | |
| CN113286194A (en) | Video processing method and device, electronic equipment and readable storage medium | |
| US10984583B2 (en) | Reconstructing views of real world 3D scenes | |
| KR101856947B1 (en) | Photographing apparatus, motion estimation apparatus, method for image compensation, method for motion estimation, computer-readable recording medium | |
| CN109618103B (en) | Anti-shake method for unmanned aerial vehicle image transmission video and unmanned aerial vehicle | |
| JP5189678B2 (en) | Image stabilization apparatus and method for moving image | |
| KR20150140812A (en) | Motion blur-free capture of low light high dynamic range images | |
| US8994838B2 (en) | Motion adaptive cropping for video stabilization | |
| CN113556464B (en) | Shooting method and device and electronic equipment | |
| CN114586337B (en) | Video anti-shake optimization processing method and device and electronic equipment | |
| CN115701125B (en) | Image anti-shake method and electronic equipment | |
| CN113132612A (en) | Image stabilization processing method, terminal shooting method, medium and system | |
| CN114390188B (en) | Image processing method and electronic equipment | |
| CN113438409B (en) | Delay calibration method, delay calibration device, computer equipment and storage medium | |
| US8964048B2 (en) | Image processing apparatus and image processing method for camera-shake correction | |
| JP6332212B2 (en) | Posture estimation apparatus, posture estimation method, and program | |
| CN115705651A (en) | Video motion estimation method, device, equipment and computer readable storage medium | |
| CN116095484B (en) | Video anti-shake method and device and electronic equipment | |
| KR101012481B1 (en) | Apparatus and Method for Correcting Moving Image Wavering | |
| JP6316030B2 (en) | Imaging apparatus and control method thereof | |
| CN116266356A (en) | Panoramic video transition rendering method and device and computer equipment | |
| CN112804444A (en) | Video processing method and device, computing equipment and storage medium | |
| CN108431867B (en) | Data processing method and terminal | |
| CN113923340B (en) | Video processing method, terminal and storage medium | |
| CN116012262B (en) | Image processing method, model training method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |