CN111246100A

CN111246100A - Anti-shake parameter calibration method and device and electronic equipment

Info

Publication number: CN111246100A
Application number: CN202010065715.0A
Authority: CN
Inventors: 戴朋飞
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2020-01-20
Filing date: 2020-01-20
Publication date: 2020-06-05
Anticipated expiration: 2040-01-20
Also published as: CN111246100B

Abstract

The application relates to an anti-shake parameter calibration method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of controlling a camera and a reference calibration object to perform relative motion, shooting the reference calibration object, and acquiring at least two frames of target images; respectively acquiring corresponding feature points from at least two frames of target images; acquiring motion information of each characteristic point; determining at least two cost values of the anti-shake parameters based on the motion information of the corresponding feature points in the at least two frames of target images; and calibrating the anti-shake parameters according to at least two cost values of the anti-shake parameters. The method and the device, the electronic equipment and the computer readable storage medium can improve the accuracy of calibrating the anti-shake parameters.

Description

Anti-shake parameter calibration method and device and electronic equipment

Technical Field

The present application relates to the field of image technologies, and in particular, to a method and an apparatus for calibrating anti-shake parameters, an electronic device, and a computer-readable storage medium.

Background

With the development of photographing devices such as smart phones and cameras, photographing technologies have emerged. When shooting, in order to shoot a clearer and better-looking picture or video, anti-shake processing is generally required to be performed on shooting equipment such as a smart phone and a camera. In order to make the photographing apparatus perform anti-shake processing more accurately, it is necessary to calibrate each anti-shake parameter of the photographing apparatus when the photographing apparatus leaves a factory.

However, the traditional anti-shake parameter calibration method has the problem of inaccurate calibration.

Disclosure of Invention

The embodiment of the application provides an anti-shake parameter calibration method and device, electronic equipment and a computer-readable storage medium, which can improve the accuracy of anti-shake parameter calibration.

An anti-shake parameter calibration method comprises the following steps:

controlling the camera and a reference calibration object to perform relative motion, shooting the reference calibration object, and acquiring at least two frames of target images;

respectively acquiring corresponding feature points from at least two frames of the target images;

acquiring motion information of each feature point;

determining at least two cost values of anti-shake parameters based on the motion information of corresponding feature points in at least two frames of the target image;

and calibrating the anti-shake parameters according to at least two cost values of the anti-shake parameters.

An anti-shake parameter calibration device, comprising:

the shooting module is used for controlling the camera and the reference calibration object to carry out relative motion, shooting the reference calibration object and acquiring at least two frames of target images;

the characteristic point acquisition module is used for respectively acquiring corresponding characteristic points from at least two frames of the target images;

the motion information acquisition module is used for acquiring the motion information of each feature point;

the cost value determining module is used for determining at least two cost values of the anti-shake parameters based on the motion information of the corresponding feature points in at least two frames of the target images;

and the calibration module is used for calibrating the anti-shake parameters according to at least two cost values of the anti-shake parameters.

An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the calibration method for anti-shake parameters.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as described above.

The anti-shake parameter calibration method, the anti-shake parameter calibration device, the electronic equipment and the computer-readable storage medium control the camera and the reference calibration object to perform relative motion, shoot the reference calibration object and acquire at least two frames of target images; respectively acquiring corresponding feature points from at least two frames of target images; acquiring motion information of each characteristic point; based on the motion information of the corresponding feature points in the at least two frames of target images, the anti-shake situation between the two frames of target images of the at least shooting reference calibration object can be known, so that at least two cost values of anti-shake parameters are determined; the anti-shake parameters can be calibrated more accurately according to at least two cost values of the anti-shake parameters.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic diagram of an image processing circuit in one embodiment;

FIG. 2 is a flowchart illustrating a method for calibrating anti-shake parameters according to an embodiment;

FIG. 3 is a flow diagram of steps in one embodiment for determining a cost value of an anti-shake parameter;

FIG. 4 is a flow diagram of steps in one embodiment for determining a target shooting strategy;

FIG. 5 is a block diagram of an apparatus for calibrating anti-shake parameters according to an embodiment;

fig. 6 is a schematic diagram of an internal structure of an electronic device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image, without departing from the scope of the present application. The first image and the second image are both images, but they are not the same image.

The embodiment of the application also provides the electronic equipment. The electronic device includes therein an Image Processing circuit, which may be implemented using hardware and/or software components, and may include various Processing units defining an ISP (Image Signal Processing) pipeline. FIG. 1 is a schematic diagram of an image processing circuit in one embodiment. As shown in fig. 1, for convenience of explanation, only aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in fig. 1, the image processing circuit includes an ISP processor 140 and control logic 150. The image data captured by the imaging device 110 is first processed by the ISP processor 140, and the ISP processor 140 analyzes the image data to capture image statistics that may be used to determine and/or control one or more parameters of the imaging device 110. The imaging device 110 may include a camera having one or more lenses 112 and an image sensor 114. The image sensor 114 may include an array of color filters (e.g., Bayer filters), and the image sensor 114 may acquire light intensity and wavelength information captured with each imaging pixel of the image sensor 114 and provide a set of raw image data that may be processed by the ISP processor 140. The sensor 120 (e.g., gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 140 based on the type of sensor 120 interface. The sensor 120 interface may utilize an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.

In addition, the image sensor 114 may also send raw image data to the sensor 120, the sensor 120 may provide the raw image data to the ISP processor 140 based on the sensor 120 interface type, or the sensor 120 may store the raw image data in the image memory 130.

The ISP processor 140 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.

The ISP processor 140 may also receive image data from the image memory 130. For example, the sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image Memory 130 may be a portion of a Memory device, a storage device, or a separate dedicated Memory within an electronic device, and may include a DMA (Direct Memory Access) feature.

Upon receiving raw image data from the image sensor 114 interface or from the sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. ISP processor 140 may also receive processed data from image memory 130 for image data processing in the raw domain and in the RGB and YCbCr color spaces. In addition, the output of the ISP processor 140 may also be sent to the image memory 130. In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

The step of the ISP processor 140 processing the image data includes: the image data is subjected to VFE (Video Front End) Processing and CPP (Camera Post Processing). The VFE processing of the image data may include modifying the contrast or brightness of the image data, modifying digitally recorded lighting status data, performing compensation processing (e.g., white balance, automatic gain control, gamma correction, etc.) on the image data, performing filter processing on the image data, etc. CPP processing of image data may include scaling an image, providing a preview frame and a record frame to each path. Among other things, the CPP may use different codecs to process the preview and record frames.

The statistical data determined by the ISP processor 140 may be transmitted to the control logic 150 unit. For example, the statistical data may include image sensor 114 statistics such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 112 shading correction, and the like. The control logic 150 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters of the imaging device 110 and control parameters of the ISP processor 140 based on the received statistical data. For example, the control parameters of the imaging device 110 may include sensor 120 control parameters (e.g., gain, integration time for exposure control), camera flash control parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), as well as lens 112 shading correction parameters.

In one embodiment, the electronic device controls the imaging device (camera) 110, i.e., the camera, to perform relative motion with respect to the reference calibration object, and controls the imaging device (camera) 110 to capture the reference calibration object via the control logic 150, to obtain at least two frames of target images, and to transmit the at least two frames of target images to the ISP processor 140 via the imaging device (camera) 110. After receiving the at least two frames of target images, the ISP processor 140 respectively obtains corresponding feature points from the at least two frames of target images; and acquiring the motion information of each characteristic point. The motion information of each feature point can be obtained by the sensor 120 and sent to the ISP processor 140. The ISP processor 140 may know the anti-shake situation between the two frames of target images of at least the photographing reference calibration object based on the motion information of the corresponding feature points in the at least two frames of target images, thereby determining at least two cost values of the anti-shake parameters; the anti-shake parameters can be calibrated more accurately according to at least two cost values of the anti-shake parameters.

Fig. 2 is a flowchart of a calibration method of anti-shake parameters in an embodiment. As shown in fig. 2, the method for calibrating the anti-shake parameter includes steps 202 to 210.

And 202, controlling the camera and the reference calibration object to perform relative motion, shooting the reference calibration object, and acquiring at least two frames of target images.

The camera is installed on the electronic equipment, can install the front side as leading camera in the electronic equipment, also can install the back side as the rear camera in the electronic equipment, also can install other places in the electronic equipment. The number of the cameras mounted on the electronic device may be one or more. The type of any one of the cameras may be one of a depth camera, a tele camera, a wide camera, a laser camera, a millimeter wave camera, and the like.

It should be noted that the camera described in this application may refer to both a lens and a camera module. The camera module comprises a lens, an image sensor, a motor and the like.

The reference calibration object refers to a reference object for calibrating the anti-shake parameters. For example, the reference calibration object may be at least one of a calibration board, a calibration drawing, a character, and the like.

Optionally, the camera can be controlled to keep still, and the reference calibration object is controlled to move; the reference calibration object can be controlled to keep still, and the camera is controlled to move; the reference calibration object and the camera can be controlled to move simultaneously, and the movement speed of the reference calibration object is different from that of the camera.

Further, the reference object and the camera may be moved from at least one of the three degrees of freedom of movement of the front-back, left-right, and up-down, may be rotated from at least one of the three degrees of freedom of rotation of the roll, pitch, and yaw, and may be moved from both the degrees of freedom of movement and the degrees of freedom of rotation.

When the camera and the reference calibration object are controlled to perform relative motion, the electronic equipment controls the camera to shoot the reference calibration object, and at least two frames of target images are obtained.

And step 204, respectively acquiring corresponding characteristic points from at least two frames of target images.

The feature point refers to a point where a certain feature exists in the target image. For example, when the reference calibration object is a calibration board, the feature point may be a corner point between a black square grid and an adjacent white square grid of the calibration board in the target image; when the reference standard object is a person, the feature point may be an eye, a nose, or the like of the person.

In each frame of the target image, the number of the acquired feature points is not limited, and may be one or more. In each target image, the feature points correspond. For example, the target image 1 acquires feature points a1 and B1, the target image 2 acquires feature points a2 and B2, the feature point a1 corresponds to the feature point a2, and the feature point B1 corresponds to the feature point B2; for another example, the feature point obtained from the target image 1 is an eye, and the feature point obtained from the target image 2 is a corresponding eye.

And step 206, acquiring the motion information of each characteristic point.

The motion information of the feature points can be coordinate positions of the feature points in the target image; or the movement direction of the feature point during shooting, wherein the movement direction can be acquired by a gyroscope in the electronic equipment; the movement speed of the feature point during shooting can be acquired by an attitude sensor in the electronic device.

And step 208, determining at least two cost values of the anti-shake parameters based on the motion information of the corresponding feature points in the at least two frames of target images.

The anti-shake parameters refer to parameters for camera anti-shake during camera shooting. The anti-shake parameters may include at least one of angular velocity data collected by the gyroscope, sampling frequency of the gyroscope, null shift value and temperature shift value of the gyroscope, delay between a collection time stamp of the gyroscope and a time stamp of taking an image, time of reading out an image by the image sensor, a rotation matrix between the gyroscope module and the image sensor, and the like. The cost value is used for representing the degree of difference between a target image actually obtained by shooting by using the anti-shake parameters and a target image which should be obtained by shooting by using the anti-shake parameters. The larger the cost value is, the more inaccurate the anti-shake parameter is represented; the smaller the cost value is, the more accurate the anti-shake parameter anti-shake is.

Two frames of target images are selected from at least two frames of target images, and one cost value of the anti-shake parameter can be determined based on the motion information of the corresponding feature points in the two selected frames of target images. And selecting other two frames of target images from the at least two frames of target images, and determining another cost value of the anti-shake parameter based on the motion information of the corresponding characteristic points in the selected other two frames of target images. The more the cost value is determined, the more accurately the anti-shake parameter can be calibrated.

And step 210, calibrating the anti-shake parameters according to at least two cost values of the anti-shake parameters.

Calibration mainly refers to using a standard measuring instrument to detect whether the accuracy (precision) of the used instrument meets the standard. And calibrating the anti-shake parameters, namely calibrating the anti-shake parameters. It can be understood that before the electronic device leaves the factory, the anti-shake parameters of the electronic device need to be calibrated, so that when a user uses the electronic device to shoot, a clearer and more stable image or video can be shot.

The anti-shake parameter calibration method controls the camera and the reference calibration object to perform relative motion, shoots the reference calibration object and obtains at least two frames of target images; respectively acquiring corresponding feature points from at least two frames of target images; acquiring motion information of each characteristic point; based on the motion information of the corresponding feature points in the at least two frames of target images, the anti-shake situation between the two frames of target images of the at least shooting reference calibration object can be known, so that at least two cost values of anti-shake parameters are determined; the anti-shake parameters can be calibrated more accurately according to at least two cost values of the anti-shake parameters.

In one embodiment, as shown in fig. 3, determining at least two cost values of the anti-shake parameters based on the motion information of the corresponding feature points in at least two frames of the target image includes:

step 302, determining a first image and a second image from at least two frames of target images; the shooting time of the second image is later than that of the first image, the characteristic point of the first image is used as a first characteristic point, and the characteristic point of the second image is used as a second characteristic point.

The first image and the second image may be two adjacent frames of target images, or may be separated from the target images by a preset number of frames.

In step 304, a reference value of the anti-shake parameter between the first image and the second image is obtained.

The reference value of the anti-shake parameter is used for carrying out anti-shake processing on the second image to obtain the anti-shake processed second image; and comparing the second image after the anti-shake processing with the first image to determine more accurate anti-shake parameters. The anti-shake process may include at least one of EIS (electronic Image Stabilization) and OIS (Optical Image Stabilization).

Specifically, the shooting time of a first image and the shooting time of a second image are obtained; acquiring a first reference value of the anti-shake parameter according to the shooting time of the first image, and acquiring a second reference value of the anti-shake parameter according to the shooting time of the second image; a reference value of an anti-shake parameter between the first image and the second image is determined based on the first reference value and the second reference value.

Optionally, the first reference value and the second reference value may be subjected to difference processing, and the obtained difference is used as a reference value of the anti-shake parameter between the first image and the second image; or obtaining a first weight of the first reference value and a second weight of the second reference value, multiplying the first reference value by the first weight to obtain a first product, multiplying the second reference value by the second weight to obtain a second product, performing difference processing on the first product and the second product, and taking the obtained difference as a reference value of the anti-shake parameter between the first image and the second image. The manner of acquiring the reference value of the anti-shake parameter between the first image and the second image is not limited.

And step 306, determining the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first characteristic point and the motion information of the second characteristic point.

The cost value is used for representing the degree of difference between a target image actually obtained by shooting by using the anti-shake parameters and a target image which should be obtained by shooting by using the anti-shake parameters. The larger the cost value is, the more inaccurate the anti-shake parameter is represented; the smaller the cost value is, the more accurate the anti-shake parameter anti-shake is.

And 308, changing the reference value of the anti-shake parameters to obtain a new reference value of the anti-shake parameters, and returning to execute the previous step until the cost value of the determined anti-shake parameters is smaller than the first generation value threshold, or the number of the obtained generation values of the anti-shake parameters reaches the first number threshold.

The reference value of the anti-shake parameters can be improved and reduced by changing the reference value of the anti-shake parameters. After the new reference value of the anti-shake parameter is obtained, the previous step is returned to, that is, the cost value of the anti-shake parameter is determined based on the reference value of the anti-shake parameter, the motion information of the first characteristic point and the motion information of the second characteristic point, so that the cost value of the new anti-shake parameter can be obtained.

In the present embodiment, a first image and a second image are determined from at least two frames of target images; the shooting time of the second image is later than that of the first image, the characteristic point of the first image is used as a first characteristic point, and the characteristic point of the second image is used as a second characteristic point; acquiring a reference value of an anti-shake parameter between a first image and a second image; determining the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first characteristic point and the motion information of the second characteristic point; and changing the reference value of the anti-shake parameters to obtain a new reference value of the anti-shake parameters, and returning to execute the previous step until the cost value of the determined anti-shake parameters is smaller than the first generation value threshold value, or the number of the obtained generation values of the anti-shake parameters reaches the first number threshold value. By changing the reference value of the anti-shake parameters, the cost values of the anti-shake parameters are obtained, and more accurate anti-shake parameters can be determined, so that the anti-shake parameters are calibrated more accurately.

In one embodiment, determining a cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point comprises: performing projection transformation on the second characteristic points based on the reference values of the anti-shake parameters and the motion information of the second characteristic points to obtain new second characteristic points and new motion information of the second characteristic points; and comparing the new motion information of the second characteristic point with the corresponding motion information of the first characteristic point to determine the cost value of the anti-shake parameter.

Projection transformation (projective transformation) is a process of transforming the coordinates of points in one image into the coordinates of points in another image. And when the second image is subjected to anti-shake processing, the second characteristic points in the second image are subjected to projection transformation by adopting anti-shake parameters to obtain new second characteristic points.

Likewise, the motion information of the new second feature point may be a coordinate position of the new second feature point in the new second image; the moving direction of the new second feature point during shooting can be obtained through a gyroscope in the electronic equipment; the movement speed of the new second feature point during shooting can be acquired by an attitude sensor in the electronic device.

It can be understood that, the second feature point in the second image corresponds to the first feature point in the first image, the second feature point is subjected to projection transformation to obtain a new second feature point, when the motion information of the new second feature point is the same as the motion information of the corresponding first feature point, the cost value of the anti-shake parameter is minimum, and the reference value of the anti-shake parameter subjected to projection transformation is most accurate; and when the difference between the new second characteristic with your motion information and the corresponding motion information of the first characteristic point is larger, the cost value of the anti-shake parameter is larger, and the reference value of the anti-shake parameter for projection transformation is more inaccurate.

For example, when the coordinates of the new second feature point are the same as the coordinates of the corresponding first feature point, the cost value of the anti-shake parameter is minimum, and the reference value of the anti-shake parameter for projection transformation is most accurate; when the difference between the coordinate of the second feature point and the coordinate of the corresponding first feature point is larger, the cost value of the anti-shake parameter is larger, and the reference value of the anti-shake parameter subjected to projection transformation is more inaccurate.

The cost value can be calculated by the following cost function:

wherein J is a cost value,

is the motion information of the feature point i in the first image t,

is the motion information of the corresponding feature point i in the second image t +1,

is to the second feature point

And obtaining new motion information of the second characteristic point after projection transformation.

In this embodiment, based on the reference value of the anti-shake parameter and the motion information of the second feature point, performing projection transformation on the second feature point to obtain a new second feature point and new motion information of the second feature point; and comparing the new motion information of the second characteristic point with the corresponding motion information of the first characteristic point, so that the cost value of the anti-shake parameter can be determined more accurately.

In one embodiment, calibrating the anti-shake parameter according to at least two cost values of the anti-shake parameter includes: determining a target cost value from at least two cost values, and taking a reference value corresponding to the target cost value as a target value of the anti-shake parameter; and calibrating the anti-shake parameters according to the target values of the anti-shake parameters.

Alternatively, the smallest cost value may be determined as the target cost value from among the at least two cost values, or the next smallest cost value may be determined as the target cost value, without being limited thereto.

After the target cost value is determined, the reference value of the anti-shake parameter corresponding to the target cost value, that is, the target value of the anti-shake parameter, is the finally determined value for calibrating the anti-shake parameter, that is, when the user is shooting, the value of the anti-shake parameter used is the target value of the anti-shake parameter.

In one embodiment, determining a target cost value from at least two cost values comprises: at least two cost values are compared, and the minimum cost value is used as the target cost value.

The reference value of the anti-shake parameter corresponding to the minimum cost value is the most accurate, and the minimum cost value is used as the target cost value, so that the anti-shake parameter can be calibrated according to the most accurate target value of the anti-shake parameter.

In other embodiments, other cost values of at least two cost values may be used as the target cost value, for example, the next smallest cost value is used as the target cost value, the cost value at the position of the median is used as the target cost value, and the like, but not limited thereto.

In other embodiments. The target cost value can also be determined from at least two cost values by methods such as a gradient descent method, a least square method, a particle swarm algorithm and the like.

In one embodiment, the method further comprises: and determining a new first image and a new second image from at least two frames of target images, and returning to the step of acquiring the reference value of the anti-shake parameter between the first image and the second image until the cost value of the determined anti-shake parameter is less than a second generation value threshold or the number of the determined first images reaches a second number threshold.

After determining the first image and the second image from the at least two frames of target images, one or more cost values corresponding to the anti-shake parameters may be determined from the first image and the second image. After the first image and the second image are processed, a new first image and a new second image can be determined from at least two frames of target images, and one or more cost values corresponding to the anti-shake parameters are determined according to the new first image and the new second image until the cost value of the determined anti-shake parameters is smaller than a second cost value threshold, or the number of the determined first images reaches a second number threshold.

In this embodiment, the cost value corresponding to the anti-shake parameter is determined according to the different first image and second image, so that the universality and stability of the determined cost value corresponding to the anti-shake parameter can be improved, more cost values of the anti-shake parameter are increased, and more accurate target cost values can be determined from more cost values, so that the anti-shake parameter can be calibrated more accurately.

In one embodiment, as shown in fig. 4, the method further includes:

and 402, when the number of the anti-shake parameters is at least two, determining a target shooting strategy.

The anti-shake parameters may include at least one of angular velocity data collected by the gyroscope, sampling frequency of the gyroscope, null shift value and temperature shift value of the gyroscope, delay between a collection time stamp of the gyroscope and a time stamp of taking an image, time of reading out an image by the image sensor, a rotation matrix between the gyroscope module and the image sensor, and the like.

The target photographing strategy refers to a strategy when a reference calibration object is photographed. The target photographing policy may include setting a photographing time length, a photographing frame rate, a photographing imaging view range, and the like. For example, the target shooting policy may be a video with a shooting duration of 5 seconds and a shooting frame rate of 10 frames/second.

And step 404, when any one of the at least two anti-shake parameters is calibrated, shooting the reference calibration object by adopting a target shooting strategy.

In a traditional anti-shake parameter calibration method, different shooting strategies are generally formulated for different anti-shake parameters, and the problem of low calibration efficiency exists. In this embodiment, when the number of anti-shake parameters is at least two, when any one anti-shake parameter of at least two anti-shake parameters is calibrated, the target shooting strategy is adopted to shoot the reference calibration object, the switching of shooting strategies when different anti-shake parameters are calibrated can be avoided, and the calibration efficiency is improved.

In one embodiment, capturing a reference calibration object to obtain at least two frames of target images includes: shooting a reference calibration object to obtain at least two frames of candidate images; at least two adjacent frame target images are determined from the at least two frame candidate images.

The candidate image refers to a photographed image. The target image refers to an image determined from the candidate images. And determining at least two adjacent frame target images from the at least two frame candidate images, wherein the shooting scenes between the two adjacent frame target images are relatively close, and the corresponding characteristic points in the target images can be more accurately determined, so that the anti-shake parameters are more accurately calibrated.

In another embodiment, candidate images separated by a preset number of frames may be further determined from the at least two frame candidate images as the at least two frame target images.

In other embodiments, the calibration method for the anti-shake parameters can also be applied to other automated tests in a factory or calibration of other parameters in the factory, so as to further improve the efficiency of a factory pipeline.

It should be understood that, although the steps in the flowcharts of fig. 2 to 4 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

Fig. 5 is a block diagram of an anti-shake parameter calibration apparatus according to an embodiment. As shown in fig. 5, an apparatus 500 for calibrating anti-shake parameters includes: a shooting module 502, a feature point obtaining module 504, a motion information obtaining module 506, a cost value determining module 508 and a calibration module 510, wherein:

and the shooting module 502 is used for controlling the camera and the reference calibration object to perform relative motion, shooting the reference calibration object and acquiring at least two frames of target images.

A feature point obtaining module 504, configured to obtain corresponding feature points from at least two frames of target images respectively.

A motion information obtaining module 506, configured to obtain motion information of each feature point.

A cost value determining module 508, configured to determine at least two cost values of the anti-shake parameter based on the motion information of the corresponding feature points in the at least two frames of target images.

And a calibration module 510, configured to calibrate the anti-shake parameter according to at least two cost values of the anti-shake parameter.

The anti-shake parameter calibration device controls the camera and the reference calibration object to perform relative motion, shoots the reference calibration object and obtains at least two frames of target images; respectively acquiring corresponding feature points from at least two frames of target images; acquiring motion information of each characteristic point; based on the motion information of the corresponding feature points in the at least two frames of target images, the anti-shake situation between the two frames of target images of the at least shooting reference calibration object can be known, so that at least two cost values of anti-shake parameters are determined; the anti-shake parameters can be calibrated more accurately according to at least two cost values of the anti-shake parameters.

In one embodiment, the cost value determining module 508 is further configured to determine a first image and a second image from at least two frames of target images; the shooting time of the second image is later than that of the first image, the characteristic point of the first image is used as a first characteristic point, and the characteristic point of the second image is used as a second characteristic point; acquiring a reference value of an anti-shake parameter between a first image and a second image; determining the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first characteristic point and the motion information of the second characteristic point; and changing the reference value of the anti-shake parameters to obtain a new reference value of the anti-shake parameters, and returning to execute the previous step until the cost value of the determined anti-shake parameters is smaller than the first generation value threshold value, or the number of the obtained generation values of the anti-shake parameters reaches the first number threshold value.

In an embodiment, the cost value determining module 508 is further configured to perform projection transformation on the second feature points based on the reference values of the anti-shake parameters and the motion information of the second feature points to obtain new second feature points and new motion information of the second feature points; and comparing the new motion information of the second characteristic point with the motion information of the first characteristic point to determine the cost value of the anti-shake parameter.

In an embodiment, the calibration module 510 is further configured to determine a target cost value from at least two cost values, and use a reference value corresponding to the target cost value as a target value of the anti-shake parameter; and calibrating the anti-shake parameters according to the target values of the anti-shake parameters.

In one embodiment, the calibration module 510 is further configured to compare at least two cost values, and use the minimum cost value as the target cost value.

In one embodiment, the cost value determining module 508 is further configured to determine a new first image and a new second image from the at least two target images, and return to the step of obtaining the reference value of the anti-shake parameter between the first image and the second image until the cost value of the determined anti-shake parameter is smaller than the second cost value threshold or the number of the determined first images reaches the second number threshold.

In an embodiment, the calibration apparatus for anti-shake parameters further includes a target shooting strategy determining module, configured to determine a target shooting strategy when the number of the anti-shake parameters is at least two; and when any one of the at least two anti-shake parameters is calibrated, a target shooting strategy is adopted to shoot the reference calibration object.

In one embodiment, the capturing module 502 is further configured to capture a reference calibration object to obtain at least two candidate images; at least two adjacent frame target images are determined from the at least two frame candidate images.

The division of each module in the calibration apparatus for anti-shake parameters is only used for illustration, and in other embodiments, the calibration apparatus for anti-shake parameters may be divided into different modules as needed to complete all or part of the functions of the calibration apparatus for anti-shake parameters.

For specific definition of the calibration apparatus for the anti-shake parameters, reference may be made to the above definition of the calibration method for the anti-shake parameters, and details are not described here. All or part of the modules in the anti-shake parameter calibration device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

Fig. 6 is a schematic diagram of an internal structure of an electronic device in one embodiment. As shown in fig. 6, the electronic device includes a processor and a memory connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement a calibration method for anti-shake parameters provided in the following embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal digital assistant), a Point of sale (POS), a vehicle-mounted computer, and a wearable device.

The implementation of each module in the calibration apparatus for anti-shake parameters provided in the embodiments of the present application may be in the form of a computer program. The computer program may be run on a terminal or a server. Program modules constituted by such computer programs may be stored on the memory of the electronic device. Which when executed by a processor, performs the steps of the method described in the embodiments of the present application.

The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of the method for calibration of anti-shake parameters.

A computer program product comprising instructions which, when run on a computer, cause the computer to perform a method of calibration of anti-shake parameters.

Any reference to memory, storage, database, or other medium used herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. An anti-shake parameter calibration method is characterized by comprising the following steps:

acquiring motion information of each feature point;

2. The method according to claim 1, wherein the determining at least two cost values of anti-shake parameters based on motion information of corresponding feature points in at least two frames of the target image comprises:

determining a first image and a second image from at least two frames of the target image; the shooting time of the second image is later than that of the first image, the characteristic point of the first image is used as a first characteristic point, and the characteristic point of the second image is used as a second characteristic point;

acquiring a reference value of an anti-shake parameter between the first image and the second image;

determining a cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first characteristic point and the motion information of the second characteristic point;

and changing the reference value of the anti-shake parameter to obtain a new reference value of the anti-shake parameter, and returning to execute the previous step until the determined cost value of the anti-shake parameter is less than a first generation value threshold value, or the obtained number of the cost values of the anti-shake parameter reaches a first number threshold value.

3. The method according to claim 2, wherein the determining the cost value of the anti-shake parameter based on the reference value of the anti-shake parameter, the motion information of the first feature point and the motion information of the second feature point comprises:

performing projection transformation on the second feature points based on the reference values of the anti-shake parameters and the motion information of the second feature points to obtain new second feature points and new motion information of the second feature points;

and comparing the new motion information of the second characteristic point with the motion information of the first characteristic point to determine the cost value of the anti-shake parameter.

4. The method according to claim 2, wherein the calibrating the anti-shake parameter according to at least two of the cost values of the anti-shake parameter comprises:

determining a target cost value from at least two cost values, and taking a reference value corresponding to the target cost value as a target value of the anti-shake parameter;

and calibrating the anti-shake parameters according to the target values of the anti-shake parameters.

5. The method of claim 4, wherein determining a target cost value from at least two of the cost values comprises:

comparing at least two of the cost values, and taking the minimum cost value as a target cost value.

6. The method of claim 2, further comprising:

determining a new first image and a new second image from at least two frames of target images, and returning to the step of acquiring the reference value of the anti-shake parameter between the first image and the second image until the cost value of the anti-shake parameter is smaller than a second cost value threshold or the number of the first images reaches a second number threshold.

7. The method of claim 1, further comprising:

when the number of the anti-shake parameters is at least two, determining a target shooting strategy;

and when any one of the at least two anti-shake parameters is calibrated, the target shooting strategy is adopted to shoot the reference calibration object.

8. The method of claim 1, wherein said capturing said reference calibration object to obtain at least two frames of target images comprises:

shooting the reference calibration object to obtain at least two frames of candidate images;

at least two adjacent frame target images are determined from the at least two frame candidate images.

9. An anti-shake parameter calibration device is characterized by comprising:

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the calibration method for anti-shake parameters according to any one of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.