CN113538252B - Image correction method and device - Google Patents
Image correction method and device Download PDFInfo
- Publication number
- CN113538252B CN113538252B CN202010307251.XA CN202010307251A CN113538252B CN 113538252 B CN113538252 B CN 113538252B CN 202010307251 A CN202010307251 A CN 202010307251A CN 113538252 B CN113538252 B CN 113538252B
- Authority
- CN
- China
- Prior art keywords
- corrected
- feature point
- feature points
- coordinate values
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000003702 image correction Methods 0.000 title claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims abstract description 75
- 238000012937 correction Methods 0.000 claims abstract description 42
- 230000015654 memory Effects 0.000 claims description 41
- 238000003062 neural network model Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000004549 pulsed laser deposition Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image correction method and device. The specific implementation scheme is as follows: acquiring configuration information of an area to be corrected in an image before correction, wherein the configuration information comprises coordinate values and offset of initial feature points; obtaining coordinate values of the predicted feature points according to the coordinate values and the offset of the initial feature points, taking the coordinate values of the predicted feature points as the coordinate values of the initial feature points, and returning to execute the prediction step; calculating coordinate values of the corrected feature points according to the coordinate values of the predicted feature points; and under the condition of stopping prediction, generating a corrected image according to the coordinate values of all the corrected feature points obtained through calculation. The image correction method avoids the operation of coordinate values of all characteristic points in the image before correction, reduces the calculated amount, reduces the power consumption of hardware, improves the image correction efficiency and accuracy, and ensures that the corrected image is closer to a real front image.
Description
Technical Field
The present application relates to the field of image processing, and in particular to the field of image correction.
Background
In the field of computer vision, detecting an object and identifying an object are widely used, and the detected object or the identified object may have angular distortion, resulting in poor accuracy of detection and identification. In order to improve the accuracy of detection and recognition, it is necessary to restore the distorted object to a front perspective view. At present, a common method is to convert data such as a distortion angle, an attitude parameter and the like into a transformation matrix, such as a rotation angle matrix, a translation matrix, a scaling matrix and the like, and multiply coordinates of a region where distortion occurs in an image by the transformation matrix to obtain a corrected image. However, the existing image correction method has large calculation amount, low efficiency and larger hardware power consumption.
Disclosure of Invention
The embodiment of the application provides an image correction method and device, so as to solve the problems in the related art, and the technical scheme is as follows:
in a first aspect, an embodiment of the present application provides an image correction method, including:
acquiring configuration information of an area to be corrected in an image before correction, wherein the configuration information comprises coordinate values and offset of initial feature points;
obtaining coordinate values of the predicted feature points according to the coordinate values and the offset of the initial feature points, taking the coordinate values of the predicted feature points as the coordinate values of the initial feature points, and returning to execute the prediction step;
calculating coordinate values of the corrected feature points according to the coordinate values of the predicted feature points;
and under the condition of stopping prediction, generating a corrected image according to the coordinate values of all the corrected feature points obtained through calculation.
In one embodiment, calculating the coordinate value of the corrected feature point from the coordinate value of the predicted feature point includes:
when the coordinate value of the predicted feature point is an integer, the coordinate value of the predicted feature point is used as the coordinate value of the corrected feature point.
In one embodiment, calculating the coordinate value of the corrected feature point from the coordinate value of the predicted feature point includes:
Under the condition that the coordinate value of the predicted feature point is not an integer, selecting the coordinate values of a plurality of adjacent feature points around the predicted feature point according to the coordinate value of the predicted feature point, wherein the coordinate values of the adjacent feature points are integers;
determining a weight ratio according to the distance between each adjacent characteristic point and the predicted characteristic point;
acquiring corresponding pixel values according to the coordinate values of the adjacent feature points, and calculating the pixel values of the corrected feature points according to the pixel values and the weight ratio of the adjacent feature points;
and converting the pixel value of the corrected feature point into a coordinate value of the corrected feature point.
In one embodiment, the method further comprises:
detecting a plurality of characteristic points in the region to be corrected by using a neural network model;
calculating a distortion angle and a posture parameter according to coordinate values of the plurality of feature points;
and obtaining the offset according to the torsion angle and the attitude parameters.
In one embodiment, the configuration information further includes a preset size parameter of the corrected image, and the condition for stopping the prediction is generated according to the preset size parameter of the corrected image.
In a second aspect, embodiments of the present application provide an image correction device, including:
the configuration information acquisition module is used for acquiring configuration information of the region to be corrected in the image before correction, wherein the configuration information comprises coordinate values of initial feature points and offset;
The coordinate value calculation module of the predicted feature point is used for obtaining the coordinate value of the predicted feature point according to the coordinate value and the offset of the initial feature point;
the calculation triggering module is used for taking the coordinate value of the predicted feature point as the coordinate value of the initial feature point and indicating the coordinate value calculation module of the predicted feature point to execute the prediction step;
the coordinate value calculation module of the corrected feature point is used for calculating the coordinate value of the corrected feature point according to the coordinate value of the predicted feature point;
and the corrected image generation module is used for generating a corrected image according to the coordinate values of all the corrected characteristic points obtained through calculation under the condition of stopping prediction.
In one embodiment, the coordinate value calculation module of the corrected feature point includes:
and the first computing sub-module is used for taking the coordinate value of the predicted feature point as the coordinate value of the corrected feature point under the condition that the coordinate value of the predicted feature point is an integer.
In one embodiment, the coordinate value calculation module of the corrected feature point includes:
the adjacent feature point selecting sub-module is used for selecting coordinate values of a plurality of adjacent feature points around the predicted feature point according to the coordinate values of the predicted feature point under the condition that the coordinate values of the predicted feature point are not completely integers, and the coordinate values of the adjacent feature points are integers;
The weight ratio calculation sub-module is used for determining weight ratio according to the distance between each adjacent characteristic point and the predicted characteristic point;
the second computing sub-module is used for obtaining corresponding pixel values according to the coordinate values of the adjacent feature points and computing the pixel values of the corrected feature points according to the pixel values and the weight ratio of the adjacent feature points;
and the coordinate value conversion sub-module is used for converting the pixel value of the corrected feature point into the coordinate value of the corrected feature point.
In one embodiment, the method further comprises:
the characteristic point detection module is used for detecting a plurality of characteristic points in the region to be corrected by using the neural network model;
the parameter calculation module is used for calculating the distortion angle and the attitude parameter according to the coordinate values of the plurality of characteristic points;
and the offset calculation module is used for obtaining the offset according to the torsion angle and the gesture parameters.
In one embodiment, the configuration information further includes a preset size parameter of the corrected image, and the condition for stopping the prediction is generated according to the preset size parameter of the corrected image.
In a third aspect, an electronic device is provided, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the above.
In a fourth aspect, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of the above.
One embodiment of the above application has the following advantages or benefits: calculating the coordinate value of the predicted feature point by using the coordinate value and the offset of the initial feature point in the region to be corrected, calculating the coordinate value of the corrected feature point based on the coordinate value of the predicted feature point, and continuously calculating the coordinate value of the next predicted feature point by using the current coordinate value and the offset of the predicted feature point, so as to obtain the coordinate value of the next corrected feature point, and obtaining the corrected image according to the coordinate values of all the corrected feature points until the condition of stopping the prediction is reached. The coordinate values of the characteristic points in the corrected image are predicted one by adopting the original data such as the offset corresponding to the image before correction, so that the coordinate values of all the characteristic points in the image before correction are prevented from being calculated, the technical problems of large calculated amount, low efficiency and large hardware power consumption are solved, the calculated amount is reduced, the hardware power consumption is reduced, the image correction efficiency and accuracy are improved, and the technical effect that the corrected image is closer to a real front image is ensured.
Other effects of the above alternative will be described below in connection with specific embodiments.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a schematic diagram of an image correction method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an image correction method according to another embodiment of the present application;
FIG. 3 is a hardware scene diagram implementing an image rectification method according to another embodiment of the present application;
FIG. 4 is a schematic diagram illustrating a calculation process of coordinate values of a predicted feature point according to another embodiment of the present application;
FIG. 5 is a schematic diagram of a process for computing pixel values of corrected feature points according to another embodiment of the present application;
FIG. 6 is a schematic diagram of an image correction device according to an embodiment of the present application;
FIG. 7 is a schematic view of an image rectification device according to another embodiment of the present application;
fig. 8 is a block diagram of an electronic device for implementing an image correction method according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As shown in fig. 1, the present embodiment provides an image correction method, including the following steps:
s110: acquiring configuration information of an area to be corrected in an image before correction, wherein the configuration information comprises coordinate values and offset of initial feature points;
s120: obtaining coordinate values of the predicted feature points according to the coordinate values and the offset of the initial feature points, taking the coordinate values of the predicted feature points as the coordinate values of the initial feature points, and returning to execute the prediction step;
s130: calculating coordinate values of the corrected feature points according to the coordinate values of the predicted feature points;
s140: and under the condition of stopping prediction, generating a corrected image according to the coordinate values of all the corrected feature points obtained through calculation.
In an example, in a scene of image detection and recognition, there is a possibility that a detected object has angular distortion, a change in posture, or the like, resulting in poor accuracy in detecting and recognizing the object. In order to correct a distorted object to obtain a frontal perspective image, the present embodiment provides an image correction method. The pre-correction image may include pedestrians, obstacles, various icons, buildings, other surrounding environments, and the like. The size of the pre-correction image may reach 256 x 256. In the pre-correction image, a specific object to be corrected and the surrounding neighboring environment can be marked by a rectangular frame to serve as an area to be corrected. For example, the first area to be corrected includes a side face of the pedestrian, and the side face is corrected to obtain a front face to identify the face of the pedestrian. The second area to be corrected comprises a warning board for front construction and no pass. The notice board is distorted, so that fonts cannot be accurately identified. The distorted warning sign is corrected, so that the warning sign displayed on the front face is obtained, and the recognition rate is improved.
And acquiring configuration information of the area to be corrected in the image before correction. The configuration information includes coordinate values of the initial feature points and offsets. In the plane coordinate system in the pre-correction image, the coordinate value of the vertex of the upper left corner of the region to be corrected may be taken as the coordinate value of the initial feature point. For example, the coordinate value of the initial feature point may be (x, y) = (0, 0). And calculating the offset according to the distortion angle and the posture parameter of the region to be corrected, wherein the offset can be expressed by a vector. And calculating the coordinate value of the first predicted feature point according to the coordinate value and the offset of the initial feature point. In order to ensure that the pixel value corresponding to the coordinate value can be acquired, the coordinate value should be an integer. Since the coordinate value of the first predicted feature point may not be an integer, in order to take the coordinate value of the integer, one or more adjacent feature points corresponding to the periphery of the first predicted feature point are obtained, the coordinate value of each adjacent feature point is an integer, and the pixel value corresponding to the coordinate value of each adjacent feature point is obtained. Meanwhile, the weight ratio may be determined according to the distance between each neighboring feature point and the first predicted feature point. And then, according to the pixel value and the weight ratio corresponding to the coordinate values of the adjacent feature points, calculating to obtain a final pixel value as the pixel value of the first corrected feature point, and converting the pixel value of the first corrected feature point into the coordinate value of the first corrected feature point.
And calculating the coordinate value of the second predicted feature point according to the coordinate value and the offset of the first predicted feature point, and executing the subsequent steps to obtain the coordinate value of the second corrected feature point. And under the condition that the stopping prediction is achieved, obtaining the coordinate value of the N (N is more than or equal to 1) th corrected characteristic point. The value of the ordinate y in the coordinate values of the corrected feature points is usually 0, which means that the N corrected feature points can be arranged according to the coordinate values of the first to nth corrected feature points to obtain a corrected image.
In this embodiment, the coordinate values of the initial feature points and the offset amount in the region to be corrected are used to calculate the coordinate values of the predicted feature points, the coordinate values of the corrected feature points are calculated based on the coordinate values of the predicted feature points, the coordinate values of the next predicted feature point are calculated by continuously using the current coordinate values of the predicted feature points and the offset amount, and the coordinate values of the next corrected feature points are further obtained, until the condition of stopping prediction is reached, the corrected image is obtained according to the coordinate values of all corrected feature points. The coordinate values of the characteristic points in the corrected image are predicted one by adopting the original data such as the offset corresponding to the image before correction, so that the coordinate values of all the characteristic points in the image before correction are prevented from being calculated, the calculated amount is reduced, the power consumption of hardware is reduced, the image correction efficiency and accuracy are improved, and the corrected image is ensured to be closer to a real front image.
In one embodiment, as shown in fig. 2, S130 includes:
s131: when the coordinate value of the predicted feature point is an integer, the coordinate value of the predicted feature point is used as the coordinate value of the corrected feature point.
In one example, the coordinate value of the predicted feature point is (a, b), and in the case where the abscissa value a and the ordinate value b are integers, the coordinate value of the predicted feature point is regarded as the coordinate value of the corrected feature point. The calculated amount is obviously reduced, the image correction efficiency is improved, and the power consumption of hardware is reduced.
In one embodiment, as shown in fig. 2, S130: comprising the following steps:
s132: under the condition that the coordinate value of the predicted feature point is not an integer, selecting the coordinate values of a plurality of adjacent feature points around the predicted feature point according to the coordinate value of the predicted feature point, wherein the coordinate values of the adjacent feature points are integers;
s133: determining a weight ratio according to the distance between each adjacent characteristic point and the predicted characteristic point;
s134: acquiring corresponding pixel values according to the coordinate values of the adjacent feature points, and calculating the pixel values of the corrected feature points according to the pixel values and the weight ratio of the adjacent feature points;
s135: and converting the pixel value of the corrected feature point into a coordinate value of the corrected feature point.
In one example, the coordinate values of the predicted feature points are not necessarily integers, and adjacent feature points whose surrounding coordinate values are integers need to be selected. For example, the coordinate values of the predicted feature points are (a, b), the adjacent feature points located at the upper left, upper right, lower left and lower right are obtained from the coordinate values (a, b) of the predicted feature points, the coordinate values of the four adjacent feature points are respectively a first coordinate value (a 1, b 1), a second coordinate value (a 2, b 2), a third coordinate value (a 3, b 3) and a fourth coordinate value (a 4, b 4), and the abscissa and the ordinate among the coordinate values of the four adjacent feature points are integers. In the case where the abscissa value a and the ordinate value b of the predicted feature point are integers, or where the abscissa value a and the ordinate value b are integers, the coordinate values of any two adjacent feature points may be selected. For example, a combination of the first coordinate value (a 1, b 1) and the second coordinate value (a 2, b 2) may be selected, and the acquired pixel values are X1 and X2. Or a combination of the second coordinate value (a 2, b 2) and the third coordinate value (a 3, b 3) may be selected, and the acquired pixel values are X2 and X3. When the abscissa value a of the predicted feature point is not an integer and the ordinate value b is not an integer, a combination of the first coordinate value (a 1, b 1), the second coordinate value (a 2, b 2), the third coordinate value (a 3, b 3) and the fourth coordinate value (a 4, b 4) is selected, and the obtained pixel values are X1, X2, X3, X4.
And determining the weight ratio between the selected adjacent feature points according to the distance between each adjacent feature point and the predicted feature point. For example, the weight ratio of the first coordinate value (a 1, b 1) and the second coordinate value (a 2, b 2) is w1: w2=2:3, or the weight ratio of the second coordinate value (a 2, b 2) and the third coordinate value (a 3, b 3) is w2: w3=4:5, or the weight ratio w1 of the first coordinate value (a 1, b 1), the second coordinate value (a 2, b 2), the third coordinate value (a 3, b 3), and the fourth coordinate value (a 4, b 4): w2: w3: w4=1:5:2:3. And obtaining the pixel value of the corrected feature point according to the pixel value and the weight ratio of each adjacent feature point. And finally, converting the pixel value of the corrected feature point into a coordinate value of the corrected feature point.
In this embodiment, the coordinate value of a predicted feature point is obtained by each calculation, and since the coordinate value of the predicted feature point may not be an integer and the corresponding pixel value cannot be directly obtained, adjacent feature points surrounding the periphery of the predicted feature point and having the integer coordinate value are selected so as to obtain the pixel values of the adjacent feature points, and the pixel values of each adjacent feature point are calculated to obtain the pixel value of the corrected feature point, and further the coordinate value of the corrected feature point is obtained. The accuracy of the coordinate values of the corrected feature points is improved, and the corrected image formed by arranging the coordinate values of the corrected feature points is more similar to the real front image.
In one embodiment, as shown in fig. 2, further comprising:
s101: detecting a plurality of characteristic points in the region to be corrected by using a neural network model;
s102: calculating a distortion angle and a posture parameter according to coordinate values of the plurality of feature points;
s103: and obtaining the offset according to the torsion angle and the attitude parameters.
In an example, in a scene of face key point detection, a neural network model may be used to detect a plurality of feature points of a side face in a face image, and an area to be corrected includes the side face. The plurality of feature points may include feature points at positions of eyebrows, eyes, nose, mouth, facial contours, and the like. The neural network model may include a Multi-tasking cascaded convolutional neural network (MTCNN, multi-task Cascaded Convolutional Networks), or the like. In the plane coordinate system, coordinate values of a plurality of detected feature points are obtained, and torsion angles and attitude parameters are calculated according to the coordinate values of the plurality of feature points. For example, the distortion angle and the attitude parameters are derived by affine variation formulas.
In one embodiment, the configuration information further includes a preset size parameter of the corrected region, and the condition for stopping the prediction is generated according to the preset size parameter of the corrected region.
In one example, when the coordinate value of the nth corrected feature point is the same as the preset size parameter, the condition of stopping prediction is reached. Assuming that the preset size parameters of the corrected image include 100 pixels in width and 50 pixels in height, there are 100 prediction feature points in each row in the horizontal direction, and 50 rows in total.
In one embodiment, a process is provided for implementing an image rectification method at a hardware level. Fig. 3 illustrates a hardware scenario diagram for implementing an image rectification method according to an embodiment.
The configuration information is sent to a prediction calculation module (coordinate_alu), and the prediction calculation module (coordinate_alu) calculates coordinate values of the predicted feature points according to the coordinate values and the offset of the initial feature points in the configuration information.
In one aspect, a prediction calculation module (prediction_alu) sends coordinate values of the prediction feature points to an address acquisition module (fetch). The address acquisition module (fetch) is used for reading coordinate values of a plurality of adjacent feature points from the memory according to the coordinate values of the predicted feature points, and acquiring corresponding register addresses (mem). The register address is a full image of the physical memory that can be used to access the physical memory. The address acquisition module (fetch) sends a plurality of register addresses (mem) to the data acquisition module (biu in). The data acquisition module (biu in) queries the coordinate values of the predicted feature points or the pixel values corresponding to the coordinate values of the adjacent feature points from the memory according to the register addresses (mem), and transmits the queried pixel values back to the data acquisition module (biu in) by utilizing the AXI (Advancede Xtensible Interface, bus protocol). AXI is an on-chip bus with high performance, high bandwidth, and low latency. The memory includes, but is not limited to, DDR (Double Data Rate), etc. The data acquisition module (biu in) sends the plurality of pixel values to the first buffer memory module (alu_buf). On the other hand, the prediction calculation module (coordinate_alu) transmits the coordinate values of the prediction feature point or the coordinate values of the plurality of neighboring feature points to the weight calculation module (cmd_fifo, command prompt, first input first output, first-in-first-out memory). The weight calculation module (cmd_fifo) is used for determining the weight ratio according to the distance between each adjacent feature point and the predicted feature point. For example, w0: w1: w2: w3, and sends the weight ratio to the first buffer memory module (alu_buf).
The first buffer memory module (alu_buf) stores pixel values and weight ratios of a plurality of adjacent feature points. The first buffer storage module (alu_buf) sends the pixel values and the weight ratios of the plurality of adjacent feature points to the correction calculation module (alu). The correction calculation module (alu) is used for obtaining the pixel value of the corrected feature point according to the pixel value and the weight ratio of each adjacent feature point. Fig. 4 shows the effective pixel values P0, P1, P2, P3 according to four neighboring feature points and the corresponding weight ratio w0: w1: w2: and w3, calculating the pixel value of the corrected feature point. P0=p0_w, p1×w1=p1_w, w2=p0_w+p1_w)/1000000 ] =p0_ 1_w; p2=p2_w, p3×w1=p3_w, w3× [ (p2_w+p3_w)/1000000 ] =p2_ 3_w; (P0_1_w+P2_3_w)/1000000 is the pixel value of the corrected feature point. The correction calculation module (alu) sends the calculated pixel values of the corrected feature points to the second buffer storage module (wb_buf) for temporary storage. The second buffer storage module (wb_buf) converts the pixel values of the corrected feature points into coordinate values of the corrected feature points, and sends the coordinate values to the data output module (biu out). The data output module (biuout) inputs the coordinate values of the corrected feature points into the memory through the bus AXI.
Fig. 5 shows a calculation process in which a prediction calculation module (coordinate_alu) obtains coordinate values of a predicted feature point from coordinate values (x, y) of an initial feature point and an offset.
First, the offset amount may be decomposed in the horizontal direction and the vertical direction to obtain a first offset amount h_delta and a second offset amount v_delta, respectively. h_delta contains two components of the abscissa axis h_delta_x and the ordinate axis h_delta_y, and v_delta contains two components of the abscissa axis v_delta_x and the ordinate axis v_delta_y.
In the horizontal direction, the abscissa value x of the initial feature point in the first row is accumulated with h_delta_x to obtain the abscissa value of the first predicted feature point. For example, if the abscissa value x=0 of the initial feature point, the abscissa value of the first predicted feature point is h_delta_x. H_delta_x is corrected before the abscissa value of the second prediction feature point is calculated, wherein the correction mode is that h_delta_x is added up with h_delta_acc_x to obtain corrected h_delta_x. Where h_delta_acc_x is a correction value of the offset in the horizontal direction, and the function is to allow a certain variation in the nonlinear value h_delta_x. And by analogy, the abscissa value of the first predicted feature point is accumulated with h_delta_x to obtain the abscissa value of the second predicted feature point, and the abscissa value of each predicted feature point is corrected again, so that the process is repeated until the first line is finished. If the ordinate value y=0 of the initial feature point, the ordinate value of the first predicted feature point is h_delta_y. The correction of h_delta_y is identical to the correction described above.
In the vertical direction, the abscissa value x of the initial feature point of the first row is accumulated with v_delta_x as the abscissa value of the initial feature point of the second row. For example, if the abscissa value x=0 of the initial feature point of the first row, the abscissa value of the initial feature point of the second row is v_delta_x. And accumulating the abscissa value of the first predicted feature point in the first row by v_change_delta to obtain the abscissa value of the first predicted feature point in the second row. v_change_delta is the first correction of the vertical offset. After the abscissa value of the first predicted feature point in the first row is accumulated with v_change_delta, the correction is continued, that is, the secondary correction value v_change_acc of the offset in the horizontal direction is accumulated again, so as to obtain the correction coordinate value of the first predicted feature point in the second row. And calculating coordinate values of a plurality of prediction feature points according to the same rule as the first line in the second line, and repeating the steps until the second line is ended. And so on until the nth row ends (N is greater than or equal to 1).
The calculation module (coordinate_alu) also selects a plurality of adjacent feature points around for the coordinate value of each predicted feature point, for example, four adjacent feature points are generally taken, which are respectively located at the upper left, upper right, lower left and lower right of the predicted feature point. The coordinate values of four adjacent feature points are obtained, and the coordinate values of the four adjacent feature points are integers, because only the integers have corresponding pixel values.
In one example, an image rectification apparatus in a hardware device receives configuration information, and parameters included in the configuration information are shown in table 1.
| Configuration information | Configuring content | Description of the invention |
| dst_width | 5 | Preset size parameter width of corrected image |
| dst_height | 8 | Preset size parameter height of corrected image |
| start_x | 3 | Coordinate values of initial feature points |
| start_y | 2 | Coordinate values of initial feature points |
| init_h_delta_x | 1.1 | Initial horizontal x offset |
| init_h_delta_y | 0.2 | Initial horizontal y offset |
| h_delta_acc_x | 0.001 | Correction of horizontal x offset |
| h_delta_acc_y | 0.002 | Correction of y offset in horizontal direction |
| v_change_delta_x | 0.01 | Correction of horizontal x offset when line change occurs |
| v_change_delta_y | 0.03 | Correction of horizontal y-offset when line change occurs |
| v_change_acc_x | 0.0001 | Secondary correction of horizontal x-offset when line change occurs |
| v_change_acc_y | 0.0003 | Secondary correction of horizontal y-offset when line change occurs |
| v_delta_x | 0.3 | Offset in vertical direction x |
| v_delta_y | 1.3 | Vertical y offset |
| v_delta_acc_x | 0.0002 | Correction of vertical x offset |
| v_delta_acc_y | 0.0004 | Correction of vertical x offset |
TABLE 1
(1) The pixel value corresponding to the coordinates (x, y) = (0, 0) of the corrected feature points is calculated, and the specific process comprises the following steps:
1.1 a prediction calculation module (correlation_alu) extracts the coordinate values (3, 2) of the initial feature points of the region to be corrected from the configuration information, and sends the coordinate values (3, 2) of the initial feature points of the region to be corrected to an address acquisition module (fetch) and a weight calculation module (cmd_fifo).
After receiving the coordinate values (3, 2) of the initial feature points, the address obtaining module (fetch) reads a plurality of adjacent feature point coordinate values from the memory, wherein the coordinate values comprise a first coordinate value (a 1, b 1) = (3, 2), a second coordinate value (a 2, b 2) = (4, 2), a third coordinate value (a 3, b 3) = (3, 3) and a fourth coordinate value (a 4, b 4) = (4, 3), and the pixel values of the four adjacent feature points are queried from the memory according to the register addresses.
After the 1.3 weight calculation module (cmd_fifo) receives the coordinate values (3, 2) of the initial feature points, four weights w0=1, w1=0, w2=1, w3=0 are calculated.
1.4 the correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=1, w1=0, w2=1, w3=0, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf). And w1 and w3 are equal to 0, and the actual equivalent is that the pixel value corresponding to the coordinate value (3, 2) of the initial characteristic point is directly selected as the pixel value of the corrected characteristic point.
1.5 writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (0, 0) point of the corrected feature point in the corrected image.
(2) The pixel value corresponding to the coordinates (x, y) = (1, 0) of the corrected feature points is calculated, and the specific process comprises the following steps:
2.1 a prediction calculation module (coordinate_alu) calculates the coordinate value (3+init_h_delta_x, 2+init_h_delta_y) of the first prediction feature point according to the coordinate value (3, 2) and the offset of the initial feature point of the region to be corrected, wherein init_h_delta_x=1.1 and init_h_delta_y=0.2, that is, the coordinate value of the first prediction feature point is (4.1,2.2), and sends the coordinate value (4.1,2.2) of the first prediction feature point to an address acquisition module (fetch) and a weight calculation module (cmd_fifo).
After receiving the coordinate value (4.1,2.2) of the first predicted feature point, the address obtaining module (fetch) reads the coordinate values of a plurality of adjacent feature points from the memory, including a first coordinate value (a 1, b 1) = (4, 2), a second coordinate value (a 2, b 2) = (5, 2), a third coordinate value (a 3, b 3) = (4, 3) and a fourth coordinate value (a 4, b 4) = (5, 3), and queries the pixel values of the four adjacent feature points from the memory according to the register address.
After the 2.3 weight calculation module (cmd_fifo) receives the coordinate value (4.1,2.2) of the first predicted feature point, four weights w0=0.9, w1=0.1, w2=0.8, w3=0.2 are calculated.
2.4 the correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=0.9, w1=0.1, w2=0.8, w3=0.2, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf).
2.5 writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (1, 0) point of the corrected feature point in the corrected image.
2.6 update init_h_delta_x, init_h_delta_y. After updating, h_delta_x=init_h_delta_x+h_delta_acc_x=1.1+0.001=1.101, and h_delta_y=init_h_delta_y+h_delta_acc_y=0.2+0.002=0.202 are obtained as the offset of the first update.
(3) The pixel value corresponding to the coordinates (x, y) = (2, 0) of the corrected feature points is calculated, and the specific process comprises the following steps:
3.1 a prediction calculation module (prediction_alu) calculates a coordinate value (4.1+h_delta_x, 2.2+h_delta_y) of a second prediction feature point according to the coordinate value (4.1,2.2) of the first prediction feature point and the offset updated for the first time, wherein h_delta_x=1.101 and h_delta_y=0.202, namely the coordinate value (5.201,2.402) of the second prediction feature point is obtained, and the coordinate value (5.201,2.402) of the second prediction feature point is sent to an address acquisition module (fetch) and a weight calculation module (cmd_fifo).
After receiving the coordinate value (5.201,2.402) of the second predicted feature point, the address obtaining module (fetch) reads the coordinate values of a plurality of adjacent feature points from the memory, including a first coordinate value (a 1, b 1) = (5, 2), a second coordinate value (a 2, b 2) = (6, 2), a third coordinate value (a 3, b 3) = (5, 3) and a fourth coordinate value (a 4, b 4) = (6, 3), and queries the pixel values of the four adjacent feature points from the memory according to the register address.
After the 3.3 weight calculation module (cmd_fifo) receives the coordinate value (5.201,2.402) of the second predicted feature point, four weights w0=0.799, w1=0.201, w2=0.598, w3=0.402 are calculated.
3.4 the correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=0.799, w1=0.201, w2=0.598, w3=0.402, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf).
And 3.5, writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (2, 0) point of the corrected feature point in the corrected image.
3.6 update h_delta_x, h_delta_y. Let h_delta_x+h_delta_acc_x=1.101+0.001=1.102 be the new h_delta_x, h_delta_y+h_delta_acc_y=0.202+0.002=0.204 be the new h_delta_y, new h_delta_x and new h_delta_y be the offsets for the second update.
4) The pixel value corresponding to the coordinates (x, y) = (3, 0) of the corrected image is calculated, and the specific process comprises the following steps:
4.1 the prediction calculation module (coordinate_alu) calculates the coordinate value (5.201+h_delta_x, 2.402+h_delta_y) of the third prediction feature point according to the coordinate value (5.201,2.402) of the second prediction feature point and the offset updated for the second time, wherein h_delta_x=1.102 and h_delta_y=0.204, so as to obtain the coordinate value (6.303,2.606) of the third prediction feature point, and sends the coordinate value (6.303,2.606) of the third prediction feature point to the address acquisition module (fetch) and the weight calculation module (cmd_fifo).
After receiving the coordinate value (6.303,2.606) of the third predicted feature point, the address obtaining module (fetch) reads the coordinate values of a plurality of adjacent feature points from the memory, including a first coordinate value (a 1, b 1) = (6, 2), a second coordinate value (a 2, b 2) = (7, 2), a third coordinate value (a 3, b 3) = (6, 3) and a fourth coordinate value (a 4, b 4) = (7, 3), and queries the pixel values of the four adjacent feature points from the memory according to the register address.
4.3 after the weight calculation module (cmd_fifo) receives the coordinates (6.303,2.606) of the third predicted feature point, four weights w0=0.697, w1=0.303, w2=0.394, w3=0.606 are calculated.
4.4 the correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=0.697, w1=0.303, w2=0.394, w3=0.606, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf).
4.5 writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (3, 0) point of the corrected feature point in the corrected image.
4.6 update h_delta_x, h_delta_y. Let h_delta_x+h_delta_acc_x=1.102+0.001=1.103 be the new h_delta_x, h_delta_y+h_delta_acc_y=0.204+0.002=0.206 be the new h_delta_y, new h_delta_x and new h_delta_y be the offsets for the second update.
5) Calculating a pixel value corresponding to coordinates (x, y) = (4, 0) of the corrected image, wherein the pixel value is the last point of the 0 th row of the corrected image, and the specific process comprises the following steps:
5.1 a prediction calculation module (coordinate_alu) calculates a coordinate value (6.303+h_delta_x, 2.606+h_delta_y) of a fourth prediction feature point according to the coordinate value (6.303,2.606) of the third prediction feature point and the offset updated for the second time, wherein h_delta_x=1.103 and h_delta_y=0.206 are obtained, namely the coordinate value of the fourth prediction feature point is (7.406,2.812), and the coordinate value (7.406,2.812) of the fourth prediction feature point is sent to an address acquisition module (fetch) and a weight calculation module (cmd_fifo).
After receiving the coordinate value (7.406,2.812) of the fourth predicted feature point, the address obtaining module (fetch) reads the coordinate values of a plurality of adjacent feature points from the memory, including a first coordinate value (a 1, b 1) = (7, 2), a second coordinate value (a 2, b 2) = (8, 2), a third coordinate value (a 3, b 3) = (7, 3) and a fourth coordinate value (a 4, b 4) = (8, 3), and queries the pixel values of the four adjacent feature points from the memory according to the register address.
After the weight calculation module (cmd_fifo) receives the coordinate value (7.406,2.812) of the fourth predicted feature point, four weights w0=0.594, w1=0.406, w2=0.188, w3=0.812 are calculated.
5.4 the correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=0.594, w1=0.406, w2=0.188, w3=0.812, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf).
And 5.5, writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (4, 0) point of the corrected feature point in the corrected image.
5.6 since a line is to be fed at this time, the updating of the offset is different from the updating described above. Init_h_delta_x, init_h_delta_y is updated. After updating, h_delta_x=init_h_delta_x+v_change_delta_x=1.1+0.01 is obtained =1.11h_delta_y=init_h_delta_y+v_change_delta_y=0.2+0.03=0.23. Update h_delta_acc_x, h_delta_acc_y. Let h_delta_acc_x+v_change_acc_x=0.001+0.0001=0.0011 be the new h_delta_acc_x, and h_delta_acc_y+v_change_acc_y=0.002+0.0003=0.0023 be the new h_delta_acc_y.
(6) Calculating a pixel value corresponding to coordinates (x, y) = (0, 1) of the corrected image, wherein the point is the first point of the 1 st row, and the specific process comprises the following steps:
6.1 the prediction calculation module (coordinate_alu) calculates the coordinate value (3+v_delta_x, 2+v_delta_y) of the first prediction feature point (fifth prediction feature point) of the second row according to the coordinate value (3, 2) and the offset of the initial feature point of the region to be corrected, wherein v_delta_x=0.3 and v_delta_y=1.3 (see table 1), namely the coordinate value (3.3) of the fifth prediction feature point is obtained, and the coordinate value (3.3) of the fifth prediction feature point is sent to the address acquisition module (fetch) and the weight calculation module (cmd_fifo).
After the address obtaining module (fetch) receives the coordinate value (3.3) of the fifth predicted feature point, it reads a plurality of adjacent feature point coordinate values from the memory, including a first coordinate value (a 1, b 1) = (3, 3), a second coordinate value (a 2, b 2) = (4, 3), a third coordinate value (a 3, b 3) = (3, 4) and a fourth coordinate value (a 4, b 4) = (4, 4), and queries the pixel values of the four adjacent feature points from the memory according to the register address.
After the weight calculation module (cmd_fifo) receives the coordinate value (3.3) of the fifth predicted feature point, four weights w0=0.7, w1=0.3, w2=0.7, w3=0.3 are calculated.
The 6.4 correction calculation module (alu) calculates the pixel value of the corrected feature point according to the pixel values of the four adjacent feature points and the four weights w0=0.7, w1=0.3, w2=0.7, w3=0.3, and writes the pixel value of the corrected feature point into the second buffer storage module (wb_buf).
And 6.5, writing the pixel value of the corrected feature point in the second buffer storage module (wb_buf) into a position corresponding to the coordinate (x, y) = (0, 1) point of the corrected feature point in the corrected image.
6.6 update v_delta_x, v_delta_y. V_delta_x+v_delta_acc_x=0.3+0.0002= 0.3002 as new v_delta_x and v_delta_y+v_delta_acc_y=1.3+0.0004= 1.3004 as new v_delta_y.
(7) And calculating a pixel value corresponding to coordinates (x, y) = (1, 1) of the corrected image. This procedure is the same as (2).
(8) And calculating a pixel value corresponding to coordinates (x, y) = (2, 1) of the corrected image. This procedure is the same as (3).
(9) And calculating a pixel value corresponding to coordinates (x, y) = (3, 1) of the corrected image. This procedure is the same as (4).
(10) The pixel value corresponding to the coordinates (x, y) = (4, 1) of the corrected image is calculated, and the process is the same as (5).
(11) The pixel value corresponding to the coordinates (x, y) = (0, 2) of the corrected image is calculated, and the process is the same as (6).
As shown in fig. 6, the embodiment provides an image correction device, including:
a configuration information obtaining module 110, configured to obtain configuration information of an area to be corrected in the image before correction, where the configuration information includes coordinate values of initial feature points and offset;
the coordinate value calculation module 120 of the predicted feature point is configured to obtain a coordinate value of the predicted feature point according to the coordinate value and the offset of the initial feature point;
a calculation triggering module 130, configured to take the coordinate value of the predicted feature point as the coordinate value of the initial feature point, and instruct the coordinate value calculating module 120 of the predicted feature point to perform the above-mentioned prediction step;
a coordinate value calculation module 140 for calculating coordinate values of the corrected feature points according to the coordinate values of the predicted feature points;
the corrected image generation module 150 is configured to generate a corrected image according to the coordinate values of all the calculated corrected feature points under the condition that the stopping prediction is reached.
In one embodiment, as shown in fig. 7, the coordinate value calculation module 140 of the corrected feature point includes:
a first calculation sub-module 141, configured to select coordinate values of at least one neighboring feature point around the predicted feature point, where the coordinate values of the neighboring feature points are integers;
In one embodiment, as shown in fig. 7, the coordinate value calculation module 140 of the corrected feature point includes:
the adjacent feature point selecting sub-module 142 is configured to select, according to the coordinate values of the predicted feature points, the coordinate values of a plurality of adjacent feature points around the predicted feature points, where the coordinate values of the predicted feature points are not completely integers;
a weight ratio calculation sub-module 143, configured to determine a weight ratio according to a distance between each adjacent feature point and the predicted feature point;
the second calculating sub-module 144 is configured to obtain corresponding pixel values according to coordinate values of each adjacent feature point, and calculate pixel values of the corrected feature points according to the pixel values and the weight ratio of each adjacent feature point;
the coordinate value conversion sub-module 145 is configured to convert the pixel value of the corrected feature point into a coordinate value of the corrected feature point.
In one embodiment, as shown in fig. 7, further comprising:
the feature point detection module 100 is configured to detect a plurality of feature points in a region to be corrected using a neural network model;
a parameter calculation module 101 for calculating a distortion angle and an attitude parameter according to coordinate values of a plurality of feature points;
The offset calculating module 102 is configured to obtain an offset according to the distortion angle and the gesture parameter.
In one embodiment, the configuration information further includes a preset size parameter of the corrected image, and the condition for stopping the prediction is generated according to the preset size parameter of the corrected image.
The functions of each module in each apparatus of the embodiments of the present application may be referred to the corresponding descriptions in the above methods, which are not described herein again.
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 8, a block diagram of an electronic device according to an image correction method according to an embodiment of the present application is shown. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 8, the electronic device includes: one or more processors 801, memory 802, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of a graphical user interface (Graphical User Interface, GUI) on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 801 is illustrated in fig. 8.
Memory 802 is a non-transitory computer-readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform an image rectification method provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform an image correction method provided by the present application.
The memory 802 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to an image correction method in the embodiments of the present application (e.g., the configuration information acquisition module 110, the first calculation module 120, the calculation triggering module 130, the second calculation module 140, and the corrected image generation module 150 shown in fig. 6). The processor 801 executes various functional applications of the server and data processing, i.e., implements an image correction method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 802.
Memory 802 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of an electronic device of an image correction method, or the like. In addition, memory 802 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 802 may optionally include memory located remotely from processor 801, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device may further include: an input device 803 and an output device 804. The processor 801, memory 802, input devices 803, and output devices 804 may be connected by a bus or other means, for example in fig. 8.
The input device 803 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of the electronic device described above, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, and the like. The output device 804 may include a display apparatus, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a light emitting diode (Light Emitting Diode, LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, application specific integrated circuits (Application Specific Integrated Circuits, ASIC), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices (programmable logic device, PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., CRT (Cathode Ray Tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (12)
1. An image correction method, comprising:
acquiring configuration information of an area to be corrected in an image before correction, wherein the configuration information comprises coordinate values and offset of initial feature points;
obtaining coordinate values of the predicted feature points according to the coordinate values of the initial feature points and the offset, taking the coordinate values of the predicted feature points as the coordinate values of the initial feature points, and returning to execute the prediction step;
Calculating coordinate values of corrected feature points according to the coordinate values of the predicted feature points;
and under the condition of stopping prediction, generating a corrected image according to the coordinate values of all the corrected feature points obtained through calculation.
2. The method according to claim 1, wherein calculating coordinate values of corrected feature points from the coordinate values of the predicted feature points comprises:
and taking the coordinate value of the predicted feature point as the coordinate value of the corrected feature point under the condition that the coordinate value of the predicted feature point is an integer.
3. The method according to claim 1, wherein calculating coordinate values of corrected feature points from the coordinate values of the predicted feature points comprises:
under the condition that the coordinate value of the predicted feature point is not an integer, selecting the coordinate values of a plurality of adjacent feature points around the predicted feature point according to the coordinate value of the predicted feature point, wherein the coordinate values of the adjacent feature points are integers;
determining a weight ratio according to the distance between each adjacent feature point and the predicted feature point;
acquiring corresponding pixel values according to the coordinate values of the adjacent feature points, and calculating the pixel values of the corrected feature points according to the pixel values of the adjacent feature points and the weight ratio;
And converting the pixel value of the corrected feature point into a coordinate value of the corrected feature point.
4. The method as recited in claim 1, further comprising:
detecting a plurality of characteristic points in the region to be corrected by using a neural network model;
calculating a torsion angle and an attitude parameter according to the coordinate values of the plurality of feature points;
and obtaining the offset according to the torsion angle and the attitude parameter.
5. The method of claim 1, wherein the configuration information further includes a preset size parameter of the corrected image, and the condition to stop prediction is generated according to the preset size parameter of the corrected image.
6. An image correction device, comprising:
the configuration information acquisition module is used for acquiring configuration information of an area to be corrected in the image before correction, wherein the configuration information comprises coordinate values of initial feature points and offset;
the coordinate value calculation module of the predicted feature point is used for obtaining the coordinate value of the predicted feature point according to the coordinate value of the initial feature point and the offset;
the calculation triggering module is used for taking the coordinate value of the prediction feature point as the coordinate value of the initial feature point and indicating the coordinate value calculation module of the prediction feature point to execute the prediction step;
The coordinate value calculation module of the corrected feature point is used for calculating the coordinate value of the corrected feature point according to the coordinate value of the predicted feature point;
and the corrected image generation module is used for generating a corrected image according to the coordinate values of all the corrected characteristic points obtained through calculation under the condition of stopping prediction.
7. The apparatus according to claim 6, wherein the coordinate value calculation module of the corrected feature point includes:
and the first calculation sub-module is used for taking the coordinate value of the predicted characteristic point as the coordinate value of the corrected characteristic point under the condition that the coordinate value of the predicted characteristic point is an integer.
8. The apparatus according to claim 6, wherein the coordinate value calculation module of the corrected feature point includes:
the adjacent feature point selecting sub-module is used for selecting coordinate values of a plurality of adjacent feature points around the predicted feature point according to the coordinate values of the predicted feature point under the condition that the coordinate values of the predicted feature point are not completely integers, and the coordinate values of the adjacent feature points are integers;
the weight ratio calculation sub-module is used for determining weight ratio according to the distance between each adjacent characteristic point and the predicted characteristic point;
The second computing sub-module is used for obtaining corresponding pixel values according to the coordinate values of the adjacent feature points and computing the pixel values of the corrected feature points according to the pixel values of the adjacent feature points and the weight ratio;
and the coordinate value conversion sub-module is used for converting the pixel value of the corrected feature point into the coordinate value of the corrected feature point.
9. The apparatus as recited in claim 6, further comprising:
the characteristic point detection module is used for detecting a plurality of characteristic points in the region to be corrected by using a neural network model;
the parameter calculation module is used for calculating a distortion angle and an attitude parameter according to the coordinate values of the plurality of characteristic points;
and the offset calculation module is used for obtaining the offset according to the torsion angle and the attitude parameter.
10. The apparatus of claim 6, wherein the configuration information further comprises a preset size parameter of the rectified image, and wherein the condition to stop prediction is generated according to the preset size parameter of the rectified image.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010307251.XA CN113538252B (en) | 2020-04-17 | 2020-04-17 | Image correction method and device |
| PCT/CN2020/118676 WO2021208369A1 (en) | 2020-04-17 | 2020-09-29 | Image correction method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010307251.XA CN113538252B (en) | 2020-04-17 | 2020-04-17 | Image correction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113538252A CN113538252A (en) | 2021-10-22 |
| CN113538252B true CN113538252B (en) | 2024-03-26 |
Family
ID=78084730
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010307251.XA Active CN113538252B (en) | 2020-04-17 | 2020-04-17 | Image correction method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113538252B (en) |
| WO (1) | WO2021208369A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114494686A (en) * | 2022-01-28 | 2022-05-13 | 北京百度网讯科技有限公司 | Text image correction method, text image correction device, electronic equipment and storage medium |
| CN115797198B (en) * | 2022-10-24 | 2023-05-23 | 北京华益精点生物技术有限公司 | Image mark correction method and related equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107945113A (en) * | 2017-11-17 | 2018-04-20 | 北京天睿空间科技股份有限公司 | The antidote of topography's splicing dislocation |
| CN108198219A (en) * | 2017-11-21 | 2018-06-22 | 合肥工业大学 | Error compensation method for camera calibration parameters for photogrammetry |
| CN108391050A (en) * | 2018-02-12 | 2018-08-10 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN110400266A (en) * | 2019-06-13 | 2019-11-01 | 北京小米移动软件有限公司 | A kind of method and device of image flame detection, storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663413B (en) * | 2012-03-09 | 2013-11-27 | 中盾信安科技(江苏)有限公司 | Multi-gesture and cross-age oriented face image authentication method |
| US10796480B2 (en) * | 2015-08-14 | 2020-10-06 | Metail Limited | Methods of generating personalized 3D head models or 3D body models |
| CN106600640B (en) * | 2016-12-12 | 2020-03-20 | 杭州视氪科技有限公司 | Face recognition auxiliary glasses based on RGB-D camera |
| US10373291B1 (en) * | 2018-01-31 | 2019-08-06 | Google Llc | Image transformation for machine learning |
-
2020
- 2020-04-17 CN CN202010307251.XA patent/CN113538252B/en active Active
- 2020-09-29 WO PCT/CN2020/118676 patent/WO2021208369A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107945113A (en) * | 2017-11-17 | 2018-04-20 | 北京天睿空间科技股份有限公司 | The antidote of topography's splicing dislocation |
| CN108198219A (en) * | 2017-11-21 | 2018-06-22 | 合肥工业大学 | Error compensation method for camera calibration parameters for photogrammetry |
| CN108391050A (en) * | 2018-02-12 | 2018-08-10 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN110400266A (en) * | 2019-06-13 | 2019-11-01 | 北京小米移动软件有限公司 | A kind of method and device of image flame detection, storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021208369A1 (en) | 2021-10-21 |
| CN113538252A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111523468B (en) | Human body key point identification method and device | |
| US20210209774A1 (en) | Image adjustment method and apparatus, electronic device and storage medium | |
| CN112785674A (en) | Texture map generation method, rendering method, device, equipment and storage medium | |
| CN111524166B (en) | Video frame processing method and device | |
| CN112381183B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN111738072A (en) | Training method and device of target detection model and electronic equipment | |
| CN112634343A (en) | Training method of image depth estimation model and processing method of image depth information | |
| CN112150551A (en) | Object pose acquisition method and device and electronic equipment | |
| CN113538252B (en) | Image correction method and device | |
| CN111275827B (en) | Edge-based augmented reality three-dimensional tracking registration method and device and electronic equipment | |
| CN110675635B (en) | Method and device for acquiring external parameters of camera, electronic equipment and storage medium | |
| CN112509058B (en) | External parameter calculating method, device, electronic equipment and storage medium | |
| CN111967297A (en) | Semantic segmentation method and device for image, electronic equipment and medium | |
| CN111191619B (en) | Method, device and equipment for detecting virtual line segment of lane line and readable storage medium | |
| CN112907496A (en) | Image fusion method and device | |
| CN112083846A (en) | Method and device for detecting click area of map display mark and electronic equipment | |
| CN113628239B (en) | Display optimization method, related device and computer program product | |
| CN113205090B (en) | Picture correction method, device, electronic equipment and computer readable storage medium | |
| CN112085814A (en) | Electronic map display method, device, equipment and readable storage medium | |
| CN111915642A (en) | Image sample generation method, device, equipment and readable storage medium | |
| CN111833391A (en) | Method and device for estimating image depth information | |
| CN111696134A (en) | Target detection method and device and electronic equipment | |
| CN112150380B (en) | Method, apparatus, electronic device, and readable storage medium for correcting image | |
| CN114565777A (en) | Data processing method and device | |
| CN114119990A (en) | Method, apparatus and computer program product for image feature point matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |