WO2023098045A1 - 图像对齐方法、装置、计算机设备和存储介质 - Google Patents
图像对齐方法、装置、计算机设备和存储介质 Download PDFInfo
- Publication number
- WO2023098045A1 WO2023098045A1 PCT/CN2022/100804 CN2022100804W WO2023098045A1 WO 2023098045 A1 WO2023098045 A1 WO 2023098045A1 CN 2022100804 W CN2022100804 W CN 2022100804W WO 2023098045 A1 WO2023098045 A1 WO 2023098045A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- target
- interest
- region
- point
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 88
- 239000011159 matrix material Substances 0.000 claims abstract description 68
- 238000013507 mapping Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 16
- 239000000284 extract Substances 0.000 claims description 54
- 238000001514 detection method Methods 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 26
- 238000010586 diagram Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 17
- 230000000694 effects Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 238000005286 illumination Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
Definitions
- the present disclosure relates to an image alignment method, device, computer equipment and storage medium.
- Image alignment plays an important role in image fusion, image stitching, and computer vision. It is to align two or more images at different times or from different angles to determine the mapping relationship between the spatial positions and intensities of the images. In this way, multiple frames of images shot continuously in a very short period of time are fused into one frame of image.
- the feature-based image alignment method in the related art first finds the feature points in the two pictures and performs feature matching, and then calculates the homography matrix to uniformly transform the pixel coordinates in the images to perform image alignment. There are uncertain factors such as illumination, depth of field, and angle in the process, which makes it difficult to extract obvious feature points from the image to be processed, which makes the method's scene adaptability poor and the accuracy of image alignment low.
- the method solves the problem in the related art that the accuracy of image alignment is not high due to inconspicuous feature points.
- an image alignment method, device, computer equipment, and storage medium are provided.
- An image alignment method comprising:
- the pixel value of each pixel in the first region of interest is mapped to the corresponding pixel in the second region of interest, so as to obtain the first region of interest after image alignment.
- acquiring the first image includes: acquiring a histogram of the second image, and determining the pixel value of each pixel in the second image; Set the pixel value of the pixel point corresponding to the pixel position in the third image; according to the pixel value of each pixel point in the second image and the pixel value of each pixel point in the third image after setting, extract At least one first corner point in the third image, and extract at least one second corner point in the second image; perform feature matching on at least one first corner point and at least one second corner point, and determine at least one first corner point A first target corner point among the points, and a second target corner point among at least one second corner point, the first target corner point matches the second target corner point according to the coordinates of the first target corner point and the second target corner point
- the coordinates of the second homography matrix are calculated; based on the second homography matrix, the pixel value of each pixel point in the third image is mapped to the corresponding pixel point of the second image, and
- the number of at least one feature matching is multiple, and performing feature matching on at least one first corner point and at least one second corner point includes: combining at least one first corner point Perform feature matching with at least one second corner point in units of N corner points; perform feature matching again with at least one first corner point and at least one second corner point in units of M corner points; Wherein, N is a positive integer greater than M.
- calculating the second homography matrix according to the coordinates of the first target corner and the coordinates of the second target corner includes: if the number of the first target corner is greater than or equal to the preset number, then calculate the second homography matrix according to the coordinates of the first target corner point and the coordinates of the second target corner point; the method also includes: if the number of the first target corner points is less than the preset number, then Extract a plurality of first ORB feature points from the third image, and extract a plurality of second ORB feature points from the second image; perform feature matching with a plurality of first ORB feature points and a plurality of second ORB feature points, and determine the plurality of ORB feature points The coordinates of at least one first target ORB feature point among the first ORB feature points, and the coordinates of at least one second target ORB feature point among a plurality of second ORB feature points, at least one first target ORB feature point and at least one Matching the second target ORB feature points; calculating
- multiple first feature points are extracted from a first region of interest in the first image
- multiple second feature points are extracted from a second region of interest in the second image
- a plurality of first feature points are extracted from the object graph
- a plurality of second feature points are extracted from the second target object.
- extracting a plurality of first feature points from the first target object map, and extracting a plurality of second feature points from the second target object includes: identifying the first target object Whether the target object is included in the figure and the second target object figure; if the target object is contained in the first target object figure and the second target object figure, then a plurality of first ECC feature points are extracted from the first target object figure, and from the second target object figure A plurality of second ECC feature points are extracted from the target object graph.
- after determining the first target object graph from the first region of interest and determining the second target object graph from the second region of interest further includes: acquiring the second target object graph and determine the pixel value of each pixel in the second object image; according to the pixel value of each pixel in the second object image, set the pixel corresponding to the pixel position in the first object image value.
- the target object detection is portrait detection, determining the first target object map from the first region of interest, and determining the second target object map from the second region of interest, including: obtaining A first portrait mask of the first region of interest obtained after the portrait detection, and a second portrait mask of the second region of interest; determining the first portrait image from the first image according to the first portrait mask, And determining a second portrait image from the second image according to the second portrait mask.
- the method further includes: extracting a third region of interest in the first image and a fourth region of interest in the second image according to a default size; If the size of the ROI is smaller than the preset size, expand the third ROI according to the preset size to obtain the first ROI; if the size of the fourth ROI is smaller than the preset size, expand the third ROI according to the preset size Extend the fourth ROI to obtain the second ROI.
- An image alignment device comprising:
- An acquisition module configured to acquire a first image and a second image, where the first image is a first image after global image alignment processing based on the second image;
- An extraction module configured to extract a plurality of first feature points from a first region of interest in the first image, and extract a plurality of second feature points from a second region of interest in the second image; the first region of interest and the second The ROIs have the same size and correspond to positions in different images;
- a feature matching module configured to perform feature matching on a plurality of first feature points and a plurality of second feature points, determine the coordinates of at least one first target feature point in the plurality of first feature points, and determine a plurality of second feature points Coordinates of at least one second target feature point, at least one first target feature point matches at least one second target feature point;
- a calculation module configured to calculate a first homography matrix for pixel mapping according to the coordinates of at least one first target feature point and the coordinates of at least one second target feature point;
- a mapping module configured to map the pixel value of each pixel in the first region of interest to the corresponding pixel in the second region of interest based on the first homography matrix, so as to obtain the first image of interest after image alignment area.
- the acquiring module is configured to acquire a histogram of the second image, and determine the pixel value of each pixel in the second image; according to the histogram of each pixel in the second image Pixel value, set the pixel value of the pixel point corresponding to the pixel position in the third image; according to the pixel value of each pixel point in the second image, and the pixel value of each pixel point in the third image after setting, extract the third At least one first corner point in the image, and extracting at least one second corner point in the second image; performing feature matching on at least one first corner point and at least one second corner point, and determining at least one of the first corner points The first target corner point of , and the second target corner point of at least one second corner point, the first target corner point matches the second target corner point; according to the coordinates of the first target corner point and the second target corner point coordinates, calculate the second homography matrix; based on the second homography matrix, map the pixel value of each pixel point in
- the acquisition module is configured to perform feature matching once with at least one first corner point and at least one second corner point in units of N corner points; One corner point and at least one second corner point, with M corner points as the unit, perform feature matching again;
- N is a positive integer greater than M.
- the obtaining module is configured to, under the condition that the number of the first target corner points is greater than or equal to the preset number, according to the coordinates of the first target corner point and the second target corner point The coordinates of , calculate the second homography matrix; under the condition that the number of the first target corner points is less than the preset number, extract a plurality of first directional fast rotation ORB feature points from the third image, and extract from the second image Extracting a plurality of second ORB feature points; performing feature matching on a plurality of first ORB feature points with a plurality of second ORB feature points, determining the coordinates of at least one first target ORB feature point in the plurality of first ORB feature points, and determining the coordinates of at least one second target ORB feature point among the plurality of second target ORB feature points, at least one first target ORB feature point matching at least one second target ORB feature point; according to the coordinates of at least one first target ORB feature point , and the coordinates of
- the extraction module is configured to perform target object detection on the first region of interest and the second region of interest; determine the first target object graph from the first region of interest, and obtain the The second region of interest determines a second object map; extracts a plurality of first feature points from the first object map, and extracts a plurality of second feature points from the second object map.
- the first feature point is a first entropy correlation coefficient ECC feature point
- the second feature point is a second ECC feature point
- An extraction module configured to identify whether the first target object graph and the second target object graph contain the target object; under the condition that the first target object graph and the second target object graph contain the target object, extract the A plurality of first ECC feature points are extracted, and a plurality of second ECC feature points are extracted from the second target object map.
- the extraction module is configured to obtain a histogram of the second target object graph, and determine the pixel value of each pixel in the second target object graph; according to the second target object graph The pixel value of each pixel in the set, set the pixel value of the pixel corresponding to the pixel position in the first object map.
- the target object detection is portrait detection
- the extraction module is configured to obtain the first portrait mask of the first region of interest obtained after the portrait detection, and the second region of interest A second portrait mask; determining a first portrait image from the first image according to the first portrait mask, and determining a second portrait map from the second image according to the second portrait mask.
- the extraction module is configured to extract the third region of interest in the first image and the fourth region of interest in the second image according to a default size; in the third region of interest If the size of the area is smaller than the preset size, expand the third ROI according to the preset size to obtain the first ROI; if the size of the fourth ROI is smaller than the preset size, expand according to the preset size The fourth ROI is used to obtain the second ROI.
- a computer device comprising a memory and one or more processors, the memory configured as a module storing computer readable instructions; the computer readable instructions, when executed by the one or more processors, cause the One or more processors execute the steps of providing the image alignment method in any one embodiment of the present disclosure.
- One or more non-volatile storage media storing computer-readable instructions.
- the computer-readable instructions are executed by one or more processors, one or more processors execute the image provided in any one of the embodiments of the present disclosure. Align the steps of the method.
- FIG. 1 is an application scenario diagram 1 of an image alignment method provided by one or more embodiments of the present disclosure
- FIG. 2 is a flow chart 1 of steps of an image alignment method provided by one or more embodiments of the present disclosure
- FIG. 3 is the second application scene diagram of the image alignment method provided by one or more embodiments of the present disclosure.
- FIG. 4 is a schematic diagram of extracting a region of interest provided by one or more embodiments of the present disclosure
- FIG. 5 is a second flow chart of the steps of the image alignment method provided by one or more embodiments of the present disclosure.
- FIG. 6 is a flowchart three of steps of an image alignment method provided by one or more embodiments of the present disclosure.
- Fig. 7 is a structural block diagram of an image alignment device in one or more embodiments of the present disclosure.
- FIG. 8 is an internal structural diagram of a computer device in one or more embodiments of the present disclosure.
- first and second and the like in the specification and claims of the present disclosure are used to distinguish different objects, rather than to describe a specific order of objects.
- first camera and the second camera are used to distinguish different cameras, not to describe a specific order of the cameras. .
- words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design described as “exemplary” or “for example” in the embodiments of the present disclosure shall not be construed as being preferred or advantageous over other embodiments or designs. To be precise, the use of words such as “exemplary” or “for example” is intended to present related concepts in a specific manner. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "plurality” refers to two one or more.
- the feature-based image alignment method first finds the feature points in the two pictures and performs feature matching, and then calculates the homography matrix to uniformly transform the pixel coordinates in the image for image alignment, but in the actual operation process, due to There are uncertain factors such as illumination, depth of field, and angle in the image shooting process, which lead to obvious time stamps of images that need to be processed, obvious depth of field, large brightness differences, large field of view differences, and large changes in image space, making it difficult to extract obvious feature points.
- This method has poor scene adaptability and low accuracy of image alignment.
- the present disclosure first performs global image alignment on the image to be processed, and then extracts feature points from the first region of interest in the image to be processed and the second region of interest in the reference image for feature matching to obtain the matched feature points
- the coordinates of the homography matrix are calculated to map the first region of interest to the second region of interest according to the homography matrix to achieve local alignment of the image to be processed by aligning all regions of interest in the image to be processed.
- the precise alignment of the images to be processed reduces the influence of uncertain factors in the image shooting process, and can extract more feature points for image alignment, which is applicable to various scenarios and improves the accuracy of image alignment.
- the image alignment method provided in the present disclosure can be applied to the application environment shown in FIG. 1 .
- the image alignment method is applied in an image alignment system.
- the image alignment system includes a terminal 102 and a server 104 . Wherein, the terminal 102 communicates with the server 104 through a network. Specifically, the image alignment method is applied to the terminal 102.
- the terminal 102 performs global image alignment on the image to be processed, and then extracts feature points from the first region of interest in the image to be processed and the second region of interest in the reference image to perform feature Matching to obtain the coordinates of the matched feature points, thereby calculating the homography matrix to map the first region of interest to the second region of interest according to the homography matrix to achieve local alignment of the image to be processed, by aligning the image to be processed All regions of interest in the image to achieve precise alignment of the image to be processed.
- the image alignment method is applied to the server 104, the server 104 receives the image to be processed 106 sent by the terminal 102, and accurately aligns the image through the above image alignment method, and then the server 104 returns the precisely aligned image 108 to the terminal 102, the The method also includes steps S202-S210:
- the terminal 102 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server 104 can be realized by an independent server or a server cluster composed of multiple servers.
- FIG. 2 is a flow chart 1 of steps of an image alignment method provided by one or more embodiments of the present disclosure.
- the present disclosure is mainly illustrated by taking the method applied to the terminal 102 in FIG. 1 as an example.
- the first image is an image obtained by performing global image alignment processing on the third image with the second image as a reference image
- the third image is an image obtained from a terminal device as a comparison device, which requires global image alignment processing
- the second image is an image obtained from the terminal device as a reference device
- the second image and the third image are images of the same scene at different angles, or images of the same scene at different times.
- the second image and the third image are acquired from different terminal devices.
- the second image is a reference image
- the third image is an image to be processed that requires global image alignment processing.
- the devices included in the figure include a mobile phone 301 , a mobile phone 303 and a server 305 ; the images include a second image 302 , a third image 304 , and a first image 306 .
- the second image 302 is obtained from the mobile phone 301 as the reference machine
- the third image 304 is obtained from the mobile phone 304 as the comparison machine
- the server 305 uses the second image 302 as the reference image to perform global image alignment processing on the third image to obtain the second image.
- An image 306 uses the second image 302 as the reference image to perform global image alignment processing on the third image to obtain the second image.
- the following will introduce the process of using the second image as a reference image to perform global image processing on the third image to obtain the first image.
- the histogram of the second image is obtained. Since the histogram represents the number of pixels of each brightness level of the image, it shows the The distribution in , intuitively reflects the brightness of the picture, so the pixel value of each pixel in the second image can be determined according to the histogram of the second image; then according to the pixel value of each pixel in the second image, use Set the pixel value of the pixel point corresponding to the pixel position in the third image by histogram equalization method, histogram specification, etc., so as to adjust the brightness of the third image according to the second image, so that the pixel value of the pixel point corresponding to the pixel position in the third image
- the pixel value is set to be the same as the pixel value of the pixel point corresponding to the corresponding pixel position in the second image, so that the brightness of the third image is the same as that of the first image, so as to extract feature points from the two pictures.
- the brightness of the third image may be adjusted by methods such as smoothing and denoising, illumination equalization processing, etc., including but not limited to the above methods, which are not limited in the present disclosure.
- each pixel in the second image and the set pixel value of each pixel in the third image extract at least one corner point in the brightness-adjusted third image, and extract the second At least one corner point in the image, and match the corner points of the two extracted images.
- the corner point is the isolated point with the greatest or smallest intensity on certain attributes, the end point of the line segment, or the point with the largest local curvature on the curve, which is used to describe the corners, boundary points and other information in the image.
- the corner points are extracted by the Harris corner detection method. It should be noted that the basic idea of the Harris corner detection method is to use a fixed window to slide in any direction on the image, and compare the before and after sliding. In both cases, the pixel grayscale changes in the window. If there is a sliding in any direction and there is a large grayscale change, it is considered that there is a corner in the window.
- the corner points are extracted by the Harris corner point detection method, which improves the robustness to noise and improves the efficiency and stability of corner point detection.
- the corner points in the extracted third image and the corner points in the second image are subjected to feature matching at least once, and firstly, when the image with a smaller scale is blurred , perform feature matching on N corner points to obtain a rough matching result of the corner points, and then increase the scale to perform feature matching on M corner points until the scale is transformed to the size of the original image to complete the precise matching of the image, where , N is a positive integer greater than M.
- the coarse matching and precise matching of the third image and the second image are realized, the false matching rate is reduced, and the matching accuracy is improved.
- the second homography matrix is a homography matrix used for global image processing on the third image.
- a plurality of Oriented Fast and Rotated Brief Oriented Fast and Rotated Brief, ORB
- Oriented Fast and Rotated Brief ORB
- second ORB feature points from the second image
- carry out feature matching with a plurality of first ORB feature points and a plurality of second ORB feature points and determine among a plurality of first ORB feature points coordinates of at least one first target ORB feature point, and determining coordinates of at least one second target ORB feature point among a plurality of second target ORB feature points, the at least one first target ORB feature point matching at least one second target ORB feature point , and then, according to the coordinates of at least one first target ORB feature point and the coordinates of at least one second target ORB feature point, calculate the second homography matrix.
- ORB feature points are extracted for feature matching, the matched ORB feature points are determined, and the coordinates corresponding to the ORB feature points are obtained, thereby calculating the homography matrix, which improves the accuracy and efficiency of feature matching.
- the present disclosure uses a random sampling consensus algorithm to calculate the homography matrix and perform iterative update to eliminate mismatching points and determine the optimal homography matrix.
- the problem of the difference in brightness between the image to be processed and the reference image is solved, and the convenience of the feature matching process is improved; the feature points of the third image and the second image are matched multiple times to reduce errors.
- the matching rate improves the accuracy of matching; the optimal homography matrix is iteratively calculated through the random consensus algorithm, so as to obtain the first image after global image alignment, and achieve a better effect of global image alignment.
- the size of the first ROI is the same as that of the second ROI, and their positions in different images correspond to each other.
- the first ROI is extracted from the first image according to the default size.
- the default size is set according to the user’s experience value.
- the default size is A ⁇ A, and the unit is pixel or centimeter.
- the extraction principle of the second ROI is the same as The principle of extracting the first region of interest is the same, and the present disclosure will not repeat them here.
- Figure 4 is a schematic diagram of extracting a region of interest from an image, setting a window 402 with a size of A ⁇ A, and extracting from the upper left corner of the image 401 to obtain a region of interest 403 with a size of A ⁇ A .
- multiple regions of interest of the image can be obtained by traversing the image according to a default size or a preset size.
- the mosaic combination of multiple ROIs is the original image before ROI extraction.
- the third region of interest in the first image after global image alignment processing and the fourth region of interest of a corresponding size in the second image are extracted according to a default size. Since the region of interest is selected according to the default size, the region of interest may be smaller and the feature points less, and the user needs to set the preset size.
- An embodiment of the present disclosure provides an implementation method. First, determine the third region of interest and Whether the fourth ROI is smaller than the preset size, if the third ROI and the fourth ROI are smaller than the preset size, expand the third ROI to obtain the first ROI, expand the fourth ROI to obtain the 2. The region of interest. Optionally, expand the boundary of the region of interest to obtain a region of interest with a larger size for feature point extraction.
- the scene in the image is detected as the target object, and the target object is first detected for the first region of interest and the second region of interest, from Determining a first target object graph from a first region of interest, and determining a second target object graph from a second region of interest; then identifying whether the first region of interest and the second region of interest contain the target object;
- first object map and the second object map When the target object is detected in the first region of interest and the second region of interest, feature extraction is performed on the first object map and the second object map, and a plurality of first object images are extracted from the first object map. feature points, and extract a plurality of second feature points from the second target object graph.
- the first feature point is the first corner point
- the second feature point is the second corner point
- the first feature point is the first ECC feature point
- second feature point is the second ECC feature point.
- the scene in the image may be a plant, an animal, a building, and other objects of concern in the field of image processing.
- the following will introduce the local alignment process in the case that the scene in the image is a person.
- portrait detection is performed on the first region of interest and the second region of interest; after the portrait detection, the first portrait mask of the first region of interest and the portrait mask of the second region of interest are obtained; according to The first portrait mask and the second portrait mask determine the first portrait image from the first region of interest, and determine the second portrait image from the second region of interest, and then extract multiple A first feature point, and extracting a plurality of second feature points from the second portrait.
- the deep learning algorithm is used to perform portrait detection on the region of interest extracted from the image, and the portrait image is extracted to achieve image alignment according to the portrait feature points in the portrait image.
- the embodiments of the present disclosure provide One embodiment: identify whether the first portrait image and the second portrait image contain portraits; if the first portrait image and the second portrait image contain portraits, then extract a plurality of first features from the first portrait image points, and extract a plurality of second feature points from the second portrait, since the first portrait image and the second portrait image contain portraits, and the feature points in the portraits are obvious and large in number, in order to improve the efficiency of feature matching , optionally, the first feature point is the first corner point, and the second feature point is the second corner point. Using feature points as corner points ensures that when the number of feature points is large and obvious, the speed of feature matching is improved, and the reliability of feature matching is improved.
- the number of matched corner points obtained is less than the preset number, then according to the first person Calculate the first entropy correlation coefficient (Entropy Corrleation Coefficient, ECC), and then extract a plurality of first ECC feature points from the first portrait image, and extract a plurality of second ECC feature points from the second portrait image.
- ECC Entropy Corrleation Coefficient
- the histogram of the second portrait image is obtained, and the second portrait image is determined.
- the pixel value of each pixel in the two-person portrait according to the pixel value of each pixel in the second portrait, set the pixel value of the pixel corresponding to the pixel position in the first portrait.
- the pixel value of each pixel in the first region of interest is set to adjust the brightness of the first region of interest using the second region of interest as a reference, and further extract at least one corner point from the first region of interest, and extract at least one corner point from the second region of interest, match the corner points of the extracted first region of interest and the second region of interest, and then calculate the first homography matrix, so that the first homography
- the property matrix is used for image alignment processing.
- the embodiment of the present disclosure provides an implementation mode.
- the region of interest extracts ECC feature points, and extracts ECC feature points from the second region of interest.
- the ECC feature points are feature points with high feature matching accuracy, so that there is no obvious difference between the first region of interest and the second ECC feature points.
- precise feature matching is carried out, and when there is no obvious scene in the image, feature matching is performed by extracting ECC feature points, which improves the accuracy of image alignment and obtains a better image alignment effect.
- feature points are extracted by performing target object detection in the first image to obtain the target object map, wherein, when the target object map contains the target object, corner points are used as feature points for extraction, in order to ensure feature matching Accuracy, using ECC feature points as feature points for extraction; in the case that the target object map does not contain the target object, adjust the brightness according to the reference area of interest, and extract the corner points of the area of interest for subsequent use At least one feature match. Whether or not the image contains the target object is processed separately to ensure the accuracy of the extracted feature points for subsequent feature point matching, thereby improving the accuracy of image alignment.
- S206 Perform feature matching on multiple first feature points and multiple second feature points, determine the coordinates of at least one first target feature point among the multiple first feature points, and determine at least one first target feature point among the multiple second feature points The coordinates of the two target feature points.
- At least one first target feature point matches at least one second target feature point.
- the present disclosure performs feature matching according to the corner points extracted after the target object is detected.
- the determined number of target feature points is more than 3 pairs required for calculating the homography matrix.
- the present disclosure extracts ECC feature points according to the above steps for feature matching.
- the feature matching method includes but is not limited to accelerated robust features (Speeded Up Robust Feature, SURF), Scale Invariant Feature Transform (SIFT), Accelerated Segmentation Test Acquired Features (Features from Accelerated Segment Test, FAST), which are not limited in this disclosure.
- the present disclosure performs feature matching based on the corner points extracted after the target object is not detected, and performs at least one feature matching that is the same as in the global image alignment process, first in the case of a smaller-scale image that is blurred Next, perform feature matching on N corner points to obtain a rough matching result of the corner points, and then increase the scale to perform feature matching on M corner points until the scale is transformed to the size of the original image to complete the precise matching of the image.
- N is a positive integer greater than M.
- the coordinates of at least one target feature point and the coordinates of at least one second target feature point are determined, wherein, The at least one first target feature point is at least one first ECC target feature point, and the at least one second target feature point is at least one second ECC target feature point determined by an ECC feature matching method.
- the random sampling consensus algorithm calculates the homography matrix and iteratively updates it, eliminates the wrong matching points, and determines the optimal homography matrix.
- the aligned first region of interest is cropped and output. If the first ROI is an expanded ROI, clipping is performed according to the size of the expanded ROI, and the expanded ROI is output.
- the region of interest is extracted multiple times from the first image and the second image, and then steps S204-S210 are performed until each region of interest in the first image is image-aligned, and then the output All the images of the first image are aligned to the processed region of interest, so as to obtain the first image after complete image alignment.
- the present disclosure first performs global image alignment on the image to be processed, and then extracts feature points from the first region of interest in the image to be processed and the second region of interest in the reference image for feature matching, and obtains the coordinates of the matched feature points , so as to calculate the homography matrix, so as to map the first region of interest to the second region of interest according to the homography matrix to realize the local alignment of the image to be processed, and realize the to-be-processed image by aligning all regions of interest in the image to be processed
- the precise alignment of images reduces the influence of uncertain factors in the image shooting process, and can extract more feature points for image alignment, which is applicable to various scenarios and improves the accuracy of image alignment.
- Fig. 5 is a flow chart 2 of the steps of the image alignment method provided by one or more embodiments of the present disclosure.
- Fig. 5 is a further expansion and optimization of the present disclosure based on the embodiment shown in Fig. 2 , wherein a part of S202 A possible implementation is as follows:
- N is a positive integer greater than M.
- S512 Determine a first target corner point in at least one first corner point, and a second target corner point in at least one second corner point.
- the first image is obtained by adjusting the brightness of the third image, which solves the problem of brightness difference between the image to be processed and the reference image, and improves the convenience of the feature matching process; iteratively calculates the optimal homography matrix through the random consensus algorithm, and thus obtains The first image after the global image alignment achieves a better effect of global image alignment; the third image is matched with the second image for multiple feature points, which reduces the false matching rate and improves the matching accuracy.
- Fig. 6 is a flowchart three of steps of an image alignment method provided by one or more embodiments of the present disclosure.
- Fig. 6 is a further expansion and optimization of the present disclosure on the basis of the embodiment shown in Fig. 2 , wherein one of S205 A possible implementation is as follows:
- S602. Perform target object detection on the first region of interest and the second region of interest.
- the target object map is obtained, and the brightness of the target object map corresponding to the image to be processed is adjusted to determine whether the important information of the target object is contained in the region of interest, and then extract the relevant information of the target object.
- ECC feature points so as to extract obvious feature points for image alignment, reduce the influence of other interference factors in the image shooting process, and improve the accuracy of image alignment.
- steps in the flow charts in FIGS. 2-4 are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 2-4 may include a plurality of sub-steps or stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, these sub-steps or stages The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.
- the embodiment of the present disclosure also provides an image alignment device, the device embodiment corresponds to the foregoing method embodiment, for the sake of easy reading, the present device embodiment does not implement the foregoing method.
- the details in the examples are described one by one, but it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiments.
- FIG. 7 is a structural block diagram of an image alignment device in one or more embodiments of the present disclosure. As shown in FIG. Module 708 and mapping module 710, wherein:
- the acquisition module 702 is configured to acquire a first image and a second image, the first image is the first image after global image alignment processing based on the second image;
- Extraction module 704 configured to extract a plurality of first feature points from the first region of interest of the first image, and extract a plurality of second feature points from the second region of interest of the second image; the first region of interest and the first region of interest The two regions of interest have the same size and correspond to the positions in different images;
- the feature matching module 706 is configured to perform feature matching on a plurality of first feature points and a plurality of second feature points, determine the coordinates of at least one first target feature point in the plurality of first feature points, and determine a plurality of second feature points coordinates of at least one second target feature point in the points, at least one first target feature point matches at least one second target feature point;
- the calculation module 708 is configured to calculate a first homography matrix configured as a pixel map according to the coordinates of at least one first target feature point and the coordinates of at least one second target feature point;
- the mapping module 710 is configured to map the pixel value of each pixel in the first region of interest to the corresponding pixel in the second region of interest based on the first homography matrix, so as to obtain the first sense image after image alignment area of interest.
- the acquisition module 702 is configured to acquire the histogram of the second image, and determine the pixel value of each pixel in the second image; Set the pixel value of the pixel point corresponding to the pixel position in the third image; according to the pixel value of each pixel point in the second image and the pixel value of each pixel point in the third image after setting, extract the first At least one first corner point in the three images, and extracting at least one second corner point in the second image; performing feature matching on at least one first corner point and at least one second corner point, and determining at least one first corner point The first target corner point in , and the second target corner point in at least one second corner point, the first target corner point matches the second target corner point; according to the coordinates of the first target corner point and the second target corner point The coordinates of the second homography matrix are calculated; based on the second homography matrix, the pixel value of each pixel point in the third image is mapped to the corresponding pixel point of the second image, and
- the acquisition module 702 is configured to perform feature matching on at least one first corner point and at least one second corner point in units of N corner points;
- the first corner point and at least one second corner point are used to perform feature matching again in units of M corner points; wherein, N is a positive integer greater than M.
- the acquisition module 702 is configured to, under the condition that the number of first target corner points is greater than or equal to the preset number, according to the coordinates of the first target corner point and the second target angle The coordinates of the points are used to calculate the second homography matrix; under the condition that the number of the first target corner points is less than the preset number, a plurality of first orientation fast rotation ORB feature points are extracted from the third image, and from the second image Extracting a plurality of second ORB feature points; performing feature matching with a plurality of first ORB feature points and a plurality of second ORB feature points, determining the coordinates of at least one first target ORB feature point in a plurality of first ORB feature points, and determining the coordinates of at least one second target ORB feature point among the plurality of second target ORB feature points, at least one first target ORB feature point matching with at least one second target ORB feature point; according to the at least one first target ORB feature point coordinates, and the coordinates of
- the extraction module 704 is configured to perform target object detection on the first region of interest and the second region of interest; determine the first target object graph from the first region of interest, and Determining a second object map from the second region of interest; extracting a plurality of first feature points from the first object map, and extracting a plurality of second feature points from the second object map.
- the first feature point is a first entropy correlation coefficient ECC feature point
- the second feature point is a second ECC feature point
- Extraction module 704 configured to identify whether the first target object graph and the second target object graph contain the target object; under the condition that the first target object graph and the second target object graph contain the target object, extract A plurality of first ECC feature points are extracted from the image, and a plurality of second ECC feature points are extracted from the second target object graph.
- the extraction module 704 is configured to: acquire the histogram of the second target object map, and determine the pixel value of each pixel in the second target object map; For the pixel value of each pixel in the object map, set the pixel value of the pixel corresponding to the pixel position in the first target object map.
- the target object detection is portrait detection
- the extraction module 704 is configured to obtain the first portrait mask of the first region of interest obtained after portrait detection, and the second A second portrait mask of the region; determining a first portrait map from the first image according to the first portrait mask, and determining a second portrait map from the second image according to the second portrait mask.
- the extracting module 704 is configured to extract the third ROI in the first image and the fourth ROI in the second image according to a default size; If the size of the ROI is smaller than the preset size, expand the third ROI according to the preset size to obtain the first ROI; if the size of the fourth ROI is smaller than the preset size, expand the third ROI according to the preset size Extend the fourth ROI to obtain the second ROI.
- the image alignment device provided by the present disclosure first performs global image alignment on the image to be processed, and then extracts feature points from the first region of interest in the image to be processed and the second region of interest in the reference image to perform feature matching to obtain the matched feature points
- the coordinates of the homography matrix are calculated to map the first region of interest to the second region of interest according to the homography matrix to achieve local alignment of the image to be processed by aligning all regions of interest in the image to be processed.
- the precise alignment of the images to be processed reduces the influence of uncertain factors in the image shooting process, and more feature points can be extracted for image alignment. It is suitable for various scenarios and improves the accuracy of image alignment.
- the image alignment device provided in this embodiment can execute the image alignment method provided in the above method embodiment, and its implementation principle and technical effect are similar, and will not be repeated here.
- Each module in the above-mentioned image alignment device can be fully or partially realized by software, hardware and a combination thereof.
- the above-mentioned modules can be embedded in or independent of one or more processors in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that one or more processors can call and execute the above-mentioned The operation corresponding to the module.
- the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, each The specific names of the functional units are only for the convenience of distinguishing each other, and are not configured to limit the protection scope of the present disclosure.
- a computer device is provided.
- the computer device may be a terminal device, and its internal structure may be as shown in FIG. 8 .
- the computer device includes one or more processors, memory, communication interfaces, databases, display screens, and input devices connected by a system bus. Wherein, one or more processors of the computer device are configured as modules providing computing and control capabilities.
- the memory of the computer equipment includes a non-volatile storage medium and an internal memory.
- the non-volatile storage medium stores an operating system and computer readable instructions.
- the internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium.
- the communication interface of the computer device is configured as a wired or wireless communication module with an external terminal, and the wireless mode can be realized through WIFI, operator network, near field communication (NFC) or other technologies.
- the computer-readable instructions are executed by one or more processors, the image alignment method provided by the above-mentioned embodiments can be realized.
- the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.
- FIG. 8 is only a block diagram of a partial structure related to the disclosed solution, and does not constitute a limitation to the computer equipment on which the disclosed solution is applied.
- the specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.
- the apparatus for displaying preview images provided by the present disclosure may be implemented in the form of computer-readable instructions, and the computer-readable instructions may be run on a computer device as shown in FIG. 8 .
- Various program modules that make up the computer device can be stored in the memory of the computer device, such as the acquisition module 702 , extraction module 704 , feature matching module 706 , calculation module 708 and mapping module 710 shown in FIG. 7 .
- Computer-readable instructions constituted by various program modules enable one or more processors to execute the steps in the image alignment method of any one embodiment of the present disclosure described in this specification.
- the computer device shown in FIG. 8 can execute the acquisition of the first image and the second image through the acquisition module 702 in the image alignment device as shown in FIG. first image.
- the computer device can perform extracting a plurality of first feature points from a first region of interest of a first image through an extraction module 704, and extracting a plurality of second feature points from a second region of interest of a second image; the first region of interest It is the same size as the second ROI and corresponds to its position in different images.
- the computer device may use the feature matching module 706 to perform feature matching on multiple first feature points and multiple second feature points, determine the coordinates of at least one first target feature point among the multiple first feature points, and determine multiple first feature points.
- the coordinates of at least one second target feature point among the two feature points, at least one first target feature point matches at least one second target feature point.
- the computer device can calculate the first homography matrix for pixel mapping according to the coordinates of at least one first target feature point and the coordinates of at least one second target feature point through the calculation module 708 .
- the computer device may use the mapping module 710 to perform mapping of the pixel value of each pixel in the first region of interest to the corresponding pixel in the second region of interest based on the first homography matrix, so as to obtain the first aligned image. a region of interest.
- a computer device comprising a memory and one or more processors, the memory storing computer readable instructions which, when executed by the one or more processors, cause One or more processors execute the steps of the image alignment method provided in any one embodiment of the present disclosure.
- the computer device provided in this embodiment can implement the image alignment method provided in the above method embodiment, and its implementation principle is similar to the technical effect, and will not be repeated here.
- One or more non-volatile storage media storing computer-readable instructions.
- the computer-readable instructions are executed by one or more processors, one or more processors execute the image provided in any one of the embodiments of the present disclosure. Align the steps of the method.
- the computer-readable storage medium may be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.
- ROM Read-Only Memory
- RAM Random Access Memory
- magnetic disk or an optical disk and the like.
- the computer-readable instructions stored on the computer-readable storage medium provided by this embodiment can realize the image alignment method provided by the above-mentioned method embodiment, and its implementation principle is similar to the technical effect, and will not be repeated here.
- Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc.
- Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory.
- RAM Random Access Memory
- SRAM Static Random Access Memory
- DRAM Dynamic Random Access Memory
- the image alignment method provided in this disclosure achieves precise alignment of the image to be processed by aligning all regions of interest in the image to be processed, reduces the influence of uncertain factors in the image shooting process, and can extract more feature points for image alignment , is applicable to various scenarios, improves the accuracy of image alignment, and has strong industrial applicability.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
本公开实施例提供了一种图像对齐方法、装置、计算机设备和存储介质,应用于图像处理技术领域,解决现有技术中由于特征点不明显从而造成图像对齐精度不高的问题,该方法包括:获取第一图像和第二图像,从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标;计算用于像素映射的第一单应性矩阵;基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
Description
本公开要求于2021年12月02日提交中国专利局、申请号为2021114598720、发明名称为“一种图像对齐方法、装置、电子设备和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本公开中。
本公开涉及一种图像对齐方法、装置、计算机设备和存储介质。
图像对齐在图像融合、图像拼接以及计算机视觉等方面发挥重要作用,它是将不同时间或不同角度的两幅或多幅图像进行对齐,以确定图像间空间位置、强度等之间的映射关系,从而将极短时间内连续拍摄的多帧图像融合成一帧图像。相关技术中基于特征的图像对齐方法首先找到两张图片中的特征点并进行特征匹配,然后计算单应性矩阵统一变换图像中的像素点坐标进行图像对齐,但实际操作过程中,由于图像拍摄过程存在光照、景深、角度等不确定因素,导致需要处理的图像难以提取明显的特征点,从而使得该方法的场景适应性差,图像对齐的精度低。
发明内容
(一)要解决的技术问题
解决相关技术中由于特征点不明显从而造成图像对齐精度不高的问题。
(二)技术方案
根据本公开公开的各种实施例,提供一种图像对齐方法、装置、计算机设备和存储介质。
一种图像对齐方法,所述方法包括:
获取第一图像和第二图像,第一图像为基于第二图像进行全局图像对齐处理后的第一图像;
从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;第一感兴趣区域与第二感兴趣区域的大小相同,且在不同图像中的位置对应;
将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标,至少一个第一目标特征点与至少一个第二目标特征点匹配;
根据至少一个第一目标特征点的坐标,以及至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵;
基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
作为本公开实施例一种可选的实施方式,获取第一图像,包括:获取第二图像的直方图,并确定第二图像中每个像素点的像素值;根据第二图像中每个像素点的像素值,设置 第三图像中像素位置对应的像素点的像素值;根据第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取第三图像中的至少一个第一角点,以及提取第二图像中的至少一个第二角点;将至少一个第一角点和至少一个第二角点进行特征匹配,确定至少一个第一角点中的第一目标角点,以及至少一个第二角点中的第二目标角点,第一目标角点与第二目标角点匹配根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;基于第二单应性矩阵,将第三图像内每个像素点的像素值映射到第二图像的对应像素点中,得到全局图像对齐后的第一图像。
作为本公开实施例一种可选的实施方式,至少一次特征匹配的次数为多次,将至少一个第一角点和至少一个第二角点进行特征匹配,包括:将至少一个第一角点和至少一个第二角点,以N个角点为单位,进行一次特征匹配;将至少一个第一角点和至少一个第二角点,以M个角点为单位,进行再一次特征匹配;其中,N为大于M的正整数。
作为本公开实施例一种可选的实施方式,根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵,包括:若第一目标角点的数量大于或等于预设数量,则根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;该方法还包括:若第一目标角点的数量小于预设数量,则从第三图像提取多个第一ORB特征点,以及从第二图像中提取多个第二ORB特征点;将多个第一ORB特征点与多个第二ORB特征点进行特征匹配,确定多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,至少一个第一目标ORB特征点与至少一个第二目标ORB特征点匹配;根据至少一个第一目标ORB特征点的坐标,以及至少一个第二目标ORB特征点的坐标,计算第二单应性矩阵。
作为本公开实施例一种可选的实施方式,从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点,包括:对第一感兴趣区域和第二感兴趣区域进行目标对象检测;从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图;从第一目标对象图中提取多个第一特征点,以及从第二目标对象中提取多个第二特征点。
作为本公开实施例一种可选的实施方式,从第一目标对象图中提取多个第一特征点,以及从第二目标对象中提取多个第二特征点,包括:识别第一目标对象图和第二目标对象图中是否包含目标对象;若第一目标对象图和第二目标对象图中包含目标对象,则从第一目标对象图中提取多个第一ECC特征点,以及从第二目标对象图中提取多个第二ECC特征点。
作为本公开实施例一种可选的实施方式,从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图之后,还包括:获取第二目标对象图的直方图,并确定第二目标对象图中每个像素点的像素值;根据第二目标对象图中每个像素点的像素值,设置第一目标对象图中像素位置对应的像素点的像素值。
作为本公开实施例一种可选的实施方式,目标对象检测为人像检测,从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图,包括:获取人像检测后得到的第一感兴趣区域的第一人像掩膜,以及第二感兴趣区域的第二人像掩膜;根据第一人像掩膜从第一图像中确定第一人像图,以及根据第二人像掩膜从第二图像中确定第二人像图。
作为本公开实施例一种可选的实施方式,所述方法还包括:根据默认尺寸提取第一图像中的第三感兴趣区域,以及第二图像中的第四感兴趣区域;在第三感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第三感兴趣区域,得到第一感兴趣区域;在第四感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第四感兴趣区域,得到第二感兴趣区域。
一种图像对齐装置,包括:
获取模块,配置成获取第一图像和第二图像,第一图像为基于第二图像进行全局图像对齐处理后的第一图像;
提取模块,配置成从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;第一感兴趣区域与第二感兴趣区域的大小相同,且在不同图像中的位置对应;
特征匹配模块,配置成将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标,至少一个第一目标特征点与至少一个第二目标特征点匹配;
计算模块,配置成根据至少一个第一目标特征点的坐标,以及至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵;
映射模块,配置成基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
作为本公开实施例一种可选的实施方式,获取模块,配置成获取第二图像的直方图,并确定第二图像中每个像素点的像素值;根据第二图像中每个像素点的像素值,设置第三图像中像素位置对应的像素点的像素值;根据第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取第三图像中的至少一个第一角点,以及提取第二图像中的至少一个第二角点;将至少一个第一角点和至少一个第二角点进行特征匹配,确定至少一个第一角点中的第一目标角点,以及至少一个第二角点中的第二目标角点,第一目标角点与第二目标角点匹配;根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;基于第二单应性矩阵,将第三图像内每个像素点的像素值映射到第二图像的对应像素点中,得到全局图像对齐后的第一图像。
作为本公开实施例一种可选的实施方式,获取模块,配置成将至少一个第一角点和至少一个第二角点,以N个角点为单位,进行一次特征匹配;将至少一个第一角点和至少一个第二角点,以M个角点为单位,进行再一次特征匹配;
其中,N为大于M的正整数。
作为本公开实施例一种可选的实施方式,获取模块,配置成在第一目标角点的数量大于或等于预设数量的条件下,根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;在第一目标角点的数量小于预设数量的条件下,从第三图像提取多个第一定向快速旋转ORB特征点,以及从第二图像中提取多个第二ORB特征点;将多个第一ORB特征点与多个第二ORB特征点进行特征匹配,确定多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,至少一个第一目标ORB特征点与至少一个第二目标ORB特征点匹配;根据至少一个第一目标ORB特征点的坐标,以及至少一个第二目标ORB特征点的坐标,计算第二 单应性矩阵。
作为本公开实施例一种可选的实施方式,提取模块,配置成对第一感兴趣区域和第二感兴趣区域进行目标对象检测;从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图;从第一目标对象图中提取多个第一特征点,以及从第二目标对象图中提取多个第二特征点。
作为本公开实施例一种可选的实施方式,第一特征点为第一熵相关系数ECC特征点,第二特征点为第二ECC特征点;
提取模块,配置成识别第一目标对象图和第二目标对象图中是否包含目标对象;在第一目标对象图和第二目标对象图中包含目标对象的条件下,从第一目标对象图中提取多个第一ECC特征点,以及从第二目标对象图中提取多个第二ECC特征点。
作为本公开实施例一种可选的实施方式,提取模块,配置成获取第二目标对象图的直方图,并确定第二目标对象图中每个像素点的像素值;根据第二目标对象图中每个像素点的像素值,设置第一目标对象图中像素位置对应的像素点的像素值。
作为本公开实施例一种可选的实施方式,目标对象检测为人像检测;提取模块,配置成获取人像检测后得到的第一感兴趣区域的第一人像掩膜,以及第二感兴趣区域的第二人像掩膜;根据第一人像掩膜从第一图像中确定第一人像图,以及根据第二人像掩膜从第二图像中确定第二人像图。
作为本公开实施例一种可选的实施方式,提取模块,配置成根据默认尺寸提取第一图像中的第三感兴趣区域,以及第二图像中的第四感兴趣区域;在第三感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第三感兴趣区域,得到第一感兴趣区域;在第四感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第四感兴趣区域,得到第二感兴趣区域。
一种计算机设备,包括存储器和一个或多个处理器,将所述存储器配置成存储计算机可读指令的模块;所述计算机可读指令被所述一个或多个处理器执行时,使得所述一个或多个处理器执行本公开任意一个实施例中提供图像对齐方法的步骤。
一个或多个存储有计算机可读指令的非易失性存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行本公开任意一个实施例中提供的图像对齐方法的步骤。
本公开的其他特征和优点将在随后的说明书中阐述,并且,部分地从说明书中变得显而易见,或者通过实施本公开而了解。本公开的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得,本公开的一个或多个实施例的细节在下面的附图和描述中提出。
为使本公开的上述目的、特征和优点能更明显易懂,下文特举可选实施例,并配合所附附图,作详细说明如下。
为了更清楚地说明本公开实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,对于本领域普通技术人员而言,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1为本公开一个或多个实施例提供的图像对齐方法的应用场景图一;
图2为本公开一个或多个实施例提供的图像对齐方法的步骤流程图一;
图3为本公开一个或多个实施例提供的图像对齐方法的应用场景图二;
图4为本公开一个或多个实施例提供的提取感兴趣区域的示意图;
图5为本公开一个或多个实施例提供的图像对齐方法的步骤流程图二;
图6为本公开一个或多个实施例提供的图像对齐方法的步骤流程图三;
图7为本公开一个或多个实施例中图像对齐装置的结构框图;
图8为本公开一个或多个实施例中计算机设备的内部结构图。
为了能够更清楚地理解本公开的上述目的、特征和优点,下面将对本公开的方案进行进一步描述。需要说明的是,在不冲突的情况下,本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开,但本公开还可以采用其他不同于在此描述的方式来实施;显然,说明书中的实施例只是本公开的一部分实施例,而不是全部的实施例。
本公开的说明书和权利要求书中的术语“第一”和“第二”等是用来区别不同的对象,而不是用来描述对象的特定顺序。例如,第一摄像头和第二摄像头是为了区别不同的摄像头,而不是为了描述摄像头的特定顺序。。
在本公开实施例中,“示例性的”或者“例如”等词来表示作例子、例证或说明。本公开实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言,使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念,此外,在本公开实施例的描述中,除非另有说明,“多个”的含义是指两个或两个以上。
在一些情形中,基于特征的图像对齐方法首先找到两张图片中的特征点并进行特征匹配,然后计算单应性矩阵统一变换图像中的像素点坐标进行图像对齐,但实际操作过程中,由于图像拍摄过程存在光照、景深、角度等不确定因素,导致需要处理的图像时间戳明显,景深比较明显,亮度差异大,视场差大,图像空域变化大从而难以提取明显的特征点,从而使得该方法的场景适应性差,图像对齐的精度低。
为解决上述问题,本公开首先对待处理图像进行全局图像对齐,然后从待处理图像中的第一感兴趣区域以及参考图像中的第二感兴趣区域提取特征点进行特征匹配,得到匹配的特征点的坐标,从而计算单应性矩阵,以根据该单应性矩阵将第一感兴趣区域映射到第二感兴趣区域实现待处理图像的局部对齐,通过对齐待处理图像中所有感兴趣区域来实现待处理图像的精准对齐,减少了图像拍摄过程中不确定因素的影响,能够提取出更多特征点来进行图像对齐,适用于多种场景,提高了图像对齐的精确度。
本公开提供的图像对齐方法,可以应用于如图1所示的应用环境中。该图像对齐方法应用于图像对齐***中。该图像对齐***包括终端102与服务器104。其中,终端102与服务器104通过网络进行通信。具体的,该图像对齐方法应用于终端102,终端102通过对待处理图像进行全局图像对齐,然后从待处理图像中的第一感兴趣区域以及参考图像中 的第二感兴趣区域提取特征点进行特征匹配,得到匹配的特征点的坐标,从而计算单应性矩阵,以根据该单应性矩阵将第一感兴趣区域映射到第二感兴趣区域实现待处理图像的局部对齐,通过对齐待处理图像中所有感兴趣区域来实现待处理图像的精准对齐。
另外,该图像对齐方法应用于服务器104,服务器104接收终端102发送的待处理图像106,通过上述图像对齐方法将图像进行精准对齐,然后服务器104将精准对齐后的图像108返回至终端102,该方法还包括步骤S202~S210:
其中,终端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备,服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
参照图2所示,图2为本公开一个或多个实施例提供的图像对齐方法的步骤流程图一。本公开主要以该方法应用于图1中的终端102来举例说明。
S202,获取第一图像和第二图像。
其中,第一图像是以第二图像作为参考图像对第三图像进行全局图像对齐处理后得到的图像;第三图像是从作为对比设备的终端设备中获取的图像,是需要进行全局图像对齐处理的图像;第二图像是从作为参***的终端设备中获取的图像;第二图像和第三图像为相同场景不同角度的图像,或者是相同场景不同时刻的图像。
在一些实施例中,在获取第一图像之前,从不同的终端设备中获取第二图像和第三图像。其中,第二图像为参考图像,第三图像为需要进行图像全局对齐处理的待处理图像。
示例性的,如图3所示,图中包括的设备有手机301、手机303服务器305;图像包括第二图像302、第三图像304、第一图像306。首先从作为参考机的手机301获取第二图像302,从作为对比机的手机304中获取第三图像304,由服务器305以第二图像302为参考图像对第三图像进行全局图像对齐处理得到第一图像306。
下述将对以第二图像作为参考图像,对第三图像进行全局图像处理得到第一图像的过程进行介绍。
在一些实施例中,从不同的终端设备中获取第二图像和第三图像之后,再获取第二图像的直方图,由于直方图表示图像的每个亮度级别的像素数量,展示了像素在图像中的分布情况,直观的体现图片的明暗程度,所以可以根据第二图像的直方图确定第二图像中每个像素点的像素值;然后根据第二图像中每个像素点的像素值,采用直方图均衡法、直方图规定化等方式设置第三图像中像素位置对应的像素点的像素值,以根据第二图像调整第三图像的亮度,使得第三图像中像素位置对应的像素点的像素值设置为与第二图像中相应像素位置对应的像素点的像素值相同,从而使得第三图像的亮度与第一图像的亮度相同,以便于从两张图片中提取特征点。
在一些实施例中,可通过平滑去噪、光照均衡化处理等方法调整第三图像的亮度,包括但不限于上述方法,本公开对此不做限定。
进一步,根据第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取亮度调整后的第三图像中的至少一个角点,并提取第二图像中至少一个角点,将提取到的两张图像的角点进行匹配。
其中,角点是在某些属性上强度最大或者最小的孤立点、线段的终点,或者是曲线上局部曲率最大的点,用于描述图像中的拐角,边界点等信息。本公开中通过哈里斯(Harris) 角点检测法提取角点,需要说明的是,Harris角点检测法基本思想是使用一个固定窗口在图像上进行任意方向上的滑动,比较滑动前与滑动后两种情况下,窗口中的像素灰度变化程度,如果存在任意方向上的滑动,都有着较大灰度变化,则认为该窗口中存在角点。通过Harris角点检测法提取角点,提高了对噪声的鲁棒性,提升了角点检测的效率和稳定性。
在一些实施例中,在角点匹配的过程中,将提取到的第三图像中的角点和第二图像中的角点进行至少一次特征匹配,先在尺度较小图像较模糊的情况下,对N个角点为单位进行特征匹配,得到角点的粗匹配结果,然后增大尺度对M个角点为单位进行特征匹配,直到尺度变换至原图大小,完成图像的精确匹配,其中,N为大于M的正整数。
示例性的,在角点匹配的过程中先以10个角点为单位进行粗匹配,然后增大尺度,以4个角点为单位进行精确匹配。
通过至少一次的特征匹配,实现对第三图像和第二图像的粗匹配和精确匹配,降低了误匹配率,提高了匹配的精确度。
在一些实施例中,在精确匹配得到匹配的角点后,可选的,若第一目标角点的数量大于或等于预设数量,则根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵。其中,第二单应性矩阵是用于对第三图像进行全局图像处理的单应性矩阵。
若第一目标角点的数量小于预设数量,则利用FAST(features from accelerated segment test)算法以及特征点邻域算法从第三图像提取多个第一定向快速旋转(Oriented Fast and Rotated Brief,ORB)特征点,以及从第二图像中提取多个第二ORB特征点,然后将多个第一ORB特征点与多个第二ORB特征点进行特征匹配,确定多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,至少一个第一目标ORB特征点与至少一个第二目标ORB特征点匹配,然后,根据至少一个第一目标ORB特征点的坐标,以及至少一个第二目标ORB特征点的坐标,计算第二单应性矩阵。
通过ORB特征匹配,提取ORB特征点进行特征匹配,确定匹配的ORB特征点,得到ORB特征点对应的坐标,从而计算单应性矩阵,提高了特征匹配的准确度和效率。
可选的,本公开使用随机抽样一致算法计算单应性矩阵并进行迭代更新,剔除误匹配点,确定最优单应性矩阵。
通过对第三图像进行亮度调整,解决待处理图像和参考图像存在亮度差异的问题,提高了特征匹配过程的便捷性;将第三图像与第二图像进行多次的特征点匹配,降低了误匹配率,提高了匹配的精确度;通过随机一致算法迭代计算最优单应性矩阵,从而得到全局图像对齐后的第一图像,实现了更好的图像全局对齐的效果。
S204,从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;
其中,第一感兴趣区域与第二感兴趣区域的大小相同,且在不同图像中的位置对应。第一感兴趣区域根据默认尺寸从第一图像中提取,默认尺寸根据用户的经验值设置,例如默认尺寸为A×A,单位为像素或厘米,相应的,第二感兴趣区域的提取原理跟第一感兴趣区域的提取原理相同,本公开在此不做赘述。
如图4所示,图4为从图像中提取感兴趣区域的示意图,在设置一个尺寸为A×A的窗口402,从图像401左上角开始提取,得到尺寸为A×A的感兴趣区域403。
需要说明的是,本公开中根据默认尺寸或预设尺寸遍历图像,可以得到图像的多个感兴趣区域。多个感兴趣区域拼接组合即为提取感兴趣区域前的原始图像。
在一些实施例中,根据默认尺寸提取全局图像对齐处理后的第一图像中的第三感兴趣区域,以及第二图像中的相应尺寸的第四感兴趣区域。由于根据默认尺寸选取感兴趣区域可能存在该感兴趣区域较小、特征点较少的情况,用户需要设置预设尺寸,本公开实施例中提供一种实施方式,首先判断第三感兴趣区域和第四感兴趣区域是否小于预设尺寸,若第三感兴趣区域和第四感兴趣区域小于预设尺寸,则扩展第三感兴趣区域得到第一感兴趣区域,扩展第四感兴趣区域得到第二感兴趣区域,可选的,从感兴趣区域的边界进行扩大,以得到更大尺寸的感兴趣区域便于提取特征点。
在一些实施例中,由于图像中的景物与背景存在明显的景深差异,所以将图像中的景物作为目标对象进行检测,首先对第一感兴趣区域和第二感兴趣区域进行目标对象检测,从第一感兴趣区域中确定第一目标对象图,以及从第二感兴趣区域中确定第二目标对象图;然后识别第一感兴趣区域和第二感兴趣区域是否包含目标对象;
在检测到第一感兴趣区域和第二感兴趣区域中存在目标对象的情况下,对第一目标对象图和第二目标对象图进行特征提取,从第一目标对象图中提取多个第一特征点,以及从第二目标对象图中提取多个第二特征点。第一特征点为第一角点,第二特征点为第二角点,或者,第一特征点为第一ECC特征点,第二特征点为第二ECC特征点。
在一些实施例中,图像中的景物可以是植物、动物、建筑等其他图像处理领域比较关注的对象,下述将在图像中的景物是人的情况下对局部对齐过程进行介绍。
在一些实施例中,对第一感兴趣区域和第二感兴趣区域进行人像检测;人像检测后得到第一感兴趣区域的第一人像掩膜以及第二感兴趣区域的人像掩膜;根据第一人像掩膜和第二人像掩膜,从第一感兴趣区域确定第一人像图,以及从第二感兴趣区域确定第二人像图,然后从第一人像图中提取多个第一特征点,以及从第二人像中提取多个第二特征点。将人像作为图像中的重点信息,通过深度学习算法对从图像中提取的感兴趣区域进行人像检测,抠出人像图,以根据人像图中的人像特征点实现图像对齐。
在一些实施例中,在从第一感兴趣区域确定第一人像图,以及从第二感兴趣区域确定第二人像图之后,需要进一步验证人像图中是否包含人像,本公开实施例中提供一种实施方式:识别第一人像图和第二人像图中是否包含人像;若第一人像图和第二人像图中包含人像,则从第一人像图中提取多个第一特征点,以及从第二人像中提取多个第二特征点,由于验证得到第一人像图像和第二人像图中包含人像,且人像中的特征点明显、数量多,为提高特征匹配的效率,可选的,第一特征点为第一角点,第二特征点为第二角点。以特征点为角点,保证在特征点数量多且明显的情况下,提高了特征匹配的速度,提升了特征匹配的可靠度。
在一些实施例中,若第一人像图和第二人像图中包含人像,在对提取到的角点进行特征匹配之后,得到的匹配的角点数量小于预设数量,则根据第一人像图计算第一熵相关系数(Entropy Corrleation Coefficient,ECC),然后从第一人像图中提取多个第一ECC特征点,以及从第二人像图中提取多个第二ECC特征点。在识别确定人像图中包含人像的情况下,为避免特征匹配后用于计算单应性矩阵的角点数量过少,需要根据ECC特征匹配法在保证对比度和亮度的光度失真不变的情况下,提取ECC特征点,以在图像中包含人像的基础上, 更好的进行特征匹配,以计算精确度更高的单应性矩阵,从而提升了图像对齐的准确度,实现了更好的图像对齐效果。
在一些实施例中,在从第一感兴趣区域确定第一人像图,以及从第二感兴趣区域确定第二人像图之后,可选的,获取第二人像图的直方图,并确定第二人像图中每个像素点的像素值;根据第二人像图中每个像素点的像素值,设置第一人像图中像素位置对应的像素点的像素值。在确定图像中的人像图之后,根据参考图像中提取的人像图的直方图调整待处理图像中人像图的亮度,以减少亮度差异对特征点提取的影响,提高特征匹配的准确度,从而提升图像对齐的精确度。
在检测到第一感兴趣区域和第二感兴趣区域中不存在目标对象的情况下,则首先获取第二感兴趣区域的直方图,确定第二感兴趣区域中每个像素点的像素值,从而据此设置第一感兴趣区域中每个像素点的像素值,以将第二感兴趣区域作为参考调整第一感兴趣区域的亮度,进一步的从第一感兴趣区域提取至少一个角点,以及从第二感兴趣区域提取至少一个角点,将提取到的第一感兴趣区域和第二感兴趣区域的角点进行匹配,然后计算第一单应性矩阵,以将该第一单应性矩阵用于图像对齐处理。
在一些实施例中,在上述提取到的第一感兴趣区域和第二感兴趣区域的角点小于预设数量的情况下,由于第一感兴趣区域和第二感兴趣区域不存在目标对象角点数量少,为了避免匹配的角点小于预设数量时计算得到第一单应性矩阵误差过大,不利于图像对齐,本公开实施例提供一种实施方式,首先从调整亮度后的第一感兴趣区域提取ECC特征点,以及从第二感兴趣区域提取ECC特征点,ECC特征点是是特征匹配精确度高的特征点,以使得第一感兴趣区域和第二ECC特征点没有明显的目标对象情况下,进行精确的特征匹配,实现了在图像中不存在明显的景物时,通过提取ECC特征点进行特征匹配,提高了图像对齐的准确度,得到更好的图像对齐效果。
上述实施例,通过在第一图像中进行目标对象检测获取目标对象图从而提取特征点,其中,在目标对象图中包含目标对象的情况下,以角点作为特征点进行提取,为保证特征匹配的准确度,以ECC特征点作为特征点进行提取;在目标对象图中不包含目标对象的情况下,根据作为参考的感兴趣区域进行亮度调整,提取感兴趣区域的角点,以用于后续至少一次的特征匹配。针对图像中是否包含目标对象的情况分别进行处理,保证提取到的特征点的准确度,以用于后续的特征点匹配,从而提升了图像对齐的精确性。
S206,将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标。
其中,至少一个第一目标特征点与至少一个第二目标特征点匹配。
在一些实施例中,本公开根据上述检测到目标对象后提取到的角点进行特征匹配。
在一些实施例中,确定的目标特征点的数量为达到计算单应性矩阵所需的3对以上。
在一些实施例中,在角点的数量未达到计算单应性矩阵所需的3对,则本公开根据上述步骤提取到ECC特征点进行特征匹配,特征匹配的方法包括但不限于加速稳健特征(Speeded Up Robust Feature,SURF)、尺度不变特征转换(Scale Invariant Feature Transform,SIFT)、加速分割测试获得特征(Features from Accelerated Segment Test,FAST),本公开对此不做限定。
在一些实施例中,本公开根据上述未检测到目标对象后提取到的角点进行特征匹配,进行与全局图像对齐处理过程中相同的至少一次特征匹配,先在尺度较小图像较模糊的情况下,对N个角点为单位进行特征匹配,得到角点的粗匹配结果,然后增大尺度对M个角点为单位进行特征匹配,直到尺度变换至原图大小,完成图像的精确匹配,其中,N为大于M的正整数。
S208,根据至少一个第一目标特征点的坐标,以及至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵。
在一些实施例中,在识别第一目标对象图中包含目标对象以及第二目标对象图中包含目标对象之后,确定至少一个目标特征点的坐标以及至少一个第二目标特征点的坐标,其中,该至少一个第一目标特征点为至少一个第一ECC目标特征点,至少一个第二目标特征点为至少一个第二ECC目标特征点通过ECC特征匹配法确定。
在一些实施例中,在识别第一目标对象图中包含目标对象以及第二目标对象图中包含目标对象之后,根据感兴趣区域Harris角点检测并进行特征匹配所确定的目标角点坐标,使用随机抽样一致算法计算单应性矩阵并进行迭代更新,剔除误匹配点,确定最优单应性矩阵。
S210,基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
在一些实施例中,剪切图像对齐后的第一感兴趣区域并输出。若第一感兴趣区域是扩展的感兴趣区域后,则根据扩展后的感兴趣区域的尺寸进行剪切,并输出扩展后的感兴趣区域。
在一些实施例中,从第一图像和第二图像中,进行多次提取感兴趣区域,再执行S204~S210的步骤,直至将第一图像中每个感兴趣区域都进行图像对齐,然后输出第一图像所有图像对齐处理后的感兴趣区域,以得到完整的图像对齐后的第一图像。通过将第一图像划分为多个感兴趣区域,对这些感兴趣区域均进行上述图像对齐操作,来实现精确图像对齐,提高了图像对齐的精准度。
综上,本公开首先对待处理图像进行全局图像对齐,然后从待处理图像中的第一感兴趣区域以及参考图像中的第二感兴趣区域提取特征点进行特征匹配,得到匹配的特征点的坐标,从而计算单应性矩阵,以根据该单应性矩阵将第一感兴趣区域映射到第二感兴趣区域实现待处理图像的局部对齐,通过对齐待处理图像中所有感兴趣区域来实现待处理图像的精准对齐,减少了图像拍摄过程中不确定因素的影响,能够提取出更多特征点来进行图像对齐,适用于多种场景,提高了图像对齐的精确度。
图5为本公开一个或多个实施例提供的图像对齐方法的步骤流程图二,图5是在图2所示的实施例的基础上,对本公开的进一步扩展与优化,其中,S202的一种可能的实现方式如下:
S502,获取第二图像的直方图,并确定第二图像中每个像素点的像素值。
S504,根据第二图像中每个像素点的像素值,设置第三图像中像素位置对应的像素点的像素值。
S506,根据第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取第三图像中的至少一个第一角点,以及提取第二图像中的至少一个第二角点。
S508,将至少一个第一角点和至少一个第二角点,以N个角点为单位,进行一次特征匹配。
S510,将至少一个第一角点和至少一个第二角点,以M个角点为单位,进行再一次特征匹配。
其中,N为大于M的正整数。
S512,确定至少一个第一角点中的第一目标角点,以及至少一个第二角点中的第二目标角点。
S514,根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;
S516,基于第二单应性矩阵,将第三图像内每个像素点的像素值映射到第二图像的对应像素点中,得到全局图像对齐后的第一图像。
通过对第三图像进行亮度调整得到第一图像,解决待处理图像和参考图像存在亮度差异的问题,提高了特征匹配过程的便捷性;通过随机一致算法迭代计算最优单应性矩阵,从而得到全局图像对齐后的第一图像,实现了更好的图像全局对齐的效果;将第三图像与第二图像进行多次的特征点匹配,降低了误匹配率,提高了匹配的精确度。
图6为本公开一个或多个实施例提供的图像对齐方法的步骤流程图三,图6是在图2所示的实施例的基础上,对本公开的进一步扩展与优化,其中,S205的一种可能的实现方式如下:
S602,对第一感兴趣区域和第二感兴趣区域进行目标对象检测。
S604,从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图。
S606,获取第二目标对象图的直方图,并确定第二目标对象图中每个像素点的像素值。
S608,根据第二目标对象图中每个像素点的像素值,设置第一目标对象图中像素位置对应的像素点的像素值。
S610,识别第一目标对象图和第二目标对象图中是否包含目标对象。
S612,若第一目标对象图和第二目标对象图中包含目标对象,则从第一目标对象图中提取多个第一ECC特征点,以及从第二目标对象图中提取多个第二ECC特征点。
通过对图像的感兴趣区域进行目标对象检测,得到目标对象图,并调整待处理图像对应的目标对象图的亮度,确定感兴趣区域中是否包含目标对象这一重要信息,然后提取目标对象相关的ECC特征点,从而提取出用于图像对齐的明显的特征点,减少了图像拍摄过程中其他干扰因素的影响,提高了图像对齐的精度。
应该理解的是,虽然图2-4的流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,图2-4中的至少一部分步骤可以包括多个子步骤或者多个阶段,这些子步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些子步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
基于同一发明构思,作为对上述方法的实现,本公开实施例还提供了一种图像对齐装置,该装置实施例与前述方法实施例对应,为便于阅读,本装置实施例不再对前述方法实 施例中的细节内容进行逐一赘述,但应当明确,本实施例中的装置能够对应实现前述方法实施例中的全部内容。
图7为本公开一个或多个实施例中图像对齐装置的结构框图,如图7所示,本实施例提供的图像对齐装置700包括:获取模块702、提取模块704、特征匹配模块706、计算模块708和映射模块710,其中:
获取模块702,配置成获取第一图像和第二图像,第一图像为基于第二图像进行全局图像对齐处理后的第一图像;
提取模块704,配置成从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;第一感兴趣区域与第二感兴趣区域的大小相同,且在不同图像中的位置对应;
特征匹配模块706,配置成将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标,至少一个第一目标特征点与至少一个第二目标特征点匹配;
计算模块708,配置成根据至少一个第一目标特征点的坐标,以及至少一个第二目标特征点的坐标,计算配置成像素映射的第一单应性矩阵;
映射模块710,配置成基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
作为本公开实施例一种可选的实施方式,获取模块702,配置成获取第二图像的直方图,并确定第二图像中每个像素点的像素值;根据第二图像中每个像素点的像素值,设置第三图像中像素位置对应的像素点的像素值;根据第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取第三图像中的至少一个第一角点,以及提取第二图像中的至少一个第二角点;将至少一个第一角点和至少一个第二角点进行特征匹配,确定至少一个第一角点中的第一目标角点,以及至少一个第二角点中的第二目标角点,第一目标角点与第二目标角点匹配;根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;基于第二单应性矩阵,将第三图像内每个像素点的像素值映射到第二图像的对应像素点中,得到全局图像对齐后的第一图像。
作为本公开实施例一种可选的实施方式,获取模块702,配置成将至少一个第一角点和至少一个第二角点,以N个角点为单位,进行一次特征匹配;将至少一个第一角点和至少一个第二角点,以M个角点为单位,进行再一次特征匹配;其中,N为大于M的正整数。
作为本公开实施例一种可选的实施方式,获取模块702,配置成在第一目标角点的数量大于或等于预设数量的条件下,根据第一目标角点的坐标与第二目标角点的坐标,计算第二单应性矩阵;在第一目标角点的数量小于预设数量的条件下,从第三图像提取多个第一定向快速旋转ORB特征点,以及从第二图像中提取多个第二ORB特征点;将多个第一ORB特征点与多个第二ORB特征点进行特征匹配,确定多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,至少一个第一目标ORB特征点与至少一个第二目标ORB特征点匹配;根据至少一个第一目标ORB特征点的坐标,以及至少一个第二目标ORB特征点的坐标,计算第二单应性矩阵。
作为本公开实施例一种可选的实施方式,提取模块704,配置成对第一感兴趣区域和第二感兴趣区域进行目标对象检测;从第一感兴趣区域确定第一目标对象图,以及从第二感兴趣区域确定第二目标对象图;从第一目标对象图中提取多个第一特征点,以及从第二目标对象图中提取多个第二特征点。
作为本公开实施例一种可选的实施方式,第一特征点为第一熵相关系数ECC特征点,第二特征点为第二ECC特征点;
提取模块704,配置成识别第一目标对象图和第二目标对象图中是否包含目标对象;在第一目标对象图和第二目标对象图中包含目标对象的条件下,从第一目标对象图中提取多个第一ECC特征点,以及从第二目标对象图中提取多个第二ECC特征点。
作为本公开实施例一种可选的实施方式,提取模块704,配置成:获取第二目标对象图的直方图,并确定第二目标对象图中每个像素点的像素值;根据第二目标对象图中每个像素点的像素值,设置第一目标对象图中像素位置对应的像素点的像素值。
作为本公开实施例一种可选的实施方式,目标对象检测为人像检测;提取模块704,配置成获取人像检测后得到的第一感兴趣区域的第一人像掩膜,以及第二感兴趣区域的第二人像掩膜;根据第一人像掩膜从第一图像中确定第一人像图,以及根据第二人像掩膜从第二图像中确定第二人像图。
作为本公开实施例一种可选的实施方式,提取模块704,配置成根据默认尺寸提取第一图像中的第三感兴趣区域,以及第二图像中的第四感兴趣区域;在第三感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第三感兴趣区域,得到第一感兴趣区域;在第四感兴趣区域的尺寸小于预设尺寸的情况下,根据预设尺寸扩展第四感兴趣区域,得到第二感兴趣区域。
本公开提供的图像对齐装置首先对待处理图像进行全局图像对齐,然后从待处理图像中的第一感兴趣区域以及参考图像中的第二感兴趣区域提取特征点进行特征匹配,得到匹配的特征点的坐标,从而计算单应性矩阵,以根据该单应性矩阵将第一感兴趣区域映射到第二感兴趣区域实现待处理图像的局部对齐,通过对齐待处理图像中所有感兴趣区域来实现待处理图像的精准对齐,减少了图像拍摄过程中不确定因素的影响,能够提取出更多特征点来进行图像对齐,适配置成多种场景,提高了图像对齐的精确度。
本实施例提供的图像对齐装置可以执行上述方法实施例提供的图像对齐方法,其实现原理与技术效果类似,此处不再赘述。上述图像对齐装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的一个或多个处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于一个或多个处理器调用执行上述各个模块对应的操作。
值得注意的是,上述图像对齐装置的实施例中,所包括的各个单元和模块只是按照功能逻辑进行划分的,但并不局限于上述的划分,只要能够实现相应的功能即可;另外,各功能单元的具体名称也只是为了便于相互区分,并不配置成限制本公开的保护范围。
在一个或多个实施例中,提供了一种计算机设备,该计算机设备可以是终端设备,其内部结构图可以如图8所示。该计算机设备包括通过***总线连接的一个或多个处理器、存储器、通信接口、数据库、显示屏和输入装置。其中,该计算机设备的一个或多个处理器配置成提供计算和控制能力的模块。该计算机设备的存储器包括非易失性存储介质、内 存储器。该非易失性存储介质存储有操作***和计算机可读指令。该内存储器为非易失性存储介质中的操作***和计算机可读指令的运行提供环境。该计算机设备的通信接口配置成与外部的终端进行有线或无线方式的通信模块,无线方式可通过WIFI、运营商网络、近场通信(NFC)或其他技术实现。该计算机可读指令被一个或多个处理器执行时以实现上述实施例提供的图像对齐方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏,该计算机设备的输入装置可以是显示屏上覆盖的触摸层,也可以是计算机设备外壳上设置的按键、轨迹球或触控板,还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解,图8中示出的结构,仅仅是与本公开方案相关的部分结构的框图,并不构成对本公开方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。
在一个或多个实施例中,本公开提供的预览图像的显示装置可以实现为一种计算机可读指令的形式,计算机可读指令可在如图8所示的计算机设备上运行。计算机设备的存储器中可存储组成该计算机设备计算机设备的各个程序模块,比如,图7所示的获取模块702、提取模块704、特征匹配模块706、计算模块708和映射模块710。各个程序模块构成的计算机可读指令使得一个或多个处理器执行本说明书中描述的本公开任意一个实施例的图像对齐方法中的步骤。
例如,图8所示的计算机设备可以通过如图7所示的图像对齐装置中的获取模块702执行获取第一图像和第二图像,第一图像为基于第二图像进行全局图像对齐处理后的第一图像。计算机设备可通过提取模块704执行从第一图像的第一感兴趣区域提取多个第一特征点,以及从第二图像的第二感兴趣区域提取多个第二特征点;第一感兴趣区域与第二感兴趣区域的大小相同,且在不同图像中的位置对应。计算机设备可通过特征匹配模块706执行将多个第一特征点与多个第二特征点进行特征匹配,确定多个第一特征点中至少一个第一目标特征点的坐标,以及确定多个第二特征点中至少一个第二目标特征点的坐标,至少一个第一目标特征点与至少一个第二目标特征点匹配。计算机设备可通过计算模块708根据至少一个第一目标特征点的坐标,以及至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵。计算机设备可通过映射模块710执行基于第一单应性矩阵将第一感兴趣区域内的每个像素点的像素值映射到第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
在一个或多个实施例中,提供了一种计算机设备,包括存储器和一个或多个处理器,该存储器存储有计算机可读指令,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行本公开任意一个实施例中提供的图像对齐方法的步骤。
本实施例提供的计算机设备,可以实现上述方法实施例提供的图像对齐方法,其实现原理与技术效果类似,此处不再赘述。
一个或多个存储有计算机可读指令的非易失性存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行本公开任意一个实施例中提供的图像对齐方法的步骤。
其中,该计算机可读存储介质可以为只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等。
本实施例提供的计算机可读存储介质上存储的计算机可读指令,可以实现上述方法实 施例提供的图像对齐方法,其实现原理与技术效果类似,此处不再赘述。
本领域普通技术人员可以理解实现上述方法实施例中的全部或部分流程,是可以通过计算机可读指令来指令相关的硬件来完成的,计算机可读指令可存储于一非易失性计算机可读取存储介质中,该计算机可读指令在执行时,可包括如上述各方法的实施例的流程。其中,本公开所提供的各实施例中所使用的对存储器、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或者外部高速缓冲存储器。作为说明而非局限,RAM以多种形式可得,比如静态随机存取存储器(Static Random Access Memory,SRAM)和动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上实施例仅表达了本公开的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对发明专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本公开构思的前提下,还可以做出若干变形和改进,这些都属于本公开的保护范围。因此,本公开专利的保护范围应以所附权利要求为准。
本公开提供的图像对齐方法,通过对齐待处理图像中所有感兴趣区域来实现待处理图像的精准对齐,减少了图像拍摄过程中不确定因素的影响,能够提取出更多特征点来进行图像对齐,适用于多种场景,提高了图像对齐的精确度,具有很强的工业实用性。
Claims (20)
- 一种图像对齐方法,其特征在于,所述方法包括:获取第一图像和第二图像,所述第一图像为基于所述第二图像进行全局图像对齐处理后的第一图像;从所述第一图像的第一感兴趣区域提取多个第一特征点,以及从所述第二图像的第二感兴趣区域提取多个第二特征点;所述第一感兴趣区域与所述第二感兴趣区域的大小相同,且在不同图像中的位置对应;将所述多个第一特征点与所述多个第二特征点进行特征匹配,确定所述多个第一特征点中至少一个第一目标特征点的坐标,以及确定所述多个第二特征点中至少一个第二目标特征点的坐标,所述至少一个第一目标特征点与所述至少一个第二目标特征点匹配;根据所述至少一个第一目标特征点的坐标,以及所述至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵;基于所述第一单应性矩阵将所述第一感兴趣区域内的每个像素点的像素值映射到所述第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
- 根据权利要求1所述的方法,其中,所述获取第一图像包括:获取第二图像的直方图,并确定所述第二图像中每个像素点的像素值;根据所述第二图像中每个像素点的像素值,设置第三图像中像素位置对应的像素点的像素值;根据所述第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取所述第三图像中的至少一个第一角点,以及提取所述第二图像中的至少一个第二角点;将所述至少一个第一角点和所述至少一个第二角点进行特征匹配,确定所述至少一个第一角点中的第一目标角点,以及所述至少一个第二角点中的第二目标角点,所述第一目标角点与所述第二目标角点匹配;根据所述第一目标角点的坐标与所述第二目标角点的坐标,计算第二单应性矩阵;基于所述第二单应性矩阵,将所述第三图像内每个像素点的像素值映射到所述第二图像的对应像素点中,得到全局图像对齐后的第一图像。
- 根据权利要求2所述的方法,其中,所述至少一次特征匹配的次数为多次,所述将所述至少一个第一角点和所述至少一个第二角点进行特征匹配,包括:将所述至少一个第一角点和所述至少一个第二角点,以N个角点为单位,进行一次特征匹配;将所述至少一个第一角点和所述至少一个第二角点,以M个角点为单位,进行再一次特征匹配;其中,N为大于M的正整数。
- 根据权利要求2所述的方法,其中,所述根据所述第一目标角点的坐标与所述第二目标角点的坐标,计算第二单应性矩阵,包括:在所述第一目标角点的数量大于或等于预设数量的条件下,根据所述第一目标角点的坐标与所述第二目标角点的坐标,计算所述第二单应性矩阵;所述方法还包括:在所述第一目标角点的数量小于预设数量的条件下,从所述第三图像提取多个第一定向快速旋转ORB特征点,以及从所述第二图像中提取多个第二ORB特征点;将所述多个第一ORB特征点与所述多个第二ORB特征点进行特征匹配,确定所述多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定所述多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,所述至少一个第一目标ORB特征点与所述至少一个第二目标ORB特征点匹配;根据所述至少一个第一目标ORB特征点的坐标,以及所述至少一个第二目标ORB特征点的坐标,计算所述第二单应性矩阵。
- 根据权利要求1所述的方法,其中,所述从所述第一图像的第一感兴趣区域提取多个第一特征点,以及从所述第二图像的第二感兴趣区域提取多个第二特征点,包括:对所述第一感兴趣区域和所述第二感兴趣区域进行目标对象检测;从所述第一感兴趣区域确定第一目标对象图,以及从所述第二感兴趣区域确定第二目标对象图;从所述第一目标对象图中提取多个第一特征点,以及从所述第二目标对象图中提取多个第二特征点。
- 根据权利要求5所述的方法,其中,所述第一特征点为第一熵相关系数ECC特征点,所述第二特征点为第二ECC特征点,所述从所述第一目标对象图中提取多个第一特征点,以及从所述第二目标对象图中提取多个第二特征点,包括:识别所述第一目标对象图和所述第二目标对象图中是否包含目标对象;在所述第一目标对象图和所述第二目标对象图中包含目标对象的条件下,从所述第一目标对象图中提取多个所述第一ECC特征点,以及从所述第二目标对象图中提取多个所述第二ECC特征点。
- 根据权利要求5所述的方法,其中,所述从所述第一感兴趣区域确定第一目标对象图,以及从所述第二感兴趣区域确定第二目标对象图之后,还包括:获取所述第二目标对象图的直方图,并确定所述第二目标对象图中每个像素点的像素值;根据所述第二目标对象图中每个像素点的像素值,设置所述第一目标对象图中像素位置对应的像素点的像素值。
- 根据权利要求5所述的方法,其中,所述目标对象检测为人像检测,所述从所述第一感兴趣区域确定第一目标对象图,以及从所述第二感兴趣区域确定第二目标对象图,包括:获取人像检测后得到的所述第一感兴趣区域的第一人像掩膜,以及所述第二感兴趣区域的第二人像掩膜;根据所述第一人像掩膜从所述第一图像中确定第一人像图,以及根据所述第二人像掩膜从所述第二图像中确定第二人像图。
- 根据权利要求1所述的方法,其中,所述方法还包括:根据默认尺寸提取所述第一图像中的第三感兴趣区域,以及第二图像中的第四感兴趣区域;在所述第三感兴趣区域的尺寸小于预设尺寸的情况下,根据所述预设尺寸扩展所述第三感兴趣区域,得到所述第一感兴趣区域;在所述第四感兴趣区域的尺寸小于所述预设尺寸的情况下,根据所述预设尺寸扩展所述第四感兴趣区域,得到所述第二感兴趣区域。
- 一种图像对齐装置,其特征在于,包括:获取模块,配置成获取第一图像和第二图像,所述第一图像为基于所述第二图像进行全局图像对齐处理后的第一图像;提取模块,配置成从所述第一图像的第一感兴趣区域提取多个第一特征点,以及从所述第二图像的第二感兴趣区域提取多个第二特征点;所述第一感兴趣区域与所述第二感兴趣区域的大小相同,且在不同图像中的位置对应;特征匹配模块,配置成将所述多个第一特征点与所述多个第二特征点进行特征匹配,确定所述多个第一特征点中至少一个第一目标特征点的坐标,以及确定所述多个第二特征点中至少一个第二目标特征点的坐标,所述至少一个第一目标特征点与所述至少一个第二目标特征点匹配;计算模块,配置成根据所述至少一个第一目标特征点的坐标,以及所述至少一个第二目标特征点的坐标,计算用于像素映射的第一单应性矩阵;映射模块,配置成基于所述第一单应性矩阵将所述第一感兴趣区域内的每个像素点的像素值映射到所述第二感兴趣区域的对应像素点中,以得到图像对齐后的第一感兴趣区域。
- 根据权利要求10所述的装置,其中,所述获取模块,配置成:获取第二图像的直方图,并确定所述第二图像中每个像素点的像素值;根据所述第二图像中每个像素点的像素值,设置第三图像中像素位置对应的像素点的像素值;根据所述第二图像中每个像素点的像素值,以及设置后的第三图像中每个像素点的像素值,提取所述第三图像中的至少一个第一角点,以及提取所述第二图像中的至少一个第二角点;将所述至少一个第一角点和所述至少一个第二角点进行特征匹配,确定所述至少一个第一角点中的第一目标角点,以及所述至少一个第二角点中的第二目标角点,所述第一目标角点与所述第二目标角点匹配;根据所述第一目标角点的坐标与所述第二目标角点的坐标,计算第二单应性矩阵;基于所述第二单应性矩阵,将所述第三图像内每个像素点的像素值映射到所述第二图像的对应像素点中,得到全局图像对齐后的第一图像。
- 根据权利要求11所述的装置,其中,所述获取模块,配置成:将所述至少一个第一角点和所述至少一个第二角点,以N个角点为单位,进行一次特征匹配;将所述至少一个第一角点和所述至少一个第二角点,以M个角点为单位,进行再一次特征匹配;其中,N为大于M的正整数。
- 根据权利要求11所述的装置,其中,所述获取模块,配置成:在所述第一目标角点的数量大于或等于预设数量的条件下,根据所述第一目标角点的 坐标与所述第二目标角点的坐标,计算所述第二单应性矩阵;在所述第一目标角点的数量小于预设数量的条件下,从所述第三图像提取多个第一定向快速旋转ORB特征点,以及从所述第二图像中提取多个第二ORB特征点;将所述多个第一ORB特征点与所述多个第二ORB特征点进行特征匹配,确定所述多个第一ORB特征点中至少一个第一目标ORB特征点的坐标,以及确定所述多个第二ORB特征点中至少一个第二目标ORB特征点的坐标,所述至少一个第一目标ORB特征点与所述至少一个第二目标ORB特征点匹配;根据所述至少一个第一目标ORB特征点的坐标,以及所述至少一个第二目标ORB特征点的坐标,计算所述第二单应性矩阵。
- 根据权利要求10所述的装置,其中,所述提取模块,配置成:对所述第一感兴趣区域和所述第二感兴趣区域进行目标对象检测;从所述第一感兴趣区域确定第一目标对象图,以及从所述第二感兴趣区域确定第二目标对象图;从所述第一目标对象图中提取多个第一特征点,以及从所述第二目标对象图中提取多个第二特征点。
- 根据权利要求14所述的装置,其中,所述第一特征点为第一熵相关系数ECC特征点,所述第二特征点为第二ECC特征点;所述提取模块,配置成:识别所述第一目标对象图和所述第二目标对象图中是否包含目标对象;在所述第一目标对象图和所述第二目标对象图中包含目标对象的条件下,从所述第一目标对象图中提取多个所述第一ECC特征点,以及从所述第二目标对象图中提取多个所述第二ECC特征点。
- 根据权利要求14所述的装置,其中,所述提取模块,配置成:获取所述第二目标对象图的直方图,并确定所述第二目标对象图中每个像素点的像素值;根据所述第二目标对象图中每个像素点的像素值,设置所述第一目标对象图中像素位置对应的像素点的像素值。
- 根据权利要求14所述的装置,其中,所述目标对象检测为人像检测;所述提取模块,配置成:获取人像检测后得到的所述第一感兴趣区域的第一人像掩膜,以及所述第二感兴趣区域的第二人像掩膜;根据所述第一人像掩膜从所述第一图像中确定第一人像图,以及根据所述第二人像掩膜从所述第二图像中确定第二人像图。
- 根据权利要求10所述的装置,其中,所述提取模块,配置成:根据默认尺寸提取所述第一图像中的第三感兴趣区域,以及第二图像中的第四感兴趣区域;在所述第三感兴趣区域的尺寸小于预设尺寸的情况下,根据所述预设尺寸扩展所述第三感兴趣区域,得到所述第一感兴趣区域;在所述第四感兴趣区域的尺寸小于所述预设尺寸的情况下,根据所述预设尺寸扩展所述第四感兴趣区域,得到所述第二感兴趣区域。
- 一种计算机设备,包括:存储器和一个或多个处理器,所述存储器中存储有计算机可读指令;所述计算机可读指令被所述一个或多个处理器执行时,使得所述一个或多个处理器执行权利要求1-9任一项所述的图像对齐方法的步骤。
- 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行权利要求1-9任一项所述的图像对齐方法的步骤。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111459872.0A CN114298902A (zh) | 2021-12-02 | 2021-12-02 | 一种图像对齐方法、装置、电子设备和存储介质 |
CN202111459872.0 | 2021-12-02 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023098045A1 true WO2023098045A1 (zh) | 2023-06-08 |
Family
ID=80964942
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/100804 WO2023098045A1 (zh) | 2021-12-02 | 2022-06-23 | 图像对齐方法、装置、计算机设备和存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114298902A (zh) |
WO (1) | WO2023098045A1 (zh) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114298902A (zh) * | 2021-12-02 | 2022-04-08 | 上海闻泰信息技术有限公司 | 一种图像对齐方法、装置、电子设备和存储介质 |
WO2023206475A1 (zh) * | 2022-04-29 | 2023-11-02 | 北京小米移动软件有限公司 | 图像处理方法、装置、电子设备及存储介质 |
CN114998097A (zh) * | 2022-07-21 | 2022-09-02 | 深圳思谋信息科技有限公司 | 图像对齐方法、装置、计算机设备和存储介质 |
CN116309760B (zh) * | 2023-05-26 | 2023-09-19 | 安徽高哲信息技术有限公司 | 谷物的图像对齐方法及谷物检测设备 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120281921A1 (en) * | 2011-05-02 | 2012-11-08 | Los Alamos National Security, Llc | Image alignment |
CN104376548A (zh) * | 2014-11-07 | 2015-02-25 | 中国电子科技集团公司第二十八研究所 | 一种基于改进型surf算法的图像快速拼接方法 |
CN109886878A (zh) * | 2019-03-20 | 2019-06-14 | 中南大学 | 一种基于由粗到精配准的红外图像拼接方法 |
CN110610457A (zh) * | 2018-06-14 | 2019-12-24 | 杭州海康威视数字技术股份有限公司 | 全景图像拼接方法、装置以及*** |
CN114298902A (zh) * | 2021-12-02 | 2022-04-08 | 上海闻泰信息技术有限公司 | 一种图像对齐方法、装置、电子设备和存储介质 |
-
2021
- 2021-12-02 CN CN202111459872.0A patent/CN114298902A/zh active Pending
-
2022
- 2022-06-23 WO PCT/CN2022/100804 patent/WO2023098045A1/zh unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120281921A1 (en) * | 2011-05-02 | 2012-11-08 | Los Alamos National Security, Llc | Image alignment |
CN104376548A (zh) * | 2014-11-07 | 2015-02-25 | 中国电子科技集团公司第二十八研究所 | 一种基于改进型surf算法的图像快速拼接方法 |
CN110610457A (zh) * | 2018-06-14 | 2019-12-24 | 杭州海康威视数字技术股份有限公司 | 全景图像拼接方法、装置以及*** |
CN109886878A (zh) * | 2019-03-20 | 2019-06-14 | 中南大学 | 一种基于由粗到精配准的红外图像拼接方法 |
CN114298902A (zh) * | 2021-12-02 | 2022-04-08 | 上海闻泰信息技术有限公司 | 一种图像对齐方法、装置、电子设备和存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN114298902A (zh) | 2022-04-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023098045A1 (zh) | 图像对齐方法、装置、计算机设备和存储介质 | |
US9275281B2 (en) | Mobile image capture, processing, and electronic form generation | |
US10284789B2 (en) | Dynamic generation of image of a scene based on removal of undesired object present in the scene | |
WO2017107700A1 (zh) | 一种实现图像配准的方法及终端 | |
US9609221B2 (en) | Image stabilization method and electronic device therefor | |
WO2019042419A1 (zh) | 图像跟踪点获取方法、设备及存储介质 | |
AU2017232186A1 (en) | Fast and robust image alignment for burst mode | |
US20130129213A1 (en) | Non-Rigid Dense Correspondence | |
US10122912B2 (en) | Device and method for detecting regions in an image | |
CN110136198B (zh) | 图像处理方法及其装置、设备和存储介质 | |
CN106296570B (zh) | 图像处理方法及装置 | |
US9147226B2 (en) | Method, apparatus and computer program product for processing of images | |
US11620730B2 (en) | Method for merging multiple images and post-processing of panorama | |
WO2022160857A1 (zh) | 图像处理方法及装置、计算机可读存储介质和电子设备 | |
US9418446B2 (en) | Method and apparatus for determining a building location based on a building image | |
WO2018184255A1 (zh) | 图像校正的方法和装置 | |
US20170351932A1 (en) | Method, apparatus and computer program product for blur estimation | |
CN114332183A (zh) | 图像配准方法、装置、计算机设备和存储介质 | |
US10861138B2 (en) | Image processing device, image processing method, and program | |
US11216961B2 (en) | Aligning digital images by selectively applying pixel-adjusted-gyroscope alignment and feature-based alignment models | |
US20230016350A1 (en) | Configurable keypoint descriptor generation | |
WO2022206679A1 (zh) | 图像处理方法、装置、计算机设备和存储介质 | |
CN115514887A (zh) | 视频采集的控制方法、装置、计算机设备和存储介质 | |
CN113056905A (zh) | 用于拍摄类长焦图像的***和方法 | |
JP5929379B2 (ja) | 画像処理装置、画像処理方法及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22899865 Country of ref document: EP Kind code of ref document: A1 |