CN113393506B

CN113393506B - Image registration method and related device and equipment

Info

Publication number: CN113393506B
Application number: CN202110713169.1A
Authority: CN
Inventors: 王求元
Original assignee: Zhejiang Shangtang Technology Development Co Ltd
Current assignee: Zhejiang Shangtang Technology Development Co Ltd
Priority date: 2021-06-25
Filing date: 2021-06-25
Publication date: 2022-12-27
Anticipated expiration: 2041-06-25
Also published as: CN113393506A

Abstract

The application discloses an image registration method, a related device and equipment, wherein the image registration method comprises the following steps: acquiring an image to be registered and a target image, wherein the target image is a curved surface, and unfolding the target image into a plane to obtain a reference image; extracting a first feature point and a first feature representation thereof in a reference image, and extracting a second feature point and a second feature representation thereof in an image to be registered; based on a coordinate system of a preset model, acquiring first coordinate information of a pixel point in the target image, corresponding to the first characteristic point, in the coordinate system, and acquiring second coordinate information of a second characteristic point in the image to be registered, wherein coordinates of the preset model are established according to a curved surface; and obtaining a registration parameter between the target image and the image to be registered by using the first coordinate information and the first characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point. By the scheme, the registration of the curved surface images can be realized.

Description

Image registration method and related device and equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image registration method, and a related apparatus and device.

Background

With the development of electronic information technology, augmented Reality (AR), virtual Reality (VR), and the like become application hotspots in the field of computer vision, and the surrounding environment can be digitized by using a camera as an input device and processing with an image algorithm, so that the use experience of interaction with a real environment is obtained.

The image registration is a research focus in the field of computer vision such as AR, VR and the like, and the registration parameters between the image to be registered and the target image shot by the camera can be obtained through the image registration technology, so that the registration position of the target image in the image to be registered can be obtained through the registration parameters subsequently. Currently, existing image registration techniques mainly face registration between planar images. However, there may be cases in which the target image is a curved surface in many scenes. In view of this, how to realize the registration of the curved surface image becomes an urgent problem to be solved.

Disclosure of Invention

The application provides an image registration method, a related device and equipment.

A first aspect of the present application provides an image registration method, including: acquiring an image to be registered and a target image, wherein the target image is a curved surface, and unfolding the target image into a plane to obtain a reference image; extracting a first feature point and a first feature representation thereof in a reference image, and extracting a second feature point and a second feature representation thereof in an image to be registered; based on a coordinate system of a preset model, acquiring first coordinate information of a pixel point in the target image, corresponding to the first characteristic point, in the coordinate system, and acquiring second coordinate information of a second characteristic point in the image to be registered, wherein the coordinate system of the preset model is established according to a curved surface; and obtaining a registration parameter between the target image and the image to be registered by using the first coordinate information and the first characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point.

Therefore, by acquiring the image to be registered and the target image, the target image is a curved surface, and the target image is expanded into a plane to obtain the reference image, so as to extract the first feature point and the first feature representation thereof in the reference image, and extract the second feature point and the second feature representation thereof in the image to be registered, on the basis, the first coordinate information of the pixel point corresponding to the first feature point in the target image in the coordinate system is acquired based on the coordinate system of the preset model, the second coordinate information of the second feature point in the image to be registered is acquired, and the coordinate system of the preset model is established according to the curved surface, so that the registration parameter between the target image and the image to be registered is obtained by utilizing the first coordinate information and the second feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point, and the registration parameter between the target image and the image to be registered can be realized through the processes of expansion of the curved surface image, matching of the preset model, and the like.

Wherein, the unfolding of the target image into a plane to obtain a reference image comprises: unfolding the target image from the unfolding line to a plane to obtain an unfolded image; wherein, the two edge lines of the unfolded image are both unfolded lines; based on the unfolded image, a reference image is obtained.

Therefore, the target image is unfolded from the unfolding line to the plane to obtain the unfolded image, the two edge lines of the unfolded image are the unfolding lines, the reference image is obtained based on the unfolded image, the curved image can be unfolded to the plane image quickly, and the registration efficiency of the curved image is improved.

Wherein obtaining a reference image based on the expanded image comprises: respectively taking at least one of the two edge lines as an initial position, and extending the expanded image by an extension area with a preset size from the initial position to obtain a reference image; wherein the image information of one side of the edge line from the extended area is at least partially identical to the image information of the other side of the edge line from the developed image.

Therefore, at least one of the two edge lines is respectively used as a starting position, the unfolded image is extended to an extended area with a preset size from the starting position to obtain a reference image, and the image information of one side of the edge line from the extended area is at least partially the same as the image information of the other side of the edge line from the unfolded image, so that the reference image obtained by unfolding the image into a plane comprises complete and continuous image information, thereby being beneficial to improving the accuracy of the first feature point extracted subsequently and the first feature representation thereof and further being beneficial to improving the accuracy of the registration parameter.

Wherein the unfolding line of the target image intersects with the X-axis or the Y-axis of the coordinate system; and/or, the altitude of the preset model is superposed with the Z axis of the coordinate system; and/or the origin of the coordinate system is any one of: the center of the preset model, the center of the bottom surface of the preset model and the center of the top surface of the preset model.

Therefore, by setting the expansion line of the target image to intersect with the X-axis or the Y-axis of the coordinate system, it is possible to contribute to reducing the complexity of acquiring the first coordinate information; the altitude of the preset model is overlapped with the Z axis of the coordinate system, so that the complexity of acquiring the first coordinate information can be reduced; by setting the origin of the coordinate system to either: the center of the preset model, the center of the bottom surface of the preset model and the center of the top surface of the preset model can be beneficial to reducing the complexity of obtaining the first coordinate information.

The method for obtaining the registration parameters between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point comprises the following steps: obtaining feature similarity between the first feature point and the second feature point by using the first feature representation of the first feature point and the second feature representation of the second feature point; taking the first characteristic point and the second characteristic point corresponding to the characteristic similarity meeting the preset condition as a group of characteristic point pairs; and obtaining a registration parameter based on the first coordinate information of the first characteristic point and the second coordinate information of the second characteristic point in the characteristic point pair.

Therefore, the feature similarity between the first feature point and the second feature point is obtained by using the first feature representation of the first feature point and the second feature representation of the second feature point, so that the first feature point and the second feature point corresponding to the feature similarity meeting the preset condition are used as a group of feature point pairs, and then the registration parameter is obtained based on the first coordinate information of the first feature point and the second coordinate information of the second feature point in the feature point pairs, that is, the registration parameter is obtained based on the feature point pair formed by the first feature point and the second feature point corresponding to the feature similarity meeting the preset condition, so that the accuracy of the registration parameter can be improved.

After obtaining the registration parameters between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point, the method further includes: acquiring a calibration result of the registration parameters; the calibration result comprises at least one of a first calibration result and a second calibration result, the first calibration result is calibrated based on third coordinate information of the first key point in the image to be calibrated in the coordinate system, and the second calibration result is calibrated by using the pixel value of the second key point in the reference image; and determining whether the registration parameters are accurate or not based on the verification result.

Therefore, after the registration parameter between the target image and the image to be registered is obtained, a verification result of the registration parameter is further obtained, the verification result includes at least one of a first verification result and a second verification result, the first verification result is obtained by verifying based on third coordinate information of the first key point in the image to be registered in the coordinate system, the second verification result is obtained by verifying the pixel value of the second key point in the reference image, and whether the registration parameter is accurate or not is determined based on the verification result, so that the registration parameter can be verified based on at least one of the first verification result and the second verification result, and the accuracy of the registration parameter can be further improved.

The checking result comprises a first checking result, and the preset model is a cylinder; acquiring a verification result of the registration parameters, comprising: acquiring a vanishing point of the edge of the side face of the cylinder in the image to be registered as a first key point; acquiring third coordinate information of the first key point by using the registration parameters; acquiring a vanishing point vector based on the third coordinate information of the first key point; and obtaining a first checking result by utilizing an included angle between the vanishing point vector and the coordinate axis of the coordinate system.

Therefore, when the verification result comprises the first verification result and the preset model is a cylinder, the vanishing point of the side edge of the cylinder in the image to be registered is obtained and used as the first key point, the third coordinate information of the first key point is obtained by utilizing the registration parameter, so that the vanishing point vector is obtained based on the third coordinate information of the first key point, and the first verification result is obtained by utilizing the included angle between the vanishing point vector and the coordinate axis of the coordinate system.

Wherein the altitude of the preset model is superposed with the Z axis of the coordinate system; based on the third coordinate information of the first key point, obtaining a vanishing point vector, including: obtaining a vanishing point vector based on the third coordinate information and the origin of the coordinate system; obtaining a first checking result by utilizing an included angle between the vanishing point vector and a coordinate axis of a coordinate system, wherein the first checking result comprises the following steps: and acquiring an included angle between the vanishing point vector and the Z axis, and acquiring a first verification result based on the magnitude relation between the included angle and a preset included angle threshold value.

Therefore, the altitude of the preset model is set to coincide with the Z axis of the coordinate system, so that a vanishing point vector is obtained based on the third coordinate information and the original point of the coordinate system, the included angle between the vanishing point vector and the Z axis is obtained, a first check result is obtained based on the size relation between the included angle and the preset included angle threshold, and the accuracy of the first check result can be improved.

Wherein the check result comprises a second check result; acquiring a verification result of the registration parameters, comprising: selecting a plurality of pixel points as second key points in a target area of a target image; the image information of the target area is contained in the image to be registered; determining a third key point corresponding to the second key point in the image to be registered by using the registration parameter; and obtaining a second check result based on the pixel value difference between the second key point and the corresponding third key point.

Therefore, under the condition that the verification result comprises a second verification result, a plurality of pixel points are selected as second key points in a target area of the target image, and the image information of the target area is contained in the image to be registered, on the basis, the registration parameters are utilized to determine third key points corresponding to the second key points in the image to be registered, and a second verification result is obtained based on the pixel value difference between the second key points and the corresponding third key points, so that the second verification result can be obtained based on the pixel value difference between the second key points selected from the reference image and the third key points corresponding to the second key points in the image to be registered, the complexity of obtaining the second verification result can be reduced, and the accuracy of the second verification result can be improved.

Wherein, in the target region of the target image, select a plurality of pixel points as the second key point, include: determining a first grid line in the target area at intervals of first numerical row pixels, and determining a second grid line in the target area at intervals of second numerical column pixels; and taking the intersection point between the first grid line and the second grid line as a second key point.

Therefore, by determining one first grid line at every first numerical row pixel in the target area and determining one second grid line at every second numerical column pixel in the target area, the intersection point between the first grid line and the second grid line is used as the second key point, which can be beneficial to improving the accuracy of the second check result.

The verification result comprises a first verification result and a second verification result, and the first verification result and the second verification result comprise whether the registration parameters are verified to be accurate or not; determining whether the registration parameters are accurate based on the verification result, comprising: determining that the registration parameters are accurate under the condition that the first check result and the second check result both comprise that the registration parameters are accurately checked; determining that the registration parameter is inaccurate if either of the first and second verification results includes that the registration parameter verifies as inaccurate.

Therefore, when the verification result includes the first verification result and the second verification result, and the first verification result and the second verification result both include whether the registration parameter is verified as accurate, if the first verification result and the second verification result both include that the registration parameter is verified as accurate, the accuracy of the registration parameter is determined, and if either of the first verification result and the second verification result includes that the registration parameter is verified as inaccurate, the registration parameter is determined to be inaccurate, that is, the registration parameter is determined to be accurate only under the condition that the first verification result and the second verification result both verify that the registration parameter is accurate, so that the accuracy of the verification result can be improved.

A second aspect of the present application provides an image registration apparatus, including: the system comprises an image acquisition module, an image expansion module, a feature extraction module, a coordinate acquisition module and a parameter acquisition module, wherein the image acquisition module is used for acquiring an image to be registered and a target image, and the target image is a curved surface; an image expansion image for expanding the target image into a plane to obtain a reference image; the characteristic extraction module is used for extracting a first characteristic point and a first characteristic representation thereof in the reference image and extracting a second characteristic point and a second characteristic representation thereof in the image to be registered; the coordinate acquisition module is used for acquiring first coordinate information of a pixel point in the target image in the coordinate system corresponding to the first characteristic point in the target image based on a coordinate system of a preset model and acquiring second coordinate information of a second characteristic point in the image to be registered, wherein the coordinate system of the preset model is established according to a curved surface; and the parameter acquisition module is used for acquiring registration parameters between the target image and the image to be registered by utilizing the first coordinate information and the first characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point.

A third aspect of the present application provides an electronic device, which includes a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the image registration method in the first aspect.

A fourth aspect of the present application provides a computer-readable storage medium having stored thereon program instructions that, when executed by a processor, implement the image registration method of the first aspect described above.

According to the scheme, the image to be registered and the target image are obtained, the target image is a curved surface, the target image is unfolded into a plane to obtain the reference image, the first characteristic point and the first characteristic representation of the first characteristic point in the reference image are extracted, the second characteristic point and the second characteristic representation of the second characteristic point in the image to be registered are extracted, on the basis, the first coordinate information of the pixel point corresponding to the first characteristic point in the target image in the coordinate system is obtained based on the coordinate system of the preset model, the second coordinate information of the second characteristic point in the image to be registered is obtained, the coordinate system of the preset model is established according to the curved surface, and therefore the registration parameter between the target image and the image to be registered is obtained by utilizing the first coordinate information and the second characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point, and the registration parameter between the target image and the image to be registered can be realized through the processes of unfolding the curved surface image, matching the preset model and the like.

Drawings

FIG. 1 is a schematic flow chart diagram of an embodiment of an image registration method of the present application;

FIG. 2 is a diagram illustrating an embodiment of the unfolding of a target image into a reference image;

FIG. 3 is a diagram illustrating an embodiment of a coordinate system of a predetermined model;

FIG. 4 is a schematic flow chart diagram of another embodiment of the image registration method of the present application;

FIG. 5 is a schematic diagram of a framework of an embodiment of an image registration apparatus of the present application;

FIG. 6 is a block diagram of an embodiment of an electronic device of the present application;

FIG. 7 is a block diagram of an embodiment of a computer-readable storage medium of the present application.

Detailed Description

The embodiments of the present application will be described in detail below with reference to the drawings.

In the following description, for purposes of explanation rather than limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, "plurality" herein means two or more than two.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of an image registration method according to the present application. Specifically, the method may include the steps of:

step S11: and acquiring an image to be registered and a target image.

In the embodiment of the present disclosure, the target image is a curved surface. Specifically, the target image may be set according to the actual application requirements. For example, the target image may be an image on a post; alternatively, the target image may be an image on a ball such as a soccer ball; alternatively, the target image may be an image on a cone, which is not limited herein.

In one implementation scenario, the image to be registered may be an image captured by a camera. For example, in application scenarios such as AR and VR, the image to be registered may be an image captured by an electronic device such as a mobile phone, a tablet computer, and smart glasses; or, in a video monitoring scene, the image to be registered may be an image captured by a monitoring camera, which is not limited herein. Other scenarios may be analogized, and are not exemplified here.

Step S12: the target image is unfolded into a plane to obtain a reference image.

In an implementation scenario, the target image may be expanded from the expanded line to a plane to obtain an expanded image, and both edge lines of the expanded image are expanded lines, on the basis, the expanded image may be directly used as a reference image. Taking the target image as an image on the side of a cylinder such as a column as an example, the edge of the side of the cylinder can be used as an unfolding line, and on this basis, the target image can be unfolded into a plane to obtain a rectangular reference image, and two edge lines (i.e. one set of opposite sides of the rectangle) of the reference image are the unfolding lines. Other cases may be analogized and are not illustrated here.

In another implementation scenario, in order to improve the accuracy of the first feature point and the first feature representation thereof extracted from the reference image, the target image may be expanded from the expansion line to a plane to obtain an expanded image, and both edge lines of the expanded image are expansion lines, and then, on the basis, at least one of the two edge lines may be further used as a starting position, and the expanded image is extended by an extension area of a preset size from the starting position to obtain the reference image, and image information derived from the expansion image on one side of the edge line is at least partially identical to image information derived from the expansion image on the other side of the edge line. Therefore, the reference image obtained by unfolding the image into a plane comprises complete and continuous image information, so that the accuracy of the first feature points extracted subsequently and the first feature representation thereof can be improved, and the accuracy of the registration parameters can be improved.

In a specific implementation scenario, the preset size of the extension area may be set according to an actual application situation. For example, when the image size of the target image is large, the preset size of the extension region may be set to be slightly larger, and conversely, when the image size of the target image is small, the preset size of the extension region may be set to be slightly smaller, that is, the preset size of the extension region may be in a positive correlation with the image size of the target image.

In another specific implementation scenario, only one of the two edge lines may be used as the starting position; alternatively, both edge lines may be used as the starting positions, which is not limited herein.

In another specific implementation scenario, please refer to fig. 2 in combination, wherein fig. 2 is a schematic diagram illustrating an embodiment of unfolding a target image into a reference image. As shown in fig. 2, the object image is a curved image matching the cylindrical side surface, the object image has a pattern "a", the unfolding line passes through the middle of the pattern "a", a unfolded image (such as a solid line rectangle in fig. 2) can be obtained by unfolding the object image from the unfolding line, and the left side edge line in the unfolded image contains image information of the right half of the pattern "a", and the right side edge line in the unfolded image contains image information of the left half of the pattern "a", so that it can be seen that the unfolded image obtained by unfolding the object image directly from the unfolding line into a plane may not contain complete and continuous image information. Therefore, it is possible to take the left edge line as a start position and extend the developed image by an extension area of a preset size from the left edge line, and the image information of the left edge line side derived from the extension area is the same as the image information of the right edge line side derived from the developed image, i.e., the image information of the left edge line side derived from the extension area and the image information of the right edge line side derived from the developed image are both the left half of the pattern "a". Further, it is also possible to take the right edge line as a start position and extend the developed image by an extension area of a preset size from the right edge line, and the image information of the right edge line side derived from the extension area is the same as the image information of the left edge line side derived from the developed image, i.e., the image information of the right edge line side derived from the extension area and the image information of the left edge line side derived from the developed image are both right half portions of the pattern "a". Other cases may be analogized, and no one example is given here. Therefore, the reference image obtained by extending and expanding into a plane includes complete and continuous image information (as shown in fig. 2, the reference image includes a complete pattern "a"), which is beneficial to improving the accuracy of the first feature points extracted subsequently and the first feature representation thereof, and is further beneficial to improving the accuracy of the registration parameters.

That is, the target image may be expanded from the expanded line to a plane to obtain an expanded image, and both edge lines of the expanded image are expanded lines, and on the basis, the reference image is obtained based on the expanded image. In one of the manners, the expanded image may be directly used as a reference image, and in the other manner, the expanded image may be extended to a certain extent to obtain the reference image, and the two manners may be selected according to actual situations, which is not limited herein. For example, the second approach may be preferentially selected in case the pattern on the target image is more continuous or the accuracy requirement for image registration is higher, while the first approach may be selected in case the image on the target image is more discrete and the accuracy requirement for image registration is looser.

Step S13: and extracting a first characteristic point and a first characteristic representation thereof in the reference image, and extracting a second characteristic point and a second characteristic representation thereof in the image to be registered.

In an implementation scenario, a first Feature point and a first Feature representation thereof in a reference image may be extracted by using a Feature extraction method, such as ORB (organized FAST and Rotated BRIEF), SIFT (Scale-innovative Feature Transform), and the like, and the Feature extraction method may be specifically selected according to actual needs, which is not limited herein.

In another implementation scenario, a second feature point and a second feature representation thereof in the image to be registered may be extracted by using a feature extraction method such as ORB, SIFT, and the like, and the feature extraction method may be specifically selected according to the actual application requirement, which is not limited herein.

In an implementation scenario, to improve the registration accuracy, a plurality of first pyramid images may be obtained based on the reference image, and the plurality of first pyramid images are obtained by scaling the reference image by using a plurality of scaling coefficients (e.g., 1.0, 0.8, 0.6, etc.), and on this basis, the plurality of first pyramid images may be extracted respectively to obtain first feature points and first feature representations thereof in the plurality of first pyramid images. Similarly, a plurality of second pyramid images may be obtained based on the image to be registered, and the plurality of second pyramid images are obtained by scaling the image to be registered by using a plurality of scaling coefficients (e.g., 1.0, 0.8, 0.6, etc.), on this basis, the plurality of second pyramid images may be extracted respectively to obtain second feature points and second feature representations thereof in the plurality of second pyramid images respectively.

It should be noted that the ORB may be used to quickly create feature representations (such as feature vectors) for feature points in an image, and these feature representations may be used to identify objects in the image, and the specific extraction process is not described herein again; in addition, the SIFT has scale invariance, so that key points can be detected in the image, and the SIFT is a local feature description, and the specific extraction process is not described in detail herein.

Step S14: based on a coordinate system of a preset model, first coordinate information of a pixel point in the target image corresponding to the first characteristic point in the coordinate system is obtained, and second coordinate information of a second characteristic point in the image to be registered is obtained.

In the embodiment of the present disclosure, the coordinate system of the predetermined model is established according to the curved surface. Specifically, a preset model can be modeled according to the curved surface, and the surface of the preset model can be matched with the curved surface, and on the basis, a coordinate system is established according to the preset model obtained through modeling.

In one implementation scenario, the preset model may be set according to the target image, for example, in a case where the target image is an image on a circular column, the preset model may be set as a cylinder; or, in the case that the target image is an image on a soccer ball, the preset model may be set as a sphere; alternatively, in the case where the target image is an image on a cone, the preset model may be set as a cone. Other cases may be analogized, and no one example is given here.

In one implementation scenario, to facilitate obtaining the first coordinate information, the unfolding line of the target image may intersect with an X-axis of a coordinate system of the preset model; alternatively, the unfolding line of the target image may intersect the Y-axis of the coordinate system.

In another implementation scenario, to facilitate obtaining the first coordinate information, the altitude of the preset model may coincide with the Z-axis of the coordinate system of the preset model.

In another implementation scenario, in order to obtain the first coordinate information, an origin of the coordinate system of the preset model may be a center of the preset model, or the origin of the coordinate system of the preset model may also be a center of a bottom surface of the preset model, or the origin of the coordinate system of the preset model may also be a center of a top surface of the preset model, which may be set according to an actual application condition, and is not limited herein.

In a specific implementation scenario, taking the preset model as a cylinder as an example, please refer to fig. 2 and fig. 3, and fig. 3 is a schematic diagram of an embodiment of a coordinate system of the preset model. As shown in fig. 3, the predetermined model whose surface is matched with the curved surface of the target image shown in fig. 2 is a cylinder, and on this basis, the center of the bottom surface of the cylinder (i.e., the black point in fig. 3) can be used as the origin O of the coordinate system, the X axis of the coordinate system passes through the unfolding line, and the Z axis coincides with the altitude of the predetermined model, so that the coordinate system of the predetermined model can be established. In the case that the origin is coincident with the Y axis by using other point locations or expansion lines, the analogy can be done, and no one example is given here.

In another specific implementation scenario, with continuing reference to fig. 3, under the condition of establishing the coordinate system as shown in fig. 3, a functional expression of the cylinder side can be obtained:

x ² +y ² ＝R ² ……(1)

in the above formula (1), X and Y respectively represent coordinate values of a point P (X, Y, z) on the side surface of the cylinder on the X axis and the Y axis of the coordinate system, and R represents the radius of the top surface (or the ground surface) of the cylinder. In addition, z ranges from 0 to h (i.e., the height of the cylinder).

In an implementation scenario, on the basis of establishing a coordinate system of a preset model, first coordinate information of a corresponding pixel point of a first feature point in a target image in the coordinate system may be obtained based on the first feature point extracted from a reference image.

In a specific implementation scenario, please continue to refer to fig. 2 and fig. 3 in combination, as shown in fig. 2, a first feature point p1 is extracted from the reference image, the first feature point corresponds to a pixel point p1 of the target image shown in fig. 3, and based on the coordinate system established in fig. 3, first coordinate information of the first feature point p1 can be obtained; and referring to another first feature point p2 extracted from the image, where the first feature point corresponds to the pixel point p2 of the target image shown in fig. 3, and based on the coordinate system established in fig. 3, the first coordinate information of the first feature point p2 may be obtained.

It should be emphasized that the first coordinate information represents the coordinate information of the pixel point corresponding to the first feature point in the target image in the coordinate system.

In an implementation scenario, a coordinate system (i.e., a two-dimensional coordinate system) of the image to be registered may be established, and on this basis, second coordinate information of the second feature point in the image to be registered may be obtained.

Step S15: and obtaining registration parameters between the target image and the image to be registered by using the first coordinate information and the first characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point.

In an implementation scenario, a feature similarity between a first feature point and a second feature point may be obtained by using a first feature representation of the first feature point and a second feature representation of the second feature point, and the first feature point and the second feature point corresponding to the feature similarity satisfying a preset condition are used as a set of feature point pairs, so that a registration parameter may be obtained based on first coordinate information of the first feature point and second coordinate information of the second feature point in the feature point pairs.

In a specific implementation scenario, the first feature representation of the first feature point and the second feature representation of the second feature point are feature vectors composed of numbers 0 and 1, and on this basis, the total number of elements in the same position of the first feature representation and the second feature representation may be counted, and the larger the total number is, the higher the feature similarity between the first feature representation and the second feature representation is. For example, taking a feature vector with a dimension of 4 as an example, the total number of elements in the same position of the first feature representation [ 01 0] and the second feature representation [ 01 0] is 4, and the dimension of the feature vector may be set according to the actual application requirement, for example, in order to improve the accuracy of feature representation as much as possible, the vector dimension of the feature representation may be improved, for example, the vector dimension may be set to 256 dimensions, which is not limited herein.

In another specific implementation scenario, the preset condition may include that the feature similarity is higher than a preset threshold. Specifically, the preset threshold may be set according to an actual application situation, for example, in a case where a requirement on accuracy of the registration parameter is high, the preset threshold may be set to be larger, and in a case where the requirement on accuracy of the registration parameter is relatively loose, the preset threshold may be set to be smaller, and a specific value of the preset threshold is not limited herein.

In yet another specific implementation scenario, the first coordinate information of the first feature Point and the second coordinate information of the second feature Point in the feature Point pair may be processed in a manner such as PnP (passive Point), so as to obtain the registration parameter. The detailed processing procedure of PnP is not described herein.

In another implementation scenario, in order to improve robustness, first coordinate information and a first feature representation of a first feature point and second coordinate information and a second feature representation of a second feature point may be processed based on a PnP manner of RANdom SAmple Consensus (RANSAC), so as to obtain a registration parameter between a target image and an image to be registered. Outliers can be effectively eliminated through random consistent sampling, robustness is improved beneficially, and the specific processing processes of random consistent sampling and PnP are not repeated.

In yet another implementation scenario, after the registration parameters are obtained, the registration position of the target image in the image to be registered may be obtained by using the registration parameters.

Referring to fig. 4, fig. 4 is a flowchart illustrating an image registration method according to another embodiment of the present application. Specifically, the method may include the steps of:

step S41: and acquiring an image to be registered and a target image.

In the embodiment of the present disclosure, the target image is a curved surface, which may specifically refer to the related description in the foregoing embodiment, and details are not repeated here.

Step S42: the target image is unfolded into a plane to obtain a reference image.

Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Step S43: and extracting a first characteristic point and a first characteristic representation thereof in the reference image, and extracting a second characteristic point and a second characteristic representation thereof in the image to be registered.

Step S44: based on a coordinate system of a preset model, first coordinate information of a pixel point in the target image corresponding to the first feature point in the coordinate system is obtained, and second coordinate information of a second feature point in the image to be registered is obtained.

In the embodiment of the present disclosure, the coordinates of the preset model are established according to a curved surface, which may specifically refer to the related description in the foregoing embodiment, and are not described herein again.

Step S45: and obtaining registration parameters between the target image and the image to be registered by using the first coordinate information and the first characteristic representation of the first characteristic point and the second coordinate information and the second characteristic representation of the second characteristic point.

Reference may be made specifically to the description related to the foregoing embodiments, which are not described herein again.

Step S46: and acquiring a checking result of the registration parameters.

In the embodiment of the present disclosure, the verification result includes at least one of a first verification result and a second verification result, the first verification result is verified based on the third coordinate information of the first keypoint in the image to be registered in the coordinate system, and the second verification result is verified by using the pixel value of the second keypoint in the reference image.

In an implementation scenario, the verification result may include a first verification result, for example, taking a preset model as a cylinder, a vanishing point of a side edge of the cylinder in the image to be registered may be obtained as a first key point, and third coordinate information of the first key point may be obtained by using the registration parameter, on this basis, a vanishing point vector may be obtained based on the third coordinate information of the first key point, so that an included angle between the vanishing point vector and a coordinate axis of a coordinate system may be used to obtain the first verification result. Therefore, the registration parameters can be verified through the vanishing points on the edges of the side faces of the cylinder under the condition that the preset model is the cylinder, namely the registration parameters can be verified from the geometric verification dimension, and therefore the accuracy of the first verification result can be improved.

In a specific implementation scenario, the third coordinate information of the first key point refers to coordinate information of the first key point in a coordinate system of the preset model. Specifically, the coordinate information of the first key point in the image coordinate system of the image to be registered may be obtained first, and the internal parameter of the camera that captures the image to be registered is obtained, so that the coordinate information of the first key point in the image coordinate system may be converted by using the internal parameter to obtain the coordinate information of the first key point in the camera coordinate system, and then the coordinate information of the first key point in the camera coordinate system may be converted by using the registration selection parameter to obtain the third coordinate information of the first key point in the coordinate system of the preset model.

In another specific implementation scenario, the vanishing point refers to a visual intersection point of parallel lines, and the side edges of the cylinder are mutually parallel line segments, so that the vanishing point of the side edge of the cylinder is the visual intersection point of the side edge. Theoretically, the vanishing point of the edge of the side surface of the cylinder is on the same straight line with the center of the top surface and the center of the bottom surface of the cylinder. Therefore, a vanishing point vector can be obtained through the third coordinate information of the first key point (namely, the vanishing point) and the origin of the preset model coordinate system, and under the condition that the Z axis of the preset model coordinate system is superposed with the high line of the cylinder, the included angle between the vanishing point vector and the Z axis is obtained, so as to verify whether the registration parameter is accurate or not through the included angle.

In another specific implementation scenario, in a case that the altitude of the preset model coincides with the Z axis of the coordinate system, an included angle between the vanishing point vector and the Z axis may be obtained in the above manner, and on the basis, the first check result may be obtained based on a magnitude relationship between the included angle and a preset included angle threshold. Specifically, when the included angle is greater than the preset included angle threshold, it may be determined that the first verification result is that the registration parameter verification is inaccurate, and conversely, when the included angle is not greater than the preset included angle threshold, it may be determined that the first verification result is that the registration parameter verification is accurate. In addition, the preset included angle threshold may be set according to an actual application situation, for example, in a case where the requirement on the accuracy of the registration parameter is high, the preset included angle threshold may be set to be slightly smaller, and in a case where the requirement on the accuracy of the registration parameter is relatively loose, the preset included angle threshold may be set to be slightly larger, and a specific numerical value of the preset included angle threshold is not limited herein.

In another implementation scenario, the verification result may include a second verification result, and on this basis, a plurality of pixel points may be selected as second key points in a target region of the target image, and image information of the target region is included in the image to be registered, and a third key point corresponding to the second key point in the image to be registered is determined by using the registration parameter, so that the second verification result may be obtained based on a pixel value difference between the second key point and the corresponding third key point. Therefore, the complexity of obtaining the second check result can be reduced, and the accuracy of the second check result can be improved.

In a specific implementation scenario, not all image information in the target image is necessarily present in the image to be registered. Therefore, a target area needs to be determined in the target image so as to select a plurality of second key points in the target area, and the image information of the target area needs to be included in the image to be registered. For example, a sphere may be photographed to obtain an image to be registered, and due to a projection relationship, only a visible portion (e.g., the front surface of the sphere) of a curved image located on the sphere is shown in the image to be registered, and an invisible portion (e.g., the back surface of the sphere) is not shown in the image to be registered, so that an image area of the visible portion in the target image may be used as the target area.

In another specific implementation scenario, a first grid line may be determined every first numerical row of pixels in the target region, and a second grid line may be determined every second numerical column of pixels in the target region, so that an intersection between the first grid line and the second grid line may be used as the second key point.

In another specific implementation scenario, the coordinate information of the second key point in the coordinate system of the preset model may be obtained first, and then the coordinate information is converted into the coordinate information in the camera coordinate system by using the registration parameter, on this basis, the coordinate information in the camera coordinate system may be converted into the coordinate information in the image coordinate system of the image to be registered by using the camera internal parameter, and further, the third key point may be obtained according to the coordinate information in the image coordinate system.

In yet another specific implementation scenario, the sum of squares of differences between pixel values of the second keypoint and pixel values of the corresponding third keypoint may be counted to obtain a pixel value difference. In addition, in the case where the pixel value difference satisfies the preset condition, it may be determined that the configuration parameter is verified as accurate, whereas in the case where the pixel value difference does not satisfy the preset condition, it may be determined that the configuration parameter is not accurate. Specifically, the preset condition may include that the pixel value difference is not greater than a preset difference threshold. The preset difference threshold may be set according to practical application situations, for example, in a case where the requirement on the accuracy of the registration parameter is relatively high, the preset difference threshold may be set slightly smaller, and in a case where the requirement on the accuracy of the registration parameter is relatively loose, the preset difference threshold may be set slightly larger, and specific values of the preset difference threshold are not limited herein.

Step S47: and determining whether the registration parameters are accurate or not based on the verification result.

In one implementation scenario, the verification result may only include the first verification result, and the first verification result may include whether the registration parameter is verified as accurate, and the registration parameter is determined to be accurate if the first verification result includes that the registration parameter is verified as accurate, whereas the registration parameter is determined to be inaccurate if the first verification result includes that the registration parameter is verified as inaccurate.

In another implementation scenario, the check result may include only the second check result, and the second check result may include whether the registration parameter is checked to be accurate, and in a case where the second check result includes that the registration parameter is checked to be accurate, the registration parameter is determined to be accurate, whereas in a case where the second check result includes that the registration parameter is checked to be inaccurate, the registration parameter is determined to be inaccurate.

In yet another implementation scenario, the verification result may include a first verification result and a second verification result, and the first verification result and the second verification result may both include whether the registration parameter is verified as accurate. On the basis, the registration parameters can be determined to be accurate under the condition that the first verification result and the second verification result both comprise the registration parameters and are verified to be accurate, and conversely, the registration parameters can be determined to be inaccurate under the condition that either one of the first verification result and the second verification result comprises the registration parameters and are verified to be inaccurate.

Different from the foregoing embodiment, after obtaining the registration parameter between the target image and the image to be registered, a verification result of the registration parameter is further obtained, where the verification result includes at least one of a first verification result and a second verification result, the first verification result is obtained by verifying based on third coordinate information of the first key point in the image to be registered in the coordinate system, the second verification result is obtained by verifying using a pixel value of the second key point in the reference image, and based on the verification result, whether the registration parameter is accurate is determined, so that the registration parameter can be verified based on at least one of the first verification result and the second verification result, which is further beneficial to further improving the accuracy of the registration parameter.

Referring to fig. 5, fig. 5 is a schematic diagram of an embodiment of an image registration apparatus 50 according to the present application. The image registration device 50 comprises an image acquisition module 51, an image expansion module 52, a feature extraction module 53, a coordinate acquisition module 54 and a parameter acquisition module 55, wherein the image acquisition module 51 is used for acquiring an image to be registered and a target image, wherein the target image is a curved surface; the image unfolding module 52 is configured to unfold the target image into a plane to obtain a reference image; the feature extraction module 53 is configured to extract a first feature point and a first feature representation thereof in the reference image, and extract a second feature point and a second feature representation thereof in the image to be registered; the coordinate obtaining module 54 is configured to obtain, based on a coordinate system of a preset model, first coordinate information of a pixel point in the target image in the coordinate system, which corresponds to the first feature point in the target image, and obtain second coordinate information of a second feature point in the image to be registered, where a coordinate of the preset model is established according to a curved surface; the parameter obtaining module 55 is configured to obtain a registration parameter between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point.

In some disclosed implementations, the image unfolding module 52 includes a plane unfolding sub-module for unfolding the target image from an unfolding line into a plane, resulting in an unfolded image; wherein, the two edge lines of the unfolded image are unfolding lines; the image expansion module 52 includes an image acquisition sub-module for obtaining a reference image based on the expanded image.

Different from the foregoing embodiment, the target image is expanded from the expansion line to the plane to obtain the expanded image, and both edge lines of the expanded image are the expansion lines, and the reference image is obtained based on the expanded image, so that the curved image can be quickly expanded to the plane image, which is beneficial to improving the registration efficiency of the curved image.

In some disclosed embodiments, the image obtaining sub-module includes a start determining unit configured to take at least one of the two edge lines as a start position, respectively, and the image obtaining sub-module includes an area extending unit configured to extend the expanded image by an extension area of a preset size from the start position to obtain a reference image; wherein the image information of one side of the edge line from the extended region is at least partially identical to the image information of the other side of the edge line from the unfolded image.

Different from the embodiment, at least one of the two edge lines is respectively used as a start position, the expanded image is expanded to an expansion area with a preset size from the start position to obtain a reference image, and image information from the expansion area on one side of the edge line is at least partially the same as image information from the expanded image on the other side of the edge line, so that the reference image expanded to a plane contains complete and continuous image information, thereby being beneficial to improving the accuracy of the first feature point extracted subsequently and the first feature representation thereof, and being further beneficial to improving the accuracy of the registration parameter.

In some disclosed implementations, the unfolding line of the target image intersects the X-axis or the Y-axis of the coordinate system; and/or the altitude of the preset model is superposed with the Z axis of the coordinate system; and/or the origin of the coordinate system is any one of: the center of the preset model, the center of the bottom surface of the preset model and the center of the top surface of the preset model.

Different from the foregoing embodiment, by setting the expansion line of the target image to intersect with the X axis or the Y axis of the coordinate system, it is possible to facilitate reduction of complexity in acquiring the first coordinate information; the altitude of the preset model is overlapped with the Z axis of the coordinate system, so that the complexity of acquiring the first coordinate information can be reduced; by setting the origin of the coordinate system to either: the center of the preset model, the center of the bottom surface of the preset model and the center of the top surface of the preset model can be beneficial to reducing the complexity of obtaining the first coordinate information.

In some disclosed implementations, the parameter obtaining module 55 includes a similarity obtaining sub-module configured to obtain a feature similarity between a first feature point and a second feature point by using a first feature representation of the first feature point and a second feature representation of the second feature point, the parameter obtaining module 55 includes a feature point pair obtaining sub-module configured to use the first feature point and the second feature point corresponding to the feature similarity meeting a preset condition as a set of feature point pairs, and the parameter obtaining module 55 includes a registration parameter obtaining sub-module configured to obtain a registration parameter based on first coordinate information of the first feature point and second coordinate information of the second feature point in the feature point pairs.

Different from the foregoing embodiment, the feature similarity between the first feature point and the second feature point is obtained by using the first feature representation of the first feature point and the second feature representation of the second feature point, so that the first feature point and the second feature point corresponding to the feature similarity meeting the preset condition are used as a group of feature point pairs, and then the registration parameter is obtained based on the first coordinate information of the first feature point and the second coordinate information of the second feature point in the feature point pairs, that is, the registration parameter is obtained based on the feature point pair formed by the first feature point and the second feature point corresponding to the feature similarity meeting the preset condition, so that the accuracy of the registration parameter can be improved.

In some disclosed implementations, the image registration apparatus 50 further includes a parameter verification module for obtaining a verification result of the registration parameter; wherein the verification result includes at least one of a first verification result and a second verification result, the first verification result is verified based on third coordinate information of the first key point in the image to be registered in the coordinate system, the second verification result is verified by using the pixel value of the second key point in the reference image, and the image registration apparatus 50 further includes a parameter determination module for determining whether the registration parameter is accurate based on the verification result.

In some public implementations, the verification result includes a first verification result, the preset model is a cylinder, the parameter verification module includes a first key point obtaining submodule and is configured to obtain a vanishing point of a side edge of the cylinder in the image to be registered as the first key point, the parameter verification module includes a coordinate information obtaining submodule and is configured to obtain third coordinate information of the first key point by using the registration parameter, the parameter verification module includes a vector obtaining submodule and is configured to obtain a vanishing point vector based on the third coordinate information of the first key point, and the parameter verification module includes an included angle verification submodule and is configured to obtain the first verification result by using an included angle between the vanishing point vector and a coordinate axis of a coordinate system.

Different from the embodiment, when the verification result includes the first verification result and the preset model is a cylinder, the vanishing point of the side edge of the cylinder in the image to be registered is obtained as the first key point, the third coordinate information of the first key point is obtained by using the registration parameter, so that the vanishing point vector is obtained based on the third coordinate information of the first key point, and the first verification result is obtained by using the included angle between the vanishing point vector and the coordinate axis of the coordinate system.

In some public implementations, the altitude of the preset model coincides with a Z axis of a coordinate system, the vector obtaining sub-module is specifically configured to obtain a vanishing point vector based on the third coordinate information and an original point of the coordinate system, and the included angle checking sub-module is specifically configured to obtain an included angle between the vanishing point vector and the Z axis, and obtain a first checking result based on a magnitude relation between the included angle and a preset included angle threshold.

Different from the embodiment, the altitude of the preset model is set to coincide with the Z axis of the coordinate system, so that the vanishing point vector is obtained based on the third coordinate information and the origin of the coordinate system, the included angle between the vanishing point vector and the Z axis is obtained, the first check result is obtained based on the size relationship between the included angle and the preset included angle threshold, and the accuracy of the first check result can be improved.

In some public implementations, the verification result comprises a second verification result, and the parameter verification module comprises a second key point acquisition submodule for selecting a plurality of pixel points as second key points in a target area of the target image; the parameter verification module comprises a pixel value verification submodule for obtaining a second verification result based on the pixel value difference between the second key point and the corresponding third key point.

Different from the foregoing embodiment, in a case that the verification result includes the second verification result, in the target region of the target image, the plurality of pixel points are selected as the second key points, and the image information of the target region is included in the image to be registered, on this basis, the third key point corresponding to the second key point in the image to be registered is determined by using the registration parameter, and the second verification result is obtained based on the pixel value difference between the second key point and the corresponding third key point, so that the second verification result can be obtained based on the pixel value difference between the second key point selected in the reference image and the third key point corresponding to the second key point in the image to be registered, which can be beneficial to reducing the complexity of obtaining the second verification result and improving the accuracy of the second verification result.

In some disclosed implementations, the second keypoint acquisition sub-module includes a grid line determination unit configured to determine a first grid line every other first numerical row of pixels in the target region and a second grid line every other second numerical column of pixels in the target region, and the second keypoint acquisition sub-module includes a keypoint acquisition unit configured to use an intersection between the first grid line and the second grid line as the second keypoint.

Different from the foregoing embodiment, by determining one first grid line every other first numerical row pixels in the target region and determining one second grid line every other second numerical column pixels in the target region, the intersection point between the first grid line and the second grid line is used as the second key point, which is beneficial to improving the accuracy of the second check result.

In some disclosed implementations, the verification result includes a first verification result and a second verification result, and the first verification result and the second verification result each include whether the registration parameter is verified as accurate, the parameter determination module includes a first determination sub-module to determine that the registration parameter is accurate if the first verification result and the second verification result both include that the registration parameter is verified as accurate, and the parameter determination module includes a second determination sub-module to determine that the registration parameter is inaccurate if either of the first verification result and the second verification result includes that the registration parameter is verified as inaccurate.

Different from the foregoing embodiment, when the verification result includes the first verification result and the second verification result, and the first verification result and the second verification result both include whether the registration parameter is verified as accurate, if the first verification result and the second verification result both include that the registration parameter is verified as accurate, the accuracy of the registration parameter is determined, and if either one of the first verification result and the second verification result includes that the registration parameter is verified as inaccurate, the inaccuracy of the registration parameter is determined, that is, the accuracy of the registration parameter is determined only when the first verification result and the second verification result both verify that the registration parameter is accurate, so that the accuracy of the verification result can be improved.

Referring to fig. 6, fig. 6 is a schematic block diagram of an embodiment of an electronic device 60 according to the present application. The electronic device 60 comprises a memory 61 and a processor 62 coupled to each other, the processor 62 being configured to execute program instructions stored in the memory 61 to implement the steps of any of the above-described embodiments of the image registration method. In one particular implementation scenario, the electronic device 60 may include, but is not limited to: a microcomputer, a server, and in addition, the electronic device 60 may also include a mobile device such as a notebook computer, a tablet computer, and the like, which is not limited herein.

In particular, the processor 62 is configured to control itself and the memory 61 to implement the steps of any of the above-described embodiments of the image recognition method. The processor 62 may also be referred to as a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip having signal processing capabilities. The Processor 62 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 62 may be collectively implemented by an integrated circuit chip.

According to the scheme, the registration of the curved surface images can be realized through the processes of curved surface image unfolding, preset model inosculation and the like.

Referring to fig. 7, fig. 7 is a block diagram illustrating an embodiment of a computer readable storage medium 70 according to the present application. The computer readable storage medium 70 stores program instructions 701 executable by the processor, the program instructions 701 being for implementing the steps of any of the above-described embodiments of the image registration method.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and for specific implementation, reference may be made to the description of the above method embodiments, and for brevity, details are not described here again.

The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, and for brevity, will not be described again herein.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is only one type of logical division, and other divisions may be implemented in practice, for example, the unit or component may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on network elements. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Claims

1. An image registration method, comprising:

acquiring an image to be registered and a target image; wherein the target image is a curved surface;

unfolding the target image into a plane to obtain a reference image;

extracting a first feature point and a first feature representation thereof in the reference image, and extracting a second feature point and a second feature representation thereof in the image to be registered;

acquiring first coordinate information of a pixel point corresponding to the first characteristic point in the target image in the coordinate system based on the coordinate system of a preset model, and acquiring second coordinate information of the second characteristic point in the image to be registered; the coordinate system of the preset model is established according to the curved surface;

obtaining a registration parameter between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point;

after obtaining the registration parameters between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point, the method further includes:

obtaining a checking result of the registration parameter; wherein the verification result comprises at least one of a first verification result and a second verification result, the first verification result is verified based on third coordinate information of the first key point in the image to be registered in the coordinate system, and the second verification result is verified by using the pixel value of the second key point in the reference image;

determining whether the registration parameter is accurate based on the verification result.

2. The method of claim 1, wherein the unfolding the target image into a plane to obtain a reference image comprises:

unfolding the target image from an unfolding line to a plane to obtain an unfolded image; wherein both edge lines of the unfolded image are the unfolded lines;

and obtaining the reference image based on the expansion image.

3. The method of claim 2, wherein the deriving the reference image based on the unfolded image comprises:

respectively taking at least one of the two edge lines as a starting position;

extending the expanded image by an extension area with a preset size from the initial position to obtain the reference image;

wherein the image information of one side of the edge line from the extended region is at least partially the same as the image information of the other side of the edge line from the developed image.

4. The method of claim 2, wherein the unfolding line of the target image intersects an X-axis or a Y-axis of the coordinate system;

and/or the altitude of the preset model is superposed with the Z axis of the coordinate system;

and/or the origin of the coordinate system is any one of: the center of the preset model, the center of the bottom surface of the preset model and the center of the top surface of the preset model.

5. The method according to any one of claims 1 to 4, wherein the obtaining the registration parameter between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point comprises:

obtaining feature similarity between the first feature point and the second feature point by using a first feature representation of the first feature point and a second feature representation of the second feature point;

taking the first characteristic points and the second characteristic points corresponding to the characteristic similarity meeting the preset condition as a group of characteristic point pairs;

and obtaining the registration parameters based on the first coordinate information of the first characteristic point and the second coordinate information of the second characteristic point in the characteristic point pair.

6. The method according to any one of claims 1 to 4, wherein the verification result comprises the first verification result, and the predetermined model is a cylinder; the obtaining of the verification result of the registration parameter includes:

acquiring a vanishing point of the edge of the side face of the cylinder in the image to be registered as the first key point;

acquiring the third coordinate information of the first key point by using the registration parameters;

acquiring a vanishing point vector based on the third coordinate information of the first key point;

and obtaining the first checking result by utilizing an included angle between the vanishing point vector and the coordinate axis of the coordinate system.

7. The method of claim 6, wherein the high line of the preset model coincides with the Z axis of the coordinate system; the obtaining of the vanishing point vector based on the third coordinate information of the first key point includes:

obtaining the vanishing point vector based on the third coordinate information and the origin of the coordinate system;

the obtaining the first check result by using an included angle between the vanishing point vector and a coordinate axis of the coordinate system includes:

and acquiring an included angle between the vanishing point vector and the Z axis, and obtaining the first verification result based on the size relation between the included angle and a preset included angle threshold value.

8. The method according to any one of claims 1 to 4, wherein the verification result comprises the second verification result; the obtaining of the verification result of the registration parameter includes:

selecting a plurality of pixel points as the second key points in a target area of the target image; wherein, the image information of the target area is contained in the image to be registered;

determining a third key point corresponding to the second key point in the image to be registered by using the registration parameter;

and obtaining the second check result based on the pixel value difference between the second key point and the corresponding third key point.

9. The method of claim 8, wherein selecting a plurality of pixel points as the second key points in the target region of the target image comprises:

determining a first grid line every other first numerical row pixels in the target area, and determining a second grid line every other second numerical column pixels in the target area;

and taking the intersection point between the first grid line and the second grid line as the second key point.

10. The method according to any one of claims 1 to 4, wherein the verification result comprises the first verification result and the second verification result, and the first verification result and the second verification result each comprise whether the registration parameter is verified as accurate; the determining whether the registration parameter is accurate based on the verification result includes:

determining that the registration parameter is accurate if both the first and second verification results include that the registration parameter verified as accurate;

determining that the registration parameter is inaccurate if either of the first and second verification results includes the registration parameter verifying as inaccurate.

11. An image registration apparatus, comprising:

the image acquisition module is used for acquiring an image to be registered and a target image; wherein the target image is a curved surface;

the image unfolding module is used for unfolding the target image into a plane to obtain a reference image;

the characteristic extraction module is used for extracting a first characteristic point and a first characteristic representation thereof in the reference image and extracting a second characteristic point and a second characteristic representation thereof in the image to be registered;

the coordinate acquisition module is used for acquiring first coordinate information of a pixel point in the target image, corresponding to the first feature point, in the target image in the coordinate system based on a coordinate system of a preset model, and acquiring second coordinate information of the second feature point in the image to be registered; establishing a coordinate system of the preset model according to the curved surface;

a parameter obtaining module, configured to obtain a registration parameter between the target image and the image to be registered by using first coordinate information and first feature representation of the first feature point and second coordinate information and second feature representation of the second feature point;

after obtaining the registration parameter between the target image and the image to be registered by using the first coordinate information and the first feature representation of the first feature point and the second coordinate information and the second feature representation of the second feature point, the image registration apparatus is further configured to:

12. An electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the image registration method of any one of claims 1 to 10.

13. A computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the image registration method of any of claims 1 to 10.