CN115457144B - Calibration pattern recognition method, calibration device and electronic equipment - Google Patents
Calibration pattern recognition method, calibration device and electronic equipment Download PDFInfo
- Publication number
- CN115457144B CN115457144B CN202211091005.0A CN202211091005A CN115457144B CN 115457144 B CN115457144 B CN 115457144B CN 202211091005 A CN202211091005 A CN 202211091005A CN 115457144 B CN115457144 B CN 115457144B
- Authority
- CN
- China
- Prior art keywords
- calibration
- suspected
- patterns
- pattern
- adjacent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012567 pattern recognition method Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 claims description 43
- 238000004590 computer program Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 13
- 238000005516 engineering process Methods 0.000 claims description 10
- 238000003909 pattern recognition Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 description 20
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 10
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/64—Analysis of geometric attributes of convexity or concavity
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The disclosure provides a calibration pattern recognition method, a calibration device and electronic equipment, wherein the calibration pattern recognition method comprises the following steps: acquiring an environment image, wherein the environment image is obtained by shooting with a camera; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting the suspected patterns which are in line with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to obtain the calibration patterns which are in line with the mark arrangement mode on the calibration plate, and eliminating the suspected patterns on the background of the calibration plate, thereby improving the robustness of the identification of the calibration patterns.
Description
Technical Field
The disclosure relates to the technical field of computers, and in particular relates to a calibration pattern recognition method, a calibration device and electronic equipment.
Background
Camera calibration is a very important technology in image processing technology, and in the image measurement process and in machine vision application, in order to determine the correlation between the spatial position of a spatial object surface point and its corresponding point in an image, so that the spatial object of the objective world can be reconstructed through the image, a camera parameter of camera imaging needs to be established, wherein the process of determining the camera parameter is camera calibration.
At present, when a calibration plate is adopted for camera calibration, all suspected patterns in a shot image are extracted to be used as calibration patterns for camera calibration, and the problem of low robustness in identifying the calibration patterns exists.
Disclosure of Invention
Aspects of the present disclosure provide a calibration pattern recognition method, calibration device, and electronic apparatus, so as to solve the problem that the robustness of recognizing the calibration pattern is low at present.
A first aspect of an embodiment of the present disclosure provides a calibration pattern recognition method, including: acquiring an environment image, wherein the environment image is obtained by shooting with a camera; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting suspected patterns which accord with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to be the calibration patterns.
A second aspect of an embodiment of the present disclosure provides a calibration method, including: the calibration pattern recognition method of the first aspect, after extracting a suspected pattern conforming to a pattern arrangement manner on a calibration board as a calibration pattern from a plurality of suspected patterns, further includes: and calibrating the camera parameters by adopting marks and calibration patterns on the calibration plate.
A third aspect of embodiments of the present disclosure provides a calibration pattern recognition apparatus for performing the calibration pattern recognition method of the first aspect, the calibration pattern recognition apparatus comprising:
the acquisition module is used for acquiring an environment image, wherein the environment image is obtained by shooting with a camera;
the first extraction module is used for extracting a plurality of suspected patterns in the environment image, and the outline of the suspected patterns accords with a preset outline;
the second extraction module is used for extracting the suspected patterns which accord with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to be the calibration patterns.
A fourth aspect of an embodiment of the present disclosure provides an electronic device, including: a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the calibration pattern recognition method of the first aspect when executing the computer program.
A fifth aspect of the disclosed embodiments provides a computer-readable storage medium having stored therein computer-executable instructions for implementing the calibration pattern recognition method of the first aspect or the calibration method of the second aspect when executed by a processor.
A sixth aspect of the disclosed embodiments provides a computer program product comprising: a computer program stored in a readable storage medium, from which the computer program can be read by at least one processor of an electronic device, the at least one processor executing the computer program causing the electronic device to perform the calibration pattern recognition method of the first aspect or the calibration method of the second aspect.
The embodiment of the disclosure is applied to a calibration scene of camera parameters, and the environment image is obtained by shooting by a camera by acquiring the environment image; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting the suspected patterns which are in line with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to obtain the calibration patterns which are in line with the mark arrangement mode on the calibration plate, and eliminating the suspected patterns on the background of the calibration plate, thereby improving the robustness of the identification of the calibration patterns.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate and explain the present disclosure, and together with the description serve to explain the present disclosure. In the drawings:
fig. 1 is an application scenario diagram of a calibration pattern recognition method according to an exemplary embodiment of the present disclosure;
FIG. 2 is a flow chart of steps of a calibration pattern recognition method provided by an exemplary embodiment of the present disclosure;
FIG. 3 is a schematic diagram I of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 4 is a flowchart of steps of another calibration pattern recognition method provided by an exemplary embodiment of the present disclosure;
FIG. 5 is a second schematic diagram of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 6 is a schematic diagram III of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 7 is a schematic diagram IV of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 8 is a schematic diagram five of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 9 is a schematic diagram six of a calibration pattern determination process provided by an exemplary embodiment of the present disclosure;
FIG. 10 is a block diagram of a calibration pattern recognition device provided in an exemplary embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
For the purposes of promoting an understanding of the principles and advantages of the disclosure, reference will now be made to the drawings and specific examples thereof, together with the following description. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present disclosure. Based on the embodiments in this disclosure, all other embodiments that a person of ordinary skill in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.
In the calibration process of camera parameters, shooting is carried out on a calibration plate to obtain corresponding calibration images, wherein a plurality of marks are arranged on the calibration plate, the calibration patterns corresponding to the marks are arranged in the calibration images, the calibration patterns of each mark need to be identified in the calibration process, and a one-to-one matching relationship between the marks and mark objects is established, so that the camera parameters can be calibrated, however, patterns in the background of the calibration plate may exist in the calibration images, the identification of the calibration patterns is influenced by the presence of the patterns in the calibration images, the obtained calibration patterns are inaccurate, and the accuracy of the calibration of the camera parameters is influenced.
Based on the above problems, the calibration pattern recognition method provided by the embodiment of the present disclosure is applied to a calibration scene of camera parameters, and the environment image is obtained by capturing an environment image by a camera; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting the suspected patterns which are in line with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to obtain the calibration patterns which are in line with the mark arrangement mode on the calibration plate, and eliminating the suspected patterns on the background of the calibration plate, thereby improving the robustness of the identification of the calibration patterns.
In this embodiment, the calibration pattern recognition method may be a calibration pattern recognition method that implements the whole by means of a server. In addition, the server device such as a server conventional server or a server array for executing the calibration pattern recognition method may be a cloud server, which is not limited herein.
In addition, an application scenario in this embodiment of the disclosure is as shown in fig. 1, where fig. 1 includes a calibration board 11 and an environmental image 12 obtained by photographing the calibration board by a camera, where the calibration board 11 is placed on an object 13, a plurality of marks a are set on the calibration board 11, and a pattern B is provided on the object 13, where the environmental image includes a plurality of suspected patterns C, and the suspected patterns C may be patterns corresponding to the marks a and/or patterns corresponding to the patterns B. In the present disclosure, a pattern corresponding to the mark a needs to be identified in the suspected pattern as a calibration pattern, so as to accurately calibrate the camera parameter.
Wherein fig. 1 is only an exemplary application scenario, and the embodiments of the present disclosure may be applied to any calibration of camera parameters. The embodiments of the present disclosure are not limited to specific application scenarios.
Fig. 2 is a flowchart illustrating steps of a calibration pattern recognition method according to an exemplary embodiment of the present disclosure. The method specifically comprises the following steps:
s201, acquiring an environment image.
The environment image is shot by a camera.
In the present disclosure, the environmental image may be obtained by photographing a calibration plate, and the first case is: the environment image includes: the mark on the calibration plate corresponds to the pattern and the pattern in the background where the calibration plate is located. The second case is: the environment image does not comprise an object corresponding to the calibration plate, and only comprises an object corresponding to the background where the calibration plate is located. In the third case, the environment image only includes the object corresponding to the calibration plate. In the present disclosure, an example is illustrated in the first case.
Referring to fig. 1, the object 13 is a background corresponding to the calibration plate 11, and the pattern B on the object 13 has an interference on the mark a.
In fig. 1, the environmental image is 12, and the environmental image 12 includes a plurality of suspected patterns C. In the actual shooting process, the outlines of the marks in the calibration plate are deformed in the environment image due to different shooting angles of the cameras. For example, if the shape of the mark is circular, the pattern corresponding to the mark in the environment image becomes elliptical. If the sign is rectangular, the pattern corresponding to the sign in the environmental image is deformed into a quadrangle.
S202, extracting a plurality of suspected patterns in the environment image.
Wherein the outline of the suspected pattern accords with a preset outline. In the present disclosure, if the sign is circular, the shape of the preset contour may be set to be elliptical, and if the sign is rectangular, the shape of the preset contour may be set to be quadrangular. And extracting a plurality of suspected patterns conforming to the preset outline from the environment image according to the preset outline. Specifically, a plurality of suspected patterns are extracted, and the coordinate position of each suspected pattern in the environment image is determined.
Illustratively, referring to fig. 3, a distribution map 21 of a plurality of suspected patterns C extracted from the environmental image 12 in fig. 1 is shown. Wherein each pattern in the profile 21 is a suspected pattern conforming to a predetermined contour, the suspected pattern comprising: the target pattern 211 corresponding to the mark on the calibration plate and the background pattern 212 corresponding to the background to which the calibration plate belongs.
S203, extracting the suspected patterns which accord with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to be the calibration patterns.
In the present disclosure, the arrangement mode of the marks on the calibration board is preset and known, where the calibration patterns can be extracted from the multiple suspected patterns in any mode, and since the extracted calibration patterns conform to the arrangement mode of the marks on the calibration board, the accurate calibration patterns can be extracted.
Referring to fig. 3, each of the pseudo patterns in the calibration image 22 is a calibration pattern, which corresponds to the symbol a in fig. 1 one by one.
In the calibration process of the camera parameters, the extracted calibration patterns are in one-to-one correspondence with marks on the calibration plate, so that the accurate calibration of the camera parameters can be ensured, and if the calibration patterns comprise background patterns, the calibration of the camera can be influenced. In the embodiment of the disclosure, the calibration patterns extracted from the plurality of suspected patterns conform to the arrangement mode of marks on the calibration plate, so that the accuracy of determining the calibration patterns is improved, and the accuracy of calibrating the camera is further improved.
Fig. 4 is a flowchart of steps of another calibration pattern recognition method according to an exemplary embodiment of the present disclosure, specifically including the following steps:
s401, acquiring an environment image.
The environment image is shot by a camera.
The specific implementation process of this step refers to S201, and will not be described here again.
S402, scaling the environment image according to a preset proportion to obtain a target image with a preset size.
In general, if an environmental image shot by a camera is large and the amount of calculation is large, if the suspected pattern is extracted from the environmental image, the environmental image is reduced to a target image with a preset size, so that the amount of calculation for extracting the suspected pattern in the follow-up process can be reduced, and the extraction efficiency can be improved.
The preset scale and the preset size are preset, specifically, the preset size is determined according to the operation resources of the operation device (such as a server) currently executing the scheme of the disclosure, so that the obtained target image with the preset size is suitable for the operation resources, for example, if the operation resources are more, the preset size is larger, and if the operation resources are less, the preset size is smaller. In addition, the present disclosure may also extract the suspected pattern directly in the environmental image, which is not limited.
S403, extracting a plurality of suspected patterns in the target image.
The specific implementation step of extracting the plurality of suspected patterns in the target image refers to S202, and is not described herein.
S404, constructing a convex hull based on the plurality of suspected patterns.
The convex hull is polygonal, the convex hull comprises at least one center suspected pattern, each center suspected pattern corresponds to a preset number of adjacent suspected patterns, and each side of the convex hull passes through the adjacent suspected patterns.
In the method, the number of marks in each row in the calibration plate is the same, the marks in adjacent rows are arranged in a staggered mode, the number of rows of the marks is greater than or equal to 5, the number of the marks in each row is greater than or equal to 4, and the preset number is 8.
Further, in the present disclosure, the number of rows of the marks on the calibration plate may be an odd number of rows or an even number of rows, where a plurality of marks on the calibration plate in the odd number of rows are non-rotationally symmetrical, so that the calibration plate may be placed at any angle, and the environmental image obtained by photographing the calibration plate may accurately determine the calibration pattern. If the number of the lines is even, the calibration plates are required to be placed according to a preset angle, and the environment images obtained by the calibration plates can be shot to accurately determine the calibration patterns.
In the present disclosure, constructing a convex hull based on a plurality of suspected patterns, includes: determining a suspected pattern with a preset number (such as 8) of adjacent suspected patterns as a center suspected pattern; and determining a suspected pattern adjacent to the central suspected pattern as an adjacent suspected pattern, and determining a convex hull covering the central suspected pattern and the adjacent suspected pattern.
Illustratively, referring to FIG. 1, the markings on calibration plate 11 are 5 rows of 4 markings each. The resulting plurality of suspected patterns is the profile 21 in fig. 3. Wherein the target pattern is determined among a plurality of suspected patterns based on the profile 21. Referring to fig. 5, the center suspected patterns are O1 and O2, and the center suspected pattern O1 includes 8 adjacent suspected patterns P1, P2, O2, P3, P4, P5, P6, and P7, respectively. The center suspected pattern O2 includes 8 adjacent suspected patterns Q1, Q2, Q3, Q4, Q5, P3, O1 and P2, respectively. The constructed convex hull is 6-sided, and the 6 sides are r1, r2, r3, r4, r5 and r6 respectively. Wherein the sides of the convex hull pass through adjacent suspected patterns P1, Q2, Q3, Q4, Q5, P4, P5, P6, and P7.
In the present disclosure, the convex hull may also be constructed in other ways, not limited herein.
S405, determining that the pseudo patterns adjacent to the convex hull are adjacent pseudo patterns outside the convex hull.
The suspected patterns adjacent to the convex hull are adjacent to the convex hull, and the suspected patterns with the distance from the edge of the convex hull smaller than a preset threshold value are adjacent suspected patterns. Wherein, referring to fig. 5, the adjacent suspected patterns include: x1, X2, X3, X4, X5 and X6.
In an alternative embodiment, if the number of rows of the flag is equal to 5, determining that the suspicious pattern adjacent to the convex hull is an adjacent suspicious pattern outside the convex hull includes: and determining the suspected patterns on the extension lines of each side of the convex hull as adjacent suspected patterns.
Specifically, the first suspected pattern on the extension line of each side of the convex hull may be determined to be an adjacent suspected pattern. Illustratively, referring to fig. 5, adjacent suspected patterns X1 and X2 are on the extension line of side r1, adjacent suspected pattern X4 is on the extension line of side r2, adjacent suspected pattern X3 is on the extension line of side r3, and adjacent suspected patterns X5 and X6 are on the extension line of side r 4.
For example, referring to fig. 6, the marks of the calibration plate 61 are 5 rows, each row has 5 marks, the image 62 is the image of the calibration plate 61 corresponding to the image including the suspected pattern, and the method includes: background pattern 621, center suspected pattern 622, adjacent suspected pattern 623, adjacent suspected pattern 624, and edge 625 of the convex hull in the suspected pattern. Wherein the adjacent suspected patterns 624 are suspected patterns on the extension of the edge 625 of the convex hull. It will be seen that the calibration pattern may also be determined for the calibration plate 61 using the method described above.
In another alternative embodiment, if the number of lines of the flag is greater than 5, determining that the suspicious pattern adjacent to the convex hull is an adjacent suspicious pattern outside the convex hull includes: determining suspected patterns on extension lines of each side of the convex hull, wherein the suspected patterns on the extension lines of the suspected patterns of the target row in the convex hull are adjacent suspected patterns, and the suspected patterns of the target row comprise: the second row of the convex hull is suspected of being patterned and the second last row is suspected of being patterned.
For example, referring to fig. 7, the marks of the calibration plate 71 are 6 rows, each row has 6 marks, and the image 72 is an image containing a suspected pattern corresponding to the calibration plate 71, wherein the image 72 includes: background patterns 721, center suspected patterns 722, adjacent suspected patterns 723, adjacent suspected patterns 724, and edges 725 of the convex hull, the noted lines 726 of the second row of marks and the noted lines 727 of the penultimate row of marks of the convex hull, wherein the adjacent suspected patterns 724 include: the suspected pattern on the extension of edge 725 of the convex hull, and the suspected pattern on the extension of the label line 726 and the label line 727.
In the present disclosure, the second and penultimate row of marks of the convex hull may be determined based on the known arrangement of marks on the calibration plate, as well as the determined convex hull.
For the case that the mark of the calibration plate is 7 rows, the corresponding heptagon of the convex hull is shown in fig. 5, and the determination parameters of the calibration pattern corresponding to the calibration plate are not described herein.
In the present disclosure, when the mark of the calibration plate is greater than or equal to 8 rows, the corresponding convex hulls are all octagons.
As another example, referring to fig. 8, the marks of the calibration plate 81 are 8 rows, the corresponding convex hulls are octagons, each row of marks is 5, and the image 82 is an image containing a suspected pattern corresponding to the calibration plate 81, wherein the image 82 includes: background pattern 821, center suspected pattern 822, adjacent suspected pattern 823, adjacent suspected pattern 824, and edge 825 of the convex hull, label line 826 of the second-to-last line label and label line 827 of the second-to-last line label of the convex hull, wherein adjacent suspected pattern 824 comprises: a suspected pattern on the extension of edge 825 of the convex hull, and a suspected pattern on the extension of label line 826 and label line 827. In fig. 8, the convex hull is 8-sided.
Still another exemplary, referring to fig. 9, the calibration plate 91 has 9 rows of marks, the convex hull has 8 polygons, each row has 5 marks, and the image 92 is an image corresponding to the calibration plate 91 and containing a suspected pattern, where the image 92 includes: background pattern 921, center suspected pattern 922, adjacent suspected pattern 923, adjacent suspected pattern 924, and edge 925 of the convex hull, label line 926 of the second-to-last line of marks and label line 927 of the second-to-last line of marks in the convex hull, wherein adjacent suspected pattern 924 includes: the edge 925 of the convex hull has a pseudo-pattern on the extension of the line, and the label lines 926 and 927 have a pseudo-pattern on the extension of the line.
S406, determining a calibration pattern.
Wherein, the calibration pattern includes: a center dummy pattern, an adjacent dummy pattern, and an adjacent dummy pattern.
In the present disclosure, the marks on the calibration plate may be set to be greater than or equal to 5 rows, and the number of marks in each row is odd or even, where the arrangement manner of the marks on the calibration plate is known. After the center, adjacent, and adjacent suspected patterns are determined, the determined center, adjacent, and adjacent suspected patterns may be determined as calibration patterns.
S407, determining the corresponding pattern of the calibration pattern in the environment image as a target calibration pattern.
In the present disclosure, since the calibration pattern is determined in the target image after the scaling of the environmental image, after the calibration pattern is determined, the corresponding pattern of the calibration pattern in the environmental image is determined as the target calibration pattern, and the target calibration pattern is taken as the pattern for calibrating the camera parameters.
S408, verifying the calibration patterns by adopting the principle of cross ratio invariance according to the distance between the marks in the calibration plate and the distance between the calibration patterns.
In the present disclosure, the distance between adjacent marks in the same row on the calibration plate is known. The line spacing between adjacent line flags is also known. Therefore, after determining the calibration patterns, the distance between adjacent calibration patterns can be obtained, and the calibration patterns are verified according to the distance between the marks corresponding to the calibration patterns in the calibration plate.
Illustratively, referring to fig. 5, for the calibration patterns X5, Q5, P4, and X6, the distance of the calibration patterns X5 and Q5 on the calibration plate is D1 and the distance in the environment image is D1. The distance between the calibration pattern Q5 and the calibration pattern P4 on the calibration plate is D2, and the distance in the environment image is D2. The distance between the calibration pattern P4 and the calibration pattern X6 on the calibration plate is D3, the distance on the environmental image is D3, and the intermediate-cross ratio in the calibration plate is identical to that in the environmental image because the cross ratio is not changed under the projective transformation (imaging process), that is, the equation (d1+d2)/(d2×d1+d2+d3) is satisfied= (d1+d2) × (d2+d3)/(d2×d1+d2+d3)). The equation can thus be used to verify each determined calibration pattern.
S409, determining the calibration pattern conforming to the principle of invariance of the cross ratio as a target calibration pattern.
In the present disclosure, by performing a test on the principle of cross-ratio invariance on each target pattern, the determined target calibration pattern is a pattern conforming to the arrangement of marks on the calibration board, and the target calibration pattern can be used for calibration of camera parameters.
Illustratively, referring to fig. 5, the target pattern includes: a suspected pattern O1, a suspected pattern O2, suspected patterns P1 to P7, suspected patterns Q1 to Q5, suspected patterns X1 to X6, and a suspected pattern Y. After the cross ratio invariance principle verification, the obtained target calibration pattern comprises: the pattern may include a pattern O1, a pattern O2, a pattern P1 to P7, a pattern Q1 to Q5, and a pattern X1 to X6.
S410, fitting the calibration pattern by adopting a sub-pixel contour detection technology and an ellipse fitting technology to obtain a target calibration pattern.
Wherein the mark is circular. Specifically, after a camera shoots a mark at the center of a circle, a mark object corresponding to the mark in an image is obtained and is elliptical or elliptical, and the outline of a suspected pattern obtained after the mark object is extracted is not smooth, so that after a target pattern is determined, a subpixel outline detection technology and an ellipse fitting technology are adopted to fit the outline of a calibration pattern, and a target calibration pattern with a smooth outline is obtained.
In the present disclosure, S407, S408, S409, S410 may select at least one of them to combine to process the target pattern, so as to obtain a target calibration pattern for calibrating the camera parameters.
The embodiment of the disclosure is applied to a calibration scene of camera parameters, and the environment image is obtained by shooting by a camera by acquiring the environment image; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting the suspected patterns which are in line with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to obtain the calibration patterns which are in line with the mark arrangement mode on the calibration plate, and eliminating the suspected patterns on the background of the calibration plate, thereby improving the robustness of the identification of the calibration patterns.
In addition, the present disclosure further provides a calibration method for calibrating camera parameters, where the calibration method includes a calibration pattern recognition method in the foregoing embodiment, and after extracting, from a plurality of suspected patterns, a suspected pattern that conforms to a pattern arrangement manner on a calibration board as a calibration pattern, the method further includes: and calibrating the camera parameters by adopting marks and calibration patterns on the calibration plate.
Specifically, after the target calibration patterns are determined, each target calibration pattern can be matched with the mark on the calibration plate according to the shape of the convex hull, so that the matching relation between each mark and each target calibration pattern is obtained, then the first coordinates of the marks on the calibration plate and the second coordinates of the target calibration patterns in the environment image are determined, and further the camera parameters can be calibrated according to the matching relation, the first coordinates and the second coordinates.
One of the characteristic marks can be determined as an initial mark for the calibration plates in the odd lines, one of the two symmetrical characteristic marks can be selected as the initial mark for the calibration plates in the even lines, then the target calibration pattern corresponding to the initial mark is determined, and then the matching relation between each mark and the target calibration pattern can be determined in a clockwise rotation or anticlockwise rotation or arrangement sequence mode according to the position relation between other marks and the initial mark and the target calibration pattern corresponding to the other target calibration patterns and the initial mark.
For example, referring to fig. 1 and 5, the mark A1 in fig. 1 may be determined as a start mark, the mark A1 is matched with the target calibration pattern Q3 in fig. 5, and then the matching relationship between the target calibration patterns Q4, Q5, P4, P5, P6, P7, P1, Q1 and Q2 and each mark on the calibration board is determined according to the clockwise direction of Q3 on the convex hull in fig. 5 and the clockwise direction of the mark A1 on the convex hull in fig. 1, and then the matching relationship between other target calibration patterns and each mark on the calibration board is determined according to the determined matching relationship. The matching relation between each target calibration pattern and the mark can be determined by constructing a convex hull, and the specific determination mode is not limited herein.
In an embodiment of the present disclosure, referring to fig. 10, in addition to providing a calibration pattern recognition method, a calibration pattern recognition apparatus 100 is provided for performing any one of the calibration pattern recognition methods, which specifically includes: an acquisition module 101, a first extraction module 102 and a second extraction module 103, wherein:
an acquisition module 101, configured to acquire an environmental image, where the environmental image is captured by a camera;
the first extraction module 102 is configured to extract a plurality of suspected patterns in the environmental image, where a contour of the suspected patterns conforms to a preset contour;
the second extraction module 103 is configured to extract, from the plurality of suspected patterns, a suspected pattern that conforms to the arrangement of the marks on the calibration board as a calibration pattern.
In an alternative embodiment, the calibration plate includes: the plurality of identical marks, the marks of adjacent rows are staggered, and the second extraction module 103 is specifically configured to: constructing a convex hull based on a plurality of suspected patterns, wherein the convex hull is polygonal, the convex hull comprises at least one center suspected pattern, each center suspected pattern corresponds to a preset number of adjacent suspected patterns, and each side of the convex hull passes through the adjacent suspected patterns; determining that the suspected patterns adjacent to the convex hull are adjacent suspected patterns outside the convex hull; determining a calibration pattern, the calibration pattern comprising: a center dummy pattern, an adjacent dummy pattern, and an adjacent dummy pattern.
In an alternative embodiment, the number of marks in each row in the calibration plate is the same, the number of rows of marks is greater than or equal to 5, the number of marks in each row is greater than or equal to 4, and the preset number is 8.
In an alternative embodiment, if the number of rows of the flag is equal to 5, the second extraction module 103 is specifically configured to, when it is determined that the pseudo-pattern adjacent to the convex hull is an adjacent pseudo-pattern, determine that the pseudo-pattern adjacent to the convex hull is outside the convex hull: and determining the suspected patterns on the extension lines of each side of the convex hull as adjacent suspected patterns.
In an alternative embodiment, if the number of rows of the flag is greater than 5, the second extraction module 103 is specifically configured to, when it is determined that the pseudo-pattern adjacent to the convex hull is an adjacent pseudo-pattern, determine that the pseudo-pattern adjacent to the convex hull is outside the convex hull: determining suspected patterns on extension lines of each side of the convex hull, wherein the suspected patterns on the extension lines of the suspected patterns of the target row in the convex hull are adjacent suspected patterns, and the suspected patterns of the target row comprise: the second row of the convex hull is suspected of being patterned and the second last row is suspected of being patterned.
In an alternative embodiment, the first extraction module 102 is specifically configured to: scaling the environment image according to a preset proportion to obtain a target image with a preset size; extracting a plurality of suspected patterns in the target image; the pattern recognition device 100 further comprises a determining module (not shown) configured to determine, from among the plurality of suspected patterns, a pattern corresponding to the calibration pattern in the environmental image after extracting the corresponding suspected pattern belonging to the mark on the calibration plate as the calibration pattern as the target calibration pattern.
In an alternative embodiment, the determining module is further configured to, after extracting, from the plurality of suspected patterns, a suspected pattern that conforms to the arrangement of the marks on the calibration board as the calibration pattern: verifying the calibration patterns by adopting the principle of cross ratio invariance according to the distance between marks in the calibration plate and the distance between the calibration patterns; and determining the calibration pattern conforming to the principle of cross ratio invariance as a target calibration pattern.
In an alternative embodiment, the mark is circular, and the determining module is further configured to, after extracting, from the plurality of suspected patterns, a suspected pattern that conforms to the arrangement of the marks on the calibration board as a calibration pattern: and fitting the calibration pattern by adopting a subpixel contour detection technology and an ellipse fitting technology to obtain a target calibration pattern.
The calibration pattern recognition device provided by the disclosure can be applied to calibration scenes of camera parameters, and the environment images are obtained by shooting by a camera; extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline; and extracting the suspected patterns which are in line with the mark arrangement mode on the calibration plate from the plurality of suspected patterns to obtain the calibration patterns which are in line with the mark arrangement mode on the calibration plate, and eliminating the suspected patterns on the background of the calibration plate, thereby improving the robustness of the identification of the calibration patterns.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations appearing in a particular order are included, but it should be clearly understood that the operations may be performed out of order or performed in parallel in the order in which they appear herein, merely for distinguishing between the various operations, and the sequence number itself does not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
Fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. As shown in fig. 11, the electronic device 110 includes: a processor 111, and a memory 112 communicatively coupled to the processor 111, the memory 112 storing computer-executable instructions.
The processor executes the computer-executed instructions stored in the memory to implement the calibration pattern recognition method or the calibration method provided in any of the above method embodiments, and specific functions and technical effects that can be implemented are not described herein.
The embodiment of the disclosure also provides a computer readable storage medium, in which computer executable instructions are stored, which when executed by a processor are used to implement the calibration pattern recognition method or the calibration method provided in any of the above method embodiments.
The disclosed embodiments also provide a computer program product comprising: computer program, the computer program is stored in a readable storage medium, and at least one processor of the electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to execute the calibration pattern recognition method or the calibration method provided by any one of the method embodiments.
In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative, e.g., the partitioning of elements is merely a logical functional partitioning, and there may be additional partitioning in actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not implemented. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, system or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform part of the steps of the methods of the various embodiments of the disclosure. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the system is divided into different functional modules to perform all or part of the functions described above. The specific working process of the system described above may refer to the corresponding process in the foregoing method embodiment, and will not be described herein.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (11)
1. A calibration pattern recognition method, comprising:
acquiring an environment image, wherein the environment image is obtained by shooting with a camera;
extracting a plurality of suspected patterns in the environment image, wherein the outline of the suspected patterns accords with a preset outline;
constructing a convex hull based on the multiple suspected patterns, wherein the convex hull is polygonal, the convex hull comprises at least one center suspected pattern, each center suspected pattern corresponds to a preset number of adjacent suspected patterns, and each side of the convex hull passes through the adjacent suspected patterns;
determining that the suspected patterns adjacent to the convex hull are adjacent suspected patterns outside the convex hull;
determining the calibration pattern, the calibration pattern comprising: the center suspected pattern, the adjacent suspected patterns and the adjacent suspected patterns, the calibration pattern is a suspected pattern conforming to the arrangement mode of marks on the calibration plate, and the calibration plate comprises: the marks of adjacent rows are staggered.
2. The calibration pattern recognition method according to claim 1, wherein the number of marks in each row in the calibration plate is the same, the number of rows of marks is greater than or equal to 5, the number of marks in each row is greater than or equal to 4, and the preset number is 8.
3. The method for identifying a calibration pattern according to claim 2, wherein if the number of lines of the flag is equal to 5, the determining that the suspected pattern outside the convex hull and adjacent to the convex hull is an adjacent suspected pattern includes: and determining the suspected patterns on the extension lines of each side of the convex hull as the adjacent suspected patterns.
4. The method for identifying a calibration pattern according to claim 2, wherein if the number of lines of the mark is greater than 5, the determining that the suspected pattern outside the convex hull and adjacent to the convex hull is an adjacent suspected pattern includes:
determining the suspected patterns on the extension lines of each side of the convex hull, and determining the suspected patterns on the extension lines of the suspected patterns of the target row in the convex hull as the adjacent suspected patterns, wherein the suspected patterns of the target row comprise: and the suspected patterns of the second row and the suspected patterns of the last-to-last row of the convex hull.
5. The calibration pattern recognition method according to any one of claims 1 to 4, wherein the extracting a plurality of suspected patterns in the environment image includes:
scaling the environment image according to a preset proportion to obtain a target image with a preset size;
extracting a plurality of suspected patterns in the target image;
said determining said calibration pattern further comprises, after said determining said calibration pattern: and determining the corresponding pattern of the calibration pattern in the environment image as a target calibration pattern.
6. The calibration pattern recognition method according to any one of claims 1 to 4, characterized by further comprising, after the determination of the calibration pattern:
verifying the calibration patterns by adopting a principle of cross ratio invariance according to the distance between marks in the calibration plate and the distance between the calibration patterns;
and determining a calibration pattern conforming to the principle of cross ratio invariance as a target calibration pattern.
7. The method of any one of claims 1 to 4, wherein the mark is circular, and the determining the calibration pattern further comprises:
and fitting the calibration pattern by adopting a subpixel contour detection technology and an ellipse fitting technology to obtain a target calibration pattern.
8. A calibration method, comprising: the calibration pattern recognition method according to any one of claims 1 to 7, further comprising, after the determination of the calibration pattern:
and calibrating the camera parameters by adopting the marks on the calibration plate and the calibration patterns.
9. A calibration pattern recognition device for performing the calibration pattern recognition method according to any one of claims 1 to 7, characterized in that the calibration pattern recognition device comprises:
the acquisition module is used for acquiring an environment image, wherein the environment image is obtained by shooting with a camera;
the first extraction module is used for extracting a plurality of suspected patterns in the environment image, and the outline of the suspected patterns accords with a preset outline;
the second extraction module is used for constructing a convex hull based on the multiple suspected patterns, the convex hull is polygonal, the convex hull comprises at least one center suspected pattern, each center suspected pattern corresponds to a preset number of adjacent suspected patterns, and each side of the convex hull passes through the adjacent suspected patterns; determining that the suspected patterns adjacent to the convex hull are adjacent suspected patterns outside the convex hull; determining the calibration pattern, the calibration pattern comprising: the center suspected pattern, the adjacent suspected patterns and the adjacent suspected patterns, the calibration pattern is a suspected pattern conforming to the arrangement mode of marks on the calibration plate, and the calibration plate comprises: the marks of adjacent rows are staggered.
10. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the calibration pattern recognition method according to any one of claims 1 to 7 or the calibration method according to claim 8 when the computer program is executed by the processor.
11. A computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, which when executed by a processor are adapted to implement the calibration pattern recognition method of any one of claims 1 to 7 or the calibration method of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211091005.0A CN115457144B (en) | 2022-09-07 | 2022-09-07 | Calibration pattern recognition method, calibration device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211091005.0A CN115457144B (en) | 2022-09-07 | 2022-09-07 | Calibration pattern recognition method, calibration device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115457144A CN115457144A (en) | 2022-12-09 |
| CN115457144B true CN115457144B (en) | 2023-08-15 |
Family
ID=84302176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211091005.0A Active CN115457144B (en) | 2022-09-07 | 2022-09-07 | Calibration pattern recognition method, calibration device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115457144B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015201707A (en) * | 2014-04-07 | 2015-11-12 | 日本放送協会 | Display pattern generation apparatus, and program for the same |
| CN109271937A (en) * | 2018-09-19 | 2019-01-25 | 深圳市赢世体育科技有限公司 | Athletic ground Marker Identity method and system based on image procossing |
| CN110288655A (en) * | 2019-06-28 | 2019-09-27 | 深圳市同为数码科技股份有限公司 | The method and device of test pattern position in a kind of automatic identification chart picture |
| CN110827357A (en) * | 2019-09-30 | 2020-02-21 | 深圳市安思疆科技有限公司 | Combined pattern calibration board and structured light camera parameter calibration method |
| CN111191557A (en) * | 2019-12-25 | 2020-05-22 | 深圳市优必选科技股份有限公司 | Mark identification positioning method, mark identification positioning device and intelligent equipment |
| CN112614188A (en) * | 2020-12-07 | 2021-04-06 | 上海交通大学 | Dot-matrix calibration board based on cross ratio invariance and identification method thereof |
| CN113052910A (en) * | 2019-12-27 | 2021-06-29 | 杭州海康机器人技术有限公司 | Calibration guiding method and camera device |
| CN113052911A (en) * | 2019-12-27 | 2021-06-29 | 杭州海康机器人技术有限公司 | Calibration board, camera calibration method and device |
-
2022
- 2022-09-07 CN CN202211091005.0A patent/CN115457144B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015201707A (en) * | 2014-04-07 | 2015-11-12 | 日本放送協会 | Display pattern generation apparatus, and program for the same |
| CN109271937A (en) * | 2018-09-19 | 2019-01-25 | 深圳市赢世体育科技有限公司 | Athletic ground Marker Identity method and system based on image procossing |
| CN110288655A (en) * | 2019-06-28 | 2019-09-27 | 深圳市同为数码科技股份有限公司 | The method and device of test pattern position in a kind of automatic identification chart picture |
| CN110827357A (en) * | 2019-09-30 | 2020-02-21 | 深圳市安思疆科技有限公司 | Combined pattern calibration board and structured light camera parameter calibration method |
| CN111191557A (en) * | 2019-12-25 | 2020-05-22 | 深圳市优必选科技股份有限公司 | Mark identification positioning method, mark identification positioning device and intelligent equipment |
| CN113052910A (en) * | 2019-12-27 | 2021-06-29 | 杭州海康机器人技术有限公司 | Calibration guiding method and camera device |
| CN113052911A (en) * | 2019-12-27 | 2021-06-29 | 杭州海康机器人技术有限公司 | Calibration board, camera calibration method and device |
| CN112614188A (en) * | 2020-12-07 | 2021-04-06 | 上海交通大学 | Dot-matrix calibration board based on cross ratio invariance and identification method thereof |
Non-Patent Citations (1)
| Title |
|---|
| Self-identifying patterns for plane-based camera calibration;Mark Fiala et.al;《Machine Vision and Applications volume》;第209-216页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115457144A (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109754426B (en) | Method, system and device for verifying camera calibration parameters | |
| CN112132907B (en) | Camera calibration method and device, electronic equipment and storage medium | |
| CN111179356A (en) | Binocular camera calibration method, device and system based on Aruco code and calibration board | |
| CN113240740B (en) | Attitude measurement method based on phase-guided binocular vision dense marking point matching | |
| CN111145271B (en) | Method and device for determining accuracy of camera parameters, storage medium and terminal | |
| CN104766309A (en) | Plane feature point navigation and positioning method and device | |
| CN111754536B (en) | Image labeling method, device, electronic equipment and storage medium | |
| CN110956660A (en) | Positioning method, robot, and computer storage medium | |
| CN110148099B (en) | Projection relation correction method and device, electronic equipment and computer readable medium | |
| CN110035280B (en) | Method and device for detecting module FOV by using checkerboard test table and readable storage medium | |
| CN114387347B (en) | Method, device, electronic equipment and medium for determining external parameter calibration | |
| CN110774319A (en) | Robot and positioning method and device thereof | |
| CN111681186A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN113465573A (en) | Monocular distance measuring method and device and intelligent device | |
| CN115201883A (en) | Moving target video positioning and speed measuring system and method | |
| CN111699513B (en) | Calibration plate, internal parameter calibration method, machine vision system and storage device | |
| CN110570354A (en) | Strip chessboard calibration plate-based close-range image splicing method | |
| CN115457144B (en) | Calibration pattern recognition method, calibration device and electronic equipment | |
| CN107945136B (en) | Fisheye image correction method, fisheye image correction system, fisheye image correction equipment and computer storage medium | |
| CN101894369B (en) | Real-time method for computing focal length of camera from image sequence | |
| CN114187363A (en) | Method and device for obtaining radial distortion parameter value and mobile terminal | |
| CN111860498B (en) | Method, device and storage medium for generating antagonism sample of license plate | |
| US20230245346A1 (en) | Method and device for corner marking and parameter calibration | |
| CN116503486A (en) | Calibration system, calibration method, calibration device and readable storage medium | |
| CN115719387A (en) | 3D camera calibration method, point cloud image acquisition method and camera calibration system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |