CN112991376B - Equipment contour labeling method and system in infrared image - Google Patents
Equipment contour labeling method and system in infrared image Download PDFInfo
- Publication number
- CN112991376B CN112991376B CN202110369188.7A CN202110369188A CN112991376B CN 112991376 B CN112991376 B CN 112991376B CN 202110369188 A CN202110369188 A CN 202110369188A CN 112991376 B CN112991376 B CN 112991376B
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- offset
- infrared
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 57
- 239000011159 matrix material Substances 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 230000001133 acceleration Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000009529 body temperature measurement Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 101100452236 Caenorhabditis elegans inf-1 gene Proteins 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000004568 cement Substances 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method and a system for labeling equipment contours in infrared images, wherein the method comprises the following steps: collecting visible light images and infrared images of equipment in a current state; matching the visible light image of the equipment in the current state with the visible light template image to obtain a visible light image feature matching matrix; calculating a first offset and a second offset of the visible light image of the device in the current state; performing equipment contour labeling on a pre-stored infrared template image to obtain coordinates of a plurality of labeling points; and determining the coordinates of a plurality of points on the outline of the device in the infrared image of the device in the current state according to the relation between the coordinates of a plurality of marking points, the first offset, the second offset, the offset generated by the rotation of the visible light image along with the rotation of the holder and the offset generated by the rotation of the infrared image along with the rotation of the holder. The invention assists the infrared image to mark the outline of the tested equipment by means of the visible light image, thereby ensuring the accuracy of the outline marking of the industrial field temperature measuring equipment.
Description
Technical Field
The invention relates to the technical field of image processing and machine vision, in particular to a method and a system for labeling equipment outlines in an infrared image.
Background
In industrial sites, such as power plants or cement plants, there are often a large number of devices, such as motors or cable joints, which need to be temperature measured at any time, in order to ensure that the devices do not experience various anomalies due to excessive temperatures, such as failure to operate properly, danger, fire or even explosion. At present, two temperature measurement modes are mainly adopted, one is that a worker carries out field temperature measurement on equipment needing temperature measurement to judge whether the equipment normally operates, and because high temperature, high pressure, high noise and high radiation are commonly existing in an industrial field, the worker is in the severe environment for a long time and has great damage to the body, in order to avoid exposing the worker in the harmful environment, the other temperature measurement mode is adopted for carrying out temperature measurement through infrared image temperature measurement equipment at present, and in order to distinguish the equipment to be measured, the infrared image temperature measurement equipment is required to acquire infrared images for contour detection and drawing.
The inventor finds that when the infrared image temperature measuring equipment detects the outline of the tested equipment, the outline of the tested equipment is not obvious in the image due to the low resolution ratio of the infrared image and the influence of the surrounding environment, particularly when the temperature of the tested equipment is close to the temperature of the surrounding environment, the tested equipment displayed by the infrared image is fuzzy, the infrared image temperature measuring equipment cannot directly detect the outline of the equipment in the infrared image, the outline of the tested equipment cannot be drawn, and the tested equipment cannot be further accurately distinguished.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person of ordinary skill in the art.
Disclosure of Invention
The invention aims to provide a device contour labeling method and a system in an infrared image, which are used for labeling the contour of a tested device by means of a visible light image to assist a low-resolution infrared image, so that the accuracy of contour labeling of industrial field temperature measuring devices is ensured.
In order to achieve the above object, the present invention provides a method for labeling an outline of a device in an infrared image, including: the method comprises the steps that a collecting device collects visible light images and infrared images of equipment in a current state, wherein a cradle head is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the cradle head; matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix; calculating a first offset of a visible light image of the device in the current state relative to the visible light template image in a transverse axis direction of an image coordinate system based on the visible light image feature matching matrix, and calculating a second offset of the visible light image of the device in the current state relative to the visible light template image in a longitudinal axis direction of the image coordinate system based on the visible light image feature matching matrix; performing equipment contour labeling on a pre-stored infrared template image to obtain coordinates of a plurality of labeling points; and determining coordinates of a plurality of points on an equipment outline in an infrared image of the equipment in the current state according to the relation among coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset and the offset generated by the rotation of a preset visible light image along with a holder and the offset generated by the rotation of the infrared image along with the holder, wherein the visible light image and the visible light template image are acquired under the condition that the acquisition device is at the same position and the shooting times of the visible light camera are the same times, and the infrared image and the infrared template image are acquired by the acquisition device at the same position.
In an embodiment of the present invention, the device profile labeling method further includes: the method comprises the steps of presetting a relation between offset generated by a visible light image rotating along with a holder and offset generated by an infrared image rotating along with the holder, wherein the relation between the offset generated by the visible light image rotating along with the holder and the offset generated by the infrared image rotating along with the holder comprises the following steps: determining the offset of the visible light image in the horizontal axis direction and the offset of the visible light image in the vertical axis direction of an image coordinate system when the cradle head rotates for each degree, and determining the offset of the infrared image in the horizontal axis direction and the offset of the infrared image in the vertical axis direction of the image coordinate system when the cradle head rotates for each degree; determining a first ratio between the offset of the infrared image in the transverse axis direction of an image coordinate system when the cradle head rotates for one degree and the offset of the visible light image in the transverse axis direction of the image coordinate system when the cradle head rotates for one degree; and determining a second ratio between the offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the holder and the offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the holder.
In an embodiment of the present invention, the matching the visible light image of the device in the current state with the pre-stored visible light template image to obtain a visible light image feature matching matrix includes: and matching the visible light image of the equipment in the current state with the pre-stored visible light template image based on an acceleration robust feature algorithm to obtain a visible light image feature matching matrix.
In one embodiment of the present invention, calculating the first offset and calculating the second offset includes: acquiring a central coordinate of the visible light template image under an image coordinate system, and carrying out coordinate transformation on the central coordinate of the visible light template image under the image coordinate system based on the visible light image characteristic matching matrix to obtain a transformed coordinate; subtracting the coordinate value of the horizontal axis of the center coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and subtracting the vertical axis coordinate value of the central coordinate on the visible light template image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset.
In an embodiment of the present invention, determining coordinates of a plurality of points on an equipment outline in an infrared image of the equipment in the current state according to a relationship between coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset, an offset generated by a preset visible light image rotating along with a cradle head, and an offset generated by an infrared image rotating along with the cradle head includes: determining coordinates of points on the device outline on the infrared image according to a first formula, wherein the first formula is: d (X, Y) = (m+x sw×Off_Rel_X,N+Ysh X off_rel_y), wherein D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, M, N is an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
Based on the same inventive concept, the invention also provides a device contour labeling system in an infrared image, which comprises: the device comprises a collecting device, a matching module, an offset calculation module, a labeling module and a contour coordinate determining module. The device comprises a collecting device, wherein a cradle head is arranged on the collecting device, a visible light camera and an infrared camera are arranged on the cradle head, and the collecting device is used for collecting visible light images and infrared images of equipment in the current state. The matching module is coupled with the acquisition device and is used for matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix. The offset calculation module is coupled with the matching module and is used for calculating a first offset of the visible light image of the device in the current state relative to the visible light template image in the transverse axis direction of an image coordinate system based on the visible light image feature matching matrix, and is also used for calculating a second offset of the visible light image of the device in the current state relative to the visible light template image in the longitudinal axis direction of the image coordinate system based on the visible light image feature matching matrix. The labeling module is used for labeling the equipment outline on a pre-stored infrared template image to obtain coordinates of a plurality of labeling points. The outline coordinate determining module is coupled with the labeling module and the offset calculating module and is used for determining coordinates of a plurality of points on the outline of the device in the infrared image of the device in the current state according to the relation among the coordinates of a plurality of labeling points on the infrared template image, the first offset, the second offset, the offset generated by the preset visible light image along with the rotation of the holder and the offset generated by the infrared image along with the rotation of the holder. The visible light image and the visible light template image are collected under the condition that the collecting device is at the same position and the shooting times of the visible light camera are the same times, and the infrared image and the infrared template image are collected by the collecting device at the same position.
In an embodiment of the present invention, the device profile labeling system further includes: the offset relation determining module is coupled with the contour coordinate determining module and is used for determining the relation between the offset of the visible light image generated by rotation of the holder and the offset of the infrared image generated by rotation of the holder, and the offset relation determining module is used for determining the offset of the visible light image in the horizontal axis direction and the offset in the vertical axis direction of the image coordinate system for each rotation of the holder and determining the offset of the infrared image in the horizontal axis direction and the offset in the vertical axis direction of the image coordinate system for each rotation of the holder; the offset relation determining module is used for determining a first ratio between the offset of the infrared image in the transverse axis direction of the image coordinate system when the cradle head rotates for one degree and the offset of the visible light image in the transverse axis direction of the image coordinate system when the cradle head rotates for one degree; the offset relation determining module is further configured to determine a second ratio between an offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the pan-tilt and an offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the pan-tilt.
In an embodiment of the present invention, the offset calculating module is configured to obtain a center coordinate of the visible light template image under an image coordinate system, and perform coordinate transformation on the center coordinate of the visible light template image under the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate; the offset calculating module is used for subtracting the coordinate value of the transverse axis of the center coordinate on the visible light template image from the coordinate value of the transverse axis of the transformed coordinate to obtain the first offset; the offset amount calculation module is further configured to subtract the vertical axis coordinate value of the center coordinate on the visible light module image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset amount.
In an embodiment of the present invention, the contour coordinate determining module is configured to determine coordinates of a point on a device contour on the infrared image according to a first formula, where the first formula is: d (X, Y) = (m+x sw×Off_Rel_X,N+Ysh X off_rel_y), wherein D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, M, N is an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
Based on the same inventive concept, the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the device contour labeling method in an infrared image according to any of the above embodiments.
Compared with the prior art, according to the equipment contour labeling method and system in the infrared image, in the equipment contour labeling process of the infrared image, the equipment in the infrared image with low resolution is subjected to auxiliary labeling by utilizing the acquisition device with the visible light camera, the infrared camera and the cloud deck to acquire the visible light image, the infrared image and the respective template images at the same position and under the same visible light camera multiple, and the problems that the resolution ratio of the visible light image is relatively high, the relation between the offset of the visible light image and the infrared image, the offset of the visible light image in the current state relative to the visible light template image, the relation between the acquired infrared template image and the infrared image and the like are utilized, so that the contour of the equipment in the infrared image can be accurately labeled, the problem that the target equipment to be measured in the infrared image is fuzzy when the resolution ratio of the infrared image is low or the target equipment to be measured is close to the ambient temperature is effectively solved, the problem that the target equipment to be measured in the infrared image is difficult to directly contour the infrared image, the method and the system in the field is suitable for drawing the equipment with high requirements for the field robustness, and the system in the method is suitable for the field temperature measurement.
Drawings
FIG. 1 is a method for device contour labeling in an infrared image according to one embodiment of the invention;
FIG. 2 is a device profile annotation system in an infrared image according to one embodiment of the invention;
FIG. 3 is a system for device contour labeling in an infrared image in accordance with an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention is, therefore, to be taken in conjunction with the accompanying drawings, and it is to be understood that the scope of the invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the term "comprise" or variations thereof such as "comprises" or "comprising", etc. will be understood to include the stated element or component without excluding other elements or components.
Aiming at the problems that the existing industrial field infrared image temperature measuring equipment is affected by low resolution of infrared images and surrounding environment and cannot accurately detect and draw the outline of the tested equipment, in one embodiment, a method for labeling equipment outline in the infrared images is provided, as shown in fig. 1, the method for labeling equipment outline comprises steps S1-S5. Firstly, it should be noted that, in the present specification, the visible light image and the visible light template image are both acquired under the condition that the acquisition device is at the same position and the shooting times of the visible light camera are all the same times, and the infrared image and the infrared template image are both acquired by the acquisition device at the same position.
In step S1, a collecting device collects a visible light image and an infrared image of a device in a current state, wherein a cradle head is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the cradle head. Preferably, the acquisition device can be a robot, and equipment can be inspected at any time according to requirements.
In step S2, the visible light image of the device in the current state is matched with a pre-stored visible light template image of the device, so as to obtain a visible light image feature matching matrix.
Optionally, an acceleration robust feature algorithm may be used to match the visible light image of the device in the current state with the pre-stored visible light template image to obtain a 2×3 visible light image feature matching matrix
In step S3, a first offset of the visible light image of the device in the current state with respect to the visible light template image in the horizontal axis direction of the image coordinate system is calculated based on the visible light image feature matching matrix, and a second offset of the visible light image of the device in the current state with respect to the visible light template image in the vertical axis direction of the image coordinate system is calculated based on the visible light image feature matching matrix.
Alternatively, the method of calculating the first offset amount and the second offset amount in step S3 is as follows. Firstly, acquiring the center coordinates of the visible light template image under an image coordinate systemWhere Width is the Width of the template image and Height is the Height of the template image. Then, based on the visible light image feature matching matrix, carrying out coordinate transformation on the central coordinate of the visible light template image under an image coordinate system to obtain transformed coordinates And finally, subtracting the coordinate value of the horizontal axis of the central coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset/>And subtracting the vertical axis coordinate value of the central coordinate on the visible light template image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset/>
And in the step S4, equipment outline marking is carried out on the pre-stored infrared template image, and coordinates of a plurality of marking points are obtained. The shape of the outline marking and the number of marking points can be determined according to the complexity of the outline. For example, a quadrilateral or other polygon may be annotated. In this embodiment, a quadrangle is labeled according to the shape of the device, and four vertex coordinates, D Left upper part (a,b),D Upper right (c,d),D Lower left (e,f),D Lower right (g, h), are obtained.
In step S5, coordinates of a plurality of points on the device outline in the infrared image of the device in the current state are determined according to the coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset, and the relation between the offset generated by the rotation of the preset visible light image along with the rotation of the cradle head and the offset generated by the rotation of the infrared image along with the rotation of the cradle head.
Specifically, the relationship between the offset amount of the visible light image generated as the pan/tilt head rotates and the offset amount of the infrared image generated as the pan/tilt head rotates is predetermined. In an alternative embodiment, determining the relationship between the offsets includes: firstly, determining the offset of the visible light image in the horizontal axis direction and the offset in the vertical axis direction of an image coordinate system when the cradle head rotates for each degree, and determining the offset of the infrared image in the horizontal axis direction and the offset in the vertical axis direction of the image coordinate system when the cradle head rotates for each degree; then determining a first ratio between the offset of the infrared image in the transverse axis direction of the image coordinate system for each rotation of the holder and the offset of the visible light image in the transverse axis direction of the image coordinate system for each rotation of the holder; and determining a second ratio between the offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the holder and the offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the holder.
In order to reduce errors and obtain a relatively accurate relation between offset values, and further obtain an accurate contour detection result, preferably, when determining the offset value of the visible light image or the infrared image when the cradle head rotates once, the cradle head can be rotated for multiple times, and multiple groups of images are collected to calculate the offset value. If the cradle head can be adjusted for 5 times, after each rotation for 1 degree, the visible light image and the infrared image are collected once, and the offset is calculated once, so that the offset x vis1、xvis2、xvis3、xvis4、xvis5 of the 5 visible light images in the horizontal axis direction of the image coordinate system and the offset y vis1、yvis2、yvis3、yvis4、yvis5 of the 5 visible light images in the vertical axis direction of the image coordinate system are obtained in total, and the offset x inf1、xinf2、xinf3、xinf4、xinf5 of the 5 infrared images in the horizontal axis direction of the image coordinate system and the offset y inf1、yinf2、yinf3、yinf4、yinf5 of the 5 infrared images in the vertical axis direction of the image coordinate system are also obtained. Then calculating the offset of the visible light image in the horizontal axis direction of the image coordinate system when the cradle head rotates once as followsAnd calculating the offset of the visible light image in the longitudinal axis direction of the image coordinate system when the cradle head rotates once as/>Calculating the offset of the infrared image in the horizontal axis direction of the image coordinate system at every rotation as/>Calculating the offset of the infrared image in the longitudinal axis direction of the image coordinate system at each rotation degree asThe first ratio is calculated as off_rel_x=x inf_ave/xvis_ave and the second ratio is calculated as off_rel_y=y inf_ave/yvis_ave.
Specifically, in step 5, coordinates of a point on the device outline on the infrared image are determined according to a first formula, where the first formula is: d (X, Y) = (m+x sw×Off_Rel_X,N+Ysh X off_rel_y), wherein D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, M, N is an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
Since four vertex coordinates are acquired in step S4, D Left upper part (a,b),D Upper right (c,d),D Lower left (e,f),D Lower right (g, h), respectively. And obtaining coordinates of four points on the equipment outline on the infrared image according to the first formula, wherein the coordinates of the four points are four vertex coordinates on the equipment outline on the infrared image, and drawing the outline of equipment in the infrared image according to the calculated four vertex coordinates on the equipment outline on the infrared image after D Left upper part new＝(a+Xsw*Off_Rel_X,b+Ysh*Off_Rel_Y);D Upper right new＝(c+Xsw*Off_Rel_X,d+Ysh*Off_Rel_Y);D Lower left new＝(e+Xsw*Off_Rel_X,f+Ysh*Off_Rel_Y);D Lower right new＝(g+Xsw*Off_Rel_X,h+Ysh*Off_Rel_Y)..
In this embodiment, in the process of labeling the outline of the device for the infrared image, the device for the infrared image with low resolution is used for auxiliary labeling, by using the acquisition device with the visible light camera, the infrared camera and the cradle head, the visible light image, the infrared image and the template images are acquired at the same position and under the same multiple of the visible light camera, the problem that the target device displayed by the infrared image is fuzzy when the resolution of the infrared image is low or the temperature of the target device to be measured is close to the ambient temperature is solved, and the problem that the outline of the target region is difficult to be drawn directly on the infrared image is solved by using the characteristics of relatively higher resolution of the visible light image, the relation between the offset of the visible light image and the infrared image, the relation between the offset of the visible light image and the visible light template image in the current state, the relation between the acquired infrared template image and the infrared image, and the like.
Based on the same inventive concept, an embodiment further provides a device contour labeling system in an infrared image, as shown in fig. 2, where the device contour labeling system in an infrared image includes: the device comprises an acquisition device 10, a matching module 11, an offset calculation module 12, a labeling module 13 and a contour coordinate determination module 14.
The collecting device 10 is provided with a cradle head, a visible light camera and an infrared camera are arranged on the cradle head, and the collecting device 10 is used for collecting visible light images and infrared images of equipment in a current state. Alternatively, the acquisition device 10 may be a robot.
The matching module 11 is coupled to the collecting device 10, and is configured to match the visible light image of the device in the current state with a pre-stored visible light template image of the device, so as to obtain a feature matching matrix of the visible light image. Optionally, an acceleration robust feature algorithm may be employed to match the visible light image of the device in the current state with the pre-stored visible light template image,
An offset calculating module 12 is coupled to the matching module 11, and is configured to calculate, based on the visible light image feature matching matrix, a first offset of the visible light image of the device in the current state with respect to the visible light template image in a direction of a horizontal axis of an image coordinate system, and the offset calculating module 12 is further configured to calculate, based on the visible light image feature matching matrix, a second offset of the visible light image of the device in the current state with respect to the visible light template image in a direction of a vertical axis of the image coordinate system.
Alternatively, the process of calculating the first offset and the second offset by the offset calculation module 12 is as follows: acquiring a central coordinate of the visible light template image under an image coordinate system, and carrying out coordinate transformation on the central coordinate of the visible light template image under the image coordinate system based on the visible light image characteristic matching matrix to obtain a transformed coordinate; subtracting the coordinate value of the horizontal axis of the center coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and subtracting the vertical axis coordinate value of the central coordinate on the visible light template image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset.
The labeling module 13 is used for labeling the equipment outline on a pre-stored infrared template image, and obtaining coordinates of a plurality of labeling points.
The outline coordinate determining module 14 is coupled to the labeling module 13 and the offset calculating module 12, and is configured to determine coordinates of a plurality of points on an outline of the device in the infrared image of the device in the current state according to a relationship between coordinates of a plurality of labeling points on the infrared template image, the first offset, the second offset, an offset generated by a preset visible light image rotating with a cradle head, and an offset generated by an infrared image rotating with the cradle head.
Optionally, in an embodiment, the device profile labeling system further includes: the offset relation determination module 15 is shown in fig. 3. The offset relation determining module 15 is coupled to the contour coordinate determining module 14, and is configured to determine a relation between an offset generated by the rotation of the visible light image along with the rotation of the pan-tilt and an offset generated by the rotation of the infrared image along with the rotation of the pan-tilt, where the offset relation determining module 15 is specifically configured to determine an offset of the visible light image in a horizontal axis direction and an offset of the infrared image in a vertical axis direction of the image coordinate system for each rotation of the pan-tilt, and determine an offset of the infrared image in the horizontal axis direction and an offset of the infrared image in the vertical axis direction of the image coordinate system for each rotation of the pan-tilt; the offset relation determining module 15 is configured to determine a first ratio between an offset of the infrared image in a transverse axis direction of the image coordinate system for each rotation of the pan-tilt and an offset of the visible light image in the transverse axis direction of the image coordinate system for each rotation of the pan-tilt; the offset relation determining module 15 is further configured to determine a second ratio between an offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the pan-tilt and an offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the pan-tilt.
Specifically, the contour coordinate determining module 14 is further configured to determine coordinates of a point on a contour of the device on the infrared image according to a first formula, where the first formula is: d (X, Y) = (a+x sw×Off_Rel_X,b+Ysh X off_rel_y), where D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, a and b are an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
Based on the same inventive concept, there is also provided in an embodiment a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the apparatus contour labeling method according to any of the above embodiments.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention are presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application to thereby enable one skilled in the art to make and utilize the invention in various exemplary embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.
Claims (6)
1. The equipment contour labeling method in the infrared image is characterized by comprising the following steps of:
The method comprises the steps that a collecting device collects visible light images and infrared images of equipment in a current state, wherein a cradle head is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the cradle head;
Matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix;
Calculating a first offset of a visible light image of the device in the current state relative to the visible light template image in a transverse axis direction of an image coordinate system based on the visible light image feature matching matrix, and calculating a second offset of the visible light image of the device in the current state relative to the visible light template image in a longitudinal axis direction of the image coordinate system based on the visible light image feature matching matrix;
Performing equipment contour labeling on a pre-stored infrared template image to obtain coordinates of a plurality of labeling points; and
Determining coordinates of a plurality of points on the equipment outline in the infrared image of the equipment in the current state according to the coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset and the relation between the offset generated by the rotation of the infrared image along with the rotation of the holder and the offset generated by the rotation of the infrared image along with the rotation of the holder,
The visible light image and the visible light template image are acquired under the condition that the acquisition device is at the same position and the shooting times of the visible light camera are the same times, and the infrared image and the infrared template image are acquired by the acquisition device at the same position;
The equipment contour labeling method further comprises the following steps: the method comprises the steps of presetting a relation between offset generated by a visible light image rotating along with a holder and offset generated by an infrared image rotating along with the holder, wherein the relation between the offset generated by the visible light image rotating along with the holder and the offset generated by the infrared image rotating along with the holder comprises the following steps:
Determining the offset of the visible light image in the horizontal axis direction and the offset of the visible light image in the vertical axis direction of an image coordinate system when the cradle head rotates for each degree, and determining the offset of the infrared image in the horizontal axis direction and the offset of the infrared image in the vertical axis direction of the image coordinate system when the cradle head rotates for each degree;
Determining a first ratio between the offset of the infrared image in the transverse axis direction of an image coordinate system when the cradle head rotates for one degree and the offset of the visible light image in the transverse axis direction of the image coordinate system when the cradle head rotates for one degree; and
Determining a second ratio between the offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the holder and the offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the holder;
Determining coordinates of a plurality of points on an equipment outline in an infrared image of the equipment in the current state according to the relation between coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset and offset generated by rotating a preset visible light image along with a holder and offset generated by rotating the infrared image along with the holder comprises the following steps:
determining coordinates of points on the device outline on the infrared image according to a first formula, wherein the first formula is: d (X, Y) = (m+x) sw Off_Rel_X，N+Ysh/>Off_rel_y), wherein D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, M, N is an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
2. The method for labeling the device outline in the infrared image according to claim 1, wherein the step of matching the visible light image of the device in the current state with the pre-stored visible light template image to obtain a visible light image feature matching matrix comprises:
and matching the visible light image of the equipment in the current state with the pre-stored visible light template image based on an acceleration robust feature algorithm to obtain a visible light image feature matching matrix.
3. The method of device contour labeling in an infrared image as defined in claim 1, wherein calculating the first offset and calculating the second offset comprises:
acquiring a central coordinate of the visible light template image under an image coordinate system, and carrying out coordinate transformation on the central coordinate of the visible light template image under the image coordinate system based on the visible light image characteristic matching matrix to obtain a transformed coordinate;
Subtracting the coordinate value of the horizontal axis of the center coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and
And subtracting the vertical axis coordinate value of the central coordinate on the visible light template image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset.
4. A device profile annotation system in an infrared image, the device profile annotation system comprising:
The device comprises a collecting device, a camera and a camera module, wherein the collecting device is provided with a cradle head, a visible light camera and an infrared camera are arranged on the cradle head, and the collecting device is used for collecting a visible light image and an infrared image of equipment in a current state;
The matching module is coupled with the acquisition device and is used for matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix;
An offset calculating module, coupled to the matching module, for calculating, based on the visible light image feature matching matrix, a first offset of the visible light image of the device in the current state with respect to the visible light template image in a horizontal axis direction of an image coordinate system, and for calculating, based on the visible light image feature matching matrix, a second offset of the visible light image of the device in the current state with respect to the visible light template image in a vertical axis direction of the image coordinate system;
The marking module is used for marking the equipment outline on a pre-stored infrared template image to obtain coordinates of a plurality of marking points; and
The contour coordinate determining module is coupled with the labeling module and the offset calculating module and is used for determining the coordinates of a plurality of points on the contour of the equipment in the infrared image of the equipment in the current state according to the relation among the coordinates of a plurality of labeling points on the infrared template image, the first offset, the second offset, the offset generated by the preset visible light image along with the rotation of the holder and the offset generated by the infrared image along with the rotation of the holder,
The visible light image and the visible light template image are acquired under the condition that the acquisition device is at the same position and the shooting times of the visible light camera are the same times, and the infrared image and the infrared template image are acquired by the acquisition device at the same position;
The equipment outline marking system in the infrared image further comprises: the offset relation determining module is coupled with the contour coordinate determining module and is used for determining the relation between the offset of the visible light image generated by rotating the holder and the offset of the infrared image generated by rotating the holder in advance, wherein the relation between the offset of the visible light image generated by rotating the holder and the offset of the infrared image generated by rotating the holder in advance comprises the following steps:
Determining the offset of the visible light image in the horizontal axis direction and the offset of the visible light image in the vertical axis direction of an image coordinate system when the cradle head rotates for each degree, and determining the offset of the infrared image in the horizontal axis direction and the offset of the infrared image in the vertical axis direction of the image coordinate system when the cradle head rotates for each degree;
Determining a first ratio between the offset of the infrared image in the transverse axis direction of an image coordinate system when the cradle head rotates for one degree and the offset of the visible light image in the transverse axis direction of the image coordinate system when the cradle head rotates for one degree; and
Determining a second ratio between the offset of the infrared image in the longitudinal axis direction of the image coordinate system for each rotation of the holder and the offset of the visible light image in the longitudinal axis direction of the image coordinate system for each rotation of the holder;
Determining coordinates of a plurality of points on an equipment outline in an infrared image of the equipment in the current state according to the relation between coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset and offset generated by rotating a preset visible light image along with a holder and offset generated by rotating the infrared image along with the holder comprises the following steps:
The contour coordinate determining module is used for determining coordinates of points on the equipment contour on the infrared image according to a first formula, wherein the first formula is as follows: d (X, Y) = (m+x) sw Off_Rel_X，N+Ysh/>Off_rel_y), wherein D is a point on the device outline on the infrared image, X, Y is an abscissa and an ordinate of the D point in the image coordinate system, M, N is an abscissa and an ordinate of a certain labeling point in the image coordinate system, X sw is the first offset, Y sh is the second offset, off_rel_x is the first ratio, and off_rel_y is the second ratio.
5. The device profile labeling system in an infrared image according to claim 4, wherein the offset calculation module is configured to obtain a center coordinate of the visible light template image in an image coordinate system, and perform coordinate transformation on the center coordinate of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate; the offset calculating module is used for subtracting the coordinate value of the transverse axis of the center coordinate on the visible light template image from the coordinate value of the transverse axis of the transformed coordinate to obtain the first offset; the offset amount calculation module is further configured to subtract the vertical axis coordinate value of the center coordinate on the visible light module image from the vertical axis coordinate value of the transformed coordinate to obtain the second offset amount.
6. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the device contour labeling method in an infrared image as claimed in any of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369188.7A CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369188.7A CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112991376A CN112991376A (en) | 2021-06-18 |
| CN112991376B true CN112991376B (en) | 2024-06-14 |
Family
ID=76339222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110369188.7A Active CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112991376B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115914792A (en) * | 2022-12-22 | 2023-04-04 | 长春理工大学 | Real-time multidimensional imaging self-adaptive adjustment system and method based on deep learning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902962A (en) * | 2012-12-28 | 2014-07-02 | 汉王科技股份有限公司 | Shielding or light source self-adaption human face recognition method and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101695728B1 (en) * | 2010-12-21 | 2017-01-23 | 엘지디스플레이 주식회사 | Display system including stereo camera and position detecting method using the same |
| CN109801336B (en) * | 2019-01-09 | 2022-09-06 | 南京理工大学 | Airborne target positioning system and method based on visible light and infrared light vision |
| CN110570454B (en) * | 2019-07-19 | 2022-03-22 | 华瑞新智科技（北京）有限公司 | Method and device for detecting foreign matter invasion |
| CN111274940A (en) * | 2020-01-19 | 2020-06-12 | 厦门中控智慧信息技术有限公司 | Face recognition method, device, equipment and storage medium |
| CN111429518B (en) * | 2020-03-24 | 2023-10-03 | 浙江大华技术股份有限公司 | Labeling method, labeling device, computing equipment and storage medium |
| CN112200840B (en) * | 2020-10-27 | 2021-05-11 | 北京深睿博联科技有限责任公司 | Moving object detection system in visible light and infrared image combination |
-
2021
- 2021-04-06 CN CN202110369188.7A patent/CN112991376B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902962A (en) * | 2012-12-28 | 2014-07-02 | 汉王科技股份有限公司 | Shielding or light source self-adaption human face recognition method and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112991376A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8744133B1 (en) | Methods and systems for locating visible differences on an object | |
| US20150239043A1 (en) | Cast Features for Location and Inspection | |
| EP3023913A1 (en) | Crack data collection method and crack data collection program | |
| US20150116481A1 (en) | Mapping damaged regions on objects | |
| US10598481B2 (en) | Crankshaft shape inspection apparatus, system and method | |
| JP2012042396A (en) | Position attitude measurement device, position attitude measurement method, and program | |
| CN106127758A (en) | A kind of visible detection method based on virtual reality technology and device | |
| CN106815866B (en) | Calibration method of fisheye camera, calibration system and target thereof | |
| CN104457577A (en) | Machine-vision-oriented non-contact type workpiece positioning and measuring method | |
| CN107271445B (en) | Defect detection method and device | |
| CN112476489B (en) | Flexible mechanical arm synchronous measurement method and system based on natural characteristics | |
| CN112991376B (en) | Equipment contour labeling method and system in infrared image | |
| CN111627070B (en) | Method, device and storage medium for calibrating rotation shaft | |
| CN107367245B (en) | Invalid point detection and elimination method in optical three-dimensional profile measurement | |
| JP5976089B2 (en) | Position / orientation measuring apparatus, position / orientation measuring method, and program | |
| JP2002132341A (en) | Field inspection device | |
| CN113983951B (en) | Three-dimensional target measuring method, device, imager and storage medium | |
| KR20150058882A (en) | Apparatus and method for motion capture using inertial sensor and optical sensor | |
| CN106643566A (en) | Method of automatically measuring initial angle of spokes of wheel hub | |
| CN109540030B (en) | Self-positioning precision detection method for handheld scanning equipment | |
| CN106123808A (en) | A kind of method measured for the deflection of automobile rearview mirror specular angle degree | |
| JPH10318732A (en) | Shape measuring device and image formation apparatus of shape measurement | |
| CN116124010B (en) | Visual displacement measurement system and method | |
| Uyanik et al. | A method for determining 3D surface points of objects by a single camera and rotary stage | |
| JP2006153654A (en) | Three-dimensional measurement instrument and three-dimensional measurement method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |