CN115376026A

CN115376026A - Key area positioning method, device, equipment and storage medium

Info

Publication number: CN115376026A
Application number: CN202211008846.0A
Authority: CN
Inventors: 刘梓权; 余佳莹; 颜大涵; 李露琼; 杨康宜; 陈文旭; 许哲源; 林佳润; 黄凯漩; 李博雯
Original assignee: Guangdong Power Grid Co Ltd; Shantou Power Supply Bureau of Guangdong Power Grid Co Ltd
Current assignee: Guangdong Power Grid Co Ltd; Shantou Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority date: 2022-08-22
Filing date: 2022-08-22
Publication date: 2022-11-22

Abstract

The invention discloses a method, a device, equipment and a storage medium for positioning a key area. The method comprises the following steps: acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting point positions; wherein the candidate template image has image association information; the image correlation information comprises position coordinates of key areas of the candidate template images; determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images; matching feature points of the infrared image to be positioned and the target template image to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; the first feature point and the second feature point are matched feature points; and determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image. The embodiment of the invention improves the positioning accuracy of the key area.

Description

Key area positioning method, device, equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for locating a key area.

Background

In order to monitor the temperature conditions of various power equipment of the transformer substation, the patrol unmanned aerial vehicle of the transformer substation can regularly fly to a preset point position to shoot infrared images of the transformer substation equipment. Through carrying out automatic analysis to the infrared image of substation equipment that unmanned aerial vehicle gathered, the hot defect that equipment probably exists can in time be found. However, since the infrared image of the substation equipment acquired by the unmanned aerial vehicle contains background elements (such as sky, ground or building) irrelevant to the analysis of the substation equipment, when the infrared image of the substation equipment is automatically analyzed, the infrared image needs to be located to a key area containing the substation equipment, so that the interference of the background elements is avoided, and the defect analysis and identification are performed on the temperature of the key area.

For the positioning of the key area of the infrared image, there are two main ways at present: one is to locate the key area based on the machine learning algorithm; the other two are matching and positioning based on the template image. The positioning result of the first mode has great contingency, and the positioning accuracy rate depends on the quality of training data. The positioning result of the second method has a certain displacement deviation, so that the positioning of the key area is not accurate enough.

Disclosure of Invention

The invention provides a method, a device, equipment and a storage medium for positioning a key area, which are used for reducing the contingency of a positioning result of the key area and improving the positioning accuracy of the key area.

According to an aspect of the present invention, there is provided a method for locating a critical area, the method including:

acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting points; wherein the candidate template image has image association information; the image correlation information comprises position coordinates of key areas of the candidate template images;

determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images;

matching feature points of the infrared image to be positioned and the target template image to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; wherein the first feature point and the second feature point are matched feature points;

and determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image.

According to another aspect of the present invention, there is provided a key area locating apparatus, the apparatus comprising:

the infrared image acquisition module is used for acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting point positions; wherein the candidate template image has image association information; the image correlation information comprises position coordinates of key areas of the candidate template images;

the template image determining module is used for determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images;

the characteristic point determining module is used for matching characteristic points of the target template image and the infrared image to be positioned to obtain a first characteristic point of the target template image and a second characteristic point of the infrared image to be positioned; wherein the first feature point and the second feature point are matched feature points;

and the key area determining module is used for determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the method for locating a critical area according to any embodiment of the present invention.

According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the method for locating a critical area according to any one of the embodiments of the present invention when the computer instructions are executed.

According to the embodiment of the invention, the infrared image to be positioned is obtained, and candidate template images are obtained by shooting at different shooting points; determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images; matching feature points of the target template image and the infrared image to be positioned to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; and determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image. According to the scheme, the key area is positioned in the characteristic point matching mode, the contingency of the positioning result of the key area is reduced, and the accuracy of the positioning result of the key area is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a method for locating a key area according to an embodiment of the present invention;

fig. 2 is a flowchart of a method for locating a key area according to a second embodiment of the present invention;

FIG. 3A is a flowchart of a stencil image collection and processing according to a third embodiment of the present invention;

fig. 3B is a flowchart illustrating a positioning process of a key area of an infrared image to be positioned according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of a key area positioning device according to a fourth embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device implementing the key area positioning method according to the embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example one

Fig. 1 is a flowchart of a method for locating a key area according to an embodiment of the present invention, where the method is applicable to locating a key area of an infrared image of a substation device, and the method may be executed by a key area locating device, where the key area locating device may be implemented in a form of hardware and/or software, and the key area locating device may be configured in an electronic device. As shown in fig. 1, the method includes:

s110, acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting points; wherein the candidate template image has image association information; the image association information includes position coordinates of key regions of the candidate template images.

The infrared image to be positioned can be a transformer substation infrared image to be subjected to key area positioning. The infrared image to be positioned can be acquired by image acquisition equipment. For example, the image acquisition device may be a drone.

The candidate template image may be an infrared image obtained by shooting at preset different shooting points by an image acquisition device. Illustratively, infrared images shot by the unmanned aerial vehicle of the transformer substation at least one preset patrol shooting point can be collected in advance to serve as candidate template images, and meanwhile, the shooting point sequence number of each candidate template image is recorded, so that position matching with the infrared images to be positioned is facilitated subsequently.

The image-related information may be information for characterizing image features of the candidate template image. The key area of the candidate template image may be an area including substation equipment in the image. It can be understood that the infrared image of the transformer substation shot by the unmanned aerial vehicle contains power equipment and background elements which are irrelevant to thermal defect analysis, such as sky, ground or buildings. Therefore, when defect analysis is performed on the infrared image of the unmanned aerial vehicle of the transformer substation, a key area containing the power equipment in the infrared image needs to be positioned first, so that the defect analysis and identification are performed on the key area, and the interference of background elements is avoided.

The position coordinates of the key area may be position coordinates of the key area on the template image. For example, the position coordinates of the key region may be the position coordinates of the pixel points in the key region corresponding to the template image. Optionally, the key area is usually a rectangular area, and therefore, when the position coordinates of the upper left area and the lower right area of the key area are determined, the position of the key area relative to the template image can be determined. Optionally, the position coordinates of the lower left area and the upper right area of the key area may also be determined, that is, the position of the key area relative to the template image may be determined.

For example, infrared images shot by the unmanned aerial vehicle at each preset patrol shooting point can be collected in advance as candidate template images. And meanwhile, the shooting point position serial numbers corresponding to the candidate template images are recorded, so that the candidate template images matched with the infrared image to be positioned can be selected from the candidate template images conveniently in the follow-up process. And for each candidate template image, marking the area where the electric power equipment is located in the image by using a rectangular frame as a key area of the candidate template image, and recording the abscissa and ordinate of the upper left corner and the abscissa and ordinate of the lower right corner of the rectangular frame.

And S120, determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images.

For example, the shooting point location number corresponding to the infrared image to be positioned may be determined, a candidate template image having the same shooting point location number as the infrared image to be positioned may be searched for from each candidate template image, and the determined candidate template image may be used as the target template image. Subsequently, the key area of the infrared image to be positioned can be positioned through the key area of the target template image.

S130, matching feature points of the infrared image to be positioned and the target template image to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; and the first characteristic point and the second characteristic point are matched characteristic points.

The first characteristic point and the second characteristic point can be respectively the characteristic points successfully matched in the infrared image to be positioned and the target template image. For example, a SIFT (Scale Invariant Feature Transform) algorithm may be used to identify Feature points in the infrared image to be located and the target template image, and a RANSAC (Random Sample Consensus) algorithm may be used to perform Feature matching on the Feature points in the template image and the infrared image to be located.

Illustratively, if 100 feature points exist in the infrared image to be positioned and 80 feature points exist in the target template image, each feature point in the infrared image to be positioned needs to be respectively matched with the 80 feature points in the target template image, and the matching can not be performed subsequently after the matching is successful.

In an optional embodiment, the image association information includes a description vector of at least one feature point corresponding to a predetermined candidate template image; the method for matching the characteristic points of the infrared image to be positioned and the target template image to obtain the first characteristic points of the target template image and the second characteristic points of the infrared image to be positioned comprises the following steps: performing feature extraction on the infrared image to be positioned to obtain a description vector of at least one feature point corresponding to the infrared image to be positioned; performing feature point matching on the infrared image to be positioned and the target template image according to the description vector corresponding to the infrared image to be positioned and the description vector corresponding to the target template image to obtain at least one feature point matching group; the characteristic point matching group comprises a first characteristic point of the matched target template image and a second characteristic point of the infrared image to be positioned.

The description vector of at least one feature point corresponding to each candidate template image may be predetermined. Illustratively, all feature points of the candidate template image may be extracted by the SIFT algorithm, and each feature point generates a 128-dimensional description vector.

Exemplarily, feature extraction is performed on the infrared image to be positioned based on the same feature extraction mode, so as to obtain a description vector of at least one feature point corresponding to the infrared image to be positioned. And matching the characteristic points of the infrared image to be positioned and the target template image by using a RANSAC algorithm based on the Euclidean distance between the characteristic points respectively corresponding to the infrared image to be positioned and the target template image and the description vectors corresponding to the characteristic points to obtain at least one characteristic matching group. And the first characteristic points and the second characteristic points included in the characteristic matching group are successfully matched characteristic points.

Exemplarily, if the feature points of the infrared image to be positioned are A1, A2, A3, A4, A5, A6, A7, A8, A9, and a10, respectively; the feature points of the target template image are B1, B2, B3, B4, B5, B6, B7, B8, B9, and B10, respectively. And if the feature points A1 and B4 are successfully matched, the feature points A5 and B10 are successfully matched, and the feature points A3 and B6 are successfully matched, obtaining 3 feature matching groups respectively comprising a feature matching group Z1, a feature matching group Z2 and a feature matching group Z3 after the infrared image to be positioned and the target template image are subjected to feature point matching. The characteristic matching group Z1 comprises a first characteristic point B4 and a second characteristic point A1; the characteristic matching group Z2 comprises a first characteristic point B10 and a second characteristic point A5; the feature matching group Z3 includes a first feature point B6 and a second feature point A3.

According to the optional embodiment, the feature points of the infrared image to be positioned and the feature points of the target template image are accurately matched in a mode of matching the feature points of the infrared image to be positioned and the target template image according to the description vector corresponding to the infrared image to be positioned and the description vector corresponding to the target template image, so that the subsequent accurate positioning of the key area of the infrared image to be positioned is facilitated.

It should be noted that, if the number of feature points successfully matched between the infrared image to be positioned and the target template image is small, it may be considered that the positioning of the key area of the infrared image to be positioned fails.

In an optional embodiment, after at least one characteristic point matching group is obtained, the number of the matching groups of the characteristic point matching group is determined; if the number of the matching groups is smaller than a preset number threshold, determining that a key area of the infrared image to be positioned cannot be positioned; and if the number of the matching groups is not less than the preset number threshold, determining a key area of the infrared image to be positioned according to the position coordinates of the first characteristic points, the position coordinates of the second characteristic points and the position coordinates of the key area of the target template image.

Exemplarily, the number of feature point matching groups for successfully matching the feature points between the infrared image to be positioned and the target template image is determined, if the number of the matching groups is less than a preset number threshold, a key area of the infrared image to be positioned cannot be positioned is determined, and after a prompt of 'failure in matching and incapability of positioning the key area' is output, key area positioning is finished; if the number of the matching groups is not less than the preset number threshold, the subsequent steps are continuously executed.

S140, determining a key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image.

For example, the key area of the infrared image to be positioned can be determined based on a linear regression analysis method according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image.

Example two

Fig. 2 is a flowchart of a method for positioning a critical area according to a second embodiment of the present invention, and this embodiment performs optimization and improvement based on the foregoing technical solutions.

Further, the image correlation information includes position coordinates of an upper left area and position coordinates of a lower right area corresponding to the key area of the candidate template image; correspondingly, the step of determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image is refined into the step of determining the position coordinates of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the upper left area corresponding to the key area of the target template image, the position coordinates of the first characteristic point and the position coordinates of the second characteristic point; determining a right-lower area position coordinate corresponding to a key area of the infrared image to be positioned according to a right-lower area position coordinate corresponding to a key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point; and determining the key area of the infrared image to be positioned according to the position coordinates of the upper left area and the position coordinates of the lower right area corresponding to the key area of the infrared image to be positioned. "to refine the determination of the critical area of the infrared image to be positioned.

As shown in fig. 2, the method comprises the following specific steps:

s210, acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting points; wherein the candidate template image has image association information; the image correlation information includes position coordinates of an upper left region and position coordinates of a lower right region corresponding to the key region of the candidate template image.

And S220, determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images.

S230, matching feature points of the target template image and the infrared image to be positioned to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; and the first characteristic point and the second characteristic point are matched characteristic points.

S240, determining the position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the upper left area corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point.

Illustratively, if the key area of the target template image is a rectangular area, the corresponding coordinates of the position of the upper left area are the coordinates of the position of the upper left corner of the rectangular frame. The position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned can be determined by selecting the characteristic point with the closer distance of the characteristic point of the position coordinate of the upper left area corresponding to the key area of the target template image in the first characteristic point and the second characteristic point.

In an optional embodiment, determining the position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the upper left area corresponding to the key area of the target template image, the position coordinate of the first feature point and the position coordinate of the second feature point includes: determining at least one characteristic point which is close to the position coordinate of the upper left area corresponding to the key area of the target template image from the first characteristic points as a first target characteristic point; taking the second characteristic points matched with the first target characteristic points as second target characteristic points; and determining the position coordinates of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the first target characteristic point, the position coordinates of the second target characteristic point and the position coordinates of the upper left area corresponding to the key area of the target template image.

Illustratively, if the first feature points are determined, the first target feature points which are closer to the position coordinate of the upper left region corresponding to the key region of the target template image are respectively P ₁ ,P ₂ ,……,P _n (ii) a The second target feature points matched with the first target feature points are respectively Q ₁ ,Q ₂ ,……,Q _n . Wherein, the position coordinates corresponding to the first target characteristic points are (x) ₁ ,y ₁ ),(x ₂ ,y ₂ ),……,(x _n ,y _n ) The position coordinates corresponding to the second target feature points are (u) ₁ ,u ₁ ),(u ₂ ,u ₂ ),……,(u _n ,u _n )。

Exemplarily, according to the abscissa of the first target feature point, the abscissa of the second target feature point and the abscissa of the upper left area corresponding to the key area of the target template image, the abscissa of the upper left area coordinate corresponding to the key area of the infrared image to be positioned is determined. Specifically, a linear regression analysis method is adopted to establish a linear regression equation as follows:

u＝a _zsx +b _zsx x；

wherein u represents the abscissa of the second target feature point of the infrared image to be positioned, x represents the abscissa of the first target feature point of the template image, and a _zsx And b _zsx Is the undetermined coefficient of the linear regression equation. Wherein, a _zsx And b _zsx The determination method of (2) is as follows:

wherein x is _i An abscissa representing an ith feature point of the n first target feature points; u. u _i And represents the abscissa of the ith feature point among the n second target feature points.

Making the horizontal coordinate x of the upper left corner of the rectangular frame of the key area of the template image _zs Substituting into linear regression equation to obtain the abscissa u of the upper left corner of the rectangular frame of the key region of the infrared image to be positioned _zs . Wherein u is _zs The determination method of (2) is as follows:

u _zs ＝a _zsx +b _zsx x _zs ；

illustratively, according to the ordinate of the first target feature point, the ordinate of the second target feature point and the ordinate of the upper left area corresponding to the key area of the target template image, the ordinate of the upper left area coordinate corresponding to the key area of the infrared image to be positioned is determined. Specifically, a linear regression analysis method is adopted to establish a linear regression equation as follows:

v＝a _zsy +b _zsy y；

wherein v represents the ordinate of the second target characteristic point of the infrared image to be positioned, y represents the ordinate of the first target characteristic point of the template image, and a _zsy And b _zsy Is the undetermined coefficient of the linear regression equation. Wherein, a _zsy And b _zsy The determination of (2) is as follows:

wherein, y _i Representing the ordinate of the ith characteristic point in the n first target characteristic points; v. of _i And the ordinate of the ith characteristic point in the n second target characteristic points is represented.

The ordinate y of the upper left corner of the rectangular frame of the key area of the template image _zs Substituting the linear regression equation into the key area to obtain the ordinate v of the upper left corner of the rectangular frame of the key area of the infrared image to be positioned _zs . Wherein v is _zs The determination method of (2) is as follows:

v _zs ＝a _zsy +b _zsy y _zs ；

and S250, determining the position coordinate of the right lower area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the right lower area corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point.

For example, if the key area of the target template image is a rectangular area, the corresponding coordinates of the lower-right area are the coordinates of the lower-right corner of the rectangular frame. The position coordinates of the right-lower area corresponding to the key area of the infrared image to be positioned can be determined by selecting the feature point with the closer distance to the feature point of the position coordinate of the right-lower area corresponding to the key area of the target template image in the first feature point and the second feature point.

In an optional embodiment, determining, according to the position coordinates of the lower right area, the position coordinates of the first feature point, and the position coordinates of the second feature point, the position coordinates of the lower right area corresponding to the key area of the infrared image to be located, includes: determining at least one characteristic point which is close to the position coordinate of a lower right area corresponding to a key area of the target template image from the first characteristic points to serve as a third target characteristic point; taking the second feature points matched with the third target feature points as fourth target feature points; and determining the position coordinates of the right lower area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the third target characteristic point, the position coordinates of the fourth target characteristic point and the position coordinates of the right lower area corresponding to the key area of the target template image.

Illustratively, if the third target feature points determined from the first feature points and having a shorter distance with the position coordinates of the corresponding lower right region of the key region of the target template image are respectively M ₁ ,M ₂ ,……,M _n (ii) a The fourth target feature points matched with the third target feature points are respectively Y ₁ ,Y ₂ ,……,Y _n . Wherein the position coordinates respectively corresponding to the third target feature points are (m) ₁ ,m ₁ ),(m ₂ ,m ₂ ),……,(m _n ,m _n ) The position coordinates respectively corresponding to the fourth target feature points are (y) ₁ ,y ₁ ),(y ₂ ,y ₂ ),……,(y _n ,y _n )。

Exemplarily, the abscissa of the coordinate of the right lower region corresponding to the key region of the to-be-positioned infrared image is determined according to the abscissa of the third target feature point, the abscissa of the fourth target feature point and the abscissa of the right lower region corresponding to the key region of the target template image. Specifically, a linear regression analysis method is adopted to establish a linear regression equation as follows:

u＝a _yxx +b _yxx x；

wherein u represents the abscissa of the third target feature point of the infrared image to be positioned, x represents the abscissa of the fourth target feature point of the template image, and a _yxx And b _yxx Is the undetermined coefficient of a linear regression equation, in which a _yxx And b _yxx The determination method of (2) is as follows:

wherein x is _i An abscissa representing an ith feature point of the n third target feature points; u. of _i And the abscissa representing the ith feature point among the n fourth target feature points.

The abscissa x of the lower right corner of a rectangular frame of a key area of the template image is measured _yx Substituting the linear regression equation into the coordinate of the lower right corner of the rectangular frame of the key area of the infrared image to be positioned _yx . Wherein u is _yx The determination of (2) is as follows:

u _yx ＝a _yxx +b _yxx x _yx ；

exemplarily, the ordinate of the coordinate of the right-lower area corresponding to the key area of the infrared image to be positioned is determined according to the abscissa of the third target feature point, the abscissa of the fourth target feature point and the ordinate of the right-lower area corresponding to the key area of the target template image. Specifically, a linear regression analysis method is adopted to establish a linear regression equation as follows:

v＝a _yxy +b _yxy y；

wherein v represents the ordinate of the fourth target characteristic point of the infrared image to be positioned, y represents the ordinate of the third target characteristic point of the template image, and a _yxy And b _yxy Is the undetermined coefficient of a linear regression equation, wherein a _yxy And b _yxy The determination method of (2) is as follows:

wherein, y _i Representing the ordinate of the ith feature point in the n third target feature points; v. of _i And the ordinate of the ith feature point in the n fourth target feature points is represented.

The ordinate y of the lower right corner of the rectangular frame of the key area of the template image _yx Substituting the linear regression equation into the infrared image to calculate the horizontal coordinate v of the lower right corner of the rectangular frame of the key area of the infrared image to be positioned _yx . Wherein v is _yx The determination method of (2) is as follows:

v _yx ＝a _yxy +b _yxy y _yx ；

s260, determining the key area of the infrared image to be positioned according to the position coordinates of the upper left area and the position coordinates of the lower right area corresponding to the key area of the infrared image to be positioned.

In an optional embodiment, after determining the key area of the infrared image to be positioned, the method further includes: and checking the area range and/or the area position of the key area of the infrared image to be positioned according to the position distance between the position coordinate of the key area of the infrared image to be positioned and the position coordinate of the key area of the target template image.

Coordinates (u) at the upper left corner of a rectangular frame of a key area for determining an infrared image to be positioned _zs ,v _zs ) And coordinates of lower right corner (u) _yx ,v _yx ) Then, checking the key area of the infrared image to be positioned, and judging the key area to be positionedAnd (3) whether the upper left corner coordinate and the lower right corner coordinate of the key area rectangular frame of the infrared image satisfy the following relation is determined:

u _zs <u _yx ；

and if the upper left corner coordinate and the lower right corner coordinate of the rectangular frame of the key area of the infrared image to be positioned do not meet the relationship, outputting a prompt of 'mapping fails and key area cannot be positioned', and then ending the positioning of the key area of the infrared image to be positioned.

And if the upper left corner coordinate and the lower right corner coordinate of the rectangular frame of the key area of the infrared image to be positioned meet the relationship, continuously checking the size and the position of the area of the key area of the infrared image to be positioned.

It should be noted that the template image and the infrared image to be positioned are obtained by shooting at the same shooting point by the unmanned aerial vehicle of the transformer substation, and although a certain displacement deviation exists between the template image and the infrared image to be positioned due to a positioning accuracy error of the unmanned aerial vehicle, the displacement deviation does not exceed a reasonable range. Therefore, the size and position difference of the rectangular frames of the key areas of the template image and the infrared image to be positioned are within a reasonable range. Therefore, the template image and the key area rectangular frame of the infrared image to be positioned are compared and checked.

And comparing and checking the size of the rectangular frame of the key area of the infrared image to be positioned. And (3) setting the length of the template image as l and the width as w, and judging whether the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively simultaneously satisfy the following relations:

if the key area rectangular frame coordinates respectively corresponding to the template image and the infrared image to be positioned simultaneously satisfy the relationship, the verification of the size comparison verification of the key area rectangular frames of the template image and the infrared image to be positioned is considered to be passed, and the subsequent position verification is continuously carried out; and if the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively do not satisfy the relationship, outputting a prompt of 'failure of verification and incapability of positioning the key area', and ending the positioning of the key area of the infrared image to be positioned.

And comparing and checking the positions of the key area rectangular frames of the infrared images to be positioned. And (3) setting the length of the template image as l and the width as w, and judging whether the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively simultaneously satisfy the following relations:

if the coordinates of the key area rectangular frames corresponding to the template image and the infrared image to be positioned respectively simultaneously satisfy the relationship, the verification of position comparison verification on the key area rectangular frames of the template image and the infrared image to be positioned is considered to be passed; and if the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively do not satisfy the relationship, outputting a prompt of 'failure of verification and incapability of positioning the key area', and ending the positioning of the key area of the infrared image to be positioned.

After the comparison check of the size and the position of the template image and the key area rectangular frame of the infrared image to be positioned is passed, according to the upper left corner coordinate (u) of the key area rectangular frame of the infrared image to be positioned _zs ,v _zs ) And coordinates of lower right corner (u) _yx ,v _yx ) And outputting a key area rectangular frame of the infrared image to be positioned, namely realizing the positioning of the key area of the infrared image to be positioned.

According to the scheme of the embodiment, the position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned is determined according to the position coordinate of the upper left area corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point; and determining the position coordinate of the right lower area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the right lower area corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point, thereby further improving the positioning accuracy of the key area of the infrared image to be positioned.

EXAMPLE III

The present embodiment further provides a preferred embodiment based on the above embodiments, and the overall idea of the present embodiment is as follows: firstly, an infrared image shot by an unmanned aerial vehicle at each shooting point is collected in advance to serve as a template image, and key region coordinate marking and feature point extraction are carried out on the template image. Secondly, after determining the infrared image to be positioned shot by the unmanned aerial vehicle, correcting deviation of the key area coordinates of the template image by using a characteristic point matching and linear regression analysis method, and mapping the coordinates into the infrared image to be positioned. And finally, comparing and checking the key area of the infrared image to be positioned and the template image, and finally positioning the key area of the infrared image to be positioned.

As shown in fig. 3A, a flow chart of template image collection and processing includes the following steps:

s1, collecting infrared images shot by the unmanned aerial vehicle of the transformer substation at each preset patrol shooting point position, taking the infrared images as template images, and recording the serial number of the shooting point position of each template image.

S2, for each template image, marking the area of the template image where the electric equipment is located by using a rectangular frame as a key area of the template image, and recording the abscissa x of the upper left corner of the rectangular frame _zs Ordinate y _zs And the abscissa x of the lower right corner _yx Ordinate y _yx 。

And S3, for each template image, extracting all feature points in the template image by using an SIFT algorithm, and generating a 128-dimensional description vector by each feature point.

As shown in fig. 3B, in the flowchart of the key region positioning of the infrared image to be positioned, after the infrared image to be positioned is obtained by the unmanned aerial vehicle, the step of performing key region positioning on the infrared image to be positioned is as follows:

s1, searching a template image with the same shooting point location serial number as the infrared image to be positioned according to the shooting point location serial number corresponding to the infrared image to be positioned, and positioning the key area of the infrared image to be positioned.

S2, extracting all feature points of the infrared image to be positioned through an SIFT algorithm, and generating a 128-dimensional description vector by each feature point.

And S3, performing feature matching on the template image and the feature points of the infrared image to be positioned by adopting a RANSAC algorithm based on the Euclidean distance between the feature point description vectors. And determining the number M of the feature points successfully matched between the infrared image to be positioned and the template image.

And S4, determining the quantity relation between the successfully matched feature point quantity M and a preset quantity threshold value. The preset quantity threshold value can be preset according to actual requirements. The present embodiment sets the number threshold to 8.

a) And if the number M of the successfully matched feature points is smaller than the preset number threshold, the fact that the difference between the infrared image to be positioned and the template image is too large is indicated, image matching is difficult to perform, and after a prompt that the matching fails and a key area cannot be positioned is output, key area positioning is finished.

b) If the number M of the successfully matched feature points is not less than the preset number threshold, 8 first feature points which are closest to the upper left corner of the rectangular frame of the key area of the template image are selected from the M successfully matched feature points and are respectively marked as P for the template image ₁ ,P ₂ ,……,P ₈ . The corresponding position coordinates are respectively noted as (x) ₁ ,y ₁ ),(x ₂ ,y ₂ ),……,(x ₈ ,y ₈ )。

S5, for the infrared image to be positioned, and P ₁ ,P ₂ ,……,P ₈ The matched second characteristic points are respectively correspondingly marked as Q ₁ ,Q ₂ ,……,Q ₈ The corresponding position coordinates are respectively noted as (u) ₁ ,u ₁ ),(u ₂ ,u ₂ ),……,(u ₈ ,u ₈ )。

S6, in order to describe the abscissa relation of the relevant matched feature points at the upper left corner of the rectangular frame of the key area of the template image and the infrared image to be positioned, a linear regression analysis method is adopted to establish a linear regression equation as follows:

u＝a _zsx +b _zsx x；

wherein u represents the abscissa of the second characteristic point of the infrared image to be positioned, x represents the abscissa of the first characteristic point of the template image, a _zsx And b _zsx Is the undetermined coefficient of the linear regression equation. Wherein, a _zsx And b _zsx The determination of (2) is as follows:

wherein x is _i The abscissa represents the ith characteristic point of the 8 first characteristic points; u. u _i The abscissa indicates the ith feature point among the 8 second feature points.

S7, enabling the abscissa x of the upper left corner of the rectangular frame of the key area of the template image to be x _zs Substituting into linear regression equation to obtain the abscissa u of the upper left corner of the rectangular frame of the key region of the infrared image to be positioned _zs . Wherein u is _zs The determination of (2) is as follows:

u _zs ＝a _zsx +b _zsx x _zs ；

s8, in order to describe the vertical coordinate relation of the relevant matching feature points at the upper left corner of the rectangular frame of the key area of the template image and the infrared image to be positioned, a linear regression analysis method is adopted to establish a linear regression equation as follows:

v＝a _zsy +b _zsy y；

wherein v represents the ordinate of the second characteristic point of the infrared image to be positioned, y represents the ordinate of the first characteristic point of the template image, and a _zsy And b _zsy Is the undetermined coefficient of the linear regression equation. Wherein, a _zsy And b _zsy The determination of (2) is as follows:

wherein, y _i Representing the ordinate of the ith characteristic point in the 8 first characteristic points; v. of _i And the ordinate of the ith feature point in the 8 second feature points is shown.

S9, enabling the ordinate y of the upper left corner of the rectangular frame of the key area of the template image _zs Substituting the linear regression equation into the key area to obtain the ordinate v of the upper left corner of the rectangular frame of the key area of the infrared image to be positioned _zs . Wherein v is _zs The determination method of (2) is as follows:

v _zs ＝a _zsy +b _zsy y _zs ；

s10, aiming at the template image, selecting key area moment of the template image from M successfully matched feature pointsThe 8 third feature points with the nearest distance from the lower right corner of the frame are respectively marked as P ₉ ,P ₁₀ ,……,P ₁₆ The corresponding position coordinates are respectively noted as (x) ₉ ,y ₉ ),(x ₁₀ ,y ₁₀ ),……,(x ₁₆ ,y ₁₆ )。

S11, for the infrared image to be positioned, and P ₉ ,P ₁₀ ,……,P ₁₆ The fourth feature points are respectively marked as Q ₉ ,Q ₁₀ ,……,Q ₁₆ The corresponding coordinates are respectively noted as (u) ₉ ,u ₉ ),(u ₁₀ ,u ₁₀ ),……,(u ₁₆ ,u ₁₆ )。

S12, in order to describe the abscissa relation of the relevant matching feature points of the template image and the lower right corner of the rectangular frame of the key area of the infrared image to be positioned, a linear regression analysis method is adopted to establish a linear regression equation as follows:

u＝a _yxx +b _yxx x；

wherein u represents the abscissa of the third characteristic point of the infrared image to be positioned, x represents the abscissa of the fourth characteristic point of the template image, a _yxx And b _yxx Is the undetermined coefficient of a linear regression equation, wherein a _yxx And b _yxx The determination method of (2) is as follows:

wherein x is _i Represents the first of 8 third feature pointsThe abscissa of the i feature points; u. u _i And the abscissa indicates the ith feature point of the 8 fourth feature points.

S13, enabling the abscissa x of the lower right corner of the rectangular frame of the key area of the template image to be x _yx Substituting the linear regression equation into the coordinate of the lower right corner of the rectangular frame of the key area of the infrared image to be positioned _yx . Wherein u is _yx The determination of (2) is as follows:

u _yx ＝a _yxx +b _yxx x _yx ；

s14, in order to describe the ordinate relationship of the relevant matching feature points of the template image and the lower right corner of the rectangular frame of the key area of the infrared image to be positioned, a linear regression analysis method is adopted to establish a linear regression equation as follows:

v＝a _yxy +b _yxy y；

wherein v represents the ordinate of the fourth characteristic point of the infrared image to be positioned, y represents the ordinate of the third characteristic point of the template image, a _yxy And b _yxy Is the undetermined coefficient of a linear regression equation, wherein a _yxy And b _yxy The determination method of (2) is as follows:

wherein, y _i Representing the ordinate of the ith feature point in the 8 third feature points; v. of _i And represents the ordinate of the ith feature point of the 8 fourth feature points.

S15, enabling the ordinate y of the lower right corner of the rectangular frame of the key area of the template image _yx Substituting the linear regression equation into the infrared image to calculate the horizontal coordinate v of the lower right corner of the rectangular frame of the key area of the infrared image to be positioned _yx . Wherein v is _yx The determination of (2) is as follows:

v _yx ＝a _yxy +b _yxy y _yx ；

s16, determining the coordinates (u) of the upper left corner of the rectangular frame of the key area of the infrared image to be positioned _zs ,v _zs ) And coordinates of lower right corner (u) _yx ,v _yx ) And then, checking the key area of the infrared image to be positioned, and judging whether the upper left corner coordinate and the lower right corner coordinate of the key area rectangular frame of the infrared image to be positioned meet the following relation:

u _zs <u _yx ；

a) And if the upper left corner coordinate and the lower right corner coordinate of the rectangular frame of the key area of the infrared image to be positioned do not meet the relationship, outputting a prompt of 'mapping failure and incapability of positioning the key area', and finishing the positioning of the key area of the infrared image to be positioned.

b) And if the upper left corner coordinate and the lower right corner coordinate of the key area rectangular frame of the infrared image to be positioned meet the relationship, continuously checking the size and the position of the area of the key area of the infrared image to be positioned.

S17, comparing and checking the size of the rectangular frame of the key area of the infrared image to be positioned. Setting the length of the template image as l and the width as w, and judging whether the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively simultaneously satisfy the following relations:

And S18, comparing and checking the positions of the key area rectangular frames of the infrared images to be positioned. And (3) setting the length of the template image as l and the width as w, and judging whether the coordinates of the rectangular frames of the key areas corresponding to the template image and the infrared image to be positioned respectively simultaneously satisfy the following relations:

S19, comparing the size and the position of the template image and the key area rectangular frame of the infrared image to be positioned, and checking the comparison result according to the upper left corner coordinate (u) of the key area rectangular frame of the infrared image to be positioned _zs ,v _zs ) And coordinates of lower right corner (u) _yx ,v _yx ) And outputting a key area rectangular frame of the infrared image to be positioned, namely positioning the key area of the infrared image to be positioned.

Example four

Fig. 4 is a schematic structural diagram of a key area positioning device according to a fourth embodiment of the present invention. The key area positioning device provided by the embodiment of the invention can be suitable for positioning the key area of the infrared image of the substation equipment, and can be realized in a hardware and/or software mode, as shown in fig. 4, and specifically comprises: an infrared image acquisition module 401, a template image determination module 402, a feature point determination module 403, and a key region determination module 404. Wherein,

the infrared image acquisition module 401 is configured to acquire an infrared image to be positioned and candidate template images obtained by shooting at different shooting points; wherein the candidate template image has image association information; the image correlation information comprises position coordinates of key areas of the candidate template images;

a template image determining module 402, configured to determine, from the candidate template images, a target template image at the same shooting point corresponding to the infrared image to be positioned;

a feature point determining module 403, configured to perform feature point matching on the target template image and the to-be-positioned infrared image to obtain a first feature point of the target template image and a second feature point of the to-be-positioned infrared image; wherein the first feature point and the second feature point are matched feature points;

a key area determining module 404, configured to determine a key area of the to-be-positioned infrared image according to the position coordinate of the first feature point, the position coordinate of the second feature point, and the position coordinate of the key area of the target template image.

According to the embodiment of the invention, the infrared image to be positioned is obtained, and candidate template images are obtained by shooting at different shooting points; determining a target template image at the same shooting point position corresponding to the infrared image to be positioned from the candidate template images; matching feature points of the target template image and the infrared image to be positioned to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; and determining the key area of the infrared image to be positioned according to the position coordinates of the first characteristic point, the position coordinates of the second characteristic point and the position coordinates of the key area of the target template image. According to the scheme, the key area is positioned in a characteristic point matching mode, the contingency of the positioning result of the key area is reduced, and the accuracy of the positioning result of the key area is improved.

Optionally, the image association information includes a predetermined description vector of at least one feature point corresponding to the candidate template image;

the feature point determining module 403 includes:

the description vector determining unit is used for extracting the characteristics of the infrared image to be positioned to obtain a description vector of at least one characteristic point corresponding to the infrared image to be positioned;

the characteristic point determining unit is used for matching characteristic points of the infrared image to be positioned and the target template image according to the description vector corresponding to the infrared image to be positioned and the description vector corresponding to the target template image to obtain at least one characteristic point matching group; the characteristic point matching group comprises a first characteristic point of the matched target template image and a second characteristic point of the infrared image to be positioned.

Optionally, the feature point determining module 403 further includes:

a matching group number determining unit, configured to determine the number of matching groups of the feature point matching group after the at least one feature point matching group is obtained;

the first key area determining unit is used for determining that the key area of the infrared image to be positioned cannot be positioned if the number of the matching groups is smaller than a preset number threshold;

and the second key area determining unit is used for determining the key area of the infrared image to be positioned according to the position coordinate of the first characteristic point, the position coordinate of the second characteristic point and the key area position coordinate of the target template image if the number of the matching groups is not less than a preset number threshold.

Optionally, the image association information includes position coordinates of an upper left region and position coordinates of a lower right region corresponding to the key region of the candidate template image;

accordingly, the critical area determining module 404 includes:

the upper left area coordinate determining unit is used for determining the upper left area position coordinate corresponding to the key area of the infrared image to be positioned according to the upper left area position coordinate corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point;

a lower right area coordinate determining unit, configured to determine, according to a lower right area position coordinate corresponding to a key area of the target template image, the position coordinate of the first feature point, and the position coordinate of the second feature point, a lower right area position coordinate corresponding to a key area of the infrared image to be located;

and the key area unit is used for determining the key area of the infrared image to be positioned according to the position coordinates of the upper left area and the position coordinates of the lower right area corresponding to the key area of the infrared image to be positioned.

Optionally, the coordinate determination unit for the upper left region includes:

a first feature point determining subunit, configured to determine, from the first feature points, at least one feature point that is closer in position coordinate distance to an upper left area corresponding to a key area of the target template image, as a first target feature point;

a second feature point determining subunit, configured to use a second feature point matched with each of the first target feature points as a second target feature point;

and the upper left area coordinate determining subunit is used for determining the upper left area position coordinate corresponding to the key area of the infrared image to be positioned according to the position coordinate of the first target feature point, the position coordinate of the second target feature point and the upper left area position coordinate corresponding to the key area of the target template image.

Optionally, the lower right area coordinate determining unit includes:

a third feature point determining subunit, configured to determine, from the first feature points, at least one feature point that is closer to a position coordinate of a lower-right area corresponding to a key area of the target template image, as a third target feature point;

a fourth feature point determining subunit, configured to use the second feature points that are matched with each of the third target feature points as fourth target feature points;

and the right-lower area coordinate determining subunit is used for determining the right-lower area position coordinate corresponding to the key area of the infrared image to be positioned according to the position coordinate of the third target feature point, the position coordinate of the fourth target feature point and the right-lower area position coordinate corresponding to the key area of the target template image.

Optionally, the apparatus further comprises:

and the area checking module is used for checking the area range and/or the area position of the key area of the infrared image to be positioned according to the position distance between the position coordinate of the key area of the infrared image to be positioned and the position coordinate of the key area of the target template image after the key area of the infrared image to be positioned is determined.

The key area positioning device provided by the embodiment of the invention can execute the key area positioning method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE five

FIG. 5 illustrates a schematic diagram of an electronic device 50 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 5, the electronic device 50 includes at least one processor 51, and a memory communicatively connected to the at least one processor 51, such as a Read Only Memory (ROM) 52, a Random Access Memory (RAM) 53, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 51 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 52 or the computer program loaded from a storage unit 58 into the Random Access Memory (RAM) 53. In the RAM 53, various programs and data necessary for the operation of the electronic apparatus 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. An input/output (I/O) interface 55 is also connected to the bus 54.

A plurality of components in the electronic apparatus 50 are connected to the I/O interface 55, including: an input unit 56 such as a keyboard, a mouse, or the like; an output unit 57 such as various types of displays, speakers, and the like; a storage unit 58 such as a magnetic disk, an optical disk, or the like; and a communication unit 59 such as a network card, modem, wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processor 51 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processors 51 include, but are not limited to, central Processing Units (CPUs), graphics Processing Units (GPUs), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processors, controllers, microcontrollers, and the like. The processor 51 performs the various methods and processes described above, such as the critical area location method.

In some embodiments, the critical area location method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and/or installed onto electronic device 50 via ROM 52 and/or communications unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the critical area positioning method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the critical area location method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for locating a key area, comprising:

acquiring an infrared image to be positioned and candidate template images obtained by shooting at different shooting point positions; wherein the candidate template image has image association information; the image correlation information comprises position coordinates of key areas of the candidate template images;

matching feature points of the target template image and the infrared image to be positioned to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned; wherein the first feature point and the second feature point are matched feature points;

2. The method according to claim 1, wherein the image association information includes a predetermined description vector of at least one feature point corresponding to the candidate template image;

the step of matching the feature points of the infrared image to be positioned and the target template image to obtain a first feature point of the target template image and a second feature point of the infrared image to be positioned comprises the following steps:

performing feature extraction on the infrared image to be positioned to obtain a description vector of at least one feature point corresponding to the infrared image to be positioned;

according to the description vector corresponding to the infrared image to be positioned and the description vector corresponding to the target template image, performing feature point matching on the infrared image to be positioned and the target template image to obtain at least one feature point matching group; and the characteristic point matching group comprises a first characteristic point of the matched target template image and a second characteristic point of the infrared image to be positioned.

3. The method of claim 2, wherein after said obtaining at least one matched set of feature points, the method further comprises:

determining the number of the matching groups of the feature point matching groups;

if the number of the matching groups is smaller than a preset number threshold, determining that a key area of the infrared image to be positioned cannot be positioned;

and if the number of the matching groups is not less than a preset number threshold, determining a key area of the infrared image to be positioned according to the position coordinate of the first characteristic point, the position coordinate of the second characteristic point and the key area position coordinate of the target template image.

4. The method according to claim 1, wherein the image association information includes position coordinates of an upper left region and position coordinates of a lower right region corresponding to the key region of the candidate template image;

correspondingly, the determining the key area of the infrared image to be positioned according to the position coordinates of the first feature point, the position coordinates of the second feature point and the position coordinates of the key area of the target template image includes:

determining the position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the upper left area corresponding to the key area of the target template image, the position coordinate of the first characteristic point and the position coordinate of the second characteristic point;

determining the position coordinates of the right lower area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the right lower area corresponding to the key area of the target template image, the position coordinates of the first characteristic point and the position coordinates of the second characteristic point;

and determining the key area of the infrared image to be positioned according to the position coordinates of the upper left area and the position coordinates of the lower right area corresponding to the key area of the infrared image to be positioned.

5. The method according to claim 4, wherein the determining the position coordinates of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the upper left area corresponding to the key area of the target template image, the position coordinates of the first feature point and the position coordinates of the second feature point comprises:

determining at least one characteristic point which is close to the position coordinate of the upper left area corresponding to the key area of the target template image from the first characteristic points as a first target characteristic point;

taking the second characteristic points matched with the first target characteristic points as second target characteristic points;

and determining the position coordinate of the upper left area corresponding to the key area of the infrared image to be positioned according to the position coordinate of the first target characteristic point, the position coordinate of the second target characteristic point and the position coordinate of the upper left area corresponding to the key area of the target template image.

6. The method according to claim 4, wherein the determining, according to the position coordinates of the lower right area corresponding to the key area of the target template image, the position coordinates of the first feature point, and the position coordinates of the second feature point, the position coordinates of the lower right area corresponding to the key area of the infrared image to be located comprises:

determining at least one feature point which is closer to the position coordinate of a lower right region corresponding to a key region of the target template image from the first feature points to serve as a third target feature point;

taking the second feature points matched with the third target feature points as fourth target feature points;

and determining the position coordinates of the right lower area corresponding to the key area of the infrared image to be positioned according to the position coordinates of the third target feature point, the position coordinates of the fourth target feature point and the position coordinates of the right lower area corresponding to the key area of the target template image.

7. The method according to any one of claims 1-6, characterized in that after determining the key area of the infrared image to be located, it further comprises:

and checking the area range and/or the area position of the key area of the infrared image to be positioned according to the position distance between the position coordinate of the key area of the infrared image to be positioned and the position coordinate of the key area of the target template image.

8. A critical area locating device, comprising:

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the critical area locating method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions for causing a processor to perform the method of locating critical areas according to any one of claims 1 to 7 when executed.