CN112991448A - Color histogram-based loop detection method and device and storage medium - Google Patents

Color histogram-based loop detection method and device and storage medium Download PDF

Info

Publication number
CN112991448A
CN112991448A CN202110300183.9A CN202110300183A CN112991448A CN 112991448 A CN112991448 A CN 112991448A CN 202110300183 A CN202110300183 A CN 202110300183A CN 112991448 A CN112991448 A CN 112991448A
Authority
CN
China
Prior art keywords
color
image
loop
hsv
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110300183.9A
Other languages
Chinese (zh)
Other versions
CN112991448B (en
Inventor
周方华
魏武
韩进
林光杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202110300183.9A priority Critical patent/CN112991448B/en
Publication of CN112991448A publication Critical patent/CN112991448A/en
Application granted granted Critical
Publication of CN112991448B publication Critical patent/CN112991448B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Probability & Statistics with Applications (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method, a device and a storage medium for loop detection based on a color histogram, wherein the method comprises the following steps: converting the RGB image of the key frame into a gray image and an HSV color image; extracting ORB characteristic points and LSD line characteristics from the image; non-uniformly quantizing an HSV color space, and calculating a color histogram; selecting a color similar image according to the main color vector in the color histogram; normalizing the color histogram, and selecting a candidate image set C according to the Papanicolaou coefficient; detecting candidate loops according to the bag-of-words model; and carrying out time consistency and space consistency detection verification on the candidate loops, and selecting real loops to carry out loop correction and eliminate accumulated errors. The invention applies the global color characteristic color histogram of the image to the loop detection, provides richer image information for the loop detection algorithm, improves the loop detection accuracy and the arithmetic efficiency of the algorithm, and can be widely applied to the field of visual positioning and navigation of mobile robots.

Description

Color histogram-based loop detection method and device and storage medium
Technical Field
The invention relates to the field of visual positioning and navigation of mobile robots, in particular to a color histogram-based loop detection method, a color histogram-based loop detection device and a storage medium.
Background
Due to the progress of computer and sensor technologies, mobile robot technology has been developed unprecedentedly, and the application scenes thereof are gradually changed from military use to commercial use and civil use, so that the huge market potential thereof prompts a large number of scientific and technological companies to be put into the research of the mobile robot technology. In the field of mobile robots, a positioning and navigation technology is one of the most important basic technologies, and an autonomous navigation robot using vision or laser as a sensor estimates the pose of the robot in a progressive manner, namely, the pose information at a certain moment is estimated according to the pose information at the previous moment, so that the accumulation of errors is inevitable, and the errors cannot be eliminated fundamentally, thereby causing the inaccuracy of long-time positioning information.
The loop detection technology is proposed to solve the problems to a great extent, and the main objective of the loop detection technology is to detect certain positions where the robot repeatedly passes in sequence according to information collected by a sensor (namely, to detect a loop route in a motion track of the robot), and eliminate accumulated errors according to comparison of positioning information of the repeated positions. The loop detection technology can eliminate accumulated errors to a great extent, and greatly improves the positioning accuracy of the robot, so that the loop detection technology has very wide application and almost becomes one of necessary modules of a robot positioning and navigation system.
The most popular loop detection algorithm at present is a Bag-of-Words (BoW) model, which is an algorithm that clusters descriptors of feature points (SIFT, SURF, ORB, etc.) in an image into individual "Words", describes a frame of image with Bag-of-Words vectors formed by a plurality of Words, and then measures similarity between images through difference between the Bag-of-Words vectors, thereby finding out loops that may exist. The bag-of-words model only uses feature points in the image, discards other information, and does not fully utilize information such as rich colors in the image, and the bag-of-words model needs to calculate a word vector of each key frame, which increases the calculation amount and reduces the operation efficiency.
Disclosure of Invention
To solve at least one of the technical problems in the prior art to some extent, an object of the present invention is to provide a method, an apparatus and a storage medium for loop detection based on a color histogram.
The technical scheme adopted by the invention is as follows:
a color histogram-based loopback detection method comprises the following steps:
acquiring a color image of a key frame, and performing color processing on the acquired color image;
the color processing includes:
converting the color image into a gray scale image and an HSV image;
extracting ORB characteristic points and LSD line characteristics according to the gray level map and the HSV image;
carrying out non-uniform quantization on an HSV color space of the HSV image, and acquiring a quantized color histogram according to the HSV color image;
constructing an image main color vector according to the color histogram, and acquiring images which belong to the same category as the current key frame according to the image main color vector;
calculating a Papanicolaou coefficient between the current key frame and a color histogram of the same kind of image of the current key frame, and constructing a candidate image set of the loopback according to the Papanicolaou coefficient;
if the loop candidate image set is empty, returning to obtain the color image of the next key frame and performing color processing;
if the loopback candidate image set is not empty, calculating bag-of-word vectors of all color images in the loopback candidate image set, calculating the similarity between the bag-of-word vector of the current key frame and the bag-of-word vectors of other image frames, determining that the detected similarity exceeds a preset threshold value, and determining that loopback exists;
and acquiring image frames according to the similarity, detecting and verifying the time consistency and the space consistency of the acquired image frames, judging that loop exists if the image frames pass the verification, and correcting the loop.
Further, the non-uniform quantization of the HSV color space of the HSV image and obtaining a quantized color histogram according to the HSV color image include:
dividing the HSV color space of the HSV image into a plurality of color intervals, wherein each color interval corresponds to one interval bin in a color histogram,
and calculating the number of pixels of which the colors are positioned in each bin interval according to the HSV color map to obtain a one-dimensional color histogram.
Further, the one-dimensional color histogram is obtained by:
dividing an H component of an HSV color space into 16 intervals, and dividing an S component and a V component into 4 intervals respectively;
and performing weighted combination on the H component, the S component and the V component to form a one-dimensional color vector:
G＝QSQVH+QVS+V
wherein Q isS、QVThe quantization levels of the saturation S and lightness V components, respectively, and the value range of the one-dimensional color vector G is [0, 1.,. 255]；
And obtaining a one-dimensional color histogram of the HSV image by calculating a one-dimensional color vector G.
Further, the constructing an image dominant color vector according to the color histogram, and obtaining an image that belongs to the same category as the current key frame according to the image dominant color vector includes:
selecting three color values with the largest proportion in a color histogram as main colors of the image, wherein each main color consists of an 8-bit binary vector viRepresenting the sequential combination of dominant colors into a 24-bit binary dominant color vector P:
P＝[v1，v2，v3]
and acquiring the images which belong to the same category as the current key frame by adopting a K-means algorithm according to the image main color vector P.
Further, the calculation formula of the Babbitt coefficient is as follows:
Figure BDA0002985929160000031
wherein p and p' are color histograms of the two images, and the value range of the Pasteur coefficient rho is [0, 1 ].
Further, the bag-of-words vector v of the current key framecBag of words vector v with other image frameskThe similarity calculation formula is as follows:
Figure BDA0002985929160000032
if s (v)c，vk) If the current key frame is larger than the predetermined threshold value, the current key frame F is determinedcAnd the k frame FkA loop exists between; otherwise no loop is formed.
Further, the ORB feature points are composed of key points and BRIEF-32 descriptors;
after extracting key points, screening the extracted key points by adopting a quadtree splitting method and a non-maximum suppression algorithm to remove edge effects;
the BRIEF-32 descriptor is a 256-bit binary vector, and each bit in the binary vector is determined by the color similarity of any two pixel blocks in the circular area;
the circular area takes a key point as a center and has m pixels of radius;
the pixel block is an area obtained in the circular area according to a preset mode.
Further, the time consistency detection verification is carried out on the obtained image frames, and the time consistency detection verification comprises the following steps:
acquiring a plurality of image frames adjacent to the current key frame, detecting whether a loop is formed according to the acquired image frames, and if the loop is formed, determining that the loop meets time consistency;
carrying out spatial consistency detection verification on the obtained image frames, wherein the method comprises the following steps:
and calculating pose transformation between the current key frame and the image frames forming the loop, and if the amplitude of the pose transformation is smaller than a threshold value, determining that the loop meets the spatial consistency.
The other technical scheme adopted by the invention is as follows:
a color histogram based loopback detection apparatus, comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement the method described above.
The other technical scheme adopted by the invention is as follows:
a storage medium having stored therein a processor-executable program for performing the method as described above when executed by a processor.
The invention has the beneficial effects that: according to the invention, the global color characteristic color histogram of the image is applied to loop detection, so that richer image information is provided for a loop detection algorithm, and the loop detection accuracy is improved; in addition, the color histogram is calculated after the HSV color space is quantized at unequal intervals, so that the calculation complexity of the algorithm is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description is made on the drawings of the embodiments of the present invention or the related technical solutions in the prior art, and it should be understood that the drawings in the following description are only for convenience and clarity of describing some embodiments in the technical solutions of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for loop detection based on a color histogram according to an embodiment of the present invention;
FIG. 2 is a flow chart of a K-means clustering algorithm in an embodiment of the present invention;
FIG. 3 is a schematic diagram of loop-back temporal consistency and spatial consistency detection according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
As shown in fig. 1, the present embodiment provides a method for detecting a loop based on a color histogram, which includes the following steps:
and S1, acquiring a color image of the key frame, and converting the color image into a gray scale image and an HSV image. Two or more key frames may be included in a video.
And S2, extracting ORB characteristic points and LSD line characteristics according to the gray level map and the HSV image.
The extracted feature points should be distributed in the whole image as uniformly as possible, and this embodiment adopts two measures to ensure uniform distribution of ORB feature points, one of which is to extract feature points for image blocks; and secondly, screening the characteristic points by adopting a quadtree splitting method and a non-maximum suppression algorithm. LSD is a feature of some straight line segments in an image whose descriptor is LBD, which is a binary descriptor.
The ORB feature points are composed of FAST key points and BRIEF descriptors, and meanwhile, in order to enable the feature points to have direction invariance, direction vectors from the geometric centers of the pixel blocks to the gray centers of the pixel blocks are adopted to represent the directions of the feature points.
When the ORB characteristic points of each frame of image are extracted, the image is converted into a gray scale image and an HSV image, and an image pyramid is respectively constructed. By establishing the image pyramid, feature points are extracted at each layer of the pyramid, so that a scale space is formed, and the scale invariance of the feature points is guaranteed.
Extracting FAST key points from each layer of the gray level image pyramid; the FAST key point is a corner point, and the judgment is carried out by comparing the gray value of a certain pixel point with the gray value of a nearby point.
Using a certain pixel point p (let its gray value be I)p) As the center of circle, there are 16 pixels on the circle with 3 pixel units as the radius, and the gray value is set as Ii(i＝1，2，...，16)：
Figure BDA0002985929160000051
Wherein:
Figure BDA0002985929160000052
in this embodiment, the threshold T is 0.2IpIf N > N0Then the p point is considered as the key point, N0Usually 12 or 9, in this example N0＝9。
In order to reduce the edge effect, the feature points should be distributed in the whole graph as uniformly as possible, before extracting the key points, the gray-scale graph is divided into a plurality of small areas of 30 × 30, and the feature points are extracted in each small area. Let the whole image extract M0Individual feature point, the expected extracted feature point number is MiThen condition M should be satisfied0＞M1。
The method for calculating the direction of each key point by adopting a gray centroid method comprises the following steps:
selecting a disc region Patch with radius of r pixels by taking the key point as the center, and calculating a central point by taking the gray value of the image block as the weight, namely a gray centroid C:
Figure BDA0002985929160000061
wherein the content of the first and second substances,
Figure BDA0002985929160000062
assuming that the centroid C of the gray scale and the geometric center O of the disk do not coincide, the direction of the key point can be represented by a vector
Figure BDA0002985929160000067
The direction angle θ of (d) represents:
θ＝atan2(m01，m10)。
in the embodiment, a BRIEF-32 descriptor is selected, that is, a 256-bit binary vector is used to describe one feature point. In an image block of 31 x 31 pixels with a key point as the center, 256 pairs of pixel points are selected in a machine learning mode, and the coordinate of each pixel point is set as (x)i，yi) 1, 2.., 512, constituting a matrix D:
Figure BDA0002985929160000063
in order to ensure the rotation invariance of the feature point descriptor, the D matrix needs to be subjected to rotation transformation by the direction angle θ of the feature point:
Dθ＝RθD
wherein R isθA rotation matrix of the direction angle θ of the feature point:
Figure BDA0002985929160000064
Dθthe matrix is formed by the coordinates of the rotated pixel points, and the coordinates of a pair of pixel points are (x'i1，y′i1)，(x′i2，y′i2) And the ith Des of the descriptoriAnd correspondingly. Calculated in the HSV three monochrome channel images to be (x'i1，y′i1)，(x′i2，y′i2) The calculation method of the pixel average value of the disc-shaped pixel block Patch with the center and the radius of 2 pixels is as follows:
Figure BDA0002985929160000065
the color similarity of the two pixel blocks is calculated:
Figure BDA0002985929160000066
among them, CdIstiAs a difference in color, BdistiIs the difference in brightness.
The ith bit Des of the descriptoriDefined according to the following method:
Figure BDA0002985929160000071
wherein epsilonc、εBThreshold values for color difference and brightness difference, respectively, when CdIstiAnd BdistiBoth less than the threshold value representing two blocks of pixelsAnd if the colors are similar, the value of the corresponding bit of the descriptor is 1, otherwise, the colors of the two pixel blocks are different, and the value of the corresponding bit of the descriptor is 0.
After the above operation is performed on 256 pixel points, a 256-bit binary vector, i.e., the R-BRIEF descriptor of the feature point, can be obtained.
S3, carrying out non-uniform quantization on the HSV color space of the HSV image, and acquiring a quantized color histogram according to the HSV color image.
And non-uniformly quantizing the HSV color space, and calculating a quantized one-dimensional color histogram according to the HSV color image. The quantization of the HSV color space is to divide the HSV color space into a plurality of small color intervals, each small color interval corresponds to one interval bin in the color histogram, and the color histogram is obtained by calculating the number of pixels of which the colors fall in each bin interval; since a color image is represented by three channels, namely hue H, saturation S and lightness V, and the types of represented colors are very many, if a color histogram is directly calculated without quantizing a color space, the calculation amount will be very large. Therefore, according to the invention, the three color components H, S, V are quantized at unequal intervals according to the perception of human eyes on colors, and then the histogram is calculated, so as to reduce the calculation complexity of the algorithm, and the specific quantization method is as follows:
according to the range of different colors in the HSV color space and the subjective perception of human vision on the colors, the H component is divided into 16 intervals, and the S component and the V component are divided into 4 intervals respectively:
Figure BDA0002985929160000072
Figure BDA0002985929160000081
the H, S, V components are weighted and combined according to the quantization levels to form a one-dimensional color vector:
G＝QSQVH+QVS+V
wherein Q isS、QVThe quantization levels of the saturation S and lightness V components are respectively, and the value range of G is [0, 1.. multidot.255]。
Further, a one-dimensional color histogram of the HSV image can be obtained by calculating G, wherein the abscissa of the color histogram is a value of G, that is, a color value of the whole HSV image after non-uniform quantization, and the ordinate is a number n of pixels having color values falling in a corresponding intervali。
And S4, constructing an image main color vector according to the color histogram, and acquiring the images which belong to the same category as the current key frame according to the image main color vector.
Selecting three color values with the largest proportion in the color histogram to form a binary vector as an image main color vector P, and finding out an image which belongs to the same class as the current frame according to the P; taking three color values with the largest proportion in a color histogram as the main color of the image, wherein each color value is composed of an 8-bit binary vector viI-1, 2, 3, which are combined in order into a 24-bit binary master color vector P:
P＝[v1，v2，v3]
further, as shown in fig. 2, the K-means algorithm is adopted to classify the dominant color vectors P of all the key frames, and a K-ary tree is established to improve the retrieval efficiency.
And S5, calculating the Papanicolaou coefficient between the current key frame and the color histogram of the same-class image of the current key frame, and constructing a candidate image set of the loopback according to the Papanicolaou coefficient.
Normalizing the color histograms, calculating the Babbitt coefficients between the current frame and the color histograms of the images in the same class, and selecting all the images with the Babbitt coefficients larger than a threshold value to form a candidate image set C; the babbit coefficient is a measure of the similarity between two statistical samples and is calculated as follows:
Figure BDA0002985929160000082
where p and p' are the color histograms of the two images, and N is 255. The value range of the Papanicolaou coefficient rho is [0, 1], and the closer the value is to 1, the higher the similarity of the two color histograms is.
And selecting images with higher similarity with the color histogram of the current key frame from the image frames obtained by clustering according to a preset threshold value to form a candidate image set C.
And S6, if the loop candidate image set is empty, returning to the step S1. If the loop candidate image set is not empty, calculating bag-of-word vectors of all color images in the loop candidate image set, calculating the similarity between the bag-of-word vector of the current key frame and the bag-of-word vectors of other image frames, determining that the detected similarity exceeds a preset threshold value, and determining that a loop exists. The bag-of-words vector is obtained by calculation according to ORB feature points and LSD line features.
Judging whether the candidate image set C is empty, if so, indicating that no image close to the color histogram of the current frame exists in the historical key frame, namely, the current key frame does not form a loop, and returning to the step S1 to continuously detect the next key frame; if the image set C is not empty, the current key frame is possible to form a loop, and a word bag model is used for further judging the current frame;
calculating the point characteristic and line characteristic bag-of-words vector of all images in the current frame and the image set C according to the bag-of-words model, and calculating the bag-of-words vector v of the current framecAnd candidate frame bag of words vector vkThe calculation method of the similarity is as follows:
Figure BDA0002985929160000091
if s (v)c，vk) If the current frame F is larger than the predetermined threshold value, the current frame F is consideredcAnd the k frame FkA loop exists between; otherwise, it means that the current frame does not constitute a loop.
And S7, acquiring image frames according to the similarity, detecting and verifying the time consistency and the space consistency of the acquired image frames, judging that a loop exists if the image frames pass the verification, and performing loop correction.
As shown in fig. 3, temporal consistency and spatial consistency detection verification is performed between image frames in which a loop may exist. The time consistency verification is to verify according to whether the same loop can be continuously detected between the previous frame and the next frame of the loop frame, and if the same loop can be detected in the next frames of the current frame of the loop, the loop is considered to meet the time consistency; the spatial consistency detection and verification is that the loop meets the spatial consistency by calculating the pose transformation between the current frame and the loop frame and considering that the loop meets the spatial consistency if the pose transformation amplitude is small.
Further, if the detected loop can meet the time consistency and the space consistency, the detected loop is regarded as a real loop, loop correction is started, all key frames on the loop are globally optimized, and the accumulated error is eliminated.
In summary, compared with the prior art, the method of the embodiment has the following beneficial effects:
(1) the global color characteristic color histogram of the image is applied to loop detection, richer image information is provided for a loop detection algorithm, and the loop detection accuracy is improved.
(2) The color histogram is calculated after the HSV color space is quantized at unequal intervals according to the perception of human eyes on colors, and the calculation complexity of the algorithm is reduced.
(3) The two-stage detection strategy based on the color histogram and the bag-of-words model reduces calculation and matching of bag-of-words vectors to a certain extent, and improves the operation efficiency of the algorithm.
The present embodiment further provides a color histogram-based loopback detection apparatus, including:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement the method described above.
The color histogram-based loopback detection device of the present embodiment can execute the loopback detection method based on the color histogram provided by the method embodiment of the present invention, can execute any combination of the implementation steps of the method embodiment, and has the corresponding functions and advantages of the method.
The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and executed by the processor to cause the computer device to perform the method illustrated in fig. 1.
The present embodiment further provides a storage medium, which stores an instruction or a program capable of executing the color histogram-based loopback detection method provided by the method embodiment of the present invention, and when the instruction or the program is executed, the method can be executed by any combination of the method embodiments, and the method has corresponding functions and advantages.
In alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flow charts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are performed independently.
Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or software module, or one or more functions and/or features may be implemented in a separate physical device or software module. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be understood within the ordinary skill of an engineer, given the nature, function, and internal relationship of the modules. Accordingly, those skilled in the art can, using ordinary skill, practice the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative of and not intended to limit the scope of the invention, which is defined by the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the foregoing description of the specification, reference to the description of "one embodiment/example," "another embodiment/example," or "certain embodiments/examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. A method for detecting a loop based on a color histogram, comprising the steps of:
acquiring a color image of a key frame, and performing color processing on the acquired color image;
the color processing includes:
converting the color image into a gray scale image and an HSV image;
extracting ORB characteristic points and LSD line characteristics according to the gray level map and the HSV image;
carrying out non-uniform quantization on an HSV color space of the HSV image, and acquiring a quantized color histogram according to the HSV color image;
constructing an image main color vector according to the color histogram, and acquiring images which belong to the same category as the current key frame according to the image main color vector;
calculating a Papanicolaou coefficient between the current key frame and a color histogram of the same kind of image of the current key frame, and constructing a candidate image set of the loopback according to the Papanicolaou coefficient;
if the loop candidate image set is empty, returning to obtain the color image of the next key frame and performing color processing;
if the loopback candidate image set is not empty, calculating bag-of-word vectors of all color images in the loopback candidate image set, calculating the similarity between the bag-of-word vector of the current key frame and the bag-of-word vectors of other image frames, determining that the detected similarity exceeds a preset threshold value, and determining that loopback exists;
and acquiring image frames according to the similarity, detecting and verifying the time consistency and the space consistency of the acquired image frames, judging that loop exists if the image frames pass the verification, and correcting the loop.
2. The method as claimed in claim 1, wherein the non-uniform quantization of HSV color space of the HSV image and obtaining a quantized color histogram from an HSV color image comprises:
dividing the HSV color space of the HSV image into a plurality of color intervals, wherein each color interval corresponds to one interval bin in a color histogram,
and calculating the number of pixels of which the colors are positioned in each bin interval according to the HSV color map to obtain a one-dimensional color histogram.
3. The method of claim 2, wherein the one-dimensional color histogram is obtained by:
dividing an H component of an HSV color space into 16 intervals, and dividing an S component and a V component into 4 intervals respectively;
and performing weighted combination on the H component, the S component and the V component to form a one-dimensional color vector:
G＝QsQVH+QVS+V
wherein Q isS、QVThe quantization levels of the saturation S and lightness V components, respectively, and the value range of the one-dimensional color vector G is [0, 1.,. 255]；
And obtaining a one-dimensional color histogram of the HSV image by calculating a one-dimensional color vector G.
4. The method according to claim 1, wherein the constructing a dominant color vector of the image according to the color histogram and obtaining the image belonging to the same category as the current key frame according to the dominant color vector of the image comprises:
selecting the color value with the largest three ratios in the color histogram as the main color of the imageColor, each dominant color consisting of an 8-bit binary vector viRepresenting the sequential combination of dominant colors into a 24-bit binary dominant color vector P:
P＝[v1，v2，v3]
and acquiring the images which belong to the same category as the current key frame by adopting a K-means algorithm according to the image main color vector P.
5. The method of claim 1, wherein the babbit coefficient is calculated by the following formula:
Figure FDA0002985929150000021
wherein p and p' are color histograms of the two images, and the value range of the Pasteur coefficient rho is [0, 1 ].
6. The method of claim 1, wherein the bag-of-words vector v of the current key frame is a color histogram-based loop detection methodcBag of words vector v with other image frameskThe similarity calculation formula is as follows:
Figure FDA0002985929150000022
if s (v)c，vk) If the current key frame is larger than the predetermined threshold value, the current key frame F is determinedcAnd the k frame FkA loop exists between; otherwise no loop is formed.
7. The method of claim 1, wherein the ORB feature points are composed of key points and BRIEF-32 descriptors;
after extracting key points, screening the extracted key points by adopting a quadtree splitting method and a non-maximum suppression algorithm to remove edge effects;
the BRIEF-32 descriptor is a 256-bit binary vector, and each bit in the binary vector is determined by the color similarity of any two pixel blocks in the circular area;
the circular area takes a key point as a center and has m pixels of radius;
the pixel block is an area obtained in the circular area according to a preset mode.
8. The color histogram-based loopback detection method as claimed in claim 1, wherein the time consistency detection verification of the obtained image frames comprises:
acquiring a plurality of image frames adjacent to the current key frame, detecting whether a loop is formed according to the acquired image frames, and if the loop is formed, determining that the loop meets time consistency;
carrying out spatial consistency detection verification on the obtained image frames, wherein the method comprises the following steps:
and calculating pose transformation between the current key frame and the image frames forming the loop, and if the amplitude of the pose transformation is smaller than a threshold value, determining that the loop meets the spatial consistency.
9. A color histogram based loopback detection apparatus, comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a color histogram based loopback detection method as recited in any of claims 1-8.
10. A storage medium having stored therein a program executable by a processor, wherein the program executable by the processor is adapted to perform the method of any one of claims 1-8 when executed by the processor.
CN202110300183.9A 2021-03-22 2021-03-22 Loop detection method, device and storage medium based on color histogram Active CN112991448B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110300183.9A CN112991448B (en) 2021-03-22 2021-03-22 Loop detection method, device and storage medium based on color histogram

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110300183.9A CN112991448B (en) 2021-03-22 2021-03-22 Loop detection method, device and storage medium based on color histogram

Publications (2)

Publication Number Publication Date
CN112991448A true CN112991448A (en) 2021-06-18
CN112991448B CN112991448B (en) 2023-09-26

Family

ID=76334255

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110300183.9A Active CN112991448B (en) 2021-03-22 2021-03-22 Loop detection method, device and storage medium based on color histogram

Country Status (1)

Country Link
CN (1) CN112991448B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118038103A (en) * 2024-04-11 2024-05-14 南京师范大学 Visual loop detection method based on improved dynamic expansion model self-adaptive algorithm

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003244469A (en) * 2002-02-19 2003-08-29 Nisca Corp Image deciding apparatus, image reader, image deciding method, program and storage medium stored with program
US20090214109A1 (en) * 2007-12-26 2009-08-27 Keisuke Nakashima Dropout color processing method and processing apparatus using same
US20100091665A1 (en) * 2008-10-14 2010-04-15 Fujitsu Limited Loopback device and mirroring method
CN103392185A (en) * 2010-12-30 2013-11-13 派尔高公司 Color similarity sorting for video forensics search
CN107025668A (en) * 2017-03-30 2017-08-08 华南理工大学 A kind of design method of the visual odometry based on depth camera
CN110047142A (en) * 2019-03-19 2019-07-23 中国科学院深圳先进技术研究院 No-manned plane three-dimensional map constructing method, device, computer equipment and storage medium
CN110188809A (en) * 2019-05-22 2019-08-30 浙江大学 A kind of winding detection method based on image block
US20200043130A1 (en) * 2018-08-04 2020-02-06 Beijing Jingdong Shangke Information Technology Co., Ltd. System and method for scan-matching oriented visual slam
CN112507778A (en) * 2020-10-16 2021-03-16 天津大学 Loop detection method of improved bag-of-words model based on line characteristics

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003244469A (en) * 2002-02-19 2003-08-29 Nisca Corp Image deciding apparatus, image reader, image deciding method, program and storage medium stored with program
US20090214109A1 (en) * 2007-12-26 2009-08-27 Keisuke Nakashima Dropout color processing method and processing apparatus using same
US20100091665A1 (en) * 2008-10-14 2010-04-15 Fujitsu Limited Loopback device and mirroring method
CN103392185A (en) * 2010-12-30 2013-11-13 派尔高公司 Color similarity sorting for video forensics search
CN107025668A (en) * 2017-03-30 2017-08-08 华南理工大学 A kind of design method of the visual odometry based on depth camera
US20200043130A1 (en) * 2018-08-04 2020-02-06 Beijing Jingdong Shangke Information Technology Co., Ltd. System and method for scan-matching oriented visual slam
CN110047142A (en) * 2019-03-19 2019-07-23 中国科学院深圳先进技术研究院 No-manned plane three-dimensional map constructing method, device, computer equipment and storage medium
CN110188809A (en) * 2019-05-22 2019-08-30 浙江大学 A kind of winding detection method based on image block
CN112507778A (en) * 2020-10-16 2021-03-16 天津大学 Loop detection method of improved bag-of-words model based on line characteristics

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
张鹏: "基于视觉的四旋翼无人机室内自主导航***的研究与实现", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 *
张鹏: "基于视觉的四旋翼无人机室内自主导航***的研究与实现", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》, no. 01, 15 January 2020 (2020-01-15), pages 3 - 4 *
方榕: "基于数字地标图的室内自定位方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
方榕: "基于数字地标图的室内自定位方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 01, 15 January 2021 (2021-01-15) *
杜鹏飞: "基于图优化的单目视觉SLAM关键技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
杜鹏飞: "基于图优化的单目视觉SLAM关键技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 12, 15 December 2018 (2018-12-15), pages 4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118038103A (en) * 2024-04-11 2024-05-14 南京师范大学 Visual loop detection method based on improved dynamic expansion model self-adaptive algorithm

Also Published As

Publication number Publication date
CN112991448B (en) 2023-09-26

Similar Documents

Publication Publication Date Title
CN106960446B (en) Unmanned ship application-oriented water surface target detection and tracking integrated method
EP3101594A1 (en) Saliency information acquisition device and saliency information acquisition method
US8406535B2 (en) Invariant visual scene and object recognition
Ardianto et al. Real-time traffic sign recognition using color segmentation and SVM
US9275447B2 (en) Method and system for describing image region based on color histogram
CN110659550A (en) Traffic sign recognition method, traffic sign recognition device, computer equipment and storage medium
WO2010043771A1 (en) Detecting and tracking objects in digital images
US20170091573A1 (en) Visual attention detector and visual attention detection method
Wu et al. Typical target detection in satellite images based on convolutional neural networks
CN112633294A (en) Significance region detection method and device based on perceptual hash and storage device
CN110958467A (en) Video quality prediction method and device and electronic equipment
CN115937552A (en) Image matching method based on fusion of manual features and depth features
Zhao et al. End-to-end visual place recognition based on deep metric learning and self-adaptively enhanced similarity metric
CN111597875A (en) Traffic sign identification method, device, equipment and storage medium
CN112991448A (en) Color histogram-based loop detection method and device and storage medium
CN114387454A (en) Self-supervision pre-training method based on region screening module and multi-level comparison
CN111105436A (en) Target tracking method, computer device, and storage medium
CN110717910B (en) CT image target detection method based on convolutional neural network and CT scanner
CN112418262A (en) Vehicle re-identification method, client and system
CN117237751A (en) Training method, recognition method, system and equipment for grabbing detection model
US20230386023A1 (en) Method for detecting medical images, electronic device, and storage medium
CN116415210A (en) Image infringement detection method, device and storage medium
CN113537158B (en) Image target detection method, device, equipment and storage medium
CN112991449B (en) AGV positioning and mapping method, system, device and medium
CN112348853B (en) Particle filter tracking method based on infrared saliency feature fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant