CN108710828B - Method, device and storage medium for identifying target object and vehicle - Google Patents
Method, device and storage medium for identifying target object and vehicle Download PDFInfo
- Publication number
- CN108710828B CN108710828B CN201810350799.5A CN201810350799A CN108710828B CN 108710828 B CN108710828 B CN 108710828B CN 201810350799 A CN201810350799 A CN 201810350799A CN 108710828 B CN108710828 B CN 108710828B
- Authority
- CN
- China
- Prior art keywords
- target
- position information
- clustering
- optical flow
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure relates to a method, an apparatus, a storage medium, and a vehicle for recognizing a target object, including: acquiring a current frame image and a previous frame image within a preset range around a vehicle; acquiring target features according to a current frame image and a previous frame image, determining first position information of the target features in the current frame image, and determining optical flow of the target features according to the first position information; clustering the target features according to the first position information and the optical flow to obtain a first target object; acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object; and identifying whether the first target object and the second target object are the same target object according to the first position information, the second position information, the optical flow and the speed information.
Description
Technical Field
The present disclosure relates to the field of vehicle technologies, and in particular, to a method, an apparatus, a storage medium, and a vehicle for identifying a target object.
Background
In order to improve the accuracy and reliability of single sensor detection, the multi-sensor information fusion technology receives more and more attention in the field of vehicle safety guarantee research.
At present, in order to improve the accuracy of identifying the front environment by an unmanned automobile, a method of information fusion by a radar sensor and a vision sensor is generally adopted, wherein the radar sensor can be arranged at the upper position of a front bumper of the automobile, and the vision sensor is arranged on a front windshield, but because the information fusion process mainly depends on position information for fusion, the vision sensor is easily influenced by changes of weather conditions and illumination conditions, the vision detection effect is correspondingly influenced, and when different target objects are superposed, the target objects cannot be distinguished by simply depending on the positions of the target objects in images, so that the target objects collected by the vision sensor and the radar sensor cannot be accurately identified by simply depending on the position information, thereby causing the failure of information fusion.
Disclosure of Invention
In order to solve the above problems, the present disclosure proposes a method, an apparatus, and a storage medium for recognizing a target object, and a vehicle.
According to a first aspect of the embodiments of the present disclosure, there is provided a method for identifying a target object, applied to a vehicle, including:
acquiring a current frame image and a previous frame image within a preset range around a vehicle;
acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information;
clustering the target features according to the first position information and the optical flow to obtain a first target object;
acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object;
identifying whether the first object and the second object are the same object or not based on the first position information, the second position information, the optical flow, and the velocity information.
Optionally, the obtaining of the target feature according to the current frame image and the previous frame image includes:
respectively detecting image feature points in the current frame image and the previous frame image;
and carrying out adjacent region matching on the image feature points to obtain the target features.
Optionally, the determining optical flow of the target feature according to the first position information comprises:
acquiring third position information of the target feature in the previous frame of image;
determining an optical flow of the target feature according to the first position information and the third position information.
Optionally, the determining optical flow of the target feature according to the first position information and the third position information comprises:
determining the moving distance of the target feature according to the first position information and the third position information;
and acquiring the optical flow of the target feature according to the moving distance and a preset acquisition period.
Optionally, the clustering the target feature according to the first position information and the optical flow to obtain a first target object includes:
clustering the target features according to the first position information to obtain a first clustering result;
and clustering the target features in the first clustering result according to the optical flow to obtain the first target object.
Optionally, the clustering the target to be clustered according to the second position information and the speed information to obtain a second target object includes:
clustering the target to be clustered according to the second position information to obtain a second clustering result;
and clustering the targets to be clustered in the second clustering result according to the speed information to obtain the second target object.
Optionally, the identifying whether the first object and the second object are the same object according to the first position information, the second position information, the optical flow, and the velocity information includes:
determining whether the first target object and the second target object are matched in position according to the first position information and the second position information;
when the first object and the second object are matched in position, continuously determining whether the optical flow of the first object is matched with the speed information of the second object;
determining that the first object and the second object are the same object when the optical flow of the first object matches the velocity information of the second object.
Optionally, before the determining whether the optical flow of the first object matches the velocity information of the second object, further comprising:
normalizing the optical flow of the first object and the velocity information of the second object respectively;
the determining whether the optical flow of the first object matches the velocity information of the second object comprises:
determining whether a difference value between the normalized optical flow and the normalized speed information is less than or equal to a preset threshold value;
determining that the optical flow of the first object matches the velocity information of the second object when the difference between the normalized optical flow and the normalized velocity information is less than or equal to the preset threshold.
According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for recognizing a target object, applied to a vehicle, including:
the acquisition module is used for acquiring a current frame image and a previous frame image within a preset range around the vehicle;
the processing module is used for acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information;
the first clustering module is used for clustering the target features according to the first position information and the optical flow to obtain a first target object;
the second clustering module is used for acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object;
an identification module, configured to identify whether the first object and the second object are the same object according to the first position information, the second position information, the optical flow, and the velocity information.
Optionally, the processing module includes:
the detection submodule is used for respectively detecting image characteristic points in the current frame image and the previous frame image;
and the matching module is used for performing adjacent region matching on the image feature points to obtain the target features.
Optionally, the processing module includes:
the obtaining submodule is used for obtaining third position information of the target feature in the previous frame of image;
a first determining submodule, configured to determine an optical flow of the target feature according to the first position information and the third position information.
Optionally, the first determining sub-module is configured to determine a moving distance of the target feature according to the first location information and the third location information; and acquiring the optical flow of the target feature according to the moving distance and a preset acquisition period.
Optionally, the first clustering module includes:
the first clustering submodule is used for clustering the target characteristics according to the first position information to obtain a first clustering result;
and the second clustering submodule is used for clustering the target features in the first clustering result according to the optical flow to obtain the first target object.
Optionally, the second clustering module includes:
the third clustering submodule is used for clustering the target to be clustered according to the second position information to obtain a second clustering result;
and the fourth clustering submodule is used for clustering the target to be clustered in the second clustering result according to the speed information to obtain the second target object.
Optionally, the identification module comprises:
a second determining submodule, configured to determine whether the first target object and the second target object are matched in position according to the first position information and the second position information;
a third determining sub-module, configured to, when the first object and the second object are satisfied with a position match, continue to determine whether the optical flow of the first object matches the velocity information of the second object;
a fourth determination sub-module configured to determine that the first object and the second object are the same object when the optical flow of the first object matches the velocity information of the second object.
Optionally, the method further comprises:
the normalization submodule is used for respectively normalizing the optical flow of the first target object and the speed information of the second target object;
the third determining submodule is used for determining whether the difference value between the normalized optical flow and the normalized speed information is smaller than or equal to a preset threshold value; determining that the optical flow of the first object matches the velocity information of the second object when the difference between the normalized optical flow and the normalized velocity information is less than or equal to the preset threshold.
According to a third aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method described above.
According to a fourth aspect of the embodiments of the present disclosure, there is provided an apparatus for recognizing a target object, including:
a memory having a computer program stored thereon; and
one or more processors configured to execute the programs in the memory to implement the steps of the method described above.
According to a fifth aspect of the embodiments of the present disclosure, there is provided a vehicle including the apparatus for recognizing a target object described above.
According to the technical scheme, the current frame image and the previous frame image within the preset range around the vehicle are obtained; acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information; clustering the target features according to the first position information and the optical flow to obtain a first target object; acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object; and identifying whether the first target object and the second target object are the same target object or not according to the first position information, the second position information, the optical flow and the speed information, so that the target object can be accurately identified, and the problem of low identification accuracy caused by identifying the target object according to the position information in the prior art is solved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
FIG. 1 is a flow chart illustrating a method of identifying a target object in accordance with an exemplary embodiment of the present disclosure;
FIG. 2 is a flow chart illustrating yet another method of identifying a target object in accordance with an exemplary embodiment of the present disclosure;
fig. 3 is a block diagram illustrating a first apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 4 is a block diagram of a second apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 5 is a block diagram illustrating a third apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 6 is a block diagram illustrating a fourth apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 7 is a block diagram illustrating a fifth apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 8 is a block diagram illustrating a sixth apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 9 is a block diagram illustrating a seventh apparatus for recognizing an object according to an exemplary embodiment of the present disclosure;
fig. 10 is a block diagram of an eighth apparatus for recognizing an object according to an exemplary embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
The present disclosure may be applied to an information fusion scenario in which a vision sensor and a radar sensor for acquiring images within a preset range around a vehicle according to a preset acquisition cycle and a processor for acquiring angle information, distance information and speed information of a target to be clustered within the preset range around the vehicle are installed in the vehicle, the processor may determine a first target object in a current frame image according to a previous frame image and the current frame image acquired by the vision sensor and determine a second target object according to the angle information, distance information and speed information of the target to be clustered acquired by the radar sensor, such that whether the first target object and the second target object are the same target is determined according to first position information and optical flow of the first target object and second position information and speed information of the second target object, and information fusion is carried out when the first target object and the second target object are the same target object, so that the target object can be accurately identified, and the problem of low identification accuracy caused by identifying the target object according to the position information in the prior art is solved.
The present disclosure is described in detail below with reference to specific examples.
Fig. 1 is a flowchart illustrating a method of identifying an object, as shown in fig. 1, applied to a vehicle, according to an exemplary embodiment of the present disclosure, the method including:
s101, obtaining a current frame image and a previous frame image in a preset range around the vehicle.
In the present disclosure, images within a preset range around a vehicle may be acquired according to a preset acquisition cycle by a vision sensor (e.g., a camera, etc.) installed in the vehicle, where the current frame image is an image acquired by the vision sensor in the current acquisition cycle, and the previous frame image is an image acquired by the vision sensor in the previous acquisition cycle.
S102, acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information.
Image feature points in the current frame image and the previous frame image may be detected respectively, for example, the image feature points may be SURF feature points, SIFT feature points, or the like; it should be noted that the target feature generally includes a plurality of target features, for example, the target feature may be each part of the same first target object existing in both the current frame image and the previous frame image, if the same vehicle exists in both the current frame image and the previous frame image, the target feature may be a left rear door, a rear window, a left rear tire, and the like of the same vehicle, and in addition, when a plurality of the same first target objects exist in both the current frame image and the previous frame image, the target feature may be each part corresponding to a plurality of the same first target objects, which is only an example and is not limited in the present disclosure.
In this step, the optical flow of the target feature may be determined by acquiring third position information of the target feature in the previous frame image, and determining the optical flow of the target feature according to the first position information and the third position information.
S103, clustering the target features according to the first position information and the optical flow to obtain a first target object.
In a possible implementation manner, the target features are clustered according to the first position information to obtain a first clustering result, and the target features in the first clustering result are clustered according to the optical flow to obtain the first target object.
S104, obtaining second position information and speed information of the target to be clustered in the preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object.
The radar sensor may divide the second target object into a plurality of parts for respective detection, and therefore the target to be clustered is each part of the second target object, and in this way, each part corresponding to a plurality of second target objects can be clustered respectively to obtain a plurality of second target objects.
In a possible implementation manner, the angle information, the distance information and the speed information of the target to be clustered in the preset range around the vehicle can be acquired through a radar sensor installed in the vehicle, and therefore the second position information of the target to be clustered can be obtained according to the angle information and the distance information.
And S105, identifying whether the first target object and the second target object are the same target object according to the first position information, the second position information, the optical flow and the speed information.
In this step, it may be determined whether the first object and the second object are position-matched based on the first position information and the second position information; when the first object and the second object are matched in position, continuously determining whether the optical flow of the first object is matched with the speed information of the second object; and when the optical flow of the first object is matched with the speed information of the second object, determining that the first object and the second object are the same object.
By adopting the method, the first target object acquired by the vision sensor and the second target object acquired by the radar sensor can be respectively acquired, and the optical flow of the first target object and the speed information of the second target object are matched after the first position information of the first target object and the two position information of the second target object are matched, so that the optical flow and the speed information are matched on the basis of the position information matching, the target object can be accurately identified, and the problem of low identification accuracy caused by identifying the target object according to the position information in the prior art is solved.
Fig. 2 is a flowchart illustrating a method of identifying an object, as shown in fig. 2, applied to a vehicle, according to an exemplary embodiment of the present disclosure, the method including:
s201, obtaining a current frame image and a previous frame image in a preset range around the vehicle.
In the present disclosure, images within a preset range around a vehicle may be acquired according to a preset acquisition cycle by a vision sensor (e.g., a camera, etc.) installed in the vehicle, where the current frame image is an image acquired by the vision sensor in the current acquisition cycle, and the previous frame image is an image acquired by the vision sensor in the previous acquisition cycle.
S202, detecting image characteristic points in the current frame image and the previous frame image respectively.
Since the SURF feature and the SIFT feature both have scale-invariant features and are suitable for detecting local features in an image, image feature points (i.e., SURF feature points or SIFT feature points) in the current frame image and the previous frame image may be extracted by using a SURF feature or SIFT feature extraction method in the present disclosure, which refers to the prior art and is not described again, the image feature point extraction method is only an example, and the present disclosure does not limit the method.
It should be noted that the image feature points may include a plurality of image feature points, where, considering that the image feature points are usually the most easily identifiable pixel points (such as corner points, inflection points, intersection points, and the like) in the image, the vicinity matching may be performed according to the image feature points in the subsequent steps.
And S203, carrying out adjacent region matching on the image feature points to obtain the target features.
In this step, the positions of the image feature points in the current frame image and the previous frame image may be determined, and the matching points existing in the preset range around the positions of the current frame image and the previous frame image may be obtained, so that the matching points and the image feature points may be determined to form the target feature.
And S204, determining first position information of the target feature in the current frame image.
And S205, acquiring third position information of the target feature in the previous frame of image.
S206, determining the optical flow of the target feature according to the first position information and the third position information.
The moving distance of the target feature can be determined according to the first position information and the third position information, and the optical flow of the target feature can be acquired according to the moving distance and a preset acquisition period.
And S207, clustering the target characteristics according to the first position information to obtain a first clustering result.
In a possible implementation manner, the current frame image may be divided into a plurality of regions, at this time, the target features in each region are grouped into one type, and the number of the regions for dividing the regions is greater than or equal to a preset number, so that the clustering accuracy is high.
And S208, clustering the target features in the first clustering result according to the optical flow to obtain a first target object.
Since the target features of different first objects may be wrongly clustered into one class when clustering is performed through the first position information, in order to avoid this problem, based on the difference of the optical flows of different first objects, the present disclosure may continue clustering the target features in the first clustering result according to the optical flows, thereby improving the accuracy of clustering different first objects.
And S209, acquiring second position information and speed information of the target to be clustered in the preset range around the vehicle.
The radar sensor may divide the second target object into a plurality of parts for respective detection, so that the target to be clustered is each part of the second target object, and in this way, each part corresponding to a plurality of second target objects may be clustered to obtain a plurality of second target objects in the subsequent step.
In a possible implementation manner, the angle information, the distance information and the speed information of the target to be clustered in the preset range around the vehicle can be acquired through a radar sensor installed in the vehicle, and therefore the second position information of the target to be clustered can be obtained according to the angle information and the distance information.
S210, clustering the target to be clustered according to the second position information to obtain a second clustering result.
The second position information obtained is generally a three-dimensional coordinate, so that a preset range around the vehicle can be divided into a plurality of three-dimensional spaces, and a target three-dimensional space where the target to be clustered is located is determined, and clustering of the target to be clustered is further achieved.
S211, clustering the target to be clustered in the second clustering result according to the speed information to obtain the second target object.
Similarly, the target to be clustered in the second clustering result may be part of different second target objects, and therefore, the parts of different second target objects in the three-dimensional space of the target may be clustered through the speed information, thereby improving the clustering accuracy of different second target objects.
S212, determining whether the first target object and the second target object are matched in position or not according to the first position information and the second position information.
Since the first position information is a two-dimensional coordinate and the second position information is a three-dimensional coordinate, the first position information and the second position information need to be subjected to coordinate conversion for position matching.
In one possible implementation, the GNN algorithm may be used to determine whether the first object and the second object are matched in position, and the above examples are only illustrative and the disclosure is not limited thereto.
When the first object and the second object are matched in position, continuing to execute step S213;
when the positions of the first target object and the second target object do not match, step S214 is executed.
S213, determining whether the optical flow of the first object is matched with the speed information of the second object.
In addition, since the optical flow and the velocity information are of different equivalent, it is not possible to directly match the optical flow and the velocity information, and in summary, before determining whether the optical flow of the first object matches the velocity information of the second object, it is necessary to normalize the optical flow of the first object and the velocity information of the second object, respectively, and thus, determining whether the optical flow of the first object matches the velocity information of the second object includes: determining whether the difference value of the normalized optical flow and the normalized speed information is smaller than or equal to a preset threshold value; and when the difference value between the normalized optical flow and the normalized speed information is larger than the preset threshold value, determining that the optical flow of the first target object is not matched with the speed information of the second target object.
When the optical flow of the first object matches the velocity information of the second object, performing step S215;
when the optical flow of the first object does not match the velocity information of the second object, step S214 is executed.
S214, determining that the first target object and the second target object are different target objects.
S215, determining that the first target object and the second target object are the same target object.
Therefore, the category information of the target object can be determined through the current frame image acquired by the vision sensor, and the category information is fused with the speed information acquired by the radar sensor, so that the vision sensor and the radar sensor are accurately fused.
It should be noted that, for the above method embodiment, for the sake of simplicity, it is described as a series of action combinations, but it should be understood by those skilled in the art that the present disclosure is not limited by the described action sequence, because some steps may be performed in other sequences or simultaneously according to the present disclosure, for example, steps S201 to S208 are a process for acquiring a first object, steps S209 to S211 are a process for acquiring a second object, and the two processes are independent processes, so steps S209 to S211 may be performed before steps S201 to S208, or the two processes may be performed simultaneously; further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required for the disclosure.
By adopting the method, the first target object acquired by the vision sensor and the second target object acquired by the radar sensor can be respectively acquired, and the optical flow of the first target object and the speed information of the second target object are matched after the first position information of the first target object and the two position information of the second target object are matched, so that the optical flow and the speed information are matched on the basis of the position information matching, the target object can be accurately identified, and the problem of low identification accuracy caused by identifying the target object according to the position information in the prior art is solved.
Fig. 3 is a block diagram of an apparatus for recognizing an object according to an exemplary embodiment of the present disclosure, as shown in fig. 3, applied to a vehicle, including:
the acquiring module 301 is configured to acquire a current frame image and a previous frame image within a preset range around a vehicle;
a processing module 302, configured to obtain a target feature according to the current frame image and the previous frame image, determine first position information of the target feature in the current frame image, and determine an optical flow of the target feature according to the first position information;
a first clustering module 303, configured to cluster the target feature according to the first position information and the optical flow to obtain a first target object;
the second clustering module 304 is configured to obtain second position information and speed information of the target to be clustered in a preset range around the vehicle, and cluster the target to be clustered according to the second position information and the speed information to obtain a second target object;
an identifying module 305, configured to identify whether the first object and the second object are the same object according to the first position information, the second position information, the optical flow, and the velocity information.
Fig. 4 is a block diagram of an apparatus for identifying an object according to an exemplary embodiment of the disclosure, and as shown in fig. 4, the processing module 302 includes:
a detection submodule 3021, configured to detect image feature points in the current frame image and the previous frame image respectively;
a matching module 3022, configured to perform neighborhood matching on the image feature point to obtain the target feature.
Fig. 5 is a block diagram of an apparatus for identifying an object according to an exemplary embodiment of the disclosure, and as shown in fig. 5, the processing module 302 includes:
an obtaining submodule 3023, configured to obtain third position information of the target feature in the previous frame of image;
a first determining submodule 3024 configured to determine an optical flow of the target feature according to the first position information and the third position information.
Optionally, the first determining submodule 3024 is configured to determine a moving distance of the target feature according to the first position information and the third position information; and acquiring the optical flow of the target feature according to the moving distance and a preset acquisition period.
Fig. 6 is a block diagram of an apparatus for identifying a target object according to an exemplary embodiment of the disclosure, and as shown in fig. 6, the first clustering module 303 includes:
the first clustering submodule 3031 is configured to cluster the target feature according to the first location information to obtain a first clustering result;
and a second clustering submodule 3032, configured to cluster the target features in the first clustering result according to the optical flow to obtain the first target object.
Fig. 7 is a block diagram of an apparatus for identifying an object according to an exemplary embodiment of the disclosure, and as shown in fig. 7, the second clustering module 304 includes:
a third clustering submodule 3041, configured to cluster the target to be clustered according to the second position information to obtain a second clustering result;
the fourth clustering submodule 3042 is configured to cluster the target to be clustered in the second clustering result according to the speed information to obtain the second target object.
Fig. 8 is a block diagram of an apparatus for identifying an object according to an exemplary embodiment of the disclosure, and as shown in fig. 8, the identification module 305 includes:
a second determination sub-module 3051, configured to determine whether the first object and the second object are matched according to the first position information and the second position information;
a third determination sub-module 3052, configured to, when the first object and the second object are satisfied with a position match, continuously determine whether the optical flow of the first object matches the velocity information of the second object;
a fourth determining sub-module 3053, configured to determine that the first object and the second object are the same object when the optical flow of the first object matches the velocity information of the second object.
Fig. 9 is a block diagram of an apparatus for identifying an object according to an exemplary embodiment of the disclosure, as shown in fig. 9, further including:
a normalization submodule 3054, configured to normalize the optical flow of the first target object and the velocity information of the second target object, respectively;
the third determining submodule 3053, configured to determine whether a difference between the normalized optical flow and the normalized speed information is smaller than or equal to a preset threshold; and when the difference value of the normalized optical flow and the normalized speed information is smaller than or equal to the preset threshold value, determining that the optical flow of the first target object is matched with the speed information of the second target object.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
By adopting the device, the first target object acquired by the vision sensor and the second target object acquired by the radar sensor can be acquired respectively, and the optical flow of the first target object and the speed information of the second target object are matched after the first position information of the first target object and the two-position information of the second target object are matched, so that the optical flow and the speed information are matched on the basis of the matching of the position information, the target object can be accurately identified, and the problem of low identification accuracy caused by identifying the target object according to the position information in the prior art is solved.
Fig. 10 is a block diagram illustrating an apparatus 1000 for identifying an object according to an example embodiment. As shown in fig. 10, the apparatus 1000 may include: a processor 1001 and a memory 1002. The device 1000 may also include one or more of a multimedia component 1003, an input/output (I/O) interface 1004, and a communications component 1005.
The processor 1001 is configured to control the overall operation of the apparatus 1000, so as to complete all or part of the steps in the method for identifying an object. The memory 1002 is used to store various types of data to support operation of the device 1000, such as instructions for any application or method operating on the device 1000, and application-related data, such as contact data, messaging, pictures, audio, video, and so forth. The Memory 1002 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk. The multimedia components 1003 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may further be stored in memory 1002 or transmitted through communication component 1005. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 1004 provides an interface between the processor 1001 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 1005 is used for wired or wireless communication between the apparatus 1000 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding Communication component 1005 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the apparatus 1000 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for performing the above-mentioned method for identifying an object.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described method of identifying an object is also provided. For example, the computer readable storage medium may be the memory 1002 comprising program instructions executable by the processor 1001 of the device 1000 to perform the method for identifying an object described above.
In still another exemplary embodiment, a vehicle is also provided, which includes the apparatus for identifying an object described above.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.
Claims (17)
1. A method of identifying an object, applied to a vehicle, comprising:
acquiring a current frame image and a previous frame image within a preset range around a vehicle;
acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information;
clustering the target features according to the first position information and the optical flow to obtain a first target object;
acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object;
identifying whether the first object and the second object are the same object according to the first position information, the second position information, the optical flow and the velocity information;
the clustering the target features according to the first position information and the optical flow to obtain a first target object comprises:
clustering the target features according to the first position information to obtain a first clustering result;
clustering the target features in the first clustering result according to the optical flow to obtain the first target object;
the clustering the target features according to the first position information to obtain a first clustering result comprises:
dividing the current frame image into a plurality of regions, and clustering the target features in each region into one type to obtain the first clustering result.
2. The method of claim 1, wherein the obtaining target features from the current frame image and the previous frame image comprises:
respectively detecting image feature points in the current frame image and the previous frame image;
and carrying out adjacent region matching on the image feature points to obtain the target features.
3. The method of claim 2, wherein the determining optical flow of the target feature from the first location information comprises:
acquiring third position information of the target feature in the previous frame of image;
determining an optical flow of the target feature according to the first position information and the third position information.
4. The method of claim 3, wherein the determining optical flow of the target feature from the first location information and the third location information comprises:
determining the moving distance of the target feature according to the first position information and the third position information;
and acquiring the optical flow of the target feature according to the moving distance and a preset acquisition period.
5. The method according to claim 1, wherein the clustering the target to be clustered according to the second position information and the speed information to obtain a second target object comprises:
clustering the target to be clustered according to the second position information to obtain a second clustering result;
and clustering the targets to be clustered in the second clustering result according to the speed information to obtain the second target object.
6. The method of claim 5, wherein said identifying whether the first object and the second object are the same object from the first position information, the second position information, and the optical flow and the velocity information comprises:
determining whether the first target object and the second target object are matched in position according to the first position information and the second position information;
when the first object and the second object are matched in position, continuously determining whether the optical flow of the first object is matched with the speed information of the second object;
determining that the first object and the second object are the same object when the optical flow of the first object matches the velocity information of the second object.
7. The method of claim 6, further comprising, prior to said determining whether the optical flow of the first object matches the velocity information of the second object:
normalizing the optical flow of the first object and the velocity information of the second object respectively;
the determining whether the optical flow of the first object matches the velocity information of the second object comprises:
determining whether a difference value between the normalized optical flow and the normalized speed information is less than or equal to a preset threshold value;
determining that the optical flow of the first object matches the velocity information of the second object when the difference between the normalized optical flow and the normalized velocity information is less than or equal to the preset threshold.
8. An apparatus for recognizing an object, applied to a vehicle, comprising:
the acquisition module is used for acquiring a current frame image and a previous frame image within a preset range around the vehicle;
the processing module is used for acquiring a target feature according to the current frame image and the previous frame image, determining first position information of the target feature in the current frame image, and determining an optical flow of the target feature according to the first position information;
the first clustering module is used for clustering the target features according to the first position information and the optical flow to obtain a first target object;
the second clustering module is used for acquiring second position information and speed information of the target to be clustered in a preset range around the vehicle, and clustering the target to be clustered according to the second position information and the speed information to obtain a second target object;
an identification module configured to identify whether the first object and the second object are the same object according to the first position information, the second position information, the optical flow, and the velocity information;
the first clustering module comprises:
the first clustering submodule is used for clustering the target characteristics according to the first position information to obtain a first clustering result;
the second clustering submodule is used for clustering the target features in the first clustering result according to the optical flow to obtain the first target object;
the first clustering submodule is specifically configured to divide the current frame image into a plurality of regions, and cluster the target features in each region into one type to obtain the first clustering result.
9. The apparatus of claim 8, wherein the processing module comprises:
the detection submodule is used for respectively detecting image characteristic points in the current frame image and the previous frame image;
and the matching module is used for performing adjacent region matching on the image feature points to obtain the target features.
10. The apparatus of claim 9, wherein the processing module comprises:
the obtaining submodule is used for obtaining third position information of the target feature in the previous frame of image;
a first determining submodule, configured to determine an optical flow of the target feature according to the first position information and the third position information.
11. The apparatus of claim 10, wherein the first determining sub-module is configured to determine a moving distance of the target feature according to the first location information and the third location information; and acquiring the optical flow of the target feature according to the moving distance and a preset acquisition period.
12. The apparatus of claim 8, wherein the second clustering module comprises:
the third clustering submodule is used for clustering the target to be clustered according to the second position information to obtain a second clustering result;
and the fourth clustering submodule is used for clustering the target to be clustered in the second clustering result according to the speed information to obtain the second target object.
13. The apparatus of claim 12, wherein the identification module comprises:
a second determining submodule, configured to determine whether the first target object and the second target object are matched in position according to the first position information and the second position information;
a third determining sub-module, configured to, when the first object and the second object are satisfied with a position match, continue to determine whether the optical flow of the first object matches the velocity information of the second object;
a fourth determination sub-module configured to determine that the first object and the second object are the same object when the optical flow of the first object matches the velocity information of the second object.
14. The apparatus of claim 13, further comprising:
the normalization submodule is used for respectively normalizing the optical flow of the first target object and the speed information of the second target object;
the third determining submodule is used for determining whether the difference value between the normalized optical flow and the normalized speed information is smaller than or equal to a preset threshold value; determining that the optical flow of the first object matches the velocity information of the second object when the difference between the normalized optical flow and the normalized velocity information is less than or equal to the preset threshold.
15. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
16. An apparatus for identifying an object, comprising:
a memory having a computer program stored thereon; and
one or more processors configured to execute the programs in the memory to implement the steps of the method of any of claims 1-7.
17. A vehicle, characterized by comprising:
the apparatus for identifying an object of any one of claims 8-14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810350799.5A CN108710828B (en) | 2018-04-18 | 2018-04-18 | Method, device and storage medium for identifying target object and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810350799.5A CN108710828B (en) | 2018-04-18 | 2018-04-18 | Method, device and storage medium for identifying target object and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108710828A CN108710828A (en) | 2018-10-26 |
| CN108710828B true CN108710828B (en) | 2021-01-01 |
Family
ID=63867209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810350799.5A Active CN108710828B (en) | 2018-04-18 | 2018-04-18 | Method, device and storage medium for identifying target object and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108710828B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262503B (en) * | 2019-07-01 | 2022-06-03 | 百度在线网络技术（北京）有限公司 | Unmanned vending vehicle scheduling method, device, equipment and readable storage medium |
| CN110633625A (en) * | 2019-07-31 | 2019-12-31 | 北京木牛领航科技有限公司 | Identification method and system |
| CN112689775B (en) * | 2020-04-29 | 2022-06-14 | 华为技术有限公司 | Radar point cloud clustering method and device |
| CN114442101B (en) * | 2022-01-28 | 2023-11-14 | 南京慧尔视智能科技有限公司 | Vehicle navigation method, device, equipment and medium based on imaging millimeter wave radar |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204640A (en) * | 2016-06-29 | 2016-12-07 | 长沙慧联智能科技有限公司 | A kind of moving object detection system and method |
| CN107507231A (en) * | 2017-09-29 | 2017-12-22 | 智造未来（北京）机器人***技术有限公司 | Trinocular vision identifies follow-up mechanism and method |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101930609B (en) * | 2010-08-24 | 2012-12-05 | 东软集团股份有限公司 | Approximate target object detecting method and device |
| CN102542843A (en) * | 2010-12-07 | 2012-07-04 | 比亚迪股份有限公司 | Early warning method for preventing vehicle collision and device |
| JP5812064B2 (en) * | 2012-11-22 | 2015-11-11 | 株式会社デンソー | Target detection device |
| CN103246896B (en) * | 2013-05-24 | 2016-02-10 | 成都方米科技有限公司 | A kind of real-time detection and tracking method of robustness vehicle |
| CN103587467B (en) * | 2013-11-21 | 2015-11-18 | 中国科学院合肥物质科学研究院 | A kind of danger is overtaken other vehicles early warning and reminding method |
| CN104658249A (en) * | 2013-11-22 | 2015-05-27 | 上海宝康电子控制工程有限公司 | Method for rapidly detecting vehicle based on frame difference and light stream |
| CN105701479B (en) * | 2016-02-26 | 2019-03-08 | 重庆邮电大学 | Intelligent vehicle multilasered optical radar fusion identification method based on target signature |
| CN106379319B (en) * | 2016-10-13 | 2019-11-19 | 上汽大众汽车有限公司 | A kind of automobile assistant driving system and control method |
| CN106842188B (en) * | 2016-12-27 | 2018-01-09 | 上海思致汽车工程技术有限公司 | A kind of object detection fusing device and method based on multisensor |
| CN107918386B (en) * | 2017-10-25 | 2021-01-01 | 北京汽车集团有限公司 | Multi-sensor data fusion method and device for vehicle and vehicle |
| CN107705560B (en) * | 2017-10-30 | 2020-10-02 | 福州大学 | Road congestion detection method integrating visual features and convolutional neural network |
-
2018
- 2018-04-18 CN CN201810350799.5A patent/CN108710828B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204640A (en) * | 2016-06-29 | 2016-12-07 | 长沙慧联智能科技有限公司 | A kind of moving object detection system and method |
| CN107507231A (en) * | 2017-09-29 | 2017-12-22 | 智造未来（北京）机器人***技术有限公司 | Trinocular vision identifies follow-up mechanism and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108710828A (en) | 2018-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108710828B (en) | Method, device and storage medium for identifying target object and vehicle | |
| CN108885699B (en) | Character recognition method, device, storage medium and electronic equipment | |
| CN109816701B (en) | Target tracking method and device and storage medium | |
| US10943141B2 (en) | Object detection device and object detection method | |
| EP3338248A1 (en) | Systems and methods for object tracking | |
| CN110268440B (en) | Image analysis device, image analysis method, and storage medium | |
| CN111369590A (en) | Multi-target tracking method and device, storage medium and electronic equipment | |
| KR101935010B1 (en) | Apparatus and method for recognizing license plate of car based on image | |
| KR20180056685A (en) | System and method for non-obstacle area detection | |
| JP2016081525A (en) | Vehicular image recognition system and corresponding method | |
| CA2993421C (en) | Object detecting method and object detecting device | |
| CN111292531B (en) | Tracking method, device and equipment of traffic signal lamp and storage medium | |
| US20200082545A1 (en) | Asset tracking systems | |
| JP7185419B2 (en) | Method and device for classifying objects for vehicles | |
| JP2014137288A (en) | Device and method for monitoring surroundings of vehicle | |
| CN111783665A (en) | Action recognition method and device, storage medium and electronic equipment | |
| JP6795224B2 (en) | Mobile detection device, mobile detection method, and program | |
| CN110866428A (en) | Target tracking method and device, electronic equipment and storage medium | |
| JP2020177648A (en) | Recognition method of false detection of abandoned object, device, and image processor | |
| CN111291749A (en) | Gesture recognition method and device and robot | |
| CN113763425A (en) | Road area calibration method and electronic equipment | |
| KR20170106823A (en) | Image processing device identifying object of interest based on partial depth map | |
| CN112292847B (en) | Image processing apparatus, mobile apparatus, method, and storage medium | |
| CN110390252B (en) | Obstacle detection method and device based on prior map information and storage medium | |
| US20240127567A1 (en) | Detection-frame position-accuracy improving system and detection-frame position correction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |