CN112464782A - Pedestrian identification method and system - Google Patents
Pedestrian identification method and system Download PDFInfo
- Publication number
- CN112464782A CN112464782A CN202011328702.4A CN202011328702A CN112464782A CN 112464782 A CN112464782 A CN 112464782A CN 202011328702 A CN202011328702 A CN 202011328702A CN 112464782 A CN112464782 A CN 112464782A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- suspected
- similarity
- visible light
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000001514 detection method Methods 0.000 claims abstract description 25
- 230000036760 body temperature Effects 0.000 claims description 11
- 230000004888 barrier function Effects 0.000 claims description 7
- 238000003384 imaging method Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000001681 protective effect Effects 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 abstract description 5
- 230000007246 mechanism Effects 0.000 abstract description 5
- 230000002411 adverse Effects 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 3
- 230000005855 radiation Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000010287 polarization Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002618 waking effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/0022—Radiation pyrometry, e.g. infrared or optical thermometry for sensing the radiation of moving bodies
- G01J5/0025—Living bodies
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/06—Systems determining position data of a target
- G01S13/08—Systems for measuring distance only
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
- G01S13/865—Combination of radar systems with lidar systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
- G01S13/867—Combination of radar systems with cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to a pedestrian identification method and a system, wherein a visible light image on a vehicle travelling route is acquired in real time, and a preset model is set to screen out a suspected pedestrian area with a confusion item; when the suspected pedestrian is judged to exist, the position information of the suspected pedestrian is obtained, and then when the suspected pedestrian is determined to have potential adverse effects on the safe driving of the vehicle, the infrared thermal image corresponding to the visible light image is obtained for secondary identification, and according to the characteristic that the human body has thermal radiation, whether the suspected pedestrian is a pedestrian can be further determined, so that the accuracy of pedestrian identification is improved, and the driving safety is improved; the pedestrian recognition system is established by taking the main control module, and the visible light camera, the infrared camera and the detection module which are connected with the main control module as main modules, and the anti-interference capability aiming at severe environment can be improved by utilizing a dual mechanism of the millimeter wave radar and the laser radar, and meanwhile, the detection precision and the detection stability are ensured.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a pedestrian identification method and system.
Background
The safety problem of automatic driving is more and more concerned by people, and because the behavior of pedestrians has uncertainty, the vehicle needs to pay more attention to the identification and protection of the pedestrians in the driving process, so that the identification of the pedestrians is the main safety problem concerned by automatic driving.
Pedestrian Detection (Pedestrian Detection) is the use of computer vision techniques to determine whether a Pedestrian is present in an image or video sequence and to provide accurate positioning. The technology can be combined with technologies such as pedestrian tracking, pedestrian re-identification and the like, and is applied to the fields of artificial intelligence systems, vehicle auxiliary driving systems, intelligent robots, intelligent video monitoring, human body behavior analysis, intelligent traffic and the like.
Due to the characteristics of rigid and flexible objects, the appearance of the pedestrian is easily affected by wearing, dimensions, shielding, postures, visual angles and other factors, so that the pedestrian detection becomes a hot topic which has research value and is very challenging in the field of computer vision.
At present, the method for identifying the pedestrian generally adopts a camera to shoot an image, and extracts the shape (outline) and the action of the pedestrian from the image so as to identify the pedestrian. However, the appearance and wearing of a person are greatly changed, for example, when the person carries an umbrella or wears a skirt or wears a raincoat, the accuracy of pedestrian identification is reduced, so that the problem of missing detection of the person occurs, and potential safety hazards are generated.
Disclosure of Invention
The invention provides a pedestrian identification method and a system, and solves the technical problems of large limitation and low identification accuracy of the existing pedestrian identification method.
In order to solve the above technical problems, the present invention provides a pedestrian identification method, comprising the steps of:
s1, acquiring a visible light image on the vehicle traveling route;
s2, identifying the visible light image, and judging whether a suspected pedestrian exists according to a preset model;
s3, when the suspected pedestrian exists, acquiring the position information of the suspected pedestrian;
s4, acquiring an infrared thermal image of an area corresponding to the suspected pedestrian according to the position information, comparing the infrared thermal image with the human body temperature distribution characteristics, and further determining whether the suspected pedestrian is a pedestrian;
and S5, when the suspected pedestrian is judged to be the pedestrian, sending out an early warning prompt.
The basic scheme acquires visible light images on a vehicle travelling route in real time, and sets a preset model to screen out a suspected pedestrian area with a confusion item (namely, whether a suspected pedestrian exists or not is judged); when the suspected pedestrian is judged to exist, the position information of the suspected pedestrian is obtained, then when the suspected pedestrian is determined to have potential adverse effects on the safe driving of the vehicle, the infrared thermal image corresponding to the visible light image is obtained for secondary recognition, and according to the characteristic that the human body has thermal radiation, whether the suspected pedestrian is a pedestrian can be further determined, so that the accuracy of pedestrian recognition is improved, and the driving safety is improved.
In a further embodiment, the step S2 includes:
s21, identifying the visible light image, and acquiring the actual contour size of the obstacle on the vehicle travelling route;
and S22, substituting the actual contour dimension into a preset model, calculating the similarity between the obstacle and a preset human body, and judging whether the obstacle is a suspected pedestrian according to the similarity.
In a further embodiment, the step S21 includes:
s211, acquiring edge characteristics of the obstacle and a corresponding azimuth angle from the visible light image;
s212, detecting the distance between the vehicle and the obstacle by adopting a radar, and determining the actual outline size of the obstacle according to the distance and the imaging proportion.
According to the scheme, the edge characteristics and the azimuth angle of the obstacle can be rapidly acquired by utilizing a mature image identification technology in the first step, the distance between the vehicle and the obstacle is acquired by radar ranging in the second step, the actual outline size of the obstacle is calculated by combining the distance with the imaging proportion of the image, and before the suspected pedestrian is judged, the obstacle imaging in different distances is converted into the actual size of the real world, so that the unified judgment on all obstacles in the whole visible light image can be facilitated.
In a further embodiment, the step S22 includes:
s221, acquiring the height and the width of the obstacle according to the actual contour size;
s222, calculating a first similarity according to the height and a preset height range;
s223, calculating a second similarity according to the width and a preset body width range;
s224, comparing the height-width ratio of the obstacle with a preset height-width ratio range to obtain a third similarity;
s225, dividing the upper part, the middle part and the lower part of the barrier by referring to the proportion of the head, the shoulder, the trunk and the legs of the preset human body, and matching the upper part, the middle part and the lower part with a model library to obtain a fourth similarity, a fifth similarity and a sixth similarity;
s226, integrating the first similarity to the sixth similarity, substituting a preset algorithm to calculate the total similarity, and further judging whether the obstacle is a suspected pedestrian.
According to the scheme, corresponding preset ranges are respectively set for local characteristics and overall characteristics of a human body, the preset ranges comprise data such as a preset height range, a preset body width range, a preset height-width ratio and a model library, the limit proportion of the human body is comprehensively covered, the probability of missing detection of pedestrians is reduced, and the accuracy of pedestrian identification is improved; in addition, the first similarity to the sixth similarity are integrated and substituted into a preset algorithm to calculate the overall similarity, and suspected pedestrians are judged according to the overall similarity, so that the fault tolerance of the identification mechanism can be further improved.
In a further embodiment, the step S4 includes:
s41, judging whether the suspected pedestrian is in a safe area or not according to the position information, if so, finishing the identification, and if not, acquiring an infrared thermal image of the area corresponding to the suspected pedestrian;
and S42, extracting a color distribution image from the infrared thermal image, comparing the color distribution image with the human body temperature distribution characteristic, and judging whether the suspected pedestrian is a pedestrian according to the comparison result.
According to the scheme, at the beginning of the stage of judging whether the suspected pedestrian is a pedestrian, whether the suspected pedestrian is in the safety area is judged in advance, the situation that the pedestrian is in the safety area and does not influence the driving of the vehicle can be directly filtered out, only the situation that the driving safety of the vehicle is influenced is analyzed, and the efficiency of pedestrian identification can be improved; when the judgment is carried out, the infrared thermal image of the area corresponding to the suspected pedestrian is directly obtained, the pertinence is carried out with the human body temperature distribution rule, and the starting frequency of the infrared camera and the data quantity to be processed can be reduced.
In a further embodiment, in the step S41, the determining whether the suspected pedestrian is in the safe area according to the position information includes:
a. judging whether the distance between the suspected pedestrian and the vehicle is larger than a safety threshold value or not according to the position information, if so, finishing the identification, and if not, entering the next step;
b. and identifying the visible light image, judging whether the suspected pedestrian is blocked by a protective guard, if so, ending the identification, and if not, judging that the suspected pedestrian is in a non-safety area.
The scheme sets the limit condition of the safety zone according to the distance between the suspected pedestrian and the vehicle and the current environment, and can basically determine the safety relation between the suspected pedestrian and the vehicle, thereby screening out the independent variables of safe driving of the vehicle.
In a further embodiment, in step S41, the acquiring an infrared thermal image of the area corresponding to the suspected pedestrian includes:
starting an infrared camera to acquire an infrared thermal image corresponding to the visible light image; or controlling the infrared camera to work continuously, and calling the infrared thermal image corresponding to the visible light image when the suspected pedestrian is determined to be in the unsafe area.
The scheme comprises two infrared thermal image acquisition modes, instant awakening shooting can be selected, electric energy loss of the infrared camera is reduced, continuous shooting can also be selected, the mode of continuously comparing and analyzing the infrared thermal image and the visible light image is not needed, and both the modes can reduce the operation load of the processor.
In further embodiments, the first similarity is 1 when the height is within the preset height range, and 0 otherwise;
when the width is within the preset body width range, the second similarity is 1, otherwise, the second similarity is 0;
when the aspect ratio is within the preset aspect ratio range, the third similarity is 1, otherwise, the third similarity is 0;
if the image corresponding to the upper part, the middle part or the lower part is successfully matched with the model library, the fourth similarity, the fifth similarity or the sixth similarity is 1, otherwise, the sixth similarity is 0.
According to the scheme, a comprehensive similarity contrast range is set for the judgment of suspected pedestrians according to human body characteristics, the similarity values of two polarizations (the similarity is 1 or 0) are set, and interference factors which cannot be pedestrians are directly screened out, so that the judgment and calculation are more concise and efficient.
The invention also provides a pedestrian recognition system for realizing the pedestrian recognition method, which comprises a main control module, and a visible light camera, an infrared camera and a detection module which are connected with the main control module;
the visible light camera is used for acquiring a visible light image on a vehicle travelling route;
the detection module is used for detecting the distance between an obstacle and a vehicle on the vehicle traveling route;
the main control module is used for identifying the visible light image according to the distance and judging whether the barrier is a suspected pedestrian or not by combining a preset model;
the infrared camera is used for acquiring an infrared thermal image of an area corresponding to the suspected pedestrian;
the main control module is further used for comparing the infrared thermal image with the human body temperature distribution characteristics, so that an early warning prompt is sent out when whether the suspected pedestrian is a pedestrian is determined.
In further embodiments, the detection module comprises a millimeter wave radar and a laser radar; the main control module is a vehicle-mounted ECU.
According to the basic scheme, the pedestrian identification system is established by taking the main control module, the visible light camera, the infrared camera and the detection module which are connected with the main control module as main modules, and the pedestrians with vital signs can be accurately identified by using dual auxiliary identification of the visible light camera and the infrared camera, so that the driving safety is improved; by utilizing the dual mechanisms of the millimeter wave radar and the laser radar, the anti-interference capability aiming at severe environment can be improved, and the detection precision and the detection stability are ensured.
Drawings
Fig. 1 is a flowchart of a pedestrian recognition method according to embodiment 1 of the present invention;
fig. 2 is a system framework diagram of a pedestrian recognition system provided in embodiment 2 of the present invention;
wherein: the system comprises a main control module 1, a visible light camera 2, an infrared camera 3, a detection module 4, a millimeter wave radar 41 and a laser radar 42.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, which are given solely for the purpose of illustration and are not to be construed as limitations of the invention, including the drawings which are incorporated herein by reference and for illustration only and are not to be construed as limitations of the invention, since many variations thereof are possible without departing from the spirit and scope of the invention.
Example 1
The pedestrian identification method provided by the embodiment of the invention is shown in figure 1 and comprises the following steps:
s1, acquiring a visible light image on the vehicle traveling route;
s2, identifying the visible light image, and judging whether a suspected pedestrian exists according to a preset model, wherein the method comprises the following steps of S21-S22:
s21, recognizing the visible light image, acquiring the actual contour size of the obstacle on the vehicle traveling route, and comprising the following steps S211 to S212:
s211, acquiring edge characteristics of the obstacle and a corresponding azimuth angle from the visible light image;
and S212, detecting the distance between the vehicle and the obstacle by adopting the radar, and determining the actual outline size of the obstacle according to the distance and the imaging proportion.
S22, substituting the actual contour dimension into a preset model, calculating the similarity between the obstacle and a preset human body, and judging whether the obstacle is a suspected pedestrian according to the similarity, wherein the method comprises the following steps of S221-S226:
s221, acquiring the height and the width of the obstacle according to the actual contour size;
s222, calculating a first similarity S according to the height and a preset height range1When the height is within the preset height range, the first similarity S1Is 1, otherwise is 0;
for example, the predetermined height range is 0.3m to 2.4 m.
S223, calculating a second similarity S according to the width and the preset body width range2When the width is within the preset body width range, the second similarity S2Is 1, otherwise is 0;
for example, the predetermined body width is in the range of 0.1m to 0.5 m.
S224, comparing the height-width ratio of the obstacle with a preset height-width ratio to obtain a third similarity S3When the aspect ratio is within the preset aspect ratio range, the third similarity S3Is 1, otherwise is 0;
for example, the preset height ratio range is 1.5-24.
S225, establishing a model library in advance according to the head shoulder, the trunk and the legs of a preset human body, dividing the upper part, the middle part and the lower part of the barrier according to the proportion of the head shoulder, the trunk and the legs of the preset human body, and matching the upper part, the middle part and the lower part with the model library to obtain a corresponding fourth similarity S4The fifth similarity S5Sixth degree of similarity S6If the matching of the image corresponding to the upper/middle/lower part with the model library is successful, the fourth similarity S4Fifth degree of similarity S5Sixth degree of similarity S6Is 1, otherwise is 0.
S226, integrating the first similarity S1Sixth degree of similarity S6And substituting the total similarity into a preset algorithm to calculate to obtain the total similarity, and further judging whether the barrier is a suspected pedestrian or not.
The preset algorithm is as follows:
wherein S is the overall similarity; a isiFor the similarity weighted value, a larger number of samples can be collected to perform sample identification simulation training, so as to obtain a better numerical value.
In the determination, the determination level may be set as needed, for example:
and directly judging the obstacles with the similarity of more than 60 percent as pedestrians, judging the obstacles with the similarity of 20 to 60 percent as suspected pedestrians, and judging the obstacles with the similarity of less than 20 percent as non-pedestrians.
In other embodiments, the level boundary of similarity may also be 50%, 30%, 10%, etc.
In the embodiment, only the area determined as the suspected pedestrian is re-identified, so that only the suspected pedestrian part in the image is extracted, and the pedestrian and the non-pedestrian are determined to be processed according to the conventional operation (for example, when the pedestrian is determined, the early warning prompt is carried out, and when the pedestrian is determined, the corresponding obstacle is ignored).
In the embodiment, the edge characteristics and the azimuth angle of the obstacle can be rapidly acquired by utilizing a mature image identification technology in the first step, the distance between the vehicle and the obstacle is acquired by radar ranging in the second step, the actual outline size of the obstacle is further calculated according to the distance and the imaging proportion of the image, and before the suspected pedestrian is judged, the obstacle imaging in different distances is converted into the actual size of the real world, so that the unified judgment on all the obstacles in the whole visible light image can be facilitated.
In the embodiment, corresponding preset ranges are respectively set for local characteristics and overall characteristics of a human body, the preset ranges comprise data such as a preset height range, a preset body width range, a preset height-width ratio and a model library, the limit proportion of the human body is comprehensively covered, the probability of missing detection of pedestrians is reduced, and the accuracy of pedestrian identification is improved; in addition, the first similarity to the sixth similarity are integrated and substituted into a preset algorithm to calculate the overall similarity, and suspected pedestrians are judged according to the overall similarity, so that the fault tolerance of the identification mechanism can be further improved.
According to human body characteristics, a comprehensive similarity contrast range is set for the judgment of suspected pedestrians, the similarity values of two polarizations (the similarity is 1 or 0) are set, and interference factors which cannot be the pedestrians are directly screened out, so that the judgment and calculation are more concise and efficient.
S3, when the suspected pedestrian exists, acquiring the position information of the suspected pedestrian;
s4, acquiring an infrared thermal image of an area corresponding to the suspected pedestrian according to the position information, comparing the infrared thermal image with the human body temperature distribution characteristics, and further determining whether the suspected pedestrian is a pedestrian, wherein the method comprises the following steps of S41-S42:
and S41, judging whether the suspected pedestrian is in a safe area or not according to the position information, if so, finishing the identification, and if not, acquiring the infrared thermal image of the area corresponding to the suspected pedestrian.
Wherein, judge whether suspected pedestrian is in safe region according to positional information, specifically:
a. and judging whether the distance between the suspected pedestrian and the vehicle is greater than a safety threshold value or not according to the position information, if so, finishing the identification, and if not, entering the next step. The safety threshold can be a fixed value, and can also be set in equal proportion according to the vehicle speed, namely the higher the vehicle speed is, the larger the safety threshold is; the lower the vehicle speed, the smaller the safety threshold.
b. And identifying the visible light image, judging whether the suspected pedestrian is blocked by the guard rail, if so, ending the identification, and otherwise, judging that the suspected pedestrian is in a non-safety area.
Specifically, whether a guard rail exists in the visible light image or not can be identified, if yes, the image of the suspected pedestrian is intercepted, the image outline comparison is carried out with a guard rail shielding model, whether the suspected pedestrian and the guard rail are located on the same side or not can be determined according to whether the suspected pedestrian shields the guard rail or not, and if the suspected pedestrian shields the guard rail, whether the suspected pedestrian is shielded by the guard rail or not can be determined, namely, the suspected pedestrian is located in a safety area; otherwise, the suspected pedestrian is judged to be not blocked by the protective guard, namely the suspected pedestrian is in an unsafe area.
In the present embodiment, the guard rail is only an example for limiting the action of the pedestrian, and is not limited to the guard rail, and may be a structure for cutting off the pedestrian traveling route, such as a ditch or a road block.
The infrared thermal image of the area corresponding to the suspected pedestrian is acquired by the following two modes which can be selected according to requirements:
firstly, immediately waking up shooting, namely immediately waking up the infrared camera to acquire an infrared thermal image corresponding to the visible light image when a suspected pedestrian is detected to be in an unsafe area, so that the electric energy loss of continuous work of the infrared camera can be reduced.
And secondly, continuous shooting, namely controlling the infrared camera to continuously work, continuously acquiring and storing the infrared thermal image at the same time as the visible light image, and calling the infrared thermal image at the same time as the visible light image when the suspected pedestrian is detected to be in the unsafe area, so that the mode of continuously comparing and analyzing the infrared thermal image and the visible light image is not needed, and the operation load of the processor can be reduced.
And S42, extracting a color distribution image from the infrared thermal image, comparing the color distribution image with the human body temperature distribution characteristic, and judging whether the suspected pedestrian is a pedestrian according to the comparison result.
In the embodiment, at the beginning of the stage of judging whether the suspected pedestrian is a pedestrian, whether the suspected pedestrian is in the safety area is judged in advance, the situation that the pedestrian is in the safety area and does not influence the driving of the vehicle can be directly filtered out, only the situation influencing the driving safety of the vehicle is analyzed, and the efficiency of pedestrian identification can be improved; when the judgment is carried out, the infrared thermal image of the area corresponding to the suspected pedestrian is directly obtained, the pertinence is carried out with the human body temperature distribution rule, and the starting frequency of the infrared camera and the data quantity to be processed can be reduced.
The safety relation between the suspected pedestrian and the vehicle can be basically determined by setting the limit condition of the safety zone according to the distance between the suspected pedestrian and the vehicle and the current environment, so that the independent variables of safe driving of the vehicle can be screened out.
And S5, when the suspected pedestrian is judged to be a pedestrian, giving an early warning prompt.
The method comprises the steps of acquiring a visible light image on a vehicle travelling route in real time, and setting a preset model to screen out a suspected pedestrian area with a confusion item (namely judging whether a suspected pedestrian exists or not); when the suspected pedestrian is judged to exist, the position information of the suspected pedestrian is obtained, then when the suspected pedestrian is determined to have potential adverse effects on the safe driving of the vehicle, the infrared thermal image corresponding to the visible light image is obtained for secondary recognition, and according to the characteristic that the human body has thermal radiation, whether the suspected pedestrian is a pedestrian can be further determined, so that the accuracy of pedestrian recognition is improved, and the driving safety is improved.
Example 2
The embodiment of the invention also provides a pedestrian recognition system, which is used for realizing the pedestrian recognition method and is shown in fig. 2, and the pedestrian recognition system comprises a main control module 1, and a visible light camera 2, an infrared camera 3 and a detection module 4 which are connected with the main control module;
the visible light camera 2 is used for acquiring a visible light image on a vehicle travelling route;
the detection module 4 is used for detecting the distance between an obstacle and the vehicle on the vehicle traveling route;
the main control module 1 is used for identifying the visible light image according to the distance and judging whether the barrier is a suspected pedestrian or not by combining a preset model;
the infrared camera 3 is used for acquiring an infrared thermal image of an area corresponding to a suspected pedestrian;
the main control module 1 is also used for comparing the infrared thermal image with the human body temperature distribution characteristics, so that when determining whether a suspected pedestrian is a pedestrian, an early warning prompt is sent out.
In the present embodiment, the detection module 4 includes a millimeter wave radar 41 and a laser radar 42; the main control module 1 is a vehicle-mounted ECU.
According to the embodiment of the invention, the pedestrian identification system is established by taking the main control module 1, and the visible light camera 2, the infrared camera 3 and the detection module 4 which are connected with the main control module as main modules, and the pedestrian with vital signs can be accurately identified by using the dual auxiliary identification of the visible light camera 2 and the infrared camera 3, so that the driving safety is improved; by using the dual mechanism of the millimeter wave radar 41 and the laser radar 42, the anti-interference capability to the severe environment can be improved, and the detection accuracy and the detection stability can be ensured.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (10)
1. A pedestrian recognition method, characterized by comprising the steps of:
s1, acquiring a visible light image on the vehicle traveling route;
s2, identifying the visible light image, and judging whether a suspected pedestrian exists according to a preset model;
s3, when the suspected pedestrian exists, acquiring the position information of the suspected pedestrian;
s4, acquiring an infrared thermal image of an area corresponding to the suspected pedestrian according to the position information, comparing the infrared thermal image with the human body temperature distribution characteristics, and further determining whether the suspected pedestrian is a pedestrian;
and S5, when the suspected pedestrian is judged to be the pedestrian, sending out an early warning prompt.
2. The pedestrian identification method according to claim 1, wherein said step S2 includes:
s21, identifying the visible light image, and acquiring the actual contour size of the obstacle on the vehicle travelling route;
and S22, substituting the actual contour dimension into a preset model, calculating the similarity between the obstacle and a preset human body, and judging whether the obstacle is a suspected pedestrian according to the similarity.
3. The pedestrian recognition method according to claim 2, wherein said step S21 includes:
s211, acquiring edge characteristics of the obstacle and a corresponding azimuth angle from the visible light image;
s212, detecting the distance between the vehicle and the obstacle by adopting a radar, and determining the actual outline size of the obstacle according to the distance and the imaging proportion.
4. A pedestrian recognition method according to claim 3, wherein said step S22 includes:
s221, acquiring the height and the width of the obstacle according to the actual contour size;
s222, calculating a first similarity according to the height and a preset height range;
s223, calculating a second similarity according to the width and a preset body width range;
s224, comparing the height-width ratio of the obstacle with a preset height-width ratio range to obtain a third similarity;
s225, dividing the upper part, the middle part and the lower part of the barrier by referring to the proportion of the head, the shoulder, the trunk and the legs of the preset human body, and matching the upper part, the middle part and the lower part with a model library to obtain a fourth similarity, a fifth similarity and a sixth similarity;
s226, integrating the first similarity to the sixth similarity, substituting a preset algorithm to calculate the total similarity, and further judging whether the obstacle is a suspected pedestrian.
5. The pedestrian identification method according to claim 1, wherein said step S4 includes:
s41, judging whether the suspected pedestrian is in a safe area or not according to the position information, if so, finishing the identification, and if not, acquiring an infrared thermal image of the area corresponding to the suspected pedestrian;
and S42, extracting a color distribution image from the infrared thermal image, comparing the color distribution image with the human body temperature distribution characteristic, and judging whether the suspected pedestrian is a pedestrian according to the comparison result.
6. The method as claimed in claim 5, wherein in the step S41, the step of determining whether the suspected pedestrian is in a safe area according to the position information includes:
a. judging whether the distance between the suspected pedestrian and the vehicle is larger than a safety threshold value or not according to the position information, if so, finishing the identification, and if not, entering the next step;
b. and identifying the visible light image, judging whether the suspected pedestrian is blocked by a protective guard, if so, ending the identification, and if not, judging that the suspected pedestrian is in a non-safety area.
7. The method according to claim 5, wherein in the step S41, the step of acquiring the infrared thermal image of the area corresponding to the suspected pedestrian comprises:
starting an infrared camera to acquire an infrared thermal image corresponding to the visible light image; or controlling the infrared camera to work continuously, and calling the infrared thermal image corresponding to the visible light image when the suspected pedestrian is determined to be in the unsafe area.
8. A pedestrian recognition method according to claim 4, characterized in that:
when the height is within the preset height range, the first similarity is 1, otherwise, the first similarity is 0;
when the width is within the preset body width range, the second similarity is 1, otherwise, the second similarity is 0;
when the aspect ratio is within the preset aspect ratio range, the third similarity is 1, otherwise, the third similarity is 0;
if the image corresponding to the upper part, the middle part or the lower part is successfully matched with the model library, the fourth similarity, the fifth similarity or the sixth similarity is 1, otherwise, the sixth similarity is 0.
9. A pedestrian recognition system for implementing a pedestrian recognition method according to claims 1 to 8, characterized in that: the system comprises a main control module, and a visible light camera, an infrared camera and a detection module which are connected with the main control module;
the visible light camera is used for acquiring a visible light image on a vehicle travelling route;
the detection module is used for detecting the distance between an obstacle and a vehicle on the vehicle traveling route;
the main control module is used for identifying the visible light image according to the distance and judging whether the barrier is a suspected pedestrian or not by combining a preset model;
the infrared camera is used for acquiring an infrared thermal image of an area corresponding to the suspected pedestrian;
the main control module is further used for comparing the infrared thermal image with the human body temperature distribution characteristics, so that an early warning prompt is sent out when whether the suspected pedestrian is a pedestrian is determined.
10. A pedestrian identification system in accordance with claim 9, wherein:
the detection module comprises a millimeter wave radar and a laser radar;
the main control module is a vehicle-mounted ECU.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011328702.4A CN112464782A (en) | 2020-11-24 | 2020-11-24 | Pedestrian identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011328702.4A CN112464782A (en) | 2020-11-24 | 2020-11-24 | Pedestrian identification method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112464782A true CN112464782A (en) | 2021-03-09 |
Family
ID=74799199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011328702.4A Pending CN112464782A (en) | 2020-11-24 | 2020-11-24 | Pedestrian identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112464782A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113246931A (en) * | 2021-06-11 | 2021-08-13 | 创新奇智(成都)科技有限公司 | Vehicle control method and device, electronic equipment and storage medium |
| CN113313023A (en) * | 2021-05-28 | 2021-08-27 | 的卢技术有限公司 | Method and system for guaranteeing driving safety when pet is intelligently sensed in vehicle |
| CN115236661A (en) * | 2022-08-22 | 2022-10-25 | 深圳市海兴科技有限公司 | Human body existence detection method based on microwave induction and microwave inductor |
| CN116183488A (en) * | 2022-12-13 | 2023-05-30 | 川易机电设备启东有限公司 | Pressure pipeline defect detection system and method based on image analysis |
| CN116311383A (en) * | 2023-05-16 | 2023-06-23 | 成都航空职业技术学院 | Intelligent building power consumption management system based on image processing |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005267030A (en) * | 2004-03-17 | 2005-09-29 | Daihatsu Motor Co Ltd | Pedestrian outline extraction method and pedestrian outline extraction device |
| CN104823218A (en) * | 2012-12-03 | 2015-08-05 | 哈曼国际工业有限公司 | System and method for detecting pedestrians using a single normal camera |
| JP2015143966A (en) * | 2013-07-25 | 2015-08-06 | 株式会社リコー | Image processor, solid object detection method, solid object detection program, and moving object control system |
| US20160019429A1 (en) * | 2014-07-17 | 2016-01-21 | Tomoko Ishigaki | Image processing apparatus, solid object detection method, solid object detection program, and moving object control system |
| WO2020117601A1 (en) * | 2018-12-03 | 2020-06-11 | Trumpore Ryan | Systems and methods for improving vehicle operations using movable sensors |
| CN111386701A (en) * | 2017-12-04 | 2020-07-07 | 索尼公司 | Image processing apparatus, image processing method, and program |
-
2020
- 2020-11-24 CN CN202011328702.4A patent/CN112464782A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005267030A (en) * | 2004-03-17 | 2005-09-29 | Daihatsu Motor Co Ltd | Pedestrian outline extraction method and pedestrian outline extraction device |
| CN104823218A (en) * | 2012-12-03 | 2015-08-05 | 哈曼国际工业有限公司 | System and method for detecting pedestrians using a single normal camera |
| JP2015143966A (en) * | 2013-07-25 | 2015-08-06 | 株式会社リコー | Image processor, solid object detection method, solid object detection program, and moving object control system |
| US20160019429A1 (en) * | 2014-07-17 | 2016-01-21 | Tomoko Ishigaki | Image processing apparatus, solid object detection method, solid object detection program, and moving object control system |
| CN111386701A (en) * | 2017-12-04 | 2020-07-07 | 索尼公司 | Image processing apparatus, image processing method, and program |
| WO2020117601A1 (en) * | 2018-12-03 | 2020-06-11 | Trumpore Ryan | Systems and methods for improving vehicle operations using movable sensors |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113313023A (en) * | 2021-05-28 | 2021-08-27 | 的卢技术有限公司 | Method and system for guaranteeing driving safety when pet is intelligently sensed in vehicle |
| CN113246931A (en) * | 2021-06-11 | 2021-08-13 | 创新奇智(成都)科技有限公司 | Vehicle control method and device, electronic equipment and storage medium |
| CN115236661A (en) * | 2022-08-22 | 2022-10-25 | 深圳市海兴科技有限公司 | Human body existence detection method based on microwave induction and microwave inductor |
| CN115236661B (en) * | 2022-08-22 | 2023-04-07 | 深圳市海兴科技有限公司 | Human body existence detection method based on microwave induction and microwave inductor |
| CN116183488A (en) * | 2022-12-13 | 2023-05-30 | 川易机电设备启东有限公司 | Pressure pipeline defect detection system and method based on image analysis |
| CN116183488B (en) * | 2022-12-13 | 2023-09-15 | 川易机电设备启东有限公司 | Pressure pipeline defect detection system and method based on image analysis |
| CN116311383A (en) * | 2023-05-16 | 2023-06-23 | 成都航空职业技术学院 | Intelligent building power consumption management system based on image processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112464782A (en) | Pedestrian identification method and system | |
| US20200175330A1 (en) | System and process for detecting, tracking and counting human objects of interest | |
| Kwak et al. | Pedestrian tracking using online boosted random ferns learning in far-infrared imagery for safe driving at night | |
| CN105373135B (en) | A kind of method and system of aircraft docking guidance and plane type recognition based on machine vision | |
| CN105809679B (en) | Mountain railway side slope rockfall detection method based on visual analysis | |
| CN102855622B (en) | A kind of infrared remote sensing image sea ship detection method based on significance analysis | |
| CN106778655B (en) | Human body skeleton-based entrance trailing entry detection method | |
| CN103034843B (en) | Method for detecting vehicle at night based on monocular vision | |
| CN108021856A (en) | Light for vehicle recognition methods, device and vehicle | |
| CN101465033A (en) | Automatic tracking recognition system and method | |
| CN115205796B (en) | Rail line foreign matter intrusion monitoring and risk early warning method and system | |
| CN112071084A (en) | Method and system for judging illegal parking by utilizing deep learning | |
| CN111667655A (en) | Infrared image-based high-speed railway safety area intrusion alarm device and method | |
| CN114140503A (en) | Power distribution network dangerous area identification device and method based on deep learning | |
| Yang et al. | On-road collision warning based on multiple FOE segmentation using a dashboard camera | |
| CN116798176A (en) | Data management system based on big data and intelligent security | |
| CN111582084A (en) | Weak supervision learning-based rail foreign matter detection method and system under empty base view angle | |
| CN105426852A (en) | Method for identifying pedestrians by vehicle-mounted monocular long-wave infrared camera | |
| CN107463886A (en) | A kind of double method and systems for dodging identification and vehicle obstacle-avoidance | |
| Guo et al. | Visibility detection based on the recognition of the preceding vehicle’s taillight signals | |
| Atiq et al. | Vehicle detection and shape recognition using optical sensors: a review | |
| CN114708544A (en) | Intelligent violation monitoring helmet based on edge calculation and monitoring method thereof | |
| CN115169673A (en) | Intelligent campus epidemic risk monitoring and early warning system and method | |
| CN114912536A (en) | Target identification method based on radar and double photoelectricity | |
| CN114581863A (en) | Vehicle dangerous state identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |