CN111856606A - Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging - Google Patents
Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging Download PDFInfo
- Publication number
- CN111856606A CN111856606A CN201910708437.3A CN201910708437A CN111856606A CN 111856606 A CN111856606 A CN 111856606A CN 201910708437 A CN201910708437 A CN 201910708437A CN 111856606 A CN111856606 A CN 111856606A
- Authority
- CN
- China
- Prior art keywords
- image
- infrared
- gray
- thermal imaging
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001931 thermography Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 22
- 241001465754 Metazoa Species 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000009432 framing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 16
- 230000004888 barrier function Effects 0.000 abstract description 2
- 238000003384 imaging method Methods 0.000 description 18
- 239000000523 sample Substances 0.000 description 13
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 238000003331 infrared imaging Methods 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 229910052732 germanium Inorganic materials 0.000 description 1
- GNPVGFCGXDBREM-UHFFFAOYSA-N germanium atom Chemical compound [Ge] GNPVGFCGXDBREM-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V8/00—Prospecting or detecting by optical means
- G01V8/10—Detecting, e.g. by using light barriers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
- G01J5/485—Temperature profile
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geophysics (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging, wherein the method comprises the steps of obtaining the current ambient temperature and the infrared temperature gray-scale value of a target; obtaining a brightness adjustment weight value of image output according to the current environment temperature and the infrared temperature gray-scale value of the target; adjusting the brightness of all gray level image data according to the brightness adjustment weight; processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image; establishing an image sample library; identifying a target in the gray-scale image source image according to the image sample library; and outputting the identification result through the whole vehicle control bus. The invention not only can solve the problem that the vision of a driver is limited under the scene of weak light or no light, but also realizes the identification and detection of the barrier, and simultaneously can solve the problem of dazzling discomfort caused by the driving of a vehicle by the opposite side by turning on a high beam.
Description
Technical Field
The invention relates to an intelligent driving auxiliary device, in particular to a forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging.
Background
The infrared thermal imaging is a technology based on passive infrared light receiving and post-processing imaging, is applied to a forward-looking intelligent driving assistance system (ADAS), makes up for driving scenes of weak visible light, and is really assisted in all-weather imaging.
An Advanced Driver Assistance System (ADAS), which is an active safety technology that collects environmental data inside and outside a vehicle at the first time by using various sensors installed on the vehicle, and performs technical processes such as identification, detection, tracking and the like of static and dynamic objects, so that a Driver can perceive possible dangers at the fastest time to draw attention and improve safety. The ADAS uses sensors, such as cameras, radars, lasers, and ultrasonic waves, which detect light, heat, pressure, or other variables used to monitor the state of the vehicle, and are usually located in the front and rear bumpers, side-view mirrors, and the inside of the steering column or on the windshield of the vehicle. Early ADAS technologies were primarily based on passive warning, which alerts motorists to abnormal vehicle or road conditions when a potential hazard is detected in the vehicle. Proactive intervention is also common with the latest ADAS technologies.
The current forward-looking intelligent driving auxiliary system mainly simulates the characteristics of human eyes to complete a series of algorithms, namely a camera based on visible light is used for detecting the front road condition in the driving process, the functions of target detection, pedestrian detection, lane line detection and the like are realized through a specific algorithm, and the driving auxiliary system combines the driving conditions of an automobile and gives an alarm to a driver when an abnormal condition occurs.
However, the forward-looking ADAS system acquires an image based on a camera of visible light, and then realizes a detection process through an identification algorithm, so that the quality of the image has a great influence on a judgment result, and the forward-looking ADAS system completely loses the effect under the condition of no visible light.
At present, the image recognition function of the infrared imaging system is realized in a system framework form of a Field Programmable Gate Array (FPGA) and a Digital Signal Processor (DSP), wherein the FPGA finishes the work of infrared imaging or image preprocessing and the like, and the DSP is responsible for the functions of image recognition and the like. The circuit structure is huge, the hardware structure is complex, and the cost is high.
Moreover, the existing infrared thermal imaging device is basically designed in an integrated manner, the detector, the image processing interface and the image output interface are integrated on one device, the device can directly output images, and when the subsequent image recognition processing is needed, the images are converted into digital video data through a special circuit and then are transmitted to a subsequent processor or other computer systems for image recognition. This is a complex and costly circuit.
Disclosure of Invention
It is to be understood that both the foregoing general description and the following detailed description of the present disclosure are exemplary and explanatory and are intended to provide further explanation of the disclosure as claimed.
Aiming at the problems, the invention not only can solve the problem that the vision field of a driver is limited under the scene with weak light or no light, but also adds the identification algorithm of target detection, realizes the identification and detection of the barrier, reminds the owner of the vehicle in time, can solve the problem of dazzling discomfort caused by the driving of a vehicle at the opposite side by turning on a high beam, ensures the driving safety of the driver, and adopts a simple hardware structure to solve the problems of complex structure, high cost and the like of an infrared image imaging and identification circuit in the prior art.
The invention discloses a forward-looking intelligent driving assistance method based on infrared thermal imaging, which is characterized in that,
acquiring a current environment temperature and an infrared temperature gray-scale value of a target;
acquiring a brightness adjustment weight of image output according to the current environment temperature and the infrared temperature gray-scale value of the target;
thirdly, adjusting the brightness of all the gray level image data according to the brightness adjustment weight value;
Processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
step five, establishing an image sample library;
sixthly, identifying the target in the gray-scale image source image according to the image sample library;
and step seven, outputting the identification result through a whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving assistance method based on infrared thermal imaging, which is characterized in that,
in the second step, the method further comprises the following steps:
and counting the temperature distribution according to the infrared temperature gray-scale value of the current environment temperature, and finding out the intermediate value of the infrared temperature gray-scale value in the maximum distribution area, wherein the weight is obtained by comparing the current environment temperature with the environment temperature value corresponding to the intermediate value.
Preferably, the invention further discloses a forward-looking intelligent driving assistance method based on infrared thermal imaging, which is characterized in that,
in the sixth step, the method further comprises:
and framing the target in the current image according to the identification result, and performing image superposition processing.
Preferably, the invention further discloses a forward-looking intelligent driving assistance method based on infrared thermal imaging, which is characterized in that,
The image sample library is formed by marking the video or the image with the target to be identified according to the data of the frame.
Preferably, the invention further discloses a forward-looking intelligent driving assistance method based on infrared thermal imaging, which is characterized in that,
the image sample library contains vehicle, human, animal and obstacle picture objects.
The invention also discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized by comprising the following components:
the infrared detector assembly receives an infrared light signal of a forward-looking object;
the FPGA special interface is connected with the output end of the infrared detector assembly and is used for processing the infrared light signal into data suitable for transmission;
the LVDS serializer is connected with the output end of the FPGA special interface and is used for serializing the infrared light signal into LVDS serial data;
the image processor is connected with the output end of the LVDS serializer, images and identifies the received LVDS serial data and then outputs the LVDS serial data, and is configured to:
acquiring the current ambient temperature and the infrared temperature gray-scale value of a target;
obtaining a brightness adjustment weight value of image output according to the current environment temperature and the infrared temperature gray-scale value of the target;
Adjusting the brightness of all the gray image data according to the brightness adjustment weight and outputting the gray image data;
processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
establishing an image sample library;
identifying a target in the gray-scale image source image according to the image sample library;
and outputting the identification result through the whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
and the image processor outputs an image superposition processing result to a whole vehicle display system and outputs the identification result to the whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
the infrared detector assembly includes a 14-bit detector.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
the control bus comprises a CAN bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
The image sample library is obtained by marking an object to be identified by providing video or image data according to frames, and the image sample library comprises vehicle, human, animal and obstacle picture objects.
By adopting the device and the method with the structure, the problems of complex structure, high cost and the like of an infrared image imaging and identifying circuit in the prior art are solved by using a simple hardware structure, and a better effect is achieved on the infrared image imaging and identifying.
Drawings
Embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Reference will now be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. Further, although the terms used in the present disclosure are selected from publicly known and used terms, some of the terms mentioned in the specification of the present disclosure may be selected by the applicant at his or her discretion, the detailed meanings of which are described in relevant parts of the description herein. Furthermore, it is required that the present disclosure is understood, not simply by the actual terms used but by the meaning of each term lying within.
The above and other objects, features and advantages of the present invention will become apparent to those skilled in the art from the following detailed description of the present invention with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram of a front-view intelligent driving assistance device based on infrared thermal imaging according to the present invention;
FIG. 2 is a flow chart of thermal imaging image processing employed in the present invention;
fig. 3 is a flow chart of an object detection algorithm employed in the present invention.
Reference numerals
11-infrared detector assembly
12-FPGA special interface
13-LVDS serializer
14-image processor
Detailed Description
This specification discloses one or more embodiments that incorporate the features of this invention. The disclosed embodiments are merely illustrative of the invention. The scope of the invention is not limited to the disclosed embodiments. The invention is defined by the appended claims.
References in the specification to "one embodiment," "an example embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but all embodiments do not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
Moreover, it should be understood that the spatial descriptions used herein (e.g., above, below, above, left, right, below, top, bottom, vertical, horizontal, etc.) are for purposes of illustration only, and that an actual implementation of the structures described herein may be spatially arranged in any orientation or manner.
All objects in nature radiate infrared radiation when their temperature is above absolute zero (i.e., -273 c). The amount of radiant energy and the distribution of energy by wavelength is determined by the surface temperature of the object. According to the characteristic that infrared rays with different wavelengths are radiated by different objects and targets due to different temperatures, after the infrared detector receives the infrared rays, an image processor processes the details of the targets to realize imaging, and the imaging is called infrared thermal imaging.
The image processing algorithm in the system is to correspond the obtained digital signals and the brightness of the images through filtering and amplifying according to the electric signal content (temperature data) output to the image processor by the detector, display the digital signals and the brightness of the images into spatial distribution images of all targets on a plane, and finally present the road condition scene in front to a vehicle owner in a gray scale image form on a vehicle-mounted display screen.
The infrared is mainly used for the weak condition of visible light, and for better driving experience and safety guarantee, a special target detection algorithm is integrated in the system, so that the system can detect and early warn targets in the front infrared sensing range during imaging, and driving safety is guaranteed. The algorithm models are trained based on image data of infrared thermal imaging, and the recognition rate is improved to a great extent.
The system consists of a thermal imaging system based on infrared radiation and an image recognition part aiming at thermal imaging.
Infrared thermal imaging section: infrared optical lens structure can be projected infrared detector to the infrared ray of thermal radiation wave band, and the preceding germanium piece of camera lens can filter the visible light and only let the infrared light pass through to guarantee infrared imaging effect. The detector receives infrared rays and converts the infrared rays into electric signals of temperature gray scale values, the electric signals are called as infrared original data, after the back-end image processor receives the original data, the processed data are sent to an image generation algorithm for imaging through a filtering and enhancing algorithm, and the result is a gray image based on brightness information.
Image generation rationale: the 14-bit detector data is filtered by a filtering algorithm to remove blind pixels and invalid data, so that normal phase metadata is guaranteed to be sent to a subsequent imaging algorithm, and the image algorithm is to convert 14-bit original data into 8-bit YUV data which can be imaged to perform imaging.
A visual recognition section: based on the R _ CNN neural network model, training the algorithm model by adopting a special image sample library of infrared thermal imaging to obtain a visual recognition algorithm model of infrared thermal imaging, transmitting video data of thermal imaging into the algorithm model, outputting a recognition result of a trained target object, and framing the target result by image superposition processing to display in a display system.
In order to enable the whole vehicle to have more prompts and driving experiences on the recognition result, the recognition result is also sent to the CAN bus in the system and is shared by other systems of the whole vehicle.
As shown in fig. 1, the forward-looking intelligent driving assistance device based on infrared thermal imaging of the present invention includes an infrared detector assembly 11, an FPGA dedicated interface 12, an LVDS serializer 13, an image processor 14 and its output.
The image processor 14 performs image imaging, image recognition and image output on the data, and respectively outputs the recognition result and the imaging to a display system and a control bus of the whole vehicle.
The data acquisition is completed by independent equipment, namely infrared data is independently acquired by an infrared detector component 11, processed into data suitable for transmission through an FPGA special interface 12 and then converted into LVDS serial data suitable for transmission by an LVDS serializer 13, and a controller end of an image processor 14 receives the data and then deserializes the data, and then inputs the data into an image processor with an ISP to perform imaging and image recognition processing.
The special interface of the invention is realized by adopting FPGA, generates data analysis of clock and detector communication, and realizes control time sequence, such as: reset, configuration, sync, and OCC, etc.
Referring to FIG. 2, the process flow of thermal imaging according to the present invention is further detailed as follows:
for example, the probe is 14 bits, and the output temperature range is 0 to (2)141), the image processor 14 counts the area where the data output by the infrared detector assembly 11 is most, and finds out the distribution relation;
step 23, the image processor 14 finds out the middle value of the infrared temperature gray-scale value in the maximum distribution area;
in this step, since the infrared detector assembly 11 senses the temperature signal, the temperature sensed by the detector 11 is usually in a certain range, such as 100 ℃, and a particular point of the present invention is that this range is not fixed in the actual environment, such as selectable (0 ℃ to 100 ℃) or (-40 ℃ to 60 ℃) according to the environmental temperature distribution, when the target scene is imaged, in order to achieve good imaging effect, the image processor 14 performs recognition processing on the respective space occupation ratios of the temperatures, the difference value of the temperatures is reflected on the brightness value of the image, and the processing is weakened when the non-main target exceeds the temperature range.
it should be noted that there are several common representation modes of an image, namely RGB, i.e. three primary colors of red, green and blue, and another is YUV, i.e. brightness and color difference, where Y is brightness and UV represents color difference. Whereas for a grey-scale image, i.e. a normally black and white image, there is only brightness data, Y values, no UV values.
The infrared thermal imaging is realized by converting a temperature gray level value output by an infrared detector into a brightness value, namely a Y value, according to a corresponding relation;
and 29, outputting the video through the vehicle-mounted video output terminal, namely imaging according to the adjusted gray level.
In summary, the image processing algorithm in the system is to correspond the brightness of the obtained digital signal and the image to display the spatial distribution image of each target on the plane through filtering and amplifying according to the content (temperature data) of the electric signal output to the image processor by the detector, and finally present the road condition scene in front to the vehicle owner in the form of a gray scale image on the vehicle-mounted display screen.
The infrared detector assembly 11 receives infrared rays and converts the infrared rays into electric signals of temperature gray scale values, the electric signals are called infrared original data, and after the image processor 14 receives the original data, the processed data are subjected to filtering and enhancement algorithms and then imaged by an image generation algorithm to form a gray scale image source image based on brightness information.
The infrared light is mainly used under the weak condition of visible light, and for better driving experience and safety guarantee, a target detection algorithm is further integrated in the system, so that the target in the infrared sensing range in front can be identified, detected and early warned during imaging, driving safety is guaranteed, and the target in the infrared image is identified through the target detection algorithm.
Meanwhile, as shown in fig. 3, the specific steps of the target detection process are as follows:
and step 35, outputting the identification result through the whole vehicle control bus.
In the process shown in fig. 3, in order to make the entire vehicle have more prompt and driving experience on the recognition result, the system also sends the recognition result to the entire vehicle control bus including the CAN bus, and the recognition result is shared by other systems of the entire vehicle.
In summary, the invention adopts a simple hardware structure to solve the problems of complex circuit structure, high cost and the like of infrared image imaging and identification in the prior art, not only can solve the problem of limited vision of a driver in a scene with weak light or no light, but also adds an identification algorithm of target detection to realize identification and detection of obstacles and prompt a vehicle owner in time, and simultaneously can solve the problem of dazzling discomfort caused by turning on a high beam of a vehicle driven by the opposite side, and ensure the driving safety of the driver.
In order to meet the long-distance transmission requirement of the vehicle gauge, the camera part and the controller part use the POC technology and adopt the FARKA connector for connection, so that the reliability is high.
The previous description of the preferred embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein may be applied to other embodiments without the use of the inventive faculty. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A forward-looking intelligent driving assistance method based on infrared thermal imaging is characterized in that,
acquiring a current environment temperature and an infrared temperature gray-scale value of a target;
acquiring a brightness adjustment weight of image output according to the current environment temperature and the infrared temperature gray-scale value of the target;
thirdly, adjusting the brightness of all the gray level image data according to the brightness adjustment weight value;
processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
step five, establishing an image sample library;
sixthly, identifying the target in the gray-scale image source image according to the image sample library;
And step seven, outputting the identification result through a whole vehicle control bus.
2. The infrared thermal imaging-based forward-looking intelligent driving assistance method according to claim 1,
in the second step, the method further comprises the following steps:
and counting the temperature distribution according to the infrared temperature gray-scale value of the current environment temperature, and finding out the intermediate value of the infrared temperature gray-scale value in the maximum distribution area, wherein the weight is obtained by comparing the current environment temperature with the environment temperature value corresponding to the intermediate value.
3. Forward looking intelligent driving assistance method based on infrared thermal imaging according to claim 1 or 2,
in the sixth step, the method further comprises:
and framing the target in the current image according to the identification result, and performing image superposition processing.
4. The infrared thermal imaging-based forward-looking intelligent driving assistance method according to claim 3,
the image sample library is formed by marking the video or the image with the target to be identified according to the data of the frame.
5. The infrared thermal imaging-based forward-looking intelligent driving assistance method according to claim 4,
the image sample library contains vehicle, human, animal and obstacle picture objects.
6. A forward-looking intelligent driving assistance device based on infrared thermal imaging is characterized by comprising:
the infrared detector assembly receives an infrared light signal of a forward-looking object;
the FPGA special interface is connected with the output end of the infrared detector assembly and is used for processing the infrared light signal into data suitable for transmission;
the LVDS serializer is connected with the output end of the FPGA special interface and is used for serializing the infrared light signal into LVDS serial data;
the image processor is connected with the output end of the LVDS serializer, images and identifies the received LVDS serial data and then outputs the LVDS serial data, and is configured to:
acquiring the current ambient temperature and the infrared temperature gray-scale value of a target;
obtaining a brightness adjustment weight value of image output according to the current environment temperature and the infrared temperature gray-scale value of the target;
adjusting the brightness of all the gray image data according to the brightness adjustment weight and outputting the gray image data;
processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
establishing an image sample library;
identifying a target in the gray-scale image source image according to the image sample library;
And outputting the identification result through the whole vehicle control bus.
7. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 6,
and the image processor outputs an image superposition processing result to a whole vehicle display system and outputs the identification result to the whole vehicle control bus.
8. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 6,
the infrared detector assembly includes a 14-bit detector.
9. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 7,
the control bus comprises a CAN bus.
10. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 6,
the image sample library is obtained by marking an object to be identified by providing video or image data according to frames, and the image sample library comprises vehicle, human, animal and obstacle picture objects.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910708437.3A CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910708437.3A CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111856606A true CN111856606A (en) | 2020-10-30 |
| CN111856606B CN111856606B (en) | 2023-06-06 |
Family
ID=72970586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910708437.3A Active CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111856606B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115376092A (en) * | 2022-10-21 | 2022-11-22 | 广州万协通信息技术有限公司 | Image recognition method and device in auxiliary driving |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101791931A (en) * | 2009-02-03 | 2010-08-04 | 上海保隆汽车科技股份有限公司 | Tire pressure monitoring system |
| CN101852869A (en) * | 2010-05-18 | 2010-10-06 | 中国民航大学 | Airfield runway foreign object detection system |
| CN102323275A (en) * | 2011-08-10 | 2012-01-18 | 宁波大学 | Imaging detection device of infrared chalcogenide glass internal macroscopic defect |
| US20180239055A1 (en) * | 2017-02-23 | 2018-08-23 | Qualcomm Incorporated | Foreign object detection using infrared sensing |
| CN109308817A (en) * | 2018-09-03 | 2019-02-05 | 山东大学 | Position is shared in construction tunnel and traffic safety hidden danger early warning system and method |
-
2019
- 2019-08-01 CN CN201910708437.3A patent/CN111856606B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101791931A (en) * | 2009-02-03 | 2010-08-04 | 上海保隆汽车科技股份有限公司 | Tire pressure monitoring system |
| CN101852869A (en) * | 2010-05-18 | 2010-10-06 | 中国民航大学 | Airfield runway foreign object detection system |
| CN102323275A (en) * | 2011-08-10 | 2012-01-18 | 宁波大学 | Imaging detection device of infrared chalcogenide glass internal macroscopic defect |
| US20180239055A1 (en) * | 2017-02-23 | 2018-08-23 | Qualcomm Incorporated | Foreign object detection using infrared sensing |
| CN109308817A (en) * | 2018-09-03 | 2019-02-05 | 山东大学 | Position is shared in construction tunnel and traffic safety hidden danger early warning system and method |
Non-Patent Citations (1)
| Title |
|---|
| 郑贵林，等: "自旋翼飞机电力巡线技术研究与应用", 《中国电力》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115376092A (en) * | 2022-10-21 | 2022-11-22 | 广州万协通信息技术有限公司 | Image recognition method and device in auxiliary driving |
| CN115376092B (en) * | 2022-10-21 | 2023-02-28 | 广州万协通信息技术有限公司 | Image recognition method and device in auxiliary driving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111856606B (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7105754B2 (en) | IMAGING DEVICE AND METHOD OF CONTROLLING IMAGING DEVICE | |
| US8767074B2 (en) | System and method of assisting visibility of driver | |
| US20070115138A1 (en) | Onboard imaging apparatus | |
| US10719949B2 (en) | Method and apparatus for monitoring region around vehicle | |
| US20220201183A1 (en) | Image recognition device and image recognition method | |
| JP2008135856A (en) | Body recognizing device | |
| JP6803989B2 (en) | Solid-state image sensor and its driving method | |
| US20220368867A1 (en) | Imaging device | |
| CN114868148A (en) | Information processing device, information processing system, information processing program, and information processing method | |
| CN111856606B (en) | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging | |
| JP2006341713A (en) | System and method for changing irradiation direction of head lamp | |
| JP2007139564A (en) | Obstacle detection system, obstacle detection method and computer program | |
| JP2005033680A (en) | Image processing apparatus for vehicle | |
| WO2017158829A1 (en) | Display control device and display control method | |
| JPH02190978A (en) | Visual sense recognizing device for vehicle | |
| JP2007069777A (en) | Obstacle detection system | |
| KR102473404B1 (en) | Apparatus for providing top view | |
| KR101030210B1 (en) | Obstacle recognition system for vehicle and method thereof | |
| KR102614494B1 (en) | Non-identical camera based image processing device | |
| JP2007030753A (en) | Obstacle display system and method | |
| TWI842952B (en) | Camera | |
| KR20200072590A (en) | Method And Apparatus for Detection of Parking Loss for Automatic Parking | |
| WO2022249562A1 (en) | Signal processing device, method, and program | |
| KR102497614B1 (en) | Adaptive univeral monitoring system and driving method threrof | |
| KR102339358B1 (en) | Vehicular device for improving visibility in adverse weather conditions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |