CN108896989B

CN108896989B - Millimeter wave radar imaging and pattern recognition

Info

Publication number: CN108896989B
Application number: CN201810443714.8A
Authority: CN
Inventors: 范立国; 陈大伟
Original assignee: Listenfor Communication Technology Co Ltd
Current assignee: Listenfor Communication Technology Co Ltd
Priority date: 2018-05-10
Filing date: 2018-05-10
Publication date: 2020-01-24
Anticipated expiration: 2038-05-10
Also published as: CN108896989A

Abstract

The invention relates to a millimeter wave radar imaging method, which comprises the steps of obtaining data under standard humidity and temperature and actual detection data of a millimeter wave radar under different humidities and temperatures, analyzing influence factors of humidity and temperature parameters on millimeter wave radar imaging through a training system, feeding back and correcting result data of millimeter wave radar imaging according to the influence factors, and fusing the corrected millimeter wave radar imaging result with high-resolution thermal imaging equipment to obtain a final imaging result. The millimeter wave radar imaging method has the advantages of good real-time performance and robustness, low false alarm rate and small influence by environment.

Description

Millimeter wave radar imaging and pattern recognition

Technical Field

The invention relates to the field of artificial intelligence, in particular to an imaging technology and a mode recognition technology of a millimeter wave radar.

Background

The machine vision system adopts the photosensitive element to simulate the vision system of human eyes and adopts the image processing technology to simulate the processing of human brain to pictures, thereby realizing the perception of real space information, having the advantages of large information acquisition quantity, rapid technical development, wide detection range and the like, and in recent years, the development of machine vision to the intelligent identification of targets is undoubtedly and greatly promoted to the deep research of computer science on artificial intelligence. Aiming at a vision system, the most important is real-time performance and robustness, the normal work of the vision system is ensured, the false alarm rate is reduced, the performance of the vision system is determined by the self performance of various machine vision sensors, and the problems of fusion of multiple sensors, sensor data correction and the like are also involved.

Among various existing machine vision sensors, millimeter wave radar is widely applied to multiple fields such as automatic driving, intelligent recognition and detection and the like due to excellent performance and low cost. Millimeter wave radars are radars that operate in the millimeter wave band (millimeter wave) for detection. The millimeter wave generally refers to the wave band of 30-300GHz frequency domain (wavelength 1-10 nm). The wavelength of the millimeter wave is between that of microwave and centimeter wave, so the millimeter wave radar has the advantages of both microwave radar and photoelectric radar. But millimeter wave radars currently exist: 1) signal attenuation in high humidity environments such as rain, fog and wet snow and 2) the problem that the measurement signal is relatively greatly affected by temperature changes.

In summary, an imaging technique based on a millimeter wave radar, which has good real-time performance and robustness, low false alarm rate and small environmental impact, needs to be provided.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the imaging technology based on the millimeter wave radar is good in instantaneity and robustness, low in false alarm rate and small in environmental influence.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a millimeter wave radar imaging method includes the steps that data under standard humidity and temperature and actual detection data of a millimeter wave radar under different humidities and temperatures are obtained, influence factors of humidity parameters on millimeter wave radar imaging are analyzed through a training system, result data of millimeter wave radar imaging are corrected according to the influence factors in a feedback mode, and the corrected millimeter wave radar imaging result is fused with high-resolution thermal imaging equipment to obtain a final imaging result.

Preferably, the standard humidity/temperature is a normal humidity/temperature for a certain area.

Preferably, the standard humidity is between 25% and 40% and the standard temperature is between 20-30 degrees celsius.

Preferably, the actual detection data includes: humidity, temperature, millimeter wave radar transmission frequency and reception frequency signal data.

Preferably, the millimeter wave radar signal data includes signal transmission-return time, transmission signal strength, transmission signal frequency, reflection signal strength, reflection signal frequency. The data detection step further includes testing of the test distance and/or product type and surface using a high resolution thermal imaging device.

Preferably, analyzing the influence factor of the humidity parameter on the millimeter wave radar imaging through the training system comprises the step of obtaining the relation of humidity-transmitting signal-testing distance-testing surface-receiving signal.

Preferably, the method further comprises the step of fusing the correction data of the millimeter wave radar and the data of the high-resolution thermal imaging device to obtain a final detection result.

Preferably, the millimeter wave radar imaging system comprises a millimeter wave radar 1, a humidity measuring device 2, a temperature measuring device 3, a high resolution thermal imaging device 4, a training learning system 5, a processor 6 and a pattern recognition device 7. The millimeter wave radar imaging system for performing the millimeter wave radar imaging method of claim 1.

Preferably, the humidity and temperature measuring device may be on the outer wall of the housing or on the inner wall of the housing of the millimeter wave radar.

Preferably, the millimeter wave radar includes a signal transmitter and a signal receiver.

The millimeter wave radar imaging and mode recognition method provided by the invention can realize the technical effects of good real-time performance and robustness, low false alarm rate and small environmental influence.

Drawings

Fig. 1 is a schematic diagram of a front-end housing of a millimeter-wave radar main body according to the present invention.

Fig. 2 is an internal structure view of the millimeter wave radar provided by the present invention with the front end case removed.

Fig. 3 is a millimeter wave radar imaging system provided by the present invention.

Detailed Description

A millimeter wave radar imaging system and method of the present invention will be described in further detail below.

The present invention will now be described in more detail with reference to the accompanying drawings, in which preferred embodiments of the invention are shown, it being understood that one skilled in the art may modify the invention herein described while still achieving the beneficial results of the present invention. Accordingly, the following description should be construed as broadly as possible to those skilled in the art and not as limiting the invention.

In the interest of clarity, not all features of an actual implementation are described. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific details must be set forth in order to achieve the developer's specific goals.

In order to make the objects and features of the present invention more comprehensible, embodiments of the present invention are described in detail below with reference to the accompanying drawings. It is to be noted that the drawings are in a very simplified form and are intended to use non-precision ratios for the purpose of facilitating and clearly facilitating the description of the embodiments of the invention.

Fig. 1 is a schematic structural diagram of a millimeter wave radar transmitter provided by the present invention, which includes a front-end housing 110, a radar millimeter wave signal transmitter 120, two millimeter wave radar signal receivers 130, an infrared thermal imaging device 140, an upper-end outer housing 150, and a lower-end outer housing 160, wherein an optical guiding device is located between the two millimeter wave radar signal receivers 130.

The shell is made of aluminum alloy and is in a curved surface shape, the front end shell 110, the upper end shell 150 and the lower end shell 160 are mechanically matched to form the shell of the transmitter, the shell is in a semi-elliptical shape integrally and used for protecting the radar transmitter, and the shell of the transmitter is used for collecting millimeter wave information transmitted and reflected by a detected target;

as shown in fig. 2, a millimeter wave transmitting and receiving antenna, which is located at the front of the housing, for transmitting and receiving a millimeter wave radar signal of a target to be detected; a core processor and a motherboard disposed within the housing.

And the training learning system is arranged in the shell, is connected with the data interfaces of the core processor and the mainboard through data lines, is connected with the control terminal through an RJ45 network transmission line, and gives equipment learning rules.

And the pattern recognition equipment is arranged among the core processor, the mainboard and the training learning system and is used for comparing the imaging data acquired by the radar receiver with the training learning data.

An optical guidance device located in the middle of the two millimeter wave radar signal receivers 130, the optical guidance device being used for providing optical tracking target guidance for users and operators, facilitating quick target selection and tracking;

the high-resolution thermal imaging device 140 is a radar imaging device fixed to the inner surface of the lower outer case 160, and further, the high-resolution thermal imaging device 140 is disposed inside the outer case and is capable of optical imaging, which provides manual determination data.

In another embodiment, the millimeter wave radar hidden dangerous goods detection device further comprises a millimeter wave radar transmitting and receiving device having 1 main receiving and transmitting device, namely, radar millimeter wave signal transmitter 120, 2 auxiliary transmitting and receiving devices, namely, millimeter wave radar signal receiver 130, 1 optical guiding device, and 1 thermal imaging device, respectively disposed at the front of the housing. The radar millimeter wave signal transmitter 120 is used for transmitting a millimeter wave radar signal of a detected target, and the two millimeter wave radar signal receivers 130 are used for receiving the millimeter wave radar signal of the detected target.

Preferably, the millimeter wave radar further includes a humidity measuring device and a temperature measuring device (not shown in the figure), and the humidity measuring device may be on the outer wall of the housing or the inner wall of the housing of the millimeter wave radar. And the temperature and humidity measuring device synchronously measures the temperature and humidity data in the environment where the millimeter wave radar is located in the millimeter wave radar measuring process.

The invention provides an imaging system of a millimeter wave radar, which is shown in figure 3. The millimeter wave radar imaging system comprises a millimeter wave radar 1, a humidity measuring device 2, a temperature measuring device 3, a high-resolution thermal imaging device 4, a training and learning system 5, a processor 6 and a pattern recognition device 7. Wherein the millimeter wave radar 1 measures the millimeter wave reflection signal (R) in real time_mw) The humidity measuring device 2 measures the ambient humidity condition (W) in real time, and the temperature measuring device 3 measures the ambient temperature condition (T) in real time. The millimeter wave radar 1, the humidity measuring device 2 and the temperature measuring device 3 are respectively connected with the training and learning system 5 through a data transmission device, and millimeter wave reflection signals (R)_mw) And the data of the environmental humidity condition (W) and the environmental temperature condition (T) are transmitted to the training learning system in real time. The high-resolution thermal imaging equipment 4 collects imaging data of a detected object and is connected with the training and learning system 5 through a data transmission device.

The training learning system 5 includes a memory including a standard database and a real-time database, and an arithmetic section. The standard database stores a large amount of standard data set in factory, including: and (3) acquiring various millimeter wave signals at standard temperature and humidity, various millimeter wave signal data at various testing gradient temperature and humidity conditions or data acquisition of different testing conditions such as different distances and different surface detection objects at standard humidity/temperature. Data in the real-time database includes temperature, humidity, signal transmission-return time, transmitted signal strength, reflected signal strength, etc. during real-time actual measurement.

The normalized humidity/temperature is typically the normal humidity/temperature for a region, such as 25%, 25 degrees celsius, or a median or average of the humidity/temperatures for the region. The standardized humidity can be set at the time of factory shipment, and can also be modified or set by a program during the use process.

The millimeter wave signal data includes a signal transmission-return time, a transmission signal intensity, a reflected signal intensity, and the like under the condition that a reference object of a known preset distance is detected.

The operation component performs PCA data analysis by extracting the data in the standard library data and the real-time database in the memory, thereby determining the influence factor analysis of the temperature and the humidity. The specific operation process is as follows:

measuring in a real-time measurement a series of (n sets of n measurements taken over n measurement periods) data comprising: real-time temperature, humidity, millimeter wave signal data. And comparing and fitting the real-time measurement data with data in a standard database to obtain influence factors of temperature and humidity on the millimeter wave signal data.

For example, for the time of signal transmission-return (t), there is a drift value (Δ t) due to the influence of temperature and humidity, Δ t ═ at + β t. N groups of data are compared with data of the same temperature and humidity in the standard data respectively to obtain n groups of a and beta values. For example, the following:

a	β
		2.5	2.4
0.5	0.7
		2.2	2.9
1.9	2.2
		3.1	3.0
2.3	2.7
		2	1.6
1	1.1
		1.5	1.6
1.1	0.9

first, average values for a and β, respectively, are found, and then for all samples, the corresponding average values are subtracted. Where the mean of a is 1.81 and the mean of β is 1.91, then one sample minus the mean is (0.69, 0.49), resulting in

a	β
		0.69	0.49
-1.31	-1.21
		0.39	0.99
0.09	0.29
		1.29	1.09
0.49	0.79
		0.19	-0.31
-0.81	-0.81
		-0.31	-0.31
-0.71	-1.01

In a second step, a characteristic covariance matrix is determined, based on the data dimensions (e.g., three dimensions), the covariance matrix is determined

Here, only a and β are solved

On the diagonal are the variances of a and β, respectively, and off-diagonal are the covariances. Covariance is a measure of the degree of change in which two variables change simultaneously. A covariance greater than 0 indicates that a and β increase if one increases and the other increases; less than 0 indicates one increase and one decrease. If a and β are statistically independent, then the covariance between the two is 0; but the covariance is 0 and cannot be said that a and β are independent. The larger the absolute value of the covariance is, the larger the influence of the two on each other is, and vice versa. Covariance is a unitless quantity, and thus if the dimensions used for the same two variables change, their covariance also produces changes in the branches.

Thirdly, solving the eigenvalue and the eigenvector of the covariance to obtain

The upper is the two eigenvalues and the lower is the corresponding a and β eigenvalue impact factors. And the computing system calculates a corresponding drift value according to the characteristic influence factor and corrects data. On the other hand, the training learning system gives the equipment high-resolution thermal imaging equipment measurement results to learn rules at the same time, and the equipment continuously has the capability of being more suitable for hidden dangerous goods detection.

The training learning system 5 is connected to the processor 6, and the processor 6 fuses the modified millimeter wave radar and the high-resolution thermal imaging device training data.

The processor 6 is connected to the pattern recognition device, and the pattern recognition of the millimeter wave radar and the high-resolution thermal imaging equipment is realized through the pattern recognition device.

The imaging method according to the above system is described in detail below, and comprises the following steps:

s1, pre-storing standard data into a standard database, wherein the standard data comprises: various millimeter wave signals under standard temperature and humidity, and various millimeter wave signal data under various testing gradient temperature and humidity conditions.

Specifically, the data measurement at the standard humidity/temperature may be data acquisition performed when the millimeter wave radar is in an original outgoing state, including data acquisition under different test conditions for different distances, different surface probes, and the like at the standard humidity/temperature.

And establishing a standard humidity database in a training system through the data acquisition for artificial intelligence training of imaging control and adjustment of the millimeter wave radar.

And S2, collecting actual measurement data in real time when the working humidity reaches the preset standard humidity during the operation of the millimeter wave radar. Measuring in a real-time measurement a series of (n sets of n measurements taken over n measurement periods) data comprising: real-time temperature, humidity, millimeter wave signal data. And establishing a functional relation between a standard humidity database and data, comparing and fitting the real-time measurement data with the data in the standard database, and training by a PCA (principal component analysis) algorithm of a training learning system to obtain influence factors of temperature and humidity on millimeter wave signal data. For the specific analysis, reference is made to the above description, which is not repeated herein.

Specifically, the signal power transmitted by the millimeter wave signal transmitter 120 and the signal power received by the millimeter wave radar signal receiver 130 are measured under the working condition, and the distance of the detected object, the product surface or the product type are collected simultaneously by the high-resolution thermal imaging device 140; and simultaneously, measuring the real-time working humidity condition through a humidity measuring device.

And S3, solving the corresponding drift value according to the characteristic influence factor, and correcting the data to obtain the corrected millimeter wave radar measurement data. And according to the humidity/temperature influence factor provided by the training system, the result of the received signal is compensated through a feedback control system in the processor, so that a more accurate measurement result is obtained.

S4 learning rules by training the learning system and giving the equipment high-resolution thermal imaging equipment measurement results, so that the equipment has the capability of adapting to hidden dangerous goods detection. And fusing the corrected millimeter wave radar and the training data of the high-resolution thermal imaging equipment through a processor.

And S5, pattern recognition of millimeter wave radar and high-resolution thermal imaging equipment imaging is realized through the pattern recognition device.

The specific identification of the product type and the surface can be performed by a pattern identification device, except for the connection mode provided in this embodiment, the pattern identification device 7 can be further disposed between the core processor and the training learning system, and compares the imaging data acquired by the radar receiver with the training learning system to determine the product type and the surface state of the measured object. The product type and the surface can be identified through pattern identification equipment, the pattern identification equipment is arranged among the core processor, the main board and the neural network learning system, imaging data acquired by the radar receiver is compared with the neural network learning data, and the product type and the surface state of the measured object are judged.

The millimeter wave radar imaging method and the millimeter wave radar imaging system improve the real-time property and the robustness, and have low false alarm rate and small influence on the environment.

The foregoing shows and describes the general principles, essential features and advantages of the invention, which is, therefore, described only as an example of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the invention, but rather that the invention includes various equivalent changes and modifications without departing from the spirit and scope of the invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims

1. A millimeter wave radar imaging method is characterized in that: the method comprises the following steps:

s1 pre-storing standard data into a standard database, the standard data including: various millimeter wave signals under standard temperature and humidity, various millimeter wave signal data under various testing gradient temperature and humidity conditions;

s2, when the working humidity reaches the preset standard humidity during the working period of the millimeter wave radar, real-time collection of actual measurement data is carried out;

s3, analyzing influence factors of the humidity parameters on millimeter wave radar imaging through a training learning system, wherein the specific influence factor operation process is as follows: acquiring n groups of data in n measurement periods of real-time measurement, wherein the data comprises: real-time signal emission-return time data t, wherein the real-time signal emission-return time data and the signal emission-return time data under standard humidity and temperature have a drift value delta t, and the delta t is fit to be alpha t + beta t; respectively comparing n groups of data with data of the same temperature and humidity in the standard data to obtain n groups of alpha and beta values;

firstly, respectively calculating the average values of alpha and beta, and then subtracting the corresponding average values from all the alpha and beta values;

second, solving a characteristic covariance matrix, and determining the covariance matrix according to the three-dimensional dimension of the data

And solving cov the vector matrix; wherein, the variance of alpha and beta is respectively on the diagonal line in the covariance matrix, and the covariance is on the off-diagonal line; the covariance is the degree of change of two variables changing at the same time, namely the covariance is greater than 0 to indicate that if one of alpha and beta is increased, the other is also increased; less than 0 indicates one increase, one decrease; if α and β are statistically independent, then the covariance between the two is 0; the larger the absolute value of the covariance is, the larger the influence of the two on each other is, and the smaller the influence is otherwise;

thirdly, solving an eigenvalue and an eigenvector value of the covariance matrix, wherein the eigenvector value comprises an upper eigenvalue and a lower eigenvalue, and the lower eigenvalue is a corresponding alpha and beta influence factor; the computing system calculates a corresponding drift value according to the characteristic influence factor and corrects millimeter wave data;

s4, training a learning system and giving a measurement result of a high-resolution thermal imaging device of the imaging system based on the millimeter wave radar to learn rules, so that the imaging system has the capability of being more suitable for detecting hidden dangerous goods continuously;

and S5, fusing the correction data of the millimeter wave radar and the data of the high-resolution thermal imaging equipment through the processor, and realizing the mode recognition of the millimeter wave radar and the high-resolution thermal imaging equipment through the mode recognition device.

2. The millimeter wave radar imaging method of claim 1, wherein: the standard humidity/temperature is the normal humidity/temperature for a certain area.

3. The millimeter wave radar imaging method according to claim 2, characterized in that: the standard humidity is 25-40%, and the standard temperature is 20-30 ℃.

4. The millimeter wave radar imaging method of claim 1, wherein: the actual measurement data includes: humidity, temperature, millimeter wave radar signal data.

5. The millimeter wave radar imaging method of claim 4, wherein: the millimeter wave radar signal data includes signal transmission-return time, transmission signal intensity, transmission signal frequency, reflection signal intensity, and reflection signal frequency.

6. An imaging system based on millimeter wave radar, comprising: millimeter wave radar (1), humidity measuring means (2), temperature measuring means (3), a high resolution thermography arrangement (4), a training learning system (5), a processor (6), pattern recognition means (7), the millimeter wave radar based imaging system being adapted to perform the millimeter wave radar imaging method according to claim 1.

7. The millimeter wave radar-based imaging system of claim 6, wherein: the humidity and temperature measuring device may be mounted on the outer wall of the housing or the inner wall of the housing of the millimeter wave radar.

8. The millimeter wave radar-based imaging system of claim 6, wherein: the millimeter wave radar includes a signal transmitter and a signal receiver.