CN115792897A - FMCW radar-based low-complexity multi-gesture recognition method and system - Google Patents
FMCW radar-based low-complexity multi-gesture recognition method and system Download PDFInfo
- Publication number
- CN115792897A CN115792897A CN202211631368.9A CN202211631368A CN115792897A CN 115792897 A CN115792897 A CN 115792897A CN 202211631368 A CN202211631368 A CN 202211631368A CN 115792897 A CN115792897 A CN 115792897A
- Authority
- CN
- China
- Prior art keywords
- signal
- gesture recognition
- fmcw
- radar
- doppler
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses a low-complexity multi-gesture recognition method and a system based on a millimeter wave radar. The invention also provides a multi-gesture recognition system which comprises a power module, an FMCW millimeter wave radar system and a multi-gesture recognition unit. The FMCW millimeter wave radar system comprises a transmitter, a receiver, a radio frequency module, a microcontroller, an analog-to-digital converter and digital signal processing. The millimeter wave radar adopted by the invention has the advantages of high resolution, good reliability, small antenna size and the like, and provides a gesture recognition solution with low complexity and high recognition accuracy.
Description
Technical Field
The invention belongs to the field of digital signal processing, and particularly relates to a low-complexity multi-gesture recognition method and system based on an FMCW radar.
Background
Natural user interfaces have been a field of active research in academia and industry for the past few years. A variety of sensing technologies have been used to seamlessly interact with computing devices. Gesture recognition has become an important component of human-computer interaction, and research and development thereof influence the naturalness and flexibility of human-computer interaction.
When the gesture detection and recognition is performed by a user, the electronic equipment analyzes and processes the data after acquiring gesture action data to finish the classification and recognition of the gesture action. The data sources that the electronic device can collect mainly include RGB images, infrared images, depth images, gyroscope and accelerator sensor data, and electromagnetic wave radio signal data. Research in gesture detection and recognition involves multiple disciplines and areas of computer vision, image processing, pattern recognition, machine learning, communications, and sensor technology. In addition, the gesture detection and recognition technology has wide application, and the application of the technology to the intelligent device can bring great commercial value. Therefore, gesture recognition technology is a research hotspot in the industry and academia.
The radar is a radio technology for detecting a target by transmitting and receiving electromagnetic waves, the application range of the radar is wide, and with the development of signal processing technology and the emergence of intelligent computation, the combination of radar signal processing and artificial intelligence technology makes a new technological development opportunity. Therefore, the radar-based smart recognition technology is gradually becoming a research focus and can be widely applied in various scenes. Such as identification of friend or foe targets, weak target detection, etc. in the military field, and automatic vehicle cruising, automobile radar, industrial liquid level detection, heartbeat and respiration detection, gesture identification, etc. in the civil field. Frequency Modulated Continuous Wave (FMCW) radar is a radar with a special modulation technique, and millimeter wave radar has an operating frequency of 30-300 GHz and a large operating bandwidth. The FMCW millimeter wave radar has remarkable advantages in the field of gesture recognition, and can accurately detect change information such as distance, speed and angle of gestures.
Frequency Modulated Continuous Wave (FMCW) radar can measure range (i.e., radial distance from the radar), velocity (relative to the velocity of the radar), and direction of arrival of a target in front of the radar. Range resolution, velocity resolution, and angular resolution are key parameters that affect radar performance. The range resolution of the radar depends on the radio frequency bandwidth of the transmitted signal. Thus, the speed resolution is largely independent of hardware constraints and can be improved by extending the frame time. The angular resolution of the radar depends on the number of antennas. Since each receive antenna requires its own receive signal chain, increasing the angular resolution means a significant increase in the area and cost of the radar. Therefore, the limited angular resolution is one of the largest limitations in the FMCW radar solutions. Therefore, radar gesture recognition algorithms should take advantage of speed and range resolution while minimizing the dependency on angular resolution.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the problem that the gesture description information amount is low in the traditional gesture recognition method is solved, and the classification accuracy of a plurality of gestures is low.
In order to solve the technical problems, the invention provides the following technical scheme: a low-complexity multi-gesture recognition method based on FMCW radar comprises the following steps:
s1, collecting gesture actions of a hand, and obtaining FMCW radar signal data corresponding to each gesture action;
s2, calculating a transmitting signal and a receiving signal of the FMCW radar according to the FMCW radar signal data, and carrying out analog-to-digital conversion on the receiving signal;
s3, performing 2D-FFT processing on the received signals after analog-to-digital conversion, and performing incoherent addition to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
s4, sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signal to obtain a second characteristic vector set related to the azimuth angle and the elevation angle of the transmitted signal and the received signal, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
and S5, dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the corresponding gesture motion as output.
Further, the foregoing step S2 includes calculating the transmission signal of the FMCW radar signal data in a sawtooth modulation manner according to the following formula:
wherein f is c Representing the center frequency of the carrier wave, f T (τ) represents a duration of a sawtooth signal of length T within one cycle, A T Is the amplitude of the transmitted signal.
Further, the step S2 includes calculating a received signal of the FMCW radar signal data according to the following formula by using a sawtooth modulation method:
wherein, A R Representing the amplitude of the echo signal, f c Representing the carrier frequency, Δ t d Representing the time-of-flight from the emission of the transmitted signal to the reception of the echo signal, f R (τ) is the echo signal frequency.
Further, in the foregoing step S3, the first set of feature vectors includes:
weighted distance:
for detecting the position of the hand;
weighted doppler:
for detecting a velocity centroid of the hand;
instantaneous energy: i = ∑ Σ i Z i For detecting the presence of a hand;
weighted distance dispersion:
the index is used for calculating the change range of the weighted distance value;
weighted doppler dispersion:
the index is used for calculating the variation range of the weighted Doppler;
wherein i is the index of RDI data, Z i Is the ith amplitude value, R i Is the ith distance value, D i The corresponding Doppler value is the ith index.
Further, the obtaining of the second feature vector set in the foregoing step S4 includes the following sub-steps:
s401, carrying out three-dimensional FFT processing on the received signal, and correspondingly obtaining peak indexes of the transmitted signal and the received signal;
s402, obtaining an arrival angle of the received signal according to the peak index of the received signal;
s403, performing two-dimensional FFT processing on the arrival angle to obtain the azimuth angle and the elevation angle of the received signal.
Further, the aforementioned step S4 of fusing the first set of feature vectors and the second set of feature vectors includes the following sub-steps:
s411, creating a sliding time window of preset n frames at the time t to obtain weighted Doppler and an azimuth vector, and then calculating the correlation between the weighted Doppler and the azimuth vector to obtain an azimuth Doppler and azimuth correlation characteristic;
s412, fusing the weighted distance, the weighted Doppler and the instantaneous energy characteristics in the first characteristic vector set with the azimuth angle, the elevation angle and the Doppler azimuth angle correlation characteristics in the second characteristic set to obtain a fused characteristic set.
Further, in the foregoing step S1, the gesture action includes: and the palm slides left, right, up, down, clockwise and anticlockwise to rotate the fingers.
The invention provides a low-complexity multi-gesture recognition system based on FMCW radar, comprising: the system comprises a power supply module, an FMCW millimeter wave radar system and a multi-gesture recognition module;
the power supply module is used for supplying power to the FMCW millimeter wave radar system;
the FMCW millimeter wave radar system is used for collecting gesture actions of a hand and obtaining FMCW radar signal data corresponding to each gesture action;
and the multi-gesture recognition module is used for generating a fusion feature set according to FMCW radar signal data and constructing and training a gesture recognition model.
Further, the FMCW millimeter wave radar system described above includes: the system comprises a microcontroller, a transmitter, a receiver, a radio frequency module, an analog-to-digital converter and a digital signal processor; wherein
The microcontroller is used for controlling the FMCW millimeter wave radar system;
the radio frequency module is used for generating frequency modulation continuous wave signals, frequency is multiplied to millimeter wave signal frequency points through a frequency multiplier, and FMCW millimeter wave radar transmitting signals are generated;
the transmitter is used for transmitting FMCW millimeter wave radar transmission signals,
the receiver is used for receiving a receiving signal returned by the transmitter for transmitting an FMCW millimeter wave radar transmitting signal,
the analog-to-digital converter is used for performing analog-to-digital conversion on the received signal;
and the digital signal processor is used for processing the received signals after analog-to-digital conversion.
The multi-gesture recognition unit includes: the device comprises a first feature vector set construction unit, a fusion feature set construction unit and a gesture recognition model construction unit;
the first feature vector set construction unit is configured to perform the following actions:
calculating the transmitting signal and the receiving signal; then, carrying out 2D-FFT processing on the received signals, and then carrying out non-coherent addition to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
the fusion feature set construction unit is configured to perform the following actions:
sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signals to obtain a second characteristic vector set related to the azimuth angle and the elevation angle of the received signals, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
the gesture recognition model building unit is configured to execute the following actions: and dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the gesture action corresponding to the feature as output.
Compared with the prior art, the technical scheme adopted by the invention has the following beneficial effects:
(1) Compared with the traditional gesture recognition method, the method combines the deep learning technology, trains data by utilizing the adaptivity of deep learning and relying on an artificial neural network, and effectively improves the recognition accuracy.
(2) The feature set adopted by the invention consists of six items, a corresponding parameter extraction method is provided from three aspects of distance, speed, angle and the like, and the diversity of data is ensured by adopting multi-dimensional data.
(3) The low complexity of the gesture recognition method is embodied in that when the feature parameters are extracted, a single number in a current heat map is extracted, the calculation and storage complexity of a classification stage is optimized, new feature parameters are introduced based on a time correlation concept, and the data volume is further optimized while the data reliability is ensured.
Drawings
FIG. 1 is an overall flow chart of the present invention.
Fig. 2 is a schematic diagram illustrating six gestures predefined to be recognized.
Figure 3 is a range-doppler plot of the signal 2D-FFT.
Fig. 4 is a diagram of a neural network structure employed in the present invention.
Detailed Description
In order to better understand the technical content of the present invention, specific embodiments are described below with reference to the accompanying drawings.
Aspects of the invention are described herein with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the invention are not limited to those illustrated in the drawings. It is to be understood that the invention is capable of implementation in any of the numerous concepts and embodiments described hereinabove or described in the following detailed description, since the disclosed concepts and embodiments are not limited to any embodiment. In addition, some aspects of the present disclosure may be used alone, or in any suitable combination with other aspects of the present disclosure.
The invention relates to a low-complexity multi-gesture recognition system based on an FMCW radar, which comprises: the system comprises a power supply module, an FMCW millimeter wave radar system and a PC terminal; and the power supply module is used for supplying power to the FMCW millimeter wave radar system. And the PC terminal is used for generating a fusion characteristic set according to FMCW radar signal data and constructing and training a gesture recognition model.
The radar signal modulation mode adopts a sawtooth wave modulation mode. When a transmitting signal of a radar system is reflected by an object and is received by a radar after time delay, and when the echo signal is mixed with the transmitting signal, an intermediate frequency signal with constant frequency, namely a receiving signal, is generated in an overlapping time period.
The FMCW millimeter wave radar system is used for collecting gesture actions of a hand and obtaining FMCW radar signal data corresponding to each gesture action, and comprises a microcontroller, a transmitter, a receiver, a radio frequency module, an analog-to-digital converter and a digital signal processor. The microcontroller is used for controlling the FMCW millimeter wave radar system; six gestures are collected as shown in fig. 2.
The radar is connected with the power supply module and the PC end by using the USB data line, a tested person needs to sit at a position about 0.4m in front of the radar, and the palm is arranged in front of the radar, so that gestures can be captured by the radar.
The radio frequency module is used for generating FMCW millimeter wave radar transmission signals; the radio frequency module is responsible for generating frequency modulation continuous wave signals and frequency doubling to millimeter wave signal frequency points through a frequency multiplier; the transmitter amplifies the signals by using a power amplifier and transmits the signals through an antenna; the antenna array comprises two transmitting antennas, namely Tx1 and Tx2, four receiving antennas, namely Rx1, rx2, rx3 and Rx4, and works in a time division/code division multiplexing mode, and can be equivalently an antenna array consisting of eight antennas.
At millimeter wave radar system's that this patent adopted signal transmission end, the signal is at first multiplied by the frequency multiplier to preset frequency point, then divide into two the tunnel with the signal, wherein one of them is enlargied the signal through power amplifier via the transmitter, accomplish the transmission of signal through the antenna, another tunnel then sends the mixer and waits to meet the object and can take place the reflection with the received signal mixing and obtain intermediate frequency signal transmitted signal in the transmission in-process, the echo signal is caught to the receiver to obtain the received signal, it is intermediate frequency signal with the transmitted signal mixing behind the low noise amplifier.
The analog-to-digital converter is used for performing analog-to-digital conversion on the received signal;
and the digital signal processor is used for processing the received signals after analog-to-digital conversion and waiting for the PC end to process. The FMCW millimeter wave radar system includes:
the PC end comprises: the device comprises a first feature vector set construction unit, a fusion feature set construction unit and a gesture recognition model construction unit;
the first feature vector set construction unit is configured to perform the following actions:
calculating the transmitting signal and the receiving signal; then, carrying out 2D-FFT processing on the received signals, and then carrying out non-coherent addition to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
the fusion feature set construction unit is configured to perform the following actions:
sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signals, further obtaining a second characteristic vector set of the azimuth angle and the elevation angle of the received signals, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
the gesture recognition model building unit is used for configuring and executing the following actions: and dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the corresponding gesture as output.
Based on the above-mentioned FMCW radar-based low-complexity multi-gesture recognition system, as shown in fig. 1, the flowchart of the present invention, a FMCW radar-based low-complexity multi-gesture recognition method, includes steps S1 to S5,
s1, collecting gesture actions of a hand, wherein the gesture actions comprise six gesture actions of left sliding, right sliding, upward sliding, downward sliding, clockwise finger rotating and anticlockwise finger rotating, and are shown in figure 2; obtaining FMCW radar signal data corresponding to each gesture;
s2, calculating a transmitting signal and a receiving signal of the FMCW radar according to the FMCW radar signal data, and carrying out analog-to-digital conversion on the receiving signal; and then ADC sampling processing is carried out, the number of ADC sampling points is 256 per sweep frequency, the number of radar sweep frequency signals is 128 per frame, and the signals are stored in an ADC cache.
Step S2 comprises the steps of adopting a sawtooth wave modulation mode for FMCW radar signal data, and calculating a transmitting signal according to the following formula:
wherein f is c Representing the center frequency of the carrier wave, f T (τ) represents a duration of a sawtooth signal of length T within one cycle, A T Is the amplitude of the transmitted signal.
Step S2 comprises the steps of adopting a sawtooth wave modulation mode for FMCW radar signal data, and calculating a receiving signal according to the following formula:
wherein A is R Representing the amplitude of the echo signal, f c Representing the carrier frequency, Δ t d Representing the time-of-flight from the emission of the transmitted signal to the reception of the echo signal, f R (τ) is the echo signal frequency.
And S3, the transmitting and receiving antenna comprises two transmitting antennas and four receiving antennas, wherein the two transmitting antennas operate in a time division/code division multiplexing mode and are equivalent to an antenna array consisting of eight antennas. Non-coherently adding the 2D-FFT outputs of multiple antennas to create a range-doppler thermal map is shown in figure 3. Extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters; to reduce the computational and memory complexity of the further classification stage, the extracted features are weighted averages in the current heatmap that reflect radar specific parameters, such as average doppler, average distance, doppler shift, etc. The amplitude of the 2D-FFT for each antenna is calculated and non-coherently added to create a Range Doppler Image (RDI) to extract amplitude-based features, which are weighted averages in the current heat map that reflect radar specific parameters, such as average Doppler, average distance, doppler shift, etc., in order to reduce the computational and memory complexity of the further classification stages.
The following were used: i is the index of the RDI data, Z i Is the ith amplitude value, R i Is the ith distance value, D i The corresponding Doppler value is the ith index.
Weighted distance:
this feature can be used to detect the position of the hand (distance centroid).
Weighted Doppler:
this feature can be used to detect the centroid of velocity of the hand.
Instantaneous energy: i = ∑ Σ i Z i This feature can be used to detect the presence of a hand.
Weighted distance dispersion:
the index may calculate a range of variation in the weighted distance value.
Weighted doppler dispersion:
the indicator can calculate the range of variation of the weighted doppler.
S4, respectively carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signals in sequence, further obtaining a second characteristic vector set related to azimuth angles and elevation angles of the transmitted signals and the received signals, fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set,
s401, carrying out three-dimensional FFT processing on the received signal, and representing the peak value index of the received signal by using 'w';
s402, obtaining an arrival angle of the received signal according to the peak index of the received signal;
s403, two-dimensional FFT processing is carried out on the arrival angle, namely the azimuth angle (theta) and the elevation angle (phi) can be obtained from the phase values corresponding to the distance and Doppler coordinates in the 2D-FFT on all the antennas.
Then, considering that a single frame of the radar will generate a single value for each feature, the frame sequence will generate a time series for each feature, and therefore the feature needs to be extracted over a sliding time window. In order to obtain the change of the doppler azimuth angle along with the time, a sliding time window of the last n frames needs to be created at the time t to obtain the weighted doppler and azimuth angle vectors, and then the correlation is calculated to obtain the doppler azimuth angle correlation characteristic.
And fusing the weighted distance, weighted Doppler and instantaneous energy characteristics in the first characteristic vector set with the azimuth angle, elevation angle and Doppler azimuth angle correlation characteristics in the second characteristic set to obtain a fused characteristic set.
And S5, dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the corresponding gesture motion as output. The neural network diagram is shown in fig. 4.
A supervised learning training method is adopted, 168 ten thousand data points (feature vectors) in total are trained on six groups of gestures according to a division mode that 65% of the data points are used for training given parameters, 15% of the data points are used for verification to select optimal parameters, and 20% of the data points are tested, and a gradient descent algorithm is adopted for optimization, and the results are shown in table 1:
TABLE 1
The invention provides a low-complexity multi-gesture recognition system based on FMCW radar, comprising: the system comprises a power supply module, an FMCW millimeter wave radar system and a multi-gesture recognition module;
the power supply module is used for supplying power to the FMCW millimeter wave radar system;
the FMCW millimeter wave radar system is used for collecting hand gesture actions and obtaining FMCW radar signal data corresponding to each hand gesture action;
the multi-gesture recognition module is used for generating a fusion feature set according to FMCW radar signal data, and constructing and training a gesture recognition model;
the FMCW millimeter wave radar system includes: the system comprises a microcontroller, a transmitter, a receiver, a radio frequency module, an analog-to-digital converter and a digital signal processor; wherein
The microcontroller is used for controlling the FMCW millimeter wave radar system;
the radio frequency module is used for generating frequency modulation continuous wave signals, frequency is multiplied to millimeter wave signal frequency points through a frequency multiplier, and FMCW millimeter wave radar transmitting signals are generated;
the transmitter is used for transmitting FMCW millimeter wave radar transmission signals,
the receiver is used for receiving a receiving signal returned by the transmitter for transmitting an FMCW millimeter wave radar transmitting signal,
the analog-to-digital converter is used for performing analog-to-digital conversion on the received signal;
and the digital signal processor is used for processing the received signals after analog-to-digital conversion.
The multi-gesture recognition unit includes: the device comprises a first feature vector set construction unit, a fusion feature set construction unit and a gesture recognition model construction unit;
the first feature vector set construction unit is configured to perform the following actions:
calculating the transmitting signal and the receiving signal; then, carrying out 2D-FFT processing on the received signals, and then carrying out non-coherent addition to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
the fusion feature set construction unit is configured to perform the following actions:
sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signals to obtain a second characteristic vector set related to the azimuth angle and the elevation angle of the received signals, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
the gesture recognition model building unit is configured to execute the following actions: and dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the corresponding gesture as output.
As can be seen from the table, after millimeter wave radar signals are processed and neural network training is carried out, the overall solution of gesture recognition has higher accuracy.
Although the invention has been described with reference to preferred embodiments, it is not intended to be limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention should be determined by the appended claims.
Claims (9)
1. A low-complexity multi-gesture recognition method based on FMCW radar is characterized by comprising the following steps:
s1, collecting gesture actions of a hand, and obtaining FMCW radar signal data corresponding to each gesture action;
s2, calculating a transmitting signal and a receiving signal of the FMCW radar according to the FMCW radar signal data, and carrying out analog-to-digital conversion on the receiving signal;
s3, performing 2D-FFT processing on the received signals after analog-to-digital conversion, and performing incoherent addition to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
s4, sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signal to obtain a second characteristic vector set related to the azimuth angle and the elevation angle of the transmitted signal and the received signal, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
and S5, dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the corresponding gesture motion as output.
2. The FMCW radar-based low complexity multi-gesture recognition method as claimed in claim 1, wherein step S2 includes applying sawtooth modulation to FMCW radar signal data to calculate its transmitted signal according to the following formula:
wherein f is c Representing the center frequency of the carrier wave, f T (τ) represents a duration of a sawtooth signal of length T within one cycle, A T Is the amplitude of the transmitted signal.
3. The FMCW radar-based low complexity multi-gesture recognition method as claimed in claim 1, wherein step S2 includes applying sawtooth modulation to FMCW radar signal data, and calculating its received signal according to the following formula:
wherein A is R Representing the amplitude of the echo signal, f c Representing the carrier frequency, Δ t d Representing the time delay of flight from the emission of the transmitted signal to the reception of the echo signal, f R (τ) is the echo signal frequency.
4. The FMCW radar-based low complexity multi-gesture recognition method of claim 3, wherein in step S3, the first set of eigenvectors includes:
weighted distance:
for detecting the position of the hand;
weighted doppler:
for detecting a velocity centroid of the hand;
instantaneous energy: i = ∑ Σ i Z i For detecting the presence of a hand;
weighted distance dispersion:
the index is used for calculating the change range of the weighted distance value;
weighted doppler dispersion:
the index is used for calculating the variation range of the weighted Doppler; where i is the index of the RDI data, Z i Is the ith amplitude value, R i Is the ith distanceFrom value, D i The corresponding Doppler value is the ith index.
5. The FMCW radar-based low complexity multi-gesture recognition method of claim 4, wherein the obtaining of the second set of feature vectors in step S4 includes the following sub-steps:
s401, carrying out three-dimensional FFT processing on the received signal, and correspondingly obtaining peak indexes of the transmitted signal and the received signal;
s402, obtaining an arrival angle of the received signal according to the peak index of the received signal;
s403, performing two-dimensional FFT processing on the arrival angle to obtain the azimuth angle and the elevation angle of the received signal.
6. A FMCW radar-based low complexity multi-gesture recognition method according to claim 4, wherein step S4 is to fuse the first set of feature vectors with the second set of feature vectors, comprising the sub-steps of:
s411, creating a sliding time window of preset n frames at the time t to obtain weighted Doppler and an azimuth vector, and then calculating the correlation between the weighted Doppler and the azimuth vector to obtain an azimuth Doppler and azimuth correlation characteristic;
s412, fusing the weighted distance, the weighted Doppler and the instantaneous energy characteristics in the first characteristic vector set with the azimuth angle, the elevation angle and the Doppler azimuth angle correlation characteristics in the second characteristic set to obtain a fused characteristic set.
7. The FMCW radar-based low complexity multi-gesture recognition method of claim 1, wherein in step S1, the gesture actions include: and the palm slides left, right, up, down, clockwise and anticlockwise to rotate the fingers.
8. A FMCW radar-based low complexity multi-gesture recognition system comprising: the system comprises a power supply module, an FMCW millimeter wave radar system and a multi-gesture recognition module;
the power supply module is used for supplying power to the FMCW millimeter wave radar system;
the FMCW millimeter wave radar system is used for collecting hand gesture actions and obtaining FMCW radar signal data corresponding to each hand gesture action;
and the multi-gesture recognition module is used for generating a fusion feature set according to FMCW radar signal data and constructing and training a gesture recognition model.
9. A FMCW radar-based low complexity multi-gesture recognition system of claim 8, wherein the FMCW millimeter wave radar system comprises: the system comprises a microcontroller, a transmitter, a receiver, a radio frequency module, an analog-to-digital converter and a digital signal processor; wherein
The microcontroller is used for controlling the FMCW millimeter wave radar system;
the radio frequency module is used for generating frequency modulation continuous wave signals, frequency is multiplied to millimeter wave signal frequency points through a frequency multiplier, and FMCW millimeter wave radar transmitting signals are generated;
the transmitter is used for transmitting FMCW millimeter wave radar transmission signals,
the receiver is used for receiving a receiving signal returned by the transmitter for transmitting an FMCW millimeter wave radar transmitting signal,
the analog-to-digital converter is used for performing analog-to-digital conversion on the received signal;
the digital signal processor is used for processing the received signals after analog-to-digital conversion;
the multi-gesture recognition unit includes: the device comprises a first feature vector set construction unit, a fusion feature set construction unit and a gesture recognition model construction unit;
the first feature vector set construction unit is configured to perform the following actions:
calculating the transmitting signal and the receiving signal; then 2D-FFT processing is carried out on the received signal, and then incoherent addition is carried out to obtain a distance and Doppler analysis heat map; extracting features related to signal amplitude from the distance and Doppler analysis heat map to obtain a first feature vector set of radar preset parameters;
the fusion feature set construction unit is configured to perform the following actions:
sequentially carrying out three-dimensional FFT processing and two-dimensional FFT processing on the received signals to obtain a second characteristic vector set related to the azimuth angle and the elevation angle of the received signals, and fusing the first characteristic vector set and the second characteristic vector set to form a fused characteristic set;
the gesture recognition model building unit is configured to execute the following actions:
and dividing the fusion feature set into a training set, a verification set and a test set according to a preset proportion, and constructing and training a gesture recognition model by taking each feature in the training set as input and the gesture action corresponding to the feature as output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211631368.9A CN115792897A (en) | 2022-12-19 | 2022-12-19 | FMCW radar-based low-complexity multi-gesture recognition method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211631368.9A CN115792897A (en) | 2022-12-19 | 2022-12-19 | FMCW radar-based low-complexity multi-gesture recognition method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115792897A true CN115792897A (en) | 2023-03-14 |
Family
ID=85425647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211631368.9A Pending CN115792897A (en) | 2022-12-19 | 2022-12-19 | FMCW radar-based low-complexity multi-gesture recognition method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115792897A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116027324A (en) * | 2023-03-24 | 2023-04-28 | 德心智能科技(常州)有限公司 | Fall detection method and device based on millimeter wave radar and millimeter wave radar equipment |
| CN117357103A (en) * | 2023-12-07 | 2024-01-09 | 山东财经大学 | CV-based limb movement training guiding method and system |
-
2022
- 2022-12-19 CN CN202211631368.9A patent/CN115792897A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116027324A (en) * | 2023-03-24 | 2023-04-28 | 德心智能科技(常州)有限公司 | Fall detection method and device based on millimeter wave radar and millimeter wave radar equipment |
| CN117357103A (en) * | 2023-12-07 | 2024-01-09 | 山东财经大学 | CV-based limb movement training guiding method and system |
| CN117357103B (en) * | 2023-12-07 | 2024-03-19 | 山东财经大学 | CV-based limb movement training guiding method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sang et al. | Micro hand gesture recognition system using ultrasonic active sensing | |
| CN115792897A (en) | FMCW radar-based low-complexity multi-gesture recognition method and system | |
| Skaria et al. | Deep-learning methods for hand-gesture recognition using ultra-wideband radar | |
| Wang et al. | A gesture air-writing tracking method that uses 24 GHz SIMO radar SoC | |
| CN113837131B (en) | Multi-scale feature fusion gesture recognition method based on FMCW millimeter wave radar | |
| Zhao et al. | Cubelearn: End-to-end learning for human motion recognition from raw mmwave radar signals | |
| Goswami et al. | Real-time multi-gesture recognition using 77 GHz FMCW MIMO single chip radar | |
| Yu et al. | mmWave radar-based hand gesture recognition using range-angle image | |
| US20230333209A1 (en) | Gesture recognition method and apparatus | |
| EP4001960A1 (en) | Radar-based tracker for target sensing | |
| Kim et al. | Radar-based human activity recognition combining range–time–Doppler maps and range-distributed-convolutional neural networks | |
| CN110988863A (en) | Novel millimeter wave radar gesture signal processing method | |
| CN113064483A (en) | Gesture recognition method and related device | |
| Lim et al. | DNN-based human face classification using 61 GHz FMCW radar sensor | |
| CN115343704A (en) | Gesture recognition method of FMCW millimeter wave radar based on multi-task learning | |
| Zhang et al. | Temporal-range-doppler features interpretation and recognition of hand gestures using MmW FMCW radar sensors | |
| Pho et al. | Radar-based face recognition: One-shot learning approach | |
| Gan et al. | Gesture recognition system using 24 GHz FMCW radar sensor realized on real-time edge computing platform | |
| Jin et al. | Interference-robust millimeter-wave radar-based dynamic hand gesture recognition using 2D CNN-transformer networks | |
| Qu et al. | Dynamic hand gesture classification based on multichannel radar using multistream fusion 1-D convolutional neural network | |
| Wang et al. | A survey of hand gesture recognition based on FMCW radar | |
| CN114708663A (en) | Millimeter wave radar sensing gesture recognition method based on few-sample learning | |
| Li et al. | Digital gesture recognition based on millimeter wave radar | |
| Song et al. | Dual-task human activity sensing for pose reconstruction and action recognition using 4d imaging radar | |
| Song et al. | High-accuracy gesture recognition using mm-wave radar based on convolutional block attention module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |