CN113203754A

CN113203754A - Material inspection method based on wireless commercial equipment

Info

Publication number: CN113203754A
Application number: CN202110520496.5A
Authority: CN
Inventors: 谷雨; 朱亚男; 刘军
Original assignee: Hefei University of Technology
Current assignee: Hefei University of Technology
Priority date: 2021-05-13
Filing date: 2021-05-13
Publication date: 2021-08-03

Abstract

The invention provides a material inspection technical method based on wireless commercial equipment, which constructs a unique dynamic fingerprint for static solid liquid materials and the like, converts the fingerprint into a picture in a visualized manner, and then performs classification and identification by using a neural network. Not only can the material be tested rapidly, but also the material can be tested in a laboratory. The wireless signal has the advantages of large coverage range, low requirement on illumination, non-invasive property, capability of penetrating through a non-metal wall and the like. Therefore, the invention has good application prospect and wide applicability.

Description

Material inspection method based on wireless commercial equipment

Technical Field

The invention relates to the field of intelligent material inspection, in particular to a material inspection technology based on wireless commercial equipment.

Background

In the prior art, non-destructive material classification methods, such as near infrared spectroscopy, are often used for drug analysis. Similarly, millimeter wave and terahertz technologies are also used for detecting substances other than long distances, for purposes of scientific exploration (e.g., finding planets) or security detection. However, whichever method is used, these methods are complex and expensive due to the high-precision tip sensors involved, not to mention the volume and power requirements.

As with identification, material classification may also employ non-destructive and less destructive vision-based methods, although this may be challenging. In a controlled environment with sufficient light or natural lighting, the problem is more easily solved if the distances are close. Similar image-based surface classification techniques also exist, such as classification using a laser optical mouse sensor, and the like. The above methods are not able to report the status of different objects (e.g. filled/unfilled cups) nor can they be used on different body parts. Vision-based material classification methods are also limited by the material quality of the object surface. This can lead to confusion because an opaque layer of material packaging hinders the sorting of the primary objects.

RFID identification solutions also exist. In contrast, although the RFID is passive and contactless, the data used is coarse-grained, and some devices cannot be used as devices commonly used in life.

Therefore, how to realize material detection with lower cost, higher reliability and more intellectualization is a great problem which needs to be solved urgently at present.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides a material inspection method based on wireless commercial equipment, which can be applied to various use scenes needing material detection, utilizes low-cost mature wireless commercial equipment, and realizes the improvement of accuracy and the advantages of low cost, simple structure and the like.

In order to achieve the aim, the invention provides a passive non-contact material inspection method based on wireless commercial equipment, which is characterized by comprising the following steps: the detection system of the detection method comprises a data preprocessing module, a Channel State Information (CSI) data visualization module and a picture classification module which are sequentially connected, and specifically comprises the following steps:

step 1, constructing a test field by utilizing a channel between a transmitting antenna and a receiving antenna of wireless commercial equipment, transmitting a reference signal by the transmitting antenna, placing a material to be tested in a transmission path between the transmitting antenna and the receiving antenna, and receiving the reference signal transmitted by the channel by the receiving antenna;

step 2, the data preprocessing module tests the channel response after the material to be tested is put in, extracts the CSI data, and preprocesses the extracted CSI data;

step 3, a CSI data visualization module visualizes the wireless signal CSI and converts the wireless signal CSI into a CSI fingerprint picture;

and 4, inputting the CSI fingerprint image into an image classification module to obtain a detection result, wherein the image classification module comprises a trained neural network model and utilizes the neural network to perform classification and identification.

Further, the wireless commercial equipment comprises a router and a mobile terminal; the transmitting antenna and the receiving antenna can belong to the same device or can be respectively arranged on different devices.

Further, after the CSI data is extracted, a Butterworth (Butterworth) filter is selected.

Further, the conversion into the CSI fingerprint image is specifically a heat map formed by translating the material from the starting point to the end point on a plurality of designated test paths in the test field at a certain speed, the channel variation caused by the material is normalized to the relative amplitude of the heat map, and the size of the amplitude determines the color shade of the heat map; the test path is set based on fresnel zone theory.

In addition, step 41 is included before step 4, a neural network model is trained, and four layers of neural networks based on a convolutional neural network are used for classifying the input pictures; before inputting a picture into a network, the picture is first converted into a unified format.

The invention has the following beneficial effects:

1. the detection method adopts wireless signal channel information as a data source, compared with the difference that X rays are only weakly absorbed by non-metallic materials, ultrasonic waves can be strongly dispersed or absorbed in composite materials, and the like, so that the detection method is harmless to human bodies, and has good non-line-of-sight monitoring effect and low requirements on equipment. Not only can inspect the material fast, not be in addition the inspection laboratory, even as long as there is the wireless place to inspect the liquid.

2. The detection method introduces a Fresnel region theory and a Rice distribution theory, so that a reliable theoretical basis is set for experimental equipment, and the system can identify more substances (such as metal, liquid and the like) on the basis of saving equipment complexity, not only can distinguish substance types, but also can be widely applied to automatic identification of substance types.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.

FIG. 1 is a system architecture diagram of a wireless business device-based material inspection method according to an embodiment of the present invention;

FIG. 2 is a heat map color chart table according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a test field configuration according to an embodiment of the present invention;

FIG. 4 is a schematic illustration of the type of solid material provided by one embodiment of the present invention;

FIG. 5 is a schematic illustration of the type of liquid material provided by one embodiment of the present invention;

fig. 6 is an experimental schematic diagram of a test result of a lead plate material according to an embodiment of the present invention;

fig. 7 is a schematic diagram illustrating an influence of a lead plate, a cardboard plate, and a wood plate on the intermediate receiving antenna CSI according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a dynamic material movement path according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of the heat corresponding to three solid materials provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of the corresponding heat levels of three liquid materials provided in accordance with one embodiment of the present invention;

FIG. 11 is a schematic diagram of solid to liquid contrast heat provided by an embodiment of the present invention;

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.

Before the present invention is proposed, the technical principle of the present invention is introduced, mainly as follows:

1. theory and application of liquid dielectric constant

Many studies based on wireless signals have led to the development of systems that use wireless signals for different purposes, such as indoor positioning, motion recognition, and even emotion detection. These systems all utilize the amplitude and phase of the signal that the object causes to change during the movement. According to the electromagnetic wave theory, when the emitted electromagnetic wave meets media with different dielectric properties, a part of the electromagnetic wave is reflected, and the other part of the electromagnetic wave is transmitted. While the amplitude and phase changes of the signal caused by the movement of the object are mainly related to the reflected signal. However, this is not intentionally ignored by these researchers, mainly because the target human body in human motion recognition cannot transmit the signal, if any, and is very little, so that the signal of the transmission part has been discarded.

The present inventors have creatively found that "the signal of the electromagnetic wave transmission section has not been fully utilized" is further studied. In fact, due to the principle of dielectric characteristics, the dielectric characteristics also affect the transmission of electromagnetic waves and cause the signal amplitude and phase of the electromagnetic waves to change. In the material testing proposed by the invention, a liquid testing is involved, as an example, the liquid thickness is 3 cm, and the transmission signal can be captured from a theoretical point of view.

2. Use of metallic materials for reflection properties of electromagnetic waves

In 1900, a scientist Porudroude proposes a Drudroude model for the motion and propagation of electrons in metal, and considers that free electrons exist in metal and are regarded as gas, and the metal can conduct electricity and heat because of the motion of the free electrons. The wavelength range of radio signals is between 7 and 12 cm, while according to the de-rod model (which mainly describes the propagation characteristics of electrons in metallic materials), the active range of electrons is much smaller than the wavelength of electromagnetic waves. That is, the maximum displacement of electrons is much smaller than the wavelength of electromagnetic wave, and it can be concluded from the conversion formula that when the maximum displacement of electrons is smaller than the average interval between media, most signals of electrons will not be scattered at this time, and it can be approximately regarded that the electrons only do reflection motion at this time. Therefore, for ordinary metals, if the incident wavelength of the electromagnetic wave is greater than 5000 nm, the absorption of the electromagnetic wave by the metal is very small.

The present invention observes that the wavelength of the wireless signal is much greater than 5000 nanometers, so when the wireless signal is transmitted to the metal surface, it can be regarded as 100% reflection. Thus, in the embodiment of the present invention, when the metal material inspection is performed, the received signal amplitude and phase change can be regarded as being caused by the reflection path difference of the metal material.

Based on the conception, the invention designs and realizes a passive non-contact material inspection method based on wireless commercial equipment and by utilizing wireless channel state information. As shown in the system architecture diagram of fig. 1, the detection system of the present invention sequentially includes a data preprocessing module, a Channel State Information (CSI) data visualization module, and a picture classification module.

The test flow comprises the following steps:

step 1, constructing a test field by utilizing a channel between a transmitting antenna and a receiving antenna of the wireless commercial equipment, transmitting a reference signal by the transmitting antenna, placing a material to be tested in a transmission path between the transmitting antenna and the receiving antenna, and receiving the reference signal transmitted by the channel by the receiving antenna.

The wireless commercial device can adopt mature wireless devices, routers, mobile terminals and the like, and the transmitting antenna and the receiving antenna can belong to the same device or can be respectively arranged on different devices. Firstly, a standard test field is constructed for static solid and liquid materials and the like through flexible experimental setting, and based on the same standard test field, channel responses caused by the materials form unique dynamic fingerprints respectively corresponding to the materials. The same standard test field is based on the same distance between the transmitting and receiving antennas and the same basic signal (the frequency and the amplitude are the same). The CSI is adopted to characterize the channel response, so that the basis for visualizing the material fingerprint is provided.

And 2, testing the channel response after the material to be tested is placed in the channel response by the data preprocessing module, extracting the CSI data, and performing smooth preprocessing on the extracted CSI data.

As an embodiment, after the CSI data is extracted, a Butterworth filter is selected. When CSI data is sampled at a rate of Fs 1000 samples/s, the cutoff frequency of the butterworth filter is set to (2 pi · f)/Fs 0.0942 rad/s.

And step 3, visualizing the wireless signal CSI by the CSI data visualization module and converting the wireless signal CSI into a CSI fingerprint picture. Preferably, the conversion into a picture is a heat map formed by translating the material at a certain speed and then moving the test path in the test field from the starting point to the end point. The heat map is drawn based on channel changes caused by the fact that a test field is placed into a material, the channel changes are normalized to relative amplitude of the heat map, the size of the amplitude determines the color depth of the heat map, and a test path is set based on Fresnel region theory.

As an implementation example, when implemented: the material is first specified to move at a preset speed v on a horizontal line parallel to and bisecting the transmitting and receiving antennas. Such as 37 cm from left to right, which is also the best experimental area calculated according to fresnel zone theory, the width of the material is 10 cm, and the material advances 5 cm each time in the experiment. Thus, the mark appears in five positions starting from the left side of the material: 0 cm, 5 cm, 10 cm, 15 cm, 20 cm. The CSI amplitude values of all the positions are converted into heat map values, thereby constructing a heat map. These five positions are taken as Y-axis coordinates, and the X-axis coordinates are 90 subcarriers in total for 3 receiving antennas. Each of which is an amplitude heat map of the material at five locations at a time. The amplitude at this time is the relative amplitude, i.e., the amplitude at the blank time is subtracted, and is used to characterize the material-induced channel variation. The amplitude determines the color depth of the heat map, and the mapping relationship can be formulated according to a color card table, such as the heat map color card table provided in fig. 2 according to an embodiment of the present invention. And performing multiple tests on multiple positions to obtain multiple pictures. The heat maps are saved in batches, ready for subsequent sorting.

And 4, inputting the CSI fingerprint image into an image classification module to obtain a detection result, wherein the image classification module comprises a trained neural network model and utilizes the neural network to perform classification and identification. The neural network model preferentially uses a convolutional neural network, and fingerprint pictures are subjected to normalized processing before being input into the network.

Before step 4, a step 41 of training the neural network model is also included: the method comprises the steps of firstly, classifying the acquired amplitude heat degree graph by using a convolutional neural network; in order to further classify the material pictures, a shallow feature embedded network is employed, preferably a four-layer neural network based on a convolutional neural network is used.

Before inputting a picture into a network, the picture is first converted into a uniform format of 100 × 100 × 3 pixel point maps. And then, two modes are adopted for sampling the training set, one mode is uniform sampling, each type selects the same number of pictures for training, and the other mode is that all data sets are disturbed and then sampled in sequence. Then, the picture is input into a neural network for training, and secondly, a K nearest neighbor classification algorithm is used for comparison with the convolutional neural network. In order to keep up with the above convolutional neural network, here too the picture is converted into a 100 × 100 × 3 matrix. After a PKL array set is obtained, a K nearest-collar algorithm is called for classification.

Furthermore, training results of two training sets can be obtained, corresponding results of the two classification methods are compared with each other, the training results of the same object are subjected to weighted comprehensive processing, a mature classification model is obtained and used as a trained neural network model, and reliability is further improved.

The invention is further described below in conjunction with the experimental validation process of the present application:

firstly, constructing a test field:

in the experiment, the invention is preferably provided with one transmitting antenna and three receiving antennas; however, the number of antennas, M transmitting antennas and N receiving antennas, may be set according to actual needs in the art, and are not described herein. Antenna placement as shown in fig. 3, the prototype system consisted of two microcomputers equipped with a mature commercial network card, here exemplified by an intel 5300 network card (5 GHz). One of the computers is externally connected with an antenna as a transmitting end, and the other computer is externally connected with three antennas as a receiving end. The antennas are all fixed on a tripod, and the sampling frequency is 1000 Hz. The three receiving antennas are arranged on the same horizontal line and are on the same horizontal line with the transmitting antenna, and the distance between the antennas is fixed. Furthermore, a lead sheet is bound at the transmitting end based on the Rice distribution theory, so that the purpose of reducing signal distortion is achieved. The environment in which the test site is located is, as an example, performed in a 7 m x 10 m office.

Secondly, preparing a material to be detected:

for experimental verification, two groups of materials are prepared, wherein the first group is a paperboard, a wood board and a lead board which have the specification of 10 centimeters (length) multiplied by 10 centimeters (width) multiplied by 0.1 (thickness); the second group is a rectangular water bottle with the specification of 17 cm (length) x 11 cm (width) x3 cm (thickness), the capacity of which is about 430 ml, and 430 ml of purified water, and contains 430 ml of fruit juice of suspended fruit particles. Shown as a solid material in fig. 4 and a liquid material in fig. 5. During testing, a material is vertically placed on the non-metal platform between the transmitting antenna and the receiving antenna. As an example, the platform is a square support of 30 cm X30 cm. In addition, the solid materials herein are all placed on cardboard supports.

Third, preliminary CSI test verification

The purpose of the preliminary verification experiment is to observe the feasibility of a material inspection method, and to observe whether different materials have different influences on channel state information in advance. The following conclusions are drawn according to experimental settings and measurement results: first, the material does have an effect on the CSI signal. As shown in fig. 6, the lead plate affected the CSI for the lead plate material in the blank state as a control. Secondly, there are differences in the impact of different materials (different lines in the figure) on the CSI.

Furthermore, for the first group of solid materials, the influence of the cardboard and the wood board on the amplitude of the signal state information is not much different, but the influence of the lead board on the signal is more obvious, and since the metal material can absorb the wireless signal, as shown in fig. 7, the solid line at the bottom is the lead board, and only the middle antenna is selected, and the middle antenna is more obvious because the middle antenna is shielded more. For the second group of liquid materials, the signal was affected similarly for the empty bottles and tap water, whereas the signal was affected more differently for the juice with more suspended particles than for tap water.

Fourthly, the test and verification of the inspection system of the invention

Based on prior experiments, adjustments were made to the experimental setup, and since the target of the material inspection was static material, multiple angles and positions were tried to define a dynamic model of the material. The test field is unchanged, the solid material in the experimental material is consistent with the previous experiment, and 430 ml of the fruit juice containing the suspended fruit particles in the liquid material is changed into 430 ml of the thick fruit juice containing no suspended fruit particles.

During experimental tests, compared with earlier experiments, the material is not only placed at the middle position of the antenna any longer, but does translational motion along the central axes parallel to the three receiving antennas, the central axis is 37 cm long, the material moves once every 5 cm from the left side, the left side is taken as a left zero point, the material positions are 0 cm, 5 cm, 10 cm, 15 cm and 20 cm in sequence, and channel state information of each position is recorded, as shown in fig. 8 below.

As the test results are analyzed, for each material, a unique dynamic fingerprint of the material can be set and visualized due to the movement in position. Specifically, three receiving antennas are provided, and each antenna has 30 subcarriers, so that there are 90 subcarriers at each position, and each position is a vertical axis (0 cm, 5 cm, 10 cm, 15 cm, 20 cm) with 90 subcarriers as a horizontal axis, and 1 packet (i.e., data at a certain time) is intercepted at each position and visualized, so that a picture with 90 × 5 — 450 pixels can be obtained. The following conclusions were drawn in conjunction with the dynamic fingerprint of each material:

1) the dynamic pixel pictures of different materials have great difference, the classification result is very perfect, and the picture pixel points can be increased by increasing the material types and subdividing more positions in specific implementation. As shown in fig. 9, the visualized heat pictures of three solid materials are respectively shown, from left to right, the three solid materials are respectively a cardboard, a wood board and a lead board, and it can be observed that: in earlier experiments, due to the fixed position, the influence of cardboard and wood on the channel state information is very similar and difficult to distinguish, and after dynamic visualization, the distinction becomes obvious and can be distinguished better. Fig. 10 shows a dynamic visualized heat picture of an empty bottle, tap water and thick juice from left to right.

2) The dynamic fingerprints of the solid, the plastic and the liquid are found to be different greatly through integration, and the two experimental materials can be combined to realize the rough classification of the solid and the liquid. As shown in fig. 11, the schematic diagram of the comparative heat of metal (lead), plastic (container) and liquid (fruit juice) is from left to right.

Fifth, analysis and evaluation of verification results

5.1, setting up the experiment

In order to confirm which experiment in the first experiment and the second experiment sets a better result, the results of the two experiments are briefly evaluated, firstly, the CSI data collected from a single position in the first experiment are classified by using a K nearest neighbor algorithm, and secondly, the heat maps formed by all the positions in the second experiment are also classified by using the K nearest neighbor algorithm. The results of the classification are shown in table 1,

table 1 table of different experimental settings

It can be seen from the above table that under the condition that the value of K is the same and the number of the test set is also the same as that of the training set, the resolution of the ten types of materials is greatly improved when the five positions are compared with a single position, so that the experiment setting of experiment two in the experiment evaluation link is the final experiment setting.

5.2 analysis of the results

On the basis of the above experiment, several solid materials, namely an aluminum plate, a lead plate, an iron plate, a copper plate, a paperboard, an acrylic plate and a balsa board are added, including empty cups, tap water and fruit juice, the total of 10 materials, the volume of liquid materials and the like are consistent with those in the above experiment, and the liquid materials are all arranged in a plastic empty cup device.

Evaluation criteria: the evaluation criterion herein is the average accuracy, P, of predicting the six materials₁₀Namely, the total of ten materials of correctly predicting wood plates, paper plates, lead plates, aluminum plates, copper plates, acrylic plates, iron plates, plastic empty cups, drinking water and fruit juice is divided by the whole test set.

From the second experiment, it can be seen that the dynamic heat map of different materials is very different, and the influence on the wireless signal is stable and constant even at different time because of the static object. The inside of the heat picture of each type of material is also very similar. The heat map in the invention has higher similarity in the whole picture under each category.

The accuracy rates of the two classification algorithms are shown in table 2, and it can be seen that the accuracy rates of the two classification algorithms are very high without fitting, and no scientific research group does relevant work in the field of material classification by using commercial wireless signals, so that no comparison with other system schemes exists.

TABLE 2 analysis of properties

The experiment proves the high feasibility of the invention, and the experimental result shows that the invention has high detection precision for material inspection.

The invention creatively sets the dynamic CSI fingerprint for the material, can realize the nondestructive detection of the material based on mature and cheap wireless commercial equipment, and improves the accuracy of the analysis result by combining a plurality of analysis methods, thereby overcoming the defects in the prior art by the conception.

Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the scope of the present invention should be determined by the following claims.

Claims

1. A passive non-contact material inspection method based on wireless commercial equipment is characterized in that: the detection system of the detection method comprises a data preprocessing module, a Channel State Information (CSI) data visualization module and a picture classification module which are sequentially connected, and specifically comprises the following steps:

2. The method of claim 1, wherein the wireless commercial device comprises a router, a mobile terminal.

3. The method of claim 1, wherein the transmitting antenna and the receiving antenna can be owned by the same device or can be separately located on different devices.

4. The method of claim 1, wherein after the CSI data is extracted, a Butterworth (Butterworth) filter is selected.

5. The method of claim 1, wherein the converting to the CSI fingerprint picture is performed by translating the material from a starting point to an end point at a speed on a plurality of designated test paths in the test field, wherein the material-induced channel variations are normalized to relative amplitudes of the thermal map, and wherein the magnitudes of the amplitudes determine the shades of colors of the thermal map.

6. The method of claim 5, wherein the test path is based on fresnel zone theory.

7. The method of claim 1, further comprising, prior to step 4, step 41 of training a neural network model.

8. The method of claim 7, wherein the input picture is classified using a four-layer neural network based on a convolutional neural network.

9. The method of claim 8, wherein the pictures are first converted to a unified format before being input to the network.