CN113049500B - Water quality detection model training and water quality detection method, electronic equipment and storage medium - Google Patents

Water quality detection model training and water quality detection method, electronic equipment and storage medium Download PDF

Info

Publication number
CN113049500B
CN113049500B CN202110296084.8A CN202110296084A CN113049500B CN 113049500 B CN113049500 B CN 113049500B CN 202110296084 A CN202110296084 A CN 202110296084A CN 113049500 B CN113049500 B CN 113049500B
Authority
CN
China
Prior art keywords
sample point
water quality
water body
water
spectral data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110296084.8A
Other languages
Chinese (zh)
Other versions
CN113049500A (en
Inventor
张昱升
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN202110296084.8A priority Critical patent/CN113049500B/en
Publication of CN113049500A publication Critical patent/CN113049500A/en
Application granted granted Critical
Publication of CN113049500B publication Critical patent/CN113049500B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/27—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands using photo-electric detection ; circuits for computing concentration
    • G01N21/274—Calibration, base line adjustment, drift correction
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/47—Scattering, i.e. diffuse reflection
    • G01N21/4738—Diffuse reflection, e.g. also for testing fluids, fibrous materials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The embodiment of the application provides a water quality detection model training and water quality detection method, an electronic device and a storage medium. The obtained spectral data of the water body at each sample point is subjected to simulated temperature drift processing, so that the condition that the spectral data of the water body at each sample point has temperature drift is simulated, and the obtained spectral data are more practical, therefore, the precision of the water quality detection model obtained by training based on the spectral data of the water body at each sample point after the simulated temperature drift processing and the actual water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.

Description

Water quality detection model training and water quality detection method, electronic equipment and storage medium
Technical Field
The application relates to the technical field of detection, in particular to a water quality detection model training and water quality detection method, electronic equipment and a storage medium.
Background
With the continuous development of society, the continuous acceleration of industrialization and urbanization processes, environmental pollution is more serious, particularly, the water quality conditions of water bodies such as rivers and lakes are continuously worsened, and phenomena such as eutrophication, water body area shrinkage and the like occur, so that the detection of the abnormal conditions of the water bodies and the correct response measures have very important significance. The water quality index refers to optical active substances influencing the optical properties of a water body under the natural environment, including chlorophyll, phycocyanin, total phosphorus, total nitrogen and the like, and can measure the eutrophication degree and transparency of the water area. The water quality index caused by the change of the optical properties of the water body is reflected on the spectral data reflected by the light waves on the surface of the water body, and corresponding water quality index parameters can be obtained by processing and analyzing the spectral data, so that the water quality detection is realized.
In the current water quality detection method, a deep learning method is adopted to detect water quality, a large amount of spectrum sample data and corresponding water quality index parameters are collected in the method, and an initial convolutional neural network is trained by utilizing the spectrum sample data and the corresponding water quality index parameters to obtain a water quality detection model; when the system is applied, the spectral data acquired in real time are input into the water quality detection model obtained by training, and corresponding water quality index parameters can be directly obtained, so that water quality detection is realized.
However, due to the influence of factors such as environment and acquisition strategy, the acquired spectrum sample data is prone to have deviation from the actual condition, which affects the accuracy of the water quality detection model training and thus affects the accuracy of water quality detection.
Disclosure of Invention
An object of the embodiment of the application is to provide a water quality detection model training method, a water quality detection method, an electronic device and a storage medium, so as to improve the precision of the water quality detection model training and further improve the accuracy of water quality detection. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a water quality detection model training method, including:
acquiring spectral data of water bodies at a plurality of sample points and actual water quality index parameters of the water bodies at the sample points;
carrying out simulated temperature drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing;
and training a preset neural network model based on the spectral data of the water body at each sample point after the temperature drift simulation treatment and the actual water quality index parameters of the water body at each sample point to obtain a water quality detection model.
Optionally, the step of performing temperature drift simulation processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing includes:
and performing spectral line drift processing and/or baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing.
Optionally, the spectral data of the water body at the sample point includes the wavelength of the light wave of the water body at the sample point and the reflectivity corresponding to the wavelength of the light wave;
the method comprises the following steps of performing spectral line drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing, wherein the steps comprise:
and performing interpolation operation according to a preset drift wavelength based on the light wave wavelength and the reflectivity corresponding to the light wave wavelength of the water body at each sample point to obtain the light wave wavelength and the reflectivity of the water body at each sample point after interpolation.
Optionally, the spectral data of the water body at the sample point includes a reflectivity corresponding to the wavelength of the light wave of the water body at the sample point;
the method comprises the following steps of performing baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing, wherein the steps comprise:
obtaining the intensity of ambient light;
and calculating the reflectivity of the water body at each sample point after baseline drift treatment based on the baseline by taking the ambient light intensity as the baseline according to the reflectivity, the ambient light intensity and the preset value corresponding to the light wave wavelength of the water body at each sample point.
Optionally, the neural network model includes a plurality of residual error layers, a dimensionality reduction layer, and an output layer connected in series; the residual error layer comprises convolution kernels of a plurality of scales, 1 × 1 convolution kernels aiming at the convolution kernels of the scales and used for fusing feature graphs output by the convolution kernels of the scales, a concat layer used for connecting the feature graphs output by the 1 × 1 convolution kernels and an activation function layer; the dimensionality reduction layer is used for carrying out channel dimensionality reduction on the feature graph output by the last residual error layer; the output layers include convolutional layers and dropout layers.
Optionally, the spectral data of the water body at the sample point includes the optical wavelength of the water body at the sample point and the reflectivity corresponding to the optical wavelength;
the residual layer further comprises a weighting layer; the weighting layer is used for respectively utilizing each preset wavelength weight to carry out point multiplication operation on the characteristic vector of the reflectivity corresponding to the light wavelength output by the activation function layer to obtain a weighted characteristic diagram; wherein, each preset wavelength weight is respectively set for different light wave wavelengths.
Optionally, the actual water quality index parameters of the water body at each sample point include: a large amount of first actual water quality index parameters obtained through non-chemical measurement and a small amount of second actual water quality index parameters obtained through chemical measurement;
training a preset neural network model based on the spectral data of the water body at each sample point after the temperature drift simulation treatment and the actual water quality index parameters of the water body at each sample point to obtain a water quality detection model, wherein the step comprises the following steps of:
initializing parameters of each network layer in a preset neural network model;
inputting the spectral data of the water body at the first sample point after the temperature drift simulation treatment corresponding to the first actual water quality index parameter into a neural network model to obtain a predicted water quality index parameter of the water body at the first sample point;
comparing the difference between the predicted water quality index parameter of the water body at the first sample point and the first actual water quality index parameter of the water body at the first sample point;
if the difference is greater than a first preset threshold value, adjusting parameters of each network layer in the neural network model based on the difference, returning to the step of inputting spectral data of the water body at a first sample point, which is subjected to temperature drift simulation processing and corresponds to the first actual water quality index parameter, into the neural network model to obtain a predicted water quality index parameter of the water body at the first sample point, and stopping training until the difference is less than the first preset threshold value or the number of times of cycle execution reaches a first preset number of times to obtain a trained neural network model;
inputting the spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain a predicted water quality index parameter of the water body at the second sample point;
calculating a gradient by using a preset loss function according to the predicted water quality index parameter of the water body at the second sample point, the actual water quality index parameter of the water body at the second sample point and a penalty term, wherein the penalty term is related to the parameter of the convolution layer in the trained neural network model;
and adjusting parameters of the convolution layer in the trained neural network model based on the gradient, returning to the step of inputting spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain the predicted water quality index parameter of the water body at the second sample point, and stopping training until the gradient is smaller than a second preset threshold or the number of times of cycle execution reaches a second preset number, and taking the currently trained neural network model as a water quality detection model.
In a second aspect, an embodiment of the present application provides a water quality detection method, including:
acquiring spectral data of water at each detection point in a water area to be detected;
sequentially inputting the spectral data of the water body at each detection point into the water quality detection model obtained by the method provided by the first aspect of the embodiment of the application to obtain the water quality index parameters of the water body at each detection point;
and analyzing the water quality index parameters of the water body at each detection point to obtain the water quality detection result of the water area to be detected.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
a memory for storing a computer program;
a processor, configured to implement the method provided by the first aspect or the method provided by the second aspect of the embodiments of the present application when executing the computer program stored in the memory.
In a fourth aspect, an embodiment of the present application provides a storage medium, where a computer program is stored in the storage medium, and when executed by a processor, the computer program implements the method provided by the first aspect or the method provided by the second aspect of the embodiment of the present application.
In a fifth aspect, embodiments of the present application further provide a computer program product containing instructions, which when executed on a computer, cause the computer to perform the method provided by the first aspect or the method provided by the second aspect of the embodiments of the present application.
According to the water quality detection model training and water quality detection method, the electronic device and the storage medium provided by the embodiment of the application, after the spectral data of the water body at the plurality of sample points are obtained, the spectral data of the water body at each sample point is subjected to simulated temperature drift processing to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing, and then the preset neural network model is trained on the basis of the spectral data of the water body at each sample point after the simulated temperature drift processing and the obtained actual water quality index parameters of the water body at each sample point to obtain the water quality detection model. The obtained spectral data of the water body at each sample point can be simulated to have temperature drift through the simulated temperature drift processing of the spectral data of the water body at each sample point, and the obtained spectral data is more practical, so that the precision of the water quality detection model obtained through training based on the spectral data of the water body at each sample point after the simulated temperature drift processing and the actual water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other embodiments can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of a water quality testing model training method according to an embodiment of the present application;
FIG. 2 is a spectral plot of an embodiment of the present application;
FIG. 3 is a schematic diagram of a model structure of a neural network model according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a residual layer structure according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a water quality detection method according to an embodiment of the present application;
FIG. 6 is a schematic structural diagram of a water quality testing model training device according to an embodiment of the present application;
FIG. 7 is a schematic structural diagram of a water quality detecting apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of protection of the present application.
In order to improve the precision of water quality detection model training and further improve the accuracy of water quality detection, the embodiment of the application provides a water quality detection model training and water quality detection method, an electronic device and a storage medium.
Next, a water quality detection model training method provided in the embodiment of the present application will be described first. The execution main body of the water quality detection model training method provided by the embodiment of the application can be an electronic device (such as a training machine) with a model training function. The method for implementing the water quality testing model training method provided by the embodiment of the application may be at least one of software, hardware circuit and logic circuit arranged in the execution main body.
As shown in fig. 1, an embodiment of the present application provides a water quality testing model training method, which may include the following steps.
S101, acquiring spectral data of the water body at a plurality of sample points and actual water quality index parameters of the water body at each sample point.
And S102, performing simulated temperature drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing.
S103, training a preset neural network model based on the spectral data of the water body at each sample point after the temperature drift simulation treatment and the actual water quality index parameters of the water body at each sample point to obtain a water quality detection model.
By applying the scheme of the embodiment of the application, after the spectral data of the water body at the plurality of sample points are obtained, the spectral data of the water body at each sample point are subjected to simulated temperature drift processing to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing, and then a preset neural network model is trained on the basis of the spectral data of the water body at each sample point after the simulated temperature drift processing and the obtained actual water quality index parameters of the water body at each sample point to obtain the water quality detection model. The obtained spectral data of the water body at each sample point can be simulated to have temperature drift by performing simulated temperature drift processing on the obtained spectral data of the water body at each sample point, and the obtained spectral data is more practical, so that the precision of the water quality detection model obtained by training based on the spectral data of the water body at each sample point after simulated temperature drift processing and the practical water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.
The water quality detection model is an end-to-end neural network model, and can realize inputting the spectral data of the water body at the detection point and directly outputting the spectral data to obtain the water quality index parameters of the water body at the detection point. The water quality detection model is obtained by training a large amount of sample data, specifically, during training, the water quality detection model is trained by taking the spectral data of the water body at a plurality of sample points as sample data input and taking the water quality index parameters of the water body at each sample point as output.
The sample points refer to a plurality of designated detection points in water areas such as rivers and lakes, the spectral data refer to data generated by the light waves at the sample points, specifically, the data can be reflectivity corresponding to the wavelength of the light waves, intensity of the light waves and the like, and in practical application, the spectral data are spectral band data, that is, the spectral data include the wavelength of the light waves and the reflectivity corresponding to the intensity of the light waves/wavelength of the light waves and the like. The spectral data can be obtained by measurement of a spectrometer or by means of image acquisition. Specifically, the manner of acquiring the spectral data of the water body at the plurality of sample points may be: the method comprises the steps of collecting images of water bodies at a plurality of sample points, and then preprocessing the image data to obtain spectral data of the water bodies at the sample points, wherein the preprocessing comprises radiometric calibration, geometric correction, atmospheric correction, water body area cutting, normalization processing and the like.
When the water quality detection model is trained, the actual water quality index parameters of the water body at each sample point are required to be acquired as nominal information during training, and the water quality index parameters are the numerical expression of the water quality index and can be the concentration, density, content and the like of the water quality index. The specific way of obtaining the actual water quality index parameters of the water body at each sample point may be: the method comprises the steps of collecting water body samples at each sample point in advance, carrying out chemical analysis or other water quality analysis on the water body samples to obtain actual water quality index parameters of the water body at each sample point, then storing the actual water quality index parameters of the water body at the sample points, and reading the actual water quality index parameters of the water body at each sample point from a warehouse during training.
If the spectral data of the water body at each original sample point is used for training, the precision of the water quality detection model is seriously reduced under the condition that the spectral data generate temperature drift, and the accuracy of the water quality detection model for water quality detection is influenced. Therefore, the embodiment of the application provides a temperature drift resistance training mode, that is, after the spectral data of the water body at each sample point is obtained, the spectral data of the water body at each sample point is subjected to temperature drift simulation treatment, so that the spectral data of the water body at each sample point after the temperature drift simulation treatment is obtained. And then training a preset neural network model based on the spectral data of the water body at each sample point after the temperature drift simulation treatment and the actual water quality index parameters of the water body at each sample point to obtain a water quality detection model.
The spectral data of the water body at each sample point after the temperature drift simulation treatment is input during training, so that the condition that the spectral data of the water body at each sample point has the temperature drift can be simulated, and the obtained spectral data is more practical, therefore, the precision of the water quality detection model obtained by training based on the spectral data of the water body at each sample point after the temperature drift simulation treatment and the actual water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.
In an implementation manner of the embodiment of the present application, S102 may specifically be: and performing spectral line drift processing and/or baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing.
In practical applications, the temperature drift of the spectral data mainly includes spectral line drift and baseline drift. The spectral line drift is the left drift or the right drift at the wavelength of the acquired spectral data; the baseline drift is a fixed value added or subtracted to the light wave intensity based on the ambient light intensity as a baseline. In a specific training process, both of the two kinds of drift processing may be performed, and of course, the probability of performing both of the two kinds of drift processing may be set to be 50%, or one kind of drift processing may be randomly selected, which is not specifically limited herein.
In an implementation manner of the embodiment of the present application, the spectral data of the water body at the sample point includes the optical wavelength of the water body at the sample point and the reflectivity corresponding to the optical wavelength. Correspondingly, the step of performing spectral line drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing may specifically be: and performing interpolation operation according to a preset drift wavelength based on the light wave wavelength and the reflectivity corresponding to the light wave wavelength of the water body at each sample point to obtain the light wave wavelength and the reflectivity of the water body at each sample point after interpolation.
The spectral line drift affects the left-right drift of the spectral data, which is equivalent to resampling the original spectral data, namely, based on the light wave wavelength and the reflectivity corresponding to the light wave wavelength of the water body at each sample point, interpolation operation is performed according to the preset drift wavelength, and the light wave wavelength and the reflectivity of the water body at each sample point after interpolation are obtained. The drift wavelength is a spectral line drift intensity parameter, which refers to the specific left drift or right drift at the wavelength of the acquired spectral data, and interpolation operation is performed according to the drift wavelength, so that resampling can be realized. In specific implementation, a spectrum curve may be first constructed based on the optical wavelength and the reflectivity corresponding to the optical wavelength of the water body at each sample point, where the horizontal axis of the spectrum curve is the optical wavelength and the vertical axis is the reflectivity corresponding to the optical wavelength (as shown in fig. 2). Then, according to a preset drift wavelength, performing interpolation operation on the spectral curve, taking bilinear interpolation operation as an example, assuming that the reflectance is 1.0 when the light wavelength is 700nm, the reflectance is 0.8 when the light wavelength is 701nm, and the preset drift wavelength is 0.4nm, performing spectral line drift processing, wherein the reflectance corresponding to the light wavelength after drift is 701nm is 0.8 (1-0.4) +1.0 =0.4 =0.88, that is, after spectral line drift processing, the reflectance corresponding to the light wavelength of 701nm is 0.88. Here, only bilinear interpolation operation is taken as an example, and more complex bicubic interpolation operation or other interpolation operation modes calculated by fields may also be used, which is not limited specifically here.
In another implementation of the embodiment of the present application, the spectral data of the water body at the sample point includes a reflectivity corresponding to a wavelength of light of the water body at the sample point. Correspondingly, the step of performing baseline shift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature shift processing may specifically be: acquiring the intensity of ambient light; and calculating the reflectivity of the water body at each sample point after baseline drift treatment based on the baseline by taking the ambient light intensity as the baseline according to the reflectivity, the ambient light intensity and the preset value corresponding to the light wave wavelength of the water body at each sample point.
When the spectral data of the water body at each sample point is collected, the spectral data of the environmental light (namely the environmental light intensity) can also be collected, and then the baseline drift processing is carried out, namely the environmental light intensity is taken as a baseline, a preset value is added or subtracted on the water body intensity spectrum at the sample point according to the reflectivity, the environmental light intensity and the preset value corresponding to the light wave wavelength of the water body at each sample point, and then the baseline drift processed water body intensity spectrum is calculatedAnd (3) the reflectivity of the water body at the sample point, wherein the preset value can be generated according to uniform distribution or Gaussian distribution. Recording the water intensity spectrum at the sample point as W w The intensity of ambient light is denoted as W as a molecule e As denominator, the reflectivity is calculated
Figure BDA0002984396600000091
Calculating the baseline drift to the water intensity spectrum, adding or subtracting a preset value count generated by uniform distribution on the water intensity spectrum, dividing by the ambient light intensity, and obtaining the reflectivity again, namely
Figure BDA0002984396600000092
Obtained W r_new The reflectivity corresponding to the light wave wavelength of the water body at the sample point after baseline drift processing based on the baseline is obtained.
The performance of data in a test set is counted, two indexes of RMSE (Root Mean Square Error) and MSE (Mean Square Error) are used, data with predicted values within the range of truth value +/-10% and truth value +/-15% account for the total number of samples in the test set, and 20-DEG C spectral line drift of a spectrum sensor is about 0.8 nm. The test results of the original model are shown in table 1, and the test results of the model after introducing the simulated temperature drift are shown in fig. 2.
TABLE 1 test results of the original model
RMSE MSE 10% ratio 15% ratio
Original model 1.47 2.17 76.5％ 87.8％
Spectral line drift of 1.2nm 1.77 3.16 66.7％ 81.6％
Spectral line drift of-1.2 nm 2.05 4.21 68.8％ 82.1％
Baseline wander 10count 1.49 2.23 75.3％ 86.5％
Baseline wander-10 count 1.74 3.05 71.2％ 83.8％
Table 2 test results of the model after introduction of the simulated temperature drift
RMSE MSE 10% by weight 15% ratio
Warm floating training model 1.61 2.60 75.2％ 87.3％
Spectral line drift of 1.2nm 1.62 2.62 74.4％ 86.8％
Spectral line drift of-1.2 nm 1.61 2.60 74.9％ 87.2％
Baseline wander 10count 1.68 2.82 73.9％ 86.6％
Baseline wander-10 count 1.75 3.06 71.3％ 84.1％
It can be seen from the above table that the model trained with normal data is less resistant to both drifts, especially spectral line drifts. After the simulated temperature drift is introduced in training for training, the testing accuracy of an original testing set is still equivalent, then four kinds of drift transformation input model testing are carried out on the testing set, the obvious spectral line drift accuracy is better kept, the baseline drift change is not large, and the baseline drift change is slightly improved. Tests show that the water quality detection model obtained by training can ensure that the output result is within a small amplitude error of a true value for drift change possibly generated within +/-20 degrees.
In an implementation manner of the embodiment of the present application, a specific training process of the water quality detection model is as follows: initializing a preset model parameter of a neural network model; inputting the spectral data of the water body at the sample point after the temperature drift simulation treatment into a neural network model to obtain a predicted water quality index parameter of the water body at the sample point; comparing the difference between the predicted water quality index parameter of the water body at the sample point and the actual water quality index parameter of the water body at the sample point; if the difference is larger than the preset threshold value, adjusting model parameters of the neural network model based on the difference, returning to the step of inputting the spectral data of the water body at the sample point after the temperature drift simulation treatment into the neural network model to obtain the predicted water quality index parameters of the water body at the sample point, stopping training until the difference is smaller than the preset threshold value or the number of times of cyclic execution reaches the preset number of times, and taking the currently trained neural network model as the water quality detection model.
In one implementation of the embodiment of the present application, the neural network model includes a plurality of residual layers, a dimensionality reduction layer, and an output layer connected in series. The residual layer comprises convolution kernels of a plurality of scales, 1 × 1 convolution kernels for fusing the feature maps output by the convolution kernels of the scales, a concat layer for connecting the feature maps output by the 1 × 1 convolution kernels and an activation function layer. The dimensionality reduction layer is used for carrying out channel dimensionality reduction on the feature graph output by the last residual error layer; the output layers include convolution layers and dropout layers.
The embodiment of the application adopts a residual error layer composed of a convolution kernel comprising a plurality of scales, a 1 × 1 convolution kernel for fusing a feature map output by the convolution kernel of each scale, and a concat layer and an activation function layer for connecting the feature maps output by the 1 × 1 convolution kernels, so as to replace a one-dimensional convolution layer in a conventional neural network model. The convolution kernels of multiple scales are adopted, the spectral data of various different receptive fields can be effectively extracted, then the convolution kernels of various scales are fused by using the 1 x 1 convolution kernels, under the condition that the effect of the convolution kernels is close to that of a conventional one-dimensional convolution layer, the calculated amount and the parameter amount which are approximate to one-dimensional channel fraction can be achieved, namely the calculated amount and the parameter amount are greatly reduced, in addition, a residual error structure is introduced, and the gradient transmission can be effectively carried out in the model training process.
Different from the pooling layer used in the current common water quality detection model, the embodiment of the application is connected with the dimensionality reduction layer behind the last residual error layer, and the dimensionality reduction layer can perform channel dimensionality reduction processing on the feature map output by the last residual error layer, so that the parameter quantity is reduced, the spatial information is kept not lost as far as possible in the feature extraction stage, and specifically, the dimensionality reduction layer can be a convolution of 1 × 1. And then connecting an output layer after the dimensionality reduction layer, wherein the output layer comprises convolution layers and dropout layers, the number of the convolution layers and the dropout layers in the output layer can be one or multiple, the convolution layers are shared by a plurality of water quality indexes, the dropout layers are used for preventing over-fitting, a specific connecting structure is that one dropout layer is connected behind each convolution layer, each water quality index is finally divided into a path to output a predicted water quality index parameter, the subsequent index increase or other modification of the water quality detection model is facilitated, compared with a full connecting layer adopted in a current common water quality detection model, the number of the convolution layers is fewer, and the parallel calculation efficiency among channels is higher.
Taking the example of including four residual layers and the output layer including two convolution layers and two dropout layers, the model structure of the neural network model is shown in fig. 3.
In an implementation manner of the embodiment of the present application, the spectral data of the water body at the sample point includes the optical wavelength of the water body at the sample point and the reflectivity corresponding to the optical wavelength. The residual error layer also comprises a weighting layer, wherein the weighting layer is used for performing point multiplication operation on the reflectivity characteristic vectors corresponding to the light wave wavelengths output by the activation function layer by using the preset wavelength weights respectively to obtain a weighted characteristic diagram. The preset wavelength weights are respectively set for different optical wavelengths.
The characteristics of the water quality detection model are combined, the known effective wave band for estimating the water quality index parameters is enhanced, so that the model can extract useful information more quickly, convergence is accelerated in the training process, and the introduced calculated amount and parameter amount are less.
The spectral characteristics of chlorophyll are: the light wave length 443nm is the maximum absorption peak of chlorophyll, the wave band 520-560nm is also the wave band reflecting the characteristics of chlorophyll and total phosphorus, and correspondingly, other water quality indexes also have similar spectral characteristics, so wavelength weights can be set respectively aiming at different light wave lengths in advance, and in the weighting layer, the point multiplication operation is performed on the characteristic vectors of the reflectivity corresponding to the light wave lengths output by the activation function layer by using the preset wavelength weights respectively to obtain a weighted characteristic diagram. As shown in fig. 3, each residual layer has a wavelength weight, and in specific implementation, a signal similar to a square wave may be generated based on the wavelength weights, and then the signal is used to smooth a curve formed by feature vectors of reflectances of corresponding light wavelengths output by the activation function layer, so that the feature vectors of effective positions may be enhanced, the feature vectors of unimportant positions may be weakened, the precision of model training may be further improved, and the accuracy of the water quality detection model may be improved.
Fig. 4 is a schematic diagram showing a specific structure of the residual layer. After passing through a 1 × 1 convolutional layer, a feature map (feature map) output by a previous layer is separated into convolutional kernels of various scales (four convolutional kernels of 1 × 1, 1 × 3, 1 × 5 and 1 × 7 shown in the figure are only examples), then the convolutional kernels are respectively fused through 1 × 1 convolution, feature connection is performed through a concat layer, then the feature map is output through an activation function layer, point multiplication operation is performed on feature vectors of reflectivity of corresponding wavelengths output by the activation function layer through weighting layers by using preset wavelength weights, the feature map is finally output after global average pooling, full connection, a smoothing function and a scale function are performed, the output feature map is accumulated with a feature map output by the previous layer to realize a residual error, and finally the feature map of the active function layer is output by the activation function layer.
In an implementation manner of the embodiment of the present application, the actual water quality index parameters of the water body at each sample point include: a large number of first actual water quality indicator parameters measured by a non-chemical method and a small number of second actual water quality indicator parameters measured by a chemical method.
Correspondingly, S103 may be specifically implemented by the following steps:
initializing parameters of each network layer in a preset neural network model;
inputting spectral data of the water body at the first sample point after the temperature drift simulation treatment corresponding to the first actual water quality index parameter into a neural network model to obtain a predicted water quality index parameter of the water body at the first sample point;
thirdly, comparing the difference between the predicted water quality index parameter of the water body at the first sample point and the first actual water quality index parameter of the water body at the first sample point;
step four, if the difference is larger than a first preset threshold value, parameters of each network layer in the neural network model are adjusted based on the difference, the second step is executed again, and the training is stopped until the difference is smaller than the first preset threshold value or the number of times of the cyclic execution reaches a first preset number, so that the trained neural network model is obtained;
fifthly, inputting the spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain a predicted water quality index parameter of the water body at the second sample point;
sixthly, calculating a gradient by using a preset loss function according to the predicted water quality index parameter of the water body at the second sample point, the actual water quality index parameter of the water body at the second sample point and a penalty term, wherein the penalty term is related to the parameter of the convolution layer in the trained neural network model;
and seventhly, adjusting parameters of the convolutional layer in the trained neural network model based on the gradient, and returning to execute the fifth step until the gradient is smaller than a second preset threshold value or the number of times of cyclic execution reaches a second preset number, stopping training, and taking the currently trained neural network model as the water quality detection model.
Many water quality indexes such as total phosphorus, total nitrogen and the like need to be measured by a chemical method, the cost of sample labeling is high, and a large number of water quality index parameters with truth value labels cannot be obtained. The water quality indexes such as chlorophyll and phycocyanin can be obtained by adopting a non-chemical method for measurement, and the like are identified by utilizing an image, so that a neural network model can be trained by utilizing spectral data of water at a first sample point after temperature drift simulation treatment corresponding to a large amount of first actual water quality index parameters obtained by the non-chemical method, parameters of each network layer in the neural network model are adjusted in the training process, after the training is finished, the spectral data of the water at a second sample point after temperature drift simulation treatment corresponding to a small amount of second actual water quality index parameters obtained by the chemical method for measurement are used for training the trained neural network model, parameters of a residual layer and a dimensionality reduction layer are fixed in the training, back propagation is not carried out, only the parameters of a convolution layer in an output layer are adjusted, and in order to prevent the detection effect of the water quality characteristics such as chlorophyll and phycocyanin from being influenced, a loss function is used when the loss function is set
Figure BDA0002984396600000131
Wherein L' (θ) is a gradient, and L (θ) is a loss function commonly usedThe number, such as MSE, is the difference result between the preset water quality index parameter and the actual water quality index parameter,
Figure BDA0002984396600000132
for penalty term, λ is a preset value, b i Is the weight of the parameter(s),
Figure BDA0002984396600000133
parameters theta of the trained convolutional layer based on data corresponding to water quality indexes such as chlorophyll and phycocyanin i Parameters of the convolutional layer being trained, b i And with
Figure BDA0002984396600000134
The absolute magnitude of the median is proportional. The original more important parameter weight is rarely changed in the training process, the water quality index can be continuously increased in the subsequent training, and meanwhile, the detection effect of the water quality characteristics such as chlorophyll, phycocyanin and the like is effectively ensured not to be influenced.
As shown in fig. 5, an embodiment of the present application provides a water quality detection method, which may include the following steps.
S501, acquiring spectral data of the water body at each detection point in the water area to be detected.
And S502, sequentially inputting the spectral data of the water body at each detection point into the water quality detection model to obtain the water quality index parameters of the water body at each detection point.
Wherein, the water quality detection model is obtained by training through the method of the embodiment shown in figure 1.
S503, analyzing the water quality index parameters of the water body at each detection point to obtain the water quality detection result of the water area to be detected.
By applying the scheme of the embodiment of the application, the water quality detection model is obtained by training by using the method shown in FIG. 1, and the accuracy of the water quality detection model is high, so that the accuracy of the water quality detection is improved when the water quality detection model is used for water quality detection.
When water quality detection is performed, spectrum data of a water body at each detection point in a water area to be detected needs to be acquired, the detection points refer to a plurality of designated positions in the water area to be detected, the spectrum data refer to data generated by light waves at the detection points, specifically, the data can be reflectivity, light wave intensity and the like, and in actual application, the spectrum data are spectrum band data, that is, the spectrum data comprise light wave wavelength and reflectivity corresponding to the light wave intensity/light wave wavelength. The spectral data can be obtained by measurement of a spectrometer and can also be obtained by an image acquisition mode. Specifically, the mode of acquiring the spectral data of the water body at each detection point in the water area to be detected may be: the method comprises the steps of collecting an image of a water area to be detected, and then preprocessing image data of corresponding positions of detection points in the image to obtain spectral data of the water body at the detection points, wherein the preprocessing comprises radiometric calibration, geometric correction, atmospheric correction, water body area cutting, normalization processing and the like.
After the spectral data of the water body at each detection point are obtained, the spectral data of the water body at each detection point are sequentially input into a water quality detection model, the water quality detection model is an end-to-end neural network model, the spectral data of the water body at the detection point can be input, and the water quality index parameters of the water body at the detection point can be directly output. The water quality detection model is obtained by training a large amount of sample data, specifically, during training, the water quality detection model is trained by taking the spectral data of the water body at a plurality of sample points as sample data input and taking the water quality index parameters of the water body at each sample point as output. The specific training mode is as shown in the embodiment of fig. 1, and is not described herein again.
After the water quality index parameters of the water body at each detection point are obtained, the water quality index parameters of the water body at each detection point can be analyzed, the specific analysis process can be concentration analysis, water quality index distribution analysis and the like, the water quality detection result of the water area to be detected can be obtained based on the analysis result, the results such as whether the water area to be detected is eutrophicated and how the water transparency is obtained, and specifically, a water quality index parameter distribution graph of the whole water area to be detected can be drawn according to the water quality index parameters of the water body at each detection point, so that the condition of the water area can be displayed more visually.
Corresponding to the above water quality testing model training method, the embodiment of the present application provides a water quality testing model training device, as shown in fig. 6, the device may include:
the first acquisition module 610 is used for acquiring spectral data of the water body at a plurality of sample points and actual water quality index parameters of the water body at each sample point;
the temperature drift processing module 620 is configured to perform temperature drift simulation processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing;
the training module 630 is configured to train a preset neural network model based on the spectral data of the water body at each sample point after the temperature drift simulation processing and the actual water quality index parameter of the water body at each sample point, so as to obtain a water quality detection model.
Optionally, the temperature drift processing module 620 may be specifically configured to: and performing spectral line drift processing and/or baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing.
Optionally, the spectral data of the water body at the sample point includes the optical wavelength of the water body at the sample point and the reflectivity corresponding to the optical wavelength;
the temperature drift processing module 620 may be specifically configured to: and based on the light wave wavelength of the water body at each sample point and the reflectivity corresponding to the light wave wavelength, performing interpolation operation according to a preset drift wavelength to obtain the light wave wavelength and the reflectivity of the water body at each sample point after interpolation.
Optionally, the spectral data of the water body at the sample point includes a reflectivity corresponding to a light wave wavelength of the water body at the sample point;
the temperature drift processing module 620 may be specifically configured to: obtaining the intensity of ambient light; and calculating the reflectivity of the water body at each sample point after baseline drift processing based on the baseline by taking the ambient light intensity as the baseline according to the reflectivity, the ambient light intensity and the preset value corresponding to the light wave wavelength of the water body at each sample point.
Optionally, the neural network model includes a plurality of residual error layers, a dimensionality reduction layer and an output layer connected in series; the residual error layer comprises convolution kernels of a plurality of scales, 1 × 1 convolution kernels aiming at the convolution kernels of the scales and used for fusing feature graphs output by the convolution kernels of the scales, a concat layer used for connecting the feature graphs output by the 1 × 1 convolution kernels and an activation function layer; the dimensionality reduction layer is used for carrying out channel dimensionality reduction on the feature graph output by the last residual error layer; the output layers include convolutional layers and dropout layers.
Optionally, the spectral data of the water body at the sample point includes the optical wavelength of the water body at the sample point and the reflectivity corresponding to the optical wavelength;
the residual layer further comprises a weighting layer; the weighting layer is used for respectively utilizing each preset wavelength weight to carry out point multiplication operation on the characteristic vector of the reflectivity corresponding to the light wavelength output by the activation function layer to obtain a weighted characteristic diagram; wherein, each preset wavelength weight is respectively set for different light wave wavelengths.
Optionally, the actual water quality index parameters of the water body at each sample point include: a large amount of first actual water quality index parameters obtained by non-chemical measurement and a small amount of second actual water quality index parameters obtained by chemical measurement;
the training module 630 may specifically be configured to:
initializing parameters of each network layer in a preset neural network model;
inputting the spectral data of the water body at the first sample point after the temperature drift simulation treatment corresponding to the first actual water quality index parameter into a neural network model to obtain a predicted water quality index parameter of the water body at the first sample point;
comparing the difference between the predicted water quality index parameter of the water body at the first sample point and the first actual water quality index parameter of the water body at the first sample point;
if the difference is greater than a first preset threshold value, adjusting parameters of each network layer in the neural network model based on the difference, returning to the step of inputting spectral data of the water body at a first sample point, which is subjected to temperature drift simulation processing and corresponds to the first actual water quality index parameter, into the neural network model to obtain a predicted water quality index parameter of the water body at the first sample point, and stopping training until the difference is less than the first preset threshold value or the number of times of cycle execution reaches a first preset number of times to obtain a trained neural network model;
inputting the spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain a predicted water quality index parameter of the water body at the second sample point;
calculating a gradient by using a preset loss function according to the predicted water quality index parameter of the water body at the second sample point, the actual water quality index parameter of the water body at the second sample point and a penalty term, wherein the penalty term is related to the parameter of the convolution layer in the trained neural network model;
and adjusting parameters of the convolution layer in the trained neural network model based on the gradient, returning to the step of inputting spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain the predicted water quality index parameter of the water body at the second sample point, and stopping training until the gradient is smaller than a second preset threshold or the number of times of cycle execution reaches a second preset number, and taking the currently trained neural network model as a water quality detection model.
By applying the scheme of the embodiment of the application, after the spectral data of the water body at the plurality of sample points are obtained, the spectral data of the water body at each sample point are subjected to simulated temperature drift processing to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing, and then a preset neural network model is trained on the basis of the spectral data of the water body at each sample point after the simulated temperature drift processing and the obtained actual water quality index parameters of the water body at each sample point to obtain the water quality detection model. The obtained spectral data of the water body at each sample point can be simulated to have temperature drift by performing simulated temperature drift processing on the obtained spectral data of the water body at each sample point, and the obtained spectral data is more practical, so that the precision of the water quality detection model obtained by training based on the spectral data of the water body at each sample point after simulated temperature drift processing and the practical water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.
Corresponding to the above water quality detection method, an embodiment of the present application provides a water quality detection apparatus, as shown in fig. 7, the apparatus may include:
the second obtaining module 710 is configured to obtain spectral data of a water body at each detection point in the water area to be detected;
the detection module 720 is configured to sequentially input the spectral data of the water body at each detection point into the water quality detection model obtained by the method shown in fig. 1, so as to obtain water quality index parameters of the water body at each detection point;
and the analysis module 730 is used for analyzing the water quality index parameters of the water body at each detection point to obtain a water quality detection result of the water area to be detected.
By applying the scheme of the embodiment of the application, the water quality detection model is obtained by training by using the method shown in FIG. 1, and the accuracy of the water quality detection model is high, so that the accuracy of water quality detection is improved when the water quality detection model is used for water quality detection.
An electronic device, as shown in fig. 8, includes a processor 801 and a memory 802. The memory 802 is used for storing computer programs; the processor 801 is configured to implement the water quality testing model training method or the water quality testing method provided in the embodiment of the present application when executing the computer program stored in the memory 802.
The Memory may include a RAM (Random Access Memory) or an NVM (Non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor including a CPU (Central Processing Unit), an NP (Network Processor), and the like; but also DSPs (Digital Signal Processing), ASICs (Application Specific Integrated circuits), FPGAs (Field-Programmable Gate arrays) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
By applying the scheme of the embodiment of the application, after the spectrum data of the water body at the plurality of sample points are obtained, the spectrum data of the water body at each sample point is subjected to simulated temperature drift processing to obtain the spectrum data of the water body at each sample point after the simulated temperature drift processing, and then the preset neural network model is trained on the basis of the spectrum data of the water body at each sample point after the simulated temperature drift processing and the obtained actual water quality index parameters of the water body at each sample point to obtain the water quality detection model. The obtained spectral data of the water body at each sample point can be simulated to have temperature drift through the simulated temperature drift processing of the spectral data of the water body at each sample point, and the obtained spectral data is more practical, so that the precision of the water quality detection model obtained through training based on the spectral data of the water body at each sample point after the simulated temperature drift processing and the actual water quality index parameters of the water body at each sample point is higher, and the accuracy of water quality detection by using the water quality detection model is improved.
In addition, the embodiment of the present application provides a storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the water quality detection model training method or the water quality detection method provided by the embodiment of the present application is implemented.
In another embodiment of the embodiments of the present application, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute the water quality testing model training method or the water quality testing method provided by the embodiments of the present application.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber, DSL (Digital Subscriber Line)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a DVD (Digital Versatile Disk)), or a semiconductor medium (e.g., a SSD (Solid State Disk)), etc.
As for the embodiments of the water quality testing model training device, the water quality testing device member identification device, the electronic device, the storage medium and the computer program product, the contents of the related methods are basically similar to the embodiments of the methods, so the description is simple, and the relevant points can be referred to the partial description of the embodiments of the methods.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the embodiments of the water quality testing model training device, the water quality testing device identity recognition device, the electronic device, the storage medium and the computer program product, since they are basically similar to the embodiments of the method, the description is relatively simple, and the relevant points can be referred to the description of the embodiments of the method.
The above description is only for the preferred embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims (9)

1. A water quality detection model training method is characterized by comprising the following steps:
acquiring spectral data of water bodies at a plurality of sample points and actual water quality index parameters of the water bodies at the sample points, wherein the actual water quality index parameters of the water bodies at the sample points comprise: a large amount of first actual water quality index parameters obtained through non-chemical measurement and a small amount of second actual water quality index parameters obtained through chemical measurement;
carrying out simulated temperature drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing;
initializing parameters of each network layer in a preset neural network model;
inputting the spectral data of the water body at the first sample point after the temperature drift simulation treatment corresponding to the first actual water quality index parameter into the neural network model to obtain a predicted water quality index parameter of the water body at the first sample point;
comparing the difference between the predicted water quality index parameter of the water body at the first sample point and the first actual water quality index parameter of the water body at the first sample point;
if the difference is larger than a first preset threshold value, adjusting parameters of each network layer in the neural network model based on the difference, returning to execute the step of inputting the spectral data of the water body at the first sample point, which is subjected to the temperature drift simulation treatment and corresponds to the first actual water quality index parameter, into the neural network model to obtain a predicted water quality index parameter of the water body at the first sample point, and stopping training until the difference is smaller than the first preset threshold value or the number of times of cycle execution reaches a first preset number of times to obtain a trained neural network model;
inputting the spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain a predicted water quality index parameter of the water body at the second sample point;
calculating a gradient by using a preset loss function according to the predicted water quality index parameter of the water body at the second sample point, the actual water quality index parameter of the water body at the second sample point and a penalty term, wherein the penalty term is related to the parameter of the convolution layer in the trained neural network model;
and adjusting parameters of a convolution layer in the trained neural network model based on the gradient, returning to execute the step of inputting the spectral data of the water body at the second sample point after the temperature drift simulation treatment corresponding to the second actual water quality index parameter into the trained neural network model to obtain the predicted water quality index parameter of the water body at the second sample point, stopping training until the gradient is smaller than a second preset threshold value or the number of times of cycle execution reaches a second preset number of times, and taking the currently trained neural network model as a water quality detection model.
2. The method according to claim 1, wherein the step of performing the simulated temperature drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing comprises:
and performing spectral line drift processing and/or baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing.
3. The method of claim 2, wherein the spectral data of the body of water at the sample point comprises a wavelength of light and a reflectance corresponding to the wavelength of light of the body of water at the sample point;
the step of performing spectral line drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the temperature drift simulation processing comprises the following steps:
and performing interpolation operation according to a preset drift wavelength based on the light wave wavelength and the reflectivity corresponding to the light wave wavelength of the water body at each sample point to obtain the light wave wavelength and the reflectivity of the water body at each sample point after interpolation.
4. The method of claim 2, wherein the spectral data of the body of water at the sample point comprises a reflectance corresponding to a wavelength of light of the body of water at the sample point;
the step of performing baseline drift processing on the spectral data of the water body at each sample point to obtain the spectral data of the water body at each sample point after the simulated temperature drift processing comprises the following steps:
obtaining the intensity of ambient light;
and calculating the reflectivity of the water body at each sample point after baseline drift treatment based on the baseline by taking the ambient light intensity as the baseline according to the reflectivity corresponding to the light wave wavelength of the water body at each sample point, the ambient light intensity and a preset value.
5. The method of claim 1, wherein the neural network model comprises a plurality of residual layers, dimensionality reduction layers, and output layers connected in series; the residual error layer comprises convolution kernels of a plurality of scales, a 1 x 1 convolution kernel for fusing the feature maps output by the convolution kernels of the scales aiming at the convolution kernels of the scales, a concat layer for connecting the feature maps output by the 1 x 1 convolution kernels and an activation function layer; the dimensionality reduction layer is used for carrying out channel dimensionality reduction on the feature map output by the last residual error layer; the output layers include convolutional layers and dropout layers.
6. The method of claim 5, wherein the spectral data of the body of water at the sample point comprises the wavelength of light and the reflectance corresponding to the wavelength of light of the body of water at the sample point;
the residual layer further comprises a weighted layer; the weighting layer is used for respectively utilizing each preset wavelength weight to carry out dot product operation on the characteristic vector of the reflectivity corresponding to the light wave wavelength output by the activation function layer to obtain a weighted characteristic diagram; and the preset wavelength weights are respectively set for different optical wavelengths.
7. A water quality detection method is characterized by comprising the following steps:
acquiring spectral data of water at each detection point in a water area to be detected;
sequentially inputting the spectral data of the water body at each detection point into the water quality detection model obtained by the method of any one of claims 1 to 6 to obtain water quality index parameters of the water body at each detection point;
and analyzing the water quality index parameters of the water body at each detection point to obtain a water quality detection result of the water area to be detected.
8. An electronic device comprising a processor and a memory;
the memory is used for storing a computer program;
the processor, when executing the computer program stored on the memory, implementing the method of any of claims 1-6 or the method of claim 7.
9. A storage medium, characterized in that the storage medium has stored therein a computer program which, when being executed by a processor, carries out the method of any one of claims 1-6 or the method of claim 7.
CN202110296084.8A 2021-03-19 2021-03-19 Water quality detection model training and water quality detection method, electronic equipment and storage medium Active CN113049500B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110296084.8A CN113049500B (en) 2021-03-19 2021-03-19 Water quality detection model training and water quality detection method, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110296084.8A CN113049500B (en) 2021-03-19 2021-03-19 Water quality detection model training and water quality detection method, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113049500A CN113049500A (en) 2021-06-29
CN113049500B true CN113049500B (en) 2022-12-06

Family

ID=76514091

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110296084.8A Active CN113049500B (en) 2021-03-19 2021-03-19 Water quality detection model training and water quality detection method, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113049500B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113567357B (en) * 2021-07-26 2024-05-24 杭州海康威视数字技术股份有限公司 Fusion method and device of spectrum data
CN117557917B (en) * 2024-01-11 2024-05-03 杭州海康威视数字技术股份有限公司 Water quality detection method and device
CN117571634B (en) * 2024-01-12 2024-04-12 杭州海康威视数字技术股份有限公司 Camera for monitoring water quality

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101968438A (en) * 2010-09-25 2011-02-09 西北农林科技大学 Method for distinguishing water injection of raw material muscles quickly
CN102636452A (en) * 2012-05-03 2012-08-15 中国科学院长春光学精密机械与物理研究所 NIR (Near Infrared Spectrum) undamaged identification authenticity method for wild ginseng
CN106769939A (en) * 2016-12-30 2017-05-31 无锡中科光电技术有限公司 The real-time calibration system and measurement calibration method of a kind of Multi-axial differential absorption spectrometer
CN108007916A (en) * 2017-03-16 2018-05-08 黑龙江八农垦大学 Hilbert Huang method establishes copolymerization Jiao's microscopic Raman measurement model of rice strain nitrogen content
CN108007917A (en) * 2017-03-16 2018-05-08 黑龙江八农垦大学 Hilbert method establishes nitrogen content raman spectroscopy measurement model method in rice strain
CN109064553A (en) * 2018-10-26 2018-12-21 东北林业大学 Solid wood board knot form inversion method based on near-infrared spectrum analysis
CN109811032A (en) * 2019-01-04 2019-05-28 山东省科学院海洋仪器仪表研究所 A kind of seawater microbial biomass spectral method of detection
CN110068544A (en) * 2019-05-08 2019-07-30 广东工业大学 Material identification network model training method and tera-hertz spectra substance identification
CN110389114A (en) * 2019-07-23 2019-10-29 上海市水文总站 A kind of medium and small water quality kind identification method based on unmanned plane imaging spectral
CN111693487A (en) * 2020-05-28 2020-09-22 济南大学 Fruit sugar degree detection method and system based on genetic algorithm and extreme learning machine
CN112014331A (en) * 2020-08-21 2020-12-01 中国第一汽车股份有限公司 Method, device and equipment for detecting water body pollution and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0029218D0 (en) * 2000-11-30 2001-01-17 Univ Ulster Improvements relating to gas discharge spectroscopy

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101968438A (en) * 2010-09-25 2011-02-09 西北农林科技大学 Method for distinguishing water injection of raw material muscles quickly
CN102636452A (en) * 2012-05-03 2012-08-15 中国科学院长春光学精密机械与物理研究所 NIR (Near Infrared Spectrum) undamaged identification authenticity method for wild ginseng
CN106769939A (en) * 2016-12-30 2017-05-31 无锡中科光电技术有限公司 The real-time calibration system and measurement calibration method of a kind of Multi-axial differential absorption spectrometer
CN108007916A (en) * 2017-03-16 2018-05-08 黑龙江八农垦大学 Hilbert Huang method establishes copolymerization Jiao's microscopic Raman measurement model of rice strain nitrogen content
CN108007917A (en) * 2017-03-16 2018-05-08 黑龙江八农垦大学 Hilbert method establishes nitrogen content raman spectroscopy measurement model method in rice strain
CN109064553A (en) * 2018-10-26 2018-12-21 东北林业大学 Solid wood board knot form inversion method based on near-infrared spectrum analysis
CN109811032A (en) * 2019-01-04 2019-05-28 山东省科学院海洋仪器仪表研究所 A kind of seawater microbial biomass spectral method of detection
CN110068544A (en) * 2019-05-08 2019-07-30 广东工业大学 Material identification network model training method and tera-hertz spectra substance identification
CN110389114A (en) * 2019-07-23 2019-10-29 上海市水文总站 A kind of medium and small water quality kind identification method based on unmanned plane imaging spectral
CN111693487A (en) * 2020-05-28 2020-09-22 济南大学 Fruit sugar degree detection method and system based on genetic algorithm and extreme learning machine
CN112014331A (en) * 2020-08-21 2020-12-01 中国第一汽车股份有限公司 Method, device and equipment for detecting water body pollution and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于神经网络模型的湛江湾水体有色溶解有机物的遥感估算;余果等;《海洋科学》;20181231;第42卷(第9期);第73-80页 *

Also Published As

Publication number Publication date
CN113049500A (en) 2021-06-29

Similar Documents

Publication Publication Date Title
CN113049500B (en) Water quality detection model training and water quality detection method, electronic equipment and storage medium
CN111738124B (en) Remote sensing image cloud detection method based on Gabor transformation and attention
CN116879297B (en) Soil moisture collaborative inversion method, device, equipment and medium
CN113283419B (en) Convolutional neural network pointer instrument image reading identification method based on attention
CN108447057A (en) SAR image change detection based on conspicuousness and depth convolutional network
Bailer-Jones The ILIUM forward modelling algorithm for multivariate parameter estimation and its application to derive stellar parameters from Gaia spectrophotometry
CN110018294B (en) Soil heavy metal detection value correction method and device and computer storage medium
CN107704802B (en) Performance prediction method for measurement of hyperspectral image clutter and anomaly detection algorithm
CN113030001B (en) Fruit sugar degree detection method and system
CN112528559B (en) Chlorophyll a concentration inversion method combining pre-classification and machine learning
CN111879709B (en) Lake water body spectral reflectivity inspection method and device
CN107632010A (en) A kind of quantitative approach of combination LIBS to steel samples
Khokhlov et al. Signatures of low-dimensional chaos in hourly water level measurements at coastal site of Mariupol, Ukraine
CN115049026A (en) Regression analysis method of space non-stationarity relation based on GSNNR
CN116858789A (en) Food safety detection system and method thereof
CN112859034B (en) Natural environment radar echo amplitude model classification method and device
CN111461923A (en) Electricity stealing monitoring system and method based on deep convolutional neural network
Küppers et al. Bayesian confidence calibration for epistemic uncertainty modelling
CN117669394A (en) Mountain canyon bridge long-term performance comprehensive evaluation method and system
CN117392564B (en) River water quality inversion method based on deep learning, electronic equipment and storage medium
CN107944474A (en) Multiple dimensioned cooperation table based on local auto-adaptive dictionary reaches hyperspectral classification method
CN111222543B (en) Substance identification method and apparatus, and computer-readable storage medium
CN103903258B (en) Method for detecting change of remote sensing image based on order statistic spectral clustering
Zhang et al. Hyperspectral images classification based on multi-scale residual network
CN113989632A (en) Bridge detection method and device for remote sensing image, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant