CN117095216B - Model training method, system, equipment and medium based on countermeasure generation network - Google Patents

Model training method, system, equipment and medium based on countermeasure generation network Download PDF

Info

Publication number
CN117095216B
CN117095216B CN202311063865.8A CN202311063865A CN117095216B CN 117095216 B CN117095216 B CN 117095216B CN 202311063865 A CN202311063865 A CN 202311063865A CN 117095216 B CN117095216 B CN 117095216B
Authority
CN
China
Prior art keywords
image
data
preset
model
soil
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311063865.8A
Other languages
Chinese (zh)
Other versions
CN117095216A (en
Inventor
万翔
谢淑云
李启铭
田野
崔倩倩
杨正论
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei Geological Survey
Original Assignee
Hubei Geological Survey
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei Geological Survey filed Critical Hubei Geological Survey
Priority to CN202311063865.8A priority Critical patent/CN117095216B/en
Publication of CN117095216A publication Critical patent/CN117095216A/en
Application granted granted Critical
Publication of CN117095216B publication Critical patent/CN117095216B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0475Generative networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/094Adversarial learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A model training method, system, equipment and medium based on an countermeasure generation network relates to the technical field of big data. In the method, soil characteristic data of an unmeasured area is acquired, and the soil characteristic data is converted into image data; processing image data according to preset parameter information to obtain an image sample; converting the file format of the image sample into a preset standard file format to obtain an input sample; inputting an input sample into a preset initial network model, acquiring historical chemical element distribution image data, training the preset initial network model, and obtaining an output result; constructing a loss function, and judging whether the loss value of the loss function is larger than a preset loss threshold value according to a preset mean square error formula; if not, the initial network model is used as a regional geochemical element prediction model. By implementing the technical scheme provided by the application, the regional geochemical element prediction model based on the antagonism generation network can be trained to realize the effect of rapidly and accurately predicting regional geochemical element concentration.

Description

Model training method, system, equipment and medium based on countermeasure generation network
Technical Field
The application relates to the technical field of big data, in particular to a model training method, system, equipment and medium based on an countermeasure generation network.
Background
With the continuous development of big data, the detection of regional geochemical data is also mature in the big data technical field. Detection of regional geochemical data is an important task in the fields of geological exploration, environmental monitoring and the like. Geochemical data is obtained by geochemical methods, including information on the content and distribution of chemical elements such as rock, soil, minerals, water, etc. The data can provide information about geological structures, deposit types, environmental pollution and the like, and has important significance in the fields of resource exploration, environmental protection and the like.
At present, the traditional geochemical element data acquisition mode is to acquire regional geochemical data through means of field sampling, laboratory analysis and the like so as to analyze and predict the data.
However, in practical application, the traditional method for determining the concentration of the geochemical element requires a large-scale field sampling by a worker and prediction of the concentration distribution of the chemical element in a target area by experience, and the method is time-consuming and inaccurate. Thus, there is a need for improved methods for evaluating the concentration profile of geochemical elements in a target area by workers.
Disclosure of Invention
The application provides a model training method, a system, equipment and a medium based on an countermeasure generation network, which can train a regional geochemical element prediction model based on the countermeasure generation network so as to realize the effect of rapidly and accurately predicting regional geochemical element concentration.
In a first aspect, the present application provides a model training method based on an countermeasure generation network, including:
acquiring historical chemical element distribution image data of a training area, wherein the historical chemical element distribution image data comprises historical soil characteristic data and corresponding historical chemical element concentrations;
Responding to image cutting operation, and processing the image data according to preset parameter information to obtain an image sample;
Converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format;
Responding to model training operation, extracting the historical soil characteristic data in the input sample as model input characteristics, inputting the model input characteristic data into a preset initial network model, dividing the corresponding historical chemical element concentration into training set data and test set data according to a preset proportion, and performing model training to obtain an output result;
Calculating an error value between the output result and preset standard data, and constructing a loss function according to the error value;
Judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value or not according to a preset mean square error formula;
and if not, taking the initial network model after training as a regional geochemical element prediction model.
By adopting the technical scheme, the system acquires the historical soil characteristic information of the training area and converts the historical soil characteristic information into image data, then cuts the image data through preset parameter information set by a worker, obtains an image sample, converts the file format of the image sample into tfrecords file format to obtain an input sample, is convenient for reading data during model training, responds to model training operation of the worker, inputs the input sample as model input characteristic input condition countermeasure generation network model to perform model training, extracts corresponding historical chemical element concentration as training set data and testing set data to perform model training, obtains an output result, calculates an error value of the output result and preset standard data and builds an error function, calculates a loss value of the error function, and if the loss value is smaller than a preset loss threshold value, takes the condition countermeasure generation network model as a regional geochemical element prediction model, so that accuracy of regional geochemical element concentration prediction can be effectively improved.
Optionally, the preset parameter information includes: an input data path, a clipping image size threshold, a clipping step size, and an output data path.
By adopting the technical scheme, the staff sets preset parameters: the system cuts the image according to the preset parameters, so that the accuracy of model training can be improved.
Optionally, extracting an original image corresponding to the image data in response to an image clipping instruction sent by a user; and cutting the image into a plurality of tif images with corresponding sizes according to a cutting mode of a sliding window and taking the tif images as the image samples according to a cutting image size threshold and a cutting step length in the preset parameter information.
By adopting the technical scheme, the image data is cut to the specified cut image size according to the cutting mode of the sliding window, so that the calculated amount of the system can be reduced.
Optionally, attaching the plurality of tif images with coordinate labels of the tif images in the original image, wherein the coordinate labels comprise a row number, a column number and a sample number; and storing the tif images attached with the same coordinate labels into the same folder to obtain a plurality of folders corresponding to the tif images stored with the same coordinate labels.
By adopting the technical scheme, the obtained multiple tif images are attached with the corresponding coordinate labels of the tif images in the original image, then the system stores the tif images with the same coordinate labels in the same folder to obtain multiple folders, the tif images can be divided, and the problem of model training efficiency caused by confusion of read data during model training is prevented.
Optionally, retrieving each tif image in the plurality of folders; converting the tif image into binary data corresponding to the tfrecords file format according to a preset serialization and inverse serialization data algorithm; and storing the binary data into a corresponding preset tfrecords file, taking the preset tfrecords file as the input sample, and using the preset tfrecords file for tensorflow frames to read data.
By adopting the technical scheme, according to the preset serialization and deserialization data algorithm, the system converts the tif image into binary data corresponding to tfrecords file format, and then stores the binary data into the corresponding preset tfrecords file, so that an input sample is obtained, a data basis can be provided for subsequent model training, and the model training efficiency is improved.
Optionally, the preset mean square error formula includes:
Wherein K is a loss value, n is the number of input samples, Y t is a preset standard element concentration, and Y p is an element concentration predicted value.
By adopting the technical scheme, the element concentration predicted value obtained by model training and the preset standard element concentration are combined to calculate to obtain the loss value, and the model training can be evaluated according to the loss value, so that the accuracy of the model training is improved.
Optionally, acquiring soil characteristic data of an unmeasured area; inputting the soil characteristic data of the non-measured area into the regional geochemical element prediction model to obtain a corresponding element distribution concentration prediction result; and converting the element distribution concentration prediction result into a tif format image and transmitting the tif format image to a user terminal.
By adopting the technical scheme, after model training is completed, soil characteristic data of an unbevelled region is obtained, the soil characteristic data of the unbevelled region is input into a regional geochemical element concentration prediction model, an element distribution concentration prediction result of a corresponding region is obtained, the prediction result is reduced into a tif format image and sent to a user terminal, the accuracy of regional geochemical element concentration prediction can be effectively improved, and the error influence of artificial acquisition and analysis of regional geochemical element concentration is reduced.
In a second aspect of the application, a system for model training methods based on an countermeasure generation network is provided.
The data acquisition module is used for acquiring historical chemical element distribution image data of the training area, wherein the historical chemical element distribution image data comprises historical soil characteristic data and corresponding historical chemical element concentrations;
The data processing module is used for responding to the image cutting operation, processing the image data according to preset parameter information and obtaining an image sample; converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format;
The model training module is used for extracting the historical soil characteristic data in the input sample as model input characteristics, inputting the model input characteristic data into a preset initial network model, dividing the corresponding historical chemical element concentration into training set data and test set data according to a preset proportion, and performing model training to obtain an output result;
the error recognition module is used for calculating an error value between the output result and preset standard data and constructing a loss function according to the error value; judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value or not according to a preset mean square error formula; and if not, taking the initial network model after training as a regional geochemical element prediction model.
In a third aspect of the application, an electronic device is provided.
In a fourth aspect of the application, a computer readable storage medium is provided.
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to implement a model training method based on an countermeasure generation network.
In summary, one or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:
1. according to the application, the historical soil characteristic information of a training area is obtained and converted into image data, then the image data is cut through preset parameter information set by a worker, an image sample is obtained, the file format of the image sample is converted into tfrecords file format to obtain an input sample, data is conveniently read during model training, then the input sample is used as model input characteristic input conditions of the worker to perform model training on a model generating network model in response to model training operation of the worker, the corresponding historical chemical element concentration is extracted as training set data and test set data to perform model training, an output result is obtained, an error value of the output result and preset standard data is calculated, an error function is constructed, and then a loss value of the error function is calculated, if the loss value is smaller than a preset loss threshold value, the condition countermeasure generating network model is used as a regional geochemical element prediction model, and the accuracy of regional geochemical element concentration prediction can be effectively improved.
2. The application cuts the image by a sliding window mode, and the sliding window mode can divide the image into a plurality of small areas, so that each small area can be independently processed, and the calculated amount of processing the image by the system is reduced.
3. According to the application, the regional geochemical element concentration is predicted by adopting the condition countermeasure generation network model, so that the cost of manual analysis of a worker in a laboratory can be reduced, a large amount of geological information is automatically fitted, and the accuracy of regional geochemical element concentration prediction is improved.
Drawings
Fig. 1 is a flow chart of a model training method based on an countermeasure generation network according to an embodiment of the present application.
Fig. 2 is a schematic diagram of an image clipping flow chart of a model training method based on an countermeasure generation network according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of a model training system based on an countermeasure generation network according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of an electronic device according to the disclosure.
Reference numerals illustrate: 301. a data acquisition module; 302. a data processing module; 303. a model training module; 304. an error recognition module; 400. an electronic device; 401. a processor; 402. a memory; 403. a user interface; 404. a network interface; 405. a communication bus.
Detailed Description
In order that those skilled in the art will better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments.
In describing embodiments of the present application, words such as "for example" or "for example" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "such as" or "for example" in embodiments of the application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "or" for example "is intended to present related concepts in a concrete fashion.
In the description of embodiments of the application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating an indicated technical feature. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
In order to facilitate understanding of the method and system provided by the embodiments of the present application, a description of the background of the embodiments of the present application is provided before the description of the embodiments of the present application.
At present, the traditional geochemical data acquisition mode is to acquire regional geochemical data through means such as field sampling, laboratory analysis and the like to perform data analysis and prediction, however, the traditional geochemical data acquisition method requires staff to input a large amount of regional geochemical characteristic information, and has the problems of high time consumption and low data acquisition accuracy.
The embodiment of the application discloses a model training method based on an antagonism generation network, which is used for obtaining regional geochemical data by acquiring image characteristic information and identifying the characteristic information according to a model and is mainly used for solving the problems of slow and inaccurate concentration of chemical elements in a target regional for evaluating the geochemical data by manual analysis.
Those skilled in the art will appreciate that the problems associated with the prior art are solved by the present application, and a detailed description of a technical solution according to an embodiment of the present application is provided below, wherein the detailed description is given with reference to the accompanying drawings.
Referring to fig. 1, a model training method based on an countermeasure generation network, the method includes S10 to S50, specifically includes the steps of:
S10: historical chemical element distribution image data of a training area is obtained, wherein the historical chemical element distribution image data comprises historical soil characteristic data and corresponding historical chemical element concentrations.
Specifically, the staff is provided with a study area comprising a training area and a non-training area. All the historical soil characteristic data of the training area, the corresponding historical chemical element concentration and the soil characteristic data of the non-area are stored in the database. The soil characteristic data comprises soil forming matrix characteristics and soil type characteristics, for example, red soil is usually rich in elements such as iron and aluminum, swamp soil is rich in elements such as organic matters and nitrogen, and soil texture and the like, for example, the proportion of particles with different particle diameters such as sand, silt and clay, and the like, and the soil with different textures can have different effects on the adsorption and release of the elements. The system acquires soil characteristic data of a training area in the database, and processes the soil characteristic data of an unmeasured area through preset geographic information software, such as arcgis and other geographic information software to obtain image data.
S20: and responding to the image clipping operation, and processing the image data according to the preset parameter information to obtain an image sample.
Specifically, a worker clicks an image clipping function button and inputs preset parameter information, the system clips image data in response to an image clipping instruction sent by the worker clicking the image clipping function button, and the clipped image is taken as an image sample.
The preset parameter information comprises four parameters of an input data path, a clipping image size threshold value, a clipping step length and an output data path. The input path is the path along which the image data is located, from which the system needs to read the image data. The size threshold of the clipping image is the size of a sub-image clipped from a complete image in the clipping process of the image data, and the setting of the size threshold of the clipping image needs to be determined according to a specific task. The cropping step is the pixel distance moved each time a sub-image is cropped from the image data. The selection of the clipping step length is set according to the clipping image size threshold value, and the appropriate clipping step length is set, so that the image data can be fully utilized, and the repeated generation data can be reduced. The output data path is a path in which the cropped image data is stored, and the model stores the cropped image data in the path for training and prediction. For example, the input data path for the telemetry image may be data/images, the crop image size threshold may be 256x256 pixels, the crop step size may be 128 pixels, and the output data path may be data/cropped_images. In the image cropping process, the system reads the original image from the data/images, crops the image according to the cropping step length of 256x256 pixels and 128 pixels, and stores the cropped image data into the data/cropping_images.
Referring to fig. 2, specific steps may include S21 to S22:
S21: and responding to an image clipping instruction sent by a user, and extracting an original image corresponding to the image data.
The system responds to an image clipping instruction sent by a worker, executes image clipping operation, and then retrieves an original image corresponding to an input path in a database according to the input path input by the worker, for example, the input path is data/images, wherein data is a root directory of a data storage, and images are subdirectories for storing image data.
S22: cutting the image into a plurality of tif images with corresponding sizes according to a cutting mode of a sliding window and taking the tif images as image samples according to a cutting image size threshold value and a cutting step length in preset parameter information.
After an original image corresponding to an input path is obtained, the system cuts the original image according to a cutting image size threshold and a cutting step length preset by a worker, and adopts a sliding window mode, wherein the sliding window is an image processing technology for extracting features from the image or performing tasks such as target detection, and the like. The clipping image size threshold corresponds to the size of the window, and the clipping step length corresponds to the distance that the window moves on the original image. After image cropping, the original image is cropped into a plurality of sub-images with the specified size of the cropping image size threshold, the sub-images are saved in a TIFF image format to obtain a plurality of tif images, and then the tif images are used as image samples by the system, and the tif images in the tif format are commonly used for storing geospatial information data such as satellite images, aerial images, digital elevation models and the like.
On the basis of the embodiment, an image storage process still exists after a plurality of tif images are obtained, and the specific steps include: for example, after obtaining a plurality of tif images, the system takes the index number of the tif images obtained after clipping in the original image as a coordinate label, and then takes the coordinate label as the file name corresponding to each tif image, namely the file name of the image sample. The coordinate tag includes a line number, a column number, and a sample number, for example, 1 in index sample_1_100_22 indicates the first image sample, 100 indicates the line number, and 22 indicates the column number. And then the system traverses each tif image and matches the tif images with the same file name into the same folder, so as to obtain a plurality of folders, wherein the tif images with the same file name, namely the tif images with the same coordinate labels, are stored in each folder.
S30: and converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format.
Specifically, since the conditional challenge-generating network algorithm is implemented based on tensorflow framework, the folders storing multiple tif images need to be converted into multiple tfrecords files before model training to facilitate the tensorflow framework to read the data. The format conversion process specifically includes: the system uses an existing image processing library, such as a file-like image reading tool to read the tif images in multiple folders and convert the tif images to tensorflow supported tensor formats, such as JPEG, PNG, or BMP. Tensors are a multi-dimensional array having any number of axes (also called dimensions) and shapes. Tensors may represent various types of data, including digital, image, audio, text, and so forth. In the tensorflow framework, all data is represented in tensors. The tif image is then converted to binary data according to a pre-set serialization data algorithm, such as the existing Protocol Buffers algorithm, and the binary data is then bundled with the folder name corresponding to the binary data and saved to a tfexample object, which tfexample object is in a serialization format for storing the data in the tfrecords file. The system then writes all tfexample objects to tfrecords files, each tfrecords file should contain the same amount of sample data, so that it can be efficiently read and processed during training. The tfrecords file serves as an input sample for model training.
S40: responding to model training operation, extracting historical soil characteristic data in an input sample as model input characteristics, inputting the model input characteristics into a preset initial network model, dividing the corresponding historical chemical element concentration into training set data and testing set data according to a preset proportion, and performing model training to obtain an output result.
Specifically, before model training, the staff needs to set parameters such as a training data path, an original data path, a predicted layer number, a conditional layer number, a training period, a batch input sample number, a training data storage period, a loss function weight value, a model storage path and the like. After the staff is set, clicking a start training function button to send a model training request, responding to the model training request, executing model training operation by the system, inputting an obtained input sample as a model input feature into a preset initial network model, generating the network model by using the preset initial network model as a conditional countermeasure, wherein the network model comprises a generator and two discriminators, acquiring historical chemical element distribution image data of a training area in a database, dividing the chemical element distribution image data into training set data and test set data according to a preset proportion, and performing model training, for example, taking ninety percent of data as training set data and ten percent of data as test set data.
The generator can convert random noise which accords with Gao Sizheng to be distributed into analog data, and the generator adopts a U-NET structure, namely jump connection, which is used for preventing information from being lost too much in the encoding process, so that training is not converged. The arbiter adopts a typical convolutional neural network structure, and needs to notice that the matrix with the format size of 30 x 30 output by the last layer of the network is the same as the design concept of the prior PatchGAN, and the image is evaluated from the global angle. The two discriminators can discriminate the analog data and the real data, and calculate a loss function according to the output discrimination matrix, namely Discrimnator Loss is used for optimizing the two discriminators; the error between the simulated data and the real data, i.e. the Euclidean distance between the two matrices, is used to construct the loss function. The output pseudo-judgment matrix is used as a basis for calculating the loss function, and the loss function of the generator can be constructed, so that the parameters of the generator are optimized, and the simulation data are more accurate.
During model training, soil characteristic data in the input samples are read, for example, the soil characteristic data include: soil-forming matrix characteristics: the soil-forming matrix refers to the original rock or sediment formed by the soil, which has an important effect on the source and content of elements in the soil. For example, if the earth-forming matrix is a limestone-rich formation, the calcium content of the soil may be higher. The soil matrix characteristics may be determined by taking soil samples of known areas and using geological survey data. Soil type characteristics: soil types are classified according to the formation process, composition, properties, and the like of the soil. Different types of soil have different elemental content and characteristics. For example, red soil is generally rich in elements such as iron, aluminum, etc., while swamp soil is rich in elements such as organic matter and nitrogen. By means of soil samples and soil classification systems of known areas, we can determine the soil type of an unknown area and infer its element concentration. Soil texture: the soil texture refers to the proportion of different particle size particles in the soil, such as sand, silt, clay, etc. The soil texture has important effects on the water retention, air permeability, nutrient retention and the like of the soil. Soil of different textures may have different effects on adsorption and release of elements. From soil samples of known areas and laboratory tests, we can determine the soil texture of the unknown areas and combine the existing data to predict element concentrations. According to the historical soil characteristic data and the corresponding historical chemical element concentration of the training area, the correlation between the soil characteristic data and the chemical element concentration can be obtained through the existing gray correlation method, so that the chemical element concentration of the non-area is predicted according to the correlation. The system takes the input image as a condition input, takes random noise as an input of a generator, and combines the random noise and the condition input to generate a target output. The discriminator uses the condition input and the generator output to discriminate the true or false of the target output, and updates the parameters of the model according to the discrimination result. The model is trained iteratively until the generator is able to generate a realistic target output. The target output is the output result of model training. For example, for an input image that includes a matrix forming characteristic, a soil type characteristic, and soil texture information, the information may be input as conditions, random noise may be input to a generator, and the generator may combine the random noise and the conditions to generate a target output, i.e., a predicted value of the concentration of the geochemical element in the region. The target output generated by the generator may evaluate the performance of the model by comparison with actual geochemical element concentration values.
S50: calculating an error value between the output result and preset standard data, and constructing a loss function according to the error value; judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value according to a preset mean square error formula; if not, the trained initial network model is used as a regional geochemical element prediction model.
Specifically, after obtaining the output result of the model training, the system compares the output result with preset standard data, that is, the geochemical element concentration predicted value and the actual geochemical element concentration value, and calculates an error value. The system collects all the trained and calculated error values, and constructs the existing Loss function L1 Loss, namely an absolute value Loss function, which is used for measuring the difference or error between the predicted value and the true value, and the calculation method of the L1 Loss is to take the average value or the sum of the absolute values of the difference between the predicted value and the true value. After obtaining the loss function, the system calculates a loss value corresponding to the output result obtained by the current training according to a preset mean square error formula during each training, then judges that the loss value corresponding to the output result obtained by the current training is compared with a preset loss threshold value, if the loss value corresponding to the output result obtained by the current training is smaller than the preset loss threshold value, the training is finished, and the trained condition countermeasure generation network model is used as a regional geochemical element prediction model. The preset mean square error formula specifically comprises:
wherein K is a loss value, n is an input sample number, Y t is a preset standard element concentration value, namely an actual geochemical element concentration value, and Y p is an element concentration predicted value, namely a geochemical element concentration predicted value obtained after training.
For example, the number of the acquired input samples is 1, the predicted value of the element concentration in the designated area is copper element concentration 5, the actual geochemical element concentration in the designated area is copper element concentration 4, the loss value is 1 through calculation of a preset mean square error formula, the preset loss threshold value is 1.5, the loss value is smaller than the preset loss threshold value, the model is judged to be converged to the actual geochemical element concentration value, and the initial network model is used as the regional geochemical element prediction model.
In a preferred embodiment of the present application, after model training is completed to obtain a regional geochemical element prediction model, the system may input regional condition data set by a worker into the regional geochemical element prediction model in response to a model prediction request sent by the worker, obtain a chemical element concentration predicted value corresponding to the regional condition data, and then automatically splice and restore the chemical element concentration predicted value into a tif format image and return the tif format image to the worker terminal.
The following are system embodiments of the present application that may be used to perform method embodiments of the present application. For details not disclosed in the platform embodiments of the present application, reference is made to the method embodiments of the present application.
Referring to fig. 3, a system for a model training method based on an countermeasure generation network according to an embodiment of the present application includes: a data acquisition module 301, a data processing module 302, a model training module 303, an error identification module 304, wherein:
the data acquisition module 301 is configured to acquire historical chemical element distribution image data of a training area, where the historical chemical element distribution image data includes historical soil feature data and corresponding historical chemical element concentrations;
The data processing module 302 is configured to process image data according to preset parameter information in response to an image cropping operation, so as to obtain an image sample; converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format;
The model training module 303 is configured to extract historical soil feature data in an input sample, input the historical soil feature data as model input features to a preset initial network model, divide the corresponding historical chemical element concentration into training set data and test set data according to a preset proportion, and perform model training to obtain an output result;
The error recognition module 304 is configured to calculate an error value between the output result and preset standard data, and construct a loss function according to the error value; judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value according to a preset mean square error formula; if not, the trained initial network model is used as a regional geochemical element prediction model.
It should be noted that: in the device provided in the above embodiment, when implementing the functions thereof, only the division of the above functional modules is used as an example, in practical application, the above functional allocation may be implemented by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the embodiments of the apparatus and the method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the embodiments of the method are detailed in the method embodiments, which are not repeated herein.
The application also discloses electronic equipment. Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 400 may include: at least one processor 401, at least one network interface 404, a user interface 403, a memory 402, at least one communication bus 405.
Wherein a communication bus 405 is used to enable connected communications between these components.
The user interface 403 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 403 may further include a standard wired interface and a standard wireless interface.
The network interface 404 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein the processor 401 may include one or more processing cores. The processor 401 connects the various parts within the entire server using various interfaces and lines, performs various functions of the server and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 402, and calling data stored in the memory 402. Alternatively, the processor 401 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 401 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, etc. The CPU mainly processes an operating system, a user interface diagram, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 401 and may be implemented by a single chip.
The Memory 402 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 402 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 402 may be used to store instructions, programs, code sets, or instruction sets. The memory 402 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described various method embodiments, etc.; the storage data area may store data or the like involved in the above respective method embodiments. The memory 402 may also optionally be at least one storage device located remotely from the aforementioned processor 401. Referring to fig. 4, an operating system, a network communication module, a user interface module, and an application program of a model training method based on a challenge-generating network may be included in a memory 402 as a computer storage medium.
In the electronic device 400 shown in fig. 4, the user interface 403 is mainly used as an interface for providing input for a user, and obtains data input by the user; and processor 401 may be used to invoke an application program in memory 402 that stores a model training method based on an antagonism generation network, which when executed by one or more processors 401, causes electronic device 400 to perform the method as in one or more of the embodiments described above. It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all of the preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. And the aforementioned memory includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a magnetic disk or an optical disk.
The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure.
This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.

Claims (9)

1. A model training method based on an countermeasure generation network, comprising:
acquiring historical chemical element distribution image data of a training area, wherein the historical chemical element distribution image data comprises historical soil characteristic data and corresponding historical chemical element concentrations;
Responding to image cutting operation, and processing the image data according to preset parameter information to obtain an image sample;
Converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format;
Responding to model training operation, extracting the historical soil characteristic data in the input sample as model input characteristics, inputting the model input characteristic data into a preset initial network model, dividing the corresponding historical chemical element concentration into training set data and test set data according to a preset proportion, and performing model training to obtain an output result;
Calculating an error value between the output result and preset standard data, and constructing a loss function according to the error value;
Judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value or not according to a preset mean square error formula;
if not, taking the trained initial network model as a regional geochemical element prediction model;
The soil characteristic data comprise soil forming matrix characteristics and soil type characteristics, for example, red soil is usually rich in elements such as iron and aluminum, swamp soil is rich in elements such as organic matters and nitrogen, and soil texture and the like, for example, the proportion of particles with different particle diameters such as sand, silt and clay;
The preset parameter information comprises an input data path, a clipping image size threshold, a clipping step length and an output data path.
2. The model training method based on the countermeasure generation network according to claim 1, wherein the processing the image data according to the preset parameter information in response to the image clipping operation to obtain the image sample includes:
responding to an image clipping instruction sent by a user, and extracting an original image corresponding to the image data;
And cutting the image into a plurality of tif images with corresponding sizes according to a cutting mode of a sliding window and taking the tif images as the image samples according to a cutting image size threshold and a cutting step length in the preset parameter information.
3. A method of model training based on an countermeasure generation network according to claim 2, further comprising, after the obtaining of the image samples:
Attaching the plurality of tif images with coordinate labels of the tif images in the original image, wherein the coordinate labels comprise row numbers, column numbers and sample numbers;
And storing the tif images attached with the same coordinate labels into the same folder to obtain a plurality of folders corresponding to the tif images stored with the same coordinate labels.
4. A model training method based on an countermeasure generation network according to claim 3, wherein the converting the file format of the image sample into a preset standard file format, to obtain an input sample, includes:
calling each tif image in the folders;
Converting the tif image into binary data corresponding to the tfrecords file format according to a preset serialization and inverse serialization data algorithm;
And storing the binary data into a corresponding preset tfrecords file, taking the preset tfrecords file as the input sample, and using the preset tfrecords file for tensorflow frames to read data.
5. The model training method based on an countermeasure generation network according to claim 1, wherein the preset mean square error formula includes:
Wherein K is a loss value, n is the number of input samples, Y t is a preset standard element concentration, and Y p is an element concentration predicted value.
6. A method of model training based on an countermeasure generation network according to claim 1, further comprising, after said taking said initial network model after training as a regional geochemical element prediction model:
acquiring soil characteristic data of an unmeasured area;
inputting the soil characteristic data of the non-measured area into the regional geochemical element prediction model to obtain a corresponding element distribution concentration prediction result;
And converting the element distribution concentration prediction result into a tif format image and transmitting the tif format image to a user terminal.
7. A model training system based on an countermeasure generation network of any of claims 1-6, the system comprising:
A data acquisition module (301) for acquiring historical chemical element distribution image data of a training area, wherein the historical chemical element distribution image data comprises historical soil characteristic data and corresponding historical chemical element concentrations; the soil characteristic data comprise soil forming matrix characteristics and soil type characteristics, for example, red soil is usually rich in elements such as iron and aluminum, swamp soil is rich in elements such as organic matters and nitrogen, and soil texture and the like, for example, the proportion of particles with different particle diameters such as sand, silt and clay;
The data processing module (302) is used for responding to the image clipping operation, processing the image data according to preset parameter information and obtaining an image sample; converting the file format of the image sample into a preset standard file format to obtain an input sample, wherein the preset standard file format is tfrecords file format; the preset parameter information comprises an input data path, a clipping image size threshold value, a clipping step length and an output data path;
The model training module (303) is used for extracting the historical soil characteristic data in the input sample as model input characteristics, inputting the model into a preset initial network model, dividing the corresponding historical chemical element concentration into training set data and testing set data according to a preset proportion, and performing model training to obtain an output result;
the error recognition module (304) is used for calculating an error value between the output result and preset standard data and constructing a loss function according to the error value; judging whether a loss value corresponding to the loss function is larger than a preset loss threshold value or not according to a preset mean square error formula; and if not, taking the initial network model after training as a regional geochemical element prediction model.
8. An electronic device comprising a processor (401), a memory (402), a user interface (403) and a network interface (404), the memory (402) being configured to store instructions, the user interface (403) and the network interface (404) being configured to communicate to other devices, the processor (401) being configured to execute the instructions stored in the memory (402) to cause the electronic device to perform a model training method based on an countermeasure generation network as claimed in any of claims 1-6.
9. A computer readable storage medium storing instructions which, when executed, perform a model training method based on an antagonism generating network according to any of claims 1-6.
CN202311063865.8A 2023-08-23 2023-08-23 Model training method, system, equipment and medium based on countermeasure generation network Active CN117095216B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311063865.8A CN117095216B (en) 2023-08-23 2023-08-23 Model training method, system, equipment and medium based on countermeasure generation network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311063865.8A CN117095216B (en) 2023-08-23 2023-08-23 Model training method, system, equipment and medium based on countermeasure generation network

Publications (2)

Publication Number Publication Date
CN117095216A CN117095216A (en) 2023-11-21
CN117095216B true CN117095216B (en) 2024-06-04

Family

ID=88778073

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311063865.8A Active CN117095216B (en) 2023-08-23 2023-08-23 Model training method, system, equipment and medium based on countermeasure generation network

Country Status (1)

Country Link
CN (1) CN117095216B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018212680A1 (en) * 2017-05-17 2018-11-22 Mineral Exploration Network (Finland) Ltd. Geochemical method for searching mineral resource deposits
CN109273059A (en) * 2018-10-15 2019-01-25 成都理工大学 A kind of regional geochemistry data correcting method
RU2728482C1 (en) * 2019-12-02 2020-07-29 Общество с ограниченной ответственностью "Газпромнефть Научно-Технический Центр" (ООО "Газпромнефть НТЦ") Methods for direct geochemical prediction of hydrocarbon deposits
CN111983715A (en) * 2020-07-23 2020-11-24 中国地质调查局发展研究中心 High and cold mountain area mineral-forming distant view area delineating method based on regional geochemistry
CN112567460A (en) * 2018-08-10 2021-03-26 日本电信电话株式会社 Abnormality detection device, probability distribution learning device, self-encoder learning device, data conversion device, and program
CN114022632A (en) * 2021-11-23 2022-02-08 中国地质大学(武汉) Geological model rapid generation method for generating countermeasure network based on deep convolution
CN115511214A (en) * 2022-10-25 2022-12-23 河海大学 Multi-scale sample unevenness-based mineral product prediction method and system
CN115602042A (en) * 2022-09-26 2023-01-13 湖北省地质调查院(Cn) Blank pattern spot geochemical element content assignment method and device and electronic equipment
CN116206700A (en) * 2023-02-04 2023-06-02 湖北省地质调查院 Calculation method and system for reserve variation of geochemical element
WO2023105255A1 (en) * 2021-12-07 2023-06-15 Totalenergies Onetech Basin-wise concentration prediction

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11054539B2 (en) * 2019-10-30 2021-07-06 Vadim Kukharev Methods of searching for mineral resources by analyzing geochemical and other anomalies during gravitational resonances

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018212680A1 (en) * 2017-05-17 2018-11-22 Mineral Exploration Network (Finland) Ltd. Geochemical method for searching mineral resource deposits
CN112567460A (en) * 2018-08-10 2021-03-26 日本电信电话株式会社 Abnormality detection device, probability distribution learning device, self-encoder learning device, data conversion device, and program
CN109273059A (en) * 2018-10-15 2019-01-25 成都理工大学 A kind of regional geochemistry data correcting method
RU2728482C1 (en) * 2019-12-02 2020-07-29 Общество с ограниченной ответственностью "Газпромнефть Научно-Технический Центр" (ООО "Газпромнефть НТЦ") Methods for direct geochemical prediction of hydrocarbon deposits
CN111983715A (en) * 2020-07-23 2020-11-24 中国地质调查局发展研究中心 High and cold mountain area mineral-forming distant view area delineating method based on regional geochemistry
CN114022632A (en) * 2021-11-23 2022-02-08 中国地质大学(武汉) Geological model rapid generation method for generating countermeasure network based on deep convolution
WO2023105255A1 (en) * 2021-12-07 2023-06-15 Totalenergies Onetech Basin-wise concentration prediction
CN115602042A (en) * 2022-09-26 2023-01-13 湖北省地质调查院(Cn) Blank pattern spot geochemical element content assignment method and device and electronic equipment
CN115511214A (en) * 2022-10-25 2022-12-23 河海大学 Multi-scale sample unevenness-based mineral product prediction method and system
CN116206700A (en) * 2023-02-04 2023-06-02 湖北省地质调查院 Calculation method and system for reserve variation of geochemical element

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Recognition of multivariate geochemical anomalies associated with mineralization using an improved generative adversarial network;Chunjie Zhang, et.al;Ore Geology Reviews;20210901(第136期);1-8 *
基于多尺度特征和元学习的智能预测找矿靶区实验研究;黄勇杰,等;计算机应用研究;20220127;第39卷(第6期);1772-1778 *
矿源层地球化学元素在黄土盖层中纵向分布的趋势提取与应用;谢淑云,等;地球科学(中国地质大学学报);20121115;第37卷(第6期);1133-1139 *
随州北部土地质量地球化学评价及空间分布研究;万翔,等;资源环境与工程;20190923;第33卷(第S1期);22-26 *

Also Published As

Publication number Publication date
CN117095216A (en) 2023-11-21

Similar Documents

Publication Publication Date Title
CN112580439B (en) Large-format remote sensing image ship target detection method and system under small sample condition
CN110598784B (en) Machine learning-based construction waste classification method and device
CN109359170B (en) Method and apparatus for generating information
CN110910343A (en) Method and device for detecting pavement cracks and computer equipment
KR101827908B1 (en) Ocean weather analysis device and method using thereof
CN109543907B (en) Complex terrain wind resource assessment method and device
CN113792667A (en) Method and device for automatically classifying properties of buildings in villages and towns based on three-dimensional remote sensing image
CN113360587B (en) Land surveying and mapping equipment and method based on GIS technology
Sun et al. Remote estimation of grafted apple tree trunk diameter in modern orchard with RGB and point cloud based on SOLOv2
CN115439753B (en) DEM-based steep river bank identification method and system
CN115457492A (en) Target detection method and device, computer equipment and storage medium
CN115272826A (en) Image identification method, device and system based on convolutional neural network
CN115292538A (en) Map line element extraction method based on deep learning
CN113901348A (en) Oncomelania snail distribution influence factor identification and prediction method based on mathematical model
CN117095216B (en) Model training method, system, equipment and medium based on countermeasure generation network
CN112837604A (en) Method and device for determining geographic coordinates of target point in map
KR101545998B1 (en) Method for Management Integration of Runoff-Hydraulic Model Data and System thereof
KR102416714B1 (en) System and method for city-scale tree mapping using 3-channel images and multiple deep learning
CN112669426B (en) Three-dimensional geographic information model rendering method and system based on generation countermeasure network
CN115062859A (en) Method and device for predicting density of gerbil unguiculatus
CN114238541A (en) Sensitive target information acquisition method and device and computer equipment
US11454733B2 (en) System and method for analysis of subsurface data
CN114239250A (en) System and method for territorial space planning design
CN116978052B (en) Subgraph layout recognition method of bridge design diagram based on improved YOLOv5
CN117195016B (en) Sewage treatment mode determining method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant