CN113837554B - Multi-mode key information matching-based food safety risk identification method and system - Google Patents
Multi-mode key information matching-based food safety risk identification method and system Download PDFInfo
- Publication number
- CN113837554B CN113837554B CN202111008264.8A CN202111008264A CN113837554B CN 113837554 B CN113837554 B CN 113837554B CN 202111008264 A CN202111008264 A CN 202111008264A CN 113837554 B CN113837554 B CN 113837554B
- Authority
- CN
- China
- Prior art keywords
- food
- information
- risk
- risk information
- key information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000013305 food Nutrition 0.000 title claims abstract description 219
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000003993 interaction Effects 0.000 claims abstract description 81
- 239000013604 expression vector Substances 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 32
- 238000005516 engineering process Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 22
- 239000013598 vector Substances 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 22
- 230000002452 interceptive effect Effects 0.000 claims description 16
- 235000021393 food security Nutrition 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000003860 storage Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 239000002994 raw material Substances 0.000 claims description 5
- 238000004519 manufacturing process Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 11
- 238000012015 optical character recognition Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000001105 regulatory effect Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 239000012467 final product Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 206010027339 Menstruation irregular Diseases 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 238000011218 seed culture Methods 0.000 description 1
- 235000021404 traditional food Nutrition 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Operations Research (AREA)
- Primary Health Care (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to a food safety risk identification method and a system based on multi-mode key information matching, wherein the method comprises the following steps: establishing a risk information database, forming a food key information tracing map, and matching the risk information database with the food key information tracing map; a risk information interaction expression vector and a food key information expression vector are obtained through a pre-training risk information database, a single-mode encoder and a multi-mode interaction encoder in a food key information traceability map; the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model; storing risk information interaction expression vectors in a grading manner according to the occurrence frequency and importance of risks, establishing a risk information interaction expression grading library, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library; the problem that the current technology cannot monitor the whole process of food and dynamically update the multi-modal risks can be solved.
Description
Technical Field
The invention relates to the technical field of food safety management, in particular to a food safety risk identification method and system based on multi-mode key information matching.
Background
With the deep development of globalization of economy, the world-wide countries have increasingly compact economic relations, and the food trade scale is continuously expanding. As a special commodity, food safety concerns are related to human health and are receiving widespread attention in all countries of the world. In order to maintain food safety, relevant regulatory authorities are established in various countries and regions, a food risk monitoring mechanism is established, foods on the market are checked, risks are compared, and risk notices are issued towards society. Meanwhile, enterprises on the food chain can actively cooperate with a supervision organization to provide information of key links such as raw material breeding, food processing, transportation, detection and the like so as to ensure that the information is transparent and traceable.
However, current regulatory conditions present a contradiction between the ever-increasing food safety needs of people and the heavy task of food regulatory. The reasons for this contradiction are mainly: 1) The food contacted by the consumer is required to pass through a series of complicated links and processes from raw materials to products, the task of a supervisory personnel is heavy, the whole process is difficult to analyze and judge risks, and only the spot inspection information of the final product can be focused generally; 2) Food risk information collected from various channels is huge in data quantity, various data modes coexist, and the food risk information is updated dynamically at an irregular period, so that the cost for comparing information is huge.
In order to ensure food safety and discover potential risks in time, labor cost is reduced, and machine learning has become one of effective ways for improving traditional food supervision means. Both Chinese patent 202010622308.5 and 202011558703.8 disclose a risk assessment method based on food safety profiles. However, the method predicts the risk level by inputting the map information of the final product into a model containing fixed risk knowledge, and cannot meet the requirements of monitoring the food process data and dynamically updating the multi-modal risk knowledge.
Disclosure of Invention
Aiming at the technical problems existing in the prior art, the invention provides a food safety risk identification method and a system based on multi-mode key information matching, and provides a food safety risk intelligent identification method and a system based on deep learning, representation learning, information retrieval and other technologies aiming at matching of food key process data and multi-mode risk information.
According to a first aspect of the present invention, there is provided a food security risk identification method based on multimodal key information matching, comprising: step 1, acquiring food risk information of multiple data modes, and establishing a risk information database containing multiple data mode combinations;
step 2, collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map, and matching the risk information database with the food key information tracing map; the label is added manually according to the notices and/or public opinion information in the risk information database to indicate whether the food key information traceability map is matched with any piece of risk information;
step 3, a risk information interaction expression vector and a food key information expression vector are obtained through a representation learning technology, a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database and a food key information traceability map;
step 4, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model;
and 5, storing the risk information interaction expression vectors in a grading manner according to the occurrence frequency and the importance of the risks, establishing a risk information interaction expression grading library, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library.
On the basis of the technical scheme, the invention can also make the following improvements.
Optionally, the step 1 includes:
step 101, acquiring food risk information of a plurality of data modes through a character recognition technology; the plurality of data modalities includes text and forms;
step 102, translating the collected food risk information into the same language type through a machine;
and step 103, collecting the acquired food risk information and the local self food risk information, and establishing the risk information database.
Optionally, the step 2 includes:
step 201, collecting the food key process information of each link of the seed cultivation, production processing and transportation on the food tracing chain, wherein the food key process information comprises: temperature, humidity, batch number, and raw materials;
step 202, summarizing and extracting the key process information, and storing entities and relations in each link of the food tracing chain in a map form to obtain the food tracing map;
and 203, marking food safety risk information matched with the food traceability map in the risk information database.
Optionally, the step 3 includes:
step 301, for a single mode encoder f of any one mode k k And respectively randomly hiding part of elements in the corresponding single-mode data, performing first-stage training by utilizing context correlation in the data, and predicting hidden elements, wherein a loss function is as follows:
wherein S is a set of hidden elements in a sample input to the single mode encoder, f k (. Cndot.) is the prediction value of hidden element of the output of the single mode encoder, y x For the true value corresponding to the hidden element, phi is a function of the difference between the measurement predicted value and the true value selected according to the corresponding modal prediction;
step 302, for a multi-modal interactive encoder f m Part of elements in co-occurrence modal data are randomly hidden, semantic association among modalities is utilized to carry out second-stage training, hidden elements are predicted, and a loss function is as follows:
wherein T is a set of hidden elements in a sample of the input multi-mode interactive encoder, f m (. Cndot.) is the hidden element predictor output by the multi-modal interactive encoder;
step 303, using pooling operation to make the multi-mode interactive encoder f m And the output of the graph encoder of the food key information traceability graph is compressed into a one-dimensional representation vector: risk information interaction representation vector h R And a food key information representation vector h G 。
Optionally, the step 4 includes:
step 401, calculating the risk information interaction expression vector h through a pre-selected distance function or a multi-layer perceptron R And a food key information representation vector h G Probability of matching between p (h R ,h G )；
Step 402, performing a third stage training by using the labeled food risk matching information fine tuning model, where the loss function is:
wherein y epsilon {0,1} is a label which indicates that the food tracing information is matched with the risk information.
Optionally, the querying whether the food to be identified has risk through the risk information interaction representation hierarchical library in the step 5 includes:
step 501, obtaining, by a graph encoder, the food key information representation vector h of the food traceability map of the food to be identified G ；
Step 503, sequentially calculating the food key information expression vector h according to the grades G Interaction with the risk information represents a vector h R Matching probability p (h) R ,h G )；
Step 504, outputting the risk level and probability p (h R ,h G )。
According to a second aspect of the present invention, there is provided a food security risk identification system based on multimodal key information matching, comprising: the system comprises a risk information database establishment module, a food key information traceability map establishment module, a representation vector calculation module, a vector matching probability calculation module and a query module;
the risk information database establishing module is used for acquiring food risk information of various data modes and establishing a risk information database containing various data mode combinations;
the food key information tracing map establishing module is used for collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map, and matching the risk information database with the food key information tracing map;
the representation vector calculation module is used for obtaining risk information interaction representation vectors and food key information representation vectors through a representation learning technology, a pre-training risk information database and a single-mode encoder and a multi-mode interaction encoder in a food key information traceability map;
the vector matching probability calculation module is used for finely adjusting a model by using tag information by calculating the matching probability of the risk information interaction expression vector and the food key information expression vector;
the query module is used for storing the risk information interaction expression vector in a grading manner according to the risk occurrence frequency and the importance, establishing a risk information interaction expression grading library, and querying whether the food to be identified has risks or not through the risk information interaction expression grading library.
According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor for implementing the steps of a food safety risk identification method based on multimodal key information matching when executing a computer management class program stored in the memory.
According to a fourth aspect of the present invention there is provided a computer readable storage medium having stored thereon a computer management class program which when executed by a processor implements the steps of a food security risk identification method based on multimodal key information matching.
According to the food safety risk identification method, the system and the storage medium based on multi-mode key information matching, provided by the embodiment of the invention, a representation learning technology is adopted, and multiple independent or co-occurrence risk data modes are mapped into one-dimensional representation vectors in a unified manner, so that the risk sources and the multi-mode risks are more relevant, and the identification precision is improved; the key process information of the food tracing is integrated into the food tracing map, so that the supervision of the key process of the whole chain of the food is realized, and the supervision strength is improved; by adopting the information retrieval technology, matching of the food tracing information and the multi-mode risk information is completed, the dynamically updated risk information can be quickly matched, and labor cost is reduced. Firstly, converting risk information and food key information collected from different countries and regions into a machine-readable unified language digital format by means of multiple digital technologies such as crawlers, machine translation, OCR (optical character recognition) and the like to form a risk information database and a food key information traceability map; then, a risk information interaction expression vector and a food key information expression vector are obtained through a representation learning technology, a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database and a food key information traceability map; further, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model; and finally, establishing a risk information interaction representation hierarchical library DH, and inquiring whether the food to be identified has risks or not.
Drawings
FIG. 1 is a flowchart of a food safety risk identification method based on multimodal key information matching provided by an embodiment of the invention;
FIG. 2 is a diagram of a multimodal key information matching framework provided by an embodiment of the present invention;
FIG. 3 is a block diagram of a food security risk identification system based on multimodal key information matching provided by an embodiment of the present invention;
fig. 4 is a schematic hardware structure of one possible electronic device according to the present invention;
fig. 5 is a schematic hardware structure of a possible computer readable storage medium according to the present invention.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
With the popularization of information retrieval technology on a large scale, a deep learning matching model represented by a search engine can help people to rapidly complete target matching tasks from large-scale information. Therefore, the information retrieval technology can be well adapted to the problem that complex process links in food safety supervision are matched with a dynamic risk library. In the context of food safety, the model generally only models the question of whether dynamic multimodal risk matches a critical process link, without predicting risk from spot check information by learning specific risk knowledge.
Fig. 1 is a flowchart of a method for identifying food security risk based on multimodal key information matching, where the method includes, as shown in fig. 1: step 1, food risk information of multiple data modes is collected, and a risk information database DR containing multiple data mode combinations is established.
And 2, collecting food key process information of each link in a food tracing chain, forming a food key information tracing map DG with a label, matching a risk information database DR with the food key information tracing map DG, and manually adding the label according to announcements and/or public opinion information in the risk information database to indicate whether the food key information tracing map is matched with any piece of risk information.
And 3, obtaining risk information interaction expression vectors and food key information expression vectors by means of expression learning technology, and pre-training a single-mode encoder and a multi-mode interaction encoder in a risk information database DR and a food key information traceability map DG.
And 4, fine-tuning the model by using the label information by calculating the matching probability of the risk information interaction expression vector and the food key information expression vector.
And 5, storing the risk information interaction expression vectors in a grading manner according to the occurrence frequency and the importance of the risks, establishing a risk information interaction expression grading library DH, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library DH.
The invention provides a food safety risk intelligent identification method aiming at matching of food key process data and multi-mode risk information based on technologies such as deep learning, representation learning and information retrieval.
Example 1
Embodiment 1 provided by the present invention is an embodiment of a method for identifying food security risk based on multimodal key information matching provided by the present invention, as shown in fig. 2, which is a multimodal key information matching framework diagram provided by the embodiment of the present invention, and as can be known from fig. 2, the embodiment includes: step 1, food risk information of multiple data modes is collected, and a risk information database DR containing multiple data mode combinations is established.
In one possible embodiment, the method for collecting and preprocessing the food risk information from the public information of the domestic and foreign websites and the local own information specifically includes:
step 101, acquiring food risk information of a plurality of data modes through a character recognition technology; the plurality of data modalities includes text, forms, and the like.
Specifically, data can be collected through a crawler technology, and character recognition can be performed through an OCR (optical character recognition) technology. And collecting information such as official notices and network public opinion which are disclosed by domestic and foreign websites and appear in various data modes simultaneously or independently.
Step 102, translating the collected food risk information into the same language type through a machine.
Food risk information containing different national and regional languages is converted into the same language through a machine translation technology.
Step 103, collecting food risk information and local own food risk information, and establishing a risk information database DR.
And 2, collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map DG, and matching the risk information database DR with the food key information tracing map DG.
In one possible embodiment, step 2 includes:
step 201, collecting food key process information of each link such as seed culture, production processing and transportation on a food traceability chain, wherein the food key process information comprises: temperature, humidity, batch number, raw materials, etc.
Step 202, summarizing and extracting key process information, and storing entities and relations in each link of a food tracing chain in a map form to obtain a food tracing map DG.
And 203, marking food safety risk information matched with the food tracing map DG in the risk information database DR.
The specific operation can be marked manually.
And 3, obtaining risk information interaction expression vectors and food key information expression vectors by means of expression learning technology, and pre-training a single-mode encoder and a multi-mode interaction encoder in a risk information database DR and a food key information traceability map DG.
In one possible embodiment, step 3 includes:
step 301, for a single mode encoder f of any one mode k k And respectively randomly hiding part of elements in the corresponding single-mode data, performing first-stage training by utilizing context correlation in the data, and predicting hidden elements, wherein a loss function is as follows:
wherein S is a set of hidden elements in a sample input to the single mode encoder, f k (. Cndot.) is the prediction value of hidden element of the output of the single mode encoder, y x For the true value corresponding to the hidden element, phi is a function of the difference between the measured predicted value and the true value selected according to the corresponding modal prediction.
Step 302, for a multi-modal interactive encoder f m Part of elements in co-occurrence modal data are randomly hidden, semantic association among modalities is utilized to carry out second-stage training, hidden elements are predicted, and a loss function is as follows:
wherein T is a set of hidden elements in a sample of the input multi-mode interactive encoder, f m (. Cndot.) is the hidden element predictor output by the multi-modal interactive encoder.
Step 303, using pooling operation to make the multi-mode interactive encoder f m And the output of the graph encoder of the food key information tracing graph DG is compressed into a one-dimensional representation vector: risk information interaction representation vector h R And a food key information representation vector h G 。
And 4, fine-tuning the model by using the label information by calculating the matching probability of the risk information interaction expression vector and the food key information expression vector.
In one possible embodiment, step 4 includes:
step 401, calculating a risk information interaction expression vector h through a pre-selected distance function or a multi-layer perceptron R And a food key information representation vector h G Probability of matching between p (h R ,h G )；
Step 402, performing a third stage training by using the labeled food risk matching information fine tuning model, where the loss function is:
wherein y epsilon {0,1} is a label which indicates that the food tracing information is matched with the risk information.
And 5, storing the risk information interaction expression vectors in a grading manner according to the occurrence frequency and the importance of the risks, establishing a risk information interaction expression grading library DH, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library DH.
In a possible embodiment, the process of querying whether the food to be identified is at risk through the risk information interaction representation hierarchical database DH in step 5 includes:
step 501, obtaining a food key information representation vector h of a food traceability map DG of a food to be identified through a graph encoder G 。
Step 503, sequentially calculating the food key information expression vector h according to the grades G Interaction with risk information represents vector h R Matching probability p (h) R ,h G )。
Step 504, outputting the level and probability p (h R ,h G )。
Example 2
Embodiment 2 provided by the present invention is an embodiment of a food security risk identification system based on multimodal key information matching provided by the present invention, and fig. 3 is a structural diagram of a food security risk identification system based on multimodal key information matching provided by the embodiment of the present invention, as can be known from fig. 3, the embodiment includes: the system comprises a risk information database establishment module, a food key information traceability map establishment module, a representation vector calculation module, a vector matching probability calculation module and a query module.
The risk information database establishing module is used for acquiring food risk information of various data modes and establishing a risk information database DR containing various data mode combinations.
And the food key information tracing map establishing module is used for collecting food key process information of each link in the food tracing chain, forming a labeled food key information tracing map DG and matching the risk information database DR with the food key information tracing map DG.
The representation vector calculation module is used for obtaining risk information interaction representation vectors and food key information representation vectors through a representation learning technology, a single-mode encoder and a multi-mode interaction encoder in the pre-training risk information database DR and the food key information traceability map DG.
And the vector matching probability calculation module is used for finely adjusting the model by using the label information by calculating the matching probability of the risk information interaction expression vector and the food key information expression vector.
And the query module is used for storing the risk information interaction expression vector in a grading manner according to the risk occurrence frequency and the importance, establishing a risk information interaction expression grading library DH, and querying whether the food to be identified has risks or not through the risk information interaction expression grading library DH.
It can be understood that the food security risk recognition system based on the matching of the multimodal key information provided by the present invention corresponds to the food security risk recognition method based on the matching of the multimodal key information provided in the foregoing embodiments, and the relevant technical features of the food security risk recognition system based on the matching of the multimodal key information may refer to the relevant technical features of the food security risk recognition method based on the matching of the multimodal key information, which is not described herein again.
Referring to fig. 4, fig. 4 is a schematic diagram of an embodiment of an electronic device according to an embodiment of the invention. As shown in fig. 4, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored on the memory 1320 and executable on the processor 1320, wherein the processor 1320 executes the computer program 1311 to implement the following steps: step 1, acquiring food risk information of multiple data modes, and establishing a risk information database DR containing multiple data mode combinations; step 2, collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map DG, and matching a risk information database DR with the food key information tracing map DG; step 3, a risk information interaction expression vector and a food key information expression vector are obtained through a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database DR and a food key information traceability map DG by means of an expression learning technology; step 4, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model; and 5, storing the risk information interaction expression vectors in a grading manner according to the occurrence frequency and the importance of the risks, establishing a risk information interaction expression grading library DH, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library DH.
Referring to fig. 5, fig. 5 is a schematic diagram of an embodiment of a computer readable storage medium according to the present invention. As shown in fig. 5, the present embodiment provides a computer-readable storage medium 1400 having stored thereon a computer program 1411, which computer program 1411, when executed by a processor, performs the steps of:
step 1, acquiring food risk information of multiple data modes, and establishing a risk information database DR containing multiple data mode combinations; step 2, collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map DG, and matching a risk information database DR with the food key information tracing map DG; step 3, a risk information interaction expression vector and a food key information expression vector are obtained through a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database DR and a food key information traceability map DG by means of an expression learning technology; step 4, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model; and 5, storing the risk information interaction expression vectors in a grading manner according to the occurrence frequency and the importance of the risks, establishing a risk information interaction expression grading library DH, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library DH.
According to the food safety risk identification method, the system and the storage medium based on multi-mode key information matching, provided by the embodiment of the invention, a representation learning technology is adopted, and multiple independent or co-occurrence risk data modes are mapped into one-dimensional representation vectors in a unified manner, so that the risk sources and the multi-mode risks are more relevant, and the identification precision is improved; the key process information of the food tracing is integrated into the food tracing map, so that the supervision of the key process of the whole chain of the food is realized, and the supervision strength is improved; by adopting the information retrieval technology, matching of the food tracing information and the multi-mode risk information is completed, the dynamically updated risk information can be quickly matched, and labor cost is reduced. Firstly, converting risk information and food key information collected from different countries and regions into a machine-readable unified language digital format by means of multiple digital technologies such as crawlers, machine translation, OCR (optical character recognition) and the like to form a risk information database and a food key information traceability map; then, a risk information interaction expression vector and a food key information expression vector are obtained through a representation learning technology, a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database and a food key information traceability map; further, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model; and finally, establishing a risk information interaction representation hierarchical library DH, and inquiring whether the food to be identified has risks or not.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (8)
1. A method for identifying food safety risk based on multimodal key information matching, the method comprising:
step 1, acquiring food risk information of multiple data modes, and establishing a risk information database containing multiple data mode combinations;
step 2, collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map, and matching the risk information database with the food key information tracing map; the label is added manually according to the notices and/or public opinion information in the risk information database to indicate whether the food key information traceability map is matched with any piece of risk information;
step 3, a risk information interaction expression vector and a food key information expression vector are obtained through a representation learning technology, a single-mode encoder and a multi-mode interaction encoder in a pre-training risk information database and a food key information traceability map;
step 4, the matching probability of the risk information interaction expression vector and the food key information expression vector is calculated, and the label information is utilized to finely adjust the model;
step 5, storing risk information interaction expression vectors in a grading manner according to the occurrence frequency and importance of risks, establishing a risk information interaction expression grading library, and inquiring whether the food to be identified has risks or not through the risk information interaction expression grading library;
the step 3 comprises the following steps:
step 301, for a single mode encoder f of any one mode k k And respectively randomly hiding part of elements in the corresponding single-mode data, performing first-stage training by utilizing context correlation in the data, and predicting hidden elements, wherein a loss function is as follows:
wherein S is a set of hidden elements in a sample input to the single mode encoder, f k (. Cndot.) is a single mode encoder outputIs the hidden element predictor, y x For the true value corresponding to the hidden element, phi is a function of the difference between the measurement predicted value and the true value selected according to the corresponding modal prediction;
step 302, for a multi-modal interactive encoder f m Part of elements in co-occurrence modal data are randomly hidden, semantic association among modalities is utilized to carry out second-stage training, hidden elements are predicted, and a loss function is as follows:
wherein T is a set of hidden elements in a sample of the input multi-mode interactive encoder, f m (. Cndot.) is the hidden element predictor output by the multi-modal interactive encoder;
step 303, using pooling operation to make the multi-mode interactive encoder f m And the output of the graph encoder of the food key information traceability graph is compressed into a one-dimensional representation vector: risk information interaction representation vector h R And a food key information representation vector h G 。
2. The food safety risk identification method according to claim 1, wherein the step 1 comprises:
step 101, acquiring food risk information of a plurality of data modes through a character recognition technology; the plurality of data modalities includes text and forms;
step 102, translating the collected food risk information into the same language type through a machine;
and step 103, collecting the acquired food risk information and the local self food risk information, and establishing the risk information database.
3. The food safety risk identification method according to claim 1, wherein the step 2 comprises:
step 201, collecting the food key process information of each link of the seed cultivation, production processing and transportation on the food tracing chain, wherein the food key process information comprises: temperature, humidity, batch number, and raw materials;
step 202, summarizing and extracting the key process information, and storing entities and relations on each link of the food tracing chain in a map form to obtain a food tracing map;
and 203, marking food safety risk information matched with the food traceability map in the risk information database.
4. The food safety risk identification method according to claim 1, wherein the step 4 comprises:
step 401, calculating the risk information interaction expression vector h through a pre-selected distance function or a multi-layer perceptron R And a food key information representation vector h G Probability of matching between p (h R ,h G )；
Step 402, performing a third stage training by using the labeled food risk matching information fine tuning model, where the loss function is:
wherein y epsilon {0,1} is a label which indicates that the food tracing information is matched with the risk information.
5. The method according to claim 1, wherein the step 5 of querying the hierarchical library for risk through the risk information interaction representation comprises:
step 501, obtaining, by a graph encoder, the food key information representation vector h of the food traceability spectrum of the food to be identified G ；
Step 503, sequentially calculating the food key information expression vector h according to the grades G Interaction with the risk information represents a vector h R Matching probability p (h) R ,h G )；
Step 504, outputting the risk level and probability p (h R ,h G )。
6. A multi-modal key information matching based food safety risk identification system, comprising: the system comprises a risk information database establishment module, a food key information traceability map establishment module, a representation vector calculation module, a vector matching probability calculation module and a query module;
the risk information database establishing module is used for acquiring food risk information of various data modes and establishing a risk information database containing various data mode combinations;
the food key information tracing map establishing module is used for collecting food key process information of each link in a food tracing chain, forming a labeled food key information tracing map, and matching the risk information database with the food key information tracing map;
the representation vector calculation module is used for obtaining risk information interaction representation vectors and food key information representation vectors through a representation learning technology, a pre-training risk information database and a single-mode encoder and a multi-mode interaction encoder in a food key information traceability map;
the vector matching probability calculation module is used for finely adjusting a model by using tag information by calculating the matching probability of the risk information interaction expression vector and the food key information expression vector;
the query module is used for storing the risk information interaction expression vector in a grading manner according to the risk occurrence frequency and the importance, establishing a risk information interaction expression grading library, and querying whether the food to be identified has risks or not through the risk information interaction expression grading library;
the processing procedure of the representation vector calculation module comprises the following steps:
step 301, for a single mode encoder f of any one mode k k Part of elements in corresponding single-mode data are randomly hidden respectively, and context association in the data is utilizedTraining in the first stage, predicting hidden elements, and obtaining a loss function as follows:
wherein S is a set of hidden elements in a sample input to the single mode encoder, f k (. Cndot.) is the prediction value of hidden element of the output of the single mode encoder, y x For the true value corresponding to the hidden element, phi is a function of the difference between the measurement predicted value and the true value selected according to the corresponding modal prediction;
step 302, for a multi-modal interactive encoder f m Part of elements in co-occurrence modal data are randomly hidden, semantic association among modalities is utilized to carry out second-stage training, hidden elements are predicted, and a loss function is as follows:
wherein T is a set of hidden elements in a sample of the input multi-mode interactive encoder, f m (. Cndot.) is the hidden element predictor output by the multi-modal interactive encoder;
step 303, using pooling operation to make the multi-mode interactive encoder f m And the output of the graph encoder of the food key information traceability graph is compressed into a one-dimensional representation vector: risk information interaction representation vector h R And a food key information representation vector h G 。
7. An electronic device comprising a memory, a processor for implementing the steps of the multimodal key information matching based food security risk identification method of any of claims 1-5 when executing a computer management class program stored in the memory.
8. A computer readable storage medium, having stored thereon a computer management class program which when executed by a processor implements the steps of the multimodal key information matching based food security risk identification method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111008264.8A CN113837554B (en) | 2021-08-30 | 2021-08-30 | Multi-mode key information matching-based food safety risk identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111008264.8A CN113837554B (en) | 2021-08-30 | 2021-08-30 | Multi-mode key information matching-based food safety risk identification method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113837554A CN113837554A (en) | 2021-12-24 |
| CN113837554B true CN113837554B (en) | 2023-07-21 |
Family
ID=78961663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111008264.8A Active CN113837554B (en) | 2021-08-30 | 2021-08-30 | Multi-mode key information matching-based food safety risk identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113837554B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723464A (en) * | 2022-04-24 | 2022-07-08 | 中国标准化研究院 | Food contact material quality safety risk monitoring model training method and application |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107391906A (en) * | 2017-06-19 | 2017-11-24 | 华南理工大学 | Health diet knowledge network construction method based on neutral net and collection of illustrative plates structure |
| CN108509601A (en) * | 2018-04-02 | 2018-09-07 | 中山大学 | A kind of flavour of food products assessment method based on big data analysis |
| US10949907B1 (en) * | 2020-06-23 | 2021-03-16 | Price Technologies Inc. | Systems and methods for deep learning model based product matching using multi modal data |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10580533B2 (en) * | 2017-12-22 | 2020-03-03 | International Business Machines Corporation | Image-based food analysis for medical condition warnings |
| CA3081168A1 (en) * | 2019-05-22 | 2020-11-22 | Royal Bank Of Canada | System and method for machine learning architecture for partially-observed multimodal data |
| CN111832922B (en) * | 2020-06-30 | 2021-10-01 | 北方工业大学 | Food safety event risk studying and judging method and device based on knowledge graph reasoning |
| CN112561064B (en) * | 2020-12-21 | 2023-03-24 | 福州大学 | Knowledge base completion method based on OWKBC model |
-
2021
- 2021-08-30 CN CN202111008264.8A patent/CN113837554B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107391906A (en) * | 2017-06-19 | 2017-11-24 | 华南理工大学 | Health diet knowledge network construction method based on neutral net and collection of illustrative plates structure |
| CN108509601A (en) * | 2018-04-02 | 2018-09-07 | 中山大学 | A kind of flavour of food products assessment method based on big data analysis |
| US10949907B1 (en) * | 2020-06-23 | 2021-03-16 | Price Technologies Inc. | Systems and methods for deep learning model based product matching using multi modal data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113837554A (en) | 2021-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110889556B (en) | Enterprise operation risk characteristic data information extraction method and extraction system | |
| CN112035453B (en) | Recommendation method, device and storage medium based on GBDT high-order feature combination | |
| CN110825644B (en) | Cross-project software defect prediction method and system | |
| CN112612902A (en) | Knowledge graph construction method and device for power grid main device | |
| CN110990590A (en) | Dynamic financial knowledge map construction method based on reinforcement learning and transfer learning | |
| CN113723632A (en) | Industrial equipment fault diagnosis method based on knowledge graph | |
| Bukhsh et al. | Maintenance intervention predictions using entity-embedding neural networks | |
| CN110310012B (en) | Data analysis method, device, equipment and computer readable storage medium | |
| CN109241199B (en) | Financial knowledge graph discovery method | |
| CN110705283A (en) | Deep learning method and system based on matching of text laws and regulations and judicial interpretations | |
| CN114610898A (en) | Method and system for constructing supply chain operation knowledge graph | |
| CN116610816A (en) | Personnel portrait knowledge graph analysis method and system based on graph convolution neural network | |
| CN113837554B (en) | Multi-mode key information matching-based food safety risk identification method and system | |
| CN116257759A (en) | Structured data intelligent classification grading system of deep neural network model | |
| CN117151222B (en) | Domain knowledge guided emergency case entity attribute and relation extraction method thereof, electronic equipment and storage medium | |
| CN112100395B (en) | Expert cooperation feasibility analysis method | |
| CN113988961A (en) | Target object prediction method and device | |
| CN114741483A (en) | Data identification method and device | |
| Jeevanantham et al. | Extension of deep learning based feature envy detection for misplaced fields and methods | |
| CN114692954A (en) | Service optimization method and platform based on user experience knowledge graph database | |
| Sengupta | Applications of data mining in library & information centres: an overview | |
| CN117708759B (en) | Method and device for positioning industry link of enterprise | |
| CN117435777B (en) | Automatic construction method and system for industrial chain map | |
| US9053422B1 (en) | Computer system and method for planning a strategy from data and model ontologies to meet a challenge | |
| CN117474457B (en) | Intelligent auxiliary system for dangerous chemical and industrial and trade equipment emergency management law enforcement inspection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |