CN113159546B

CN113159546B - Crop supply chain hazard risk judging method and system based on unsupervised dimension reduction density clustering

Info

Publication number: CN113159546B
Application number: CN202110386361.4A
Authority: CN
Inventors: 孔建磊; 杨成才; 王小艺; 金学波; 王建利; 李柳生
Original assignee: Beijing Technology and Business University
Current assignee: Beijing Technology and Business University
Priority date: 2021-04-12
Filing date: 2021-04-12
Publication date: 2024-05-14
Anticipated expiration: 2041-04-12
Also published as: CN113159546A

Abstract

The invention provides a crop supply chain hazard risk judging method of unsupervised dimension-reduction density clustering, which comprises the following steps: acquiring a data sample set, wherein the data sample set comprises risk indexes; vector coding and standard normalization preprocessing are carried out on risk indexes in the data sample set to obtain a data sample set containing standardized data vectors; performing feature dimension reduction on the data sample set containing the standardized data vector to obtain a data sample set containing a high-dimension feature vector; performing clustering calculation screening on the data sample set containing the high-dimensional feature vector to obtain a risk level clustering center; and carrying out neighborhood search on the peripheral data of the risk level clustering center to obtain a risk judgment result. The invention can carry out grading evaluation on the risk conditions of different crops and foods.

Description

Crop supply chain hazard risk judging method and system based on unsupervised dimension reduction density clustering

Technical Field

The application relates to the field of food safety, in particular to a crop supply chain hazard risk judging method and system based on unsupervised dimension-reduction density clustering.

Background

Crops are indispensable products in daily life of people, and include food crops and cash crops, which influence national main security and stable economic and social development. However, in recent years, the quality safety problem of crops is more happening, and in order to reduce the risk threat of food sources, the risk grading evaluation is becoming a powerful guarantee for strengthening the food safety system of grains. The basic content of the method is to technically evaluate the risk possibly generated by the dangerous substances and the potential hazards based on a scientific level, namely, the pollution level of the food-borne dangerous substances is classified by combining various factors such as food characteristics, food pollution level, diet exposure and the like, and the risk level is quantified and the risk priority is identified in a plurality of complex food safety problems. The World Health Organization (WHO) and the United nations grain and agricultural organization (FAO) point out that food security risk classification evaluation is a structured decision-making process and is closely related to risk management, risk communication, risk prevention and control and the like, so that risk evaluators can be helped to accurately grasp the risk probability differences of different dangerous matters, guide risk management to clearly give priority to important supervision objects, and reasonably allocate resource decision corresponding management measures. The current risk classification evaluation method mainly comprises two aspects:

1) The index system method comprises the following steps: and extracting significant factors and potential factors from data such as food sampling inspection, investigation statistics and the like, and constructing a risk classification evaluation index system to carry out risk quantification classification. The method needs to integrate the characteristics of outbreak epidemiological data of chemical hazardous substances and food-borne diseases, evaluates food safety supervision spot inspection data of corresponding areas, builds a food risk classification index system on the basis of a qualitative or semi-qualitative risk evaluation mechanism, and analyzes and evaluates the food safety high risk combination needing to pay attention to.

2) Hierarchical model method: taking the diversity of foods, the diversity of the hazards, the toxicity difference of each hazard and the relevance of evaluation indexes into consideration, and taking factors such as growth, diffusion and the like into consideration, carrying out quantitative assignment, weight calculation and rank ordering on the possibility and severity of risk occurrence in a quantitative or semi-quantitative mode. The common models comprise a probability exposure evaluation model, a decision evaluation model, a sQMRA model, a FIRRM model, a iRisk method, a fuzzy comprehensive evaluation method, an integrated analysis method and the like, wherein the methods are used for constructing corresponding hazard risk evaluation models by using various statistical analysis and machine learning algorithms on the basis of comprehensively considering the diversity of foods and hazards and the difference of hazard toxicity, mining the inherent correlation among various indexes, and exploring the risk grades and probability indexes of different types of hazards.

At present, various risk evaluation technologies are applied to food safety evaluation, prevention and supervision, but crop supply is a multi-link process involving planting, production and processing, circulation and storage, sales and consumption and the like, wherein any link has hazard risk factors with different categories and degrees, and each factor is influenced by food diversity, data multisource isomerism, regional distribution difference, time variability and the like. The risk classification evaluation of the crop supply chain dangers involves a plurality of indexes, and the disaster degree and the influence on the social stability are various, so that the traditional risk classification method is difficult to be applied. In particular, the traditional method relies on metering data and statistical data in the risk analysis process, excessive human settings exist in index quantification and weight distribution in the analysis process, real sampling inspection supervision objective data verification is lacked, the action mechanism process of multidimensional heterogeneous food data on hazards in a supply chain link is ignored, and the coupling action mechanism of the hazards in the supply chain link is difficult to excavate in practical application, so that a false conclusion against practical association rules is easy to obtain in mathematical statistics.

Disclosure of Invention

In order to solve one of the technical problems, the invention provides a crop supply chain hazard risk judging method and system for unsupervised dimension-reduction density clustering.

An embodiment of the present invention provides a method for determining risk of crop supply chain hazards in an unsupervised dimension-reduction density cluster, where the method includes:

acquiring a data sample set, wherein the data sample set comprises risk indexes;

vector coding and standard normalization preprocessing are carried out on risk indexes in the data sample set to obtain a data sample set containing standardized data vectors;

Performing feature dimension reduction on the data sample set containing the standardized data vector to obtain a data sample set containing a high-dimension feature vector;

performing clustering calculation screening on the data sample set containing the high-dimensional feature vector to obtain a risk level clustering center;

And carrying out neighborhood search on the peripheral data of the risk level clustering center to obtain a risk judgment result.

A second aspect of embodiments of the present invention provides an unsupervised dimension-reduction density clustered crop supply chain hazard risk determination system, the system comprising a processor configured with processor-executable operating instructions to perform operations comprising:

The beneficial effects of the invention are as follows: aiming at main dangers in a supply chain, the invention constructs an unsupervised dimension-reduction density clustering hazard risk judging method, which can carry out grading evaluation on risk conditions of different crops and foods. According to the invention, risk evaluation indexes set in advance manually are not needed, internal relations of various dangerous matters in different supply chain link distributions are mined in a purely data driving mode, the risk weights of the dangerous matters factors are calculated in a self-adaptive mode, the risk classification situation of various dangerous matters on various supply chain links of crops is evaluated more scientifically, the subjective preset risk class number and limit value division interference are avoided, a targeted and credible scheme is provided for work such as important determination and supervision priority order arrangement of the supply chain dangerous matters, the management efficiency is improved, and the occurrence of related food safety accidents is reduced.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

FIG. 1 is a flow chart of a method for determining risk of crop supply chain hazards in an unsupervised dimension-reduction density cluster according to embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of principal component analysis;

FIG. 3 is a schematic diagram of a principal component analysis method according to example 1 of the present invention;

FIG. 4 is a schematic view of the local density according to example 1 of the present invention;

FIG. 5 is a schematic diagram of a comprehensive evaluation parameter screening clustering center according to embodiment 1 of the present invention;

fig. 6 is a schematic diagram of a density peak clustering principle according to embodiment 1 of the present invention, where (a) is two-dimensional data clustering, and (b) is a clustering center partition diagram;

FIG. 7 is a schematic view of the risk level density clustering results described in the examples;

FIG. 8 is a three-dimensional view of the risk level density clustering results described in the examples;

FIG. 9 is a graph showing the statistics of the risk level of a part of the dangerous objects described in the example;

FIG. 10 is a schematic diagram showing the risk level distribution of various types of pests in a crop supply chain as described in the examples;

FIG. 11 is a schematic diagram of risk level distribution of hazards within each supply chain link as described in the examples.

Detailed Description

In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of exemplary embodiments of the present application is provided in conjunction with the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application and not exhaustive of all embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.

Example 1

As shown in fig. 1, the embodiment provides a method for determining risk of crop supply chain hazards in an unsupervised dimension-reduction density cluster, which includes:

s101, acquiring a data sample set.

Specifically, the embodiment builds an unsupervised dimension-reduction density clustering hazard risk judging method aiming at main hazards (heavy metals, mycotoxins, pesticide residues, food additives and the like) existing in a supply chain based on food safety spot inspection data. The example analysis is carried out by using various dangerous matters in province and city of national main agricultural production. Information and data published by websites of the national grain bureau, the national quality inspection bureau and the like are collected, classified, sorted and preprocessed according to the types of hazard safety problems such as heavy metals, mycotoxins, microorganisms, food additives, pesticide residues and the like and sources of supply chain links thereof, and the types of hazard and sources of a large amount of grain processed product spot inspection data in a period of time are collected, so that the main production area and great consumption province of Chinese grain foods are covered.

In view of the fact that grain safety is affected by various factors such as policy and regulation, economy and society, and the like, a grain supply chain covers various links from planting, production and processing, storage logistics, to final sales and consumption in markets, supermarkets and other catering places, and potential safety hazards and risk factors can appear in each link. According to the embodiment, through analysis and preprocessing of professional websites and news media information in the literature and food industry, the hazard category of food safety accidents and source data thereof (for example, in 2013-2018), are obtained, are combined into spot check data to complete data expansion, a data set of 21027 samples is constructed, each data sample comprises 30 risk indexes, and each risk factor condition in a supply chain is reflected.

S102, vector coding and standard normalization preprocessing are carried out on risk indexes in the data sample set to obtain the data sample set containing the standardized data vector.

Specifically, in view of the fact that each risk index in the data sample set has category differences such as English, chinese characters, placeholders and the like, and has various structure type data such as numerical value type, logic type, character type, floating point type and the like. If these risk indicators are directly input into a cloud platform server or computer for processing, they will directly result in failure to store and analyze. In addition, each risk index value does not accord with the approximate normal distribution, and poor grading results can be caused by directly inputting the risk judgment model of the dangerous matters. Therefore, it is necessary to encode risk indicators of different categories and structures first to convert semi-structured, structured data into structured numeric.

In this embodiment, independent thermal encoding (One Hot Encoder) and Embedding encoding are used to encode the data sample sets so that the high-dimensional distances between different risk indicators are approximately the same. Firstly, performing 0/1 binarization processing on different types of characteristic values of each column of risk indexes in a data sample set by adopting independent thermal coding. Taking the production province index as an example, 5 types of characteristics of Henan, shandong, heilongjiang, jiangsu and Anhui are selected as the basis, and after vector coding operation, the method becomes Beijing 10000, hunan 01000, hebei 00100, hubei 00010 and Guangxi 00001, unstructured regional index codes are structured digital types, and the problem of digital size does not exist. However, the production provinces in the food data comprise a plurality of areas, a large amount of redundant sparse matrixes can be generated along with the rapid increase of dimensions, the calculation amount of a computer is increased, the computer does not perform useless operations, and the performance of the hazard risk judging process is not improved. Therefore, the embodiment continues to use Embedding codes to perform matrix weight embedding, and projects the ith characteristic value of a certain index in the sample to a high-dimensional space to perform coding, where the following formula is shown:

Where δ is a kronecker function, α represents all eigenvalues within the risk indicator, the function output is 1 when α=x _i, and 0 when not equal. If the number of possible eigenvalues of index x _i is m, δ is the eigenvector of length m. And ω _β is the inter-layer weight matrix connecting the one-hot coding and Embedding coding, β is the embedded corresponding index. Taking a certain characteristic value vector 'Beijing 10000' in the provincial index as an example, the vector is embedded into a low-dimensional matrix space after vector coding processing Through linear projection mapping of an omega weight matrix with n×m dimensions, let n=3 and m=5, the embedded vector features are shown as follows:

After Embedding codes are used for embedding to represent all types of characteristics, the inputs of all continuous variables are mapped and connected, so that compared with single-hot codes, the characteristic dimension is reduced, a large amount of computing resources and memory use are effectively avoided, and subsequent data processing and judgment model training are facilitated.

Through the data sample set, the meaning and the numerical range of each risk index of different samples are different, the characteristic values of the same risk index are far different, and the model is correctly trained due to abnormal size data and discrete distribution rules. In order to make different attribute indexes have comparability, the model better understands the meaning of data, and meanwhile eliminates the influence of attribute dimension, the embodiment performs linear normalization and standardization processing on the numerical value attribute obtained after vector encoding. And (3) linearly changing the numerical attribute to map the result to the range of [0,1] so as to realize the equal-ratio scaling of the numerical attribute. The normalization process is shown in the following formula:

Wherein the method comprises the steps of For the index feature value after vector coding, X _norm is the normalized result, and X _max、X_min is the data maximum value and minimum value respectively. And providing a basic data set for the subsequent risk scoring and early warning model by the normalized data. And then, carrying out normal distribution standardization processing on the characteristic results of each risk index, and aggregating each risk index to the distribution condition of approximate normal distribution with the mean value of 0 and the variance of 1. The normalization formula is as follows:

And (6) mu is the average standard value of the training set, s is the standard mean square error of the training set, y is the standard normalized data result, and finally, the data with far difference of all risk indexes are limited to the same range.

And S103, performing feature dimension reduction on the data sample set containing the standardized data vector to obtain the data sample set containing the high-dimension feature vector.

In particular, considering that the data sample set presents characteristics such as nonlinearity, multi-source heterogeneous distribution, too many dimensions and data structures are sometimes too late, and worse performance is generated in practical analysis application. The traditional simple linear principal component analysis (PRINCIPAL COMPONENT ANALYSIS, PCA) method transforms the related variables in the original vector space into a new coordinate in a mapping mode to form new independent vectors, and can greatly reserve the information of the original variables. As shown in fig. 2, the PCA reduces the feature of the n-dimensional data set to p principal component vectors of new composition, and if the maximum variance among the principal component vectors is obtained, it means that the amount of information contained is also maximum. Let the input data set x= (X ₁,X₂...,X_n)^T, note E (X) =u, cov (X) = Σ, and the following is obtained by linear transformation:

Wherein a ₁,a₂,...,a_p are p-dimensional unit vectors orthogonal to each other, namely vector characteristic results of the new transformed coordinate system. And solving to maximize the Pc variance, and sequencing the eigenvalues to obtain a principal component containing enough density information, thereby realizing the dimension reduction and parameter compression of the data. PCA, however, can only process linear data and does not work well in the face of non-linear food data. Therefore, the embodiment improves the PCA algorithm, provides a membership optimization kernel principal component analysis method (KERNEL PRINCIPAL Component Analysis, KPCA), maps the linear inseparable input vector into a high-dimensional nonlinear feature space, quantitatively analyzes the membership condition of each principal component by adopting a kernel function, realizes the unsupervised dimensionality reduction processing of the data in the data sample set through the effective dimensionality reduction processing of the nonlinear data, and can also avoid the dimensionality disaster problem. The schematic diagram is shown in fig. 3. Based on the kernel function inner product transformation principle, x _i and x _j are set as data sample points of an input space, linear inseparable standardized data vectors in a data sample set containing the standardized data vectors are projected into a high-dimensional feature space through a nonlinear mapping kernel function phi, and the kernel function is expressed in a Gaussian form, wherein the expression is as follows:

(x_i,x_j)→K(x_i,x_j)＝Φ(x_i)·Φ(x_j)

Where x _i,x_j is the sample point of the input space and σ is the gaussian kernel bandwidth. N samples x _k (k=1, 2,., n) of the input space, given that each index is subjected to coding and normalization preprocessing, are known The principal component covariance matrix C is obtained as follows:

Introducing a kernel function to transform an input original sample point into a sample point in a high-dimensional feature space, and obtaining a feature space covariance matrix as shown in the following formula:

The eigenvalue λ of the solution eigenvalue and the corresponding eigenvector v are shown as follows:

λ(Φ(x_k)·ν)＝Φ(x_k)·Cν

since the feature vector can be linearly represented by the dataset, v can be linearly represented by Φ (x _i):

Wherein s _i represents a membership value (0<s _i.ltoreq.1), representing the importance of each sample to the principal component partitioning hyperplane.

In this embodiment, the membership value is used to calculate the class center of each data sample from the positive class and the negative class, and the euclidean distance between the sample point and the positive class and the negative class is compared to determine the membership condition. The membership function is expressed as:

When the distance from the positive class sample to the center of the positive class is smaller than that from the center of the negative class, the membership degree is 1, and when the distance from the center of the positive class to the center of the negative class is larger than that from the center of the positive class, the membership degree is regarded as a noise point, and the membership degree value is calculated according to a distance function, so that different data are more reasonably divided into corresponding feature vectors.

According to the method for analyzing the principal components of the kernel with optimized membership, a data sample set containing standardized data vectors is projected into a nonlinear separable high-dimensional space, principal component classification representing key risk factors in each index is extracted, unsupervised dimension reduction and data quantity and calculation quantity compression are realized, and a foundation is provided for adaptively mining risk judgment grades of supply chain dangers.

And S104, carrying out clustering calculation screening on the data sample set containing the high-dimensional feature vector to obtain a risk level clustering center, and carrying out neighborhood search on surrounding data of the risk level clustering center to obtain a risk judgment result.

Specifically, on the basis of performing dimension reduction processing on different characteristic values of each index and extracting main component components of key risk factors, an unsupervised clustering analysis is performed on risk distribution and grades of various dangerous matters by using a data-driven density peak clustering method, and a risk grade clustering center is automatically screened according to three basic principles: (1) The local density of the risk clustering centers in the feature space is larger, and the periphery of any one risk clustering center is surrounded by other samples with smaller density and smaller distance; (2) The risk clustering centers have larger distances from other samples with higher densities, namely the distances between the clustering centers with different risk levels are longer; (3) The samples where the outlier noise is present have a smaller local density and are farther apart than the other samples. The specific implementation process is as follows:

s1041, estimating the local density in a nonlinear way.

Describing the risk distribution situation of the dangerous substances based on a density peak clustering method, setting the local density value of the risk level as rho _i, the high-density distance value as delta _i, and the expression is as follows:

Wherein D _ij represents the feature distance between two samples, and D _c represents the discrimination value for dividing the cluster to which the two samples belong, abbreviated as the truncated radius. The local density ρ _i indicates that the distance between the jth sample and the ith sample is smaller than the total number of samples of D _c and is therefore directly affected by the truncation distance D _c, with the following distribution function relationship ρ _i＝f(D_ij,D_c therebetween. As can be seen from data statistics, the local densities of all samples in the high-dimensional feature space conform to a specific Gaussian distribution, so that the local density estimation expression of the dimension-reduction feature vector v _i is obtained by estimating a distribution function by adopting a non-parametric method in the embodiment:

Where w represents a dimensional constant and f represents a non-parametric kernel function. The constraint kernel function expression meets the origin symmetry constraint, and the total value of the uncertain integral of the kernel function is 1, so that a Gaussian estimated kernel function is obtained:

carrying out Gaussian kernel function to obtain a Gaussian kernel density estimation form of local density:

From the above equation, the local density of a certain feature vector is directly affected by the cut-off distance. When the value of D _c is smaller, the closer the feature distance between the two feature vectors v _a and v _b is, the smaller the value of the intermediate variable z is obtained and gradually goes to zero, so that the closer the overall Gaussian distribution curve is to the intermediate symmetry real axis, the larger the obtained kernel function value is, the higher the local density is, and only the samples which are particularly close to each other have larger probability density and influence capability. As D _c increases, the distribution of the local density function tends to be flattened, so that the local density can be affected by samples at different distances, and the contribution degree of samples at different distances to the local density is scattered too far from the kernel estimator, so that the actual situation is not met. Therefore, the ratio of 2% is selected in this embodiment to select the cut-off distance, that is, the largest distance among the smallest 2% distances is D _c. The strategy of determining the cut-off distance by means of a scaling factor reduces the dependency of the parameter on the specific problem to some extent. The local density estimation process is shown in fig. 4, and it can be seen that the local density of the rightmost point is maximum, which has a value of 4; the local density value of the point pointed by the arrow with the rightmost point is the smallest, and the value is 1; the other points tend to be intermediate, with values of 2,3, respectively.

S1042, calculating the minimum value of the high-density distance.

After determining the optimal solution of the cut-off distance, for example, k samples with high local density of a certain sample v _i are selected, and according to the local density sorting of each sample, the samples with higher local density are required to be selected as risk level clustering centers. For sample v _j with higher local density values, the minimum distance between it and the v _i sample is calculated as follows:

δ_i＝W_min

wherein δ _i is the minimum value of the high-density distance. Each sample feature vector may be redefined with a local density value ρ _i and a high density distance minimum δ _i, i.e., v _i(ρ_i,δ_i). And each feature vector which can be used as a cluster center is required to satisfy the following expression:

v_c＝v_i(ρ_i＞ρ_Δ,δ_i＞δ_Δ)

wherein ρ _Δ,δ_Δ represents the division threshold of the two parameters, respectively. If delta _i of some samples is relatively large and its density is less than the threshold, then this indicates that the sample has outlier noise, which needs to be removed, and the noise discrimination is as follows:

v_noise＝v_j(ρ_i＜ρ_Δ,δ_i＞δ_Δ)

S1043, comprehensive evaluation parameter optimization clustering center screening

In a two-dimensional plane constructed by two parameters of a local density value and a high density distance minimum value, the traditional density peak clustering method can be used for dividing different clustering centers by empirically setting a local density threshold ρ _Δ and a distance threshold delta _Δ under the condition of manual observation. However, in the application of evaluating the risk level of the hazard in the actual crop supply chain, excessive human interference brings about evaluation subjectivity, does not have fault tolerance capability, and easily causes the change of the number of the risk level clustering centers and the neighborhood range when the distribution of data samples is different or different observer classification standards are different. Therefore, the embodiment proposes that the comprehensive evaluation parameters optimize the density peak clustering method, and the expression is as follows:

γ_i＝δ_i·ρ_i

It can be seen that when gamma _i is larger, both delta _i and ρ _i may be larger, and the probability that this sample point is the cluster center is larger. However, for some noise points ρ _i, though small, δ _i is large, ultimately resulting in γ _i being large. Thus, the normalization of δ _i and ρ _i is used as follows:

Where u _δ,u_ρ is the variance of delta _i and p _i respectively, For the average of δ _i and ρ _i, i represents the number of samples, after normalizing the two parameters, the two parameters are distributed together, and the new comprehensive evaluation function is shown as follows:

γ_i'＝δ_i'·ρ_i'

The comprehensive evaluation function value of each sample is calculated, and the following formulas are arranged in ascending order:

γ₁'＜…＜γ_i'＜…＜γ_k’

After the cluster center is determined according to the comprehensive evaluation function value of each sample, drawing a gamma _i value plane distribution diagram according to the ascending label serial number of the sample which is not the cluster center, as shown in fig. 5. It is observed that gamma _i of the non-cluster center is smoother, and when the transition from the non-cluster center to the cluster center occurs, a clear jump process occurs, and the value can be detected through the value. And gamma _i presents a straight line in the non-clustered central region, while the distribution in the clustered central region satisfies the curtain law. The search procedure from the super voxel feature vectors F ₁ to F _K of the present embodiment is thus as follows: x=x ₁→...→x_k, i.e. from X ₁ to X _k, after progressive approximation, until a jump occurs at X _i, so a jump threshold is set, and when X _k-x_k-1||≥ε₁ is i, all data after X _k is considered as a cluster center point, so the data is divided into different clusters.

When c samples are cluster centers, other samples in the data set need to be divided into corresponding class clusters according to the local density to which the samples belong. Assuming that the comprehensive evaluation parameters v _i all meet the threshold dividing condition and are the center point of a certain risk level cluster, samples nearby the threshold dividing condition are attracted by local density to form the domain division of the risk level cluster, and the specific expression is as follows:

V_k：{v₁,...,v_k}→{peak₁,...,peak_k}

C_k：near(v₁,...,v_k-m)→peak_k

Wherein k risk level clustering center sets { v ₁,...,v_k } exist, and k class cluster center sets { peak ₁,...,peak_k } are formed independently after analysis by the optimized density peak clustering method, and correspond to k risk level class groups. The rest is to divide other samples around the cluster center and distribute the samples to the corresponding risk level clusters to realize the unsupervised risk level evaluation of the supply chain hazards. The whole risk level clustering center dividing process is as shown in fig. 5, 15 different samples are distributed in a two-dimensional plane constructed by two parameters of a local density value and a high density distance minimum value, an original data space is provided with two risk level clusters A, B, the delta _i of each sample is calculated by calculating the rho _i of each sample point and then sorting, the sample point 6 and the sample point 11 can be seen to have larger delta _i and rho _i at the same time through fig. 6 (b), the cluster centers are respectively cluster A, B, the cluster centers are respectively larger delta _i and rho _i, and the sample points are peak points with higher density than other samples. Although the sample points 14, 15 have a larger minimum high density distance δ _i, they are far from the cluster A, B, defined as noise points, which need to be culled.

Example 2

Corresponding to embodiment 1, this embodiment proposes an unsupervised dimension-reduction density clustering crop supply chain hazard risk determination system, the system comprising a processor configured with processor-executable operating instructions to perform the following operations:

The specific working principle and calculation process of the system can be referred to the content described in embodiment 1, and will not be described herein.

On the basis of the pretreatment of the coding and standardized data, the method uses the Gaussian kernel main formation method to carry out dimension reduction treatment on the original data, extracts key risk factors in the multi-source heterogeneous indexes, and reduces the complexity and the calculation difficulty of the original input data; and then an optimized density peak clustering method is used for carrying out unsupervised clustering analysis, a comprehensive evaluation function is designed to select an optimized clustering center point screening and neighborhood searching process, the interference influence of artificial subjective factors is reduced, the automatic evaluation and high-low distribution ordering of the risk levels of various dangerous matters in a crop supply chain are realized in an unsupervised data driving mode, the problems that the traditional evaluation method is low in accuracy, long in running time, large in labor cost and the like are effectively solved, the risk evaluation accuracy and reliability of government regulatory authorities, enterprise merchants and consumers on the crop supply chain are improved, and the crop quality safety and the production of the supply chain are powerfully ensured. The practical effects of the present application will be described in further detail with reference to examples.

First, data used in experiments using a grain dataset comprising 30 risk indicators, containing 21027 sample numbers, will be described. The purpose of this example is how to achieve an automatic assessment of the risk level of supply chain hazards without human tagging, the risk indicators in the supply chain being shown in table 1.

TABLE 1

It can be seen from table 1 that there are indexes that can be quantized, such as production and consumption, and indexes that cannot be quantized, such as food category and links, among the risk indexes, so that the data are digitized using independent thermal encoding and Embedding encoding. In order to make different attribute indexes have comparability, the model better understand the meaning of data, and eliminate the influence of attribute dimension, linear normalization and normalization are performed on the numerical attribute obtained by preprocessing, and the processed partial data are shown in table 2.

TABLE 2

/>

As can be seen from table 2, the risk indicators of these non-digital quantities of food category, links, etc. in the data after independent thermal encoding and Embedding encoding and normalization have been changed to digital representations, and all the data has been shrunk to the interval 0-1.

The multidimensional heterogeneous data containing 30 risk indexes and 21027 sample numbers are subjected to dimension reduction treatment by using a KPCA method, and after KPCA extraction, the total variance data of the risk index components are shown in table 3, and the risk index component matrix is shown in table 4.

TABLE 3 Table 3

As can be seen from Table 3, the feature values larger than 1 in the initial feature column have only the first 6 components, and the information of the first 6 components accounts for 93.325% of the total information, so that the information of the original variables is basically reserved, and the first 6 components are extracted as main components. The weight of the principal component can be calculated from the weight calculation method of the principal component, which is shown as follows:

Wherein ω _i represents the weight of the i-th principal component, Representing the variance of the ith component, i=1, 2, …,6,/>Representing the cumulative variance of the selected components. The final calculated weights are shown below.

Assuming that the score of the i-th principal component extracted is F _i, a score model of each principal component can be calculated from table 4, and the model is as follows:

F_i＝m₁x₁+m₂x₂+…+m₃₀x₃₀

wherein m ₁,m₂,…,m₃₀ represents the component value of the corresponding risk indicator.

TABLE 4 Table 4

/>

And then, establishing a comprehensive evaluation score mathematical model according to the weight of each main component, wherein the formula is as follows:

where Q represents the composite score, ω _k is the weight of the principal component, and n is the total number of principal components extracted.

According to the risk level list analysis of certain dangers, the risk levels of the dangers in different regional links obtained according to the comprehensive evaluation scores are shown below, and are divided into 8 levels of safety level I, safety level II, early warning level III, low risk level IV, medium risk level V, high risk level VI, high risk level VII and ultra-high risk level VIII by comparing the total scores of all the dangers. As shown in table 5:

TABLE 5

According to the risk level of the dangerous substances, the risk levels of mycotoxins, heavy metals, microorganisms and illegal additives in the grain data in rural areas and cities are counted, as shown in table 6.

It can be seen that the risk level of zearalenone, chromium in a city is lower than in rural areas, and the risk of the rest of the pests is higher than the risk level or leveling of the pests in rural areas, because the city's grain supply chain is relatively more complex. Overall, the risk level of illegal additives is highest, the risk level of microorganisms is lowest, and the risk level of mycotoxins and heavy metals is in the middle.

TABLE 6

/>

And extracting a risk principal component according to a kernel principal component analysis method to perform characteristic dimension reduction characterization, and adopting self-adaptive mining of supply chain hazard risk evaluation from data based on unsupervised density peak clustering. The result is shown in FIG. 7 after performing unsupervised clustering operation by using scikit-learn machine learning library based on Python development environment.

As can be seen by looking at the three-dimensional scatter plot of fig. 8, the categories are well differentiated. The automatic risk level classification of the unsupervised clustering is realized based on the unsupervised density peak clustering, and the automatic risk level classification is respectively a safety level I, a safer level II, an early warning level III, a lower risk level IV, a medium risk level V, a higher risk level VI, a high risk level VII and an ultrahigh risk level VIII, wherein the total number of the risk levels is 8, the risk level of the safety level I is the lowest, the risk level of the ultrahigh risk level VIII is the highest, and the risk level classification is shown in a table 7.

TABLE 7

/>

The safety level I can be 7200, the safety level II is 7025, the early warning level III is 2917, the low risk level IV is 1884, the medium risk level V is 764, the high risk level VI is 482, the high risk level VII is 450, and the ultra-high risk level VIII is 305. The different classes are obviously separated on the whole, two classes have cross overlapping parts, the clustering effect is good, and the automatic division of risk levels is realized by an unsupervised clustering mode. The proportion of dangerous grade proportion is not large in 21027 samples, which indicates that the grain safety risk of China is optimistic from the whole aspect in recent years, and the proportion is mainly related to the reason that the law and regulation related to China are more and more perfect in recent years and the food safety monitoring system is continuous and sound. However, the early warning level still exists, so that the establishment of sound related regulations is required to be continued, and the important supervision of foods in some high-risk areas is carried out, so that the safety of a grain supply chain is ensured.

The present example adopts the contour coefficient to evaluate the quality of the clustering result, and a point p is given, and the contour coefficient of the point is defined as follows:

Where a (p) is the distance between the point p and the other points p in the same cluster and b (p) is the minimum average distance between the point p and a different cluster. a (p) reflects the degree of compactness of data in the cluster to which p belongs, and b (p) reflects the degree of separation of the cluster from other neighboring clusters. Obviously, the larger b (p), the smaller a (p), and the better the corresponding clustering quality, the contour coefficient s (p) is adopted for averaging to measure the quality of the final clustering result. The range of the contour coefficient is [ -1,1], and the more the similar samples are separated from each other and the samples of different types are separated from each other, the higher the score is. The contour coefficient of the cluster of the example is calculated to be 0.56, which indicates that the cluster quality is good.

FIG. 9 shows the ratio of each hazard in the sample of the statistical part, and the residual quantity of aluminum and the content of cadmium are detected in most samples in the statistical sample, which are important subjects for hazard risk detection; fig. 10 shows the hazard class of the sample of the part counted by the clustering method of this example, and it can be seen that the ratio of the aluminum to cadmium content belonging to the safety class (II) is very low, indicating that the aluminum to cadmium content should be detected with emphasis in the supply chain.

In addition, by counting the risk level of each dangerous object, the distribution of risk levels in each link of the supply chain can be analyzed, as shown in fig. 11. It can be seen that the risk level distribution of the hazard varies in different supply chain links. Overall, the safety level I has a higher proportion in each link, while the risk level VIII has a higher proportion in the production link, which means that some hazard risks are easier to generate in the production stage, and the sample with high risk rarely appears in the consumption link, which means that the risk control in the supply chain consumption link is better. According to the result, the links which are easy to have high risk can be controlled in a focused way.

In conclusion, according to the experimental results of the examples, the method for clustering the unsupervised density peaks can realize the self-adaptive classification of the risk of the hazard in the crop supply chain according to various indexes, and implement crop safety early warning.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. An unsupervised dimension-reduction density clustering crop supply chain hazard risk judging method, which is characterized by comprising the following steps:

carrying out neighborhood search on the peripheral data of the risk level clustering center to obtain a risk judgment result;

The process of performing feature dimension reduction on the data sample set containing the standardized data vector to obtain the data sample set containing the high-dimension feature vector comprises the following steps:

Based on the kernel function inner product transformation principle, x _i and x _j are set as data sample points of an input space, linear inseparable standardized data vectors in a data sample set containing the standardized data vectors are projected into a high-dimensional feature space through a nonlinear mapping kernel function phi, and the kernel function is expressed in a Gaussian form, wherein the expression is as follows:

(x_i,x_j)→K(x_i,x_j)＝Φ(x_i)·Φ(x_j)

where x _i,x_j is the sample point of the input space, σ is the gaussian kernel bandwidth, n samples of the input space x _k (k=1, 2,., n), given that each index is subjected to coding and standardized preprocessing, it is known that The principal component covariance matrix C is obtained as follows:

the eigenvalue λ of the solution eigenvalue and the corresponding eigenvector v are as follows:

λ(Φ(x_k)·v)＝Φ(x_k)·Cν

Wherein s _i represents a membership value (0<s _i.ltoreq.1), represents the importance of each sample to the principal component partitioning hyperplane,

Calculating class centers of each data sample from the positive class and the negative class through the membership value, judging membership conditions of the data samples by comparing Euclidean distances between sample points and the positive class and the negative class, wherein membership functions are expressed as follows:

When the distance from the positive class sample to the center of the positive class is smaller than that from the center of the negative class, the membership degree is 1, and when the distance from the center of the positive class to the center of the negative class is larger than that from the center of the positive class, the membership degree is regarded as a noise point, the membership degree value is calculated according to a distance function, and different data are more reasonably divided into corresponding feature vectors.

2. The method of claim 1, wherein vector encoding and standard normalization preprocessing the risk indicator in the set of data samples to obtain the set of data samples containing the normalized data vector comprises:

performing binarization processing on different types of characteristic values of each column of risk indexes in the data sample set;

the data output after binarization processing is subjected to weight embedding to obtain vector features;

Performing linear transformation on the numerical attribute features in the vector features to obtain normalized data;

And carrying out normal distribution standardization processing on the normalized data, and aggregating the numerical attribute characteristics to an approximate normal distribution state with the mean value of 0 and the variance of 1 to obtain a data sample set containing standardized data vectors.

3. The method according to claim 1, wherein the process of performing cluster computation screening on the data sample set containing the high-dimensional feature vector to obtain a risk level cluster center comprises:

performing distance standardization processing on each component of the high-dimensional feature vector in the data sample set containing the high-dimensional feature vector, and calculating the local density and the higher density distance difference value of each component of the high-dimensional feature vector according to density peak clustering to obtain the local density value and the high density distance minimum value of the high-dimensional feature vector;

and constructing a comprehensive evaluation function according to the local density value and the high-density distance minimum value of the high-dimensional feature vector, and obtaining a risk level clustering center according to the comprehensive evaluation function.

4. The method of claim 3, wherein the performing a neighborhood search on the risk level clustering center peripheral data to obtain a risk judgment result includes:

Performing field searching on the peripheral data of the risk level clustering center, and sorting local density values and high-density distance minimum values of the high-dimensional feature vectors around the risk level clustering center to obtain sorting results;

and removing noise points according to the sorting result to obtain a risk judgment result.

5. An unsupervised dimension-reduction density clustered crop supply chain hazard risk determination system, the system comprising a processor configured with processor-executable operating instructions to perform the method of claim 1.

6. The system of claim 5, wherein the processor is configured with processor-executable operating instructions to perform operations comprising:

7. The system of claim 5, wherein the processor is configured with processor-executable operating instructions to perform operations comprising:

8. The system of claim 7, wherein the processor is configured with processor-executable operating instructions to perform operations comprising: