CN113096079A

CN113096079A - Image analysis system and construction method thereof

Info

Publication number: CN113096079A
Application number: CN202110338180.4A
Authority: CN
Inventors: 廖欣
Original assignee: West China Second University Hospital of Sichuan University
Current assignee: West China Second University Hospital of Sichuan University
Priority date: 2021-03-30
Filing date: 2021-03-30
Publication date: 2021-07-09
Anticipated expiration: 2041-03-30
Also published as: CN113096079B

Abstract

The invention provides an image analysis system and a construction method thereof. The system comprises an image database construction unit, a convolutional neural network unit and an analysis unit; the image database construction unit comprises an image data acquisition unit, an image data annotation unit and an image database construction unit; the convolutional neural network unit comprises a convolutional neural network model construction unit and a convolutional neural network model training unit; and the analysis unit analyzes the specific image structure in the image to be analyzed by using the trained image anomaly detection model. The image analysis system is simple in structure, can quickly identify the image and output an analysis result, and improves the judgment accuracy, the working efficiency and the working continuous state.

Description

Image analysis system and construction method thereof

Technical Field

The invention relates to the field of image analysis, in particular to an image analysis system and a construction method thereof.

Background

At present, with the research and progress of artificial intelligence technology, the artificial intelligence technology is being applied to various fields, the artificial intelligence technology is a comprehensive subject and relates to a wide range of fields, and the technology of the existing hardware level and the technology of the software level are also available. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly includes computer vision technology, voice processing technology, natural language processing technology, Machine Learning (Machine Learning)/Deep Learning (Deep Learning), and the like.

Machine learning specializes in studying how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to improve their performance. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. At present, machine learning models in various forms have thoroughly changed many fields of artificial intelligence, and are widely applied to intelligent analysis and recognition of images.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention aims to provide an image analysis system and a construction method thereof.

In order to achieve the above object of the present invention, the present invention provides an image analysis system, comprising an image database construction unit, a convolutional neural network unit, and an analysis unit;

the image database construction unit comprises an image data acquisition unit for acquiring input image data, an image data labeling unit for labeling different image structures in each input image data, and an image database construction unit for classifying and sorting the labeled image data provided by the image data labeling unit;

the convolutional neural network unit comprises a convolutional neural network model construction unit and a convolutional neural network model training unit, and the convolutional neural network model construction unit is used for constructing an image anomaly detection model; the convolutional neural network model training unit trains the image anomaly detection model, inputs the model into a training image set, loss function weights, a feature extraction network and a classification network, and outputs a feature set S comprising the training set, a trained feature extraction network and weights f thereof_θ；

And the analysis unit analyzes the specific image structure in the image to be analyzed by using the trained image anomaly detection model.

The image analysis system disclosed by the invention is simple in structure, can be used for rapidly identifying the image and outputting an analysis result, and can be used for improving the judgment accuracy, the working efficiency and the working continuous state.

In the preferable scheme of the image analysis system, the convolutional neural network construction unit comprises a feature extraction network and a classification network;

the feature extraction network comprises a lower layer formed by M convolution modules BLOCK-A, a higher layer formed by N residual convolution modules BLOCK-B, P residual convolution modules BLOCK-C, a convolution layer connected with the higher layer and a tanh () activation function;

the convolution module BLOCK-A consists of a convolution layer and a LeakyReLU () activation function;

the residual convolution module BLOCK-B is composed of superposed convolution kernels of 1x1 and 3x3 and a skip layer;

the residual convolution module BLOCK-C is composed of a convolution kernel and a layer jump, wherein the convolution kernel is superposed with 1x1, 1x3 and 3x 1;

the classification network consists of a full connectivity layer and a LeakyReLU () activation function.

The feature extraction network has a plurality of convolution kernels with different scales on the same layer, so that sparse and non-sparse features can be learned at the same time, and layer jump ensures that deep and shallow network features can be considered by the network at the same time. The two characteristics in the network structure design increase the feature expression capability of the network.

the feature extraction network comprises a lower layer formed by K convolution modules BLOCK-D, a higher layer formed by Q self-supervision convolution modules BLOCK-F, a convolution layer connected with the higher layer and a tanh () activation function;

the convolution module BLOCK-D consists of a convolution layer and a LeakyReLU () activation function;

the self-supervision convolution module BLOCK-F module comprises a plurality of 1x1 and 3x3 convolution kernels and a mean pooling layer which are mutually overlapped;

The feature extraction network enables the same layer to have convolution kernels with various scales, and can learn sparse and non-sparse features simultaneously, so that the feature expression capability of the network is improved.

the feature extraction network comprises a plurality of convolution layers, and a BLOCK-G module is introduced in the middle of the convolution layers;

the BLOCK-G module comprises a plurality of superimposed 1x1, 3x3, 5x5 convolution kernels and a max pooling layer;

The feature extraction network of the invention enables the same layer to have convolution kernels with various scales, thereby increasing the feature expression capability of the network.

The application also provides a construction method of the image analysis system, which comprises the steps of constructing the image analysis system, obtaining an original image by an image database construction unit, labeling different image structures in the original image, classifying and sorting the labeled image data, and dividing a training set, a check set and a test set; training the image anomaly detection model by adopting a training set to obtain an ideal image anomaly detection model, and verifying the ideal image anomaly detection model by adopting a verification set to detect the accuracy of the ideal image anomaly detection model; testing an ideal image anomaly detection model by adopting a test set to detect the robustness of the ideal image anomaly detection model;

if the difference between the accuracy of the image anomaly detection model on the test set and the accuracy in the training of the check set exceeds a preset value, the model is over-fitted, the model returns to the convolutional neural network training unit, the network structure or the parameters are adjusted to perform retraining again, so that the image anomaly detection model with the difference between the accuracy on the test set and the accuracy in the training of the check set within the preset value is obtained, and at the moment, the robustness of the image anomaly detection model is high.

According to the system construction method, due to the introduction of the check set and the test set, the phenomena of under-fitting and over-fitting of the ideal weight of the abnormal detection model can be avoided, and the robustness of the ideal weight of the abnormal detection model obtained through training is guaranteed.

The preferable scheme of the image analysis system construction method is that the training flow of the training set to the image anomaly detection model is as follows:

b1: training a feature extraction network of an image anomaly detection model by using images in a training set;

b2: using the trained feature extraction network to obtain and store a feature set S, S ← S { (f) } corresponding to the training set_θ(p) }, i.e.: randomly extracting an image block aiming at each image in the training set, wherein the image block has the same receiving field as the feature extraction network, and acquiring a feature vector f of the image block through the feature extraction network obtained by training_θ(p) the feature vectors as a whole constitute a feature vector set S;

b3: saving trained feature extraction network weights f_θAnd a feature vector set S corresponding to the training set.

The method and the device obtain the ideal weight after the training of the anomaly detection model is finished, and obtain the image characteristic set of the specified image structure according to the ideal weight and the images of the training set, so that the method and the device are used for realizing the anomaly detection of subsequent test images, can quickly identify the images and output the analysis result, and improve the judgment accuracy.

In a preferred embodiment of the image analysis system construction method, the step B1 includes the following steps:

b11: for each image in the training set, randomly selecting an image block p in an eight-neighborhood of a 3x3 grid of the image, wherein the scale of the image block p is the same as the receiving field of the feature extraction network, then randomly dithering the center of the image block p to obtain an image block p1, calculating the cross entropy of the image block p and the image block p1 as a sub-term Loss function Loss _1,

wherein the image block p₁Relative to pThe true relative positions are y {0,1, …,7}, y_iReferring to the number of image blocks of the 8 relative positions of the category i in y {0,1, …,7} in the training set; classifier C_φIs trained to correctly predict image block p₁Relative to the image block p, i.e. y ═ C_φ(f_θ(p),f_θ(p₁))，a_iThe confidence of the category i calculated by the classifier, and N is the total number of samples in the training set;

for an image block p, randomly selecting an image block p which is in the same row or column but not adjacent to the image block p in the four neighborhoods of the 5 × 5 grid₂,p₂The scale is the same as the acceptance field of the feature extraction network, the cross entropy of the image blocks p and p2 is calculated as a sub-term Loss function Loss _2,

wherein the image block p₂The true relative position with respect to p is y {0,1,2,3}, y_iReferring to the number of image blocks of the 4 relative positions of the category i in the training set, namely y {0,1,2,3 }; classifier C_φIs trained to correctly predict image block p₂Relative to the image block p, i.e. y ═ C_φ(f_θ(p),f_θ(p₂))，b_iIs the confidence of class i calculated by the classifier;

aiming at an image block p, acquiring 2-4 image blocks p3, p4, p5 and p6 of four adjacent intersection areas of p, calculating L2 norm distances between p and selected image blocks p3, p4, p5 and p6, and averaging the distances to be used as a subitem Loss function Loss _3,

||f_θ(p)-f_θ(p_2+i)||₂the L2 norm distance between the image block p and a selected image block in p3, p4, p5 and p 6;

b12: calculating Loss function Loss of network model as lambda₁*Loss_1+λ₂*Loss_2+Loss_3，λ₁、λ₂The weight value in the loss function is larger than 0, and the Adam optimizer is utilized to carry out back propagation so as to realize the network weight of the feature extraction network modelRepeating iteration and optimizing;

b13: and C, repeatedly executing the steps B11-B12 until the number of turns is specified, and selecting and storing the optimal weight of the feature extraction network and the classification network according to the loss function of each turn of training. In a preferred embodiment of the image analysis system construction method, the step B1 includes the following steps:

step 1: for each image in the training set, randomly selecting an image block p7 in the eight neighborhood of the 3x3 grid of the image block p, calculating the cross entropy of the image blocks p and p7 as a sub-item Loss function Loss _4,

wherein the image block p₇The true relative position with respect to p is y {0,1, …,7}, y_iReferring to the number of image blocks of the 8 relative positions of the category i in y {0,1, …,7} in the training set; classifier C_φIs trained to correctly predict image block p₇Relative to the image block p, i.e.

c_iIs the probability value of the category i calculated by the classifier, and N is the total number of samples in the training set;

for an image block p, randomly taking an image block p8 which is in the same row or column but not adjacent to the image block p in the four neighborhoods of the 5X5 network, calculating the cross entropy of p and p8 as a sub-term Loss function Loss _5,

wherein the image block p₈The true relative position with respect to p is y {0,1,2,3}, y_iReferring to the number of image blocks of the 4 relative positions of the category i in the training set, namely y {0,1,2,3 }; classifier C_φIs trained to correctly predict image block p₈Relative to the image block p, i.e.

d_iIs the probability value of the category i calculated by the classifier;

step 2: calculating a Loss function Loss of the network model, wherein lambda is a weight value in the Loss function and is larger than 0, and performing back propagation by using an Adam optimizer to realize network weight iteration and optimization of the feature extraction network model;

and step 3: and (3) repeatedly executing the steps 1-2 to designate the number of rounds, and selecting and storing the optimal weights of the feature extraction network and the classification network according to the loss function of each round of training.

The training process of the invention solves the problems that the number of the feature centers of the complex image is uncertain, and the workload of distributing corresponding image blocks to different feature centers is extremely large. Because the image blocks randomly selected directly from the training images have larger intra-class variance variation, part of the image blocks correspond to the background and part of the image blocks contain the target, and the situation that the image blocks contain both the background and the target can also exist. Therefore, mapping all features of different image blocks to one center, performing unimodal clustering, will weaken the link between features and content. In order to solve the problem, the scheme does not clearly define a center and divide corresponding image blocks, on the contrary, spatial adjacent image blocks are sampled to obtain image blocks with similar semantics, then a feature extraction network is trained to automatically collect the image blocks with similar features and semantics, and when the trained feature extraction network can well solve the pre-task, the network is considered to be capable of extracting effective features.

The system construction method can select the optimal training result from the results of the appointed training rounds of the anomaly detection model according to the loss function, and obtain the ideal weight of the anomaly detection model. Due to the design of the deep anomaly detection network structure and the self-supervision learning technology introduced in the construction of the loss function, the method can complete model training under the condition of a small sample data set, and further realize the analysis work of the target image structure.

The invention has the beneficial effects that: the invention has the advantages of high accuracy, short time consumption, long work duration and wide application range, can be widely applied to the fields of medical treatment, traffic safety and the like, particularly in the medical treatment field, is beneficial to solving the problem of uneven medical resource distribution, can realize remote high-quality medical treatment and the like, and provides more convenient and more accurate pathological diagnosis service for patients.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic diagram of the structure of an image analysis system;

FIG. 2 is a schematic diagram of a first convolutional neural network construction unit network structure;

FIG. 3 is a schematic diagram of a convolution module BLOCK-A network structure;

FIG. 4 is a schematic diagram of a convolution module BLOCK-B network structure;

FIG. 5 is a schematic diagram of a convolution module BLOCK-C network structure;

FIG. 6 is a schematic diagram of a second convolutional neural network construction unit network structure;

FIG. 7 is a schematic diagram of a convolution module BLOCK-D network structure;

FIG. 8 is a schematic diagram of a convolution module BLOCK-F network structure;

FIG. 9 is a schematic diagram of a third convolutional neural network construction unit network structure;

FIG. 10 is a schematic diagram of a convolution module BLOCK-G network structure;

FIG. 11 is a schematic diagram of eight neighborhoods of a 3 × 3 grid of image blocks p;

FIG. 12 is a diagram of four neighborhood domains of a 5 × 5 grid of image blocks p;

fig. 13 is a schematic diagram of four adjacent intersection areas of an image block p.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.

As shown in fig. 1, the present invention provides an image analysis system, which includes an image database construction unit, a convolutional neural network unit, and an analysis unit.

The image database construction unit comprises an image data acquisition unit, an image data annotation unit and an image database construction unit. The image data acquisition unit is used for acquiring input image data, the image data labeling unit is used for labeling different image structures in each input image data, and the image database construction unit is used for classifying and sorting the labeled image data provided by the image data labeling unit, dividing a training set, a check set and a test set and constructing an image database.

The convolutional neural network unit comprises a convolutional neural network model construction unit and a convolutional neural network model training unit. The convolutional neural network model construction unit is used for constructing an image anomaly detection model; the convolutional neural network model training unit trains the image anomaly detection model to obtain an ideal image anomaly detection model, the input of the convolutional neural network model training unit comprises a training image set, a loss function weight parameter, a feature extraction network and a classification network, and the output of the convolutional neural network model training unit comprises a feature set S of the training set, the trained feature extraction network and a weight f thereof_θ；

In addition, the input terminal is used for inputting the existing image into the image data obtaining unit, and the input data is finally classified and collected by the image database construction unit for supporting the subsequent image analysis work. The output terminal is used for presenting the analysis result (specific image structure and corresponding area ratio) of the robust and ideal feature extraction network model obtained by the convolutional neural network model training unit to the input image to a doctor as a clinical diagnosis reference so as to improve the accuracy, the working efficiency and the working continuous state of the working personnel.

In this embodiment, the convolutional neural network constructing unit includes a feature extraction network and a classification network, and the feature extraction network extracts feature information of the image block, so that the subsequent classification network can correctly predict the relative position of the image block. In this embodiment, the classification network is composed of a fully connected layer and a LeakyReLU () activation function, and once training is completed, the classification network is discarded.

Feature extraction network this embodiment provides three models:

first, as shown in fig. 2, the feature extraction network is constructed by using the idea of modular concatenation, so that the width and depth of the network can be expanded as required, the lower layer of the feature extraction network is constructed by M convolution modules BLOCK-a, the upper layer of the feature extraction network is constructed by introducing N residual convolution modules BLOCK-B and P residual convolution modules BLOCK-C, and then a convolution layer and a tanh () activation function are connected with the upper layer. M is configurable, the value range is an integer between 3 and 6, the default value is 4, N is configurable, the value range is an integer between 1 and 3, and the default value is 2; p can be configured, the value range is a positive integer between 1 and 3, and the default value is 2.

The convolution module BLOCK-a is composed of a convolution layer and a LeakyReLU () activation function, as shown in FIG. 3; the residual convolution module BLOCK-B is formed by superimposing convolution kernels of 1x1 and 3x3 and layer skipping, as shown in fig. 4; the residual convolution module BLOCK-C is constructed by superimposing convolution kernels of 1x1, 1x3, 3x1 and layer jumps, as shown in fig. 5. Because the same layer has convolution kernels with different scales, sparse and non-sparse characteristics can be learned at the same time, and the jump layer (shortcuts) ensures that the network can consider deep and shallow network characteristics at the same time. The two characteristics in the network structure design increase the feature expression capability of the network.

Secondly, as shown in fig. 6, the feature extraction network is formed by using a modular splicing idea, so that the width and the depth of the network can be amplified as required, the lower layer of the feature extraction network is formed by K convolution modules BLOCK-D, the upper layer of the feature extraction network is introduced with Q self-supervision convolution modules BLOCK-F, and a convolution layer and a tanh () activation function are connected with the upper layer; k can be configured, the value range is an integer between 4 and 6, the default value is 5, Q can be configured, the value range is an integer between 1 and 3, and the default value is 1.

The convolution module BLOCK-D is composed of a convolution layer and a LeakyReLU () activation function, as shown in FIG. 7; the self-supervision convolution module BLOCK-F enables the same layer to have convolution kernels of various scales through a plurality of convolution kernels of 1x1 and 3x3 and a mean pooling layer which are mutually overlapped, so that sparse and non-sparse features can be simultaneously learned, and the feature expression capability of the network is increased, as shown in fig. 8.

Third, as shown in fig. 9, the feature extraction network includes a plurality of convolutional layers, and a BLOCK-G module is introduced in the middle of the plurality of convolutional layers. The feature extraction network extracts feature information of the image blocks so that a subsequent classification network can correctly predict the relative positions of the image blocks.

The BLOCK-G module includes convolution kernels of 1x1, 3x3, 5x5 and a maximum pooling layer, so that the same layer has convolution kernels of various scales, and the feature expression capability of the network is increased, as shown in fig. 10.

The convolution modules referred to in the present application all use conventional convolution layer modules, unless otherwise specified.

In order to improve the accuracy and robustness of the image anomaly detection model, the convolutional neural network unit further comprises a convolutional neural network model inspection unit; the convolutional neural network model checking unit comprises a model checking unit and a model testing unit, and the model checking unit is used for detecting the accuracy of the convolutional network model obtained by training; the model testing unit is used for detecting whether the convolutional network model obtained through training is over-fitted or not so as to screen out the network model with high robustness.

In this embodiment, when the analysis unit analyzes the specific image structure in the image to be analyzed by using the trained image anomaly detection model, the analysis unit may analyze the specific image structure by using the existing method, or may analyze the specific image structure by using the following method.

Image to be analyzed I_testAnd performing sliding window blocking, and dividing image blocks with the same size as the receiving field of the feature extraction network according to sliding step S pixels to obtain an image block sequence, wherein the size of the blocked image is W multiplied by W pixels, and S is more than or equal to 1 and less than or equal to W.

Carrying out self-adaptive segmentation on the image block subjected to sliding window partitioning, and distinguishing a target and a blank background in the image block; namely segmenting the foreground and the background, discarding the image blocks with the target ratio smaller than a threshold T1, and not performing subsequent processing; reserving image BLOCKs with a ratio greater than a threshold T1 to form a sequence of image BLOCKs { BLOCK }_i,jAnd j and i are the counts of the image blocks on x and y coordinates respectively, and the counts and the y coordinates together form the number of the image blocks.

Image BLOCK sequence { BLOCK_i,jInputting image blocks in the image block extraction network to obtain an image I_testThe abnormal feature map M of (1). The method specifically comprises the following steps: image BLOCK sequence { BLOCK_i,jCalculating the abnormal value score abnormal thereof through a feature extraction network_i,jAnd image BLOCK BLOCK_i,jAbnormal value score of abnormal_i,jAs an initial outlier score, abnormal, for each pixel in the image block_i,j＝min_h∈S||f(p)-h||₂Wherein the image BLOCK sequence { BLOCK_i,jThe feature vector acquired by the input feature extraction network is f (BLOCK)_i,j) H is any one of the feature vectors in the feature vector set S, | · | | non-calculation₂Represents the L2 norm distance, then min_h∈S||f(p)-h||₂Then represents the image BLOCK BLOCK_i,jIs spaced from the minimum L2 norm of any feature vector in the set S of feature vectors.

Then calculating the image I to be analyzed_testAn abnormal feature map M after the network is extracted through the features:

calculating an image I to be analyzed_testThe abnormal score value P of each pixel in the image_i，j，

Image to be analyzed I_testAbnormal score value p of all pixels in_i,jThe corresponding abnormal feature maps M, M, N are constructed to refer to the total number of image blocks in the x and y directions, respectively.

Performing threshold segmentation on the abnormal feature map M, and calculating the structural type of a specific image in the image I to be analyzed by using the segmented binary image_testArea percentage of (1). The method specifically comprises the following steps:

performing threshold segmentation on the abnormal feature map M according to a threshold T2, and calculating the area percentage of a specific image structure

Wherein, AREA_GCTIs an image I to be analyzed_testArea of region in (1), corresponding to sequence { BLOCK_i,jSum of foreground AREA of each image block in }, AREA_STRUCTIs an image I to be analyzed_testThe area of the specific image structure in the image is obtained by subtracting the sequence { BLOCK } from the foreground area after the threshold segmentation of the corresponding abnormal feature map M_i,jThe sum of the background areas of each image block in the pixel.

The application also provides a method for constructing the image analysis system, which specifically comprises the following steps: constructing the image analysis system, acquiring an original image by an image database construction unit, labeling different image structures in the original image, classifying and sorting the labeled image data, and dividing a training set, a check set and a test set; training the image anomaly detection model by adopting a training set to obtain an ideal image anomaly detection model, and verifying the ideal image anomaly detection model by adopting a verification set to detect the accuracy of the ideal image anomaly detection model; and testing the ideal image anomaly detection model by adopting a test set to detect the robustness of the model.

The training flow of the training set to the image anomaly detection model is as follows:

b1: and training a feature extraction network of the image anomaly detection model by adopting the images in the training set.

In this embodiment, two training procedures are provided as follows:

the first method comprises the following steps:

b11: for each image in the training set, as shown in fig. 11, an image block p is arbitrarily selected from eight neighborhoods of a 3 × 3 grid, the scale of p is the same as the receiving field of the feature extraction network, then random dithering is performed on the center of the image block p to obtain an image block p1, the cross entropy of the image block p and p1 is calculated to be used as a sub-item Loss function Loss _1,

wherein the image block p₁The true relative position with respect to p is y {0,1, …,7}, y_iReferring to the number of image blocks of the 8 relative positions of the category i in y {0,1, …,7} in the training set; classifier C_φIs trained to correctly predict image block p₁Relative to the image block p, i.e. y ═ C_φ(f_θ(p),f_θ(p₁) Representation of an image block p predicted by a classifier₁Relative position to image block p, a_iIs the confidence of class i calculated by the classifier, and N is the total number of samples in the training set.

For the image block p, as shown in fig. 12, in the four neighborhoods of the 5 × 5 grid, an image block p in the same row or column but not adjacent to the image block p is randomly selected₂,p₂The scale is the same as the acceptance field of the feature extraction network, the cross entropy of the image blocks p and p2 is calculated as a sub-term Loss function Loss _2,

wherein the image block p₂The true relative position with respect to p is y {0,1,2,3}, y_iReferring to the number of image blocks of the 4 relative positions of the category i in the training set, namely y {0,1,2,3 }; classifier C_φIs trained to correctly predict image block p₂And imagesRelative position of block p, i.e. y ═ C_φ(f_θ(p),f_θ(p₂) Representation of an image block p predicted by a classifier₂Relative position to image block p, b_iIs the confidence level for class i calculated by the classifier.

For an image block p, as shown in fig. 13, for four adjacent intersection areas of p, 2 to 4 of the image blocks p3, p4, p5, and p6 are obtained, that is, for four edge points (upper left corner, upper right corner, lower left corner, and lower right corner) of the image block p as new image block centers, four new image blocks p3, p4, p5, and p6 with the same scale as the image block p can be obtained, L2 norm distances of selected image blocks among p and p3, p4, p5, and p6 are calculated, and averaged to be used as a sub-term Loss function Loss _3,

||f_θ(p)-f_θ(p_2+i)||₂the L2 norm distance between the image block p and the selected image block in p3, p4, p5 and p6 is pointed out.

B12: calculating Loss function Loss of network model as lambda₁*Loss_1+λ₂*Loss_2+Loss_3，λ₁、λ₂And performing back propagation by using an Adam optimizer to realize network weight iteration and optimization of the feature extraction network model, wherein the weight values in the loss function are all larger than 0.

B13: and C, repeatedly executing the steps B11-B12 until the number of turns is specified, and selecting and storing the optimal weight of the feature extraction network and the classification network according to the loss function of each turn of training.

And the second method comprises the following steps:

wherein the image block p₇The true relative position with respect to p is y {0,1, …,7}, y_iIn the training set, the class i is relative to the class i in the 8 pairs of y {0,1, …,7}The number of image blocks of a location; classifier C_φIs trained to correctly predict image block p₇Relative to the image block p, i.e.

Representing a predicted image block p by a classifier₇Relative position to image block p, c_iIs the confidence of class i calculated by the classifier, and N is the total number of samples in the training set.

Randomly taking an image block p8 which is in the same row or column but is not adjacent to the image block p at intervals of four neighborhoods of a 5X5 network aiming at the image block p, calculating the cross entropy of p and p8 as a sub-term Loss function Loss _5,

wherein the image block p₈The true relative position with respect to p is y {0,1,2,3}, y_iReferring to the number of image blocks of the 4 relative positions of the category i in the training set, namely y {0,1,2,3 }; classifier C_φIs trained to correctly predict image block p₈With respect to the relative position of the image block p,

representing a predicted image block p by a classifier₈Relative position to image block p, d_iIs the confidence level for class i calculated by the classifier.

Step 2: and calculating a Loss function Loss of the network model, wherein the Loss is lambda Loss _4+ Loss _5, the lambda is a weight value in the Loss function and is larger than 0, and performing back propagation by using an Adam optimizer to realize network weight iteration and optimization of the feature extraction network model.

It should be noted that if the number of image data in the training set is too small, the minimum number of network layers can be set in a configurable range for the network structure, so as to avoid overfitting when the training data is insufficient; if training is carried out on the conventional convolutional neural network model, the model accuracy cannot be increased, more network layers can be set in a configurable range according to the network structure, namely, the fitting capability of the model is improved by increasing the depth of the convolutional model.

Then the next step is performed:

b2: using the trained feature extraction network to obtain and store a feature set S, S ← S { (f) } corresponding to the training set_θ(p) }, i.e.: randomly extracting an image block aiming at each image in the training set, wherein the image block has the same receiving field as the feature extraction network, and acquiring a feature vector f of the image block through the feature extraction network obtained by training_θ(p) the feature vectors as a whole constitute a feature vector set S.

B3: saving trained feature extraction network weights f_θAnd training a feature set S corresponding to the image set to finish the training of the image anomaly detection model.

After the image anomaly detection model is trained, an ideal image anomaly detection model is obtained, and the ideal image anomaly detection model can be verified by adopting a verification set to detect the accuracy of the ideal image anomaly detection model; testing an ideal image anomaly detection model by adopting a test set to detect the robustness of the ideal image anomaly detection model; if the difference between the accuracy of the image anomaly detection model on the test set and the accuracy in the training of the check set exceeds a preset value, the model is over-fitted, the network structure or the parameters are adjusted to train again to obtain the image anomaly detection model with the difference between the accuracy on the test set and the accuracy in the training of the check set within the preset value, and at the moment, the robustness of the image anomaly detection model is high. The accuracy here is: the method comprises the steps of conducting sliding window blocking on images in a check set or a test set according to step length pixels to obtain an image block sequence (the image block sequence is the same as a receiving field of a feature extraction network), then obtaining an abnormal feature map of the images through the feature extraction network, conducting threshold segmentation on the abnormal feature map, calculating the area percentage of a specific image structure type in the images by utilizing the segmented binary images, and comparing the percentage with an artificial labeling result to obtain accuracy. And if the accuracy of the images in the verification set is within the acceptable range, the image anomaly detection model obtained by training is considered as an ideal model.

The image analysis system and the construction method can be used for analyzing the pathological image, but not limited to analyzing the pathological image, so as to obtain the following image structure of tumor cells in the pathological image: the intelligent, efficient and quantitative analysis of the follicular structure, the island structure, the funicular structure, the ribbon structure, the diffuse structure and the like is taken as an example, 8 pathological doctors with diagnosis experience of ovarian granulosa cytoma for more than 5 years are selected, 30 pathological section images of the ovarian granulosa cytoma are provided for each person respectively, the tumor cell image structure in the pathological section images is analyzed, then the accuracy and the average time are calculated, the diagnosis state of the doctors is counted, and compared with the scheme provided by the invention, the results are shown in the following table 1.

TABLE 1 comparison of the results of the image analysis of ovarian granulosa cell tumors in pathological sections

As can be seen from table 1, the accuracy of the analysis of the tumor cell image structure (follicular structure, island structure, funicular structure, ribbon structure, diffuse structure, etc.) in the pathological section by using the scheme provided by the present invention is higher than that of a professional pathologist, and a quantitative conclusion can be obtained (the pathologist can only obtain a subjective qualitative or semi-quantitative conclusion by visual analysis). In addition, the method of the invention has shorter analysis time and long work duration.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An image analysis system is characterized by comprising an image database construction unit, a convolutional neural network unit and an analysis unit;

2. The image analysis system according to claim 1, wherein the convolutional neural network construction unit includes a feature extraction network and a classification network;

3. The image analysis system according to claim 1, wherein the convolutional neural network construction unit includes a feature extraction network and a classification network;

4. The image analysis system according to claim 1, wherein the convolutional neural network construction unit includes a feature extraction network and a classification network;

5. The image analysis system of claim 1, wherein the convolutional neural network unit further comprises a convolutional neural network model verification unit;

the convolutional neural network model checking unit comprises a model checking unit and a model testing unit, and the model checking unit is used for detecting the accuracy of the convolutional network model obtained by training; the model testing unit is used for detecting whether the convolutional network model obtained through training is over-fitted or not so as to screen out the network model with high robustness.

6. An image analysis system construction method is characterized in that the image analysis system according to any one of claims 1 to 5 is constructed, an image database construction unit obtains an original image and labels different image structures in the original image, the labeled image data is classified and sorted, and a training set, a check set and a test set are divided; training the image anomaly detection model by adopting a training set to obtain an ideal image anomaly detection model, and verifying the ideal image anomaly detection model by adopting a verification set to detect the accuracy of the ideal image anomaly detection model; testing an ideal image anomaly detection model by adopting a test set to detect the robustness of the ideal image anomaly detection model;

7. The image analysis system construction method according to claim 6, wherein the training flow of the training set to the image anomaly detection model is as follows:

b2: using the trained feature extraction network to obtain and store a feature set S, S ← S { (f) } corresponding to the training set_θ(p) }, i.e.: randomly extracting an image block for each image in the training set, wherein the image block is the same as the receiving field of the feature extraction network, and acquiring the image block by the feature extraction network obtained by trainingThe feature vector f of the image block_θ(p) the feature vectors as a whole constitute a feature vector set S;

8. The image analysis system construction method according to claim 7, wherein the step B1 includes the steps of:

wherein the image block p₁The true relative position with respect to p is y {0,1, …,7}, y_iReferring to the number of image blocks of the 8 relative positions of the category i in y {0,1, …,7} in the training set; classifier C_φIs trained to correctly predict image block p₁Relative to the image block p, i.e. y ═ C_φ(f_θ(p),f_θ(p₁))，a_iThe confidence of the category i calculated by the classifier, and N is the total number of samples in the training set;

wherein the image block p₂The true relative position with respect to p is y {0,1,2,3}, y_iReferring to the number of image blocks of the 4 relative positions of the category i in the training set, namely y {0,1,2,3 }; classifier C_φIs trained to correctly predict image block p₂Phase with image block pTo position, i.e. y ═ C_φ(f_θ(p),f_θ(p₂))，b_iIs the confidence of class i calculated by the classifier;

b12: calculating Loss function Loss of network model as lambda₁*Loss_1+λ₂*Loss_2+Loss_3，λ₁、λ₂The weighted value in the loss function is larger than 0, and backward propagation is carried out by using an Adam optimizer, so that network weight iteration and optimization of the feature extraction network model are realized;

9. The image analysis system construction method according to claim 7, wherein the step B1 includes the steps of:

wherein the image block p₇The true relative position with respect to p is y {0,1, …,7}, y_iReferring to the number of image blocks of the 8 relative positions of the category i in y {0,1, …,7} in the training set; classifier C_φIs trained to correctly predict image block p₇Relative bit to image block pPut into, i.e.

d_iIs the probability value of the category i calculated by the classifier;