CN112000827A - Hardware image retrieval method and system based on deep learning - Google Patents
Hardware image retrieval method and system based on deep learning Download PDFInfo
- Publication number
- CN112000827A CN112000827A CN202010876226.3A CN202010876226A CN112000827A CN 112000827 A CN112000827 A CN 112000827A CN 202010876226 A CN202010876226 A CN 202010876226A CN 112000827 A CN112000827 A CN 112000827A
- Authority
- CN
- China
- Prior art keywords
- layer
- image
- hardware image
- hardware
- category label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013135 deep learning Methods 0.000 title claims abstract description 26
- 238000013528 artificial neural network Methods 0.000 claims abstract description 61
- 238000007781 pre-processing Methods 0.000 claims abstract description 30
- 238000010606 normalization Methods 0.000 claims description 27
- 238000006243 chemical reaction Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 18
- 230000004913 activation Effects 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 229910052751 metal Inorganic materials 0.000 claims 1
- 239000002184 metal Substances 0.000 claims 1
- 150000002739 metals Chemical class 0.000 claims 1
- 238000013145 classification model Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 1
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Library & Information Science (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a hardware image retrieval method and system based on deep learning. The method comprises the steps of obtaining a hardware image with a category label; carrying out data preprocessing on the hardware image with the category label; constructing a neural network according to the preprocessed hardware image with the category label; and searching the hardware image to be searched by utilizing the trained neural network. The hardware image retrieval method and system based on deep learning provided by the invention improve the speed and accuracy of hardware image retrieval.
Description
Technical Field
The invention relates to the field of image retrieval, in particular to a hardware image retrieval method and system based on deep learning.
Background
At present, the industrial image retrieval application mainly comprises two modes. Firstly, the image features are extracted by a deep ranking model, and then the similarity is compared by calculating the distance between the two image features, the main technical problem of the method is that the triplets data required by the training model are very difficult to acquire, and some acquisition methods need additional data, such as [ the click quantity combined with the images ]. The second approach is adopted by the paper Deep Learning of Binary Hash Codes for Fast Image Retrieval (hereinafter referred to as Binary Hash Codes) and the paper Visual Search at eBay (hereinafter referred to as eBay). Features (including features of a full connection layer and Binary Hash code features proposed in the Binary Hash Codes paper) are extracted through a common image classification model, and similarity is compared by calculating the distance between the two image features. The main problem with this approach is that the accuracy of image searching is generally not as good as the first approach, but this approach has two major advantages. Firstly, a great deal of energy is not needed to be spent on collecting triplets data, secondly, the model training time is shorter, the requirement on the performance of the server is lower, and therefore the method is suitable for small and medium-sized data. However, the method proposed by the binary hash codes paper is not suitable for large and medium-sized data volumes (hundreds of thousands of images), because coarse-grained retrieval (retrieval based on binary hash codes) is to search the whole image database, and a lot of time is wasted in searching irrelevant image categories; the method proposed in the ebay paper is not suitable for small and medium size data (less than 5 ten thousand graphs), and the final search accuracy is not high due to no reordering for fine-grained retrieval (retrieval based on full-connected layer fc 6).
Disclosure of Invention
The invention aims to provide a hardware image retrieval method and a hardware image retrieval system based on deep learning, which are used for improving the speed and the accuracy of hardware image retrieval.
In order to achieve the purpose, the invention provides the following scheme:
a hardware image retrieval method based on deep learning comprises the following steps:
acquiring a hardware image with a category label;
carrying out data preprocessing on the hardware image with the category label; the pretreatment comprises the following steps: adjusting the size of an image, converting the type of data, normalizing, standardizing and enhancing the image;
constructing a neural network according to the preprocessed hardware image with the category label; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes a hardware image as input and takes a category label of the hardware image as output; the output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval;
and searching the hardware image to be searched by utilizing the trained neural network.
Optionally, the data preprocessing is performed on the hardware image with the category label, and specifically includes:
adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment;
converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion;
carrying out normalization processing on the hardware image with the class label after the data type conversion to obtain the hardware image with the class label after the normalization processing;
standardizing the hardware image with the category label after the normalization processing to obtain the hardware image with the category label after the standardization processing;
and carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the pretreatment.
Optionally, the method of constructing a neural network according to the preprocessed hardware image with the category label further includes:
and training the neural network to obtain the trained neural network.
Optionally, the training is utilized to search the hardware image to be searched by the neural network, and the method specifically includes:
carrying out data preprocessing on the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved;
determining a plurality of category labels (the first m labels with the maximum probability), binary retrieval vectors and features of an fc6 layer of the preprocessed hardware image to be retrieved according to the first layer network and the second layer network of the neural network;
performing coarse-grained search according to the category labels and the binary retrieval vector to obtain a plurality of images with the difference smaller than a set threshold value with the preprocessed hardware image to be retrieved; and performing fine-grained search by using the characteristics of the fc6 layer.
A hardware image retrieval system based on deep learning comprises:
the hardware image acquisition module is used for acquiring hardware images with category labels;
the preprocessing module is used for preprocessing the data of the hardware image with the category label; the pretreatment comprises the following steps: adjusting the size of an image, converting the type of data, normalizing, standardizing and enhancing the image;
the neural network construction module is used for constructing a neural network according to the preprocessed hardware images with the class labels; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes a hardware image as input and takes a category label of the hardware image as output; the output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval;
and the retrieval module is used for retrieving the hardware image to be retrieved by utilizing the neural network.
Optionally, the preprocessing module specifically includes:
the image size adjusting unit is used for adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment;
the data type conversion unit is used for converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion;
the normalization processing unit is used for performing normalization processing on the hardware image with the category label after the data type conversion to obtain the hardware image with the category label after the normalization processing;
the standardized processing unit is used for carrying out standardized processing on the hardware image with the classified label after the standardized processing to obtain the hardware image with the classified label after the standardized processing;
and the image enhancement processing unit is used for carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the preprocessing.
Optionally, the method further includes:
and the trained neural network determining module is used for training the neural network to obtain the trained neural network.
Optionally, the retrieving module specifically includes:
the preprocessing unit is used for preprocessing the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved;
the characteristic determining unit is used for determining a plurality of categories, binary retrieval vectors and fc 6-layer characteristics of the preprocessed hardware images to be retrieved according to the trained neural network;
the retrieval unit is used for performing coarse-grained search according to the plurality of category labels and the binary retrieval vector to obtain a plurality of images, the difference between the images and the preprocessed hardware image to be retrieved is smaller than a set threshold value; and performing fine-grained search by using the characteristics of the fc6 layer.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the hardware image retrieval method and system based on deep learning, the preprocessed hardware images with the class labels are used for constructing the neural network, the data for constructing the neural network is simple, the neural network can be used for retrieving a large number of images, and the retrieval accuracy is improved on the basis of improving the retrieval speed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of a hardware image retrieval method based on deep learning according to the present invention;
fig. 2 is a schematic structural diagram of a hardware image retrieval system based on deep learning according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a hardware image retrieval method and a hardware image retrieval system based on deep learning, which are used for improving the speed and the accuracy of hardware image retrieval.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a hardware image retrieval method based on deep learning, as shown in fig. 1, the hardware image retrieval method based on deep learning provided by the present invention includes:
and S101, acquiring the hardware image with the category label.
S102, carrying out data preprocessing on the hardware image with the category label; the pretreatment comprises the following steps: image resizing, data type conversion, normalization, and image enhancement.
S102 specifically comprises the following steps:
and adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment. The resized labeled hardware image is typically 244 x 244.
And converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion. The hardware image with the category label after data type conversion is an array of (n, 244, 244, 3), where n is the number of images, 244 is the size of the image, and 3 is the RGB value.
And carrying out normalization processing on the hardware image with the category label after the data type conversion to obtain the hardware image with the category label after the normalization processing. I.e. to pixel point values xiCarry out xiAnd carrying out treatment of/255.0 to convert the value range of the pixel value from 0-255 to 0-1.
And carrying out standardization processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the standardization processing. I.e. for pixel point value xiIs carried out (x)iMean)/std treatment. Wherein mean and std are each XtrainAll pixelsMean and standard deviation of point values. The image standardization is to realize centralized processing of data through mean value removal, and the data centralization accords with a data distribution rule according to convex optimization theory and data probability distribution related knowledge, so that a generalization effect after training is obtained more easily.
And carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the pretreatment. The specific process of image enhancement processing is as follows:
an angle is randomly selected from the range of 0 to 90 degrees to rotate the image.
Randomly selecting a multiple from 0.9 to 1.1 to make the image simultaneously perform the same degree of scaling operation in the length direction and the width direction. When the parameter is greater than 0 and less than 1, the zoom-in operation is performed, and when the parameter is greater than 1, the zoom-out operation is performed.
The image is randomly flipped horizontally and flipped up and down.
S103, constructing a neural network according to the preprocessed hardware image with the category label; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes the hardware image as input, and takes the category label of the hardware image as output. The convolutional layer is a convolutional layer of VGG 16. The output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval.
Wherein, add the Batch Normalization layer and not only can make the process of convergence accelerate greatly, greatly must promote the training speed, can effectively alleviate the gradient moreover and disappear and the gradient explosion problem. The addition of the Dropout layer can obviously reduce the overfitting phenomenon, thereby improving the robustness of the model.
After S103, further comprising:
and training the neural network to obtain the trained neural network.
The specific process of training the neural network is as follows:
when the first layer network is trained, the convolutional layer part adopts a convolutional layer which is trained in advance through imagenet. The weights of the layer two network are initialized randomly. The training process can be divided into four parts, namely forward conduction, loss function, backward conduction and weight updating.
In forward conduction, let XtrainThrough the whole network, assuming that the pictures have n categories in total, the output value of the second layer network is sout＝[s1，s2，...sn]TWill SoutSubstituting the softmax function into the obtained q is softmax (S)out) Passing through a loss function
To calculate the loss, piFor the true class of picture i, qiIs the softmax value of the ith graph. Then conducting backward, according to the formula
The weight is updated, alpha is the learning rate, and the weight can be adjusted by self. And the above steps are repeated.
When training the second layer of network, take Dropout already trained by the first layer of network and the previous network layers, and then add a fully-connected hidden layer and a new fully-connected layer after the Dropout layer. And then training the network according to four steps of forward conduction, loss function, backward conduction and weight updating. It should be noted that the network model is also a picture classification model, so that the output of the full connection layer needs to be substituted into the softmax function, and the loss function is also a loss function
And S104, retrieving the hardware image to be retrieved by utilizing the trained neural network.
S104 specifically comprises the following steps:
and carrying out data preprocessing on the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved.
And determining a plurality of class labels, binary retrieval vectors and the characteristics of the fc6 layer of the preprocessed hardware image to be retrieved according to the first layer network and the second layer network of the neural network.
Performing coarse-grained search according to the category labels and the binary retrieval vector to obtain a plurality of images with the difference smaller than a set threshold value with the preprocessed hardware image to be retrieved; and performing fine-grained search by using the characteristics of the fc6 layer.
As a specific embodiment, a picture to be retrieved without a category label is given, and data preprocessing is performed on the picture to be retrieved to obtain an image data set X.
And carrying out forward conducting operation of the neural network on the image array X. The feature vector Y of the network can be obtained through the first layer networkfc6And the first k classes with the highest predicted probability (topk accuracy)>99.9%), and is noted as Ck＝[c1，c2，...ck]。CkClass c of middle row first1Is the prediction category of the picture to be retrieved. The feature vector Y of the sigmoid layer can be obtained through the second layer networksigmoid. Further will be
Converted into binary search vector with the calculation formula of
Wherein j represents
The j-th bit element of (1). The output of this step is the feature vector Y of the FC6 layer of the search imagefc6The first k classes Ck＝[c1，c2，...ck]Binary search vector YbinAnd a prediction class c of pictures1。
According to CkAnd YbinA coarse grain search is performed. Through CkTo reduce the search space, i.e. search only for category Ck＝[c1，c2，...ck]The pictures of (2) do not need to search for other types of pictures. Followed by YbinAnd comparing the Hamming distances of the binary retrieval vectors of the pictures in the search space, and finally selecting m pictures with the closest Hamming distances from the search space. The Hamming distance is calculated by the following formula
Assuming that x, y are both n-bit encoded,
representing an exclusive or. The output of this step is m pictures screened from the picture database.
According to CkAnd YbinFine-grained search is carried out, the search space is changed into m pictures screened in the steps, and Y of the picture to be retrieved is usedfc6Comparing Y of pictures in search spacefc6And (4) reordering m graphs of the search space from near to far according to the Euclidean distance to obtain a final ordering result. And then determining the picture most similar to the picture to be searched.
Fig. 2 is a schematic structural diagram of a hardware image retrieval system based on deep learning, as shown in fig. 2, the hardware image retrieval system based on deep learning provided by the present invention includes: the hardware image acquisition module 201, the preprocessing module 202, the neural network construction module 203 and the retrieval module 204.
The hardware image obtaining module 201 is used for obtaining a hardware image with a category label.
The preprocessing module 202 is configured to perform data preprocessing on the hardware image with the category label; the pretreatment comprises the following steps: image resizing, data type conversion, normalization, and image enhancement.
The neural network construction module 203 is used for constructing a neural network according to the preprocessed hardware images with the category labels; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes the hardware image as input, and takes the category label of the hardware image as output. The output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval.
The retrieval module 204 is configured to retrieve the hardware image to be retrieved by using the neural network.
The preprocessing module 202 specifically includes: the image processing device comprises an image size adjusting unit, a data type converting unit, a normalization processing unit and an image enhancement processing unit.
And the image size adjusting unit is used for adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment.
The data type conversion unit is used for converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion.
And the normalization processing unit is used for performing normalization processing on the hardware image with the category label after the data type conversion to obtain the hardware image with the category label after the normalization processing.
And the standardization processing unit is used for carrying out standardization processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the standardization processing.
And the image enhancement processing unit is used for carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the preprocessing.
The invention provides a hardware image retrieval system based on deep learning, which further comprises: and a trained neural network determination module.
And the trained neural network determining module is used for training the neural network to obtain the trained neural network.
The retrieval module specifically comprises: the device comprises a preprocessing unit, a feature determining unit and a retrieval unit.
And the preprocessing unit is used for preprocessing the data of the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved.
The feature determination unit is used for determining a plurality of categories, binary retrieval vectors and features of the fc6 layer of the preprocessed hardware image to be retrieved according to the trained neural network.
The retrieval unit is used for performing coarse-grained search according to the plurality of category labels and the binary retrieval vector to obtain a plurality of images, the difference between the images and the preprocessed hardware image to be retrieved is smaller than a set threshold value; and performing fine-grained search by using the characteristics of the fc6 layer.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. A hardware image retrieval method based on deep learning is characterized by comprising the following steps:
acquiring a hardware image with a category label;
carrying out data preprocessing on the hardware image with the category label; the pretreatment comprises the following steps: adjusting the size of an image, converting the type of data, normalizing, standardizing and enhancing the image;
constructing a neural network according to the preprocessed hardware image with the category label; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes a hardware image as input and takes a category label of the hardware image as output; the output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval;
and searching the hardware image to be searched by utilizing the trained neural network.
2. The hardware image retrieval method based on deep learning of claim 1, wherein the hardware image with the category label is subjected to data preprocessing, and specifically comprises:
adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment;
converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion;
carrying out normalization processing on the hardware image with the class label after the data type conversion to obtain the hardware image with the class label after the normalization processing;
standardizing the hardware image with the category label after the normalization processing to obtain the hardware image with the category label after the standardization processing;
and carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the pretreatment.
3. The deep learning-based hardware image retrieval method according to claim 1, wherein a neural network is constructed according to the preprocessed hardware images with the class labels, and then the method further comprises:
and training the neural network to obtain the trained neural network.
4. The hardware image retrieval method based on deep learning of claim 3, wherein the retrieval of the hardware image to be retrieved by using the trained neural network specifically comprises:
carrying out data preprocessing on the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved;
determining a plurality of categories, binary retrieval vectors and fc6 layer characteristics of the preprocessed hardware images to be retrieved according to the first layer network and the second layer network of the neural network;
performing coarse-grained search according to the category labels and the binary retrieval vector to obtain a plurality of images with the difference smaller than a set threshold value with the preprocessed hardware image to be retrieved; and performing fine-grained search by using the characteristics of the fc6 layer.
5. The utility model provides a five metals image retrieval system based on deep learning which characterized in that includes:
the hardware image acquisition module is used for acquiring hardware images with category labels;
the preprocessing module is used for preprocessing the data of the hardware image with the category label; the pretreatment comprises the following steps: adjusting the size of an image, converting the type of data, normalizing, standardizing and enhancing the image;
the neural network construction module is used for constructing a neural network according to the preprocessed hardware images with the class labels; the neural network comprises a convolution layer, a full connection layer, a Batch Normalization layer, an activation function Relu layer and a Dropout layer which are connected in sequence; the Dropout layer is respectively connected with the first layer network and the second layer network; wherein the first layer network comprises a fully connected layer; the second layer network comprises two fully connected layers and a sigmoid layer between the two fully connected layers; the neural network takes a hardware image as input and takes a category label of the hardware image as output; the output of the Relu layer of the activation function is the feature of the fc6 layer, and the feature of the fc6 layer is used for fine-grained retrieval; the output of the sigmoid layer is converted into a binary hash code, and the binary hash code is used for coarse-grained retrieval;
and the retrieval module is used for retrieving the hardware image to be retrieved by utilizing the neural network.
6. The hardware image retrieval system based on deep learning of claim 5, wherein the preprocessing module specifically comprises:
the image size adjusting unit is used for adjusting the image size of the hardware image with the category label to obtain the hardware image with the category label after size adjustment;
the data type conversion unit is used for converting the data type of the hardware image with the category label after the size adjustment into a numpy array data type to obtain the hardware image with the category label after the data type conversion;
the normalization processing unit is used for performing normalization processing on the hardware image with the category label after the data type conversion to obtain the hardware image with the category label after the normalization processing;
the standardized processing unit is used for carrying out standardized processing on the hardware image with the classified label after the standardized processing to obtain the hardware image with the classified label after the standardized processing;
and the image enhancement processing unit is used for carrying out image enhancement processing on the hardware image with the category label after the standardization processing to obtain the hardware image with the category label after the preprocessing.
7. The deep learning-based hardware image retrieval system according to claim 5, further comprising:
and the trained neural network determining module is used for training the neural network to obtain the trained neural network.
8. The hardware image retrieval system based on deep learning of claim 7, wherein the retrieval module specifically comprises:
the preprocessing unit is used for preprocessing the hardware image to be retrieved to obtain a preprocessed hardware image to be retrieved;
the characteristic determining unit is used for determining a plurality of categories, binary retrieval vectors and fc 6-layer characteristics of the preprocessed hardware images to be retrieved according to the trained neural network;
the retrieval unit is used for performing coarse-grained search according to the plurality of category labels and the binary retrieval vector to obtain a plurality of images, the difference between the images and the preprocessed hardware image to be retrieved is smaller than a set threshold value; and performing fine-grained search by using the characteristics of the fc6 layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010876226.3A CN112000827A (en) | 2020-08-27 | 2020-08-27 | Hardware image retrieval method and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010876226.3A CN112000827A (en) | 2020-08-27 | 2020-08-27 | Hardware image retrieval method and system based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112000827A true CN112000827A (en) | 2020-11-27 |
Family
ID=73470483
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010876226.3A Pending CN112000827A (en) | 2020-08-27 | 2020-08-27 | Hardware image retrieval method and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000827A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107480261A (en) * | 2017-08-16 | 2017-12-15 | 上海荷福人工智能科技（集团）有限公司 | One kind is based on deep learning fine granularity facial image method for quickly retrieving |
| CN110059206A (en) * | 2019-03-29 | 2019-07-26 | 银江股份有限公司 | A kind of extensive hashing image search method based on depth representative learning |
| CN110297931A (en) * | 2019-04-23 | 2019-10-01 | 西北大学 | A kind of image search method |
| CN111460200A (en) * | 2020-03-04 | 2020-07-28 | 西北大学 | Image retrieval method and model based on multitask deep learning and construction method thereof |
-
2020
- 2020-08-27 CN CN202010876226.3A patent/CN112000827A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107480261A (en) * | 2017-08-16 | 2017-12-15 | 上海荷福人工智能科技（集团）有限公司 | One kind is based on deep learning fine granularity facial image method for quickly retrieving |
| CN110059206A (en) * | 2019-03-29 | 2019-07-26 | 银江股份有限公司 | A kind of extensive hashing image search method based on depth representative learning |
| CN110297931A (en) * | 2019-04-23 | 2019-10-01 | 西北大学 | A kind of image search method |
| CN111460200A (en) * | 2020-03-04 | 2020-07-28 | 西北大学 | Image retrieval method and model based on multitask deep learning and construction method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 何智海: "基于深度学习的五金件图像检索方法及其应用研究", 中国优秀硕士学位论文全文数据库信息科技辑, no. 2, 15 February 2020 (2020-02-15) * |
| 陈超逸: "基于深度学习特征的图像检索技术研究与实现", 中国优秀硕士学位论文全文数据库 信息科技辑, no. 11, 15 November 2018 (2018-11-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107480261B (en) | Fine-grained face image fast retrieval method based on deep learning | |
| CN107330127B (en) | Similar text detection method based on text picture retrieval | |
| CN108334574B (en) | Cross-modal retrieval method based on collaborative matrix decomposition | |
| CN110222218B (en) | Image retrieval method based on multi-scale NetVLAD and depth hash | |
| CN113221641B (en) | Video pedestrian re-identification method based on generation of antagonism network and attention mechanism | |
| Huang et al. | Cost-effective vehicle type recognition in surveillance images with deep active learning and web data | |
| Prudviraj et al. | Attentive contextual network for image captioning | |
| CN108446334B (en) | Image retrieval method based on content for unsupervised countermeasure training | |
| CN113377981B (en) | Large-scale logistics commodity image retrieval method based on multitask deep hash learning | |
| CN112651940B (en) | Collaborative visual saliency detection method based on dual-encoder generation type countermeasure network | |
| CN111008224B (en) | Time sequence classification and retrieval method based on deep multitasking representation learning | |
| CN111104555A (en) | Video hash retrieval method based on attention mechanism | |
| CN111461175B (en) | Label recommendation model construction method and device of self-attention and cooperative attention mechanism | |
| CN112163114B (en) | Image retrieval method based on feature fusion | |
| CN115937655A (en) | Target detection model of multi-order feature interaction, and construction method, device and application thereof | |
| CN116383422B (en) | Non-supervision cross-modal hash retrieval method based on anchor points | |
| CN115795065A (en) | Multimedia data cross-modal retrieval method and system based on weighted hash code | |
| CN113656700A (en) | Hash retrieval method based on multi-similarity consistent matrix decomposition | |
| CN116049450A (en) | Multi-mode-supported image-text retrieval method and device based on distance clustering | |
| Yu et al. | Text-image matching for cross-modal remote sensing image retrieval via graph neural network | |
| CN114780767A (en) | Large-scale image retrieval method and system based on deep convolutional neural network | |
| Xu et al. | Residual spatial attention kernel generation network for hyperspectral image classification with small sample size | |
| Zhang et al. | DHNet: Salient object detection with dynamic scale-aware learning and hard-sample refinement | |
| Gong et al. | Erroneous pixel prediction for semantic image segmentation | |
| CN111259176B (en) | Cross-modal Hash retrieval method based on matrix decomposition and integrated with supervision information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |