CN111209428A - Image retrieval method, device, equipment and computer readable storage medium - Google Patents
Image retrieval method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111209428A CN111209428A CN202010008668.6A CN202010008668A CN111209428A CN 111209428 A CN111209428 A CN 111209428A CN 202010008668 A CN202010008668 A CN 202010008668A CN 111209428 A CN111209428 A CN 111209428A
- Authority
- CN
- China
- Prior art keywords
- image
- loss function
- preset
- total
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses an image retrieval method, an image retrieval device, image retrieval equipment and a computer readable storage medium, wherein the method comprises the following steps: constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model; extracting first image characteristics of various images in a preset image database according to the network model; and when the image to be retrieved is received, extracting second image characteristics of the image to be retrieved based on the network model, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics. The total loss function constructed by the method can measure the accuracy of the model from multiple aspects, so that the generated network model is more accurate; the accuracy of the target image searched according to the second image features extracted by the network model and various first image features is ensured, and the accuracy of image retrieval is improved.
Description
Technical Field
The present invention relates to the field of financial technology (Fintech), and in particular, to an image retrieval method, apparatus, device, and computer-readable storage medium.
Background
With the continuous development of financial technology (Fintech), especially internet technology and finance, more and more technologies (such as artificial intelligence, big data, cloud storage and the like) are applied to the financial field, but the financial industry also puts higher requirements on various technologies, such as the requirement of conveniently and accurately inquiring images required by users from mass remote sensing images.
With the rapid development of the aerospace technology, the number of remote sensing images acquired by a high-resolution remote sensing mode is greatly increased, a user can acquire various types of images from a large number of remote sensing images, and the images required by the user are acquired by a retrieval model. However, the existing index for measuring the accuracy of the retrieval model has the defect of insufficient accuracy, so that the retrieval model for retrieving the image is inaccurate. The images retrieved through the retrieval model have large characteristic difference between the same type of images, and the classification is not accurate enough; the difference of the characteristics of different types of images is small, and the different types of images are difficult to distinguish; finally, the retrieved image is not the image required by the user, and the retrieval accuracy is low.
Disclosure of Invention
The invention mainly aims to provide an image retrieval method, an image retrieval device, image retrieval equipment and a computer readable storage medium, and aims to solve the technical problems that in the prior art, the image required by a user is not accurately acquired and the image retrieval accuracy rate is low.
In order to achieve the above object, the present invention provides an image retrieval method, including the steps of:
constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model;
extracting first image characteristics of various images in a preset image database according to the network model;
when an image to be retrieved is received, extracting second image characteristics of the image to be retrieved based on the network model, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics.
Optionally, the step of transmitting a preset sample image to a base model for training according to the total loss function, and generating a network model includes:
classifying the preset sample images based on the basic model to obtain a plurality of preset sample image groups, and calculating the total loss function based on the plurality of preset sample image groups to generate a total function value;
judging whether the total function value is smaller than a preset threshold value, if so, finishing the training of the basic model and generating a network model;
if the total function value is not smaller than a preset threshold value, classifying the preset sample image groups again based on the basic model until the total function value generated by calculating the total loss function based on the classified preset sample image groups is smaller than the preset threshold value.
Optionally, the calculating the total loss function based on the plurality of preset sample image sets, and the generating a total function value includes:
calculating a central loss function in the total loss function based on a plurality of preset sample image groups to generate a central loss function value;
calculating a weight loss function in the total loss function based on a plurality of preset sample image groups to generate a weight loss function value;
and calculating a preset loss function in the total loss function to generate a cross entropy loss function value, and generating a total function value of the total loss function according to the central loss function value, the weight loss function value and the cross entropy loss function value.
Optionally, the step of calculating a central loss function in the total loss functions based on a plurality of preset sample image groups, and generating a central loss function value includes:
randomly screening each preset sample image group to obtain a sample image to be processed of each preset sample image group;
respectively transmitting the characteristic values of the sample images to be processed of each preset sample image group to a preset formula to generate a central point of each preset sample image group;
and calculating a central loss function in the total loss function according to the central point to obtain a central loss function value.
Optionally, the step of searching for the target image corresponding to the image to be retrieved according to the second image feature and each of the first image features includes:
respectively carrying out Euclidean distance calculation on the second image features and each first image feature to generate a similarity value between the second image features and each first image feature;
and searching a target image corresponding to the image to be retrieved according to each similarity value.
Optionally, the step of searching for the target image corresponding to the image to be retrieved according to each of the similarity values includes:
comparing the similarity values, determining a target similarity value with the largest value in the similarity values, and searching a first image feature of a target corresponding to the target similarity value;
and determining each image in the image type corresponding to the target first image characteristic as the target image, and outputting the target image.
Optionally, the step of extracting, according to the network model, first image features of each type of image in a preset image database includes:
transmitting images in a preset database to the network model, classifying the images based on the network model, and obtaining various images in the preset database;
and extracting the characteristics of the various images based on the network model to obtain the first image characteristics of the various images.
Further, to achieve the above object, the present invention also provides an image retrieval apparatus including:
the training module is used for constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model;
the extraction module is used for extracting first image characteristics of various images in a preset image database according to the network model;
and the searching module is used for extracting second image characteristics of the image to be retrieved based on the network model when the image to be retrieved is received, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics.
Further, to achieve the above object, the present invention also provides an image retrieval device, which includes a memory, a processor, and an image retrieval program stored on the memory and executable on the processor, wherein the image retrieval program, when executed by the processor, implements the steps of the image retrieval method as described above.
Further, to achieve the above object, the present invention also provides a computer readable storage medium having stored thereon an image retrieval program, which when executed by a processor, implements the steps of the image retrieval method as described above.
The image retrieval method comprises the steps of constructing a total loss function comprising a central loss function and a weight loss function, training a preset sample image based on the total loss function, and generating a network model; extracting and storing first image characteristics of each image in a preset image database according to the network model, and extracting second image characteristics of the image to be retrieved through the network model when the image to be retrieved is received; and then, searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics. The accuracy of the model can be measured from multiple aspects due to the constructed total loss function, so that the generated network model is more accurate; the images in the preset image database are classified through the network model, and the accuracy of extracting the first graphic features of the images is high; the target image searched and obtained is the image required by the user according to the second image characteristics and the first image characteristics, and the accuracy of image retrieval is improved.
Drawings
FIG. 1 is a schematic structural diagram of an apparatus hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of an image retrieval method according to the present invention;
FIG. 3 is a functional block diagram of an image retrieving device according to a preferred embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an image retrieval device, and referring to fig. 1, fig. 1 is a schematic structural diagram of a device hardware operating environment according to an embodiment of the image retrieval device of the invention.
As shown in fig. 1, the image retrieval apparatus may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the hardware configuration of the image retrieval device shown in fig. 1 does not constitute a limitation of the image retrieval device, and may include more or less components than those shown, or combine some components, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and an image retrieval program. The operating system is a program for managing and controlling the image retrieval equipment and software resources, and supports the running of a network communication module, a user interface module, an image retrieval program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
In the hardware structure of the image retrieval device shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; the processor 1001 may call an image retrieval program stored in the memory 1005 and perform the following operations:
constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model;
extracting first image characteristics of various images in a preset image database according to the network model;
when an image to be retrieved is received, extracting second image characteristics of the image to be retrieved based on the network model, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics.
Further, the step of transmitting a preset sample image to a base model for training according to the total loss function to generate a network model includes:
classifying the preset sample images based on the basic model to obtain a plurality of preset sample image groups, and calculating the total loss function based on the plurality of preset sample image groups to generate a total function value;
judging whether the total function value is smaller than a preset threshold value, if so, finishing the training of the basic model and generating a network model;
if the total function value is not smaller than a preset threshold value, classifying the preset sample image groups again based on the basic model until the total function value generated by calculating the total loss function based on the classified preset sample image groups is smaller than the preset threshold value.
Further, the step of calculating the total loss function based on the plurality of preset sample image sets, and generating a total function value includes:
calculating a central loss function in the total loss function based on a plurality of preset sample image groups to generate a central loss function value;
calculating a weight loss function in the total loss function based on a plurality of preset sample image groups to generate a weight loss function value;
and calculating a preset loss function in the total loss function to generate a cross entropy loss function value, and generating a total function value of the total loss function according to the central loss function value, the weight loss function value and the cross entropy loss function value.
Further, the step of calculating a central loss function in the total loss functions based on the plurality of preset sample image groups, and generating a central loss function value includes:
randomly screening each preset sample image group to obtain a sample image to be processed of each preset sample image group;
respectively transmitting the characteristic values of the sample images to be processed of each preset sample image group to a preset formula to generate a central point of each preset sample image group;
and calculating a central loss function in the total loss function according to the central point to obtain a central loss function value.
Further, the step of searching for the target image corresponding to the image to be retrieved according to the second image feature and each of the first image features includes:
respectively carrying out Euclidean distance calculation on the second image features and each first image feature to generate a similarity value between the second image features and each first image feature;
and searching a target image corresponding to the image to be retrieved according to each similarity value.
Further, the step of searching for the target image corresponding to the image to be retrieved according to each of the similarity values includes:
comparing the similarity values, determining a target similarity value with the largest value in the similarity values, and searching a first image feature of a target corresponding to the target similarity value;
and determining each image in the image type corresponding to the target first image characteristic as the target image, and outputting the target image.
Further, the step of extracting the first image features of the various types of images in the preset image database according to the network model includes:
transmitting images in a preset database to the network model, classifying the images based on the network model, and obtaining various images in the preset database;
and extracting the characteristics of the various images based on the network model to obtain the first image characteristics of the various images.
The specific implementation of the image retrieval device of the present invention is substantially the same as the following embodiments of the image retrieval method, and is not described herein again.
The invention also provides an image retrieval method.
Referring to fig. 2, fig. 2 is a flowchart illustrating an image retrieval method according to a first embodiment of the present invention.
While a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than that shown or described herein. Specifically, the image retrieval method in the present embodiment includes:
and step S10, constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model.
The image retrieval method in the embodiment is applied to a server, and the server has a trained network model so as to retrieve images in a database through the network model to obtain images required by a user. The retrieved image may be an image generated by photographing a natural scene, or an image acquired by a remote sensing technology, and is preferably retrieved for the image acquired by the remote sensing technology.
Understandably, before image retrieval is carried out through the trained network model, the network model is trained; and in order to characterize the accuracy of the trained network model, a loss function needs to be constructed. In the embodiment, a loss function is constructed by combining various factors, and the loss function at least comprises a central loss function and a weight loss function besides a preset loss function which is preset. The loss function constructed by the preset loss function, the central loss function and the weight loss function is used as a total loss function, so that the network model obtained by training has the characteristics that the characteristic difference between the similar images is small and the characteristic difference between the different images is large when the images are classified and retrieved, and the accuracy of classification and retrieval is improved.
It should be noted that the preset loss function in this embodiment is preferably a softmax loss function, and the functional formula of the preset loss function is shown in formula (1):
wherein i is 1, 2, 3, j is 1, 2, 3, sjThe jth value of the output vector s of softmax represents the probability that the picture belongs to the jth class, y is a vector of 1 × T with T values, the value at the jth position is 1, the values at the rest positions are 0, and log is a logarithmic function.
Further, the functional formula of the center loss function is shown in formula (2):
where x represents the input image, the image corresponds to a category of y, and a center is set for each category, denoted as cyAnd f (x) represents a value calculated for the input image x by the network model.
Further, the function of the weight loss function is shown in equation (3):
wherein i is 1, 2, 3. wiAnd (3) representing the parameters of the ith layer of neural network in the network model, wherein n represents all the layers of the neural network, and i is less than or equal to n.
The functional formula of the total loss function obtained by composing the preset loss function, the central loss function and the weight loss function is shown in formula (4):
Ltotal＝Lsoftmax+λ1Lcenter+λ2Lw(4)；
wherein λ is1And λ2Are two parameters for adjusting the balance between the pre-set loss function, the central loss function and the weight loss function.
Further, after the total loss function is obtained through construction, a preset sample image for training which is preset can be transmitted to the basic model, the basic model is trained, and whether the training of the basic model is finished or not is judged by calculating the total function value of the total loss function after training. If the accuracy of the trained total function value representation basic model is higher, the accuracy is not greatly influenced by continuous training, and system resources are occupied instead, namely the value of the continuous training is not large; and stopping training at the moment, wherein the trained basic model is the network model. And when the accuracy of the total function value representation basic model after training is low, the influence of continuous training on the improvement of the accuracy is large, and the value of the continuous training is high, the training is continued until the accuracy of the basic model is high, so that the network model is obtained.
Step S20, extracting first image characteristics of various images in a preset image database according to the network model;
further, a preset image database is previously provided, in which various types of images are stored, so that the user can conveniently search for a desired image therefrom. After the network model is obtained through training, various images in a preset image database can be classified according to the network model, the images with similar characteristics are divided into the same class, and the same characteristics in the various images are extracted as first image characteristics. Specifically, according to the network model, the step of extracting the first image features of each type of image in the preset image database comprises the following steps:
step S21, transmitting images in a preset database to the network model, and classifying the images based on the network model to obtain various images in the preset database;
and step S22, extracting the characteristics of each type of image based on the network model to obtain the first image characteristics of each type of image.
And reading the images in the preset data volume and transmitting the images to the network model so as to classify the images based on the trained model parameters of the network model. And dividing the images with the similar characteristics larger than a certain value among the images into the same class, and dividing the images with the similar characteristics not larger than the certain value into different classes, so as to obtain various images in a preset database. And then, respectively extracting the features of the various images, extracting the same features of the images among the various images, and obtaining the first image features of the various images so as to represent the characteristics of the various images.
It should be noted that after the first image features of each type of image are extracted, the first image features of each type of image may be stored in another database to form a feature database, so as to search for an image required by a user directly according to the first image features in the feature database.
Step S30, when receiving the image to be retrieved, extracting the second image characteristic of the image to be retrieved based on the network model, and searching the target image corresponding to the image to be retrieved according to the second image characteristic and each first image characteristic.
Further, when the user has a requirement for retrieving a certain image, the image is taken as an image to be retrieved for uploading operation. And when receiving the image to be retrieved, the server calls the network model to extract the features of the image to be retrieved, and takes the extracted features as second image features to represent the characteristics of the image to be retrieved.
Furthermore, the target image corresponding to the image to be retrieved is determined according to the characteristics of the image to be retrieved represented by the second image characteristics and the characteristics of each type of image represented by each first image characteristic. When one characteristic of the characteristics of the various images represented by the characteristics of the first image is very close to the characteristic of the image to be retrieved represented by the second image, the image type with the characteristic is the image which needs to be searched by the user, and the image is used as a target image to meet the acquisition requirement of the user on the image to be retrieved.
The image retrieval method comprises the steps of constructing a total loss function comprising a central loss function and a weight loss function, training a preset sample image based on the total loss function, and generating a network model; extracting and storing first image characteristics of each image in a preset image database according to the network model, and extracting second image characteristics of the image to be retrieved through the network model when the image to be retrieved is received; and then, searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics. The accuracy of the model can be measured from multiple aspects due to the constructed total loss function, so that the generated network model is more accurate; the images in the preset image database are classified through the network model, and the accuracy of extracting the first graphic features of the images is high; the target image searched and obtained is the image required by the user according to the second image characteristics and the first image characteristics, and the accuracy of image retrieval is improved.
Further, based on the first embodiment of the image retrieval method of the present invention, a second embodiment of the image retrieval method of the present invention is proposed.
The second embodiment of the image retrieval method is different from the first embodiment of the image retrieval method in that the step of transmitting a preset sample image to a base model for training according to the total loss function and generating a network model comprises:
step S11, classifying the preset sample images based on the basic model to obtain a plurality of preset sample image groups, and calculating the total loss function based on the plurality of preset sample image groups to generate a total function value;
in the basic model training process, the preset sample images are classified by the basic model to obtain each preset sample image group, the total loss function is calculated on the basis of each classified preset sample image group to generate a total function value, and the accuracy of the trained basic model is represented by the size of the total function value. The total loss function comprises a preset loss function, a central loss function and a weight loss function, so that the total function value is correspondingly related to the preset loss function, the central loss function and the weight loss function; specifically, the step of calculating the total loss function based on a plurality of preset sample image sets, and generating the total function value includes:
step S111, calculating a central loss function in the total loss function based on a plurality of preset sample image groups to generate a central loss function value;
further, after obtaining each preset sample image group by the base model classification, on the basis of each preset sample image group, a central loss function in the total loss functions is calculated, that is, calculated by formula (2), so as to obtain a central loss function value. The calculation of the central loss function is related to the central point of each preset sample image group, that is, the central point of each preset sample image group needs to be determined in the process of calculating the central loss function value. Specifically, a central loss function in the total loss functions is calculated based on a plurality of preset sample image groups, and the step of generating the central loss function value includes:
step a, randomly screening each preset sample image group to obtain a sample image to be processed of each preset sample image group;
b, respectively transmitting the characteristic values of the sample images to be processed of each preset sample image group to a preset formula, and generating the central point of each preset sample image group;
and c, calculating a central loss function in the total loss function according to the central point to obtain a central loss function value.
Considering that there are many images in each preset sample image, calculating the average of the feature values of all the images in each preset sample image may result in complicated calculation and low calculation efficiency. Therefore, in order to improve the calculation efficiency, the present embodiment provides a partial image calculation mechanism when calculating the average point. Specifically, each preset sample image group is randomly screened to obtain a to-be-processed sample image of each preset sample image group for calculating an average point.
Furthermore, a preset formula for calculating the central point is preset, and after the to-be-processed sample images of each preset sample image group are obtained through random screening, the characteristic value of each to-be-processed sample image is extracted to represent the characteristics of the images in each preset sample image group. And then, respectively transmitting the characteristic values of the characteristic groups of the preset sample images to a preset formula, and obtaining the central point of each preset sample image group through calculation of the preset formula. Wherein, the preset formula is shown as formula (5):
wherein i is 1, 2, 3, n is the number of sample images to be processed in the preset sample image group, and i is less than or equal to n, f (x)i) The characteristic value is corresponding to the ith picture.
Further, after the central points of the preset sample image groups are obtained through calculation in formula (5), the central loss function is calculated according to the central points, and the central loss function value can be obtained.
Step S112, calculating a weight loss function in the total loss function based on a plurality of preset sample image groups to generate a weight loss function value;
step S113, calculating a preset loss function in the total loss function to generate a cross entropy loss function value, and generating a total function value of the total loss function according to the central loss function value, the weight loss function value, and the cross entropy loss function value.
Further, after the central loss function in the total loss function is calculated on the basis of each preset sample image group to obtain a central loss function value, the weight loss function and the preset loss function in the total loss function are also required to be calculated respectively, that is, the weight loss function value and the cross entropy loss function value are obtained by calculating through formulas (3) and (1) respectively. And then, calculating the calculated central loss function value, the weight loss function value and the cross entropy loss function value according to the formula (5) to obtain a total function value of the total loss function so as to represent the accuracy of the trained basic model.
Step S12, judging whether the total function value is smaller than a preset threshold value, if so, finishing the training of the basic model and generating a network model;
step S13, if the total function value is not less than the preset threshold, re-classifying the preset sample image groups based on the basic model until the total function value generated by calculating the total loss function based on the classified preset sample image groups is less than the preset threshold.
Furthermore, in order to represent the accuracy of the trained basic model, a preset threshold value is set empirically in advance. After the total loss function is generated and calculated to obtain a total function value, comparing the total function value with a preset threshold value, judging whether the total function value is smaller than the preset threshold value, if so, indicating that the accuracy of the trained basic model is higher, finishing the training of the basic model, and taking the trained basic model as a network model.
If the total function value is judged to be not less than the preset threshold value, the accuracy of the trained basic model is still to be improved, so that the basic model is classified again on the basis of each classified preset sample image group, and the total function value of the total loss function after the classification is calculated again. If the total function value is smaller than a preset threshold value, finishing training the basic model and generating a network model; otherwise, if the total function value is still not smaller than the preset threshold, the total function value is classified again on the basis of each current preset sample image group and generated, and training of the basic model is not completed until the generated total function value is smaller than the preset threshold, so that the network model is generated.
In the process of training the basic model, the accuracy of the basic model is determined by calculating the total function value of the total loss function; because the total loss function comprises the central loss function and the weight loss function, a plurality of factors influencing the accuracy of the model are considered, and the network model obtained by training is more accurate.
Further, based on the first or second embodiment of the image retrieval method of the present invention, a third embodiment of the image retrieval method of the present invention is proposed.
The third embodiment of the image retrieval method is different from the first or second embodiment of the image retrieval method in that the step of searching for the target image corresponding to the image to be retrieved according to the second image feature and each of the first image features comprises:
step S31, performing euclidean distance calculation on the second image features and each of the first image features, respectively, to generate similarity values between the second image features and each of the first image features;
and step S32, searching a target image corresponding to the image to be retrieved according to each similarity value.
Further, in the embodiment, an image to be retrieved, which is required to be acquired by a user, is searched from a preset image database in an euclidean distance manner; wherein the euclidean distance refers to the real distance between two points in the m-dimensional space, or the natural length of the vector (i.e. the distance from the point to the origin), and has the characteristic that the smaller the distance value, the higher the similarity. Specifically, after the second image features of the image to be retrieved are extracted, euclidean distances between the first image features of each type of image in the preset database and the second image features are calculated, and the distance values obtained through calculation are similarity values between the second feature image and each first image feature, so that the similarity degree between each second image feature and each first image feature is represented through each similarity value.
Furthermore, the similarity values are compared, the magnitude relation among the similarity values is determined, and the similarity value with the largest value is searched as the target similarity value. The first image feature with the maximum value of the target similarity value, that is, the first image feature with the highest similarity to the second image feature is generated as the target first image feature. After the target first image feature corresponding to the target first image feature is found according to the target similarity value with the maximum numerical value, an image type with the target first image feature can be found from a preset image database, and the image type is an image which needs to be searched by a user. Therefore, all images in the preset image database with the image type of the target first image feature are determined as target images which need to be retrieved by the user, and all the target images are output and displayed, so that the user can conveniently acquire the target images corresponding to the images to be retrieved.
In the embodiment, the similarity between the image to be retrieved and various images in the preset image database is calculated through the Euclidean distance, so that compared with the problem that the binary code of the image obtained by retrieving the image in a hash algorithm mode is too long or too short, a large amount of storage space is needed or the retrieval accuracy is low, the method is favorable for ensuring the image compression accuracy.
The invention also provides an image retrieval device.
Referring to fig. 3, fig. 3 is a functional block diagram of the image retrieving device according to the first embodiment of the present invention. The image retrieval apparatus includes:
the training module 10 is configured to construct a total loss function including a central loss function and a weight loss function, and transmit a preset sample image to a base model for training according to the total loss function to generate a network model;
the extraction module 20 is configured to extract first image features of various types of images in a preset image database according to the network model;
the searching module 30 is configured to, when an image to be retrieved is received, extract a second image feature of the image to be retrieved based on the network model, and search for a target image corresponding to the image to be retrieved according to the second image feature and each of the first image features.
Further, the training module 10 includes:
the calculation unit is used for classifying the preset sample images based on the basic model to obtain a plurality of preset sample image groups, and calculating the total loss function based on the plurality of preset sample image groups to generate a total function value;
the judging unit is used for judging whether the total function value is smaller than a preset threshold value or not, and finishing the training of the basic model and generating a network model if the total function value is smaller than the preset threshold value;
and the classification unit is used for reclassifying the preset sample image groups based on the basic model if the total function value is not less than a preset threshold value until the total function value generated by calculating the total loss function based on the classified preset sample image groups is less than the preset threshold value.
Further, the computing unit is further configured to:
calculating a central loss function in the total loss function based on a plurality of preset sample image groups to generate a central loss function value;
calculating a weight loss function in the total loss function based on a plurality of preset sample image groups to generate a weight loss function value;
and calculating a preset loss function in the total loss function to generate a cross entropy loss function value, and generating a total function value of the total loss function according to the central loss function value, the weight loss function value and the cross entropy loss function value.
Further, the computing unit is further configured to:
randomly screening each preset sample image group to obtain a sample image to be processed of each preset sample image group;
respectively transmitting the characteristic values of the sample images to be processed of each preset sample image group to a preset formula to generate a central point of each preset sample image group;
and calculating a central loss function in the total loss function according to the central point to obtain a central loss function value.
Further, the lookup module 30 includes:
a generating unit, configured to perform euclidean distance calculation on the second image features and each of the first image features, and generate similarity values between the second image features and each of the first image features;
and the searching unit is used for searching the target image corresponding to the image to be retrieved according to each similarity value.
Further, the search unit is further configured to:
comparing the similarity values, determining a target similarity value with the largest value in the similarity values, and searching a first image feature of a target corresponding to the target similarity value;
and determining each image in the image type corresponding to the target first image characteristic as the target image, and outputting the target image.
Further, the extraction module 20 further includes:
the transmission unit is used for transmitting the images in a preset database to the network model, classifying the images based on the network model and obtaining various images in the preset database;
and the extraction unit is used for extracting the characteristics of all types of images based on the network model to obtain the first image characteristics of all types of images.
The specific implementation of the image retrieval apparatus of the present invention is substantially the same as the embodiments of the image retrieval method described above, and will not be described herein again.
In addition, the embodiment of the invention also provides a computer readable storage medium.
The computer readable storage medium has stored thereon an image retrieval program which, when executed by a processor, implements the steps of the image retrieval method as described above.
The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the embodiments of the image retrieval method described above, and is not described herein again.
The present invention is described in connection with the accompanying drawings, but the present invention is not limited to the above embodiments, which are only illustrative and not restrictive, and those skilled in the art can make various changes without departing from the spirit and scope of the invention as defined by the appended claims, and all changes that come within the meaning and range of equivalency of the specification and drawings that are obvious from the description and the attached claims are intended to be embraced therein.
Claims (10)
1. An image retrieval method, characterized by comprising the steps of:
constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model;
extracting first image characteristics of various images in a preset image database according to the network model;
when an image to be retrieved is received, extracting second image characteristics of the image to be retrieved based on the network model, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics.
2. The image retrieval method of claim 1, wherein the step of transmitting a preset sample image to a base model for training according to the total loss function, and generating a network model comprises:
classifying the preset sample images based on the basic model to obtain a plurality of preset sample image groups, and calculating the total loss function based on the plurality of preset sample image groups to generate a total function value;
judging whether the total function value is smaller than a preset threshold value, if so, finishing the training of the basic model and generating a network model;
if the total function value is not smaller than a preset threshold value, classifying the preset sample image groups again based on the basic model until the total function value generated by calculating the total loss function based on the classified preset sample image groups is smaller than the preset threshold value.
3. The image retrieval method of claim 2, wherein the step of calculating the total loss function based on the plurality of preset sample image sets, generating a total function value comprises:
calculating a central loss function in the total loss function based on a plurality of preset sample image groups to generate a central loss function value;
calculating a weight loss function in the total loss function based on a plurality of preset sample image groups to generate a weight loss function value;
and calculating a preset loss function in the total loss function to generate a cross entropy loss function value, and generating a total function value of the total loss function according to the central loss function value, the weight loss function value and the cross entropy loss function value.
4. The image retrieval method of claim 3, wherein the step of calculating a central loss function of the total loss functions based on the plurality of preset sample image groups, and generating a central loss function value comprises:
randomly screening each preset sample image group to obtain a sample image to be processed of each preset sample image group;
respectively transmitting the characteristic values of the sample images to be processed of each preset sample image group to a preset formula to generate a central point of each preset sample image group;
and calculating a central loss function in the total loss function according to the central point to obtain a central loss function value.
5. The image retrieval method of any one of claims 1 to 4, wherein the step of finding a target image corresponding to the image to be retrieved according to the second image feature and each of the first image features comprises:
respectively carrying out Euclidean distance calculation on the second image features and each first image feature to generate a similarity value between the second image features and each first image feature;
and searching a target image corresponding to the image to be retrieved according to each similarity value.
6. The image retrieval method of claim 5, wherein the step of finding a target image corresponding to the image to be retrieved according to each of the similarity values comprises:
comparing the similarity values, determining a target similarity value with the largest value in the similarity values, and searching a first image feature of a target corresponding to the target similarity value;
and determining each image in the image type corresponding to the target first image characteristic as the target image, and outputting the target image.
7. The image retrieval method of any one of claims 1 to 4, wherein the step of extracting the first image feature of each type of image in the preset image database according to the network model comprises:
transmitting images in a preset database to the network model, classifying the images based on the network model, and obtaining various images in the preset database;
and extracting the characteristics of the various images based on the network model to obtain the first image characteristics of the various images.
8. An image retrieval apparatus characterized by comprising:
the training module is used for constructing a total loss function comprising a central loss function and a weight loss function, and transmitting a preset sample image to a basic model for training according to the total loss function to generate a network model;
the extraction module is used for extracting first image characteristics of various images in a preset image database according to the network model;
and the searching module is used for extracting second image characteristics of the image to be retrieved based on the network model when the image to be retrieved is received, and searching a target image corresponding to the image to be retrieved according to the second image characteristics and the first image characteristics.
9. An image retrieval device, characterized in that the image retrieval device comprises a memory, a processor and an image retrieval program stored on the memory and executable on the processor, which image retrieval program, when executed by the processor, implements the steps of the image retrieval method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon an image retrieval program which, when executed by a processor, implements the steps of the image retrieval method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010008668.6A CN111209428A (en) | 2020-01-03 | 2020-01-03 | Image retrieval method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010008668.6A CN111209428A (en) | 2020-01-03 | 2020-01-03 | Image retrieval method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111209428A true CN111209428A (en) | 2020-05-29 |
Family
ID=70788332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010008668.6A Pending CN111209428A (en) | 2020-01-03 | 2020-01-03 | Image retrieval method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209428A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360700A (en) * | 2021-06-30 | 2021-09-07 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for training image-text retrieval model and image-text retrieval |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169974A (en) * | 2017-05-26 | 2017-09-15 | 中国科学技术大学 | It is a kind of based on the image partition method for supervising full convolutional neural networks more |
| CN107679078A (en) * | 2017-08-29 | 2018-02-09 | 银江股份有限公司 | A kind of bayonet socket image vehicle method for quickly retrieving and system based on deep learning |
| WO2019085793A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classification method, computer device and computer readable storage medium |
| CN109784366A (en) * | 2018-12-07 | 2019-05-21 | 北京飞搜科技有限公司 | The fine grit classification method, apparatus and electronic equipment of target object |
| CN109800754A (en) * | 2018-12-06 | 2019-05-24 | 杭州电子科技大学 | A kind of ancient character body classification method based on convolutional neural networks |
-
2020
- 2020-01-03 CN CN202010008668.6A patent/CN111209428A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169974A (en) * | 2017-05-26 | 2017-09-15 | 中国科学技术大学 | It is a kind of based on the image partition method for supervising full convolutional neural networks more |
| CN107679078A (en) * | 2017-08-29 | 2018-02-09 | 银江股份有限公司 | A kind of bayonet socket image vehicle method for quickly retrieving and system based on deep learning |
| WO2019085793A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classification method, computer device and computer readable storage medium |
| CN109800754A (en) * | 2018-12-06 | 2019-05-24 | 杭州电子科技大学 | A kind of ancient character body classification method based on convolutional neural networks |
| CN109784366A (en) * | 2018-12-07 | 2019-05-21 | 北京飞搜科技有限公司 | The fine grit classification method, apparatus and electronic equipment of target object |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360700A (en) * | 2021-06-30 | 2021-09-07 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for training image-text retrieval model and image-text retrieval |
| CN113360700B (en) * | 2021-06-30 | 2023-09-29 | 北京百度网讯科技有限公司 | Training of image-text retrieval model, image-text retrieval method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902708B (en) | Recommendation model training method and related device | |
| CN109993102B (en) | Similar face retrieval method, device and storage medium | |
| CN109299344A (en) | The generation method of order models, the sort method of search result, device and equipment | |
| CN111553428B (en) | Method, apparatus, device and readable storage medium for training discriminant model | |
| CN109993627B (en) | Recommendation method, recommendation model training device and storage medium | |
| CN109033261B (en) | Image processing method, image processing apparatus, image processing device, and storage medium | |
| CN112306829B (en) | Method and device for determining performance information, storage medium and terminal | |
| CN111159563A (en) | Method, device and equipment for determining user interest point information and storage medium | |
| US20220139085A1 (en) | Method and apparatus for video frame processing | |
| CN116108393A (en) | Power sensitive data classification and classification method and device, storage medium and electronic equipment | |
| CN114676279B (en) | Image retrieval method, device, equipment and computer readable storage medium | |
| CN111210022B (en) | Backward model selecting method, apparatus and readable storage medium | |
| CN111143555A (en) | Big data-based customer portrait generation method, device, equipment and storage medium | |
| CN115545103A (en) | Abnormal data identification method, label identification method and abnormal data identification device | |
| CN112817563A (en) | Target attribute configuration information determination method, computer device, and storage medium | |
| CN114926832A (en) | Feature extraction model training method, material chartlet processing method, device and electronic equipment | |
| CN111209428A (en) | Image retrieval method, device, equipment and computer readable storage medium | |
| CN112686312A (en) | Data classification method, device and system | |
| CN110955774A (en) | Word frequency distribution-based character classification method, device, equipment and medium | |
| CN105843930A (en) | Video search method and device | |
| WO2023102844A1 (en) | Method and apparatus for determining pruning module, and computer-readable storage medium | |
| CN115757900A (en) | User demand analysis method and system applying artificial intelligence model | |
| CN114677578A (en) | Method and device for determining training sample data | |
| CN114511715A (en) | Driving scene data mining method | |
| CN112070162A (en) | Multi-class processing task training sample construction method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |