CN116363407A - Image classification method, device, equipment and storage medium - Google Patents
Image classification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116363407A CN116363407A CN202111618688.6A CN202111618688A CN116363407A CN 116363407 A CN116363407 A CN 116363407A CN 202111618688 A CN202111618688 A CN 202111618688A CN 116363407 A CN116363407 A CN 116363407A
- Authority
- CN
- China
- Prior art keywords
- information
- image
- preset
- target
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000000605 extraction Methods 0.000 claims abstract description 137
- 238000002372 labelling Methods 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000013527 convolutional neural network Methods 0.000 description 15
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to the field of image processing technologies, and in particular, to an image classification method, apparatus, device, and storage medium. According to the method, the frame images to be classified of the target video are subjected to feature extraction through the preset first feature extraction model, different images can be distinguished so as to facilitate subsequent image classification, feature information corresponding to the frame images is matched with historical feature information in the preset database, basis is provided for image classification, so that category information of the frame images is determined, and the technical problem that the image classification method for classifying after labeling the images is low in efficiency is solved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image classification method, apparatus, device, and storage medium.
Background
In the image processing technology, the image is usually manually marked first to classify the image, but if the image is manually marked first to classify the frame image in the same video, a great deal of manpower resources are wasted, and the image classification efficiency is not high.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide an image classification method, an image classification device and a storage medium, and aims to solve the technical problem that an image classification method for classifying images after labeling is low in efficiency in the prior art.
To achieve the above object, the present invention provides an image classification method comprising the steps of:
obtaining a frame image in a target video, and traversing the frame image to obtain a frame image to be classified;
performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
matching the target characteristic information with historical characteristic information in a preset database;
and determining the category information of the frame images to be classified according to the matching result.
Optionally, after extracting the features of the frame image to be classified by a preset first feature extraction model, the method further includes:
performing image processing on the images to be classified to obtain a contrast image;
performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information;
And updating the preset second feature extraction model according to the target feature information and the contrast feature information.
Optionally, the updating the preset second feature extraction model according to the target feature information and the contrast feature information includes:
determining a contrast loss parameter according to the target characteristic information and the contrast characteristic information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
Optionally, the determining a contrast loss parameter according to the target feature information and the contrast feature information includes:
obtaining similarity information of the target characteristic information and the contrast characteristic information;
and determining a contrast loss parameter according to the similarity information.
Optionally, after updating the preset contrast feature extraction parameter model according to the contrast loss parameter, the method further includes:
obtaining model parameters of an updated preset second feature extraction model;
and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
Optionally, the determining the category information of the frame image to be classified according to the matching result includes:
when the matching is successful, detecting whether the target video is consistent with a source video corresponding to the historical characteristic information;
If the target video is the same as the source video, acquiring category information of the historical characteristic information;
and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
Optionally, after detecting whether the target video is consistent with the source video corresponding to the history feature information when the matching is successful, the method further includes:
when the matching fails, generating corresponding target category information in a preset database according to the target characteristic information;
and updating the preset database according to the target category information.
Optionally, the updating the preset database according to the target category information includes:
acquiring a category number threshold value and the current category number in a preset database;
and when the current category number is not greater than the category number threshold, adding the target category information to the preset database to update the preset database.
Optionally, after obtaining the category information of the history feature information if the target video is the same as the source video, the method further includes:
if the target video is different from the source video, generating corresponding target category information in a preset database according to the characteristic information in the target video;
Updating the preset database, and executing the step of acquiring the category number threshold value and the current category number in the preset database.
Optionally, before the matching the target feature information with the history feature information in the preset database, the method further includes:
acquiring a historical video sample and a frame image type sample in the historical video;
and presetting a database according to the historical video sample and the frame image category sample component.
In addition, to achieve the above object, the present invention also proposes an image classification apparatus including:
the device comprises an image acquisition module, a feature extraction module, a feature matching module and a category determination module;
the image acquisition module is used for acquiring frame images in the target video for traversing to obtain frame images to be classified;
the feature extraction module is used for carrying out feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
the feature matching module is used for matching the target feature information with the historical feature information in a preset database;
and the category determining module is used for determining the category information of the frame images to be classified according to the matching result.
Optionally, the feature extraction module is further configured to perform image processing on the image to be classified to obtain a comparison image;
performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information;
and updating the preset second feature extraction model according to the target feature information and the contrast feature information.
Optionally, the feature extraction module is further configured to determine a contrast loss parameter according to the target feature information and the contrast feature information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
Optionally, the feature extraction module is further configured to obtain similarity information of the target feature information and the contrast feature information;
and determining a contrast loss parameter according to the similarity information.
Optionally, the feature extraction module is further configured to obtain model parameters of the updated preset second feature extraction model;
and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
Optionally, the category determining module is further configured to detect whether the target video is consistent with a source video corresponding to the historical feature information when the matching is successful;
If the target video is the same as the source video, acquiring category information of the historical characteristic information;
and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
Optionally, the category determining module is further configured to generate corresponding target category information in a preset database according to the target feature information when the matching fails;
and updating the preset database according to the target category information.
In addition, to achieve the above object, the present invention also proposes an image classification apparatus including: a memory, a processor, and an image classification program stored on the memory and executable on the processor, the image classification program configured to implement the steps of the image classification method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon an image classification program which, when executed by a processor, implements the steps of the image classification method as described above.
The invention discloses a method for acquiring frame images in a target video for traversing to acquire frame images to be classified; performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information; matching the target characteristic information with historical characteristic information in a preset database; determining the category information of the frame images to be classified according to the matching result; compared with the prior art, the method has the advantages that the characteristic extraction is carried out on the frame images to be classified of the target video through the preset first characteristic extraction model, different images can be distinguished so as to facilitate the subsequent image classification, the characteristic information corresponding to the frame images is matched with the historical characteristic information in the preset database, the basis is provided for image classification, the category information of the frame images is determined, and the technical problem that the image classification method for classifying after labeling the images is low in efficiency is solved.
Drawings
FIG. 1 is a schematic diagram of an image classification device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of an image classification method according to the present invention;
FIG. 3 is a schematic view of image features of an embodiment of an image classification method according to the present invention;
FIG. 4 is a flowchart of a second embodiment of an image classification method according to the present invention;
FIG. 5 is a flowchart of a third embodiment of an image classification method according to the present invention;
fig. 6 is a block diagram showing the structure of a first embodiment of the image classification apparatus according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic diagram of an image classification device of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the image classification apparatus may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 does not constitute a limitation of the image classification apparatus, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and an image classification program may be included in the memory 1005 as one type of storage medium.
In the image classification apparatus shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the image classification apparatus of the present invention may be provided in an image classification apparatus which calls an image classification program stored in the memory 1005 through the processor 1001 and performs the image classification method provided by the embodiment of the present invention.
An embodiment of the present invention provides an image classification method, referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of an image classification method according to the present invention.
In this embodiment, the image classification method includes the following steps:
step S10: and obtaining a frame image in the target video, and traversing to obtain the frame image to be classified.
It should be noted that, the execution subject of the method of the present embodiment may be an image classification device having functions of data processing, network communication and program running, such as a mobile phone, a computer, or other electronic devices capable of implementing the same or similar functions, which is not limited in this embodiment, and in the following embodiments, a control computer is exemplified.
It should be noted that the target video may be a source video used by the user for image classification; the frame image refers to a single still image picture with the smallest unit in one video, because the processing speed of human eyes for continuous images is limited, as long as the image refresh speed exceeds the recognition limiting speed within one second, the false image of the dynamic video is formed, so each single still image picture, namely, the frame image is continuously played, the dynamic video can be formed, and the frame image in one video is the same as the frame number of the video, for example: each shot in the movie is played quickly to form a dynamic video consisting of each shot.
It will be appreciated that the frame image to be classified may be any frame image in the target video, since the present embodiment classifies images in one video, i.e. it is required to classify each of the needle images that make up the video.
In a specific implementation, each frame image in the target video is queried, each frame image is output, and each output frame image is recorded as a frame image to be classified.
Step S20: and carrying out feature extraction on the frame images to be classified through a preset first feature extraction model to obtain target feature information.
It should be understood that the preset first feature extraction model is used for extracting feature information of the frame image to be classified, so that the judging of the category information of the frame image to be classified according to the feature information of the image and the feature information in the preset database is convenient to follow.
The preset first feature extraction model may be a convolutional neural network model (Convolutional Neural Networks, CNN), or may be another neural network model or algorithm with a feature extraction function for an image, which is not particularly limited in this embodiment, and in this embodiment, a convolutional neural network model is taken as an example of the preset first feature extraction model.
It is easy to understand that the target feature information refers to frame image feature information obtained after feature extraction of a frame image to be classified by a preset first feature extraction model, and is used for marking features of the image so as to facilitate feature distinction from other images, for example: in fig. 3, the feature information corresponding to the image a is square, the side length of the square is 10cm, the feature information corresponding to the image B is rectangular, the length and width of the rectangle are respectively 10cm and 5cm, and the two images can be classified by the feature information of the image a and the image B through a control computer to obtain two different types of image categories.
Step S30: and matching the target characteristic information with the historical characteristic information in a preset database.
It should be noted that, the preset database is used to store the category information and the feature information of the frame images of other videos, the category information and the corresponding category information and the feature information corresponding to the video frame images may be good samples trained before, the samples are stored to obtain a sample queue, or the database obtained by updating according to the image classification result before is not limited in this embodiment.
The historical characteristic information refers to characteristic information of frame images of other videos stored in a historical database, and in this embodiment, a preset sample storage amount corresponding to the database is limited, and if a sample is added after the sample upper limit stored in the database has been reached, a sample added earliest in the database needs to be deleted, so that a new sample can be successfully stored.
It can be understood that the samples stored in the preset database are classified according to each video, and in each video category, the corresponding frame images are classified, that is, different pictures of the same video belong to the same video category, but in the same video category, the frame images are divided into different categories, so that the problem of non-convergence caused by the fact that the intra-category difference is larger than the inter-category difference when similar images in the video are directly forced into different categories is prevented.
In specific implementation, after the frame image to be classified is determined, the frame image to be classified is subjected to feature extraction through a convolutional neural network model to obtain feature information of the frame image to be classified, the feature information of the frame image to be classified is matched with the feature information of other images stored in a preset database, if the matching is successful, the category information corresponding to the frame image to be classified can be determined according to the image category information corresponding to the history feature information which is successfully matched, if the matching is failed, the category of the frame image to be classified can be added in the database, the adding rule obeys first-in first-out, namely, the number of samples in the database reaches the upper storage limit of the database, and the samples which are added into the database at the earliest time are required to be deleted.
Step S40: and determining the category information of the frame images to be classified according to the matching result.
It should be noted that the category information refers to information obtained by classifying according to the feature information of different images, in general, the feature information obtained by extracting features of the same image through the same convolutional neural network is the same, and the feature information of similar images is also similar, and the category information of the images can be determined according to the similarity between the frame image to be classified and the images in the database.
The embodiment discloses that frame images in a target video are acquired and traversed to obtain frame images to be classified; performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information; matching the target characteristic information with historical characteristic information in a preset database; determining the category information of the frame images to be classified according to the matching result; according to the method, the frame images to be classified of the target video are subjected to feature extraction through the preset first feature extraction model, different images can be distinguished so as to facilitate subsequent image classification, feature information corresponding to the frame images is matched with historical feature information in the preset database, basis is provided for image classification, so that category information of the frame images is determined, and the technical problem that the image classification method for classifying after labeling the images is low in efficiency is solved.
Referring to fig. 4, fig. 4 is a flowchart illustrating a second embodiment of an image classification method according to the present invention.
Based on the first embodiment, in this embodiment, after step S20, the method further includes:
step S210: and performing image processing on the image to be classified to obtain a contrast image.
It should be noted that, the contrast image refers to an image obtained after the frame image to be classified is subjected to image change, and in this embodiment, the image processing may be performed by performing sharpness, brightness, blurring, and the like on the image, and the processing manner of the image is not particularly limited in this embodiment.
Step S220: and carrying out feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information.
It is worth to say that the preset second feature extraction model is used for extracting feature information of the comparison image so as to facilitate the subsequent comparison according to the feature information of the image and the feature information of the frame image to be classified, a comparison result is obtained, and parameter updating of the preset first feature extraction model and the preset second feature extraction model is achieved according to the comparison result, so that a better image feature extraction effect is obtained.
The preset second feature extraction model is consistent with the network structure of the preset first feature extraction model, and may be a convolutional neural network model (Convolutional Neural Networks, CNN), or may be another neural network model or algorithm with a feature extraction function for an image, which is not particularly limited in this embodiment.
In addition, in this embodiment, two kinds of image transformation may be performed on the frame image to be classified, so as to implement feature extraction according to different changed images, compare similarity of feature information of the two kinds of image transformation, obtain a comparison loss value of the two kinds of image transformation, and implement parameter updating of the preset second feature extraction model and the preset first feature extraction model.
Further, the step S220 includes:
determining a contrast loss parameter according to the target characteristic information and the contrast characteristic information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
It can be understood that the contrast loss parameter is used for adjusting parameter information of the preset second feature extraction model, so that the feature information extracted by the preset second feature extraction model has higher similarity with the feature information extracted by the preset first feature extraction model, and the accuracy of feature information extraction is realized.
Step S230: and updating the preset second feature extraction model according to the target feature information and the contrast feature information.
It should be noted that, the step of determining the contrast loss parameter according to the target feature information and the contrast feature information includes: obtaining similarity information of the target characteristic information and the contrast characteristic information; and determining a contrast loss parameter according to the similarity information.
After the updating of the preset contrast feature extraction parameter model according to the contrast loss parameter, the method further comprises: obtaining model parameters of an updated preset second feature extraction model; and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
It should be understood that, the updating of the parameters of the preset second feature extraction model may be achieved by the comparison loss value between the feature information obtained by respectively performing feature extraction on the two images by the two models, but when the parameters of the preset first feature extraction model are updated, the updated comparison loss value is combined with an equal ratio weight, in this embodiment, the equal ratio weight may be 0.9, which is not particularly limited in this embodiment; the formula for updating the preset first feature extraction model is as follows:
A`＝H×Q+A×(1-Q)
wherein A' is an updated preset first feature extraction model parameter, A is an original preset first feature extraction model parameter, H is a contrast loss value, and Q is an equal-ratio weight value.
For example: through calculating the contrast loss value between the two feature information to be 2, the parameter of the original preset second feature extraction model is 1, then the parameter of the updated preset second feature extraction model is 3, when the parameter of the preset first feature extraction model is updated, the contrast loss value is required to be multiplied by the preset weight to be 0.9, and then the parameter of the updated preset first feature extraction model is 1.1.
The embodiment discloses that the image to be classified is subjected to image processing to obtain a contrast image; performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information; updating the preset second feature extraction model according to the target feature information and the contrast feature information; according to the embodiment, the contrast loss value between the two images is detected through different characteristic information obtained through different processing of the same image, so that parameter updating of a characteristic extraction model is realized, more accurate characteristic extraction is realized, and the characteristic extraction efficiency is improved.
In this embodiment, before the step S30, the method further includes:
step S210': and acquiring a historical video sample and a frame image category sample in the historical video.
It should be noted that, the historical video sample may be a training sample before the component presets the database, or may be a source video corresponding to an image sample used for image classification.
It is readily understood that each frame image in the historical video sample is a sample of the frame image category of the historical video.
In a specific implementation, samples stored in a preset database are classified according to videos, in each video category, corresponding frame images are classified, namely different pictures of the same video belong to the same video category, but in the same video category, frame images are divided into different categories, so that the problem of non-convergence caused by the fact that intra-category differences are larger than inter-category differences when similar images in the video are directly forced into different categories is solved.
Step S220': and presetting a database according to the historical video sample and the frame image category sample component.
In this embodiment, the storage amount of the sample corresponding to the preset database is limited, and if a sample is to be added after the upper limit of the samples stored in the database has been reached, it is necessary to delete the sample added earliest in the database, so that the new sample can be stored successfully.
For example: the upper limit of the samples of the preset database is 2000, and 2000 samples are already in the preset database, if one sample needs to be added into the preset database, the sample of the database needs to be deleted by the earliest addition in the 2000 samples, and a sample adding space is reserved for sample addition.
The embodiment discloses obtaining a historical video sample and a frame image category sample in the historical video; presetting a database according to the historical video sample and the frame image category sample component; according to the embodiment, the data is continuously updated by determining the sample class adding rule of the database, so that the invalid data is prevented from occupying a large amount of storage space, and the resource utilization rate is improved.
Referring to fig. 5, fig. 5 is a flowchart of a third embodiment of an image classification method according to the present invention.
Based on the above second embodiment, in this embodiment, the step S40 includes:
step S401: and when the matching is successful, detecting whether the target video is consistent with the source video corresponding to the historical characteristic information.
It should be noted that, when the matching of the target feature information corresponding to the frame image to be classified and the historical feature information in the preset database is successful, that is, the similarity between the target feature information and the historical feature information exceeds the preset threshold, it indicates that the feature information of the frame image to be classified exists in the preset database, that is, the frame image to be classified may be similar to the category of one image in the database.
In this embodiment, the frame image to be classified and the successfully matched image may not belong to the same video, and may be forcedly belonging to different video classes, so that the situation of misclassification of the images may occur.
Further, after the step S401, the method further includes:
when the matching fails, generating corresponding target category information in a preset database according to the target characteristic information;
and updating the preset database according to the target category information.
It should be noted that, when matching of the target feature information corresponding to the frame image to be classified with the historical feature information in the preset database fails, it indicates that the feature information of the frame image to be classified does not exist in the preset database, and at this time, a new image category needs to be established in the database.
In addition, the storage amount of the sample corresponding to the preset database is an upper limit, if the upper limit of the sample stored in the database is reached, a sample added earliest in the database needs to be deleted to enable a new sample to be stored successfully, so that in order to establish a new image category in the database, a category number threshold value and a current category number in the preset database need to be acquired, and when the current category number is not greater than the category number threshold value, the target category information is added to the preset database to update the preset database.
It can be understood that the category number threshold value refers to a sample storage upper limit value of a preset database, and the current category number refers to the number of category samples already stored in the current database, wherein when the current category number is greater than the category number threshold value, the category information to be deleted is determined; and updating the preset database according to the target category information and the category information to be deleted.
In a specific implementation, if the number of the current categories is not greater than the threshold of the number of the categories, the redundant storage space is indicated in the preset database, the redundant storage space can be used for storing the newly established category information of the frame images to be classified, and the category information of the frame images to be classified can be added into the preset database for storage; if the number of the current categories is larger than the threshold value of the number of the categories, which means that no redundant storage space exists in the preset database, the earliest stored category information in the source preset database needs to be deleted, so that the category information corresponding to the frame images to be classified can be successfully stored.
Step S402: and if the target video is the same as the source video, acquiring category information of the historical characteristic information.
It should be understood that if the video to which the frame image to be classified belongs is the same as the source video of the matched image, i.e., the two videos are the same video, the frame image to be classified is an image belonging to the source video category in the database.
Further, after the step S402, the method further includes:
if the target video is different from the video source, generating corresponding target category information in a preset database according to the characteristic information in the target video;
updating the preset database, and executing the step of acquiring the category number threshold value and the current category number in the preset database.
In a specific implementation, if the target video is different from the source video, the frame image to be classified and the matched image do not necessarily belong to the same class, and should belong to two video classes.
Step S403: and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
It can be understood that the feature information similar to the target feature information is matched in the preset database, and the images corresponding to the feature information and the target feature information belong to the same video, so that the frame image to be classified and the matched image belong to the same class.
The embodiment discloses that when matching is successful, whether a target video is consistent with a source video corresponding to historical feature information or not is detected; if the target video is the same as the source video, acquiring category information of the historical characteristic information; determining the category information of the frame images to be classified according to the category information of the history characteristic information; according to the method and the device, whether the frame image to be classified and the image in the matched database are images in the same video or not is detected, so that the problem of non-convergence caused by the fact that intra-class differences are larger than inter-class differences when similar images in the same video are directly forced into different classes is prevented.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium stores an image classification program, and the image classification program realizes the steps of the image classification method when being executed by a processor.
Because the storage medium adopts all the technical schemes of all the embodiments, the storage medium has at least all the beneficial effects brought by the technical schemes of the embodiments, and the description is omitted here.
Referring to fig. 6, fig. 6 is a block diagram illustrating a first embodiment of an image classification apparatus according to the present invention.
As shown in fig. 6, an image classification apparatus according to an embodiment of the present invention includes: an image acquisition module 10, a feature extraction module 20, a feature matching module 30, and a category determination module 40;
the image acquisition module 10 is used for acquiring frame images in the target video for traversing to obtain frame images to be classified.
It should be noted that the target video may be a source video used by the user for image classification; the frame image refers to a single still image picture with the smallest unit in one video, and as the human naked eye has a certain limit on the processing speed of continuous images, as long as the image refresh speed exceeds the recognition limit speed within one second, the false image of the dynamic video is formed, so that each single still image picture, namely, the frame image, is continuously played, and the dynamic video can be formed, and the frame image in one video is the same as the frame number of the video, for example: each shot in the movie is played quickly to form a dynamic video consisting of each shot.
It will be appreciated that the frame image to be classified may be any frame image in the target video, since the present embodiment classifies images in one video, i.e. it is required to classify each of the needle images that make up the video.
In a specific implementation, each frame image in the target video is queried, each frame image is output, and each output frame image is recorded as a frame image to be classified.
The feature extraction module 20 is configured to perform feature extraction on the frame image to be classified through a preset first feature extraction model, so as to obtain target feature information.
It should be understood that the preset first feature extraction model is used for extracting feature information of the frame image to be classified, so that the judging of the category information of the frame image to be classified according to the feature information of the image and the feature information in the preset database is convenient to follow.
The preset first feature extraction model may be a convolutional neural network model (Convolutional Neural Networks, CNN), or may be another neural network model or algorithm with a feature extraction function for an image, which is not particularly limited in this embodiment, and in this embodiment, a convolutional neural network model is taken as an example of the preset first feature extraction model.
It is easy to understand that the target feature information refers to frame image feature information obtained after feature extraction of a frame image to be classified by a preset first feature extraction model, and is used for marking features of the image so as to facilitate feature distinction from other images, for example: in fig. 3, the feature information corresponding to the image a is square, the side length of the square is 10cm, the feature information corresponding to the image B is rectangular, the length and width of the rectangle are respectively 10cm and 5cm, and the two images can be classified by the feature information of the image a and the image B through a control computer to obtain two different types of image categories.
And the feature matching module 30 is used for matching the target feature information with the history feature information in a preset database.
It should be noted that, the preset database is used to store the category information and the feature information of the frame images of other videos, the category information and the corresponding category information and the feature information corresponding to the video frame images may be good samples trained before, the samples are stored to obtain a sample queue, or the database obtained by updating according to the image classification result before is not limited in this embodiment.
The historical characteristic information refers to characteristic information of frame images of other videos stored in a historical database, and in this embodiment, a preset sample storage amount corresponding to the database is limited, and if a sample is added after the sample upper limit stored in the database has been reached, a sample added earliest in the database needs to be deleted, so that a new sample can be successfully stored.
It can be understood that the samples stored in the preset database are classified according to each video, and in each video category, the corresponding frame images are classified, that is, different pictures of the same video belong to the same video category, but in the same video category, the frame images are divided into different categories, so that the problem of non-convergence caused by the fact that the intra-category difference is larger than the inter-category difference when similar images in the video are directly forced into different categories is prevented.
In specific implementation, after the frame image to be classified is determined, the frame image to be classified is subjected to feature extraction through a convolutional neural network model to obtain feature information of the frame image to be classified, the feature information of the frame image to be classified is matched with the feature information of other images stored in a preset database, if the matching is successful, the category information corresponding to the frame image to be classified can be determined according to the image category information corresponding to the history feature information which is successfully matched, if the matching is failed, the category of the frame image to be classified can be added in the database, the adding rule obeys first-in first-out, namely, the number of samples in the database reaches the upper storage limit of the database, and the samples which are added into the database at the earliest time are required to be deleted.
And the category determining module 40 is configured to determine category information of the frame image to be classified according to the matching result.
It should be noted that the category information refers to information obtained by classifying according to the feature information of different images, in general, the feature information obtained by extracting features of the same image through the same convolutional neural network is the same, and the feature information of similar images is also similar, and the category information of the images can be determined according to the similarity between the frame image to be classified and the images in the database.
The embodiment discloses that frame images in a target video are acquired and traversed to obtain frame images to be classified; performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information; matching the target characteristic information with historical characteristic information in a preset database; determining the category information of the frame images to be classified according to the matching result; according to the method, the frame images to be classified of the target video are subjected to feature extraction through the preset first feature extraction model, different images can be distinguished so as to facilitate subsequent image classification, feature information corresponding to the frame images is matched with historical feature information in the preset database, basis is provided for image classification, so that category information of the frame images is determined, and the technical problem that the image classification method for classifying after labeling the images is low in efficiency is solved.
It should be understood that the foregoing is illustrative only and is not limiting, and that in specific applications, those skilled in the art may set the invention as desired, and the invention is not limited thereto.
It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
In addition, technical details that are not described in detail in this embodiment may refer to the image classification method provided in any embodiment of the present invention, and are not described herein.
Furthermore, it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. Read Only Memory)/RAM, magnetic disk, optical disk) and including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
The invention discloses an image classification method A1, which comprises the following steps:
obtaining a frame image in a target video, and traversing the frame image to obtain a frame image to be classified;
performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
matching the target characteristic information with historical characteristic information in a preset database;
and determining the category information of the frame images to be classified according to the matching result.
A2, the image classification method according to A1, wherein the feature extraction is performed on the frame image to be classified through a preset first feature extraction model, and after target feature information is obtained, the method further comprises:
performing image processing on the images to be classified to obtain a contrast image;
performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information;
And updating the preset second feature extraction model according to the target feature information and the contrast feature information.
A3, the image classification method according to A2, wherein the updating the preset second feature extraction model according to the target feature information and the contrast feature information comprises the following steps:
determining a contrast loss parameter according to the target characteristic information and the contrast characteristic information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
A4, determining a contrast loss parameter according to the target feature information and the contrast feature information according to the image classification method as described in A3, wherein the method comprises the following steps:
obtaining similarity information of the target characteristic information and the contrast characteristic information;
and determining a contrast loss parameter according to the similarity information.
A5, the image classification method according to A3, wherein after updating the preset contrast feature extraction parameter model according to the contrast loss parameter, further comprises:
obtaining model parameters of an updated preset second feature extraction model;
and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
A6, determining the category information of the frame image to be classified according to the matching result according to the image classification method as described in A1, wherein the method comprises the following steps:
When the matching is successful, detecting whether the target video is consistent with a source video corresponding to the historical characteristic information;
if the target video is the same as the source video, acquiring category information of the historical characteristic information;
and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
A7, the image classification method according to A6, after detecting whether the target video is consistent with the source video corresponding to the history feature information when the matching is successful, further comprises:
when the matching fails, generating corresponding target category information in a preset database according to the target characteristic information;
and updating the preset database according to the target category information.
A8, the image classification method according to A7, wherein the updating the preset database according to the target category information comprises the following steps:
acquiring a category number threshold value and the current category number in a preset database;
and when the current category number is not greater than the category number threshold, adding the target category information to the preset database to update the preset database.
A9, the image classification method according to A8, after obtaining the threshold number of categories and the number of current categories in the preset database, further includes:
When the number of the current categories is larger than the threshold value of the number of the categories, determining category information to be deleted;
and updating the preset database according to the target category information and the category information to be deleted.
A10, the image classification method according to A6, wherein if the target video is the same as the source video, the method further comprises, after obtaining the category information of the history feature information:
if the target video is different from the source video, generating corresponding target category information in a preset database according to the characteristic information in the target video;
updating the preset database, and executing the step of acquiring the category number threshold value and the current category number in the preset database.
A11, the image classification method according to A1, before the matching the target feature information with the history feature information in the preset database, further includes:
acquiring a historical video sample and a frame image type sample in the historical video;
and presetting a database according to the historical video sample and the frame image category sample component.
The invention also discloses a B12 and an image classification device, wherein the image classification device comprises: the device comprises an image acquisition module, a feature extraction module, a feature matching module and a category determination module;
The image acquisition module is used for acquiring frame images in the target video for traversing to obtain frame images to be classified;
the feature extraction module is used for carrying out feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
the feature matching module is used for matching the target feature information with the historical feature information in a preset database;
and the category determining module is used for determining the category information of the frame images to be classified according to the matching result.
B13, the image classification device as described in B12, wherein the feature extraction module is further configured to perform image processing on the image to be classified to obtain a comparison image;
performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information;
and updating the preset second feature extraction model according to the target feature information and the contrast feature information.
The image classification device as in B13, wherein the feature extraction module is further configured to determine a contrast loss parameter according to the target feature information and the contrast feature information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
The image classification device as described in B15, wherein the feature extraction module is further configured to obtain similarity information between the target feature information and the contrast feature information;
and determining a contrast loss parameter according to the similarity information.
The image classification device as in B16, wherein the feature extraction module is further configured to obtain model parameters of the updated preset second feature extraction model;
and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
The image classification device as in B17, where the class determination module is further configured to detect whether the target video is consistent with a source video corresponding to the historical feature information when the matching is successful;
if the target video is the same as the source video, acquiring category information of the historical characteristic information;
and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
B18, the image classification device as described in B17, wherein the category determination module is further configured to generate corresponding target category information in a preset database according to the target feature information when the matching fails;
and updating the preset database according to the target category information.
The invention also discloses C19, an image classification device, the image classification device includes: a memory, a processor, and an image classification program stored on the memory and executable on the processor, the image classification program configured to implement the image classification method as described above.
The invention also discloses D20, a storage medium, wherein the storage medium stores an image classification program, and the image classification program realizes the image classification method when being executed by a processor.
Claims (10)
1. An image classification method, characterized in that the image classification method comprises:
obtaining a frame image in a target video, and traversing the frame image to obtain a frame image to be classified;
performing feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
matching the target characteristic information with historical characteristic information in a preset database;
and determining the category information of the frame images to be classified according to the matching result.
2. The method for classifying images according to claim 1, wherein the step of extracting features of the frame image to be classified by a preset first feature extraction model to obtain target feature information, further comprises:
Performing image processing on the images to be classified to obtain a contrast image;
performing feature extraction on the comparison image through a preset second feature extraction model to obtain comparison feature information;
and updating the preset second feature extraction model according to the target feature information and the contrast feature information.
3. The image classification method of claim 2, wherein updating the preset second feature extraction model according to the target feature information and the contrast feature information comprises:
determining a contrast loss parameter according to the target characteristic information and the contrast characteristic information;
and updating the preset second feature extraction parameter model according to the contrast loss parameters.
4. The image classification method of claim 3, wherein said determining a contrast loss parameter based on said target feature information and said contrast feature information comprises:
obtaining similarity information of the target characteristic information and the contrast characteristic information;
and determining a contrast loss parameter according to the similarity information.
5. The image classification method of claim 3, further comprising, after updating the preset contrast feature extraction parameter model according to the contrast loss parameter:
Obtaining model parameters of an updated preset second feature extraction model;
and updating the preset first feature extraction model according to the model parameters and a preset parameter updating strategy.
6. The image classification method according to claim 1, wherein the determining the class information of the frame image to be classified according to the matching result includes:
when the matching is successful, detecting whether the target video is consistent with a source video corresponding to the historical characteristic information;
if the target video is the same as the source video, acquiring category information of the historical characteristic information;
and determining the category information of the frame images to be classified according to the category information of the historical characteristic information.
7. The image classification method according to claim 6, wherein after detecting whether the target video is consistent with the source video corresponding to the history feature information when the matching is successful, further comprising:
when the matching fails, generating corresponding target category information in a preset database according to the target characteristic information;
and updating the preset database according to the target category information.
8. An image classification apparatus, characterized in that the image classification apparatus comprises: the device comprises an image acquisition module, a feature extraction module, a feature matching module and a category determination module;
The image acquisition module is used for acquiring frame images in the target video for traversing to obtain frame images to be classified;
the feature extraction module is used for carrying out feature extraction on the frame image to be classified through a preset first feature extraction model to obtain target feature information;
the feature matching module is used for matching the target feature information with the historical feature information in a preset database;
and the category determining module is used for determining the category information of the frame images to be classified according to the matching result.
9. An image classification apparatus, characterized in that the image classification apparatus comprises: a memory, a processor, and an image classification program stored on the memory and executable on the processor, the image classification program configured to implement the image classification method of any one of claims 1 to 7.
10. A storage medium having stored thereon an image classification program which, when executed by a processor, implements the image classification method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111618688.6A CN116363407A (en) | 2021-12-27 | 2021-12-27 | Image classification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111618688.6A CN116363407A (en) | 2021-12-27 | 2021-12-27 | Image classification method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116363407A true CN116363407A (en) | 2023-06-30 |
Family
ID=86910341
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111618688.6A Pending CN116363407A (en) | 2021-12-27 | 2021-12-27 | Image classification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116363407A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117671784A (en) * | 2023-12-04 | 2024-03-08 | 北京中航智信建设工程有限公司 | Human behavior analysis method and system based on video analysis |
-
2021
- 2021-12-27 CN CN202111618688.6A patent/CN116363407A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117671784A (en) * | 2023-12-04 | 2024-03-08 | 北京中航智信建设工程有限公司 | Human behavior analysis method and system based on video analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8750573B2 (en) | Hand gesture detection | |
| CN112348117B (en) | Scene recognition method, device, computer equipment and storage medium | |
| JP2020515983A (en) | Target person search method and device, device, program product and medium | |
| US20120027252A1 (en) | Hand gesture detection | |
| CN110602527A (en) | Video processing method, device and storage medium | |
| CN110136198B (en) | Image processing method, apparatus, device and storage medium thereof | |
| CN112348089B (en) | Working state identification method, server, storage medium and device | |
| CN111553302B (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| US11694331B2 (en) | Capture and storage of magnified images | |
| CN112381092B (en) | Tracking method, tracking device and computer readable storage medium | |
| CN111553838A (en) | Model parameter updating method, device, equipment and storage medium | |
| CN113657370B (en) | Character recognition method and related equipment thereof | |
| CN113222921A (en) | Image processing method and system | |
| CN112613508A (en) | Object identification method, device and equipment | |
| CN116363407A (en) | Image classification method, device, equipment and storage medium | |
| CN115240203A (en) | Service data processing method, device, equipment and storage medium | |
| CN110458202B (en) | Picture processing method, device and equipment and computer readable storage medium | |
| CN114255493A (en) | Image detection method, face detection device, face detection equipment and storage medium | |
| CN112948630B (en) | List updating method, electronic equipment, storage medium and device | |
| CN113657369B (en) | Character recognition method and related equipment thereof | |
| CN114677578A (en) | Method and device for determining training sample data | |
| CN110348898B (en) | Information pushing method and device based on human body identification | |
| US9092694B2 (en) | Method and system for creating a digital image album | |
| CN113361426A (en) | Vehicle loss assessment image acquisition method, medium, device and electronic equipment | |
| CN111797922A (en) | Text image classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |