CN112560970A

CN112560970A - Abnormal picture detection method, system, equipment and storage medium based on self-coding

Info

Publication number: CN112560970A
Application number: CN202011517572.9A
Authority: CN
Inventors: 翟步中
Original assignee: Shanghai Minglue Artificial Intelligence Group Co Ltd
Current assignee: Shanghai Minglue Artificial Intelligence Group Co Ltd
Priority date: 2020-12-21
Filing date: 2020-12-21
Publication date: 2021-03-26

Abstract

The application discloses a method, a system, equipment and a storage medium for detecting abnormal pictures based on self-encoding, wherein the method comprises the following steps: inputting an original picture into a recognition model to obtain a first class confidence coefficient; judging whether the first category confidence exceeds a first threshold value; if so, acquiring the category of the original picture according to the first category confidence; if not, the original picture is converted and further judged. By the method and the device, the category of the picture can be systematically judged, the accuracy of identifying the model is improved, and the abnormal picture can be effectively identified.

Description

Abnormal picture detection method, system, equipment and storage medium based on self-coding

Technical Field

The invention relates to the technical field of image processing. More particularly, the present invention relates to a method, system, device and storage medium for detecting abnormal pictures based on self-encoding.

Background

At present, the depth recognition model deployed in the production environment often encounters the situation that the pictures sent into the model exceed the recognizable range of the model, that is, theoretically, the model cannot judge which class the pictures belong to, and such pictures are called abnormal pictures. The main reason for this is that the model runs in an unsupervised environment and the picture does not know whether it belongs to a known category before it is fed into the model.

For the above situation, two ways are generally used for identifying the abnormal picture:

1. after the model judges the type of the picture, generating confidence coefficient which is judged as each known type, sequencing the confidence coefficient, and judging that the corresponding type is the type of the picture if the highest value of the confidence coefficient is higher than a threshold value set by a system;

2. and adding data and categories, and changing the abnormal pictures into recognizable categories.

However, the above two methods still have the following problems:

1. the first approach, which relies strictly on the setting of the category threshold, leads to the following problems: if the threshold is set too high, the accuracy is high, but the recall rate is reduced, namely more pictures of the same type are judged to be abnormal pictures because the pictures do not reach the threshold; if the threshold value is lowered, the recall rate is increased, but the accuracy rate is reduced, namely, a plurality of abnormal pictures are judged wrongly;

2. the second method generally reduces the effect of the abnormal picture by adding data and increasing the class ratio, but in some production environments, it is difficult to avoid the effect of the abnormal picture by amplifying data and amplifying class;

3. the two modes have narrow application range, for example, in the field of identifying fast-moving products, a lot of new products and new packages are available on the market every day, the market share of the new products on the market is not high, the sample collection is difficult, and meanwhile, the new packages are often similar to old packages or similar to other friend products, so that the two modes for judging the abnormal pictures fail.

Disclosure of Invention

The embodiment of the application provides an abnormal picture detection method based on self-coding and solves the problem of subjective factor influence in the related technology.

The invention provides an abnormal picture detection method based on self-encoding, which comprises the following steps:

a first confidence degree obtaining step: inputting an original picture into a recognition model to obtain a first class confidence coefficient;

a first threshold value judging step: judging whether the first category confidence exceeds a first threshold value;

a first category acquisition step: if so, acquiring the category of the original picture according to the first category confidence;

a conversion step: if not, the original picture is converted and further judged.

As a further improvement of the present invention, the first threshold determining step specifically includes the following steps:

a first sequencing step: sequencing the first class confidence coefficients to obtain the maximum value of the first class confidence coefficients;

a first judgment step: and judging whether the maximum value in the first category confidence exceeds the first threshold value.

As a further improvement of the present invention, the converting step specifically includes the steps of:

a picture conversion step: inputting the original picture into a self-coding model to obtain a converted picture;

a second confidence degree obtaining step: inputting the converted picture into the recognition model to obtain a second category confidence coefficient;

a second threshold value judging step: judging whether the confidence of the second category exceeds a second threshold value;

a second category acquisition step: if so, acquiring the category of the original picture according to the second category confidence;

an abnormality determination step: and if not, judging the original picture to be an abnormal picture.

As a further improvement of the present invention, the second threshold value determining step specifically includes the following steps:

a second sorting step: sequencing the second category confidence degrees to obtain the maximum value in the second category confidence degrees;

a second judgment step: and judging whether the maximum value in the second category confidence exceeds the second threshold value.

Based on the same invention idea, the invention also discloses an abnormal picture detection method based on self-coding based on any invention creation, discloses an abnormal picture detection system based on self-coding,

the abnormal picture detection system based on self-coding comprises:

the first confidence coefficient acquisition module is used for inputting the original picture into the recognition model to acquire a first category confidence coefficient;

the first threshold judging module is used for judging whether the first class confidence coefficient exceeds a first threshold;

the first type obtaining module is used for obtaining the type of the original picture according to the first type confidence coefficient if the original picture is the original picture;

and if not, converting the original picture for further judgment.

As a further improvement of the present invention, the first threshold determining module specifically includes:

the first sequencing unit is used for sequencing the first class confidence coefficients to obtain the maximum value of the first class confidence coefficients;

and the first judging unit is used for judging whether the maximum value in the first class confidence exceeds the first threshold.

As a further improvement of the present invention, the conversion module specifically includes:

the picture conversion unit is used for inputting the original picture into a coding model to obtain a converted picture;

a second confidence acquisition unit which inputs the converted picture into the recognition model to acquire a second category confidence;

a second threshold value judging unit that judges whether the second category confidence exceeds a second threshold value;

a second category obtaining unit, configured to obtain a category of the original picture according to the second category confidence if the original picture is the original picture;

and an abnormal judging unit, if not, judging the original picture as an abnormal picture.

As a further improvement of the present invention, the second threshold value determining unit specifically includes:

the second sorting unit is used for sorting the second category confidence coefficients to obtain the maximum value in the second category confidence coefficients;

a second determination unit that determines whether or not a maximum value of the second category confidence exceeds the second threshold.

In addition, to achieve the above object, the present invention further provides an apparatus, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements a self-encoding based abnormal picture detection method when executing the computer program.

Further, to achieve the above object, the present invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements a self-encoding based abnormal picture detection method.

Compared with the prior art, the invention has the following beneficial effects:

1. the abnormal picture detection method based on self-coding is provided, a self-coding model is used for converting a picture which is possibly abnormal, and the problem that the accuracy rate of identifying the abnormal picture by a deep learning model is low is solved;

2. the accuracy of the recognition model is improved by setting two thresholds;

3. the additional cost of the self-coding model is low, including the labor cost for training the self-coding model and the time cost for using the self-coding model in a production environment;

4. by adopting an unsupervised training process, training data does not need to be marked additionally, and in a production environment, only pictures with uncertain recognition models can be sent into the self-coding model for conversion, so that the conversion process and the recognition process can be realized in parallel, and the working efficiency is greatly improved.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a self-coding model provided by an embodiment of the present invention;

fig. 2 is a flowchart of an overall method for detecting an abnormal picture based on self-encoding according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating the abnormal picture detection scheme according to the present embodiment;

FIG. 4 is a flowchart illustrating the overall process of step S2 disclosed in FIG. 2;

FIG. 5 is a flowchart illustrating the whole step S4 disclosed in FIG. 2;

FIG. 6 is a flowchart illustrating the whole step S43 disclosed in FIG. 5;

fig. 7 is a structural frame diagram of an abnormal picture detection system based on self-encoding according to this embodiment;

fig. 8 is a block diagram of a computer device according to an embodiment of the present invention.

In the above figures:

10. a first confidence acquisition module; 20. a first threshold judgment module; 30. a first category acquisition module; 40. a conversion module; 21. a first sequencing unit; 22. a first judgment unit; 41. a picture conversion unit; 42. a second confidence acquisition unit; 43. a second threshold value judging unit; 44. a second category acquisition unit; 45. an abnormality determination unit; 431. a second sorting unit; 432. a second judgment unit 80, a bus; 81. a processor; 82. a memory; 83. a communication interface.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.

It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference to the terms "first," "second," "third," and the like in this application merely distinguishes similar objects and is not to be construed as referring to a particular ordering of objects.

The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and those skilled in the art should understand that the functional, methodological, or structural equivalents of these embodiments or alternatives thereof fall within the scope of the present invention.

Before describing in detail the various embodiments of the present invention, the core inventive concepts of the present invention are summarized and described in detail by the following several embodiments.

The method can realize abnormal picture detection based on the self-coding model, and improves the detection accuracy of the recognition model.

The first embodiment is as follows:

referring to fig. 1 to 6, the present example discloses a specific implementation of an abnormal picture detection method (hereinafter referred to as "method") based on self-encoding.

In particular, the method is performed based on a self-coding model. Fig. 1 is a schematic diagram of a self-coding model, and the principle of the self-coding model is explained as follows: an unsupervised self-encoding model is trained using a training data set identifying the model, the training process requiring that the output and input of the model be as similar as possible, and an auto-encoder (auto-encoder) is learned using the training data. The model obtained with reference to fig. 1 should have such a variation in the test set: the output of a normal picture (picture of the target data set, normal cola in the figure) after the self-coding model is very similar to the original picture; the output of an abnormal picture (a picture in the non-target dataset, in the illustration a packaged lipstick looking like a cola) after the self-encoding model is very different from the original picture.

Specifically referring to fig. 2 and 3, the method disclosed in this embodiment mainly includes the following steps:

and step S1, inputting the original picture into the recognition model, and acquiring a first category confidence.

Specifically, in some embodiments, the original image is input into the recognition model, and then the confidence level of each category (classification confidence level one) is obtained.

Then, step S2 is executed to determine whether the first category confidence exceeds a first threshold.

Specifically, in some embodiments, referring to fig. 4, step S2 specifically includes the following steps:

s21, sequencing the first category confidence coefficients to obtain the maximum value of the first category confidence coefficients;

and S22, judging whether the maximum value in the first category confidence exceeds the first threshold.

Specifically, if the first category confidence exceeds a first threshold, step S3 is executed, and if yes, the category of the original picture is obtained according to the first category confidence.

Specifically, in some embodiments, after sorting the first classification confidence levels, if the confidence level is the highest and exceeds the set first threshold (threshold one), the classification result may be obtained according to the highest confidence level in the first classification confidence levels.

Specifically, if the first category confidence does not exceed the first threshold, step S4 is executed, otherwise, the original picture is converted for further determination.

Specifically, in some embodiments, referring to fig. 5, step S4 specifically includes the following steps:

s41, inputting the original picture into a coding model to obtain a converted picture;

s42, inputting the converted picture into the recognition model to obtain a second category confidence;

s43, judging whether the confidence of the second category exceeds a second threshold value;

s44, if yes, acquiring the type of the original picture according to the second type confidence coefficient;

and S45, if not, judging the original picture to be an abnormal picture.

Specifically, in some embodiments, referring to fig. 6, step S43 specifically includes the following steps:

s431, sequencing the second category confidence coefficients to obtain the maximum value in the second category confidence coefficients;

s432, judging whether the maximum value in the second category confidence exceeds the second threshold value.

Specifically, in some embodiments, if the first class confidence does not exceed the first threshold, the original picture is sent to a self-coding model to obtain a converted picture, and then the converted picture is sent to an identification model to obtain a second class confidence (second classification confidence), if the class with the highest second middle confidence of the classification confidence exceeds the second threshold (second threshold), the corresponding class is determined, and if the class with the highest second middle confidence of the classification confidence is lower than the second threshold, the abnormal picture is determined.

According to the abnormal picture detection method based on self-coding disclosed by the embodiment of the application, the self-coding model is utilized to convert the possibly abnormal picture, so that the problem of low accuracy rate of identifying the abnormal picture by the deep learning model is solved; the accuracy of the recognition model is improved by setting two thresholds; the additional cost of the self-coding model is low, including the labor cost for training the self-coding model and the time cost for using the self-coding model in a production environment; by adopting an unsupervised training process, training data does not need to be marked additionally, and in a production environment, only pictures with uncertain recognition models can be sent into the self-coding model for conversion, so that the conversion process and the recognition process can be realized in parallel, and the working efficiency is greatly improved.

Example two:

in combination with the method for detecting an abnormal picture based on self-encoding disclosed in the first embodiment, the present embodiment discloses a specific implementation example of an abnormal picture detection system (hereinafter referred to as "system") based on self-encoding.

Referring to fig. 7, the system includes:

the first confidence coefficient obtaining module 10 is used for inputting the original picture into the recognition model to obtain a first category confidence coefficient;

a first threshold judgment module 20, which judges whether the first class confidence exceeds a first threshold;

a first category obtaining module 30, if yes, obtaining the category of the original picture according to the first category confidence;

and if not, the conversion module 40 performs conversion and further judgment on the original picture.

Specifically, in some embodiments, the first threshold determining module 20 specifically includes:

the first sorting unit 21 sorts the first category confidence coefficients to obtain a maximum value of the first category confidence coefficients;

the first determination unit 22 determines whether or not the maximum value of the first class confidence exceeds the first threshold.

Specifically, in some embodiments, the conversion module 40 specifically includes:

a picture conversion unit 41, which inputs the original picture from a coding model to obtain a converted picture;

a second confidence acquiring unit 42, which inputs the converted picture into the recognition model to acquire a second category confidence;

a second threshold value judging unit 43 that judges whether or not the second category confidence exceeds a second threshold value;

a second category obtaining unit 44, if yes, obtaining the category of the original picture according to the second category confidence;

and an abnormal judgment unit 45, if not, judging that the original picture is an abnormal picture.

Specifically, in some embodiments, the second threshold determining unit 43 specifically includes:

a second sorting unit 431, which sorts the second category confidence coefficients to obtain a maximum value of the second category confidence coefficients;

the second determination unit 432 determines whether or not the maximum value of the second class confidence exceeds the second threshold.

Please refer to the description of the first embodiment, which will not be repeated herein.

Example three:

referring to FIG. 8, this embodiment discloses an embodiment of a computer device. The computer device may comprise a processor 81 and a memory 82 in which computer program instructions are stored.

Specifically, the processor 81 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.

Memory 82 may include, among other things, mass storage for data or instructions. By way of example, and not limitation, memory 82 may include a Hard Disk Drive (Hard Disk Drive, abbreviated to HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 82 may include removable or non-removable (or fixed) media, where appropriate. The memory 82 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 82 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, Memory 82 includes Read-Only Memory (ROM) and Random Access Memory (RAM). The ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), Electrically rewritable ROM (EAROM), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.

The memory 82 may be used to store or cache various data files for processing and/or communication use, as well as possible computer program instructions executed by the processor 81.

The processor 81 reads and executes the computer program instructions stored in the memory 82 to implement any one of the self-encoding based abnormal picture detection methods in the above-described embodiments.

In some of these embodiments, the computer device may also include a communication interface 83 and a bus 80. As shown in fig. 8, the processor 81, the memory 82, and the communication interface 83 are connected via the bus 80 to complete communication therebetween.

The communication interface 83 is used for implementing communication between modules, devices, units and/or equipment in the embodiment of the present application. The communication port 83 may also be implemented with other components such as: the data communication is carried out among external equipment, image/data acquisition equipment, a database, external storage, an image/data processing workstation and the like.

Bus 80 includes hardware, software, or both to couple the components of the computer device to each other. Bus 80 includes, but is not limited to, at least one of the following: data Bus (Data Bus), Address Bus (Address Bus), Control Bus (Control Bus), Expansion Bus (Expansion Bus), and Local Bus (Local Bus). By way of example, and not limitation, Bus 80 may include an Accelerated Graphics Port (AGP) or other Graphics Bus, an Enhanced Industry Standard Architecture (EISA) Bus, a Front-Side Bus (FSB), a Hyper Transport (HT) Interconnect, an ISA (ISA) Bus, an InfiniBand (InfiniBand) Interconnect, a Low Pin Count (LPC) Bus, a memory Bus, a microchannel Architecture (MCA) Bus, a PCI (Peripheral Component Interconnect) Bus, a PCI-Express (PCI-X) Bus, a Serial Advanced Technology Attachment (SATA) Bus, a Video Electronics Bus (audio Electronics Association), abbreviated VLB) bus or other suitable bus or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although specific buses are described and shown in the embodiments of the application, any suitable buses or interconnects are contemplated by the application.

The computer device may implement the abnormal picture detection based on self-encoding, thereby implementing the method described in connection with fig. 1.

In addition, in combination with the abnormal picture detection method based on self-encoding in the foregoing embodiments, the embodiments of the present application may provide a computer-readable storage medium to implement the method. The computer readable storage medium having stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement any one of the above embodiments of the self-encoding based abnormal picture detection method.

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

In summary, the method for detecting the abnormal picture based on the self-coding has the advantages that the self-coding model is used for converting the picture which is possibly abnormal, so that the problem that the accuracy rate of the deep learning model for identifying the abnormal picture is low is solved; the accuracy of the recognition model is improved by setting two thresholds; the additional cost of the self-coding model is low, including the labor cost for training the self-coding model and the time cost for using the self-coding model in a production environment; by adopting an unsupervised training process, training data does not need to be marked additionally, and in a production environment, only pictures with uncertain recognition models can be sent into the self-coding model for conversion, so that the conversion process and the recognition process can be realized in parallel, and the working efficiency is greatly improved.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. An abnormal picture detection method based on self-encoding is characterized by comprising the following steps:

2. The method according to claim 1, wherein the first threshold determination step specifically comprises the following steps:

3. The self-coding based abnormal picture detection method according to claim 1, wherein the converting step specifically comprises the steps of:

4. The method according to claim 3, wherein the second threshold determination step specifically comprises the following steps:

5. An abnormal picture detection system based on self-coding, comprising:

and if not, converting the original picture for further judgment.

6. The system according to claim 5, wherein the first threshold determination module specifically comprises:

7. The self-encoding-based abnormal picture detection system according to claim 5, wherein the conversion module specifically comprises:

8. The system according to claim 7, wherein the second threshold determining unit specifically comprises:

9. An apparatus comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the self-encoding based exceptional picture detection method as claimed in any one of claims 1 to 4 when executing the computer program.

10. A storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the self-encoding based abnormal picture detection method according to any one of claims 1 to 4.