CN118052998A

CN118052998A - Feature processing method, device, apparatus, storage medium, and computer program product

Info

Publication number: CN118052998A
Application number: CN202410455621.2A
Authority: CN
Inventors: 胡健龙; 陈旭; 彭瑾龙; 张江宁; 甘振业
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2024-04-16
Filing date: 2024-04-16
Publication date: 2024-05-17
Anticipated expiration: 2044-04-16
Also published as: CN118052998B

Abstract

The application provides a feature processing method, a device, equipment, a storage medium and a computer program product, relating to the artificial intelligence technology; the method comprises the following steps: extracting abnormal sample characteristics of each of a plurality of abnormal image samples; the abnormal image sample belongs to a target object, and when the target object is detected based on the abnormal image sample, the obtained detection result represents that the target object has an abnormality; clustering the abnormal sample features to obtain at least one feature cluster; noise is added to a first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain a second abnormal sample feature corresponding to each first abnormal sample feature, the second abnormal sample feature is used for training an abnormal detection model, and the abnormal detection model is used for detecting whether an object in an image to be detected is abnormal or not. According to the application, the accuracy of the generated abnormal sample characteristics can be improved.

Description

Feature processing method, device, apparatus, storage medium, and computer program product

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a feature processing method, apparatus, device, storage medium, and computer program product.

Background

With the development of artificial intelligence technology, the artificial intelligence technology is gradually applied to an abnormality detection scene of an object, for example, a machine learning model is adopted to realize abnormality detection of the object. While training of machine learning models often requires a large number of outlier samples, outlier samples in real-world scenarios tend to be limited. Thus, an abnormal sample can be constructed. In the related art, the construction mode of the abnormal sample is mostly implemented on the pixel level, for example, some random images and normal samples are used for synthesizing the abnormal sample on the pixel level, but the construction mode is generally complicated in steps, and the constructed abnormal sample has a larger difference from the real abnormal sample.

Disclosure of Invention

The embodiment of the application provides a feature processing method, a device, equipment, a storage medium and a computer program product, which can improve the accuracy of generated abnormal sample features.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a feature processing method, which comprises the following steps:

Extracting abnormal sample characteristics of each of a plurality of abnormal image samples;

the abnormal image sample belongs to a target object, and when the target object is detected based on the abnormal image sample, the obtained detection result represents that the target object has an abnormality;

Clustering the abnormal sample features to obtain at least one feature cluster;

Noise is added to a first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain a second abnormal sample feature corresponding to each first abnormal sample feature, the second abnormal sample feature is used for training an abnormal detection model, and the abnormal detection model is used for detecting whether an object in an image to be detected is abnormal or not.

The embodiment of the application also provides a feature processing device, which comprises:

The extraction module is used for extracting the abnormal sample characteristics of each of the plurality of abnormal image samples;

the clustering module is used for carrying out clustering processing on the abnormal sample characteristics to obtain at least one characteristic cluster;

the noise adding module is used for adding noise to first abnormal sample characteristics corresponding to the cluster center of each characteristic cluster to obtain second abnormal sample characteristics corresponding to each first abnormal sample characteristic, the second abnormal sample characteristics are used for training an abnormal detection model, and the abnormal detection model is used for detecting whether an object in an image to be detected has an abnormality or not.

In the above scheme, the noise adding module is further configured to generate at least one first noise before adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature, where each first noise is different; the noise adding module is further configured to perform the following processing for each of the feature clusters: and respectively adding the first noise to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain at least one second abnormal sample feature corresponding to the first abnormal sample feature.

In the above scheme, the noise adding module is further configured to sample the data according with the first distribution at least once to obtain at least one sampling result, and take each sampling result as each first noise, where each sampling result is different; or acquiring data conforming to at least one second distribution, and sampling the data conforming to the second distribution to obtain the first noise for each second distribution, wherein the second distributions are different.

In the above scheme, the noise adding module is further configured to generate second noise corresponding to each feature cluster before adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain second abnormal sample feature corresponding to each first abnormal sample feature; the noise adding module is further configured to perform the following processing for each of the feature clusters: and adding second noise corresponding to the feature cluster to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain a second abnormal sample feature corresponding to the first abnormal sample feature.

In the above solution, the noise adding module is further configured to perform, for each of the feature clusters, the following processing: determining the feature distribution of the feature cluster; generating data conforming to the feature distribution of the feature cluster; and sampling from the data conforming to the characteristic distribution of the characteristic cluster to obtain second noise corresponding to the characteristic cluster.

In the above scheme, the noise adding module is further configured to add noise to a first abnormal sample feature corresponding to a cluster center of each feature cluster, and obtain a second abnormal sample feature corresponding to each first abnormal sample feature, and then obtain an exogenous image, where the exogenous image does not include the target object; extracting exogenous image characteristics of the exogenous image; one of the following steps is performed: fusing the second abnormal sample feature and the exogenous image feature to obtain a third abnormal sample feature; and determining the exogenous image characteristic as a third abnormal sample characteristic.

In the above scheme, the noise adding module is further configured to add noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster, obtain the second abnormal sample feature corresponding to each first abnormal sample feature, extract a normal sample feature of a normal image sample, and generate third noise; the normal image sample belongs to a target object, and when the target object is detected based on the normal image sample, the obtained detection result represents that the target object is normal; calling an abnormal characteristic generating model to generate a fourth abnormal sample characteristic based on the normal sample characteristic and the third noise; and fusing the second abnormal sample characteristic and the fourth abnormal sample characteristic to obtain a fifth abnormal sample characteristic.

In the above scheme, the noise adding module is further configured to obtain an initial abnormal feature generating model and a first normal sample feature of the first normal image sample, and generate fourth noise; invoking the initial abnormal feature generation model to generate a sixth abnormal sample feature based on the first normal sample feature and the fourth noise; invoking a feature discrimination model to discriminate the sixth abnormal sample feature to obtain a feature discrimination result, wherein the feature discrimination result indicates the possible degree of the sixth abnormal sample feature being a true abnormal sample feature; determining a value of a loss function of the initial abnormal feature generation model based on the real abnormal sample feature and the feature discrimination result; and training the initial abnormal feature generation model based on the value of the loss function to obtain the abnormal feature generation model.

In the above solution, the noise adding module is further configured to obtain a first weight of the second abnormal sample feature, and obtain a second weight of the fourth abnormal sample feature; and weighting the second abnormal sample feature and the fourth abnormal sample feature based on the first weight and the second weight to obtain the fifth abnormal sample feature.

In the above scheme, the noise adding module is further configured to add noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster, obtain second abnormal sample features corresponding to each first abnormal sample feature, and score a plurality of the second abnormal sample features respectively to obtain scoring results of each second abnormal sample feature, where the scoring results indicate a degree of proximity between the second abnormal sample feature and a true abnormal sample feature; and selecting target abnormal sample characteristics with scoring results meeting scoring conditions from a plurality of second abnormal sample characteristics based on scoring results of the second abnormal sample characteristics.

In the above scheme, the clustering module is further configured to determine a first feature cluster from at least one feature cluster after the clustering is performed on the plurality of abnormal sample features to obtain at least one feature cluster, and determine distances between each second feature cluster and the first feature cluster, where the second feature cluster is a feature cluster different from the first feature cluster in the at least one feature cluster; determining a third feature cluster with the distance higher than a distance threshold value and a fourth feature cluster with the distance not higher than the distance threshold value from the at least one feature cluster; the noise adding module is further configured to add noise to a first abnormal sample feature corresponding to a cluster center of each third feature cluster, so as to obtain a second abnormal sample feature corresponding to each third feature cluster; noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the fourth characteristic clusters, and second abnormal sample characteristics corresponding to the fourth characteristic clusters are obtained; the number of the second abnormal sample features corresponding to each third feature cluster is smaller than the number of the second abnormal sample features corresponding to each fourth feature cluster.

The embodiment of the application also provides electronic equipment, which comprises:

A memory for storing computer executable instructions;

And the processor is used for realizing the characteristic processing method provided by the embodiment of the application when executing the computer executable instructions stored in the memory.

The embodiment of the application also provides a computer readable storage medium which stores computer executable instructions or a computer program, and when the computer executable instructions or the computer program are executed by a processor, the characteristic processing method provided by the embodiment of the application is realized.

The embodiment of the application also provides a computer program product, which comprises computer executable instructions or a computer program, and the computer executable instructions or the computer program realize the characteristic processing method provided by the embodiment of the application when being executed by a processor.

The embodiment of the application has the following beneficial effects:

By applying the embodiment of the application, firstly, the characteristics of the abnormal sample of each of a plurality of abnormal image samples are extracted, the abnormal image samples are attributed to the target object, and when the target object is detected based on the abnormal image samples, the obtained detection result represents that the target object has abnormality; then clustering is carried out on the plurality of abnormal sample features to obtain at least one feature cluster; and adding noise to the first abnormal sample characteristics corresponding to the cluster center of each characteristic cluster to obtain second abnormal sample characteristics corresponding to each first abnormal sample characteristic. Here, the second abnormal sample feature is used for expansion of the abnormal sample feature, so that training of the abnormal detection model can be performed by the second abnormal sample feature. In this way, the expansion of the abnormal sample features is realized from the feature level, the abnormal sample features exist in the form of a plurality of feature cluster clusters in the feature space, the noise is added to the cluster centers of the cluster clusters to generate second abnormal sample features, the generated second abnormal sample features are ensured to be closer to the original abnormal sample features, the accuracy of the generated abnormal sample features is improved, the expansion effect of the abnormal sample features is further improved, and the training effect of the abnormal detection model is improved.

Drawings

FIG. 1 is a schematic diagram of a feature processing system according to an embodiment of the present application;

Fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 3A is a schematic flow chart of a feature processing method according to an embodiment of the present application;

FIG. 3B is a schematic diagram of a second flow chart of a feature processing method according to an embodiment of the present application;

FIG. 3C is a third flow chart of a feature processing method according to an embodiment of the present application;

FIG. 3D is a fourth flowchart of a feature processing method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a feature cluster provided by an embodiment of the present application;

fig. 5 is a schematic diagram of an architecture of a feature processing method according to an embodiment of the present application.

It should be noted that the "first" and "second" are only used to distinguish between different schemes, and do not represent the degree of preference or priority in implementation.

Detailed Description

The present application will be further described in detail with reference to the accompanying drawings, for the purpose of making the objects, technical solutions and advantages of the present application more apparent, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

In the following description, the terms "first", "second", "third" and the like are merely used to distinguish similar objects and do not represent a specific ordering of the objects, it being understood that the "first", "second", "third" may be interchanged with a specific order or sequence, as permitted, to enable embodiments of the application described herein to be practiced otherwise than as illustrated or described herein.

In the present embodiment, the term "module" or "unit" refers to a computer program or a part of a computer program having a predetermined function and working together with other relevant parts to achieve a predetermined object, and may be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Also, a processor (or multiple processors or memories) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of an overall module or unit that includes the functionality of the module or unit.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of the application is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

In the embodiment of the application, the relevant data collection processing should be strictly according to the requirements of relevant laws and regulations when the example is applied, so as to acquire the informed consent or independent consent of the personal information body, and develop the subsequent data use and processing behaviors within the authorized range of the laws and regulations and the personal information body.

Before describing embodiments of the present application in further detail, the terms and terminology involved in the embodiments of the present application will be described, and the terms and terminology involved in the embodiments of the present application will be used in the following explanation.

1) Client side: applications running in electronic devices for providing various services, such as clients supporting feature processing.

2) In response to: for representing a condition or state upon which an operation is performed, one or more operations performed may be in real-time or with a set delay when the condition or state upon which the operation is dependent is satisfied; without being specifically described, there is no limitation in the execution sequence of the plurality of operations performed.

3) Exogenous data: the data that can be contacted in the training task is called intra-domain data, and the other data can be called exogenous data.

4) Computer Vision technology (CV): the method is a science for researching how to make the machine "look at", and further means that a camera and a computer are used to replace human eyes to recognize and measure targets and other machine vision, and further graphic processing is performed, so that the computer is used to process images which are more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision technologies generally include technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, map construction, etc., and also include common biological feature recognition technologies such as face recognition and fingerprint recognition, anomaly detection technologies applied to industrial quality inspection, etc.

The embodiment of the application provides a feature processing method, a device, equipment, a storage medium and a computer program product, which can improve the accuracy of generated abnormal sample features. Next, embodiments of the present application will be described in detail based on the above description of nouns and terms involved in the embodiments of the present application.

The following describes a feature processing system provided by an embodiment of the present application. Referring to fig. 1, fig. 1 is a schematic architecture diagram of a feature processing system according to an embodiment of the present application. To enable support for one exemplary application, feature processing system 100 includes: server 200, network 300, and terminal 400. The terminal 400 is connected to the server 200 through the network 300, where the network 300 may be a wide area network or a local area network, or a combination of both, and the data transmission is implemented using a wireless or wired link.

Here, the terminal 400 (e.g., a client running with support for feature processing) transmits a feature processing request for requesting generation of an abnormal sample feature to the server 200 in response to a feature processing instruction; the server 200 receives a feature processing request transmitted from the terminal 400; responding to the feature processing request, acquiring a plurality of abnormal image samples, wherein the abnormal image samples belong to a target object, and when the target object is detected based on the abnormal image samples, the obtained detection result represents that the target object has an abnormality; extracting abnormal sample characteristics of each of a plurality of abnormal image samples; clustering is carried out on the plurality of abnormal sample features to obtain at least one feature cluster; noise is added to first abnormal sample features corresponding to the cluster centers of the feature clusters, and second abnormal sample features corresponding to the first abnormal sample features are obtained; returning a notification message that the abnormal sample feature has been generated to the terminal 400; the terminal 400 receives and displays the notification message returned from the server 200. In this way, the abnormal sample characteristics are generated, the expansion of the abnormal sample characteristics can be realized, the abnormal sample characteristics can be used for training an object detection model, the object detection model is used for detecting an object in an image to be detected, a detection result used for representing whether the object has an abnormality or not is obtained, the expansion of the abnormal sample characteristics is realized by the characteristic processing method provided by the embodiment of the application, the training effect of the object detection model can be improved, and the detection precision of the object detection model obtained by training is improved.

In some embodiments, the feature processing method provided by the embodiments of the present application is implemented by an electronic device, for example, may be implemented by a terminal alone, may be implemented by a server alone, or may be implemented by a terminal and a server cooperatively. Embodiments of the present application may be applied to a variety of scenarios including, but not limited to, cloud Technology (Cloud Technology), artificial intelligence (ARTIFICIAL INTELLIGENCE, AI), intelligent transportation, assisted driving, computer vision Technology, anomaly detection of objects in images (e.g., industrial products), and the like.

In some embodiments, the electronic device implementing the feature processing method provided by the embodiment of the present application may be a terminal or a server of various types. The server (e.g., server 200) may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers. The terminal (e.g., terminal 400) may be, but is not limited to, a notebook computer, tablet computer, desktop computer, smart phone, smart voice interaction device (e.g., smart speaker), smart home appliance (e.g., smart television), smart watch, vehicle-mounted terminal, wearable device, virtual Reality (VR) device, aircraft, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited by the embodiment of the present application.

In some embodiments, the feature processing method provided by the embodiment of the application can be implemented by means of cloud technology. Cloud technology refers to a hosting technology for unifying serial resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is a generic term of network technology, information technology, integration technology, management platform technology, application technology and the like based on cloud computing business model application, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical network systems require a large amount of computing and storage resources. As an example, a server (e.g., server 200) may also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, web services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

In some embodiments, the feature processing method provided by the embodiment of the application can be implemented by means of artificial intelligence technology. Artificial intelligence is a comprehensive technology of computer science, and by researching the design principle and implementation method of various intelligent machines, the machines have the functions of sensing, reasoning and decision. Artificial intelligence infrastructure technologies generally include, for example, sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, pre-training model technologies, operation/interaction systems, mechatronics, and the like. The pre-training model is also called a large model and a basic model, and can be widely applied to all large-direction downstream tasks of artificial intelligence after fine adjustment. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions. As an example, the abnormal sample feature generated by the feature processing method provided by the embodiment of the present application may be used to train an object detection model, where the object detection model is used to detect an object, and obtain a detection result used to characterize whether the object has an abnormality. Here, the embodiment of the application realizes the expansion of the abnormal sample characteristics, and can improve the training effect of the object detection model, thereby improving the detection precision of the object detection model obtained by training.

In some embodiments, the terminal or server may implement the feature processing method provided by the embodiments of the present application by running various computer executable instructions or computer programs. For example, the computer-executable instructions may be commands at the micro-program level, machine instructions, or software instructions. The computer program may be a native program or a software module in an operating system; a Native (APP) Application, i.e. a program that needs to be installed in an operating system to run; or an applet that can be embedded in any APP, i.e., a program that can be run only by being downloaded into the browser environment. In general, the computer-executable instructions may be any form of instructions and the computer program may be any form of application, module, or plug-in.

The electronic device for implementing the feature processing method provided by the embodiment of the application is described below. Referring to fig. 2, fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 500 provided in the embodiment of the present application may be a terminal or a server. As shown in fig. 2, the electronic device 500 includes: at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in electronic device 500 are coupled together by bus system 540. It is appreciated that the bus system 540 is used to enable connected communications between these components. The bus system 540 includes a power bus, a control bus, and a status signal bus in addition to the data bus. The various buses are labeled as bus system 540 in fig. 2 for clarity of illustration.

The Processor 510 may be an integrated circuit chip having signal processing capabilities such as a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc., where the general purpose Processor may be a microprocessor or any conventional Processor, etc.

The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and/or one or more visual displays. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The memory 550 may be removable, non-removable, or a combination thereof. Memory 550 may include one or more storage devices physically located away from processor 510. Memory 550 includes volatile memory or nonvolatile memory, and may also include both volatile and nonvolatile memory. The non-volatile Memory may be a Read Only Memory (ROM) and the volatile Memory may be a random access Memory (Random Access Memory, RAM). The memory 550 described in embodiments of the present application is intended to comprise any suitable type of memory.

In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules and data structures, or subsets or supersets thereof, as exemplified below.

An operating system 551 including system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

Network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: bluetooth, wireless compatibility authentication (WiFi), and universal serial bus (Universal Serial Bus, USB), etc.;

A presentation module 553 for enabling presentation of information (e.g., a user interface for operating a peripheral device and displaying content and information) via one or more output devices 531 (e.g., a display screen, speakers, etc.) associated with the user interface 530;

The input processing module 554 is configured to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

In some embodiments, the feature processing apparatus provided in the embodiments of the present application may be implemented in software, and fig. 2 shows the feature processing apparatus 555 stored in the memory 550, which may be software in the form of a program, a plug-in, or the like, including the following software modules: the extraction module 5551, the clustering module 5552 and the noise addition module 5553 are logical, and thus may be arbitrarily combined or further split according to the implemented functions, the functions of each module will be described below.

The feature processing method provided by the embodiment of the application is described below. As described above, the feature processing method provided by the embodiment of the present application is implemented by the electronic device, for example, may be implemented by the server or the terminal alone, or implemented by the server and the terminal cooperatively. The execution subject of each step will not be repeated hereinafter. Referring to fig. 3A, fig. 3A is a schematic flow chart of a feature processing method provided by an embodiment of the present application, where the feature processing method provided by the embodiment of the present application includes:

Step 101: and extracting the abnormal sample characteristics of each of the plurality of abnormal image samples.

The abnormal image sample belongs to the target object, and when the target object is detected based on the abnormal image sample, the obtained detection result represents that the target object has an abnormality.

For step 101, an outlier image sample may be first acquired. The abnormal image sample comprises a target object, which is an object to be detected, and for example, the target object can be an industrial product, such as a notebook computer, a display screen, a wood floor, a desk and chair, a ceramic tile, a square box, a license plate, a door plate, a vehicle accessory, a book and the like. The image content of the abnormal image sample may include all or part of the target object. The detection of the target object may be object condition (e.g., quality, presence or absence of defect, etc.) detection, such as capturing an image of the target object, and then detecting whether the target object in the image is abnormal (e.g., whether there is dirt, defect, scratch, etc. on the industrial product). Therefore, the abnormal image sample is an image sample of which the detection result indicates that the target object is abnormal, that is, when the target object is detected based on the abnormal image sample, the obtained detection result indicates that the target object is abnormal (for example, the target object is defective).

Continuously, after a plurality of abnormal image samples are acquired, extracting the characteristics of each abnormal image sample to obtain the abnormal sample characteristics of each abnormal image sample. For example, one feature extraction model may be trained in advance, and the abnormal sample features of each abnormal image sample may be extracted by the feature extraction model trained in advance. In practice, the feature extraction model may be a pre-trained Backbone model on a large-scale dataset (e.g., imageNet).

It should be noted that, the inspection of the target object based on the image of the target object may be implemented based on a pre-trained machine learning model (i.e., an object detection model). While a large amount of sample data is often required for training the object detection model, including an abnormal image sample and a normal image sample (when the target object is detected based on the normal image sample, the obtained detection result indicates that the target object is normal, such as no defect exists), in an actual scene, the number and variety of the abnormal image samples that can be obtained are often limited, which leads to a problem that the model training faces data imbalance, that is, the number of the abnormal image samples may be far less than the number of the normal image samples, which leads to an unsatisfactory training effect of the object detection model and further leads to low detection accuracy of the object detection. Therefore, sample expansion is required for the abnormal image samples. When the object detection model is trained based on the image sample, the sample characteristics of the image sample are often processed, and based on the processing, the embodiment of the application expands the abnormal sample characteristics of the abnormal image sample from the characteristic level so as to realize the expansion of the abnormal sample data for training.

Step 102: and clustering the plurality of abnormal sample features to obtain at least one feature cluster.

And 102, clustering the abnormal sample features acquired in the step 101 to obtain at least one feature cluster. For example, a clustering algorithm (such as a K-means algorithm) may be used to perform clustering on the plurality of abnormal sample features to obtain at least one feature cluster. Wherein each feature cluster includes at least one outlier sample feature; the cluster center (i.e., cluster center) of each feature cluster corresponds to a first abnormal sample feature of the plurality of abnormal sample features; for example, referring to fig. 4, a plurality of feature clusters are obtained by performing clustering processing on a plurality of abnormal sample features. In a feature space formed by a plurality of feature clusters, each feature cluster represents an abnormal sample feature of one mode, and if feature sampling is performed in a certain area around the cluster center of the feature cluster, the obtained feature can also be used as the abnormal sample feature so as to realize the expansion of the original abnormal sample feature.

Step 103: noise is added to first abnormal sample features corresponding to cluster centers of the feature clusters, and second abnormal sample features corresponding to the first abnormal sample features are obtained.

The second abnormal sample feature is used for training an abnormal detection model, and the abnormal detection model is used for detecting whether an object in the image to be detected has an abnormality or not.

For step 103, the effect of feature sampling in a certain area around the cluster center of each feature cluster can be achieved by adding noise to the first abnormal sample features corresponding to the cluster center of each feature cluster, and the second abnormal sample features corresponding to each first abnormal sample feature can be obtained. The second abnormal sample features corresponding to the first abnormal sample features are obtained, namely, the second abnormal sample features are used for expanding a plurality of abnormal image features.

The noise added in step 103 may be pre-generated. By way of example, the noise may be sampled from a target distribution; the target distribution includes a plurality of random data conforming to a particular data distribution type (e.g., normal distribution, standard normal distribution, uniform distribution), which may be generated based on a random data generation algorithm.

The second abnormal sample features generated in step 103 are used for training an abnormal detection model, and specifically, the first abnormal sample features and the second abnormal sample features corresponding to the first abnormal sample features can be combined together to train the abnormal detection model. The anomaly detection model is used for detecting whether an anomaly exists in an object in an image to be detected. The anomaly detection model may be constructed based on a neural network, such as a convolutional neural network, a deep neural network, a recurrent neural network, or the like. Whether an object (such as an industrial product) in an image to be detected (such as an image of the industrial product) is abnormal or not is detected through the abnormal detection model, so that the detection efficiency can be improved, and the second abnormal sample characteristics expanded by the embodiment of the application participate in training of the abnormal detection model, so that the training effect of the abnormal detection model can be improved, and the abnormal detection precision is improved.

In some embodiments, referring to fig. 3B, before performing step 103 "adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature" shown in fig. 3A, step 1031a may be further performed: generating at least one first noise, wherein the first noise is different; based on this, step 103 shown in fig. 3A can be implemented by performing step 1032 a: the following processing is respectively executed for each feature cluster: and respectively adding first noise to first abnormal sample features corresponding to the cluster centers of the feature clusters to obtain at least one second abnormal sample feature corresponding to the first abnormal sample feature.

For step 1031a, at least one first noise may be generated, each first noise being different from each other. For example, at least one first noise may be generated by: at least one sampling result is obtained by sampling the data conforming to the first distribution at least once, each sampling result is used as each first noise, and each sampling result is different; or acquiring data conforming to at least one second distribution, and sampling the data conforming to the second distribution for each second distribution to obtain first noise, wherein each second distribution is different. Specifically, for example, the generation sources of the respective first noises may be the same, for example, sampling is performed at least once from data conforming to the first distribution (e.g., normal distribution, poisson distribution), and it is ensured that the sampling result obtained by sampling is different for each sampling, and the sampling result obtained by sampling is regarded as the first noise. For example, the generation sources may be different between the respective first noises. The first noise is sampled from data conforming to each of at least one second distribution, which may include a normal distribution, a uniform distribution, a poisson distribution, and so forth, which differs from one second distribution to another.

For step 1032a, the following processing may be performed for each feature cluster: and respectively adding first noise to first abnormal sample features corresponding to the cluster centers of the feature clusters to obtain at least one second abnormal sample feature corresponding to the first abnormal sample feature. Namely, adding a first noise to the first abnormal sample feature to obtain a second abnormal sample feature; under the condition that the first noise added each time is different, at least one second abnormal sample characteristic corresponding to the first abnormal sample characteristic can be obtained, and each second abnormal sample characteristic is also different. Thus, diversified second abnormal sample characteristics can be generated, the abnormal sample characteristics are enriched, and the expansion effect of the abnormal sample characteristics is improved.

In some embodiments, referring to fig. 3C, before performing step 103 "adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature" shown in fig. 3A, step 1031b may be further performed: generating second noise corresponding to each characteristic cluster; based on this, step 103 shown in fig. 3A may also be implemented by performing step 1032 b: the following processing is respectively executed for each feature cluster: and adding second noise corresponding to the feature cluster to the first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain a second abnormal sample feature corresponding to the first abnormal sample feature.

For step 1031b, the following processing may be performed for each feature cluster: firstly, determining the characteristic distribution of the characteristic cluster, then generating data conforming to the characteristic distribution of the characteristic cluster, and further sampling from the data conforming to the characteristic distribution of the characteristic cluster to obtain second noise corresponding to the characteristic cluster. In this way, a corresponding second noise is generated for each feature cluster, and each second noise is obtained from data samples conforming to the feature distribution of the feature cluster, which will cause each second noise to also conform to the feature distribution of the corresponding feature cluster, thereby improving the proximity of the second abnormal sample feature generated by adding the second noise to the true abnormal sample feature.

For step 1032b, the following processing may be performed for each feature cluster: and adding second noise corresponding to the feature cluster to the first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain a second abnormal sample feature corresponding to the first abnormal sample feature. In some examples, the second noise generated for each feature cluster may also be multiple, and each second noise is different from one another; thus, each second noise corresponding to the feature cluster can be added to the first abnormal sample feature corresponding to the cluster center of the feature cluster, so as to obtain a plurality of second abnormal sample features corresponding to the first abnormal sample feature. Namely, adding a second noise to the first abnormal sample characteristic to obtain a second abnormal sample characteristic; under the condition that the second noise added each time is different, a plurality of second abnormal sample characteristics corresponding to the first abnormal sample characteristics can be obtained, and each second abnormal sample characteristic is different. Therefore, the added second noise is ensured to be in line with the feature distribution of the corresponding feature cluster, the diversity of the second noise is increased, so that diversified second abnormal sample features can be generated, the proximity degree of the generated diversified second abnormal sample features and real abnormal sample features can be improved, and the expansion effect of the abnormal sample features is increased.

In some embodiments, after performing step 103 "adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature", the following steps may be further performed: acquiring an exogenous image, wherein the exogenous image does not comprise a target object; extracting exogenous image characteristics of exogenous images; and fusing the second abnormal sample characteristics and the exogenous image characteristics to obtain third abnormal sample characteristics.

It should be noted that, the exogenous image is an image that does not include the target object (including the target object in the normal image sample and the target object in the abnormal image sample), specifically, the abnormal sample feature is used for the target training task (i.e., the training task of the object detection model of the target object), so that the data that the target training task can contact becomes intra-domain data, i.e., the image sample related to the target object, and other images are exogenous images. Extracting features of the exogenous image to obtain exogenous image features; for example, a feature extraction model may be pre-trained, and the exogenous image features of each exogenous image may be extracted by the pre-trained feature extraction model. In practice, the feature extraction model may be a pre-trained Backbone model on a large-scale dataset (e.g., imageNet). In some examples, after the exogenous image is acquired, data enhancement may also be performed on the exogenous image to obtain an enhanced exogenous image, so that feature extraction is performed on the enhanced exogenous image to obtain exogenous image features.

And continuing, fusing the extracted exogenous image characteristics with the second abnormal sample characteristics to obtain third abnormal sample characteristics, wherein the third abnormal sample characteristics are also used for expanding the abnormal sample characteristics. During fusion, corresponding weighting weights can be set for the second abnormal sample feature and the exogenous image feature respectively, so that the second abnormal sample feature and the exogenous image feature are subjected to weighted summation processing based on the respective weighting weights of the second abnormal sample feature and the exogenous image feature, and a third abnormal sample feature is obtained. In some embodiments, the exogenous image feature may also be directly used as a third abnormal sample feature. Thus, by introducing the exogenous image features of the exogenous image, the content diversity of the generated third abnormal sample features is increased, thereby increasing the diversity of the expanded abnormal sample features.

In some embodiments, referring to fig. 3D, after performing step 103 "adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature", the following steps 104-106 may be further performed: step 104, extracting normal sample characteristics of the normal image sample, and generating third noise; the normal image sample belongs to the target object, and when the target object is detected based on the normal image sample, the obtained detection result represents that the target object is normal; step 105, calling an abnormal feature generation model to generate a fourth abnormal sample feature based on the normal sample feature and the third noise; and step 106, fusing the second abnormal sample characteristics and the fourth abnormal sample characteristics to obtain fifth abnormal sample characteristics.

For step 104, a normal image sample may be obtained, which also belongs to the target object, and may include all or part of the target object, and when the target object is detected based on the normal image sample, the obtained detection result indicates that the target object is normal. And then extracting normal sample features of the normal image sample. For example, one feature extraction model may be trained in advance, and normal sample features of each normal image sample may be extracted by the feature extraction model trained in advance. In practice, the feature extraction model may be a pre-trained Backbone model on a large-scale dataset (e.g., imageNet). Meanwhile, a third noise is generated in step 104, for example, the third noise may be sampled from a target distribution, where the target distribution includes a plurality of random data conforming to a specific data distribution type (e.g., normal distribution, standard normal distribution, uniform distribution), and the random data conforming to the specific data distribution type may be generated based on a random data generation algorithm.

For step 105, the normal sample feature and the third noise are input into the abnormal feature generation model, thereby invoking the abnormal feature generation model, and generating a fourth abnormal sample feature based on the normal sample feature and the third noise.

And for the step 106, fusing the fourth abnormal sample feature and the second abnormal sample feature to obtain a fifth abnormal sample feature. During fusion, acquiring a first weight of a second abnormal sample feature and acquiring a second weight of a fourth abnormal sample feature; and weighting the second abnormal sample characteristic and the fourth abnormal sample characteristic based on the first weight and the second weight to obtain a fifth abnormal sample characteristic. That is, the respective weighting weights may be set for the second abnormal sample feature and the fourth abnormal sample feature, so that the weighted summation processing is performed on the second abnormal sample feature and the fourth abnormal sample feature based on the respective weighting weights of the second abnormal sample feature and the fourth abnormal sample feature, to obtain the fifth abnormal sample feature. Therefore, the third abnormal sample feature generated by the trained abnormal feature generation model is closer to the real abnormal sample feature, and the fourth abnormal sample feature and the second abnormal sample feature are fused, so that the approximation degree of the obtained fifth abnormal sample feature and the real abnormal sample feature is improved, and the generation effect of the abnormal sample feature is improved.

In some embodiments, the anomaly characteristic generation model may be derived by performing the steps of: step 201, acquiring an initial abnormal feature generation model and first normal sample features of a first normal image sample, and generating fourth noise; step 202, calling an initial abnormal feature generation model to generate a sixth abnormal sample feature based on the first normal sample feature and the fourth noise; step 203, calling a feature discrimination model to discriminate the feature of the sixth abnormal sample, so as to obtain a feature discrimination result, wherein the feature discrimination result indicates the possible degree of the feature of the sixth abnormal sample as a true abnormal sample feature; step 204, determining the value of a loss function of the initial abnormal feature generation model based on the real abnormal sample features and the feature discrimination results; and step 205, training the initial abnormal feature generation model based on the value of the loss function to obtain an abnormal feature generation model.

For step 201, the initial anomaly characteristic generation model may be pre-constructed, illustratively, the initial anomaly characteristic generation model includes a cross-attention (cross attention) module and two residual neural networks (ResBlock), each ResBlock may include a convolutional layer Conv, a fully-connected layer (BatchNorm), and an activation function layer (e.g., leakyReLU layers). The generation manner of the fourth noise may refer to the generation manner of the third noise, which is not described herein. For step 202, an initial abnormal feature generation model is invoked to generate a sixth abnormal sample feature based on the first normal sample feature and the fourth noise. For step 203, based on the real abnormal sample feature, a feature discrimination model is called to discriminate the sixth abnormal sample feature, so as to obtain a feature discrimination result. For step 204, firstly, obtaining the real abnormal sample characteristics, and then, determining the value of the loss function of the initial abnormal characteristic generation model based on the real abnormal sample characteristics and the characteristic discrimination result; thus, in step 205, based on the value of the loss function, model parameters of the initial abnormal feature generation model are updated to train the initial abnormal feature generation model to obtain an abnormal feature generation model.

In some embodiments, after noise is added to the first abnormal sample feature corresponding to the cluster center of each feature cluster, so as to obtain the second abnormal sample feature corresponding to each first abnormal sample feature, the following processing may be further performed: scoring the plurality of second abnormal sample features to obtain scoring results of the second abnormal sample features, wherein the scoring results indicate the proximity degree between the second abnormal sample features and the real abnormal sample features; and selecting target abnormal sample characteristics, of which the scoring results meet the scoring conditions, from the plurality of second abnormal sample characteristics based on the scoring results of the second abnormal sample characteristics. For example, the target abnormal sample feature may be a second abnormal sample feature of the target number with the scoring result sorted in descending order, or a second abnormal sample feature with the scoring result higher than the score threshold. Therefore, the target abnormal sample characteristics of the approaching degree between the target abnormal sample characteristics and the real abnormal sample characteristics can be selected, the abnormal sample characteristics are expanded, and the expansion effect of the abnormal sample characteristics is improved.

In some embodiments, after clustering the plurality of abnormal sample features to obtain at least one feature cluster, the following process may be further performed: determining a first feature cluster from at least one feature cluster, and determining the distance between each second feature cluster and the first feature cluster, wherein the second feature cluster is a different feature cluster from the first feature cluster in the at least one feature cluster; determining a third feature cluster with a distance higher than a distance threshold value and a fourth feature cluster with a distance not higher than the distance threshold value from the at least one feature cluster; based on the above, noise is added to the first abnormal sample feature corresponding to the cluster center of each feature cluster, and a second abnormal sample feature corresponding to each first abnormal sample feature is obtained, including: noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the third characteristic clusters to obtain second abnormal sample characteristics corresponding to the third characteristic clusters; noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the fourth characteristic clusters to obtain second abnormal sample characteristics corresponding to the fourth characteristic clusters; the number of the second abnormal sample features corresponding to each third feature cluster is smaller than the number of the second abnormal sample features corresponding to each fourth feature cluster.

Here, a target feature space distributed in a feature cluster set is first determined, and the first feature cluster set may be a center cluster set in the target feature space; and then determining the distance between each second feature cluster and the first feature cluster, so that a third feature cluster with the distance higher than a distance threshold value and a fourth feature cluster with the distance not higher than the distance threshold value are determined in at least one feature cluster. Based on the above, noise is added to the first abnormal sample feature corresponding to the cluster center of each third feature cluster to obtain second abnormal sample feature corresponding to each third feature cluster, and noise is added to the first abnormal sample feature corresponding to the cluster center of each fourth feature cluster to obtain second abnormal sample feature corresponding to each fourth feature cluster, so that the number of the second abnormal sample features corresponding to each third feature cluster is smaller than the number of the second abnormal sample features corresponding to each fourth feature cluster. Therefore, the common abnormal sample characteristics can be expanded more, the relatively rare abnormal sample characteristics can be expanded appropriately, and the distribution rationality of the expanded abnormal sample characteristics can be improved.

An exemplary application of the embodiments of the present application in a practical application scenario is described below with reference to an industrial quality inspection scenario. The industrial quality detection is to detect the quality of industrial products. Industrial quality inspection industrial products are manually inspected one by technically trained workers. With the development of AI technology, AI systems combining industrial cameras and machine learning are beginning to be adopted to replace manual detection, so that labor cost is reduced. However, machine learning models typically rely on large amounts of data, whereas in the industrial field, the number and variety of defective products (i.e., products with anomalies) is relatively small, but defects that may actually occur are endless. This makes it difficult to directly collect a true anomaly sample (i.e., an image that includes an anomaly product) and apply it to training of a machine learning model. In some scenarios, training of an object detection model for product anomaly detection is an unsupervised approach, meaning that only data of normal samples (images of non-defective products) are used for learning during training, and the trained model is required to identify whether the test sample is normal (non-defective) or abnormal (defective) during testing. In a real product anomaly detection scene, a small amount of anomaly samples and labels thereof can be obtained, so that an object detection model can be trained in a semi-supervision mode.

But neither unsupervised training nor semi-supervised training is faced with the problem of data imbalance. Data imbalance is embodied as a number of abnormal samples (images of defective products) being much smaller than a number of normal samples (images of non-defective products), so it is a common practice to generate some falsified abnormal samples to expand the number of abnormal samples at the time of training. In the related art, most of the generation methods of the abnormal samples are implemented on the pixel level, for example, the abnormal samples are synthesized by using the exogenous image and the random normal samples, but the generation method is generally complicated in steps, and the generated abnormal samples have larger differences from the real abnormal samples.

Based on this, the inventors found that it is more efficient to construct an abnormal sample feature in the feature space than to construct an abnormal sample in the pixel space, and that the abnormal sample feature of a true abnormal sample is closer. Therefore, the embodiment of the application provides a generation method of abnormal sample characteristics, which can utilize the existing abnormal sample characteristics to construct the abnormal sample characteristics which are closer to the real abnormal sample characteristics. In practical application, the embodiment of the application can be applied to any practical industrial AI quality inspection project, and training of an object detection model can be realized by using a normal sample and a small amount of available abnormal samples, so as to be used for detecting the abnormality of industrial products.

(1) Generating an abnormal sample feature in a first mode. Here, feature extraction is performed on a plurality of available abnormal samples to obtain a plurality of abnormal sample features, for example, feature extraction may be implemented using a pre-trained feature extraction model (e.g., a pre-trained backhaul model on a large-scale data set (ImageNet)). In an actual scene, the characteristics of the abnormal samples are relatively discrete, and because the types of the abnormal samples are various, but different anomalies still have a certain commonality, for a plurality of extracted abnormal sample characteristics, clustering can be performed on the plurality of abnormal sample characteristics to obtain at least one characteristic cluster, as shown in fig. 4 (fig. 4 is a schematic diagram of the characteristic cluster provided by the embodiment of the present application), and the abnormal sample characteristics exist in a plurality of cluster forms in a characteristic space. Each feature cluster represents a pattern of abnormal sample features, so that sampling around the feature clusters can obtain potentially true abnormal sample features to realize the expansion of the abnormal sample features. Specifically, a clustering algorithm (such as K-means) may be used to perform clustering processing on multiple abnormal sample features to obtain K feature clusters, and determine K cluster centers (i.e., cluster centers):

；

wherein, Representing the number of cluster centers,Is a super parameter and can be adjusted according to actual conditions; a plurality of anomaly sample features representing an available anomaly image sample; representing the result of clustering A set of cluster centers. Next, sampling around each cluster center, i.e., for each cluster center, adding noise (e.g., random noise sampled from a gaussian distribution) to the outlier sample features corresponding to that cluster center, resulting in the outlier sample features generated：

；

Wherein,Is a cluster center.

(2) And a second generation mode of abnormal sample characteristics. Since the mode presented by the exogenous data and the currently used data field have larger difference, the difference also exists in the characteristic space, so that some exogenous images can be selected, and the exogenous image characteristics of the exogenous images are extracted as generated abnormal sample characteristics:

wherein, Representing a feature extraction model (such as a pre-trained Backbone model on a large-scale dataset (ImageNet)),Representing a random image of an external source,Representing the exogenous image features.

In some examples, for the exogenous image, the data enhancement may be performed on the exogenous image first, and then feature extraction may be performed on the exogenous image obtained by the data enhancement, so as to obtain the generated abnormal sample feature.

(3) And generating an abnormal sample characteristic. Here, the generation of the abnormal sample feature is achieved by adopting an countermeasure generation manner. Referring to fig. 5, fig. 5 is a schematic diagram of an architecture of a feature processing method according to an embodiment of the present application. The countermeasure generation framework shown in FIG. 5, during training, an abnormal feature generatorSum feature discriminatorAnd playing games with each other, and generating the finally obtained abnormal sample characteristics closer to the real abnormal sample characteristics by the finally obtained abnormal characteristic generator G.

With continued reference to FIG. 5, the countermeasure generation framework is composed of an anomaly feature generatorSum feature discriminatorThe composition is formed. Since the normal image sample and the abnormal image sample are data of the same category in the scene of industrial quality inspection, the abnormal sample characteristics and the normal sample characteristics are associated to a certain extent, and the normal sample characteristics are based on the abnormal sample characteristicsRandom noise(E.g. Gaussian noise sampled from Gaussian distribution) as an outlier generatorThe purpose of introducing random noise here is to give the generated outlier sample features more diversity.

Exemplary, anomaly characteristic GeneratorCan be composed of one cross attention module and two ResBlock (Conv+ BatchNorm + leakyReLU) modules, in the cross-attention module, the random noise is inputAs query, the normal sample characteristics of the inputAs keys and values. Exemplary, normal sample characteristicsRandom noiseAbnormality feature generatorThe size of the output may be 1024x1536. For the structure of the abnormal feature generator, the abnormal feature generator can be replaced according to the actual scene, and the abnormal feature generator is not limited in the embodiment of the application.

Exemplary feature discriminantCan be composed of a series of downsampling blocks (Downsample), a feature discriminatorFor abnormal feature generatorEach generated feature is judged, a judgment score is output, and the closer the judgment score is to 1, the more abnormal the generated abnormal sample feature is (namely, the closer the judgment score is to the real abnormal sample feature); conversely, the more normal the generated abnormal sample feature (i.e., the closer to the true normal sample feature) is accounted for.

The countermeasure generation framework takes the manner of countermeasure training, namely: first fix an outlier feature generatorTraining feature discriminantThen fix the feature discriminantTraining anomaly feature generatorThis process is repeated a number of times until the training end condition is reached (e.g., the number of training times reaches the number of times threshold). By way of example, the loss function employed for training may be as follows:

；

wherein, In order to be a value of the loss function,For a normal sample feature,As a result of the random noise,Is a true abnormal sample feature.

After this challenge-generating framework training is completed, an anomaly feature generator may be usedTo generate an outlier sample feature that approximates the true outlier sample feature:

；

wherein, Representing the generated abnormal sample features.

By applying the embodiment of the application, 1) the abnormal sample characteristics of the abnormal image sample which can be obtained in the actual scene are used as effective supervision information, and new abnormal sample characteristics are constructed on the characteristic level, so that compared with the construction of the abnormal sample characteristics in the pixel space, the construction of the abnormal sample characteristics is more direct and efficient; 2) The abnormal sample characteristics generated based on the embodiment of the application are applied to the training of the semi-supervised object detection model, and the promotion of 0.3% and 0.1% is respectively realized on the indexes of the image level AUROC and the pixel level AUROC.

Continuing with the description below of an exemplary architecture of feature processing device 555 implemented as a software module provided by embodiments of the present application, in some embodiments, as shown in fig. 2, the software modules stored in feature processing device 555 of memory 550 may comprise: an extraction module 5551, configured to extract an abnormal sample feature of each of the plurality of abnormal image samples; the abnormal image sample belongs to a target object, and when the target object is detected based on the abnormal image sample, the obtained detection result represents that the target object has an abnormality; a clustering module 5552, configured to perform clustering processing on a plurality of abnormal sample features to obtain at least one feature cluster; the noise adding module 5553 is configured to add noise to a first abnormal sample feature corresponding to a cluster center of each feature cluster, so as to obtain a second abnormal sample feature corresponding to each first abnormal sample feature.

In some embodiments, the noise adding module 5553 is further configured to generate at least one first noise before adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain the second abnormal sample feature corresponding to each first abnormal sample feature, where each first noise is different; the noise adding module 5553 is further configured to perform the following processing for each of the feature clusters: and respectively adding the first noise to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain at least one second abnormal sample feature corresponding to the first abnormal sample feature.

In some embodiments, the noise adding module 5553 is further configured to sample the data according to the first distribution at least once to obtain at least one sampling result, and take each sampling result as each first noise, where each sampling result is different; or acquiring data conforming to at least one second distribution, and sampling the data conforming to the second distribution to obtain the first noise for each second distribution, wherein the second distributions are different.

In some embodiments, the noise adding module 5553 is further configured to generate a second noise corresponding to each feature cluster before adding noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster to obtain a second abnormal sample feature corresponding to each first abnormal sample feature; the noise adding module 5553 is further configured to perform the following processing for each of the feature clusters: and adding second noise corresponding to the feature cluster to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain a second abnormal sample feature corresponding to the first abnormal sample feature.

In some embodiments, the noise adding module 5553 is further configured to perform the following processing for each of the feature clusters: determining the feature distribution of the feature cluster; generating data conforming to the feature distribution of the feature cluster; and sampling from the data conforming to the characteristic distribution of the characteristic cluster to obtain second noise corresponding to the characteristic cluster.

In some embodiments, the noise adding module 5553 is further configured to add noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster, and obtain a second abnormal sample feature corresponding to each first abnormal sample feature, where the second abnormal sample feature corresponds to each first abnormal sample feature, and the exogenous image does not include the target object; extracting exogenous image characteristics of the exogenous image; one of the following steps is performed: fusing the second abnormal sample feature and the exogenous image feature to obtain a third abnormal sample feature; and determining the exogenous image characteristic as a third abnormal sample characteristic.

In some embodiments, the noise adding module 5553 is further configured to add noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster, obtain the second abnormal sample feature corresponding to each first abnormal sample feature, extract a normal sample feature of a normal image sample, and generate third noise; the normal image sample belongs to a target object, and when the target object is detected based on the normal image sample, the obtained detection result represents that the target object is normal; calling an abnormal characteristic generating model to generate a fourth abnormal sample characteristic based on the normal sample characteristic and the third noise; and fusing the second abnormal sample characteristic and the fourth abnormal sample characteristic to obtain a fifth abnormal sample characteristic.

In some embodiments, the noise adding module 5553 is further configured to acquire an initial abnormal feature generation model and a first normal sample feature of the first normal image sample, and generate a fourth noise; invoking the initial abnormal feature generation model to generate a sixth abnormal sample feature based on the first normal sample feature and the fourth noise; invoking a feature discrimination model to discriminate the sixth abnormal sample feature to obtain a feature discrimination result, wherein the feature discrimination result indicates the possible degree of the sixth abnormal sample feature being a true abnormal sample feature; determining a value of a loss function of the initial abnormal feature generation model based on the real abnormal sample feature and the feature discrimination result; and training the initial abnormal feature generation model based on the value of the loss function to obtain the abnormal feature generation model.

In some embodiments, the noise adding module 5553 is further configured to obtain a first weight of the second abnormal sample feature and obtain a second weight of the fourth abnormal sample feature; and weighting the second abnormal sample feature and the fourth abnormal sample feature based on the first weight and the second weight to obtain the fifth abnormal sample feature.

In some embodiments, the noise adding module 5553 is further configured to add noise to the first abnormal sample feature corresponding to the cluster center of each feature cluster, obtain second abnormal sample features corresponding to each first abnormal sample feature, and score a plurality of the second abnormal sample features respectively, so as to obtain a scoring result of each second abnormal sample feature, where the scoring result indicates a degree of proximity between the second abnormal sample feature and a true abnormal sample feature; and selecting target abnormal sample characteristics with scoring results meeting scoring conditions from a plurality of second abnormal sample characteristics based on scoring results of the second abnormal sample characteristics.

In some embodiments, the clustering module 5552 is further configured to determine, after the clustering processing is performed on the plurality of abnormal sample features to obtain at least one feature cluster, a first feature cluster from the at least one feature cluster, and determine a distance between each second feature cluster and the first feature cluster, where the second feature cluster is a feature cluster different from the first feature cluster in the at least one feature cluster; determining a third feature cluster with the distance higher than a distance threshold value and a fourth feature cluster with the distance not higher than the distance threshold value from the at least one feature cluster; the noise adding module 5553 is further configured to add noise to a first abnormal sample feature corresponding to a cluster center of each third feature cluster, so as to obtain a second abnormal sample feature corresponding to each third feature cluster; noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the fourth characteristic clusters, and second abnormal sample characteristics corresponding to the fourth characteristic clusters are obtained; the number of the second abnormal sample features corresponding to each third feature cluster is smaller than the number of the second abnormal sample features corresponding to each fourth feature cluster.

It should be noted that, the description of the embodiments of the device in the present disclosure is similar to the description of the embodiments of the method described above, and has similar beneficial effects as those of the embodiments of the method, which are not described herein. The technical details that are not described in the feature processing apparatus provided in the embodiment of the present application may be understood based on the description of the technical details in the foregoing method embodiment.

Embodiments of the present application also provide a computer program product comprising computer-executable instructions or a computer program stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instructions or the computer program from the computer readable storage medium, and the processor executes the computer executable instructions or the computer program, so that the electronic device executes the feature processing method provided by the embodiment of the application.

The embodiment of the application also provides a computer readable storage medium, in which computer executable instructions or a computer program are stored, which when executed by a processor, cause the processor to execute the feature processing method provided by the embodiment of the application.

In some embodiments, the computer readable storage medium may be RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; but may be a variety of devices including one or any combination of the above memories.

In some embodiments, computer-executable instructions may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, in the form of programs, software modules, scripts, or code, and they may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

As an example, computer-executable instructions may, but need not, correspond to files in a file system, may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext markup language (Hyper Text Markup Language, HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

As an example, computer-executable instructions may be deployed to be executed on one electronic device or on multiple electronic devices located at one site or distributed across multiple sites and interconnected by a communication network.

The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1.A method of feature processing, the method comprising:

Clustering the abnormal sample features to obtain at least one feature cluster;

2. The method of claim 1, wherein before adding noise to the first abnormal sample feature corresponding to the cluster center of each of the feature clusters to obtain the second abnormal sample feature corresponding to each of the first abnormal sample features, the method further comprises:

generating at least one first noise, wherein each first noise is different;

noise is added to a first abnormal sample feature corresponding to a cluster center of each feature cluster, so as to obtain a second abnormal sample feature corresponding to each first abnormal sample feature, including:

The following processing is respectively executed for each characteristic cluster:

And respectively adding the first noise to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain at least one second abnormal sample feature corresponding to the first abnormal sample feature.

3. The method of claim 2, wherein the generating at least one first noise comprises:

At least one sampling result is obtained by sampling the data conforming to the first distribution at least once, each sampling result is used as each first noise, and the sampling results are different;

Or acquiring data conforming to at least one second distribution, and sampling the data conforming to the second distribution to obtain the first noise for each second distribution, wherein the second distributions are different.

4. The method of claim 1, wherein before adding noise to the first abnormal sample feature corresponding to the cluster center of each of the feature clusters to obtain the second abnormal sample feature corresponding to each of the first abnormal sample features, the method further comprises:

Generating second noise corresponding to each characteristic cluster;

and adding second noise corresponding to the feature cluster to a first abnormal sample feature corresponding to the cluster center of the feature cluster to obtain a second abnormal sample feature corresponding to the first abnormal sample feature.

5. The method of claim 4, wherein generating the second noise for each of the feature clusters comprises:

determining the feature distribution of the feature cluster;

generating data conforming to the feature distribution of the feature cluster;

And sampling from the data conforming to the characteristic distribution of the characteristic cluster to obtain second noise corresponding to the characteristic cluster.

6. The method of claim 1, wherein after adding noise to the first abnormal sample feature corresponding to the cluster center of each of the feature clusters to obtain the second abnormal sample feature corresponding to each of the first abnormal sample features, the method further comprises:

Acquiring an exogenous image, wherein the exogenous image does not comprise the target object;

extracting exogenous image characteristics of the exogenous image;

one of the following steps is performed:

Fusing the second abnormal sample feature and the exogenous image feature to obtain a third abnormal sample feature;

And determining the exogenous image characteristic as a third abnormal sample characteristic.

7. The method of claim 1, wherein after adding noise to the first abnormal sample feature corresponding to the cluster center of each of the feature clusters to obtain the second abnormal sample feature corresponding to each of the first abnormal sample features, the method further comprises:

Extracting normal sample characteristics of a normal image sample and generating third noise;

the normal image sample belongs to a target object, and when the target object is detected based on the normal image sample, the obtained detection result represents that the target object is normal;

Calling an abnormal characteristic generating model to generate a fourth abnormal sample characteristic based on the normal sample characteristic and the third noise;

and fusing the second abnormal sample characteristic and the fourth abnormal sample characteristic to obtain a fifth abnormal sample characteristic.

8. The method of claim 7, wherein the method further comprises:

Acquiring an initial abnormal characteristic generation model and first normal sample characteristics of a first normal image sample, and generating fourth noise;

Invoking the initial abnormal feature generation model to generate a sixth abnormal sample feature based on the first normal sample feature and the fourth noise;

invoking a feature discrimination model to discriminate the sixth abnormal sample feature to obtain a feature discrimination result, wherein the feature discrimination result indicates the possible degree of the sixth abnormal sample feature being a true abnormal sample feature;

Determining a value of a loss function of the initial abnormal feature generation model based on the real abnormal sample feature and the feature discrimination result;

And training the initial abnormal feature generation model based on the value of the loss function to obtain the abnormal feature generation model.

9. The method of claim 7, wherein said fusing the second abnormal sample feature and the fourth abnormal sample feature to obtain a fifth abnormal sample feature comprises:

Acquiring a first weight of the second abnormal sample feature and a second weight of the fourth abnormal sample feature;

And weighting the second abnormal sample feature and the fourth abnormal sample feature based on the first weight and the second weight to obtain the fifth abnormal sample feature.

10. The method of claim 1, wherein after adding noise to the first abnormal sample feature corresponding to the cluster center of each of the feature clusters to obtain the second abnormal sample feature corresponding to each of the first abnormal sample features, the method further comprises:

Scoring the plurality of second abnormal sample features to obtain scoring results of the second abnormal sample features, wherein the scoring results indicate the degree of closeness between the second abnormal sample features and the real abnormal sample features;

And selecting target abnormal sample characteristics with scoring results meeting scoring conditions from a plurality of second abnormal sample characteristics based on scoring results of the second abnormal sample characteristics.

11. The method of claim 1, wherein after clustering the plurality of abnormal sample features to obtain at least one feature cluster, the method further comprises:

Determining a first feature cluster from the at least one feature cluster, and determining a distance between each second feature cluster and the first feature cluster, wherein the second feature cluster is a feature cluster different from the first feature cluster in the at least one feature cluster;

determining a third feature cluster with the distance higher than a distance threshold value and a fourth feature cluster with the distance not higher than the distance threshold value from the at least one feature cluster;

Noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the third characteristic clusters, and second abnormal sample characteristics corresponding to the third characteristic clusters are obtained;

Noise is added to the first abnormal sample characteristics corresponding to the cluster centers of the fourth characteristic clusters, and second abnormal sample characteristics corresponding to the fourth characteristic clusters are obtained;

the number of the second abnormal sample features corresponding to each third feature cluster is smaller than the number of the second abnormal sample features corresponding to each fourth feature cluster.

12. A feature processing apparatus, the apparatus comprising:

13. An electronic device, the electronic device comprising:

A memory for storing computer executable instructions;

A processor for implementing the feature processing method of any one of claims 1 to 11 when executing computer-executable instructions stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, which, when executed by a processor, implements the feature processing method of any one of claims 1 to 11.

15. A computer program product comprising computer-executable instructions or a computer program which, when executed by a processor, implements the feature processing method of any one of claims 1 to 11.