WO2022166325A1 - Multi-label class equalization method and device - Google Patents

Multi-label class equalization method and device Download PDF

Info

Publication number
WO2022166325A1
WO2022166325A1 PCT/CN2021/132897 CN2021132897W WO2022166325A1 WO 2022166325 A1 WO2022166325 A1 WO 2022166325A1 CN 2021132897 W CN2021132897 W CN 2021132897W WO 2022166325 A1 WO2022166325 A1 WO 2022166325A1
Authority
WO
WIPO (PCT)
Prior art keywords
instance
subclass
class
target
instances
Prior art date
Application number
PCT/CN2021/132897
Other languages
French (fr)
Chinese (zh)
Inventor
王俊凯
李亚敏
刘新春
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022166325A1 publication Critical patent/WO2022166325A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present application relates to the technical field of artificial intelligence and computer vision, and in particular, to a multi-label class equalization method and device thereof.
  • Object detection is an important research direction in the field of computer vision and image processing.
  • most of the data types of target detection in real life have long-tail effects, that is, the head category accounts for the majority of samples, and the tail category has very few samples.
  • the number of instances of trains and cars in the 2D dataset BDD differs by a factor of 600
  • the number of instances of bicycles and cars in the 3D dataset nuScenes differs by a factor of 4.
  • sampling techniques address the long tail of the data.
  • sampling techniques often result in scene loss or indirectly increase the number of other classes.
  • An embodiment of the present invention provides a multi-label class equalization method, the method includes: acquiring a sample set including multiple instances, each instance including one or more class labels; The instances in the set are classified according to categories to form multiple categories, each instance is divided into one or more categories, and each category corresponds to a category; according to the number of target category labels in each instance in the subcategory and each The number of all class labels in the instance, to determine the target sampling weight of each instance in each subclass, where the subclass is a plurality of classes whose number is less than the first threshold, and the target class label is each The class label corresponding to the subclass in each instance, the target sampling weight is the sampling weight corresponding to the subclass in each instance; according to the target sampling weight of each instance in the subclass, for each subclass Instances of the class are weighted for sampling.
  • This application establishes a weight model for the subclasses, calculates the target sampling weight of each instance of each class, and performs weighted sampling on the instances of each subclass to add new instances. Only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all category labels, and the impact on other categories is small, which can avoid scene loss and indirectly increase other categories. number of categories.
  • the method further includes: constructing the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
  • the number of instances in the subclass is increased by constructing the subclass, so that the long tail effect can be further avoided.
  • the method further includes: constructing instances in the subclass to a first number; judging whether the number of instances in each subclass reaches a second number; The number of instances does not reach the second number, continue to classify the instances in the sample set according to the class labels to form multiple classes, continue to determine the target sampling weight of each instance in each subclass, continue to Weighted sampling is performed on instances of each subclass, and the target instance determined according to the weighted sampling result is continued to construct the subclass as a new instance until the number of instances in each subclass reaches the second number.
  • the amount of data of small categories can be increased in a targeted manner, and based on the iterative sampling strategy, due to new instances, the preset target number of instances, the first number and the second number will be adjusted in each round of sampling, and the total number of samples will not increase sharply , while further avoiding indirect increases in the number of other categories.
  • the method further includes: if the number of instances in each subclass reaches a second number, outputting instances of all classes.
  • the instances of all classes are output, so that the training device can train the initial AI model according to the instances to obtain the target model.
  • the method before the constructing the instances in the subclass to a first number, the method further comprises: determining the first number.
  • the first quantity can be adjusted in each round of sampling.
  • the determining the first number includes: determining the first number to be an average of the numbers of current instances of all classes.
  • the first number is the average value of the number of current instances of all classes, it avoids that the N aim is too large and the total number of instances in the sample set is too large, and that the N aim is too small. The resulting number of iterations is too large and the convergence is slow.
  • the method before the judging whether the number of instances in each subclass reaches a second number, the method further includes: determining the second number.
  • the second quantity can be adjusted in each round of sampling.
  • the determining the second number includes: determining the second number as r*N max , where r ⁇ [0.5, 0.8], and N max is the number of instances of the largest class.
  • the second number is r*N max , where r ⁇ [0.5,0.8], and N max is the number of instances of the largest class, which avoids that the second number is too large to easily lead to duplicate samples in the new sample set If the second number is too small, it is easy to cause overfitting of the training process, and it is avoided that the second number is too small and the equalization effect of each category is not obvious.
  • the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
  • the weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method, so that the newly added instances mainly include instances with more target category labels, which will reduce the imbalance of various categories.
  • the performing weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass includes: generating a preset interval for each instance in the subclass According to the random number and the target sampling weight, the sampling score of each instance in the subclass is determined by the weighted random sampling method WRS; the target instance with the largest sampling score is taken as the newly added instance.
  • the weighted random sampling method (WRS) is used to make the sampling score of an instance with a larger target sampling weight greater. been sampled.
  • the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include: for all instances in the subclass, according to the target sampling weight Sort from small to large; generate random numbers in a preset interval; determine new instances by naive weighted sampling method according to the random number and the sorting of the target sampling weight.
  • the naive weighted sampling method is used to make the instances with larger target sampling weights have larger accumulated values one by one.
  • the instances with larger target sampling weights will be easier to be collected. sample out.
  • the target sampling weight of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Before, the method further comprises: determining a subclass of the plurality of classes.
  • the subclass can be sampled.
  • the determining of the subclasses in the plurality of the classes includes: determining the sampling rate of each class according to the number of current instances of each class and the number of preset target instances; determining the sampling rate A class greater than zero is a subclass of a plurality of said classes.
  • the determining the sampling rate of each class according to the number of current instances of each class and the preset target instance number includes: according to the formula The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t.
  • the case passed the formula The sampling rate, and thus the subclass, can be determined.
  • determining the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance includes the following steps: : According to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the proportion of target class labels for each instance in each subclass; The proportion of target class labels for each instance determines the target sampling weight for each instance in each subclass.
  • the weight model is adopted, so that the greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the target sampling weight of the instance of the class; and The lower the number of target class labels for instances in each subclass relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class.
  • the target class label of each instance in each subclass determines the target class label of each instance in each subclass.
  • the proportion includes: according to the formula Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t.
  • the number of labels, m ti is the number of all class labels in instance i in subclass t.
  • the determining the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance in each subclass includes: according to the formula Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t, is the sum of the proportions of the target class labels of all instances in the small class t.
  • An embodiment of the present invention further provides a multi-label class equalization device, the device includes: an acquisition unit configured to acquire a sample set including multiple instances, each instance including one or more class labels ; Sampling unit, the sampling unit is used to classify the instances in the sample set according to the categories to form a plurality of categories according to the category labels, each instance is divided into one or more categories, and each category corresponds to a category; the sampling unit is used to determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the The subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is the corresponding class label in each instance.
  • the sampling weight of the subclass the sampling unit is further configured to perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
  • This application establishes a weight model for the subclasses, calculates the target sampling weight of each instance of each class, and performs weighted sampling on the instances of each subclass to add new instances. Only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all category labels, and the impact on other categories is small, which can avoid scene loss and indirectly increase other categories. number of categories.
  • the sampling unit is further configured to construct the subclass by using the target instance determined according to the weighted sampling result as a newly added sample.
  • the number of instances in the subclass is increased by constructing the subclass, so that the long tail effect can be further avoided.
  • the sampling unit is further configured to construct a first number of instances in the subclass; the sampling unit is further configured to determine whether the number of instances in each subclass reaches a second number ; The sampling unit is also used for, if the number of instances in each subclass does not reach the second number, continue to classify the instances in the sample set according to the class according to the class label to form multiple classes, and continue to determine A target sampling weight for each instance in each subclass, and continuing weighted sampling of instances in each subclass until the number of instances in each subclass reaches the second number.
  • the amount of data of small categories can be increased in a targeted manner, and based on the iterative sampling strategy, due to new instances, the preset target number of instances, the first number and the second number will be adjusted in each round of sampling, and the total number of samples will not increase sharply , while further avoiding indirect increases in the number of other categories.
  • the sampling unit is further configured to output instances of all classes if the number of instances in each subclass reaches the second number.
  • the instances of all classes are output, so that the training device can train the initial AI model according to the instances to obtain the target model.
  • the sampling unit is further configured to determine the first number.
  • the first quantity can be adjusted during each sampling.
  • the sampling unit is further configured to determine that the first number is an average value of the number of current instances of all classes.
  • the first number is the average value of the number of current instances of all classes, it avoids that the N aim is too large and the total number of instances in the sample set is too large, and that the N aim is too small. The resulting number of iterations is too large and the convergence is slow.
  • the sampling unit is further configured to determine the second number.
  • the second quantity can be adjusted in each sampling.
  • the sampling unit is further configured to determine the second number as r*N max , where r ⁇ [0.5, 0.8], and N max is the number of instances of the largest class.
  • the second number is r*N max , where r ⁇ [0.5, 0.8], and N max is the number of instances of the largest class, which avoids that the second number is too large to easily lead to repetition in new sample sets If the number of samples is large, it is easy to cause overfitting, and it is avoided that the second number is too small and the equalization effect of each category is not obvious.
  • the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
  • the weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method, so that the newly added instances mainly include instances with more target category labels, which will reduce the imbalance of various categories.
  • the sampling unit is further configured to generate a random number of a preset interval for each instance in the subclass; according to the random number and the target sampling weight, the weighted random sampling method WRS is used to determine The sampling score of each instance in the subclass; the target instance with the largest sampling score is taken as the newly added instance.
  • the weighted random sampling method (WRS) is used to make the sampling score of an instance with a larger target sampling weight higher. been sampled.
  • the sampling unit is further configured to sort all the instances in the subclass according to the target sampling weight in ascending order; generate a random number in a preset interval; according to the random number and the The ordering of the target sampling weights determines the newly added instances through the naive weighted sampling method.
  • the naive weighted sampling method is used to make the instances with larger target sampling weights have larger accumulated values one by one.
  • the instances with larger target sampling weights will be easier to be collected. sample out.
  • the target sampling weight of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Before: the sampling unit is also used to determine subclasses of the plurality of classes.
  • the subclass can be sampled.
  • the sampling unit is further configured to determine the sampling rate of each class according to the number of current instances of each class and the preset target number of instances; determine that there are multiple classes with the sampling rate greater than zero A subclass within the described class.
  • the sampling unit is further configured to The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t.
  • the case passed the formula The sampling rate, and thus the subclass, can be determined.
  • the sampling unit is configured to determine, according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, the Proportion of target class labels; the sampling unit is further configured to determine the target sampling weight of each instance in each subclass according to the ratio of target class labels of each instance of each subclass.
  • the weight model is adopted, so that the greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the target sampling weight of the instance of the class; and The lower the number of target class labels for instances in each subclass relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class.
  • the sampling unit is further configured to Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t
  • the number of labels, m ti is the number of all class labels in instance i in subclass t.
  • the sampling unit is further configured to Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t, is the sum of the proportions of the target class labels of all instances in the small class t.
  • An embodiment of the present invention further provides an electronic device, the electronic device includes a processor and a memory, the memory is used for storing program instructions, when the processor calls the program instructions, the implementation of any one of the above The multi-label class equalization method.
  • An embodiment of the present invention also provides a vehicle including the electronic device as described above.
  • An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a program, and the program enables a computer device to implement the multi-tag equalization-like method described in any one of the above.
  • An embodiment of the present invention also provides a computer program product, the computer program product comprising computer-executable instructions stored in a computer-readable storage medium; at least one processor of the device can be obtained from the computer-executable instructions.
  • the computer-executable instructions are read from the reading storage medium, and the at least one processor executes the computer-executable instructions to cause the device to perform the multi-tag equalization-like method described in any one of the above.
  • FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of a multi-label class equalization method according to the first embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an example of an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of the number of instances in corresponding categories after the instances in the sample set are classified according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a category label of category t according to an embodiment of the present invention.
  • FIG. 6 is a schematic flowchart of a multi-label class equalization method according to a second embodiment of the present invention.
  • FIG. 7 is a schematic diagram of constructing instances of the subclass so that the number of instances of each subclass reaches a first number according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of iteratively constructing a subclass for the subclass according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a multi-label equalizer-like apparatus according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • words such as “for example” are used to represent examples, illustrations or illustrations. Any embodiment or design described in the embodiments of the present application as “for example,” should not be construed as preferred or advantageous over other embodiments or designs. Rather, use of words such as “such as” is intended to present the related concepts in a specific manner.
  • Existing mixed-sampling-based data equalization methods form new datasets by instance classification, computing averages, random upsampling, random downsampling, and combining.
  • the instance classification includes counting the number of instances corresponding to each category.
  • the calculating the average value includes accumulating the number of instances of each category, and then calculating the average value.
  • the random up-sampling includes performing up-sampling by random sampling with replacement on the category whose number of instances is lower than the average value, so that the number of instances is equal to the average value.
  • the random down-sampling includes down-sampling the classes whose number of instances exceeds the average, so that the number of instances is equal to the average.
  • the combining includes recombining the class instances to form a new dataset.
  • Example It generally refers to the image samples (Image) in the 2D data set and the point cloud samples (Point Cloud) in the 3D data set.
  • Sample set a set of several instances, each element in the set is an instance.
  • Category refers to the distinction between different things, which is a distinction made according to different types.
  • Class A subset defined on the set of samples composed of all instances, samples of the same class are indistinguishable in a property we care about, that is, have the same pattern.
  • Small class refers to the class whose number is less than the first threshold among multiple classes.
  • the model may be an AI model.
  • AI model is a machine learning model, which is essentially a mathematical model including a large number of parameters and mathematical formulas (or mathematical rules), the purpose is to learn some mathematical expressions, so that the mathematical expressions can provide input value x and output value y
  • the correlation between, and the mathematical expression that can provide the correlation between x and y is the trained AI model.
  • the initial AI model needs to be trained before using the AI model for detection.
  • the factors affecting AI model training mainly include three aspects: training data set, initial AI model and machine computing power.
  • training data set With the increase of application scenarios of current AI models, the scenarios that AI models need to face become more complex, resulting in more and more complex AI models for training.
  • data volume requirements for the training data set are also increasing, and the data types are also increasing.
  • the amount of calculation in the training process increases, the requirements for machine computing power are also increasing, and the time required for training is getting longer and longer. How to optimize the AI model training process to obtain an AI model with better accuracy in the shortest time is the focus of the industry.
  • the developer can create an instance through the multi-label class equalization method provided by the embodiment of this application, and can use the structure including the structure of this application.
  • the training data set of the instance to be trained is used to train the AI model to be trained, so as to improve the accuracy of model inference.
  • the neural network model to be trained is the initial AI model that needs to be trained.
  • the trained AI model is capable of predictive analytics and can be used for a variety of different purposes.
  • data of interest eg people or objects in images/videos
  • target data eg video streams, picture streams or images
  • the object of interest is called the target
  • the type of the target can be determined according to the application scenario and business direction.
  • the target in the field of monitoring and security, can be but not limited to vehicles, human bodies, faces, etc.
  • the target can be but not limited to roads, license plate numbers, road facilities, traffic signs, pedestrians, etc.
  • the target can be but not limited to container number, courier number, obstacle, package number, etc.
  • the target can be but not limited to terrain, flying equipment, etc.
  • the target can be but not limited to organs, tissues, etc.
  • the trained AI model can also be applied to other scenarios other than the above target detection scenarios, such as recognition, segmentation and other scenarios.
  • the following is an example of the application of the trained AI model to target detection scenarios, but this case does not limit it to only target detection scenarios.
  • FIG. 1 is a schematic diagram of a system architecture 10 according to an embodiment of the present invention.
  • the data collection device 160 is used to collect training data.
  • the training data may include training instances, where the results of the training instances may be manually pre-labeled results or automatically marked by an existing labeling system. Annotated results, i.e. each training instance includes one or more class labels.
  • the data collecting device 160 stores the training data in the database 130 .
  • the training device 120 obtains the target model 101 by training based on the training data maintained in the database 130 .
  • the training device 120 trains the target model 101 based on the training data, and the target model 101 is able to find the data of interest (eg images/people in the video) in the target data (eg video streams, picture streams or images). or object), to determine the position and size of the object in the image or video.
  • the data of interest eg images/people in the video
  • the target data eg video streams, picture streams or images. or object
  • the training device 120 may acquire a sample set including multiple instances, each instance including one or more category labels; and classify the instances in the sample set according to categories according to the category labels Form multiple classes, each instance is divided into one or more classes, and each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, Determine the target sampling weight of each instance in each subclass, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, and the target class label is the corresponding subclass in each instance.
  • the target sampling weight is the sampling weight corresponding to the subclass in each instance; weighted sampling is performed on the instance of each subclass according to the target sampling weight of each instance in the subclass.
  • the training device 120 may also train the initial AI model according to the example to obtain the target model 101 .
  • the training data maintained in the database 130 may not necessarily come from the collection of the data collection device 160, and may also be received from other devices.
  • the training device 120 may not necessarily train the target model 101 entirely based on the training data maintained by the database 130, and may also obtain training data from the cloud or other places for model training, for example, from the 2D data set BDD or 3D data. Collect training data such as nuScenes to perform model training, and the above description should not be taken as a limitation on the embodiments of the present application.
  • the target model 101 trained according to the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in FIG. 1
  • the execution device 110 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer , AR/VR, vehicle terminal, etc., it can also be a server or cloud, etc.
  • the execution device 110 is configured with an I/O interface 112, which is used for data interaction with external devices, and a user can input data to the I/O interface 112 through the client device 140, and the input data is in the embodiment of the present application It can include: the image to be processed input by the client device.
  • the data storage system 150 is used for receiving and storing the parameters of the target model sent by the training device 120, and for storing the data of the target detection results obtained by the target model 101, and of course, it may also include the data required for the normal operation of the data storage system 150.
  • Program code or instructions
  • the data storage system 150 may be a distributed storage cluster composed of one device or multiple devices deployed outside the execution device 110. At this time, when the execution device 110 needs to use the data on the data storage system 150, the data storage system 150 The data required by the execution device 110 is sent to the execution device 110, and accordingly, the execution device 110 receives and stores (or caches) the data. Of course, the data storage system 150 can also be deployed in the execution device 110.
  • the distributed storage system can include one or more memories.
  • different memories different memories
  • the calculation module 111 uses the target model 101 to process the input image to be processed (such as a video stream, picture stream or image), for example, to find data of interest (such as a person or object in the image/video) from the input image to be processed ) to determine the position and size of objects in an image or video.
  • the input image to be processed such as a video stream, picture stream or image
  • data of interest such as a person or object in the image/video
  • the I/O interface 112 returns the processing result, such as the position and size of the object in the image or video obtained as described above, to the client device 140, so as to be provided to the user.
  • the training device 120 can generate corresponding target models 101 based on different training data for different goals or tasks, and the corresponding target models 101 can be used to achieve the above-mentioned goals or complete the above-mentioned tasks. This provides the user with the desired result.
  • the user can manually specify the input data, which can be operated through the interface provided by the I/O interface 112 .
  • the client device 140 can automatically send the input data to the I/O interface 112 . If the user's authorization is required to request the client device 140 to automatically send the input data, the user can set the corresponding permission in the client device 140 .
  • the user can view the result output by the execution device 110 on the client device 140, and the specific presentation form can be a specific manner such as display, sound, and action.
  • the client device 140 can also be used as a data collection terminal to collect the input data of the input I/O interface 112 and the output result of the output I/O interface 112 as new sample data as shown in the figure, and store them in the database 130 .
  • the I/O interface 112 directly uses the input data input into the I/O interface 112 and the output result of the output I/O interface 112 as shown in the figure as a new sample The data is stored in database 130 .
  • FIG. 1 is only a schematic diagram of a system architecture provided by an embodiment of the present invention, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data storage system 150 is an external memory relative to the execution device 110 , and in other cases, the data storage system 150 may also be placed in the execution device 110 .
  • the training device 120, the execution device 110, and the client device 140 may be three different physical devices, or the training device 120 and the execution device 110 may be on the same physical device or a cluster. , it is also possible that the execution device 110 and the client device 140 are on the same physical device or a cluster.
  • FIG. 2 is a schematic flowchart of a multi-label class equalization method according to the first embodiment of the present invention.
  • the method may be specifically performed by the training device 120 shown in FIG. 1 , and the sample sets of the multiple instances in the method may be the training data maintained in the database 130 shown in FIG. 1 .
  • the Steps S202 to S206 of the method may be executed in the training device 120, or may be pre-executed by other functional modules before the training device 120, that is, preprocessing the training data received or acquired from the database 130 first , in the sampling process described in steps S202 to S206, a new sample set is obtained, which is used as the input of the training device 120, and the training device performs model training.
  • the method may be processed by the CPU, or may be jointly processed by the CPU and the GPU, or other processors suitable for processing may be used without using the GPU, which is not limited in this application.
  • the method includes but is not limited to the following steps:
  • S201 Acquire a sample set including multiple instances, each instance including one or more category labels.
  • the sample set may be collected by the data collection device 160 shown in FIG. 1 , wherein each instance in the sample set collected by the data collection device 160 is a result of manual pre-labeling or is automatically labelled by an existing labeling system. result.
  • the sample set may also be obtained from a 2D data set BDD or a 3D data set nuScenes, etc., or a data set from a memory or other places, and the result of automatic labeling by an existing labeling system.
  • the sample set may include all the data sets in the 2D data set BDD or the 3D data set nuScenes, or may only include part of the data set, and this case is described by taking only part of the data set as an example.
  • the sample set as an autonomous driving data set.
  • there are 10 categories in the sample set which are buses, traffic lights, signs, people, bicycles, cars, trucks, motorcycles, trains, and riders.
  • the categories of the sample set are not limited to the above categories, but can also be other categories.
  • the sample set is not limited to the autonomous driving data set, the sample set can also be a data set of dogs from Stanford Dogs Dataset, or a data set of face images from Labelled Faces in the Wild, etc.
  • the categories of the sample sets are also different.
  • the example may be a picture sample in a 2D dataset.
  • the instance may also be a point cloud sample in a 3D dataset.
  • Such an example may be as shown in FIG. 3 .
  • the example includes 6 class labels, namely traffic sign (traffic light), traffic sign (traffic light), rider (rider), bike (bicycle), truck (truck), and car (car) .
  • each object in the instance is marked by a two-dimensional frame.
  • Each category label is used to represent an object.
  • the class labels are indicated on the instance, and the labeling of these class labels helps determine the type of each object.
  • the category label -car indicates that the object selected by the box is a car
  • the category label -traffic sign indicates that the object selected by the box is a traffic light
  • the category label -rider indicates that the object selected is a rider
  • the category label -bike The box-selected object is a bicycle
  • the category label -truck indicates that the box-selected object is a truck.
  • the same class label belongs to a class.
  • the above 6 category labels belong to 5 categories, namely car, traffic light, rider, bicycle, and truck.
  • S202 Classify the instances in the sample set according to the categories to form multiple categories according to the category labels, each instance is divided into one or more categories, and each category corresponds to a category.
  • an instance if an instance includes a category label, the index of the instance is added under the category. If an instance includes multiple category labels, the indices of the instances are respectively added under the corresponding categories. If an instance includes multiple labels of the same category, then add the instance's index to the category once.
  • the above example includes 2 traffic sign (traffic light) class labels, 1 car (car) class label, 1 rider (rider) class label, and 1 bike (bicycle) class label.
  • a truck (truck) category label then add the index number of the above instance under the category traffic light, the category car, the category rider, the category bicycle, and the category truck.
  • the index number of the instance to be added under the category can be shown in Table 1 below:
  • the number of instances in different classes may be the same or different, as shown in FIG. 4 .
  • the car class has the largest number of instances at 27558
  • the bicycle class has the least number of instances at 6263.
  • the head category has the majority of instances
  • the tail category has very few instances. For example, cars and humans account for the majority of instances, and train, rider, motorcycle, and bicycle categories account for very few instances.
  • S203 Determine subclasses in the plurality of classes.
  • the subclass is a class whose number is less than the first threshold among the plurality of classes.
  • the determining of the sub-classes in the plurality of the classes includes: determining the sampling rate of each class according to the number of current instances of each class and the preset target number of instances; determining that the classes with the sampling rate greater than zero are A subclass of multiple said classes. Determining the sampling rate of each class according to the number of current instances of each class and the preset target instance number includes: according to the formula The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t.
  • the N aim may be an average value of the number of instances of each category, or a value slightly larger than the average value of the number of instances of each category, or a value slightly smaller than the average value of the number of instances of each category. Wherein, if the N aim is too large, the total number of instances in the sample set is likely to be too large, and if the N aim is too small, it is likely to cause too many iterations and slow convergence.
  • the class with the sampling rate greater than zero is a large class among the plurality of classes.
  • the number of current instances of the bicycle class is 6263, if the preset target instance number is 14000, then according to the formula If it is determined that the sampling rate is about 1.24, which is greater than zero, it can be determined that the bicycle class is a subclass of the plurality of classes.
  • S204 Determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the target class label is each The class label corresponding to the subclass in each instance, and the target sampling weight is the sampling weight corresponding to the subclass in each instance.
  • each instance may include one or more category labels, and may be grouped into one or more categories.
  • the proportions of different categories included in different instances may or may not be the same.
  • the above examples include categories of traffic lights, categories of cars, categories of riders, categories of bicycles, and categories of trucks that account for 1/3, 1/6, 1/6, and 1/6, respectively. , and 1/6.
  • the proportions of target category labels of different instances under one category may be the same or different, so the new sample sets formed by upsampling according to different instances under the same category are also different.
  • determining the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance includes: The number of target class labels in each instance in the subclass and the number of all class labels in each embodiment, determine the proportion of target class labels for each instance in each subclass; The proportion of target class labels for each instance determines the target sampling weight for each instance in each subclass.
  • the proportion of the target class labels of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Include: According to the formula Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
  • the subclass m is a bicycle
  • the subclass m includes n instances, namely instance_1, . . . , instance_i, . . . , and instance_n.
  • the category labels included in different instances may or may not be the same.
  • the class labels included in instance_1 are bike, truck, bus
  • the class labels included in instance_i are bike, bike, rider, motor
  • the class labels included in instance_n are bike, truck, car .
  • the proportion of the bike category label of the instance i in the bicycle class in all the category labels of the instance i is It can be seen from this that if the number of target class labels of instances in the subclass is greater relative to the number of all class labels in the instance, the proportion of target class labels of the instances of the class is higher ; If the number of target class labels of instances in the subclass is less relative to the number of all class labels in the instance, the proportion of target class labels of the instances of the class is lower.
  • the determining the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance in each subclass includes: according to the formula Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t, is the sum of the proportions of the target class labels of all instances in the small class t.
  • subclass t includes instance i and instance j
  • the proportion of bike category labels of instance i is The proportion of bike category labels for instance j is Then the sum of the proportions of the bike category labels of all instances of the small class t is
  • the target sampling weight of instance i in subclass t is It can be seen from this that if the proportion of the target class label of the instances in each subclass is higher, the target sampling weight of the instance of the class is higher; if the target class label of the instances in each subclass has a higher proportion The lower the proportion, the lower the target sampling weight for the instance of the class.
  • the target sampling weight of the instance of the class may be higher; if each subclass The lower the number of target class labels for an instance in a class relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class may be.
  • S205 Perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
  • the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
  • the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include:
  • the target instance with the largest sampling score is used as the new instance.
  • the preset interval is a (0, 1) interval.
  • the preset interval can also be other intervals, such as (0, 2).
  • a random number generator can be used to generate random numbers in multiple (0, 1) intervals.
  • the determining the sampling score of each instance in the subclass by the weighted random sampling method WRS according to the random number and the target sampling weight includes: according to the formula ti ⁇ tn determines the sampling rate, where S ti is the sampling score of instance i in class t, R ti is the random number of instance i in class t, and w ti is the target sampling weight of instance i in small class t , ti is the ith instance in subclass t, and tn is the nth instance in subclass t.
  • the target sampling weight of instance i in subclass t is The target sampling weight of instance j in subclass t is If the random number of instance i in subclass t is 0.8, and the random number of instance j in subclass t is 0.6, it can be determined that the sampling scores of instance i and instance j in subclass t are 0.69 and 0.28 respectively, then the sampling score Instance i with a value of 0.69 is used as a new instance.
  • the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include:
  • the newly added instance is determined by the naive weighted sampling method according to the order of the random number and the target sampling weight.
  • the preset interval is a (0, 1) interval.
  • the preset interval can also be other intervals, such as (0, 2).
  • a random number generator can be used to generate random numbers in multiple (0, 1) intervals.
  • the determining of the newly added instance by the naive weighted sampling method according to the ordering of the random number and the target sampling weight includes: accumulating the instances of the instance one by one starting from the target sampling weight of the first instance according to the ordering of the target sampling weight. The target sampling weight reaches the random number when the accumulated target sampling weight reaches the target instance; the target instance is regarded as a newly added instance.
  • the target sampling weight of instance i in subclass t is The target sampling weight of instance j in subclass t is Then the target sampling weights are sorted from small to large as If the random number is 0.5, the target sampling weight of the first instance is 0.4, and it does not reach 0.5, and the target sampling weight of the second instance is accumulated, the accumulated target sampling weight is 1, and it reaches 0.5, then the target sampling weight can be determined as The instance of 0.6 is the target instance, and the target instance is regarded as the newly added instance.
  • the weighted random sampling method WRS is used to determine the sampling score of each instance of the sub-category, and the sampling score of an instance with a larger target sampling weight will be higher.
  • the target instance with the largest sampling score as the new instance, the instance with the larger target sampling weight will be easier to be sampled, and the newly added instance will mainly include the instance with more target category labels, which will reduce the Minor categories are not balanced.
  • the naive weighted sampling method is used to determine the accumulated value one by one in the order of the target sampling weight from small to large, and the one-by-one accumulated value of the instance with a larger target sampling weight is larger.
  • the instance with a larger target sampling weight will be easier to be sampled, and the newly added instance will mainly include more target category labels. It will reduce the imbalance of various categories.
  • S206 Construct the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
  • the step of adding an instance is repeated multiple times to construct the subclass. Specifically, the step of adding an instance is repeated several times, that is, step S205, and the new instance is added to the original subclass to construct the subclass. For example, continue to take subclass t as an example to illustrate, subclass t includes instance i and instance j, if the new instance includes instance k and instance l, the constructed subclass includes: instance i, instance j, instance k and Example l.
  • the method may further train the initial AI model according to the instance to obtain the target model.
  • a sample set including multiple instances is obtained, and each instance includes one or more class labels; the instances in the sample set are classified according to classes to form multiple classes according to the class labels, and each instance is classified into multiple classes.
  • each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the value of each instance in each subclass Target sampling weight, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is each
  • Each instance corresponds to the sampling weight of the subclass; according to the target sampling weight of each instance in the subclass, weighted sampling is performed on the instance of each subclass, and only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all
  • FIG. 6 is a schematic flowchart of a multi-label class equalization method according to a second embodiment of the present invention.
  • the method may be specifically performed by the training device 120 shown in FIG. 1 , and the sample sets of the multiple instances in the method may be the training data maintained in the database 130 shown in FIG. 1 .
  • the Steps S602 to S612 of the method may be executed in the training device 120, or may be pre-executed by other functional modules before the training device 120, that is, preprocessing the training data received or acquired from the database 130 first , according to the sampling process described in steps S602 to S612, a new sample set is obtained, which is used as the input of the training device 120, and the training device performs model training.
  • the method may be processed by the CPU, or may be jointly processed by the CPU and the GPU, or other processors suitable for processing may be used without using the GPU, which is not limited in this application.
  • the method includes but is not limited to the following steps:
  • S601 Acquire a sample set including multiple instances, each instance including one or more category labels.
  • Step S601 in the second embodiment is similar to step S201 in the first embodiment.
  • Step S601 in the second embodiment is similar to step S201 in the first embodiment.
  • S602 Classify the instances in the sample set into multiple classes according to the class labels, each instance is classified into one or more classes, and each class corresponds to a class.
  • Step S602 in the second embodiment is similar to step S202 in the first embodiment.
  • Step S602 in the first embodiment please refer to the detailed description of step S202 in the first embodiment in FIG. 2 , which will not be repeated here.
  • S603 Determine subclasses among the plurality of classes.
  • step S203 The process of determining subclasses in the plurality of classes in the second embodiment is similar to the process of determining subclasses in the classes in step S203 of the first embodiment.
  • the detailed description of step S203 in an embodiment is not repeated here.
  • the iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling.
  • a subclass within a class is adopted.
  • S604 Determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the target class label is each The class label corresponding to the subclass in each instance, and the target sampling weight is the sampling weight corresponding to the subclass in each instance.
  • Step S604 in the second embodiment is similar to step S204 in the first embodiment. For details, please refer to the detailed description of step S204 in the first embodiment in FIG. 2, which will not be repeated here.
  • S605 Perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
  • Step S605 in the second embodiment is similar to step S205 in the first embodiment.
  • Step S605 in the first embodiment please refer to the detailed description of step S205 in the first embodiment in FIG. 2 , which will not be repeated here.
  • S606 Construct the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
  • Step S606 in the second embodiment is similar to step S206 in the first embodiment.
  • Step S606 in the first embodiment please refer to the detailed description of step S206 in the first embodiment in FIG. 2 , which will not be repeated here.
  • the determining of the first number includes: determining the first number to be an average of the numbers of current instances of all classes.
  • the first number may also be other numbers, for example, a value slightly larger than the average value of the number of instances of each category, or a value slightly smaller than the average value of the number of instances of each category.
  • the first number may be the same as the preset target instance number, or may be different from the preset target instance number.
  • an iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling, and the first number needs to be determined during each round of sampling.
  • S608 Construct instances in the subclass to a first number.
  • constructing the instances in the subclass to the first number includes:
  • step S605 to step S606 are continued until the number of instances in the subclass reaches the first number.
  • the method before generating a random number of a preset interval for each instance of the shown subclass, the method further includes: determining, for the subclass, the number of current instances of the subclass and the first number to be The number of instances constructed.
  • the instances in the subclass are repeatedly weighted Sampling to get delta new instances.
  • the number of instances in each subclass can be brought up to the first number.
  • FIG. 7 the schematic diagram after weighted sampling is performed on each subclass is shown in FIG. 7 .
  • the number of instances of all subcategories such as sign, bus, train, rider, motorcycle, and bicycle are increased to the first number.
  • the determining of the second number includes: determining the second number as r*N max , where r ⁇ [0.5, 0.8], and N max is the number of instances of the largest class.
  • the r is not limited to the above-mentioned values, and the r can also be other values, such as 0.48, 0.81, and the like. Wherein, if the second number is too large, the number of repeated samples in the new sample set is likely to be large, which may easily lead to over-fitting, and if the second number is too small, the equalization effect of each category is likely to be insignificant.
  • the iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling, and the second number needs to be determined during each round of sampling.
  • S611 If the number of instances in each sub-category does not reach the second number, continue to classify the instances in the sample set according to categories to form multiple categories according to the category labels, and continue to determine the number of instances in each sub-category. the target sampling weight of each instance, continue to perform weighted sampling on the instances of each subclass, and continue to construct the subclasses with the target instances determined according to the weighted sampling results as new instances until the number of instances in each subclass reaches second quantity.
  • steps S602-S610 are repeated, so that the number of instances in each subclass can reach the second number.
  • FIG. 8 a schematic diagram of iteratively constructing a subclass for each subclass is shown in FIG. 8 .
  • the number of instances of all subcategories such as truck, traffic light, sign, bus, train, rider, motorcycle, and bicycle, is increased to the second number.
  • the method may further train the initial AI model according to the instance to obtain the target model.
  • a sample set including multiple instances is obtained, and each instance includes one or more class labels; the instances in the sample set are classified according to classes to form multiple classes according to the class labels, and each instance is classified into multiple classes.
  • each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the value of each instance in each subclass Target sampling weight, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is each The sampling weights corresponding to the subclasses in each instance; according to the target sampling weights of each instance in the subclasses, weighted sampling is performed on the instances of each subclass; the target instances determined according to the weighted sampling results are used as new Instance constructs the subclass; constructs the instances in the subclass to the first number; determines whether the number of instances in each subclass
  • Retain do not lose instances, and at the same time make the number of target class labels in the subclass easier to be upsampled compared to instances with more class labels, and has less impact on other classes.
  • the preset target number of instances the first number and the second number will be adjusted in each round of sampling, which can ensure that the number of instances in each category reaches the second number before exiting the iteration, further avoiding the impact on other categories. , which can avoid scene loss and indirectly increase the number of other categories, thus preserving the original data and solving the problem of category imbalance.
  • FIG. 9 is a schematic structural diagram of a multi-label quasi-equalization apparatus according to an embodiment of the present invention.
  • the multi-label quasi-equalization apparatus 900 may include an acquisition unit 901 and a sampling unit 902 .
  • the multi-label class equalization apparatus 900 may be used to execute the steps of the multi-label class equalization method of the embodiment of the present application.
  • the acquiring unit 901 may be configured to perform step S201 in the method shown in FIG. 2
  • the sampling unit 902 may be configured to perform steps S202 to S206 in the method shown in FIG. 2 .
  • the acquiring unit 901 may be configured to perform step S601 in the method shown in FIG. 6
  • the sampling unit 902 may be configured to perform steps S602 to S612 in the method shown in FIG. 6 .
  • the multi-label equalization-like apparatus 900 may further include a training unit 903 .
  • the training unit 903 is configured to train the initial AI model according to the instance to obtain the target model.
  • FIG. 10 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.
  • the electronic device 1000 shown in FIG. 10 includes a memory 1001 , a processor 1002 , a communication interface 1003 and a bus 1004 .
  • the memory 1001 , the processor 1002 , and the communication interface 1003 are connected to each other through the bus 1004 for communication.
  • the memory 1001 may be a read-only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM).
  • the memory 1001 may store a program. When the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 and the communication interface 1003 are used to execute each step of the multi-tag equalization-like method of the embodiment of the present application.
  • the processor 1002 may adopt a general-purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a graphics processing unit (graphics processing unit, GPU) or one or more
  • the integrated circuit is used to execute the relevant program to realize the functions required to be performed by the units in the multi-tag equalization-like apparatus of the embodiment of the present application, or to execute the multi-tag equalization-like method of the method embodiment of the present application.
  • the processor 1002 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the multi-tag quasi-equalization method of the present application may be completed by hardware integrated logic circuits in the processor 1002 or instructions in the form of software.
  • the above-mentioned processor 1002 can also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processing
  • ASIC application-specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • FPGA Field Programmable Gate Array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001 and, in combination with its hardware, completes the functions required to be performed by the units included in the multi-tag equalizer-like device of the embodiment of the present application, or executes the method implementation of the present application.
  • the communication interface 1003 implements communication between the electronic device 1000 and other devices or a communication network using a transceiver such as but not limited to a transceiver.
  • a transceiver such as but not limited to a transceiver.
  • data can be acquired through the communication interface 1003 .
  • Bus 1004 may include a pathway for communicating information between various components of electronic device 1000 (eg, memory 1001, processor 1002, communication interface 1003).
  • the electronic device 1000 may further include an output component, such as a display, a sound box, etc., the output component is used to display the parameters to be used for training the model to the developer, so the developer can know these parameters, or These parameters are modified, and the modified parameters are input into the electronic device 1000 through an input component (eg, mouse, keyboard, etc.).
  • the electronic device 1000 can also display the trained target model to the developer through the output component.
  • the acquisition unit 901 in the multi-tag equalization-like apparatus 900 is equivalent to the communication interface 1003 in the electronic device 1000
  • the sampling unit 902 may be equivalent to the processor 1002 .
  • FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • the electronic device 1100 includes a memory 1101 , a processor 1102 , a communication interface 1103 and a bus 1104 .
  • the memory 1101 , the processor 1102 , and the communication interface 1103 are connected to each other through the bus 1104 for communication.
  • the memory 1101 may be a read only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM).
  • the memory 1101 can store programs, and when the programs stored in the memory 1101 are executed by the processor 1102, the processor 1102 and the communication interface 1103 are used to execute various steps of the target detection method.
  • the processor 1102 may adopt a general-purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a graphics processing unit (graphics processing unit, GPU) or one or more An integrated circuit for executing the associated program to perform the object detection method.
  • CPU Central Processing Unit
  • ASIC Application Specific Integrated Circuit
  • GPU graphics processing unit
  • the processor 1102 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the target detection method may be completed by hardware integrated logic circuits in the processor 1102 or instructions in the form of software.
  • the above-mentioned processor 1102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices. , discrete gate or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processing
  • ASIC application-specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • FPGA Field Programmable Gate Array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and completes the target detection method in combination with its hardware.
  • the communication interface 1103 uses a transceiver such as but not limited to a transceiver to implement communication between the electronic device 1100 and other devices or a communication network.
  • a transceiver such as but not limited to a transceiver to implement communication between the electronic device 1100 and other devices or a communication network.
  • the communication interface 1103 establishes a communication connection with the above-mentioned training device, and the parameters of the target model sent by the above-mentioned training device can be received through the communication interface 1103 .
  • the parameters of the target model can be stored in the memory 1101 for recall.
  • the bus 1104 may include a pathway for communicating information between the various components of the electronic device 1100 (eg, the memory 1101, the processor 1102, the communication interface 1103).
  • the electronic device 1100 may further include an output component, such as a display, a sound box, etc., where the output component is used to display the target detection result obtained by the target model to the user.
  • an output component such as a display, a sound box, etc.
  • the target detection method may be any existing target detection method, which will not be repeated here.
  • the electronic device 1000 and the electronic device 1100 shown in FIG. 10 and FIG. 11 only show a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the electronic device 1000 and the electronic device 1100 The electronic device 1100 also includes other components necessary for normal operation. Meanwhile, according to specific needs, those skilled in the art should understand that the electronic device 1000 and the electronic device 1100 may further include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device electronic device 1000 and the electronic device 1100 may also only include the necessary devices for implementing the embodiments of the present application, and do not necessarily include all the devices shown in FIG. 10 or FIG. 11 .
  • the electronic device 1000 is equivalent to the training device 120 in 1, and the electronic device 1100 is equivalent to the execution device 110 in FIG. 1 .
  • the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.
  • an embodiment of the present invention also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium runs on a processor, the implementation shown in FIG. 2 or FIG. 6 is implemented.
  • the multi-label class equalization method is implemented.
  • a computer program product comprising computer-executable instructions stored in a computer-readable storage medium; from which the computer-readable storage medium can be read by at least one processor of a device Computer-executed instructions, the at least one processor executing the computer-executed instructions causes the device to implement the multi-tag equalization-like method shown in FIG. 2 or FIG. 6 .
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Stored Programmes (AREA)

Abstract

A multi-label class equalization method. The method comprises: obtaining a sample set comprising multiple instances, each instance comprising one or more category labels; classifying, according to the category labels, the instances in the sample set on the basis of categories to form multiple classes, each instance being divided into one or more classes, and each class corresponding to a category; determining a target sampling weight of each instance in each subclass according to the number of target category labels in each instance in each subclass and the number of all category labels in each instance, wherein the subclasses are classes, among the plurality of classes, of which the number is less than a first threshold, the target category labels are category labels corresponding to the subclass in each instance, and the target sampling weight is a sampling weight corresponding to the subclass in each instance; and performing weighted sampling on the instances of each subclass according to the target sampling weight in each instance in the subclass. The present invention further provides a device, such that scenario loss and indirect increase of the number of other categories are avoided.

Description

多标签的类均衡方法及其装置Multi-label class equalization method and device
本申请要求于2021年02月05日提交中国国家知识产权局、申请号为202110164841.6、发明名称为“多标签的类均衡方法及其装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202110164841.6 and the invention titled "Multi-label quasi-equilibrium method and device thereof" filed with the State Intellectual Property Office of China on February 5, 2021, the entire contents of which are incorporated by reference in this application.
技术领域technical field
本申请涉及人工智能与计算机视觉技术领域，尤其涉及一种多标签的类均衡方法及其装置。The present application relates to the technical field of artificial intelligence and computer vision, and in particular, to a multi-label class equalization method and device thereof.
背景技术Background technique
目标检测是计算机视觉和图像处理领域一个重要的研究方向，目前广泛地应用于视频直播、安防监控、机器人和人机交互等相关领域。然而现实生活中大部分目标检测的数据类型都存在长尾效应，即头部类别占多数样本，尾部类别样本数极少。例如，2D数据集BDD中火车与汽车的实例数相差600倍，3D数据集nuScenes中自行车与汽车的实例数相差4倍。目前，采样技术可解决数据的长尾效应。但是，采样技术通常会导致场景丢失或者间接增加其他类别的数量。Object detection is an important research direction in the field of computer vision and image processing. However, most of the data types of target detection in real life have long-tail effects, that is, the head category accounts for the majority of samples, and the tail category has very few samples. For example, the number of instances of trains and cars in the 2D dataset BDD differs by a factor of 600, and the number of instances of bicycles and cars in the 3D dataset nuScenes differs by a factor of 4. Currently, sampling techniques address the long tail of the data. However, sampling techniques often result in scene loss or indirectly increase the number of other classes.
发明内容SUMMARY OF THE INVENTION
鉴于以上内容，有必要提供一种多标签的类均衡方法及其装置，可避免场景丢失及间接增加其他类别的数量。In view of the above, it is necessary to provide a multi-label class equalization method and device thereof, which can avoid scene loss and indirectly increase the number of other classes.
本发明的一实施例提供一种多标签的类均衡方法，所述方法包括：获取包括多个实例的样本集，每个实例包括一个或多个类别标签；根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。An embodiment of the present invention provides a multi-label class equalization method, the method includes: acquiring a sample set including multiple instances, each instance including one or more class labels; The instances in the set are classified according to categories to form multiple categories, each instance is divided into one or more categories, and each category corresponds to a category; according to the number of target category labels in each instance in the subcategory and each The number of all class labels in the instance, to determine the target sampling weight of each instance in each subclass, where the subclass is a plurality of classes whose number is less than the first threshold, and the target class label is each The class label corresponding to the subclass in each instance, the target sampling weight is the sampling weight corresponding to the subclass in each instance; according to the target sampling weight of each instance in the subclass, for each subclass Instances of the class are weighted for sampling.
本申请通过对小类建立权重模型，计算每个类别每个实例的目标采样权重，并对每个小类的实例进行加权采样来新增实例，可仅新增实例，原始的实例会被全部保留，不丢失实例，同时可使得小类中的目标类别标签的数量相对于所有类别标签的数量越多的实例越容易被上采样，对其他类别影响较小，可避免场景丢失及间接增加其他类别的数量。This application establishes a weight model for the subclasses, calculates the target sampling weight of each instance of each class, and performs weighted sampling on the instances of each subclass to add new instances. Only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all category labels, and the impact on other categories is small, which can avoid scene loss and indirectly increase other categories. number of categories.
根据本申请的一些实施例，所述方法还包括：将根据加权采样结果确定的目标实例作为新增实例构造所述小类。According to some embodiments of the present application, the method further includes: constructing the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
本申请通过构造小类来增加小类中的实例数，从而可进一步避免长尾效应。In this application, the number of instances in the subclass is increased by constructing the subclass, so that the long tail effect can be further avoided.
根据本申请的一些实施例，所述方法还包括：构造所述小类中的实例至第一数量；判断每个小类中的实例的数量是否达到第二数量；若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，继续对每个小类的实例进行加权采样，及继续将根据加权采样结果确定的目标实例作为新增实例构造所述小类直至每个小类中的实例的数量达到第二数量。According to some embodiments of the present application, the method further includes: constructing instances in the subclass to a first number; judging whether the number of instances in each subclass reaches a second number; The number of instances does not reach the second number, continue to classify the instances in the sample set according to the class labels to form multiple classes, continue to determine the target sampling weight of each instance in each subclass, continue to Weighted sampling is performed on instances of each subclass, and the target instance determined according to the weighted sampling result is continued to construct the subclass as a new instance until the number of instances in each subclass reaches the second number.
本案可定向增加小类的数据量，且基于迭代采样策略，由于新增实例，每轮采样时预设目标实例数、第一数量及第二数量皆会发生调整，样本总量不会暴增，同时进一步避免间接增加其他类别的数量。In this case, the amount of data of small categories can be increased in a targeted manner, and based on the iterative sampling strategy, due to new instances, the preset target number of instances, the first number and the second number will be adjusted in each round of sampling, and the total number of samples will not increase sharply , while further avoiding indirect increases in the number of other categories.
根据本申请的一些实施例，所述方法还包括：若每个小类中的实例的数量达到第二数量，输出所有类的实例。According to some embodiments of the present application, the method further includes: if the number of instances in each subclass reaches a second number, outputting instances of all classes.
本案通过输出所有类的实例，使得所述训练设备可根据所述实例对初始AI模型进行训练来得到目标模型。In this case, the instances of all classes are output, so that the training device can train the initial AI model according to the instances to obtain the target model.
根据本申请的一些实施例，在所述构造所述小类中的实例至第一数量之前，所述方法还包括：确定所述第一数量。According to some embodiments of the present application, before the constructing the instances in the subclass to a first number, the method further comprises: determining the first number.
本案通过确定第一数量，可在每轮采样时，调整第一数量。In this case, by determining the first quantity, the first quantity can be adjusted in each round of sampling.
根据本申请的一些实施例，所述确定所述第一数量包括：确定所述第一数量为所有类的当前实例的数量的平均值。According to some embodiments of the present application, the determining the first number includes: determining the first number to be an average of the numbers of current instances of all classes.
本案通过所述第一数量为所有类的当前实例的数量的平均值，避免了所述N aim太大容易导致的样本集中的实例的总数量过大，及避免了所述N aim太小容易导致的迭代次数过多，收敛慢。 In this case, because the first number is the average value of the number of current instances of all classes, it avoids that the N aim is too large and the total number of instances in the sample set is too large, and that the N aim is too small. The resulting number of iterations is too large and the convergence is slow.
根据本申请的一些实施例，在所述判断每个小类中的实例的数量是否达到第二数量之前，所述方法还包括：确定所述第二数量。According to some embodiments of the present application, before the judging whether the number of instances in each subclass reaches a second number, the method further includes: determining the second number.
本案通过确定第二数量，可在每轮采样时，调整第二数量。In this case, by determining the second quantity, the second quantity can be adjusted in each round of sampling.
根据本申请的一些实施例，所述确定所述第二数量包括：确定所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量。 According to some embodiments of the present application, the determining the second number includes: determining the second number as r*N max , where r∈[0.5, 0.8], and N max is the number of instances of the largest class.
本案通过所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量，避免了所述第二数量太大容易导致新的样本集中的重复样本的数量较多，容易导致训练过程过拟合，及避免了所述第二数量太小容易各类别的均衡效果不明显。 In this case, the second number is r*N max , where r∈[0.5,0.8], and N max is the number of instances of the largest class, which avoids that the second number is too large to easily lead to duplicate samples in the new sample set If the second number is too small, it is easy to cause overfitting of the training process, and it is avoided that the second number is too small and the equalization effect of each category is not obvious.
根据本申请的一些实施例，所述加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种。According to some embodiments of the present application, the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
本案通过加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种，使得新增的实例主要是包括较多目标类别标签的实例，将会减小各类别的不均衡。In this case, the weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method, so that the newly added instances mainly include instances with more target category labels, which will reduce the imbalance of various categories.
根据本申请的一些实施例，所述根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样包括：针对所述小类中每个实例生成预设区间的随机数；根据所述随机数及所述目标采样权重通过加权随机采样法WRS确定所述小类中每个实例的采样分值；将采样分值最大的目标实例作为新增实例。According to some embodiments of the present application, the performing weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass includes: generating a preset interval for each instance in the subclass According to the random number and the target sampling weight, the sampling score of each instance in the subclass is determined by the weighted random sampling method WRS; the target instance with the largest sampling score is taken as the newly added instance.
本案通过加权随机采样法WRS，使得目标采样权重越大的实例的采样分值越会越大， 通过将采样分值最大的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来。In this case, the weighted random sampling method (WRS) is used to make the sampling score of an instance with a larger target sampling weight greater. been sampled.
根据本申请的一些实施例，所述根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样可包括：针对所述小类中所有实例按照目标采样权重从小到大的顺序排序；生成预设区间的随机数；根据所述随机数及所述目标采样权重的排序通过朴素加权采样法确定新增实例。According to some embodiments of the present application, the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include: for all instances in the subclass, according to the target sampling weight Sort from small to large; generate random numbers in a preset interval; determine new instances by naive weighted sampling method according to the random number and the sorting of the target sampling weight.
本案通过朴素加权采样法，使得目标采样权重越大的实例的逐一累加值越大，通过将逐一累加值达到随机数的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来。In this case, the naive weighted sampling method is used to make the instances with larger target sampling weights have larger accumulated values one by one. By taking the target instances whose accumulated values reach random numbers as new instances, the instances with larger target sampling weights will be easier to be collected. sample out.
根据本申请的一些实施例，在所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重之前，所述方法还包括：确定多个所述类中的小类。According to some embodiments of the present application, the target sampling weight of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Before, the method further comprises: determining a subclass of the plurality of classes.
本案通过确定小类，进而可对小类进行采样。In this case, by determining the subclass, the subclass can be sampled.
根据本申请的一些实施例，所述确定多个所述类中的小类包括：根据每个类的当前实例的数量及预设目标实例数确定每个类的采样率；确定所述采样率大于零的类为多个所述类中的小类。According to some embodiments of the present application, the determining of the subclasses in the plurality of the classes includes: determining the sampling rate of each class according to the number of current instances of each class and the number of preset target instances; determining the sampling rate A class greater than zero is a subclass of a plurality of said classes.
根据本申请的一些实施例，所述根据每个类的当前实例的数量及预设目标实例数确定每个类的采样率包括：根据公式
Figure PCTCN2021132897-appb-000001
确定采样率，其中，f t为类t的采样率，N aim为所述预设目标实例数，N i为类t的当前实例的数量。 According to some embodiments of the present application, the determining the sampling rate of each class according to the number of current instances of each class and the preset target instance number includes: according to the formula
Figure PCTCN2021132897-appb-000001
The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t.
本案通过公式
Figure PCTCN2021132897-appb-000002
可确定采样率，进而可确定小类。 The case passed the formula
Figure PCTCN2021132897-appb-000002
The sampling rate, and thus the subclass, can be determined.
根据本申请的一些实施例，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重包括：根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比；根据每个小类中每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重。According to some embodiments of the present application, determining the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance includes the following steps: : According to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the proportion of target class labels for each instance in each subclass; The proportion of target class labels for each instance determines the target sampling weight for each instance in each subclass.
本案通过权重模型，使得每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越多，则所述类的所述实例的目标采样权重越高；而每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标采样权重越低。In this case, the weight model is adopted, so that the greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the target sampling weight of the instance of the class; and The lower the number of target class labels for instances in each subclass relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class.
根据本申请的一些实施例，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比包括：根据公式
Figure PCTCN2021132897-appb-000003
确定每个小类中每个实例的目标类别标签的占比，其中，p ti为小类t中的实例i的目标类别标签的占比，k ti为小类t中的实例i的目标类别标签的数量，m ti为小类t中的实例i中的所有类别标签的数量。 According to some embodiments of the present application, according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the target class label of each instance in each subclass. The proportion includes: according to the formula
Figure PCTCN2021132897-appb-000003
Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
本案通过公式
Figure PCTCN2021132897-appb-000004
使得每个小类中的实例的目标类别标签的数量相对于所述实例中 的所有类别标签的数量越多，则所述类的所述实例的目标类别标签的占比越高；每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标类别标签的占比越低。 The case passed the formula
Figure PCTCN2021132897-appb-000004
The greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the proportion of target class labels of the instances of the class; each subclass The smaller the number of target class labels of instances in a class relative to the number of all class labels in the instance, the lower the proportion of target class labels of the instances of the class.
根据本申请的一些实施例，所述根据每个小类每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重包括：根据公式
Figure PCTCN2021132897-appb-000005
确定每个小类中每个实例的目标采样权重，其中，w ti为小类t中的实例i的目标采样权重，p ti为小类t中的实例i的目标类别标签的占比，tn为小类t中的所有实例的总数量，
Figure PCTCN2021132897-appb-000006
为小类t中的所有实例的目标类别标签的占比之和。 According to some embodiments of the present application, the determining the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance in each subclass includes: according to the formula
Figure PCTCN2021132897-appb-000005
Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t,
Figure PCTCN2021132897-appb-000006
is the sum of the proportions of the target class labels of all instances in the small class t.
本案通过公式
Figure PCTCN2021132897-appb-000007
使得每个小类中的实例的目标类别标签的占比越高，则所述类的所述实例的目标采样权重越高；如果每个小类中的实例的目标类别标签的占比越低，则所述类的所述实例的目标采样权重越低。 The case passed the formula
Figure PCTCN2021132897-appb-000007
The higher the proportion of the target class label of the instance in each subclass, the higher the target sampling weight of the instance of the class; if the proportion of the target class label of the instance in each subclass is lower , the lower the target sampling weight of the instance of the class.
本发明的一实施例还提供一种多标签的类均衡装置，所述装置包括：获取单元，所述获取单元用于获取包括多个实例的样本集，每个实例包括一个或多个类别标签；采样单元，所述采样单元用于根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；所述采样单元用于根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；所述采样单元还用于根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。An embodiment of the present invention further provides a multi-label class equalization device, the device includes: an acquisition unit configured to acquire a sample set including multiple instances, each instance including one or more class labels ; Sampling unit, the sampling unit is used to classify the instances in the sample set according to the categories to form a plurality of categories according to the category labels, each instance is divided into one or more categories, and each category corresponds to a category; the sampling unit is used to determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the The subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is the corresponding class label in each instance. The sampling weight of the subclass; the sampling unit is further configured to perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
本申请通过对小类建立权重模型，计算每个类别每个实例的目标采样权重，并对每个小类的实例进行加权采样来新增实例，可仅新增实例，原始的实例会被全部保留，不丢失实例，同时可使得小类中的目标类别标签的数量相对于所有类别标签的数量越多的实例越容易被上采样，对其他类别影响较小，可避免场景丢失及间接增加其他类别的数量。This application establishes a weight model for the subclasses, calculates the target sampling weight of each instance of each class, and performs weighted sampling on the instances of each subclass to add new instances. Only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all category labels, and the impact on other categories is small, which can avoid scene loss and indirectly increase other categories. number of categories.
根据本申请的一些实施例，所述采样单元还用于将根据加权采样结果确定的目标实例作为新增样本构造所述小类。According to some embodiments of the present application, the sampling unit is further configured to construct the subclass by using the target instance determined according to the weighted sampling result as a newly added sample.
本申请通过构造小类来增加小类中的实例数，从而可进一步避免长尾效应。In this application, the number of instances in the subclass is increased by constructing the subclass, so that the long tail effect can be further avoided.
根据本申请的一些实施例，所述采样单元还用于构造所述小类中的实例至第一数量；所述采样单元还用于判断每个小类中的实例的数量是否达到第二数量；所述采样单元还用于若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，及继续对每个小类的实例进行加权采样直至每个小类中的实例的数量达到第二数量。According to some embodiments of the present application, the sampling unit is further configured to construct a first number of instances in the subclass; the sampling unit is further configured to determine whether the number of instances in each subclass reaches a second number ; The sampling unit is also used for, if the number of instances in each subclass does not reach the second number, continue to classify the instances in the sample set according to the class according to the class label to form multiple classes, and continue to determine A target sampling weight for each instance in each subclass, and continuing weighted sampling of instances in each subclass until the number of instances in each subclass reaches the second number.
本案可定向增加小类的数据量，且基于迭代采样策略，由于新增实例，每轮采样时预设目标实例数、第一数量及第二数量皆会发生调整，样本总量不会暴增，同时进一步避免间接增加其他类别的数量。In this case, the amount of data of small categories can be increased in a targeted manner, and based on the iterative sampling strategy, due to new instances, the preset target number of instances, the first number and the second number will be adjusted in each round of sampling, and the total number of samples will not increase sharply , while further avoiding indirect increases in the number of other categories.
根据本申请的一些实施例，所述采样单元还用于若每个小类中的实例的数量达到第二数量，输出所有类的实例。According to some embodiments of the present application, the sampling unit is further configured to output instances of all classes if the number of instances in each subclass reaches the second number.
本案通过输出所有类的实例，使得所述训练设备可根据所述实例对初始AI模型进行训练来得到目标模型。In this case, the instances of all classes are output, so that the training device can train the initial AI model according to the instances to obtain the target model.
根据本申请的一些实施例，在所述构造所述小类中的实例至第一数量之前：所述采样单元还用于确定所述第一数量。According to some embodiments of the present application, before the constructing the instances in the subclass to a first number: the sampling unit is further configured to determine the first number.
本案通过确定第一数量，可在每次采样时，调整第一数量。In this case, by determining the first quantity, the first quantity can be adjusted during each sampling.
根据本申请的一些实施例，所述采样单元还用于确定所述第一数量为所有类的当前实例的数量的平均值。According to some embodiments of the present application, the sampling unit is further configured to determine that the first number is an average value of the number of current instances of all classes.
本案通过所述第一数量为所有类的当前实例的数量的平均值，避免了所述N aim太大容易导致的样本集中的实例的总数量过大，及避免了所述N aim太小容易导致的迭代次数过多，收敛慢。 In this case, because the first number is the average value of the number of current instances of all classes, it avoids that the N aim is too large and the total number of instances in the sample set is too large, and that the N aim is too small. The resulting number of iterations is too large and the convergence is slow.
根据本申请的一些实施例，在所述判断每个小类中的实例的数量是否达到第二数量之前：所述采样单元还用于确定所述第二数量。According to some embodiments of the present application, before the judging whether the number of instances in each subclass reaches a second number: the sampling unit is further configured to determine the second number.
本案通过确定第二数量，可在每次采样时，调整第二数量。In this case, by determining the second quantity, the second quantity can be adjusted in each sampling.
根据本申请的一些实施例，所述采样单元还用于确定所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量。 According to some embodiments of the present application, the sampling unit is further configured to determine the second number as r*N max , where r∈[0.5, 0.8], and N max is the number of instances of the largest class.
本案通过所述第二数量为为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量，避免了所述第二数量太大容易导致新的样本集中的重复样本的数量较多，容易导致过拟合，及避免了所述第二数量太小容易各类别的均衡效果不明显。 In this case, the second number is r*N max , where r∈[0.5, 0.8], and N max is the number of instances of the largest class, which avoids that the second number is too large to easily lead to repetition in new sample sets If the number of samples is large, it is easy to cause overfitting, and it is avoided that the second number is too small and the equalization effect of each category is not obvious.
根据本申请的一些实施例，所述加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种。According to some embodiments of the present application, the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
本案通过加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种，使得新增的实例主要是包括较多目标类别标签的实例，将会减小各类别的不均衡。In this case, the weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method, so that the newly added instances mainly include instances with more target category labels, which will reduce the imbalance of various categories.
根据本申请的一些实施例，所述采样单元还用于针对所述小类中每个实例生成预设区间的随机数；根据所述随机数及所述目标采样权重通过加权随机采样法WRS确定所述小类中每个实例的采样分值；将采样分值最大的目标实例作为新增实例。According to some embodiments of the present application, the sampling unit is further configured to generate a random number of a preset interval for each instance in the subclass; according to the random number and the target sampling weight, the weighted random sampling method WRS is used to determine The sampling score of each instance in the subclass; the target instance with the largest sampling score is taken as the newly added instance.
本案通过加权随机采样法WRS，使得目标采样权重越大的实例的采样分值越会越大，通过将采样分值最大的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来。In this case, the weighted random sampling method (WRS) is used to make the sampling score of an instance with a larger target sampling weight higher. been sampled.
根据本申请的一些实施例，所述采样单元还用于针对所述小类中所有实例按照目标采样权重从小到大的顺序排序；生成预设区间的随机数；根据所述随机数及所述目标采样权重的排序通过朴素加权采样法确定新增实例。According to some embodiments of the present application, the sampling unit is further configured to sort all the instances in the subclass according to the target sampling weight in ascending order; generate a random number in a preset interval; according to the random number and the The ordering of the target sampling weights determines the newly added instances through the naive weighted sampling method.
本案通过朴素加权采样法，使得目标采样权重越大的实例的逐一累加值越大，通过将逐一累加值达到随机数的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来。In this case, the naive weighted sampling method is used to make the instances with larger target sampling weights have larger accumulated values one by one. By taking the target instances whose accumulated values reach random numbers as new instances, the instances with larger target sampling weights will be easier to be collected. sample out.
根据本申请的一些实施例，在所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重之前：所述采样单元还用于确定多个所述类中的小类。According to some embodiments of the present application, the target sampling weight of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Before: the sampling unit is also used to determine subclasses of the plurality of classes.
本案通过确定小类，进而可对小类进行采样。In this case, by determining the subclass, the subclass can be sampled.
根据本申请的一些实施例，所述采样单元还用于根据每个类的当前实例的数量及预设目 标实例数确定每个类的采样率；确定所述采样率大于零的类为多个所述类中的小类。According to some embodiments of the present application, the sampling unit is further configured to determine the sampling rate of each class according to the number of current instances of each class and the preset target number of instances; determine that there are multiple classes with the sampling rate greater than zero A subclass within the described class.
根据本申请的一些实施例，所述采样单元还用于根据公式
Figure PCTCN2021132897-appb-000008
确定采样率，其中，f t为类t的采样率，N aim为所述预设目标实例数，N i为类t的当前实例的数量。 According to some embodiments of the present application, the sampling unit is further configured to
Figure PCTCN2021132897-appb-000008
The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t.
本案通过公式
Figure PCTCN2021132897-appb-000009
可确定采样率，进而可确定小类。 The case passed the formula
Figure PCTCN2021132897-appb-000009
The sampling rate, and thus the subclass, can be determined.
根据本申请的一些实施例，所述采样单元用于根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比；所述采样单元还用于根据每个小类每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重。According to some embodiments of the present application, the sampling unit is configured to determine, according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, the Proportion of target class labels; the sampling unit is further configured to determine the target sampling weight of each instance in each subclass according to the ratio of target class labels of each instance of each subclass.
本案通过权重模型，使得每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越多，则所述类的所述实例的目标采样权重越高；而每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标采样权重越低。In this case, the weight model is adopted, so that the greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the target sampling weight of the instance of the class; and The lower the number of target class labels for instances in each subclass relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class.
根据本申请的一些实施例，所述采样单元还用于根据公式
Figure PCTCN2021132897-appb-000010
确定每个小类中每个实例的目标类别标签的占比，其中，p ti为小类t中的实例i的目标类别标签的占比，k ti为小类t中的实例i的目标类别标签的数量，m ti为小类t中的实例i中的所有类别标签的数量。 According to some embodiments of the present application, the sampling unit is further configured to
Figure PCTCN2021132897-appb-000010
Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
本案通过公式
Figure PCTCN2021132897-appb-000011
使得每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越多，则所述类的所述实例的目标类别标签的占比越高；每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标类别标签的占比越低。 The case passed the formula
Figure PCTCN2021132897-appb-000011
The greater the number of target class labels of instances in each subclass relative to the number of all class labels in the instance, the higher the proportion of target class labels of the instances of the class; each subclass The smaller the number of target class labels of instances in a class relative to the number of all class labels in the instance, the lower the proportion of target class labels of the instances of the class.
根据本申请的一些实施例，所述采样单元还用于根据公式
Figure PCTCN2021132897-appb-000012
确定每个小类中每个实例的目标采样权重，其中，w ti为小类t中的实例i的目标采样权重，p ti为小类t中的实例i的目标类别标签的占比，tn为小类t中的所有实例的总数量，
Figure PCTCN2021132897-appb-000013
为小类t中的所有实例的目标类别标签的占比之和。 According to some embodiments of the present application, the sampling unit is further configured to
Figure PCTCN2021132897-appb-000012
Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t,
Figure PCTCN2021132897-appb-000013
is the sum of the proportions of the target class labels of all instances in the small class t.
本案通过公式
Figure PCTCN2021132897-appb-000014
使得每个小类中的实例的目标类别标签的占比越高，则所述类的所述实例的目标采样权重越高；如果每个小类中的实例的目标类别标签的占比越低，则所述类的所述实例的目标采样权重越低。 The case passed the formula
Figure PCTCN2021132897-appb-000014
The higher the proportion of the target class label of the instance in each subclass, the higher the target sampling weight of the instance of the class; if the proportion of the target class label of the instance in each subclass is lower , the lower the target sampling weight of the instance of the class.
本发明的一实施例还提供一种电子设备，所述电子设备包括处理器和存储器，所述存储器用于存储程序指令，所述处理器调用所述程序指令时，实现如上任一项所述的多标签的类均衡方法。An embodiment of the present invention further provides an electronic device, the electronic device includes a processor and a memory, the memory is used for storing program instructions, when the processor calls the program instructions, the implementation of any one of the above The multi-label class equalization method.
本发明的一实施例还提供一种车辆，所述车辆包括如上所述的电子设备。An embodiment of the present invention also provides a vehicle including the electronic device as described above.
本发明的一实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有程序，所述程序使得计算机设备实现如上任一项所述的多标签的类均衡方法。An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a program, and the program enables a computer device to implement the multi-tag equalization-like method described in any one of the above.
本发明的一实施例还提供一种计算机程序产品，所述计算机程序产品包括计算机执行指 令，所述计算机执行指令存储在计算机可读存储介质中；设备的至少一个处理器可以从所述计算机可读存储介质中读取所述计算机执行指令，所述至少一个处理器执行所述计算机执行指令使得所述设备执行如上任一项所述的多标签的类均衡方法。An embodiment of the present invention also provides a computer program product, the computer program product comprising computer-executable instructions stored in a computer-readable storage medium; at least one processor of the device can be obtained from the computer-executable instructions. The computer-executable instructions are read from the reading storage medium, and the at least one processor executes the computer-executable instructions to cause the device to perform the multi-tag equalization-like method described in any one of the above.
附图说明Description of drawings
图1为本发明实施例的***架构示意图。FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present invention.
图2为本发明第一实施例的多标签的类均衡方法的流程示意图。FIG. 2 is a schematic flowchart of a multi-label class equalization method according to the first embodiment of the present invention.
图3为本发明实施例的实例的示意图。FIG. 3 is a schematic diagram of an example of an embodiment of the present invention.
图4为本发明实施例的样本集中的实例分类后对应类别下的实例的数量示意图。FIG. 4 is a schematic diagram of the number of instances in corresponding categories after the instances in the sample set are classified according to an embodiment of the present invention.
图5为本发明实施例的类别t的类别标签示意图。FIG. 5 is a schematic diagram of a category label of category t according to an embodiment of the present invention.
图6为本发明第二实施例的多标签的类均衡方法的流程示意图。FIG. 6 is a schematic flowchart of a multi-label class equalization method according to a second embodiment of the present invention.
图7为本发明实施例的对所述小类构造实例使得每个小类的实例的数量达到第一数量的示意图。FIG. 7 is a schematic diagram of constructing instances of the subclass so that the number of instances of each subclass reaches a first number according to an embodiment of the present invention.
图8为本发明实施例的对所述小类迭代构造小类的示意图。FIG. 8 is a schematic diagram of iteratively constructing a subclass for the subclass according to an embodiment of the present invention.
图9为本发明实施例提供的一种多标签的类均衡装置的结构示意图。FIG. 9 is a schematic structural diagram of a multi-label equalizer-like apparatus according to an embodiment of the present invention.
图10为本发明实施例提供的一种电子设备的硬件结构示意图。FIG. 10 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.
图11为本发明实施例提供的一种电子装置的结构示意图。FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
主要元件符号说明Description of main component symbols
***架构                    10 System Architecture 10
数据采集设备                160 Data acquisition equipment 160
数据库                      130Database 130
训练设备                    120Training equipment 120
目标模型                    101target model 101
执行设备                    110Execution equipment 110
I/O接口                     112I/O interface 112
客户设备                    140 Customer equipment 140
数据存储***                150Data storage system 150
计算模块                    111Calculation module 111
多标签的类均衡装置          900 Multi-label class equalizer 900
获取单元                    901Get unit 901
采样单元                    902 Sampling unit 902
训练单元                    903 Training Unit 903
电子设备                    1000 Electronic equipment 1000
电子装置                    1100 Electronic device 1100
存储器                      1001,1101 Memory 1001,1101
处理器                      1002,1102 Processor 1002,1102
通信接口                    1003,1103 Communication interface 1003,1103
总线                        1004,1104 Bus 1004,1104
具体实施方式Detailed ways
在本申请实施例的描述中，“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“例如”等词旨在以具体方式呈现相关概念。In the description of the embodiments of the present application, words such as "for example" are used to represent examples, illustrations or illustrations. Any embodiment or design described in the embodiments of the present application as "for example," should not be construed as preferred or advantageous over other embodiments or designs. Rather, use of words such as "such as" is intended to present the related concepts in a specific manner.
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请中的技术领域的技术人员通常理解的含义相同。本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。应理解，本申请中除非另有说明，“多个”是指两个或多于两个。Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field in this application. Terms used in the specification of the present application are for the purpose of describing specific embodiments only, and are not intended to limit the present invention. It should be understood that in this application, unless stated otherwise, "plurality" refers to two or more than two.
现有的基于混合采样的数据均衡方法通过实例归类、计算平均值、随机上采样、随机下采样、及组合来形成新的数据集。所述实例归类包括统计每个类别对应的实例数量。所述计算平均值包括将各类别的实例数累加，再求平均值。所述随机上采样包括对实例数低于平均值的类别通过有放回随机采样进行上采样，使其实例数等于平均值。所述随机下采样包括对实例数超出平均值的类别进行下采样，使其实例数等于平均值。所述组合包括将各类别实例重新组合形成新的数据集。但是，在随机下采样时，会直接放弃部分场景，从而会造成场景丢失。在随机上采样时，由于为通过有放回随机采样进行上采样，不能确保每个实例一定被选中，可能会造成场景丢失。同时，由于一个实例可能包括多个标签，基于某一标签进行上采样时，可能会同时大量增加其他标签的数量，这将会导致各类别不均衡。Existing mixed-sampling-based data equalization methods form new datasets by instance classification, computing averages, random upsampling, random downsampling, and combining. The instance classification includes counting the number of instances corresponding to each category. The calculating the average value includes accumulating the number of instances of each category, and then calculating the average value. The random up-sampling includes performing up-sampling by random sampling with replacement on the category whose number of instances is lower than the average value, so that the number of instances is equal to the average value. The random down-sampling includes down-sampling the classes whose number of instances exceeds the average, so that the number of instances is equal to the average. The combining includes recombining the class instances to form a new dataset. However, during random downsampling, part of the scene is directly discarded, resulting in scene loss. During random upsampling, since upsampling is performed by random sampling with replacement, each instance cannot be guaranteed to be selected, which may cause scene loss. At the same time, since an instance may include multiple labels, when upsampling based on a certain label, the number of other labels may be greatly increased at the same time, which will lead to imbalance of each category.
为了更好地理解本申请实施例的方法，下面先对本案所涉及的一些术语和概念进行介绍。In order to better understand the methods of the embodiments of the present application, some terms and concepts involved in the present application are first introduced below.
实例：泛指2D数据集中的图片样本(Image)和3D数据集中的点云样本(Point Cloud)。Example: It generally refers to the image samples (Image) in the 2D data set and the point cloud samples (Point Cloud) in the 3D data set.
样本集：若干实例构成的集合，所述集合中的每个元素就是一个实例。Sample set: a set of several instances, each element in the set is an instance.
类别：是指不同事物的区分，是按种类的不同而做出的区别。Category: refers to the distinction between different things, which is a distinction made according to different types.
类：在所有实例所构成的样本集上定义的一个子集合，同一类的样本在我们所关心的某种性质上是不可区分的，即具有相同的模式。Class: A subset defined on the set of samples composed of all instances, samples of the same class are indistinguishable in a property we care about, that is, have the same pattern.
小类：是指多个类中数量小于第一阈值的类。Small class: refers to the class whose number is less than the first threshold among multiple classes.
在对本申请实施例进行详细的解释说明之前，先对本申请实施例涉及的应用场景予以介绍。Before the detailed explanation of the embodiments of the present application, the application scenarios involved in the embodiments of the present application are introduced.
当前，训练数据集中的实例可用于模型。所述模型可以是一种AI模型。AI模型是一种机器学习模型，其本质是一种包括大量参数和数学公式(或数学规则)的数学模型，目的是学习一些数学表达，使所述数学表达能提供输入值x和输出值y之间的相关性，能够提供x和y之间的相关性的数学表达即为训练后的AI模型。在利用AI模型进行检测之前需要对初始AI模型进行训练。Currently, instances in the training dataset are available for the model. The model may be an AI model. AI model is a machine learning model, which is essentially a mathematical model including a large number of parameters and mathematical formulas (or mathematical rules), the purpose is to learn some mathematical expressions, so that the mathematical expressions can provide input value x and output value y The correlation between, and the mathematical expression that can provide the correlation between x and y is the trained AI model. The initial AI model needs to be trained before using the AI model for detection.
对AI模型训练有影响的因素主要包括三个方面：训练数据集、初始AI模型及机器算力。随着当前AI模型应用场景的增加，AI模型需要面对的场景复杂化，导致训练用的AI模型越来越复杂。同时，为了提高AI模型的训练效果，对训练数据集的数据量要求也越来越大，数据的类也越来越多。由此导致训练过程的计算量增加，对机器算力要求也在不断增加，训练需要的时间也越来越长。如何对AI模型训练过程进行优化，以在最短时间获得一 个准确率更好的AI模型是行业关注的重点。The factors affecting AI model training mainly include three aspects: training data set, initial AI model and machine computing power. With the increase of application scenarios of current AI models, the scenarios that AI models need to face become more complex, resulting in more and more complex AI models for training. At the same time, in order to improve the training effect of the AI model, the data volume requirements for the training data set are also increasing, and the data types are also increasing. As a result, the amount of calculation in the training process increases, the requirements for machine computing power are also increasing, and the time required for training is getting longer and longer. How to optimize the AI model training process to obtain an AI model with better accuracy in the shortest time is the focus of the industry.
基于此，为了提高AI模型训练的性能，开发人员在编写好最初的待训练的神经网络模型之后，可以通过本申请实施例提供的多标签的类均衡方法建立实例，并可利用包括本申请构造的实例的训练数据集对待训练的AI模型进行训练，提高模型推理的准确性。其中，待训练的神经网络模型是需要进行训练的初始AI模型。Based on this, in order to improve the performance of AI model training, after writing the initial neural network model to be trained, the developer can create an instance through the multi-label class equalization method provided by the embodiment of this application, and can use the structure including the structure of this application. The training data set of the instance to be trained is used to train the AI model to be trained, so as to improve the accuracy of model inference. Among them, the neural network model to be trained is the initial AI model that needs to be trained.
训练后的AI模型具备预测分析的能力，可用于各种不同的作用。例如，可从目标数据(例如视频流、图片流或图像)中找到感兴趣的数据(例如图像/视频中的人或者物体)，来确定物体在图像或视频中的位置和大小。其中，感兴趣的物体称为目标，目标的类型可以根据应用场景以及业务方向确定。例如，应用在监控安防领域，目标可以而不限于是车辆、人体、人脸等。应用在交通领域，目标可以而不限于是道路、车牌号、道路设施、交通标志、行人等。应用在物流领域，目标可以而不限于是集装箱号、快递单号、障碍物、包裹号等。应用在军事以及国防领域，目标可以而不限于是地形、飞行设备等。应用在医疗领域，目标可以而不限于是器官、组织等。The trained AI model is capable of predictive analytics and can be used for a variety of different purposes. For example, data of interest (eg people or objects in images/videos) can be found from target data (eg video streams, picture streams or images) to determine the location and size of objects in the image or video. Among them, the object of interest is called the target, and the type of the target can be determined according to the application scenario and business direction. For example, in the field of monitoring and security, the target can be but not limited to vehicles, human bodies, faces, etc. Applied in the field of transportation, the target can be but not limited to roads, license plate numbers, road facilities, traffic signs, pedestrians, etc. Applied in the field of logistics, the target can be but not limited to container number, courier number, obstacle, package number, etc. Applied in the military and national defense fields, the target can be but not limited to terrain, flying equipment, etc. Applied in the medical field, the target can be but not limited to organs, tissues, etc.
显然，训练后的AI模型还可适用于上述的目标检测场景外的其他场景，例如识别、分割等场景。下面以训练后的AI模型适用于目标检测场景为例进行说明，但是本案并不将此作为限制于仅适用于目标检测场景。Obviously, the trained AI model can also be applied to other scenarios other than the above target detection scenarios, such as recognition, segmentation and other scenarios. The following is an example of the application of the trained AI model to target detection scenarios, but this case does not limit it to only target detection scenarios.
参考图1，图1为本发明实施例的***架构10示意图。数据采集设备160用于采集训练数据。针对本申请实施例的目标检测方法(AI模型的应用示例中的一例)来说，训练数据可以包括训练实例，其中，训练实例的结果可以是人工预先标注的结果或者通过现有的标注***自动标注的结果，即每个训练实例包括一个或多个类别标签。在采集到训练数据之后，数据采集设备160将这些训练数据存入数据库130。训练设备120基于数据库130中维护的训练数据训练得到目标模型101。下面将更详细地描述训练设备120如何基于训练数据训练得到目标模型101，目标模型101能够在目标数据(例如视频流、图片流或图像)中找到感兴趣的数据(例如图像/视频中的人或者物体)，来确定物体在图像或视频中的位置和大小。Referring to FIG. 1 , FIG. 1 is a schematic diagram of a system architecture 10 according to an embodiment of the present invention. The data collection device 160 is used to collect training data. For the target detection method in the embodiment of the present application (an example of an application example of an AI model), the training data may include training instances, where the results of the training instances may be manually pre-labeled results or automatically marked by an existing labeling system. Annotated results, i.e. each training instance includes one or more class labels. After collecting the training data, the data collecting device 160 stores the training data in the database 130 . The training device 120 obtains the target model 101 by training based on the training data maintained in the database 130 . The following will describe in more detail how the training device 120 trains the target model 101 based on the training data, and the target model 101 is able to find the data of interest (eg images/people in the video) in the target data (eg video streams, picture streams or images). or object), to determine the position and size of the object in the image or video.
在本实施例中，所述训练设备120可获取包括多个实例的样本集，每个实例包括一个或多个类别标签；根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。所述训练设备120还可根据所述实例对初始AI模型进行训练来得到目标模型101。In this embodiment, the training device 120 may acquire a sample set including multiple instances, each instance including one or more category labels; and classify the instances in the sample set according to categories according to the category labels Form multiple classes, each instance is divided into one or more classes, and each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, Determine the target sampling weight of each instance in each subclass, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, and the target class label is the corresponding subclass in each instance. Category label, the target sampling weight is the sampling weight corresponding to the subclass in each instance; weighted sampling is performed on the instance of each subclass according to the target sampling weight of each instance in the subclass. The training device 120 may also train the initial AI model according to the example to obtain the target model 101 .
需要说明的是，在实际的应用中，所述数据库130中维护的训练数据不一定都来自于数据采集设备160的采集，也有可能是从其他设备接收得到的。另外需要说明的是，训练设备120也不一定完全基于数据库130维护的训练数据进行目标模型101的训练，也有可能从云端或其他地方获取训练数据进行模型训练，例如从2D数据集BDD或者3D数据集nuScenes等获取训练数据进行模型训练，上述描述不应该作为对本申请实施例的限定。It should be noted that, in practical applications, the training data maintained in the database 130 may not necessarily come from the collection of the data collection device 160, and may also be received from other devices. In addition, it should be noted that the training device 120 may not necessarily train the target model 101 entirely based on the training data maintained by the database 130, and may also obtain training data from the cloud or other places for model training, for example, from the 2D data set BDD or 3D data. Collect training data such as nuScenes to perform model training, and the above description should not be taken as a limitation on the embodiments of the present application.
根据训练设备120训练得到的目标模型101可以应用于不同的***或设备中，如应用于图1所示的执行设备110，所述执行设备110可以是终端，如手机终端，平板电脑，笔记本 电脑，AR/VR，车载终端等，还可以是服务器或者云端等。在图1中，执行设备110配置有I/O接口112，用于与外部设备进行数据交互，用户可以通过客户设备140向I/O接口112输入数据，所述输入数据在本申请实施例中可以包括：客户设备输入的待处理图像。The target model 101 trained according to the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in FIG. 1 , the execution device 110 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer , AR/VR, vehicle terminal, etc., it can also be a server or cloud, etc. In FIG. 1, the execution device 110 is configured with an I/O interface 112, which is used for data interaction with external devices, and a user can input data to the I/O interface 112 through the client device 140, and the input data is in the embodiment of the present application It can include: the image to be processed input by the client device.
数据存储***150用于接收和存储训练设备120发送的目标模型的参数，以及用于存储通过目标模型101得到的目标检测结果的数据，当然还可能包括所述数据存储***150正常运行所需的程序代码(或指令)。数据存储***150可以为部署在执行设备110以外的一个设备或者多个设备构成的分布式存储集群，此时，当执行设备110需要使用数据存储***150上的数据时，可以由数据存储***150向执行设备110发送所述执行设备110所需的数据，相应地，所述执行设备110接收并存储(或者缓存)所述数据。当然数据存储***150也可以部署在执行设备110内，当部署在执行设备110内时，所述分布式存储***可以包括一个或者多个存储器，可选的，存在多个存储器时，不同的存储器用于存储不同类型的数据，如通过训练设备生成的目标模型的模型参数和通过目标模型101得到的目标检测结果的数据可以分别存储在两个不同的存储器上。The data storage system 150 is used for receiving and storing the parameters of the target model sent by the training device 120, and for storing the data of the target detection results obtained by the target model 101, and of course, it may also include the data required for the normal operation of the data storage system 150. Program code (or instructions). The data storage system 150 may be a distributed storage cluster composed of one device or multiple devices deployed outside the execution device 110. At this time, when the execution device 110 needs to use the data on the data storage system 150, the data storage system 150 The data required by the execution device 110 is sent to the execution device 110, and accordingly, the execution device 110 receives and stores (or caches) the data. Of course, the data storage system 150 can also be deployed in the execution device 110. When deployed in the execution device 110, the distributed storage system can include one or more memories. Optionally, when there are multiple memories, different memories The data for storing different types of data, such as the model parameters of the target model generated by the training device and the data of the target detection result obtained by the target model 101, may be stored in two different memories respectively.
计算模块111使用目标模型101对输入的待处理图像(例如视频流、图片流或图像)进行处理，例如，从输入的待处理图像中找到感兴趣的数据(例如图像/视频中的人或者物体)，来确定物体在图像或视频中的位置和大小。The calculation module 111 uses the target model 101 to process the input image to be processed (such as a video stream, picture stream or image), for example, to find data of interest (such as a person or object in the image/video) from the input image to be processed ) to determine the position and size of objects in an image or video.
最后，I/O接口112将处理结果，如上述得到的物体在图像或视频中的位置和大小返回给客户设备140，从而提供给用户。Finally, the I/O interface 112 returns the processing result, such as the position and size of the object in the image or video obtained as described above, to the client device 140, so as to be provided to the user.
值得说明的是，训练设备120可以针对不同的目标或称不同的任务，基于不同的训练数据生成相应的目标模型101，所述相应的目标模型101即可以用于实现上述目标或完成上述任务，从而为用户提供所需的结果。It is worth noting that the training device 120 can generate corresponding target models 101 based on different training data for different goals or tasks, and the corresponding target models 101 can be used to achieve the above-mentioned goals or complete the above-mentioned tasks. This provides the user with the desired result.
在图1中所示情况下，用户可以手动给定输入数据，所述手动给定可以通过I/O接口112提供的界面进行操作。另一种情况下，客户设备140可以自动地向I/O接口112发送输入数据，如果要求客户设备140自动发送输入数据需要获得用户的授权，则用户可以在客户设备140中设置相应权限。用户可以在客户设备140查看执行设备110输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备140也可以作为数据采集端，采集如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果作为新的样本数据，并存入数据库130。当然，也可以不经过客户设备140进行采集，而是由I/O接口112直接将如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果，作为新的样本数据存入数据库130。In the case shown in FIG. 1 , the user can manually specify the input data, which can be operated through the interface provided by the I/O interface 112 . In another case, the client device 140 can automatically send the input data to the I/O interface 112 . If the user's authorization is required to request the client device 140 to automatically send the input data, the user can set the corresponding permission in the client device 140 . The user can view the result output by the execution device 110 on the client device 140, and the specific presentation form can be a specific manner such as display, sound, and action. The client device 140 can also be used as a data collection terminal to collect the input data of the input I/O interface 112 and the output result of the output I/O interface 112 as new sample data as shown in the figure, and store them in the database 130 . Of course, it is also possible not to collect through the client device 140, but the I/O interface 112 directly uses the input data input into the I/O interface 112 and the output result of the output I/O interface 112 as shown in the figure as a new sample The data is stored in database 130 .
值得注意的是，附图1仅是本发明实施例提供的一种***架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在附图1中，数据存储***150相对执行设备110是外部存储器，在其它情况下，也可以将数据存储***150置于执行设备110中。It is worth noting that FIG. 1 is only a schematic diagram of a system architecture provided by an embodiment of the present invention, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG. 1 , the data storage system 150 is an external memory relative to the execution device 110 , and in other cases, the data storage system 150 may also be placed in the execution device 110 .
在本申请实施例中，所述训练设备120、执行设备110、客户设备140可以分别为三个不同的物理设备，也可能所述训练设备120和执行设备110在同一个物理设备或者一个集群上，也可能所述执行设备110与所述客户设备140在同一个物理设备或者一个集群上。In this embodiment of the present application, the training device 120, the execution device 110, and the client device 140 may be three different physical devices, or the training device 120 and the execution device 110 may be on the same physical device or a cluster. , it is also possible that the execution device 110 and the client device 140 are on the same physical device or a cluster.
请参考图2，为本发明第一实施例的多标签的类均衡方法的流程示意图。所述方法具体可以由如图1所示的训练设备120执行，所述方法中的多个实例的样本集可以是如图1所示 的数据库130中维护的训练数据，可选的，所述方法的步骤S202至步骤S206可以在所述训练设备120中执行，也可以在训练设备120之前由其他功能模块预先执行，即先对从所述数据库130中接收或者获取到的训练数据进行预处理，如步骤S202至步骤S206所述的采样过程，得到新的样本集，作为所述训练设备120的输入，并由所述训练设备执行模型的训练。Please refer to FIG. 2 , which is a schematic flowchart of a multi-label class equalization method according to the first embodiment of the present invention. The method may be specifically performed by the training device 120 shown in FIG. 1 , and the sample sets of the multiple instances in the method may be the training data maintained in the database 130 shown in FIG. 1 . Optionally, the Steps S202 to S206 of the method may be executed in the training device 120, or may be pre-executed by other functional modules before the training device 120, that is, preprocessing the training data received or acquired from the database 130 first , in the sampling process described in steps S202 to S206, a new sample set is obtained, which is used as the input of the training device 120, and the training device performs model training.
可选地，所述方法可以由CPU处理，也可以由CPU和GPU共同处理，也可以不用GPU，而使用其他适合用于处理处理的处理器，本申请不做限制。Optionally, the method may be processed by the CPU, or may be jointly processed by the CPU and the GPU, or other processors suitable for processing may be used without using the GPU, which is not limited in this application.
所述方法包括但不限于如下步骤：The method includes but is not limited to the following steps:
S201：获取包括多个实例的样本集，每个实例包括一个或多个类别标签。S201: Acquire a sample set including multiple instances, each instance including one or more category labels.
所述样本集可为通过图1所示的数据采集设备160采集，其中所述数据采集设备160采集的样本集中的每个实例为人工预先标注的结果或者为通过现有的标注***自动标注的结果。所述样本集也可为从2D数据集BDD或者3D数据集nuScenes等获取，或者为从存储器等其他地方来的数据集，并通过现有的标注***自动标注的结果。所述样本集可包括所述2D数据集BDD或者3D数据集nuScenes中的全部数据集，也可仅包括部分数据集，本案以仅包括部分数据集为例进行说明。The sample set may be collected by the data collection device 160 shown in FIG. 1 , wherein each instance in the sample set collected by the data collection device 160 is a result of manual pre-labeling or is automatically labelled by an existing labeling system. result. The sample set may also be obtained from a 2D data set BDD or a 3D data set nuScenes, etc., or a data set from a memory or other places, and the result of automatic labeling by an existing labeling system. The sample set may include all the data sets in the 2D data set BDD or the 3D data set nuScenes, or may only include part of the data set, and this case is described by taking only part of the data set as an example.
本案以所述样本集为自动驾驶数据集进行简要说明。在本实施例中，所述样本集中的类别有10个，分别为公共汽车、交通灯、指示牌、人、自行车、汽车、卡车、摩托车、火车、及骑手。但是，可以理解的是，对于自动驾驶数据集，所述样本集的类别不仅局限于上述的类别，还可为其他的类别。所述样本集不仅局限于自动驾驶数据集，所述样本集还可为来自Stanford Dogs Dataset的狗的数据集，或者来自Labelled Faces in the Wild的人脸图像的数据集等。显然，对于不同的样本集，所述样本集的类别也不同。This case briefly describes the sample set as an autonomous driving data set. In this embodiment, there are 10 categories in the sample set, which are buses, traffic lights, signs, people, bicycles, cars, trucks, motorcycles, trains, and riders. However, it can be understood that, for the automatic driving data set, the categories of the sample set are not limited to the above categories, but can also be other categories. The sample set is not limited to the autonomous driving data set, the sample set can also be a data set of dogs from Stanford Dogs Dataset, or a data set of face images from Labelled Faces in the Wild, etc. Obviously, for different sample sets, the categories of the sample sets are also different.
所述实例可为2D数据集中的图片样本。所述实例还可为3D数据集中的点云样本。所述实例可为如图3所示。在图3中，所述实例包括6个类别标签，分别为traffic sign(交通灯)、traffic sign(交通灯)、rider(骑手)、bike(自行车)、truck(卡车)、及car(汽车)。在本实施例中，所述实例中的每个对象通过二维边框标示。每个类别标签用于表示一个对象。所述类别标签标示在所述实例上，这些类别标签的标示可帮助确定每个对象的类型。例如，类别标签-car表示框选的所述对象为汽车，类别标签-traffic sign表示框选的所述对象为交通灯，类别标签-rider表示框选的所述对象为骑手，类别标签-bike表示框选的所述对象为自行车，及类别标签-truck表示框选的所述对象为卡车。相同的类别标签属于一个类。上述6个类别标签属于5个类别，分别为汽车、交通灯、骑手、自行车、及卡车。The example may be a picture sample in a 2D dataset. The instance may also be a point cloud sample in a 3D dataset. Such an example may be as shown in FIG. 3 . In Figure 3, the example includes 6 class labels, namely traffic sign (traffic light), traffic sign (traffic light), rider (rider), bike (bicycle), truck (truck), and car (car) . In this embodiment, each object in the instance is marked by a two-dimensional frame. Each category label is used to represent an object. The class labels are indicated on the instance, and the labeling of these class labels helps determine the type of each object. For example, the category label -car indicates that the object selected by the box is a car, the category label -traffic sign indicates that the object selected by the box is a traffic light, the category label -rider indicates that the object selected is a rider, and the category label -bike The box-selected object is a bicycle, and the category label -truck indicates that the box-selected object is a truck. The same class label belongs to a class. The above 6 category labels belong to 5 categories, namely car, traffic light, rider, bicycle, and truck.
S202：根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别。S202: Classify the instances in the sample set according to the categories to form multiple categories according to the category labels, each instance is divided into one or more categories, and each category corresponds to a category.
在本实施例中，如果一个实例包括一个类别标签，则将所述实例的索引添加至所述类别下。如果一个实例包括多个类别标签，则将所述实例的索引分别添加至相应的类别下。如果一个实例包括多个相同类别标签，则将所述实例的索引添加至所述类别一次。以上述图3中的实例为例，上述实例包括2个traffic sign(交通灯)类别标签，1个car(汽车)类别标签，1个rider(骑手)类别标签，1个bike(自行车)类别标签、及1个truck(卡车)类别标签，则分别在类别交通灯、类别汽车、类别骑手、类别自行车、及类别卡车下添加上述实例的索引号。所述在类别下添加实例的索引号可如下表1所示：In this embodiment, if an instance includes a category label, the index of the instance is added under the category. If an instance includes multiple category labels, the indices of the instances are respectively added under the corresponding categories. If an instance includes multiple labels of the same category, then add the instance's index to the category once. Taking the example in Figure 3 above as an example, the above example includes 2 traffic sign (traffic light) class labels, 1 car (car) class label, 1 rider (rider) class label, and 1 bike (bicycle) class label. , and a truck (truck) category label, then add the index number of the above instance under the category traffic light, the category car, the category rider, the category bicycle, and the category truck. The index number of the instance to be added under the category can be shown in Table 1 below:
表1Table 1
类别category 索引号The index number
公共汽车bus 实例_1,…,实例_3000instance_1,...,instance_3000
交通灯traffic light 实例_2,…,实例_3000instance_2,...,instance_3000
指示牌indicator 实例_2,…,实例_2999instance_2,...,instance_2999
…… ……
骑手rider 实例_3,…,实例_3000instance_3,...,instance_3000
所述样本集中的实例分类后，形成多个类，不同类中的实例的数量可相同或不相同，如图4所示。在图4中，汽车类中的实例的数量最多，为27558例，所述自行车类中的实例的数量最少，为6263例。在图4中，头部类别占多数实例，尾部类别占极少实例，例如，例如汽车类及人类占多数实例，火车类、骑手类、摩托车类及自行车类占极少实例。After the instances in the sample set are classified, multiple classes are formed, and the number of instances in different classes may be the same or different, as shown in FIG. 4 . In Figure 4, the car class has the largest number of instances at 27558, and the bicycle class has the least number of instances at 6263. In Figure 4, the head category has the majority of instances, and the tail category has very few instances. For example, cars and humans account for the majority of instances, and train, rider, motorcycle, and bicycle categories account for very few instances.
S203：确定多个所述类中的小类。S203: Determine subclasses in the plurality of classes.
所述小类为多个所述类中数量小于第一阈值的类。具体地，所述确定多个所述类中的小类包括：根据每个类的当前实例的数量及预设目标实例数确定每个类的采样率；确定所述采样率大于零的类为多个所述类中的小类。所述根据每个类的当前实例的数量及预设目标实例数确定每个类的采样率包括：根据公式
Figure PCTCN2021132897-appb-000015
确定采样率，其中，f t为类t的采样率，N aim为所述预设目标实例数，N i为类t的当前实例的数量。所述N aim可为各类别实例数的平均值，或者略大于各类别实例数的平均值的值，或者略小于各类别实例数的平均值的值。其中，如果所述N aim太大，容易导致样本集中的实例的总数量过大，如果所述N aim太小，容易导致迭代次数过多，收敛慢。所述采样率大于零的类为多个所述类中的大类。 The subclass is a class whose number is less than the first threshold among the plurality of classes. Specifically, the determining of the sub-classes in the plurality of the classes includes: determining the sampling rate of each class according to the number of current instances of each class and the preset target number of instances; determining that the classes with the sampling rate greater than zero are A subclass of multiple said classes. Determining the sampling rate of each class according to the number of current instances of each class and the preset target instance number includes: according to the formula
Figure PCTCN2021132897-appb-000015
The sampling rate is determined, where f t is the sampling rate of class t, N aim is the number of the preset target instances, and N i is the number of current instances of class t. The N aim may be an average value of the number of instances of each category, or a value slightly larger than the average value of the number of instances of each category, or a value slightly smaller than the average value of the number of instances of each category. Wherein, if the N aim is too large, the total number of instances in the sample set is likely to be too large, and if the N aim is too small, it is likely to cause too many iterations and slow convergence. The class with the sampling rate greater than zero is a large class among the plurality of classes.
例如，以图4为例，自行车类的当前实例的数量为6263，如果预设目标实例数为14000，则根据公式
Figure PCTCN2021132897-appb-000016
确定采样率约为1.24，大于零，则可确定自行车类为多个所述类中的小类。 For example, taking Figure 4 as an example, the number of current instances of the bicycle class is 6263, if the preset target instance number is 14000, then according to the formula
Figure PCTCN2021132897-appb-000016
If it is determined that the sampling rate is about 1.24, which is greater than zero, it can be determined that the bicycle class is a subclass of the plurality of classes.
S204：根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重。S204: Determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the target class label is each The class label corresponding to the subclass in each instance, and the target sampling weight is the sampling weight corresponding to the subclass in each instance.
在本实施例中，每个实例可包括一个或多个类别标签，可被分至一个或多个类中。不同实例包括的不同类别的占比可能相同或者不相同。以上述图3中的实例为例，上述实例包括的类别交通灯、类别汽车、类别骑手、类别自行车、及类别卡车的占比分别为1/3，1/6，1/6，1/6，及1/6。则，一个类别下的不同实例的目标类别标签的占比可相同或者不相同，因此根据同一类别下不同实例进行上采样所形成的新的样本集也不同。所以，在进行上采样之前，需要确定小类中的不同类中的每个实例的目标类别标签在其所有类别标签的占比，与所述类中的所有实例的类别标签的占比之和的比值，即确定每个小类中每个实例的目标采样权重。In this embodiment, each instance may include one or more category labels, and may be grouped into one or more categories. The proportions of different categories included in different instances may or may not be the same. Taking the example in Figure 3 above as an example, the above examples include categories of traffic lights, categories of cars, categories of riders, categories of bicycles, and categories of trucks that account for 1/3, 1/6, 1/6, and 1/6, respectively. , and 1/6. Then, the proportions of target category labels of different instances under one category may be the same or different, so the new sample sets formed by upsampling according to different instances under the same category are also different. Therefore, before performing upsampling, it is necessary to determine the proportion of the target class label of each instance in different classes in the subclass in all its class labels, and the sum of the proportion of the class labels of all instances in the class The ratio of , that is, to determine the target sampling weight for each instance in each subclass.
在本实施例中，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重包括：根据小类中每个实例中的目标类别标签的数量及每个实施例中的所有类别标签的数量，确定每个小类中每个实例的目 标类别标签的占比；根据每个小类中每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重。In this embodiment, determining the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance includes: The number of target class labels in each instance in the subclass and the number of all class labels in each embodiment, determine the proportion of target class labels for each instance in each subclass; The proportion of target class labels for each instance determines the target sampling weight for each instance in each subclass.
在本实施例中，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比包括：根据公式
Figure PCTCN2021132897-appb-000017
确定每个小类中每个实例的目标类别标签的占比，其中，p ti为小类t中的实例i的目标类别标签的占比，k ti为小类t中的实例i的目标类别标签的数量，m ti为小类t中的实例i中的所有类别标签的数量。 In this embodiment, the proportion of the target class labels of each instance in each subclass is determined according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance Include: According to the formula
Figure PCTCN2021132897-appb-000017
Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
以小类m为例进行说明，如图5所示，小类m为自行车，所述小类m包括n个实例，分别为实例_1，…，实例_i，…，及实例_n。不同实例所包括的类别标签可相同或者不相同。例如，在图5中，实例_1包括的类别标签为bike，truck，bus，实例_i包括的类别标签为bike，bike，rider，motor，实例_n包括的类别标签为bike，truck，car。则，自行车类中的实例i的bike类别标签在实例i的所有类别标签的占比，即自行车类中的实例i的所述bike类别标签的占比，为
Figure PCTCN2021132897-appb-000018
由此可知，如果所述小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越多，则所述类的所述实例的目标类别标签的占比越高；如果所述小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标类别标签的占比越低。 Taking the subclass m as an example for illustration, as shown in FIG. 5 , the subclass m is a bicycle, and the subclass m includes n instances, namely instance_1, . . . , instance_i, . . . , and instance_n. The category labels included in different instances may or may not be the same. For example, in Figure 5, the class labels included in instance_1 are bike, truck, bus, the class labels included in instance_i are bike, bike, rider, motor, and the class labels included in instance_n are bike, truck, car . Then, the proportion of the bike category label of the instance i in the bicycle class in all the category labels of the instance i, that is, the proportion of the bike category label of the instance i in the bicycle class, is
Figure PCTCN2021132897-appb-000018
It can be seen from this that if the number of target class labels of instances in the subclass is greater relative to the number of all class labels in the instance, the proportion of target class labels of the instances of the class is higher ; If the number of target class labels of instances in the subclass is less relative to the number of all class labels in the instance, the proportion of target class labels of the instances of the class is lower.
在本实施例中，所述根据每个小类中每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重包括：根据公式
Figure PCTCN2021132897-appb-000019
确定每个小类中每个实例的目标采样权重，其中，w ti为小类t中的实例i的目标采样权重，p ti为小类t中的实例i的目标类别标签的占比，tn为小类t中的所有实例的总数量，
Figure PCTCN2021132897-appb-000020
为小类t中的所有实例的目标类别标签的占比之和。 In this embodiment, the determining the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance in each subclass includes: according to the formula
Figure PCTCN2021132897-appb-000019
Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t,
Figure PCTCN2021132897-appb-000020
is the sum of the proportions of the target class labels of all instances in the small class t.
以小类t为例进行说明，小类t包括实例i及实例j，实例i的bike类别标签的占比为
Figure PCTCN2021132897-appb-000021
实例j的bike类别标签的占比为
Figure PCTCN2021132897-appb-000022
则小类t的所有实例的bike类别标签的占比之和为
Figure PCTCN2021132897-appb-000023
由此可得，小类t中实例i的目标采样权重为
Figure PCTCN2021132897-appb-000024
由此可知，如果每个小类中的实例的目标类别标签的占比越高，则所述类的所述实例的目标采样权重越高；如果每个小类中的实例的目标类别标签的占比越低，则所述类的所述实例的目标采样权重越低。即如果每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越多，则所述类的所述实例的目标采样权重可能越高；如果每个小类中的实例的目标类别标签的数量相对于所述实例中的所有类别标签的数量越少，则所述类的所述实例的目标采样权重可能越低。 Taking subclass t as an example to illustrate, subclass t includes instance i and instance j, and the proportion of bike category labels of instance i is
Figure PCTCN2021132897-appb-000021
The proportion of bike category labels for instance j is
Figure PCTCN2021132897-appb-000022
Then the sum of the proportions of the bike category labels of all instances of the small class t is
Figure PCTCN2021132897-appb-000023
Thus, the target sampling weight of instance i in subclass t is
Figure PCTCN2021132897-appb-000024
It can be seen from this that if the proportion of the target class label of the instances in each subclass is higher, the target sampling weight of the instance of the class is higher; if the target class label of the instances in each subclass has a higher proportion The lower the proportion, the lower the target sampling weight for the instance of the class. That is, if the number of target class labels of instances in each subclass is greater relative to the number of all class labels in the instance, the target sampling weight of the instance of the class may be higher; if each subclass The lower the number of target class labels for an instance in a class relative to the number of all class labels in that instance, the lower the target sampling weight for that instance of that class may be.
S205：根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。S205: Perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
在本实施例中，在对小类进行上采样时，如果主要是对所述小类中的目标类别标签的数量相对于所述实例中的所有类别标签的数量较多的实例进行上采样，将会使得新增的实例主要是包括较多目标类别标签的实例，将会减小各类别的不均衡。In this embodiment, when up-sampling a sub-class, if the up-sampling is mainly performed on an instance in which the number of target class labels in the sub-class is larger than the number of all class labels in the instance, This will make the newly added instances mainly include instances with more target category labels, which will reduce the imbalance of various categories.
在本实施例中，所述加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种。In this embodiment, the weighted sampling includes at least one of a weighted random sampling method WRS and a naive weighted sampling method.
在本实施例中，所述根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样可包括：In this embodiment, the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include:
针对所述小类中每个实例生成预设区间的随机数；Generate a random number in a preset interval for each instance in the subclass;
根据所述随机数及所述目标采样权重通过加权随机采样法WRS确定所述小类中每个实例的采样分值；Determine the sampling score of each instance in the subclass by the weighted random sampling method WRS according to the random number and the target sampling weight;
将采样分值最大的目标实例作为新增实例。The target instance with the largest sampling score is used as the new instance.
在本实施例中，所述预设区间为(0，1)区间。所述预设区间还可为其他区间，例如(0,2)等。本案可利用随机数生成器，生成多个(0，1)区间的随机数。所述根据所述随机数及所述目标采样权重通过加权随机采样法WRS确定所述小类中每个实例的采样分值包括：根据公式
Figure PCTCN2021132897-appb-000025
ti∈tn确定采样率，其中，S ti为类t中的实例i的采样分值，R ti为类t中的实例i的随机数，w ti为小类t中的实例i的目标采样权重，ti为小类t中的第i个实例，tn为小类t中的第n个实例。例如，继续以上述的类t为例进行说明，小类t中实例i的目标采样权重为
Figure PCTCN2021132897-appb-000026
小类t中实例j的目标采样权重为
Figure PCTCN2021132897-appb-000027
若小类t中实例i的随机数为0.8，小类t中实例j的随机数为0.6，可确定小类t中实例i和实例j的采样分值分别为0.69及0.28，则将采样分值为0.69的实例i作为新增实例。 In this embodiment, the preset interval is a (0, 1) interval. The preset interval can also be other intervals, such as (0, 2). In this case, a random number generator can be used to generate random numbers in multiple (0, 1) intervals. The determining the sampling score of each instance in the subclass by the weighted random sampling method WRS according to the random number and the target sampling weight includes: according to the formula
Figure PCTCN2021132897-appb-000025
ti ∈ tn determines the sampling rate, where S ti is the sampling score of instance i in class t, R ti is the random number of instance i in class t, and w ti is the target sampling weight of instance i in small class t , ti is the ith instance in subclass t, and tn is the nth instance in subclass t. For example, continuing to take the above class t as an example, the target sampling weight of instance i in subclass t is
Figure PCTCN2021132897-appb-000026
The target sampling weight of instance j in subclass t is
Figure PCTCN2021132897-appb-000027
If the random number of instance i in subclass t is 0.8, and the random number of instance j in subclass t is 0.6, it can be determined that the sampling scores of instance i and instance j in subclass t are 0.69 and 0.28 respectively, then the sampling score Instance i with a value of 0.69 is used as a new instance.
在本实施例中，所述根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样可包括：In this embodiment, the weighted sampling of the instances of each subclass according to the target sampling weight of each instance in the subclass may include:
针对所述小类中所有实例按照目标采样权重从小到大的顺序排序；Sort all the instances in the subclass according to the order of the target sampling weight from small to large;
生成预设区间的随机数；Generate random numbers in preset intervals;
根据所述随机数及所述目标采样权重的排序通过朴素加权采样法确定新增实例。The newly added instance is determined by the naive weighted sampling method according to the order of the random number and the target sampling weight.
在本实施例中，所述预设区间为(0，1)区间。所述预设区间还可为其他区间，例如(0,2)等。本案可利用随机数生成器，生成多个(0，1)区间的随机数。所述根据所述随机数及所述目标采样权重的排序通过朴素加权采样法确定新增实例包括：按照所述目标采样权重的排序从第一个实例的目标采样权重开始逐一累加所述实例的目标采样权重直至目标实例时累加的目标采样权重达到所述随机数；将所述目标实例作为新增实例。例如，继续以上述的类t为例进行说明，小类t中实例i的目标采样权重为
Figure PCTCN2021132897-appb-000028
小类t中实例j的目标采样权重为
Figure PCTCN2021132897-appb-000029
则目标采样权重从小到大排序为
Figure PCTCN2021132897-appb-000030
若随机数为0.5，第一个实例的目标采样权重为0.4，没有达到0.5，累加第二个实例的目标采样权重，累加的目标采样权重为1，达到了0.5，则可确定目标采样权重为0.6的实例为目标实例，并将目标实例作为新增实例。 In this embodiment, the preset interval is a (0, 1) interval. The preset interval can also be other intervals, such as (0, 2). In this case, a random number generator can be used to generate random numbers in multiple (0, 1) intervals. The determining of the newly added instance by the naive weighted sampling method according to the ordering of the random number and the target sampling weight includes: accumulating the instances of the instance one by one starting from the target sampling weight of the first instance according to the ordering of the target sampling weight. The target sampling weight reaches the random number when the accumulated target sampling weight reaches the target instance; the target instance is regarded as a newly added instance. For example, continuing to take the above class t as an example, the target sampling weight of instance i in subclass t is
Figure PCTCN2021132897-appb-000028
The target sampling weight of instance j in subclass t is
Figure PCTCN2021132897-appb-000029
Then the target sampling weights are sorted from small to large as
Figure PCTCN2021132897-appb-000030
If the random number is 0.5, the target sampling weight of the first instance is 0.4, and it does not reach 0.5, and the target sampling weight of the second instance is accumulated, the accumulated target sampling weight is 1, and it reaches 0.5, then the target sampling weight can be determined as The instance of 0.6 is the target instance, and the target instance is regarded as the newly added instance.
在本实施例中，通过加权随机采样法WRS确定所述小类每个实例的采样分值，目标采样权重越大的实例的采样分值越会越大。通过将采样分值最大的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来，将会使得新增的实例主要是包括较多目标类别标签的实例，将会减小各类别的不均衡。In this embodiment, the weighted random sampling method WRS is used to determine the sampling score of each instance of the sub-category, and the sampling score of an instance with a larger target sampling weight will be higher. By taking the target instance with the largest sampling score as the new instance, the instance with the larger target sampling weight will be easier to be sampled, and the newly added instance will mainly include the instance with more target category labels, which will reduce the Minor categories are not balanced.
通过朴素加权采样法确定按照目标采样权重从小到大的排序中的逐一累加值，目标采样权重越大的实例的逐一累加值越大。通过将逐一累加值达到随机数的目标实例作为新增实例，将使得目标采样权重越大的实例越容易被取样出来，将会使得新增的实例主要是包括较多目标类别标签的实例，将会减小各类别的不均衡。The naive weighted sampling method is used to determine the accumulated value one by one in the order of the target sampling weight from small to large, and the one-by-one accumulated value of the instance with a larger target sampling weight is larger. By taking the target instance whose accumulated value reaches a random number one by one as a new instance, the instance with a larger target sampling weight will be easier to be sampled, and the newly added instance will mainly include more target category labels. It will reduce the imbalance of various categories.
S206：将根据加权采样结果确定的目标实例作为新增实例构造所述小类。S206: Construct the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
在本实施例中，重复多次新增实例步骤来构造所述小类。具体地，重复多次新增实例步骤，即步骤S205，并将新增实例加入原小类中来构造所述小类。例如，继续以小类t为例进 行说明，小类t中包括实例i及实例j，若新增实例包括实例k及实例l，则构造的小类包括：实例i、实例j、实例k及实例l。In this embodiment, the step of adding an instance is repeated multiple times to construct the subclass. Specifically, the step of adding an instance is repeated several times, that is, step S205, and the new instance is added to the original subclass to construct the subclass. For example, continue to take subclass t as an example to illustrate, subclass t includes instance i and instance j, if the new instance includes instance k and instance l, the constructed subclass includes: instance i, instance j, instance k and Example l.
在本实施例中，所述方法还可根据所述实例对初始AI模型进行训练来得到目标模型。In this embodiment, the method may further train the initial AI model according to the instance to obtain the target model.
本案通过获取包括多个实例的样本集，每个实例包括一个或多个类别标签；根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样，可仅新增实例，原始的实例会被全部保留，不丢失实例，同时可使得小类中的目标类别标签的数量相对于所有类别标签的数量越多的实例越容易被上采样，对其他类别影响较小，可避免场景丢失及间接增加其他类别的数量。In this case, a sample set including multiple instances is obtained, and each instance includes one or more class labels; the instances in the sample set are classified according to classes to form multiple classes according to the class labels, and each instance is classified into multiple classes. to one or more classes, each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the value of each instance in each subclass Target sampling weight, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is each Each instance corresponds to the sampling weight of the subclass; according to the target sampling weight of each instance in the subclass, weighted sampling is performed on the instance of each subclass, and only new instances can be added, and the original instances will be all Retain, do not lose instances, and at the same time, the number of target category labels in the sub-class can be easily upsampled relative to the number of all category labels, and the impact on other categories is small, which can avoid scene loss and indirectly increase other categories. number of categories.
请参考图6，为本发明第二实施例的多标签的类均衡方法的流程示意图。Please refer to FIG. 6 , which is a schematic flowchart of a multi-label class equalization method according to a second embodiment of the present invention.
所述方法具体可以由如图1所示的训练设备120执行，所述方法中的多个实例的样本集可以是如图1所示的数据库130中维护的训练数据，可选的，所述方法的步骤S602至步骤S612可以在所述训练设备120中执行，也可以在训练设备120之前由其他功能模块预先执行，即先对从所述数据库130中接收或者获取到的训练数据进行预处理，如步骤S602至步骤S612所述的采样过程，得到新的样本集，作为所述训练设备120的输入，并由所述训练设备执行模型的训练。The method may be specifically performed by the training device 120 shown in FIG. 1 , and the sample sets of the multiple instances in the method may be the training data maintained in the database 130 shown in FIG. 1 . Optionally, the Steps S602 to S612 of the method may be executed in the training device 120, or may be pre-executed by other functional modules before the training device 120, that is, preprocessing the training data received or acquired from the database 130 first , according to the sampling process described in steps S602 to S612, a new sample set is obtained, which is used as the input of the training device 120, and the training device performs model training.
可选地，所述方法可以由CPU处理，也可以由CPU和GPU共同处理，也可以不用GPU，而使用其他适合用于处理处理的处理器，本申请不做限制。Optionally, the method may be processed by the CPU, or may be jointly processed by the CPU and the GPU, or other processors suitable for processing may be used without using the GPU, which is not limited in this application.
所述方法包括但不限于如下步骤：The method includes but is not limited to the following steps:
S601：获取包括多个实例的样本集，每个实例包括一个或多个类别标签。S601: Acquire a sample set including multiple instances, each instance including one or more category labels.
第二实施例中的步骤S601与第一实施例中的步骤S201相似，具体请参考图2中对第一实施例中的步骤S201的详细描述，在此不再进行赘述。Step S601 in the second embodiment is similar to step S201 in the first embodiment. For details, please refer to the detailed description of step S201 in the first embodiment in FIG. 2 , which will not be repeated here.
S602：根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别。S602: Classify the instances in the sample set into multiple classes according to the class labels, each instance is classified into one or more classes, and each class corresponds to a class.
第二实施例中的步骤S602与第一实施例中的步骤S202相似，具体请参考图2中对第一实施例中的步骤S202的详细描述，在此不再进行赘述。Step S602 in the second embodiment is similar to step S202 in the first embodiment. For details, please refer to the detailed description of step S202 in the first embodiment in FIG. 2 , which will not be repeated here.
S603：确定多个所述类中的小类。S603: Determine subclasses among the plurality of classes.
第二实施例的所述确定多个所述类中的小类的流程与第一实施例的步骤S203中确定多个所述类中的小类的流程相似，具体请参考图2中对第一实施例中的步骤S203的详细描述，在此不再进行赘述。The process of determining subclasses in the plurality of classes in the second embodiment is similar to the process of determining subclasses in the classes in step S203 of the first embodiment. The detailed description of step S203 in an embodiment is not repeated here.
在本实施例中，采用迭代采样的方式，每轮采样后皆会存在一个或多个类的实例的数量发生变化，则每轮采样时，需要预设目标实例数，并确定多个所述类中的小类。In this embodiment, the iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling. A subclass within a class.
S604：根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重。S604: Determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the target class label is each The class label corresponding to the subclass in each instance, and the target sampling weight is the sampling weight corresponding to the subclass in each instance.
第二实施例中的步骤S604与第一实施例中的步骤S204相似，具体请参考图2中对第一 实施例中的步骤S204的详细描述，在此不再进行赘述。Step S604 in the second embodiment is similar to step S204 in the first embodiment. For details, please refer to the detailed description of step S204 in the first embodiment in FIG. 2, which will not be repeated here.
S605：根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。S605: Perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
第二实施例中的步骤S605与第一实施例中的步骤S205相似，具体请参考图2中对第一实施例中的步骤S205的详细描述，在此不再进行赘述。Step S605 in the second embodiment is similar to step S205 in the first embodiment. For details, please refer to the detailed description of step S205 in the first embodiment in FIG. 2 , which will not be repeated here.
S606：将根据加权采样结果确定的目标实例作为新增实例构造所述小类。S606: Construct the subclass by using the target instance determined according to the weighted sampling result as a newly added instance.
第二实施例中的步骤S606与第一实施例中的步骤S206相似，具体请参考图2中对第一实施例中的步骤S206的详细描述，在此不再进行赘述。Step S606 in the second embodiment is similar to step S206 in the first embodiment. For details, please refer to the detailed description of step S206 in the first embodiment in FIG. 2 , which will not be repeated here.
S607：确定所述第一数量。S607: Determine the first quantity.
所述确定所述第一数量包括：确定所述第一数量为所有类的当前实例的数量的平均值。所述第一数量还可为其他数量，例如略大于各类别实例数的平均值的值，或者略小于各类别实例数的平均值的值。所述第一数量可与所述预设目标实例数相同，也可与所述预设目标实例数不相同。The determining of the first number includes: determining the first number to be an average of the numbers of current instances of all classes. The first number may also be other numbers, for example, a value slightly larger than the average value of the number of instances of each category, or a value slightly smaller than the average value of the number of instances of each category. The first number may be the same as the preset target instance number, or may be different from the preset target instance number.
在本实施例中，采用迭代采样的方式，每轮采样后皆会存在一个或多个类的实例的数量发生变化，则每轮采样时，需要确定所述第一数量。In this embodiment, an iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling, and the first number needs to be determined during each round of sampling.
S608：构造所述小类中的实例至第一数量。S608: Construct instances in the subclass to a first number.
在本实施例中，构造所述小类中的实例至第一数量包括：In this embodiment, constructing the instances in the subclass to the first number includes:
判断所述小类中的实例的数量是否达到第一数量；Determine whether the number of instances in the subclass reaches the first number;
若所述小类中的实例的数量没有达到第一数量，继续根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样来构造小类直至所述小类中的实例的数量达到第一数量，即继续执行步骤S605至步骤S606直至所述小类中的实例的数量达到第一数量。If the number of instances in the sub-class does not reach the first number, continue to perform weighted sampling on the instances of each sub-class according to the target sampling weight of each instance in the sub-class to construct a sub-class until the sub-class is When the number of instances in the class reaches the first number, step S605 to step S606 are continued until the number of instances in the subclass reaches the first number.
具体地，在针对所示小类每个实例生成预设区间的随机数之前，所述方法还包括：针对所述小类根据所述小类的当前实例的数量及所述第一数量确定需构造的实例的数量。需构造的实例的数量可为所述第一数量与所述小类的当前实例的数量的差值。例如，对于小类t，假设其实例数为n，则小类t需构造的实例的数量为delta＝avg-n。其中，delta为所述需构造的实例的数量，avg为所述第一数量，n为小类t的当前实例的数量。在本实施例中，判断所述小类中的实例的数量是否达到第一数量，若所述小类中的实例的数量没有达到第一数量，则重复对所述小类中的实例进行加权采样来得到delta个新增实例。从而，可使每个小类中的实例的数量达到第一数量。以上述的图4为例，在对每个小类进行加权采样后的示意图如图7所示。在图7中，所有小类，例如指示牌、公共汽车、火车、骑手、摩托车、及自行车的实例的数量皆增加至第一数量。Specifically, before generating a random number of a preset interval for each instance of the shown subclass, the method further includes: determining, for the subclass, the number of current instances of the subclass and the first number to be The number of instances constructed. The number of instances to be constructed may be the difference between the first number and the number of current instances of the subclass. For example, for a small class t, assuming that the number of its instances is n, the number of instances to be constructed for the small class t is delta=avg-n. Wherein, delta is the number of instances to be constructed, avg is the first number, and n is the current number of instances of subclass t. In this embodiment, it is determined whether the number of instances in the subclass reaches the first number, and if the number of instances in the subclass does not reach the first number, the instances in the subclass are repeatedly weighted Sampling to get delta new instances. Thus, the number of instances in each subclass can be brought up to the first number. Taking the above-mentioned FIG. 4 as an example, the schematic diagram after weighted sampling is performed on each subclass is shown in FIG. 7 . In Figure 7, the number of instances of all subcategories such as sign, bus, train, rider, motorcycle, and bicycle are increased to the first number.
S609：确定第二数量。S609: Determine the second quantity.
所述确定第二数量包括：确定所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量。所述r不仅局限于上述的值，所述r还可为其他值，例如0.48，0.81等。其中，如果所述第二数量太大，容易导致新的样本集中的重复样本的数量较多，容易导致过拟合，如果所述第二数量太小，容易各类别的均衡效果不明显。 The determining of the second number includes: determining the second number as r*N max , where r∈[0.5, 0.8], and N max is the number of instances of the largest class. The r is not limited to the above-mentioned values, and the r can also be other values, such as 0.48, 0.81, and the like. Wherein, if the second number is too large, the number of repeated samples in the new sample set is likely to be large, which may easily lead to over-fitting, and if the second number is too small, the equalization effect of each category is likely to be insignificant.
在本实施例中，采用迭代采样的方式，每轮采样后皆会存在一个或多个类的实例的数量发生变化，则每轮采样时，需要确定所述第二数量。In this embodiment, the iterative sampling method is adopted, and the number of instances of one or more classes will change after each round of sampling, and the second number needs to be determined during each round of sampling.
S610：判断每个小类中的实例的数量是否达到第二数量。S610: Determine whether the number of instances in each subclass reaches the second number.
S611：若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样 本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，继续对每个小类的实例进行加权采样，及继续将根据加权采样结果确定的目标实例作为新增实例构造所述小类直至每个小类中的实例的数量达到第二数量。S611: If the number of instances in each sub-category does not reach the second number, continue to classify the instances in the sample set according to categories to form multiple categories according to the category labels, and continue to determine the number of instances in each sub-category. the target sampling weight of each instance, continue to perform weighted sampling on the instances of each subclass, and continue to construct the subclasses with the target instances determined according to the weighted sampling results as new instances until the number of instances in each subclass reaches second quantity.
在本实施例中，若所述小类中的实例的数量没有达到第二数量，则重复执行步骤S602-S610，从而，可使每个小类中的实例的数量达到第二数量。以上述的图4为例，在对每个小类迭代构造小类的示意图如图8所示。在图8中，经过了3轮采样，所有小类，例如卡车、交通灯、指示牌、公共汽车、火车、骑手、摩托车、及自行车的实例的数量皆增加至第二数量。In this embodiment, if the number of instances in the subclass does not reach the second number, steps S602-S610 are repeated, so that the number of instances in each subclass can reach the second number. Taking the above-mentioned FIG. 4 as an example, a schematic diagram of iteratively constructing a subclass for each subclass is shown in FIG. 8 . In Figure 8, after 3 rounds of sampling, the number of instances of all subcategories, such as truck, traffic light, sign, bus, train, rider, motorcycle, and bicycle, is increased to the second number.
S612：若每个小类中的实例的数量达到第二数量，输出所有类中的实例。S612: If the number of instances in each subclass reaches the second number, output the instances in all classes.
在本实施例中，所述方法还可根据实例对初始AI模型进行训练来得到目标模型。In this embodiment, the method may further train the initial AI model according to the instance to obtain the target model.
本案通过获取包括多个实例的样本集，每个实例包括一个或多个类别标签；根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样；将根据加权采样结果确定的目标实例作为新增实例构造所述小类；构造所述小类中的实例至第一数量；判断每个小类中的实例的数量是否达到第二数量；若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，继续对每个小类的实例进行加权采样，及继续将根据加权采样结果确定的目标实例作为新增实例构造所述小类直至每个小类中的实例的数量达到第二数量，可仅新增实例，原始的实例会被全部保留，不丢失实例，同时可使得小类中的目标类别标签的数量相对于所有类别标签的数量越多的实例越容易被上采样，对其他类别影响较小，基于迭代采样策略，由于新增实例，每轮采样时预设目标实例数、第一数量及第二数量皆会发生调整，可确保各类别中的实例的数量皆达到第二数量才退出迭代，进一步避免了对其他类别的影响，从而可避免场景丢失及间接增加其他类别的数量，进而保留了原始数据，并解决了类别不均衡的问题。In this case, a sample set including multiple instances is obtained, and each instance includes one or more class labels; the instances in the sample set are classified according to classes to form multiple classes according to the class labels, and each instance is classified into multiple classes. to one or more classes, each class corresponds to a class; according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the value of each instance in each subclass Target sampling weight, wherein the subclass is a class whose number is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is each The sampling weights corresponding to the subclasses in each instance; according to the target sampling weights of each instance in the subclasses, weighted sampling is performed on the instances of each subclass; the target instances determined according to the weighted sampling results are used as new Instance constructs the subclass; constructs the instances in the subclass to the first number; determines whether the number of instances in each subclass reaches the second number; if the number of instances in each subclass does not reach the second number continue to classify the instances in the sample set according to the categories to form multiple categories according to the category labels, continue to determine the target sampling weight of each instance in each sub-category, and continue to perform Weighted sampling, and continue to use the target instance determined according to the weighted sampling result as a new instance to construct the subclass until the number of instances in each subclass reaches the second number, only new instances can be added, and all the original instances will be deleted. Retain, do not lose instances, and at the same time make the number of target class labels in the subclass easier to be upsampled compared to instances with more class labels, and has less impact on other classes. Based on the iterative sampling strategy, due to the new For example, the preset target number of instances, the first number and the second number will be adjusted in each round of sampling, which can ensure that the number of instances in each category reaches the second number before exiting the iteration, further avoiding the impact on other categories. , which can avoid scene loss and indirectly increase the number of other categories, thus preserving the original data and solving the problem of category imbalance.
请参考图9，图9为本发明实施例提供的一种多标签的类均衡装置的结构示意图，所述多标签的类均衡装置900可以包括获取单元901和采样单元902。Please refer to FIG. 9 . FIG. 9 is a schematic structural diagram of a multi-label quasi-equalization apparatus according to an embodiment of the present invention. The multi-label quasi-equalization apparatus 900 may include an acquisition unit 901 and a sampling unit 902 .
多标签的类均衡装置900可以用于执行本申请实施例的多标签的类均衡方法的步骤。例如，获取单元901可以用于执行图2所示方法中的步骤S201，采样单元902可以用于执行图2所示方法中的步骤S202至步骤S206。又例如，获取单元901可以用于执行图6所示方法中的步骤S601，采样单元902可以用于执行图6所示方法中的步骤S602至步骤S612。The multi-label class equalization apparatus 900 may be used to execute the steps of the multi-label class equalization method of the embodiment of the present application. For example, the acquiring unit 901 may be configured to perform step S201 in the method shown in FIG. 2 , and the sampling unit 902 may be configured to perform steps S202 to S206 in the method shown in FIG. 2 . For another example, the acquiring unit 901 may be configured to perform step S601 in the method shown in FIG. 6 , and the sampling unit 902 may be configured to perform steps S602 to S612 in the method shown in FIG. 6 .
所述多标签的类均衡装置900还可包括训练单元903。所述训练单元903用于根据所述实例对初始AI模型进行训练来得到目标模型。The multi-label equalization-like apparatus 900 may further include a training unit 903 . The training unit 903 is configured to train the initial AI model according to the instance to obtain the target model.
请参见图10，图10为本发明实施例提供的一种电子设备的硬件结构示意图。图10所示的电子设备1000包括存储器1001、处理器1002、通信接口1003以及总线1004。其中，存储器1001、处理器1002、通信接口1003通过总线1004实现彼此之间的通信连接。Please refer to FIG. 10. FIG. 10 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention. The electronic device 1000 shown in FIG. 10 includes a memory 1001 , a processor 1002 , a communication interface 1003 and a bus 1004 . The memory 1001 , the processor 1002 , and the communication interface 1003 are connected to each other through the bus 1004 for communication.
存储器1001可以是只读存储器(Read Only Memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(Random Access Memory，RAM)。存储器1001可以存储程序，当存储器1001中存储的程序被处理器1002执行时，处理器1002和通信接口1003用于执行本申请实施例的多标签的类均衡方法的各个步骤。The memory 1001 may be a read-only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM). The memory 1001 may store a program. When the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 and the communication interface 1003 are used to execute each step of the multi-tag equalization-like method of the embodiment of the present application.
处理器1002可以采用通用的中央处理器(Central Processing Unit，CPU)，微处理器，应用专用集成电路(Application Specific Integrated Circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序，以实现本申请实施例的多标签的类均衡装置中的单元所需执行的功能，或者执行本申请方法实施例的多标签的类均衡方法。The processor 1002 may adopt a general-purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a graphics processing unit (graphics processing unit, GPU) or one or more The integrated circuit is used to execute the relevant program to realize the functions required to be performed by the units in the multi-tag equalization-like apparatus of the embodiment of the present application, or to execute the multi-tag equalization-like method of the method embodiment of the present application.
处理器1002还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的多标签的类均衡方法的各个步骤可以通过处理器1002中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1002还可以是通用处理器、数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者所述处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。所述存储介质位于存储器1001，处理器1002读取存储器1001中的信息，结合其硬件完成本申请实施例的多标签的类均衡装置中包括的单元所需执行的功能，或者执行本申请方法实施例的多标签的类均衡方法。The processor 1002 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the multi-tag quasi-equalization method of the present application may be completed by hardware integrated logic circuits in the processor 1002 or instructions in the form of software. The above-mentioned processor 1002 can also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001 and, in combination with its hardware, completes the functions required to be performed by the units included in the multi-tag equalizer-like device of the embodiment of the present application, or executes the method implementation of the present application. Example of a multi-label class equalization method.
通信接口1003使用例如但不限于收发器一类的收发装置，来实现电子设备1000与其他设备或通信网络之间的通信。例如，可以通过通信接口1003获取数据(如本申请实施例一及实施例二中的包括多个实例的样本集)。The communication interface 1003 implements communication between the electronic device 1000 and other devices or a communication network using a transceiver such as but not limited to a transceiver. For example, data (such as the sample set including multiple instances in Embodiment 1 and Embodiment 2 of the present application) can be acquired through the communication interface 1003 .
总线1004可包括在电子设备1000各个部件(例如，存储器1001、处理器1002、通信接口1003)之间传送信息的通路。 Bus 1004 may include a pathway for communicating information between various components of electronic device 1000 (eg, memory 1001, processor 1002, communication interface 1003).
可选的，所述电子设备1000还可以包括输出组件，例如，显示器、音响等，所述输出组件用于向开发人员展示训练模型要用到的参数，因此开发人员可以获知这些参数，也可以对这些参数进行修改，并通过输入组件(例如，鼠标、键盘等)将修改后的参数输入到所述电子设备1000中。另外，所述电子设备1000还可以通过输出组件将训练出的目标模型展示给开发人员。Optionally, the electronic device 1000 may further include an output component, such as a display, a sound box, etc., the output component is used to display the parameters to be used for training the model to the developer, so the developer can know these parameters, or These parameters are modified, and the modified parameters are input into the electronic device 1000 through an input component (eg, mouse, keyboard, etc.). In addition, the electronic device 1000 can also display the trained target model to the developer through the output component.
应理解，多标签的类均衡装置900中的获取单元901相当于所述电子设备1000中的通信接口1003，采样单元902可以相当于处理器1002。It should be understood that the acquisition unit 901 in the multi-tag equalization-like apparatus 900 is equivalent to the communication interface 1003 in the electronic device 1000 , and the sampling unit 902 may be equivalent to the processor 1002 .
请参见图11，图11为本发明实施例提供的一种电子装置的结构示意图，所述电子装置1100包括存储器1101、处理器1102、通信接口1103以及总线1104。其中，存储器1101、处理器1102、通信接口1103通过总线1104实现彼此之间的通信连接。Please refer to FIG. 11 . FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 1100 includes a memory 1101 , a processor 1102 , a communication interface 1103 and a bus 1104 . The memory 1101 , the processor 1102 , and the communication interface 1103 are connected to each other through the bus 1104 for communication.
存储器1101可以是只读存储器(Read Only Memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(Random Access Memory，RAM)。存储器1101可以存储程 序，当存储器1101中存储的程序被处理器1102执行时，处理器1102和通信接口1103用于执行目标检测方法的各个步骤。The memory 1101 may be a read only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM). The memory 1101 can store programs, and when the programs stored in the memory 1101 are executed by the processor 1102, the processor 1102 and the communication interface 1103 are used to execute various steps of the target detection method.
处理器1102可以采用通用的中央处理器(Central Processing Unit，CPU)，微处理器，应用专用集成电路(Application Specific Integrated Circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序，以执行目标检测方法。The processor 1102 may adopt a general-purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a graphics processing unit (graphics processing unit, GPU) or one or more An integrated circuit for executing the associated program to perform the object detection method.
处理器1102还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，目标检测方法的各个步骤可以通过处理器1102中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1102还可以是通用处理器、数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者所述处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。所述存储介质位于存储器1101，处理器1102读取存储器1101中的信息，结合其硬件完成目标检测方法。The processor 1102 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the target detection method may be completed by hardware integrated logic circuits in the processor 1102 or instructions in the form of software. The above-mentioned processor 1102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices. , discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and completes the target detection method in combination with its hardware.
通信接口1103使用例如但不限于收发器一类的收发装置，来实现电子装置1100与其他设备或通信网络之间的通信。例如，所述通信接口1103与上述训练设备建立了通信连接，可以通过通信接口1103接收上述训练设备发送的目标模型的参数。所述目标模型的参数可以存储在存储器1101中以供调用。The communication interface 1103 uses a transceiver such as but not limited to a transceiver to implement communication between the electronic device 1100 and other devices or a communication network. For example, the communication interface 1103 establishes a communication connection with the above-mentioned training device, and the parameters of the target model sent by the above-mentioned training device can be received through the communication interface 1103 . The parameters of the target model can be stored in the memory 1101 for recall.
总线1104可包括在电子装置1100各个部件(例如，存储器1101、处理器1102、通信接口1103)之间传送信息的通路。The bus 1104 may include a pathway for communicating information between the various components of the electronic device 1100 (eg, the memory 1101, the processor 1102, the communication interface 1103).
可选的，所述电子装置1100还可以包括输出组件，例如，显示器、音响等，所述输出组件用于向用户展示由目标模型得到的目标检测结果。Optionally, the electronic device 1100 may further include an output component, such as a display, a sound box, etc., where the output component is used to display the target detection result obtained by the target model to the user.
所述目标检测方法可为现有中的任何目标检测方法，在此不再进行赘述。The target detection method may be any existing target detection method, which will not be repeated here.
应注意，尽管图10和图11所示的电子设备1000和电子装置1100仅仅示出了存储器、处理器、通信接口，但是在具体实现过程中，本领域的技术人员应当理解，电子设备1000和电子装置1100还包括实现正常运行所必须的其他器件。同时，根据具体需要，本领域的技术人员应当理解，电子设备1000和电子装置1100还可包括实现其他附加功能的硬件器件。此外，本领域的技术人员应当理解，装置电子设备1000和电子装置1100也可仅仅包括实现本申请实施例所必须的器件，而不必包括图10或图11中所示的全部器件。It should be noted that although the electronic device 1000 and the electronic device 1100 shown in FIG. 10 and FIG. 11 only show a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the electronic device 1000 and the electronic device 1100 The electronic device 1100 also includes other components necessary for normal operation. Meanwhile, according to specific needs, those skilled in the art should understand that the electronic device 1000 and the electronic device 1100 may further include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device electronic device 1000 and the electronic device 1100 may also only include the necessary devices for implementing the embodiments of the present application, and do not necessarily include all the devices shown in FIG. 10 or FIG. 11 .
可以理解，所述电子设备1000相当于1中的所述训练设备120，所述电子装置1100相当于图1中的所述执行设备110。本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。It can be understood that the electronic device 1000 is equivalent to the training device 120 in 1, and the electronic device 1100 is equivalent to the execution device 110 in FIG. 1 . Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.
除以上方法和装置外，本发明实施例还提供一种计算机可读存储介质，所述计算机可读 存储介质中存储有指令，当其在处理器上运行时，实现图2或图6所示的多标签的类均衡方法。In addition to the above method and apparatus, an embodiment of the present invention also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium runs on a processor, the implementation shown in FIG. 2 or FIG. 6 is implemented. The multi-label class equalization method.
一种计算机程序产品，所述计算机程序产品包括计算机执行指令，所述计算机执行指令存储在计算机可读存储介质中；设备的至少一个处理器可以从所述计算机可读存储介质中读取所述计算机执行指令，所述至少一个处理器执行所述计算机执行指令使得所述设备实现图2或图6所示的多标签的类均衡方法。A computer program product comprising computer-executable instructions stored in a computer-readable storage medium; from which the computer-readable storage medium can be read by at least one processor of a device Computer-executed instructions, the at least one processor executing the computer-executed instructions causes the device to implement the multi-tag equalization-like method shown in FIG. 2 or FIG. 6 .
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (22)

  1. 一种多标签的类均衡方法，其特征在于，所述方法包括：A multi-label class equalization method, characterized in that the method comprises:
    获取包括多个实例的样本集，每个实例包括一个或多个类别标签；Obtain a sample set including multiple instances, each instance including one or more class labels;
    根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；Classify the instances in the sample set according to the class labels to form a plurality of classes, each instance is classified into one or more classes, and each class corresponds to a class;
    根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；The target sampling weight of each instance in each subclass is determined according to the number of target class labels in each instance and the number of all class labels in each instance, wherein the subclass is a plurality of the The number of classes in the class is less than the first threshold, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is the sampling weight corresponding to the subclass in each instance;
    根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。Instances of each subclass are sampled weighted according to the target sampling weight of each instance in the subclass.
  2. 如权利要求1所述的多标签的类均衡方法，其特征在于，所述方法还包括：The multi-label class equalization method according to claim 1, wherein the method further comprises:
    将根据加权采样结果确定的目标实例作为新增实例构造所述小类。The subclass is constructed by taking the target instance determined according to the weighted sampling result as a newly added instance.
  3. 如权利要求2所述的多标签的类均衡方法，其特征在于，所述方法还包括：The multi-label class equalization method according to claim 2, wherein the method further comprises:
    构造所述小类中的实例至第一数量；constructing instances in the subclass to a first number;
    判断每个小类中的实例的数量是否达到第二数量；Determine whether the number of instances in each subclass reaches the second number;
    若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，继续对每个小类的实例进行加权采样，及继续将根据加权采样结果确定的目标实例作为新增实例构造所述小类直至每个小类中的实例的数量达到第二数量。If the number of instances in each subclass does not reach the second number, continue to classify the instances in the sample set according to the class according to the class label to form multiple classes, and continue to determine each instance in each subclass the target sampling weight, continue to perform weighted sampling on the instances of each subclass, and continue to construct the subclasses with the target instances determined according to the weighted sampling results as new instances until the number of instances in each subclass reaches the second quantity.
  4. 如权利要求3所述的多标签的类均衡方法，其特征在于，在所述构造所述小类中的实例至第一数量之前，所述方法还包括：The multi-label class balancing method according to claim 3, characterized in that before the constructing the instances in the subclass to the first number, the method further comprises:
    确定所述第一数量为所有类的当前实例的数量的平均值。The first number is determined to be the average of the number of current instances of all classes.
  5. 如权利要求3所述的多标签的类均衡方法，其特征在于，在所述判断每个小类中的实例的数量是否达到第二数量之前，所述方法还包括：The multi-label class balancing method according to claim 3, wherein before judging whether the number of instances in each subclass reaches the second number, the method further comprises:
    确定所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量。 The second number is determined to be r*N max , where r∈[0.5,0.8] and N max is the number of instances of the largest class.
  6. 如权利要求1-5任意一种所述的多标签的类均衡方法，其特征在于：The multi-label class equalization method according to any one of claims 1-5, characterized in that:
    所述加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种。The weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method.
  7. 如权利要求1-6任意一种所述的多标签的类均衡方法，其特征在于，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重包括：The multi-label class balancing method according to any one of claims 1-6, wherein the method is based on the number of target class labels in each instance in the subclass and the number of all class labels in each instance , determining the target sampling weight for each instance in each subclass includes:
    根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比；According to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determine the proportion of target class labels for each instance in each subclass;
    根据每个小类中每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重。The target sampling weight of each instance in each subclass is determined according to the proportion of the target class label of each instance in each subclass.
  8. 如权利要求7所述的多标签的类均衡方法，其特征在于，所述根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比包括：The multi-label class balancing method according to claim 7, wherein, according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, determining each subclass The proportion of target class labels for each instance in the class includes:
    根据公式
    Figure PCTCN2021132897-appb-100001
    确定每个小类中每个实例的目标类别标签的占比，其中，p ti为小类t中的实例i的目标类别标签的占比，k ti为小类t中的实例i的目标类别标签的数量，m ti为小类t中的实例i中的所有类别标签的数量。     According to the formula
    Figure PCTCN2021132897-appb-100001
    Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
  9. 如权利要求7所述的多标签的类均衡方法，其特征在于，所述根据每个小类每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重包括：The multi-label class balancing method according to claim 7, wherein the determining the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance of each subclass comprises:
    根据公式
    Figure PCTCN2021132897-appb-100002
    确定每个小类中每个实例的目标采样权重，其中，w ti为小类t中的实例i的目标采样权重，p ti为小类t中的实例i的目标类别标签的占比，tn为小类t中的所有实例的总数量，
    Figure PCTCN2021132897-appb-100003
    为小类t中的所有实例的目标类别标签的占比之和。     According to the formula
    Figure PCTCN2021132897-appb-100002
    Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t,
    Figure PCTCN2021132897-appb-100003
    is the sum of the proportions of the target class labels of all instances in the small class t.
  10. 一种多标签的类均衡装置，其特征在于，所述装置包括：A multi-label equalizer-like device, characterized in that the device comprises:
    获取单元，所述获取单元用于获取包括多个实例的样本集，每个实例包括一个或多个类别标签；an acquisition unit, the acquisition unit is configured to acquire a sample set including multiple instances, each instance including one or more class labels;
    采样单元，所述采样单元用于根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，每个实例被分至一个或多个类，每个类对应一类别；a sampling unit, the sampling unit is configured to classify the instances in the sample set according to the categories to form a plurality of categories according to the category labels, each instance is divided into one or more categories, and each category corresponds to a category ;
    所述采样单元用于根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标采样权重，其中所述小类为多个所述类中数量小于第一阈值的类，所述目标类别标签为每个实例中所述小类对应的类别标签，所述目标采样权重为每个实例中对应所述小类的采样权重；The sampling unit is configured to determine the target sampling weight of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance, wherein the small class The class is a class whose quantity is less than the first threshold in the plurality of classes, the target class label is the class label corresponding to the subclass in each instance, and the target sampling weight is the class corresponding to the subclass in each instance The sampling weight of ;
    所述采样单元还用于根据所述小类中的每个实例的目标采样权重，对每个小类的实例进行加权采样。The sampling unit is further configured to perform weighted sampling on the instances of each subclass according to the target sampling weight of each instance in the subclass.
  11. 如权利要求10所述的多标签的类均衡装置，其特征在于：The multi-label quasi-equilibrium device according to claim 10, wherein:
    所述采样单元还用于将根据加权采样结果确定的目标实例作为新增样本构造所述小类。The sampling unit is further configured to construct the subclass by using the target instance determined according to the weighted sampling result as a newly added sample.
  12. 如权利要求11所述的多标签的类均衡装置，其特征在于：The multi-label equalizer-like device according to claim 11, wherein:
    所述采样单元还用于构造所述小类中的实例至第一数量；The sampling unit is further configured to construct instances in the subclass to a first number;
    所述采样单元还用于判断每个小类中的实例的数量是否达到第二数量；The sampling unit is also used for judging whether the number of instances in each subclass reaches the second number;
    所述采样单元还用于若每个小类中的实例的数量没有达到第二数量，继续根据所述类别标签将所述样本集中的所述实例按照类别进行分类形成多个类，继续确定每个小类中每个实例的目标采样权重，及继续对每个小类的实例进行加权采样直至每个小类中的实例的数量达到第二数量。The sampling unit is further configured to, if the number of instances in each subclass does not reach the second number, continue to classify the instances in the sample set according to the classes to form multiple classes according to the class labels, and continue to determine each class. target sampling weights for each instance in each subclass, and continue weighted sampling of instances in each subclass until the number of instances in each subclass reaches the second number.
  13. 如权利要求12所述的多标签的类均衡装置，其特征在于，在所述构造所述小类中的实例至第一数量之前：The multi-label class equalization device according to claim 12, wherein, before the constructing the instances in the subclass to the first number:
    所述采样单元还用于确定所述第一数量为所有类的当前实例的数量的平均值。The sampling unit is further configured to determine that the first number is an average value of the numbers of current instances of all classes.
  14. 如权利要求12所述的多标签的类均衡装置，其特征在于，在所述判断每个小类中的实例的数量是否达到第二数量之前：The multi-label class equalization device according to claim 12, characterized in that, before said judging whether the number of instances in each subclass reaches the second number:
    所述采样单元还用于确定所述第二数量为r*N max，其中r∈[0.5,0.8]，N max为最大类的实例的数量。 The sampling unit is further configured to determine the second number as r*N max , where r∈[0.5, 0.8], and N max is the number of instances of the largest class.
  15. 如权利要求10-14任意一种所述的多标签的类均衡装置，其特征在于：The multi-label quasi-equilibrium device according to any one of claims 10-14, characterized in that:
    所述加权采样包括加权随机采样法WRS及朴素加权采样法中的至少一种。The weighted sampling includes at least one of the weighted random sampling method WRS and the naive weighted sampling method.
  16. 如权利要求10-15任意一种所述的多标签的类均衡装置，其特征在于：The multi-label quasi-equilibrium device according to any one of claims 10-15, characterized in that:
    所述采样单元用于根据小类中每个实例中的目标类别标签的数量及每个实例中的所有类别标签的数量，确定每个小类中每个实例的目标类别标签的占比；The sampling unit is used to determine the proportion of the target class label of each instance in each subclass according to the number of target class labels in each instance in the subclass and the number of all class labels in each instance;
    所述采样单元还用于根据每个小类每个实例的目标类别标签的占比确定每个小类中每个实例的目标采样权重。The sampling unit is further configured to determine the target sampling weight of each instance in each subclass according to the proportion of the target class label of each instance in each subclass.
  17. 如权利要求16所述的多标签的类均衡装置，其特征在于：The multi-label equalizer-like device according to claim 16, wherein:
    所述采样单元还用于根据公式
    Figure PCTCN2021132897-appb-100004
    确定每个小类中每个实例的目标类别标签的占比，其中，p ti为小类t中的实例i的目标类别标签的占比，k ti为小类t中的实例i的目标类别标签的数量，m ti为小类t中的实例i中的所有类别标签的数量。     The sampling unit is also used according to the formula
    Figure PCTCN2021132897-appb-100004
    Determine the proportion of target class labels of each instance in each subclass, where p ti is the proportion of target class labels of instance i in subclass t, and k ti is the target class of instance i in subclass t The number of labels, m ti is the number of all class labels in instance i in subclass t.
  18. 如权利要求16所述的多标签的类均衡装置，其特征在于：The multi-label equalizer-like device according to claim 16, wherein:
    所述采样单元还用于根据公式
    Figure PCTCN2021132897-appb-100005
    确定每个小类中每个实例的目标采样权重，其中，w ti为小类t中的实例i的目标采样权重，p ti为小类t中的实例i的目标类别标签的占比，tn为小类t中的所有实例的总数量，
    Figure PCTCN2021132897-appb-100006
    为小类t中的所有实例的目标类别标签的占比之和。     The sampling unit is also used according to the formula
    Figure PCTCN2021132897-appb-100005
    Determine the target sampling weight of each instance in each subclass, where w ti is the target sampling weight of instance i in subclass t, pti is the proportion of target class labels of instance i in subclass t, tn is the total number of all instances in subclass t,
    Figure PCTCN2021132897-appb-100006
    is the sum of the proportions of the target class labels of all instances in the small class t.
  19. 一种电子设备，其特征在于，所述电子设备包括处理器和存储器，所述存储器用于存储程序指令，所述处理器调用所述程序指令时，实现如权利要求1至9中任一项所述的多标签的类均衡方法。An electronic device, characterized in that the electronic device includes a processor and a memory, the memory is used to store program instructions, and when the processor invokes the program instructions, any one of claims 1 to 9 is implemented. The described multi-label class equalization method.
  20. 一种车辆，其特征在于，所述车辆包括如权利要求19所述的电子设备。A vehicle, characterized in that the vehicle includes the electronic device of claim 19 .
  21. 一种计算机可读存储介质，其特征在于，该计算机可读存储介质存储有程序，所述程序使得计算机设备实现如权利要求1至9中任一项所述的多标签的类均衡方法。A computer-readable storage medium, characterized in that the computer-readable storage medium stores a program, the program enables a computer device to implement the multi-label equalization-like method according to any one of claims 1 to 9.
  22. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机执行指令，所述计算机执行指令存储在计算机可读存储介质中；设备的至少一个处理器可以从所述计算机可读存储介质中读取所述计算机执行指令，所述至少一个处理器执行所述计算机执行指令使得所述设备执行如权利要求1至9中任一项所述的多标签的类均衡方法。A computer program product, characterized in that the computer program product comprises computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium; at least one processor of the device can be downloaded from the computer-readable storage medium Reading the computer-executable instructions, the at least one processor executing the computer-executable instructions causes the apparatus to perform the multi-tag equalization-like method of any one of claims 1-9.
PCT/CN2021/132897 2021-02-05 2021-11-24 Multi-label class equalization method and device WO2022166325A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110164841.6 2021-02-05
CN202110164841.6A CN114881096A (en) 2021-02-05 2021-02-05 Multi-label class balancing method and device

Publications (1)

Publication Number Publication Date
WO2022166325A1 true WO2022166325A1 (en) 2022-08-11

Family

ID=82666992

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/132897 WO2022166325A1 (en) 2021-02-05 2021-11-24 Multi-label class equalization method and device

Country Status (2)

Country Link
CN (1) CN114881096A (en)
WO (1) WO2022166325A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116644296B (en) * 2023-07-27 2023-10-03 北京斯年智驾科技有限公司 Data enhancement method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170270429A1 (en) * 2016-03-21 2017-09-21 Xerox Corporation Methods and systems for improved machine learning using supervised classification of imbalanced datasets with overlap
CN110717529A (en) * 2019-09-25 2020-01-21 南京旷云科技有限公司 Data sampling method and device
CN111079811A (en) * 2019-12-06 2020-04-28 西安电子科技大学 Sampling method for multi-label classified data imbalance problem
CN111240279A (en) * 2019-12-26 2020-06-05 浙江大学 Confrontation enhancement fault classification method for industrial unbalanced data
CN111652268A (en) * 2020-04-22 2020-09-11 浙江盈狐云数据科技有限公司 Unbalanced stream data classification method based on resampling mechanism

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170270429A1 (en) * 2016-03-21 2017-09-21 Xerox Corporation Methods and systems for improved machine learning using supervised classification of imbalanced datasets with overlap
CN110717529A (en) * 2019-09-25 2020-01-21 南京旷云科技有限公司 Data sampling method and device
CN111079811A (en) * 2019-12-06 2020-04-28 西安电子科技大学 Sampling method for multi-label classified data imbalance problem
CN111240279A (en) * 2019-12-26 2020-06-05 浙江大学 Confrontation enhancement fault classification method for industrial unbalanced data
CN111652268A (en) * 2020-04-22 2020-09-11 浙江盈狐云数据科技有限公司 Unbalanced stream data classification method based on resampling mechanism

Also Published As

Publication number Publication date
CN114881096A (en) 2022-08-09

Similar Documents

Publication Publication Date Title
Lin et al. A Real‐Time Vehicle Counting, Speed Estimation, and Classification System Based on Virtual Detection Zone and YOLO
CN107545263B (en) Object detection method and device
Li et al. Robust vehicle detection in high-resolution aerial images with imbalanced data
Fan et al. Real-time and accurate abnormal behavior detection in videos
Cao et al. Pedestrian detection algorithm for intelligent vehicles in complex scenarios
Wickramarachchi et al. Knowledge-infused learning for entity prediction in driving scenes
Zhang et al. Adaptive dense pyramid network for object detection in UAV imagery
Mane et al. Traffic density classification for multiclass vehicles using customized convolutional neural network for smart city
WO2022166325A1 (en) Multi-label class equalization method and device
CN115375781A (en) Data processing method and device
Bouderbal et al. How would image down-sampling and compression impact object detection in the context of self-driving vehicles?
CN115457288A (en) Multi-target tracking method and device based on aerial view angle, storage medium and equipment
Cervera-Uribe et al. U19-Net: a deep learning approach for obstacle detection in self-driving cars
Zhang et al. CE-RetinaNet: A channel enhancement method for infrared wildlife detection in UAV images
Xu et al. Tiny FCOS: A lightweight anchor-free object detection algorithm for mobile scenarios
Alqarqaz et al. An Object Classification Approach for Autonomous Vehicles Using Machine Learning Techniques
Chowdhury et al. Vehicle detection and classification using deep neural networks
Jiang et al. A multiobject detection scheme based on deep learning for infrared images
US20230035995A1 (en) Method, apparatus and storage medium for object attribute classification model training
WO2021174513A1 (en) Image processing system and method, and autonomous vehicle comprising said system
Schennings Deep convolutional neural networks for real-time single frame monocular depth estimation
Wu et al. Forward collision warning system using multi-modal trajectory prediction of the intelligent vehicle
Grannas Real-time uncertainty estimation for semantic segmentation: Improving uncertainty estimates with temperature scaling and predicted dirichlet distributions
Bello et al. Motorcycling-Net: A Segmentation Approach for Detecting Motorcycling Near Misses
Truong et al. Rotated Mask Region-Based Convolutional Neural Network Detection for Parking Space Management System

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21924320

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21924320

Country of ref document: EP

Kind code of ref document: A1