CN111857793A

CN111857793A - Network model training method, device, equipment and storage medium

Info

Publication number: CN111857793A
Application number: CN201910360267.4A
Authority: CN
Inventors: 王翔
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2020-10-30

Abstract

The application discloses a network model training method, a network model training device, a network model training equipment and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes; receiving a first configuration instruction based on a first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in a plurality of reference data enhancement modes; and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on the original training sample according to a data enhancement mode responded by the target configuration content. Therefore, after the network model to be trained is trained based on the enhanced training sample, the network model obtained by training can be better adapted to the application scene.

Description

Network model training method, device, equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for training a network model.

Background

Currently, network models such as object detection, classification, recognition, etc. are widely used. This type of network model typically requires training before use. At present, a deep learning algorithm is generally adopted, and features are automatically learned from a large number of training samples in an iterative training mode, so that the training of a model is completed.

In the deep learning algorithm, a large number of training samples are generally required, and therefore, a data enhancement mode can be generally adopted to perform data enhancement processing on the existing training samples so as to expand the existing training samples, thereby obtaining a large number of training samples, and training the network model based on the obtained large number of training samples.

However, currently, a given data enhancement method is used to perform data enhancement processing on training samples, and the given data enhancement method cannot be applied to various application scenarios, so that training effects of the training samples subjected to data enhancement processing on the network model in some application scenarios are poor.

Disclosure of Invention

The embodiment of the application provides a training method, a training device, equipment and a storage medium of a network model, and can solve the problem that a set data enhancement mode cannot be applied to various different application scenes in the related technology. The technical scheme is as follows:

In one aspect, a method for training a network model is provided, where the method includes:

displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes;

receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the multiple reference data enhancement modes;

and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on the training sample according to a data enhancement mode responded by the target configuration content.

In another aspect, an apparatus for training a network model is provided, the apparatus including:

the display module is used for displaying a first configuration interface, and the first configuration interface comprises a plurality of reference data enhancement modes;

a receiving module, configured to receive a first configuration instruction based on the first configuration interface, where the first configuration instruction is triggered after a user configures at least one reference data enhancement mode of the multiple reference data enhancement modes;

And the acquisition module is used for acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on the training sample according to a data enhancement mode responded by the target configuration content.

In another aspect, an electronic device is provided, including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the training method of any one of the network models described above.

In another aspect, a computer-readable storage medium is provided, which has instructions stored thereon, and when executed by a processor, the instructions implement any of the above network model training methods.

In another aspect, a computer program product is provided comprising instructions which, when run on a computer, cause the computer to perform any of the above described methods of training a network model.

The technical scheme provided by the embodiment of the application has the following beneficial effects:

the method comprises the steps of displaying a first configuration interface comprising multiple reference data enhancement modes, enabling a user to configure at least one data enhancement mode in the first configuration interface, correspondingly receiving a first configuration instruction triggered after user configuration, obtaining target configuration content according to the first configuration instruction, and carrying out data enhancement processing on an original training sample according to the data enhancement mode responded by the target configuration content to obtain an enhanced training sample. Because the target configuration content is configured by the user according to the actual requirements of the application scene, the obtained enhanced training sample can meet the training requirements of the network model in the application scene, and the network model obtained by training can be better adapted to the application scene.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow diagram illustrating a method for training a network model in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a method for training a network model in accordance with another exemplary embodiment;

FIG. 3 is a display diagram illustrating a configuration interface in accordance with an exemplary embodiment;

FIG. 4 is a display diagram illustrating a configuration interface in accordance with an exemplary embodiment;

FIG. 5 is a flow diagram illustrating a method for training a network model in accordance with another exemplary embodiment;

FIG. 6 is a schematic diagram illustrating an architecture of a training apparatus for a network model in accordance with an exemplary embodiment;

FIG. 7 is a schematic diagram of a network model training apparatus according to another exemplary embodiment;

FIG. 8 is a schematic diagram of a network model training apparatus according to another exemplary embodiment;

Fig. 9 shows a block diagram of an electronic device 900 according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Before describing the network model training method provided by the embodiment of the present application in detail, the application scenario and the implementation environment related to the embodiment of the present application are briefly described.

First, a brief description is given of an application scenario related to an embodiment of the present application.

In the technical fields of target detection, identification, classification and the like, most of algorithms based on traditional pattern identification and machine learning adopt artificially designed features for training, and the problems of low precision and poor generalization capability exist. Therefore, the deep learning algorithm is provided for training the network model, the principle is that characteristics are automatically learned from a large number of training samples through iterative training, and therefore training of the network model is achieved. The deep learning algorithm generally requires a large number of training samples, for example, thousands or even tens of thousands of training samples are generally required, and a large number of training samples are difficult to obtain in some application scenarios, for example, in the field of defect detection. Based on this, the common processing method is to process the existing training samples by using a data enhancement method to expand the existing training samples, so as to obtain a large number of training samples. However, when the fixed data enhancement method is used for network training in different application scenes, it may be difficult to adapt the fixed data enhancement method to some application scenes, for example, an application scene is a high-brightness scene, and the fixed data enhancement method includes randomly adjusting the sample brightness from low to high in a wide range, so that the adaptability to the application scene is limited. Therefore, the method for training the network model can support a user to configure the data enhancement mode on line according to the actual requirements of the application scene, so that the training sample processed by the configured data enhancement mode can meet the training requirements of the network model in the application scene, and the network model obtained by training has strong adaptability in the application scene. For specific implementation, refer to the following embodiments.

Next, a brief description will be given of an implementation environment related to the embodiments of the present application.

The training method of the network model provided by the embodiment of the application can be executed by electronic equipment, and the electronic equipment can be configured with a display screen so as to realize human-computer interaction through the display screen. The electronic device may be a computer device or an embedded device, and in some embodiments, the computer device may be a device such as a tablet computer, a notebook computer, a desktop computer, and a portable computer, which is not limited in this application.

After the application scenarios and the implementation environments related to the embodiments of the present application are described, the method for training the network model provided by the embodiments of the present application will be described in detail with reference to the accompanying drawings.

Fig. 1 is a flowchart illustrating a method for training a network model according to an exemplary embodiment, where the method for training the network model may include the following steps:

step 101: displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes;

step 102: and receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the multiple reference data enhancement modes.

Step 103: and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on an original training sample according to a data enhancement mode responded by the target configuration content.

In the embodiment of the application, a first configuration interface including multiple reference data enhancement modes is presented, a user can configure at least one data enhancement mode in the first configuration interface, correspondingly, a first configuration instruction triggered after the user configuration is received, target configuration content is obtained according to the first configuration instruction, and data enhancement processing is performed on an original training sample according to the data enhancement mode responded by the target configuration content to obtain an enhanced training sample. Because the target configuration content is configured by the user according to the actual requirements of the application scene, the obtained enhanced training sample can meet the training requirements of the network model in the application scene, and the network model obtained by training can be better adapted to the application scene.

In a possible implementation manner of the present application, acquiring target configuration content according to the first configuration instruction includes:

Determining a reference data enhancement mode selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction;

displaying a second configuration interface based on the determined reference data enhancement mode;

receiving a second configuration instruction in the second configuration interface, wherein the second configuration instruction is triggered after a user performs parameter configuration on the parameters of the determined reference data enhancement mode;

and determining the target configuration content based on the reference data enhancement mode after the parameter configuration operation according to the second configuration instruction.

determining two or more reference data enhancement modes selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction;

receiving a third configuration instruction in the second configuration interface, wherein the third configuration instruction is triggered after a user performs parameter configuration on the determined parameter of each reference data enhancement mode;

and determining the target configuration content based on the reference data enhancement mode after the parameter configuration operation according to the third configuration instruction.

In a possible implementation manner of the present application, after obtaining the target configuration content according to the first configuration instruction, the method further includes:

performing data enhancement processing on the original training sample based on the data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

and training the network model to be trained based on the enhanced training sample.

In a possible implementation manner of the present application, before training a network model to be trained based on the enhanced training sample, the method further includes:

obtaining a test sample;

this training is carried out to the network model of treating training based on this reinforcing training sample, includes:

performing iterative training on a network model to be trained based on the enhanced training sample;

when the iterative training times reach a training time threshold value, evaluating the currently trained network model through the test sample;

when the evaluation result does not reach the reference performance, adjusting the target configuration content;

and performing data enhancement processing on the original training sample based on a data enhancement mode responded by the adjusted target configuration content, re-determining the obtained training sample as an enhanced training sample, and returning the operation of performing iterative training on the network model to be trained based on the enhanced training sample until the evaluation result reaches the reference performance or the total number of iterative training reaches the reference training number.

In one possible implementation manner of the present application, adjusting the target configuration content includes:

displaying a third configuration interface, receiving an adjusting instruction for the target configuration content based on the third configuration interface, and adjusting the target configuration content according to the adjusting instruction;

alternatively, the first and second electrodes may be,

respectively obtaining the feature information of various features of the test sample and the enhanced training sample, correspondingly obtaining a first feature information set and a second feature information set, and adjusting the target configuration content according to the first feature information set and the second feature information set, wherein various features correspond to a reference data enhancement mode.

In a possible implementation manner of the present application, adjusting the target configuration content according to the first feature information set and the second feature information set includes:

for any feature in the features, respectively acquiring feature information of any feature from the first feature information set and the second feature information set;

according to the obtained feature information, counting a first feature range and a second feature range corresponding to any feature, wherein the first feature range is a feature range corresponding to the test sample, and the second feature range is a feature range corresponding to the enhanced training sample;

When the first characteristic range is different from the second characteristic range, if the target configuration content does not include the reference data enhancement mode corresponding to any characteristic, configuring the reference data enhancement mode corresponding to any characteristic in the target configuration content to obtain the adjusted target configuration content; and if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting the parameter of the reference data enhancement mode corresponding to any feature according to the adjustment step threshold.

In one possible implementation of the present application, the plurality of reference data enhancements include at least two of rotation, mirroring, scaling, noising, blurring, brightness adjustment, and cropping.

All the above optional technical solutions can be combined arbitrarily to form an optional embodiment of the present application, and the present application embodiment is not described in detail again.

Fig. 2 is a flowchart illustrating a method for training a network model, which may be applied to an electronic device, according to another exemplary embodiment, where the method for training the network model may include the following steps:

step 201: and displaying a first configuration interface, wherein the first configuration interface comprises a plurality of reference data enhancement modes.

In a possible implementation manner, the electronic device may present the first configuration interface when receiving an enhanced manner configuration instruction, where the enhanced manner configuration instruction may be triggered by a user, and the user may trigger through a reference operation, where the reference operation may include a click operation, a sliding operation, and the like, which is not limited in this embodiment of the present application.

For example, the electronic device may be provided with an enhanced mode configuration option, when a user wants to configure a data enhanced mode, the enhanced mode configuration option may be clicked to trigger an enhanced mode configuration instruction, and accordingly, after receiving the enhanced mode configuration instruction, the electronic device indicates that the user wants to configure a data enhanced mode, and displays the first configuration interface, where the first configuration interface displays a plurality of preset reference data enhanced modes.

That is, in this embodiment, the electronic device may provide the first configuration interface, and display the multiple reference data enhancement modes in the first configuration interface, so that the user may configure the data enhancement modes based on the multiple reference data enhancement modes according to actual requirements of different application scenarios, thereby avoiding using fixed data enhancement modes in different application scenarios.

Wherein the plurality of reference data enhancements may include at least two of rotation, mirroring, scaling, noising, blurring, brightness adjustment, and cropping.

Of course, it should be noted that, the reference data enhancement modes include rotation, mirroring, scaling, adding noise, blurring, adjusting brightness, and cropping, which are only used as examples for description herein, and in another embodiment, the reference data enhancement modes may further include other data enhancement modes, which are not limited in this embodiment of the present application.

Step 202: receiving a first configuration instruction based on the first configuration interface, wherein the first configuration instruction is triggered after a user configures at least one reference data enhancement mode in the plurality of reference data enhancement modes.

After the electronic device displays the multiple reference data enhancement modes through the first configuration interface, a user can select which reference data enhancement mode or reference data enhancement modes are required to be adopted in the first configuration interface to trigger the first configuration instruction.

For example, each of the plurality of reference data enhancement modes may correspond to an option for selection, and when a user wants to select a certain reference data enhancement mode, the option corresponding to the certain reference data enhancement mode may be selected. Further, the first configuration interface may be provided with a confirmation selection option, and when the user selects the reference data enhancement mode to be adopted, the confirmation selection option may be clicked to trigger the first configuration instruction.

Step 203: and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on the original training sample according to a data enhancement mode responded by the target configuration content.

According to different numbers of the reference data enhancement modes selected by the user, the content carried by the first configuration instruction is different, and according to different content carried by the first configuration instruction, the specific implementation mode for acquiring the target configuration content is different. As an example, the implementation of obtaining the target configuration content according to the first configuration instruction may include the following possible implementation manners:

the first implementation mode comprises the following steps: determining a reference data enhancement mode selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction, displaying a second configuration interface based on the determined reference data enhancement mode, receiving a second configuration instruction in the second configuration interface, wherein the second configuration instruction is triggered after the user performs parameter configuration on the parameter of the determined reference data enhancement mode, and determining target configuration content based on the reference data enhancement mode after the reference configuration operation according to the second configuration instruction.

In this implementation, the user has only selected one reference data enhancement mode, where the second configuration interface may be used for the user to perform parameter configuration on the selected reference data enhancement mode. In some embodiments, the electronic device may display the reference data enhancement mode in the second configuration interface, and display a reference configuration item in an area corresponding to the reference data enhancement mode, for example, as shown in fig. 3.

As an example, the parameters required to be configured for the reference data enhancement mode may include an enhancement parameter range and a probability, where the probability is used to indicate a sample proportion of the training samples processed by using the corresponding reference data enhancement mode, for example, if the probability corresponding to a certain reference data enhancement mode is 60%, 60% of the training samples in the training samples are processed by using the reference data enhancement mode in the subsequent processing.

The user may perform reference configuration on the reference data enhancement mode displayed on the second configuration interface according to actual needs, so as to trigger a second configuration instruction, which may include, as an example, the configured parameter value, and further may include indication information of the selected reference data enhancement mode. For example, if the application scenario is that an article with a fixed size needs to be detected, and the selected reference data enhancement mode is a scaling enhancement mode, the user may configure the scaling parameter of the scaling enhancement mode in a smaller range near 1 time, and set the probability to 100%, at this time, the second configuration instruction includes the scaling parameter near 1 time, and the probability is 100%, and further includes indication information of the scaling enhancement mode; if the application scenario is a defect requiring a large difference in size to be detected, the scaling parameter may be configured in a relatively large range (which may be configured according to actual requirements) on a reasonable premise, and the probability is set to 80%, at this time, the second configuration instruction includes the range in which the scaling parameter is large (i.e., the value configured according to actual requirements), and the probability is 80%.

It should be noted that, the above description is only made by taking two types of parameters corresponding to the reference data enhancement method as an example, in another embodiment, the reference data enhancement method may further include other parameters, and the embodiment of the present application does not limit this.

And the electronic equipment determines target configuration content according to the second configuration instruction, wherein the target configuration content comprises the reference data enhancement mode selected by the user and the enhancement parameter range and probability of the reference data enhancement mode.

The second implementation mode comprises the following steps: determining two or more reference data enhancement modes selected by a user from the plurality of reference data enhancement modes according to the first configuration instruction, displaying a second configuration interface based on the determined reference data enhancement modes, receiving a third configuration instruction in the second configuration interface, wherein the third configuration instruction is triggered after the user performs parameter configuration on parameters of each determined reference data enhancement mode, and determining the target configuration content based on the reference data enhancement modes after the parameter configuration operation according to the third configuration instruction.

In this implementation, the user selects two or more reference data enhancement modes, wherein the second configuration interface may be used for the user to perform parameter configuration on all the selected reference data enhancement modes. In some embodiments, the electronic device may present each selected reference data enhancement mode in the second configuration interface, and present a reference configuration item in an area corresponding to each reference data enhancement mode, for example, as shown in fig. 4.

As an example, the parameters required to be configured for each reference data enhancement mode may include the enhancement parameter ranges and probabilities described above. The user can perform reference configuration on each reference data enhancement mode displayed on the second configuration interface according to actual requirements, so that a second configuration instruction is triggered. As an example, the second configuration instruction may include the configured parameter value, and further, may further include indication information of each selected reference data enhancement mode. For example, if the application scenario is that an article with a fixed size needs to be detected, the selected reference data enhancement mode includes a scaling enhancement mode, the user may configure the scaling parameter of the scaling enhancement mode in a smaller range near 1 time, and set the probability to 100%, at this time, the second configuration instruction includes the scaling parameter near 1 time and the probability to 100%, and further includes indication information of the scaling enhancement mode; if the application scenario is a defect requiring a large difference in size to be detected, the scaling parameter may be configured in a relatively large range (which may be configured according to actual requirements) on a reasonable premise, and the probability is set to 80%, at this time, the second configuration instruction includes the range in which the scaling parameter is large (i.e., the value configured according to actual requirements), and the probability is 80%.

In addition, when the user selects two or more reference data enhancement modes, the electronic device may further perform parameter configuration on other reference data enhancement modes according to the brightness, blur, noise and other conditions of the image acquired in the application scene.

It should be noted that, the above description is only an example of first selecting at least one reference data enhancement mode from the plurality of reference data enhancement modes, and then performing parameter configuration on the selected at least one reference data enhancement mode. In another embodiment, one or some of the multiple reference data enhancement modes may be closed according to actual requirements, so as to obtain the at least one reference data enhancement mode, and then parameter configuration may be performed on the obtained at least one reference data enhancement mode.

In another embodiment, in order to enable the network model to be further adapted to the application scenario, the method provided in the embodiment of the present application may further include the following implementation steps.

Step 204: and performing data enhancement processing on the original training sample based on the data enhancement mode responded by the configuration content to obtain an enhanced training sample.

For example, when at least one of the reference data enhancement modes in the configuration content comprises a scaling and a noise adding mode, the electronic equipment scales and adds noise to the original training sample to obtain an extended enhanced training sample. Wherein the original training sample can be obtained by collection.

Step 205: and training the network model to be trained based on the enhanced training sample.

After the enhanced training sample is obtained, the enhanced training sample can be used for training the network model to be trained by adopting a deep learning algorithm.

Further, in the deep learning training process, the performance of the network model on some types of the enhanced training samples in the enhanced training samples may be weak, and for this case, the electronic device may further obtain the test sample, so as to evaluate the network model according to the test sample, and modify the data enhancement mode according to the evaluation result.

Specifically, the electronic device obtains the test sample, and at this time, the specific implementation of training the network model to be trained based on the enhanced training sample may include the following implementation steps:

2051: and performing iterative training on the network model to be trained based on the enhanced training sample.

Referring to fig. 5, the electronic device performs a certain number of iterative training on the network model to be trained based on the enhanced training samples.

2052: and when the iterative training times reach the training time threshold, evaluating the currently trained network model through the test sample.

And after the electronic equipment iteratively trains the to-be-trained network model for a certain number of times based on the enhanced training sample, the performance of the currently-trained network model is evaluated by using the test sample. Wherein, the test sample can be obtained by sample collection in advance.

The training time threshold may be set by a user according to actual needs in a user-defined manner, or may be set by the electronic device as a default, which is not limited in the embodiment of the present application.

2053: and when the evaluation result does not reach the reference performance, adjusting the target configuration content.

Wherein, the reference performance can be preset according to actual requirements. When the evaluation result does not reach the reference performance, the currently trained network model is not in line with the actual requirement, and the target configuration content can be adjusted at the moment. In one possible implementation, adjusting the target configuration content may include two implementations as follows:

In a first possible implementation manner, a third configuration interface is displayed, an adjustment instruction for the target configuration content is received based on the third configuration interface, and the target configuration content is adjusted according to the adjustment instruction.

When the electronic device determines that the evaluation result does not reach the reference performance, the third configuration interface may be presented, so that the user may reconfigure the target configuration content based on the third configuration interface to trigger an adjustment instruction. For example, a parameter of a certain reference data enhancement mode may be modified in the target configuration content, or a reference data enhancement mode may be deleted or added in the target configuration content, and further, when the reference data enhancement mode is added, a parameter configuration operation of the added reference data enhancement mode by the user may be received.

As an example, the third configuration interface may be the same as the first configuration interface, that is, when it is determined that the evaluation result does not reach the reference performance, the electronic device may re-display the first configuration interface to facilitate the user to reconfigure the reference data enhancement mode based on the first configuration interface. As another example, the third configuration interface may also be different from the first configuration interface, for example, the third configuration interface may show the currently configured target configuration content and provide a reconfiguration option, and the user may click on the reconfiguration option to trigger an adjustment instruction to readjust the target configuration content.

In a second possible implementation manner, feature information of various features of the test sample and the enhanced training sample is respectively obtained, a first feature information set and a second feature information set are correspondingly obtained, and the target configuration content is adjusted according to the first feature information set and the second feature information set, wherein various features correspond to a reference data enhancement manner.

The target configuration content may also be automatically adjusted when the electronic device determines that the evaluation result does not reach the reference performance. In implementation, feature information of various features may be obtained from the test sample to obtain a first feature information set, and feature information of the various features may be obtained from the enhanced training sample to obtain a second feature information set. The various features may include, but are not limited to, color, size, blur, brightness, noise, among others.

Then, the electronic equipment adjusts the target configuration content according to the first characteristic information set and the second characteristic information set. In one possible implementation, the process may include: for any one of the features, feature information of the any one feature is respectively acquired from the first feature information set and the second feature information set, and a first feature range and a second feature range corresponding to the any one feature are counted according to the acquired feature information, wherein the first feature range is a feature range corresponding to the test sample, and the second feature range is a feature range corresponding to the enhanced training sample. When the first characteristic range is different from the second characteristic range, if the target configuration content does not include the reference data enhancement mode corresponding to any characteristic, configuring the reference data enhancement mode corresponding to any characteristic in the target configuration content to obtain the adjusted target configuration content; and if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting the parameter of the reference data enhancement mode corresponding to any feature according to the adjustment step threshold.

As an example, according to the acquired feature information, the implementation of counting the first feature range and the second feature range corresponding to any feature may be: and counting the characteristic information corresponding to any one characteristic by a certain proportion of pixel points in the test sample, taking the minimum value in the counted characteristic information as the lower limit of a first characteristic range, and taking the maximum value in the counted characteristic information as the upper limit of the first characteristic range, thereby obtaining the first characteristic range. Similarly, for the feature information corresponding to any feature, the pixel points in the same proportion in the enhanced training sample are counted, the minimum value in the counted feature information is used as the lower limit of the second feature range, and the maximum value in the counted feature information is used as the upper limit of the second feature range, so that the second feature range is obtained.

For any feature, when the feature range corresponding to the feature information in the first feature information set is different from the feature range corresponding to the feature information in the second feature information set, it indicates that the difference between the test sample and the enhanced training sample is larger for the feature, and at this time, the reference data enhancement mode corresponding to any feature may be reconfigured to adjust the target configuration content.

For example, for the feature of brightness, when the feature range corresponding to the test sample is different from the feature range corresponding to the enhanced training sample, for example, the feature range corresponding to the test sample is [128,255], the feature range corresponding to the enhanced training sample is [128,200], which indicates that the brightness difference between the enhanced training sample and the test sample is large, at this time, the reference data enhancement mode of brightness adjustment corresponding to brightness may be reconfigured to increase the specific gravity of the class of training samples in the enhanced training sample.

As an example, it may be detected whether the configured target configuration content includes the reference data enhancement mode corresponding to the any feature, and if the configured target configuration content does not include the reference data enhancement mode corresponding to the any feature, the reference data enhancement mode corresponding to the feature may be added to the target configuration content, and the corresponding parameter may be automatically configured. If the configured target configuration content includes the reference data enhancement mode corresponding to any feature, the current configured parameter is unreasonable, so that the corresponding parameter can be adjusted according to the adjustment step threshold.

The adjustment step threshold may be set by a user according to actual needs in a self-defined manner, or may be set by the electronic device as a default, which is not limited in the embodiment of the present application. And, the adjustment step size threshold values corresponding to different parameters may be the same or different.

For example, after a certain number of iterative trainings, the performance of the current network model may be evaluated by using the test sample, and when it is determined that the processing capability of the network model for the test sample with relatively high noise in the application scenario is relatively weak, the training sample needs to be re-expanded by modifying the reference data enhancement mode, so that the proportion of the noise training sample in the obtained enhanced training sample is increased, and the robustness of the network model for the noise sample is improved.

It should be noted that, for the test sample and the enhanced training sample, when there is a difference in the feature ranges corresponding to the multiple features, the configuration of the reference data enhancement mode corresponding to one feature of the multiple features may be modified each time, and then the reference data enhancement modes corresponding to other features may be adjusted after the next training.

2054: and performing data enhancement processing on the original training sample based on the adjusted target configuration content, re-determining the obtained training sample as an enhanced training sample, and returning the operation of performing iterative training on the network model to be trained based on the enhanced training sample until the evaluation result reaches the reference performance or the total iterative training times reaches the reference training times.

That is, after the electronic device adjusts the target configuration content, the original training sample may be re-subjected to data enhancement processing according to the adjusted target configuration content, so as to re-expand the original training sample to obtain an enhanced training sample. The electronic device then continues to iteratively train the network model to be trained using the retrieved enhanced training samples, i.e., returns to step 2051. If the network model to be trained is iteratively trained for a certain number of times based on the re-determined enhanced training sample, and the evaluation result of evaluating the trained network model by using the test sample does not reach the reference performance yet, the target configuration content is continuously re-adjusted, data enhancement processing is performed on the original training sample based on the adjusted target configuration content, the obtained training sample is re-determined to be the enhanced training sample, and the step 2051 is returned. And repeating iteration in the training process until the evaluation result reaches the reference performance, finishing the training, and outputting the currently obtained network model as a final network model.

Or, when the total number of iterative training times reaches the reference training times, the performance of the network model does not reach the reference performance, and the electronic device terminates training, and at this time, the trained network model with the best performance can be used as the final network model. The reference training times can be preset according to actual requirements.

It is worth mentioning that the training samples are re-expanded in an online data modification enhancement mode to increase the proportion of certain classes of training samples until the network model can reach the reference performance, so that a deep learning model with strong adaptability to the application scene is finally obtained, and the model performance is improved. In addition, the method provided by the application is particularly suitable for the scenes operated by users rather than developers in the training process, so that the deep learning product can adapt to different application scenes, and the secondary development cost of the product is reduced.

Fig. 6 is a schematic structural diagram illustrating a training apparatus of a network model according to an exemplary embodiment, where the training apparatus of the network model may be implemented by software, hardware, or a combination of the two. The training device of the network model may include:

a display module 610, configured to display a first configuration interface, where the first configuration interface includes multiple reference data enhancement modes;

a receiving module 620, configured to receive a first configuration instruction based on the first configuration interface, where the first configuration instruction is triggered after a user configures at least one reference data enhancement manner of the multiple reference data enhancement manners;

an obtaining module 630, configured to obtain target configuration content according to the first configuration instruction, where the target configuration content is used to obtain an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on the training sample according to a data enhancement mode responded by the target configuration content.

In a possible implementation manner of the present application, the obtaining module 630 is configured to:

determining two or more reference data enhancement modes selected from the plurality of reference data enhancement modes by a user according to the first configuration instruction;

In a possible implementation manner of the present application, please refer to fig. 7, the apparatus further includes:

The processing module 640 is configured to perform data enhancement processing on an original training sample in a data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

and a training module 650 for training the network model to be trained based on the enhanced training samples.

In a possible implementation manner of the present application, please refer to fig. 8, the apparatus further includes:

a sample obtaining module 660 for obtaining a test sample;

the training module 650 is configured to:

when the iterative training times reach a training time threshold, evaluating a network model obtained by current training through the test sample;

In one possible implementation manner of the present application, the training module 650 is configured to:

alternatively, the first and second electrodes may be,

respectively obtaining feature information of various features of the test sample and the enhanced training sample, correspondingly obtaining a first feature information set and a second feature information set, and adjusting the target configuration content according to the first feature information set and the second feature information set, wherein various features correspond to a reference data enhancement mode.

for any one of the features, respectively acquiring feature information of any one feature from the first feature information set and the second feature information set;

When the first characteristic range is different from the second characteristic range, if the target configuration content does not include the reference data enhancement mode corresponding to any characteristic, configuring the reference data enhancement mode corresponding to any characteristic in the target configuration content to obtain the adjusted target configuration content;

and if the target configuration content comprises the reference data enhancement mode corresponding to any feature, adjusting the parameter of the reference data enhancement mode corresponding to any feature according to the adjustment step threshold.

In one possible implementation manner of the present application, the plurality of reference data enhancement manners include at least two of rotation, mirroring, scaling, adding noise, blurring, brightness adjustment, and cropping.

It should be noted that: in the training apparatus for a network model provided in the foregoing embodiment, when implementing the training method for a network model, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the network model training device provided in the above embodiments and the network model training method embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.

Fig. 9 shows a block diagram of an electronic device 900 according to an exemplary embodiment of the present application. The electronic device 900 may be: a tablet, laptop, or desktop computer. The electronic device 900 may also be referred to by other names such as user equipment, portable terminals, laptop terminals, desktop terminals, and the like.

In general, the electronic device 900 includes: a processor 901 and a memory 902.

Processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 901 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 901 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 902 is used to store at least one instruction for execution by processor 901 to implement a method of training a network model provided by method embodiments herein.

In some embodiments, the electronic device 900 may further optionally include: a peripheral interface 903 and at least one peripheral. The processor 901, memory 902, and peripheral interface 903 may be connected by buses or signal lines. Various peripheral devices may be connected to the peripheral interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a touch display screen 905, a camera 906, an audio circuit 907, a positioning component 908, and a power supply 909.

The peripheral interface 903 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, memory 902, and peripheral interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902 and the peripheral interface 903 may be implemented on a separate chip or circuit board, which is not limited by this embodiment.

The Radio Frequency circuit 904 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 904 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 904 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 904 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 904 may communicate with other devices via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 904 may also include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 905 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to capture touch signals on or over the surface of the display screen 905. The touch signal may be input to the processor 901 as a control signal for processing. At this point, the display 905 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 905 may be one, providing the front panel of the electronic device 900; in other embodiments, the number of the display panels 905 may be at least two, and the at least two display panels are respectively disposed on different surfaces of the electronic device 900 or are in a folding design; in still other embodiments, the display 905 may be a flexible display disposed on a curved surface or on a folded surface of the electronic device 900. Even more, the display screen 905 may be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display panel 905 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 906 is used to capture images or video. Optionally, camera assembly 906 includes a front camera and a rear camera. Generally, a front camera is disposed on a front panel of an electronic apparatus, and a rear camera is disposed on a rear surface of the electronic apparatus. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 906 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

Audio circuit 907 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 901 for processing, or inputting the electric signals to the radio frequency circuit 904 for realizing voice communication. For stereo capture or noise reduction purposes, the microphones may be multiple and located at different locations of the electronic device 900. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuit 907 may also include a headphone jack.

The positioning component 908 is used to locate a current geographic location of the electronic device 900 to implement navigation or LBS (location based Service). The positioning component 908 may be a positioning component based on the GPS (global positioning System) in the united states, the beidou System in china, or the galileo System in russia.

The power supply 909 is used to supply power to various components in the electronic device 900. The power source 909 may be alternating current, direct current, disposable or rechargeable. When the power source 909 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the electronic device 900 also includes one or more sensors 910. The one or more sensors 910 include, but are not limited to: acceleration sensor 911, gyro sensor 912, pressure sensor 913, fingerprint sensor 914, optical sensor 915, and proximity sensor 916.

The acceleration sensor 911 may detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the electronic device 900. For example, the acceleration sensor 911 may be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 901 can control the touch display 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 912 may detect a body direction and a rotation angle of the electronic device 900, and the gyro sensor 912 and the acceleration sensor 911 cooperate to acquire a 3D motion of the user on the electronic device 900. The processor 901 can implement the following functions according to the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

The pressure sensor 913 may be disposed on a side bezel of the electronic device 900 and/or underneath the touch display screen 905. When the pressure sensor 913 is disposed on the side frame of the electronic device 900, the user's holding signal of the electronic device 900 may be detected, and the processor 901 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed at a lower layer of the touch display 905, the processor 901 controls the operability control on the UI interface according to the pressure operation of the user on the touch display 905. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 914 is used for collecting a fingerprint of the user, and the processor 901 identifies the user according to the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, processor 901 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 914 may be disposed on the front, back, or side of the electronic device 900. When a physical button or vendor Logo is provided on the electronic device 900, the fingerprint sensor 914 may be integrated with the physical button or vendor Logo.

The optical sensor 915 is used to collect ambient light intensity. In one embodiment, the processor 901 may control the display brightness of the touch display 905 based on the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 905 is turned down. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 according to the ambient light intensity collected by the optical sensor 915.

The proximity sensor 916, also known as a distance sensor, is typically disposed on the front panel of the electronic device 900. The proximity sensor 916 is used to capture the distance between the user and the front of the electronic device 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front face of the electronic device 900 gradually decreases, the processor 901 controls the touch display 905 to switch from the bright screen state to the dark screen state; when the proximity sensor 916 detects that the distance between the user and the front surface of the electronic device 900 becomes gradually larger, the processor 901 controls the touch display 905 to switch from the breath screen state to the bright screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 9 does not constitute a limitation of the electronic device 900, and may include more or fewer components than those shown, or combine certain components, or employ a different arrangement of components.

Embodiments of the present application further provide a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method for training a network model provided in the foregoing embodiments.

The embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, causes the computer to execute the method for training a network model provided in the above embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a network model, the method comprising:

and acquiring target configuration content according to the first configuration instruction, wherein the target configuration content is used for acquiring an enhanced training sample, and the enhanced training sample is obtained by performing data enhancement on an original training sample according to a data enhancement mode responded by the target configuration content.

2. The method of claim 1, wherein obtaining target configuration content according to the first configuration instruction comprises:

3. The method of claim 1, wherein obtaining target configuration content according to the first configuration instruction comprises:

4. The method of claim 1, wherein after obtaining the target configuration content according to the first configuration instruction, further comprising:

performing data enhancement processing on an original training sample based on a data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

5. The method of claim 4, wherein prior to training the network model to be trained based on the enhanced training samples, further comprising:

obtaining a test sample;

the training of the network model to be trained based on the enhanced training samples comprises:

6. The method of claim 5, wherein the adjusting the target configuration content comprises:

alternatively, the first and second electrodes may be,

7. The method of claim 6, wherein said adjusting the target configuration content according to the first set of feature information and the second set of feature information comprises:

for any one of the features, respectively acquiring feature information of the any one feature from the first feature information set and the second feature information set;

8. The method of any of claims 1-7, wherein the plurality of reference data enhancements include at least two of rotation, mirroring, scaling, noising, blurring, brightness adjustment, and cropping.

9. An apparatus for training a network model, the apparatus comprising:

10. The apparatus of claim 9, wherein the acquisition module is to:

11. The apparatus of claim 9, wherein the acquisition module is to:

12. The apparatus of claim 9, wherein the apparatus further comprises:

the processing module is used for performing data enhancement processing on an original training sample in a data enhancement mode responded by the target configuration content to obtain an enhanced training sample;

and the training module is used for training the network model to be trained based on the enhanced training sample.

13. The apparatus of any of claims 9-12, wherein the plurality of reference data enhancements include at least two of rotation, mirroring, scaling, noising, blurring, brightness adjustment, and cropping.

14. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the steps of any of the methods of claims 1-8.

15. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of any of the methods of claims 1-8.