CN109919300B - Neural network training method and device and image processing method and device - Google Patents
Neural network training method and device and image processing method and device Download PDFInfo
- Publication number
- CN109919300B CN109919300B CN201910138574.8A CN201910138574A CN109919300B CN 109919300 B CN109919300 B CN 109919300B CN 201910138574 A CN201910138574 A CN 201910138574A CN 109919300 B CN109919300 B CN 109919300B
- Authority
- CN
- China
- Prior art keywords
- neural network
- training
- processing
- states
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The disclosure relates to a neural network training method and device and an image processing method and device, wherein the method comprises the following steps: respectively inputting the sample images in the training set into first neural networks in N intermediate training states for processing to obtain first processing results in the N intermediate training states; and training a second neural network according to the training set and the first processing results of the N intermediate training states. The embodiment of the disclosure can enable the learning process of the second neural network to be constrained by the optimized path of the first neural network, thereby obtaining the second neural network with higher performance.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a neural network training method and apparatus, and an image processing method and apparatus.
Background
In the field of artificial intelligence technology, deep neural networks achieve very good performance in many tasks (e.g., vision tasks). Generally, the larger the number of parameters and computations for the network, the better the performance. Deploying such large network models on resource-constrained systems (e.g., terminals) is difficult, and only smaller-scale networks can be deployed. However, training a small network directly yields a model with much lower performance than a large network. How to improve the performance of a small network without increasing the sample image is an urgent problem to be solved.
Disclosure of Invention
The present disclosure provides a neural network training technical solution.
According to an aspect of the present disclosure, there is provided a neural network training method, including:
respectively inputting the sample images in the training set into first neural networks in N intermediate training states for processing to obtain first processing results in the N intermediate training states;
training a second neural network according to the training set and the first processing results of the N intermediate training states,
the second neural network is used for processing the image to be processed, and the first neural network is used for training the second neural network.
In one possible implementation, training a second neural network according to the training set and the first processing result of the N intermediate training states includes:
inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
determining a first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and training the second neural network according to the first loss, and determining the trained second neural network as the second neural network in the (i + 1) th state.
In one possible implementation, the first neural network includes M training states, M being an integer and M > N, the method further includes:
a first neural network of N intermediate training states is determined from the first neural networks of M training states.
In one possible implementation, determining a first neural network of N intermediate training states from among the first neural networks of M training states includes:
and sampling the first neural networks in the M training states at equal state quantity intervals to obtain the first neural networks in the N intermediate training states.
In one possible implementation, determining a first neural network of N intermediate training states from among the first neural networks of M training states includes:
a first neural network of N intermediate training states is randomly selected from the first neural networks of M training states.
In one possible implementation, the method further includes: training the first neural network according to the training set,
wherein training the first neural network according to the training set comprises:
inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
determining a second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
In one possible implementation, the image to be processed and the sample image include a face image, and the second neural network is used for identifying a class of the face image.
According to an aspect of the present disclosure, there is provided an image processing method including:
inputting the image to be processed into a neural network for processing to obtain an image processing result,
wherein the neural network comprises a second neural network trained according to any one of the methods described above.
According to an aspect of the present disclosure, there is provided a neural network training apparatus including:
the processing module is used for respectively inputting the sample images in the training set into the first neural networks in the N intermediate training states for processing to obtain first processing results in the N intermediate training states;
a first training module for training a second neural network according to the training set and first processing results of the N intermediate training states,
the second neural network is used for processing the image to be processed, and the first neural network is used for training the second neural network.
In one possible implementation, the first training module includes:
the first processing submodule is used for inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
the first loss determining submodule is used for determining first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and the training sub-module is used for training the second neural network according to the first loss and determining the trained second neural network as the second neural network in the (i + 1) th state.
In one possible implementation, the first neural network includes M training states, M being an integer and M > N, the apparatus further includes:
and the determining module is used for determining the first neural network in the N intermediate training states from the first neural networks in the M training states.
In one possible implementation manner, the determining module includes:
and the first determining submodule is used for sampling the first neural network in the M training states at equal state quantity intervals to obtain the first neural network in the N intermediate training states.
In one possible implementation manner, the determining module includes:
and the second determining submodule is used for randomly selecting the first neural network in the N intermediate training states from the first neural networks in the M training states.
In one possible implementation, the apparatus further includes: a second training module for training the first neural network according to the training set,
wherein the second training module comprises:
the second processing submodule is used for inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
the second loss determining submodule is used for determining second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and the network adjusting submodule is used for adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
In one possible implementation, the image to be processed and the sample image include a face image, and the second neural network is used for identifying a class of the face image.
According to an aspect of the present disclosure, there is provided an image processing apparatus including:
the image processing module is used for inputting the image to be processed into the neural network for processing to obtain an image processing result,
wherein the neural network comprises a second neural network trained according to any one of the apparatus described above.
According to an aspect of the present disclosure, there is provided an electronic device including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to: performing the method of any of the above.
According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method of any one of the above.
In the embodiment of the disclosure, the processing results of the first neural network (teacher network) in the intermediate training state on the sample images can be obtained, and the second neural network (student network) is trained according to the processing results and the sample images, so that the learning process of the second neural network is constrained by the optimized path of the first neural network, and the second neural network with higher performance is obtained.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.
Fig. 1 shows a flow diagram of a neural network training method according to an embodiment of the present disclosure.
Fig. 2 shows a schematic diagram of a neural network training process in accordance with an embodiment of the present disclosure.
FIG. 3 illustrates a block diagram of a neural network training device, in accordance with an embodiment of the present disclosure.
Fig. 4 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.
Fig. 5 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
Fig. 1 shows a flow diagram of a neural network training method according to an embodiment of the present disclosure. As shown in fig. 1, the method includes:
step S11, respectively inputting the sample images in the training set into N first neural networks in intermediate training states for processing to obtain N first processing results;
step S12, training a second neural network according to the training set and the N first processing results,
the second neural network is used for processing the image to be processed, and the first neural network is used for training the second neural network.
According to the neural network training method disclosed by the embodiment of the disclosure, the processing results of the first neural network (teacher network) in the intermediate training state on the sample images can be obtained, and the second neural network (student network) is trained according to the processing results and the sample images, so that the learning process of the second neural network is constrained by the optimized path of the first neural network, and the second neural network with higher performance is obtained.
In one possible implementation, the neural network training method may be performed by an electronic device such as a terminal device or a server, the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, and the method may be implemented by a processor calling a computer readable instruction stored in a memory. Alternatively, the method may be performed by a server.
In one possible implementation, the second neural network is a student network for processing the image to be processed, and the first neural network is a teacher network for training the second neural network. The network structures and parameter quantities of the first neural network and the second neural network are different, for example, the first neural network can be applied to a server side, and the network structures are complex and the parameter quantities are large; the second neural network can be applied to the mobile terminal, and is simple in network structure and small in parameter quantity. The performance of the trained student network is similar to that of the teacher network.
In one possible implementation, a training set may be preset, and the training set may include a large number of sample images conforming to a mathematical distribution for training the first neural network and the second neural network.
In one possible implementation, the image to be processed and the sample images in the training set may be, for example, face images, and the second neural network is, for example, used to identify the class of the face images. It should be understood that the image to be processed and the sample image may be other types of images, and the second neural network may be applied to other types of usage scenarios, which are not limited by the present disclosure.
In one possible implementation, before training the second neural network, the neural network training method may further include: training the first neural network according to the training set.
For example, a training set X × Y may be given, where X ═ X (X)1、x2、……、xn) Representing a set of sample images; y ═ Y1、y2、……、yn) Annotation information (e.g., labels) indicating respective sample images, and n indicates the number of sample images. From this training set, an optimization algorithm (e.g., SGD algorithm or Adam algorithm, etc.) may be employed to train the first neural network T with a larger number of parameters. The present disclosure is not limited to the particular optimization employed to train the first neural network.
In one possible implementation, the step of training the first neural network according to the training set may include:
inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
determining a second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
For example, during the training process, the network parameters of the first neural network T may be updated through a plurality of iterations, each iteration resulting in the first neural network T in an intermediate training state.
In a possible implementation manner, assuming that the total number of rounds (MaxIters) of training iterations of the first neural network T is M (M is an integer greater than 1), for the first neural network T in the current M-th state (which is one of M training states and 1 ≦ M < M), the plurality of sample images may be respectively input into the first neural network T in the M-th state for processing, and the first processing result in the M-th state, that is, the M-th first processing result, may be output.
In a possible implementation manner, according to the labeling information of each sample image and the first processing results of the corresponding multiple mth states, the network loss (second loss) of the first neural network in the mth state may be determined according to a preset optimization algorithm (e.g., an SGD algorithm or an Adam algorithm); and reversely adjusting the network parameters of the first neural network according to the network loss to obtain the first neural network in the m +1 th state. When the first neural network meets the training condition, the trained first neural network can be obtained.
In this way, the first neural network can be trained through multiple iterations, and a high-precision teacher network is obtained.
After the first neural network (teacher network) is trained, the teacher network (large network) can be used to train the student network (small network) by knowledge distillation, for example, by minimizing the cross entropy between the outputs of the large network and the small network, so as to train the small network better and reduce the performance gap between the large network and the small network. In the related art, a trained large network may be directly used as a learning target of a small network, but the learning target is difficult and it is difficult to train a high-performance small network.
In a possible implementation manner, a plurality of teacher networks in intermediate states on a training path of the teacher network can be selected to train the student network, so that the difficulty of learning targets is gradually increased, the training difficulty of the student network is reduced, and the performance of the student network is further improved.
In one possible implementation, the method may further include: a first neural network of N intermediate training states is determined from the first neural networks of M training states.
For example, a first neural network of partial intermediate training states (N intermediate training states, N < M) may be selected from the first neural network of M iterations (M training states). And storing the first neural network in the N intermediate training states as a check point (check point model), so as to embody the optimization path of the first neural network.
In one possible implementation, the step of determining the first neural network of N intermediate training states from the first neural network of M training states may include: and sampling the first neural networks in the M training states at equal state quantity intervals to obtain the first neural networks in the N intermediate training states. That is, equal interval selection (sampling) may be performed, for example, the sampled state number interval may be set to be M/N, and the specific value of the state number interval is not limited in the present disclosure.
By the mode, the check points can be uniformly selected, so that the check points represent more uniform optimized paths, and the training difficulty of the student network is reduced.
In one possible implementation, the step of determining the first neural network of N intermediate training states from the first neural network of M training states may include: a first neural network of N intermediate training states is randomly selected from the first neural networks of M training states. That is, a random culling (sampling) may be performed to sample a first neural network of N intermediate training states from a first neural network of M training states (checkpoints).
By the mode, the check points can be randomly selected, so that the check points represent unfixed optimized paths, and the training effect of the student network is improved.
In a possible implementation manner, after the first neural network of N intermediate training states is determined, in step S11, each sample image in the training set may be respectively input into the first neural network of N intermediate training states for processing, so as to obtain first processing results of N intermediate training states; from the first processing results of the respective intermediate training states, a second neural network may be trained in step S12.
In one possible implementation, step S12 may include:
inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
determining a first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and training the second neural network according to the first loss, and determining the trained second neural network as the second neural network in the (i + 1) th state.
For example, let the second neural network S have N training states, corresponding to N intermediate training states of the first neural network T. For the second neural network S in the current i-th state (one of N training states, and i is more than or equal to 1 and less than or equal to i < N), a plurality of sample images can be respectively input into the second neural network S in the i-th state for processing, and a second processing result in the i-th state is output.
In one possible implementation, a first loss of the second neural network in the i-th state may be determined according to the labeling information (label) of the sample image, the second processing result of the i-th state, and the first processing result of the i-th intermediate training state. Wherein the first loss may for example comprise a cross-entropy loss function, as shown in equation (1):
in the formula (1), LOSSiIndicating a first loss, S, of the i-th stateiA second neural network representing an i-th state;
representing the jth training image xjInput a second neural network SiThen obtaining a second processing result; y isiRepresenting the jth training image xjThe label of (1);
representing the cross entropy loss.
In the formula (1), the first and second groups,
representing an image xjIn the second neural network SiThe last softmax layer is preceded by a feature, which is applied
Dividing by the relaxation factor tau and then performing softmax processing to obtain the processing result
Similarly, TiA first neural network representing an ith intermediate training state;
representing an image xjIn the first neural network TiThe last softmax layer is preceded by a feature, which is applied
Dividing by the relaxation factor tau and then performing softmax processing to obtain the processing result
λ represents the cross entropy loss
The weight of (c).
The value of the relaxation factor τ may be, for example, 2 to 10, and the value of the relaxation factor τ is not limited by the present disclosure.
In one possible implementation, the LOSS is based on the i-th stateiThe second neural network may be trained. For example, LOSS according to first LOSSiAdjusting the network parameters of the second neural network in a reverse gradient manner; after multiple adjustments, when the second neural network meets the preset training condition, the training process of the ith state can be completed, and the trained second neural network is determined to be the second neural network of the (i + 1) th state. The present disclosure does not limit the specific training mode of the second neural network.
Fig. 2 shows a schematic diagram of a neural network training process in accordance with an embodiment of the present disclosure. As shown in fig. 2, a block 21 represents a training process of the first neural network T, and a block 22 represents a training process of the second neural network S. T is1、T2、……、TNFirst neural network, S, representing N intermediate training states, respectively1、S2、……、SNRepresenting a second neural network of N training states, respectively.
In a possible implementation, the first neural network T of N intermediate training states may be determined first1、T2、……、TN(ii) a Inputting the training images in the training set 23 into T1In the method, a first processing result of a first intermediate training state is output, and a second neural network S is trained through a training image and the first processing result1Obtaining a second neural network S of the next state2. By analogy, a second neural network S of the Nth training state can be obtainedN。
By the method, the student network (the second neural network) can learn the teacher network with the number of the training rounds before comparison and then learn the teacher network with the number of the training rounds after comparison, so that the network parameters of the student network are gradually updated, and the accuracy of the finally obtained second neural network is improved.
According to an embodiment of the present disclosure, there is also provided an image processing method including:
inputting the image to be processed into a neural network for processing to obtain an image processing result, wherein the neural network comprises a second neural network obtained by training according to the method.
In one possible implementation, the image to be processed may be, for example, a face image, and the face image may be input into a trained student network (second neural network) for image recognition, so as to determine the class of the face image. In this way, the recognition accuracy of the image can be improved.
The student network obtained by the neural network training method can be deployed in embedded equipment (such as a terminal) and can be applied to various use scenes such as mobile phone unlocking and pedestrian recognition. The deployment mode and the use scene of the student network are not limited by the disclosure.
According to the embodiment of the disclosure, the optimized path of the large network (teacher network) can be used as the learning target of knowledge distillation, and knowledge distillation based on optimized path constraint is realized, so that the small network (student network) can learn the large network from easy to difficult in sequence, and better performance is obtained on the small network with the same computation amount. According to the embodiment of the disclosure, the performance of the small network can be effectively improved no matter whether the check points are randomly selected (teacher network in the middle training state), or the check points are uniformly selected at equal intervals, and no matter what the number of the check points is (1).
It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted.
In addition, the present disclosure also provides a neural network training device, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the neural network training methods provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions in the methods section are not repeated.
It will be understood by those skilled in the art that the order of writing of the steps in the above methods of the embodiments does not imply a strict order of execution and that the particular order of execution of the steps should be determined by their function and possibly their inherent logic.
Fig. 3 shows a block diagram of a neural network training device according to an embodiment of the present disclosure, as shown in fig. 3, the neural network training device including:
the processing module 31 is configured to input the sample images in the training set into the first neural networks in the N intermediate training states respectively for processing, so as to obtain first processing results in the N intermediate training states;
a first training module 32 for training a second neural network according to the training set and the first processing results of the N intermediate training states,
the second neural network is used for processing the image to be processed, and the first neural network is used for training the second neural network.
In one possible implementation, the first training module 32 includes:
the first processing submodule is used for inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
the first loss determining submodule is used for determining first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and the training sub-module is used for training the second neural network according to the first loss and determining the trained second neural network as the second neural network in the (i + 1) th state.
In one possible implementation, the first neural network includes M training states, M being an integer and M > N, the apparatus further includes:
and the determining module is used for determining the first neural network in the N intermediate training states from the first neural networks in the M training states.
In one possible implementation manner, the determining module includes:
and the first determining submodule is used for sampling the first neural network in the M training states at equal state quantity intervals to obtain the first neural network in the N intermediate training states.
In one possible implementation manner, the determining module includes:
and the second determining submodule is used for randomly selecting the first neural network in the N intermediate training states from the first neural networks in the M training states.
In one possible implementation, the apparatus further includes: a second training module for training the first neural network according to the training set,
wherein the second training module comprises:
the second processing submodule is used for inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
the second loss determining submodule is used for determining second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and the network adjusting submodule is used for adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
In one possible implementation, the image to be processed and the sample image include a face image, and the second neural network is used for identifying a class of the face image.
According to an embodiment of the present disclosure, there is also provided an image processing apparatus including: and the image processing module is used for inputting the image to be processed into a neural network for processing to obtain an image processing result, wherein the neural network comprises a second neural network obtained by training according to any one of the devices.
In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.
Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a non-volatile computer readable storage medium.
An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the above method.
The electronic device may be provided as a terminal, server, or other form of device.
Fig. 4 illustrates a block diagram of an electronic device 800 in accordance with an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.
Referring to fig. 4, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.
The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.
Fig. 5 illustrates a block diagram of an electronic device 1900 in accordance with an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 5, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (16)
1. A neural network training method, the method comprising:
respectively inputting the sample images in the training set into first neural networks in N intermediate training states for processing to obtain first processing results in the N intermediate training states;
training a second neural network according to the training set and the first processing results of the N intermediate training states,
wherein the second neural network is a network for processing an image to be processed, the first neural network is a network for training the second neural network,
wherein training a second neural network according to the training set and the first processing results of the N intermediate training states comprises:
inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
determining a first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and training the second neural network according to the first loss, and determining the trained second neural network as the second neural network in the (i + 1) th state.
2. The method of claim 1, wherein the first neural network comprises M training states, M being an integer and M > N, the method further comprising:
a first neural network of N intermediate training states is determined from the first neural networks of M training states.
3. The method of claim 2, wherein determining the first neural network of N intermediate training states from the first neural network of M training states comprises:
and sampling the first neural networks in the M training states at equal state quantity intervals to obtain the first neural networks in the N intermediate training states.
4. The method of claim 2, wherein determining the first neural network of N intermediate training states from the first neural network of M training states comprises:
a first neural network of N intermediate training states is randomly selected from the first neural networks of M training states.
5. The method of claim 2, further comprising: training the first neural network according to the training set,
wherein training the first neural network according to the training set comprises:
inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
determining a second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
6. The method according to any one of claims 1 to 5, wherein the image to be processed and the sample image comprise a face image, and the second neural network is used for identifying a class of the face image.
7. An image processing method, characterized in that the method comprises:
inputting the image to be processed into a neural network for processing to obtain an image processing result,
wherein the neural network comprises a second neural network trained according to the method of any one of claims 1-6.
8. A neural network training device, comprising:
the processing module is used for respectively inputting the sample images in the training set into the first neural networks in the N intermediate training states for processing to obtain first processing results in the N intermediate training states;
a first training module for training a second neural network according to the training set and first processing results of the N intermediate training states,
wherein the second neural network is a network for processing an image to be processed, the first neural network is a network for training the second neural network,
wherein the first training module comprises:
the first processing submodule is used for inputting the sample images in the training set into a second neural network in an ith state for processing to obtain a second processing result in the ith state, wherein the ith state is one of N preset training states, and i is more than or equal to 1 and is less than N;
the first loss determining submodule is used for determining first loss of the second neural network in the ith state according to the labeling information of the sample image, the second processing result in the ith state and the first processing result in the ith intermediate training state;
and the training sub-module is used for training the second neural network according to the first loss and determining the trained second neural network as the second neural network in the (i + 1) th state.
9. The apparatus of claim 8, wherein the first neural network comprises M training states, M being an integer and M > N, the apparatus further comprising:
and the determining module is used for determining the first neural network in the N intermediate training states from the first neural networks in the M training states.
10. The apparatus of claim 9, wherein the determining module comprises:
and the first determining submodule is used for sampling the first neural network in the M training states at equal state quantity intervals to obtain the first neural network in the N intermediate training states.
11. The apparatus of claim 9, wherein the determining module comprises:
and the second determining submodule is used for randomly selecting the first neural network in the N intermediate training states from the first neural networks in the M training states.
12. The apparatus of claim 9, further comprising: a second training module for training the first neural network according to the training set,
wherein the second training module comprises:
the second processing submodule is used for inputting the sample images in the training set into a first neural network in an M-th state for processing to obtain an M-th first processing result, wherein the M-th state is one of the M training states, and M is more than or equal to 1 and less than M;
the second loss determining submodule is used for determining second loss of the first neural network according to the labeling information of the sample image and the mth first processing result;
and the network adjusting submodule is used for adjusting the network parameters of the first neural network according to the second loss to obtain the first neural network in the m +1 th state.
13. The apparatus according to any one of claims 9-12, wherein the image to be processed and the sample image comprise a face image, and the second neural network is configured to identify a class of the face image.
14. An image processing apparatus, characterized in that the apparatus comprises:
the image processing module is used for inputting the image to be processed into the neural network for processing to obtain an image processing result,
wherein the neural network comprises a second neural network trained by the apparatus of any one of claims 8-13.
15. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to: performing the method of any one of claims 1 to 7.
16. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138574.8A CN109919300B (en) | 2019-02-25 | 2019-02-25 | Neural network training method and device and image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138574.8A CN109919300B (en) | 2019-02-25 | 2019-02-25 | Neural network training method and device and image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109919300A CN109919300A (en) | 2019-06-21 |
| CN109919300B true CN109919300B (en) | 2021-03-05 |
Family
ID=66962246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910138574.8A Active CN109919300B (en) | 2019-02-25 | 2019-02-25 | Neural network training method and device and image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109919300B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110569709A (en) * | 2019-07-16 | 2019-12-13 | 浙江大学 | Scene analysis method based on knowledge reorganization |
| CN110472681A (en) * | 2019-08-09 | 2019-11-19 | 北京市商汤科技开发有限公司 | The neural metwork training scheme and image procossing scheme of knowledge based distillation |
| CN110826344B (en) | 2019-10-24 | 2022-03-01 | 北京小米智能科技有限公司 | Neural network model compression method, corpus translation method and apparatus thereof |
| CN112825143A (en) * | 2019-11-20 | 2021-05-21 | 北京眼神智能科技有限公司 | Deep convolutional neural network compression method, device, storage medium and equipment |
| CN111191722B (en) * | 2019-12-30 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | Method and device for training prediction model through computer |
| CN111242303B (en) * | 2020-01-14 | 2023-12-01 | 北京市商汤科技开发有限公司 | Network training method and device, and image processing method and device |
| CN111428613A (en) * | 2020-03-19 | 2020-07-17 | 北京市商汤科技开发有限公司 | Data processing method, device, equipment and storage medium |
| CN112541577A (en) * | 2020-12-16 | 2021-03-23 | 上海商汤智能科技有限公司 | Neural network generation method and device, electronic device and storage medium |
| CN113660038B (en) * | 2021-06-28 | 2022-08-02 | 华南师范大学 | Optical network routing method based on deep reinforcement learning and knowledge distillation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108898168B (en) * | 2018-06-19 | 2021-06-01 | 清华大学 | Compression method and system of convolutional neural network model for target detection |
| CN108921294A (en) * | 2018-07-11 | 2018-11-30 | 浙江大学 | A kind of gradual piece of knowledge distillating method accelerated for neural network |
-
2019
- 2019-02-25 CN CN201910138574.8A patent/CN109919300B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109919300A (en) | 2019-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109919300B (en) | Neural network training method and device and image processing method and device | |
| CN110210535B (en) | Neural network training method and device and image processing method and device | |
| CN110837761B (en) | Multi-model knowledge distillation method and device, electronic equipment and storage medium | |
| CN110647834B (en) | Human face and human hand correlation detection method and device, electronic equipment and storage medium | |
| CN110909815B (en) | Neural network training method, neural network training device, neural network processing device, neural network training device, image processing device and electronic equipment | |
| CN109766954B (en) | Target object processing method and device, electronic equipment and storage medium | |
| CN110458218B (en) | Image classification method and device and classification network training method and device | |
| CN109165738B (en) | Neural network model optimization method and device, electronic device and storage medium | |
| CN109615006B (en) | Character recognition method and device, electronic equipment and storage medium | |
| CN109543537B (en) | Re-recognition model increment training method and device, electronic equipment and storage medium | |
| CN109858614B (en) | Neural network training method and device, electronic equipment and storage medium | |
| CN111242303B (en) | Network training method and device, and image processing method and device | |
| CN111539410B (en) | Character recognition method and device, electronic equipment and storage medium | |
| CN110858924B (en) | Video background music generation method and device and storage medium | |
| CN110188865B (en) | Information processing method and device, electronic equipment and storage medium | |
| CN112001364A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN109934240B (en) | Feature updating method and device, electronic equipment and storage medium | |
| CN109685041B (en) | Image analysis method and device, electronic equipment and storage medium | |
| CN109447258B (en) | Neural network model optimization method and device, electronic device and storage medium | |
| CN109903252B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN113807498B (en) | Model expansion method and device, electronic equipment and storage medium | |
| CN112559673A (en) | Language processing model training method and device, electronic equipment and storage medium | |
| CN111988622B (en) | Video prediction method and device, electronic equipment and storage medium | |
| CN109635926B (en) | Attention feature acquisition method and device for neural network and storage medium | |
| CN112102300A (en) | Counting method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |