WO2019034129A1 - 神经网络结构的生成方法和装置、电子设备、存储介质 - Google Patents
神经网络结构的生成方法和装置、电子设备、存储介质 Download PDFInfo
- Publication number
- WO2019034129A1 WO2019034129A1 PCT/CN2018/100914 CN2018100914W WO2019034129A1 WO 2019034129 A1 WO2019034129 A1 WO 2019034129A1 CN 2018100914 W CN2018100914 W CN 2018100914W WO 2019034129 A1 WO2019034129 A1 WO 2019034129A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- network
- neural network
- sampling
- accuracy
- layer
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Definitions
- the present application relates to artificial intelligence technology, and in particular to a method and device for generating a neural network structure, an electronic device, and a storage medium.
- Neural network is an algorithmic mathematical model for distributed parallel information processing by mimicking the behavioral characteristics of animal neural networks.
- neural network belongs to the core technology in image recognition system, neural network has been widely used in security monitoring, finance, and even driverless technology. There are differences in the structure and parameters of the required neural networks for different tasks or different application scenarios.
- the embodiment of the present application provides a technical solution for generating a neural network structure.
- a first aspect of the embodiments of the present application provides a method for generating a neural network structure, including:
- the sampled neural network is trained based on the sample data, and the accuracy corresponding to the sampled neural network is obtained;
- a device for generating a neural network structure includes:
- a sampling unit configured to sample a neural network structure, to generate a network block, where the network block includes at least one network layer;
- a network building unit configured to construct a sampling neural network based on the network block
- a training unit configured to train the sampled neural network based on the sample data, and obtain an accuracy corresponding to the sampled neural network
- a network output unit configured to regenerate a new network block according to the accuracy, in response to the preset condition not satisfying the accuracy, until the sampling neural network constructed by the new network block satisfies a preset condition, and the preset is satisfied
- Conditional sampling neural networks serve as target neural networks.
- an electronic device comprising a processor comprising a generation device of a neural network structure as described above.
- an electronic device including: a memory, configured to store executable instructions;
- a processor for communicating with the memory to execute the executable instructions to perform the operations of the method of generating a neural network structure as described above.
- a computer storage medium for storing computer readable instructions that, when executed, perform the operations of the method of generating a neural network structure as described above.
- a method and device for generating a neural network structure, an electronic device, and a computer storage medium sampling a neural network structure to generate a network block; constructing a sampling neural network based on the network block; constructing a sampling nerve by sampling The network avoids the manual setting of the network structure; the sampling neural network is trained based on the sample data, and the accuracy of the sampling neural network is obtained; by training, the parameters in the sampling neural network are better in the current structure, and the corresponding sampling neural network can be obtained.
- a new network block is generated again according to the accuracy, until the sampling neural network constructed by the new network block satisfies the preset condition, and the sampling nerve satisfying the preset condition is satisfied.
- the network is used as the target neural network; based on the sampled data generated by the reinforcement learning, the accuracy of the task and/or data corresponding to the sample data can reach the expected, reducing the cost and time of the network structure design.
- FIG. 1 is a flow chart of an embodiment of a method for generating a neural network structure according to the present application.
- FIG. 2 is a schematic structural diagram of a network block constructed based on a network layer according to an embodiment of the present application.
- FIG. 3 is a schematic structural diagram of another network block constructed based on a network layer in an embodiment of the present application.
- FIG. 4 is a schematic structural diagram of a distributed system constructed by the method of the present application.
- FIG. 5 is a schematic structural diagram of an embodiment of a neural network structure generating apparatus according to the present application.
- FIG. 6 is a schematic structural diagram of an embodiment of an electronic device according to the present application.
- Embodiments of the present application can be applied to computer systems/servers that can operate with numerous other general purpose or special purpose computing system environments or configurations.
- Examples of well-known computing systems, environments, and/or configurations suitable for use with computer systems/servers include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, based on Microprocessor systems, set-top boxes, programmable consumer electronics, networked personal computers, small computer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above, and the like.
- the computer system/server can be described in the general context of computer system executable instructions (such as program modules) being executed by a computer system.
- program modules may include routines, programs, target programs, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types.
- the computer system/server can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communication network.
- program modules may be located on a local or remote computing system storage medium including storage devices.
- FIG. 1 is a flow chart of an embodiment of a method for generating a neural network structure according to the present application. As shown in FIG. 1, the method of this embodiment includes:
- Step 101 Sample a neural network structure to generate a network block, where the network block includes at least one network layer.
- the network structure includes a network layer, a network parameter corresponding to the network layer, a setting connection manner between the network layers, and the like.
- the network layer may include some or all types of network layers in the prior art, which is not limited in this application.
- the type and number of network layers constituting the network block, the network layer referred to in the present application refers to a unit for constructing a neural network that can implement a certain function separately, such as a convolution layer, a pooling layer, a full connection layer, and the like.
- the step 101 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a sampling unit 51 that is executed by the processor.
- Step 102 constructing a sampling neural network based on the network block.
- the neural network includes at least one network block; the network block is used as a network sub-structure.
- the trained sampling neural network may be preset by K network blocks, and the structure of the K network blocks may be the same. Or not the same, K is an integer greater than or equal to one. The value of K depends on the trade-off between training efficiency and accuracy in practical applications.
- the step 102 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a network building unit 52 that is executed by the processor.
- step 103 the sampled neural network is trained based on the sample data, and the accuracy of the sampled neural network is obtained.
- the step 103 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a training unit 53 that is executed by the processor.
- Step 104 Respond to the preset condition in the accuracy, and generate a new network block again according to the accuracy, until the sampling neural network constructed by the new network block satisfies the preset condition, and the sampling neural network that satisfies the preset condition is targeted. Neural Networks.
- the sample data implements a target neural network that automatically sets the structure and parameters for a specific task.
- the step 104 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a network output unit 54 that is executed by the processor.
- the neural network structure is sampled to generate a network block; the sampling neural network is constructed based on the network block; and the sampling neural network is constructed by sampling to avoid manual setting of the network structure;
- the sampling neural network is trained, and the accuracy of the sampling neural network is obtained.
- the preset condition is met, and a new network block is generated again according to the accuracy, until the sampling neural network constructed by the new network block satisfies the preset condition, and the sampling neural network satisfying the preset condition is used as the target neural network; the reinforcement learning is performed based on the sample data.
- the generated sampling neural network, the accuracy of the task and/or data corresponding to the sample data of the sampling neural network can reach the expected, reducing the cost and time of the network structure design.
- step 101 includes:
- the neural network structure is sampled based on a probability vector, and at least one network layer, at least one network layer corresponding network parameter, and at least one network layer connection manner are obtained, and at least one network layer stack forms a network. Piece.
- sampling is performed by using a probability vector corresponding to the number of network layer types, and at least one network layer may be sampled, and corresponding network parameters exist in the network layer, in the sampling process.
- the network parameters of the corresponding network layer are obtained at the same time.
- the preset connection mode is selected.
- the connection manner between the network layers may also be obtained, where the probability vector is Each probability value corresponds to a network layer.
- the probability value corresponding to a certain network layer is 0, the network layer formed by sampling does not include the network layer, and the sampling value can be determined according to the probability value in the probability vector.
- the number of each network layer is not limited to one for each network layer.
- the network layer includes any one or more of the following:
- the above network layers enumerated in this embodiment are for understanding and implementation by those skilled in the art, and are not intended to limit the present application.
- the convolution realizes the result of summation of two variables after multiplication in a certain range, convolution.
- Layer the convolutional layer can be composed of several convolutional units, and the parameters of the convolutional unit are optimized by the backpropagation algorithm; the purpose of the convolution operation is to extract different features of the input, and the first layer of convolutional layer may only extract some low-level Features such as edges, lines, and corners, and more layers of networks can iteratively extract more complex features from low-level features.
- the main function of the max pooling layer is to reduce the downsampling by maximizing the feature points in the neighborhood without damaging the recognition result; the main function of the average pooling layer is through the neighboring layer.
- the intra-domain feature points are averaged to achieve downsampling without damaging the recognition result;
- the output of the identity layer is equivalent to its input, mainly for the convenience of combining different network layers;
- the feature overlay (Elemental Add)
- the function of the different network layers received is added together;
- the main function of the concat is to connect the features of different network layers received together;
- FIG. 2 is a network layer-based construction in the embodiment of the present application. Schematic diagram of the network block structure. These include: Identity, Convolution, Elemental Add, Max Pooling, Average Pooling, and Concat.
- FIG. 1 Schematic diagram of the network block structure. These include: Identity, Convolution, Elemental Add, Max Pooling, Average Pooling, and Concat.
- FIG. 3 is a schematic structural diagram of another network block constructed based on a network layer in an embodiment of the present application. These include: Identity, Convolution, Elemental Add, Average Pooling, and Concat. As can be seen from the network block structure shown in FIG. 2 and FIG. 3, some or all types of network layers may be included in the network block, and the number of each network layer is not fixed.
- a network block is obtained by at least one network layer stack, including:
- Analyzing the obtained network layer obtaining at least one type of network layer, a quantity of at least one type of network layer, and a connection order of at least one network layer;
- the obtained network layer obtains a network block by connecting the connection relationship between the at least one network layer according to the connection order corresponding to the at least one network layer.
- an element (network layer) included therein should be determined, and determining a network layer includes determining a type and a quantity. After knowing the type and number of the network layer, it is necessary to determine a connection order and a connection relationship of the network layer.
- the obtained network layers may be stacked together to form a network block; the same type And a number of network layers, through different stacking methods can form multiple network blocks.
- step 104 includes:
- the probability vector of the neural network structure is adjusted according to the accuracy of the sampling neural network, and the neural network structure is sampled by the adjusted probability vector to generate a new network block.
- the structure of the network block is adjusted by using reinforcement learning.
- the accuracy of the neural network does not reach the set value, the neural network structure is not good enough, and a lower return is returned.
- Re-sampling constitutes a new network block, and then constructs a neural network with new network blocks until a neural network with a standard of accuracy is obtained.
- the accuracy it is determined whether the neural network meets the standard.
- the accuracy of the neural network does not reach the set value, it indicates that the structure of the neural network is not applicable to the current sample data, and at this time, the parameters in the neural network are already trained.
- the network structure needs to be adjusted at this time, and the network structure can be adjusted mainly by the network block; therefore, the structure of the network block needs to be adjusted, and the structure of the network block can be adjusted by adjusting the network in the network block.
- the number of layers or the connection relationship of the network layer in the network block is adjusted, since each of the sampled network layers has constructed at least one network block through different connection relationships (between at least one network block: network layer type and quantity) Similarly, the location of the network layer is different from the connection relationship between the network layers. Therefore, by changing the sampling probability vector, the number of network layers constituting the network block is changed, and the structure of the network block is changed.
- step 103 trains the sampling neural network based on the sample data, including:
- the sample data is processed by using a sampling neural network to obtain a processing result
- the sampled neural network is trained based on the error value until the sampled neural network satisfies the convergence condition.
- the sample data is marked with supervised information, which is a target result that the sample data is known to be processed corresponding to the sampled neural network, and the content of the supervised information determines the function of the trained neural network.
- the neural network in order to ensure that the structure of the constructed neural network is adapted to specific task data, the neural network needs to be trained by using the sample data corresponding to the task data to identify whether the neural network can accurately process the task data.
- the process of training the neural network can be implemented by using a reverse gradient algorithm commonly used in the prior art.
- the result of the training is that the parameters in the neural network are better, and the error value between the obtained supervisory information and the processing result can determine the error value.
- the accuracy and accuracy of the neural network are inversely proportional to the error value. The larger the error value, the smaller the accuracy. The worse the processing effect of the neural network on the sample data is. The smaller the error value is, the higher the accuracy is, indicating the neural network. The better the processing of the sample data is.
- the sampling neural network is trained based on the error value until the sampling neural network satisfies the convergence condition, including:
- the convergence speed in response to the loss function is greater than or equal to a preset value, and according to the obtained error value, the parameters in the sampling neural network are adjusted by the inverse gradient algorithm to obtain the adjusted sampling neural network; until the adjusted neural network corresponding loss The convergence speed of the function is less than the preset value.
- the convergence condition cannot be achieved by setting the magnitude of the error value (there may be a neural network that cannot make the error value meet the requirements no matter how many times the training is performed)
- the convergence speed of the loss function When the convergence speed of the loss function is reduced below the preset value, it means that the effect of continuing iteration on the error value is not great. End the training, output the corresponding error value, and further calculate the accuracy of the neural network for the sample data.
- the sampling neural network is trained based on the error value until the sampling neural network satisfies the convergence condition, including:
- the number of times the error value is calculated in response to the loss function is less than a preset value.
- the parameters in the sampling neural network are adjusted by the inverse gradient algorithm to obtain the adjusted sampling neural network, and the error value is calculated for the loss function.
- the adjusted neural network corresponding loss function calculates the error value the number of times is greater than or equal to the preset value.
- a preset value is set to control the number of iterations.
- the training is stopped, and the generation speed of the neural network structure is accelerated.
- the method further includes:
- the network complexity of the preset ratio and the network density of the preset ratio are reduced, and the accuracy of the corresponding sampling neural network is obtained.
- the network computing complexity and the network density correspond to the sampling neural network.
- the correction method can be implemented by formula (1):
- FLOPs is the number of floating-point operations per second (also known as the peak speed per second) is the number of floating-point operations performed per second (English: Floating-point operations per second; abbreviation The abbreviation of :FLOPs) is used to estimate computer performance, especially in the field of scientific computing using a large number of floating-point operations.
- FLOPs are used as the network computational complexity of neural networks, the theoretical estimation formula can be, but is not limited to, formulas ( 2):
- n l-1 is the number of input channels of this layer of sampling neural network
- n l is the number of output channels of this layer of sampling neural network
- s l is the size of the convolution kernel (kernel size)
- m l is this The size of the feature map of the output of the layer.
- Density is the network density, which is the number of edges in the graph (in the mathematical field) divided by the number of graphs used to measure the complexity of the graph.
- Still another embodiment of the method for generating a neural network structure of the present application is based on the above embodiments.
- Step 101 may include sampling the neural network structure to generate N network blocks, where N is an integer greater than zero.
- sampling process can be implemented using an agent and send N network blocks to the network controller.
- the step 101 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a sampling unit 51 that is executed by the processor.
- Step 102 can include constructing N sampled neural networks based on N network blocks, respectively.
- the process of constructing a sampling neural network may be implemented by a network structure controller that distributes N sampling neural networks to N training nodes (environment variables, env).
- N training nodes environment variables, env
- env represents an external command
- program file that lists all environment variables and their assignments.
- the step 102 can be performed by a processor invoking a corresponding instruction stored in a memory, or can be performed by a network building unit 52 that is executed by the processor.
- Step 103 may include separately training the N sampled neural networks based on the sample data, and obtaining N accuracy corresponding to the N sampled neural networks.
- N sampled neural networks are separately trained by N training nodes (environment variables, env), and the sample data received by these training nodes are the same or similar (based on extraction from sample data sets of the same training target).
- the step 103 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a training unit 53 that is executed by the processor.
- Step 104 may include: in response to the accuracy of the preset condition not present in the N degrees of accuracy, respectively generating a new N network block according to the N accuracys, and respectively, N samples respectively constructed by the new N network blocks. There is a neural network in the neural network that satisfies the preset condition, and the sampling neural network that satisfies the preset condition is used as the target neural network.
- the N accuracys obtained by the N training nodes are transmitted to the agent by the network controller, and the agent determines, according to the accuracy, whether the sampled neural network is the target neural network, when the sampled neural network is not When the target neural network is used, the sampling probability is corrected, a new network block is obtained, and a new sampling neural network is reconstructed.
- the step 104 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a network output unit 54 that is executed by the processor.
- FIG. 4 is a schematic structural diagram of a distributed system constructed by the method of the present application.
- constructing including an agent, a network structure controller, and at least one training node (environment variable, env, corresponding to the environment 1, environment 2, environment N in the figure, that is, in the figure
- the distributed system of each environment represents a training node
- simultaneously trains multiple different neural networks thereby improving the generation speed of the neural network structure, saving training time and improving the efficiency of neural network training.
- the accuracy of a neural network is evaluated by the accuracy of the corresponding neural network.
- the accuracy reaches the set condition, the neural network can be considered to meet the requirements, and the task data corresponding to the sample data can be processed and obtained better.
- the result of course, the higher the accuracy, the more accurate the results of the neural network for task data processing.
- the preset condition may be: after looping X times, selecting Y sampling neural networks with higher accuracy among N "sampling neural networks" as the output target neural network, wherein X, N and Y are respectively greater than An integer equal to 1.
- the method further includes:
- a sampling neural network that satisfies a preset condition is used as a target neural network in response to the accuracy meeting the preset condition;
- the current sampled neural network is taken as the target neural network.
- the sampling neural network formed by the network block satisfies the preset condition for the first time, the sampling neural network is directly used as the target neural network; or when the sampling reaches a preset number of times, the sampling neural network to be obtained is taken as Target neural network to increase the speed of neural network structure generation.
- the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
- the foregoing steps include the steps of the foregoing method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
- FIG. 5 is a schematic structural diagram of an embodiment of a neural network structure generating apparatus according to the present application.
- the apparatus of this embodiment can be used to implement the various method embodiments described above. As shown in FIG. 5, the apparatus of this embodiment includes:
- the sampling unit 51 is configured to sample the neural network structure to generate a network block, where the network block includes at least one network layer.
- the network construction unit 52 is configured to construct a sampling neural network based on the network block.
- the training unit 53 is configured to train the sampling neural network based on the sample data, and obtain the accuracy corresponding to the sampling neural network.
- the network output unit 54 is configured to generate a new network block according to the accuracy, in response to the preset condition not satisfying the accuracy, until the sampling neural network constructed by the new network block satisfies the preset condition, and the sampling that meets the preset condition is met.
- the neural network acts as a target neural network.
- the apparatus for generating a neural network structure samples the neural network structure to generate a network block; constructs a sampling neural network based on the network block; constructing the sampling neural network by sampling avoids manually setting the network structure;
- the sample data trains the sampling neural network and obtains the accuracy of the sampling neural network. By training, the parameters in the sampling neural network reach the current structure better, and the better accuracy of the corresponding sampling neural network can be obtained; the response is accurate.
- the preset condition is not met, and a new network block is generated again according to the accuracy, until the sampling neural network constructed by the new network block satisfies the preset condition, and the sampling neural network satisfying the preset condition is used as the target neural network; based on the sample data
- the accuracy of the task and/or data corresponding to the sample data of the sampled neural network can reach the expected, reducing the cost and time of the network structure design.
- the sampling unit 51 includes:
- a probability sampling module configured to sample the neural network structure based on a probability vector, to obtain at least one network layer, a network parameter corresponding to the at least one network layer, and a connection manner between the at least one network layer;
- a network block generating module is configured to form a network block by at least one network layer stack.
- sampling is performed by using a probability vector corresponding to the number of network layer types, and at least one network layer may be sampled, and corresponding network parameters exist in the network layer, in the sampling process.
- the network parameters of the corresponding network layer are obtained at the same time.
- the preset connection mode is selected.
- the connection manner between the network layers may also be obtained, where the probability vector is Each probability value corresponds to a network layer.
- the probability value corresponding to a certain network layer is 0, the network layer formed by sampling does not include the network layer, and the sampling value can be determined according to the probability value in the probability vector.
- the number of each network layer is not limited to one for each network layer.
- the network layer includes any one or more of the following:
- Convolution layer maximum pooling layer, average pooling layer, characteristic layer, feature overlay layer, and connection layer.
- the network block generating module may be configured to analyze the network layer obtained by sampling, and obtain at least one type of network layer, at least one type of network layer. The number of connections, and the connection order of at least one network layer; the obtained network layer is connected to the connection order between the at least one network layer according to the connection order corresponding to the at least one network layer to obtain a network block.
- the network output unit 54 may be configured to adjust a probability vector for sampling the structure of the neural network according to the accuracy of the sampling neural network.
- the neural network structure is sampled by the adjusted probability vector to generate a new network block.
- the structure of the network block is adjusted by using reinforcement learning.
- the accuracy of the neural network does not reach the set value, the neural network structure is not good enough, and a lower return is returned.
- Re-sampling constitutes a new network block, and then constructs a neural network with new network blocks until a neural network with a standard of accuracy is obtained.
- the training unit 53 includes:
- a network processing module configured to process sample data by using a sampling neural network to obtain a processing result
- the error calculation module is configured to calculate the error value between the supervision information of the sample data annotation and the processing result by using the loss function; the sample data is marked with supervision information;
- the network training module is configured to train the sampling neural network based on the error value until the sampling neural network satisfies the convergence condition.
- the sample data is marked with supervised information, which is a target result that the sample data is known to be processed corresponding to the sampled neural network, and the content of the supervised information determines the function of the trained neural network.
- the neural network in order to ensure that the structure of the constructed neural network is adapted to specific task data, the neural network needs to be trained by using the sample data corresponding to the task data to identify whether the neural network can accurately process the task data.
- the process of training the neural network can be implemented by using a reverse gradient algorithm commonly used in the prior art.
- the result of the training is that the parameters in the neural network are better, and the error value between the obtained supervisory information and the processing result can determine the error value.
- the accuracy and accuracy of the neural network are inversely proportional to the error value. The larger the error value, the smaller the accuracy. The worse the processing effect of the neural network on the sample data is. The smaller the error value is, the higher the accuracy is, indicating the neural network. The better the processing of the sample data is.
- the network training module may be used to:
- the convergence speed in response to the loss function is greater than or equal to a preset value, and according to the obtained error value, the parameters in the sampling neural network are adjusted by the inverse gradient algorithm to obtain the adjusted sampling neural network; until the adjusted neural network corresponding loss The convergence speed of the function is less than the preset value.
- the network training module may be used to:
- the number of times the error value is calculated in response to the loss function is less than a preset value.
- the parameters in the sampling neural network are adjusted by the inverse gradient algorithm to obtain the adjusted sampling neural network, and the error value is calculated for the loss function.
- the adjusted neural network corresponding loss function calculates the error value the number of times is greater than or equal to the preset value.
- the training unit 53 may further include:
- the accuracy prediction module is configured to calculate the prediction accuracy of the sampled neural network according to the error value of the sampled neural network completed by the training;
- the accuracy adjustment module is configured to reduce the network complexity of the preset ratio and the network density of the preset ratio based on the prediction accuracy, and obtain the accuracy of the corresponding sampling neural network; the network computing complexity and the network density correspond to the sampling Neural Networks.
- Still another embodiment of the apparatus for generating a neural network structure of the present application is based on the above embodiments.
- the sampling unit 51 can be configured to sample the neural network structure to generate N kinds of network blocks, where N is an integer greater than zero;
- the network construction unit 52 can be configured to construct N sampling neural networks based on N network blocks respectively;
- the training unit 53 can be configured to separately train the N sampled neural networks based on the sample data, and obtain N accuracy corresponding to the N sampled neural networks;
- the network output unit 54 may be configured to regenerate a new N network block according to the N accuracy, respectively, in response to the accuracy of the N presets not satisfying the preset condition, until the new N network blocks are separately constructed.
- a neural network that satisfies a preset condition exists in the N sampling neural networks, and a sampling neural network that satisfies a preset condition is used as a target neural network.
- a plurality of different neural networks are simultaneously trained by constructing a distributed system including an agent, a network structure controller, and at least one training node (environment variable, env), thereby improving the nerve.
- the generation speed of the network structure saves training time and improves the efficiency of neural network training.
- the method further includes:
- a target judging unit configured to use a sampling neural network that satisfies a preset condition as a target neural network, in response to the accuracy meeting the preset condition;
- the current sampled neural network is used as the target neural network.
- an electronic device including a processor, where the processor includes any one of the embodiments of the generating device of the neural network structure of the present application.
- an electronic device includes: a memory, configured to store executable instructions;
- a processor for communicating with the memory to execute executable instructions to perform the operations of any of the various embodiments of the method for generating a neural network structure of the present application.
- a computer storage medium for storing computer readable instructions, and when the instructions are executed, performing any one of the embodiments of the method for generating a neural network structure of the present application Operation.
- the embodiment of the present application further provides an electronic device, such as a mobile terminal, a personal computer (PC), a tablet computer, a server, and the like.
- an electronic device such as a mobile terminal, a personal computer (PC), a tablet computer, a server, and the like.
- FIG. 6 there is shown a schematic structural diagram of an electronic device 600 suitable for implementing a terminal device or a server of an embodiment of the present application.
- the computer system 600 includes one or more processors and a communication unit.
- the one or more processors for example: one or more central processing units (CPUs) 601, and/or one or more image processing units (GPUs) 613, etc.
- the processors may be stored in a read-only memory ( Executable instructions in ROM) 602 or executable instructions loaded from random access memory (RAM) 603 from storage portion 608 perform various appropriate actions and processes.
- Communication portion 612 can include, but is not limited to, a network card, which can include, but is not limited to, an IB (Infiniband) network card, and the processor can communicate with read only memory 602 and/or random access memory 603 to execute executable instructions over bus 604.
- the communication unit 612 is connected to the communication unit 612 and communicates with other target devices, thereby performing operations corresponding to any of the methods provided by the embodiments of the present application, for example, sampling a neural network structure to generate a network block, where the network block includes At least one network layer; constructing a sampling neural network based on the network block; training the sampling neural network based on the sample data, and obtaining the accuracy of the sampling neural network; in response to the accuracy not meeting the preset condition, generating a new one according to the accuracy The network block, until the sampling neural network constructed by the new network block satisfies the preset condition, uses the sampling neural network that satisfies the preset condition as the target neural network.
- RAM 603 various programs and data required for the operation of the device can be stored.
- the CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
- ROM 602 is an optional module.
- the RAM 603 stores executable instructions or writes executable instructions to the ROM 602 at runtime, the executable instructions causing the processor 601 to perform operations corresponding to the above-described communication methods.
- An input/output (I/O) interface 605 is also coupled to bus 604.
- the communication unit 612 may be integrated or may be provided with a plurality of sub-modules (e.g., a plurality of IB network cards) and on the bus link.
- the following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a storage portion 608 including a hard disk or the like. And a communication portion 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the Internet.
- Driver 610 is also coupled to I/O interface 605 as needed.
- a removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage portion 608 as needed.
- FIG. 6 is only an optional implementation manner.
- the number and types of components in FIG. 6 may be selected, deleted, added, or replaced according to actual needs.
- an embodiment of the present application includes a computer program product comprising a computer program tangibly embodied on a machine readable medium, the computer program comprising program code for executing the method illustrated in the flowchart, the program code comprising Executing instructions corresponding to the method steps provided by the embodiments of the present application, for example, sampling a neural network structure, generating a network block, including at least one network layer in the network block; constructing a sampling neural network based on the network block; and sampling the neural network based on the sample data Perform training and obtain the accuracy corresponding to the sampled neural network; in response to the accuracy, the preset condition is not met, and a new network block is generated again according to the accuracy, until the sampled neural network constructed by the new network block satisfies the preset condition, A sampling neural network that satisfies a preset condition is used as a target neural network.
- the computer program can be downloaded and installed from the network via communication portion 609, and/or installed from removable media 611.
- the computer program is executed by the central processing unit (CPU) 601, the above-described functions defined in the method of the present application are performed.
- the methods and apparatus of the present application may be implemented in a number of ways.
- the methods and apparatus of the present application can be implemented in software, hardware, firmware, or any combination of software, hardware, and firmware.
- the above-described sequence of steps for the method is for illustrative purposes only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specifically stated.
- the present application can also be implemented as a program recorded in a recording medium, the programs including machine readable instructions for implementing the method according to the present application.
- the present application also covers a recording medium storing a program for executing the method according to the present application.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Feedback Control In General (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
Claims (25)
- 一种神经网络结构的生成方法,其特征在于,包括:对神经网络结构进行采样,生成网络块,所述网络块中包括至少一个网络层;基于所述网络块构建采样神经网络;基于样本数据对所述采样神经网络进行训练,并得到所述采样神经网络对应的准确度;响应于所述准确度不满足预设条件,根据所述准确度再次生成新的网络块,直至新的网络块构建的采样神经网络满足预设条件,将满足预设条件的采样神经网络作为目标神经网络。
- 根据权利要求1所述的方法,其特征在于,所述对神经网络结构进行采样,生成网络块,包括:基于一个概率向量对神经网络结构进行采样,获得至少一种网络层、所述至少一种网络层对应的网络参数和所述至少一种网络层之间的连接方式,由所述至少一种网络层堆叠获得一种网络块。
- 根据权利要求1或2所述的方法,其特征在于,所述网络层包括以下任意一种或多种:卷积层、最大池化层、平均池化层、特性层、特征叠加层和连接层。
- 根据权利要求2或3所述的方法,其特征在于,所述由所述至少一种网络层堆叠获得一种网络块,包括:分析采样获得的网络层,得到至少一种所述网络层的类型、至少一种类型的所述网络层的数量、和至少一种的所述网络层的连接顺序;将获得的所述网络层按照所述至少一种网络层对应的连接顺序,通过所述至少一种网络层之间的连接关系连接获得一种网络块。
- 根据权利要求1-4任一所述的方法,其特征在于,所述根据所述准确度再次生成新的网络块,包括:根据所述采样神经网络对应的准确度调整对神经网络结构进行采样的概率向量,通过调整后的概率向量对神经网络结构采样,生成新的一种网络块。
- 根据权利要求1-5任一所述的方法,其特征在于,所述基于样本数据对所述采样神经网络进行训练,包括:利用所述采样神经网络对所述样本数据进行处理,获得处理结果;利用损失函数,计算所述样本数据标注的监督信息与所述处理结果之间的误差值;所 述样本数据标注有监督信息;基于所述误差值对所述采样神经网络进行训练,直至所述采样神经网络满足收敛条件。
- 根据权利要求6所述的方法,其特征在于,所述基于所述误差值对所述采样神经网络进行训练,直至所述采样神经网络满足收敛条件,包括:响应于所述损失函数的收敛速度大于或等于预设值,根据获得的误差值,通过反向梯度算法调整所述采样神经网络中的参数,得到调整后的采样神经网络;直到调整后的神经网络对应的损失函数的收敛速度小于预设值。
- 根据权利要6所述的方法,其特征在于,所述基于所述误差值对所述采样神经网络进行训练,直至所述采样神经网络满足收敛条件,包括:响应于所述损失函数计算误差值的次数小于预设值,根据获得的误差值,通过反向梯度算法调整所述采样神经网络中的参数,得到调整后的采样神经网络,并对所述损失函数计算误差值的次数加一;直到调整后的神经网络对应的损失函数计算误差值的次数大于或等于预设值。
- 根据权利要6-8任一所述的方法,其特征在于,所述得到所述采样神经网络对应的准确度,包括:根据训练完成的所述采样神经网络的误差值计算得到所述采样神经网络的预测准确度;基于所述预测准确度,减掉预设比例的网络计算复杂度和预设比例的网络密度,得到对应所述采样神经网络的准确度;所述网络计算复杂度和所述网络密度对应所述采样神经网络。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述对神经网络结构进行采样,生成网络块,包括:对神经网络结构进行采样,生成N种网络块,所述N为大于零的整数;相应的,所述基于所述网络块构建采样神经网络,包括:分别基于所述N种网络块构建N个采样神经网络;相应的,所述基于样本数据对所述采样神经网络进行训练,并得到所述采样神经网络对应的准确度,包括:基于样本数据分别对所述N个采样神经网络进行训练,并得到所述N个采样神经网络对应的N个准确度;相应的,所述响应于所述准确度中不满足预设条件,根据所述准确度再次生成新的网 络块,直至新的网络块构建的采样神经网络满足预设条件,将满足预设条件的采样神经网络作为目标神经网络,包括:响应于所述N个准确度中不存在满足预设条件的准确度,分别根据所述N个准确度再次生成新的N种网络块,直至新的N种网络块分别构建的N个采样神经网络中存在满足预设条件的神经网络,将满足预设条件的采样神经网络作为目标神经网络。
- 根据权利要求1所述的方法,其特征在于,所述得到所述采样神经网络对应的准确度之后,还包括:响应于所述准确度满足预设条件,将满足预设条件的采样神经网络作为目标神经网络;或,响应于所述准确度不满足预设条件,且所述对神经网络结构进行采样已经达到预设次数,则将当前的采样神经网络作为目标神经网络。
- 一种神经网络结构的生成装置,其特征在于,包括:采样单元,用于对神经网络结构进行采样,生成网络块,所述网络块中包括至少一个网络层;网络构建单元,用于基于所述网络块构建采样神经网络;训练单元,用于基于样本数据对所述采样神经网络进行训练,并得到所述采样神经网络对应的准确度;网络输出单元,用于响应于所述准确度中不满足预设条件,根据所述准确度再次生成新的网络块,直至新的网络块构建的采样神经网络满足预设条件,将满足预设条件的采样神经网络作为目标神经网络。
- 根据权利要求12所述的装置,其特征在于,所述采样单元,包括:概率采样模块,用于基于一个概率向量对神经网络结构进行采样,获得至少一种网络层、所述至少一种网络层对应的网络参数和所述至少一种网络层之间的连接方式;网络块生成模块,用于由所述至少一种网络层堆叠获得一种网络块。
- 根据权利要求12或13所述的装置,其特征在于,所述网络层包括以下任意一种或多种:卷积层、最大池化层、平均池化层、特性层、特征叠加层和连接层。
- 根据权利要求13或14所述的装置,其特征在于,所述网络块生成模块,具体用于分析采样获得的网络层,得到至少一种所述网络层的类型、至少一种类型所述网络层的数量、和至少一种的所述网络层的连接顺序;将获得的所述网络层按照所述至少一种网络 层对应的连接顺序,通过所述至少一种网络层之间的连接关系连接获得一种网络块。
- 根据权利要求12-15任一所述的装置,其特征在于,所述网络输出单元,具体用于根据所述采样神经网络对应的准确度调整对神经网络结构进行采样的概率向量,通过调整后的概率向量对神经网络结构采样,生成新的一种网络块。
- 根据权利要求12-16任一所述的装置,其特征在于,所述训练单元,包括:网络处理模块,用于利用所述采样神经网络对所述样本数据进行处理,获得处理结果;误差计算模块,用于利用损失函数,计算所述样本数据标注的监督信息与所述处理结果之间的误差值;所述样本数据标注有监督信息;网络训练模块,用于基于所述误差值对所述采样神经网络进行训练,直至所述采样神经网络满足收敛条件。
- 根据权利要求17所述的装置,其特征在于,所述网络训练模块,具体用于:响应于所述损失函数的收敛速度大于或等于预设值,根据获得的误差值,通过反向梯度算法调整所述采样神经网络中的参数,得到调整后的采样神经网络;直到调整后的神经网络对应的损失函数的收敛速度小于预设值。
- 根据权利要17所述的装置,其特征在于,所述网络训练模块,具体用于:响应于所述损失函数计算误差值的次数小于预设值,根据获得误差值,通过反向梯度算法调整所述采样神经网络中的参数,得到调整后的采样神经网络,并对所述损失函数计算误差值的次数加一;直到调整后的神经网络对应的损失函数计算误差值的次数大于或等于预设值。
- 根据权利要17-19任一所述的装置,其特征在于,所述训练单元,还包括:准确度预测模块,用于根据训练完成的所述采样神经网络的误差值计算得到所述采样神经网络的预测准确度;准确度调整模块,用于基于所述预测准确度,减掉预设比例的网络计算复杂度和预设比例的网络密度,得到对应所述采样神经网络的准确度;所述网络计算复杂度和所述网络密度对应所述采样神经网络。
- 根据权利要求12-20任一所述的装置,其特征在于,所述采样单元,具体用于对神经网络结构进行采样,生成N种网络块,所述N为大于零的整数;所述网络构建单元,具体用于分别基于所述N种网络块构建N个采样神经网络;所述训练单元,具体用于基于样本数据分别对所述N个采样神经网络进行训练,并得 到所述N个采样神经网络对应的N个准确度;所述网络输出单元,用于响应于所述N个准确度中不存在满足预设条件的准确度,分别根据所述N个准确度再次生成新的N种网络块,直至新的N种网络块分别构建的N个采样神经网络中存在满足预设条件的神经网络,将满足预设条件的采样神经网络作为目标神经网络。
- 根据权利要求12所述的装置,其特征在于,还包括:目标判断单元,用于响应于所述准确度满足预设条件,将满足预设条件的采样神经网络作为目标神经网络;或,用于响应于所述准确度不满足预设条件,且所述对神经网络结构进行采样已经达到预设次数,则将当前的采样神经网络作为目标神经网络。
- 一种电子设备,其特征在于,包括处理器,所述处理器包括权利要求12至22任意一项所述的神经网络结构的生成装置。
- 一种电子设备,其特征在于,包括:存储器,用于存储可执行指令;以及处理器,用于与所述存储器通信以执行所述可执行指令从而完成权利要求1至11任意一项所述神经网络结构的生成方法的操作。
- 一种计算机存储介质,用于存储计算机可读取的指令,其特征在于,所述指令被执行时执行权利要求1至11任意一项所述神经网络结构的生成方法的操作。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
SG11201912129TA SG11201912129TA (en) | 2017-08-18 | 2018-08-17 | Neural network structure generation method and device, electronic equipment and storage medium |
JP2020508039A JP6811894B2 (ja) | 2017-08-18 | 2018-08-17 | ニューラルネットワーク構造の生成方法および装置、電子機器、ならびに記憶媒体 |
KR1020207005617A KR102170105B1 (ko) | 2017-08-18 | 2018-08-17 | 신경 네트워크 구조의 생성 방법 및 장치, 전자 기기, 저장 매체 |
US16/200,140 US11270190B2 (en) | 2017-08-18 | 2018-11-26 | Method and apparatus for generating target neural network structure, electronic device, and storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710718042.2A CN108229647A (zh) | 2017-08-18 | 2017-08-18 | 神经网络结构的生成方法和装置、电子设备、存储介质 |
CN201710718042.2 | 2017-08-18 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/200,140 Continuation US11270190B2 (en) | 2017-08-18 | 2018-11-26 | Method and apparatus for generating target neural network structure, electronic device, and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019034129A1 true WO2019034129A1 (zh) | 2019-02-21 |
Family
ID=62655197
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/100914 WO2019034129A1 (zh) | 2017-08-18 | 2018-08-17 | 神经网络结构的生成方法和装置、电子设备、存储介质 |
Country Status (6)
Country | Link |
---|---|
US (1) | US11270190B2 (zh) |
JP (1) | JP6811894B2 (zh) |
KR (1) | KR102170105B1 (zh) |
CN (1) | CN108229647A (zh) |
SG (1) | SG11201912129TA (zh) |
WO (1) | WO2019034129A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109948795A (zh) * | 2019-03-11 | 2019-06-28 | 驭势科技(北京)有限公司 | 一种确定网络结构精度和延时优化点的方法和装置 |
Families Citing this family (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018033137A1 (zh) * | 2016-08-19 | 2018-02-22 | 北京市商汤科技开发有限公司 | 在视频图像中展示业务对象的方法、装置和电子设备 |
WO2018176000A1 (en) | 2017-03-23 | 2018-09-27 | DeepScale, Inc. | Data synthesis for autonomous control systems |
US11409692B2 (en) | 2017-07-24 | 2022-08-09 | Tesla, Inc. | Vector computational unit |
US11157441B2 (en) | 2017-07-24 | 2021-10-26 | Tesla, Inc. | Computational array microprocessor system using non-consecutive data formatting |
US11893393B2 (en) | 2017-07-24 | 2024-02-06 | Tesla, Inc. | Computational array microprocessor system with hardware arbiter managing memory requests |
US10671349B2 (en) | 2017-07-24 | 2020-06-02 | Tesla, Inc. | Accelerated mathematical engine |
CN108229647A (zh) | 2017-08-18 | 2018-06-29 | 北京市商汤科技开发有限公司 | 神经网络结构的生成方法和装置、电子设备、存储介质 |
US11561791B2 (en) | 2018-02-01 | 2023-01-24 | Tesla, Inc. | Vector computational unit receiving data elements in parallel from a last row of a computational array |
US11215999B2 (en) | 2018-06-20 | 2022-01-04 | Tesla, Inc. | Data pipeline and deep learning system for autonomous driving |
US11361457B2 (en) | 2018-07-20 | 2022-06-14 | Tesla, Inc. | Annotation cross-labeling for autonomous control systems |
US11636333B2 (en) | 2018-07-26 | 2023-04-25 | Tesla, Inc. | Optimizing neural network structures for embedded systems |
US11562231B2 (en) | 2018-09-03 | 2023-01-24 | Tesla, Inc. | Neural networks for embedded devices |
CN113039556B (zh) | 2018-10-11 | 2022-10-21 | 特斯拉公司 | 用于使用增广数据训练机器模型的***和方法 |
US11196678B2 (en) | 2018-10-25 | 2021-12-07 | Tesla, Inc. | QOS manager for system on a chip communications |
CN111105029B (zh) * | 2018-10-29 | 2024-04-16 | 北京地平线机器人技术研发有限公司 | 神经网络的生成方法、生成装置和电子设备 |
CN109635920B (zh) * | 2018-11-12 | 2021-09-03 | 北京市商汤科技开发有限公司 | 神经网络优化方法及装置、电子设备和存储介质 |
CN109598332B (zh) * | 2018-11-14 | 2021-04-09 | 北京市商汤科技开发有限公司 | 神经网络生成方法及装置、电子设备和存储介质 |
US11816585B2 (en) | 2018-12-03 | 2023-11-14 | Tesla, Inc. | Machine learning models operating at different frequencies for autonomous vehicles |
US11537811B2 (en) | 2018-12-04 | 2022-12-27 | Tesla, Inc. | Enhanced object detection for autonomous vehicles based on field view |
CN109359727B (zh) * | 2018-12-07 | 2022-01-11 | 北京字节跳动网络技术有限公司 | 神经网络的结构确定方法、装置、设备及可读介质 |
CN111325311B (zh) * | 2018-12-14 | 2024-03-29 | 深圳云天励飞技术有限公司 | 用于图像识别的神经网络模型生成方法及相关设备 |
US11610117B2 (en) | 2018-12-27 | 2023-03-21 | Tesla, Inc. | System and method for adapting a neural network model on a hardware platform |
CN113688933B (zh) * | 2019-01-18 | 2024-05-24 | 北京市商汤科技开发有限公司 | 分类网络的训练方法及分类方法和装置、电子设备 |
US11150664B2 (en) | 2019-02-01 | 2021-10-19 | Tesla, Inc. | Predicting three-dimensional features for autonomous driving |
US10997461B2 (en) | 2019-02-01 | 2021-05-04 | Tesla, Inc. | Generating ground truth for machine learning from time series elements |
US11567514B2 (en) | 2019-02-11 | 2023-01-31 | Tesla, Inc. | Autonomous and user controlled vehicle summon to a target |
US10956755B2 (en) | 2019-02-19 | 2021-03-23 | Tesla, Inc. | Estimating object properties using visual image data |
CN110070120B (zh) * | 2019-04-11 | 2021-08-27 | 清华大学 | 基于判别采样策略的深度度量学习方法及*** |
CN110070029B (zh) * | 2019-04-17 | 2021-07-16 | 北京易达图灵科技有限公司 | 一种步态识别方法及装置 |
CN110110861B (zh) * | 2019-05-09 | 2021-11-26 | 北京市商汤科技开发有限公司 | 确定模型超参数及模型训练的方法和装置、存储介质 |
CN110147883B (zh) * | 2019-05-28 | 2022-06-03 | 航天科工***仿真科技(北京)有限公司 | 用于作战仿真的模型的训练方法、装置、设备和存储介质 |
WO2020237688A1 (zh) * | 2019-05-31 | 2020-12-03 | 深圳市大疆创新科技有限公司 | 网络结构搜索的方法及装置、计算机存储介质和计算机程序产品 |
CN110390385B (zh) * | 2019-06-28 | 2021-09-28 | 东南大学 | 一种基于bnrp的可配置并行通用卷积神经网络加速器 |
CN112446462B (zh) * | 2019-08-30 | 2024-06-18 | 华为技术有限公司 | 目标神经网络模型的生成方法和装置 |
CN110647990A (zh) * | 2019-09-18 | 2020-01-03 | 无锡信捷电气股份有限公司 | 基于灰色关联分析的深度卷积神经网络模型的裁剪方法 |
CN111105031B (zh) * | 2019-11-11 | 2023-10-17 | 北京地平线机器人技术研发有限公司 | 网络结构搜索方法和装置、存储介质、电子设备 |
CN112990461B (zh) * | 2019-12-16 | 2023-09-19 | 杭州海康威视数字技术股份有限公司 | 构建神经网络模型的方法、装置、计算机设备和存储介质 |
CN111222637B (zh) * | 2020-01-17 | 2023-11-28 | 上海商汤智能科技有限公司 | 神经网络模型部署方法及装置、电子设备和存储介质 |
CN111325343B (zh) * | 2020-02-20 | 2022-09-09 | 北京市商汤科技开发有限公司 | 神经网络的确定、目标检测、智能行驶控制方法及装置 |
CN111582474B (zh) * | 2020-04-24 | 2023-08-25 | 中科物栖(南京)科技有限公司 | 神经网络结构探测方法、结构探测模型的训练方法及装置 |
CN111783937A (zh) * | 2020-05-19 | 2020-10-16 | 华为技术有限公司 | 一种神经网络构建方法以及*** |
CN111797983A (zh) * | 2020-05-25 | 2020-10-20 | 华为技术有限公司 | 一种神经网络构建方法以及装置 |
CN113723603A (zh) * | 2020-05-26 | 2021-11-30 | 华为技术有限公司 | 一种更新参数的方法、装置及存储介质 |
CN113873539A (zh) * | 2020-06-30 | 2021-12-31 | 华为技术有限公司 | 获取神经网络的方法和装置 |
CN111985644B (zh) * | 2020-08-28 | 2024-03-08 | 北京市商汤科技开发有限公司 | 神经网络生成方法及装置、电子设备及存储介质 |
KR102455681B1 (ko) * | 2020-12-16 | 2022-10-18 | 경북대학교 산학협력단 | 긴 시퀀스 데이터 내 존재하는 관계 정보를 저장 가능한 분산 연관 메모리 네트워크 시스템 |
WO2022126448A1 (zh) * | 2020-12-16 | 2022-06-23 | 华为技术有限公司 | 一种基于演化学习的神经网络结构搜索方法和*** |
CN112818788B (zh) * | 2021-01-25 | 2022-05-03 | 电子科技大学 | 一种基于无人机集群的分布式卷积神经网络分层匹配方法 |
CN113344181B (zh) * | 2021-05-31 | 2022-10-18 | 北京市商汤科技开发有限公司 | 神经网络的结构搜索方法、装置、计算机设备及存储介质 |
CN113869496A (zh) * | 2021-09-30 | 2021-12-31 | 华为技术有限公司 | 一种神经网络的获取方法、数据处理方法以及相关设备 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101726742A (zh) * | 2009-12-01 | 2010-06-09 | 中国地质大学(北京) | 农作物污染胁迫水平遥感测量方法 |
CN106203330A (zh) * | 2016-07-08 | 2016-12-07 | 西安理工大学 | 一种基于卷积神经网络的车辆分类方法 |
CN106778902A (zh) * | 2017-01-03 | 2017-05-31 | 河北工业大学 | 基于深度卷积神经网络的奶牛个体识别方法 |
CN108229647A (zh) * | 2017-08-18 | 2018-06-29 | 北京市商汤科技开发有限公司 | 神经网络结构的生成方法和装置、电子设备、存储介质 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2880556B2 (ja) | 1990-04-25 | 1999-04-12 | 松下冷機株式会社 | 冷蔵庫 |
JPH04353963A (ja) * | 1991-05-30 | 1992-12-08 | Toshiba Corp | 神経回路網の構築装置および方法 |
JPH087483A (ja) | 1994-06-14 | 1996-01-12 | Sony Corp | ディスク再生装置 |
JPH0887483A (ja) * | 1994-09-16 | 1996-04-02 | Hitachi Ltd | ネットワーク合成方法 |
JP3315890B2 (ja) * | 1996-05-31 | 2002-08-19 | 株式会社東芝 | データ処理システム |
JP6042274B2 (ja) * | 2013-06-28 | 2016-12-14 | 株式会社デンソーアイティーラボラトリ | ニューラルネットワーク最適化方法、ニューラルネットワーク最適化装置及びプログラム |
US9679258B2 (en) | 2013-10-08 | 2017-06-13 | Google Inc. | Methods and apparatus for reinforcement learning |
CN108475345A (zh) * | 2015-11-12 | 2018-08-31 | 谷歌有限责任公司 | 生成较大神经网络 |
CN105701540B (zh) | 2016-01-11 | 2017-12-19 | 清华大学 | 一种自生成神经网络构建方法 |
KR102415506B1 (ko) * | 2016-10-26 | 2022-07-01 | 삼성전자주식회사 | 뉴럴 네트워크 간소화 방법 및 장치 |
KR101932835B1 (ko) * | 2017-02-01 | 2019-03-20 | 성균관대학교산학협력단 | 행동 결정 장치 및 방법, 컴퓨터 판독 가능한 저장 매체 |
CN109902186B (zh) | 2019-03-12 | 2021-05-11 | 北京百度网讯科技有限公司 | 用于生成神经网络的方法和装置 |
-
2017
- 2017-08-18 CN CN201710718042.2A patent/CN108229647A/zh active Pending
-
2018
- 2018-08-17 KR KR1020207005617A patent/KR102170105B1/ko active IP Right Grant
- 2018-08-17 SG SG11201912129TA patent/SG11201912129TA/en unknown
- 2018-08-17 JP JP2020508039A patent/JP6811894B2/ja active Active
- 2018-08-17 WO PCT/CN2018/100914 patent/WO2019034129A1/zh active Application Filing
- 2018-11-26 US US16/200,140 patent/US11270190B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101726742A (zh) * | 2009-12-01 | 2010-06-09 | 中国地质大学(北京) | 农作物污染胁迫水平遥感测量方法 |
CN106203330A (zh) * | 2016-07-08 | 2016-12-07 | 西安理工大学 | 一种基于卷积神经网络的车辆分类方法 |
CN106778902A (zh) * | 2017-01-03 | 2017-05-31 | 河北工业大学 | 基于深度卷积神经网络的奶牛个体识别方法 |
CN108229647A (zh) * | 2017-08-18 | 2018-06-29 | 北京市商汤科技开发有限公司 | 神经网络结构的生成方法和装置、电子设备、存储介质 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109948795A (zh) * | 2019-03-11 | 2019-06-28 | 驭势科技(北京)有限公司 | 一种确定网络结构精度和延时优化点的方法和装置 |
CN109948795B (zh) * | 2019-03-11 | 2021-12-14 | 驭势科技(北京)有限公司 | 一种确定网络结构精度和延时优化点的方法和装置 |
Also Published As
Publication number | Publication date |
---|---|
CN108229647A (zh) | 2018-06-29 |
JP2020526855A (ja) | 2020-08-31 |
KR102170105B1 (ko) | 2020-10-26 |
SG11201912129TA (en) | 2020-02-27 |
JP6811894B2 (ja) | 2021-01-13 |
US11270190B2 (en) | 2022-03-08 |
US20190095780A1 (en) | 2019-03-28 |
KR20200031163A (ko) | 2020-03-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019034129A1 (zh) | 神经网络结构的生成方法和装置、电子设备、存储介质 | |
US11301719B2 (en) | Semantic segmentation model training methods and apparatuses, electronic devices, and storage media | |
WO2019137104A1 (zh) | 基于深度学习的推荐方法和装置、电子设备、介质、程序 | |
WO2018099473A1 (zh) | 场景分析方法和***、电子设备 | |
CN108154222B (zh) | 深度神经网络训练方法和***、电子设备 | |
WO2019111118A1 (en) | Robust gradient weight compression schemes for deep learning applications | |
WO2020143225A1 (zh) | 神经网络的训练方法、训练装置和电子设备 | |
WO2019214344A1 (zh) | ***增强学习方法和装置、电子设备、计算机存储介质 | |
CN115147680B (zh) | 目标检测模型的预训练方法、装置以及设备 | |
CN114065863A (zh) | 联邦学习的方法、装置、***、电子设备及存储介质 | |
US20210166131A1 (en) | Training spectral inference neural networks using bilevel optimization | |
CN117351299A (zh) | 图像生成及模型训练方法、装置、设备和存储介质 | |
CN113344213A (zh) | 知识蒸馏方法、装置、电子设备及计算机可读存储介质 | |
WO2023197857A1 (zh) | 一种模型切分方法及其相关设备 | |
CN112784967B (zh) | 信息处理方法、装置以及电子设备 | |
WO2022160897A1 (zh) | 双目视差估计方法、模型训练方法以及相关设备 | |
CN115294396A (zh) | 骨干网络的训练方法以及图像分类方法 | |
CN113688920A (zh) | 模型训练及目标检测方法、装置、电子设备以及路侧设备 | |
CN114844889B (zh) | 视频处理模型的更新方法、装置、电子设备及存储介质 | |
CN111428741A (zh) | 网络社区的发现方法、装置、电子设备及可读存储介质 | |
CN113591906B (zh) | 图的分割方法、装置及电子设备 | |
CN114495236B (zh) | 图像分割方法、装置、设备、介质及程序产品 | |
CN115018009B (zh) | 对象描述方法、网络模型的训练方法及装置 | |
CN114239840B (zh) | 量子信道噪声系数估计方法及装置、电子设备和介质 | |
WO2023236900A1 (zh) | 一种项目推荐方法及其相关设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18846151 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2020508039 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 20207005617 Country of ref document: KR Kind code of ref document: A |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 08/09/2020) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18846151 Country of ref document: EP Kind code of ref document: A1 |