CN112116104A - Method, apparatus, medium, and electronic device for automatically integrating machine learning - Google Patents
Method, apparatus, medium, and electronic device for automatically integrating machine learning Download PDFInfo
- Publication number
- CN112116104A CN112116104A CN202010982700.0A CN202010982700A CN112116104A CN 112116104 A CN112116104 A CN 112116104A CN 202010982700 A CN202010982700 A CN 202010982700A CN 112116104 A CN112116104 A CN 112116104A
- Authority
- CN
- China
- Prior art keywords
- integration
- machine learning
- model
- parameter
- hyper
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000010354 integration Effects 0.000 claims abstract description 170
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000003860 storage Methods 0.000 claims description 23
- 238000005457 optimization Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Feedback Control In General (AREA)
- Machine Translation (AREA)
Abstract
The embodiment of the invention provides a method, a device, a medium and electronic equipment for automatically integrating machine learning, wherein the method comprises the following steps: step S1, acquiring at least two pre-configured machine learning models, a hyper-parameter search space, an integration algorithm and an integration algorithm parameter set; step S2, inputting a first hyper-parameter selected in the range of the hyper-parameter search space into each machine learning model; step S3, integrating the at least two machine learning models according to the integration algorithm and the first integration algorithm parameters to generate a first integration model; step S4, training the first integrated model and scoring the model to obtain a scoring result; and step S5, taking the step S2, the step S3 and the step S4 which are executed in sequence as a loop, and after the loop is repeatedly executed, determining a target integration model according to the grading result of each loop. The technical scheme of the embodiment of the invention can obtain an integrated model with better overall quality.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for automatically integrating machine learning, a computer readable storage medium and electronic equipment.
Background
Machine Learning (Machine Learning) is a multi-field cross subject, and relates to a plurality of subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. Machine learning specializes in studying how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to improve their performance. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and thus is also increasingly widely used in various business scenarios.
By processing and selecting the service data, processing and selecting the characteristics, selecting the model and adjusting the parameters, the optimal machine learning model can be obtained. This process involves a great deal of repetitive and empirical labor.
The automatic machine learning technology automatically generates a machine learning model through means of automatic characteristic engineering, automatic model selection, automatic parameter adjustment optimization and the like, and reduces the labor amount of algorithm personnel.
In order to improve the effect of the machine learning model, the multiple models can be integrated by methods such as ensemble learning from multiple aspects such as accuracy, precision and recall rate to obtain an integrated model with higher effect score.
Ensemble learning is a technique that uses multiple compatible learning algorithms/models to perform a single task in order to achieve better predictive performance.
The prior art typically optimizes individual models over-parameters by automatic machine learning. The ensemble learning is generally performed after the basic model is optimized, and parameters of the ensemble learning also need to be adjusted and optimized. When the processes of carrying out hyper-parameter optimization on a single model and carrying out parameter optimization on ensemble learning are carried out in series according to the sequence, the optimization of adjusting and optimizing by stages is actually carried out, and although the obtained final model is adjusted and optimized in each stage, the final integral model is not necessarily the optimal model.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.
Disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, a computer-readable storage medium, and an electronic device for automatically integrating machine learning, so as to obtain an integrated model with overall superiority to at least a certain extent.
Additional features and advantages of the invention will be set forth in the detailed description which follows, or may be learned by practice of the invention.
According to a first aspect of embodiments of the present invention, there is provided a method of automatically integrating machine learning, the method comprising: step S1, acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, wherein the integration algorithm parameter group comprises at least one integration algorithm parameter; step S2, inputting a first hyper-parameter selected in the range of the hyper-parameter search space into each machine learning model; step S3, integrating the at least two machine learning models according to the integration algorithm and a first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model; step S4, training the first integrated model and scoring the model to obtain a scoring result; and step S5, taking the step S2, the step S3 and the step S4 which are executed in sequence as a loop, and after N loops are repeatedly executed, determining a target integration model according to the grading result of each loop, wherein N is a positive integer.
In some embodiments, the step S2 includes: integrating the at least two machine learning models according to any one of the following integration algorithms: stacking integration algorithms, bagging integration algorithms, and lifting integration algorithms.
In some embodiments, before the step S1, the method further comprises: generating a configuration file in a json format, wherein the configuration file comprises the at least two machine learning models, a hyper-parameter search space of each machine learning model, at least two integration algorithms and integration algorithm parameters of the integration algorithms.
In some embodiments, the determining the target integration model according to the scoring result of each cycle includes: and determining a target integration model according to the grading result of each cycle and the cycle execution times.
In some embodiments, after the step S4, the method further includes: and selecting the hyper-parameters and the integration algorithm parameters of the next cycle according to the grading result and the optimization algorithm.
In some embodiments, the machine learning model is derived from any one of the following machine learning frameworks: tensorflow symbolic math system, a pytorch deep learning framework, a sklern machine learning language library.
In some embodiments, the machine learning model has weight parameters, the method further comprising: and determining the weight of each machine learning model in the integration process according to the weight parameter of each machine learning model.
According to a second aspect of embodiments of the present invention, there is provided an apparatus for automatically integrating machine learning, the apparatus comprising: the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, and the integration algorithm parameter group comprises at least one integration algorithm parameter; a parameter input unit configured to input a first hyper-parameter selected within a range of the hyper-parameter search space into each of the machine learning models; the integration unit is used for integrating the at least two machine learning models according to an integration algorithm and a first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model; the training scoring unit is used for training and scoring the first integrated model to obtain a scoring result; and the preference unit is used for determining a target integrated model according to the grading result of each cycle after the input unit, the integration unit and the training grading unit repeat N times of cycles of input, integration, training and grading in sequence, wherein N is a positive integer.
In some embodiments, the integrated unit is further configured to: integrating the at least two machine learning models according to any one of the following integration algorithms: stacking integration algorithms, bagging integration algorithms, and lifting integration algorithms.
In some embodiments, the apparatus further comprises: a configuration unit, configured to generate a configuration file in a json format, where the configuration file includes the at least two machine learning models, a hyper-parameter search space of each machine learning model, at least two integration algorithms, and an integration algorithm parameter set of each integration algorithm.
In some embodiments, the target integration model includes an optimal hyper-parameter and an optimal integration algorithm parameter, and the preference unit is further configured to determine the target integration model according to the scoring result of each loop and the number of times of loop execution.
In some embodiments, the apparatus further comprises: and the selecting unit is used for selecting the hyper-parameters and the integrated algorithm parameters of the next cycle according to the grading result and the optimization algorithm.
In some embodiments, the machine learning models have weight parameters, and the apparatus further comprises a determining unit for determining the weight of each of the machine learning models in the integration process according to the weight parameters of each of the machine learning models.
According to a third aspect of embodiments of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of automatic integrated machine learning as described in the first aspect of the embodiments above.
According to a fourth aspect of embodiments of the present invention, there is provided an electronic apparatus, including: one or several processors; storage means for storing one or several programs which, when executed by the one or several processors, cause the one or several processors to carry out the method of automatic integrated machine learning as described in the first aspect of the embodiments above.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in the technical solutions provided in some embodiments of the present invention, an overall optimal integrated model can be obtained by integrating the machine learning model, training and scoring the model, repeating the cycle for a plurality of times, and determining a target integrated model according to the scoring result.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 schematically illustrates a flow diagram of a method of automatically integrating machine learning, in accordance with one embodiment of the present invention;
FIG. 2 schematically illustrates a flow diagram of a method of automatically integrating machine learning, in accordance with another embodiment of the present invention;
FIG. 3 schematically illustrates a block diagram of an apparatus for automated integrated machine learning, in accordance with one embodiment of the present invention;
FIG. 4 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or several hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 1 is a flow chart illustrating a method for automatically integrating machine learning according to an embodiment of the present invention. The method provided by the embodiment of the invention can be executed by any electronic equipment with computer processing capability, such as a terminal device and/or a server. As shown in fig. 1, the method of automatically integrating machine learning includes:
and step S1, acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, wherein the integration algorithm group comprises at least one integration algorithm parameter.
Step S2, inputting the first hyper-parameter selected in the range of the hyper-parameter search space into each machine learning model.
And step S3, integrating at least two machine learning models according to the integration algorithm and the first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model.
And step S4, training the first integrated model and scoring the model to obtain a scoring result.
And step S5, taking the step S2, the step S3 and the step S4 which are sequentially executed as a loop, repeatedly executing N loops, and determining the target integrated model according to the scoring result of each loop, wherein N is a positive integer.
In the technical scheme of the embodiment of the invention, a first hyper-parameter is selected in the range of a hyper-parameter search space, a first integration algorithm parameter is selected from an integration algorithm parameter group, a plurality of machine learning models are subjected to parameter input and integration according to the first hyper-parameter and the first integration algorithm parameter, the obtained integration models are subjected to model scoring to form a cycle, and after the cycle is executed for a plurality of times, the overall optimal integration model can be determined as a target integration model according to the scoring result of each cycle.
In the embodiment of the invention, different integration algorithms can be selected in different cycles.
The technical scheme of the embodiment of the invention combines the super-parameter tuning of the machine learning model with the integrated algorithm selection, the integrated learning parameter selection and other steps, and the like to operate, and performs integrated learning at the same time in the automatic machine learning stage to obtain the final overall optimization model, thereby reducing the labor amount of algorithm personnel, and simultaneously obtaining the final integrated model which is the result of overall scoring optimization, namely the overall optimized integrated model.
With the help of the technical scheme of the embodiment of the invention, an algorithm worker can easily perform automatic ensemble learning training after compiling the hyper-parameter search space file according to a certain format rule, and finally generate an optimized ensemble model. In the whole process, a large amount of repetitive labor of algorithm personnel is avoided, a large amount of time is saved, and the production efficiency is improved.
The implementation of the technical scheme of the embodiment of the invention can depend on the python dependent library. Before step S1, the python library on which the solution of the embodiment of the present invention depends is also automatically installed. And then generating a configuration file in a json format, wherein the configuration file comprises at least two machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter of the integration algorithm.
json (JS object notation) is a lightweight data exchange format. It stores and represents data in a text format that is completely independent of the programming language, based on a subset of ECMAScript (js specification set by the european computer association).
The configuration file is mainly used for defining the configuration of the hyper-parameter search space and the integration model. The configuration file has a certain format rule, and an algorithm worker writes the configuration file according to the format rule so as to load the configuration file when performing automatic machine integration learning.
In the technical scheme of the embodiment of the invention, the configuration file and the optimization algorithm are used for determining the hyper-parameters. The hyper-parameter selected in each cycle is a selection result of an adjustable hyper-parameter search space in a configuration file, the selected hyper-parameter is input into a written model, several models can be integrated according to an integration algorithm and the configuration of parameters of the integration algorithm, and then training and model scoring are performed to complete a cycle.
The hyper-parameters in the first cycle may be chosen randomly or empirically. After the first cycle is completed, the framework selects the next group of hyper-parameters according to the scoring result and the optimization algorithm, starts the next round of training and scoring, and repeats the steps in the cycle until the optimal hyper-parameters and the optimal integration model parameters are searched, so that the target integration model, namely the optimal integration model, is obtained.
The configuration of the parametric search space and the integration model is defined by a json-formatted file, the format of which is represented by the following example:
as seen from the example json configuration file, four machine learning models 0-3 and adjustable hyper-parameter search spaces of the models are arranged in a nested configuration mode, the four machine learning models perform automatic hyper-parameter tuning in the adjustable hyper-parameter spaces, and the last line of configuration is to integrate the four models configured in the front by adopting VotingClassifier (voting classifier).
In practical applications, the number of machine learning models in the configuration file is not limited to four.
In step S2, at least two machine learning models may be integrated according to any one of the following integration algorithms: stacking integration algorithms, bagging integration algorithms, and lifting integration algorithms.
Stacking (Stacking), bagging (bagging), and Boosting are the main methods of three ensemble studies.
In the embodiment of the present invention, the machine learning models have weight parameters, and in step S3, the weight of each machine learning model in the integration process may be determined according to the weight parameters of each machine learning model.
For the json configuration file of the above example, the weight parameter of each machine learning model is selected from the parameter space defined in the configured value, so that tuning of ensemble learning is achieved, and an algorithm person can adopt different ensemble learning according to the algorithm effect.
In step S5, the target integration model may be determined based on the score result of each loop, or may be determined based on the score result of each loop and the number of loop executions.
Here, the target integration model is an optimal integration model, and the target integration model includes an optimal hyper-parameter and an optimal integration algorithm parameter. When the optimal integration model is selected, one way is to consider that the optimal integration model is obtained when the scoring result reaches a set threshold value. In another mode, a curve graph of the grading result and the number of times of cycle execution is constructed, and the highest point of the curve is taken as an optimal integrated model.
After step S4, the hyper-parameters and the parameters of the integrated algorithm of the next cycle are selected according to the scoring result and the optimization algorithm. That is, after step S4, an appropriate hyper-parametric input machine learning model is selected for step S2 based on the scoring results obtained in step S4.
Here, the optimization algorithm may include a bayesian optimization algorithm, a TPE (Tree-structured park Estimator Approach) optimization algorithm, and is not limited thereto.
In one embodiment of the present invention, after 40 times of loop execution, an optimal integration model can be obtained.
In the embodiment of the invention, various parameters of the machine learning model and the deep learning model can be adjusted, so that the performance of the machine learning model is optimal.
In particular, the machine learning model may be derived from any one of the following machine learning frameworks: tensorflow symbolic math system, a pytorch deep learning framework, a sklern machine learning language library.
Among them, the tensflow is a symbolic mathematical system based on dataflow programming, and is widely applied to programming realization of various machine learning (machine learning) algorithms, and the predecessor of the tensflow is the neural network algorithm library distebrief of ***. Pythrch is a popular deep learning framework. skearn is a machine learning Python language library of open source (BSD).
Deep Learning (Deep Learning) is a branch of machine Learning that attempts algorithms that perform high-level abstractions of data using multiple processing layers that contain complex structures or are composed of multiple nonlinear transformations. Deep learning is a characterization learning method in machine learning. The observations of an image may be represented in a number of ways, such as a vector of intensity values for each pixel, or more abstractly as a series of edges, specially shaped regions, and so forth. And tasks are easier to learn from the examples using some specific representation methods. Such as face recognition or facial expression recognition. The benefit of deep learning is that the manual acquisition of features will be replaced by efficient algorithms of unsupervised or semi-supervised feature learning and hierarchical feature extraction.
Since the steps of model hyper-parameter optimization and ensemble learning are all realized by python, machine learning framework libraries supporting all python interfaces, such as popular tensorflow, pyrrch, sklern, are supported. And the integrated algorithm of all python interfaces is supported, and the integrated learning such as stacking, bagging and lifting is realized.
As shown in fig. 2, an automatic integrated machine learning method according to an embodiment of the present invention includes the following steps:
step S601, a plurality of machine learning models which are pre-configured, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter of the integration algorithm are obtained.
In step S602, one hyper-parameter selected within the range of the hyper-parameter search space is input to each machine learning model.
Step S603, integrating a plurality of machine learning models according to an integration algorithm and an integration algorithm parameter to generate an integrated model.
And step S604, training the integrated model and scoring the model to obtain a scoring result.
And step S605, judging whether the integration model is the optimal integration model according to the grading result. If yes, executing step S607; if the determination result is no, step S606 is executed.
Step S606, selecting a new hyper-parameter according to the scoring result.
And step S607, determining the optimal integration model as the target integration model.
In the method for automatically integrating machine learning provided by the embodiment of the invention, the machine learning model is integrated, then training and model scoring are carried out, the cycle is repeated for multiple times, and the target integration model is determined according to the scoring result, so that the integrally optimal integration model can be obtained.
Embodiments of the apparatus of the present invention are described below that can be used to perform the above-described methods of automated integrated machine learning of the present invention. Referring to fig. 3, an apparatus based on automatic integrated machine learning according to an embodiment of the present invention includes:
an obtaining unit 302, configured to obtain at least two preconfigured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm, and an integration algorithm parameter set of the integration algorithm, where the integration algorithm parameter set includes at least one integration algorithm parameter.
A parameter input unit 304 for inputting a first hyper-parameter selected within the range of the hyper-parameter search space into each machine learning model.
An integrating unit 306, configured to integrate the at least two machine learning models according to the integration algorithm and the first integration algorithm parameter of the integration algorithm, so as to generate a first integration model.
And the training scoring unit 308 is configured to train and score the first integrated model to obtain a scoring result.
And the preference unit 310 is configured to determine a target integration model according to a scoring result of each cycle after the input unit, the integration unit, and the training scoring unit repeat N cycles of input, integration, training, and scoring operations in sequence, where N is a positive integer.
The integration unit 306 is further configured to: integrating at least two machine learning models according to any one of the following integration algorithms: stacking integration algorithms, bagging integration algorithms, and lifting integration algorithms.
The device further comprises a configuration unit, which is used for generating a configuration file in a json format, wherein the configuration file comprises at least two machine learning models, a hyper-parameter search space of each machine learning model, at least two integration algorithms and an integration algorithm parameter group of each integration algorithm.
The target integration model includes an optimal hyper-parameter and an optimal integration algorithm parameter, and the optimization unit 310 is further configured to determine the target integration model according to the scoring results of each cycle and the number of times of cycle execution.
The device also comprises a selection unit used for selecting the hyper-parameters and the integrated algorithm parameters of the next cycle according to the grading result and the optimization algorithm.
The machine learning models have weight parameters, and the device further comprises a determining unit for determining the weight of each machine learning model in the integration process according to the weight parameters of each machine learning model.
For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the above-described embodiments of the method for automatic integrated machine learning of the present invention for details which are not disclosed in the embodiments of the apparatus of the present invention, since the respective functional modules of the apparatus for automatic integrated machine learning based on the exemplary embodiments of the present invention correspond to the steps of the above-described exemplary embodiments of the method for automatic integrated machine learning.
In the device based on the automatic integrated machine learning provided by the embodiment of the invention, the machine learning model is integrated firstly, then training and model grading are carried out, the cycle is repeated for multiple times, and the target integrated model is determined according to the grading result, so that the integrally optimal integrated model can be obtained.
Referring now to FIG. 4, a block diagram of a computer system 400 suitable for use with the electronic device implementing an embodiment of the invention is shown. The computer system 400 of the electronic device shown in fig. 4 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in fig. 4, the computer system 400 includes a Central Processing Unit (CPU)401 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 404 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.
In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable storage medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. The above-described functions defined in the system of the present application are executed when the computer program is executed by a Central Processing Unit (CPU) 401.
It should be noted that the computer readable storage medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable storage medium carries one or several programs, which when executed by an electronic device, cause the electronic device to implement the method for automatically integrating machine learning as described in the above embodiments.
For example, the electronic device may implement the following as shown in fig. 1: step S1, acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, wherein the integration algorithm group comprises at least one integration algorithm parameter; step S2, inputting a first hyper-parameter selected in the range of the hyper-parameter search space into each machine learning model; step S3, integrating the at least two machine learning models according to an integration algorithm and a first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model; step S4, training the first integrated model and scoring the model to obtain a scoring result; and step S5, taking the step S2, the step S3 and the step S4 which are sequentially executed as a loop, repeatedly executing N loops, and determining a target integration model according to the grading result of each loop, wherein N is a positive integer.
As another example, the electronic device may also implement the steps as described in fig. 2.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by several modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. A method of automatically integrating machine learning, the method comprising the steps of:
step S1, acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, wherein the integration algorithm parameter group comprises at least one integration algorithm parameter;
step S2, inputting a first hyper-parameter selected in the range of the hyper-parameter search space into each machine learning model;
step S3, integrating the at least two machine learning models according to the integration algorithm and a first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model;
step S4, training the first integrated model and scoring the model to obtain a scoring result;
and step S5, taking the step S2, the step S3 and the step S4 which are executed in sequence as a loop, and after the loop is repeatedly executed, determining a target integration model according to the grading result of each loop.
2. The method according to claim 1, wherein the step S3 includes:
integrating the at least two machine learning models according to any one of the following integration algorithms:
stacking integration algorithms, bagging integration algorithms, and lifting integration algorithms.
3. The method according to claim 1, wherein before the step S1, the method further comprises:
generating a configuration file in a json format, wherein the configuration file comprises the at least two machine learning models, a hyper-parameter search space of each machine learning model, at least two integration algorithms and an integration algorithm parameter set of each integration algorithm.
4. The method of claim 1, wherein the target integration model comprises an optimal hyper-parameter and an optimal integration algorithm parameter, and wherein determining the target integration model according to the scoring results of each of the cycles comprises:
and determining a target integration model according to the grading result of each cycle and the cycle execution times.
5. The method according to claim 1, wherein after the step S4, the method further comprises:
and selecting the hyper-parameters and the integrated algorithm parameters of the next cycle according to the scoring result and the optimization algorithm.
6. The method of claim 1, wherein the machine learning model is derived from any one of the following machine learning frameworks:
tensorflow symbolic math system, a pytorch deep learning framework, a sklern machine learning language library.
7. The method of claim 1, wherein the machine learning model has weight parameters, the method further comprising:
and determining the weight of each machine learning model in the integration process according to the weight parameter of each machine learning model.
8. An apparatus for automatically integrating machine learning, the apparatus comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring at least two pre-configured machine learning models, a hyper-parameter search space of each machine learning model, an integration algorithm and an integration algorithm parameter group of the integration algorithm, and the integration algorithm parameter group comprises at least one integration algorithm parameter;
a parameter input unit configured to input a first hyper-parameter selected within a range of the hyper-parameter search space into each of the machine learning models;
the integration unit is used for integrating the at least two machine learning models according to an integration algorithm and a first integration algorithm parameter in the integration algorithm parameter group to generate a first integration model;
the training scoring unit is used for training and scoring the first integrated model to obtain a scoring result;
and the preference unit is used for determining a target integrated model according to the grading result of each cycle after the input unit, the integration unit and the training grading unit repeatedly and sequentially carry out the cycle of input, integration, training and grading operation.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of automatically integrating machine learning according to any one of claims 1 to 7.
10. An electronic device, comprising:
one or several processors;
storage means for storing one or several programs which, when executed by the one or several processors, cause the one or several processors to carry out the method of automated integrated machine learning according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010982700.0A CN112116104B (en) | 2020-09-17 | 2020-09-17 | Method, device, medium and electronic equipment for automatically integrating machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010982700.0A CN112116104B (en) | 2020-09-17 | 2020-09-17 | Method, device, medium and electronic equipment for automatically integrating machine learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112116104A true CN112116104A (en) | 2020-12-22 |
| CN112116104B CN112116104B (en) | 2024-06-18 |
Family
ID=73801087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010982700.0A Active CN112116104B (en) | 2020-09-17 | 2020-09-17 | Method, device, medium and electronic equipment for automatically integrating machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112116104B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113837323A (en) * | 2021-11-08 | 2021-12-24 | 中国联合网络通信集团有限公司 | Satisfaction prediction model training method and device, electronic equipment and storage medium |
| CN114003393A (en) * | 2021-12-30 | 2022-02-01 | 南京大学 | Method and system for improving integrated automatic machine learning operation performance |
| WO2022193408A1 (en) * | 2021-03-17 | 2022-09-22 | 中奥智能工业研究院(南京)有限公司 | Automatic data analysis and modeling process |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110210556A (en) * | 2019-05-29 | 2019-09-06 | 中国科学技术大学 | Pedestrian identifies data creation method again |
| CN110222848A (en) * | 2019-05-05 | 2019-09-10 | 阿里巴巴集团控股有限公司 | The determination method and device for the integrated model that computer executes |
| CN110503208A (en) * | 2019-08-26 | 2019-11-26 | 第四范式（北京）技术有限公司 | Resource regulating method and resource scheduling device in multi-model exploration |
| CN110517492A (en) * | 2019-08-27 | 2019-11-29 | 中国科学院自动化研究所 | Based on the traffic route recommended method of parallel integrated study, system, device |
| CN110763660A (en) * | 2019-10-22 | 2020-02-07 | 华南理工大学 | LIBS quantitative analysis method based on ensemble learning |
| CN110914839A (en) * | 2017-06-26 | 2020-03-24 | D5Ai有限责任公司 | Selective training of error decorrelation |
| US20200193313A1 (en) * | 2018-12-14 | 2020-06-18 | Parallel Machines, Inc. | Interpretability-based machine learning adjustment during production |
| US20200242400A1 (en) * | 2019-01-25 | 2020-07-30 | Oath Inc. | Systems and methods for hyper parameter optimization for improved machine learning ensembles |
| CN111597758A (en) * | 2020-05-14 | 2020-08-28 | 河海大学 | Medium and small river integrated forecasting method based on negative correlation learning |
-
2020
- 2020-09-17 CN CN202010982700.0A patent/CN112116104B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110914839A (en) * | 2017-06-26 | 2020-03-24 | D5Ai有限责任公司 | Selective training of error decorrelation |
| US20200193313A1 (en) * | 2018-12-14 | 2020-06-18 | Parallel Machines, Inc. | Interpretability-based machine learning adjustment during production |
| US20200242400A1 (en) * | 2019-01-25 | 2020-07-30 | Oath Inc. | Systems and methods for hyper parameter optimization for improved machine learning ensembles |
| CN110222848A (en) * | 2019-05-05 | 2019-09-10 | 阿里巴巴集团控股有限公司 | The determination method and device for the integrated model that computer executes |
| CN110210556A (en) * | 2019-05-29 | 2019-09-06 | 中国科学技术大学 | Pedestrian identifies data creation method again |
| CN110503208A (en) * | 2019-08-26 | 2019-11-26 | 第四范式（北京）技术有限公司 | Resource regulating method and resource scheduling device in multi-model exploration |
| CN110517492A (en) * | 2019-08-27 | 2019-11-29 | 中国科学院自动化研究所 | Based on the traffic route recommended method of parallel integrated study, system, device |
| CN110763660A (en) * | 2019-10-22 | 2020-02-07 | 华南理工大学 | LIBS quantitative analysis method based on ensemble learning |
| CN111597758A (en) * | 2020-05-14 | 2020-08-28 | 河海大学 | Medium and small river integrated forecasting method based on negative correlation learning |
Non-Patent Citations (1)
| Title |
|---|
| 杨峻山;纪震;谢维信;朱泽轩;: "基于粒子群优化的生物组学数据分类模型选择", 深圳大学学报(理工版), no. 03, 30 May 2016 (2016-05-30) * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022193408A1 (en) * | 2021-03-17 | 2022-09-22 | 中奥智能工业研究院(南京)有限公司 | Automatic data analysis and modeling process |
| CN113837323A (en) * | 2021-11-08 | 2021-12-24 | 中国联合网络通信集团有限公司 | Satisfaction prediction model training method and device, electronic equipment and storage medium |
| CN113837323B (en) * | 2021-11-08 | 2023-07-04 | 中国联合网络通信集团有限公司 | Training method and device of satisfaction prediction model, electronic equipment and storage medium |
| CN114003393A (en) * | 2021-12-30 | 2022-02-01 | 南京大学 | Method and system for improving integrated automatic machine learning operation performance |
| CN114003393B (en) * | 2021-12-30 | 2022-06-14 | 南京大学 | Method and system for improving integrated automatic machine learning operation performance |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112116104B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110807515B (en) | Model generation method and device | |
| CN110766142A (en) | Model generation method and device | |
| WO2021136365A1 (en) | Application development method and apparatus based on machine learning model, and electronic device | |
| Hernández-Lobato et al. | Probabilistic backpropagation for scalable learning of bayesian neural networks | |
| CN112699991A (en) | Method, electronic device, and computer-readable medium for accelerating information processing for neural network training | |
| CN112116104B (en) | Method, device, medium and electronic equipment for automatically integrating machine learning | |
| CN111406267A (en) | Neural architecture search using performance-predictive neural networks | |
| CN110503192A (en) | The effective neural framework of resource | |
| WO2019155064A1 (en) | Data compression using jointly trained encoder, decoder, and prior neural networks | |
| CN109906460A (en) | Dynamic cooperation attention network for question and answer | |
| CN111602148A (en) | Regularized neural network architecture search | |
| US11989657B2 (en) | Automated machine learning pipeline for timeseries datasets utilizing point-based algorithms | |
| US20220188645A1 (en) | Using generative adversarial networks to construct realistic counterfactual explanations for machine learning models | |
| US11900263B2 (en) | Augmenting neural networks | |
| US11861469B2 (en) | Code generation for Auto-AI | |
| US20220198277A1 (en) | Post-hoc explanation of machine learning models using generative adversarial networks | |
| CN111373406B (en) | Accelerated simulation setup process using a priori knowledge extraction of problem matching | |
| US20190228297A1 (en) | Artificial Intelligence Modelling Engine | |
| US20230267307A1 (en) | Systems and Methods for Generation of Machine-Learned Multitask Models | |
| CN113557534A (en) | Deep forest model development and training | |
| CN112200296A (en) | Network model quantification method and device, storage medium and electronic equipment | |
| CN112420125A (en) | Molecular attribute prediction method and device, intelligent equipment and terminal | |
| CN110782016A (en) | Method and apparatus for optimizing neural network architecture search | |
| Solazzo et al. | Hardware design automation of convolutional neural networks | |
| CN114462582A (en) | Data processing method, device and equipment based on convolutional neural network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |