CN116702923A

CN116702923A - Feature screening method, device, equipment and medium based on federal learning

Info

Publication number: CN116702923A
Application number: CN202310720291.0A
Authority: CN
Inventors: 瞿晓阳; 王健宗; 刘承昊
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2023-06-16
Filing date: 2023-06-16
Publication date: 2023-09-05

Abstract

The present invention relates to the field of data processing technologies, and in particular, to a feature screening method, apparatus, device, and medium based on federal learning. The method is applied to the financial field, a training feature set of a local end to which a model to be trained belongs is obtained, M update parameters of corresponding feature categories after M times of federal training are collected, a change trend corresponding to each feature category is determined according to the M update parameters of the feature categories, a preset change condition is obtained, and feature categories which do not meet the change condition are determined.

Description

Feature screening method, device, equipment and medium based on federal learning

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a feature screening method, apparatus, device, and medium based on federal learning.

Background

In the field of credit wind control, aiming at the problems of scarcity, incomplete credit review data, insufficient historical information precipitation and the like of small and medium-sized enterprises, a bank can fuse multi-source information such as enterprise operation data, tax data, business data, payment data and the like under the condition of ensuring data safety and privacy protection of a data provider through federal learning, so that a modeling characteristic system is enriched, the effectiveness of a model is jointly improved, and federal learning is also known as federal machine learning, joint learning and federal learning. Federal learning is a machine learning framework, and can effectively help a plurality of institutions to perform data use and machine learning modeling under the condition that the requirements of user privacy protection and data security are met, so that data sharing is realized. Screening out the appropriate features in federal learning business floor is a key point to improve the overall effect of the model.

In the prior art, a manual screening method is generally adopted to perform feature screening, however, the above feature screening method is excessively dependent on subjective experience, and modeling staff of different participants in federal modeling have different experience and cognition on feature screening, so that the screening precision is poor, and therefore, how to improve the screening precision in the feature screening process is a problem to be solved urgently.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a feature screening method, device, apparatus and medium based on federal learning to solve the problem of low screening accuracy.

A first aspect of an embodiment of the present application provides a feature screening method based on federal learning, where the feature screening method includes:

acquiring a training feature set of a local end to which a model to be trained belongs, wherein the training feature set comprises feature quantities corresponding to N feature categories, the training feature set is used for training the model to be trained, N parameters of the trained model are sent to a central server end for federal training, and N is an integer greater than 1;

collecting N updating parameters sent by the central server after M times of federal training to obtain M updating parameters corresponding to characteristic categories, wherein M is an integer greater than 1;

for any feature category, determining the change trend of the update parameter corresponding to the feature category according to M update parameters of the feature category, and obtaining the change trend corresponding to each feature category;

and acquiring a preset change condition, comparing the change trend of all feature categories with the change condition, determining the feature category which does not meet the change condition, and eliminating the corresponding feature quantity from the training feature set.

A second aspect of an embodiment of the present application provides a feature screening apparatus based on federal learning, the feature screening apparatus including:

the training feature set comprises feature quantities corresponding to N feature categories, and is used for training the model to be trained, and transmitting N parameters of the trained model to a central server for federal training, wherein N is an integer greater than 1;

the acquisition module is used for acquiring N updating parameters sent by the central server after M times of federal training to obtain M updating parameters of corresponding feature categories, wherein M is an integer greater than 1;

the determining module is used for determining the change trend of the update parameters corresponding to the feature categories according to M update parameters of the feature categories aiming at any feature category to obtain the change trend corresponding to each feature category;

the screening module is used for acquiring preset change conditions, comparing the change trend of all feature categories with the change conditions, determining feature categories which do not meet the change conditions, and eliminating corresponding feature quantities from the training feature set.

In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the feature screening method according to the first aspect when executing the computer program.

In a fourth aspect, embodiments of the present invention provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the feature screening method according to the first aspect.

Compared with the prior art, the invention has the beneficial effects that:

the method comprises the steps of obtaining a training feature set of a local end to which a model to be trained belongs, wherein the training feature set comprises feature quantities corresponding to N feature categories, the training feature set is used for training the model to be trained, N parameters of the trained model are sent to a central server end for federal training, N is an integer larger than 1, N updated parameters sent by the central server end after M times of federal training are collected to obtain M updated parameters corresponding to the feature categories, M is an integer larger than 1, for any feature category, according to the M updated parameters of the feature categories, the change trend of the updated parameters corresponding to the feature categories is determined, the change trend corresponding to each feature category is obtained, preset change conditions are obtained, the change trend of all the feature categories is compared with the change conditions, the feature categories which do not meet the change conditions are determined, and the corresponding feature quantities are removed from the training feature sets. In the invention, the change trend of the feature type updating parameters is detected by using the updating parameters of the feature type in the model to be trained, whether the feature type is a real feature type is judged, the feature quantity corresponding to the real feature type is reserved, in the screening process, the authenticity corresponding to each feature type can be judged under the condition that priori knowledge is not needed, and the feature type screening is carried out, so that the screening result is more reasonable, and the accuracy of the screening result is improved. According to the federal learning model obtained through the feature training after screening, the precision of the model can be improved, and therefore the risk of credit operation based on the federal learning model is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of an application environment of a feature screening method based on federal learning according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a feature screening method based on federal learning according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a feature screening device based on federal learning according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

The embodiment of the invention can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

It should be understood that the sequence numbers of the steps in the following embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not be construed as limiting the implementation process of the embodiments of the present invention.

In order to illustrate the technical scheme of the invention, the following description is made by specific examples.

The feature screening method based on federal learning provided by an embodiment of the present invention can be applied in an application environment as shown in fig. 1, where a local end communicates with a server end. The local terminal includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), and the like. The server may be implemented as a stand-alone server or as a cluster of servers generated by multiple servers.

Referring to fig. 2, a flow chart of a feature screening method based on federal learning according to an embodiment of the present invention is shown, where the feature screening method based on federal learning may be applied to a server in fig. 1, and the server is connected to a corresponding local terminal, as shown in fig. 2, and the feature screening method based on federal learning may include the following steps.

S201: and acquiring a training feature set of a local terminal to which the model to be trained belongs.

In step S201, a training feature set of a local end to which a model to be trained belongs is obtained, where the training feature set of the local end is a feature training set of the local end participating in federal learning, the training feature set includes feature amounts corresponding to N feature categories, the training feature set is used for training the model to be trained, N parameters of the trained model are sent to a central server end for federal training, and N is an integer greater than 1.

In this embodiment, a financial system for constructing a federal learning architecture is obtained, where the financial system includes a server and N banking terminals, each banking terminal is connected to the server, and a data set included in each bank may be the same or different, for example, each bank includes the same batch of trusted people list data, some of the banking data sets include some business data handled by a user in the bank, and some of the banks do not include corresponding business data. Wherein, the number of data of each data set corresponding to each bank can be the same or different.

In this embodiment, local ends connected with a server are obtained according to the server, a specific number of local ends are selected, corresponding training feature sets are obtained from the selected local ends, each local end has different training feature sets of users, and joint modeling is performed according to the feature data, so that the server needs to face multiple training feature set providers, and contribution or value of the training feature sets provided by the multiple data providers to a model may be uneven. The training feature set is filtered.

And N parameters of the trained model are sent to a central server for federal training, the central server updates the corresponding parameters based on the parameters sent by the local end in federal training, iterative training is sequentially carried out, and model updating parameters generated during model training operation are sent to the corresponding local end, so that the local end updates the local federal learning model according to the training feature set, and a trained federal learning model corresponding to each local end is obtained. The training feature set comprises feature quantities corresponding to N feature categories, is used for training a model to be trained, and sends N parameters of the trained model to the central server for federal training.

Optionally, obtaining the training feature set of the local end to which the model to be trained belongs includes:

and collecting a plurality of feature categories provided by the local terminal, and randomly selecting N feature categories from the plurality of feature categories to form a training feature set of the local terminal to which the model to be trained belongs.

In this embodiment, in order to improve universality of the screened features and avoid overfitting, N feature classes are randomly selected from a plurality of feature classes provided by each local end of federal learning in a feature dimension sampling-based manner, and then subsequent feature screening links are performed in parallel based on the N feature classes. There are various sampling methods, such as full search, heuristic search or random search.

The exhaustive full search mode is simple. Heuristic searches may compute some index of all training features prior to federal feature screening to exploratory addition of features from the empty set or cull features from the full set of features. The search range can be reduced, and the complexity of the problem can be reduced.

The multiple feature categories provided by each local end of federal learning may include identification information that may distinguish training features, e.g., numbers, letters, names, etc. The user privacy protection function can be achieved by distinguishing the actual names of the training feature sets. The training feature sets provided by the local terminals are used for hiding the feature names, so that the user data privacy can be protected, and feature screening can be performed under the condition of ensuring safety.

In this embodiment, the communication cost of federal modeling is high, and feature dimensions provided by multiple parties in federal modeling are often more, so that a random search mode can be adopted. Sampling a plurality of feature categories provided by each local end of federal learning to obtain N feature categories, including: and randomly sampling a plurality of characteristic categories provided by each participant of federal learning to obtain N characteristic categories. The feature categories comprise identification information, the identification information is a number, a plurality of feature categories provided by each local end of the Union learning are randomly sampled, N feature categories are obtained, and a training feature set of the local end to which the model to be trained belongs is formed. Comprising the following steps: sequencing according to the numbers of the feature categories to obtain a feature sequence; and randomly sampling the feature sequence according to a randomly generated sequence selection algorithm to obtain N feature categories. Specifically, different subsets are formed on the complete feature sets provided by the local ends, for example, feature categories of the local ends can be formed into a feature sequence according to numbers, and then a random generation sequence selection algorithm (Random Generation plus Sequential Selection, RGSS) is used for randomly generating the multiple subsets to form a training feature set of the local end to which the model to be trained belongs.

S202: and acquiring N updating parameters sent by the central server after M times of federal training to obtain M updating parameters of the corresponding feature class.

In step S202, the federation training is to obtain the model parameters sent by the local end, integrate the model parameters, send the integrated model parameters to the local end, and perform retraining by the local end according to the received updated parameters, so that after performing federation training for M times, the central server end sends the updated parameters for M times to the local end, and obtains M updated parameters of corresponding feature categories, where each feature category in the N feature categories obtains M updated parameters respectively.

In this embodiment, the local training model is initialized, the local training times, the learning rate, the global model (random initialization), the local mask, the scaling factor, and the learning rate preheating times, the parameters of the initial state of the local training model are the same, and the model parameters of the local training models constructed between different participating local ends are the same, so that it is ensured that the different participating local ends respectively perform model training on the same local training model.

For example, the embodiment includes a participant local terminal a1, a participant local terminal a2, and a participant local terminal a3, where the participant local terminal a1, the participant local terminal a2, and the participant local terminal a3 respectively train a model to be trained to obtain a model update parameter b1, a model update parameter b2, and a model update parameter b3, the model update parameter b1, the model update parameter b2, and the model update parameter b3 are sent to a central server, and the central server integrates the model update parameter b1, the model update parameter b2, and the model update parameter b3 to obtain new model parameters, sends the new model parameters to each local terminal, and obtains M update parameters of corresponding feature types after M times of federal training.

When the model to be trained is trained, the local end needs to create corresponding task configuration information based on the federal training task. Specifically, the local side may determine and create task configuration information of the federation training task by responding to a federation training setting operation of a user, where the task configuration information of the federation training task includes, but is not limited to: the system comprises information such as task types, engine frameworks, automatic parameter adjusting algorithms, early termination algorithms, characteristic engineering information processing methods, data preprocessing methods and the like.

After the local end determines task configuration information corresponding to the federation training task, the local end sends the task configuration information to the central server end, so that the central server end obtains the task configuration information of at least two local ends participating in the federation training task. Since the task configuration information does not involve data security privacy issues, the local side may send the task configuration information to the central server side without encryption. And initializing the configuration information of the model to be trained in the Union training task according to the task configuration information. The configuration information of the model to be trained may refer to configuration information adopted when training a machine learning model of the bang training task.

For example, the central server determines configuration information such as a target task type, a target engine frame, a target automatic parameter adjusting algorithm, a target early termination algorithm, a target feature engineering, a target data preprocessing scheme, a model to be trained and the like aiming at the federation training task information according to the task configuration information, so as to perform initialization setting on the configuration information of the model to be trained in the federation training task, and complete deployment arrangement of the whole training task.

And executing model training operation of the federal training task based on the initialized configuration information of the model to be trained. The trained federal training model is used for carrying out service processing on the received user data by the corresponding local terminal. In practical application, the trained federal training model can be applied to business systems such as advertisement recommendation systems, video recommendation systems, user credit prediction systems and the like.

The trained federal training model is a cooperative training model obtained by federal training of a central server and a local server. After the central server finishes initializing the configuration information of the model to be trained in the Union training task, the central server performs model training operation of the Union training task by utilizing automatic machine learning based on the initialized configuration information of the model to be trained. Specifically, the server may schedule each local terminal to perform automatic data preprocessing and automatic feature engineering on the training feature set stored locally at the local terminal according to the initialized configuration information of the model to be trained.

For example, after the automatic machine learning engine obtains the configuration information of the model to be trained, the automatic machine learning engine selects the training framework required to be used in the federal training task, the model to be trained to be used, the model screening evaluation mode, the parameter model group and other information. The automated machine learning engine then performs training based on the selected training framework and the attributes of the model to be trained. The automatic machine learning engine starts a training framework engine to train the data according to the framework configuration, and evaluates the result at the middle or end of the training. And (3) sorting and summarizing the output evaluation information and model information, and feeding back to a global automatic machine learning engine, thereby realizing model training operation for executing federal training tasks. The global automatic machine learning engine is an automatic machine learning engine corresponding to the central server. And after M times of federal training, collecting N updating parameters sent by a central server after M times of federal training to obtain M updating parameters of corresponding feature categories.

Optionally, collecting N update parameters sent by the central server after M times of federal training to obtain M update parameters corresponding to the feature class, including:

aiming at each federal training, the local end trains the model to be trained to obtain N local updating parameters corresponding to N characteristic categories in the local end;

n local update parameters are sent to a central server;

for any feature category, the central server integrates the local update parameters corresponding to the feature category to obtain the update parameters corresponding to the central server, and the central server sends the update parameters to the local terminal;

and the local end carries out M times of federal training on the model to be trained according to the received updated parameters to obtain M updated parameters of the corresponding feature categories.

In this embodiment, the central server issues an initialization to-be-trained model to the local end in a manner of issuing a training task, and each local end that receives the initialization to-be-trained model uses its own local data to perform one round of model training on the initialization model, so as to obtain each trained iterative model. And each local end transmits the iteration model obtained after the model training of the model to be trained is initialized back to the central server end to obtain a federal training, so that each local end executes a round of model training and transmits the iteration model obtained by training back to the central server end every time the central server end transmits the model to each local end, and the server end obtains the iteration model transmitted back after a plurality of local ends execute the model training in the federal training model training process. For each federal training, the local end trains the model to be trained to obtain N local update parameters corresponding to N feature categories in the local end, the N local update parameters are sent to the central server, for any feature category, the central server integrates the local update parameters corresponding to the feature categories to obtain update parameters corresponding to the central server, the central server sends the update parameters to the local end, and the local end performs M federal training on the model to be trained according to the received update parameters to obtain M update parameters corresponding to the feature categories.

Optionally, for any feature class, the central server integrates the local update parameters corresponding to the feature class to obtain the update parameters corresponding to the central server, including:

when the number of the local terminals is multiple, the central server calculates the average value of the local update parameters, and the average value is used as the update parameter corresponding to the central server.

In this embodiment, when the central server calculates the corresponding update parameters, when the corresponding local ends are multiple, the average value of the parameters of the corresponding feature categories is calculated when the parameters of the corresponding feature categories sent by the multiple local ends are obtained, and the average value is used as the update parameter corresponding to the central server.

S203: and determining the change trend of the update parameters corresponding to the feature categories according to M update parameters of the feature categories aiming at any feature category, and obtaining the change trend corresponding to each feature category.

In step S203, for any feature class, according to M update parameters of the feature class, a change trend of the update parameters corresponding to the feature class is determined, so as to obtain a change trend corresponding to each feature class, so that the parameters are filtered according to the change trend of the parameters.

In this embodiment, after M times of federal training are acquired, the corresponding feature classes are filtered through the obtained M update parameters, and the stability of the corresponding feature classes is determined and the unstable feature classes are deleted by determining the change trend of the update parameters corresponding to the feature classes.

The update parameter variation trend is used for detecting whether the model update parameter is a false parameter, and in the step, the update parameter variation trend can be used for detecting the model update parameter in different local terminals so as to respectively judge whether the model update parameter in different local terminals is a false parameter.

The variation trend of the update parameter corresponding to the feature class can be represented by calculating the variance of the update parameter, wherein the larger the variance of the update parameter in the corresponding feature class is, the smaller the variance of the update parameter in the corresponding feature class is.

According to M updating parameters of the feature class, when the variation trend of the updating parameters corresponding to the feature class is determined, the variation trend of the M updating parameters can be determined by determining the discrete degree of the M updating parameters by using a method of twice standard deviation, after the corresponding variance is obtained by calculation, the standard deviation of the updating parameters is calculated according to the variance, then the mean value of the M updating parameters is calculated, finally the sum of the twice standard deviation and the mean value is calculated, the number of updating parameters which are not in the corresponding interval is calculated according to the sum of the twice standard deviation and the mean value, when the number of the updating parameters which are not in the corresponding interval is larger, the variation trend of the M updating parameters is considered to be unstable, and when the number of the updating parameters which are not in the corresponding interval is smaller, the variation trend of the M updating parameters is considered to be stable.

For any feature class, according to M update parameters of the feature class, when the change trend of the update parameters corresponding to the feature class is determined, the parameter change rate representation between the adjacent update parameters of the corresponding feature class can also be used, after the M update parameters of the corresponding feature class are obtained, the M update parameters are arranged according to the update sequence, the change rate of the adjacent update parameters is calculated to obtain M-1 change rates, the change trend of the update parameters corresponding to the feature class is determined according to the average value of the M-1 change rates, when the change rate average value is larger, the M update parameters are considered to be unstable, and when the change rate average value is smaller, the M update parameters are considered to be stable.

Optionally, for any feature class, determining a change trend of the update parameter corresponding to the feature class according to M update parameters of the feature class, to obtain a change trend corresponding to each feature class, including:

for any feature class, calculating the variance of M update parameters according to M update parameters of the feature class;

and determining the change trend of the update parameters corresponding to the feature categories according to the variance, and obtaining the change trend corresponding to each feature category.

In this embodiment, for any feature class, the variance of the M update parameters is calculated according to the M update parameters of the feature class, and the change trend of the update parameters corresponding to the feature class is determined according to the variance, so as to obtain the change trend corresponding to each feature class. When calculating the variance of M update parameters, filtering the M update parameters to obtain more accurate update parameters, wherein the filtering mode can be performed in a clustering mode, and as the variance of the update parameters in each characteristic category needs to be calculated and the characteristic category is screened through the variance, when the update parameters are subjected to clustering filtering, the number of the update parameters which are correspondingly filtered is set so as to avoid filtering the update parameters far away from the central point, and the screening of the characteristic category is difficult.

And determining the change trend of the update parameters corresponding to the feature categories according to the variance, and obtaining the change trend corresponding to each feature category, wherein when the variance is larger, the change trend of the update parameters is considered to be unstable, and when the variance is smaller, the change trend of the update parameters is considered to be stable.

S204: and acquiring preset change conditions, comparing the change trend of all feature categories with the change conditions, determining feature categories which do not meet the change conditions, and eliminating corresponding feature quantities from the training feature set.

In step S204, a preset change condition is obtained, a change trend of all feature categories is compared with the change condition, feature categories which do not meet the change condition are determined, corresponding feature quantities are removed from the training feature set, wherein the preset change condition can be a corresponding threshold value, and when the change trend does not meet the threshold value condition, the corresponding feature quantities are removed from the training feature set.

In this embodiment, the change trend of all feature classes is scored, so as to obtain a score value corresponding to each feature class, a preset change condition is a corresponding threshold value, when the score value corresponding to the feature class is greater than the threshold value, the feature quantity corresponding to the corresponding feature class is reserved, and when the score value corresponding to the feature class is less than the threshold value, the feature quantity corresponding to the corresponding feature class is deleted.

For example, when the change trend of all feature categories is classified, the higher the corresponding score is when the change trend of the feature category is small, the lower the corresponding score is when the change trend of the feature category is large, the corresponding feature quantity of the corresponding feature category is reserved when the score value of the corresponding feature category is larger than a threshold value, and the corresponding feature quantity of the corresponding feature category is deleted when the score value of the corresponding feature category is smaller than the threshold value.

The method comprises the steps of obtaining a training feature set of a local end to which a model to be trained belongs, wherein the training feature set comprises feature quantities corresponding to N feature categories, the training feature set is used for training the model to be trained, N parameters of the trained model are sent to a central server end for federal training, N is an integer larger than 1, N updated parameters sent by the central server end after M times of federal training are collected to obtain M updated parameters corresponding to the feature categories, M is an integer larger than 1, for any feature category, according to the M updated parameters of the feature categories, the change trend of the updated parameters corresponding to the feature categories is determined, the change trend corresponding to each feature category is obtained, preset change conditions are obtained, the change trend of all the feature categories is compared with the change conditions, the feature categories which do not meet the change conditions are determined, and the corresponding feature quantities are removed from the training feature sets. In the invention, the change trend of the feature type updating parameters is detected by using the updating parameters of the feature type in the model to be trained, whether the feature type is a real feature type is judged, the feature quantity corresponding to the real feature type is reserved, in the screening process, the authenticity corresponding to each feature type can be judged under the condition that priori knowledge is not needed, and feature type screening is carried out, so that the screening result is more reasonable, and the accuracy of the screening result is improved.

Optionally, a preset change condition is obtained, the change trend of all feature categories is compared with the change condition, the feature category which does not meet the change condition is determined, and after the corresponding feature quantity is removed from the training feature set, the method further comprises the steps of:

removing the corresponding feature quantity from the training feature set to obtain a training feature set after feature screening;

and performing federal training on the model to be trained by using the screened characteristic training set to obtain a trained model corresponding to the model to be trained.

In this embodiment, the federal training is performed on the model to be trained by using the feature training set after screening, so as to obtain a trained model corresponding to the model to be trained. During training, the central server transmits the updated parameters of the feature categories in the feature training set after screening to the local end, the local end uses corresponding local data to iteratively train the model to be trained M+1 times, and training is ended until the updated parameters of the feature categories transmitted to the central server tend to be converged, so that a trained model corresponding to the model to be trained is obtained. In practical application, the trained model can be applied to business systems such as advertisement recommendation systems, video recommendation systems, user credit prediction systems and the like.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a feature screening device based on federal learning according to an embodiment of the present invention. The terminal in this embodiment includes units for executing the steps in the embodiment corresponding to fig. 2. Refer specifically to fig. 2 and the related description in the embodiment corresponding to fig. 2. For convenience of explanation, only the portions related to the present embodiment are shown. As shown in fig. 3, the feature screening apparatus 30 includes: the system comprises an acquisition module 31, an acquisition module 32, a determination module 33 and a screening module 34.

The obtaining module 31 is configured to obtain a training feature set of a local side to which the model to be trained belongs, where the training feature set includes feature amounts corresponding to N feature categories, and the training feature set is configured to train the model to be trained, and send N parameters of the trained model to a central server side for federal training, where N is an integer greater than 1.

The acquisition module 32 is configured to acquire N update parameters sent by the central server after M times of federal training, and obtain M update parameters corresponding to the feature class, where M is an integer greater than 1.

The determining module 33 is configured to determine, for any feature class, a variation trend of the update parameter corresponding to the feature class according to M update parameters of the feature class, so as to obtain a variation trend corresponding to each feature class;

The screening module 34 is configured to obtain a preset change condition, compare the change trend of all feature categories with the change condition, determine feature categories that do not satisfy the change condition, and reject corresponding feature quantities from the training feature set.

Optionally, the feature screening apparatus 30 further includes:

the obtaining module is used for eliminating the corresponding feature quantity from the training feature set to obtain the training feature set after feature screening.

And the training module is used for carrying out federal training on the model to be trained by using the screened characteristic training set to obtain a trained model corresponding to the model to be trained.

Optionally, the acquiring module 31 includes:

the selecting unit is used for collecting a plurality of feature categories provided by the local terminal, randomly selecting N feature categories from the plurality of feature categories, and forming a training feature set of the local terminal to which the model to be trained belongs.

Optionally, the collecting module 32 includes:

the local updating parameter determining unit is used for training the model to be trained by the local terminal aiming at each federal training to obtain N local updating parameters corresponding to N characteristic categories in the local terminal.

The first sending unit is used for sending the N local updating parameters to the central server.

The second sending unit is used for integrating the local updating parameters corresponding to the feature categories aiming at any feature category by the central server to obtain the updating parameters corresponding to the central server, and the central server sends the updating parameters to the local terminal.

And the federation training unit is used for carrying out federation training on the model to be trained for M times by the local end according to the received updated parameters to obtain M updated parameters of the corresponding feature class.

Optionally, the second transmitting unit includes:

and the calculating subunit is used for calculating the average value of the local updating parameters by the central server when the local ends are a plurality of, and taking the average value as the updating parameters corresponding to the central server.

Optionally, the determining module 33 includes:

and the variance determining unit is used for calculating variances of M updating parameters according to M updating parameters of the feature category aiming at any feature category.

And the change trend determining unit is used for determining the change trend of the update parameter corresponding to the feature category according to the variance, and obtaining the change trend corresponding to each feature category.

It should be noted that, because the content of information interaction and execution process between the above units is based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.

Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention. As shown in fig. 4, the computer device of this embodiment includes: at least one processor (only one shown in fig. 4), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor executing the computer program performing any of the individual feature screening method steps described above.

The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 4 is merely an example of a computer device and is not intended to limit the computer device, and that a computer device may include more or fewer components than shown, or may combine certain components, or different components, such as may also include a network interface, a display screen, an input device, and the like.

The processor may be a CPU, but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory includes a readable storage medium, an internal memory, etc., where the internal memory may be the memory of the computer device, the internal memory providing an environment for the execution of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments may be an external storage device of the computer device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. that are provided on the computer device. Further, the memory may also include both internal storage units and external storage devices of the computer device. The memory is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs such as program codes of computer programs, and the like. The memory may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiment, and may be implemented by a computer program to instruct related hardware, and the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of the method embodiment described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

The present invention may also be implemented as a computer program product for implementing all or part of the steps of the method embodiments described above, when the computer program product is run on a computer device, causing the computer device to execute the steps of the method embodiments described above.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer device and method may be implemented in other manners. For example, the apparatus/computer device embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physically located, or may be distributed over a plurality of network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims

1. A feature screening method based on federal learning, the feature screening method comprising:

2. The feature screening method of claim 1, wherein the obtaining the training feature set of the local side to which the model to be trained belongs includes:

3. The feature screening method of claim 1, wherein the acquiring N update parameters sent by the central server after M times of federal training to obtain M update parameters of corresponding feature classes includes:

For each federal training, the local end trains the model to be trained to obtain N local updating parameters corresponding to N characteristic categories in the local end;

the N local updating parameters are sent to the central server;

for any feature category, the central server integrates local update parameters corresponding to the feature category to obtain update parameters corresponding to the central server, and the central server sends the update parameters to the local terminal;

and the local end performs M times of federal training on the model to be trained according to the received updated parameters to obtain M updated parameters of the corresponding feature categories.

4. The feature screening method of claim 3, wherein the integrating, by the central server, the local update parameter corresponding to the feature class for any feature class to obtain the update parameter corresponding to the central server includes:

when the local ends are multiple, the central server calculates the average value of the local update parameters, and the average value is used as the update parameter corresponding to the central server.

5. The feature screening method of claim 1, wherein the determining, for any feature class, a trend of the feature class corresponding to the update parameter according to the M update parameters of the feature class, to obtain a trend of the feature class corresponding to each feature class, includes:

For any feature class, calculating variances of M update parameters according to M update parameters of the feature class;

6. The feature screening method of claim 1, wherein the obtaining a preset change condition, comparing the change trend of all feature categories with the change condition, determining feature categories that do not satisfy the change condition, and removing corresponding feature quantities from the training feature set, further comprises:

7. Feature screening device based on federal study, characterized in that, the feature screening device includes:

8. The feature screening apparatus of claim 6, wherein the feature screening apparatus further comprises:

the obtaining module is used for removing the corresponding feature quantity from the training feature set to obtain a training feature set after feature screening;

and the training module is used for performing federal training on the model to be trained by using the screened characteristic training set to obtain a trained model corresponding to the model to be trained.

9. A computer device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, which processor implements the feature screening method according to any of claims 1 to 6 when the computer program is executed.

10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the feature screening method according to any one of claims 1 to 6.