CN117407718B - Training method, application method and system of battery replacement prediction model - Google Patents
Training method, application method and system of battery replacement prediction model Download PDFInfo
- Publication number
- CN117407718B CN117407718B CN202311726033.XA CN202311726033A CN117407718B CN 117407718 B CN117407718 B CN 117407718B CN 202311726033 A CN202311726033 A CN 202311726033A CN 117407718 B CN117407718 B CN 117407718B
- Authority
- CN
- China
- Prior art keywords
- battery
- training
- prediction model
- basic
- battery replacement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000008859 change Effects 0.000 claims abstract description 63
- 238000006243 chemical reaction Methods 0.000 claims abstract description 49
- 230000009471 action Effects 0.000 claims abstract description 27
- 230000002452 interceptive effect Effects 0.000 claims abstract description 19
- 230000002787 reinforcement Effects 0.000 claims description 24
- 238000013135 deep learning Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 230000005611 electricity Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 238000003860 storage Methods 0.000 description 7
- 210000004027 cell Anatomy 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0499—Feedforward networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/092—Reinforcement learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a training method, an application method and a system of a battery replacement prediction model, wherein the training method comprises the following steps: based on the basic characteristics of the battery, the basic characteristics of the user and the interactive characteristics, constructing and obtaining the power conversion state characteristics; discretizing the basic characteristics of the battery to construct a battery changing electric characteristic; constructing a training data set for model training based on the power change state features, the power change action features and the reward value features, wherein the reward value features are obtained in a questionnaire form; based on the training data set, training of the battery replacement prediction model is completed. Through the method and the device, the training of the battery replacement prediction model based on the graph network structure is realized for the first time, the battery replacement according with the user requirement can be accurately output in real time through the model, and the problem of how to reasonably recommend the battery replacement is solved.
Description
Technical Field
The application relates to the technical field of new energy, in particular to a training method, an application method and a system of a power conversion prediction model.
Background
The traditional battery power conversion system generally carries out the replacement of the battery of the electric vehicle based on the residual electric quantity of the battery and/or the capacity of the battery, such as randomly selecting a battery with highest electric quantity from a power conversion cabinet for replacement, or preferably selecting a battery with double high electric quantity and capacity for replacement, but under the actual electric vehicle power conversion scene, the battery recommendation is a very complex problem, and the scheme of judging whether the battery is suitable for a rider or not simply based on the residual electric quantity of the battery and/or the capacity of the battery is unreasonable, which easily leads to insufficient utilization of battery resources.
With the continuous development of machine learning technology, some schemes for performing power change recommendation based on a machine learning model also appear, for example, patent application number 201911386061.5 discloses a power change prediction method, and particularly based on estimated riding probability, estimated riding duration and current battery power, whether power change is needed is judged, so that factors considered in judging whether power change is needed are still few, and how to recommend a power change battery is not disclosed.
At present, no effective solution is proposed for the problem of reasonably recommending the battery replacement in the related art.
Disclosure of Invention
The embodiment of the application provides a training method, an application method and a system of a battery replacement prediction model, which are used for at least solving the problem of how to reasonably recommend a battery replacement in the related technology.
In a first aspect, an embodiment of the present application provides a method for training a power conversion prediction model, where the method includes:
constructing a power conversion state characteristic, wherein the power conversion state characteristic is based on a battery basic characteristic, a user basic characteristic and an interactive characteristic;
constructing a battery replacement operation feature, wherein the battery replacement operation feature is obtained by discretizing the basic feature of the battery;
constructing a training data set, wherein the training data set is based on the power change state characteristics, the power change action characteristics and the reward value characteristics, and the reward value characteristics are obtained through questionnaire survey;
and based on the training data set, completing the training of the power conversion prediction model.
In some of these embodiments, the constructing the training data set includes:
clustering the power change state features through a preset clustering algorithm to obtain a limited number of power change state features; clustering the electric change operation features through the preset clustering algorithm to obtain a limited electric change operation features;
and constructing a training data set for model training based on the limited power change state features, the limited power change operation features and the reward value features.
In some of these embodiments, the constructing the power change state feature includes:
extracting basic battery characteristics, basic user characteristics and interactive characteristics through a graph network structure;
and constructing and obtaining the power conversion state characteristic based on the battery basic characteristic, the user basic characteristic and the interactive characteristic.
In some of these embodiments, the training of the complete battery replacement prediction model includes:
inputting the training data set into a deep learning network and a reinforcement learning network in a power conversion prediction model;
and the parameters of the deep learning network and the reinforcement learning network are adjusted through feedback of a preset loss function until the value of the preset loss function is minimum, and training of the power conversion prediction model is completed.
In some of these embodiments, inputting the training data set into the deep learning network and the reinforcement learning network in the power-on prediction model comprises:
and taking 80% of the training data set as a training data set and 20% of the training data set as a test data set, and inputting the training data set into a deep Learning network and a reinforcement Learning network in a power conversion prediction model, wherein the deep Learning network is an MLP fully-connected neural network, and the reinforcement Learning network is a Q Learning algorithm network.
In some embodiments, adjusting parameters of the deep learning network and the reinforcement learning network through a preset loss function feedback until a value of the preset loss function is minimum, and completing training of the power conversion prediction model includes:
and measuring the difference between the predicted output and the real output of the power conversion prediction model by using a mean square error loss function, and adjusting the parameters of the deep learning network and the reinforcement learning network through feedback of an optimizer so as to minimize the value of the mean square error loss function, thereby completing the training of the power conversion prediction model, wherein the optimizer is an Adam optimizer.
In some of these embodiments, prior to constructing a training data set for model training based on the battery change status feature, the battery change action feature, and the reward value feature, the method includes:
and after the user returns the battery, acquiring the use experience of the user in the form of a questionnaire, and obtaining the rewarding value characteristic based on the use experience.
In some of these embodiments, before constructing the battery change status feature based on the battery base feature, the user base feature, and the interactivity feature, the method comprises:
the method comprises the steps of obtaining basic battery characteristics, user basic characteristics and interactive characteristics, wherein the basic battery characteristics comprise the quantity of battery cells, the electric quantity of the battery, the capacity of the battery, the SOC value of the battery, the temperature of the battery and the voltage difference between the battery cells, the basic user characteristics comprise gender, age and place of birth, and the interactive characteristics comprise the accumulated electric quantity of the battery and the mileage of riding of the battery.
In a second aspect, an embodiment of the present application provides a method for applying a power conversion prediction model, where the method is performed based on the power conversion prediction model obtained by the training method in any one of the first aspect, and the method includes:
and selecting a target battery through the battery replacement prediction model based on the user basic information and the battery basic information in the battery replacement cabinet, which are acquired in real time, wherein the target battery is used for replacing the battery for the user.
In a third aspect, an embodiment of the present application provides a system for a power conversion prediction model, where the system is configured to implement the training method according to any one of the first aspect, and the system includes:
the model training module is used for training a battery replacement prediction model;
and the model application module is used for selecting a target battery through the power change prediction model, wherein the target battery is used for changing power for a user.
Compared with the related art, the training method, the application method and the system of the battery replacement prediction model provided by the embodiment of the application, wherein the training method is used for obtaining the battery replacement state characteristics through construction based on the basic characteristics of the battery, the basic characteristics of the user and the interactive characteristics; discretizing the basic characteristics of the battery to construct a battery changing electric characteristic; constructing a training data set for model training based on the power change state features, the power change action features and the reward value features, wherein the reward value features are obtained in a questionnaire form; based on the training data set, the training of the power conversion prediction model is completed, the training of the power conversion prediction model based on the graph network structure is realized for the first time, and the power conversion battery meeting the user requirement can be accurately output in real time through the model, so that the problem of how to reasonably recommend the power conversion battery is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flow chart of steps of a battery replacement prediction model training method according to an embodiment of the present application;
FIG. 2 is a flow chart diagram of a battery replacement prediction model training and application method according to an embodiment of the present application;
fig. 3 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described and illustrated below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden on the person of ordinary skill in the art based on the embodiments provided herein, are intended to be within the scope of the present application.
It is apparent that the drawings in the following description are only some examples or embodiments of the present application, and it is possible for those of ordinary skill in the art to apply the present application to other similar situations according to these drawings without inventive effort. Moreover, it should be appreciated that while such a development effort might be complex and lengthy, it would nevertheless be a routine undertaking of design, fabrication, or manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as having the benefit of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by those of ordinary skill in the art that the embodiments described herein can be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar terms herein do not denote a limitation of quantity, but rather denote the singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein refers to two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., "a and/or B" may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
In the related art, the conventional battery power conversion system generally has the following two decision modes:
1. the system randomly selects a battery with the highest electric quantity from the battery changing cabinet for recommendation.
Therefore, the decision mode only considers the electric quantity information of the battery, does not consider the delivery date, capacity, performance and the like of the battery, and secondly, the system does not consider the daily power change habit of a rider, namely the daily power change requirement; the two reasons can lead to inconsistent system recommended batteries and expected batteries of a rider, and reduce the power exchange experience of the rider.
2. The system considers the battery and the capacity information comprehensively, and preferably one battery and a battery with double high capacity are selected.
It can be seen that this decision mode maximally meets the daily demands of the rider, but maximally meets the daily demands of the rider may result in uneven distribution of the batteries in the battery changing cabinet, even if the current rider gets the best battery, when the next rider changes the battery, there is a possibility that the best battery is not obtained, or the battery he wants is not obtained, in which case, there is no way to adapt to the battery changing demands of each rider. Second, mass production and distribution of high-quality batteries increases the production costs and pressures of the enterprise, so a more rational solution is needed to solve this problem.
In summary, in the actual power-changing scenario, battery recommendation is a very complex problem, and it is unreasonable to judge whether the battery is suitable for the rider or not by unilaterally depending on the battery power or the battery capacity index, and the resource cannot be fully utilized. Therefore, in the operation process of the power conversion system, a strategy model with the combined action of multiple factors, namely a deep reinforcement learning data model based on a graph network is required to be provided so as to meet the personalized requirements of riders and protect the riders. The scheme provided by the application outputs the battery replacement most suitable for users based on the deep reinforcement learning model of the graph network, collects all factors affecting battery endurance as much as possible, utilizes the factors to estimate the battery value in the battery replacement cabinet at each point in the city, adopts the estimation method based on the deep reinforcement learning of the graph network, fully considers the daily behavior habit of a rider and the battery parameter information, reduces the production cost of enterprises, and provides safe and reasonable endurance guarantee for the power replacement of the rider.
The following is a specific embodiment of a method, an application method and a system for training a battery replacement prediction model provided in the present application.
An embodiment of the present application provides a training method of a power conversion prediction model, and fig. 1 is a step flowchart of the training method of the power conversion prediction model according to an embodiment of the present application, as shown in fig. 1, and the method includes the following steps:
step S102, constructing and obtaining a power conversion state characteristic based on basic characteristics of a battery, basic characteristics of a user and interactive characteristics;
specifically, step S102, fig. 2 is a flow chart of a power-change prediction model training and application method according to an embodiment of the present application, as shown in fig. 2, three elements of a power-change prediction model (deep reinforcement learning model) are first required to be reinforced by a graph network structure: a power change State feature State (S), a power change Action feature Action (A) and a Reward value feature Reward (R).
For the power change State feature State (S): based on the basic battery characteristics, the basic user characteristics and the interactive characteristics, combining the three types of characteristics, constructing a power conversion State characteristic State (S), wherein the basic battery characteristics comprise the number of battery cells, the battery power quantity, the battery capacity, the battery SOC value, the temperature of the battery, the voltage difference among the battery cells and the like; the user basic characteristics include gender, age, gender, birth place, etc.; the interactive features of the battery with the rider include the use of battery power and the mileage the rider uses to ride on the battery, among others.
Before step S102, the method further includes step S101, extracting a battery basic feature, a user basic feature and an interactive feature through a graph network structure, where the graph network structure includes graph nodes and edges, the graph nodes are user basic information or battery basic information, and the edges are user and battery interactive information; and constructing and obtaining the power conversion state characteristic based on the battery basic characteristic, the user basic characteristic and the interactive characteristic. The basic characteristics of the battery comprise the quantity of battery cells, the quantity of battery electricity, the capacity of the battery, the SOC value of the battery, the temperature of the battery and the voltage difference between the battery cells, the basic characteristics of the user comprise gender, age and birth place, and the interactive characteristics comprise the accumulated electricity consumption of the battery and the riding mileage of the battery.
It should be noted that, the application fully utilizes multiple feature dimension information, including battery basic information, rider basic information and rider-battery interaction information, and utilizes the network structure of the graph to deeply explore hidden variables contained in the information, so that the battery basic feature, the user basic feature and the interaction feature can be extracted more comprehensively and accurately.
Step S104, discretizing the basic characteristics of the battery to construct and obtain the battery replacement operation characteristics;
specifically, as shown in fig. 2, step S104 is performed for the commutation Action feature Action (a): and discretizing the continuity characteristic by taking the basic characteristic of the battery as a reference, and constructing and obtaining the multi-dimensional battery change characteristic Action (A).
After step S104, the method further includes step S105, after the user returns the battery, obtaining a user experience in the form of a questionnaire, and obtaining a bonus value feature based on the use experience.
Specifically, as shown in fig. 2, for the bonus value feature Reward (R): with the discrete prize value distribution, if the rider returns to the battery, "satisfaction" is filled in by the questionnaire, a +10 prize value is given, if the rider returns to the battery, "dissatisfaction" is filled in by the questionnaire, a-10 prize value is given, and if the rider has no feedback, a 0 prize value is given. Further, the acquisition of the Reward value feature Reward may also consider a subsequent rate, in particular, the subsequent rate will be counted according to the city dimension, which is equal to the number of current month's continuous rate/the number of passengers in the previous month's operation, and the final Reward value reward=city's continuous rate, such as, for example, a city continuous rate is: 0.8, the satisfaction of questionnaires is-10,0,10 respectively; the final prize value Reward is-8, 0, 8, respectively.
Step S106, a training data set for model training is constructed based on the power change state characteristics, the power change action characteristics and the rewarding value characteristics, wherein the rewarding value characteristics are obtained in a questionnaire form;
step S106 specifically further includes the steps of:
step S1061, clustering the power change state features through a preset clustering algorithm to obtain a limited number of power change state features; clustering the power change action features through a preset clustering algorithm to obtain a limited number of power change action features;
specifically, as shown in fig. 2, step S1061 quantifies a limited number of power change State features State (S) and power change Action features actions (a) using a k-means++ algorithm:
it should be noted that too many power change State features State (S) or power change Action features (a) consume a large amount of operation resources, which makes reinforcement learning difficult to train, so that the k-means++ algorithm is adopted to cluster the multi-dimensional power change State features State (S) and power change Action features (a) respectively. The set parameter k represents the class number of the cluster, the value range is preferably [2,50], the Euclidean distance index is utilized to evaluate the quality of the k value, the k value with the minimum Euclidean distance is finally selected (namely, the condition of good clustering effect is obtained), the optimal k value of the power change State characteristic State (S) is 26, the optimal k value of the power change State characteristic Action (A) is 10, and the number of the final configuration State Action pairs < S, A > is 260.
Step S1062, a training data set for model training is constructed based on the limited number of battery change status features, the limited number of battery change action features, and the bonus value features.
In step S1062, based on the limited power change status features, the limited power change operation features, and the rewarding value features, corresponding power change status information, power change operation information, and rewarding value information are obtained, so as to construct a training data set for model training.
It should be noted that, training and application of the deep reinforcement learning model (power conversion prediction model) are based on a graph network data set, a deep learning network and a reinforcement learning network, wherein the training data set is constructed through interaction information of a rider, an order and a battery, different riders may use the same battery or different batteries, the basic information of the rider and the basic information of the battery form a graph node, and the interaction information of the rider and the battery form an edge. According to the scheme, compared with a scheme of recommending batteries according to battery electric quantity only, the cost is lower, the productivity pressure is smaller, the supply and demand relationship is more balanced, and compared with a scheme of recommending batteries according to battery electric quantity and capacity, consideration factors are more comprehensive, and the supply side pressure of a high-quality battery is further reduced.
Step S108, based on the training data set, training of the power-change prediction model is completed.
Step S108 further includes the steps of:
step S1081, inputting the training data set into a deep learning network and a reinforcement learning network in the power-change prediction model;
specifically, as shown in fig. 2, in step S1081, 80% of the training data set is used as the training data set and 20% of the training data set is used as the test data set, and the training data set is input to the deep Learning network and the reinforcement Learning network in the power-change prediction model, where the deep Learning network is an MLP fully-connected neural network, and the reinforcement Learning network is a Q Learning algorithm network.
The graph nodes and edges in the training data set are output to the downstream deep learning network by graph network aggregation. The deep learning network is an MLP fully-connected neural network, and the MLP consists of four layers of neural networks, namely an input layer (one layer), a hidden layer (two layers) and an output layer (one layer). The input layer is composed of 26 x 2 = 52 neurons, the first hidden layer is composed of 128 neurons, the second hidden layer is composed of 64 neurons, the output layer is composed of 10 neurons (representing the probability of adopting different power change characteristics (A) under the current power change State characteristic State (S)), the input layer and the hidden layer and the output layer are activated by adopting a leaky_Relu activation function, and the output layer is connected with a Softmax activation function. The reinforcement Learning model selects a Q Learning algorithm network, and the Q value calculation formula is as follows:
Q(s, a)←Q(s, a)+α(r + γQ w- (s', argmax a' Q w- (s', a'))
wherein Q (s, a) represents the current state action pair<S,A>Prize value, Q w- (s ', a') represents the prize value w-estimated using the deep neural network MLP parameter, α is the learning rate, R is the current step R function value (i.e. the return desire), and γ is the decay factor. 80% of the data are used for training, the other 20% are used for evaluating the model convergence effect, and all the data are in a row index disorder order so as to balance the randomness of the selection of the power change State characteristic State (S).
And S1082, feeding back and adjusting parameters of the deep learning network and the reinforcement learning network through a preset loss function until the value of the preset loss function is minimum, and finishing training of the power-change prediction model.
In step S1082, specifically, the difference between the predicted output and the actual output of the power conversion prediction model is measured by a mean square error loss function, and parameters of the deep learning network and the reinforcement learning network are adjusted by an optimizer, which is an Adam optimizer, in a feedback manner so as to minimize the value of the mean square error loss function, thereby completing the training of the power conversion prediction model.
It should be noted that, in the training stage of the power conversion prediction model (deep learning network+reinforcement learning network), a Loss function Loss and an Optimizer are constructed, the Loss adopts a mean square error Loss function, and the expression is:
wherein N represents the number of training samples, s i For the ith power change State feature State (S), a i Action (A), Q) which is characteristic of the ith commutation w For a state action pair<S i ,A i >Is a prize value for (a). The final goal of the deep reinforcement learning network training is to minimize the MSE_loss value; the Optimizer employs an Adam Optimizer with a learning rate learning_rate of 0.0002. The algorithm Model is trained for 1000 rounds until the MSE_loss value is reduced to the lowest, the long-term accumulated Q value is ensured to be maximized, and a power-conversion prediction Model is stored for the next strategy generation.
Through the steps in the embodiment of the application, the training of the power conversion prediction model based on the graph network structure is realized for the first time, and the power conversion battery meeting the user requirement can be accurately output in real time through the model, so that the problem of how to reasonably recommend the power conversion battery is solved.
The embodiment of the application provides an application method of a battery replacement prediction model, the method comprises the steps of
And selecting a target battery through a battery replacement prediction model based on the user basic information and the battery basic information in the battery replacement cabinet, which are acquired in real time, wherein the target battery is used for replacing the battery for the user.
Specifically, according to a battery replacement prediction Model stored in a network training stage, basic information of a rider and basic information of each battery in a cabinet during battery replacement, an index battery item with the maximum expected Q value is selected, and the index battery item is provided for the rider to replace the battery.
In addition, the battery replacement prediction Model in the application also has an automatic updating function, specifically, three elements of a battery replacement State feature State (S), a battery replacement Action feature Action (A) and a Reward value feature Reward (R) are built according to daily updating data updating, and periodic parameter updating is carried out on the battery replacement prediction Model.
It should be noted that the steps illustrated in the above-described flow or flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.
The embodiment of the application provides a power conversion system based on a graph network, which comprises a model training module and a model application module;
the model training module is used for constructing and obtaining the power conversion state characteristics according to the basic characteristics of the battery, the basic characteristics of the user and the interactive characteristics; discretizing the basic characteristics of the battery to construct a battery changing electric characteristic; constructing a training data set for model training based on the power change state features, the power change action features and the reward value features, wherein the reward value features are obtained in a questionnaire form; based on the training data set, completing the training of the power conversion prediction model;
and the model application module is used for selecting a target battery through the battery replacement prediction model according to the user basic information and the battery basic information in the battery replacement cabinet, which are acquired in real time, wherein the target battery is used for replacing the battery for the user.
Through the model training module and the model application module in the embodiment of the application, the training of the power conversion prediction model based on the graph network structure is realized for the first time, and the power conversion battery meeting the user requirement can be accurately output in real time through the model, so that the problem of how to reasonably recommend the power conversion battery is solved.
The above-described respective modules may be functional modules or program modules, and may be implemented by software or hardware. For modules implemented in hardware, the various modules described above may be located in the same processor; or the above modules may be located in different processors in any combination.
The present embodiment also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.
It should be noted that, specific examples in this embodiment may refer to examples described in the foregoing embodiments and alternative implementations, and this embodiment is not repeated herein.
In addition, in combination with the training and application method of the power conversion prediction model in the above embodiment, the embodiment of the application may be implemented by providing a storage medium. The storage medium has a computer program stored thereon; the computer program, when executed by a processor, implements the training and application method of any of the power-change prediction models of the above embodiments.
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method of training and applying a battery replacement prediction model. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
In one embodiment, fig. 3 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application, and as shown in fig. 3, an electronic device is provided, which may be a server, and an internal structure diagram thereof may be shown in fig. 3. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected by an internal bus, where the non-volatile memory stores an operating system, computer programs, and a database. The processor is used for providing computing and control capability, the network interface is used for communicating with an external terminal through network connection, the internal memory is used for providing environment for the operation of an operating system and a computer program, the computer program is executed by the processor to realize a training and application method of a power conversion prediction model, and the database is used for storing data.
It will be appreciated by those skilled in the art that the structure shown in fig. 3 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the electronic device to which the present application is applied, and that a particular electronic device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It should be understood by those skilled in the art that the technical features of the above-described embodiments may be combined in any manner, and for brevity, all of the possible combinations of the technical features of the above-described embodiments are not described, however, they should be considered as being within the scope of the description provided herein, as long as there is no contradiction between the combinations of the technical features.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (5)
1. A method of training a battery replacement prediction model, the method comprising:
extracting basic battery characteristics, basic user characteristics and interactive characteristics through a graph network structure, wherein the graph network structure comprises graph nodes and edges, the graph nodes are basic user information or basic battery information, and the edges are interactive information between a user and a battery;
the method comprises the steps of constructing and obtaining a battery replacement state characteristic based on a battery basic characteristic, a user basic characteristic and an interactive characteristic, wherein the battery basic characteristic comprises the number of battery cells, the battery electricity quantity, the battery capacity, a battery SOC value, the battery temperature and the voltage difference between the battery cells, the user basic characteristic comprises gender, age and birth place, and the interactive characteristic comprises the accumulated electricity consumption of a battery and the riding mileage of the battery;
constructing a battery replacement operation feature, wherein the battery replacement operation feature is obtained by discretizing the basic feature of the battery;
constructing a training data set, wherein the training data set is based on the power change state characteristics, the power change action characteristics and the reward value characteristics, and the reward value characteristics are obtained through questionnaire survey;
and based on the training data set, completing the training of the power conversion prediction model.
2. The method of claim 1, wherein constructing the training data set comprises:
clustering the power change state features through a preset clustering algorithm to obtain a limited number of power change state features; clustering the electric change operation features through the preset clustering algorithm to obtain a limited electric change operation features;
and constructing a training data set for model training based on the limited power change state features, the limited power change operation features and the reward value features.
3. The method of claim 1, wherein the completing the training of the battery replacement prediction model comprises:
inputting the training data set into a deep learning network and a reinforcement learning network in a power conversion prediction model;
and the parameters of the deep learning network and the reinforcement learning network are adjusted through feedback of a preset loss function until the value of the preset loss function is minimum, and training of the power conversion prediction model is completed.
4. A method for applying a battery replacement prediction model, wherein the method is performed based on the battery replacement prediction model obtained by the training method according to any one of claims 1 to 3, and the method comprises:
and selecting a target battery through the battery replacement prediction model based on the user basic information and the battery basic information in the battery replacement cabinet, which are acquired in real time, wherein the target battery is used for replacing the battery for the user.
5. A system of a battery replacement prediction model for implementing the training method of any one of claims 1 to 3, the system comprising:
the model training module is used for training a battery replacement prediction model;
and the model application module is used for selecting a target battery through the power change prediction model, wherein the target battery is used for changing power for a user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311726033.XA CN117407718B (en) | 2023-12-15 | 2023-12-15 | Training method, application method and system of battery replacement prediction model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311726033.XA CN117407718B (en) | 2023-12-15 | 2023-12-15 | Training method, application method and system of battery replacement prediction model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117407718A CN117407718A (en) | 2024-01-16 |
| CN117407718B true CN117407718B (en) | 2024-03-26 |
Family
ID=89500390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311726033.XA Active CN117407718B (en) | 2023-12-15 | 2023-12-15 | Training method, application method and system of battery replacement prediction model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117407718B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696370A (en) * | 2020-06-16 | 2020-09-22 | 西安电子科技大学 | Traffic light control method based on heuristic deep Q network |
| CN111861647A (en) * | 2020-07-03 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Method and system for recommending boarding points |
| CN111966484A (en) * | 2020-06-23 | 2020-11-20 | 北京大学 | Cluster resource management and task scheduling method and system based on deep reinforcement learning |
| CN114004149A (en) * | 2021-10-29 | 2022-02-01 | 深圳市商汤科技有限公司 | Intelligent agent training method and device, computer equipment and storage medium |
| CN115742865A (en) * | 2022-11-01 | 2023-03-07 | 江苏正力新能电池技术有限公司 | Electric automobile thermal management method and system based on mileage prediction |
| CN116767024A (en) * | 2023-04-14 | 2023-09-19 | 联合汽车电子有限公司 | Battery equalization method and device based on reinforcement learning |
| CN116878535A (en) * | 2023-09-05 | 2023-10-13 | 杭州宇谷科技股份有限公司 | Intelligent power conversion guiding method and system based on hybrid time sequence network |
| WO2023207663A1 (en) * | 2022-04-29 | 2023-11-02 | 阿里云计算有限公司 | Traffic scheduling method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11707197B2 (en) * | 2017-12-22 | 2023-07-25 | Resmed Sensor Technologies Limited | Apparatus, system, and method for physiological sensing in vehicles |
-
2023
- 2023-12-15 CN CN202311726033.XA patent/CN117407718B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696370A (en) * | 2020-06-16 | 2020-09-22 | 西安电子科技大学 | Traffic light control method based on heuristic deep Q network |
| CN111966484A (en) * | 2020-06-23 | 2020-11-20 | 北京大学 | Cluster resource management and task scheduling method and system based on deep reinforcement learning |
| CN111861647A (en) * | 2020-07-03 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Method and system for recommending boarding points |
| CN114004149A (en) * | 2021-10-29 | 2022-02-01 | 深圳市商汤科技有限公司 | Intelligent agent training method and device, computer equipment and storage medium |
| WO2023207663A1 (en) * | 2022-04-29 | 2023-11-02 | 阿里云计算有限公司 | Traffic scheduling method |
| CN115742865A (en) * | 2022-11-01 | 2023-03-07 | 江苏正力新能电池技术有限公司 | Electric automobile thermal management method and system based on mileage prediction |
| CN116767024A (en) * | 2023-04-14 | 2023-09-19 | 联合汽车电子有限公司 | Battery equalization method and device based on reinforcement learning |
| CN116878535A (en) * | 2023-09-05 | 2023-10-13 | 杭州宇谷科技股份有限公司 | Intelligent power conversion guiding method and system based on hybrid time sequence network |
Non-Patent Citations (3)
| Title |
|---|
| A Game-Based Battery Swapping Station Recommendation Approach for Electric Vehicles;Lili Ran .etal;IEEE Transactions on Intelligent Transportation Systems;20230504;第24卷(第9期);全文 * |
| 基于深度强化学习的无人艇航行控制;张法帅 等;计测技术;20180630(第S1期);全文 * |
| 武玉伟 编著.深度学习基础与应用.北京理工大学出版社,2020,15. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117407718A (en) | 2024-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sun et al. | Learning multiple-question decision trees for cold-start recommendation | |
| Salant et al. | Statistical inference in games | |
| CN104779680B (en) | Electric automobile reservation charging control method and device | |
| Cai et al. | Learning collaborative filtering and its application to people to people recommendation in social networks | |
| CN105405061A (en) | Evaluation method of power supply reliability of distribution network based on demand response | |
| CN110766438B (en) | Method for analyzing user behavior of power grid user through artificial intelligence | |
| Niknam et al. | Impact of heat, power and hydrogen generation on optimal placement and operation of fuel cell power plants | |
| CN117407718B (en) | Training method, application method and system of battery replacement prediction model | |
| Li et al. | Deep reinforcement learning based explainable pricing policy for virtual storage rental service | |
| CN111667298A (en) | User personalized demand prediction method and system | |
| CN115329907A (en) | Electric load completion method and system based on DBSCAN clustering | |
| CN110163717A (en) | A kind of Products Show method, apparatus, computer readable storage medium and server | |
| CN113300375B (en) | Temperature control load demand response control method considering user participation intention difference | |
| CN116316537A (en) | Transmission line operation control method, device, equipment, medium and program product | |
| Ma et al. | Incentive-based demand response under incomplete information based on the deep deterministic policy gradient | |
| CN114331556A (en) | Energy service provider benefit evaluation method, system, device and storage medium | |
| CN113610599A (en) | Electric power spot-shipment day-ahead market auxiliary quotation method | |
| El Alami et al. | Toward an effective hybrid collaborative filtering: a new approach based on matrix factorization and heuristic-based neighborhood | |
| Karahodza et al. | Feature enhanced time-aware recommender system | |
| CN116707019B (en) | Electric quantity distribution method, system and computer equipment for daily electric power market | |
| CN112510679A (en) | Distributed energy storage optimization scheduling method and device, computer equipment and storage medium | |
| CN116542498A (en) | Battery scheduling method, system, device and medium based on deep reinforcement learning | |
| CN117217364A (en) | Power cost value prediction method, device, equipment, medium and computer product | |
| CN117473122A (en) | Market participation energy storage profit optimization method and system based on graph calculation | |
| CN117391748A (en) | EV user customized electricity price package design method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |