CN113159281A - Data processing method and data processing device - Google Patents
Data processing method and data processing device Download PDFInfo
- Publication number
- CN113159281A CN113159281A CN202110322243.7A CN202110322243A CN113159281A CN 113159281 A CN113159281 A CN 113159281A CN 202110322243 A CN202110322243 A CN 202110322243A CN 113159281 A CN113159281 A CN 113159281A
- Authority
- CN
- China
- Prior art keywords
- loss function
- output layer
- prediction model
- time length
- task information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 86
- 238000003672 processing method Methods 0.000 title abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 73
- 230000006870 function Effects 0.000 claims description 94
- 238000000034 method Methods 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 6
- 235000012054 meals Nutrition 0.000 description 6
- 235000013305 food Nutrition 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
- G06Q10/0838—Historical data
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Development Economics (AREA)
- Health & Medical Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention discloses a data processing method and a data processing device. After acquiring the task information of a plurality of historical tasks including the actual completion time and the actual processing time of the historical tasks, the embodiment of the invention trains the prediction model based on the task information of the plurality of historical tasks until the loss function of the actual completion time and the loss function of the actual processing time corresponding to the prediction model are converged. The prediction model comprises a shared structure and an unshared structure, the unshared structure comprises a first output layer and a second output layer of the prediction model, the first output layer takes actual completion time as a training target, and the second output layer takes actual processing time as a training target. The embodiment of the invention trains the prediction model simultaneously based on a plurality of training targets, thereby improving the training efficiency of the model.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and a data processing apparatus.
Background
With the continuous popularization of the internet and logistics industry and the rapid development of the online shopping industry, more and more people select to purchase goods of the heart instrument through the online shopping platform. For the logistics distribution industry, the article distribution time length is an important factor influencing the satisfaction degree of users, so how to determine the article distribution time length is very important for the logistics distribution industry. The article delivery task often includes a plurality of links, and the existing training mode of the model for predicting the article delivery duration can only train the model based on the duration consumed by one link in the article delivery task, so the training efficiency of the model by the training mode is not high.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method and a data processing apparatus, which are used for training a model in a multi-task learning manner, so as to improve the training efficiency of the model.
In a first aspect, an embodiment of the present invention provides a data processing method, where the method includes:
acquiring a task information set, wherein the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
training a prediction model by taking the actual completion time length and the actual processing time length as training targets on the basis of the historical task information until a loss function of the prediction model converges, wherein the loss function comprises a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion time length, and the second loss function is a loss function corresponding to the actual processing time length;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
Preferably, the method further comprises:
acquiring target task information of a target task;
and determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
Preferably, the first output layer and the second output layer are all fully connected layers.
Preferably, the predictive model is a neural network based factorisation machine.
Preferably, the second loss function is a regression loss function with quantiles.
In a second aspect, an embodiment of the present invention provides a data processing apparatus, where the apparatus includes:
the system comprises a first information acquisition unit, a second information acquisition unit and a processing unit, wherein the first information acquisition unit is used for acquiring a task information set, the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
a model training unit, configured to train a prediction model with the actual completion duration and the actual processing duration as training targets based on the historical task information until a loss function of the prediction model converges, where the loss function includes a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion duration, and the second loss function is a loss function corresponding to the actual processing duration;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
Preferably, the apparatus further comprises:
the second information acquisition unit is used for acquiring target task information of the target task;
and the time length prediction unit is used for determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
Preferably, the first output layer and the second output layer are all fully connected layers.
In a third aspect, the present invention provides a computer-readable storage medium on which computer program instructions are stored, the computer program instructions, when executed by a processor, implementing the method according to any one of the first aspect.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, the memory being configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of the first aspect.
After acquiring the task information of a plurality of historical tasks including the actual completion time and the actual processing time of the historical tasks, the embodiment of the invention trains the prediction model based on the task information of the plurality of historical tasks until the loss function of the actual completion time and the loss function of the actual processing time corresponding to the prediction model are converged. The prediction model comprises a shared structure and an unshared structure, the unshared structure comprises a first output layer and a second output layer of the prediction model, the first output layer takes actual completion time as a training target, and the second output layer takes actual processing time as a training target. The embodiment of the invention trains the prediction model simultaneously based on a plurality of training targets, thereby improving the training efficiency of the model.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
FIG. 1 is a flow chart of a data processing method of a first embodiment of the present invention;
FIG. 2 is a schematic diagram of a prediction model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a data processing apparatus according to a second embodiment of the present invention;
fig. 4 is a schematic diagram of an electronic device according to a third embodiment of the invention.
Detailed Description
The present invention will be described below based on examples, but the present invention is not limited to only these examples. In the following detailed description of the present invention, certain specific details are set forth. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details. Well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present invention.
Further, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.
Unless the context clearly requires otherwise, throughout the description, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".
In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the embodiment of the present invention, the historical task and the target task are taken as examples in the technical field of takeout delivery, but it is easily understood by those skilled in the art that the method of the present embodiment is also applicable when the historical task and the target task are other logistics delivery tasks or tasks with multiple links.
The object distribution task often includes multiple links. Taking a takeaway delivery task as an example, the takeaway delivery task generally comprises several links of ordering by a user, ordering by a merchant, taking a meal by a commodity, ordering by a delivery person, taking a meal by a delivery person, and delivering a meal by a delivery person, and the existing training method for the model of the commodity delivery duration can only be based on one link of the commodity delivery task, for example, after the order is taken by the merchant, the duration consumed by the link of the commodity serving is used for training the model, and the training method does not relate to training the model based on the durations consumed by other links, so the training efficiency of the training method of the model is not high.
Fig. 1 is a flowchart of a data processing method according to a first embodiment of the present invention. As shown in fig. 1, the method of the present embodiment includes the following steps:
and step S100, acquiring a task information set.
After a user orders a takeout food item through the online takeout platform to issue a takeout order, the online takeout platform generates a takeout logistics distribution task, and meanwhile, various information of the takeout order and data generated in various links in the takeout logistics distribution task, such as the order taking time of a merchant and the order taking time of distribution personnel, are issued to the online takeout platform, so that a platform side server (hereinafter also referred to as a server) can acquire order information of the takeout order including the generation time of the takeout order (also referred to as the order taking time of the user) from the online takeout platform and store the order information of the takeout order in a database.
In this embodiment, the server may acquire order information of take orders generated by the online take-out platform in the current period from the database at a predetermined period (for example, every two weeks) as the task information set. The task information set is used for multi-task learning, namely training a prediction model. The set of task information may include historical task information for a plurality of historical tasks, and the historical task information includes at least an actual completion time length and an actual processing time length for the historical tasks. The actual completion time length is used for representing the time length consumed from task release to task completion, namely the time length consumed from ordering by the user to taking the takeaway food by the user; the actual processing time length is used for representing the time length consumed by the order processing personnel from the processing task of accepting the takeaway order to the completion of the processing of the takeaway order, namely the time length consumed by the merchant from the order acceptance to the meal delivery (namely, the takeaway meal preparation), or the time length consumed by the delivery personnel from the order acceptance to the delivery completion.
For example, the predetermined period is two weeks, the current period is 3 months and 1 day to 3 months and 14 days, the server may obtain, from the database, order information of take-out orders generated by the online take-out platform in 3 months and 1 day to 3 months and 14 days as a set of task information, and each of the order information includes a time length consumed by a corresponding take-out order from an order placing time to a time when a user takes a take-out food item and a time length consumed by a delivery person from the taking of the take-out order to the completion of delivery (that is, the user takes the take-out food item).
Optionally, the order information of the take-away order may further include other information, such as a merchant score, a distance between a location of the merchant and an address set by the user, a current day weather, whether to be holiday or not, an order price, a score of a delivery person, a timeout rate of the delivery person, and the like, which is not specifically limited in this embodiment.
And S200, training the prediction model by taking the actual completion time length and the actual processing time length as training targets based on the historical task information until the loss function of the prediction model is converged.
For the same take-away order, the corresponding actual completion duration and actual processing duration have a correlation, that is, the actual completion duration and actual processing duration may be affected by the same information, for example, the longer the time for a distributor to obtain a take-away meal of any take-away order, the longer the actual completion duration and actual processing duration corresponding to the take-away order. Therefore, in this embodiment, the correlation between the actual completion time and the actual processing time is learned by means of multi-task learning, which is a process of training the prediction model. Multi-task learning (multi task learning) is a kind of integrated learning algorithm, and a plurality of tasks with relevance (in this embodiment, training tasks with actual completion duration and actual processing duration) are trained simultaneously, so that the prediction model has better generalization capability, that is, higher accuracy and stronger adaptability.
Fig. 2 is a schematic structural diagram of a prediction model according to an embodiment of the present invention. It will be readily appreciated that the number of layers of the predictive model shown in fig. 2 is merely illustrative. As shown in fig. 2, the predictive model has a shared structure 21 and an unshared structure 22. The shared structure 21 is also a structure shared by two training tasks, namely the actual completion time length and the actual processing time length, and the weights in the shared structure 21 are common. Unshared structure 22 is a structure unique to each training task, each training task corresponding to a portion of the unshared structure. In this embodiment, the output layers of the non-shared structure, i.e., the prediction model, specifically include a first output layer 221 with the actual completion duration as the training target and a second output layer 222 with the actual processing duration as the training target. In this embodiment, the first output layer 221 and the second output layer 222 are all fully connected layers.
In this embodiment, the prediction model may be deep fm (a factor Machine Based Neural Network). The Deep learning model is composed of two structures, namely a Deep Neural Network (DNN) structure and a Factorization Machine (FM) structure, namely, a shared structure of the prediction model is the FM structure, and a non-shared structure of the prediction model is the DNN structure. The deep FM can learn the feature intersection of the high-dimensional features and the low-dimensional features at the same time, and has high prediction accuracy. Alternatively, the prediction model may also be other models, such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), and the like, and this embodiment is not limited in particular.
The prediction model of this embodiment has two training targets, so the loss function of the prediction model may include two parts, one part is a loss function (i.e., a first loss function) corresponding to the actual completion duration, and the other part is a loss function (i.e., a second loss function) corresponding to the actual processing duration. In this embodiment, the first loss function and the second loss function may be both regression loss functions, and specifically may be MAE (Mean Absolute Error) loss functions. And generated in order to balance the merchant or distributor in processing the take-out orderIn the embodiment, the second loss function is adjusted through quantiles to increase punishment for a smaller value in the processing duration output by the prediction model and decrease punishment for a larger value. In particular, the first Loss function Loss1Can be expressed by the following formula:
wherein m is the number of historical tasks in the task information set, y1，iIs the actual completion time length, y 'corresponding to the ith historical task'1，iAnd the completion time length corresponding to the ith historical task output by the prediction model. The second Loss function Loss2 may be expressed by the following equation:
wherein, y2，iIs the actual processing time length, y 'corresponding to the ith historical task'2，iFor the processing time length corresponding to the ith historical task output by the prediction model, α is a quantile, and in this embodiment, α may be 0.8. In this embodiment, the activation functions adopted in the first output layer and the second output layer of the prediction model may both be sigmoid functions, so that the completion time length y 'corresponding to the ith historical task output by the prediction model'1，iCan be expressed by the following formula:
wherein the content of the first and second substances,
output of FM structure for prediction modelThe output values corresponding to the i historical tasks,
and outputting an output value corresponding to the ith historical task for the first output layer of the prediction model. Similarly, the processing time length y corresponding to the ith historical task output by the prediction model2，iCan be expressed by the following formula:
wherein the content of the first and second substances,
the output value corresponding to the ith historical task output for the FM structure of the prediction model,
and outputting an output value corresponding to the ith historical task for the second output layer of the prediction model.
In this embodiment, the loss function of the prediction model may be the sum of the first loss function and the second loss function, and therefore, in the training process of the prediction model, when the loss functions converge, it may be considered that both the first loss function and the second loss function are converged.
Optionally, in this embodiment, the server may update the prediction model according to the task information sets obtained at each period at a predetermined period, so as to ensure the accuracy of the prediction model.
Optionally, after training the prediction model is completed, the method of this embodiment may further include the following steps:
step S300, target task information of the target task is obtained.
In this embodiment, the target task may be a newly generated takeout order acquired by the server from the online takeout platform, and the target task information may include a merchant score corresponding to the target takeout order, a distance between a location of the merchant and an address set by the user, a weather of the current day, whether to save or leave a holiday, an order price, a score of a delivery person, a timeout rate of the delivery person, and the like.
And step S400, determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
In this step, the server may use the target task information of the target task as an input of the prediction model, so that the prediction model may simultaneously output the predicted completion duration and the predicted processing duration of the target task, thereby improving the duration prediction efficiency of the target task.
After acquiring the task information of the plurality of historical tasks including the actual completion time and the actual processing time of the historical tasks, the embodiment trains the prediction model based on the task information of the plurality of historical tasks until the loss function of the actual completion time and the loss function of the actual processing time corresponding to the prediction model are both converged. The prediction model comprises a shared structure and an unshared structure, the unshared structure comprises a first output layer and a second output layer of the prediction model, the first output layer takes actual completion time as a training target, and the second output layer takes actual processing time as a training target. The embodiment trains the prediction model based on a plurality of training targets simultaneously, thereby improving the training efficiency of the model.
Fig. 3 is a schematic diagram of a data processing apparatus according to a second embodiment of the present invention. As shown in fig. 3, the apparatus of the present embodiment includes a first information acquisition unit 301 and a model training unit 302.
The first information acquiring unit 301 is configured to acquire a task information set, where the task information set includes historical task information of a plurality of historical tasks, and the historical task information includes actual completion time and actual processing time of the historical tasks. The model training unit 302 is configured to train a prediction model with the actual completion duration and the actual processing duration as training targets based on the historical task information until a loss function of the prediction model converges, where the loss function includes a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion duration, and the second loss function is a loss corresponding to the actual processing duration. The prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
Further, the apparatus further includes a second information acquisition unit 303 and a duration prediction unit 304.
The second information acquiring unit 303 is configured to acquire target task information of a target task. The duration prediction unit 304 is configured to determine a predicted completion duration and a predicted processing duration of the target task based on the prediction model according to the target task information.
Further, the first output layer and the second output layer are all fully connected layers.
Further, the predictive model is a neural network based factorization machine.
Further, the second loss function is a regression loss function with quantiles.
After acquiring the task information of the plurality of historical tasks including the actual completion time and the actual processing time of the historical tasks, the embodiment trains the prediction model based on the task information of the plurality of historical tasks until the loss function of the actual completion time and the loss function of the actual processing time corresponding to the prediction model are both converged. The prediction model comprises a shared structure and an unshared structure, the unshared structure comprises a first output layer and a second output layer of the prediction model, the first output layer takes actual completion time as a training target, and the second output layer takes actual processing time as a training target. The embodiment trains the prediction model based on a plurality of training targets simultaneously, thereby improving the training efficiency of the model.
Fig. 4 is a schematic diagram of an electronic device according to a third embodiment of the invention. In this embodiment, the electronic device includes a server, a terminal, and the like. As shown in fig. 4, the electronic device: at least one processor 401; and a memory 402 communicatively coupled to the at least one processor 401; and a communication component 403 communicatively coupled to the scanning device, the communication component 403 receiving and transmitting data under control of the processor 401; the memory 402 stores instructions executable by the at least one processor 401, and the instructions are executed by the at least one processor 401 to implement the data processing method.
Specifically, the electronic device includes: one or more processors 401 and a memory 402, one processor 401 being exemplified in fig. 4. The processor 401 and the memory 402 may be connected by a bus or other means, and fig. 4 illustrates the connection by a bus as an example. Memory 402, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 401 executes various functional applications of the device and data processing, i.e., implements the above-described data processing method, by executing nonvolatile software programs, instructions, and modules stored in the memory 402.
The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store a list of options, etc. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 402 may optionally include memory located remotely from processor 401, which may be connected to an external device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory 402 and when executed by the one or more processors 1501 perform the data processing method in any of the method embodiments described above.
The product can execute the method provided by the embodiment of the application, has corresponding functional modules and beneficial effects of the execution method, and can refer to the method provided by the embodiment of the application without detailed technical details in the embodiment.
After acquiring the task information of the plurality of historical tasks including the actual completion time and the actual processing time of the historical tasks, the embodiment trains the prediction model based on the task information of the plurality of historical tasks until the loss function of the actual completion time and the loss function of the actual processing time corresponding to the prediction model are both converged. The prediction model comprises a shared structure and an unshared structure, the unshared structure comprises a first output layer and a second output layer of the prediction model, the first output layer takes actual completion time as a training target, and the second output layer takes actual processing time as a training target. The embodiment trains the prediction model based on a plurality of training targets simultaneously, thereby improving the training efficiency of the model.
A fourth embodiment of the invention relates to a non-volatile storage medium for storing a computer-readable program for causing a computer to perform some or all of the above-described method embodiments.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The embodiment of the invention discloses A1 and a data processing method, wherein the method comprises the following steps:
acquiring a task information set, wherein the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
training a prediction model by taking the actual completion time length and the actual processing time length as training targets on the basis of the historical task information until a loss function of the prediction model converges, wherein the loss function comprises a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion time length, and the second loss function is a loss function corresponding to the actual processing time length;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
A2, the method of a1, the method further comprising:
acquiring target task information of a target task;
and determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
A3, the method of A1, wherein the first output layer and the second output layer are all connected layers.
A4, the method of A1, wherein the predictive model is a neural network-based factoring machine.
In the method of a5, as in a1 or a4, the second loss function is a regression loss function with quantiles.
The embodiment of the invention also discloses B1 and a data processing device, wherein the device comprises:
the system comprises a first information acquisition unit, a second information acquisition unit and a processing unit, wherein the first information acquisition unit is used for acquiring a task information set, the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
a model training unit, configured to train a prediction model with the actual completion duration and the actual processing duration as training targets based on the historical task information until a loss function of the prediction model converges, where the loss function includes a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion duration, and the second loss function is a loss corresponding to the actual processing duration;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
B2, the apparatus of B1, further comprising:
the second information acquisition unit is used for acquiring target task information of the target task;
and the time length prediction unit is used for determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
B3 the device of B1, wherein the first output layer and the second output layer are all connected layers.
B4, the device as described in B1, wherein the prediction model is a neural network-based factorization machine.
B5, the device according to B1 or B4, wherein the second loss function is a regression loss function with quantiles.
The embodiment of the invention also discloses C1, a computer readable storage medium, and computer program instructions stored thereon, wherein the computer program instructions realize the method according to any one of A1-A5 when being executed by a processor.
The embodiment of the invention also discloses D1, an electronic device, comprising a memory and a processor, wherein the memory is used for storing one or more computer program instructions, and the processor executes the one or more computer program instructions to realize the method according to any one of A1-A5.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of data processing, the method comprising:
acquiring a task information set, wherein the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
training a prediction model by taking the actual completion time length and the actual processing time length as training targets on the basis of the historical task information until a loss function of the prediction model converges, wherein the loss function comprises a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion time length, and the second loss function is a loss function corresponding to the actual processing time length;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
2. The method of claim 1, further comprising:
acquiring target task information of a target task;
and determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
3. The method of claim 1, wherein the first output layer and the second output layer are both fully connected layers.
4. The method of claim 1, wherein the predictive model is a neural network-based factorization machine.
5. The method of claim 1 or 4, wherein the second loss function is a regression loss function with quantiles.
6. A data processing apparatus, characterized in that the apparatus comprises:
the system comprises a first information acquisition unit, a second information acquisition unit and a processing unit, wherein the first information acquisition unit is used for acquiring a task information set, the task information set comprises historical task information of a plurality of historical tasks, and the historical task information comprises actual completion time and actual processing time of the historical tasks;
a model training unit, configured to train a prediction model with the actual completion duration and the actual processing duration as training targets based on the historical task information until a loss function of the prediction model converges, where the loss function includes a first loss function and a second loss function, the first loss function is a loss function corresponding to the actual completion duration, and the second loss function is a loss function corresponding to the actual processing duration;
the prediction model comprises a shared structure and an unshared structure, the unshared structure is an output layer of the prediction model, the unshared structure comprises a first output layer and a second output layer, the first output layer takes the actual completion time length as a training target, and the second output layer takes the actual processing time length as a training target.
7. The apparatus of claim 6, further comprising:
the second information acquisition unit is used for acquiring target task information of the target task;
and the time length prediction unit is used for determining the predicted completion time length and the predicted processing time length of the target task based on the prediction model according to the target task information.
8. The apparatus of claim 6, wherein the first output layer and the second output layer are both fully connected layers.
9. A computer-readable storage medium on which computer program instructions are stored, which, when executed by a processor, implement the method of any one of claims 1-5.
10. An electronic device comprising a memory and a processor, wherein the memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322243.7A CN113159281A (en) | 2021-03-25 | 2021-03-25 | Data processing method and data processing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322243.7A CN113159281A (en) | 2021-03-25 | 2021-03-25 | Data processing method and data processing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113159281A true CN113159281A (en) | 2021-07-23 |
Family
ID=76884958
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110322243.7A Pending CN113159281A (en) | 2021-03-25 | 2021-03-25 | Data processing method and data processing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113159281A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066105A (en) * | 2022-01-11 | 2022-02-18 | 浙江口碑网络技术有限公司 | Training method of waybill distribution timeout estimation model, storage medium and electronic equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504595A (en) * | 2014-12-19 | 2015-04-08 | 上海点啥网络科技有限公司 | Method with function of estimating pick-up time based on online ordering and application thereof |
| CN110378522A (en) * | 2019-07-02 | 2019-10-25 | 北京三快在线科技有限公司 | Method, apparatus, storage medium and the electronic equipment of prediction dispatching status information |
| CN110390503A (en) * | 2019-07-03 | 2019-10-29 | 北京三快在线科技有限公司 | Method and apparatus, storage medium and the electronic equipment that model training, distribution time determine |
| CN111144743A (en) * | 2019-12-26 | 2020-05-12 | 北京每日优鲜电子商务有限公司 | Resource allocation method, device, server and computer readable storage medium |
| CN111598487A (en) * | 2020-06-22 | 2020-08-28 | 拉扎斯网络科技（上海）有限公司 | Data processing and model training method and device, electronic equipment and storage medium |
| CN111598277A (en) * | 2020-04-15 | 2020-08-28 | 杭州优行科技有限公司 | Delivery method and device for reserved delivery piece order, electronic equipment and storage medium |
| CN112036788A (en) * | 2020-07-28 | 2020-12-04 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, readable storage medium and electronic equipment |
| CN112183856A (en) * | 2020-09-27 | 2021-01-05 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, readable storage medium and electronic equipment |
| WO2021017609A1 (en) * | 2019-07-30 | 2021-02-04 | 北京三快在线科技有限公司 | Determination of estimated time of arrival |
-
2021
- 2021-03-25 CN CN202110322243.7A patent/CN113159281A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504595A (en) * | 2014-12-19 | 2015-04-08 | 上海点啥网络科技有限公司 | Method with function of estimating pick-up time based on online ordering and application thereof |
| CN110378522A (en) * | 2019-07-02 | 2019-10-25 | 北京三快在线科技有限公司 | Method, apparatus, storage medium and the electronic equipment of prediction dispatching status information |
| CN110390503A (en) * | 2019-07-03 | 2019-10-29 | 北京三快在线科技有限公司 | Method and apparatus, storage medium and the electronic equipment that model training, distribution time determine |
| WO2021017609A1 (en) * | 2019-07-30 | 2021-02-04 | 北京三快在线科技有限公司 | Determination of estimated time of arrival |
| CN111144743A (en) * | 2019-12-26 | 2020-05-12 | 北京每日优鲜电子商务有限公司 | Resource allocation method, device, server and computer readable storage medium |
| CN111598277A (en) * | 2020-04-15 | 2020-08-28 | 杭州优行科技有限公司 | Delivery method and device for reserved delivery piece order, electronic equipment and storage medium |
| CN111598487A (en) * | 2020-06-22 | 2020-08-28 | 拉扎斯网络科技（上海）有限公司 | Data processing and model training method and device, electronic equipment and storage medium |
| CN112036788A (en) * | 2020-07-28 | 2020-12-04 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, readable storage medium and electronic equipment |
| CN112183856A (en) * | 2020-09-27 | 2021-01-05 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, readable storage medium and electronic equipment |
Non-Patent Citations (5)
| Title |
|---|
| 《神经网络与深度学习》, 机械工业出版社 * |
| HUIFENG GUO等: "DeepFM: A Factorization-Machine based Neural Network for CTR Prediction", 《ARXIV:1703.04247V1》, 13 March 2017 (2017-03-13) * |
| HUIFENG GUO等: "DeepFM: A Factorization-Machine based Neural Network for CTR Prediction", 《ARXIV:1703.04247V1》, pages 242 - 244 * |
| 吴纲维: "基于文本挖掘和集成学习的外卖订单出餐时长预测", 《万方数据》 * |
| 吴纲维: "基于文本挖掘和集成学习的外卖订单出餐时长预测", 《万方数据》, 12 July 2018 (2018-07-12) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066105A (en) * | 2022-01-11 | 2022-02-18 | 浙江口碑网络技术有限公司 | Training method of waybill distribution timeout estimation model, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wangphanich et al. | Analysis of the bullwhip effect in multi-product, multi-stage supply chain systems–a simulation approach | |
| CN106462795B (en) | System and method for allocating capital to trading strategies for big data trading in financial markets | |
| Li et al. | Reinforcement learning for joint pricing, lead-time and scheduling decisions in make-to-order systems | |
| CN110276495A (en) | Commercial articles ordering method, apparatus, computer and storage medium based on artificial intelligence | |
| CN111340244B (en) | Prediction method, training method, device, server and medium | |
| KR102384728B1 (en) | Method and system for outputting producer-centric auction price in reverse of agricultural product based on auction price of producing area | |
| CN109508917B (en) | Order distribution method, order distribution device, storage medium and electronic equipment | |
| CN112085355A (en) | Trade order management method and device for Internet e-commerce | |
| CN112041874A (en) | Computer-implemented method for generating suggestion list and system for generating order list | |
| WO2023020255A1 (en) | Data processing method and apparatus, device, and storage medium | |
| Hong et al. | Neural network controller with on-line inventory feedback data in RFID-enabled supply chain | |
| CN115631014A (en) | Intelligent commodity recommendation method and system | |
| CN113159281A (en) | Data processing method and data processing device | |
| CN108351887A (en) | The presentation of numerical data | |
| CN117522256A (en) | Purchasing screening method, system, storage medium and terminal based on supplier portrait | |
| CN111489196A (en) | Prediction method and device based on deep learning network, electronic equipment and medium | |
| CN113313562B (en) | Product data processing method and device, computer equipment and storage medium | |
| KR102309891B1 (en) | INTERNET SHOPPINGMALL OPERATION MANAGEMENT SYSTEM and THE METHOD BASED ON INTIGRATING THE MULTIPLE OPEN MARKET TRADING AVAILABLE FOR PRODUCT INVENTORY CONTROL | |
| CN112581153B (en) | Resource allocation method, resource allocation device, storage medium, and electronic apparatus | |
| JP6686208B1 (en) | Information processing device, information processing method, and program | |
| CN111582408A (en) | Data processing method, data processing apparatus, storage medium, and electronic device | |
| CN111047354A (en) | Time-sharing pricing implementation method and device, electronic equipment and storage medium | |
| CN113762993A (en) | Data processing method, system, device, equipment and storage medium | |
| Bhuiya et al. | A fuzzy random periodic review inventory model involving controllable back-order rate and variable lead-time | |
| Li et al. | Online Single-Machine Scheduling via Reinforcement Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210723 |
|
| RJ01 | Rejection of invention patent application after publication |