CN109376981B - Data processing mode determining method and device, server and data processing method - Google Patents

Data processing mode determining method and device, server and data processing method Download PDF

Info

Publication number
CN109376981B
CN109376981B CN201811014471.2A CN201811014471A CN109376981B CN 109376981 B CN109376981 B CN 109376981B CN 201811014471 A CN201811014471 A CN 201811014471A CN 109376981 B CN109376981 B CN 109376981B
Authority
CN
China
Prior art keywords
data processing
data
processing mode
decision
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811014471.2A
Other languages
Chinese (zh)
Other versions
CN109376981A (en
Inventor
张亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Advanced New Technologies Co Ltd
Original Assignee
Advanced New Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Advanced New Technologies Co Ltd filed Critical Advanced New Technologies Co Ltd
Priority to CN201811014471.2A priority Critical patent/CN109376981B/en
Publication of CN109376981A publication Critical patent/CN109376981A/en
Application granted granted Critical
Publication of CN109376981B publication Critical patent/CN109376981B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0637Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3495Performance evaluation by tracing or monitoring for systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0639Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393Score-carding, benchmarking or key performance indicator [KPI] analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Computer Hardware Design (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Tourism & Hospitality (AREA)
  • Game Theory and Decision Science (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The specification provides a method and a device for determining a data processing mode, a server and a data processing method. The method for determining the data processing mode comprises the following steps: acquiring target data to be processed and system operation parameters; and determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule of the processing mode is used for determining a data processing mode of the target data by combining the target data and the system operation parameters. In the embodiment of the specification, a determination rule of a processing mode matched with the system operation parameters is automatically determined by using a trained preset decision matching model; and determining a target processing mode which can take the factors of system operation stability and the like into consideration and is suitable for data processing of the target data according to the determination rule of the processing mode so as to process the data of the target data, thereby reducing the implementation cost and improving the efficiency and the accuracy of determining the data processing mode.

Description

Data processing mode determining method and device, server and data processing method
Technical Field
The present specification belongs to the field of internet technologies, and in particular, to a method and an apparatus for determining a data processing mode, a server, and a data processing method.
Background
In data processing, decision judgment of a selected data processing mode is often involved, and a proper data processing mode can be selected through the decision judgment to perform specific data processing.
For example, in the data processing for generating data B from data a, there are two alternative data processing methods to complete the data processing. At this time, the decision module in the data processing link may make a decision by a determination rule of a processing manner set based on factors such as the own needs of data processing and the operating environment of the system, and select an appropriate data processing manner from the two data processing manners to perform specific data processing.
At present, most of the existing methods rely on technical personnel to analyze the system environment, and combine their own experience to modify and set the determination rule of the processing mode applied by the decision module in the data processing link; and then the decision module can determine a data processing mode aiming at the target data according to the determination rule of the processing mode so as to perform specific data processing on the target data. In the implementation process of the method, technicians are required to manually adjust and change the determination rule of the processing mode applied in the decision module according to the change condition of the operating environment of the system by combining own experience, so that the operation and maintenance cost is relatively high, and the determination efficiency and accuracy are difficult to control. Therefore, a method for determining a data processing method is needed to reduce the implementation cost and improve the efficiency and accuracy of determining the data processing method.
Disclosure of Invention
The present specification aims to provide a method, an apparatus, a server and a data processing method for determining a data processing mode, so as to reduce implementation cost and improve efficiency and accuracy of determining the data processing mode.
The determination method, the determination device, the server and the data processing method for the data processing mode provided by the specification are realized as follows:
a method for determining a data processing mode comprises the following steps: acquiring target data to be processed and system operation parameters; and determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule of the processing mode is used for determining a data processing mode of the target data by combining the target data and the system operation parameters.
A method of data processing, comprising: acquiring target data to be processed and system operation parameters; determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model; determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters; and performing data processing on the target data according to the target processing mode.
A data processing mode determining device comprises: the acquisition module is used for acquiring target data to be processed and system operation parameters; the decision management module is used for determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model and sending the determination rule of the processing mode to the decision module, wherein the determination rule of the processing mode at least comprises index parameters and decision rules; and the decision module is used for determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters.
A server comprises a processor and a memory for storing processor executable instructions, wherein the processor executes the instructions to acquire target data to be processed and system operation parameters; determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model; and determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters.
A computer readable storage medium having stored thereon computer instructions which, when executed, enable acquisition of target data to be processed and system operating parameters; determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model; and determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters.
According to the method, the device, the server and the data processing method for determining the data processing mode, the determination rule of the processing mode matched with the system operation parameters is determined automatically according to the specific operation environment of the system by using the trained preset decision matching model; and then, according to the determination rule of the processing mode, determining a target processing mode which can better take the factors such as the system operation stability into consideration and is suitable for the data processing of the target data so as to perform specific data processing on the target data, thereby reducing the implementation cost, improving the efficiency and the accuracy of the determination of the data processing mode, and efficiently and stably performing the data processing of the target data based on the target processing mode determined by the rule.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
FIG. 1 is a schematic diagram of an embodiment of data processing based on a prior art method;
fig. 2 is a schematic diagram of an embodiment of a method for determining a data processing manner provided by an embodiment of the present specification, in a scenario example;
fig. 3 is a schematic diagram of another embodiment of a method for determining a data processing manner provided by an embodiment of the present specification, in a scenario example;
fig. 4 is a schematic flowchart of a method for determining a data processing manner according to an embodiment of the present disclosure;
FIG. 5 is a flow chart of a data processing method provided by an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of an embodiment of a server provided in an embodiment of the present specification;
fig. 7 is a schematic diagram of an embodiment of a data processing mode determining apparatus provided in an embodiment of the present specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
In consideration of that, most of existing determination methods for data processing methods rely on technical staff to manually set a determination rule for determining a suitable data processing method according to a system operating environment (e.g., memory occupation condition, etc.) in combination with specific target data processing, so that a decision module (or decision logic) in a data processing link can perform decision judgment based on the determination rule for the data processing method to determine a suitable and stable data processing method to perform specific data processing on target data. For example, as shown in fig. 1, when the operation load of the whole system is large and there is a risk of system abnormality, a technician needs to manually adjust and modify the determination rule of the processing mode applied by the decision module in the data processing link according to the system condition and by combining experience; therefore, the decision module can determine a more appropriate data processing mode 2 from three preset data processing modes as a target processing mode according to the determination rule of the adjusted and modified processing mode and the target data and the system operation parameters, and perform data processing on the target data.
In the implementation process of the method, a technician is required to modify and adjust the determination rule of the corresponding processing mode according to the real-time change condition of the system operation environment, and the determination rule of the processing mode is modified and adjusted manually according to the experience of the technician, so that the cost for manual operation and maintenance is relatively high, and the processing efficiency and accuracy are not stable enough in the implementation process.
In view of the above situation, the present specification considers that a trained preset decision matching model can be used to replace a technician, automatically determine a determination rule of a data processing mode matching with the current system operation situation in time and efficiently according to the real-time transformation situation of the system operation environment, and then perform decision judgment according to the determination rule of the processing mode, and determine a suitable target processing mode to stably and efficiently perform data processing on target data, thereby reducing implementation cost and improving the efficiency and accuracy of determining the data processing mode.
The embodiment of the specification provides a method for determining a data processing mode, and the method for determining the data processing mode can be applied to a system architecture comprising a plurality of servers (or processors) or a server comprising a plurality of program modules.
In this embodiment, one or more of the plurality of servers are responsible for the decision management link, and the other servers are responsible for the data processing link.
Specifically, a decision management module (for example, a program code in charge of decision management logic) is disposed on the decision management link, and a decision module and a processing module (for example, a program code in charge of data processing main logic) are disposed on the data processing link. The decision management link and the data processing link are independent from each other, so that the decision management module can send the determination rule of the processing mode to the decision module in the data processing link under the condition that the decision management module generates the determination rule of the processing mode, so that the decision module determines the data processing mode for processing the target data based on the determination rule of the processing mode; meanwhile, the logic of the data processing link is not influenced by the fact that the logic of the decision management link modifies the determination rule of the processing mode used by the decision module because the logic of the decision management link is not coupled with the data processing link.
In this embodiment, the decision management module may be specifically configured to obtain target data to be processed and system operation parameters; determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model (for example, the determination rule or algorithm to be used in the processing mode and the related index parameters can be determined); and sending the determination rule of the processing mode to a decision module.
In this embodiment, on one hand, the decision module may receive, through network communication (or through a synchronization module), a determination rule of a processing manner sent by the decision management module in the decision management link, and modify and update the determination rule of the processing manner used in the decision module according to the determination rule of the processing manner, so that the decision module may be synchronized with the decision management module. And on the other hand, target data of the processing module and system operation parameters can be received through the data processing link, decision judgment is carried out according to a determination rule of the processing mode, and a proper processing mode is determined to be used as the target processing mode.
In this embodiment, the processing module may be specifically configured to receive the target processing manner determined by the decision module, and perform specific data processing on the target data according to the target processing manner.
In this embodiment, the server may be an electronic device having data operation, storage function and network interaction function; software may also be provided that runs in the electronic device to support data processing, storage, and network interaction. The number of servers is not particularly limited in the present embodiment. The server may be one server, several servers, or a server cluster formed by several servers.
In a scenario example, as shown in fig. 2, by applying the method for determining a processing manner provided by the embodiment of the present specification, stability of a system operating environment can be considered, and an appropriate processing manner is automatically and efficiently determined to perform data processing on target data.
In this scenario example, in implementation, a decision management module in the decision management link may collect current system operation parameters in real time or periodically.
The system operating parameters may be specifically understood as performance parameters for characterizing the system operating environment condition. Specifically, the system operating parameters may include: memory occupancy rate, hard disk remaining capacity, read-write speed of an I/O interface (input/output interface), health index of a system environment, and the like. Of course, it should be noted that the above listed system operation parameters are only for better illustration of the embodiments of the present specification. In specific implementation, other types of performance parameters may be introduced according to specific application scenarios and implementation requirements, for example, the access amount of a system database may be introduced as the system operation parameter. The present specification is not limited to these.
After the decision management module obtains the system operation parameters, in order to ensure that the system can operate stably and efficiently, a trained preset decision matching model can be called to determine a determination rule of a processing mode suitable for the current system operation environment, namely a determination rule of a processing mode matched with the system operation parameters according to the system operation condition. Therefore, the decision module on the subsequent data processing link can determine the target processing mode of the target data suitable for the current system operation condition according to the determination rule of the processing mode.
The determination rule of the processing manner may be specifically understood as a decision method and basis for determining a data processing manner of the target data, that is, a decision judgment may be performed according to the determination rule of the processing manner to determine the data processing manner of the target data.
The above data processing mode may be specifically understood as an optional implementation mode for performing specific data processing on the target data in the data processing link. In general, data processing of target data can be implemented by various optional embodiments, that is, data processing of target data can be performed by various optional data processing methods. In the implementation of different data processing methods, there may be differences in the requirements for resource occupation and performance and the processing results obtained.
The target processing method may be specifically understood as a data processing method that is matched with the current system operation condition among multiple data processing methods and can stably and efficiently complete data processing on target data.
For example, the target data may be input data W to be calculated, and the data W needs to be calculated to obtain a corresponding calculation result E. At this time, two optional data processing modes, namely a data processing mode a and a data processing mode b, may exist in the data processing link. The calculation result E can be calculated from the data W according to the data processing method a or the data processing method b. However, the data processing mode a corresponds to a simpler and more convenient calculation mode, which occupies relatively less resources, has relatively lower requirements on system performance, and obtains a calculation result E with relatively lower accuracy. The data processing mode b corresponds to a more complex calculation mode, which occupies relatively large resources, has relatively high requirements on system performance, and has relatively high accuracy of a calculation result E obtained through calculation. If the available resources of the current system are less, the accuracy requirement of the data processing on the calculation result E is lower, and in order to stably and efficiently process the data W, the matched target processing mode may be the data processing mode a. If the available resources of the current system are more, the accuracy requirement of the data processing on the calculation result E is higher, and the matched target processing mode can be the data processing mode b.
The determination rule of the processing manner may be specifically understood as a method and a basis for determining a target processing manner from a plurality of selectable processing manners. The above determination rule of the processing method matched with the system operation parameter may be specifically understood as a determination rule of a processing method based on the current system operation condition, which can better consider the system operation stability.
The rule for determining the processing manner may specifically include: decision rules, index parameters, and the like. The decision rule may be a judgment algorithm (also called a decision algorithm), a judgment rule, or the like. Specifically, based on the decision rule, whether the system operation parameter meets the preset condition or not can be determined to adopt which corresponding data processing mode. For example, the judgment rule: if the occupancy rate of the CPU is less than the threshold j, selecting the first data processing mode as a target processing mode, and if the occupancy rate of the CPU is more than or equal to the threshold j, selecting the second data processing mode as the target processing mode; or a corresponding data processing mode and the like can be understood as a decision rule when the classification algorithm judges that a certain type of condition is met. The index parameter may be specifically understood as an index parameter or a parameter threshold value used for comparing and determining whether or not a system operation parameter and the like meet a preset condition when the condition is determined by using the decision rule. For example, the judgment rule: if the occupancy rate of the CPU is less than the threshold j, the first data processing mode is selected as the threshold j in the target processing mode, and the threshold j can be regarded as a parameter index. Of course, it should be noted that the above-listed decision rules and parameter indexes are only for better illustration of the embodiments of the present specification. In specific implementation, the corresponding decision rule and index parameter can be set according to specific conditions and processing requirements. The present specification is not limited to these.
The preset decision matching model can be specifically understood as a model which is obtained by learning and training historical data of data processing in advance and can determine a determination rule of a proper processing mode according to the specific situation of the operating environment of the system. For example, the current system has a poor operating environment, and the obtained determination rule of the processing method tends to determine a data processing method with a low requirement on the operating environment as a target processing method. Therefore, the subsequent data processing of the target data can be efficiently, accurately and stably performed by determining the target processing mode which can perform data processing on the target data and is matched with the current system operation condition based on the determination rule of the determined processing mode.
In this scenario example, referring to fig. 3, the decision management module may input the current system operation parameter as an input to the preset decision matching model; the preset decision matching model may output a result of matching with the system operation parameter, that is, a determination rule (including a corresponding decision rule and an index parameter) of a processing manner to be determined, which matches with the system operation parameter, based on the input system operation parameter.
After the decision management module obtains a determination rule of a processing mode matched with the system operation parameters, corresponding configuration data can be generated according to the determination rule of the processing mode, wherein the configuration data at least comprises the corresponding decision rule and index parameters; and sending the configuration data to a decision module in a data processing link through network communication and the like.
It should be noted that the decision module and the decision management module may specifically use the same set of protocol and standard. Therefore, after receiving the configuration data, the decision module can obtain the determination rule of the processing mode determined by the decision management module, namely the decision rule and the index parameter, from the configuration data through data analysis according to the same set of protocol and standard; furthermore, the original decision rule and index parameter locally used by the decision module can be modified and updated according to the determination rule of the processing mode, that is, the decision rule and index parameter locally used by the decision module are modified and synchronized into the decision rule and index parameter determined by the decision management module.
Further, the decision module may determine, based on the modified and synchronized decision rule and index parameter, a data processing mode to be adopted by the processing module in the data processing link according to the target data to be processed and the system operation parameter. Specifically, based on the modified and synchronized decision rule and index parameter, a data processing mode capable of stably and efficiently performing data processing on target data in a current system operation environment is determined as a target processing mode according to target data to be processed and system operation parameters in a data processing link, so that a processing module can stably and efficiently perform data processing according to the target processing mode.
Specifically, the decision module may determine a plurality of preset data processing modes matched with the target data according to the target data; then comparing the system operation parameters with the index parameters to obtain a comparison result; and finally, according to the decision rule, determining a preset data processing mode matched with the comparison result from the plurality of preset data processing modes as the target processing mode.
For example, the decision module may first screen out a plurality of data processing manners suitable for the target data according to characteristics such as the type and the number of the target data, and use the data processing manners as preset data processing manners matched with the target data. And then the obtained current system operation parameters and the index parameters can be compared to obtain a specific comparison result. For example, it is determined that: and the memory occupancy rate is greater than the occupancy rate threshold, the remaining capacity of the hard disk is greater than the remaining capacity threshold, the health index of the system environment is lower than the index threshold (the downtime risk exists), and the like are taken as the comparison results. And further, a preset condition met by the current system operation environment can be judged according to the decision rule and the comparison result, and a data processing mode corresponding to the preset condition can be selected to perform a data processing mode matched with the comparison result, so that a target processing mode which is suitable for the current system operation condition and can perform data processing on target data is obtained.
After the decision module determines the target processing mode, the decision module can provide the target processing mode to a processing module in the data processing link, and then the processing module can efficiently and stably perform specific data processing on the target data according to the target processing mode.
As can be seen from the above scene example, the method for determining a processing manner provided in this specification automatically determines, according to a specific operating environment of a system, a determination rule of a processing manner that matches an operating parameter of the system by using a trained preset decision matching model; and then, according to the determination rule of the processing mode, determining a target processing mode which can better take the factors such as the system operation stability into consideration and is suitable for the data processing of the target data so as to perform specific data processing on the target data, thereby reducing the implementation cost, improving the efficiency and the accuracy of the determination of the data processing mode, and efficiently and stably performing the data processing of the target data through the target processing mode determined by the determination rule based on the processing mode.
In another scenario example, it is emphasized that the data processing link is independent of the decision management link. The decision management link may be provided with a decision management module, and the data processing link may be provided with a decision module and a processing model. The decision management link is responsible for generating a determination rule of a processing mode, and the determination rule of the processing mode is sent to a decision module on the data processing link through modes such as network communication and the like. And the data processing link is used for determining a target processing mode based on the determination rule of the processing mode through the decision module and completing data processing on the target data according to the target processing mode through the processing module.
Based on the structure and the connection mode, the decision module can acquire configuration data which is generated by the decision management module and contains the determination rule of the processing mode on the decision management link through network communication, and modify and update the determination rule of the processing mode used by the decision module according to the configuration data so as to keep synchronization with the decision management module. Meanwhile, the decision module is encapsulated in the data processing link and connected with the processing module, so that the decision module can be regarded as a decision logic coupled in the data processing link. Based on the decision module, the corresponding target processing mode can be determined efficiently and accurately according to the determination rule of the mode to be processed and the processed target data and the system operation parameters, and then the processing module can process the target data according to the target processing mode.
Because the decision management link is set independently of the data processing link and is weakly coupled with the data processing link, the determination rule of the processing mode is generated, and the determination rule of the processing mode is sent to the decision module without being influenced by the data processing link, so that the determination rule of the processing mode matched with the system operation parameter can be more accurately and quickly determined, the determination rule of the processing mode is sent to the decision module through network communication, the decision rule and the index parameter locally used by the decision module are modified and updated, the decision module arranged in the data processing link can determine the target processing mode according to the modified and updated decision rule and index parameter, and the processing module in the data processing link can process data according to the target processing mode. Therefore, the influence on other program logics in the data processing link is reduced, the difficulty and the cost of change are reduced, other data are introduced from the outside of the data processing link more easily and conveniently, and the processing efficiency is further improved.
Referring to fig. 4, an embodiment of the present disclosure provides a method for determining a data processing manner. In particular implementations, the method may include the following.
S41: and acquiring target data to be processed and system operation parameters.
In this embodiment, the target data may be specifically understood as service data that needs to be processed. The target data may be text data, numerical data, image data, or the like. The specific form and content of the target data are not limited in the present specification.
In this embodiment, the system operation parameter may be specifically understood as a performance parameter for characterizing a system operation environment condition. Specifically, the system operating parameters may include: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface, health index of system environment and the like. Of course, it should be noted that the above listed system operation parameters are only for better illustration of the embodiments of the present disclosure. During specific implementation, other types of performance parameters can be introduced as system operation parameters according to specific application scenarios and implementation requirements. The present specification is not limited to these.
S42: and determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule of the processing mode is used for determining a data processing mode of the target data by combining the target data and the system operation parameters.
In this embodiment, the processing manner (i.e., the data processing manner) may be specifically understood as an optional implementation manner in the data processing link for performing specific data processing on the target data. Generally, the data processing of the target data can be realized by various alternative embodiments, that is, there may be various alternative data processing manners that can realize the data processing of the target data. Different data processing methods may have different requirements on resource occupation and performance, and different results.
In this embodiment, the determination rule of the processing method may be specifically understood as a decision method and basis for determining a data processing method for target data, and in a specific implementation, the target processing method for performing data processing on the target data may be determined by decision judgment according to the determination rule of the processing method.
In this embodiment, the above rule for determining the processing manner matched with the system operation parameters may be specifically understood as a rule for determining a target processing manner that is suitable for the current system operation environment, and can tend to determine to make the system operate stably and healthily, and avoid operation problems such as downtime and database abnormality as much as possible.
The rule for determining the processing manner may specifically include: decision rules, index parameters, and the like. The decision rule may be a judgment algorithm (or a decision algorithm), a judgment rule, or the like. Specifically, based on the decision rule, whether the system operation parameter meets the preset condition or not can be determined to adopt which corresponding data processing mode. For example, the judgment rule: if the occupancy rate of the CPU is less than the threshold j, selecting the first data processing mode as a target processing mode, and if the occupancy rate of the CPU is more than or equal to the threshold j, selecting the second data processing mode as the target processing mode; or a corresponding data processing mode and the like can be understood as a decision rule when the classification algorithm judges that a certain type of condition is met. The index parameter may be specifically understood as an index parameter or a parameter threshold value used for comparing and determining whether or not a system operation parameter and the like meet a preset condition when the condition is determined by using the decision rule. Of course, it should be noted that the decision rules and parameter indexes listed above are only for better illustration of the embodiments of the present disclosure. In specific implementation, the corresponding decision rule and index parameter can be set according to specific conditions and processing requirements. The present specification is not limited to these.
In this embodiment, the preset decision matching model may be specifically understood as a model obtained by learning and training historical data of data processing in advance, and a determination rule for determining an appropriate processing manner according to a specific situation of an operating environment of the system. In specific implementation, the system operation parameters may be used as model input and input into the preset decision matching model, so as to obtain a determination rule for determining a processing mode of a target processing mode matched with the current system operation condition.
In this embodiment, the determining rule for determining the processing mode matched with the system operating parameter according to the preset decision matching model may include the following steps: and calling the preset decision matching model through a decision management module in a decision management link to determine a determination rule of a processing mode matched with the system operation parameters, wherein the determination rule of the processing mode at least comprises index parameters and decision rules.
In this embodiment, the decision management link may be specifically configured to determine, by a decision management module (or a decision management logic) in the decision management link, a determination rule of a processing manner that matches the current system operating condition according to the system operating parameter; and sending the determination rule of the processing mode through network communication so as to modify and update the determination rule of the processing mode used in a decision module (or decision logic) in the data processing link into the determination rule of the processing mode matched with the system operation parameters. The data processing link can be specifically used for determining a target processing mode through decision judgment by combining system operation parameters and target data to be processed according to a determination rule of the processing mode through a decision module in the data link; and processing the target data according to the target processing mode by a processing module in the data processing link.
In this embodiment, the decision management link and the data processing link are independent of each other, and have relatively weak coupling and relatively small correlation influence. Therefore, based on the structure and the connection relationship, the decision management module determines a determination rule of a processing mode matched with the system operation parameters, and sends the determination rule of the processing mode to the decision module in the data processing link, so that the implementation process of the determination rule of the processing mode used by the decision module is changed has relatively small influence and change on other logic modules except the decision module in the data processing link, and the implementation cost of modifying the determination rule of the processing mode in the decision module is reduced; meanwhile, through the decision management link, other data introduced from the outside of the data processing link are more convenient and efficient to decide and interpret.
In this embodiment, the rule for determining the processing manner may be specifically used to determine a data processing manner for the target data by combining the target data and the system operation parameter.
In this embodiment, after the decision module in the data processing link obtains the determination rule of the processing mode matching the system operation parameter, further, it can be determined according to the determination rule of the processing mode, the target data, and the system operation parameter that not only the target data can be better processed, but also the stability of the system operation can be considered, and the data processing mode suitable for the current system operation environment is taken as the target processing mode. And then, the processing module in the data processing link is used for processing the target data according to the target processing mode, so that the data processing of the target data is more stable and efficient.
In this embodiment, the determining a target processing manner according to the determination rule of the processing manner, the target data, and the system operation parameter may be implemented as a data processing manner for the target data, and the following may be included: determining a plurality of preset data processing modes matched with the target data according to the target data; comparing the system operation parameters with the index parameters to obtain a comparison result; and according to the decision rule, determining a preset data processing mode matched with the comparison result from the plurality of preset data processing modes as the target processing mode.
In this embodiment, the determining, according to the target data, a plurality of preset data processing manners matched with the target data may specifically include: and selecting a data processing mode suitable for the characteristics of the target data (namely matched with the target data) as a preset data processing mode according to the characteristics of the type, the quantity and the like of the target data.
In this embodiment, the preset data processing mode may be a predefined and set optional data processing mode selected and determined by the decision module in the data processing link according to a specific situation.
In this embodiment, the comparing the system operation parameters and the index parameters to obtain the comparison result may specifically be comparing each of the system operation parameters with the corresponding index parameter to obtain the corresponding comparison result. For example, the remaining capacity of the hard disk (system operation parameter) is smaller than a capacity threshold (corresponding index parameter).
In this embodiment, the determining, according to the decision rule, a preset data processing manner that matches the comparison result from among the plurality of preset data processing manners as the target processing manner may specifically include: and according to the decision rule, taking the comparison result as a decision judgment basis, determining the current target data processing and which preset condition the system operation environment meets, and further selecting a preset data processing mode corresponding to the met preset condition as a target processing mode.
After the target processing mode is determined, the processing module in the data processing link can perform efficient and stable data processing on the target data according to the target processing mode, and meanwhile, the stability and health of the whole system can be considered.
As can be seen from the above, the method for determining a processing mode provided in this specification automatically determines a determination rule of a processing mode matching with system operation parameters according to a specific operation environment of a system by using a trained preset decision matching model; and then, according to the determination rule of the processing mode, determining a suitable target processing mode for the data processing of the target data, which can better take the factors such as the system operation stability into consideration, so as to perform specific data processing on the target data, thereby reducing the implementation cost, improving the efficiency and the accuracy of the determination of the data processing mode, and efficiently and stably performing the data processing of the target data through the target processing mode determined based on the rule.
In an embodiment, a determination rule of a processing manner matched with the system operation parameter is determined according to a preset decision matching model, and the specific implementation may include: calling the preset decision matching model through a decision management module in a decision management link to determine a determination rule of a processing mode matched with the system operation parameters, wherein the determination rule of the processing mode at least comprises index parameters and decision rules;
correspondingly, after determining the determination rule of the processing mode matched with the system operation parameter according to the preset decision matching model, the method may further include the following steps: determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters through a decision module in a data processing link; processing the target data by a processing module in the data processing link according to the target processing mode; wherein the decision management link is independent of the data processing link.
It should be noted that the decision management link is independent of the data processing link, which can be specifically understood as that in terms of task function, the decision management link and the data processing link are different in division of labor and are independent of each other; structurally, the decision management link and the data processing link are weak in coupling and have small mutual influence.
In this embodiment, the system operation parameter may be specifically understood as a performance parameter for characterizing a system operation environment condition. Specifically, the system operating parameters may include: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface, health index of system environment and the like. The health index of the system environment may refer to an evaluation parameter of a health condition of the entire system operating environment. For example, the system is safe to operate as a whole, or the system has a relatively high risk of being down, abnormal and the like. Of course, it should be noted that the above listed system operation parameters are only illustrative and should not be construed as an improper limitation of the present specification.
In this embodiment, the decision management link may specifically include decision management, and specifically may be configured to determine, according to the system operation parameter, a determination rule of a processing manner that matches the operation condition of the current system through a decision management module (or a decision management logic) in the decision management link; and the determination rule of the processing mode is sent to a decision module (or decision logic) in a data processing link through network communication so as to change the determination rule modification of the processing mode locally used by the decision module. The data processing link may specifically include a decision module and a processing module, and is specifically configured to determine, by a decision module in the data link, a suitable target processing mode through decision determination in combination with system operation parameters and target data according to a determination rule of the processing mode; and the processing module in the data processing link can be used for processing the target data according to the target processing mode.
However, in the existing method, there is usually no decision management module, no decision management link independent of the data processing link, and only the decision module coupled with the data processing link. Based on the structure and the connection relation in the existing method, the decision module and the data processing link have stronger coupling and larger relevance. Therefore, when the determination rule of the processing manner in the decision module is modified, it will inevitably affect the logic modules in other parts of the data processing link, for example, the logic modules such as the processing module in the data processing link may need to be modified and adjusted accordingly. The cost for changing the determination rule of the processing mode adopted by the decision module is relatively high; furthermore, the difficulty of introducing other relevant data from outside the data processing link is also relatively great.
Based on the structure and the connection relationship in the embodiment, the decision management module determines a determination rule of a processing mode matched with the system operation parameters, and sends the determination rule of the processing mode to the decision module, so that the decision module updates and modifies the determination rule of the processing mode used locally according to the received determination rule of the processing mode, and the influence and the change of the process on logic modules of other parts except the decision module in a data processing link are relatively small, thereby reducing the implementation cost of modifying and changing the determination rule of the processing mode in the decision module; meanwhile, through the decision management link, other data introduced from the outside of the data processing link are more convenient and efficient to make decision judgment.
In an embodiment, the determining a target processing manner according to the determination rule of the processing manner, the target data, and the system operation parameter may include the following steps:
s1: determining a plurality of preset data processing modes matched with the target data according to the target data;
s2: comparing the system operation parameters with the index parameters to obtain a comparison result;
s3: and according to the decision rule, determining a preset data processing mode matched with the comparison result from the plurality of preset data processing modes as the target processing mode.
In this embodiment, the determined target processing mode may be specifically understood as a data processing mode that can perform data processing on target data well and simultaneously consider the operation stability and safety of the whole system.
In an embodiment, the preset decision matching model may be specifically established as follows: acquiring historical data of data processing of a system, wherein the historical data of the data processing at least comprises the following steps: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing; and training by using the historical data of the data processing, and establishing to obtain the preset decision matching model.
In this embodiment, the historical data of the data processing may specifically include data related to a hardware part in the historical data processing process, such as a determination rule of a processing manner used in the data processing, an operation parameter of a system of the data processing, and the like, and may also include data related to a data processing service in the historical data processing process, such as result data obtained by the data processing, feedback information of the data processing, a success rate and a failure rate of the data processing, and the like. In specific implementation, data of an appropriate type can be introduced as the history data of the data processing according to specific situations. The present specification is not limited to these.
In the present embodiment, the historical data of the data processing of the system may specifically be determination rules (e.g., used decision rules and index parameters) of a processing method based on which data processing is performed in the system every time in history, operation parameters (e.g., CPU occupancy, remaining hard disk capacity, and the like) of the system every time data processing is performed, and result data obtained by data processing each time, and feedback information (e.g., processing failure, occurrence of downtime abnormality, and the like) of the data processing. Of course, the above listed history data of the data processing is only a schematic illustration. The present specification does not limit the specific contents of the history data of the data processing.
In this embodiment, the acquiring historical data of the data processing of the system may include, in specific implementation: acquiring log records in a system and feedback results of a user on each data processing; analyzing the log record, and extracting a determination rule of a processing mode used for data processing, an operation parameter of the data processing and result data obtained by the data processing; and determining feedback information of data processing and the like according to the feedback result of the user to each data processing. Of course, the above-listed manner of obtaining the history data of the data processing of the system is only an illustrative description. In specific implementation, according to a specific scenario, other reasonable manners may be adopted to obtain historical data of data processing of the system. The present specification is not limited to these.
In this embodiment, after obtaining the historical data of the data processing of the system, the historical data of the data processing of the system may be used as sample data to perform learning training on models such as a neural network, so as to establish a model that can reflect a mapping relationship between a system operation parameter, a determination rule of a data processing mode, and a system operation stability, and thus obtain the preset decision matching model. Therefore, the determination rule of the processing mode which tends to make the system operate stably and safely can be found according to the system operation parameters by using the preset decision matching model as the determination rule of the processing mode matched with the system parameters.
In one embodiment, the system operating parameter may specifically include at least one of: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface, health index of system environment and the like. Of course, it should be noted that the above listed system operating parameters are only illustrative. In specific implementation, other types of parameters can be introduced as the system operation parameters according to specific situations. The present specification is not limited to these.
As can be seen from the above, the method for determining a processing mode provided in this specification automatically determines a determination rule of a processing mode matching with system operation parameters according to a specific operation environment of a system by using a trained preset decision matching model; then, a target processing mode which can better take the factors such as system operation stability and the like into consideration and is suitable for data processing of the target data is determined according to the determination rule of the processing mode so as to perform specific data processing on the target data, thereby reducing the implementation cost, improving the efficiency and the accuracy of determining the data processing mode, and efficiently and stably performing data processing on the target data through the target processing mode determined based on the rule; the decision management module independent of the data processing link is used for determining and sending the determination rule of the processing mode, so that the decision module in the data processing link can change the determination rule of the used processing mode, the influence of stronger logic coupling on the change of the decision module in the data processing link in the existing structure is weakened, the difficulty and the implementation cost for determining and modifying the determination rule of the processing mode are reduced, and the processing efficiency is improved.
Referring to fig. 5, an embodiment of the present disclosure provides a data processing method. In particular, the method may comprise the following
S51: acquiring target data to be processed and system operation parameters;
s52: determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model;
s53: determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters;
s54: and performing data processing on the target data according to the target processing mode.
In this embodiment, the preset decision matching model may be specifically understood as a model obtained by learning and training historical data of data processing in advance, and a determination rule for determining an appropriate processing manner according to a specific situation of an operating environment of the system. In specific implementation, the system operation parameters may be used as model input and input into the preset decision matching model, so as to obtain a determination rule for determining a processing mode of a target processing mode matched with the current system operation condition.
In this embodiment, in specific implementation, a determination rule of a processing mode matched with a system operation parameter may be determined by obtaining and according to the current system operation parameter; determining a data processing mode which can take the factors of the data processing of the target data, the system operation stability and the like into consideration as a target processing mode according to the determination rule of the processing mode and by combining the target data and the system operation parameters; further, according to the target data processing method, specific data processing can be efficiently and stably performed on the target data.
In an embodiment, the determining, according to a preset decision matching model, a determination rule of a processing manner matching with the system operation parameter may include: and calling the preset decision matching model through a decision management module in a decision management link to determine a determination rule of a processing mode matched with the system operation parameters, wherein the determination rule of the processing mode at least comprises index parameters and decision rules. Correspondingly, according to the determination rule of the processing mode, the target data and the system operation parameters, determining a target processing mode as a data processing mode for the target data; according to the target processing mode, performing data processing on the target data may specifically include: determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters through a decision module in a data processing link; and processing the target data according to the target processing mode through the processing module in the data processing link.
In an embodiment, when the preset decision matching model is implemented, the preset decision matching model may be established as follows: acquiring historical data of data processing of a system, wherein the historical data of the data processing at least comprises the following steps: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing; and training by using the historical data of the data processing, and establishing the preset decision matching model.
As can be seen from the above, the data processing method provided in this specification automatically determines, according to the specific operating environment of the system, a determination rule of a processing manner that matches the operating parameters of the system by using a trained preset decision matching model; and determining a target processing mode which can better take the factors such as the system operation stability into consideration and is suitable for the data processing of the target data according to the determination rule of the processing mode so as to perform specific data processing on the target data, so that the system can efficiently and stably perform the data processing of the target data, the risks of downtime, abnormality and the like of the system are reduced, and the safety and the stability of the system are improved.
Embodiments of the present specification further provide a server, including a processor and a memory for storing processor-executable instructions, where the processor, when implemented, may perform the following steps according to the instructions: acquiring target data to be processed and system operation parameters; and determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule of the processing mode is used for determining a data processing mode of the target data by combining the target data and the system operation parameters.
In order to more accurately complete the above instructions, referring to fig. 6, the present specification further provides another specific server, wherein the server includes a network communication port 601, a processor 602 and a memory 603, and the above structures are connected by an internal cable, so that the structures can perform specific data interaction.
The network communication port 601 may be specifically configured to obtain target data to be processed and system operation parameters;
the processor 602 may be specifically configured to determine, according to a preset decision matching model, a determination rule of a processing manner that matches the system operation parameter, where the determination rule of the processing manner is used to determine, by combining the target data and the system operation parameter, a data processing manner for the target data.
The memory 603 may be specifically configured to store target data and system operating parameters, intermediate data generated by the processor 602, and store corresponding instruction programs.
In this embodiment, the network communication port 601 may be a virtual port bound with different communication protocols so as to transmit or receive different data. For example, the network communication port may be port No. 80 responsible for web data communication, port No. 21 responsible for FTP data communication, or port No. 25 responsible for mail data communication. In addition, the network communication port can also be a communication interface or a communication chip of an entity. For example, it may be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it may also be a bluetooth chip.
In this embodiment, the processor 602 may be implemented in any suitable manner. For example, the processor may take the form of, for example, a microprocessor or processor and a computer-readable medium that stores computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, an embedded microcontroller, and so forth. The description is not intended to be limiting.
In this embodiment, the memory 603 may include multiple layers, and in a digital system, the memory may be any memory as long as binary data can be stored; in an integrated circuit, a circuit without a physical form and with a storage function is also called a memory, such as a RAM, a FIFO and the like; in the system, the storage device in physical form is also called a memory, such as a memory bank, a TF card and the like.
The present specification further provides a computer storage medium based on the above payment method, where the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the computer storage medium implements: acquiring target data to be processed and system operation parameters; determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model; and determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters.
In the present embodiment, the storage medium includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Cache (Cache), a Hard Disk Drive (HDD), or a Memory Card (Memory Card). The memory may be used to store computer program instructions. The network communication unit may be an interface for performing network connection communication, which is set in accordance with a standard prescribed by a communication protocol.
In this embodiment, the functions and effects specifically realized by the program instructions stored in the computer storage medium can be explained by comparing with other embodiments, and are not described herein again.
Referring to fig. 7, in a software level, an embodiment of the present specification further provides a device for determining a data processing manner, where the device may specifically include the following structural modules:
the obtaining module 701 may be specifically configured to obtain target data to be processed and system operation parameters;
the decision management module 702 may be specifically configured to determine, according to a preset decision matching model, a determination rule of a processing manner that matches the system operation parameter, and send the determination rule of the processing manner to the decision module, where the determination rule of the processing manner at least includes an index parameter and a decision rule;
the decision module 703 may be specifically configured to determine a target processing manner according to the determination rule of the processing manner, the target data, and the system operation parameter, and serve as a data processing manner for the target data.
In an embodiment, the apparatus may further include a processing module, which may be specifically configured to perform data processing on the target data according to the target processing manner, where the decision module is connected to the processing module and disposed in a data processing link; the decision management module is arranged in a decision management link, which is independent of the data processing link.
In one embodiment, the decision module 702 may specifically include;
the first determining unit may be specifically configured to determine, according to the target data, a plurality of preset data processing manners matched with the target data;
the comparison unit is specifically used for comparing the system operation parameters with the index parameters to obtain a comparison result;
the second determining unit may be specifically configured to determine, according to the decision rule, a preset data processing manner that matches the comparison result from the plurality of preset data processing manners as the target processing manner.
In an embodiment, the apparatus may further specifically include an establishing module, which may specifically be configured to establish a preset decision matching model.
In one embodiment, the establishing module may specifically include:
the obtaining unit may be specifically configured to obtain historical data of data processing of a system, where the historical data of data processing at least includes: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing;
and the training unit can be specifically used for training by using the historical data of the data processing to establish the preset decision matching model.
In one embodiment, the system operating parameters include at least one of: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface, health index of system environment and the like. Of course, the above listed system operating parameters are merely illustrative. In specific implementation, other types of parameters can be introduced as the system operation parameters according to specific situations. The present specification is not limited to these.
It should be noted that, the units, devices, modules, etc. illustrated in the above embodiments may be implemented by a computer chip or an entity, or implemented by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. It is to be understood that, in implementing the present specification, functions of each module may be implemented in one or more pieces of software and/or hardware, or a module that implements the same function may be implemented by a combination of a plurality of sub-modules or sub-units, or the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
As can be seen from the above, the data processing mode determining device provided in the embodiments of the present specification automatically determines, according to the specific operating environment of the system, a determining rule of a processing mode matching with the system operating parameters by using a trained preset decision matching model through a decision management module; and then a decision module determines a target processing mode which can better take the factors such as system operation stability and the like into consideration and is suitable for data processing of the target data according to the determination rule of the processing mode so as to perform specific data processing on the target data, thereby reducing the implementation cost, improving the efficiency and accuracy of determining the data processing mode, and efficiently and stably performing data processing on the target data through the target processing mode determined based on the rule.
Although the present specification provides method steps as described in the examples or flowcharts, additional or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an apparatus or client product in practice executes, it may execute sequentially or in parallel (e.g., in a parallel processor or multithreaded processing environment, or even in a distributed data processing environment) according to the embodiments or methods shown in the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded. The terms first, second, etc. are used to denote names, but not to denote any particular order.
Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
From the above description of the embodiments, it is clear to those skilled in the art that the present specification can be implemented by software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the present specification may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes instructions for causing a computer device (which may be a personal computer, a mobile terminal, a server, or a network device) to execute the method according to the embodiments or some parts of the embodiments of the present specification.
The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The description is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
While the specification has been described with examples, those skilled in the art will appreciate that there are numerous variations and permutations of the specification without departing from the spirit of the specification, and it is intended that the appended claims encompass such variations and modifications without departing from the spirit of the specification.

Claims (13)

1. A method for determining a data processing mode comprises the following steps:
acquiring target data to be processed and system operation parameters;
determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule of the processing mode is used for determining a data processing mode of the target data by combining the target data and the system operation parameters, and the determination rule of the processing mode at least comprises an index parameter and a decision rule; the preset decision matching model is obtained by learning and training historical data of data processing in advance;
determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model, wherein the determination rule comprises the following steps: calling the preset decision matching model through a decision management link to determine a determination rule of a processing mode matched with the system operation parameters; the decision management link is provided with a decision management module;
correspondingly, a target processing mode is determined through a data processing link according to the determination rule of the processing mode, the target data and the system operation parameters, and the target processing mode is used as a data processing mode for the target data; performing data processing on the target data according to the target processing mode through the data processing link; the data processing link is provided with a decision module and a processing module, and the decision management link is independent of the data processing link.
2. The method of claim 1, wherein determining a target processing mode according to the determination rule of the processing mode, the target data and the system operation parameter comprises:
determining a plurality of preset data processing modes matched with the target data according to the target data;
comparing the system operation parameters with the index parameters to obtain a comparison result;
and according to the decision rule, determining a preset data processing mode matched with the comparison result from the plurality of preset data processing modes as the target processing mode.
3. The method of claim 1, wherein the predetermined decision matching model is established as follows:
acquiring historical data of data processing of a system, wherein the historical data of the data processing at least comprises the following steps: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing;
and training by using the historical data of the data processing, and establishing the preset decision matching model.
4. The method of claim 1, the system operating parameters comprising at least one of: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface and health index of system environment.
5. A method of data processing, comprising:
acquiring target data to be processed and system operation parameters;
determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model; the determining rule of the processing mode at least comprises an index parameter and a decision rule; the preset decision matching model is obtained by learning and training historical data of data processing in advance;
determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters;
according to the target processing mode, carrying out data processing on the target data;
the determining rule of the processing mode matched with the system operation parameters is determined according to a preset decision matching model, and the determining rule comprises the following steps: calling the preset decision matching model through a decision management link to determine a determination rule of a processing mode matched with the system operation parameters; the decision management link is provided with a decision management module;
correspondingly, a target processing mode is determined through a data processing link according to the determination rule of the processing mode, the target data and the system operation parameters, and the target processing mode is used as a data processing mode for the target data; performing data processing on the target data according to the target processing mode through the data processing link; the data processing link is provided with a decision module and a processing module, and the decision management link is independent of the data processing link.
6. The method of claim 5, wherein the predetermined decision matching model is established as follows:
acquiring historical data of data processing of a system, wherein the historical data of the data processing at least comprises the following steps: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing;
and training by using the historical data of the data processing, and establishing the preset decision matching model.
7. A data processing mode determining device comprises:
the acquisition module is used for acquiring target data to be processed and system operation parameters;
the decision management module is used for determining a determination rule of a processing mode matched with the system operation parameters according to a preset decision matching model and sending the determination rule of the processing mode to the decision module, wherein the determination rule of the processing mode at least comprises index parameters and decision rules; the preset decision matching model is obtained by learning and training historical data of data processing in advance;
the decision module is used for determining a target processing mode as a data processing mode for the target data according to the determination rule of the processing mode, the target data and the system operation parameters;
the decision management module is arranged on a decision management link; the decision module is arranged on the data processing link; the data processing link is also provided with a processing module; the decision management link is independent of the data processing link;
correspondingly, calling the preset decision matching model through a decision management link to determine a determination rule of a processing mode matched with the system operation parameters; determining a target processing mode as a data processing mode for the target data through a data processing link according to the determination rule of the processing mode, the target data and the system operation parameters; and performing data processing on the target data according to the target processing mode through the data processing link.
8. The apparatus of claim 7, the decision module comprising;
the first determining unit is used for determining a plurality of preset data processing modes matched with the target data according to the target data;
the comparison unit is used for comparing the system operation parameters with the index parameters to obtain a comparison result;
and a second determining unit, configured to determine, according to the decision rule, a preset data processing manner that matches the comparison result from the plurality of preset data processing manners as the target processing manner.
9. The apparatus of claim 7, further comprising a building module for building a predetermined decision matching model.
10. The apparatus of claim 9, the establishing means comprising:
an obtaining unit, configured to obtain historical data of data processing of a system, where the historical data of data processing at least includes: determining rules of processing modes used by data processing, operating parameters of the data processing, result data obtained by the data processing and feedback information of the data processing;
and the training unit is used for training by utilizing the historical data of the data processing and establishing the preset decision matching model.
11. The apparatus of claim 7, the system operating parameters comprising at least one of: memory occupancy rate, hard disk residual capacity, read-write speed of an I/O interface and health index of system environment.
12. A server comprising a processor and a memory for storing processor-executable instructions that when executed by the processor implement the steps of the method of any one of claims 1 to 4.
13. A computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of the method of any one of claims 1 to 4.
CN201811014471.2A 2018-08-31 2018-08-31 Data processing mode determining method and device, server and data processing method Active CN109376981B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811014471.2A CN109376981B (en) 2018-08-31 2018-08-31 Data processing mode determining method and device, server and data processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811014471.2A CN109376981B (en) 2018-08-31 2018-08-31 Data processing mode determining method and device, server and data processing method

Publications (2)

Publication Number Publication Date
CN109376981A CN109376981A (en) 2019-02-22
CN109376981B true CN109376981B (en) 2022-09-02

Family

ID=65404169

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811014471.2A Active CN109376981B (en) 2018-08-31 2018-08-31 Data processing mode determining method and device, server and data processing method

Country Status (1)

Country Link
CN (1) CN109376981B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113077277A (en) * 2020-01-06 2021-07-06 北京京邦达贸易有限公司 Information processing apparatus and method
CN111240938A (en) * 2020-01-10 2020-06-05 深圳市五联云科技有限公司 Data congestion processing method and device based on face-brushing payment
CN112015838B (en) * 2020-08-28 2022-09-09 苏州智加科技有限公司 Road test data processing method and system and server
CN114140259A (en) * 2021-11-29 2022-03-04 中国平安财产保险股份有限公司 Artificial intelligence-based wind control method, device, equipment and storage medium for underwriting

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106970837A (en) * 2017-03-29 2017-07-21 联想(北京)有限公司 A kind of information processing method and electronic equipment
CN108173698A (en) * 2018-01-17 2018-06-15 阿里巴巴集团控股有限公司 Network service management method, apparatus, server and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107122369B (en) * 2016-02-25 2021-05-11 创新先进技术有限公司 Service data processing method, device and system
CN108287708B (en) * 2017-12-22 2021-06-25 深圳市云智融科技有限公司 Data processing method and device, server and computer readable storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106970837A (en) * 2017-03-29 2017-07-21 联想(北京)有限公司 A kind of information processing method and electronic equipment
CN108173698A (en) * 2018-01-17 2018-06-15 阿里巴巴集团控股有限公司 Network service management method, apparatus, server and storage medium

Also Published As

Publication number Publication date
CN109376981A (en) 2019-02-22

Similar Documents

Publication Publication Date Title
CN109376981B (en) Data processing mode determining method and device, server and data processing method
WO2018103595A1 (en) Authorization policy recommendation method and device, server, and storage medium
CN110222880B (en) Service risk determining method, model training method and data processing method
CN108833458B (en) Application recommendation method, device, medium and equipment
CN109726763B (en) Information asset identification method, device, equipment and medium
CN110442712B (en) Risk determination method, risk determination device, server and text examination system
CN104541293A (en) Architecture for client-cloud behavior analyzer
US10834183B2 (en) Managing idle and active servers in cloud data centers
CN114503108A (en) Adding countermeasure robustness to a trained machine learning model
CN110955586A (en) System fault prediction method, device and equipment based on log
CN111698303A (en) Data processing method and device, electronic equipment and storage medium
CN105487970B (en) A kind of method for showing interface and device
CN106844550B (en) Virtualization platform operation recommendation method and device
CN111581258A (en) Safety data analysis method, device, system, equipment and storage medium
CN114356712A (en) Data processing method, device, equipment, readable storage medium and program product
CN116521344A (en) AI algorithm scheduling method and system based on resource bus
US11782923B2 (en) Optimizing breakeven points for enhancing system performance
CN110532267A (en) Determination method, apparatus, storage medium and the electronic device of field
CN111815442B (en) Link prediction method and device and electronic equipment
CN115529315A (en) Cloud edge coordination system
CN114841267A (en) Real-time prediction method and device, electronic equipment and computer program product
CN112905987B (en) Account identification method, device, server and storage medium
CN107528728A (en) A kind of method and device for obtaining user experience data
CN113792435A (en) Data model establishing method and device and server
CN113610225A (en) Quality evaluation model training method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20200925

Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant after: Innovative advanced technology Co.,Ltd.

Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant before: Advanced innovation technology Co.,Ltd.

Effective date of registration: 20200925

Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant after: Advanced innovation technology Co.,Ltd.

Address before: Greater Cayman, British Cayman Islands

Applicant before: Alibaba Group Holding Ltd.

GR01 Patent grant
GR01 Patent grant