CN116662387A

CN116662387A - Service data processing method, device, equipment and storage medium

Info

Publication number: CN116662387A
Application number: CN202310554450.4A
Authority: CN
Inventors: 詹敏; 陈仲宇; 胡伟; 侯美雪
Original assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Current assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Priority date: 2023-05-17
Filing date: 2023-05-17
Publication date: 2023-08-29

Abstract

The application provides a business data processing method, a device, equipment and a storage medium, relating to the technical field of data management, wherein the method comprises the following steps: and obtaining a to-be-detected service data set from a plurality of data sources, and determining a data detection rule corresponding to the to-be-detected service data from a data detection rule base containing the latest data detection rule of each service according to the corresponding service type and the data source for each to-be-detected service data in the to-be-detected service data set so as to detect the data compliance of the to-be-detected service data. If the detection result indicates that target service data which does not meet the data detection rule exists in each service data to be detected, generating a corresponding data modification task for each target service data, determining a data modification mechanism corresponding to the data modification task according to the service object identification of each target service data, and sending each data modification task to the corresponding data modification mechanism so as to improve the data processing efficiency of the service data.

Description

Service data processing method, device, equipment and storage medium

Technical Field

The present application relates to the field of data management technologies, and in particular, to a service data processing method, device, equipment, and storage medium.

Background

With the development of internet technology, the requirements of various services on service data are changed, and the user can not meet the latest service data quality requirement in the past when transacting the compliance service data provided by related services, so that in order to ensure that the user information required to be provided when transacting the services by the user meets the latest service compliance and risk controllability requirements, the financial institution is required to modify the compliance service data according to the latest data compliance requirement so as to improve the quality level of the service data.

At present, when data correction is performed on service data, a financial institution checks and corrects massive service data manually, the operation is complex, the error rate is high, and the service data correction efficiency is low.

Therefore, how to improve the data processing efficiency of the service data is a problem to be solved.

Disclosure of Invention

The embodiment of the application provides a service data processing method, device, equipment and storage medium, which are used for improving the processing efficiency of service data.

In one aspect, a service data processing method is provided, and the method includes:

obtaining a to-be-detected service data set from a plurality of data sources, wherein each piece of to-be-detected service data in the to-be-detected service data set is object information provided by a corresponding service object when corresponding service is handled in a history manner;

For each piece of to-be-detected business data in the to-be-detected business data set, determining a data detection rule corresponding to each piece of to-be-detected business data from a data detection rule base based on a business type and a data source corresponding to each piece of to-be-detected business data; wherein, the data detection rule base comprises the latest data detection rule of each service;

based on each obtained data detection rule, respectively carrying out data compliance detection on corresponding service data to be detected;

if the target service data which does not meet the data detection rule exists in each piece of service data to be detected based on the detection result, corresponding data rectification tasks are respectively generated for each piece of target service data;

and determining a data modification mechanism corresponding to the corresponding data modification task based on the service object identifier corresponding to each item of label service data, and sending each data modification task to the corresponding data modification mechanism.

In one aspect, there is provided a service data processing apparatus, the apparatus comprising:

the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a to-be-detected service data set from a plurality of data sources, and each piece of to-be-detected service data in the to-be-detected service data set is object information provided by a corresponding service object when the corresponding service object transacts corresponding service in history;

The determining unit is used for determining the data detection rule corresponding to each piece of the to-be-detected business data from the data detection rule base based on the business type and the data source corresponding to each piece of the to-be-detected business data; wherein, the data detection rule base comprises the latest data detection rule of each service;

the detection unit is used for respectively carrying out data compliance detection on corresponding service data to be detected based on the obtained data detection rules;

the generating unit is used for generating corresponding data rectifying tasks for each item of target service data if the target service data which does not meet the data detection rule exists in each item of service data to be detected based on the detection result;

and the sending unit is used for determining the data rectifying mechanism corresponding to the corresponding data rectifying task based on the service object identifier corresponding to each item of standard service data and sending each data rectifying task to the corresponding data rectifying mechanism.

Optionally, the detection unit is specifically configured to:

respectively carrying out format conversion on each piece of service data to be detected according to a preset format rule;

And based on the obtained data detection rules, respectively carrying out data compliance detection on the business data to be detected after format conversion.

Optionally, the detection unit is specifically configured to:

for each item of label service data, the following operations are respectively executed:

for one item of target service data, matching a service object identifier corresponding to the target service data with a service object identifier corresponding to a data rectification task generated in a history;

if the matching is successful, updating the data modification task which is successfully matched based on the target service data, and taking the updated data modification task as a data modification task corresponding to the target service data;

and if the matching is unsuccessful, generating a new data rectifying task for the target service data.

Optionally, the sending unit is further configured to:

for each data modification task, the following operations are respectively executed:

for one data modification task, determining whether the total number of current data modification tasks of a data modification mechanism corresponding to the data modification task exceeds a first preset number threshold;

if the number of the data modification tasks exceeds the preset number threshold, the data modification mechanism is redistributed for the data modification tasks based on the data modification mechanism with the number of the data modification tasks smaller than a second preset number threshold;

And if the first preset quantity threshold value is not exceeded, taking the determined data modification mechanism as a data modification mechanism corresponding to the data modification task.

Optionally, the device further includes an updating unit, configured to:

receiving a task completion result returned by a data modification mechanism aiming at a corresponding data modification task, wherein the task completion result represents that the data modification mechanism has completed the data modification task, and the task completion result carries target service data modified by the data modification mechanism;

based on the corresponding data detection rule, carrying out data compliance detection on the rectified target service data;

and if the rectified target service data passes the data compliance detection, updating the task state of the data rectifying task to a finished state.

Optionally, the generating unit is further configured to:

acquiring a task state of at least one data rectification task generated within a preset time range;

determining task processing conditions of each data modification mechanism based on the task state of the at least one data modification task and the data modification tasks corresponding to each data modification mechanism;

Based on the task processing conditions of the data rectification mechanisms, a corresponding task schedule is generated, and the task schedule is sent to the terminal equipment of the appointed object.

Optionally, the sending unit is further configured to:

determining whether an overdue data modification task exists based on the task state and the task deadline of the at least one data modification task;

if overdue data modification tasks exist, task reminding information is sent to the data modification mechanism corresponding to each overdue data modification task.

In one aspect, a computer device is provided comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the methods described above when the computer program is executed.

In one aspect, there is provided a computer storage medium having stored thereon computer program instructions which, when executed by a processor, perform the steps of any of the methods described above.

In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the steps of any of the methods described above.

In the embodiment of the application, a to-be-detected service data set containing object information provided by a service object when corresponding service is handled in a history is obtained from a plurality of data sources, and for each piece of to-be-detected service data in the to-be-detected service data set, a data detection rule corresponding to the to-be-detected service data is determined from a data detection rule base containing the latest data detection rule of each service according to the corresponding service type and the data source so as to detect the data compliance. If the detection result indicates that target service data which does not meet the data detection rule exists in each service data to be detected, generating a corresponding data modification task for each target service data, determining a data modification mechanism corresponding to the data modification task according to the service object identification of each target service data, and finally transmitting each data modification task to the corresponding data modification mechanism.

According to the method, the high-efficiency detection and modification of massive service data are realized, the flow of manual intervention and the probability of manual errors are reduced, and the efficiency and accuracy of service data modification are greatly improved by automatically completing the acquisition of the service data set to be detected from a plurality of data sources, the data compliance detection of the service data to be detected, the automatic generation of the data modification task according to the detection result and the determination of the data modification mechanism corresponding to the data modification task according to the service object identification.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;

fig. 2 is a system architecture diagram of a data management device according to an embodiment of the present application;

fig. 3 is a flow chart of a service data processing method according to an embodiment of the present application;

fig. 4 is a schematic diagram of service data to be detected before and after format conversion according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a data modification task according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a task schedule provided by an embodiment of the present application;

fig. 7 is a schematic structural diagram of a service data processing device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application. Embodiments of the application and features of the embodiments may be combined with one another arbitrarily without conflict. Also, while a logical order is depicted in the flowchart, in some cases, the steps depicted or described may be performed in a different order than presented herein.

In order to facilitate understanding of the technical solution provided by the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

The resource: refers to an objective existence form with a certain amount of accumulation, which can be used for creating material wealth and mental wealth in the nature and human society, and comprises virtual resources and physical resources such as electronic funds or points.

Service data: the method refers to user information provided by a user in the financial field in the business handling process, such as identity information of the name, the identity card number, the address and the like of the user, account information of an electronic resource account number, the electronic resource account and the like.

Data detection rules: the detection rules established for the business data of different business types and different business data according to the related data quality requirements are used for detecting whether the business data accords with corresponding specifications and standards. For example, when a user opens an electronic resource account at a financial institution, it is necessary to detect whether the user's identification card number is correct and belongs to the user's own real information.

Task schedule: refers to a table or report showing the progress and status of the processing of the data rectification task. The related personnel can quickly know the processing condition and the completion state of each data rectification task according to the visualized task schedule.

The following briefly describes the design concept of the embodiment of the present application:

With the development of the age, the user may not meet the quality requirement of the latest service data in the past when transacting the compliance service data provided by the related service, so that in order to ensure that the user information required to be provided meets the latest service compliance and risk controllability requirement when transacting the service by the user, the financial institution needs to modify the compliance service data according to the latest data compliance requirement so as to improve the quality level of the service data.

At present, a data processing method aiming at service data is lacking, and a financial institution autonomously checks and rectifies massive service data manually, so that the operation is complex, the error rate is high, and the efficiency of service data rectification is low.

In view of the above problems, an embodiment of the present application provides a service data processing method, where a to-be-detected service data set including object information provided by a service object when handling corresponding services in history may be obtained from a plurality of data sources, and for each piece of to-be-detected service data in the to-be-detected service data set, according to a corresponding service type and a data source thereof, a data detection rule corresponding to the to-be-detected service data is determined from a data detection rule base including latest data detection rules of each service, so as to perform data compliance detection on the to-be-detected service data. If the detection result indicates that target service data which does not meet the data detection rule exists in each service data to be detected, generating a corresponding data modification task for each target service data, determining a data modification mechanism corresponding to the data modification task according to the service object identification of each target service data, and finally transmitting each data modification task to the corresponding data modification mechanism.

According to the method, the high-efficiency detection and modification of massive service data are realized, the intervention of manual operation and the probability of manual error are reduced, and the efficiency and accuracy of service data modification are greatly improved by automatically completing the acquisition of the service data set to be detected from a plurality of data sources, the data compliance detection of the service data to be detected, the automatic generation of the data modification task according to the detection result and the determination of the data modification mechanism corresponding to the data modification task according to the service object identification.

In order to further improve the accuracy of service data rectification, the application also establishes a data detection rule base containing the latest data detection rules of each service, and service data which does not meet the current service data quality requirement can be timely found through the latest data detection rules, thereby avoiding missing detection and misjudgment, ensuring the compliance of the service data and further improving the controllability of service risks.

In order to further improve the accuracy and efficiency of service data detection, the application also provides a method for converting the format of the service data to be detected through a preset format rule so as to ensure the uniformity and specification of the format of the service data to be detected, thereby reducing the complexity and error probability of processing the service data to be detected. Meanwhile, the business data to be detected after format conversion can be identified and processed by the data detection rule base more quickly, and the accuracy and the efficiency of data compliance detection are further improved.

In order to further improve the efficiency of service data modification, the application also matches the service object identification of the target service data with the service object identification of the historically generated data modification task, and when the matching is successful, the service object is indicated to have generated the corresponding data modification task, so that the target service data is directly updated to the existing data modification task without generating the data modification task again, and the efficiency of service data modification is improved.

In order to further improve the accuracy of the service data modification, the application also receives the task completion result returned by the data modification mechanism, timely knows the progress situation of the data modification task of each data modification mechanism, and carries out secondary detection on the modified target service data through the corresponding data detection rule so as to ensure the compliance of the service data.

In order to improve the flexibility of service data modification task allocation, after the corresponding data modification mechanism of the data modification task is determined through the service object identification, the application also judges whether the determined data modification mechanism is overloaded according to the preset number threshold, so that the data modification task is allocated to the data modification mechanism with fewer current tasks, the task load of all the data modification mechanisms is balanced, backlog and hysteresis of the data modification task caused by the overload of the data modification mechanism task are avoided, and the processing efficiency of the data modification task is further improved.

In order to achieve traceability of the service data modification task, the application also determines the task processing condition of each data modification mechanism through the obtained task state of each data modification task within a certain time range and the data modification task corresponding to each data modification mechanism, generates a task schedule and sends the task schedule to the terminal equipment of the appointed object, and can monitor the progress condition of the data modification task in real time through the task schedule so as to present the task schedule in a visual schedule mode, thereby improving the convenience of task schedule query, facilitating the supervision of related personnel on the service data modification task and promoting the timely processing of the data modification task.

In order to further improve the efficiency of service data processing, the application also judges whether the overdue data modification task exists according to the task state and the task period of each data modification task, and sends task reminding information to the data modification mechanism corresponding to the overdue data modification task, thereby effectively avoiding overdue risk of the service data modification task and improving the efficiency of service data processing.

The following description is made for some simple descriptions of application scenarios applicable to the technical scheme of the embodiment of the present application, and it should be noted that, in the technical scheme of the present application, the collection, propagation, use, etc. of data all meet the requirements of relevant national laws and regulations, and the application scenarios described below are only used for illustrating the embodiment of the present application, but not for limiting. In the specific implementation process, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

The technical scheme provided by the embodiment of the application can be suitable for the scene that the financial institutions carry out quality detection and rectification on business data. As shown in fig. 1, an application scenario provided in an embodiment of the present application may include a data management device 100 and a data modification device 101.

The data management device 100 may be a computer device with a certain processing capability, for example, a mobile phone, a personal computer (personal computer, PC), a server, etc. may be configured to execute any one of the method apparatuses provided in the embodiments of the present application, which is not illustrated here. For convenience of description, hereinafter, embodiments of the method will be described taking an execution subject of the method as a server capable of executing the method as an example. The server may be a server device corresponding to a data center responsible for design, development and maintenance of a data system of a financial institution, and based on the data rectification method provided by the embodiment of the present application, a to-be-detected service data set is obtained from a plurality of data sources such as other data management departments and systems of the financial institution, and a plurality of to-be-detected service data are subjected to data compliance detection, so that corresponding data rectification tasks are generated for target service data which do not meet the data detection rule, and each data rectification task is sent to the corresponding data rectification institution. It will be appreciated that the subject matter of the method being performed by the server is merely an exemplary illustration and should not be construed as limiting the method. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, but is not limited thereto. The data management device 100 may also configure a database, where the database may be used to store data detection rules, mapping relationships between service data and service object identifiers, intermediate data generated in a service data modification process, information to be stored sent by the data modification device, and so on, which are related in the scheme provided by the embodiment of the present application.

The data modification apparatus 101 is a mobile internet device (Mobile Internet Device, MID) of various forms corresponding to the data modification mechanism, that is, a terminal device that accesses the internet through a wireless network technology, for example, a smart phone, a vehicle-mounted smart terminal, a smart television, a notebook computer, a tablet computer, a Point of sale (POS), an automatic teller machine (Automated Teller Machine, ATM), a multimedia terminal, an internet banking machine, a cash deposit machine (Cash Deposit Machine, CDM), or any device that can be connected to the data management apparatus, receive a data modification task, and provide local services to personnel related to the data modification mechanism, and the embodiment is not limited specifically. The data rectifying mechanism can be a department or a mechanism responsible for data rectifying work in the financial mechanism and is responsible for ensuring that service data of the financial mechanism meet the latest data quality requirement and service requirement, for example, a branch mechanism of each place of the financial mechanism can be used as the data rectifying mechanism, a data rectifying group or a data rectifying department is arranged in the branch mechanism and is responsible for distributing data of a headquarter data center to the branch mechanism for rectifying work, and the safety and stability of the data are ensured.

The data management device 100 and the data modification device 101 may be connected through a network 102, where the network 102 may be a wired network, or may be a Wireless network, for example, a Wireless network may be a mobile cellular network, for example, a fourth generation mobile communication (4 g) network, a fifth generation mobile communication (5 g) network, or a New Radio (NR) network, or may be a Wireless-Fidelity (WIFI) network, or may be other possible networks, which may not be limited by the embodiments of the present application.

It should be noted that, the number of the data management device 100 and the data modification device 101 is not limited in practice, and is not particularly limited in the embodiment of the present application, as shown in fig. 1. And the components and structures shown in fig. 1 are exemplary only and not limiting, and other components and structures may be provided as desired in a practical scenario.

As shown in fig. 2, a system architecture diagram of a data management device according to an embodiment of the present application, where the data management device specifically includes the following modules:

(1) The data acquisition module is used for acquiring the service data set to be detected from a plurality of data sources and sending the service data set to the data detection module.

(2) And the rule management module is used for storing the latest data detection rule of each service and sending the latest data detection rule to the data detection module so as to enable the latest data detection rule to carry out data compliance detection on the service data to be detected.

(3) The data detection module is used for receiving the to-be-detected service data set transmitted by the data acquisition module, acquiring corresponding data detection rules from the rule management module according to the service types and the data sources of each piece of to-be-detected service data, carrying out data compliance detection on the to-be-detected service data according to the data detection rules, determining whether target service data which does not meet the data detection rules exist in the to-be-detected service data according to detection results, and transmitting the target service data to the task generation module.

(4) The task generating module is used for receiving the determined target service data which does not meet the data detection rule from the data detection module, generating a corresponding data rectifying task for the target service data and sending the corresponding data rectifying task to the task distributing module.

(5) The task distribution module is used for receiving the data modification tasks generated by the task generation module, determining the data modification mechanisms corresponding to the data modification tasks according to the service object identifiers, and sending the data modification tasks to the corresponding data modification mechanisms so that the data modification mechanisms modify the target service data which does not meet the data detection rules.

It should be noted that the components and structures of the functional block diagram shown in fig. 2 are merely exemplary and not limiting, and that other components and structures may be provided as desired in a practical scenario.

The service data processing method provided by the exemplary embodiment of the present application will be described below with reference to the accompanying drawings in conjunction with the application scenario described above, and it should be noted that the application scenario described above is only shown for the convenience of understanding the spirit and principle of the present application, and the embodiment of the present application is not limited in any way in this respect.

Referring to fig. 3, a flow chart of a service data processing method provided by an embodiment of the present application is shown, an execution main body of the method is a data management device shown in fig. 1 and fig. 2, and a specific implementation flow of the method is as follows:

step 301: a set of traffic data to be detected is obtained that originates from a plurality of data sources.

In the embodiment of the application, in order to detect and rectify the service data such as the object information and the like provided by the service object when the service object transacts the service in history, a service data set to be detected containing a plurality of pieces of service data to be detected is comprehensively obtained from a plurality of data sources, and the subsequent data compliance detection and data rectification flows are carried out.

Specifically, taking a financial institution as an example, the data source may be various business systems in the financial institution, including but not limited to a customer relationship management system (Customer Relationship Management, CRM) storing business data such as personal identity information, contact information, family members, etc. of the customer, a resource account management system storing business data such as balance of the customer, running water of the resource account, type of the resource account, status of the resource account, transaction record, etc., and a product management system storing relevant information of financial products such as stocks, bonds, foreign exchange, etc. handled by the customer.

Step 302: for each piece of to-be-detected business data in the to-be-detected business data set, determining the data detection rule corresponding to each piece of to-be-detected business data from the data detection rule base based on the business type and the data source corresponding to each piece of to-be-detected business data.

In the embodiment of the application, in order to improve the accuracy of service data detection, the data detection rule base comprises the latest data detection rule corresponding to each service, and each data detection rule has a mapping relation with the service type and the data source of the service data to be detected, so that the data detection rule matched with the service data to be detected can be determined from the data detection rule base according to the data type and the data source of the service data to be detected, thereby completing the data compliance detection of the service data to be detected.

Specifically, taking a financial institution as an example, the service data to be detected may come from a resource account management system of the financial institution, the service type may be user salary flow information, and according to the data source and the service type, a corresponding latest data detection rule may be determined, for example, may be: the user needs to provide a continuous payroll record for the last 6 months while transacting the resource account. When the salary flow record information indicated by the service data to be detected has a break or does not meet the condition of 6 months, the target service data which is not in accordance with the data detection rule can be determined.

The service data set to be detected can also come from a credit investigation system of a financial institution, the service type of the service data to be detected can be a personal credit report provided by a user when the user transacts the financial service, and the corresponding data detection rule can be that the credit investigation rating of the user is not lower than the B level and no overdue record exists when the user transacts the financial service. If the credit rating of the customer is lower than the B level or an overdue record exists, the customer can be considered to be target business data which does not accord with the data detection rule.

Step 303: and based on the obtained data detection rules, respectively carrying out data compliance detection on the corresponding service data to be detected.

In the embodiment of the application, after the corresponding data detection rule is determined according to the service type and the data source of the service data to be detected, the corresponding data to be detected is subjected to data compliance detection according to the data detection rule so as to determine whether each data to be detected meets the latest data quality requirement of the corresponding service.

In one possible implementation manner, since each service data to be detected may be derived from data sources of different departments, different systems and different areas, and each service type is different, each service data to be detected may have a problem of difference in data format, for example, each obtained service data to be detected may include each data type of text, picture, audio, video and the like, and each service data to be detected which is the same as the text data type may also have different text structures and the like, so that data compliance detection may not be performed normally or correct results may not be detected accurately. Therefore, in order to reduce the complexity and error probability of performing data compliance detection on the service data to be detected in different formats and improve the efficiency of data compliance detection, before performing data compliance detection on the service data to be detected, format conversion can be performed on each piece of service data to be detected according to a preset format rule, and data compliance detection can be performed on each piece of service data to be detected after format conversion.

In one possible implementation manner, the preset format rule associated with the above parameter may be preset according to the service types and data sources of different service data to be detected and the corresponding data detection rules.

Specifically, referring to fig. 4, taking the to-be-detected service data as a transaction record derived from the resource account management system as an example, fig. 4 (a) is an original format of the transaction record directly obtained from the resource account management system, and fig. 4 (b) is a transaction record after format conversion. Because the transaction record contains a plurality of parameter information such as transaction number, resource account number, transaction date, transaction amount, transaction type, transfer account number and the like, the corresponding data detection rule needs to detect part of specific parameters in the transaction record according to the corresponding judgment condition, for example, whether the transaction amount exceeds the balance of the corresponding resource account; detecting whether the transaction date is legal or not and is earlier than the account opening date of the resource account or not; detecting whether the transfer account is abnormal, such as whether the account is a blacklist account of a financial institution; it is detected whether the transaction type is legal, such as a money laundering transaction type. Therefore, the converted transaction record data can directly and quickly identify parameters to be detected and the matched judging conditions thereof, so that the efficiency of data compliance detection is improved.

Step 304: if the target service data which does not meet the data detection rule exists in each piece of service data to be detected based on the detection result, corresponding data modification tasks are respectively generated for each piece of target service data.

In the embodiment of the application, in order to improve the efficiency of service data modification, when the target service data which does not meet the data detection rule is determined to exist, corresponding data modification tasks are generated for each target service data according to a preset task generation strategy, so that a data modification mechanism can modify the target service data through the data modification tasks.

In one possible implementation manner, the data modification task may include parameters such as task content, task deadline, task status, and the like, and the parameter fields are filled with information carried by the determined target service data to generate a corresponding data modification task.

Specifically, as shown in fig. 5, taking the service data set to be detected as the resource account management system of the self-owned financial institution, the service type can be taken as an example of user account opening information, the user account opening information includes information such as a customer name, an identity card number, a mobile phone number and the like, and the corresponding data detection rule is determined according to the data source and the service type, so that the customer name is required to be Chinese, the identity card number is required to be 18 digits, and the mobile phone number is required to be 11 digits. When the mobile phone number of the customer account opening information data is determined to be only 10 digits through data compliance detection, and the corresponding data detection rule is not met, a corresponding data rectifying task is automatically generated according to the target service data, the task content is that the mobile phone number of the customer account opening information data is modified to be the correct 11 digits, the task deadline can be determined from a preset mapping relation between the task deadline and the service type through the service type corresponding to the target service data, and the mobile phone number can also be set by a person, and the default task state is incomplete when the data rectifying task is just generated. The data modification mechanism receiving the data modification task is responsible for modifying the mobile phone number of the account opening information data according to the task parameters so as to ensure the accuracy and compliance of the data.

In a possible implementation manner, after the target service data is determined, the service object identifier corresponding to the target service data is matched with the service object identifier corresponding to the data modification task generated in history, when the matching is successful, the data modification task successfully matched with the target service data is updated through the target service data, and the updated data modification task is directly used as the data modification task corresponding to the target service data, so that a plurality of data modification tasks can be prevented from being repeatedly generated for the same service object, calculation resources are saved, the data modification efficiency is improved, and the data modification mechanism can also directly modify all the non-compliant target service data related to the same service object through one data modification task, so that the same service object is prevented from being disturbed repeatedly, and the user experience is prevented from being influenced. When the matching fails, a new data rectifying task can be directly generated for the target service data.

Specifically, taking a financial institution related business scenario as an example, the business object identifier may be a user number, a user resource account number, a transaction flow number, and the like. For example, a user may open a resource account at a financial institution, where each resource account has a unique resource account number corresponding to the user identity, so that service data related to the user may use the resource account number as a service object identifier, thereby inducing all non-compliant target service data related to the same service object to a data modification task, and improving data modification efficiency.

Step 305: and determining a data modification mechanism corresponding to the corresponding data modification task based on the service object identifier corresponding to each item of label service data, and sending each data modification task to the corresponding data modification mechanism.

In the embodiment of the application, after the corresponding data modification task is generated for the target service data, each data modification task is distributed to the corresponding data modification mechanism according to the service object identification, and the modification of the service data is automatically completed, so that the high-efficiency detection and modification of mass service data are realized, the intervention of manual operation and the probability of manual error are reduced, and the efficiency and accuracy of service data modification are improved.

In one possible implementation manner, in order to improve flexibility of service data modification task allocation, after determining a data modification mechanism corresponding to each data modification task, it may be determined whether the total number of current data modification tasks of the data modification mechanism exceeds a first preset number threshold, if the total number of current data modification tasks exceeds the preset number threshold, the data modification mechanism is indicated to be overloaded, and according to the data modification mechanism with the number of data modification tasks smaller than a second preset number threshold, that is, the data modification mechanism with the smaller number of current tasks, the corresponding data modification mechanism is redistributed to the data modification task, and when the first preset number threshold is not exceeded, the determined data modification mechanism is still used as the data modification mechanism corresponding to the data modification task, so that task loads of all the data modification mechanisms are balanced, and backlog and hysteresis of the data modification task due to overload of the data modification mechanism are avoided, and further processing efficiency of the data modification task is improved.

In one possible implementation manner, the first preset number threshold and the second preset number threshold may be determined by comprehensively analyzing the situation of processing tasks of each data modification mechanism in history, for example, according to the task number and the completion situation of each data modification mechanism in history, it is determined that the maximum number of data modification tasks that can be processed simultaneously by the data modification mechanism is 10, and under the processing speed meeting the requirement, the maximum number of data modification tasks that can be processed simultaneously by the data modification mechanism is 5, so that the same first preset number threshold and second preset number threshold can be uniformly set for all the data modification mechanisms to be 10 and 5 respectively, and also the historical processing situation of each data modification mechanism can be combined, the processing capacity of each data modification mechanism is analyzed, and different preset number thresholds are respectively set for each data modification mechanism by combining factors such as business requirement and budget.

Specifically, taking the current financial institution having three data modification institutions A, B, C responsible for modifying the target business data, the first preset number of the first tasks is 10, and the second preset number of the second tasks is 5 as an example. When the data modification mechanism corresponding to the current data modification task is determined to be A through the service object identifier, and under the condition that one data modification task is newly added, the number of the current data modification tasks of the data modification mechanism A is 11 and exceeds the first preset number threshold, so that the data modification task needs to be reassigned to the data modification mechanism C which does not exceed the second preset number threshold, and the data modification efficiency is ensured.

In a possible implementation manner, the application can also indicate that the data modification task is completed by receiving the task completion result of the target service data which is returned by the data modification mechanism for the corresponding data modification task, timely know the progress condition of the data modification task of each data modification mechanism, and perform data compliance detection on the modified target service data through the corresponding data detection rule, and update the task state of the data modification task to the completed state when the modified target service data is determined to pass the data compliance detection, and ensure the accuracy of service data modification through secondary data compliance detection.

In the embodiment of the application, after each data modification task is distributed to the corresponding data modification mechanism, the statistics of the processing condition of each data modification mechanism can be realized, and the statistical flow of the processing condition of the data modification mechanism is introduced below.

In order to enable a service data modification task to have traceability, the application determines the task processing condition of each data modification mechanism and generates a corresponding task schedule according to the task state and the data modification task corresponding to each data modification mechanism by acquiring the task state of the data modification task generated in a preset time range and sends the task state and the data modification task corresponding to each data modification mechanism to the terminal equipment of an appointed object.

In a possible implementation manner, as shown in fig. 6, the task schedule includes, besides the task processing conditions of each data modification mechanism and the corresponding data modification task, the task completion ratio of each data modification mechanism, so that the designated object can monitor the progress condition of the data modification task in real time from multiple levels of the data modification mechanism, the data modification task and the like through the visual chart or the table-form task schedule, thereby improving the convenience of task progress query, facilitating the designated object to query the progress of the business data modification task, and promoting the data modification task to be processed in time.

In one possible implementation manner, the query authority of different task progress targets can be set according to the identity type of the query object, and different levels of task progress information can be provided for different objects so as to realize hierarchical and angular task management. For example, a supervisor can view the progress of all data modification tasks, while a general employee can only view the progress of the data modification tasks that he is responsible for processing.

In a possible implementation manner, the application can also determine whether the overdue data modification task exists or not through the task state and the task period of the data modification task, and when the overdue data modification task exists, the application sends the task reminding information to the corresponding data modification mechanism, so that the data modification task can be processed in time, and the service data processing efficiency is improved.

Referring to fig. 7, based on the same inventive concept, an embodiment of the present application further provides a service data processing apparatus 70, including:

an obtaining unit 701, configured to obtain to-be-detected service data sets from a plurality of data sources, where each to-be-detected service data in the to-be-detected service data sets is object information provided by a corresponding service object when the corresponding service object handles a corresponding service in a history;

a determining unit 702, configured to determine, for each to-be-detected service data in the to-be-detected service data set, a data detection rule corresponding to each to-be-detected service data from the data detection rule base based on a service type and a data source corresponding to each to-be-detected service data; the data detection rule base comprises the latest data detection rule of each service;

a detection unit 703, configured to perform data compliance detection on corresponding service data to be detected based on the obtained data detection rules;

a generating unit 704, configured to generate a corresponding data modification task for each item of target service data if it is determined, based on the detection result, that the target service data that does not satisfy the data detection rule exists in each item of service data to be detected;

And the sending unit 705 is configured to determine a data modification mechanism corresponding to the corresponding data modification task based on the service object identifier corresponding to each item of standard service data, and send each data modification task to the corresponding data modification mechanism.

Optionally, the detecting unit 703 is specifically configured to:

respectively carrying out format conversion on each piece of business data to be detected according to a preset format rule;

Optionally, the detecting unit 703 is specifically configured to:

if the matching is unsuccessful, a new data rectifying task is generated for the target service data.

Optionally, the sending unit 705 is further configured to:

for one data modification task, determining whether the total number of current data modification tasks of the data modification mechanism corresponding to the data modification task exceeds a first preset number threshold;

if the number of the data modification tasks exceeds the preset number threshold, the data modification mechanism is reassigned to the data modification task based on the data modification mechanism with the number of the data modification tasks smaller than the second preset number threshold;

Optionally, the apparatus further comprises an updating unit 706, specifically configured to:

receiving a task completion result returned by the data modification mechanism aiming at the corresponding data modification task, wherein the task completion result characterizes that the data modification mechanism has completed the data modification task, and the task completion result carries target service data modified by the data modification mechanism;

based on the corresponding data detection rule, carrying out data compliance detection on the target service data after modification;

and if the target service data after the rectification is determined to pass the data compliance detection, updating the task state of the data rectification task to a finished state.

Optionally, the generating unit 704 is further configured to:

determining the task processing condition of each data modification mechanism based on the task state of at least one data modification task and the data modification task corresponding to each data modification mechanism;

based on the task processing conditions of each data modification mechanism, a corresponding task schedule is generated, and the task schedule is sent to the terminal equipment of the appointed object.

Optionally, the sending unit 705 is further configured to:

determining whether an overdue data modification task exists based on a task state and a task deadline of at least one data modification task;

By the device, the to-be-detected service data set containing object information provided by the service object when corresponding service is handled in history is obtained from a plurality of data sources, and the data detection rule corresponding to the to-be-detected service data is determined from the data detection rule base containing the latest data detection rule of each service according to the corresponding service type and data source of each to-be-detected service data in the to-be-detected service data set so as to detect the data compliance. If the detection result indicates that target service data which does not meet the data detection rule exists in each service data to be detected, generating a corresponding data modification task for each target service data, determining a data modification mechanism corresponding to the data modification task according to the service object identification of each target service data, and finally transmitting each data modification task to the corresponding data modification mechanism. According to the method, the high-efficiency detection and modification of massive service data are realized, the flow of manual intervention and the probability of manual errors are reduced, and the efficiency and accuracy of service data modification are greatly improved by automatically completing the acquisition of the service data set to be detected from a plurality of data sources, the data compliance detection of the service data to be detected, the automatic generation of the data modification task according to the detection result and the determination of the data modification mechanism corresponding to the data modification task according to the service object identification.

For convenience of description, the above parts are respectively described as being functionally divided into unit modules (or modules). Of course, the functions of each unit (or module) may be implemented in the same piece or pieces of software or hardware when implementing the present application. The apparatus may be used to perform the methods shown in the embodiments of the present application, and therefore, the description of the foregoing embodiments may be referred to for the functions that can be implemented by each functional module of the apparatus, and the like, which are not repeated.

Referring to fig. 8, based on the same technical concept, the embodiment of the application further provides a computer device. In one embodiment, the computer device may be, for example, the data management device shown in fig. 1 and 2. The computer device, as shown in fig. 8, may include a memory 801, a communication module 803, and one or more processors 802.

A memory 801 for storing a computer program for execution by the processor 802. The memory 801 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system; the storage data area may store various sets of operation instructions, etc.

The memory 801 may be a volatile memory (RAM) such as a random-access memory (RAM); the memory 801 may also be a nonvolatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk (HDD) or a Solid State Drive (SSD); or memory 801, is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 801 may be a combination of the above memories.

The processor 802 may include one or more central processing units (central processing unit, CPU) or digital processing units, etc. A processor 802 for implementing the above-described service data processing method when calling the computer program stored in the memory 801.

The communication module 803 is configured to communicate with a data rectification device or other network device.

The specific connection medium between the memory 801, the communication module 803, and the processor 802 is not limited in the embodiment of the present application. The embodiment of the present application is illustrated in fig. 8 by a bus 804 between the memory 801 and the processor 802, where the bus 804 is illustrated in fig. 8 by a bold line, and the connection between other components is merely illustrative, and not limiting. The bus 804 may be classified as an address bus, a data bus, a control bus, or the like. For ease of description, only one thick line is depicted in fig. 8, but only one bus or one type of bus is not depicted.

The memory 801 stores a computer storage medium in which computer executable instructions for implementing the service data processing method provided by the embodiment of the present application are stored. The processor 802 is configured to execute the service data processing method provided in the foregoing embodiments.

Based on the same inventive concept, the embodiments of the present application also provide a storage medium having stored thereon a computer program which, when executed on a computer, causes a computer processor to perform the steps in the service data processing method provided according to the various embodiments of the present application described above in the present specification.

In some possible embodiments, aspects of the service data processing method provided by the present application may also be implemented in the form of a program product, which includes a program code for causing a computer device to perform the steps of the service data processing method according to the various exemplary embodiments of the present application as described in the present specification, when the program product is run on the computer device, for example, the computer device may perform the steps of the embodiments.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product of embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code and may run on a computing device. However, the program product of the present application is not limited thereto, and in the present application, the readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with a command execution system, apparatus, or device.

The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a command execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's equipment, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the elements described above may be embodied in one element in accordance with embodiments of the present application. Conversely, the features and functions of one unit described above may be further divided into a plurality of units to be embodied.

Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to either imply that the operations must be performed in that particular order or that all of the illustrated operations be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A method for processing service data, the method comprising:

2. The method of claim 1, wherein prior to separately performing data compliance detection on the corresponding service data to be detected based on each obtained data detection rule, the method further comprises:

based on each obtained data detection rule, respectively performing data compliance detection on the corresponding service data to be detected, including:

3. The method of claim 1, wherein if it is determined, based on the detection result, that the target service data that does not satisfy the data detection rule exists in each piece of service data to be detected, generating corresponding data modification tasks for each piece of target service data, respectively, includes:

4. The method of claim 1, wherein after determining the data modification mechanism corresponding to the corresponding data modification task based on the service object identifier corresponding to each item of standard service data, the method further comprises:

5. The method of claim 1, wherein after each data modification task is sent to a corresponding data modification facility, the method further comprises:

6. The method of any one of claims 1-5, further comprising:

7. The method of claim 6, wherein after acquiring the task state of the at least one data rectification task generated within the preset time range, the method further comprises:

8. A traffic data processing apparatus, the apparatus comprising:

9. The apparatus of claim 8, wherein the generating unit is specifically configured to:

10. The apparatus of claim 8, wherein the transmitting unit is further for:

11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that,

the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 7.

12. A computer storage medium having a computer program stored thereon, characterized in that,

Which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

13. A computer program product comprising a computer program, characterized in that,