CN113495902A

CN113495902A - Data processing method and data standard management system

Info

Publication number: CN113495902A
Application number: CN202010197689.7A
Authority: CN
Inventors: 柴永明; 宋国英; 崔静
Original assignee: Ministry Of Industry And Information Technology Of People's Republic Of China; Huawei Technologies Co Ltd
Current assignee: Ministry Of Industry And Information Technology Of People's Republic Of China; Huawei Technologies Co Ltd
Priority date: 2020-03-19
Filing date: 2020-03-19
Publication date: 2021-10-12
Also published as: WO2021184995A1

Abstract

The application discloses a data processing method and a data standard management system, and belongs to the field of data processing. The method comprises the following steps: acquiring information of a first database table structure of a first business system to be online, wherein the information of the first database table structure comprises information of at least one data element; checking information of the first database table structure based on a data standard library, wherein the data standard library comprises information of a plurality of standard data elements; and after the information of the first database table structure is verified, acquiring a target database table structure of the first business system, wherein the target database table structure is determined based on the verified first database table structure. The method and the device can improve the quality of the data provided by the service system after the service system is online and reduce the probability of data conversion after the service system is online. The method and the device are used for data processing of the business system.

Description

Data processing method and data standard management system

Technical Field

The present application relates to the field of data processing, and in particular, to a data processing method and a data standard management system.

Background

Data standards are data meaning and business rules that need to be followed together within a given group (e.g., an industry or an organization). Data standards are typically represented in the form of a table structure (also known as a database table structure) that includes a plurality of business fields. Each service field in the table structure is called a data element (also called a data element) and is a basic building block of the data standard. The information of the data elements is all in accordance with the business rules, and the information comprises the contents of names, definitions, structures, value-taking rules and the like.

Currently, some business systems maintain their own database table structures, which are data processing conditions under which the business systems provide data services. In order to implement the management of data standards, a data standard management system is proposed, which stores information of data standards to be complied with by one or more industries, and the data standards may include national standards (national standards for short), industry standards (line standards for short) and/or local standards (landmarks for short). After a certain service system is on line, the data standard management system can check the information of the database table structure maintained in the service system; if the database table structure does not meet the requirement of the target data standard corresponding to the business system and stored by the data standard management system, the data standard management system establishes the mapping relation between the database table structure and the target data standard (namely the mapping relation between the data elements of the database table structure and the data elements of the target data standard), and when the business system provides data service for the outside every time, the data standard management system converts the data meeting the conditions defined by the database table structure into the data meeting the target data standard based on the mapping relation and outputs the converted data.

However, the service system managed by the data standard management system needs to perform data conversion each time a data service is provided to the outside, which affects the efficiency of providing the data service.

Disclosure of Invention

The embodiment of the application provides a data processing method and a data standard management system. The technical scheme is as follows:

in a first aspect, a data processing method is provided, which includes:

acquiring information of a first database table structure of a first business system to be online, wherein the information of the first database table structure comprises information of at least one data element; checking the information of the first database table structure based on a data standard library, wherein the data standard library comprises information of a plurality of standard data elements; and after the information of the first database table structure is verified, acquiring a target database table structure of the first business system, wherein the target database table structure is determined based on the verified first database table structure.

According to the business data processing method provided by the embodiment of the application, before the business system is on line, the information of the database table structure of the business system is checked, so that the business system can adopt an accurate target database table structure after being on line. Compared with the prior art, the reliability of the target database table structure is higher, so that the quality of data provided after the business system is online is improved, the probability of data conversion after the business system is online is reduced, and the cost of later-stage data management is reduced.

The data standard management system may have various ways of acquiring the information of the first database table structure of the first service system to be online, and the following two ways are taken as examples in the embodiment of the present application for explanation:

in a first alternative, the information of the first database table structure is obtained by means of online obtaining.

The information of the first database table structure and the information of the standard data element both include data element identification and data element description information, and the obtaining of the information of the first database table structure of the first service system to be online includes:

receiving a data element identifier of the first data element, wherein the data element identifier of the first data element is one of data element identifiers of a plurality of standard data elements stored in the data standard library; and acquiring data element description information corresponding to the data element identification of the first data element in the data standard library.

In one possible implementation, the data criteria management system supports a keyword search function. The data element identification of the first data element is obtained by searching the data element identifications of a plurality of standard data elements stored in the data standard library through the keyword corresponding to the first data element.

In one possible implementation, the data criteria management system also supports a search prompt function to ensure that business personnel efficiently determine the data element identification for the first data element.

In one possible implementation, prior to the receiving the data element identification of the first data element, the method further comprises: matching the received keyword with data element identifications of a plurality of standard data elements stored in the data standard library; and outputting a matching result, wherein the matching result comprises the information of the at least one second standard data element, and the data element identification of each second standard data element in the at least one second standard data element is matched with the keyword.

In one possible implementation, the aforementioned algorithm for matching the received keyword with the data element identifications of the plurality of standard data elements may be a fuzzy matching algorithm, wherein the fuzzy matching algorithm refers to giving a certain degree of accuracy to matching according to the proposed conditions or requirements. The fuzzy matching principle is that the content which is the same as the searched content is searched first, and the content which is very close to the searched content is not searched any more. In the present application, the fuzzy matching algorithm also allows the parts of the keywords used for searching to be out of word or spaced. The search content may include synonyms of keywords, near synonyms, related words, phrases containing keywords, and the like.

The matching result obtained by adopting the fuzzy matching algorithm can comprise an accurate matching result and a result except the accurate matching, compared with the method of simply adopting the accurate matching algorithm to obtain the information of the second standard data element, the fuzzy matching content is wider, the number of the obtained information of the second standard data element is more, and the reference of the matching result to business personnel is improved. The exact matching algorithm means that matching is determined only when the searched keyword and the data element identifier of the standard data element are completely consistent in literal, and the matching limit is exact and strict.

Optionally, if there are a plurality of second standard data elements obtained by matching, in the matching result, the information of the plurality of second standard data elements may be arranged according to a first specified order. For example, this first specified order may be implemented by the following two exemplary implementations:

in a first exemplary implementation manner, the information of the plurality of second standard data elements is sorted in descending order of the matching degree of the data element identification of the second standard data element and the keyword (i.e. sorted in the order of the matching degree from large to small).

For example, for any second standard data element, the calculation manner of the matching degree of the data element identifier of the second standard data element and the keyword may be various, for example, the matching degree P1 satisfies the first matching degree calculation formula:

P1＝M/N；

wherein, M is the number of characters of the data element identification of the second standard data element which is the same as the number of characters of the key word, and is equivalent to the number of characters corresponding to the intersection of the data element identification of the second standard data element and the key word; n is the maximum number of characters in the number of characters of the data element identification and the number of characters of the keyword of the second standard data element.

In a second exemplary implementation, the information of the plurality of second-criterion data elements is sorted in descending order of priority of the data criterion to which the second-criterion data elements belong (i.e. sorted in order of priority from high to low).

Optionally, the priority of the data criteria may include a criteria priority or a time priority. The standard priority refers to the priority of the standard itself. The standard priority is sequentially ranked from high to low as national standard, row standard and landmark. Temporal priority is generally the higher the priority the closer the publication time or the implementation time is to the current.

It is noted that the first designated order can also be in other manners, for example, the first designated order is an order determined by combining the first and second exemplary implementations described above. That is, the information of the plurality of second standard data elements is sorted according to the matching degree of the data element identifier of the second standard data element and the keyword and the priority of the data standard to which the second standard data element belongs. For example, for each second standard data element, the data standard management system may obtain a matching degree between the data element identifier of the second standard data element and the keyword, obtain a priority of the data standard to which the second standard data element belongs, and assign a value to the priority of the data standard to which the second standard data element belongs according to a specified rule, where the priority is positively correlated with the assigned value, that is, the higher the priority is, the higher the value is; and then, according to weights respectively allocated to the matching degree and the priority in advance, determining the ordering indication value of the second standard data element in a weighted summation mode based on the matching degree and the priority corresponding to the second standard data element. And finally, the data standard management system carries out the sequencing of the information of each second standard data element according to the sequencing indication value corresponding to each second standard data element. Typically sorted in descending order according to the sorting indicator.

By sequencing the information of the plurality of second standard data elements according to the first designated sequence, the prompt effective for the service personnel can be improved, and the prompt hit rate can be improved.

In a second alternative, the information of the first database table structure is obtained by receiving an offline edited data standard document.

In one possible implementation, the obtaining information of the first database table structure of the first business system to be online includes:

a data standard document is received, the data standard document including information of the first database table structure.

The third-party modeling tool can access (such as query) the data standard library, obtain information of the standard data elements stored in the data standard library, and generate the data standard document based on the information of the standard data elements. Accordingly, the process of receiving a data standard document includes: receiving the data standard document generated by the third party modeling tool based on the data standard library.

The third-party modeling tool can support generation of data standard documents meeting the requirements of the data standard library, and all or part of information of the corresponding first database table structure meets the requirements of the data standard library, so that the operation cost of verification can be reduced, and the verification cost is reduced.

outputting a data dictionary template, wherein the data dictionary template is a reference template of the information of the first database table structure;

information of the first database table structure input based on the data dictionary template is received.

The data standard management system is used for business personnel to refer by outputting the data dictionary template, so that the business personnel do not only formulate the information of the data elements by relying on own experience but also formulate the information of the data elements according to the ground, thereby improving the accuracy of the obtained information of the first database table structure and reducing the complexity and the operation cost of the subsequent verification process.

In this embodiment of the present application, checking information of the first database table structure may include at least two selectable modes, that is, data standard symbolic checking and data standard normative checking:

the first alternative is data standard symbolic check. The verification process refers to verifying information of at least one data element based on information of standard data elements in a data standard library. The information of the standard data elements is the aforementioned "symbol".

In one possible implementation, the process of checking the information of the first database table structure based on the database standard includes:

when the information of the first data element is not matched with the information of the plurality of standard data elements, sending first modification prompt information, wherein the first modification prompt information indicates that the information of the first data element is updated, and the first data element is one of the at least one data element; and after receiving the updated information of the first data element matched with the information of any one of the plurality of standard data elements, determining that the information verification of the first data element is successful.

The business personnel can realize the multiple modification of the information of the first data element through the first modification prompt information sent by the data standard management system for multiple times so as to meet the requirement of the standard data element in the data standard library, so that the business personnel can define the information of the data element consistent with the information of the standard data element in the data standard library.

In one possible implementation, the information of the first database table structure and the information of the standard data elements each include a data element identification, the first modification hint information includes information of at least one first standard data element, and the data element identification of each of the at least one first standard data element is in fuzzy match with the data element identification of the first data element. For example, the fuzzy matching algorithm is a search algorithm in the elastic search.

The matching result obtained by adopting the fuzzy matching algorithm can comprise an accurate matching result and a result except the accurate matching, compared with the method of simply adopting the accurate matching algorithm to obtain the information of the first standard data element, the fuzzy matching content is wider, the number of the obtained information of the first standard data element is more, and the reference of the matching result to business personnel is improved.

In one possible implementation, if there are a plurality of first standard data elements obtained by matching, the information of the plurality of first standard data elements may be arranged in a second specified order in the first modification hint information. For example, this second designated order may be implemented by the following two exemplary implementations:

in a first exemplary implementation, the information of the plurality of first standard data elements is sorted in descending order of the matching degree of the data element identification of the first standard data element and the keyword (i.e. sorted in the order of the matching degree from large to small).

In a second exemplary implementation, the information of the plurality of first-criterion data elements is sorted in descending order of priority of the data criterion to which the first-criterion data elements belong (i.e. sorted in order of priority from high to low).

It is noted that the second designated order can also be defined in other ways, for example, the second designated order is an order determined by combining the first and second exemplary implementations described above. That is, the information of the plurality of first standard data elements is sorted according to the matching degree of the data element identifier of the first standard data element and the data element identifier of the first data element and the priority of the data standard to which the first standard data element belongs. For example, for each first standard data element, the data standard management system may obtain a matching degree between the data element identifier of the first standard data element and the data element identifier of the first data element, obtain a priority of the data standard to which the first standard data element belongs, and assign a value to the priority of the data standard to which the first standard data element belongs according to a specified rule, where the priority is positively correlated with the assigned value; and then, according to weights respectively allocated to the matching degree and the priority in advance, determining the sorting indication value of the first standard data element in a weighted summation mode based on the matching degree and the priority corresponding to the first standard data element. And finally, the data standard management system carries out sequencing on the information of each first standard data element according to the sequencing indication value corresponding to each first standard data element. Typically sorted in descending order according to the sorting indicator.

By sequencing the information of the plurality of first standard data elements according to the second specified sequence, the effective prompt for service personnel can be improved, and the prompt hit rate can be improved.

In one possible implementation, the method further comprises: receiving the updated information of the first database table structure;

determining an incremental data element in the updated information of the first database table structure, and determining the information of the first data element in the incremental data element information; alternatively, the information for the first data element is determined from the information for the full amount of data elements of the updated information for the first database table structure.

The second alternative, data standardization check. The verification process refers to verifying the normalization of the information of the data standard. The format of the information of the data standard is mainly checked.

The checking the information of the first database table structure based on the data standard base comprises the following steps:

when the format of the information of the first database table structure does not meet the specified format requirement, sending second modification prompt information, wherein the second modification prompt information indicates the format of the information of the first database table structure to be updated; and after receiving the updated information of the first database table structure with the format conforming to the format requirement, determining that the format verification of the information of the first database table structure is successful.

The business personnel can realize the multiple modification of the format of the information of the first database table structure through the second modification prompt information sent by the data standard management system for multiple times so as to meet the requirement of the data standard management system on the format of the standard information, and the business personnel can define the information of the database table structure which meets the composite requirement.

In one possible implementation, after the obtaining the information of the first database table structure of the first business system to be online, the method further includes:

when the value corresponding to the second data element is an enumeratable value, adding data element remark information for the second data element, where the data element remark information is used to identify the enumeratable value corresponding to the second data element, and the second data element is one of the at least one data element.

In this way, after the information verification of the subsequent first database table structure is completed, adding the data element remark information for the second data element may still be included in the target database table structure determined based on the first database table structure. After the first service system is on-line, if data corresponding to the second data element needs to be acquired, the data can be directly acquired according to the format of the enumerable numerical value corresponding to the second data element, so as to ensure that the finally acquired data meets the format requirement of the data standard library, namely, meets the relevant standard.

In one possible implementation, the method further comprises:

receiving a data standard library operation request, wherein the data standard library operation request comprises a standard data element adding request, a standard data element updating request, a standard data element deleting request or a standard data element inquiring request; and after the authentication of the operation request of the data standard library is successful, executing the operation corresponding to the operation request of the data standard library on the data standard library.

The embodiment of the application schematically provides the following authentication modes:

the first authentication mode comprises the following steps: authentication of data operations at high security levels.

When the data operation request is a standard data element addition request, a standard data element update request or a standard data element deletion request, the data standard management system detects whether an account carried in the data standard request is an account of a first level, where the first level is greater than a specified level threshold, for example, the account of the first level is an account of a system administrator.

And when the data standard management system detects that the account carried in the data standard request is not the account of the first grade, determining that the authentication of the operation request of the data standard library fails.

When the data standard management system detects that the account carried in the data standard request is an account of a first grade, in an optional mode, the data standard management system determines that the authentication on the operation request of the data standard library is successful; in another optional mode, the data standard management system sends the data standard library operation request to the terminal device corresponding to the account of the second level, and after receiving an allowing instruction indicating that the operation on the data standard library is allowed, the data standard management system determines that the authentication on the data standard library operation request is successful; and after receiving a prohibition instruction indicating that the operation on the data standard library is not allowed, determining that the authentication on the operation request of the data standard library fails. And the second level is higher than or equal to the first level, and the account number of the second level is different from the account number of the first level. For example, if the account at the second level is an account of a project administrator, the terminal device corresponding to the corresponding account at the second level is the first terminal device. After receiving a data standard library operation request through a first terminal device, a project manager determines whether corresponding personnel are allowed to operate the data standard library or not according to the content of the request and an account carried in the request, if the operation on the data standard library is allowed, an allowing instruction is sent through the first terminal device, and if the operation on the data standard library is not allowed, a forbidding instruction is sent through the first terminal device.

The second authentication mode comprises the following steps: authentication of data operations of low security level.

When the data operation request is a standard data element query request, the data standard management system detects whether an account carried in the data standard request is an account allocated by the data standard management system, that is, a legal account in the data standard management system.

And when the data standard management system detects that the account carried in the data standard request is not the account distributed by the data standard management system, determining that the authentication on the operation request of the data standard library fails.

When the data standard management system detects that the account carried in the data standard request is the account allocated by the data standard management system, in an optional mode, the data standard management system determines that the authentication on the operation request of the data standard library is successful; in another optional mode, the data standard management system sends the data standard library operation request to the terminal device corresponding to the account of the third level, and after receiving an allowing instruction indicating that the operation on the data standard library is allowed, the data standard management system determines that the authentication on the data standard library operation request is successful; and after receiving a prohibition instruction indicating that the operation on the data standard library is not allowed, determining that the authentication on the operation request of the data standard library fails.

The third grade is higher than or equal to the grade of the account number carried in the data standard request, and the account number of the third grade is different from the account number carried in the data standard request. For example, the account of the third level is an account of a project administrator or a system administrator, and if the account of the third level is an account of the system administrator, the terminal device corresponding to the corresponding account of the third level is the third terminal device. After receiving the data standard library operation request through the third terminal device, the system administrator determines whether to allow the corresponding personnel to operate the data standard library according to the content of the request and the account number carried in the request, if the operation on the data standard library is allowed, the system administrator sends an allowing instruction through the third terminal device, and if the operation on the data standard library is not allowed, the system administrator sends a prohibiting instruction through the third terminal device.

In the two authentication manners, if the data standard management system successfully authenticates the operation request of the data standard library, the data standard library is subjected to the operation corresponding to the operation request of the data standard library. In one possible implementation, the data standard management system may further send a data operation response indicating that the operation corresponding to the request for executing the data standard library operation on the data standard library is completed or indicating that the operation is successful.

And if the data standard management system fails to authenticate the operation request of the data standard library, forbidding to execute the operation corresponding to the operation request of the data standard library on the data standard library. In one possible implementation, the data standard management system may further send a data operation response indicating that the operation corresponding to the data standard library operation request is prohibited from being executed on the data standard library, or indicating that the operation fails.

In one possible implementation, the method further comprises: inquiring an operation log corresponding to the data standard library; and when the operation log comprises an abnormal operation log, sending an abnormal alarm. A system administrator can determine whether data operation backtracking needs to be carried out or not based on the abnormal alarm, so that data rollback is realized, and the data standard management system is effectively maintained.

In the embodiment of the application, the data standard management system also supports a data governance function. In an optional example, the data standard management system is pre-established with an artificial intelligence model, and data governance is performed through the artificial intelligence model. The data governance process may include the steps of:

acquiring information of a second database table structure of the second business system after the second business system is online, wherein the information of the second database table structure comprises information of at least one data element; when it is detected based on the artificial intelligence model that the information of the third data element does not match with the information of the plurality of standard data elements, determining a target standard data element corresponding to the third data element in the plurality of standard data elements, wherein the third data element is one of at least one data element included in the information of the second database table structure; and establishing a mapping relation between the third data element and the target standard data element based on the artificial intelligence model.

In one possible implementation, the data standard management system supports script output functionality. After the obtaining of the target database table structure of the first business system, the method further includes: and outputting the script corresponding to the target database table structure. The script is used to generate information of a target database table structure, which includes a database table building statement of the first business system. The service person can receive the script via the second terminal device. After the first service system is online, service personnel can load and run the script in the first service system, the running script can generate information of a target database table structure, and a corresponding database table structure is obtained according to the information construction of the target database table structure. Therefore, business personnel do not need to write scripts by themselves, the workload of the business personnel is reduced, and the labor cost is saved.

In a second aspect, there is provided a data standard management system, which may include at least one module, which may be configured to implement the data processing method provided in the first aspect or various possible implementations of the first aspect.

In a third aspect, the present application provides a computing device comprising a processor and a memory. The memory stores computer instructions; the processor executes the computer instructions stored by the memory to cause the computing device to perform the methods provided by the first aspect or the various possible implementations of the first aspect, to cause the computing device to deploy the data standard management system provided by the second aspect or the various possible implementations of the second aspect.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer instructions that instruct a computing device to execute the method provided by the above first aspect or various possible implementations of the first aspect, or instruct the computing device to deploy the data standard management system provided by the above second aspect or various possible implementations of the second aspect.

In a fifth aspect, the present application provides a computer program product comprising computer instructions stored in a computer readable storage medium. A processor of the computing device may read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computing device to perform the method provided by the above-mentioned first aspect or various possible implementations of the first aspect, to cause the computing device to deploy the data standard management system provided by the above-mentioned second aspect or various possible implementations of the second aspect.

A sixth aspect provides a chip which may comprise programmable logic circuits and/or program instructions for implementing a data processing method as claimed in any one of the first aspects when the chip is in operation.

Drawings

Fig. 1 is a schematic application environment diagram of a data standard management system according to a data processing method provided in an embodiment of the present application;

fig. 2 is a schematic application environment diagram of a data standard management system according to another data processing method provided in the embodiment of the present application;

FIG. 3 is a flow chart of a data processing method provided by an exemplary embodiment of the present application;

FIG. 4 is a schematic interface diagram provided by the data standard management system in the embodiment of the present application;

FIG. 5 is a schematic diagram of another exemplary interface provided by the data standard management system in an embodiment of the present application;

fig. 6 is a schematic structural diagram of a data standard management system according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of another data standard management system provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

For the convenience of the reader to understand, the embodiments of the present application first explain terms related to a data processing method provided by the embodiments of the present application.

Data standards are data meaning and business rules that are to be followed together within a given group (e.g., an industry or an organization). Data standards are typically represented in the form of a table structure that includes a plurality of service fields. Since data standard correspondence table structures are typically stored in databases, the data standard correspondence table structures are also referred to as database table structures, and the tables defined by the database table structures are database tables (also referred to as physical tables or data entities).

The data element refers to a business field in the database table structure and is a basic composition unit of the database table structure. The information of one data element includes information for describing an attribute of the data element. I.e. attribute information of a data element. The attribute information may include information for describing parameters of the attribute of the data element itself, such as information of parameters of name, definition, structure, and value-taking rule, and may also include information for describing parameters of the environment to which the data element belongs, such as name of the database table structure to which the data element belongs. Wherein, the data elements in the data standard are called standard data elements.

A codeset refers to a definition of an enumeratable number in a database table structure. An enumerable value means that there are many instances of the value. For example, a data element corresponds to an enumeratable number, which means that there are a plurality of values in the data element, but the number of values is limited. Because the name of a data element, the value in the content of the data element, the description of the value, etc. can be numerical values, for a data element, the code set refers to the definition of the enumerable numerical value when the numerical value corresponding to the data element is the enumerable numerical value. For example, the name of a data element is: gender, data element content of the data element has male and female 2 enumerable values, and the code set defines that the 2 enumerable values are represented by 0 and 1, respectively.

An application scenario related to the data processing method provided by the embodiment of the present application is described below.

Within a business or an organization, executing a project involves one or more business systems. The one or more business systems need to comply with the same data standard, such as national, line or landmark. Therefore, the working efficiency and the data quality of the project can be improved, and the working cost is reduced.

For example, assuming that an enterprise belonging to an organization X establishes a project, the project is implemented by a plurality of manufacturers, each manufacturer maintains one or more business systems (usually maintains one business system), the data standard management system related to the data processing method provided by the embodiment of the present application may implement consistency of data standards adopted by the business systems maintained by the plurality of manufacturers (that is, database table structures adopted by the manufacturers all meet requirements of the data standards). In the embodiment of the present application, an administrator of an enterprise is referred to as a project administrator, an administrator of a manufacturer is referred to as a service person, and a worker maintaining a data standard management system is referred to as a system administrator.

Optionally, the aforementioned organization X may be a communication organization, an electric power organization, a water conservancy organization or an agricultural organization, and correspondingly, the service system may be a communication service system, an electric power service system, a water conservancy service system or an agricultural service system, and so on.

Fig. 1 is a schematic application environment diagram of a data standard management system according to a data processing method provided in an embodiment of the present application. Referring to fig. 1, the data standard management system 10 includes:

a data standard library 101 and a data standard management device 102. The data standard library 101 and the data standard management device 102 are connected in wired or wireless communication. The database 101 may be a server or a server cluster consisting of a plurality of servers. The data standard library 101 is used for storing information of a plurality of standard data elements. Standard data elements refer to data elements in an established data standard, such as an established national standard, a line standard, or a landmark. The data standard management device 102 may be a server or a server cluster consisting of a plurality of servers or other computer devices. The data standard management device 102 is configured to manage some or all functions of one or more business systems, and check information of a database table structure of the managed business system through standard data elements stored in the data standard library 101.

Optionally, as shown in fig. 2, the data standard management system 10 may further include:

a lifecycle management device 103 and a data governance device 104. The life cycle management device 103 and the data governance device 104 are respectively in wired or wireless communication connection with the data standard library 101. The lifecycle management device 103 can be one server or a server cluster consisting of multiple servers. The lifecycle management device 103 is used to manage operations on the database criteria 101, such as add operations, update operations, delete operations, or query operations of the criteria data elements. Data governance device 104 may be a server or a cluster of servers or a cloud computing center. The data governance device 104 is used for performing data governance of the data standard library, such as establishing a mapping relationship between data elements in a database table structure and standard data elements in the data standard library 101.

It should be noted that, when the data amount required to be processed by the data standard management system is small or there is a device with high data processing performance, the multiple functions of the data standard library 101, the data standard management device 102, the life cycle management device 103 and the data governance device 104 may be implemented by corresponding modules of one device, that is, the functions of at least two devices of the data standard library 101, the data standard management device 102, the life cycle management device 103 and the data governance device 104 are integrated on one device. For example, the device may include a data standard storage module, a data standard management module, a life cycle management module, and a data governance module, which respectively implement the functions of the data standard library 101, the data standard management device 102, the life cycle management device 103, and the data governance device 104.

Optionally, the data standard management system further supports a task management function, and is configured to manage each service system and allocate a management right to a service person of each service system. Each service person can only manage databases and the like associated with the service systems assigned to him.

Further optionally, the data standard management system 10 may further include a first terminal device, a second terminal device, and a third terminal device corresponding to the project manager, the service person, and the system manager, respectively. The project manager, the service personnel and the system manager can access the data standard management system through corresponding terminal equipment.

Fig. 3 is a flowchart illustrating a data processing method according to an exemplary embodiment of the present application, where the data processing method may be applied to the data standard management system shown in fig. 1 or fig. 2. In the subsequent embodiments, assuming that the first service system to be online needs to check the first database table structure based on the data standard library, where the data standard library includes information of a plurality of standard data elements, as shown in fig. 3, the method includes:

step 201, the data standard management system obtains information of a first database table structure of a first service system to be online.

When a business person newly builds or rebuilds the first business system, the business person may build a database table structure for the first business system. The database table structure of the first business system can be one or more, the database table structure is self-defined by business personnel, and the business personnel can provide the database table structure maintained by the business personnel to the data standard management system so as to check the information of the database table structure by the data standard management system. For example, the business personnel can input the information of the database table structure into the data standard management system through one-time editing request or multiple editing requests by using the second terminal equipment.

The first database table structure is assumed to be one database table structure in the first business system, and the processing modes of other database table structures refer to the first database table structure. In an embodiment of the application, the information of the first database table structure includes information of at least one data element. The information of each data element in the information of the first database table structure comprises attribute information of the data element; in some scenarios, the information of a data element also includes a set of codes when the value to which the data element corresponds is an enumerable value. In this embodiment of the present application, information of a data element may also be divided into a data element identifier and data element description information, where the data element identifier is used to identify a corresponding data element, the data element identifier may include a data element name and/or a data element code (also referred to as a data element number), the data element name may include a chinese name and/or an english name, and the data element code may be a character string composed of numbers and/or letters used to identify the corresponding data element. The data element description information is used for describing the data element and comprises a part of the attribute information except the data element identification. In some optional cases, the data element description information further includes a code set, such as when the corresponding value of the data element is an enumerable value, the data element description information further includes the code set.

Optionally, the information of the first database table structure further includes information of the first database and first table information. The database table structure established by the business system needs to be stored in a designated space, so that the maintenance is convenient. Therefore, before the database table structure, the business personnel usually needs to establish a database for storing the information of the data elements included in the first database table structure. The aforementioned first database is a database for storing a first database table structure. The information of the first database may include attribute information such as the identity, address and/or structure of the database. The database identifier is used to identify the corresponding database, and may include a database name and/or a database code. The database name may include a chinese name and/or an english name, and the database code may be a character string composed of numbers and/or letters for identifying the corresponding database. The first table information is information in the first database table except for information of data elements, and the first table information can comprise attribute information such as identification and/or structure of the database table. The database table identification is used for identifying the corresponding database table, the database table identification can comprise a database table name and/or a database table code, the database table name can comprise a Chinese name and/or an English name, and the database table code can be a character string consisting of numbers and/or letters for identifying the corresponding database table.

The information of the first database table structure can be carried in an online editable data standard document. The information of the first database table structure mainly includes information of at least one data element, and the process of acquiring the information of the first database table structure of the first business system to be online mainly includes the process of acquiring the information of the at least one data element. The service personnel can input the information of a plurality of data elements to the data standard management system one by one in an online editing mode through the second terminal equipment used by the service personnel, and correspondingly, the data standard management system receives the information of the plurality of data elements one by one in an online obtaining mode. Other information in the information of the first database table structure, such as the information of the first database and the first table information, may also be input to the data standard management system in an online editing manner, and the data standard management system may receive the other information in the information of the first database table structure.

In the embodiment of the present application, it is assumed that the data element identifier of the first data element is one of data element identifiers of a plurality of standard data elements stored in the data standard library, and both the information of the first database table structure and the information of the standard data elements stored in the data standard library include a data element identifier and data element description information. The data standard management system may obtain information of the first data element through a data element identification of the first data element. The acquisition process of the information of the other data elements refers to the acquisition process of the information of the first data element.

The service person can input the data element identification of the first data element to the data standard management system through the second terminal device used by the service person. Correspondingly, the data standard management system receives a data element identifier of the first data element; since the first data element belongs to a plurality of standard data elements stored in the data standard library, the information of the first data element is stored in the data standard library. The data standard management system may query the data standard library based on the data element identifier of the first data element, so as to obtain the data element description information corresponding to the data element identifier of the first data element in the data standard library, so as to obtain the information of the first data element based on the data element identifier of the first data element and the data element description information.

In an alternative example, the service person may input the data element identifier of the first data element by himself or herself according to his or her own experience. For example, the data criteria management system may provide an input box at the user interface in which the business person enters the data element identification for the first data element.

In another alternative example, the data criteria management system supports a keyword search function. The data element identifier of the first data element can be obtained by searching the data element identifiers of a plurality of standard data elements stored in the data standard library through the keyword corresponding to the first data element. For example, the data criterion management system may provide a search box on the user interface, and the service person may input a keyword corresponding to the first data element in the search box, and the data criterion management system searches the data element identifiers of the plurality of standard data elements stored in the data criterion library based on the keyword to obtain the data element identifier of the first data element. Wherein the keyword is a character used for indexing the data element identification of the first data element, and may be composed of one or more characters, and the keyword may include: english characters, chinese characters, and/or numeric characters (also called numeric characters).

Optionally, the data standard management system further supports a search prompt function to ensure that the service personnel effectively determines the data element identifier of the first data element. For example, prior to receiving a data element identification for a first data element, the data criteria management system may match the received keyword with data element identifications for a plurality of criteria data elements stored by a data criteria library; and outputting a matching result, wherein the matching result comprises information of at least one second standard data element, and the data element identification of each second standard data element in the at least one second standard data element is matched with the keyword. In this way, the service person can obtain information of each second standard data element related to the keyword in the matching result, so as to select the first data element which is desired by the service person from at least one second standard data element. Correspondingly, the data standard management system receives the data element identification of the corresponding first data element after detecting the selection operation.

It should be noted that the matching result usually includes information of a specified number of second standard data elements, where the specified number is an integer greater than 1, so as to improve the referential of the matching result to the service personnel. And the algorithm for the data standard management system to match the received keyword with the data element identifications of the plurality of standard data elements may be a fuzzy matching algorithm, for example, a search algorithm in ElasticSearch (ES for short). Where fuzzy matching algorithms refer to giving a certain degree of accuracy of the matching according to the conditions or requirements set forth. The fuzzy matching principle is that the content which is the same as the searched content is searched first, and the content which is very close to the searched content is not searched any more. In the present application, the fuzzy matching algorithm also allows the parts of the keywords used for searching to be out of word or spaced. The search content may include synonyms of keywords, near synonyms, related words, phrases containing keywords, and the like.

The matching result obtained by adopting the fuzzy matching algorithm can comprise an accurate matching result and a result except the accurate matching, compared with the method of simply adopting the accurate matching algorithm to obtain the information of the second standard data element, the fuzzy matching content is wider, the number of the obtained information of the second standard data element is more, and the reference of the matching result to business personnel is improved. The exact matching algorithm means that matching is determined only when the searched keyword and the data element identifier of the standard data element are completely consistent in literal, and the matching limit is exact and strict. The matching algorithm provided in the embodiment of the present application may also be other algorithms, which is not limited herein.

It is to be noted that the data standard management system supports the condition search function in addition to the keyword search function. The corresponding search prompt function not only includes the result corresponding to the keyword search function, but also includes the result corresponding to the condition search function.

The data standard management system can provide a condition input box on the user interface, business personnel can input search conditions in the condition input box, and the data standard management system searches data element identifications of a plurality of standard data elements stored in the data standard base based on the search conditions to obtain the data element identifications meeting the search conditions. Further, the service personnel can search the keyword and simultaneously search the condition, and correspondingly, the data standard management system can accord with the search condition and identify the information of the second standard data element matched with the keyword. Therefore, the business personnel can be provided with more accurate recommendation results of the information of the second standard data element. The query duration of the business personnel in the information of the plurality of second standard data elements is reduced, and the business personnel can quickly select the information of the desired standard data elements.

Wherein the search condition may include: time of release, department of administration information and/or criteria category, and the like. Wherein, the publishing time refers to the publishing time of the data standard to which the standard data element in the data standard library belongs, such as 2018; the administrative department information refers to information of a manager of the data standard to which the standard data elements in the data standard library belong, such as a certain electrical industry association; the standard category refers to a category of data standard, such as a security class or a product class, to which a standard data element in the data standard library belongs.

Fig. 4 is a schematic interface diagram provided by the data standard management system in the embodiment of the present application. Suppose that the keywords entered by the service person in the search box are: "shaping" is performed, content is not input in a condition input box (in fig. 4, the condition input box is used for inputting the publishing time), the matching result output by the data standard management system comprises information of 2 second standard data elements of which the data element identifiers (in fig. 4, the data element identifiers are described as examples of the names of the data elements) are "shaping test field" and "shaping test field 1", the publishing time limitation is not performed in the process of retrieving the information of the 2 second standard data elements, and the service personnel can select one second standard data element as the first data element based on the matching result.

In practical implementation of the embodiment of the present application, the information included in the matching result may also have other forms. For example, the matching result comprises only the data element identification of the at least one second criterion data element; or, the matching result only includes the data element description information of the at least one second standard data element; or the matching result further comprises the matching degree of the data element identification of each second standard data element in the at least one second standard data element and the keyword on the basis of the information comprising the at least one second standard data element, so that the referential of the matching result to business personnel is improved.

P1＝M/N；

wherein, M is the number of characters of the data element identification of the second standard data element which is the same as the number of characters of the key word, and is equivalent to the number of characters corresponding to the intersection of the data element identification of the second standard data element and the key word; n is the maximum number of characters in the number of characters of the data element identification and the number of characters of the keyword of the second standard data element. For example, if the data element of the second standard data element is identified as "integer test field" and the key is "integer", then M ═ 2, N ═ 6, and P1 ≈ 1/3 ≈ 33.3%.

It should be noted that the foregoing calculation of the matching degree by using the first matching degree calculation formula is only an exemplary method for obtaining the matching degree provided in the embodiment of the present application, and the method for obtaining the matching degree may also have other manners, and may refer to a conventional method for obtaining the matching degree, such as a method for obtaining the matching degree of a fuzzy matching algorithm (e.g., ES).

With continued reference to fig. 4, the matching degree P1 of the second standard data element with the keyword "integer" when the data element identification is "integer test field 1" is 2/7 ≈ 28.6%. Therefore, in fig. 4, after sorting in descending order according to the matching degree between the data element identifier of the second standard data element and the keyword, the information of the second standard data element whose data element identifier is the "integer test field" in the matching result output by the data standard management system is arranged before the information of the second standard data element whose data element identifier is the "integer test field 1".

Referring to fig. 5, after sorting in fig. 5 according to the standard priority of the data standard to which the second standard data element belongs in a descending order, the information of the second standard data element whose data element is identified as "integer test field 1" in the matching result output by the data standard management system is arranged before the information of the second standard data element whose data element is identified as "integer test field".

For example, if a second standard data element has a matching degree a, a priority value b, and weights X and Y assigned to the matching degree and the priority in advance, respectively, the second standard data element has a corresponding sorting indication value c, where c is aX + bY.

The business personnel can generate a data standard document in an offline editing mode through the second terminal equipment used by the business personnel, the data standard document comprises the information of the first database table structure, and the generated data standard document is transmitted to the data standard management system. Accordingly, the data standard management system receives the data standard document. The data standard document may be a document of multiple data types, such as a database table type of data document.

In a first alternative implementation manner, the business personnel may run a third-party modeling tool on the second terminal device, generate a data standard document through the third-party modeling tool, and input the data standard document into the data standard management system. By way of example, the third party modeling tool may be an E-Rwin or PowerDesigner like data modeling tool.

In a second optional implementation manner, the data standard management system provided in the embodiment of the present application may further support access of a third party modeling tool, where the third party modeling tool may access (e.g., query) the data standard library, obtain information of a standard data element stored in the data standard library, and generate a data standard document based on the information of the standard data element. That is, the third party modeling tool may support the generation of data standard documents that meet the requirements of a data standard library. For example, a business person may run a third party modeling tool at the second terminal device, with the data standard documents generated by the third party modeling tool based on the database of data standards. Accordingly, the data standard management system receives a data standard document generated by a third party modeling tool based on the data standard library. By way of example, the third party modeling tool may be an E-Rwin or PowerDesigner like data modeling tool.

In the first optional implementation manner, since the generation rule of the data standard document of the third-party modeling tool is different from the generation rule required by the data standard library, after the data standard management system acquires the data standard document generated by the third-party modeling tool, the data standard management system needs to verify the information of the first database table structure in the data standard document, so as to obtain the information of the first database table structure meeting the requirement of the data standard library; in the second optional mode, the third-party modeling tool can support generation of the data standard document meeting the requirements of the data standard library, and all or part of the information of the corresponding first database table structure meets the requirements of the data standard library, so that the operation cost of verification can be reduced, and the verification cost can be reduced.

Optionally, in step 201, the data standard management system may further output a data dictionary template, where the data dictionary template is a reference template of the information of the first database table structure; the service person may input information of the first database table structure, such as information of single or batch data elements, to the data standard management system through the second terminal device based on the data dictionary template, and correspondingly, the data standard management system receives the information of the first database table structure input based on the data dictionary template. The specific obtaining mode of the information of the first database table structure may refer to the first optional mode or the second optional mode for obtaining the information of the first database table structure of the first service system to be online.

By way of example, the data dictionary template may be as shown in Table 1.

TABLE 1

Referring to table 1, the data dictionary template includes parameters related to information of data elements shown in the first row of table 1, and the parameters related in table 1 include parameters such as database name, english name of physical table, english name of data element, and chinese name of data element; the data dictionary template further includes interpretation information (or description information) of parameters related to the information of the data elements as shown in the second row of table 1, the interpretation information being used to interpret the meaning represented by each corresponding parameter. For example, the parameters: optional values, the corresponding interpretation information is: "optional values and descriptions of data, such as gender: f, female; m, male ". The data dictionary template further includes filling examples of information of 6 data elements (i.e. information of one data element per line) as shown in the third to ninth lines of table 1, and is used for prompting the service personnel how to fill in the information of the data elements.

The foregoing table 1 is merely an illustrative example provided in the embodiment of the present application, and when the embodiment of the present application is actually implemented, the data dictionary template may have other forms as long as the purpose of providing reference for the business personnel to input the information of the data element is achieved.

Step 202, the data standard management system checks the information of the first database table structure based on the data standard library.

Because the first database table structure is a database table structure which is customized by business personnel and is planned to be used in the first business system, the data standard management system needs to check the information based on the data standard database after acquiring the information of the first database table structure. In this embodiment of the present application, checking information of the first database table structure may include at least two aspects of the following data standard symbolic check and data standard normative check:

in a first aspect, data is checked for standard symbology. The verification process refers to verifying information of at least one data element based on information of standard data elements in a data standard library. The information of the standard data elements is the aforementioned "symbol".

Optionally, assuming that the first data element is one of the at least one data element included in the information of the first database table structure, the process of comparing the information of the standard data element in the standard database with the information of the at least one data element includes:

step a1, the data standards management system compares the information of the first data element with the information of a plurality of standard data elements stored in the data standards repository.

It is worth mentioning that, in order to effectively match the information of the first data element with the information of the plurality of standard data elements stored in the data standard library, the information of the standard data element is usually translated into information, for example, in text form, which can be compared with the information of the first data element.

In the embodiment of the application, the data standard management system compares the information of the first data element with the information of the plurality of standard data elements stored in the data standard library, mainly to determine whether the information of the plurality of standard data elements has information matching with the information of the first data element. I.e. to determine whether the information of each standard data element matches the information of the first data element. Wherein the determination of whether the information of one standard data element matches the information of the first data element depends on the matching condition.

In a first alternative, the matching condition is that the information of the standard data element and the information of the corresponding parameter in the information of the first data element are the same. As shown in table 1, the information of the parameters, such as the database name of the standard data element, the english name of the physical table, the english name of the data element, and the chinese name of the data element, corresponds to the same information of the corresponding parameter of the first data element.

As described above, the information of the data element includes attribute information (information of parameters other than optional values in the first row of table 1 all belong to the attribute information); in some scenarios it is also possible to include a code set (e.g. the parameters in the first row of table 1: information of selectable values belong to the code set). Since the information of the first database table structure is customized by business personnel, at least one data element is also customized, and the information of the code set may not be defined in the information of some data elements. Therefore, if the matching condition is that the information of the standard data element is the same as the information of the corresponding parameter in the information of the first data element, the probability that the information matching the information of the first data element exists in the data standard library is low, and the matching efficiency is low.

In a second alternative, the matching condition is that the attribute information of the standard data element is the same as the information of the corresponding parameter in the attribute information of the first data element. As shown in table 1, information of parameters (i.e., information of parameters other than the optional values), i.e., a database name of the standard data element, an english name of the physical table, an english name of the data element, and a chinese name of the data element, etc., corresponds to the information of the corresponding parameter of the first data element. A process for a data standard management system to compare information of a first data element with information of a plurality of standard data elements, comprising: and a process of comparing the attribute information of the first data element with the attribute information of the plurality of standard data elements.

Similarly to the first optional manner, if the matching condition is that the attribute information of the standard data element is the same as the attribute information of the corresponding parameter in the information of the first data element, and the parameters related to the attribute information of the standard data element and the attribute information of the first data element are more, the probability that the information matched with the information of the first data element exists in the data standard library is lower, and the matching efficiency is lower.

In a third alternative, the matching condition is that the information of the specified parameter in the attribute information of the standard data element is the same as the information of the corresponding parameter in the attribute information of the first data element. The specified parameter generally belongs to a parameter describing the attribute of the data element, such as a name, a definition, a structure and/or a value rule, wherein the name may include parameters such as an english name of the data element, and the definition may include parameters such as a character type, a character length and/or a character precision. The designated parameters can describe the self attributes of the data elements, and the number of the parameters is small, so that the matching condition can ensure that the probability of the information matched with the information of the first data element in the data standard library is high, and the matching efficiency is improved.

Step A2, when the information of the first data element does not match with the information of the plurality of standard data elements in the data standard library, the data standard management system sends a first modification prompt message, and the first modification prompt message indicates to update the information of the first data element.

For example, the data standard management system may send a first modification prompt message to the second terminal device, and the second terminal device presents the first modification prompt message to the service staff so that the service staff updates the information of the first data element.

Step a3, the data standard management system determines that the information verification of the first data element is successful after receiving the updated information of the first data element matching the information of any one of the plurality of standard data elements.

The data standard management system, after receiving the updated information of the first data element, compares the updated information of the first data element with the information of the plurality of standard data elements, and the comparing step may refer to the foregoing step a 1; when the information of the first data element does not match with the information of the plurality of standard data elements in the data standard library, the data standard management system sends the first modification prompting information data element identification again, and the prompting step can refer to the step A2; and after receiving the updated information of the first data element again, the data standard management system repeatedly executes the comparison step, the prompt step and the receiving step of the information of the first data element until the information of any standard data element in the plurality of standard data elements of the data standard library is matched with the updated information of the first data element.

And when the information of any standard data element in the plurality of standard data elements of the data standard library is matched with the updated information of the first data element, the data standard management system determines that the information verification of the first data element is successful.

Therefore, business personnel can realize multiple modification of the information of the first data element through the first modification prompt information sent by the data standard management system for multiple times so as to meet the requirement of the standard data element in the data standard library, and the business personnel can define the information of the data element consistent with the information of the standard data element in the data standard library.

As described above, the information of the first database table structure is carried in the data standard document that can be edited online or offline, so when the business personnel updates the information of the first data element, the business personnel usually update the information of the first database table structure in which the first data element is located, that is, update the information in the data standard document. The data standard management system needs to first locate the updated information of the first data element in the updated information of the first database table structure, and then verify the updated information of the first data element.

In an alternative, after receiving the updated information of the first database table structure, the data standard management system determines the information of the first data element from the information of the total amount of data elements of the updated information of the first database table structure. For example, the data standard management system may scan all of the data elements of the information of the first database table structure in order to locate the information of the first data element among the information of all of the data elements. This process is referred to as a full check.

In another optional mode, after the data standard management system receives the updated information of the first database table structure; incremental data elements may be determined in the updated information of the first database table structure and information of the first data element may be determined in the information of the incremental data elements. This process is called incremental verification.

Optionally, the data standard management system establishes an update indication rule, and when the service personnel updates the information of the data element through the second terminal device, the service personnel may update the information of the data element according to the update indication rule, so that the data standard management system effectively locates the incremental data element.

For example, the data standard management system carries the information of the first database table structure in the first modification prompt information, and adds the remark field in the information of the first database table structure, so that the process of sending the first modification prompt information by the data standard management system is equivalent to performing rollback on the information of the first database table structure. And if the information of the first database table structure is carried in the data standard document, the rollback of the document is realized. After receiving the information of the first database table structure through the second terminal device, the service personnel updates the information of the data elements therein, and adds target remark information in the remark field, wherein the target remark information indicates the updated data elements. Then, after receiving the updated information of the first database table structure, the data standard management system can determine the updated data element, namely the incremental data element, by querying the remark field, and then locate the first data element in the incremental data element. For example, the information of the first database table structure includes information of 6 rows of data elements, and after the business personnel receives the information of the first database table structure through the second terminal device, the business personnel update the information of the first row of data elements and the third row of data elements and add target remark information indicating the first row of data elements and the third row of data elements in the remark field. The data standard management system may determine that the first and third rows of data elements are incremental data elements by querying the remark field.

For another example, the data standard management system carries the information of the first database table structure in the first modification prompt information, and adds a plug-in to the information of the first database table structure. After receiving the information of the first database table structure through the second terminal device, the service personnel updates the information of the data elements therein, and the plug-in automatically identifies the updated data elements. Then, after receiving the updated information of the first database table structure, the data standard management system can determine the updated data element, namely the incremental data element, through the identification of the plug-in, and then locate the first data element in the incremental data element. Wherein the plug-in may identify the updated data element by adding annotations, highlighting, and/or adding a specified color, etc. For example, the information of the first database table structure includes information of 6 rows of data elements, the business personnel updates the information of the first row of data elements and the information of the third row of data elements after receiving the information of the first database table structure through the second terminal device, and the plug-in component highlights the information of the first row of data elements and the third row of data elements. The data criteria management system determines the highlighted first and third rows of data elements as incremental data elements.

The number of data elements inquired by the data standard management system can be reduced through incremental verification, and the efficiency of determining the updated data elements is improved.

Optionally, the first modification hint information may include information of at least one first standard data element, and the data element identifier of each first standard data element in the at least one first standard data element matches with the data element identifier of the first data element. In this way, the service person can obtain the information of each first standard data element related to the data element identifier of the first data element in the first modification prompt message, so as to select the first data element which the service person wants to modify from at least one first standard data element.

It should be noted that the first modification notice usually includes information specifying a number of first standard data elements, and the specified number is an integer greater than 1, so that the referential of the first modification notice to the service personnel can be improved. The algorithm for the data standard management system to match the received data element identifier of the first data element with the data element identifiers of the plurality of standard data elements may be a fuzzy matching algorithm, that is, the data element identifier of each first standard data element in the at least one first standard data element is matched with the data element identifier of the first data element. For example, the fuzzy matching algorithm is a search algorithm in the elastic search.

The matching result obtained by adopting the fuzzy matching algorithm can comprise an accurate matching result and a result except the accurate matching, compared with the method of simply adopting the accurate matching algorithm to obtain the information of the first standard data element, the fuzzy matching content is wider, the number of the obtained information of the first standard data element is more, and the reference of the matching result to business personnel is improved. The matching algorithm may also be other algorithms, which is not limited in this embodiment of the present application.

Optionally, if there are multiple first standard data elements obtained by matching, in the first modification prompt information, the information of the multiple first standard data elements. May be arranged in a second specified order. For example, this second designated order may be implemented by the following two exemplary implementations:

It should be noted that, for a specific process of the foregoing two exemplary implementation manners and a manner of combining the two exemplary implementation manners, reference may be made to the two exemplary implementation manners corresponding to the first specified order in step 201 and a manner of combining the two exemplary implementation manners, where step 202 is different from step 201 in that the content of the data element identifier of the first data element and the content of the keyword may be different, but both include one or more characters, and details of the embodiment of the present application are not repeated here.

Further optionally, in the verification process provided by the first aspect, the data standard management system may further add some remark fields according to the verification condition, so as to prompt information that needs to be noticed by the service staff.

For example, assume that the second data element is one of at least one data element included in the information of the first database table structure. And when the numerical value corresponding to the second data element is an enumeratable numerical value, adding data element remark information for the second data element, wherein the data element remark information is used for identifying the enumeratable numerical value corresponding to the second data element. Referring to step a1, since the information of the first database table structure is customized by the service personnel, there is a possibility that the information of the code set is not defined and the information of the defined code set is not accurate. Therefore, in this embodiment of the application, when the value corresponding to the second data element is an enumerable value, the data element remark information may be added to the second data element, so as to add the information of the accurate code set. For example, the name of the second data element is: and age, adding data element remark information to the second data element, wherein the data element remark information is used for identifying 120 enumerable values from 1 to 120 corresponding to the second data element.

In a second aspect, data standardization checks. The verification process refers to verifying the normalization of the information of the data standard. The format of the information of the data standard is mainly checked.

Optionally, the process of checking the information of the first database table structure based on a data standard library, where the data standard library includes information of a plurality of standard data elements includes:

and step B1, the data standard management system detects whether the format of the information of the first database table structure meets the specified format requirement.

The information of the first database table structure includes information of at least one data element, and may also include information of the first database and first table information, as described in step 201. The process of the data standard management system detecting whether the format of the information of the first database table structure meets the specified format requirement includes: detecting whether the format of the information of each data element meets the specified data element format requirement, such as whether the English name of the data element is composed of specified characters (such as capital English letters); detecting whether the format of the information of the first database meets the specified database format requirement, for example, whether the English name of the database is composed of specified characters (such as capital English letters) and whether the length of the database code is smaller than a first specified length threshold, wherein the first specified length threshold can be 60 bits (bits refer to the number of digits of a numerical value); whether the format of the first table information meets the structural formula requirement of a specified database table is detected, for example, whether the database table identification is composed of specified characters (such as capital English letters) or not, whether the length of the database table code is smaller than a second specified length threshold value or not is detected, and the second specified length threshold value can be 60 bits.

And step B2, when the format of the information of the first database table structure does not meet the specified format requirement, the data standard management system sends second modification prompt information, and the second modification prompt information indicates the format of the information for updating the first database table structure.

For example, the data standard management system may send second modification prompt information to the second terminal device, and the second modification prompt information is presented to the business personnel by the second terminal device, so that the business personnel updates the format of the information of the first database table structure.

And step B3, after receiving the updated information of the first database table structure with the format meeting the format requirement, the data standard management system determines that the format check of the information of the first database table structure is successful.

After receiving the updated information of the first database table structure, the data standard management system detects whether the format of the information of the first database table structure meets the specified format requirement, and the detecting step may refer to step B1; when the format of the information of the first database table structure still does not meet the specified format requirement, the data standard management system sends a second modification prompt message again, and the prompt step can refer to the step B2; and after the data standard management system receives the updated information of the first database table structure again, repeatedly executing the detection step, the prompt step and the receiving step of the information of the first database table structure until the format of the information of the first database table structure meets the specified format requirement.

And when the format of the updated information of the first database table structure meets the specified format requirement, the data standard management system determines that the format verification of the information of the first database table structure is successful.

It can be seen from the above that, the service personnel can implement multiple modifications of the format of the information of the first database table structure through the second modification prompt information sent by the data standard management system for multiple times, so as to meet the requirement of the data standard management system on the format of the standard information, and enable the service personnel to define the information of the database table structure meeting the requirement.

It should be noted that the second modification hint information generally indicates which specific information in the information of the first database table structure does not meet the specified format requirement, such as the information of the data element or the information of the first database or the information of the first table does not meet the corresponding format requirement.

In an alternative, after receiving the updated information of the first database table structure, it is detected whether all the information in the first database table structure meets the specified format requirement. For example, the data standard management system may scan all of the information of the first database table structure in the database and detect whether the scanned information meets the corresponding format requirements.

In another alternative, after receiving the updated information of the first database table structure, the information that was last time not compliant with the corresponding format requirement is detected without detecting all the information in the first database table structure. The information amount inquired by the data standard management system can be reduced by only detecting the information which does not meet the corresponding format requirement last time, and the detection efficiency is improved.

It should be noted that the verification processes in the foregoing two aspects may be executed simultaneously or separately, and the order of the first execution in the embodiment of the present application is not limited.

Through the verification process in the step 202, automatic matching verification can be performed by using the information of the first database table structure created for the first business system, such as the first table information, the information of the data elements, the code set and the like, and a system administrator only needs to perform simple inspection, so that the labor input is greatly saved, and because the manual auditing link is reduced, the manual error can be reduced, and the accuracy of the finally obtained target database table structure is improved. Compared with the traditional technology, the labor cost can be saved by more than 75%. And the processes of filling, transferring, comparing, filling in auditing opinions, feeding back, modifying, rechecking and the like are performed on line through the database table originally, so that the processes of automatic checking, comparison modifying, submitting auditing and auditing result feedback are simplified, the offline information transfer process and links are reduced, the information transfer time is saved, the work flow is further simplified, the work efficiency is improved, and the work efficiency can be improved by over 70 percent generally.

Step 203, after checking the information of the first database table structure, the data standard management system acquires a target database table structure of the first business system, wherein the target database table structure is determined based on the first database table structure.

The target database table structure is a database table structure adopted by the first service system after the first service system is online, and is substantially obtained in advance before the first service is online, and only the target database table structure needs to be used after the first service is online. The information of the target database table structure includes part or all of the information of the first database table structure. In one alternative, the information of the first database table structure may be determined directly as the information of the target database table structure.

In another alternative, the information of the first database table structure may need to be further adjusted to obtain the information of the target database table structure.

Since the data elements of different business systems may vary according to actual conditions for different periods of time, although the information of the corresponding standard data element is not recorded in the data standard library, the information of the data element is allowed to be added in a certain business system. However, the information of these data elements cannot be verified effectively by the data standard management system, so after the foregoing step 202, that is, after the data standard management system performs automatic verification on the information of the first database table structure, the data standard management system also supports manual secondary verification on the information of the first database table structure.

For example, the secondary verification process may include:

corresponding to the first aspect, after determining that the information verification on the first data element is successful, the data standard management system sends the information of the first data element to the specified terminal device. The designated terminal device is a terminal device of a verifier for performing secondary verification, and may be a terminal device of a system administrator or a project administrator. For example, assuming that the verifier is a system administrator, the terminal device is designated as the third terminal device. After receiving the information of the first data element through the appointed terminal equipment, the verifying personnel judges whether the information of the first data element needs to be modified or not, and sends first verifying response information to the data standard management system through the appointed terminal equipment based on the judgment result, wherein the first verifying response information is used for indicating that the information of the first data element is modified or indicating that the secondary verification of the information of the first data element is successful. The data standard management system receives the first verification response information and sends the first verification response information to second terminal equipment of the service personnel.

When the first verification response information is used for indicating to modify the information of the first data element, the service personnel can modify the information of the first data element through the second terminal equipment, and then send the modified information to the appointed terminal equipment through the data standard management system again, and the verification personnel performs verification until the second verification of the information of the first data element is indicated to be successful by the first verification response information received by the second terminal equipment.

When the first verification response information is used for indicating that the second verification of the information of the first data element is successful, the service personnel does not need to modify the information of the first data element.

Corresponding to the second aspect, after determining that the format check of the information of the first database table structure is successful, the data standard management system sends the information of the first database table structure to the specified terminal device. After receiving the information of the first database table structure through the appointed terminal equipment, the verifier judges whether the format of the information of the first database table structure needs to be modified or not, and sends second verification response information to the data standard management system through the appointed terminal equipment based on the judgment result, wherein the second verification response information is used for indicating that the format of the information of the first database table structure is modified or indicating that the secondary verification of the format of the information of the first database table structure is successful. And the data standard management system receives the second check response information and sends the second check response information to the second terminal equipment of the service personnel.

When the second check response information is used for indicating that the format of the information of the first database table structure is modified, the business personnel can modify the format of the information of the first data element through the second terminal device, and then send the modified information to the specified terminal device through the data standard management system, and the check personnel checks the modified information until the second check response information received by the second terminal device indicates that the second check of the format of the information of the first data element is successful.

When the second check response information is used for indicating that the second check on the format of the information of the first data element is successful, the business personnel do not need to modify the format of the information of the first data element.

It should be noted that the information of the first data element and the information of the first database table structure may be carried in the same check request (also referred to as a check application) and sent to the specified terminal device, and the first check response information and the second check response information may be the same information, so as to reduce the number of times of interaction between the data standard management system and each terminal device, and save network overhead.

Through carrying out manual secondary verification on the information of the first database table structure, the flexibility and the reliability of the finally obtained information of the target database table structure can be ensured.

Optionally, the information of the target database table structure may be information of the first database table structure after the manual second verification is successful.

Optionally, after the verification is successful and/or the manual secondary verification is successful, the service staff may be prompted to further adjust the information of the first database table structure by sending third modification prompt information, so as to obtain the information of the target database table structure. For example, the first database table structure whose information meets the requirement of the standard drop rate and/or the requirement of the matching rate is finally determined as the target database table structure.

And the requirement of the standard falling rate means that the standard falling rate of the information of the first database table structure is greater than a specified standard falling rate threshold value. And the label falling rate is the ratio of the number of actual label falling data elements in the information of the first database table structure to the number of corresponding label falling data elements in the information of the first database table structure. The actual landmarked data element is a data element matching the standard data element, and the definition of the matching can refer to the definition of the aforementioned step a 1. The data element should be marked as a data element whose service identifier is the same as the service identifier (such as the english name of the data element) of the standard data element, but not matched with the standard data element (that is, only the condition that the service identifiers in the matching condition are the same is satisfied). For example, if the number of actual tagging data elements in the information in the first database table structure is 5 and the number of target tagging data elements is 10, the matching rate is 5/10-50%.

The requirement of the matching rate means that the matching rate of the information of the first database table structure is greater than a specified matching rate threshold. Wherein, the matching rate is: the number of the data elements to be marked and the total number of the data elements contained in the database table structure of the first business system.

As mentioned above, there may be one or more database table structures of the first business system, and the total number of the data elements contained in the database table structure of the first business system is the total number of the data elements contained in all the database table structures in the first business system. For example, if the number of database table structures of the first business system is 3, the total number of data elements is 30, and the number of data elements to be marked in the first database table structure is 6, the matching rate is 6/30-20%.

And step 204, outputting a script corresponding to the target database table structure by the data standard management system.

In the embodiment of the application, the data standard management system supports a script output function. After the information of the first database table structure is checked and verified to obtain the information of the target database table structure, the data standard management system can generate and output a script (also called a table building script) corresponding to the target database table structure. The script is used to generate information of a target database table structure, which includes a database table building statement of the first business system. The service person can receive the script via the second terminal device. After the first service system is online, service personnel can load and run the script in the first service system, the running script can generate information of a target database table structure, and a corresponding database table structure is obtained according to the information construction of the target database table structure.

In the embodiment of the application, the data standard management system outputs the script without the need of self-writing of the script by service personnel, so that the workload of the service personnel is reduced, and the labor cost is saved.

Optionally, when the target database table structure is determined, the data standard management system may further send a target database table structure use request to the first terminal device of the project administrator, where the request carries information of the target database table structure, and the project administrator may establish the database table after the first service system is online based on the information of the target database table structure. And after the establishment is finished, sending a database table use notice to second terminal equipment of the service personnel through the data standard management system so as to inform the service personnel that the target database table structure can be used after the first service system is on line. Therefore, business personnel can not need to build a database table.

The foregoing steps 201 to 204 may be performed by the data standard management device 102 in the data standard management system 10 shown in fig. 1, where the steps 201 to 203 correspond to a data verification function; step 204 corresponds to a script output function. Optionally, the data standard management system 10 further supports a life cycle management function corresponding to the following steps 205 to 206, and a data governance function shown in step 207. Where the lifecycle management functions are performed by lifecycle management device 103 and the data governance functions are performed by data governance device 104. Wherein, the steps 205 to 206 are as follows:

step 205, the data standard management system receives a data standard library operation request, where the data standard library operation request includes a standard data element addition request, a standard data element update request, a standard data element deletion request, or a standard data element query request.

In the embodiment of the application, various data standard library operations on the data standard library are supported, such as a standard data element adding operation, a standard data element updating operation, a standard data element deleting operation and a standard data element inquiring operation. The corresponding data standard library operation requests are standard data element adding requests, standard data element updating requests, standard data element deleting requests or standard data element inquiring requests respectively.

Wherein the standard data element adding request is used for requesting to add information of one or more data elements in the data standard library. For example, a system administrator may collect a national standard, a line standard, or a landmark, split the collected data criteria into a plurality of data elements, and add information for the plurality of data elements to the data criteria library via one or more criteria data element addition requests. Each standard data element addition request may carry information of a single or a batch of data elements. For example, if a landmark adds a part of data elements to a newly-built business system, the part of data elements needs to be added to a data standard library for guiding the construction of the business system of an industry or subordinate unit, and if the data standard is unified, a system administrator adds the part of data elements through a standard data element adding request. For another example, if a standard data element is added by national standard or industry standard, the system administrator adds the data element by a standard data element adding request to keep synchronization with national standard or line standard.

It is worth noting that a system administrator may manage the system to collect data elements based on data criteria. In an alternative mode, the data standard management system may further output a data element collection template, where the data element collection template is a reference template for data element collection, and the structure of the data element collection template may refer to the data dictionary template; the service person can input information of the data elements, such as information of single or batch data elements, to the data standard management system through the second terminal device based on the data element collection template. Correspondingly, after receiving the information of the data element, the data standard management system sends the received information of the data element to a third terminal device of a system administrator for reference by the system administrator. For example, the service person may directly input the data element in the data element collection template through the second terminal device to obtain an updated data element collection template, and the data standard management system sends the updated data element collection template to the third terminal device of the system administrator.

The standard data element update request is for requesting an update of information for one or more data elements in a data standard repository. For example, a national standard or industry standard updates a standard data element, and the system administrator updates the data element via a standard data element update request to keep synchronization with the national standard or the line standard.

The standard data element deletion request is for requesting deletion of information for one or more data elements in the data standard library. For example, when the data standard corresponding to a certain national standard, a certain row standard or a certain landmark stops being used, the system administrator deletes the information of a plurality of data elements corresponding to the data standard in the data standard library through one or more standard data element deletion requests. Optionally, when the data standard management system performs a deletion operation on a certain standard data element in the data standard library in a subsequent process, a deletion flag may be added to the standard data element, and the standard data element is not physically deleted, so as to facilitate a subsequent query. The delete flag indicates that the standard data element has been discarded, which may also carry a reason for the discarding of the standard data element, such as that the data standard has been decommissioned.

The standard data element query request is used for requesting information for querying data elements in the data standard library.

It is worth mentioning that, when adding a data element, the data standard management system may further output a data element template, which is a reference template of the added data element; based on the data element template, an operator can input the information of the data elements required to be carried in the standard data element adding request to the data standard management system through the corresponding terminal equipment, and correspondingly, the data standard management system receives the information of the data elements input based on the data element template. Wherein the data element template may be as shown in table 2.

TABLE 2

Referring to table 2, the data element template includes parameters related to information of the data element as shown in the first row of table 2, and the parameters related in table 2 include basic data classification name, basic data classification code, identifier of the data element, and chinese name of the data element, etc. as an example; the data element template further includes interpretation information (or description information) of parameters related to the information of the data elements as shown in the second row of table 2, and the interpretation information is used for interpreting the meaning represented by each corresponding parameter. For example, the parameters: the basic data classification name, the corresponding interpretation information is: "the classification to which the data element belongs takes the following values: XX common: an XX basis; the public of the bureau: XX bureau _ public information; a bureau system: XX bureau _ system name ". The data element template further includes filling examples of information of 3 data elements (i.e., information of one data element per line) as shown in the third to fifth lines of table 2, and is used for prompting the operator how to fill in the information of the data elements.

The foregoing table 2 is merely an illustrative example provided in the embodiment of the present application, and when the embodiment of the present application is actually implemented, the data element template may have other forms as long as the purpose of providing reference for the operator to input the information of the data element is achieved.

The data standard management system is used for the reference of an operator by outputting the data element template, so that the operator does not only formulate the information of the data element by the experience of the operator but also formulate the information of the data element according to the ground, and the accuracy of the information of the standard data element input to the standard database can be improved.

Step 206, after the data standard management system successfully authenticates the operation request of the data standard library, the data standard management system executes the operation corresponding to the operation request of the data standard library.

Because the data standard library stores information of a plurality of standard data elements, the information of the standard data elements is the basis for checking the database table structure of each business system, if the information of the standard data elements is added, deleted or modified at will, management confusion of the information in the data standard library can be caused, and the information of the data standard library loses effective referential of checking. Therefore, when the data standard library is operated, the operation request of the data standard library needs to be authenticated, and after the authentication is successful, the data standard management system can perform the operation corresponding to the operation request of the data standard library on the data standard library.

As described above, in the application scenario related to the data processing method, the related staff members include a project manager, a business person, and a system manager. All the workers need to register corresponding account numbers in the data standard management system, and when the data standard management system is operated, the corresponding operation information carries the account numbers so as to identify the identities of the operators. The account numbers of the workers with different identities have different grades, and the operation authorities of the data standard management system are different, so that the corresponding authentication modes are different.

In addition, because the types of the data standard library operation are various, the security levels related to different types of data operations are different, and the corresponding authentication modes are different.

When the data operation is a standard data element addition operation, a standard data element update operation, or a standard data element deletion operation, the security level involved with the data operation is higher. The general service personnel are not allowed to perform the relevant operation. And because the data standard management system is maintained by a system administrator, the system administrator generally has the authority to manage the data standard library, and the data standard library can perform operations with higher security level. However, the system administrator is not necessarily a professional of the corresponding project, so that the data standard corresponding to the project may not be known, and therefore, the related personnel is required to perform auxiliary authentication of the data operation. When the data operation is a data meta query operation, the data operation involves a lower level of security that is typically viewable by project managers, business personnel, and system administrators. However, in order to avoid some malicious accesses and reduce the leakage of the secret, related personnel can be added to perform auxiliary authentication of data operation.

Based on the above principle, the embodiments of the present application schematically propose the following authentication methods:

In the two authentication manners, if the data standard management system successfully authenticates the operation request of the data standard library, the data standard library is subjected to the operation corresponding to the operation request of the data standard library. Optionally, the data standard management system may further send a data operation response indicating that the operation corresponding to the data standard library execution data standard library operation request is completed, or indicating that the operation is successful.

And if the data standard management system fails to authenticate the operation request of the data standard library, forbidding to execute the operation corresponding to the operation request of the data standard library on the data standard library. Optionally, the standard management system may further send a data operation response indicating that the operation corresponding to the data standard library operation request is prohibited from being executed on the data standard library, or indicating that the operation fails.

Optionally, the database operations of the data standard management system further include operations on the code set, such as a code set addition operation, a code set update operation, a code set deletion operation, and a code set query operation. The corresponding data standard library operation requests are respectively a code set adding request, a code set updating request, a code set deleting request or a code set inquiring request. The corresponding process may refer to the process of adding, updating, deleting, and querying the standard data element, which is not described in this embodiment.

By authenticating the operation request of the data standard library, the safety of data in the data standard library can be ensured, the reliability of the information of the standard data elements in the data standard library is ensured, and the disclosure is effectively prevented.

The data standard management system supports addition (also called release), update, deletion and query of data elements, and can also support functions of maintenance of code sets, auditing and other maintenance of data elements and the like. It is worth mentioning that the data standard management system can also manage information of a plurality of data standards in the form of database tables or documents for users to refer and refer to. These series of functions for a data standard or data element may be referred to as lifecycle management of the data element. The data standard can be comprehensively controlled in a whole process based on the generation period management. In the embodiment of the application, the data standard management system can also perform systematic management on the documents corresponding to the data standards, for example, different pages are established in different industries, grades and versions in a webpage form, so that users can conveniently look up and use the pages.

Optionally, in the process of generating cycle management, the data standard management system may generate an operation log for each data operation. After receiving the query instruction, the data standard management system may query the operation log corresponding to the data standard library periodically or; and when the operation log comprises the abnormal operation log, sending an abnormal alarm. A system administrator can determine whether data operation backtracking needs to be carried out or not based on the abnormal alarm, so that data rollback is realized, and the data standard management system is effectively maintained.

And step 207, the data standard management system carries out data governance based on the artificial intelligence model.

In the embodiment of the application, the data standard management system also supports a data governance function.

In an alternative example, the data governance process corresponding to the data governance function may refer to a conventional data governance process.

In another alternative example, the data standard management system is pre-established with an artificial intelligence model, and data governance is performed through the artificial intelligence model. The data governance process may include the steps of:

step C1, the data standard management system obtains the information of the second database table structure of the second business system after online, and the information of the second database table structure includes the information of at least one data element.

The process of step C1 may refer to the process of step 201, which is not limited in this embodiment.

And step C2, when the data standard management system detects that the information of the third data element does not match the information of the plurality of standard data elements based on the artificial intelligence model, determining a target standard data element corresponding to the third data element in the plurality of standard data elements, wherein the third data element is one of at least one data element included in the information of the second database table structure.

The data standard management system may input information of the second database table structure into the artificial intelligence model, detect, by the artificial intelligence model, whether information of the third data element matches with information of the plurality of standard data elements, and determine, when the information of the third data element does not match with the information of the plurality of standard data elements, a target standard data element corresponding to the third data element among the plurality of standard data elements.

And step C3, the data standard management system establishes the mapping relation between the third data element and the target standard data element based on the artificial intelligence model.

After the artificial intelligence model establishes the mapping relation between the third data element and the target standard data element, the mapping relation can be output for use in subsequently providing data service.

The artificial intelligence model can be obtained by training information of a plurality of standard data elements and information of sample data elements. By adopting the artificial intelligence model to establish the mapping relation, the establishing accuracy and the establishing efficiency of the mapping relation can be improved, and the data management effect is improved.

The sequence of the steps of the service data processing method provided in the embodiment of the present application may be appropriately adjusted, and the steps may also be increased or decreased according to the situation, for example, steps 206 and 207 may be located before step 201, and any method that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application shall be covered by the protection scope of the present application, and therefore, no further description is given.

With the advancement of the cloud of the traditional business system, the micro-service or business system new establishment for some reason, the new establishment of a database table and a data element is necessarily involved; the original service system may be built by multiple manufacturers at different time periods, and the data standard referred by each manufacturer and the normative requirement of the database are also different, for example: data element standards, codeset standards, and the like; in the traditional data standard management system, after a certain service system is on line, the database table structure maintained in the service system is checked; if the database table structure does not meet the requirement of the target data standard corresponding to the business system and stored by the data standard management system, the data standard management system establishes the mapping relation between the database table structure and the target data standard. This process is called a data governance process.

In the embodiment of the application, when the service system is clouded/micro-serviced/newly built, the database table structure of the service system is verified before the service system is on line. In the embodiment of the present application, the online of the service system may be the first online (i.e., the online after the new establishment), or the online after the modification. Through the data standard symbolic verification and the data standard normative verification, unified requirements of data standards, database design and the like in a service system can be met, service manufacturers can build according to the unified requirements, data understanding consistency, data standards consistency and database standardization consistency are achieved, and data quality is improved.

For the convenience of reader understanding, the embodiment of the present application schematically provides a scene of online after service system modification. The scenario is a Platform as a Service (PaaS) scenario. Suppose that a project needs to perform business system microservice transformation on a plurality of business systems corresponding to the project. The service system needs to be split and newly built according to the micro-service. When the business system is newly built, the new construction of a database table structure is involved. Since the old business system may be built by multiple manufacturers at different time, the design of the database table structure by the manufacturers is inconsistent. Therefore, by adopting the data processing method provided by the embodiment of the application, the database table structure can be defined before the online of the business systems of all manufacturers. The management system can also realize the floor audit of the unified standard and monitor the floor, and can issue the data standard internally or externally according to the requirements of business personnel and standard administrators.

Further, for the convenience of understanding of readers, the embodiment of the present application schematically illustrates an actual implementation flow of the processing method of the foregoing database table structure. Firstly, a service person A of a first service system applies for an account number in a data standard management system, the data standard management system pushes the application to a system administrator, the application is approved by the system administrator, and if the application is approved by the system administrator, the service person A is allowed to operate the first service system. Business person A can newly build a database, edit the structure of the database and/or delete the database in the first business system. In the process of editing the database, the service person a may establish the database table by online editing or offline editing (including importing the database table or importing information of data elements, etc.). During editing of the database table, data elements may be created, edited, or deleted. During the process of creating or editing data elements, the adjustment of data elements is performed based on the verification of the data standard management system, and then the adjustment of database tables and data elements can also be performed based on manual verification, for example, the database tables and data elements are verified by a system administrator. And after the manual verification is passed, the label falling of the database table is carried out to obtain the structure of the target database table. Finally, the service person a can construct a database table in the database of the first service system based on the script corresponding to the target database table structure provided by the data standard management system. For each link, reference may be made to explanations in the foregoing steps, which are not described in detail in this embodiment of the application.

To sum up, the service data processing method provided in this embodiment of the present application checks information of a database table structure of a service system before the service system is online, thereby ensuring that the service system can adopt an accurate target database table structure after the service system is online. Compared with the prior art, the reliability of the target database table structure is higher, so that the quality of data provided after the business system is online is improved, the probability of data conversion after the business system is online is reduced, and the cost of later-stage data management is reduced.

An embodiment of the present application provides a data standard management system, as shown in fig. 6, the data standard management system includes:

a first obtaining module 301, configured to obtain information of a first database table structure of a first service system to be online, where the information of the first database table structure includes information of at least one data element; a checking module 302, configured to check information of the first database table structure based on a data standard library, where the data standard library includes information of a plurality of standard data elements; a second obtaining module 303, configured to obtain a target database table structure of the first business system after checking the information of the first database table structure, where the target database table structure is determined based on the checked first database table structure.

To sum up, the data standard management system provided by the embodiment of the present application checks the information of the database table structure of the business system by the check module before the business system is online, thereby ensuring that the business system can adopt an accurate target database table structure after being online. Compared with the prior art, the reliability of the target database table structure is higher, so that the quality of data provided after the business system is online is improved, the probability of data conversion after the business system is online is reduced, and the cost of later-stage data management is reduced.

Optionally, the checking module 302 is configured to: when the information of the first data element is not matched with the information of the plurality of standard data elements, sending first modification prompt information, wherein the first modification prompt information indicates that the information of the first data element is updated, and the first data element is one of the at least one data element; and after receiving the updated information of the first data element matched with the information of any one of the plurality of standard data elements, determining that the information verification of the first data element is successful.

Optionally, the information of the first database table structure and the information of the standard data elements each include a data element identifier, the first modification hint information includes information of at least one first standard data element, and the data element identifier of each first standard data element in the at least one first standard data element is in fuzzy matching with the data element identifier of the first data element.

Optionally, the first modification prompt message includes a plurality of pieces of information of the first standard data element, and the pieces of information of the first standard data element are sorted in descending order according to the matching degree between the data element identifier of the first standard data element and the data element identifier of the first data element; and/or sorting in descending order according to the priority of the data standard to which the first standard data element belongs.

Optionally, the data standard management system further includes: the first receiving module is used for receiving the updated information of the first database table structure;

a first determination module to: determining an incremental data element in the updated information of the first database table structure, and determining the information of the first data element in the incremental data element information; alternatively, the information for the first data element is determined from the information for the full amount of data elements of the updated information for the first database table structure.

Optionally, the checking module 302 is configured to: when the format of the information of the first database table structure does not meet the specified format requirement, sending second modification prompt information, wherein the second modification prompt information indicates the format of the information of the first database table structure to be updated; and after receiving the updated information of the first database table structure with the format conforming to the format requirement, determining that the format verification of the information of the first database table structure is successful.

Optionally, the data standard management system further includes:

and a remark module, configured to add remark information of a data element to a second data element when a value corresponding to the second data element is an enumerable value after the information of the first database table structure of the first service system to be online is acquired, where the remark information of the data element is used to identify the enumerable value corresponding to the second data element, and the second data element is one of the at least one data element.

Optionally, the information of the first database table structure and the information of the standard data element both include a data element identifier and data element description information, and the first obtaining module 301 is configured to: receiving a data element identifier of the first data element, wherein the data element identifier of the first data element is one of data element identifiers of a plurality of standard data elements stored in the data standard library; and acquiring data element description information corresponding to the data element identification of the first data element in the data standard library.

Optionally, the first obtaining module 301 is configured to: a data standard document is received, the data standard document including information of the first database table structure.

Optionally, the first obtaining module 301 is configured to: receiving the data standard document generated by the third party modeling tool based on the data standard library.

Optionally, the first obtaining module 301 is configured to: outputting a data dictionary template, wherein the data dictionary template is a reference template of the information of the first database table structure;

Optionally, the data standard management system further includes: the second receiving module is used for receiving a data standard library operation request, wherein the data standard library operation request comprises a standard data element adding request, a standard data element updating request, a standard data element deleting request or a standard data element inquiring request; and the operation module is used for executing the operation corresponding to the operation request of the data standard library to the data standard library after the operation request of the data standard library is successfully authenticated.

Optionally, the data standard management system further includes: the third obtaining module is used for obtaining information of a second database table structure of the second business system after the online, wherein the information of the second database table structure comprises information of at least one data element; a second determining module, configured to determine, when it is detected based on the artificial intelligence model that information of a third data element does not match information of the plurality of standard data elements, a target standard data element corresponding to the third data element in the plurality of standard data elements, where the third data element is one of at least one data element included in the information of the second database table structure; and the establishing module is used for establishing the mapping relation between the third data element and the target standard data element based on the artificial intelligence model.

Optionally, the data standard management system further includes:

and the output module is used for outputting the script corresponding to the target database table structure after the target database table structure of the first service system is acquired.

It should be noted that, when the data standard management system provided in the foregoing embodiment performs the database table structure processing, only the division of the functional modules is illustrated, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the above described functions.

For example, the structure of the data standard management system provided in the embodiment of the present application may also refer to the data standard management systems shown in fig. 1 and fig. 2, where, as shown in fig. 7, the data standard management device 102 may include a business modeling module 1021, a standard auditing module 1022, and a business use module 1023; the life cycle management device 103 may include a standard formulation module 1031 and a data standard library operation module 1032; data governance device 104 includes a data criteria module 1041 and a metadata module 1042.

The business modeling module 1021 can complete the functions of the first obtaining module 301, the verifying module 302 and the second obtaining module 303, that is, execute the actions of the steps 201 to 202; the standard auditing module 1022 may be configured to perform the aforementioned secondary verification process; the service use module 1023 is used for sending a target database table structure use request to the first terminal device of the project administrator after the target database table structure is determined, and sending a database table use notice to the second terminal device of the service personnel after the project administrator completes the construction of the database table through the first terminal device.

The standard making module 1031 is used for establishing information of standard data elements in a data standard database, and a system administrator can specify the information of the standard data elements on line and upload the information to the standard database; the data standard library operation module 1032 may perform the functions of the aforementioned second receiving module and operation module, i.e. perform the actions of the aforementioned step 205 to step 206.

The data standard module 1041 may complete the functions of the aforementioned third obtaining module, the second determining module and the establishing module, i.e. perform the actions of step 207. The metadata module 1042 is configured to set a timing task, periodically collect information (such as metadata) of a database table of the managed service system, check the information of the database table and the data element that are newly created or updated by the service system, and perform data standard and normative verification, where the process may refer to the corresponding process in step 202. This can reduce the amount of information in the service system that is not in a satisfactory format.

Optionally, fig. 8 schematically provides one possible basic hardware architecture of the computing device described herein.

Referring to fig. 8, computing device 400 includes a processor 401, memory 402, a communication interface 403, and a bus 404.

In the computing device 400, the number of the processors 401 may be one or more, and fig. 8 only illustrates one of the processors 401. Alternatively, the processor 401 may be a Central Processing Unit (CPU). If the computing device 400 has multiple processors 401, the types of the multiple processors 401 may be different, or may be the same. Optionally, the plurality of processors 401 of the computing device 400 may also be integrated as a multi-core processor.

Memory 402 stores computer instructions and data; the memory 402 may store computer instructions and data necessary to implement the data redistribution methods provided herein, e.g., the memory 402 stores instructions for implementing the steps of the data redistribution methods. The memory 402 may be any one or any combination of the following storage media: nonvolatile memory (e.g., Read Only Memory (ROM), Solid State Disk (SSD), hard disk (HDD), optical disk), volatile memory.

The communication interface 403 may be any one or any combination of the following devices: a network interface (e.g., an ethernet interface), a wireless network card, etc. having a network access function.

The communication interface 403 is used for the computing device 400 to communicate data with other computing devices or terminals.

The bus 404 may connect the processor 401 with the memory 402 and the communication interface 403. Thus, via bus 404, processor 401 may access memory 402 and may also interact with other computing devices or terminals using communication interface 403.

In the present application, the computing device 400 executes computer instructions in the memory 402, causing the computing device 400 to implement the data redistribution methods provided herein, or causing the computing device 400 to deploy a data standard management system.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, executable by a processor of a server to perform a data processing method as shown in various embodiments of the present application is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In addition, the data standard management system provided by the above embodiment and the data processing method embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.

The term "and/or" in this application is only one kind of association relationship describing the associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

In this application, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" means two or more unless expressly limited otherwise. "A refers to B" and means that A is the same as B or A is simply modified based on B.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of data processing, the method comprising:

acquiring information of a first database table structure of a first business system to be online, wherein the information of the first database table structure comprises information of at least one data element;

checking information of the first database table structure based on a data standard library, wherein the data standard library comprises information of a plurality of standard data elements;

and after the information of the first database table structure is verified, acquiring a target database table structure of the first business system, wherein the target database table structure is determined based on the verified first database table structure.

2. The method of claim 1, wherein the checking the information of the first database table structure based on the database standard comprises:

when the information of a first data element is not matched with the information of the plurality of standard data elements, sending first modification prompt information, wherein the first modification prompt information indicates that the information of the first data element is updated, and the first data element is one of the at least one data element;

after receiving the updated information of the first data element matched with the information of any one of the plurality of standard data elements, determining that the information verification of the first data element is successful.

3. The method of claim 2, wherein the information of the first database table structure and the information of the standard data elements each include a data element identification, and wherein the first modification hint information includes information of at least one first standard data element, the data element identification of each of the at least one first standard data element being a fuzzy match with the data element identification of the first data element.

4. The method according to claim 3, wherein the first modification hint information comprises a plurality of information of the first standard data elements, the plurality of information of the first standard data elements being sorted in descending order of matching degree of the data element identification of the first standard data element with the data element identification of the first data element; and/or sorting in descending order according to the priority of the data standard to which the first standard data element belongs.

5. The method of any of claims 2 to 4, further comprising:

receiving updated information of the first database table structure;

determining an incremental data element in the updated information of the first database table structure, and determining the information of the first data element in the incremental data element information;

or, determining the information of the first data element in the information of the total amount of data elements of the updated information of the first database table structure.

6. The method of any of claims 1 to 5, wherein the checking the information of the first database table structure based on the database standard comprises:

when the format of the information of the first database table structure does not meet the specified format requirement, sending second modification prompt information, wherein the second modification prompt information indicates that the format of the information of the first database table structure is updated;

and after receiving the updated information of the first database table structure with the format meeting the format requirement, determining that the format verification of the information of the first database table structure is successful.

7. The method according to any one of claims 1 to 6, wherein after said obtaining information of the first database table structure of the first business system to be online, the method further comprises:

when the value corresponding to the second data element is an enumeratable value, adding data element remark information to the second data element, where the data element remark information is used to identify the enumeratable value corresponding to the second data element, and the second data element is one of the at least one data element.

8. The method according to any one of claims 1 to 7, wherein the information of the first database table structure and the information of the standard data element each include data element identification and data element description information, and the acquiring the information of the first database table structure of the first business system to be online includes:

receiving a data element identifier of the first data element, wherein the data element identifier of the first data element is one of data element identifiers of a plurality of standard data elements stored in the data standard library;

and acquiring data element description information corresponding to the data element identification of the first data element in the data standard library.

9. The method according to any one of claims 1 to 7, wherein the obtaining information of the first database table structure of the first business system to be online comprises:

receiving a data standard document, wherein the data standard document comprises information of the first database table structure.

10. The method of claim 9, wherein receiving the data standard document comprises:

receiving the data standard document generated by a third party modeling tool based on the data standard library.

11. The method according to claim 8 or 9, wherein the obtaining information of the first database table structure of the first business system to be online comprises:

receiving information of the first database table structure input based on the data dictionary template.

12. The method according to any one of claims 1 to 10, further comprising:

receiving a data standard library operation request, wherein the data standard library operation request comprises a standard data element adding request, a standard data element updating request, a standard data element deleting request or a standard data element inquiring request;

and after the authentication of the operation request of the data standard library is successful, executing the operation corresponding to the operation request of the data standard library on the data standard library.

13. The method of any one of claims 1 to 12, further comprising:

acquiring information of a second database table structure of a second business system after the second business system is online, wherein the information of the second database table structure comprises information of at least one data element;

when it is detected based on an artificial intelligence model that information of a third data element does not match information of the plurality of standard data elements, determining a target standard data element corresponding to the third data element in the plurality of standard data elements, wherein the third data element is one of at least one data element included in the information of the second database table structure;

and establishing a mapping relation between the third data element and the target standard data element based on the artificial intelligence model.

14. The method according to any one of claims 1 to 13, wherein after said obtaining the target database table structure of the first business system, the method further comprises:

and outputting the script corresponding to the target database table structure.

15. A data standard management system, the system comprising:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring information of a first database table structure of a first business system to be online, and the information of the first database table structure comprises information of at least one data element;

the verification module is used for verifying the information of the first database table structure based on a data standard library, and the data standard library comprises information of a plurality of standard data elements;

and the second acquisition module is used for acquiring the target database table structure of the first business system after checking the information of the first database table structure, and the target database table structure is determined based on the checked first database table structure.

16. The system of claim 15, wherein the verification module is configured to:

17. The system of claim 16, wherein the information of the first database table structure and the information of the standard data elements each include a data element identification, and wherein the first modification hint information includes information of at least one first standard data element, the data element identification of each of the at least one first standard data element being a fuzzy match with the data element identification of the first data element.

18. The system according to claim 17, wherein the first modification hint information includes a plurality of information about the first standard data elements, the plurality of information about the first standard data elements being sorted in descending order of matching of the data element identifiers of the first standard data elements with the data element identifiers of the first data elements; and/or sorting in descending order according to the priority of the data standard to which the first standard data element belongs.

19. The system of any one of claims 16 to 18, further comprising:

the first receiving module is used for receiving the updated information of the first database table structure;

a first determination module to:

20. The system of any one of claims 15 to 19, wherein the verification module is configured to:

21. The system of any one of claims 15 to 20, further comprising:

22. The system according to any one of claims 15 to 21, wherein the information of the first database table structure and the information of the standard data elements each include a data element identification and a data element description information, the first obtaining module is configured to:

23. The system according to any one of claims 15 to 22, wherein the first obtaining module is configured to:

24. The system of claim 23, wherein the first obtaining module is configured to:

25. The system according to claim 23 or 24, wherein the first obtaining module is configured to:

26. The system of any one of claims 15 to 25, further comprising:

the second receiving module is used for receiving a data standard library operation request, wherein the data standard library operation request comprises a standard data element adding request, a standard data element updating request, a standard data element deleting request or a standard data element inquiring request;

and the operation module is used for executing the operation corresponding to the operation request of the data standard library to the data standard library after the operation request of the data standard library is successfully authenticated.

27. The system of any one of claims 15 to 26, further comprising:

the third obtaining module is used for obtaining information of a second database table structure of the second business system after the online, wherein the information of the second database table structure comprises information of at least one data element;

a second determining module, configured to determine, when it is detected based on an artificial intelligence model that information of a third data element does not match information of the plurality of standard data elements, a target standard data element corresponding to the third data element in the plurality of standard data elements, where the third data element is one of at least one data element included in the information of the second database table structure;

and the establishing module is used for establishing the mapping relation between the third data element and the target standard data element based on the artificial intelligence model.

28. The system of any one of claims 15 to 27, further comprising:

and the output module is used for outputting the script corresponding to the target database table structure after the target database table structure of the first service system is obtained.

29. A computer device, comprising:

a processor and a memory;

the memory to store computer instructions;

the processor, configured to execute the computer instructions stored by the memory, to cause the computing device to perform the data processing method of any of claims 1 to 14.

30. A computer-readable storage medium comprising computer instructions that direct a computing device to perform the data processing method of any of claims 1 to 14.