CN115794833A - Data processing method, server and computer storage medium - Google Patents

Data processing method, server and computer storage medium Download PDF

Info

Publication number
CN115794833A
CN115794833A CN202211632995.4A CN202211632995A CN115794833A CN 115794833 A CN115794833 A CN 115794833A CN 202211632995 A CN202211632995 A CN 202211632995A CN 115794833 A CN115794833 A CN 115794833A
Authority
CN
China
Prior art keywords
field
server
data
association
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211632995.4A
Other languages
Chinese (zh)
Inventor
刘永华
赵学亮
曾广锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN202211632995.4A priority Critical patent/CN115794833A/en
Publication of CN115794833A publication Critical patent/CN115794833A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a data processing method, a server and a computer storage medium, and relates to the technical and financial field, in particular to the data processing field. The method comprises the following steps: the server obtains the access data and the operation instruction when the interface is called, and further obtains the interface execution information. The server analyzes the interface execution information to obtain field association information and data table association information. And the server acquires a service object main table and an association table corresponding to the target service from the data association information according to the target service. And the server determines a data template according to the business object main table and the association table. And the server updates the field value in the data template according to the field rule to obtain the data to be generated. And the server uses the data to be generated to realize the number of the target service. The method improves the accuracy of acquiring the incidence relation and the field incidence relation of the data table, and further improves the processing efficiency and the number making effectiveness of the data.

Description

Data processing method, server and computer storage medium
Technical Field
The present application relates to the field of financial technology, and in particular, to a data processing method, a server, and a computer storage medium.
Background
With the development of computer technology, more and more financial businesses realize the electronization of technology and can be realized in a system, and the traditional financial industry is gradually shifting to financial technology (Fintech).
In the system database, each financial transaction will typically correspond to at least one data table. Each data table may include a plurality of fields therein. Each field corresponds to a field name. There may be some correlation between these data tables. In the prior art, a server may determine whether there is an association between two data tables by determining similarity of field names in the two data tables.
However, as the amount of data in the database increases, there may be situations where fields with different field names in different data tables correspond to the same content, or where fields with the same field name in different data tables correspond to different content. The occurrence of these situations easily causes a problem that the accuracy of the determination of the relevance of the data table is low, and further the data processing efficiency is reduced.
Disclosure of Invention
The application provides a data processing method, a server and a computer storage medium, which are used for solving the problem of reducing the data processing efficiency.
In a first aspect, the present application provides a data processing method, including:
acquiring and analyzing access data and operation instructions of a plurality of interfaces of a system during multiple calls to obtain interface execution information of each interface during each call, wherein the interface execution information at least comprises an interface number, an access field value and an associated field;
analyzing and obtaining field associated information and data table associated information of each data table in a database of the system according to the interface execution information of the plurality of interfaces of the system during multiple calls;
determining a data template corresponding to the target service according to the data table association information and the target service;
and generating the data to be generated of the target according to the data template and the field association information, and finishing the number of the target service by using the data to be generated.
In a second aspect, the present application provides a data processing apparatus comprising:
the acquisition module is used for acquiring and analyzing the access data and the operation instructions of a plurality of interfaces of the system during multiple calls to obtain interface execution information of each interface during each call, wherein the interface execution information at least comprises an interface number, an access field value and an associated field;
the processing module is used for analyzing and obtaining field associated information and data table associated information of each data table in a database of the system according to the interface execution information of the plurality of interfaces of the system during multiple calls; determining a data template corresponding to the target service according to the data table association information and the target service; and generating the data to be generated of the target according to the data template and the field association information, and finishing the number of the target service by using the data to be generated.
In a third aspect, the present application provides a server, comprising: a memory and a processor;
the memory is used for storing a computer program; the processor is configured to perform the data processing method of the first aspect and any one of the possible designs of the first aspect according to a computer program stored in the memory.
In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored therein, where the computer program is executed by at least one processor of a server, and the server executes the data processing method of the first aspect and any possible design of the first aspect.
In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by at least one processor of a server, causes the server to perform the data processing method of the first aspect and any one of the possible designs of the first aspect.
According to the data processing method, the server and the computer storage medium, the monitoring process is added in the system, and the access data and the operation instruction when the interface is called are obtained; sorting the input data and the operation instructions of each call of each interface one by one to obtain interface execution information of each call of each interface; after accumulating enough interface execution information, uniformly analyzing the interface execution information to obtain field association information and data table association information of each data table in a database of the system; acquiring a target service; acquiring a service object main table and an association table corresponding to the target service from the data association information according to the target service; acquiring latest inserted data from the main table and the associated table of the business object in a query mode; determining a field rule of each field in the data template according to the field type; updating the field value in the data template according to the field rule of each field to obtain data to be generated; and inserting the data to be generated into the corresponding business object main table and the associated table through a database insertion statement to realize the means of the number of the target business, realize the generation of the associated information of the data table, improve the acquisition accuracy of the associated relation and the field associated relation of the data table, and further improve the processing efficiency of the data and the effect of the validity of the number of the business.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a flowchart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a tree structure of an operation instruction according to an embodiment of the present application;
fig. 4 is a flowchart of a data processing method according to an embodiment of the present application;
FIG. 5 is a flowchart of an example of data processing provided by an embodiment of the present application;
FIG. 6 is a flowchart of an algorithm for analyzing association relationship of data tables according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic hardware structure diagram of a server according to an embodiment of the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged where appropriate. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The word "if," as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination," depending on the context.
Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise.
The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.
With the development of computer technology, more and more financial businesses realize the electronization of technology and can be realized in a system, and the traditional financial industry is gradually shifting to financial technology (Fintech). In the system database, each financial transaction will typically correspond to at least one data table. There may be some correlation between these data tables. Each data table may include a plurality of fields therein. Each field corresponds to a field name. There may be at least one field in each data table that is the primary key of the data table. In the following, the server introduces the concepts of business object, business object main field, and business object main table. In the system using the relational database, if one data object has one data table as the basic information table and at least one associated table, the data object is the service object of the corresponding service system. When the field a is the only primary key field in the table where the field a is located, and there is a field B in the association table that is consistent with the usage of the field a, the server may further determine whether the field a is used more frequently than the field B in the association table in the operation instruction where the interface call is yes. If yes, the field A is the main field of the business object. The table in which the main field of the service object is located is the main table of the service object.
Currently, a business object in a system has a basic information table of the business object as a main table of the business object. The only primary key field in the table represents this business object. If the other data table contains the field, the other data table represents that the data table is the association table of the business object. In the prior art, a server may determine whether there is an association between two data tables by determining similarity of field names in the two data tables. That is, the server may find an association table of a business object by determining whether the similarity between two field names is greater than a threshold. In the similarity judging process, the server may judge whether the two fields are the same by calculating the similarity of the character strings corresponding to the field names after underlining all the fields and uniformly converting the case. In the process, the server can also use the module names of different modules and the table names of different data tables as prefixes of the field names so as to distinguish the fields from different modules and different data tables.
However, as the usage time of the system increases, the number of data tables in the database of the system increases, the field names in the data tables also increase with the increase of the data tables, and the complexity of the system increases. The system may not be uniform in naming fields in a data table in a database. For example, it may happen that fields of different field names in different data tables correspond to the same content, or that fields of the same field name in different data tables correspond to different content, and so on. For example, a fund product is a business object of a financial system. The server may use the fund product information table "fund _ info" as the business object master table. The only primary key field in the business object primary table is "prod _ code". The fund product limit table 'fund _ limit', the transaction flow table 'trans _ log' and the fund product information table 'fund _ info' are three association tables of the fund product (service object). The "fund _ code" field in the fund product limit table "fund _ limit", the "inner _ code" field and the "ta _ sub _ code" field in the transaction flow meter "trans _ log", and the "prod _ code" field in the fund product information table "fund _ info", although not named, are used consistently and correspond to the same content. As another example, the fund product (business object) may further include an association table insurance product information table "entity _ info". The "prod _ code" field in the insurance product information table "interior _ info" and the "prod _ code" field in the fund product information table "fund _ info" have the same field name, but the usage of the two fields is not consistent and corresponds to different substantive contents. Therefore, in the prior art, the problems of low judgment accuracy and poor comprehensiveness exist in the association table of the business object directly analyzed and judged according to the similarity of the field names.
During system testing, the server usually needs to generate a large number of test numbers. The server may use these test metrics to complete the testing of the system. Currently, inserting data into a data stream is a common method of data stream generation. When inserting the data number, the server determines a service object according to the target service. The server can find the basic information table of the business object according to the definition of the business object. The server may use the basic information table as a business object master table. The server can find other data tables containing the unique primary key field in the business object primary table in a field name similarity searching mode. The server may use these data tables as the association table for the business object. When the business object is needed to be used for system test, the server can select part or all of the data tables from the business object main table and the association table according to the actual requirement of the number of the manufacture. The server can insert new data into the business object main table and the association table of the business object in a data inserting mode to realize the number making. Wherein the database may be an SQL database. According to the method for manufacturing the number, the server can not accurately and effectively realize the test manufacturing when the server can not accurately obtain the association table of the business object.
In order to solve the above technical problem, the inventive concept of the data processing method provided in the embodiment of the present application is that the server may obtain the access data and the operation instruction when the interface is called each time through the interface of the monitoring service system. When the database corresponding to the business system is an SQL database, the operation instruction may be an SQL statement. The server can analyze the input parameter data of the interface call and determine the interface execution information containing the relationship of the fields in each data table in the database. The server may determine the field association information based on the interface execution information. The server can also obtain the interface execution information in the operation instruction by analyzing the operation instruction during each interface calling. The operation instruction may include a plurality of database control instructions. The instruction types of the database control instruction can include adding, deleting, modifying and inquiring. The server may analyze each database control instruction to obtain a main field, a direct association field, and an indirect association field included in the database control instruction. The main field, the direct association field and the indirect association field constitute field association information of the operation instruction. Further, the server may determine the data table association information based on the field association information. The data table association information may include a service object master table and an association table corresponding to the service object master table. The method comprises the steps of obtaining all service object main fields and other fields which are directly related and indirectly related to the corresponding service object main fields by analyzing field related data by using an improved association analysis Apriori algorithm, and further obtaining a service object main table and an association table thereof. In addition, when the operation instruction executed by single interface calling is analyzed, a dynamic programming and breadth-first algorithm is used for the complex query database control instruction. The server may determine all associated fields in the database control instruction according to the field after finding the field having the direct value in the database control instruction. In addition, after the data table association information is obtained, the server can also obtain all fields corresponding to the target service according to the main table and the association table in the data table association information. The server may generate a corresponding data template from these fields. The server may generate a new input according to the data template and insert the new data into the corresponding master table and association table to implement the manufacture. According to the method, after the retrieval accuracy of the business object main table and the business object association table is improved, correct number making basis can be provided for technicians, and therefore the number making efficiency is improved.
In the present application, a server is used as an execution agent to execute the data processing method of the following embodiment. Specifically, the execution body may be a hardware device of the server, or a software application in the server, or a computer-readable storage medium on which the software application implementing the following embodiment is installed, or code of the software application implementing the following embodiment.
The technical solution of the present application will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 1 shows a flowchart of a data processing method according to an embodiment of the present application. As shown in fig. 1, with a server as an execution subject, the method of this embodiment may include the following steps:
s101, obtaining and analyzing access data and operation instructions of a plurality of interfaces of the system during multiple calls, and obtaining interface execution information of each interface during each call, wherein the interface execution information at least comprises an interface number, an access field value and an associated field.
In this embodiment, the server may add a monitoring process to the system. The monitoring process can acquire the reference data and the operation instruction when the interface of the system is called. The server can sort the input parameter data and the operation instruction of each call of each interface one by one to obtain the interface execution information of each call of each interface. The interface execution information may include the interface execution pipeline and the interface number of the call. Wherein the interface execution pipeline is used to uniquely identify a call of an interface. The interface execution information can also comprise an entry field and an entry field value which are determined according to the entry parameter data. The interface execution information can also be determined according to the access data and the operation instruction, and the related field of the interface during the calling is determined.
In one implementation, the parameter data and the operation instruction obtained by the server may be as shown in table 1. Wherein, each row corresponds to the access data and the operation instruction acquired by one interface of the server when being executed at one time.
TABLE 1
Figure BDA0004006543240000071
Figure BDA0004006543240000081
Wherein the interface execution sequence number is generated when the interface is called. The interface execution serial number is used to uniquely identify a call to an interface in the system. For example, as shown in table 1, the call information of 6 times of interface call is recorded. The 6-call interface executes the pipelined XXXX00001, XXXX00002, … …, respectively. This interface execution pipeline is merely an example to indicate that the interface execution pipeline may be composed of strings and numbers. Where the string may be preset by a technician and the number may be an incremental number. The interface number is used to uniquely identify an interface. For example, as shown in table 1, the 6 interface calls collectively involve three interfaces interface001, interface002 and interface 003. When the interface is called each time, the server can obtain the interface number of the called interface and record the interface number together with the interface execution serial number. The interface call may include multiple entries. For example, as shown in table 1, when the interface001 interface call with the serial number XXXX00001 is executed, the entry parameter data acquired by the server includes 5 fields. Wherein, the field E corresponds to a value of 5, the field D corresponds to a value of C, the field C corresponds to a value of C, and the field X corresponds to values [ { "prod _ code": 000001', "B": B "}, {" prod _ code ": 000002", "B": B2 ] }. The value corresponding to the field X may specifically include a field prod _ code and a field B. Wherein the field prod _ code may have values of 000001 and 000002. The corresponding value of field B may include B and B2. The operation instruction of the interface may specifically include a plurality of database control instructions. In the example used in the present application, the database used is an SQL database, and therefore, the operation instruction may specifically be an SQL instruction. For example, as shown in table 1, when the interface001 interface call with the pipeline number XXXX00001 is executed, the operation instructions acquired by the server may include 6 instructions for implementing data insertion, data deletion, data update, and data query.
In one implementation, the interface execution information obtained by the server may be as shown in table 2, where each row corresponds to interface execution information generated after one execution of one interface.
TABLE 2
Figure BDA0004006543240000082
Figure BDA0004006543240000091
The interface execution pipeline and the interface number in table 2 are the same as those in table 1. The server may analyze the entry field, the entry field value, and the association field in the interface execution information of table 2 according to the entry field and the operation instruction in table 1. Wherein, at least one field can be included in the parameter field. For example, as shown in Table 2, the interface001 interface with pipeline number XXXX00001 is executed, corresponding to an entry field. Its entry field is prod _ code. The argument field values correspond to argument fields, each of which may correspond to the contents enclosed by middle brackets "[ ]" in the argument field values. For example, as shown in table 2, the entry field values corresponding to the entry field prod _ code of the interface001 interface with the execution pipeline number XXXX00001 are [ "000001", "000002" ]. The association field is used to represent other fields in other data tables associated with the reference field. The association corresponds to the same content as the reference field, and has the same usage mode. For example, as shown in table 2, the interface001 interface with the pipeline number XXXX00001 is executed, the entry field value may be, and the association field may include [ "fund _ info.prod _ code", "trans _ log.inner _ code", "trans _ log.ta _ sub _ code", "fund _ limit.fund _ code", "e1.E11", "f1.F11", "g1.G11", "j1.J11" ]. That is, the field prod _ code in the data table tend _ info corresponds to the same contents and has the same usage as the field inner _ code in the data table trans _ log, the field ta _ sub _ code in the data table trans _ log, the field fund _ code in the data table tend _ limit, the field E11 in the data table E1, the field F11 in the data table F1, the field G11 in the data table G1, and the field J11 in the data table J1.
The specific process that the server can generate the interface execution information shown in table 2 according to the entry data and the operation instruction shown in table 1 may be as shown in fig. 2.
S102, analyzing and obtaining field associated information and data table associated information of each data table in a database of the system according to interface execution information of a plurality of interfaces of the system during multiple calls.
In this embodiment, the server may generate one piece of interface execution information for each interface call. The server may record the interface execution information in table 2. The server can perform unified analysis on the interface execution information after accumulating enough interface execution information.
In one example, the server may use index information consisting of an interface number, a reference field, and a reference field value of the interface execution information. Each index may correspond to a row in table 2. When the server detects the same index information, the server may delete the row corresponding to the index. For example, in the interface execution information with the interface execution pipeline of XXXX00017, the interface number may be interface001, the entry field may be prod _ code, and the entry field value may be [ "000001", "000002" ]. The index information generated by the interface execution information whose interface execution pipeline is XXXX00017 is the same as the index information generated by the interface execution information whose interface number may be interface 001. At this time, in order to improve the effectiveness of association calculation, the server may delete the interface execution information corresponding to the interface execution pipeline XXXX00017, so as to ensure the uniqueness of the index information.
In one example, the server may accumulate pieces of index information. When the number of pieces of index information reaches the preset number, the server may perform the analysis of step S102. The number of pieces of index information corresponds to the number of rows in table 2, or the number of pieces of interface execution information acquired by the server. For example, the preset number may be 1000.
The server may count the associated fields in the execution information of the preset number of interfaces, and determine non-repeating fields contained therein. This field may be represented by a "table name field name". The non-repeating field is a single entry field. The server may also count the number of occurrences of each individual field. The server may analyze the association between these single-term fields using the modified Apriori algorithm. The goal of the Apriori algorithm is to find the largest frequent set of K terms. The server can improve the Apriori algorithm according to the requirements, and find the maximum number of associated fields by supporting the relevance transmitted among different fields. Specifically, the server may determine whether an association relationship exists between two single fields by calculating a support degree, a confidence degree, and a promotion degree between the two single fields. The server can determine the field association information according to the association relationship among the single fields.
In one implementation, the field association information and the data table association information may be presented in table 3.
TABLE 3
Figure BDA0004006543240000101
The service in table 3 forms a single field with the main field and the line main table, and the association table and the association field form a single field. For example, the service object main table fund _ info and main field prod _ code in the first row of table 3 may constitute a single field fund _ info. Wherein, the support degree, the confidence degree and the promotion degree form the associated indexes of two single fields. The content as recorded in table 3 is a single entry field where there is an association. Wherein, the associated fields of the first row in table 3 include inner _ code and ta _ sub _ code. The two association fields may constitute a single field trans _ log.inner _ code and a single field trans _ log.ta _ sub _ code with an association table. The two singleton fields are both associated with a singleton field, fund _ info. The server may calculate the association indexes of the one-entry field fund _ info.prod _ code with the one-entry field trans _ log.inner _ code and the one-entry field trans _ log.ta _ sub _ code, respectively. The server may record a larger set of correlation metrics in this table 3. The business object main table and the association table in table 3 constitute data table association information.
The judgment of the association relationship between every two single fields and the calculation of the support degree, the confidence degree and the promotion degree index may be specifically shown in fig. 4.
S103, determining a data template corresponding to the target service according to the data table association information and the target service.
In this embodiment, the server may obtain the target service. The target service is used to indicate a service object that needs to generate a test number. The server can obtain a service object main table and an association table corresponding to the target service from the data association information according to the target service. The server can obtain the latest inserted data from the business object main table and the association table in a query mode. The server may generate a data template from these fields.
In one example, the process of determining the data template by the server specifically includes the following steps:
step 1, determining a corresponding business object main table according to a target business.
In this step, the server may select the service object master table according to the target service. The business object master table may be determined according to the data table association information.
And 2, determining an association table with association according to the data table association information and the business object main table.
In this step, the server may determine the association table corresponding to the service object master table according to the data table association information.
In one example, the server may sort the association tables from high to low according to their support degrees. The server may display the sorted association table. The technician may select one or more association tables from the association tables after viewing the association tables. The server may use these selected association tables as the association table to be used in the next step.
In another example, the server may select a part or all of the association tables from the association tables according to a preset rule as the association table used in the next step.
And 3, searching the business object main table and the association table to obtain the data template.
In this step, the server may obtain a row of data that is newly inserted from the service object master table. The server may retrieve from these association tables the data rows corresponding to the most recently inserted row of data of the business object master table. Alternatively, the server may obtain the field value corresponding to the main field of the business object input by the technician. The server may obtain all data corresponding to the field value by querying from the service object main table and the association table according to the field value. For example, the service object main table is fund _ info, and the service object main field is prod _ code. The server may obtain the value of the field of the main field of the business object as "000003". The server may generate a query statement for the field value of the field from the business object master table and the association table. The SQL query statement for querying the fund _ info table is select from fund _ info where prod _ code = '000003'. The association table may include trans _ log and fund _ limit. The query statement of the association table may include select from trans log where inner _ code = '000003' and ta sub code = '000003' and select from fund _ limit where word fund _ code = '000003'.
The server may generate a corresponding insert statement according to the data obtained by the query. The insert statement is the data template.
In one example, if the server cannot obtain data from the service object master table and the association table during query, the server cannot generate a data template corresponding to the secondary number. The manufacturing is finished.
And S104, generating target data to be generated according to the data template and the field association information, and completing the number making of the target service by using the data to be generated.
In this embodiment, the server may determine the field rule of each field in the data template according to the field type. For example, the field rule may be that the primary key field cannot be duplicated with existing data in the table, that a field of the serial number type should be generated incrementally, and so on. The server may update the field value in the data template according to the field rule of each field, to obtain the data to be generated. The server can insert the data to be generated into the corresponding business object main table and the associated table through the database insertion statement to realize the number of the target business. The server may also obtain the number of artifacts that need to be generated. The server can update according to the number of the data to be generated to obtain a plurality of data to be generated, and the number of the data to be generated is completed in a data insertion mode.
In one example, the process of generating the data to be generated may specifically include the following steps:
step 1, resetting the relevant fields in the data template to the same value according to the field relevant information.
In this step, the server may first reset the plurality of associated fields in the data template to the same value according to the field association information. The reset value is different from the values already in the data table. The reset value conforms to the rules of the field.
And 2, resetting the values of other fields in the data template according to the field rule.
In this step, when the field is a unique primary key or a unique index, the server may update the field value of the field to an appropriate and non-repeating value. When the field type of the field is the serial number type, the server may generate a non-repeating serial number for the field according to a specified serial number generation rule. When the field type of the field is the self-increment ID type, the server can generate a corresponding appropriate new value for the field according to the specified self-increment ID type. When the field type of the field is a numeric type or a non-serial number or a non-self-increment ID, and the field is not a unique primary key or a unique index, the server can keep the field value unchanged.
In one example, during the manufacturing process, the server inserts the number of data pieces and the query result of each data sheet for each manufacturing. For example, the query statement for querying the association table trans _ log may be select from trans _ log where inner _ code = '000003' and ta _sub _code = '000003'. The query result of the server may be matched to 3 pieces of data satisfying the query condition in trans _ log. The server may automatically generate the 3 pieces of data corresponding to the data to be generated at the time of the manufacture. The server can process differently according to different field types of each piece of data. For example, if the field fund12 in the trans _ log table is the running number type field, the server can generate 3 corresponding new values for the 3 pieces of data according to the running number generation rule. As another example, if field fund13 in the trans _ log table is a type of auto-ID field, the server can generate 3 appropriate new values for the 3 pieces of data according to the auto-ID generation rule. As another example, if the field fund14 is a numeric type field, the server may keep the values of the 3 pieces of data unchanged. Or, the technician may also delete 1 of the 3 pieces of data obtained by querying the trans _ log table, and modify some fields in the remaining 2 pieces of data to generate the data to be generated.
According to the data processing method, the server can add a monitoring process in the system. The monitoring process can acquire the reference data and the operation instruction when the interface of the system is called. The server can sort the input parameter data and the operation instruction of each call of each interface one by one to obtain the interface execution information of each call of each interface. The server can perform unified analysis on the interface execution information after accumulating enough interface execution information to obtain field association information and data table association information of each data table in a database of the system. The server may obtain the target service. The server can obtain the business object main table and the association table corresponding to the target business from the data association information according to the target business. The server can obtain the latest inserted data from the business object main table and the association table in a query mode. The server may determine the field rules for each field in the data template based on the field type. The server may update the field value in the data template according to the field rule of each field, to obtain the data to be generated. The server can insert the data to be generated into the corresponding business object main table and the associated table through the database insertion statement to realize the number of the target business. According to the method and the device, the generation of the associated information of the data table is realized through the analysis of the access data and the operation instruction, the acquisition accuracy of the associated relation of the data table and the associated relation of the field is improved, the associated relation of the data table is accurately acquired, and the processing efficiency and the manufacture number effectiveness of the data are further improved.
Fig. 2 shows a flowchart of a data processing method according to an embodiment of the present application. On the basis of the embodiment shown in fig. 1, in step S101, the specific step of generating the interface execution instruction by the server according to the parameter and the operation instruction may be as shown in fig. 2, where the server is used as an execution subject, and the method of this embodiment may include the following steps:
s201, acquiring the parameter entering data and the operation instruction of the interface when the interface is called through the monitoring interface.
In this embodiment, the server may call the monitoring program in the system background running interface. When an interface is called and executed by a tested service system, a program acquires the parameter data requested to be input and the operating instruction correspondingly executed in the process of executing and calling the secondary interface by monitoring the thread execution record when the corresponding interface runs. The operation instruction may be a database control instruction. For example, the database control instructions may be SQL statements.
In one example, the server may further store the reference data and the corresponding executed operation instructions when the single interface is called to the database. The database control command may include addition, deletion, modification, and search statements. Specifically, the SQL statement may be as shown by the operation instructions of table 1.
S202, analyzing the parameter data to obtain a parameter dictionary. And analyzing the operation instruction to obtain an instruction dictionary.
In this embodiment, the server may analyze the parameter entering data and the operation instruction called by the single interface, respectively, to obtain a parameter entering dictionary and an instruction dictionary. In the following processing procedures, the method can automatically analyze and directly extract the parameter entering field and the parameter entering field value from parameter entering data and operation instructions. Since the argument field and the argument field value can be directly extracted, the argument field value can be a direct value. Alternatively, the server may obtain the direct value according to the key field of the service object. The direct value has the characteristic of a non-processed value. The server can also determine the direct association relationship between the parameter-entering field and the fields in each data table in the database according to the parameter-entering field after the parameter-entering data is analyzed. In the process of analyzing the operation instruction, when the operation instruction is an SQL statement, the operation instruction may specifically include four statements of add, delete, modify, and check. For the three kinds of SQL statements of add, delete, and change, the server may generally obtain a direct value and/or a direct association relationship. For the queried SQL statement, because the complexity of the SQL query statement may be high, the server may use dynamic programming and breadth-first algorithm to find all the associated fields in SQL and their corresponding data tables corresponding to the fields having direct values in the SQL.
In one example, the specific process of the server parsing the access data includes the following steps:
step 1, analyzing the parameter input data to obtain a parameter input dictionary of the parameter input data, wherein the parameter input dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value.
In this step, the server may process the entry parameter data of the interface to obtain the key-value key value pair in the entry parameter data. When the entry parameter data contains nested key-value pairs, the server can sort the entry parameter data into a data dictionary with only one layer of key-value pairs. The data dictionary is the reference dictionary.
The server may include an interface parameter data handler for handling the interface parameter data. The server may initialize the interface input parameter data handler upon receiving the input parameter data for a single interface call, place the interface input parameter data handler in an INIT state, and begin analysis. For example, for the reference data shown in table 1, the server may first process the reference data into a reference dictionary with only one layer of key-value key values. The processed reference dictionary may be:
Figure BDA0004006543240000141
and 2, deleting the key value pair if the data type of the field value of the key value pair in the reference dictionary is different from the preset type.
In this step, the server may filter the reference dictionary. The server may obtain the data type defined by the primary key field in the primary table of the business object. The data type of the primary key field is a preset type. The server may delete key-value pairs in the data dictionary that are different from the preset type. For example, when the preset type is a string type, the server may delete a key-value pair of a non-string type such as a reshape, a float, a list, and the like. Even if the server does not speak the key-value pair deletion of the data type different from the preset type in this step, the server will not analyze the key-value pairs in the subsequent processing. For example, for the above-mentioned reference dictionary, the data type of the field value corresponding to the field E is the shaping. Thus, the server may delete the key-value pair "E": 5. In another example, the data type of the field value corresponding to the field B is a list. Thus, the server may delete the key-value pair "B" [ "B", "B2" ].
And 3, deleting a plurality of field values if the field values of a plurality of key value pairs in the reference dictionary are the same.
In this step, the server may exclude key-value pairs with equal field values from the reference dictionary. The deletion of these key-value pairs can preclude accidental equality from interfering with the analysis. For example, for the above-described reference dictionary, the field C and the field B have the same field value. Thus, the server may delete the key-value pair "C": C and the key-value pair "D": C ".
The execution of step 2 and step 3 is not limited by the described order of actions, and step 2 and step 3 may be performed in other orders or simultaneously. After the server completes the execution of the above 3 steps, the execution of the input parameter data processing program is finished, and the input parameter data processing program is in a FINISH state. For the reference dictionary of the reference data shown in table 1, the final reference dictionary obtained by the server may be:
{
"prod_code":["000001","000002"]
}
in one example, the operation instruction may include at least one database control instruction. When the database is an SQL database, the database control command may be an SQL statement. As shown in table 1, the operation command called by the interface at one time may include a strip SQL statement. The server can analyze the plurality of SQL sentences one by one. The specific process of the server parsing the SQL statement may include:
step 1, determining an instruction type corresponding to each database control instruction, wherein the instruction types at least comprise addition, deletion, update and query.
In this step, the server may run an SQL statement analysis program. The SQL statement analysis program is used for analyzing one SQL statement. When the server receives the operation instruction called by the single interface, the SQL statement analysis program is initialized to be in an INIT state, and the SQL statements are processed one by one. The server may first determine the instruction type of each SQL statement. The SQL statement can include four types of addition, deletion, modification and check. Therefore, the instruction type can include four types, namely adding, deleting, updating and inquiring.
And 2, analyzing the database control instructions according to the instruction types of the database control instructions to obtain an instruction dictionary of the database control instructions, wherein the instruction dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value.
In this step, the server may select different parsing manners to process the SQL statement according to different instruction types.
In one implementation, when the command type is new, deleted, or modified, the server may obtain the key-value pair corresponding to the direct value from the SQL statement. The server may sort multiple key value pairs in the instruction dictionary based on these direct values.
For example, as shown in table 1, when the SQL statement is "INSERT intra full info _ info (prod _ code, a22, … …) VALUES ('000001', 'a21', … …), ('000002', 'a22', … …)," the server may directly obtain the data table as fund _ info from the SQL statement. The fields are prod _ code, a22, etc. Wherein field values corresponding to the field prod _ code are "000001" and "000002". The field a22 corresponds to field values "a21" and "a22". The corresponding key-value pair can be as follows:
Figure BDA0004006543240000161
for another example, as shown in table 1, when SQL statements are "delete from train _ log where _ inner _ code = '000001' and ta _sub _code = '000001'" and "delete from train _ log where _ inner _ code = '000002' and ta _sub _code = '000002', the server may directly acquire the data table as trans _ log from the SQL statement. The fields are inner _ code and ta _ sub _ code. Wherein, the field value corresponding to the field inner _ code is "000001" and "000002". The field values corresponding to the field ta _ sub _ code are "000001" and "000002". The corresponding key-value pairs may be as follows:
Figure BDA0004006543240000162
for example, as shown in table 1, when the SQL statement is "update function _ limit set B13= 'B3' where function _ code = '000001' and B12= 'B' ″) and" update function _ limit set B13= 'B3' where function _ limit _ code = '000002' and B12= 'B2' ″, the server may directly acquire the data table as function _ limit from the SQL statement. Fields are B13, fund _ code and B12. The field B13 corresponds to field values "B3" and "B3". The field value corresponding to the field fund _ code is "000001" and "000002". The field B12 corresponds to field values of "B" and "B2". The corresponding key-value pairs may be as follows:
Figure BDA0004006543240000171
in another example, when the instruction type is a query, the server may analyze the SQL statement using a dynamic programming and breadth first algorithm, since the query statement may be a complex statement. The method comprises the following specific steps:
and step 21, analyzing the field names and the field values in the database control instruction to obtain at least one key value pair of the database control instruction.
In this step, the server may first analyze the SQL statement according to the syntax of the SQL statement and obtain a direct value from the SQL statement, which is the same as the above step. The server may determine a key-value pair in the SQL statement from the direct value. For example, as shown in Table 1, the SQL statement of the query may be
Figure BDA0004006543240000172
Figure BDA0004006543240000173
In the above-described complex SQL query statement, there are various cases such as sub-queries and join-related queries. The server first needs to find the nested tables corresponding to all the sub-queries. Second, the server can find field data in the nested tables that each original table has a direct value. For example, the server may determine that there are fields of direct values in the nested table "(select t1.Prod _ code, t2.A22, t2.C13, t3.E12from fund _ info t1left join C1t2on t1.A22= t2.C12left join E1t3on t1.Prod _ code = t3.E11where t1.Prod _ code = '000001') t4". Wherein, the original table is fund _ info. The field is prod _ code. The field value is 000001. The key-value pair may be:
{
"fund_info.prod_code":["000001"]
}
in one implementation, if the server does not obtain the key-value pair from the SQL query statement, the server may end the subsequent analysis of the SQL query statement.
And step 22, analyzing the character string nesting relation corresponding to the database control instruction according to the first preset character and the second preset character, and constructing a tree structure of the database control instruction.
In this step, the server may use a stacking mode to segment the sub-queries in the process of traversing the SQL string one by one. The server may be pushed when encountering a first preset character and pushed when encountering a second preset character. And the server can acquire the character string between the first preset character and the second preset character every time the stack is popped. Wherein the first preset character may be "(". The second preset character may be ")". If the second predetermined character of the string is followed by a separate string and is not an SQL key (and, or, then, etc.), or if the second predetermined character of the string is followed by an "as" key, then the server determines that the string is a nested table. After finding the nested table, the server can analyze the SQL substrings corresponding to the nested table and obtain the data of the field with the direct value in the original table. The server may set each original table as a node. If the corresponding original table node already exists, the server does not add the original table node and uses the existing corresponding original table node. Otherwise, the server adds the original table node. The server may construct these nodes into a tree data structure based on the root nodes of these original table nodes. When all the characters in the SQL statement are popped, the server can generate a root node t0 for the nested table.
For example, in the above SQL statement, the server first encounters 3 first preset characters, and then encounters 1 second preset character for popping. The server may obtain a nested table between the second preset character and a first preset character closest to the second preset character. The nested table may comprise "(select t1.Prod _ code, t2.A22, t2.C13, t3.E12from fund _ info t1left join C1t2on t1.A22= t2.C12left join E1t3on t1.Prod _ code = t3.E1 where t1.Prod _ code = '000001') t4". According to the substrings corresponding to the nested table, the server can determine that the t4 node of the nested table is the root node of the fund _ info, C1 and E1 nodes of the original table. The tree structure constructed by these nodes may be as shown in fig. 3.
The server may continue traversing the SQL statement and encounter the fourth first preset character and push it. Thereafter, the server may encounter a second preset character during the traversal and pop. The server can get the nested table "(select E2.E21, f1.F1 f rom E2t5inner join on f1t6t5.E22= t6. F12) t7". According to the substrings corresponding to the nested table, the server can determine that the node t7 of the nested table is the root node of the node E2 and the node F1 of the original table.
The server may continue traversing the SQL statement and encounter the third second preset character. The server may obtain a nested table between the second first predetermined character and the third second predetermined character previously stacked. The nested table may be "(select t4.Prod _ code, t4.A22, t4.C13, t4.E12, t7.E2 from t4left join t7on t4.Prod _ code = t7. F11) t8". According to the substrings corresponding to the nested table, the server can determine that the node t8 of the nested table is the root node of the nodes t4 and t7 of the original table.
The server may continue traversing the SQL statement and encounter the fourth second preset character. The server may obtain a nesting table between the first preset character and the fourth second preset character previously pushed. The nested table may be "(select t8.Prod _ code, t8.A22, t8.C13, t8.E12, t8.E21, t9.G21from t8left join G1t9on t8.Prod _ code = t9. G11) t10". According to the substrings corresponding to the nested table, the server can determine that the node t10 of the nested table is the root node of the node t8 and the node G1 of the original table.
The SQL statement continues to be traversed and the fifth first string is encountered. The server may stack the fifth first string. Thereafter, the server may encounter the fifth second string. The server may obtain a nested table between the fifth first string and the fifth second string. The nested table can be "(select t11.H12, t12.J11from H1t11
inner join J1t12on t11.H11= t12. J12) t13". According to the substrings corresponding to the nested table, the server can determine that the node t13 of the nested table is the root node of the node H1 and the node J1 of the original table.
The server may continue traversing the SQL statement. And the server does not acquire the next first character string until the traversal of the SQL sentence is finished. The server may add root node t0 as the root node of the original tables t10 and t13. The server completes the construction of the tree structure as shown in fig. 3.
And step 23, determining an instruction dictionary of the database control instruction according to at least one key value pair of the database control instruction and the tree structure of the database control instruction.
In this step, the server may perform indirect association on the SQL query statement according to the key-value pair and the tree structure obtained from the SQL query statement. The server can find the root node corresponding to each data according to the node of each data, and then analyzes the SQL sentence corresponding to the root node. In particular, the server may use a SQL analysis method corresponding to the non-leaf nodes. The analysis method may specifically comprise both down analysis and up analysis. During this analysis, the server may continue to merge nodes that have already been analyzed, recursively analyzing the nodes to continue analysis. Finally, when all the nodes to be analyzed are in a state of terminating the analysis, the server completes the analysis. The server may find the fields in the data pair in the associated fields of other data tables during the analysis.
In one example, the SQL analysis method corresponding to the non-leaf node may include the following four cases:
and 11, if the current node is the first root node and the SQL statement of the nested table corresponding to the current node is the SQL statement corresponding to the non-root node t0, the server analyzes the SQL statement in a mode of firstly analyzing downwards and then analyzing upwards.
And 12, if the current node is not the first root node, the SQL statement of the nested table corresponding to the current node is the SQL statement corresponding to the non-root node t0, and the server finds the current node in an upward analysis manner, the server may analyze the current node in a downward analysis manner and then in an upward analysis manner.
And 13, if the current node is not the first root node, the SQL statement of the nested table corresponding to the current node is the SQL statement corresponding to the non-root node t0, and the server finds the current node in a downward analysis mode, the server can directly analyze in the downward analysis mode.
And 14, if the current node is not the first root node and the SQL statement of the nested table corresponding to the current node is the SQL statement corresponding to the root node t0, the server may directly perform analysis in a downward analysis manner.
In one example, in the downward analysis method, the server only needs to analyze the association relationship between the fields in the data table corresponding to the child nodes of the current node and the fields in the data pair. The server can determine the association relationship between the fields by analyzing the SQL statement corresponding to the current node. The server can also analyze the association fields among the data tables connected by the join according to the SQL statement corresponding to the current node. During this analysis, the server may determine whether the child node is a leaf node or has been marked as analyzed. If the child node is a leaf node or has been marked as analyzed, the server terminates the analysis of the child node. If the child node is not a leaf node and/or is not marked as analyzed and the data table corresponding to the child node contains an associated field, the server continues to analyze the child node. When the analysis is completed, the server will mark the analyzed nodes as analyzed. In this downward analysis process, the server may use a breadth-first algorithm to implement the analysis of the plurality of child nodes of the current node. The server may analyze all children nodes of the current node one by one. Thereafter, the server may continue the analysis for child nodes that satisfy the condition. And for the child nodes which do not meet the conditions, the server terminates the analysis and obtains the result of the field association relationship. The process may specifically include the following three cases:
and 21, if the existing field of the data table is equal to the field to be searched, and the node corresponding to the data table is not a leaf node, the child node needs to continue the analysis.
And 22, if the field in the data table is equal to the field to be searched, and the node corresponding to the data table is a leaf node, the server determines that the field corresponding to the data table is the field to be searched, and the server finishes the analysis of the node.
Case 23, if no field in the data table is equal to the field to be looked up, the server terminates the analysis of the node, whether it is a leaf node or not.
The server may continue to analyze the child node determined to need to continue the analysis, which belongs to the case 21, after the analysis of all the child nodes of the current node is completed.
In one example, in the upward analysis method, the server may analyze whether a data table corresponding to the current node contains a field to be looked up. Thereafter, the server may classify the analysis process into the following three cases according to whether the current node is the root node t 0:
and 31, if the current node is not the root node t0, the server continues to analyze the root node corresponding to the current node.
Case 32, if it is contained and is the root node t0, the server does not analyze the root node corresponding to the current node any more.
Case 33, if not, the server does not analyze the root node corresponding to the current node.
For example, according to the tree structure shown in fig. 3 and the SQL query statement corresponding to the tree structure, the analysis process in this step may specifically include the following procedures:
the server may analyze the nested table "(select t1.Prod _ code, t2.A22, t2.C13, t3.E12from fund _ info t1left join C1t2on t1.A22= t2.C12left join E1t3on t1.Prod _ code = t3.E1 where t1.Prod _ code = '000001') t4" to get the data pairs corresponding to the direct values. The data pair corresponding to the direct value may be:
{
"fund_info.prod_code":["000001"]
}
the server can determine that the original table is fund _ info and the field is prod _ code according to the nested table. The server may determine the node corresponding to the original table fund _ info and the root node corresponding to the node from the tree structure shown in fig. 3. The root node corresponding to the node fund _ info is t4. The SQL statement of the nested table corresponding to the root node t4 may be "(select t1.Prod _ code, t2.A22, t2.C13, t3.E12from A2t1left join C1t2on t1.A22= t2.C12left join E1t3on t1.Prod _ code = t3.E1 where t1.Prod _ code = 'a') t4". The server may determine that the node t4 is the first root node, and the SQL statement of the nested table corresponding to the node t4 is the SQL statement corresponding to the non-t 0 root node. According to the SQL analysis method corresponding to the non-leaf node, the server can determine that the node t4 belongs to the condition 11, and therefore the server performs analysis in a mode of firstly performing downward analysis and then performing upward analysis.
The server may start from node t4 and analyze the child nodes of node t4 downward. The node t4 includes three child nodes of fund _ info, C1, and E1. The server first parses the child node fund _ info. The server may determine from the data pair that the data table, fund _ info, is the data table in which the field, fund _ info. Therefore, the server terminates the analysis of the child node fund _ info section. This case corresponds to case 22. Thereafter, the server may analyze child node C1. According to the SQL statement of the nested table, the server may determine that no field in the data table corresponding to the child node C1 is equal to the prod _ code field of the fund _ info table, and the child node C1 is a leaf node. Thus, the server terminates the analysis of child node C1. This case corresponds to case 23. Finally, the server may analyze the child node E1. According to the SQL statement of the nested table, the server can obtain that the field E11 in the data table E1 is equal to the field prod _ code in the data table fund _ info. Since the child node E1 corresponding to the data table E1 is a leaf node, the server terminates the analysis of the child node E1. This case corresponds to the case E22. The server may mark child nodes fund _ info, C1, E1, and t4 as analyzed. Thereafter, the server may add the field E11 in the data table E1 to the instruction dictionary. The updated instruction dictionary may be:
Figure BDA0004006543240000221
the server may continue to perform the upward analysis after completing the downward analysis of the node t4. In the process of the upward analysis, the server first determines whether the data table corresponding to the current node contains a field to be searched. For example, the current node is node t4. The node t4 includes a field to be searched for, fund _ info. Thereafter, the server determines whether the current node is the root node t0. Since node t4 is not root node t0, the server continues to analyze the root node corresponding to the current node. This case corresponds to case 31. The root node corresponding to the node t4 is a node t8. The server continues to analyze node t8. The server may continue to determine which of the SQL analysis methods that the node t8 belongs to. First, the server may determine that the node t8 is not the first root node according to the tree structure shown in fig. 3, and the SQL statement of the nested table corresponding to the node t8 is the SQL statement corresponding to the non-root node t0. Secondly, according to the foregoing analysis, the server can determine that the node t8 is found by the analysis of the upward analysis. This case corresponds to case 12. The server may perform the analysis by first analyzing downward and then analyzing upward. The SQL statement of the nested table corresponding to the node t8 may be "(select t4.Prod _ code, t4.A22, t4.C13, t4.E12, t7.E21from t4left join t7on t4.Prod _ code = t7. F11) t8".
The server may start from the node t8 and analyze the child nodes of the node t8 downward. The children of the node t8 may include t4 and t7. The server first analyzes the child node t4. Since this child node t4 has already been analyzed, the server terminates the downward analysis of the child node t4. Thereafter, the server may analyze the child node t7. According to the SQL statement corresponding to the nested table of the child node t7, the server may determine that the field F11 in the child node t7 is equal to the field prod _ code in the data table fund _ info. And since the child node t7 is not a leaf node, the server needs to continue analyzing the child node t7 according to the condition 21.
The server may record the analysis requirements that need to continue analyzing child node t7. Meanwhile, the server may start from the node t8 and analyze upwards. In this upward analysis process, the server first determines whether or not the field to be searched for is contained in the node t8, fund _ info. Prod code since this field is already contained in the child node t4 of the node t8. Thus, the server can determine that the field to be searched for is contained in the node t8. The server may continue to determine whether the node t8 is the root node t0. Since this node t8 is not the root node t0, the server may continue to analyze the root node corresponding to the node t8, corresponding to case 31. The root node corresponding to the node t8 is a node t10.
Before continuing to analyze the node t10, the server may mark the already analyzed nodes t8, t7 as analyzed. In addition, the server needs to analyze the node t7 according to the previous record.
The SQL statement for the nested table corresponding to the node t7 node may be "(select E2.E21, f1.F1 from E2t5inner join on f1t 6. E22= t6. F12) t7". The server may determine that the node t7 is not the first root node according to the tree structure shown in fig. 3, and the SQL statement of the nested table corresponding to the node t7 is the SQL statement corresponding to the non-root node t0. From the foregoing analysis, the server can also determine that the node t7 is found by the downward analysis. This case corresponds to case 13. The server can directly analyze in a downward analysis mode. The server first analyzes child node E2. According to the SQL statement of the nested table corresponding to the node t7, the server may determine that no field in the data table E2 corresponding to the node E2 is equal to the field prod _ code in the data table fund _ info. And child node E2 is a leaf node. This case corresponds to case 23, the server terminates the analysis of child node E2. The server may continue to analyze child node F1. According to the SQL statement of the nested table corresponding to the node t7, the server may determine that the field F11 in the data table F1 is equal to the field prod _ code in the data table fund _ info. And since child node F1 is a leaf node. This case corresponds to case 22. The server may determine that field F11 in the data table F1 is the field to be sought. The server may terminate the analysis of child node F1. The server may update the instruction dictionary. The updated instruction dictionary is:
Figure BDA0004006543240000231
the server may mark the nodes t7, E2, F1 as analyzed after completing the downward analysis of the node t7. The server may continue to analyze node t10. Since the node t10 is not the first root node, the node t10 is not the root node t0, and the node t10 is found by the upward analysis, the node t10 belongs to the case 12. The server may perform the analysis by first analyzing downward and then analyzing upward. The server may obtain the SQL statement of the nested table corresponding to the node t10 as "(select t8.Prod _ code, t8.A22, t8.C13, t8.E12, t8.E21, t9.G21from t8left join G1t9on t8.Prod _ code = t9. G11) t10".
When the server analyzes the node t10 downwards, it may be determined that the node t10 includes two child nodes t8 and G1. The server first analyzes the child node t8. Since this child node t8 has already been analyzed, the server terminates the downward analysis of the child node t8. Thereafter, the server may analyze the child node G1. From the above SQL statement, the server may determine that the field G11 in the data table G1 is equal to the field prod _ code in the data table fund _ info. And since the node G1 is a leaf node. This case corresponds to case 22. The server may determine that field G11 in the data table G1 is the field to be sought. The server may terminate the analysis of child node G1. The server may update the instruction dictionary. The updated instruction dictionary is:
Figure BDA0004006543240000241
the server may continue to perform the upward analysis after completing the downward analysis of the node t4. Since this node t10 includes a field to be searched for and the node t10 is not the root node t0, it corresponds to the case 31. The server obtains the root node t0 of the node t10. The server continues to analyze the node t0. Since the node t0 is not the first root node, and the node t0 is the root node t0, this corresponds to the case 14. The server directly adopts a downward analysis mode to analyze. The SQL statement of the nested table corresponding to the node t0 is "(select t10.Prod _ code, t10.A22, t10.C13, t10.E12, t10.E21, t10.G21, t13.H12from t10left join t13on t10.Prod _ code = t13. J11) t0".
The server may start from node t0 and analyze node t0 downward. The child nodes of the node t0 may include t10 and t13. The server first analyzes the child node t10. Since this child node t10 has already been analyzed, the server terminates the downward analysis of the child node t10. Thereafter, the server may analyze the child node t13. From the SQL statement of the nested table corresponding to node t0, the server can determine that field J11 in data table t13 is equal to field prod _ code in data table fund _ info. And since t13 is not a leaf node, it corresponds to case 21. The server needs to continue analyzing the t13 node. The SQL statement for the nested table corresponding to this t13 node is "(select t11.H12, t12. J1from H1t11inner join J1t12on t11.H11= t12. J12) t13". Since node t13 is not the first root node and is not root node t0, and this node t13 is found by the downward analysis. Thus, this case corresponds to case 13, and the server can analyze directly in a downward analysis.
The child nodes of the node t13 may include H1 and J1. The server first analyzes child node H1. According to the SQL statement of the nested table, the server can determine that no field of the data table H1 is equal to the field to be searched. And since the node H1 is a leaf node, the server ends the analysis of the node H1 corresponding to the case 23. Second, the server analyzes the J1 node. According to the SQL statement of the nested table, the server can determine that the field J11 in the data table J1 is equal to the field to be searched. And since node J1 is a leaf node, corresponding to case 22, the server may terminate the analysis of node J1. The server may add field J11 in data table J1 to the instruction dictionary. The updated instruction dictionary may be:
Figure BDA0004006543240000251
at this point, the server finishes analyzing all SQL statements in the operation instruction shown in table 1. The server may merge the same field and the same field value in the instruction dictionary to obtain a new instruction dictionary. The instruction dictionary corresponding to the merged operation instruction may include:
Figure BDA0004006543240000252
when the analysis of the operation instruction in a single interface call is completed, the SQL statement parser enters a FINISH state.
The server may repeat the parsing of the parameter entry data and the operation instruction in step S202 to obtain parameter entry data of multiple calls of multiple interfaces and parameter entry dictionaries and instruction dictionaries corresponding to the operation instruction, which have been acquired by the server. According to the parameter entering data and the operation instruction in each interface call shown in table 1, the server may analyze to obtain a parameter entering dictionary and an instruction dictionary shown in table 4.
TABLE 4
Figure BDA0004006543240000253
Figure BDA0004006543240000261
And S203, generating interface execution information of the interface during the calling according to the reference dictionary and the instruction dictionary.
In this embodiment, the server may perform analysis on the reference data and the operation instruction respectively to obtain a reference dictionary and an instruction dictionary, and then sort the instruction dictionary according to the reference dictionary. The server may obtain the interface execution information as shown in table 2 after the arrangement. In this step, the server may complete the operation through the relational analysis program. And after the server completes the analysis of the reference dictionary and the instruction dictionary, the server enters the INIT state by the relation analysis program. As shown in table 2 and table 4, the server may record the interface execution flow and the interface number when the interface call is performed once. Thereafter, the server may determine the reference fields and reference field values in table 2from the reference dictionary in table 4. For example, as shown in the entry dictionary with interface serial number XXXX00001, the server may determine that the entry field is prod _ code and the entry field value is [ "000001", "000002" ].
The server may then filter the data pairs in the instruction dictionary based on the parameter field and the parameter field value. The server may select a data pair having an association relationship with the parameter entry field from the plurality of data pairs in the instruction dictionary shown in table 4, and delete a data pair not related to the parameter entry. The server may use the field names of the data pairs with the association relationship as the association fields in the interface execution information. For example, the field name of the data pair "fund _ info.prod _ code" [ "000001", "000002" ] is the same as the value of the entry field, and the field value is the same as the value of the entry field. Thus, the data pair is retained. A field name and a field value of a data pair "function _ info.A22": a21 and a22 are different from an entry field and an entry field value, and the data pair has no association with the data pair "function _ info.prod _ code": 000001 and 000002". Thus, the data pair is deleted. The field name of the data pair "trans _ log.inner _ code": f "000001", "000002" ] is different from the entry field, but the field value is the same as the entry field value, and the data has an association relationship with the data pair "fund _ info.prod _ code": f "000001", "000002" ] that has been reserved. Thus, the data pair is retained. Correspondingly, the server holds the data pairs "trans _ log.ta _ sub _ code": [ "000001", "000002" ], "fund _ limit.fund _ code": [ "000001", "000002" ], "E1.E11": [ "000001" ], "F1.F11": [ "000001" ], "G1.G11": [ "000001" ], "J1.J11": [ "000001" ]. And the server deletes the data pair "fund _ limit.B12": b "," b2"]. The field names corresponding to the retained data pairs may be as shown in the associated fields of table 2.
According to the data processing method, the server can operate the interface at the background of the system to call the monitoring program. When an interface is called and executed by a tested service system, a program acquires the parameter data requested to be input and the operating instruction correspondingly executed in the process of executing and calling the secondary interface by monitoring the thread execution record when the corresponding interface runs. The server can analyze the parameter entering data and the operation instruction called by the single interface respectively to obtain a parameter entering dictionary and an instruction dictionary. The server can sort the instruction dictionary by the reference dictionary to obtain interface execution information. According to the method and the device, the generation of the associated information of the data table is realized through the analysis of the access data and the operation instruction, the acquisition accuracy of the associated relation of the data table and the associated relation of the field is improved, the associated relation of the data table is accurately acquired, and the processing efficiency and the manufacture number effectiveness of the data are further improved. In addition, in the analysis process of the complex SQL statement, the server can also construct a corresponding tree structure according to the nested table of the complex SQL statement, so as to determine the incidence relation of the fields in the complex SQL statement.
Fig. 4 shows a flowchart of a data processing method according to an embodiment of the present application. Based on the embodiments shown in fig. 1 to fig. 3, taking a server as an execution subject, the process of determining the association relationship between two single fields in step S102 shown in fig. 2 may include the following steps:
s301, a plurality of single fields and the occurrence frequency of each single field are obtained through sorting according to the associated fields in the interface execution information of a plurality of interfaces of the system during multiple calls.
In this embodiment, the server may run a related analysis program. The association analysis program may be preset to start on a daily schedule. When the association analysis program is started, the association analysis program enters an INIT state. The server can sort the associated fields in the interface execution information to obtain a single list. The fields in the single item list are fields present in the plurality of interface execution information. The number of times each single item field is also included in the list of single items. The single-item fields in the single-item list are all fields which appear in the associated fields in the interface execution information and are not repeated. For example, the server may sort the associated fields in table 2from the interface execution information with interface execution serial number XXXX00001 to the interface execution information with interface execution serial number XXXX 01000.
In one example, before the server sorts the single field, the server may determine whether the number of pieces of interface execution information currently acquired is greater than a preset number. If the number is less than the preset number, the server may end the operation of the association analysis program of the current day and cause the association analysis program to enter a FINISH state. Otherwise, if the number is greater than or equal to the preset number, the server may continue to perform the following steps. The preset number may be 1000.
S302, sorting the single fields according to the sequence of the occurrence times from large to small.
In this embodiment, the server may sort the single fields in the single list according to the occurrence frequency of each single field. The sorting rule may be sorting in order of the number of occurrences from large to small.
S303, traversing the sorted single field, and screening out the main field of the service object and the directly associated field corresponding to the main field of each service object from the single field.
In this embodiment, the server may traverse the sorted singleton fields. In the traversal process, the server may determine whether a single field is a main field of the service object after acquiring the single field. When the single field is the main field of the business object, the server can traverse the sorted single field again. The server may determine whether the single field is a directly associated field corresponding to the main field of the service object after each single field is obtained. For ease of understanding, the list of items traversed by the server for the first time may be named the first list of items. The single item field obtained by the server in the first single item list may be the first single item field. The list of singletons traversed by the server a second time may also be named a second list of singletons. The single field obtained by the server in the second single list may be the second single field. The first list of items and the second list of items may be the same list. Alternatively, for ease of understanding, the first list of singletons and the second list of singletons may be considered as two real-time synchronized lists.
In one example, based on the first single list and the second single list, the process of filtering, by the server, the main field of the service object and the directly associated field corresponding to the main field of each service object may specifically include the following steps:
and 31, sequentially acquiring a first single field from the sorted single fields.
In this step, the server may sequentially obtain a first single field from the first single list.
And step 32, when the first single field is not marked as being associated and the first single field is the only primary key in the data table where the first single field is located, determining that the first single field is the main field of the business object.
In this step, the server needs to determine whether the first single-item field is a main field of the service object. The server preferably determines whether the first single-item field is marked as associated. If already noted as associated, the single item field is not a business object primary field. The server returns to step 31 and proceeds to fetch the next first singleton field. Otherwise, if the first single field is not marked as associated, the server further judges whether the first single field is the only primary key in the data table where the first single field is located. If the first singleton field is not the only primary key, the server returns to step 31 and proceeds to fetch the next first singleton field. Otherwise, if the first single-item field is the only primary key, the server determines that the first single-item field is the main field of the business object.
And step 33, acquiring the second single item fields from the sorted single item fields in a reordering mode.
In this step, the server may sequentially obtain a second single field from the second single list. It should be noted that each time the server obtains a first single-item field, the server needs to traverse the second single-item list from the beginning. After the server completes traversing the second list, the server may return to step 31 and continue to obtain the next first list field.
And 34, when the second single-item field is not marked as being associated, calculating an association index of the first single-item field and the second single-item field.
In this step, the server may first determine whether the second single-item field is marked as associated. If the second singleton field is marked as associated, the server returns to step 33 and continues to retrieve the next second singleton field. Otherwise, if the second single item field is not marked as associated, the server may calculate an association indicator for the first single item field and the second single item field.
In one example, the association index may specifically include three indexes, namely, a support index, a confidence index and a lift index. The specific process of calculating the correlation index by the server may include:
step 341, determining a support degree index according to the ratio of the number of times of the first single field and the second single field appearing simultaneously in the interface execution information to the total number of the interface execution information.
In this step, the server may obtain the total number num of data samples (AllSamples). The total number of the data samples is the total number of the interface execution information obtained by the server. The server can also count the number (XY) of times that the first single-item field X and the second single-item field Y appear simultaneously in the interface execution information. The first single field and the second single field appear at the same time, that is, in the associated field of the interface execution information corresponding to one interface call, the first single field and the second single field exist at the same time. The calculation formula of the support degree index between the first single-item field and the second single-item field may be:
Support(X,Y)=P(XY)=number(XY)/num(AllSamples)
for example, the server may obtain 1000 pieces of interface execution information, and perform the calculation of the support degree index by using the 1000 pieces of interface execution information as a sample. Assuming that the first one-entry field is fund _ info.prod _ code and the second one-entry field is trans _ log.inner _ code. Among the samples, there are 100 samples including the field fund _ info.prod _ code and 12 samples including the field trans _ log.inner _ code. And wherein there are 10 samples containing both the field fund _ info.prod _ code and the field trans _ log.inner _ code. According to the above formula, the server can calculate the supported index to be 1%.
And 342, determining a confidence index according to the ratio of the support index to the number of times of the second single-item field in the total number of interface execution information.
In this step, the calculation formula of the confidence indexes of the first single-term field X and the second single-term field Y is:
Confidence(X<-Y)=P(X|Y)=P(XY)/P(Y)
wherein, P (XY) is a support index. P (Y) is the support index of the second single-item field. The support index of the second single field is the ratio of the number of the interface execution information with the second single field to the total number of the interface execution information. For example, the server may calculate the confidence indicator to be 83.3% according to the above example and formula.
And 343, determining a promotion index according to the ratio of the confidence index to the number of times of occurrence of the first single field in the total number of interface execution information.
In this step, the calculation formula of the lifting degree index of the first single-item field X and the second single-item field Y is:
Lift(X<-Y)=P(X|Y)/P(X)=Confidence(X<-Y)/P(X)
wherein Confidence (X < -Y) is a Confidence index. P (X) is a support index of the first single-item field. The support index of the first single field is the ratio of the number of the interface execution information with the first single field to the total number of the interface execution information. For example, according to the above example and formula, the server may calculate that the lifting degree index is 8.3.
And step 35, when the association index is larger than a first preset threshold value, determining that the second single field is a direct association field of the first single field.
In this step, the server may obtain a first preset threshold. The server may compare the calculated correlation indicator with the first preset threshold. The first preset threshold may include a support threshold, a confidence threshold and a lift threshold. The three thresholds may be set by a technician. For example, the support threshold may be 0.5%, the confidence threshold may be 60%, and the boost threshold may be 1. And if the association index is larger than a first preset threshold value, the server determines that the second single-item field is a directly associated field of the first single-item field. Otherwise, if the association index is less than or equal to the first preset threshold, the server determines that the second single-item field is not a directly associated field of the first single-item field. After the judgment is completed, the server may return to step 3 to obtain the next second single-item field, until the second single-item list is traversed, the server may return to step 1 to obtain the next first single-item field.
When the second single-item field is a directly related field of the first single-item field, the server may record the first single-item field and the second single-item field machine related indexes, of which the related indexes are greater than the first preset threshold, in the field related table as described in table 5. The main field of the business object is a first single field. The direct association field is a direct association field corresponding to the main field of the service object. For example, the server may determine, according to the support degree indicator, the confidence indicator, the promotion degree indicator, and the first preset threshold calculated in the above example, that the support degree indicator is 1% >0.5%, the confidence indicator is 83.3% >60%, and the promotion degree indicator is 8.3> -1. Therefore, the association index of the first single-item field and the second single-item field is greater than the first preset threshold value. Namely, an association relationship exists between the first single item field and the second single item field. The server may record the first single field fund _ info.prod _ code and the second single field trans _ log.inner _ code in table 5. For another example, when the server calculates that the association index between the first single field fund _ info, prod _ code and the second single field trans _ log.ta _ sub _ code includes that the support degree is 1.1%, the confidence degree is 90%, and the lift degree is 9, the server may determine that an association relationship exists between the first single field fund _ info, prod _ code and the second single field trans _ log.ta _ sub _ code after comparing the association index with the first preset threshold. Thus, the server may record the first one-item field fund _ info.prod _ code and the second one-item field trans _ log.ta _ sub _ code in table 5. For another example, when the server calculates that the association index between the first single field fund _ info.prod _ code and the second single field e1.e11 includes that the support degree is 1.1%, the confidence degree is 90%, and the promotion degree is 0.6, the server may determine that the promotion degree index is smaller than the first preset threshold after comparing the association index with the first preset threshold. The server may determine that there is no association between the first one-item field fund _ info.prod _ code and the second one-item field e1.e 11. Therefore, the server does not need to record the first one field fund _ info.prod _ code and the second one field e1.e11 in table 5.
TABLE 5
Figure BDA0004006543240000311
S304, marking the main field and the direct association field of the business object as associated.
In this embodiment, the server may mark the main field and the directly associated field of the business object recorded in table 5 as associated. It should be noted that the execution of this step and the above step S303 are performed in an interspersed manner. That is, in the server traversal process, when it is determined that an association relationship exists between the first single item field and the second single item field, the server marks the first single item field and the second single item field as associated.
In one example, when a first list of items and a second list of items are included, consistency is maintained due to the first list of items and the second list of items. Thus, when a first single item field in the first single item list is marked as associated, the first single item field in the second single item list is also marked as associated. Correspondingly, when the second single item field in the second single item list is marked as associated, the second single item field in the first single item list is also marked as associated.
S305, traversing the single-item fields which are not marked to be associated, and screening out indirect associated fields corresponding to the main fields of the business objects.
In this embodiment, the server may traverse the single item fields in the single item list that are not marked as associated. The server can calculate the association index between the single item field and the single item field marked as associated in the single item list after acquiring the single item field. The server can determine whether an association relationship exists between the single field which is not marked as associated and the single field which is marked as associated according to the association index and a preset threshold. When there is an association relationship between the two single fields, the server may determine, according to the single field marked as associated, a business object main field corresponding to the single field marked as associated. The server may determine that the single-item field that is not marked as associated is an indirectly associated field of the business object's main field.
In one example, the process of determining the indirect association field may specifically include the following steps:
and step 51, generating an associated list and an unassociated list according to the mark of each single field.
In this step, in order to facilitate subsequent traversal, the server may sort the single item list into an associated list and an unassociated list. And the associated list comprises single item fields which are marked as associated. The unassociated list includes a single entry field that is not marked as being associated. The individual item fields in the associated list and unassociated list are ordered the same as the individual item list. The server may sort the individual single-item fields according to their number of occurrences.
Step 52, the third single-entry field is sequentially retrieved from the unassociated list.
In this step, the server may sequentially obtain a third single-entry field from the unassociated list.
And step 53, calculating the association indexes of the third single-item fields and each fourth single-item field in the associated list one by one.
In this step, the server may sequentially obtain a fourth single field from the associated list. The server may calculate an association indicator for the third singleton field and the fourth singleton field. The correlation index may be calculated as shown in step 34. The correlation index may include a support index, a confidence index, and a lift index.
And step 54, determining that the third single field is associated with the fourth single field when the association index is greater than a second preset threshold.
In this step, the server may compare the correlation index with a second preset threshold. The second preset threshold may be the same as the first preset threshold in step 35. Alternatively, the second preset threshold may be different from the first preset threshold. For example, in the second preset threshold, the support threshold may be 0.6%, the confidence threshold may be 65%, and the lift threshold may be 1.5. When all three of the associated indexes are greater than a second preset threshold, the server may determine that the third single-item field is associated with the fourth single-item field.
In one example, the server may further delete the third associated field from the unassociated list and add to the associated list upon determining that an association exists between the third individual field and the fourth individual field. When the third associated field is added to the associated list, the individual fields in the associated list are still sorted according to their number of occurrences.
And step 55, determining that the third single field is an indirect association field of the business object main field corresponding to the fourth single field.
In this step, since the fourth single-entry field is a single-entry field in the associated list, the server may further determine, according to table 5, a main field of the service object corresponding to the fourth single-entry field. The fourth single field may be any one of a main field, a direct association field, and an indirect association field of the service object. The server may determine that the third singleton field is an indirectly associated field of the business object's main field. The server can write the third single-item field and the fourth single-item field into the field association table. It should be noted that, the third and fourth single fields may be newly added lines. And the fourth single-item field is an indirect association field transfer field between the third single-item field and the main field of the business object.
After the server completes the determination of the third single-item field and the fourth single-item field, the server may return to step 53 and obtain the next fourth single-item field from the associated list. Until the associated list is traversed, the server may return to step 51 and obtain the next third single-item field from the unassociated list.
For example, the third one field may be a fund _ limit. There are 20 samples in the sample containing the field fund _ limit. There are 12 samples of the sample containing the field trans log inner code. There are 8 samples of the sample that contain both the field fund _ limit. According to the above formula, the server can calculate the supported index to be 0.8%. The confidence index is 66.7%, and the promotion index is 3.3. The preset threshold may include a support threshold of 0.5%, a confidence threshold of 60%, and a boost threshold of 1. The server may determine, according to step 54, that the support indicator is 0.8% >0.5%, the confidence indicator is 66.7% >60%, and the promotion indicator is 3.3 >. The server may determine that there is an association between the third one-term field fund _ limit. The server may determine that the service object main field corresponding to the fourth single field trans _ log.inner _ code is fund _ info.prod _ code according to table 5. The server may add the third single field, function _ limit.function _ code, the fourth single field, trans _ log.inner _ code, and the service object main field, function _ info.prod _ code, to table 5. Table 5 after this addition can be as follows:
TABLE 5
Figure BDA0004006543240000331
S306, determining field association information and data table association information of each data table according to the main field, the direct association field and the indirect association field of the business object.
In this embodiment, the server may analyze the field association table shown in table 5 obtained in the above steps, and determine field association information between fields of each data table and data table association information between each data table. In the analysis process, the server may analyze the business object main field, the direct association field and the indirect association field of each row of data in the field association table. The server can determine the main field and the data table where the main field is located according to the main field of the service object. The data table may be a business object master table. The server can also determine the association field corresponding to the main field and the association table corresponding to the association field according to the direct association field and the indirect association field. The server can write the main table, the main field, the association table and the association field of the business object into the association information relation table. The association information relationship table may be as shown in table 5. The association information relation table may further include a service object main table, a main field, an association table, and an association index between the service object main field and a direct association field or between the service object main field and an indirect association field corresponding to the association field.
In one example, to reduce data duplication, the server may generate an index in the association information relationship table according to the business object master table and the association table. One index uniquely corresponds to one row of data in the associated information relationship table. For example, after the server obtains a row of data from table 5, the server may generate an index according to the service object master table and the association table obtained through parsing. The server may determine whether the index already exists. If the index already exists, the server compares the support degree in a row of data corresponding to the already existing index. And if the support degree in a row of data corresponding to the existing index is greater than or equal to the support degree of the data corresponding to the index obtained just after the analysis, the server does not add the main table, the main field, the association table and the association field of the service object obtained through the analysis into the association information relation table. Otherwise, if the support degree of the data corresponding to the existing index in the row is smaller than the support degree of the data corresponding to the index obtained through analysis, the server uses the service object main table, the main field, the association table, the association field and the association index obtained through analysis to replace the data corresponding to the existing index in the row. If the index does not exist, the server can add the main table, the main field, the association table and the association field of the service object obtained by analysis to the association information relation table.
In one example, the server may sort the rows of data in the association information relationship table according to the support degree index.
The association information relationship table may be as shown in table 6.
TABLE 6
Figure BDA0004006543240000341
According to the data processing method, the server can sort the associated fields in the interface execution information of the plurality of interfaces of the system when the plurality of interfaces are called for multiple times to obtain the single list. The single item list includes a plurality of single item fields and the number of occurrences of each single item field. The server can sort the single-item fields in the single-item list according to the sequence of the occurrence times from large to small. The server may traverse the sorted list of items and screen out the main fields of the business objects and the directly associated fields corresponding to the main fields of each business object from the list of items. The server may mark the business object main field and the directly associated field as associated. The server may also re-traverse the single item fields in the single item list that are not marked as associated. The server can screen the indirect association fields corresponding to the main fields of the business objects from the single fields which are not marked as being associated. The server can determine the field association information and the data table association information of each data table according to the main field, the direct association field and the indirect association field of the service object. According to the method and the device, through analysis of the access data and the operation instruction, generation of the associated information of the data table is achieved, the acquisition accuracy of the associated relation of the data table and the associated relation of the field is improved, the associated relation of the data table is accurately acquired, and the processing efficiency and the number making effectiveness of the data are further improved. In addition, the index is set, repeated data are reduced, and data effectiveness is improved.
On the basis of the above embodiments, fig. 5 is a flowchart illustrating an example of data processing provided in the present embodiment. On the basis of the embodiments shown in fig. 1 to 4, as shown in fig. 5, with a server as an execution subject, this example may include the following steps:
s401, monitoring and acquiring the access data and the corresponding executed operation instruction during single interface calling.
In this example, the server may implement monitoring on each interface in the server by adding a monitoring process to the system. When the monitoring process monitors that the interface is called for a single time, the server can acquire the entry data and the corresponding input operation instruction of the interface in the single call.
S402, storing the parameter data and the corresponding operation instruction during single interface calling.
In this example, the server may store the entry data and the operation instruction of the interface call in the database. Alternatively, the server may store the entry data and the operation instruction in a row of a data table in the database. Optionally, when storing, the server may use the interface execution serial number called by the secondary interface as the primary key of the line of data. Optionally, the server may also store the information such as the interface number together in the database.
And S403, analyzing the input parameter data called by the single interface to obtain the input parameter dictionary.
S404, analyzing the operation instruction called by the single interface to obtain an instruction dictionary.
In this example, the implementation manners of step S403 and step S404 are similar to the implementation manner of step S202 in fig. 2, and are not described herein again.
S405, analyzing the single interface call reference dictionary and the instruction dictionary to obtain interface execution information of the single interface call.
In this example, the implementation manner of step S405 is similar to the implementation manner of step S203 in fig. 2, and this embodiment is not described here again. The interface execution information may be as shown in table 2. Wherein each row corresponds to interface execution information for a single interface call. A plurality of interface execution information for a plurality of interface calls may be included in the table 2.
S406, analyzing data of the execution information of the plurality of interfaces to obtain field association information of the data table.
And S407, analyzing the field association information to obtain the data table association information.
In this example, the implementation manners of steps S406 and S407 are similar to the implementation manner of step S102 in fig. 1, and this embodiment is not described herein again. Wherein, the field association information and the data table association information may be as shown in table 3. The server may store the field association information and the data table association information in the server. The server can also periodically update the field association information and the data table association information according to the monitored and acquired entry data and operation instructions called by the word interface.
S408, testing the number.
In this example, the server may obtain an instruction that the user needs to generate the test number. This test number is typically used for traffic testing.
And S409, selecting a business object main table.
In this example, after determining that the user has a requirement for the number of artifacts, the server needs to select the service object master table of the target service according to an instruction for generating a test number of the user. The target service is the service tested at this time. Each service may correspond to a master table of service objects.
S410, obtaining the latest data of the business object main table and the business object association table.
In this example, the server may acquire the field association information and the data table association information stored in S407. The server can obtain the latest association table corresponding to the service master table according to the data table association information.
S411, selecting a manufacturing mode.
In this example, the server may generate an instruction for testing the number according to the user requirement, and determine the number mode. The number making mode can comprise cloning and direct generation. When the user selects the clone mode, the server performs steps S413, S414, and S415. Otherwise, when the user selects the direct generation, the server performs steps S417 and S418.
S412, cloning.
In this example, when the mode of generating the test number indicated by the instruction that the user needs to generate the test number is clone, the following steps may be included.
And S413, searching data of the corresponding business object main table and the corresponding business object association table in the current test environment.
In this example, the server may determine data corresponding to the target service according to the service object master table and the association table. The data may include a plurality of fields from the business object master table and the association table.
And S414, generating data of a corresponding table according to a certain rule by taking the searched data as a template.
In this example, the server may generate the data template according to a field corresponding to the data of the target service. The template may include information of all fields corresponding to the data, field rules of each field, at least one reference value of each field, and the like.
And S416, directly generating.
In this example, when the generation manner of the test number indicated by the instruction that the user needs to generate the test number is direct generation, the following steps may be included.
S417, selecting the main table and the association table of the service object requiring the number.
In this example, after determining the target service requiring the number, the server may determine the corresponding service object master table. Then, the server may determine the association table corresponding to the business object master table according to the business object master table. The server can also select the association table needed to be used from the association tables according to the requirement of the manufacture number, and abandon the association tables which have association but are not needed in the manufacture number.
And S418, generating data of the corresponding table according to a certain plan.
In this example, the server may select fields in the business object main table and the association table according to a certain rule. These fields constitute the data that the application needs to use.
And S415, executing the manufacture number.
In this example, when the number making method is clone, the server may generate a new field value according to the field rule of each field. The server may compose these new field values into new data. The new data is the data obtained by the manufacture number. When the number generation mode is direct generation, the server can directly generate each field value in the data according to the field included in the data.
And S419, finishing the manufacture.
In the data processing method provided by the application, the server can obtain the field associated information and the data table associated information of the data table by analyzing the access data and the operation instruction obtained by multiple times of interface calling. The server can also obtain an association table corresponding to the service object main table according to the data table association information when determining that the number of the target service needs to be created, and further generate a data template. The server may complete the numbering according to the data template. By the method, the server can acquire the latest association table of the main table of the service object when determining that the number of the target service needs to be manufactured, and the effectiveness of the number manufacturing is improved.
On the basis of the above embodiment, as shown in fig. 6, an algorithm flowchart of data table association analysis provided in this embodiment is shown. On the basis of the embodiments shown in fig. 1 to 5, as shown in fig. 6, with a server as an execution subject, this example may include the following steps:
s501, starting.
And S502, analyzing the sample data to obtain a single list.
In this example, the sample data is interface execution information obtained by the server according to a single interface call analysis. The server can analyze the interface execution information obtained for multiple times and called by a single interface to obtain a single list. The interface execution information may be as shown in table 2. The server may retrieve the fields in the association fields of table 2. Each field may be a single entry field. The server may compose the non-duplicate single-entry fields of table 2 into a single-entry list.
S503, sorting the single item fields in the single item list from large to small according to the occurrence times.
In this example, the server may also count the number of occurrences of each single-item field in table 2 in the process of acquiring the single-item field. That is, the single entry list may include the single entry fields and the number of times each single entry field occurs. The server may sort each individual field in order of arrival from the time of arrival according to the number of occurrences of the individual field.
S504, traversing the first single-item list to obtain 1 first single-item field.
In this example, the server may generate the first list of items and the second list of items from the list of items. For ease of understanding, the list of items traversed by the server for the first time may be named the first list of items. The single item field obtained by the server in the first single item list may be the first single item field. The list of singletons traversed by the server a second time may also be named a second list of singletons. The single field obtained by the server in the second single list may be the second single field. The first list of items and the second list of items may be the same list. Alternatively, for ease of understanding, the first list of singletons and the second list of singletons may be considered as two real-time synchronized lists. The server may traverse the first list of entries to obtain 1 first entry field.
And S505, judging that the first single field is in an unassociated state.
In this example, the server may determine whether the first single-entry field is in an unassociated state. If so, the server continues to perform step S506 and makes other determinations. If not, the server executes step S514, determines whether the traversal is completed, and returns to step S504 when the traversal is not completed, to obtain the next first single-item field.
S506, judging that the first single-item field is the only primary key in the data table where the first single-item field is located.
In this example, the server needs to continue to determine whether the first single-entry field is the only primary key in the data table where the first single-entry field is located when determining that the first single-entry field is in the unassociated state. If yes, the server continues to execute step S507, and determines that the first single-item field is the business object main field. If not, the server executes step S514 to determine whether the traversal is completed, and returns to step S504 to obtain the next first singleton field if the traversal is not completed.
S507, setting the first single field as a main field of the business object.
In this example, the server may set the first single-entry field as the business object main field when determining that the first single-entry field is in the unassociated state and whether the first single-entry field is the only primary key in the data table where the first single-entry field is located.
And S508, traversing the second single-item list to obtain 1 second single-item field.
In this example, the server may traverse the second list of items and obtain a second item field from the second list of items.
S509, judging that the second single field is not the main field of the business object and is in an unassociated state.
In this example, the server may determine whether the second single-entry field is a main field of the business object, and whether the second single-entry field is in an unassociated state. If the second single field is not the main field of the business object and is in an unassociated state, the server continues to execute step S510, and performs subsequent judgment on the second single field. Otherwise, if the second single field is already marked as the main field of the service object in step S507, or the second single field is already in an associated state, the server performs step S513 to determine whether the traversal is completed, and returns to step S508 when the traversal is not completed, and acquires the next second single field.
S510, judging the association relationship between the main field of the business object and the second single field.
In this example, the server may determine whether there is an association relationship between the second single field and the service object main field determined in step S507 when determining that the second single field is not the main field of the service object and is in an unassociated state. If yes, the server will continue to execute step S511 and record the association relationship between the second single-item field and the main field of the service object. If not, the server executes step S513 to determine whether the traversal is completed, and returns to step S508 when the traversal is not completed, and acquires the next second singleton field.
S511, the main field of the service object and the second single-item field relation data are added into a field relation table.
In this example, the server may record an association relationship between the main field of the business object and the second single-item field in the field relationship table when determining that the association relationship exists between the main field of the business object and the second single-item field. The field relationship table may be as shown in table 5.
S512, setting the main field of the business object and the second single-item field to be in an associated state in a single-item list.
In this example, after completing the recording of the association relationship between the main field of the service object and the second single-item field, the server may set the main field of the service object and the second single-item field to be in an associated state in the first single-item list and the second single-item list, respectively.
And S513, judging that the second single item list is traversed completely.
In this example, the server may determine whether the second list of singletons completes the traversal. If not, indicating that the traversal has not been completed, the server may return to step S508 and continue to acquire the next second singleton field. If yes, it indicates that the traversal is completed, the server may determine that the first singleton field has completed the determination, and the server may return to step S504 and continue to acquire the next first singleton field for determination.
And S514, judging that the first single list is traversed completely.
In this example, the server may determine whether the first list of singletons completes the traversal. If not, it indicates that the traversal is not completed, the server may return to step S504 and continue to acquire the next first singleton field. If yes, the traversal is completed, and the server finishes pairwise judgment between each single field in the single list. At this time, the server may continue to perform step S515 to supplement the association relationship.
And S515, splitting the single item list into an associated list and an unassociated list.
In this example, the server may split the single item list into two lists, an associated list and an unassociated list, according to the association status of each single item field in the single item list.
S516, traversing the unassociated list to obtain 1 unassociated field.
In this example, the server may traverse the unassociated list to obtain an unassociated field.
S517, traversing the associated list to obtain 1 associated field.
In this example, the server may traverse the associated list to obtain an associated field.
S518, judging the association relationship between the unassociated field and the associated field.
In this example, the server may determine whether an association exists between the unassociated field and the associated field. If so, the server may proceed to S519, completing the recording of the association. If not, the server executes step S525 to determine whether the traversal of the associated list is completed. And returns to step S517 when the traversal of the associated list is not completed, and acquires the next associated field. Alternatively, returning to step S516 when the already associated list has completed the traversal, the next unassociated field is acquired.
S519, searching the main field of the business object corresponding to the associated field.
In this example, the server may search for the main field of the business object corresponding to the associated field.
S520, adding the related fields and the relation data of the main fields and the unassociated fields of the corresponding business objects into a field relation table.
In this example, the server may add the associated field and the relationship data of the main field and the unassociated field of the corresponding business object to the field relationship table. The field relationship table may be as shown in table 5.
And S521, adding the unassociated fields into the associated list.
In this example, after determining that there is an association relationship between the unassociated field and the associated field and completing the recording of the association relationship, the server may delete the unassociated field from the unassociated list and add the unassociated field to the associated list. The position of the unassociated field in the associated list may be determined according to the order of the number of occurrences of the unassociated field and the number of occurrences of each associated field in the associated list.
S522, the traversal of the associated list is terminated.
In this example, after determining an association relationship of the unassociated field, the server may determine the field association information and the data table association information of each data table according to the association relationship. Therefore, the server may determine that the determination that the unassociated field is complete after adding the unassociated field to the associated list. Thus, the server may terminate the traversal of the associated list.
S523, the unassociated list is judged to be traversed and completed.
In this example, the server may determine whether the unassociated list completes the traversal. If yes, the judgment of the incidence relation is finished. If not, the server needs to return to step S516 and obtain the next unassociated field.
And S524, judging that the traversal of the associated list is completed.
In this example, the server may determine whether the traversal of the associated list is complete. If so, the server returns to step S516 to obtain the next unassociated field. If not, the server returns to step S517 to obtain the next associated field.
And S525, ending.
According to the data processing method, the server can obtain the main field of the service object, the direct association relationship between every two fields and the indirect association relationship between every two fields through two traversal screening in the single item list. According to the method and the device, the generation of the associated information of the data table is realized through the analysis of the association relationship twice, and the acquisition accuracy of the association relationship of the data table and the field association relationship is improved. The data table association relation is accurately obtained, and the data processing efficiency and the data making effectiveness are further improved.
Fig. 7 shows a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, and as shown in fig. 7, a data processing apparatus 10 according to the present embodiment is used for implementing an operation corresponding to a server in any of the method embodiments described above, where the data processing apparatus 10 according to the present embodiment includes:
the obtaining module 11 is configured to obtain and analyze access data and an operation instruction of multiple interfaces of the system during multiple calls, and obtain interface execution information of each interface during each call, where the interface execution information at least includes an interface number, an access field value, and an association field.
And the processing module 12 is configured to analyze, according to the interface execution information of the multiple interfaces of the system during multiple calls, field association information and data table association information of each data table in the database of the system. And determining a data template corresponding to the target service according to the data table association information and the target service. And generating data to be generated of the target according to the data template and the field association information, and completing the number making of the target service by using the data to be generated.
In one example, the obtaining module 11 is specifically configured to:
and acquiring the parameter entering data and the operation instruction of the interface when the interface is called through the monitoring interface.
And analyzing the parameter to obtain a parameter dictionary. And analyzing the operation instruction to obtain an instruction dictionary.
And generating interface execution information of the interface during the calling according to the reference dictionary and the instruction dictionary.
In one example, the obtaining module 11 is specifically configured to:
analyzing the parameter data to obtain a parameter dictionary of the parameter data, wherein the parameter dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value.
And if the data type of the field value of the key value pair in the reference dictionary is different from the preset type, deleting the key value pair.
If the field values of a plurality of key-value pairs in the reference dictionary are the same, deleting a plurality of field values.
In one example, the operation instruction includes at least one database control instruction, and the obtaining module 11 is specifically configured to:
and determining the instruction type corresponding to each database control instruction, wherein the instruction types at least comprise addition, deletion, updating and query.
And analyzing the database control instructions according to the instruction types of the database control instructions to obtain an instruction dictionary of the database control instructions, wherein the instruction dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value.
In one example, when the database control instruction is a query operation, the obtaining module 11 is specifically configured to:
and analyzing the field name and the field value in the database control instruction to obtain at least one key value pair of the database control instruction.
And analyzing the character string nesting relation corresponding to the database control instruction according to the first preset character and the second preset character, and constructing a tree structure of the database control instruction.
And determining an instruction dictionary of the database control instruction according to at least one key value pair of the database control instruction and the tree structure of the database control instruction.
In one example, the processing module 12 is specifically configured to:
and sorting to obtain a plurality of single fields and the occurrence times of each single field according to the associated fields in the interface execution information of a plurality of interfaces of the system during multiple calls.
And sorting the single-item fields according to the sequence of the occurrence times from large to small.
And traversing the sorted single field, and screening out the main field of the service object and the directly associated field corresponding to the main field of each service object from the single field.
And marking the main field and the direct association field of the business object as associated.
And traversing the single field which is not marked as the associated field, and screening out the indirect associated field corresponding to the main field of the business object.
And determining field association information and data table association information of each data table according to the main field, the direct association field and the indirect association field of the service object.
In one example, the processing module 12 is specifically configured to:
and sequentially acquiring the first single item field from the sorted single item fields.
And when the first single-item field is not marked as being associated and the first single-item field is the only primary key in the data table where the first single-item field is located, determining that the first single-item field is the main field of the business object.
And sequentially acquiring the second single item field from the sorted single item fields again.
When the second single item field is not marked as being associated, an association index of the first single item field and the second single item field is calculated.
And when the correlation index is larger than a first preset threshold value, determining that the second single-item field is a direct correlation field of the first single-item field.
In one example, the association index includes a support degree index, a confidence degree index, and a lift degree index, and the processing module 12 is specifically configured to:
and determining a support degree index according to the ratio of the number of times of the simultaneous occurrence of the first single field and the second single field in the interface execution information to the total number of the interface execution information.
And determining a confidence index according to the ratio of the support index to the number of times of the second single field in the total number of interface execution information.
And determining a promotion index according to the ratio of the confidence index to the number of times of the first single field in the total number of interface execution information.
In one example, the processing module 12 is specifically configured to:
and generating an associated list and an unassociated list according to the mark of each single field.
The third single-entry field is sequentially fetched from the unassociated list.
And calculating the association index of the third single-item field and each fourth single-item field in the associated list one by one.
And when the association index is larger than a second preset threshold value, determining that the third single field is associated with the fourth single field.
And determining the third single field as the indirect association field of the business object main field corresponding to the fourth single field.
In one example, the processing module 12 is further configured to:
and accumulating the index information consisting of the interface number, the parameter field and the parameter field value in the interface execution information until the number of the non-repeated index information reaches the preset number.
In one example, the processing module 12 is specifically configured to:
and determining a corresponding business object main table according to the target business.
And determining the association table with association according to the data table association information and the business object main table.
And searching the main table and the association table of the business object to obtain a data template.
In one example, the processing module 12 is specifically configured to:
and resetting the associated fields in the data template to the same value according to the field association information.
The values of the other fields in the data template are reset according to the field rules.
The data processing apparatus 10 provided in the embodiment of the present application may implement the method embodiment, and for details of implementation principles and technical effects, reference may be made to the method embodiment, which is not described herein again.
Fig. 8 shows a hardware structure diagram of a server provided in an embodiment of the present application. As shown in fig. 8, the server 20 is configured to implement the operation corresponding to the server in any of the above method embodiments, where the server 20 of this embodiment may include: memory 21, processor 22.
A memory 21 for storing a computer program. The Memory 21 may include a Random Access Memory (RAM), a Non-Volatile Memory (NVM), at least one disk Memory, a usb flash drive, a removable hard drive, a read-only Memory, a magnetic disk or an optical disk.
And a processor 22 for executing the computer program stored in the memory to implement the data processing method in the above-described embodiments. Reference may be made in particular to the description relating to the method embodiments described above. The Processor 22 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
Alternatively, the memory 21 may be separate or integrated with the processor 22.
When memory 21 is a separate device from processor 22, server 20 may also include bus 23. The bus 23 is used to connect the memory 21 and the processor 22. The bus 23 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The server provided in this embodiment may be used to execute the data processing method, and the implementation manner and the technical effect thereof are similar, and this embodiment is not described herein again.
The present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used for implementing the methods provided by the above-mentioned various embodiments when being executed by a processor.
The computer-readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media can be any available media that can be accessed by a general purpose or special purpose computer. For example, a computer readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the computer readable storage medium. Of course, the computer readable storage medium may also be integral to the processor. The processor and the computer-readable storage medium may reside in an Application Specific Integrated Circuit (ASIC). Additionally, the ASIC may reside in user equipment. Of course, the processor and the computer-readable storage medium may also reside as discrete components in a communication device.
In particular, the computer-readable storage medium may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random-Access Memory (SRAM), electrically-Erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The present application also provides a computer program product comprising a computer program stored in a computer readable storage medium. The computer program can be read by at least one processor of the device from a computer-readable storage medium, and execution of the computer program by the at least one processor causes the device to implement the methods provided by the various embodiments described above.
Embodiments of the present application further provide a chip, where the chip includes a memory and a processor, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device in which the chip is installed executes the method in the above various possible embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of modules may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
Wherein the modules may be physically separated, e.g. mounted at different locations of one device, or mounted on different devices, or distributed over multiple network elements, or distributed over multiple processors. The modules may also be integrated, for example, in the same device, or in a set of codes. The respective modules may exist in the form of hardware, or may also exist in the form of software, or may also be implemented in the form of software plus hardware. The method and the device can select part or all of the modules according to actual needs to achieve the purpose of the scheme of the embodiment.
When the respective modules are implemented as integrated modules in the form of software functional modules, they may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor to execute some steps of the methods according to the embodiments of the present application.
It should be understood that, although the respective steps in the flowcharts in the above-described embodiments are sequentially shown as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, in different orders, and may be performed alternately or at least partially with respect to other steps or sub-steps of other steps.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: it is also possible to modify the solutions described in the previous embodiments or to substitute some or all of the technical features. And the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims (14)

1.A method of data processing, the method comprising:
acquiring and analyzing access data and operation instructions of a plurality of interfaces of a system during multiple calls to obtain interface execution information of each interface during each call, wherein the interface execution information at least comprises an interface number, an access field value and an associated field;
analyzing and obtaining field associated information and data table associated information of each data table in a database of the system according to the interface execution information of the plurality of interfaces of the system during multiple calls;
determining a data template corresponding to the target service according to the data table association information and the target service;
and generating the data to be generated of the target according to the data template and the field association information, and finishing the number of the target service by using the data to be generated.
2. The method according to claim 1, wherein the obtaining and analyzing the access data and the operation command of the multiple interfaces of the system during multiple calls to obtain the interface execution information of each interface during each call comprises:
acquiring the access data and the operation instruction of the interface when the interface is called by monitoring the interface;
analyzing the parameter input data to obtain a parameter input dictionary; analyzing the operation instruction to obtain an instruction dictionary;
and generating interface execution information of the interface during the calling according to the reference dictionary and the instruction dictionary.
3. The method according to claim 2, wherein the parsing the entry parameter data to obtain an entry parameter dictionary specifically includes:
analyzing the parameter data to obtain a parameter dictionary of the parameter data, wherein the parameter dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value;
if the data type of the field value of the key value pair in the reference dictionary is different from the preset type, deleting the key value pair;
and if the field values of a plurality of key value pairs in the reference dictionary are the same, deleting a plurality of field values.
4. The method according to claim 2, wherein the operation instruction includes at least one database control instruction, and the analyzing the operation instruction to obtain an instruction field specifically includes:
determining an instruction type corresponding to each database control instruction, wherein the instruction types at least comprise addition, deletion, updating and query;
analyzing the database control instructions according to the instruction types of the database control instructions to obtain an instruction dictionary of the database control instructions, wherein the instruction dictionary comprises at least one key value pair, and each key value pair comprises a field name and a field value.
5. The method according to claim 4, wherein when the database control instruction is a query operation, the analyzing the database control instruction to obtain the instruction dictionary of the database control instruction specifically includes:
analyzing the field name and the field value in the database control instruction to obtain at least one key value pair of the database control instruction;
analyzing a character string nesting relation corresponding to the database control instruction according to the first preset character and the second preset character, and constructing a tree structure of the database control instruction;
determining the instruction dictionary of the database control instruction according to at least one key-value pair of the database control instruction and the tree structure of the database control instruction.
6. The method according to any one of claims 1 to 5, wherein the analyzing, according to the interface execution information of the plurality of interfaces of the system during multiple calls, to obtain the field association information and the data table association information of each data table in the database of the system specifically includes:
according to the associated fields in the interface execution information of a plurality of interfaces of the system during multiple calls, a plurality of single fields and the occurrence times of each single field are obtained through sorting;
sequencing the single fields according to the sequence of the occurrence times from large to small;
traversing the sorted single fields, and screening out main fields of the business objects and directly associated fields corresponding to the main fields of each business object from the single fields;
marking the main field of the business object and the directly related field as related;
traversing the single field which is not marked as being associated, and screening out an indirect association field corresponding to the main field of the business object;
and determining field association information and data table association information of each data table according to the main field of the service object, the direct association field and the indirect association field.
7. The method according to claim 6, wherein traversing the sorted single fields and screening out the main fields of the service objects and the directly associated fields corresponding to the main fields of each service object from the single fields specifically comprises:
sequentially acquiring a first single field from the sorted single fields;
when the first single field is not marked as being associated and the first single field is the only primary key in the data table where the first single field is located, determining that the first single field is the main field of the business object;
sequentially acquiring a second single field from the sorted single fields;
when the second single-item field is not marked as being associated, calculating an association index of the first single-item field and the second single-item field;
and when the association index is larger than a first preset threshold value, determining that the second single field is a direct association field of the first single field.
8. The method according to claim 7, wherein the correlation index includes a support degree index, a confidence degree index and a lifting degree index, and the calculating the correlation index of the first single-item field and the second single-item field specifically includes:
determining the support degree index according to the ratio of the number of times of the simultaneous occurrence of the first single field and the second single field in the interface execution information to the total number of the interface execution information;
determining a confidence index according to the ratio of the support index to the number of times of occurrence of the second single field in the total number of interface execution information;
and determining a promotion index according to the ratio of the confidence index to the number of times of occurrence of the first single field in the total number of interface execution information.
9. The method according to claim 8, wherein traversing the single field that is not marked as associated and screening out an indirect associated field corresponding to a main field of a business object specifically comprises:
generating an associated list and an unassociated list according to the mark of each single field;
sequentially acquiring a third single field from the unassociated list;
calculating association indexes of the third single-item fields and each fourth single-item field in the associated list one by one;
when the correlation index is larger than a second preset threshold value, determining that the third single field is correlated with the fourth single field;
and determining that the third single field is an indirect association field of the business object main field corresponding to the fourth single field.
10. The method according to any one of claims 1-5, wherein before analyzing the field association information and the data table association information of each data table in the database of the system according to the interface execution information of the plurality of interfaces of the system during multiple calls, the method further comprises:
and accumulating the index information consisting of the interface number, the parameter field and the parameter field value in the interface execution information until the number of the non-repeated index information reaches a preset number.
11. The method according to any one of claims 1 to 5, wherein the determining, according to the data table association information and the target service, the data template corresponding to the target service specifically includes:
determining a corresponding business object main table according to the target business;
determining an association table with association according to the data table association information and the business object main table;
and searching the main table and the association table of the service object to obtain a data template.
12. The method according to any one of claims 1 to 5, wherein the generating data to be generated of the target according to the data template and the field association information specifically includes:
resetting the associated fields in the data template to the same value according to the field associated information;
and resetting the values of other fields in the data template according to the field rule.
13. A server, characterized in that the server comprises: a memory, a processor; the memory is used for storing a computer program; the processor is configured to implement the data processing method according to any one of claims 1 to 12, in accordance with the computer program stored in the memory.
14. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the data processing method of any one of claims 1 to 12.
CN202211632995.4A 2022-12-19 2022-12-19 Data processing method, server and computer storage medium Pending CN115794833A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211632995.4A CN115794833A (en) 2022-12-19 2022-12-19 Data processing method, server and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211632995.4A CN115794833A (en) 2022-12-19 2022-12-19 Data processing method, server and computer storage medium

Publications (1)

Publication Number Publication Date
CN115794833A true CN115794833A (en) 2023-03-14

Family

ID=85425761

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211632995.4A Pending CN115794833A (en) 2022-12-19 2022-12-19 Data processing method, server and computer storage medium

Country Status (1)

Country Link
CN (1) CN115794833A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116561132A (en) * 2023-07-06 2023-08-08 深圳市明源云科技有限公司 Service data query method, electronic device and readable storage medium
CN116777597A (en) * 2023-06-19 2023-09-19 中国银行保险信息技术管理有限公司 Financial risk assessment method, device, storage medium and computer equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116777597A (en) * 2023-06-19 2023-09-19 中国银行保险信息技术管理有限公司 Financial risk assessment method, device, storage medium and computer equipment
CN116561132A (en) * 2023-07-06 2023-08-08 深圳市明源云科技有限公司 Service data query method, electronic device and readable storage medium
CN116561132B (en) * 2023-07-06 2023-12-22 深圳市明源云科技有限公司 Service data query method, electronic device and readable storage medium

Similar Documents

Publication Publication Date Title
CN115794833A (en) Data processing method, server and computer storage medium
CN101131706B (en) Query amending method and system thereof
CN109284293B (en) Data migration method for upgrading business charging system of water business company
US5875445A (en) Performance-related estimation using pseudo-ranked trees
US20080005106A1 (en) System and method for automatic weight generation for probabilistic matching
US9442991B2 (en) Ascribing actionable attributes to data that describes a personal identity
JP2014533417A (en) Data clustering based on candidate queries
CN108509505B (en) Character string retrieval method and device based on partition double-array Trie
WO2015010509A1 (en) One-dimensional liner space-based method for implementing trie tree dictionary search
CN111858567A (en) Method and system for cleaning government affair data through standard data elements
CN114625748A (en) SQL query statement generation method and device, electronic equipment and readable storage medium
CN117290376A (en) Two-stage Text2SQL model, method and system based on large language model
CN117251517B (en) Annual-authentication administrative division information matching method and model under large data field of view
CN110990423B (en) SQL statement execution method, device, equipment and storage medium
CN116126918A (en) Data generation method, information screening method, device and medium
CN111221864B (en) Intelligent index recommendation method based on mysql slow query log word frequency analysis
CN114327607A (en) Automatic generation method of BS code
CN109785099B (en) Method and system for automatically processing service data information
CN117390064B (en) Database query optimization method based on embeddable subgraph
CN116680299B (en) Database query method, system, storage medium and device
CN114090627B (en) Data query method and device
CN109977269B (en) Data self-adaptive fusion method for XML file
CN114943004B (en) Attribute graph query method, attribute graph query device, and storage medium
JPH0352068A (en) Logical operation system
Magnani et al. A study on company name matching for database integration

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication