CN114237762A

CN114237762A - Point file processing method, device, equipment, medium and program product

Info

Publication number: CN114237762A
Application number: CN202111585672.XA
Authority: CN
Inventors: 丁欢; 邱晓海; 陈磊; 丁明翼; 王勇
Original assignee: China Construction Bank Corp
Current assignee: China Construction Bank Corp
Priority date: 2021-12-22
Filing date: 2021-12-22
Publication date: 2022-03-25

Abstract

The present disclosure provides a method for processing an integral file, including: acquiring point files of M channels in N channels, wherein the N channels comprise channels for transacting services for a user, and the point files comprise user data generated in response to the services transacted by the user; processing a score file of each channel of the M channels based on a preset configuration file, wherein each channel corresponds to at least one configuration file, and the configuration files comprise processing logic for the user data; and obtaining the user points according to the processing result corresponding to each channel, wherein the processing result is obtained by executing the processing logic on the user data in the point file. The present disclosure also provides a point file processing apparatus, a device, a storage medium, and a program product.

Description

Point file processing method, device, equipment, medium and program product

Technical Field

The present disclosure relates to the field of data processing, and more particularly, to a method, an apparatus, a device, a medium, and a program product for processing a point file.

Background

Some companies can accumulate points according to user data, so that users can exchange rights and interests by taking the accumulated points as exchange carriers, and the purposes of rewarding the users and enhancing the user stickiness are achieved. In performing the user point calculation, the user data may relate to different point activities, which in turn may relate to different point calculation rules. Therefore, the user data is usually matched with corresponding integral calculation conditions, and the user integral is obtained through calculation, wherein the integral calculation conditions comprise processing logic of the user data.

In the related art, the processing logic of the user data may be directly embedded in the program, for example, the related variables are directly replaced by the fixed calculation rules in the source code of the program, and cannot be changed freely. The credit activities of each company are various, and the service scenes are various, so that the content and the format of user data are complex and various. The user data processing logic is also dynamically variable and complex due to integration activity, changes in service scenarios, or correlation to each other. Therefore, how to dynamically and extendably configure processing logic and efficiently process complex user data in a multi-service scenario is a problem to be solved urgently at present.

Disclosure of Invention

In view of the foregoing, the present disclosure provides a method, apparatus, device, medium, and program product for processing a point file of a configuration processing logic that is dynamically and extensible in an abstract, centralized, and configurable manner.

In an aspect of the embodiments of the present disclosure, there is provided a method for processing an integral file, including: acquiring point files of M channels in N channels, wherein the N channels comprise channels for transacting services for a user, the point files comprise user data generated in response to the services transacted by the user, N, M are integers greater than or equal to 1 respectively, and M is less than or equal to N; processing a score file of each channel in the M channels based on a preset configuration file, wherein each channel corresponds to at least one configuration file, and the configuration files comprise processing logic for the user data; and obtaining the user scores according to the processing result corresponding to each channel, wherein the processing result is obtained by executing the processing logic on the user data in the score file.

According to an embodiment of the present disclosure, the configuration file includes an analytic configuration file, the processing logic includes an analytic logic, and processing the score file of each of the M channels based on a preset configuration file includes: determining the corresponding analysis configuration file based on the channel identification of each channel; and executing analysis logic in the analysis configuration file, and analyzing the integral file of each channel, wherein the analysis logic is used for converting the user data into user data in a preset format.

According to an embodiment of the present disclosure, the parsing configuration file includes X parsing units, the executing parsing logic in the parsing configuration file, and parsing the point file of each channel includes executing the X parsing units, which specifically includes: determining X first target fields from the user data, wherein X is an integer greater than or equal to 1; and correspondingly mapping the values of the X first target fields to an X column in a first database table, wherein the mapped X column comprises the user data in the preset format, and the X analysis units comprise X analysis logics corresponding to the X column.

According to an embodiment of the present disclosure, the method further comprises: executing a preset storage process, and merging at least two first database tables with an association relationship into a second database table, wherein the association relationship is determined by the relationship between the channels in the N channels.

According to an embodiment of the present disclosure, the configuration file includes a process configuration file, the processing logic includes a process logic, and processing the score file of each of the M channels based on a preset configuration file further includes: determining the corresponding process configuration file based on the channel identification of each channel; and executing the flow logic in the flow configuration file, and processing the analyzed integral file of each channel, wherein the flow logic is used for processing the user data in the preset format to obtain a processing result.

According to an embodiment of the present disclosure, the process configuration file includes Y process unit sets, the executing the process logic in the process configuration file, and the processing the parsed point file of each channel includes: sequentially executing each flow unit set in the Y flow unit sets based on the preset sequence of the Y flow unit sets, wherein each flow unit set corresponds to each processing flow for obtaining the processing result; wherein each flow unit set comprises Z flow units, each flow unit in the Z flow units corresponds to each sub-process flow in each process flow, each flow unit comprises at least one flow logic, and Y, Z is an integer greater than or equal to 1.

According to an embodiment of the present disclosure, the obtaining the user score according to the processing result corresponding to each channel includes: determining a second target field from the analyzed integral file of each channel; querying a corresponding configuration condition based on the second target field, wherein the configuration condition is used for determining a calculation condition for obtaining the user point; and obtaining the user points according to the configuration conditions and the processing result.

According to the embodiment of the present disclosure, before querying the corresponding configuration condition based on the second target field, the method further includes presetting the configuration condition, specifically including: presetting at least one parameter identifier in a third database table; and presetting at least one scene field corresponding to each parameter identifier in the at least one parameter identifier in a fourth database table.

According to an embodiment of the present disclosure, the querying the corresponding configuration condition based on the second target field includes: determining a target parameter identifier based on the at least one parameter identifier in the third database table; determining at least one scene field corresponding to the target parameter identifier from the fourth database table; matching the second target field with the at least one scene field to obtain a matching result; and determining corresponding configuration conditions based on the matching result.

According to an embodiment of the present disclosure, further comprising: distributing N channel identifications for the N channels; setting a file storage path corresponding to each channel identifier and an integral file identifier; wherein, the obtaining of the score files of M channels in the N channels comprises: scanning a file storage path corresponding to each channel identifier to acquire at least one storage file; determining the score file from the at least one stored file based on the score file identification.

Another aspect of the disclosed embodiments provides a score file processing apparatus, including: the system comprises a file acquisition module, a file acquisition module and a service management module, wherein the file acquisition module is used for acquiring score files of M channels in N channels, the N channels comprise channels for transacting services for a user, the score files comprise user data generated in response to the services transacted by the user, and N, M are integers greater than or equal to 1 respectively; the file processing module is used for processing the integral file of each channel in the M channels based on a preset configuration file, wherein each channel corresponds to at least one configuration file, and the configuration files comprise processing logic for the user data; and the point acquisition module is used for acquiring the user points according to the processing result corresponding to each channel, wherein the processing result is acquired by executing the processing logic on the user data in the point file.

Another aspect of the disclosed embodiments provides an electronic device, including: one or more processors; a storage device to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method as described above.

Yet another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having stored thereon executable instructions, which when executed by a processor, cause the processor to perform the method as described above.

Yet another aspect of the disclosed embodiments provides a computer program product comprising a computer program that when executed by a processor implements the method as described above.

One or more of the above embodiments have the following advantageous effects: firstly, abstracting steps or objects needing to be processed according to the characteristics of each channel in N channels to form processing logic adapted to each channel. Then, the configuration files of the N channels are managed in a centralized mode, and each channel corresponds to at least one configuration file. And realizing the configuration design of the processing logic adapted to the N channels by the form of the configuration file. Therefore, dynamic change and expandable effect of processing logic can be realized through operation on the configuration file, for example, the processing logic in the corresponding configuration file can be executed aiming at the score files of M channels to process the user data in the score files. And finally, obtaining the user points according to the processing result.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an architecture diagram of a user point processing system according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a score file processing method according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow chart for processing a points file according to an embodiment of the present disclosure;

FIG. 4 schematically illustrates a flow chart for parsing a points file according to an embodiment of the disclosure;

FIG. 5 schematically illustrates a flow diagram for processing a points file according to another embodiment of the present disclosure;

FIG. 6 schematically shows a flow chart for obtaining user points according to an embodiment of the present disclosure;

FIG. 7 schematically illustrates a flow chart for presetting the configuration conditions according to an embodiment of the present disclosure;

FIG. 8 schematically illustrates a flow diagram for querying configuration conditions, in accordance with an embodiment of the present disclosure;

FIG. 9 schematically illustrates a flow chart for obtaining a points file according to an embodiment of the disclosure;

fig. 10 schematically shows a block diagram of the structure of the integral file processing apparatus according to the embodiment of the present disclosure;

FIG. 11 schematically shows a block diagram of an electronic device adapted to implement the method of integral file processing according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

Embodiments of the disclosure relate to terms having the following meanings:

and (4) channel: the method is used for distinguishing factors such as the types of services handled by users, the sources of the provided services, data sources, user behaviors, service scenes and the like. For example, data generated by a user consuming with a credit card can be considered to belong to a credit card channel, data generated by a user consuming with a debit card can be considered to belong to a debit card channel, and the channel is not particularly limited by the present disclosure, and the distinguishing function can be achieved.

The processing logic: the data processing process of one or more processing actions is completed according to specific requirements. One or more program statements are executed in accordance with the content of the user data, e.g., based on preset program execution logic and execution conditions.

A database table: an object in a database used to store data is a collection of structured data.

And (3) a storage process: a set of SQL statements for accomplishing a specific function is compiled and stored in a database, and is invoked to execute by specifying the name of the stored procedure and giving a parameter (if the stored procedure carries a parameter).

In the technical scheme of the disclosure, the processing of acquisition, collection, storage, use, processing, transmission, provision, disclosure, application and the like of user data is carried out under the permission of a user, and the user data conforms to the regulations of related laws and regulations, and necessary confidentiality measures are taken without violating the good custom of the public order.

Fig. 1 schematically shows an architecture diagram of a user point processing system according to an embodiment of the present disclosure.

As shown in fig. 1, the user point processing system 100 according to this embodiment may include a data source 110, a first cluster 120, a second cluster 130, a third cluster 140, a rule engine 150, a fourth cluster 160, message middleware 170, a fifth cluster 180, and a sixth cluster 190. The data source 110 may include a static data source 111 and a real-time data source 112, among others. The first cluster 120, the second cluster 130, the third cluster 140, the fourth cluster 160, the fifth cluster 180, and the sixth cluster 190 may be distributed clusters, respectively. Message middleware 170 may be a message cluster to provide message queue services.

The data source 110 is configured to receive a static data source, such as first user data generated based on a first activity of a user, and may also be configured to receive a real-time data source, such as second user data generated based on a second activity. The point file may include a static point file and a dynamic point file, and the corresponding point file includes the first user data or the second user data.

The first cluster 120 may include a plurality of server clusters, where each server cluster may include several servers. The first cluster 120 may be used to pre-process static data.

The second cluster 130 may comprise a Hadoop cluster for receiving the preprocessed data from the first cluster 120 for further processing the calculations for the preprocessing. The Hadoop cluster is a platform suitable for distributed storage and distributed computing of mass data, and can comprise a distributed storage framework (HDFS) component, a distributed computing framework (MapReduce) component and a resource scheduling platform (yann) component.

The third cluster 140 may include a Spark cluster, which is used to read the files processed by the second cluster 130 for concurrent processing. Specifically, rule matching may be performed item by item, and integral calculation may be performed according to the matched rule. The Spark cluster can be used as a caller for resource allocation, wherein Spark applications running in the Spark cluster run independently and are isolated from each other to a certain extent.

The rules engine 150 may be configured to invoke one or more score rules, and in response to invocation of the third cluster 140, filter the score rules and return matching score rules.

The fourth cluster 160 may include a plurality of online application clusters for providing the first online service.

Message middleware 170 may include Kafka message clusters to provide message queue services. The Kafka message cluster can be classified according to topic when storing messages, each topic can be divided into a plurality of part groups, and each part is an ordered queue. Each message in the partition is assigned an ordered id.

The fifth cluster 180 may comprise a streaming Storm cluster for listening to messages in the Kafka message cluster and streaming.

The sixth cluster 190 may include a plurality of online application clusters for providing second online services.

The user point processing system 100 may be used to implement the user point processing method according to the embodiment of the present disclosure, and may also be used to deploy the user point processing apparatus according to the embodiment of the present disclosure.

The user point processing method according to the embodiment of the present disclosure will be described in detail below with reference to fig. 2 to 9, taking the financial institution as an example to perform user point processing and obtain a static point file from a static data source, based on the user point processing system described in fig. 1.

Fig. 2 schematically shows a flow chart of a method of processing a point file according to an embodiment of the present disclosure.

As shown in fig. 2, the integration file processing method of this embodiment may include operations S210 to S230.

In operation S210, point files of M channels of N channels including a channel for transacting a service for a user are acquired, the point files including user data generated in response to the service transacted by the user, N, M being integers greater than or equal to 1, respectively, and M being less than or equal to N.

For example, where the financial institution is a bank, the channels may include credit card channels, debit card channels, standard-reaching activity channels, and the like. For example, a credit card channel may provide credit card consumption services to a user and generate user data in response to the user's credit card consumption behavior. The debit-card channel may provide debit-card consumption services to the user and generate user data in response to the user's debit-card consumption behavior. The standard activity channel can respond to the activity participation of the user, provide services such as activity entries, activity descriptions and the like for the user and generate user data.

The points file may include user data for one or more users over a period of time. For example, the credit card integration file may include information on how current credit card consumption is for a number of users over a period of time. The debit card integration file may include debit card consumption chronological information for a plurality of users over a period of time. The compliance recording points file may include compliance recording information for a plurality of users over a period of time.

Referring to fig. 1, after transacting business in M channels, a user will generate a score file and push the score file to a static data source. The static data source may include a storage device for storing a static points file. The points file may be scanned out of the storage device periodically. Since it may not be that the credit file of each channel has been pushed to the static data source during the timed scan, the credit files of M channels are obtained.

In operation S220, a score file for each of the M channels is processed based on a preset profile, where each channel corresponds to at least one profile, and the profile includes processing logic for user data.

The score files generated by the various channels may have different formats, such as file formats, or the same value in each file may correspond to different fields. Therefore, at least one configuration file can be correspondingly set for each channel, and the configuration files are used for processing data in the integral files.

Different channels may adopt different data generation processes, so that the content and format of the score files received from each channel are different, and the score calculation cannot be performed through the unified processing logic. Therefore, by adopting the configuration file corresponding to each channel, targeted processing can be realized. The processing logic may be determined for the object to be processed based on a requirement for processing the partial data into the target data content or the data format.

Due to the abstraction, a certain processing logic can be applied to a plurality of channels with corresponding requirements in a generalized manner, that is, a part of the processing logic between one or more channels can be shared (for example, a common logic can be written in respective configuration files, or a configuration file shared by the channels can also be set). In addition, through the adapted configuration file, the processing logic specific to each channel can also be written into the configuration file.

In operation S230, a user score is obtained according to a processing result corresponding to each channel, wherein the processing result is obtained by executing a processing logic on user data in the score file.

According to the embodiment of the disclosure, firstly, steps or objects needing to be processed are abstracted according to the characteristics of each channel in N channels, and processing logic adapted to each channel is formed. Then, the configuration files of the N channels are managed in a centralized mode, and each channel corresponds to at least one configuration file. And realizing the configuration design of the processing logic adapted to the N channels by the form of the configuration file. Therefore, dynamic change and extensible effect of user processing logic can be realized through operation on the configuration file. The processing logic in the corresponding configuration file can be executed for the score files of the M channels to process the user data in the score files. And finally, obtaining the user points according to the processing result. Therefore, the problems of processing of various service scenes and complex and various data formats can be partially solved.

Fig. 3 schematically shows a flowchart of processing the integral file in operation S220 according to an embodiment of the present disclosure.

As shown in fig. 3, the processing of the score file for each of the M channels based on the preset profile in operation S220 may include operations S310 to S320. Wherein the configuration file comprises an analytic configuration file and the processing logic comprises an analytic logic.

In operation S310, a corresponding parsing profile is determined based on the channel identification of each channel.

In some embodiments, a channel identifier may be pre-assigned to each channel, and the parsing configuration file has a parsing identifier corresponding to the channel identifier, for example, the name of the parsing configuration file is named by the channel identifier, and then the name is the parsing identifier. Or writing the channel identifier in the analysis configuration file as the analysis identifier.

In operation S320, parsing logic in the parsing configuration file is executed to parse the score file of each channel, wherein the parsing logic is used to convert the user data into user data in a predetermined format.

Referring to fig. 1, the first cluster 120 may run a bulk application 1, a bulk application 2, a bulk application 3 … … a bulk application N. Each batch application may interface with a corresponding channel and execute parsing logic to process user data upon determining a parsing configuration file.

The predetermined format may refer to a user data format that conforms to a uniform specification. For example, fields representing the same value in different score files are not the same, i.e., are assumed to be in different formats, and the values may be mapped to the same fields. For example, the type of the user data in different score files is integer type or floating point type, i.e. the format is determined to be different, and the format can be unified into character string type. For example, the values of the same business meaning of different score files are in different units, namely, the values are considered to be in different formats, and the values can be unified into units, such as dollars, Japanese dollars or Euros, which are converted into RMB.

One of the functions of the parsing logic is to preprocess the integral file in the static data source, so as to facilitate the subsequent unified operation on the preprocessed user data. For example, a user may consume using a credit card at a mobile banking client of bank a, or may consume using a credit card at applications such as wechat, pay treasure, unionpay, and the like, and although the point file generated in the above process may be defined as a credit card channel, due to different merchants consuming the credit card, the consumption application is different, and accordingly, different user data formats are generated. By means of preprocessing, user data in different formats can be converted into user data in a predetermined format.

The Format configuration can be carried out by using a Format expression in an XML Format, the channel identification is used as the identification of the configuration section, and the analysis unit is used as a node to configure the analysis logic of each line of user data of the integral file.

Fig. 4 schematically shows a flowchart of parsing the point file in operation S320 according to an embodiment of the present disclosure.

As shown in fig. 4, the parsing logic in the configuration file in operation S320 may include performing the X parsing units, specifically including operations S410 to S420. The analysis configuration file comprises X analysis units.

In operation S410, X first target fields are determined from the user data, where X is an integer greater than or equal to 1.

For example, in a credit card channel point file, each line of user data is a consumption stream of one user. Each row of User data may include, for example only, a User identification User ID, a credit Card Number, a transaction serial Number, etc. The parsing configuration unit may include 3 parsing units A, B, C corresponding to the User ID, the credit Card Number, and the transaction serial Number one to one. For example, parsing unit A is used to determine the first destination field User ID, and the other two parsing units B, C are used to determine the Card Number and the Serials Number.

In operation S420, the values of the X first target fields are mapped to X columns in the first database table, where the mapped X columns include user data in a predetermined format, and the X parsing units include X parsing logics corresponding to the X columns.

For example, the parsing unit a traverses the User data of each row, first determining the first target field User ID. The value corresponding to the User ID is then populated into the column in the first database table corresponding to the User identification, such as the User Number column. The parsing unit B, C has a similar parsing process as described above, and is not described herein.

The function of the X kinds of analysis logics is that, on one hand, data meeting requirements are filled into corresponding columns, and files in a static data source are conveniently put into a database for subsequent processing. Moreover, the value of the User ID is mapped to a User Number column, so that the data format of the front end (facing to the User) can be converted into a format which is convenient to process by the back end (the integration processing background). On the other hand, the analysis unit is used as a unit to configure the analysis logic, which is beneficial to clearly standardizing the processing steps of the user data. In some embodiments, each parsing logic may be composed of a plurality of program statements, such as statements used for data cleansing, field matching, traversal, and the like functions.

According to an embodiment of the present disclosure, after the operation S420 is performed, a preset storing process is further performed to merge at least two first database tables having an association relationship into one second database table, where the association relationship is determined by a relationship between channels of the N channels.

Referring to FIG. 1, after the first cluster 120 obtains the first database table, the first database table may be unloaded to the second cluster 130 through a storage process. The first database table obtained by preprocessing using the parsing configuration file in the first cluster 120 may be a temporary table, and then the first database table is further processed and merged to obtain the target table in the second cluster 130.

Relationships between channels may be obtained through the association of user data for the various channels. For example, the credit card channels may include a credit card A channel, a credit card B channel, and a credit card C channel. The credit card channel A can generate a transaction serial number under the credit card, the corresponding point file can comprise the transaction serial number and a user identifier to indicate that the user has credit card transaction behavior, and repeated flowing is prevented through the transaction serial number and the user identifier. The credit card B channel is used for processing transaction detail information of a scene in a bank, for example, the credit card B channel may include a transaction serial number, and corresponding merchant information, whether to sign a contract, a transaction amount, a transaction type, and the like. The credit card C channel is used for processing transaction detail information of a scene outside a bank, such as transaction detail information applied by a user in WeChat, Payment treasures, Unionpay and the like, for example, a point file may include a transaction serial number, and corresponding information of a merchant, whether to sign a contract, a transaction amount, a transaction type and the like.

After performing operations S410 to S420, the credit card a channel, the credit card B channel, and the credit card C channel correspond to a first database table, respectively. And to calculate the user points of the credit card channel, all the data of the three first database tables of the three channels can be processed uniformly. Therefore, the first database tables of the three channels can be merged into a second database table (i.e. the target table) based on the transaction serial number based on the storage process.

Fig. 5 schematically shows a flowchart of processing the integral file in operation S220 according to another embodiment of the present disclosure.

As shown in fig. 5, the processing of the score file for each of the M channels based on the preset profile in operation S220 may include operations S510 to S520. Wherein operations S510 to S520 may be performed after operation S320. The configuration file comprises a parsing configuration file and the processing logic comprises parsing logic.

In operation S510, a corresponding process profile is determined based on the channel identification of each channel.

The configuration method of the process configuration file is similar to that of the analysis configuration file, and the process configuration file can be distributed with an identifier corresponding to the channel identifier so as to be convenient to determine.

In operation S520, a flow logic in the flow configuration file is executed, and the parsed point file of each channel is processed, wherein the flow logic is configured to process the user data in the predetermined format to obtain the processing result.

The parsed per-channel points file may be a first database table per channel. If the channel A of the credit card, the channel B of the credit card and the channel C of the credit card have an association relationship, the analyzed point files of the three channels can all correspond to the same second database table. Similarly, three channels may also correspond to the same process profile, as determined by the channel identification of the credit card a channel, for example.

According to an embodiment of the present disclosure, the process configuration file includes Y sets of process units, and operation S520 includes: and sequentially executing each flow unit set in the Y flow unit sets based on the preset sequence of the Y flow unit sets, wherein each flow unit set corresponds to each processing flow for obtaining the processing result. Each flow unit set comprises Z flow units, each flow unit in the Z flow units corresponds to each sub-processing flow in each processing flow, each flow unit comprises at least one flow logic, and Y, Z is an integer greater than or equal to 1.

Referring to fig. 1, the third cluster 140 may be utilized to take out the parsed score file of each channel from the second cluster 130, and determine a process configuration file and then sequentially execute each process unit set therein.

For example, in the third cluster 140, statements "Card 01 ═ Filter1, Check1, Check2, Deal 1" may be configured in properties type files of JAVA engineering. Wherein "Card 01" is a channel identifier of the credit Card a channel, and "Filter 1", "Check 1", "Check 2" and "Deal 1" are respectively a variable for calling a flow unit set corresponding to each processing flow. Based on the preset sequence in the above statements, "Filter 1", "Check 1", "Check 2" and "Deal 1" are executed in sequence during execution. For example, "Filter 1" is executed to Filter out invalid data in the user data, "Check 1" is executed to Check whether the merchant ID exists in the current user data in the merchant blacklist, "Check 2" is executed to Check whether the user ID exists in the user blacklist in the current user data, "Deal 1" is executed to perform integral calculation to obtain a processing result, and a rule engine may be invoked to calculate and obtain the processing result while or after "Deal 1" is executed. The processing result may include an integrated value that can be accumulated for each piece of user data.

Taking the execution of "Filter 1" as an example, the called flow unit set may include a plurality of flow units, and the process of executing the flow units is a process of completing the sub-processing flow. For example, a flow unit for filtering first invalid data (e.g., null), a flow unit for filtering second invalid data (e.g., scrambling code), and so on.

It should be noted that the flow logic in the flow configuration file is different from the rules that the rules engine 150 can invoke. The execution flow logic is different from the user data content processed by the execution rule, for example, "Check 1" is used to execute the corresponding flow logic, and the merchant ID is checked whether it is in the merchant blacklist, then the rule may not be configured for checking the merchant ID.

Fig. 6 schematically shows a flowchart of obtaining user points in operation S230 according to an embodiment of the present disclosure.

As shown in fig. 6, the obtaining of the user points according to the processing result corresponding to each channel in operation S230 may include operations S610 to S630.

Referring to fig. 1, after the third cluster 140 processes the parsed point file to obtain a processing result, the processing result may be pushed to the message middleware 170, and the processing result may be encapsulated into a corresponding message. In particular, the third cluster 140 may compute one or more processing results, encapsulated as one or more messages, in response to one or more activities involved with each transaction pipeline data.

The sixth cluster 190 may perform operations S610 to S630 to obtain the user points. For example, the sixth cluster 190 may include a router or gateway, an online application 1, an online application 2, an online application 3 … …, an online application N, and a database. Among them, the online applications 1 to N may provide an online service, and the online service may update the user points in units of the user point account in response to a request generated by consuming the message from the fifth cluster 180.

In operation S610, a second target field is determined from the parsed point file of each channel.

In an alternative embodiment, each message may be considered as a parsed point file, and the second target field may be included in each message.

Another alternative is to include the Serials Number, User ID and the value of the credit in each message. The fifth cluster 180 may retrieve a second database table from the database and determine a second target field based on the Serials Number.

In operation S620, a corresponding configuration condition is queried based on the second target field, wherein the configuration condition is used to determine a calculation condition for obtaining the user score.

In operation S630, a user score is obtained according to the configuration condition and the processing result.

For example, the online service first calls the current running tally based on the User ID. Then, the row of data where a certain Serial Number corresponding to the User ID is located in the second database table is accessed, and the corresponding point activity field in the point activity column is determined to be used as a second target field. Then, the corresponding configuration condition is inquired, for example, whether the accumulated integral of the integral activity has the maximum value limit, if so, whether the accumulated integral of the integral activity exceeds the maximum value is further determined according to the current accumulated integral value. If not, the integration is continued directly on the basis of the current integration integral.

Fig. 7 schematically shows a flow chart of preset configuration conditions according to an embodiment of the present disclosure.

Before performing operation S620, configuration conditions may also be preset, and as shown in fig. 7, the preset configuration conditions may include operations S710 to S720.

In operation S710, at least one parameter identifier is preset in a third database table.

Parameter identification may refer to configuring the computation conditions in a parameterized manner. Each calculation condition may be assigned a corresponding parameter identification.

In operation S720, at least one scene field corresponding to each of the at least one parameter identifier is preset in the fourth database table.

The scene field may refer to detailed scene information specifically related to the user data, such as fields corresponding to information of a plurality of point activities, point star levels, consumption amounts, merchant names, and the like.

The third database table can be used as a parameter identification set definition table for defining parameter identification (such as name, number, etc.) and type. The fourth database table may be used as a parameter identifier specific value table for defining enumerated values of the parameter identifier, i.e., scene fields.

According to the embodiment of the present disclosure, in one aspect, the parameterized configuration is implemented in a database table, and the dynamically adjustable parameter identifier can be stored through a database wide table design. Compared with a file configuration mode, the database table mode is more flexible and specific, the database table mode takes effect quickly, and centralized configuration is more efficient. On the other hand, the third database table and the fourth database table are combined with each other to realize parameterization configuration, so that management of various calculation conditions can be facilitated, and efficiency can be improved by operating according to management specifications during programming or condition updating.

Fig. 8 schematically shows a flowchart of querying the configuration condition in operation S620 according to an embodiment of the present disclosure.

As shown in fig. 8, querying the corresponding configuration condition based on the second target field in operation S620 may include operations S810 to S840.

In operation S810, a target parameter identifier is determined based on at least one parameter identifier in the third database table.

Each parameter identification may be determined as a target parameter identification in turn, in order in the third database table.

In operation S820, at least one scene field corresponding to the target parameter identifier is determined from the fourth database table.

In operation S830, the second target field is matched with at least one scene field to obtain a matching result.

In some embodiments, the parsed points file may be a second database table. Each row of data of the second database table may be treated as a unit of processing to determine whether a field of a column in each row of data meets which calculation condition. For example, first, a certain column of fields is treated as a second target field, such as Credit Activity A. Then, a target parameter identification, such as the point accumulation maximum limit Max _ jifen, is determined from a third database table. Next, at least one scene field corresponding to Max _ jifen is determined from the fourth database table. And matching Activity A with at least one scene field to obtain a matching result.

According to the embodiment of the disclosure, in order to avoid frequent query of the third database table and the fourth database table, multiple access to the databases causes system stress. The third database table and the fourth database table can be stored by uniformly utilizing a redis cache, and when a parameter identifier or a scene field is changed, the cache is triggered to be updated, so that the effect of cluster effect is achieved.

In operation S840, a corresponding configuration condition is determined based on the matching result.

An alternative embodiment is to take the point activity with the point accumulation maximum limit set as the scene field, corresponding to Max _ jifen in the fourth database table. If the Activity A can be matched with the corresponding scene field Activity A, the matching result is successful, and the configuration condition sets the maximum integral accumulation limit value for the Activity A. If the matching is not successful, the matching result is failure, and the configuration condition is that Activity A does not set the maximum integral accumulation limit value.

Another optional implementation is to use all the integration activities as scene fields, correspond to Max _ jifen in the fourth database table, set the flag to 1 for activities that set the maximum integration limit, and set the flag to 0 for activities that do not set the maximum integration limit. Therefore, the corresponding scene field Activity a can be matched based on Activity a, then the identifier is queried, if the result is 1, the matching result is started, and the configuration condition sets the maximum integral accumulation limit value for Activity a. If the value is 0, the matching result is closed, and the configuration condition is that Activity A does not set the integral accumulation maximum limit value.

Fig. 9 schematically shows a flowchart for obtaining a points file according to an embodiment of the present disclosure.

As shown in fig. 9, acquiring the point file of this embodiment may include operations S910 to S940, wherein operation S210 may include operations S930 to S940.

In operation S910, N channel identifications are assigned to the N channels.

In operation S920, a file storage path corresponding to each channel identifier and a score file identifier are set.

A monitoring configuration table may be preset for distributing channel identification, setting file storage paths, and point file identification. The monitoring configuration table can be an Excel table or a database table.

Table 1 schematically illustrates monitoring configuration content in some embodiments.

TABLE 1

Channel identification	File storage path	Point file identification
			Card01	/home/ap/nas/file1	Card01_***_％DATEyyyyMMdd％.dat
Card02	/home/ap/nas/file2	Card02_***_％DATEyyyyMMdd％.dat

The credit file identifier is, for example, in the form of "channel identifier _ write sequence number _ write date", and the file of each channel may have the credit file identifier as a file name.

In operation S930, a file storage path corresponding to each channel identifier is scanned to obtain at least one storage file.

In operation S940, a score file is determined from the at least one storage file based on the score file identification.

The NFT scripts can be pushed into each monitoring application server by the FTP script deploying machine for batch updating, so that the monitoring application servers do not need to be restarted, for example, linux system reading on the monitoring servers can be immediately effective in the NFT scripts. Wherein, the NFT script may be invoked to execute the monitoring flow. Referring to table 1, the NFT script may be called at regular time to scan whether parsable storage files are already stored in the file storage path corresponding to each channel identifier. And under the condition of having the storage file, identifying the file name, and determining the file name as the score file if the file name accords with the score file identifier.

Based on the integral file processing method, the disclosure also provides an integral file processing device. The apparatus will be described in detail below with reference to fig. 10.

Fig. 10 schematically shows a block diagram of the structure of the integral file processing apparatus according to the embodiment of the present disclosure.

As shown in fig. 10, the score file processing apparatus 1000 of this embodiment includes a file acquisition module 1010, a file processing module 1020, and a score acquisition module 1030.

The file acquiring module 1010 may perform operation S210 of acquiring score files of M channels of N channels, where the N channels include channels for transacting services for a user, and the score files include user data generated in response to the services transacted by the user, N, M being integers greater than or equal to 1, respectively.

The file obtaining module 1010 may further perform operations S910 to S940, which are not described herein.

The file processing module 1020 may perform operation S220 for processing the score file of each of the M channels based on a preset profile, where each channel corresponds to at least one profile, and the profile includes processing logic for user data.

The file processing module 1020 may also perform operations S310 through S320, and operations S410 through S420. Operations S510 to S520 may also be performed, which are not described herein.

The point obtaining module 1030 may perform operation S230 to obtain user points according to a processing result corresponding to each channel, where the processing result is obtained by performing processing logic on user data in a point file.

The integral obtaining module 1030 may further perform operations S610 to S630, and operations S810 to S840, which are not described herein.

The point file processing device 1000 can monitor the file storage path and the storage file name corresponding to the channel identifier through file monitoring configuration, and trigger analysis processing when the point file is determined to arrive. Then, the analysis processing task performs row disassembly by reading the analysis configuration file. And then, triggering a flow configuration file after the analysis is finished, and sequentially processing the flow configuration file by configuration steps in the flow unit set. And finally, judging whether the records meet the conditions by combining corresponding parameters of the database table to finally obtain a target state result set so as to obtain the user integral. The problem of complicated formats under multiple scenes is efficiently processed through the configuration design.

According to the embodiment of the present disclosure, any plurality of the file obtaining module 1010, the file processing module 1020, and the score obtaining module 1030 may be combined into one module to be implemented, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module.

According to an embodiment of the present disclosure, at least one of the file obtaining module 1010, the file processing module 1020, and the integral obtaining module 1030 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware. Alternatively, at least one of the file acquisition module 1010, the file processing module 1020 and the score acquisition module 1030 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.

As shown in fig. 11, an electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. The processor 1101 may comprise, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 1101 may also include on-board memory for caching purposes. The processor 1101 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to the embodiments of the present disclosure.

In the RAM1103, various programs and data necessary for the operation of the electronic device 1100 are stored. The processor 1101, the ROM 1102, and the RAM1103 are connected to each other by a bus 1104. The processor 1101 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1102 and/or the RAM 1103. It is to be noted that the programs may also be stored in one or more memories other than the ROM 1102 and the RAM 1103. The processor 1101 may also perform various operations of the method flows according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 1100 may also include input/output (I/O) interface 1105, input/output (I/O) interface 1105 also connected to bus 1104, according to an embodiment of the disclosure. Electronic device 1100 may also include one or more of the following components connected to I/O interface 1105: an input section 1106 including a keyboard, mouse, etc. Including an output portion 1107 such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), or the like, as well as a speaker or the like. A storage section 1108 including a hard disk and the like. And a communication section 1109 including a network interface card such as a LAN card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. A driver 1110 is also connected to the I/O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 1102 and/or the RAM1103 and/or one or more memories other than the ROM 1102 and the RAM1103 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the item recommendation method provided by the embodiment of the disclosure.

The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 1101. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication part 1109, and/or installed from the removable medium 1111. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. The computer program, when executed by the processor 1101, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A method of processing a points file, comprising:

acquiring point files of M channels in N channels, wherein the N channels comprise channels for transacting services for a user, the point files comprise user data generated in response to the services transacted by the user, N, M are integers greater than or equal to 1 respectively, and M is less than or equal to N;

processing a score file of each channel in the M channels based on a preset configuration file, wherein each channel corresponds to at least one configuration file, and the configuration files comprise processing logic for the user data;

and obtaining the user scores according to the processing result corresponding to each channel, wherein the processing result is obtained by executing the processing logic on the user data in the score file.

2. The method of claim 1, wherein the profile comprises a parsing profile, the processing logic comprises parsing logic, and the processing the score file for each of the M channels based on the preset profile comprises:

determining the corresponding analysis configuration file based on the channel identification of each channel;

and executing analysis logic in the analysis configuration file, and analyzing the integral file of each channel, wherein the analysis logic is used for converting the user data into user data in a preset format.

3. The method of claim 2, wherein the parsing configuration file includes X parsing units, the executing parsing logic in the parsing configuration file, and parsing the point file of each channel includes executing the X parsing units, which specifically includes:

determining X first target fields from the user data, wherein X is an integer greater than or equal to 1;

and correspondingly mapping the values of the X first target fields to an X column in a first database table, wherein the mapped X column comprises the user data in the preset format, and the X analysis units comprise X analysis logics corresponding to the X column.

4. The method of claim 3, wherein the method further comprises:

executing a preset storage process, and merging at least two first database tables with an association relationship into a second database table, wherein the association relationship is determined by the relationship between the channels in the N channels.

5. The method of claim 2, wherein the profile comprises a flow profile, the processing logic comprises flow logic, and the processing the score file for each of the M channels based on the preset profile further comprises:

determining the corresponding process configuration file based on the channel identification of each channel;

and executing the flow logic in the flow configuration file, and processing the analyzed integral file of each channel, wherein the flow logic is used for processing the user data in the preset format to obtain the processing result.

6. The method of claim 5, wherein the process profile includes Y sets of process units, the executing the process logic in the process profile, and the processing the parsed per-channel score file includes:

sequentially executing each flow unit set in the Y flow unit sets based on the preset sequence of the Y flow unit sets, wherein each flow unit set corresponds to each processing flow for obtaining the processing result;

wherein each flow unit set comprises Z flow units, each flow unit in the Z flow units corresponds to each sub-process flow in each process flow, each flow unit comprises at least one flow logic, and Y, Z is an integer greater than or equal to 1.

7. The method of claim 5, wherein said obtaining the user points according to the processing result corresponding to each channel comprises:

determining a second target field from the analyzed integral file of each channel;

querying a corresponding configuration condition based on the second target field, wherein the configuration condition is used for determining a calculation condition for obtaining the user point;

and obtaining the user points according to the configuration conditions and the processing result.

8. The method according to claim 7, wherein, before querying the corresponding configuration condition based on the second target field, the method further includes presetting the configuration condition, specifically including:

presetting at least one parameter identifier in a third database table;

and presetting at least one scene field corresponding to each parameter identifier in the at least one parameter identifier in a fourth database table.

9. The method of claim 8, wherein the querying for a corresponding configuration condition based on the second target field comprises:

determining a target parameter identifier based on the at least one parameter identifier in the third database table;

determining at least one scene field corresponding to the target parameter identifier from the fourth database table;

matching the second target field with the at least one scene field to obtain a matching result;

and determining corresponding configuration conditions based on the matching result.

10. The method of claim 1, comprising:

distributing N channel identifications for the N channels;

setting a file storage path corresponding to each channel identifier and an integral file identifier;

wherein, the obtaining of the score files of M channels in the N channels comprises:

scanning a file storage path corresponding to each channel identifier to acquire at least one storage file;

determining the score file from the at least one stored file based on the score file identification.

11. A point file processing apparatus comprising:

the system comprises a file acquisition module, a file acquisition module and a service management module, wherein the file acquisition module is used for acquiring score files of M channels in N channels, the N channels comprise channels for transacting services for a user, the score files comprise user data generated in response to the services transacted by the user, and N, M are integers greater than or equal to 1 respectively;

the file processing module is used for processing the integral file of each channel in the M channels based on a preset configuration file, wherein each channel corresponds to at least one configuration file, and the configuration files comprise processing logic for the user data;

and the point acquisition module is used for acquiring the user points according to the processing result corresponding to each channel, wherein the processing result is acquired by executing the processing logic on the user data in the point file.

12. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-10.

13. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 10.

14. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 10.