CN110674100A

CN110674100A - User demand prediction method and framework based on full-channel operation data

Info

Publication number: CN110674100A
Application number: CN201910928706.7A
Authority: CN
Inventors: 李虎; 曾毅峰; 王之良; 徐飞
Original assignee: Shanghai Pudong Development Bank Co Ltd Credit Card Center
Current assignee: Shanghai Pudong Development Bank Co Ltd Credit Card Center
Priority date: 2019-09-28
Filing date: 2019-09-28
Publication date: 2020-01-10
Anticipated expiration: 2039-09-28
Also published as: CN110674100B

Abstract

The invention relates to a user demand prediction method and a framework based on full-channel operation data, wherein the method comprises the following steps: step 1: data collection, namely acquiring original operation data of a user from all systems of a whole channel and storing the original operation data in a big data frame file system; step 2: data processing, namely, constructing user portrait data after data cleaning and processing the acquired user original operation data in each system of the whole channel; and step 3: establishing a machine learning model, dividing user image data into a data set for machine learning model training and a test data set for machine learning model verification, and then training and verifying the machine learning model by using the test data set and the training data set to obtain a finally trained and verified machine learning model; and 4, step 4: and recommending the service, namely performing model evaluation on the final machine learning model, predicting the corresponding requirement of the original operation data of the user by using the model evaluation, and recommending the service. Compared with the prior art, the method is more suitable for large-capacity data and has good user experience.

Description

User demand prediction method and framework based on full-channel operation data

Technical Field

The invention relates to the technical field of computers, in particular to a user demand prediction method and a user demand prediction framework based on full-channel operation data.

Background

With the maturity of new technology, new and advanced applications come from the fusion of 5G, artificial intelligence and the Internet of things in the future, and an intelligent connected world is created, which affects all individuals, industries, society and economy. Among them, the emergence of artificial intelligence has brought a new era for the great potential of mobile applications. For several years, mobile application developers have employed artificial intelligence in their innovations. Such as Siri from apple inc. Machine learning is developing rapidly and users now need a flexible algorithm to enhance the experience.

After 2013, the explosion development of internet finance pushes big data to a new climax. At present, due to the vigorous development of internet finance and consumption finance, the traditional customer operation mode of banking industry is tested seriously, and a new customer operation means is urgently needed. The traditional customer operation mode has the following defects:

(1) the information coverage is not enough: at present, the number of natural people collected by a personal credit investigation center in the central row reaches 8.6 hundred million people, but only 3 hundred million people have credit records;

(2) the information validity is insufficient: credit records mainly come from financial institutions such as commercial banks and rural credit agencies, and have serious shortcuts in data timeliness, comprehensiveness and hierarchy.

(3) The services are various, and the client cannot find the required services;

(4) the system is not intelligent enough;

(5) the customer experience result cannot be fed back in time.

Disclosure of Invention

The present invention aims to overcome the defects of the prior art and provide a user demand prediction method and architecture based on full-channel operation data.

The purpose of the invention can be realized by the following technical scheme:

a user demand prediction method based on full-channel operation data comprises the following steps:

step 1: acquiring original operation data of a user from each system of a whole channel, and storing the original operation data in a big data frame file system;

step 2: constructing user portrait data after data cleaning and processing the acquired user original operation data in each system of the whole channel;

and step 3: establishing a machine learning model, dividing user image data into a test data set for machine learning model verification and a training data set for machine learning model training, and then training and verifying the machine learning model by using the test data set and the training data set to obtain a finally trained and verified machine learning model;

and 4, step 4: and after model evaluation is carried out on the machine learning model which is finally trained and verified, the corresponding requirements of the original operation data of the user are predicted by using the model, and the service is recommended.

Further, the full channels in step 1 include an animation channel, an APP channel, a WeChat channel, an IVR (Interactive Voice Response) channel, a Payment service window channel, and a CSR (customer service Resepresenceive) artificial customer service channel.

Further, the user portrait data in step 2 includes customer basic information, customer transaction information, customer activity information, and customer buried point information.

Further, the method also comprises the step 5: and after receiving the service corresponding to the requirement, the user collects user feedback and trains and optimizes the machine learning model.

Further, the step 1 specifically includes: and (4) acquiring user original operation data from all the systems of the whole channel by using an ETL tool, and storing the user original operation data in a big data frame file system.

Further, the big data frame file system in step 1 adopts an HDFS file system.

Further, the indexes of the model evaluation in the step 4 include classification, regression, ranking, clustering, topical models and recommendation.

The invention also provides a framework of the user demand prediction method based on the full-channel operation data, and the framework comprises the following steps:

the operation data acquisition module is used for acquiring original data by using an ETL tool and storing the original data in an HDFS file system;

the characteristic engineering module is used for carrying out characteristic construction, characteristic extraction and characteristic selection from the original data;

the model training module is used for training the machine learning model;

the model verification module is used for verifying the machine learning model;

and the model application module is used for running the trained and verified machine learning model on line.

Compared with the prior art, the invention has the following advantages:

(1) the invention adopts Hadoop correlation technique; hadoop is a large data processing framework that can be used for storage and computing services from a single server cluster to thousands of server clusters. Hadoopdistributed File System (HDFS) provides a big data storage service that can span multiple computers, while MapReduce provides a framework for parallel processing. The HDFS file system is adopted for data storage, so that infinite capacity storage can be carried out theoretically, and the bottleneck that data storage of a traditional database is limited is solved. MapReduce provides a calculation solution for massive data.

(2) The invention adopts machine learning (deep learning) related technology; as shown in fig. 2, in the conventional programming, a person inputs rules (i.e., programs) and data to be processed according to the rules, and a system outputs answers; in the machine learning modeling process, people input data and answers expected from the data, and the system outputs rules or called models. These rules can then be applied to new data and cause the computer to generate answers autonomously, with the machine learning system being trained rather than explicitly programmed. Many examples relating to a task are input into the machine learning system, which finds statistical structures in these examples, and eventually finds rules to automate the task.

Drawings

FIG. 1 is a schematic diagram of an architecture for practicing the present invention;

FIG. 2 is a schematic diagram of the technical advantage of the present invention;

FIG. 3 is a flow chart of a method of the present invention;

FIG. 4 is a flowchart of a method according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.

Examples

Nowadays, banks are developing rapidly in informationization, a large amount of valuable business data and user data are accumulated, user requirements are mined by means of big data, appropriate products are recommended for users, operation benefits are increased under the condition that user disturbance and marketing cost are reduced, and the method is a key point for urgently promoting the business of credit cards of various banks.

The invention establishes a customer service recommendation system based on related concepts of interconnected finance, big data and machine learning, mainly comprises data acquisition, feature engineering, model training, model verification and model application, the overall structure is shown in figure 1,

the figure includes:

1. operational data collection

The original data is obtained from each system by using an ETL tool and is stored on an HDFS file system. The raw data needs to have the following characteristics:

a. have "representativeness".

b. For the classification problem, the data skew cannot be too severe, and the number of different classes of data does not have a difference of several orders of magnitude.

c. And estimating the memory consumption of the training model for the magnitude of the evaluation data, the number of samples and the number of characteristics. If the amount of data is too large, it may be considered to reduce training samples, reduce dimensions, or use a distributed machine learning system.

2. Feature engineering

The method comprises the steps of feature construction, feature extraction and feature selection from original data. The maximum effectiveness of original data can be exerted well by characteristic engineering, the effect and performance of an algorithm can be obviously improved, and the effect of a simple model is better than that of a complex model sometimes. Most of the time of data mining is spent on feature engineering, and is a very basic and necessary step for machine learning.

3. Model training

In machine learning, training is an important step and can directly influence the result of machine learning, and no matter which content needs a model in machine learning, a reasonable algorithm is selected for corresponding modeling, so that people can make the model better and more accurate.

4. Model validation

Through test data, the effectiveness of the model is verified, error samples are observed, and the reasons for error generation are analyzed, so that a breakthrough point for improving the performance of the algorithm can be found. The error analysis mainly analyzes error sources, data, characteristics and algorithms.

5. Model application

The success or failure of the model is directly determined by the on-line operation effect of the model. The method does not simply comprise the conditions of accuracy, errors and the like, and also comprises whether the running speed (time complexity), the resource consumption degree (space complexity) and the stability are acceptable.

The raw operation data of the customers in the whole channel are gathered by days, ① customer portrait (or called characteristic) data are obtained after simple data cleaning, processing and storing, reasonable sample data are selected from ① to construct a ② training data set and a ③ testing data set for machine learning of the data, and the training data set and the testing data set are crucial to the accuracy of model effect, and the relevant specific information in the graph is as follows:

the method mainly focuses on considering the demand direction of users in different channels, forms independent customer figures for each user by collecting user behavior information (mainly by buried point information collection) of the users in each channel, arranges scattered unstructured data information to form historical operation information of the users, combines the historical transaction behavior and other information of the users, calculates what services each customer may need in a future time period by using a model algorithm by establishing a model and a training model and taking the customer figures as input parameters, collects behavior information of the users after the customers contact through each channel, provides sufficient data samples for the evolution of the model in reaction to the training and optimization of the model, forms information closed loops of the users, channels and the model, and realizes quasi-real-time cyclic customer behavior prediction, the service which is needed most by the client is accurately predicted. As shown in particular in fig. 4.

The method and the architecture system have obvious benefits before and after production; the whole channel service demand prediction system is applied to the cartoon customer service channel, and the staged transaction amount is obviously increased compared with 1-5 months in 2018.

The method and the architecture system improve the hit rate before and after production, and have good user experience; the full-channel service demand prediction system is applied to the cartoon customer service channel, the calling times of the cartoon channel interface reach 1.2 ten thousand times every day, the hit rate is maintained at 74%, the calling times of the cartoon channel interface are obviously increased compared with the calling times of the cartoon channel interface before the cartoon channel is online in 2018 in 1-5 months, the viscosity of a user is improved, the user is more willing to select the cartoon channel to handle business when the user has service demands, and the user experience is good.

The method and the architecture system of the invention are used for personalized recommendation service for different customers; the personalized recommendation service adopted by the full-channel service demand prediction system recommends different service nodes aiming at different customers, and can meet different customer demands from person to person, and the recommended service nodes are the same for different customers before the system is on line, so that the demands of target customers cannot be met, the loss of the target customers is caused, and the loss of the customers can cause irreparable loss to a credit card center.

While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A user demand prediction method based on full-channel operation data is characterized by comprising the following steps:

2. The method according to claim 1, wherein the full channel in step 1 comprises an animation channel, an APP channel, a WeChat channel, an IVR channel, a Payment service Window channel and a CSR customer service channel.

3. The method of claim 1, wherein the user profile data in step 2 comprises customer basic information, customer transaction information, customer activity information and customer site information.

4. The method for predicting the demand of the user based on the channel-wide operation data as claimed in claim 1, further comprising the step 5: and after receiving the service corresponding to the requirement, the user collects user feedback and trains and optimizes the machine learning model.

5. The method for predicting the user demand based on the full channel operation data as claimed in claim 1, wherein the step 1 specifically comprises: and (4) acquiring user original operation data from all the systems of the whole channel by using an ETL tool, and storing the user original operation data in a big data frame file system.

6. The method for predicting user demand based on full channel operation data according to claim 1, wherein the big data frame file system in step 1 adopts an HDFS file system.

7. The method according to claim 1, wherein the model evaluation indexes in step 4 include classification, regression, ranking, clustering, topical models and recommendations.

8. An architecture based on the full-channel operation data based user demand prediction method according to any one of claims 1 to 7, characterized in that the architecture comprises:

the model training module is used for training the machine learning model;

the model verification module is used for verifying the machine learning model;