CN107944059A - A kind of user behavior analysis method and system based on stream calculation - Google Patents
A kind of user behavior analysis method and system based on stream calculation Download PDFInfo
- Publication number
- CN107944059A CN107944059A CN201711477395.4A CN201711477395A CN107944059A CN 107944059 A CN107944059 A CN 107944059A CN 201711477395 A CN201711477395 A CN 201711477395A CN 107944059 A CN107944059 A CN 107944059A
- Authority
- CN
- China
- Prior art keywords
- user
- analysis
- data
- user behavior
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- Accounting & Taxation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Game Theory and Decision Science (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
An embodiment of the present invention provides a kind of user behavior analysis method and system based on stream calculation, are used for realization the real-time analysis of user behavior, improve the accuracy of user behavior analysis.Present invention method includes：User behavior data is gathered from least two channels, and the JSON data packets of unified form are converted to, the user behavior data includes user behavior path, time index, user base label data, transaction data, active retention, click behavior, Platform Type；Active user behavioural analysis carries out the user behavior data using Spark Streaming stream calculations engines, the user behavior analysis includes event analysis, user's retention analysis, funnel analysis, user behavior path analysis, transaction analysis, user's portrait and clicks on analysis；User behavior analysis is stored as a result, and showing corresponding analysis result according to the inquiry request of user.
Description
Technical field
The present invention relates to big data processing technology field, and in particular to a kind of user behavior analysis method based on stream calculation
And system.
Background technology
User behavior analysis, refers to be collected all data during using the product by the user, arranges, counts, point
The rule used the product by the user is analysed, strong data supporting is provided for the follow-up developments of product, optimization or marketing.In information explosion
Epoch, information increases with the speed of geometry level daily so that the major Internet firm of in the market all suffers from choosing for sternness
War.
In existing scheme, the collection for user behavior data is often single channel, such as single Web ends or list
Only App ends, it is relatively fewer for the user behavior data collection capacity of unique user, influence the accurate of user behavior analysis result
Property.Secondly, existing user behavior data engine analysis mechanism using under Hadoop framework Map/Reduce batch at
Reason mechanism, can not handle user behavior data in real time, causes the analysis result of user behavior data to have delay.
In view of this, it is necessary to propose a kind of new user behavior analysis method.
The content of the invention
An embodiment of the present invention provides a kind of user behavior analysis method and system based on stream calculation, user is used for realization
The real-time analysis of behavior, improves the accuracy of user behavior analysis.
First aspect of the embodiment of the present invention provides a kind of user behavior analysis method based on stream calculation, its feature exists
In, including：
User behavior data is gathered from least two channels, and is converted to the JSON data packets of unified form, the user
Behavioral data includes user behavior path, time index, user base label data, transaction data, active retention, click row
For, Platform Type；
Active user behavioural analysis is carried out to the user behavior data using Spark Streaming stream calculations engines,
The user behavior analysis include event analysis, user retain analysis, funnel analysis, user behavior path analysis, transaction analysis,
User draws a portrait and clicks on analysis；
User behavior analysis is stored as a result, and showing corresponding analysis result according to the inquiry request of user.
Optionally, the storage user behavior analysis according to the corresponding analysis of the inquiry request of user displaying as a result, and tie
Fruit includes：
The user behavior analysis result is saved into distributed caching and relational database；
According to the inquiry request of user, preferentially obtained from distributed caching and show corresponding analysis result data, if
Inquiry from relational database cluster less than then obtaining and show corresponding analysis result data from distributed caching.
Optionally, the method further includes：
Using the double message queue treatment mechanisms of distributed type open formula message system Rocket MQ to the JSON data packets into
Row transmission, double message queues include main message queue and are used for JSON data packets from message queue, the main message queue
Transmission, the JSON data packet retransmissions for being used to lose, be delayed or malfunction from message queue.
Optionally, user's portrait includes：
According to the event analysis, user behavior path analysis and transaction analysis, the user property number for corresponding to user is obtained
According to the user attribute data includes at least social property, life attribute, consumer behavior data；
The user attribute data and the user base label data associated storage are formed into user's portrait.
Optionally, the user base label data includes：User type, user's gender, age, user role, user
Grade, registration type, user are accessed in regional information, user access device type, App version informations and browser version information
It is one or more.
Second aspect of the embodiment of the present invention provides a kind of user behavior analysis system based on stream calculation, its feature exists
In, including：
Data acquisition module, for gathering user behavior data from least two channels, and is converted to unified form
JSON data packets, the user behavior data include user behavior path, time index, user base label data, number of deals
According to, active retain, click on behavior, Platform Type；
Spark Streaming stream calculation engines, for using Spark Streaming stream calculations mechanism to the user
Behavioral data carries out active user behavioural analysis, and the user behavior analysis includes event analysis, user retains analysis, funnel point
Analysis, user behavior path analysis, transaction analysis, user's portrait and click analysis；
Storage and display module, for storing user behavior analysis as a result, and showing correspondence according to the inquiry request of user
Analysis result.
Optionally, the storage is specifically included with display module：
Storage unit, for the user behavior analysis result to be saved into distributed caching and relational database；
Display unit, for the inquiry request according to user, preferentially obtains from distributed caching and shows corresponding point
Result data is analysed, if inquiry from relational database cluster less than obtaining and show corresponding analysis from distributed caching
Result data.
Optionally, which further includes：
Data transmission module, for using the double message queue treatment mechanisms of distributed type open formula message system Rocket MQ
The JSON data packets are transmitted, double message queues include main message queue and from message queue, the main message
Queue is transmitted for JSON data packets, the JSON data packet retransmissions for being used to lose, be delayed or malfunction from message queue.
Optionally, the Spark Streaming stream calculation engines include：
Data acquisition unit, for according to the event analysis, user behavior path analysis and transaction analysis, obtaining and corresponding to
The user attribute data of user, the user attribute data include at least social property, life attribute, consumer behavior data；
Associated storage unit, for by the user attribute data and the user base label data associated storage shape
Draw a portrait into user.
Optionally, the user base label data includes：User type, user's gender, age, user role, user
Grade, registration type, user are accessed in regional information, user access device type, App version informations and browser version information
It is one or more.
As can be seen from the above technical solutions, the embodiment of the present invention has the following advantages：
In the embodiment of the present invention, user behavior data can be gathered from multiple channel, and be converted into unified JSON
Formatted data bag can collect the more user behavior datas of unique user in order to handle, relative to existing scheme, favorably
In the accuracy for improving user behavior analysis result, Spark can be used for the user behavior data collected
Streaming stream calculations mechanism carries out active user behavioural analysis to user behavior data, reduces user behavior data analysis
As a result delay.
Brief description of the drawings
Fig. 1 is a kind of one embodiment signal of user behavior analysis method based on stream calculation in the embodiment of the present invention
Figure；
Fig. 2 is a kind of one embodiment signal of user behavior analysis system based on stream calculation in the embodiment of the present invention
Figure；
Fig. 3 is a kind of another embodiment signal of user behavior analysis system based on stream calculation in the embodiment of the present invention
Figure；
Fig. 4 is a kind of another embodiment signal of user behavior analysis system based on stream calculation in the embodiment of the present invention
Figure.
Embodiment
An embodiment of the present invention provides a kind of user behavior analysis method and system based on stream calculation, user is used for realization
The real-time analysis of behavior, improves the accuracy of user behavior analysis.
In order to make those skilled in the art more fully understand the present invention program, below in conjunction with the embodiment of the present invention
Attached drawing, is clearly and completely described the technical solution in the embodiment of the present invention, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
Member's all other embodiments obtained without making creative work, should all belong to the model that the present invention protects
Enclose.
Term " comprising " and " having " in description and claims of this specification and above-mentioned attached drawing and they
Any deformation, it is intended that cover it is non-exclusive include, for example, containing the process of series of steps or unit, method, being
System, product or equipment are not necessarily limited to those steps or the unit clearly listed, but may include not list clearly or
For the intrinsic other steps of these processes, method, product or equipment or unit.
In order to make it easy to understand, the idiographic flow in the embodiment of the present invention is described below, referring to Fig. 1, of the invention
A kind of one embodiment of the user behavior analysis method based on stream calculation may include in embodiment：
101st, user behavior data is gathered from least two channels, and is converted to the JSON data packets of unified form；
In order to gather the more user behavior datas of unique user, system can be acquired user behavior from multiple support channels
Data, for example, the corresponding user behavior in one or more Web ends and one or more App ends that can be associated from user
Data, specific user behavior data can include but is not limited to user behavior path, time index, user base number of tags
According to, transaction data, it is active retain, click on behavior, Platform Type, specific user behavior data type can be according to the need of user
Ask and reasonably set, do not limited herein specifically.Further, data processing is carried out for the ease of system, will can adopted
The data collected are converted to the JSON data packets of unified form, and specific JSON data formats are no longer superfluous herein for the prior art
State.
102nd, active user behavioural analysis is carried out to user behavior data using Spark Streaming stream calculations engines；
After the behavioral data of user is collected, system can use Spark Streaming stream calculations engines according to
Preset analysis model analyzes user behavior, and specific user behavior, which carries out analysis, can include event analysis, user
Retain analysis, funnel analysis, user behavior path analysis, transaction analysis, user's portrait and click on analysis.
Wherein, event be tracking or record user behavior or business procedure (specifically may include as registered, logging in, thumbing up,
Comment, concern etc.).Buried a little by predeterminable event, user behavior data when triggering such event can be recorded.Event analysis
Be event is subdivided into browse, gently interaction, again interaction, merchandise four major classes, by event alternate analysis, see clearly event correlation,
So as to optimize product function.By taking customer registration affair as an example, specifically it may include：
The 1st, event behavior is set：Click on user's registration button；
The 2nd, event argument and property value are set：Event argument, user's registration information (user name, phone number, mailbox, property
Not, age, password etc.), user sources IP information, access equipment type, App version informations, browser version information etc.；Event
Property value：The corresponding parameters/properties key-value pair data of above-mentioned parameter；
3rd, customer registration affair is set to bury a little：Web can be embedded in JS track of issues codes, and App can be embedded in sdk, integrate
In being applied to App；
4th, event triggering result data is recorded.When customer registration affair triggers, system records user log-on message, user comes
Source IP information, access equipment type, App version informations, browser version information etc., analysis event result data can be combined with
User retains analysis, funnel analysis etc., counting user registration conversion data.
User retains analysis and refers to that analyzing user is converted into any active ues from the volatile user at initial stage, stablizes user, loyalty
The overall process of real user, supports User Defined to retain index, next day can be supported to retain on time dimension, retains within 3rd, seven
Day retains, and January retains, and March retains analysis etc..
Next day retains：New user's ratio that next day reuses after the first use.When concern next day retention can be with first
Between find product up-gradation after experience effect；Retain within 3rd：The ratio that new user reuses after three days after the first use；Seven days
Retain：The ratio that new user reuses after seven days after the first use.Concern is retained on the 7th can analyze user one completely
The retention situation in cycle on probation；January retains：Retention in January Statistical Criteria and so on.Analyzing moon retention can be with analysis product liter
Stability after level；
March retains：Retention in March Statistical Criteria and so on.The analysis moon retains the stabilization after can upgrading with analysis product
Property.
Funnel analysis can support User Defined funnel, it is possible to achieve the funnel model of arbitrary act, while support have
Sequence, the switching of unordered funnel so that funnel analysis is suitable for a variety of conversion scenes, and passage time, event dimension are further analyzed
The difference of different funnel conversion ratios.Funnel analysis can analyze conversion and loss of the user behavior path in each step, to being lost in
Refinement multi dimensional analysis is carried out compared with multipath, finds out leak source lifting conversion.For example, funnel may include the step of analysis：
The 1st, funnel name is set, such as：Homepage registration conversion；
The 2nd, funnel type is set, according to business needs, funnel type is set for orderly funnel or the (leakage in order of unordered funnel
Bucket：Order of occurrence between the multiple steps of considered critical funnel；Unordered funnel：Event occurs between not limiting the multiple steps of funnel
Order)；
The 3rd, funnel step is set, each step corresponds to an index (browsing pages, trigger event).With " homepage registration turns
Exemplified by this funnel of change ", funnel step is set to may include：Setting browsing pages are website homepage, and setting trigger event is user
Registration；
4th, preserve funnel and analyzed, by taking " homepage registration conversion " this funnel as an example, it is assumed that there are 100 people to access certain electricity
Business website, has 30 people to click on registration, has 10 people to succeed in registration.This process shares three steps, the conversion ratio of the first step to second step
For 30%, turnover rate 70%, second step to the 3rd step conversion ratio is 33%, turnover rate 67%；The conversion ratio of whole process is
10%, turnover rate 90%.
User behavior path analysis refers to gather and analyzes the behavior of user path, and setting conversion target, intellectual analysis is completed
Object event conversion pathway combines, and conversion path data is intuitively shown, easily understands user's conversion pathway.
Transaction analysis can include order analyzing, shopping cart analysis, commercial analysis, Shopping Behaviors analysis etc., effectively facilitate
Conversion ratio is lifted.
User's portrait can specifically include：
According to event analysis, user behavior path analysis and transaction analysis, the user attribute data of corresponding user is obtained, is used
Family attribute data includes at least social property, life attribute, consumer behavior data；
User attribute data and user base label data associated storage are formed into user's portrait.
Optionally, user base label data includes：User type, user's gender, the age, user role, user gradation,
Registration type, user access one in regional information, user access device type, App version informations and browser version information
Item is multinomial.
Click on analysis and interbehavior, the support linking point such as refer to gather mouse rollovers screen comprehensively, browse, click on, stopping
Hit, the page is clicked on, browsed, analysis is clicked in the subdivision of split screen, notice various dimensions, interaction of the accurate and visual displaying user on the page
Behavior.
It is understood that above-mentioned specific user behavior is analyzed, for example, event analysis, user retain analysis, leakage
Bucket analysis, user behavior path analysis, transaction analysis, user's portrait and click analysis etc. can be closed according to the demand of user
The configuration of reason, does not limit specifically herein.
103rd, user behavior analysis is stored as a result, and showing corresponding analysis result according to the inquiry request of user.
User behavior is divided in real time according to preset analysis model by Spark Streaming stream calculations engines
After analysis, system can store user behavior analysis as a result, and showing corresponding analysis result, use according to the inquiry request of user
Family can greatly reducing the delay of data analysis with the corresponding real-time analysis result of real time inspection.
Optionally, as a kind of possible embodiment, in the embodiment of the present invention, specific implementation storage user behavior analysis
As a result and show that the process of corresponding analysis result may include：
User behavior analysis result is saved into distributed caching and relational database；According to the inquiry request of user,
Preferentially obtained from distributed caching and show corresponding analysis result data, if from distributed caching inquiry less than, from
Obtained in relational database cluster and show corresponding analysis result data.
Specifically, distributed caching Redis and MySQL can be used in order to improve the inquiry of system and displaying response speed
Cluster secondary storage mechanism preserves user behavior analysis result data.User behavior is inquired about provides Web, App channel with displaying
Query interface and interface, visualize user behavior analysis data.
In the embodiment of the present invention, user behavior data can be gathered from multiple channel, and be converted into unified JSON
Formatted data bag can collect the more user behavior datas of unique user in order to handle, relative to existing scheme, favorably
In the accuracy for improving user behavior analysis result, Spark can be used for the user behavior data collected
Streaming stream calculations mechanism carries out active user behavioural analysis to user behavior data, reduces user behavior data analysis
As a result delay.
On the basis of the embodiment shown in above-mentioned Fig. 1, since the data employed in the embodiment of the present invention by all kinds of means are adopted
Collection mechanism, in order to ensure the reliability of data transfer, it is necessary to which the transmitting procedure of the data to collecting optimizes, optional
, as a kind of possible embodiment, in the embodiment of the present invention, system can use distributed type open formula message system
The double message queue treatment mechanisms of Rocket MQ are transmitted JSON data packets, double message queues include main message queue and from
Message queue, main message queue are transmitted for JSON data packets, the JSON data for being used to lose, be delayed or malfunction from message queue
The double message queue treatment mechanisms of packet retransmission, wherein distributed type open formula message system Rocket MQ are the prior art, herein no longer
Repeat.
Above-described embodiment retouches a kind of user behavior analysis method based on stream calculation in the embodiment of the present invention
State, a kind of user behavior analysis system based on stream calculation in the embodiment of the present invention will be described below, refer to figure
2, in the embodiment of the present invention, a kind of one embodiment of the user behavior analysis system based on stream calculation may include：
Data acquisition module 201, for gathering user behavior data from least two channels, and is converted to unified form
JSON data packets, user behavior data include user behavior path, time index, user base label data, transaction data, work
Jump is retained, clicks on behavior, Platform Type；
Spark Streaming stream calculations engine 202, for using Spark Streaming stream calculation mechanism to user
Behavioral data carries out active user behavioural analysis, user behavior analysis includes event analysis, user retains analysis, funnel is analyzed,
User behavior path analysis, transaction analysis, user's portrait and click analysis；
Storage and display module 203, for storing user behavior analysis as a result, and according to the displaying pair of the inquiry request of user
The analysis result answered.
Optionally, referring to Fig. 3, as a kind of possible embodiment, in the embodiment of the present invention, storage and display module
203 specifically include：
Storage unit 2031, for user behavior analysis result to be saved into distributed caching and relational database；
Display unit 2032, for the inquiry request according to user, preferentially obtains from distributed caching and shows correspondence
Analysis result data, if inquiry from relational database cluster less than obtaining and show corresponding from distributed caching
Analysis result data.
Optionally, as a kind of possible embodiment, in the embodiment of the present invention, which further includes：
Data transmission module 204, for using the double message queue processors of distributed type open formula message system Rocket MQ
System is transmitted JSON data packets, and double message queues include main message queue and are used for from message queue, main message queue
JSON data packets are transmitted, the JSON data packet retransmissions for being used to lose, be delayed or malfunction from message queue.
Optionally, referring to Fig. 4, as a kind of possible embodiment, in the embodiment of the present invention, Spark
Streaming stream calculations engine 202 can specifically include：
Data acquisition unit 2021, for according to event analysis, user behavior path analysis and transaction analysis, obtaining and corresponding to
The user attribute data of user, user attribute data include at least social property, life attribute, consumer behavior data；
Associated storage unit 2022, is used for user attribute data and user base label data associated storage to be formed
Draw a portrait at family.
Optionally, user base label data includes：User type, user's gender, the age, user role, user gradation,
Registration type, user access one in regional information, user access device type, App version informations and browser version information
Item is multinomial.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, may be referred to the corresponding process in preceding method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed system, apparatus and method can be with
Realize by another way.For example, device embodiment described above is only schematical, for example, the unit
Division, is only a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing
Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be the indirect coupling by some interfaces, device or unit
Close or communicate to connect, can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separate, be shown as unit
The component shown may or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
In network unit.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also
That unit is individually physically present, can also two or more units integrate in a unit.Above-mentioned integrated list
Member can both be realized in the form of hardware, can also be realized in the form of SFU software functional unit.
If the integrated unit is realized in the form of SFU software functional unit and is used as independent production marketing or use
When, it can be stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially
The part to contribute in other words to the prior art or all or part of the technical solution can be in the form of software products
Embody, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server, or network equipment etc.) performs the complete of each embodiment the method for the present invention
Portion or part steps.And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can store journey
The medium of sequence code.
The above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although with reference to before
Embodiment is stated the present invention is described in detail, it will be understood by those of ordinary skill in the art that：It still can be to preceding
State the technical solution described in each embodiment to modify, or equivalent substitution is carried out to which part technical characteristic；And these
Modification is replaced, and the essence of appropriate technical solution is departed from the spirit and scope of various embodiments of the present invention technical solution.
Claims (10)
- A kind of 1. user behavior analysis method based on stream calculation, it is characterised in that including：User behavior data is gathered from least two channels, and is converted to the JSON data packets of unified form, the user behavior Data include user behavior path, time index, user base label data, transaction data, active retain, click on behavior, is flat Platform type；Active user behavioural analysis is carried out to the user behavior data using Spark Streaming stream calculations engines, it is described User behavior analysis includes event analysis, user retains analysis, funnel analysis, user behavior path analysis, transaction analysis, user Portrait and click analysis；User behavior analysis is stored as a result, and showing corresponding analysis result according to the inquiry request of user.
- 2. according to the method described in claim 1, it is characterized in that, the storage user behavior analysis is as a result, and according to user Inquiry request show that corresponding analysis result includes：The user behavior analysis result is stored in distributed caching and relational database；According to the inquiry request of user, preferentially obtained from distributed caching and show corresponding analysis result data, if from point Inquiry from relational database cluster less than then obtaining and show corresponding analysis result data in cloth caching.
- 3. according to the method described in claim 2, it is characterized in that, further include：The JSON data packets are passed using distributed type open formula message system Rocket MQ double message queue treatment mechanisms Defeated, double message queues include main message queue and are transmitted from message queue, the main message queue for JSON data packets, The JSON data packet retransmissions for being used to lose, be delayed or malfunction from message queue.
- 4. according to the method in any one of claims 1 to 3, it is characterised in that user's portrait includes：According to the event analysis, user behavior path analysis and transaction analysis, the user attribute data for corresponding to user, institute are obtained User attribute data is stated including at least social property, life attribute, consumer behavior data；The user attribute data and the user base label data associated storage are formed into user's portrait.
- 5. according to the method described in claim 4, it is characterized in that, the user base label data includes：User type, use Family gender, age, user role, user gradation, registration type, user access regional information, user access device type, App It is one or more in version information and browser version information.
- A kind of 6. user behavior analysis system based on stream calculation, it is characterised in that including：Data acquisition module, for gathering user behavior data from least two channels, and is converted to the JSON numbers of unified form According to bag, the user behavior data includes user behavior path, time index, user base label data, transaction data, active Retain, click on behavior, Platform Type；Spark Streaming stream calculation engines, for using Spark Streaming stream calculations mechanism to the user behavior Data carry out active user behavioural analysis, the user behavior analysis includes event analysis, user retains analysis, funnel is analyzed, User behavior path analysis, transaction analysis, user's portrait and click analysis；Storage and display module, for storing user behavior analysis as a result, and showing corresponding point according to the inquiry request of user Analyse result.
- 7. system according to claim 6, it is characterised in that the storage is specifically included with display module：Storage unit, for the user behavior analysis result to be stored in distributed caching and relational database；Display unit, for the inquiry request according to user, preferentially obtains from distributed caching and shows corresponding analysis knot Fruit data, if inquiry from relational database cluster less than obtaining and show corresponding analysis result from distributed caching Data.
- 8. system according to claim 7, it is characterised in that further include：Data transmission module, for using the double message queue treatment mechanisms of distributed type open formula message system Rocket MQ to institute State JSON data packets to be transmitted, double message queues include main message queue and from message queue, the main message queue Transmitted for JSON data packets, the JSON data packet retransmissions for being used to lose, be delayed or malfunction from message queue.
- 9. the system according to any one of claim 6 to 8, it is characterised in that the SparkStreaming stream calculations Engine includes：Data acquisition unit, for according to the event analysis, user behavior path analysis and transaction analysis, obtaining corresponding user User attribute data, the user attribute data include at least social property, life attribute, consumer behavior data；Associated storage unit, is used for the user attribute data and the user base label data associated storage to be formed Draw a portrait at family.
- 10. system according to claim 9, it is characterised in that the user base label data includes：User type, User's gender, the age, user role, user gradation, registration type, user access regional information, user access device type, It is one or more in App version informations and browser version information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477395.4A CN107944059A (en) | 2017-12-29 | 2017-12-29 | A kind of user behavior analysis method and system based on stream calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477395.4A CN107944059A (en) | 2017-12-29 | 2017-12-29 | A kind of user behavior analysis method and system based on stream calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107944059A true CN107944059A (en) | 2018-04-20 |
Family
ID=61937940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711477395.4A Pending CN107944059A (en) | 2017-12-29 | 2017-12-29 | A kind of user behavior analysis method and system based on stream calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107944059A (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595562A (en) * | 2018-04-12 | 2018-09-28 | 西安邮电大学 | User's evaluation data analysing method based on accurate sex determination |
| CN109033355A (en) * | 2018-07-25 | 2018-12-18 | 北京易观智库网络科技有限公司 | Carry out the method, apparatus and storage medium of funnel analysis |
| CN109379330A (en) * | 2018-09-07 | 2019-02-22 | 阿里巴巴集团控股有限公司 | The implementation method and device that content is launched |
| CN109408559A (en) * | 2018-10-09 | 2019-03-01 | 北京易观智库网络科技有限公司 | Retain the method, apparatus and storage medium of analysis |
| CN109451423A (en) * | 2018-11-14 | 2019-03-08 | 重庆雾都科技有限公司 | A kind of intelligent box of accurate marketing |
| CN109683494A (en) * | 2018-11-30 | 2019-04-26 | 上海五零盛同信息科技有限公司 | Internet of things equipment coordinated control system |
| CN109710876A (en) * | 2018-12-26 | 2019-05-03 | 咪咕文化科技有限公司 | A kind of information recommendation method and device, computer readable storage medium |
| CN109783512A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
| CN110083459A (en) * | 2019-03-16 | 2019-08-02 | 平安城市建设科技(深圳)有限公司 | The data in cross-page face bury point methods, device, equipment and storage medium |
| CN110119416A (en) * | 2019-05-16 | 2019-08-13 | 重庆八戒传媒有限公司 | A kind of service data analysis system and method |
| CN110389974A (en) * | 2019-07-30 | 2019-10-29 | 中原银行股份有限公司 | Data analysing method and system |
| CN110781061A (en) * | 2019-09-20 | 2020-02-11 | 平安科技（深圳）有限公司 | Method and device for recording user behavior link |
| WO2020037917A1 (en) * | 2018-08-22 | 2020-02-27 | 平安科技（深圳）有限公司 | User behavior data recommendation method, server and computer readable medium |
| CN110990473A (en) * | 2019-11-28 | 2020-04-10 | 北京海益同展信息科技有限公司 | Tag data processing system and method |
| CN111126737A (en) * | 2018-11-01 | 2020-05-08 | 百度在线网络技术（北京）有限公司 | Cross-scene cross analysis method and device, electronic equipment and storage medium |
| CN111127059A (en) * | 2018-10-31 | 2020-05-08 | 北京国双科技有限公司 | User quality analysis method and device |
| CN111291042A (en) * | 2019-12-23 | 2020-06-16 | 创意信息技术股份有限公司 | Power data processing system and method for power supply service |
| CN111311326A (en) * | 2020-02-18 | 2020-06-19 | 平安科技（深圳）有限公司 | User behavior real-time multidimensional analysis method and device and storage medium |
| CN111382183A (en) * | 2018-12-29 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data query method and device |
| CN111694874A (en) * | 2020-06-17 | 2020-09-22 | 科技谷（厦门）信息技术有限公司 | User behavior analysis system based on big data platform |
| CN111767201A (en) * | 2020-06-29 | 2020-10-13 | 百度在线网络技术（北京）有限公司 | User behavior analysis method, terminal device, server and storage medium |
| CN111930508A (en) * | 2020-08-18 | 2020-11-13 | 上海浦东发展银行股份有限公司 | User behavior analysis platform system, method and storage medium based on behavior data |
| CN111949867A (en) * | 2020-08-10 | 2020-11-17 | 中国平安人寿保险股份有限公司 | Cross-APP user behavior analysis model training method, analysis method and related equipment |
| CN111966726A (en) * | 2020-07-22 | 2020-11-20 | 武汉极意网络科技有限公司 | System and method for generating self-adaptive data analysis report based on different types of clients |
| CN112016027A (en) * | 2020-08-20 | 2020-12-01 | 杭州小电科技股份有限公司 | Method and device for monitoring access data operation platform |
| CN112506733A (en) * | 2020-10-30 | 2021-03-16 | 福建亿能达信息技术股份有限公司 | Method, device, equipment and medium for finely analyzing user behavior data |
| CN113220657A (en) * | 2021-05-14 | 2021-08-06 | 上海哔哩哔哩科技有限公司 | Data processing method and device and computer equipment |
| CN113344604A (en) * | 2021-04-16 | 2021-09-03 | 广州迅捷微风信息科技有限公司 | User subdivision method based on user behavior data and flow calculation |
| CN113901325A (en) * | 2021-10-27 | 2022-01-07 | 首都信息发展股份有限公司 | User behavior analysis device and method |
| CN114092138A (en) * | 2021-11-10 | 2022-02-25 | 建信金融科技有限责任公司 | User behavior analysis method, device, equipment and storage medium |
| CN114610204A (en) * | 2022-03-14 | 2022-06-10 | 中国农业银行股份有限公司 | Auxiliary device and method for data processing, storage medium and electronic equipment |
| CN115408586A (en) * | 2022-08-25 | 2022-11-29 | 广东博成网络科技有限公司 | Intelligent channel operation data analysis method, system, equipment and storage medium |
| CN116187869A (en) * | 2023-04-28 | 2023-05-30 | 北京合力金智科技发展有限公司 | User behavior analysis method and system applied to government enterprise informatization system |
| CN116501778A (en) * | 2023-05-16 | 2023-07-28 | 湖北省珍岛数字智能科技有限公司 | Real-time user behavior data analysis method based on ClickHouse |
| CN117354356A (en) * | 2023-12-04 | 2024-01-05 | 四川才子软件信息网络有限公司 | APP region retention statistical method, system and equipment |
| CN111311326B (en) * | 2020-02-18 | 2024-05-28 | 平安科技（深圳）有限公司 | User behavior real-time multidimensional analysis method, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893628A (en) * | 2016-05-17 | 2016-08-24 | 中国农业银行股份有限公司 | Real-time data collection system and method |
| CN105912582A (en) * | 2016-03-31 | 2016-08-31 | 畅捷通信息技术股份有限公司 | Control method for users' behavior analyses and control system for users' behavior analyses |
| CN106504099A (en) * | 2015-09-07 | 2017-03-15 | 国家计算机网络与信息安全管理中心 | A kind of system for building user's portrait |
| CN107193643A (en) * | 2017-03-28 | 2017-09-22 | 中国科学院信息工程研究所 | A kind of highly reliable distributed traffic real-time statistical method and system |
| CN107480189A (en) * | 2017-07-11 | 2017-12-15 | 上海精数信息科技有限公司 | A kind of various dimensions real-time analyzer and method |
-
2017
- 2017-12-29 CN CN201711477395.4A patent/CN107944059A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106504099A (en) * | 2015-09-07 | 2017-03-15 | 国家计算机网络与信息安全管理中心 | A kind of system for building user's portrait |
| CN105912582A (en) * | 2016-03-31 | 2016-08-31 | 畅捷通信息技术股份有限公司 | Control method for users' behavior analyses and control system for users' behavior analyses |
| CN105893628A (en) * | 2016-05-17 | 2016-08-24 | 中国农业银行股份有限公司 | Real-time data collection system and method |
| CN107193643A (en) * | 2017-03-28 | 2017-09-22 | 中国科学院信息工程研究所 | A kind of highly reliable distributed traffic real-time statistical method and system |
| CN107480189A (en) * | 2017-07-11 | 2017-12-15 | 上海精数信息科技有限公司 | A kind of various dimensions real-time analyzer and method |
Non-Patent Citations (1)
| Title |
|---|
| GUOHECANG: ""Spark Streaming实践和优化"", 《HTTPS://BLOG.CSDN.NET/GUOHECANG/ARTICLE/DETAILS/51583214》 * |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595562A (en) * | 2018-04-12 | 2018-09-28 | 西安邮电大学 | User's evaluation data analysing method based on accurate sex determination |
| CN109033355A (en) * | 2018-07-25 | 2018-12-18 | 北京易观智库网络科技有限公司 | Carry out the method, apparatus and storage medium of funnel analysis |
| WO2020037917A1 (en) * | 2018-08-22 | 2020-02-27 | 平安科技（深圳）有限公司 | User behavior data recommendation method, server and computer readable medium |
| CN109379330A (en) * | 2018-09-07 | 2019-02-22 | 阿里巴巴集团控股有限公司 | The implementation method and device that content is launched |
| CN109408559A (en) * | 2018-10-09 | 2019-03-01 | 北京易观智库网络科技有限公司 | Retain the method, apparatus and storage medium of analysis |
| CN111127059B (en) * | 2018-10-31 | 2023-04-18 | 北京国双科技有限公司 | User quality analysis method and device |
| CN111127059A (en) * | 2018-10-31 | 2020-05-08 | 北京国双科技有限公司 | User quality analysis method and device |
| CN111126737B (en) * | 2018-11-01 | 2022-09-13 | 百度在线网络技术（北京）有限公司 | Cross-scene cross analysis method and device, electronic equipment and storage medium |
| CN111126737A (en) * | 2018-11-01 | 2020-05-08 | 百度在线网络技术（北京）有限公司 | Cross-scene cross analysis method and device, electronic equipment and storage medium |
| CN109451423A (en) * | 2018-11-14 | 2019-03-08 | 重庆雾都科技有限公司 | A kind of intelligent box of accurate marketing |
| CN109683494A (en) * | 2018-11-30 | 2019-04-26 | 上海五零盛同信息科技有限公司 | Internet of things equipment coordinated control system |
| CN109783512A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
| CN109710876A (en) * | 2018-12-26 | 2019-05-03 | 咪咕文化科技有限公司 | A kind of information recommendation method and device, computer readable storage medium |
| CN111382183B (en) * | 2018-12-29 | 2023-06-27 | 阿里巴巴集团控股有限公司 | Data query method and device |
| CN111382183A (en) * | 2018-12-29 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data query method and device |
| CN110083459A (en) * | 2019-03-16 | 2019-08-02 | 平安城市建设科技(深圳)有限公司 | The data in cross-page face bury point methods, device, equipment and storage medium |
| CN110119416A (en) * | 2019-05-16 | 2019-08-13 | 重庆八戒传媒有限公司 | A kind of service data analysis system and method |
| CN110389974A (en) * | 2019-07-30 | 2019-10-29 | 中原银行股份有限公司 | Data analysing method and system |
| CN110781061A (en) * | 2019-09-20 | 2020-02-11 | 平安科技（深圳）有限公司 | Method and device for recording user behavior link |
| CN110781061B (en) * | 2019-09-20 | 2022-11-25 | 平安科技（深圳）有限公司 | Method and device for recording user behavior link |
| CN110990473B (en) * | 2019-11-28 | 2023-11-03 | 京东科技信息技术有限公司 | Tag data processing system and method |
| CN110990473A (en) * | 2019-11-28 | 2020-04-10 | 北京海益同展信息科技有限公司 | Tag data processing system and method |
| CN111291042A (en) * | 2019-12-23 | 2020-06-16 | 创意信息技术股份有限公司 | Power data processing system and method for power supply service |
| WO2021164253A1 (en) * | 2020-02-18 | 2021-08-26 | 平安科技（深圳）有限公司 | Method and device for real-time multidimensional analysis of user behaviors, and storage medium |
| CN111311326B (en) * | 2020-02-18 | 2024-05-28 | 平安科技（深圳）有限公司 | User behavior real-time multidimensional analysis method, device and storage medium |
| CN111311326A (en) * | 2020-02-18 | 2020-06-19 | 平安科技（深圳）有限公司 | User behavior real-time multidimensional analysis method and device and storage medium |
| CN111694874A (en) * | 2020-06-17 | 2020-09-22 | 科技谷（厦门）信息技术有限公司 | User behavior analysis system based on big data platform |
| CN111767201B (en) * | 2020-06-29 | 2023-08-29 | 百度在线网络技术（北京）有限公司 | User behavior analysis method, terminal device, server and storage medium |
| CN111767201A (en) * | 2020-06-29 | 2020-10-13 | 百度在线网络技术（北京）有限公司 | User behavior analysis method, terminal device, server and storage medium |
| CN111966726A (en) * | 2020-07-22 | 2020-11-20 | 武汉极意网络科技有限公司 | System and method for generating self-adaptive data analysis report based on different types of clients |
| CN111966726B (en) * | 2020-07-22 | 2023-09-26 | 武汉极意网络科技有限公司 | System and method for generating self-adaptive data analysis report based on different types of clients |
| CN111949867A (en) * | 2020-08-10 | 2020-11-17 | 中国平安人寿保险股份有限公司 | Cross-APP user behavior analysis model training method, analysis method and related equipment |
| CN111930508B (en) * | 2020-08-18 | 2022-08-12 | 上海浦东发展银行股份有限公司 | User behavior analysis platform system, method and storage medium based on behavior data |
| CN111930508A (en) * | 2020-08-18 | 2020-11-13 | 上海浦东发展银行股份有限公司 | User behavior analysis platform system, method and storage medium based on behavior data |
| CN112016027A (en) * | 2020-08-20 | 2020-12-01 | 杭州小电科技股份有限公司 | Method and device for monitoring access data operation platform |
| CN112506733A (en) * | 2020-10-30 | 2021-03-16 | 福建亿能达信息技术股份有限公司 | Method, device, equipment and medium for finely analyzing user behavior data |
| CN113344604A (en) * | 2021-04-16 | 2021-09-03 | 广州迅捷微风信息科技有限公司 | User subdivision method based on user behavior data and flow calculation |
| CN113220657A (en) * | 2021-05-14 | 2021-08-06 | 上海哔哩哔哩科技有限公司 | Data processing method and device and computer equipment |
| CN113901325A (en) * | 2021-10-27 | 2022-01-07 | 首都信息发展股份有限公司 | User behavior analysis device and method |
| CN114092138A (en) * | 2021-11-10 | 2022-02-25 | 建信金融科技有限责任公司 | User behavior analysis method, device, equipment and storage medium |
| CN114610204A (en) * | 2022-03-14 | 2022-06-10 | 中国农业银行股份有限公司 | Auxiliary device and method for data processing, storage medium and electronic equipment |
| CN114610204B (en) * | 2022-03-14 | 2024-03-26 | 中国农业银行股份有限公司 | Auxiliary device and method for data processing, storage medium and electronic equipment |
| CN115408586A (en) * | 2022-08-25 | 2022-11-29 | 广东博成网络科技有限公司 | Intelligent channel operation data analysis method, system, equipment and storage medium |
| CN115408586B (en) * | 2022-08-25 | 2024-01-23 | 广东博成网络科技有限公司 | Intelligent channel operation data analysis method, system, equipment and storage medium |
| CN116187869A (en) * | 2023-04-28 | 2023-05-30 | 北京合力金智科技发展有限公司 | User behavior analysis method and system applied to government enterprise informatization system |
| CN116501778A (en) * | 2023-05-16 | 2023-07-28 | 湖北省珍岛数字智能科技有限公司 | Real-time user behavior data analysis method based on ClickHouse |
| CN117354356A (en) * | 2023-12-04 | 2024-01-05 | 四川才子软件信息网络有限公司 | APP region retention statistical method, system and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107944059A (en) | A kind of user behavior analysis method and system based on stream calculation | |
| US11620300B2 (en) | Real-time measurement and system monitoring based on generated dependency graph models of system components | |
| US10853382B2 (en) | Interactive punchcard visualizations | |
| US10853383B2 (en) | Interactive parallel coordinates visualizations | |
| US20200104402A1 (en) | System Monitoring Driven By Automatically Determined Operational Parameters Of Dependency Graph Model With User Interface | |
| CN108334517A (en) | A kind of webpage rendering intent and relevant device | |
| US8694514B2 (en) | Collaborative filtering engine | |
| US11037342B1 (en) | Visualization modules for use within a framework for displaying interactive visualizations of event data | |
| CN107451861B (en) | Method for identifying user internet access characteristics under big data | |
| US20210279232A1 (en) | Chatbot Search System, Chatbot Search Method, and Program | |
| CN104217031B (en) | A kind of method and apparatus that user's classification is carried out according to server search daily record data | |
| US9613155B2 (en) | System and framework for multi-dimensionally visualizing and interacting with large data sets | |
| CN104850546B (en) | Display method and system of mobile media information | |
| US10853380B1 (en) | Framework for displaying interactive visualizations of event data | |
| CN106295382B (en) | A kind of Information Risk preventing control method and device | |
| CN103136335A (en) | Data control method based on data platforms | |
| CN108874812B (en) | Data processing method, server and computer storage medium | |
| CN106339891A (en) | Intelligent analysis method and system based on large data acquisition | |
| CN104182506A (en) | Log management method | |
| CN109242553A (en) | A kind of user behavior data recommended method, server and computer-readable medium | |
| US10861202B1 (en) | Sankey graph visualization for machine data search and analysis system | |
| CN107688568A (en) | Acquisition method and device based on web page access behavior record | |
| CN112394861B (en) | Page jump method and device, storage medium and electronic device | |
| US10552996B2 (en) | Systems and techniques for determining associations between multiple types of data in large data sets | |
| CN108416627A (en) | A kind of brand influence force monitoring method and system based on internet data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200423 Address after: 518000 two 601-B1 Software Park, Shenzhen Road, Nanshan District hi tech, Shenzhen, Guangdong, 8 Applicant after: SHENZHEN ZHONGRUN SIFANG INFORMATION TECHNOLOGY Co.,Ltd. Address before: 710000 room B 603, room B, No. 38 Tengfei innovation center, Xi'an Hi-tech Zone, Xi'an hi tech Zone Applicant before: XI'AN BRANCH COMPANY OF SHENZHEN CHINASFAN INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180420 |
|
| RJ01 | Rejection of invention patent application after publication |