CN110175281A - A kind of user data processing, exchange method, apparatus and system - Google Patents
A kind of user data processing, exchange method, apparatus and system Download PDFInfo
- Publication number
- CN110175281A CN110175281A CN201910273174.8A CN201910273174A CN110175281A CN 110175281 A CN110175281 A CN 110175281A CN 201910273174 A CN201910273174 A CN 201910273174A CN 110175281 A CN110175281 A CN 110175281A
- Authority
- CN
- China
- Prior art keywords
- data
- user
- user behavior
- behavior data
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/604—Tools and structures for managing or administering access control systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Automation & Control Theory (AREA)
- Fuzzy Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of user data processing, exchange method, apparatus and system, which includes: to obtain the interface message and/or component software of user terminal access；User behavior data is obtained according to the interface message and/or component software, the user behavior data includes at least: the identification information and operation information of the user；Cleaning conversion is carried out to the user behavior data according to preset data processing work stream and/or Data Analysis Model；User behavior data after cleaning conversion is added into label, and is stored into database.
Description
Technical field
The present invention relates to big data processing technology fields, and in particular to a kind of user data processing, exchange method, device and
System.
Background technique
In the information age, the development of database was formd using the data modeling induction and conclusion past and will be given a forecast to future
" computational science "；And internet/big data age is consumed, by collecting a large amount of data, forms and computer is allowed to go rule
Study and Optimized model " data science ".Big data (Bigdata) is commonly used to describe that a company creates a large amount of non-
Structural data and semi-structured data, with the arriving in industry internet AI intelligent epoch, big data (Bigdata) needs more
Effectively it is evolved into data-intensive scientific discovery (ScientificDiscovery).And any one scientific discovery, it requires
Meet the requirement of Reproducible Result (repeatable result).Because it is based on data as a result, be all from data -> from
Therefore reason -> result (Data- > Process- > Result) such a process will meet Reproducible Result's
It is required that a kind of more intelligent, more flexible real-time (not single only in face of user) data processing system is then needed to handle data.
In the epoch for consuming internet, data processing system used in each large size technology platform, is by providing data
Technology of sharing by data collection and is stored on themselves platform (centralization and publicly-owned cloud), and applies super id machine
System, has more permission to freely using for user by platform provider centralized control, these are brought using limitation to user
Inconvenience has dragged slowly industry to interconnect the development progress of technology required for network process decentralization more significantly.
Summary of the invention
In view of this, the embodiment of the invention provides a kind of user data processing, exchange method, apparatus and system, with solution
Certainly the above-mentioned problems in the prior art.
According in a first aspect, the embodiment of the invention provides a kind of user data processing methods, comprising: obtain user's termination
The interface message and/or component software entered；User behavior data is obtained according to the interface message and/or component software, it is described
User behavior data includes at least: the identification information and operation information of the user；According to preset data processing work stream and/
Or Data Analysis Model carries out cleaning conversion to the user behavior data；User behavior data after cleaning conversion is added
It tags, and stores into database.
With reference to first aspect, in first aspect first embodiment, according to preset data processing work stream and/or number
Cleaning conversion is carried out to the user behavior data according to analysis model, comprising: extract the data characteristics of the user behavior data；
It is user behavior data marking according to the data characteristics and preset data marking table.
With reference to first aspect, in first aspect second embodiment, according to preset data processing work stream and/or number
Cleaning conversion is carried out to the user behavior data according to analysis model, comprising: data screening rule is generated according to the instruction of user；
The user behavior data is screened according to the data screening rule；User behavior data after retaining screening.
With reference to first aspect, in first aspect third embodiment, according to preset data processing work stream and/or number
Cleaning conversion is carried out to the user behavior data according to analysis model, comprising: according to the instruction of user under shared data platform
Carry Data Analysis Model；The user behavior data is inputted into the Data Analysis Model and carries out data analysis, generates analysis knot
Fruit.
According to second aspect, the embodiment of the invention provides a kind of user data exchange methods, comprising: obtains user information
And interaction request；The corresponding label of user behavior data, the behavioral data are read from database according to the interaction request
To use the user data exchange method as described in first aspect or first aspect any one embodiment to store to database
User behavior data；According to the user information and interaction request, the label is transmitted.
According to the third aspect, the embodiment of the invention provides a kind of user data processing units, comprising: acquisition of information mould
Block, for obtaining the interface message and/or component software of user terminal access；Behavioral data obtains module, for connecing according to
Message breath and/or component software obtain user behavior data, and the user behavior data includes at least: the identification letter of the user
Breath and operation information；Data scrubbing module is used for according to preset data processing work stream and/or Data Analysis Model to described
User behavior data carries out cleaning conversion；Data mark module, for that will be added by the user behavior data after cleaning conversion
Label, and store into database.
In conjunction with the third aspect, in first aspect first embodiment, the data scrubbing module includes: that data characteristics mentions
Submodule is taken, for extracting the data characteristics of the user behavior data；Give a mark submodule, for according to the data characteristics with
And preset data marking table is user behavior data marking.
In conjunction with the third aspect, in first aspect second embodiment, the data scrubbing module includes: data screening rule
Submodule is then generated, for generating data screening rule according to the instruction of user；Data screening submodule, for according to the number
The user behavior data is screened according to screening rule；Data retention submodule, for retaining the user behavior after screening
Data.
In conjunction with the third aspect, in first aspect third embodiment, the data scrubbing module includes: carrier under model
Module, for according to the instruction of user from shared data platform downloading data analysis model；Data analyze submodule, are used for institute
It states user behavior data and inputs the Data Analysis Model progress data analysis, generate analysis result.
According to fourth aspect, the embodiment of the invention provides a kind of user data interactive devices, comprising: interaction request obtains
Module, for obtaining user information and interaction request；Data label read module, for according to the interaction request from database
The corresponding label of middle reading user behavior data, the behavioral data are using such as first aspect or first aspect any one reality
User data interactive device described in mode is applied to store to the user behavior data of database；Data interaction module is used for basis
The user information and interaction request transmit the label.
According to the 5th aspect, the embodiment of the invention provides a kind of user data interactive systems, comprising: such as first aspect or
User data processing unit described in any one embodiment of first aspect and as second aspect or second aspect any one
User data interactive device described in embodiment.
According to the 6th aspect, the embodiment of the invention provides a kind of big data processing equipments, comprising: memory and processing
Device communicates with each other connection between the memory and the processor, computer instruction, the place are stored in the memory
Device is managed by executing the computer instruction, thereby executing institute in any one of first aspect or first aspect embodiment
The user data processing method stated, alternatively, executing user data exchange method described in second aspect.
It is described computer-readable the embodiment of the invention provides a kind of computer readable storage medium according to the 7th aspect
Storage medium stores computer instruction, and the computer instruction is for making the computer execute first aspect or first aspect
Any one embodiment described in user data processing method, alternatively, execute second aspect described in user data hand over
Mutual method.
The beneficial effect that the embodiment of the present invention has is, by applying the present invention, can be provided for user it is more open,
The Data Share System of hommization；User data is being obtained, in the process of processing to user data, for different users
Data are added corresponding label data and are stored, and convenient for being managed to user data, and realize going for industry internet
Centralization.Also, corresponding flow chart of data processing or Processing Algorithm are provided based on shared platform, can be made to measure respectively for user
From data processing service, improve the use feeling of user.
Detailed description of the invention
The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, and attached drawing is schematically without that should manage
Solution is carries out any restrictions to the present invention, in the accompanying drawings:
Fig. 1 shows the flow chart of the user data processing method of the embodiment of the present invention；
Fig. 2 shows the flow charts of the user data exchange method of the embodiment of the present invention；
Fig. 3 shows the structural schematic diagram of the user data processing unit of the embodiment of the present invention；
Fig. 4 shows the structural schematic diagram of the user data interactive device of the embodiment of the present invention；
Fig. 5 shows the hardware structural diagram of the big data processing equipment of the embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those skilled in the art are not having
Every other embodiment obtained under the premise of creative work is made, shall fall within the protection scope of the present invention.
The embodiment of the present invention provides a kind of user data processing method, as shown in Figure 1, the processing method includes following step
It is rapid:
Step S101: the interface message and/or component software of user terminal access are obtained；In practical application, user can be
It is linked into data processing platform (DPP) by ustomer premises access equipment, at this point, obtaining the interface of user terminal access by the data processing platform (DPP)
Information (API) and/or component software (SDK), and the channel of user's access can be varied, e.g. trade company, enterprise are certainly
The access of the channels such as oneself website, social media, APP.
Step S102: user behavior data is obtained according to interface message and/or component software, user behavior data at least wraps
It includes: the identification information and operation information of the user.Specifically, the identification information of the user can include but is not limited to: 1) using
Family cookie, the Uid of user, the memberid etc. of trade company.And the operation information of user then refers to through user journal record
The various operation behaviors of user or user are for the operation datas such as the browsing of recommendation, click, closing, these user's rows
It can be handled, converted, be recorded by event tag processor, the API of electric business website and log API for data.Therefore, may be used
To be obtained by above-mentioned API and/or SDK.
Step S103: according to preset data processing work stream and/or Data Analysis Model to the user behavior data
Carry out cleaning conversion (ETL processing).
Optionally, in some embodiments of the invention, the ETL treatment process executed in step S103 refers to: extracting institute
State the data characteristics of user behavior data；It is the user behavior according to the data characteristics and preset data marking table
Data marking.
This process major embodiment can realize user behavior scoring system: can extract wherein after getting user behavior data
Data characteristics, duration, frequency, depth such as user's browsing " skiing the page ", to label to user, while can be to this label
A marking is had, therefore, similar " skiing ", " double plates ", " veneer ", " primary skiing ", " bone ash grade skiing in the label of user
Player " just has different score values, description user's feature that can be relatively accurate.
Optionally, in some embodiments of the invention, the ETL treatment process executed in step S103 refers to: according to
The instruction at family generates data screening rule；The user behavior data is screened according to the data screening rule；Retain
User behavior data after screening.
The rule that is previously set according to user is adopted or is given up to data, can carry out as desired among these
Configuration, the execution, management of rule, different industries can configure to one's name different rule bases, the present invention not with
This is limited.
Optionally, in some embodiments of the invention, the ETL treatment process executed in step S103 refers to: according to
The instruction at family is from shared data platform downloading data analysis model；The user behavior data is inputted into the Data Analysis Model
Data analysis is carried out, analysis result is generated.
In practical application, in most cases, data are fuzzy concepts, for example, when needing to filter out in larger scope
When target user, it may be necessary to use different data screening algorithm (e.g. crowd's broadcast algorithm (lookalike/
Actalike data processing)) is carried out.At this point, corresponding calculation can be downloaded from correspondingly shared platform according to the instruction that user inputs
Method model or Data Analysis Model obtain analysis result to analyze user behavior data.
By ETL treated the user behavior data of above-mentioned steps S103, step S104 can be passed through: will turned by cleaning
User behavior data after changing adds label, and stores into database.
By the data that ETL is handled, meet the requirement of corresponding data screening, therefore stamped corresponding label,
It is stored in database, for subsequent query, calling, interaction etc..
User data processing method through the embodiment of the present invention can provide more open, hommization number for user
According to shared mechanism；Also, this method can by obtain user data generate user data label, according to data label be without
Enterprise customer pushes user data, is not violating general data conservation regulation (General Data Protection
Regulation, GDPR) data exchange is promoted under the premise of policy.User data is being obtained, user data is being handled
During, corresponding label data, which is added, for different user data is stored, convenient for being managed to user data,
And realize the decentralization of industry internet.
The embodiment of the present invention also provides a kind of user data exchange method, as shown in Fig. 2, user data exchange method master
Include:
Step S201: user information and interaction request are obtained；In embodiments of the present invention, the interactive process master of user data
To be started by the request of user, when user initiates data interaction request, while some user informations can be inputted, at this point, obtaining
Take the user information and interaction request.
Step S202: the corresponding label of user behavior data, the row are read from database according to the interaction request
It is to be stored using user data exchange method described in above-mentioned any embodiment to the user behavior data of database for data.
Step S203: according to the user information and interaction request, the label is transmitted.In above-mentioned steps S202
In the user behavior data that is extracted from database, corresponding label (tagging) is added after having already passed through analysis processing
Data, when interacting process, only interaction is the label for representing corresponding data, to guarantee that the information of user and data exist
Safety in interactive process.
In practical applications, above-mentioned user behavior data is different according to different application fields, for example, when answering
For hospital HIS data, disease type, number of hospitalized trend can be filtered out, average length of hospitalization is calculated.Applied to traffic system
System, can differentiate congestion path, charge station's bayonet can identify doubtful fee evasion vehicle.Applied to electric business system, can differentiate
User preferences, potential purchase intention, the present invention is not limited thereto.
The embodiment of the present invention also provides a kind of user data processing unit, as shown in figure 3, the processing unit includes:
Data obtaining module 101, for obtaining the interface message and/or component software of user terminal access；Detailed content can
Referring to the associated description of the step S101 of above method embodiment.
Behavioral data obtains module 102, for obtaining user behavior number according to the interface message and/or component software
According to；Detailed content can be found in the associated description of the step S102 of above method embodiment.
Data scrubbing module 103 is used for according to preset data processing work stream and/or Data Analysis Model to the use
Family behavioral data carries out cleaning conversion；Detailed content can be found in the associated description of the step S103 of above method embodiment.
Optionally, in some embodiments of the invention, which includes: that data characteristics extracts submodule
Block, for extracting the data characteristics of the user behavior data；Marking submodule is used for according to the data characteristics and presets
Data marking table be the user behavior data give a mark.
Optionally, in some embodiments of the invention, which includes: that data screening rule generates son
Module, for generating data screening rule according to the instruction of user；Data screening submodule, for being advised according to the data screening
Then the user behavior data is screened；Data retention submodule, for retaining the user behavior data after screening.
Optionally, in some embodiments of the invention, which includes: model downloading submodule, is used
According to the instruction of user from shared data platform downloading data analysis model；Data analyze submodule, are used for the user
Behavioral data inputs the Data Analysis Model and carries out data analysis, generates analysis result.
Data mark module 104 for that will add label by the user behavior data after cleaning conversion, and is stored to number
According in library；Detailed content can be found in the associated description of the step S104 of above method embodiment.
User data processing unit through the embodiment of the present invention can provide more open, hommization number for user
According to shared mechanism；Also, this method can by obtain user data generate user data label, according to data label be without
Enterprise customer pushes user data, is not violating general data conservation regulation (General Data Protection
Regulation, GDPR) data exchange is promoted under the premise of policy.User data is being obtained, user data is being handled
During, corresponding label data, which is added, for different user data is stored, convenient for being managed to user data,
And realize the decentralization of industry internet.
The embodiment of the present invention also provides a kind of user data interactive device, as shown in figure 4, the user data interactive device packet
It includes:
Interaction request obtains module 201, for obtaining user information and interaction request；Detailed content can be found in the above method
The associated description of the step S201 of embodiment.
Data label read module 202, for reading user behavior data pair from database according to the interaction request
The label answered；Detailed content can be found in the associated description of the step S202 of above method embodiment.
Data interaction module 203, for according to the user information and interaction request, the label to be transmitted；In detail
Thin content can be found in the associated description of the step S203 of above method embodiment.
The embodiment of the present invention also provides a kind of user data interactive system, mainly includes use described in above-mentioned any embodiment
User data processing unit and user data interactive device.Wherein, the function which mainly realizes such as 1 institute of table
Show:
Table 1
In practical applications, which can be framework in enterprises, access the original business of enterprise
The data of system.The Operation Definition provided by the user data processing unit, user design the process of oneself, operation obtain by
The data needed, for example user are added to corresponding label (tagging).User data processing unit and user data interactive device
When carrying out data interaction, the interaction of the data interaction being related to i.e. label (Tagging), the not interaction of initial data,
To the safety of effective guarantee user data.
The embodiment of the invention also provides a kind of big data processing equipments, as shown in figure 5, the big data processing equipment can be with
Including processor 51 and memory 52, wherein processor 51 can be connected with memory 52 by bus or other modes, Fig. 5
In by by bus connect for.
Processor 51 can be central processing unit (Central Processing Unit, CPU).Processor 51 can be with
For other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
The combination of the chips such as discrete hardware components or above-mentioned all kinds of chips.
Memory 52 is used as a kind of non-transient computer readable storage medium, can be used for storing non-transient software program, non-
Transient computer executable program and module, such as the user data processing method or user data interaction in the embodiment of the present invention
Corresponding program instruction/the module of method is (for example, data obtaining module shown in Fig. 3 101, behavioral data obtain module 102, number
Module 201 is obtained according to cleaning modul 103 and data mark module 104 or interaction request shown in Fig. 4, data label is read
Module 202, data interaction module 203).Processor 51 is by running the non-transient software program being stored in memory 52, referring to
It enables and module is realized in above method embodiment thereby executing the various function application and data processing of processor
User data processing method or user data exchange method.
Memory 52 may include storing program area and storage data area, wherein storing program area can storage program area,
Application program required at least one function；It storage data area can the data etc. that are created of storage processor 51.In addition, storage
Device 52 may include high-speed random access memory, can also include non-transient memory, for example, at least a magnetic disk storage
Part, flush memory device or other non-transient solid-state memories.In some embodiments, it includes relative to place that memory 52 is optional
The remotely located memory of device 51 is managed, these remote memories can pass through network connection to processor 51.The reality of above-mentioned network
Example includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
One or more of modules are stored in the memory 52, when being executed by the processor 51, are executed
User data processing method or user data exchange method in embodiment as Figure 1-Figure 2.
Above-mentioned big data processing equipment detail can correspond to corresponding phase in embodiment referring to FIG. 1 to 4
It closes description and effect is understood that details are not described herein again.
It is that can lead to it will be understood by those skilled in the art that realizing all or part of the process in above-described embodiment method
Computer program is crossed to instruct relevant hardware and complete, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can for magnetic disk,
CD, read-only memory (Read-Only Memory, ROM), random access memory (Random Access
Memory, RAM), flash memory (Flash Memory), hard disk (Hard Disk Drive, abbreviation: HDD) or solid state hard disk
(Solid-State Drive, SSD) etc.；The storage medium can also include the combination of the memory of mentioned kind.
Although being described in conjunction with the accompanying the embodiment of the present invention, those skilled in the art can not depart from the present invention
Spirit and scope in the case where various modifications and variations can be made, such modifications and variations are each fallen within by appended claims institute
Within the scope of restriction.
Claims (13)
1. a kind of user data processing method characterized by comprising
Obtain the interface message and/or component software of user terminal access；
User behavior data is obtained according to the interface message and/or component software, the user behavior data includes at least: institute
State the identification information and operation information of user；
Cleaning conversion is carried out to the user behavior data according to preset data processing work stream and/or Data Analysis Model；
User behavior data after cleaning conversion is added into label, and is stored into database.
2. user data processing method according to claim 1, which is characterized in that according to preset data processing work stream
And/or Data Analysis Model carries out cleaning conversion to the user behavior data, comprising:
Extract the data characteristics of the user behavior data；
It is user behavior data marking according to the data characteristics and preset data marking table.
3. user data processing method according to claim 1, which is characterized in that according to preset data processing work stream
And/or Data Analysis Model carries out cleaning conversion to the user behavior data, comprising:
Data screening rule is generated according to the instruction of user；
The user behavior data is screened according to the data screening rule；
User behavior data after retaining screening.
4. user data processing method according to claim 1, which is characterized in that according to preset data processing work stream
And/or Data Analysis Model carries out cleaning conversion to the user behavior data, comprising:
According to the instruction of user from shared data platform downloading data analysis model；
The user behavior data is inputted into the Data Analysis Model and carries out data analysis, generates analysis result.
5. a kind of user data exchange method characterized by comprising
Obtain user information and interaction request；
The corresponding label of user behavior data is read from database according to the interaction request, the behavioral data is using such as
The user data exchange method of any one of claim 1-4 is stored to the user behavior data of database；
According to the user information and interaction request, the label is transmitted.
6. a kind of user data processing unit characterized by comprising
Data obtaining module, for obtaining the interface message and/or component software of user terminal access；
Behavioral data obtains module, for obtaining user behavior data, the use according to the interface message and/or component software
Family behavioral data includes at least: the identification information and operation information of the user；
Data scrubbing module is used for according to preset data processing work stream and/or Data Analysis Model to the user behavior
Data carry out cleaning conversion；
Data mark module for that will add label by the user behavior data after cleaning conversion, and is stored into database.
7. user data processing unit according to claim 6, which is characterized in that the data scrubbing module includes:
Data characteristics extracting sub-module, for extracting the data characteristics of the user behavior data；
Marking submodule, for being that the user behavior data is beaten according to the data characteristics and preset data marking table
Point.
8. user data processing unit according to claim 6, which is characterized in that the data scrubbing module includes:
Data screening rule generates submodule, for generating data screening rule according to the instruction of user；
Data screening submodule, for being screened according to the data screening rule to the user behavior data；
Data retention submodule, for retaining the user behavior data after screening.
9. user data processing unit according to claim 6, which is characterized in that the data scrubbing module includes:
Model downloads submodule, for according to the instruction of user from shared data platform downloading data analysis model；
Data analyze submodule, carry out data analysis for the user behavior data to be inputted the Data Analysis Model, raw
At analysis result.
10. a kind of user data interactive device characterized by comprising
Interaction request obtains module, for obtaining user information and interaction request；
Data label read module, for reading the corresponding mark of user behavior data from database according to the interaction request
Label, the behavioral data are to be stored using the user data interactive device such as any one of claim 1-4 to the use of database
Family behavioral data；
Data interaction module, for according to the user information and interaction request, the label to be transmitted.
11. a kind of user data interactive system characterized by comprising the number of users as described in any one of claim 6-9
According to processing unit and user data interactive device as claimed in claim 10.
12. a kind of big data processing equipment characterized by comprising
Memory and processor communicate with each other connection, are stored in the memory between the memory and the processor
Computer instruction, the processor is by executing the computer instruction, thereby executing as described in any one of claim 1-4
User data processing method, alternatively, executing user data exchange method as claimed in claim 5.
13. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has computer to refer to
It enables, the computer instruction is handled for making the computer execute user data of any of claims 1-4 such as
Method, alternatively, executing user data exchange method as claimed in claim 5.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910037309 | 2019-01-15 | ||
| CN2019100373090 | 2019-01-15 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110175281A true CN110175281A (en) | 2019-08-27 |
Family
ID=67569442
Family Applications (5)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910272512.6A Pending CN110134711A (en) | 2019-01-15 | 2019-04-04 | Processing method, device, equipment and the computer readable storage medium of big data |
| CN201910274466.3A Pending CN110196933A (en) | 2019-01-15 | 2019-04-04 | A kind of user data label processing method and device |
| CN201910273168.2A Pending CN110188144A (en) | 2019-01-15 | 2019-04-04 | A kind of user data digitalized processing method, device and equipment |
| CN201910273174.8A Pending CN110175281A (en) | 2019-01-15 | 2019-04-04 | A kind of user data processing, exchange method, apparatus and system |
| CN201910274370.7A Pending CN110188264A (en) | 2019-01-15 | 2019-04-04 | A kind of user data exchange method and device |
Family Applications Before (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910272512.6A Pending CN110134711A (en) | 2019-01-15 | 2019-04-04 | Processing method, device, equipment and the computer readable storage medium of big data |
| CN201910274466.3A Pending CN110196933A (en) | 2019-01-15 | 2019-04-04 | A kind of user data label processing method and device |
| CN201910273168.2A Pending CN110188144A (en) | 2019-01-15 | 2019-04-04 | A kind of user data digitalized processing method, device and equipment |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910274370.7A Pending CN110188264A (en) | 2019-01-15 | 2019-04-04 | A kind of user data exchange method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (5) | CN110134711A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046382A (en) * | 2019-12-30 | 2020-04-21 | 武汉英迈信息科技有限公司 | Database auditing method, device, storage medium and device |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241081A (en) * | 2020-01-09 | 2020-06-05 | 杭州涂鸦信息技术有限公司 | IOT platform data collection method and system, readable storage medium and computer equipment |
| CN111291267B (en) * | 2020-02-17 | 2024-04-12 | 中国农业银行股份有限公司 | APP user behavior analysis method and device |
| CN112073367B (en) * | 2020-07-28 | 2022-12-09 | 智维云图(上海)智能科技有限公司 | Protocol analysis method and system for visual fire-fighting Internet of things data |
| CN112184103A (en) * | 2020-09-15 | 2021-01-05 | 苏州牧星智能科技有限公司 | Order processing method, device and system |
| CN112287015B (en) * | 2020-10-14 | 2024-03-01 | 北京易观数智科技股份有限公司 | Image generation system, image generation method, electronic device, and storage medium |
| CN112632597A (en) * | 2020-12-08 | 2021-04-09 | 国家计算机网络与信息安全管理中心 | Data desensitization method and device readable storage medium |
| CN114520815A (en) * | 2021-12-31 | 2022-05-20 | 乐爱创新(北京)科技有限公司 | Method, device, equipment and storage medium for processing big data of water environment |
-
2019
- 2019-04-04 CN CN201910272512.6A patent/CN110134711A/en active Pending
- 2019-04-04 CN CN201910274466.3A patent/CN110196933A/en active Pending
- 2019-04-04 CN CN201910273168.2A patent/CN110188144A/en active Pending
- 2019-04-04 CN CN201910273174.8A patent/CN110175281A/en active Pending
- 2019-04-04 CN CN201910274370.7A patent/CN110188264A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046382A (en) * | 2019-12-30 | 2020-04-21 | 武汉英迈信息科技有限公司 | Database auditing method, device, storage medium and device |
| CN111046382B (en) * | 2019-12-30 | 2024-04-02 | 武汉英迈信息科技有限公司 | Database auditing method, equipment, storage medium and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188144A (en) | 2019-08-30 |
| CN110188264A (en) | 2019-08-30 |
| CN110196933A (en) | 2019-09-03 |
| CN110134711A (en) | 2019-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110175281A (en) | A kind of user data processing, exchange method, apparatus and system | |
| WO2020037918A1 (en) | Risk control strategy determining method based on predictive model, and related device | |
| CN111190939B (en) | User portrait construction method and device | |
| US20210117509A1 (en) | Creating a knowledge graph based on text-based knowledge corpora | |
| US20150095267A1 (en) | Techniques to dynamically generate real time frequently asked questions from forum data | |
| CN105721629B (en) | User identifier matching process and device | |
| CN105023165A (en) | Method, device and system for controlling release tasks in social networking platform | |
| CN108549909B (en) | Object classification method and object classification system based on crowdsourcing | |
| CN111753006B (en) | Prediction system and method based on federal learning | |
| CN108491267A (en) | Method and apparatus for generating information | |
| CN115511501A (en) | Data processing method, computer equipment and readable storage medium | |
| CN106097043A (en) | The processing method of a kind of credit data and server | |
| CN106815254A (en) | A kind of data processing method and device | |
| CN108320045A (en) | Student performance prediction technique and device | |
| CN111209931A (en) | Data processing method, platform, terminal device and storage medium | |
| CN108764332A (en) | A kind of Channel Quality analysis method, computing device and storage medium | |
| CN110543603B (en) | Collaborative filtering recommendation method, device, equipment and medium based on user behaviors | |
| CN112529702A (en) | User credit granting strategy allocation method and device and electronic equipment | |
| CN107977678A (en) | Method and apparatus for output information | |
| CN111523324A (en) | Training method and device for named entity recognition model | |
| CN107291826A (en) | File search processing method and processing device | |
| CN110069686A (en) | User behavior analysis method, apparatus, computer installation and storage medium | |
| CN115576973B (en) | Service deployment method, device, computer equipment and readable storage medium | |
| CN110215703A (en) | The selection method of game application, apparatus and system | |
| CN110069691A (en) | For handling the method and apparatus for clicking behavioral data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |