CN109388657A - Data processing method, device, computer equipment and storage medium - Google Patents
Data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109388657A CN109388657A CN201811050633.8A CN201811050633A CN109388657A CN 109388657 A CN109388657 A CN 109388657A CN 201811050633 A CN201811050633 A CN 201811050633A CN 109388657 A CN109388657 A CN 109388657A
- Authority
- CN
- China
- Prior art keywords
- data
- address
- cached
- cache cluster
- different types
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Data processing method, device, computer equipment and storage medium provided in an embodiment of the present invention.This method comprises: multiple cache clusters are constructed, the corresponding IP address of a cache cluster；Cached configuration table is constructed, the cached configuration table includes the corresponding relationship between the type of data and the IP address of cache cluster；If receiving to data cached, classify described by preset rules to data cached to obtain different types of data；The IP address of the corresponding multiple cache clusters of a variety of different types of data is obtained from the cached configuration table；According to acquired IP address, a variety of different types of data are saved respectively into corresponding cache cluster.The embodiment of the present invention utilizes the different different types of data of cache cluster distributed caching, according to the type of data, data distribution formula is cached to corresponding cache cluster based on Distributed Message Queue technology, the unique caching of same type data is conducive to the uniformity of business processing.
Description
Technical field
The present invention relates to technical field of information processing more particularly to a kind of data processing method, device, computer equipment and
Storage medium.
Background technique
Multiple cache clusters are generally included in distributed cache system, each cache cluster has multiple data cached
Machine, the data that all machines in a cache cluster save are just the same, the data saved between different cache clusters
It is different.
Existing distributed cache system is during a data buffer storage, by all data buffer storages a to cache cluster
In, the cache cluster for currently needing to save data is selected by data volume that each cache cluster saves, so may be implemented
Data volume balance between each cache cluster, however this property that will cause the data that each cache cluster saves is to be not fixed
, when needing to carry out certain a kind of business, the cache cluster that may need to use every time is all inconsistent, when certain cache sets are mass-sended
When giving birth to failure, some can be normally carried out same class business possibility, some can not be normally carried out, this is unfavorable for the system of business processing
One property.
Summary of the invention
The embodiment of the invention provides a kind of data processing method, device, computer equipment and storage mediums, it is intended to will be same
The data of one type or property are stored in identical cache cluster, guarantee the uniformity of business.
In a first aspect, the embodiment of the invention provides a kind of data processing methods, this method comprises: constructing multiple cache sets
Group, the corresponding IP address of a cache cluster；Construct multiple cache clusters, the corresponding IP address of a cache cluster；Structure
Cached configuration table is built, the cached configuration table includes the corresponding relationship between the type of data and the IP address of cache cluster；If
It receives to data cached, classifies described by preset rules to data cached to obtain different types of data；From institute
State the IP address that the corresponding multiple cache clusters of a variety of different types of data are obtained in cached configuration table；According to acquired
IP address, a variety of different types of data are saved respectively into corresponding cache cluster.
Second aspect, the embodiment of the invention also provides a kind of data processing equipment, the data processing equipment includes using
In the unit for realizing data processing method described in first aspect.
The third aspect, the embodiment of the invention also provides a kind of computer equipments, including memory, and with the storage
The connected processor of device；The memory is used to store the computer program for realizing data processing method；The processor is used for
The computer program stored in the memory is run, to execute the method as described in above-mentioned first aspect.
Fourth aspect, the embodiment of the invention provides a kind of storage medium, the storage medium is stored with one or one
A above computer program, the one or more computer program can be held by one or more than one processor
Row, to realize method described in above-mentioned first aspect.
The embodiment of the invention provides a kind of data processing method, device, computer equipment and storage medium, wherein method
It include: the multiple cache clusters of building, the corresponding IP address of a cache cluster；Construct cached configuration table, the cached configuration
Table includes the corresponding relationship between the type of data and the IP address of cache cluster；If receiving to data cached, will it is described to
It is data cached to classify by preset rules to obtain different types of data；It is obtained from the cached configuration table described a variety of
The IP address of the corresponding multiple cache clusters of different types of data；It, will be a variety of different types of according to acquired IP address
Data are saved respectively into corresponding cache cluster.The embodiment of the present invention is by delaying data using multiple cache clusters
It deposits, the corresponding IP address of a cache cluster, according to the one-to-one relationship of the type of data and the IP address of cache cluster
It searches the cache cluster for caching different types of data, the data of same type is stored in the same cache cluster, number
According to unique caching be conducive to the uniformity of business processing.
Detailed description of the invention
Technical solution in order to illustrate the embodiments of the present invention more clearly, below will be to needed in embodiment description
Attached drawing is briefly described, it should be apparent that, drawings in the following description are some embodiments of the invention, general for this field
For logical technical staff, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of flow diagram for data processing method that one embodiment of the invention provides；
Fig. 2 is a kind of sub-process schematic diagram for data processing method that one embodiment of the invention provides；
Fig. 3 be another embodiment of the present invention provides a kind of data processing method flow diagram；
Fig. 4 be another embodiment of the present invention provides a kind of data processing method sub-process schematic diagram；
Fig. 5 is a kind of schematic block diagram for data processing equipment that one embodiment of the invention provides；
Fig. 6 is a kind of subelement schematic block diagram for data processing equipment that one embodiment of the invention provides；
Fig. 7 is a kind of schematic block diagram for data processing equipment that one embodiment of the invention provides；
Fig. 8 be another embodiment of the present invention provides a kind of data processing equipment subelement schematic block diagram；
Fig. 9 is a kind of structural representation block diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
It should be appreciated that ought use in this specification and in the appended claims, term " includes " and "comprising" instruction
Described feature, entirety, step, operation, the presence of element and/or component, but one or more of the other feature, whole is not precluded
Body, step, operation, the presence or addition of element, component and/or its set.
It is also understood that referring in description of the invention to term "and/or" used in the appended claims related
Join any combination and all possible combinations of one or more of item listed, and including these combinations.
It will also be understood that these elements are not answered although term first, second etc. can be used to describe various elements herein
This is limited to these terms, these terms are only used to for these elements being distinguished from each other out.
Fig. 1 is a kind of flow diagram of data processing method provided in an embodiment of the present invention.This method is applied to desk-top
In the terminals such as computer, laptop computer, tablet computer.As shown in Figure 1, the method comprising the steps of S101~S105.
S101, the multiple cache clusters of building, the corresponding IP address of a cache cluster.
It constructs multiple cache clusters and forms data buffer storage center, distributed caching, a cache cluster pair are carried out to data
Answer an IP address.
S102, building cached configuration table, the cached configuration table include data type and cache cluster IP address it
Between corresponding relationship.
The type of data and the corresponding relationship of IP address are established, when needing data cached, the data that cache as needed
Type, the data buffer storage that the type of data and the corresponding relationship of IP address can find which kind from caching allocation list exists
Which cache cluster.
If S103, receiving to data cached, by it is described classify by preset rules to data cached it is a variety of to obtain
Different types of data.
In order to reduce the pressure of database, need to carry out the total data in database into backup caching, i.e., in database
Data be to need the data that cache, after each database update, the data of update are issued into caching end.
Generate different business datums to data cached all kinds of, different business, the embodiment of the present invention wait cache
Data include multiple business data.For example, having conglomerate's employee information, conglomerate's information, vehicle insurance contract to data cached
Information, life insurance contract, casualty insurance contract, operation information of each system of conglomerate etc., between different business data
May general character, therefore, it is necessary to sort out by these data summarizing.
In one embodiment, as shown in Fig. 2, step S103 the following steps are included:
S1031, the characteristic attribute for obtaining each business datum.
S1032, sort out the business datum in the multiple business data with same characteristic features attribute to form same type
Data.
Characteristic attribute refers to the service feature of data, is to indicate that a certain data belong to the data of any type of service.
For example, conglomerate's employee information has the characteristic attributes such as enterprise, employee, conglomerate's information has the characteristic attributes such as enterprise,
Conglomerate's employee information, conglomerate's information etc. belong to the business datum of enterprises property；Vehicle insurance contract information, life insurance are protected
Dangerous contract, casualty insurance contract have the characteristic attributes such as insurance, belong to insurance class business datum, and the system of conglomerate runs letter
The characteristic attribute of breath is systematic name etc., belongs to system class business datum.According to the general character between the characteristic attribute of different data,
Data with same characteristic features are classified as same type of data, such as by conglomerate's employee information and group's company information
It is classified as enterprise's category information, vehicle insurance contract information, life insurance contract, casualty insurance contract are classified as insurance category information, by group
The system operation information of enterprise is classified as system category information.
Different business data with same characteristic features attribute are sorted out to be formed same type of data be stored in it is identical slow
Center is deposited, the uniformity saved conducive to data.
S104, the corresponding multiple cache clusters of a variety of different types of data are obtained from the cached configuration table
IP address.
S105, according to acquired IP address, a variety of different types of data are saved respectively to corresponding cache cluster
In.
The type of data and the IP address of cache cluster have one-to-one relationship, are postponed according to the type of different data
The cache cluster for depositing the available data for being used to save the type in allocation list, the data of same type is stored in identical
In cache cluster, the uniformity for the uniformity and business processing that data save is realized, by being stored in data cached for flood tide
Multiple cache clusters realize the distributed caching of big data.
Data processing method provided in an embodiment of the present invention caches data using multiple cache clusters, and one slow
The corresponding IP address of cluster is deposited, caching is searched according to the one-to-one relationship of the type of data and the IP address of cache cluster
The data of same type are stored in the same cache cluster by the cache cluster of different types of data, and the unification of data is slow
There is the uniformity conducive to business processing.
Fig. 3 is a kind of flow diagram of data processing method provided in an embodiment of the present invention, as shown in figure 3, in step
It further include step S106-S108 after S105.
If S106, receiving inquiry instruction for inquiring data, the inquiry instruction includes querying condition, described in analysis
Querying condition is to obtain the type of required inquiry data.
It can analyze the data that the data that user needs to inquire are which kind according to querying condition.For example, querying condition
For insurance contract number, then the insurance category information it is known that the data for needing to inquire, for another example, inquiry are numbered according to insurance contract
Condition is enterprise staff ID number, then according to the enterprise staff information for the information inquired needed for enterprise staff ID number, i.e. enterprise
Class data.
S107, the IP address that the corresponding cache cluster of the type is read from the cached configuration table.
S108, data are inquired from the corresponding cache cluster of the IP address.
The type of data and the IP address of cache cluster have one-to-one relationship, are matched according to the type of data from caching
The data of the type can be inquired by setting in table is stored in for which cache cluster, and data are inquired from the cache cluster.
Include multiple main frames in one cache cluster, only needs to inquire on one host when inquiring data.
In one embodiment, as shown in figure 4, step S108 includes:
S1081, the operating index for obtaining the multiple host.
The operating index of host includes one kind or more of the utilization rate of CPU, network I/O interface occupancy and memory usage
Kind.
S1082, the load factor that each host is analyzed according to the operating index.
The utilization rate of CPU is higher, and network I/O interface occupancy is higher, and memory usage is higher, and the load factor of host is got over
It is high.
When using multiple operating index as judge host load factor when, can be commented using the utilization rate of CPU as main
Sentence index, network I/O interface occupancy is taken second place, memory usage third, i.e., when the utilization rate of the CPU of different hosts is different
Or not close (the utilization rate difference of the CPU of two hosts illustrates that the utilization rate of the CPU of two hosts is close within 10%)
When, consider the utilization rate of CPU only to evaluate the load factor of host, i.e. the minimum host of the utilization rate of CPU is that load factor is minimum
Host considers I/O interface occupancy when the utilization rate of CPU is the same or close to evaluate the load factor of host, if different hosts
I/O interface occupancy as or close (the I/O interface occupancy difference of two hosts illustrates two within 10%
The I/O interface occupancy of host is close), consider memory usage, the minimum host of memory usage is minimum as load factor
Host, if different or not close, the minimum host conduct of I/O interface occupancy of the I/O interface occupancy of different hosts
The minimum host of load factor.
S1083, the inquiry instruction is assigned to the minimum host of load factor to inquire required data.
Inquiry instruction is assigned on the minimum host of load factor and executes, and to realize the load balancing of each host, improves industry
Business concurrent power.
The data processing method of the present embodiment refers to according to the load factor of hosts different in cache cluster to select to execute inquiry
The host of order realizes the load balancing of each host in cache cluster, improves service concurrence power.
Fig. 5 is a kind of schematic block diagram of data processing equipment 100 provided in an embodiment of the present invention.The data processing equipment
100 include the unit for executing above-mentioned data processing method, which can be configured in desktop computer, tablet computer, hand
Mention computer, etc. in terminals.The data processing equipment 100 includes the first construction unit 101, the second construction unit 102, taxon
103, first acquisition unit 104 and storage unit 105
First construction unit 101 is for constructing multiple cache clusters, the corresponding IP address of a cache cluster.
For second construction unit 102 for constructing cached configuration table, the cached configuration table includes the type and caching of data
Corresponding relationship between the IP address of cluster.
If taxon 103 is classified to data cached by preset rules for receiving to data cached by described
To obtain a variety of different types of data.
First acquisition unit 104 is corresponding for obtaining a variety of different types of data from the cached configuration table
The IP address of multiple cache clusters.
Storage unit 105 is used to be saved a variety of different types of data to corresponding slow according to acquired IP address
It deposits in cluster.
In one embodiment, it is described to it is data cached include multiple business data.
As shown in fig. 6, the taxon 103 includes following subelement:
First obtains subelement 1031, for obtaining the characteristic attribute of each business datum；And
Sort out subelement 1032, for sorting out the business datum in the multiple business data with same characteristic features attribute
Form same type of data.
In one embodiment, the data processing equipment 100 further includes receiving unit 106, analytical unit 107, reads list
Member 108 and query unit 109.
Receiving unit 106 is used to receive the inquiry instruction for inquiring data, and the inquiry instruction includes querying condition.
Analytical unit 107 is used to analyze the querying condition to obtain the type of required inquiry data.
Reading unit 108 is used to read the IP address of the corresponding cache cluster of the type from the cached configuration table.
Query unit 109 is for inquiring data from the corresponding cache cluster of the IP address.
In one embodiment, as shown in figure 8, the query unit 109 includes following subelement:
Second obtains subelement 1091, for obtaining the operating index of the multiple host；
Subelement 1092 is analyzed, for analyzing the load factor of each host according to the operating index；And
Subelement 1093 is inquired, the inquiry instruction is assigned to the minimum host of load factor to inquire required data.
The specific descriptions of above-mentioned data processing equipment 100 and each unit, the not detailed place of the embodiment of the present invention can join
Embodiment of the method is stated before examination, is not repeated herein.
Above-mentioned data processing equipment 100 can be implemented as a kind of form of computer program, and computer program can be such as
It is run in computer equipment shown in Fig. 9.
Fig. 9 is a kind of structural representation block diagram of computer equipment 200 provided in an embodiment of the present invention.The computer equipment
200, which can be terminal, be also possible to server, wherein terminal can be smart phone, plate electricity
Brain, laptop, desktop computer, personal digital assistant and wearable device etc. have the electronic equipment of communication function.Service
Device can be independent server, be also possible to the server cluster of multiple server compositions.
The computer equipment 200, including processor 202, memory and the network interface connected by system bus 201
205, wherein memory may include non-volatile memory medium 203 and built-in storage 204.
The non-volatile memory medium 203 of the computer equipment 200 can storage program area 2031 and computer program
2032, which is performed, and processor 202 may make to execute a kind of data processing method.The built-in storage
204 provide environment for the operation of the computer program 2032 in non-volatile memory medium 203.The place of the computer equipment 200
Device 202 is managed for providing calculating and control ability, supports the operation of entire computer equipment 200.The network of computer equipment 200
Interface 205 is for carrying out network communication, such as task, the reception data of transmission distribution.
It will be understood by those skilled in the art that the embodiment of computer equipment shown in Fig. 9 is not constituted to computer
The restriction of equipment specific composition, in other embodiments, computer equipment may include components more more or fewer than diagram, or
Person combines certain components or different component layouts.For example, in some embodiments, computer equipment can only include depositing
Reservoir and processor, in such embodiments, the structure and function of memory and processor are consistent with embodiment illustrated in fig. 9,
Details are not described herein.
Processor 202 run non-volatile memory medium 203 in computer program 2032 when, processor 202 execute with
Lower step: multiple cache clusters, the corresponding IP address of a cache cluster are constructed；Cached configuration table is constructed, the caching is matched
Setting table includes the corresponding relationship between the type of data and the IP address of cache cluster；It, will be described if receiving to data cached
Classify to data cached by preset rules to obtain different types of data；It is obtained from the cached configuration table described more
The IP address of the corresponding multiple cache clusters of the different types of data of kind；According to acquired IP address, by a variety of different types
Data saved respectively into corresponding cache cluster.
In one embodiment, it is described to it is data cached include multiple business data；The processor 202 is executing described incite somebody to action
It is described when it is data cached classify by preset rules to obtain the step of different types of data when, it is specific to execute following step
It is rapid: to obtain the characteristic attribute of each business datum；Different business data with same characteristic features attribute are sorted out to be formed it is same
The data of type.
In one embodiment, the processor 202 described saves different types of data to corresponding caching executing
After the step of cluster, also execute following steps: if receiving the inquiry instruction of inquiry data, the inquiry instruction includes inquiry
Condition；Analyze the type of inquiry data needed for the querying condition obtains；The type pair is read from the cached configuration table
The IP address for the cache cluster answered；Data are inquired from the corresponding cache cluster of the IP address.
In one embodiment, the cache cluster includes multiple main frames；The processor 202 is in the step of execution, tool
Body executes following steps: obtaining the operating index of the multiple host；The load of each host is analyzed according to the operating index
Rate；The inquiry instruction is assigned to the minimum host of load factor to inquire required data.
In one embodiment, the operating index includes the utilization rate of CPU, network I/O interface occupancy and EMS memory occupation
Rate it is one or more.
It should be appreciated that in the embodiment of the present application, processor 202 can be central processing unit (Central
Processing Unit, CPU), which can also be other general processors, digital signal processor (Digital
Signal Processor, DSP), specific integrated circuit (Application Specific Integrated Circuit,
ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic
Device, discrete gate or transistor logic, discrete hardware components etc..Wherein, general processor can be microprocessor or
Person's processor is also possible to any conventional processor etc..
Those of ordinary skill in the art will appreciate that be realize above-described embodiment method in all or part of the process,
It is that relevant hardware can be instructed to complete by computer program.The computer program includes program instruction, computer journey
Sequence can be stored in a storage medium, which is computer readable storage medium.The program instruction is by the department of computer science
At least one processor in system executes, to realize the process step of the embodiment of the above method.
Therefore, the present invention also provides a kind of storage medium, the storage medium is stored with one or more than one calculating
Machine program, the one or more computer program can be executed by one or more than one processor, it can be achieved that
Following steps: multiple Pre-Evaluation prices by preset order arrangement are obtained；Using the multiple Pre-Evaluation calculation of price difference because
Subsystem number；The corresponding weighted value of each Pre-Evaluation price is obtained according to the discrimination factor coefficient；According to the multiple Pre-Evaluation
Price and the corresponding weighted value of each Pre-Evaluation price calculate the comprehensive assessment price of the house property to be evaluated.
In one embodiment, described the step of utilizing the multiple Pre-Evaluation calculation of price discrimination factor coefficient is being realized
When, it implements following steps: constructing multiple cache clusters, the corresponding IP address of a cache cluster；Construct cached configuration
Table, the cached configuration table include the corresponding relationship between the type of data and the IP address of cache cluster；If receiving to slow
Deposit data is classified described by preset rules to data cached to obtain different types of data；From the cached configuration
The IP address of the corresponding multiple cache clusters of a variety of different types of data is obtained in table；According to acquired IP address,
A variety of different types of data are saved respectively into corresponding cache cluster.
In one embodiment, it is described to it is data cached include multiple business data；It is described by the number to be cached realizing
When according to classifying by preset rules to obtain the step of different types of data, implementing following steps: obtaining each
The characteristic attribute of business datum；Different business data with same characteristic features attribute are sorted out to form same type of data.
In one embodiment, realize described the step of saving different types of data to corresponding cache cluster it
Afterwards, if also performing the steps of the inquiry instruction for receiving inquiry data, the inquiry instruction includes querying condition；Analysis institute
State the type of inquiry data needed for querying condition obtains；The corresponding cache cluster of the type is read from the cached configuration table
IP address；Data are inquired from the corresponding cache cluster of the IP address.
In one embodiment, the cache cluster includes multiple main frames；In the step of realization, following step is implemented
It is rapid: to obtain the operating index of the multiple host；The load factor of each host is analyzed according to the operating index；By the inquiry
Instruction is assigned to the minimum host of load factor to inquire required data.
In one embodiment, the operating index includes the utilization rate of CPU, network I/O interface occupancy and EMS memory occupation
Rate it is one or more.
The storage medium can be USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), magnetic disk
Or the various computer readable storage mediums that can store program code such as CD.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Specially
Industry technical staff can use different methods to achieve the described function each specific application, but this realization is not
It is considered as beyond the scope of this invention.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary.For example, the division of each unit, only
Only a kind of logical function partition, there may be another division manner in actual implementation.Such as multiple units or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.
The steps in the embodiment of the present invention can be sequentially adjusted, merged and deleted according to actual needs.This hair
Unit in bright embodiment device can be combined, divided and deleted according to actual needs.In addition, in each implementation of the present invention
Each functional unit in example can integrate in one processing unit, is also possible to each unit and physically exists alone, can also be with
It is that two or more units are integrated in one unit.
If the integrated unit is realized in the form of SFU software functional unit and when sold or used as an independent product,
It can store in one storage medium.Based on this understanding, technical solution of the present invention is substantially in other words to existing skill
The all or part of part or the technical solution that art contributes can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, terminal or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection scope subject to.
Claims (10)
1. a kind of data processing method characterized by comprising
Construct multiple cache clusters, the corresponding IP address of a cache cluster；
Cached configuration table is constructed, the cached configuration table includes the corresponding pass between the type of data and the IP address of cache cluster
System；
If receiving to data cached, by it is described classify by preset rules to data cached it is a variety of different types of to obtain
Data；
The IP address of the corresponding multiple cache clusters of a variety of different types of data is obtained from the cached configuration table；With
And
According to acquired IP address, a variety of different types of data are saved respectively into corresponding cache cluster.
2. data processing method according to claim 1, which is characterized in that it is described to it is data cached include multiple business number
According to；
It is described to classify described to data cached by preset rules to obtain a variety of different types of data, comprising:
Obtain the characteristic attribute of each business datum；And
Sort out the business datum in the multiple business data with same characteristic features attribute to form same type of data.
3. data processing method according to claim 2, which is characterized in that described to save different types of data to right
After the cache cluster answered, further includes:
If receiving the inquiry instruction for inquiring data, the inquiry instruction includes querying condition, analyzes the querying condition
To obtain the type of required inquiry data；
The IP address of the corresponding cache cluster of the type is read from the cached configuration table；And
Data are inquired from the corresponding cache cluster of the IP address.
4. data processing method according to claim 3, which is characterized in that the cache cluster includes multiple main frames；
It is described to inquire data from the corresponding cache cluster of the IP address, comprising:
Obtain the operating index of the multiple host；
The load factor of each host is analyzed according to the operating index；And
The inquiry instruction is assigned to the minimum host of load factor to inquire required data.
5. data processing method according to claim 4, which is characterized in that the operating index include CPU utilization rate,
Network I/O interface occupancy and memory usage it is one or more.
6. a kind of data processing equipment characterized by comprising
First construction unit, for constructing multiple cache clusters, the corresponding IP address of a cache cluster；
Second construction unit, for constructing cached configuration table, the cached configuration table includes the type and cache cluster of data
Corresponding relationship between IP address；
Taxon, if classifying for receiving to data cached by described to data cached by preset rules to obtain
A variety of different types of data；
First acquisition unit, it is corresponding multiple slow for obtaining a variety of different types of data from the cached configuration table
Deposit the IP address of cluster；And
Storage unit, for according to acquired IP address, a variety of different types of data to be saved to corresponding cache cluster
In.
7. data processing equipment according to claim 6, which is characterized in that it is described to it is data cached include multiple business number
According to；
The taxon includes:
First obtains subelement, for obtaining the characteristic attribute of each business datum；And
Sort out subelement, for will in the multiple business data with same characteristic features attribute business datum sort out to be formed it is same
The data of type.
8. data processing equipment according to claim 6, which is characterized in that further include:
Receiving unit, for receiving the inquiry instruction for inquiring data, the inquiry instruction includes querying condition；
Analytical unit, the type of inquiry data needed for being obtained for analyzing the querying condition；
Reading unit, for reading the IP address of the corresponding cache cluster of the type from the cached configuration table；And
Query unit, for inquiring data from the corresponding cache cluster of the IP address.
9. a kind of computer equipment, which is characterized in that including memory and the processor being connected with the memory；
The memory is used to store the computer program for realizing data processing method；
The processor is for running the computer program stored in the memory, to execute such as any one of claim 1 to 5
The method.
10. a kind of storage medium, which is characterized in that the storage medium is stored with one or more than one computer program,
The one or more computer program can be executed by one or more than one processor, to realize as right is wanted
Seek 1 to 5 described in any item methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811050633.8A CN109388657B (en) | 2018-09-10 | 2018-09-10 | Data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811050633.8A CN109388657B (en) | 2018-09-10 | 2018-09-10 | Data processing method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109388657A true CN109388657A (en) | 2019-02-26 |
| CN109388657B CN109388657B (en) | 2023-08-08 |
Family
ID=65418765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811050633.8A Active CN109388657B (en) | 2018-09-10 | 2018-09-10 | Data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109388657B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110166429A (en) * | 2019-04-12 | 2019-08-23 | 深圳壹账通智能科技有限公司 | Data package processing method, device, computer readable storage medium and server |
| CN110399272A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Log processing equipment, method, electronic equipment and computer readable storage medium |
| CN111046070A (en) * | 2019-11-21 | 2020-04-21 | 深圳前海环融联易信息科技服务有限公司 | Intelligent data caching method and device, computer equipment and storage medium |
| CN111367672A (en) * | 2020-03-05 | 2020-07-03 | 北京奇艺世纪科技有限公司 | Data caching method and device, electronic equipment and computer storage medium |
| CN111414424A (en) * | 2020-03-23 | 2020-07-14 | 北京思特奇信息技术股份有限公司 | Method, system, medium and device for automatically synchronizing redis of configuration data |
| CN112286971A (en) * | 2020-11-04 | 2021-01-29 | 中国电力财务有限公司 | Cache data management method and device, server and computer storage medium |
| CN113448997A (en) * | 2021-06-22 | 2021-09-28 | 深信服科技股份有限公司 | Cache processing method and device, electronic equipment and storage medium |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101631140A (en) * | 2009-08-03 | 2010-01-20 | 中兴通讯股份有限公司 | Cluster server of instant communication system and method of inter-clusters communication |
| CN101751394A (en) * | 2008-12-16 | 2010-06-23 | 青岛海信传媒网络技术有限公司 | Method and system for synchronizing data |
| CN102316160A (en) * | 2011-06-14 | 2012-01-11 | 贵阳朗玛信息技术股份有限公司 | Website system and communication method thereof |
| US20150153961A1 (en) * | 2011-10-18 | 2015-06-04 | Hitachi Ltd. | Method for assigning storage area and computer system using the same |
| CN105045237A (en) * | 2015-07-22 | 2015-11-11 | 浙江大丰实业股份有限公司 | Intelligent distributed stage data mining system |
| CN105630812A (en) * | 2014-10-30 | 2016-06-01 | 阿里巴巴集团控股有限公司 | Refreshing method and device of cluster application cache |
| CN105744006A (en) * | 2016-05-10 | 2016-07-06 | 中国民航大学 | Particle swarm optimization user request dispatching method facing multi-type service |
| US20160197878A1 (en) * | 2015-01-07 | 2016-07-07 | Sony Corporation | Method and system for processing a geographical internet protocol (ip) lookup request |
| CN105827687A (en) * | 2015-11-17 | 2016-08-03 | 广东亿迅科技有限公司 | Cluster management method and management system thereof |
| CN106095796A (en) * | 2016-05-30 | 2016-11-09 | 中国邮政储蓄银行股份有限公司 | Distributed data storage method, Apparatus and system |
| CN106372136A (en) * | 2010-12-30 | 2017-02-01 | 脸谱公司 | Distributed cache system and method and storage medium |
| CN106874425A (en) * | 2017-01-23 | 2017-06-20 | 福州大学 | Real time critical word approximate search algorithm based on Storm |
| CN107395669A (en) * | 2017-06-01 | 2017-11-24 | 华南理工大学 | A kind of collecting method and system based on the real-time distributed big data of streaming |
| US20170345293A1 (en) * | 2016-05-24 | 2017-11-30 | Iheartmedia Management Services, Inc. | Broadcast traffic information bounding areas |
| CN107426332A (en) * | 2017-08-10 | 2017-12-01 | 华南理工大学 | The load-balancing method and system of a kind of web server cluster |
| CN107704582A (en) * | 2017-10-08 | 2018-02-16 | 安徽康佳电子有限公司 | A kind of closed loop Ecological feed-back catenary system based on server and webpage |
| CN107798062A (en) * | 2017-09-20 | 2018-03-13 | 中国电力科学研究院 | A kind of transformer station's historical data unifies storage method and system |
| CN108418874A (en) * | 2018-02-12 | 2018-08-17 | 平安科技（深圳）有限公司 | Guiding method, device, computer equipment and storage medium are returned across wide area network data |
-
2018
- 2018-09-10 CN CN201811050633.8A patent/CN109388657B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101751394A (en) * | 2008-12-16 | 2010-06-23 | 青岛海信传媒网络技术有限公司 | Method and system for synchronizing data |
| CN101631140A (en) * | 2009-08-03 | 2010-01-20 | 中兴通讯股份有限公司 | Cluster server of instant communication system and method of inter-clusters communication |
| CN106372136A (en) * | 2010-12-30 | 2017-02-01 | 脸谱公司 | Distributed cache system and method and storage medium |
| CN102316160A (en) * | 2011-06-14 | 2012-01-11 | 贵阳朗玛信息技术股份有限公司 | Website system and communication method thereof |
| US20150153961A1 (en) * | 2011-10-18 | 2015-06-04 | Hitachi Ltd. | Method for assigning storage area and computer system using the same |
| CN105630812A (en) * | 2014-10-30 | 2016-06-01 | 阿里巴巴集团控股有限公司 | Refreshing method and device of cluster application cache |
| US20160197878A1 (en) * | 2015-01-07 | 2016-07-07 | Sony Corporation | Method and system for processing a geographical internet protocol (ip) lookup request |
| CN105045237A (en) * | 2015-07-22 | 2015-11-11 | 浙江大丰实业股份有限公司 | Intelligent distributed stage data mining system |
| CN105827687A (en) * | 2015-11-17 | 2016-08-03 | 广东亿迅科技有限公司 | Cluster management method and management system thereof |
| CN105744006A (en) * | 2016-05-10 | 2016-07-06 | 中国民航大学 | Particle swarm optimization user request dispatching method facing multi-type service |
| US20170345293A1 (en) * | 2016-05-24 | 2017-11-30 | Iheartmedia Management Services, Inc. | Broadcast traffic information bounding areas |
| CN106095796A (en) * | 2016-05-30 | 2016-11-09 | 中国邮政储蓄银行股份有限公司 | Distributed data storage method, Apparatus and system |
| CN106874425A (en) * | 2017-01-23 | 2017-06-20 | 福州大学 | Real time critical word approximate search algorithm based on Storm |
| CN107395669A (en) * | 2017-06-01 | 2017-11-24 | 华南理工大学 | A kind of collecting method and system based on the real-time distributed big data of streaming |
| CN107426332A (en) * | 2017-08-10 | 2017-12-01 | 华南理工大学 | The load-balancing method and system of a kind of web server cluster |
| CN107798062A (en) * | 2017-09-20 | 2018-03-13 | 中国电力科学研究院 | A kind of transformer station's historical data unifies storage method and system |
| CN107704582A (en) * | 2017-10-08 | 2018-02-16 | 安徽康佳电子有限公司 | A kind of closed loop Ecological feed-back catenary system based on server and webpage |
| CN108418874A (en) * | 2018-02-12 | 2018-08-17 | 平安科技（深圳）有限公司 | Guiding method, device, computer equipment and storage medium are returned across wide area network data |
Non-Patent Citations (1)
| Title |
|---|
| 涂敬伟等: "基于MapReduce和分布式缓存的KNN分类算法研究", 《微型机与应用》, pages 18 - 21 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110166429A (en) * | 2019-04-12 | 2019-08-23 | 深圳壹账通智能科技有限公司 | Data package processing method, device, computer readable storage medium and server |
| CN110166429B (en) * | 2019-04-12 | 2022-03-22 | 深圳壹账通智能科技有限公司 | Data packet processing method and device, computer readable storage medium and server |
| CN110399272A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Log processing equipment, method, electronic equipment and computer readable storage medium |
| CN110399272B (en) * | 2019-07-29 | 2022-02-18 | 中国工商银行股份有限公司 | Log processing device, method, electronic device, and computer-readable storage medium |
| CN111046070A (en) * | 2019-11-21 | 2020-04-21 | 深圳前海环融联易信息科技服务有限公司 | Intelligent data caching method and device, computer equipment and storage medium |
| CN111367672A (en) * | 2020-03-05 | 2020-07-03 | 北京奇艺世纪科技有限公司 | Data caching method and device, electronic equipment and computer storage medium |
| CN111414424A (en) * | 2020-03-23 | 2020-07-14 | 北京思特奇信息技术股份有限公司 | Method, system, medium and device for automatically synchronizing redis of configuration data |
| CN111414424B (en) * | 2020-03-23 | 2023-08-04 | 北京思特奇信息技术股份有限公司 | Method, system, medium and equipment for automatically synchronizing redis of configuration data |
| CN112286971A (en) * | 2020-11-04 | 2021-01-29 | 中国电力财务有限公司 | Cache data management method and device, server and computer storage medium |
| CN113448997A (en) * | 2021-06-22 | 2021-09-28 | 深信服科技股份有限公司 | Cache processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109388657B (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109388657A (en) | Data processing method, device, computer equipment and storage medium | |
| US10715598B1 (en) | Implementation of a web-scale data fabric | |
| US9043317B2 (en) | System and method for event-driven prioritization | |
| CN107729376B (en) | Insurance data auditing method and device, computer equipment and storage medium | |
| CN108388675A (en) | Circulation method and terminal device are drawn in a kind of identity | |
| CN108446210A (en) | Measure, storage medium and the server of system performance | |
| WO2015148159A1 (en) | Determining a temporary transaction limit | |
| US10067964B2 (en) | System and method for analyzing popularity of one or more user defined topics among the big data | |
| CN110309110A (en) | A kind of big data log monitoring method and device, storage medium and computer equipment | |
| CN109299087A (en) | Data cache method, device, computer equipment and storage medium | |
| CN109617646A (en) | Message forwarding method, device, computer equipment and computer readable storage medium | |
| CN109344157A (en) | Read and write abruption method, apparatus, computer equipment and storage medium | |
| CN109165975A (en) | Label recommendation method, device, computer equipment and storage medium | |
| CN105740434B (en) | Network information methods of marking and device | |
| CN111598360A (en) | Service policy determination method and device and electronic equipment | |
| CN111541628A (en) | Power communication network service resource allocation method and related device | |
| CN107145574A (en) | database data processing method, device and storage medium and electronic equipment | |
| CN109871368A (en) | Database detection method, apparatus, computer installation and storage medium | |
| CN112651826A (en) | Credit limit management and control system, method and readable storage medium | |
| CN111639077A (en) | Data management method and device, electronic equipment and storage medium | |
| CN110309143A (en) | Data similarity determines method, apparatus and processing equipment | |
| CN114781717A (en) | Network point equipment recommendation method, device, equipment and storage medium | |
| CN110543426A (en) | software performance risk detection method and device | |
| CN111652281B (en) | Information data classification method, device and readable storage medium | |
| CN103488693A (en) | Data processing device and data processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |