CN104090901B - A kind of method that data are processed, device and server - Google Patents
A kind of method that data are processed, device and server Download PDFInfo
- Publication number
- CN104090901B CN104090901B CN201310756095.5A CN201310756095A CN104090901B CN 104090901 B CN104090901 B CN 104090901B CN 201310756095 A CN201310756095 A CN 201310756095A CN 104090901 B CN104090901 B CN 104090901B
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- checked
- storage engines
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method processed data, device and server, belong to computer realm.Methods described includes：The query statement that receiving terminal sends, query statement carries the storage location information of data to be checked and the processing mode for the treatment of data to be checked, storage location information according to data to be checked, it is determined that the database of storage data to be checked is data warehouse or storage engines；If it is determined that being data warehouse, then query statement is sent to data warehouse, data warehouse is obtained data to be checked and is processed data to be checked according to processing mode；If it is determined that being storage engines, then query statement is sent to storage engines, storage engines is obtained data to be checked and is processed data to be checked according to processing mode.So that storage engines and data warehouse can play respective advantage, the step of processing data and time can be saved, high degree saves stand-by period and the system resource of user.
Description
Technical field
The present invention relates to computer realm, more particularly to a kind of method processed data, device and server.
Background technology
Data warehouse is a kind of data system for data storage and the function with data analysis, at present usually by it
As the Database Systems on internet backstage, the data for storing magnanimity.And the data analysis function that data warehouse has can
To provide the user data processing service.
At present, when user needs inquiry and processing data, be sent to for the storage location and processing mode of the data by user
Data warehouse, data warehouse obtains the data according to the storage location, and carries out data to the data according to the processing mode.Example
Such as, the data of user's inquiry are the wage that certain company provided in 2013, store table name as number of the payroll of the data
According to storage location, and processing mode is to count the total wages got in 2013 of each employee, and data warehouse is according to the table
Name obtains the wage that the said firm provided in 2013, and according to the processing mode count that each employee got in 2013 it is total
Wage.
Wherein, the time rank that data warehouse is processed data is minute rank, and the big data larger to quantity is entered
Row treatment when, it is necessary to rather plant or longer time, when processing the less small data of data volume, it is also desirable to rather plant or more
For a long time, this process time for small data is oversize, it is impossible to meet requirement of the small data to process time, treatment effeciency compared with
It is low.
The content of the invention
In order to improve the efficiency processed data, the invention provides a kind of method processed data, dress
Put and server.The technical scheme is as follows：
A kind of method processed data, methods described includes：
The query statement that receiving terminal sends, the query statement carries storage location information and the place of data to be checked
The processing mode of the data to be checked is managed, the storage location information at least includes data where the data to be checked
The table name of storehouse table；
According to the storage location information of the data to be checked, it is determined that the database of the data to be checked is stored,
The database includes data warehouse or storage engines；
If it is determined that being data warehouse, then the query statement is sent to the data warehouse, makes the data bins
Storehouse data to be checked according to the storage location acquisition of information and according to the processing mode to the number to be checked
According to being processed；
If it is determined that being storage engines, then the query statement is sent to the storage for storing the data to be checked
Engine, makes storage engines data to be checked according to the storage location acquisition of information and according to the processing mode
The data to be checked are processed.
A kind of device processed data, described device includes：
First receiver module, for the query statement that receiving terminal sends, the query statement carries data to be checked
Storage location information and the treatment data to be checked processing mode, the storage location information at least includes described treating
The table name of database table where the data of inquiry；
First determining module, for the storage location information according to the data to be checked, it is determined that storage is described to be checked
The database of the data of inquiry, the database includes data warehouse or storage engines；
First sending module, for if it is determined that be data warehouse, then the query statement is sent to the data
Warehouse, makes data warehouse data to be checked according to the storage location acquisition of information and according to the processing mode
The data to be checked are processed；
Second sending module, for if it is determined that being storage engines, then it is described the query statement to be sent into storage
The storage engines of data to be checked, the data for making the storage engines to be checked according to the storage location acquisition of information
And the data to be checked are processed according to the processing mode.
A kind of server processed data, the server includes：
One or more processors；With
Memory；
The memory storage has one or more programs, one or more of programs be configured to by one or
Multiple computing devices, one or more of program bags are containing the finger for carrying out a kind of method processed data
Order.
In embodiments of the present invention, when the big data larger to data volume is processed, the data warehouse in process time
Have great advantage relative to storage engines tool, when processing the less small data of data volume, stored in process time and drawn
Hold up and have great advantage relative to data warehouse tool；The load store engine in data warehouse so that storage engines and data warehouse
Respective advantage can be played, short slab when solving the less small data of data warehouse processing data amount in process time,
The process step and process time processed data can be saved, high degree reduces the stand-by period of user and saving
System resource.
Brief description of the drawings
Fig. 1 is a kind of storage system provided in an embodiment of the present invention；
Fig. 2 is a kind of method flow diagram processed data that the embodiment of the present invention 1 is provided；
Fig. 3-1 is a kind of method flow diagram of storage small data that the embodiment of the present invention 2 is provided；
Fig. 3-2 is a kind of method flow diagram of storage big data that the embodiment of the present invention 2 is provided；
Fig. 3-3 is a kind of method flow diagram of data storage that the embodiment of the present invention 2 is provided；
Fig. 3-4 is the method flow diagram of a kind of inquiry that the embodiment of the present invention 2 is provided and processing data；
Fig. 3-5 is the method flow diagram of a kind of inquiry that the embodiment of the present invention 2 is provided and processing data；
Fig. 4 is a kind of apparatus structure schematic diagram processed data that the embodiment of the present invention 3 is provided；
Fig. 5 is a kind of server architecture schematic diagram processed data that the embodiment of the present invention 4 is provided.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing to embodiment party of the present invention
Formula is described in further detail.
Referring to Fig. 1, a kind of storage system is the embodiment of the invention provides, including：
Control end 1, the storage engines 3 of data warehouse 2 and at least one, control end 1 can be integrated in data warehouse 2.
Data warehouse is a kind of data system for data storage and the function with data analysis, and it is to storage
When data are processed, processing procedure is cumbersome, specially：The SQL that will be received first（Structured Query Language,
SQL）Statement translation is multiple MapReduce（Mapping and stipulations programming model）Program；Each MapReduce
Program initializes respective process and starts the process of initialization simultaneously respectively, then is obtained from the database table of data warehouse and treat
The data for importing, then by pending data importing process, are carried out Map computings and Reduce fortune by the data for the treatment of afterwards
Calculate, wherein Map computings are that the data that will be imported are grouped by predetermined way, and Reduce computings are by treatment side by the data after packet
Formula is processed, final output result, while will interim findings that constantly clean up process is obtained in calculating process.
Data warehouse can simultaneously run multiple processes, when process mass data when, on the required time, its relative to
Storage engines have very big advantage；But no matter data volume is much, it is required for being performed according to above-mentioned steps, therefore required time
At least it is also required to a few minutes.
Storage engines can data storage and can according to processing mode to store data process.Storage engines have
There is very powerful Statistical information system and query optimizer, the two is combined can classify the data of storage and sort, to enable
Enough storage locations for clearly obtaining each class data, the speed when data are inquired about is very fast, you can with the very short time
Obtain the storage location and quick obtaining data to be checked of the data to be checked for needing inquiry, its speed ratio for obtaining data
The speed that data warehouse obtains data is many soon, and storage engines can directly perform SQL statement, and can directly according to
Processing mode is processed the data for obtaining, without the SQL statement of reception is translated as into multiple MapReduce programs,
Respective process need not be initialized and start the process of initialization, it is not necessary to which the data of importing are carried out into Map computings and Reduce
Computing, and need not interim findings that constantly clean up process is obtained in calculating process, therefore the acquisition of storage engines treats
The processing procedure that data of the speed of the data of inquiry quickly and to obtaining are processed is easy, less small in processing data amount
During data, the consuming time only needs several milliseconds；But storage engines only one of which computing node, i.e., can only simultaneously start one and enter
Journey, the larger data of processing data amount are provided no advantage against, it usually needs spend dozens of minutes.
Therefore, in the less small data of processing data amount, the time needed for the processing data of data warehouse 2 draws more than storage
Time needed for holding up 3 processing datas, for example, the time needed for the processing data of data warehouse 2 is a few minutes, more than storage engines 3
Several seconds of time needed for processing data；When therefore processing the less small data of data volume, stored in process time
Engine 3 has great advantage relative to the tool of data warehouse 2.
In the larger big data of processing data amount, the time needed for the processing data of storage engines 3 is more than at data warehouse 2
Time needed for reason data, for example, the time needed for the processing data of storage engines 3 is dozens of minutes, processed more than data warehouse 2
A few minutes time needed for data；Therefore, when the big data larger to data volume is processed, the data warehouse in process time
2 have great advantage relative to the tool of storage engines 3.
The load store engine 3 in data warehouse 2 so that storage engines 3 and data warehouse 2 can play respective excellent
Gesture, short slab when solving the less small data of the processing data amount of data warehouse 2 in process time, can save to enter data
The process step and process time of row treatment, can reduce stand-by period and the saving system resource of user with high degree.
If user needs the less small data of data storage amount, such as decimal of the data volume less than or equal to predetermined threshold value
According to predetermined threshold value can be 8GB, 10GB or 12GB etc., then can store in storage engines 3 small data.
If user needs the larger big data of data storage amount, such as data volume then may be used more than the big data of predetermined threshold value
Stored in data warehouse 2 with by big data.
In the storage system shown in Fig. 1, data warehouse 2 and storage engines 3 are all a kind of data for data storage
System and can to store data be analyzed.
In the less small data of processing data amount, very short and general of process time needs several milliseconds to storage engines 3, and counts
According to warehouse 2 in the less small data of processing data amount, process time is very long typically at least to need a few minutes.Therefore, to data
When measuring less small data and being processed, storage engines 3 have great advantage relative to the tool of data warehouse 2 in process time, energy
Enough save substantial amounts of time and system resource.
, in the larger big data of processing data amount, process time is very short and typically only needs a few minutes, and deposits for data warehouse 2
In the larger big data of processing data amount, process time is very long typically at least to need dozens of minutes to storage engine 3.Therefore, logarithm
When being processed according to the larger big data of amount, data warehouse 2 has great advantage relative to the tool of storage engines 3 in process time,
Substantial amounts of time and system resource can be saved.
Embodiment 1
Referring to Fig. 2, a kind of method processed data is the embodiment of the invention provides, the method is used for shown in Fig. 1
Storage system, the method flow includes：
Step 101：The query statement that receiving terminal sends, query statement carries the storage location information of data to be checked
With the processing mode for processing data to be checked, storage location information is at least including the table of database table where data to be checked
Name；
Step 102：Storage location information according to data to be checked, it is determined that the database of storage data to be checked,
Database includes data warehouse or storage engines；
Step 103：If it is determined that being data warehouse, then query statement is sent to data warehouse, makes data warehouse root
Data to be checked are processed according to storage location acquisition of information data to be checked and according to processing mode；
Step 104：If it is determined that being storage engines, then query statement is sent to the storage of storage data to be checked
Engine, makes storage engines enter to data to be checked according to storage location acquisition of information data to be checked and according to processing mode
Row treatment.
In embodiments of the present invention, when the big data larger to data volume is processed, the data warehouse in process time
Have great advantage relative to storage engines tool, when processing the less small data of data volume, stored in process time and drawn
Hold up and have great advantage relative to data warehouse tool；The load store engine in data warehouse so that storage engines and data warehouse
Respective advantage can be played, short slab when solving the less small data of data warehouse processing data amount in process time,
The process step and process time processed data can be saved, high degree reduces the stand-by period of user and saving
System resource.
Embodiment 2
Referring to Fig. 3-1, if user needs data storage amount less than or equal to the small data of predetermined threshold value, the small data is
Data to be stored, then set up a database table in storage engines first, then by data storage to be stored in foundation
In database table, can specifically be realized by the step of following 201 to 209.
201：Terminal obtains the mark of the storage engines of user's selection, sends first and builds table statement to control end；First builds
Table statement at least carries the mark of the storage engines, the table name of database table to be created, what database table to be created included
The row name and data type of each row；
When user is needed data storage to be stored on storage engines, a mark for storage engines is selected in terminal
Know, and the table name and data to be created of the database table to be created of establishment on the storage engines are needed to terminal input
The row name and data type of each row that storehouse table includes.
Terminal obtains the mark of the storage engines of user's selection, and the database table to be created for receiving user input again
Table name and each row for including of database table to be created row name and data type, then retransmit first and build table statement to control
End processed.
For example, it is assumed that predetermined threshold value is 10GB, user needs to store the wage data that AA companies provide in the January, the work
Money data are data to be stored, and data to be stored only account for 100MB capacity, and the data volume of data to be stored is less than default threshold
Value 10GB, therefore user can be specifically as follows by data storage to be stored in storage engines：User is first in terminal
One storage engines of selection are designated ID1, then payroll from the storage engines to terminal input that need to be created in table name
Be " AA companies-January-payroll ", and each row for including of payroll row name and data type, the payroll includes two
Row, the row entitled " employee name " and data type of first row are " string ", the row entitled " wage number " of secondary series with
And data type is " int ".
Then terminal sends first and builds table statement to control end, and first builds the mark that table statement at least carries the storage engines
ID1, the table name of payroll, the row name and data type of each row that payroll includes.
202：Control end receives first and builds table statement, obtains the mark of the first storage engines for building table statement carrying；
Further, control end also obtains the number of each row that the first database table to be created for building table statement carrying includes
According to type, the data type that the storage engines are not supported, the data type that will be determined are determined from the data type of each row
The data type of storage engines support is converted to, is implemented as follows：
Control end obtains the data type of each row that the first database table to be created for building table statement carrying includes, according to
The mark of the first storage engines for building table statement carrying, obtains the corresponding data type conversion relation table of the storage engines, should
Data type conversion relation table is used to store data type to be converted pass corresponding with the data type that the storage engines are supported
System, for example, table 1 is the corresponding data type conversion relation table of certain storage engines, first is classified as data type to be converted, the
Two are classified as the data type of storage engines support, and all data types of storage engines support are stored in secondary series.
Further according to the data type conversion relation table, determine what the storage engines were not supported from the data type of each row
Data type, each data type that will be determined as data type to be converted, and from the data type conversion relation table
The corresponding data type of each data type that acquisition is determined, each correspondence is respectively converted into by each data type for determining
Data type.For example, the data type determined is int, using data type int as data type to be converted,
And it is integer that the corresponding data types of data type int are obtained from data type conversion table as shown in table 1, will wait to turn
The data type int for changing is converted to data type integer.
Table 1
| The data type that storage engines are supported | Data type to be converted |
| integer | int |
| varchar | string |
| numeric | double |
| bit boolean | boolean |
| tinyint | tinyint |
| bigint | bigint |
203：Control end sends first and builds table statement to storage engines according to the mark of storage engines；
Further, the table name of the database table to be created that control end can also build table statement carrying by first and storage
In the table name of database table with the corresponding relation of the mark of storage engines, tool entity is realized as follows for the mark storage of engine：
Control end is searched in the table name of database table with the corresponding relation of the mark of storage engines includes that first builds predicative
The record of the mark of the storage engines that sentence is carried, if not finding the mark including the first storage engines for building table statement carrying
Record, then by first build table statement carrying database table to be created table name and storage engines mark store in data
In the corresponding relation of the table name of storehouse table and the mark of storage engines, if find drawn including the first storage for building table statement carrying
The record of the mark held up, then the table name of the database table to be created of table statement carrying is built in addition first in the record.
For example, control end is searched in the table name of the database table shown in table 2 with the corresponding relation of the mark of storage engines
The record of the mark ID1 including storage engines, and find the record of the mark ID1 including storage engines, then in the record
The table name " AA companies-January-payroll " of payroll is added, table 3 is obtained
The corresponding relation of the table name of shown database table and the mark of storage engines.
Table 2
| The table name of database table | The mark of storage engines |
| " class " | ID1 |
| " student performance ", " school ", " age per capita " | ID2 |
| " per tour average achievement ", " order commodity " | ID3 |
Table 3
| The table name of database table | The mark of storage engines |
| " class ", " AA companies-January-payroll " | ID1 |
| " student performance ", " school ", " age per capita " | ID2 |
| " per tour average achievement ", " order commodity " | ID3 |
204：Storage engines receive first and build table statement, and building table statement according to first creates database table to be created；
Specifically, the row name of each row that storage engines include according to the first database table to be created for building table statement carrying
Determine the columns of database table to be created, the database table of blank is created according to the columns for determining, table statement is built according to first
The row name of each row that the table name and database table to be created of the database table to be created for carrying include, sets the number of the blank
According to storehouse table.
Further, storage engines send the first notification message to control end, the first notification message be used for notify user into
Work(creates database table；Control end receives the first notification message and is transmitted to terminal；Terminal receives the first notification message, points out to use
Family storage engines have successfully created database table to be created.
After terminal shows the prompt message, user submits storage order to terminal, makes terminal by following flow by terminal
Data storage to be stored in storage engines.
205：Terminal sends the table name and data to be stored of database table to control end, and the table name of the database table is upper
State the table name of database table to be created；
For example, the name " Zhang San " and the corresponding wage of Zhang San " 1000 yuan " of data to be stored for employee, the surname of employee
Name " Li Si " and the corresponding wage of Li Si " 2000 yuan ", and employee name " king five " and the corresponding wage " 3000 of king five
Unit ", and the table name of data place database table to be stored is " AA companies-January-payroll "；" Zhang San ", " Li Si " and
The row of " king five " corresponding row are entitled " employee name ", and the row of " 1000 yuan ", " 2000 yuan " and " 3000 yuan " corresponding row are entitled
" wage number ".
206：Control end receives the table name and data to be stored of the database table；
For example, control end receive data to be stored and database table where data to be stored table name " AA companies-
January-payroll ".
207：Control end obtains the mark of the storage engines for storing data to be stored according to the table name of the database table
Know；
Specifically, according to the table name of the database table, look-up table list of file names, table name list is used for data storage storehouse to control end
The table name of each database table that storehouse includes, and find out the table name in table name list in the absence of the database table, it is determined that go out
Need by data storage to be stored in storage engines, according to the table name of the database table, table name from database table with deposit
The mark of corresponding storage engines is obtained in the mark for storing up engine, using the mark of the storage engines of acquisition as waiting to deposit for storing
The mark of the storage engines of the data of storage.
208：According to the mark of the storage engines for obtaining, by the table name of the database table and data is activation to be stored to right
The storage engines answered；
209：Storage engines receive the table name and data to be stored of the database table, by data storage to be stored at this
In the corresponding database table of table name of database table.
Specifically, storage engines receive the table name and data to be stored of the database table, according to the table of the database table
Name obtains corresponding database table, the row name of the row where obtaining every data that data to be stored include, according to every number
Stored in corresponding row per data according to the row star of the row at place.
For example, the name " Zhang San " of employee and the corresponding wage of Zhang San " 1000 yuan " are respectively stored in into the entitled " employee of row
In the row of name " and " wage number ", the name " Li Si " of employee and the corresponding wage of Li Si " 2000 yuan " are respectively stored in
In arranging the row of entitled " employee name " and " wage number ", and by the name " king five " and the corresponding wage of king five of employee
" 3000 yuan " are respectively stored in the row of row entitled " employee name " and " wage number ".
Referring to Fig. 3-2, if user needs data storage amount more than the big data of predetermined threshold value, the big data is to be stored
Data, then set up a database table in data warehouse first, then by data storage to be stored in the database set up
In table, specifically realized by the step of following 301 to 305.
301：Terminal sends second and builds table statement to control end, and second builds table statement at least carries database table to be created
Table name, each row that database table to be created includes row name and data type；
When user is needed data storage to be stored on data warehouse, directly needed in data warehouse to terminal input
The row name and data type of each row that the table name and database table to be created of the database table to be created of upper establishment include.
What the table name and database table to be created that terminal receives only the database table to be created of user input included
The row name and data type of each row, and determine that user is needed by data storage to be stored in data warehouse, and send second
Table statement is built to control end.
For example, user needs to store sales figure data of the BB companies in 2013, the sales figure data are to be stored
Data, data to be stored account for 15GB capacity, and the data volume of data to be stored is more than predetermined threshold value 10GB, therefore user can
With designated terminal by data storage to be stored in data warehouse, specially：User is needed in data warehouse to terminal input
The table name of the sales figure table of middle establishment is " BB companies -2013 years-sales figure table ", and each row that sales figure table includes
Row name and data type, the sales figure table include two row, the row entitled " trade name " and data type of first row are
" string ", the row entitled " sales volume " and data type of secondary series are " int ".
302：Control end receives second and builds table statement, and table statement is built according to second, creates to be created in data warehouse
Database table；
Specifically, the row name of each row that control end includes according to the second database table to be created for building table statement carrying is true
The columns of fixed database table to be created, the database table of blank is created according to the columns for determining, is built table statement according to second and is taken
The row name and data type of each row that the table name and database table to be created of the database table to be created of band include, setting should
The database table of blank.
Further, the table name storage of the database table to be created that control end builds table statement carrying by second is in table name row
In table.
Further, control end sends second notification message to terminal, and second notification message is used to notify that user successfully creates
Build database table to be created；Terminal receives second notification message, points out user data warehouse to successfully create to be created
Database table.
After terminal shows the prompt message, user submits storage order to terminal, makes terminal by following flow by terminal
Data storage to be stored in data warehouse.
303：Terminal sends the table name and data to be stored of database table to control end, and the table name of the database table is upper
State the table name of database table to be created；
For example, data to be stored are trade name " shampoo " and the corresponding sales volume of shampoo " 4000 ", commodity
Title " toothpaste " and the corresponding sales volume of toothpaste " 5000 ", and trade name " perfumed soap " and the corresponding sales volume of perfumed soap
The table name of database table where " 6000 ", and data to be stored is " BB companies -2013 years-sales figure table "；" shampoo ",
The row of " toothpaste " and " perfumed soap " corresponding row are entitled " trade name ", the row name of " 4000 ", " 5000 " and " 6000 " corresponding row
It is " sales volume ".
304：Control end receives the table name and data to be stored of the database table, according to the table name of the database table, really
Determine data storage to be stored in data warehouse；
For example, control end receive data to be stored and database table where data to be stored table name " BB companies-
The table of-sales figure in 2013 ", it is determined that by data storage to be stored in data warehouse.
305：Control end in data warehouse by data storage to be stored the database table the corresponding data of table name
In the table of storehouse.
Specifically, control end obtains corresponding database table according to the table name of the database table, obtains data to be stored
Including every data where row row name, the row star of the row according to where every data stored corresponding per data
In row.
For example, it is entitled that trade name " shampoo " and the corresponding sales volume of shampoo " 4000 " are respectively stored in into row
In the row of " trade name " and " wage number "；Trade name " toothpaste " and the corresponding sales volume of toothpaste " 5000 " are deposited respectively
Store up in the row of row entitled " employee name " and " wage number "；By trade name " perfumed soap " and the corresponding sales volume of perfumed soap
" 6000 " are respectively stored in the row of row entitled " employee name " and " wage number ".
In embodiments of the present invention, it is also possible to by the following method flow by data storage to be stored in data warehouse or
In storage engines.User selects the data for needing storage in terminal, and is input into database table to be created to terminal
Table name, each row that database table to be created includes row name and data type.The data that terminal selects user are used as treating
The data of storage, the data volume size of acquisition data to be stored, and receive the database table to be created of user input
Table name, the row name and data type of each row that database table to be created includes, transmission builds table statement to control end, and this builds predicative
Sentence carries the data volume size of data to be stored, the table name of database table to be created, what database table to be created included
The row name and data type of each row.
Control end receives this and builds table statement, the data volume size according to data to be stored, it is determined that for storing to be stored
Data database,（If for example, the data volume size of data to be stored is more than predetermined threshold value, it is determined that for storing
The database of data to be stored is data warehouse, if the population size of data to be stored is less than or equal to predetermined threshold value,
Then determine that for the database for storing data to be stored be storage engines）.
If it is determined that being data warehouse, then control end creates database table to be created in data warehouse, and in number
According to storing data to be stored in warehouse, and the step of implement process and may refer to above-mentioned steps 302 to 305, herein not
Describe in detail again.
If it is determined that being storage engines, then a mark for storage engines is selected（For example, selection residual memory space is most
The mark of big storage engines）, according to the mark of the storage engines, sending this and build table statement and give the storage engines, the storage is drawn
Hold up and create database table to be created and store data to be stored, and implement process may refer to above-mentioned steps 204 to
209 the step of, no longer describe in detail herein.
For example, with reference to Fig. 3-3, next carried out with the process of specific example data to be stored to above-mentioned storage
Describe in detail, data warehouse can be TDW（Tencent Distributed Warehouse, Tengxun's Distributed Data Warehouse）,
Storage engines can be PostgreSQL（Storage engines）.
Establishment database table to be created that control end receiving terminal sends builds table statement create stored as
Pgdata, this builds the mark that table statement carries storage engines PostgreSQL, the table name of database table to be created, to be created
The row name and data type of each row that database table includes, send this and build table statement to PostgreSQL；PostgreSQL is received
This is built table statement and attempts basis and builds table statement establishment database table；
If creating failed, notify that control end PostgreSQL creates database table not successfully；Control end is then logical
Know that terminal PostgreSQL creates database table, end operation not successfully.
If created successfully, notify that control end PostgreSQL successfully creates database table；Control end will be to be created
The mark storage of the table name and PostgreSQL of database table is closed in the correspondence of the mark of the table name and storage engines of database table
In system, and notify that terminal PostgreSQL successfully creates database table；Terminal will need the data to be stored of storage and wait to deposit
The table name of database table is sent to control end where the data of storage；Control end receives data and data institute to be stored to be stored
In the table name of database table, the table name of database table, draws in table name and the storage of database table according to where data to be stored
The mark of corresponding storage engines PostgreSQL is obtained in the corresponding relation of the mark held up, the mark hair according to PostgreSQL
Data to be stored are sent to PostgreSQL；PostgreSQL receives data to be stored, is deposited in the database table for having created
Storage data to be stored, end operation.
Referring to Fig. 3-4, if user needs to inquire about and process the data for having stored, user can be sent with designated terminal and be inquired about
Sentence is to control end, so that control end obtains data according to the query statement, the data to obtaining are processed, and treatment is tied
Fruit returns to terminal, can specifically be realized by the step of following 401 to 408.
401：Terminal obtains the query statement of user input, sends the query statement to control end, and the query statement is carried
The storage location information of data to be checked and the processing mode for the treatment of data to be checked, storage location information at least include treating
The table name of database table where the data of inquiry；
When user needs to inquire about and process the data for having stored, directly to terminal input inquiry sentence, the query statement is taken
The storage location information of the data to be checked with inquiry is needed and the processing mode for the treatment of data to be checked, storage location letter
Breath at least includes the table name of database table where data to be checked.
Terminal obtains the query statement of user input and sends the query statement to control end.
For example, user needs to inquire about the average salary number that AA companies provide in the January, can be input into terminal and looked into
Sentence is ask, the query statement carries the storage of the wage data for needing the data AA companies to be checked of inquiry to be provided in the January
Positional information, and processing mode is to calculate the average salary number that AA companies provide in the January, and the storage location information is extremely
The table name " AA companies-January-payroll " of the payroll where including data to be checked less.
402：The query statement that control end receiving terminal sends, the number according to where the data to be checked that query statement is carried
According to the table name of storehouse table, look-up table list of file names, if table name list includes the table name of database table where data to be checked,
Data storage to be checked is determined in data warehouse, performs step 403；If not including data to be checked in table name list
The table name of place database table, it is determined that data storage to be checked performs step 406 in storage engines；
For example, the query statement that control end receiving terminal sends, obtains the data place to be checked that query statement includes
Payroll table name " AA companies-January-payroll ", search table name list as shown in table 4, and the table name shown in table 4
Do not include the table name of entitled " AA companies-January-payroll " in list, perform step 406.
Table 4
| " student performance " |
| " class " |
| " employee's age " |
403：The query statement is sent to data warehouse by control end；
404：Data warehouse receives the query statement, and the storage location acquisition of information according to data to be checked is to be checked
Data to be checked are processed, and result is sent into control by data according to the processing mode that the query statement is carried
End processed；
405：Control end reception processing result is simultaneously sent to terminal, end operation；
406：The table name of control end database table according to where data to be checked, obtains for storing data to be checked
Storage engines mark, and send query statement to obtain storage engines the corresponding storage engines of mark；
Specifically, the table name of control end database table according to where data to be checked, from the database table for having stored
Table name and the mark that corresponding storage engines are found out in the corresponding relation of the mark of storage engines, the storage engines that will be found out
Mark as be used for store data to be checked storage engines mark；And send query statement and draw to the storage for finding out
The corresponding storage engines of mark held up.
For example, searched in the table name of the database table shown in table 2 with the corresponding relation of the mark of storage engines including table
The record of name " AA companies-January-payroll ", the corresponding database table of the table name is the wage that AA companies provide in the January
The payroll of data, and the record including table name " AA companies-January-payroll " is found, obtain what the record included
The mark ID1 of storage engines, sends mark ID1 corresponding storage engines of the query statement to storage engines.
407：Storage engines receive the query statement, and the storage location acquisition of information according to data to be checked is to be checked
Data to be checked are processed, and result is sent into control by data according to the processing mode that the query statement is carried
End processed；
For example, storage engines receive query statement, database table where obtaining data to be checked is obtained in the tables of data
The wage number " 1000 yuan ", " 2000 yuan " and " 3000 yuan " that AA companies provide in the January to each employee is taken, and calculates hair
The average salary number " 2000 yuan " put, the average salary number that will be calculated " 2000 yuan " is sent to control end.
408：Control end reception processing result is simultaneously transmitted to terminal, end operation.
, wherein it is desired to explanation, if storage engines cannot be successfully processed data to be checked, sends treating for acquisition
The data of inquiry are to control end；Control end receives data to be checked, and the processing mode according to query statement carrying is to be checked
The data of inquiry are processed, and result is sent into terminal.
For example, with reference to Fig. 3-5, the mistake of the data for next having been stored to above-mentioned inquiry and treatment with a specific example
Journey is described in detail, and the query statement that control end receiving terminal sends, query statement carries the number to be checked for needing inquiry
According to the table name and the processing mode for the treatment of data to be checked of place database table, in table name and the storage engines of database table
Being searched in the corresponding relation of mark includes the record of the table name of the database table that query statement is carried；If found out including inquiry
The record of the table name of the database table that sentence is carried, then obtain the mark of the storage engines PostgreSQL in the record, according to
The mark of PostgreSQL, PostgreSQL is sent to by query statement；PostgreSQL receives query statement, according to inquiry language
The table name of database table where the data to be checked that sentence includes obtains data to be checked, and attempts to be checked according to processing
The processing mode of data is processed data to be checked；If processed successfully, result is returned into control end；Control
End receives result and is sent to terminal, end operation；If treatment is unsuccessful, the data is activation to be checked that will be obtained
To control end, control end receives data to be checked, and according to processing the processing mode of data to be checked to number to be checked
According to being processed, result is sent to terminal, end operation.
Wherein, data warehouse only supports the renewal of subregion granularity formula when the data of its storage are updated, for example, it is desired to
When deleting some data in a certain database table that it includes, it is necessary first to determine the data that need to delete and need not
The data of deletion, then delete all data in the database table, and the data that need not be deleted finally are re-write into the number
According in the table of storehouse；Whole process complex steps, required time is very long and wastes substantial amounts of system resource.And storage engines are updating
During the data of its storage, the data deleted can will be needed directly to be deleted from database table, compared with data warehouse, be walked in operation
Suddenly, the aspect such as time tool has great advantage.
In addition, the outer computer language interface of data warehouse does not enrich, only the interface of a few computers language,
It is only capable of the data that the program by being write by a small number of computer languages obtains data warehouse storage；But the outside of storage engines
Computer language interface enriches very much, and almost can obtain storage engines by the program write by any computer language stores
Data；Connection is set up by the interface of data warehouse and the interface of storage engines between, when user is used by calculating
When the program that machine language is write obtains the data stored in data warehouse, even if the computer language data warehouse that user uses is not
Possess the computer language interface that user uses, but because storage engines possess the computer language interface that user uses, user
The data in data warehouse can also be obtained, is specifically as follows：Program is inputed to storage engines by user；Storage engines are obtained to be used
The program of family input, by the linkup transmit program between storage engines and data warehouse to data warehouse；Data warehouse is received
Program simultaneously obtains data according to program, sends data to storage engines；User can obtain the data stored in data warehouse,
Ground connection enriches the external language interface of data warehouse.
In embodiments of the present invention, when the big data larger to data volume is processed, the data warehouse in process time
Have great advantage relative to storage engines tool, when processing the less small data of data volume, stored in process time and drawn
Hold up and have great advantage relative to data warehouse tool；The load store engine on data warehouse so that user can by data volume compared with
Small small data is stored in storage engines, and the larger big data of data volume is stored in data warehouse；User needs inquiry
And when processing the data for having stored, if the data are small data and store in storage engines, query statement is sent to
Storage engines are so that storage engines obtain the data and the data are processed；If the data are big data and store in number
According to warehouse, then query statement is sent to data warehouse so that data warehouse obtains the data and the data are processed；
So that storage engines and data warehouse can play respective advantage；Load store engine solves number in data warehouse
Short slab during small data less according to warehouse processing data amount in process time, can save the treatment processed data
Step and process time, high degree reduce the stand-by period of user and save system resource.
Embodiment 3
Referring to Fig. 4, a kind of device processed data is the embodiment of the invention provides, including：
First receiver module 501, for the query statement that receiving terminal sends, query statement carries data to be checked
The processing mode of storage location information and treatment data to be checked, storage location information is at least included where data to be checked
The table name of database table；
First determining module 502, for the storage location information according to data to be checked, it is determined that storage number to be checked
According to database, database include data warehouse or storage engines；
First sending module 503, for if it is determined that be data warehouse, then query statement is sent to data warehouse,
Make data warehouse according to storage location acquisition of information data to be checked and according to processing mode to data to be checked at
Reason；
Second sending module 504, for if it is determined that being storage engines, then it is to be checked query statement to be sent into storage
Data storage engines, storage engines is treated according to storage location acquisition of information data to be checked and according to processing mode
The data of inquiry are processed.
Preferably, the first determining module 502 includes：
Searching unit, for the table name look-up table list of file names of the database table according to where data to be checked, table name list
The table name of the database table included for data storage warehouse；
First determining unit, if including the table name of database table where data to be checked for table name list,
Data storage to be checked is determined in data warehouse, if not including database table where data to be checked in table name list
Table name, it is determined that data storage to be checked is in storage engines.
Preferably, the second sending module 504 includes：
Acquiring unit, for the table name of the database table according to where data to be checked, from the database table for having stored
The mark of the storage engines of the table name data to be checked with acquisition storage in the corresponding relation of the mark of storage engines；
Transmitting element, for the mark according to storage engines, depositing for storage data to be checked is sent to by query statement
Storage engine.
Further, the device also includes：
Processing module, if data to be checked cannot be processed for storage engines, according to depositing for data to be checked
Storage positional information, obtains data to be checked from storage engines, and data to be checked are processed according to processing mode.
Further, the device also includes：
Second receiver module, table statement is built for what receiving terminal sent, builds the mark that table statement at least carries storage engines
Know, the table name of database table to be created, the row name and data type of each row that database table to be created includes；
3rd sending module, for the mark according to storage engines, transmission is built table statement to storage engines, makes storage engines
Create database table to be created.
Further, the device also includes：
3rd receiver module, table statement is built for what receiving terminal sent, is built table statement and is at least carried data to be stored
Data volume size, the table name of database table to be created, the row name and data class of each row that database table to be created includes
Type；
Second determining module, for according to data volume size, it is determined that the database for storing data to be stored；
4th sending module, for if it is determined that be data warehouse, then data to be created are created in data warehouse
Storehouse table；
5th sending module, for if it is determined that be storage engines, then a mark for storage engines being selected, according to depositing
The mark of engine is stored up, transmission is built table statement to storage engines, storage engines is created database table to be created.
Preferably, the second determining module includes：
Second determining unit, if being more than predetermined threshold value for data volume size, it is determined that for storing number to be stored
According to database be data warehouse；
3rd determining unit, if being less than or equal to predetermined threshold value for data volume size, it is determined that wait to deposit for storing
The database of the data of storage is storage engines.
Further, the device also includes：
3rd determining module, for determining that storage is drawn in the data type of each row included from database table to be created
Hold up the data type do not supported；
Modular converter, the data class that the data type conversion for will be determined in table statement is built is supported for storage engines
Type.
Further, the device also includes：
Memory module, the table name of database table to be created and the mark of storage engines for will build table statement carrying are deposited
Store up in the table name of database table with the corresponding relation of the mark of storage engines.
In embodiments of the present invention, when the big data larger to data volume is processed, the data warehouse in process time
Have great advantage relative to storage engines tool, when processing the less small data of data volume, stored in process time and drawn
Hold up and have great advantage relative to data warehouse tool；The load store engine in data warehouse so that storage engines and data warehouse
Respective advantage can be played, short slab when solving the less small data of data warehouse processing data amount in process time,
The process step and process time processed data can be saved, high degree reduces the stand-by period of user and saving
System resource.
Embodiment 4
Referring to Fig. 5, a kind of server processed data is the embodiment of the invention provides, including：
CPU（CPU）601st, including random access memory（RAM）602 and read-only storage（ROM）603
System storage 604, and connection system memory 604 and CPU 601 system bus 605.The server
600 basic input/outputs for also including transmission information between each device in help computer（I/O systems）606, and
For the mass-memory unit 607 of storage program area 613, application program 610 and other program modules 614.
The basic input/output 606 is included for the display 608 of display information and for user input letter
The input equipment 609 of such as mouse, keyboard etc of breath.Wherein described display 608 and input equipment 609 are all by being connected to
The IOC 610 of system bus 605 is connected to CPU 601.The basic input/output 606
Can also including IOC 610 for receive and process from etc. keyboard, mouse or electronic touch pen it is multiple its
The input of his equipment.Similarly, IOC 610 also provides output to display screen, printer or other kinds of defeated
Go out equipment.
The mass-memory unit 607 is by being connected to the bulk memory controller of system bus 605（It is not shown）
It is connected to CPU 601.The mass-memory unit 607 and its associated computer-readable medium are server
600 provide non-volatile memories.That is, the mass-memory unit 607 can include such as hard disk or CD-ROM
The computer-readable medium of driver etc（It is not shown）.
Without loss of generality, the computer-readable medium can include computer-readable storage medium and communication media.Computer
Storage medium is including for storage computer-readable instruction, data structure, program module or information etc. other data
Volatibility and non-volatile, removable and irremovable medium that any method or technique is realized.Computer-readable storage medium includes
RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storages its technologies, CD-ROM, DVD or other optical storages, tape
Box, tape, disk storage or other magnetic storage apparatus.Certainly, skilled person will appreciate that the computer-readable storage medium
It is not limited to above-mentioned several.Above-mentioned system storage 604 and mass-memory unit 607 may be collectively referred to as memory.
According to various embodiments of the present invention, the server 600 can also be arrived by network connections such as internets
Remote computer operation on network.Namely server 600 can be by the network interface that is connected on the system bus 605
Unit 611 is connected to network 612, in other words, it is also possible to be connected to using NIU 611 other kinds of network or
Remote computer system（It is not shown）.
The memory also include one or more than one program, one or more than one program storage in
In memory, one or more than one program bag for carry out offer of the present invention containing a kind of being processed data
The instruction of method.
One of ordinary skill in the art will appreciate that realizing that all or part of step of above-described embodiment can be by hardware
To complete, it is also possible to instruct the hardware of correlation to complete by program, described program can be stored in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only storage, disk or CD etc..
The foregoing is only presently preferred embodiments of the present invention, be not intended to limit the invention, it is all it is of the invention spirit and
Within principle, any modification, equivalent substitution and improvements made etc. should be included within the scope of the present invention.
Claims (17)
1. a kind of method that data are processed, it is characterised in that methods described includes：
The query statement that receiving terminal sends, the query statement carries storage location information and the treatment institute of data to be checked
The processing mode of data to be checked is stated, the storage location information at least includes database table where the data to be checked
Table name；
According to the storage location information of the data to be checked, it is determined that the database of the storage data to be checked, described
Database includes data warehouse or storage engines；
If it is determined that being data warehouse, then the query statement is sent to the data warehouse, makes the data warehouse root
The data to be checked are entered according to data to be checked described in the storage location acquisition of information and according to the processing mode
Row treatment；
If it is determined that be storage engines, then the query statement be sent to and store the storage of the data to be checked and draw
Hold up, make storage engines data to be checked according to the storage location acquisition of information and according to the processing mode pair
The data to be checked are processed；
If the storage engines cannot process the data to be checked, according to the storage location of the data to be checked
Information, obtains the data to be checked from the storage engines, according to the processing mode to the data to be checked
Processed.
2. the method for claim 1, it is characterised in that described to be believed according to the storage location of the data to be checked
Breath, it is determined that the database of the storage data to be checked, including：
The table name look-up table list of file names of database table according to where the data to be checked, the table name list is used to store institute
State the table name of the database table that data warehouse includes；
If the table name list includes the table name of database table where the data to be checked, it is determined that described to be checked
Data storage in the data warehouse, if not including database where the data to be checked in the table name list
The table name of table, it is determined that the data storage to be checked is in storage engines.
3. the method for claim 1, it is characterised in that described that the query statement is sent into storage is described to be checked
Data storage engines, including：
The table name of database table according to where the data to be checked, from the table name and storage engines of the database table for having stored
Mark corresponding relation in obtain the storage data to be checked storage engines mark；
According to the mark of the storage engines, the query statement is sent to and is stored the storage of the data to be checked and is drawn
Hold up.
4. the method for claim 1, it is characterised in that methods described also includes：
The table statement of building of the terminal transmission is received, it is described to build the mark that table statement at least carries storage engines, number to be created
According to the table name of storehouse table, the row name and data type of each row that the database table to be created includes；
According to the mark of the storage engines, table statement is built to the storage engines described in transmission, create the storage engines
The database table to be created.
5. the method for claim 1, it is characterised in that methods described also includes：
The table statement of building of the terminal transmission is received, it is described to build the data volume size that table statement at least carries data to be stored,
The table name of database table to be created, the row name and data type of each row that the database table to be created includes；
According to the data volume size, it is determined that the database for storing the data to be stored；
If it is determined that being data warehouse, then the database table to be created is created in the data warehouse；
If it is determined that being storage engines, then a mark for storage engines is selected, according to the mark of the storage engines, sent
The table statement of building makes the storage engines create the database table to be created to the storage engines.
6. method as claimed in claim 5, it is characterised in that described according to the data volume size, it is determined that for storing institute
The database of data to be stored is stated, including：
If the data volume size is more than predetermined threshold value, it is determined that for storing the database of the data to be stored to count
According to warehouse；
If the data volume size is less than or equal to the predetermined threshold value, it is determined that for storing the data to be stored
Database is storage engines.
7. the method as described in claim 4 or 5, it is characterised in that the mark according to the storage engines, sends described
Table statement is built to before the storage engines, is also included：
The number that the storage engines are not supported is determined in the data type of each row included from the database table to be created
According to type；
It is described build table statement in by the data type conversion determined be data type that the storage engines are supported.
8. method as claimed in claim 4, it is characterised in that what the receiving terminal sent builds after table statement, also includes：
The mark of the table name of the database table to be created for building table statement carrying and storage engines is stored in database table
Table name and storage engines mark corresponding relation in.
9. a kind of device that data are processed, it is characterised in that described device includes：
First receiver module, for the query statement that receiving terminal sends, the query statement carries depositing for data to be checked
The processing mode of storage positional information and the treatment data to be checked, the storage location information at least includes described to be checked
Data where database table table name；
First determining module, for the storage location information according to the data to be checked, it is determined that storage is described to be checked
The database of data, the database includes data warehouse or storage engines；
First sending module, for if it is determined that be data warehouse, then the query statement is sent to the data warehouse,
Make data warehouse data to be checked according to the storage location acquisition of information and according to the processing mode to institute
Data to be checked are stated to be processed；
Second sending module, for if it is determined that being storage engines, then it is described to be checked the query statement to be sent into storage
The storage engines of the data of inquiry, make storage engines data to be checked according to the storage location acquisition of information and root
The data to be checked are processed according to the processing mode；
Processing module, if the data to be checked cannot be processed for the storage engines, according to described to be checked
The storage location information of data, obtains the data to be checked, according to the processing mode to institute from the storage engines
Data to be checked are stated to be processed.
10. device as claimed in claim 9, it is characterised in that first determining module includes：
Searching unit, for the table name look-up table list of file names of the database table according to where the data to be checked, the table name
List is used to store the table name of the database table that the data warehouse includes；
First determining unit, if including the table of database table where the data to be checked for the table name list
Name, it is determined that the data storage to be checked is in the data warehouse, if not including described treating in the table name list
The table name of database table where the data of inquiry, it is determined that the data storage to be checked is in storage engines.
11. devices as claimed in claim 9, it is characterised in that second sending module includes：
Acquiring unit, for the table name of the database table according to where the data to be checked, from the database table for having stored
Table name stores the mark of the storage engines of the data to be checked with acquisition in the corresponding relation of the mark of storage engines；
Transmitting element, for the mark according to the storage engines, is sent to storage described to be checked by the query statement
The storage engines of data.
12. devices as claimed in claim 9, it is characterised in that described device also includes：
Second receiver module, table statement is built for receive that the terminal sends, and the table statement of building at least carries storage engines
Mark, the table name of database table to be created, the row name and data type of each row that the database table to be created includes；
3rd sending module, for the mark according to the storage engines, table statement is built to the storage engines described in transmission, is made
The storage engines create the database table to be created.
13. devices as claimed in claim 9, it is characterised in that described device also includes：
3rd receiver module, table statement is built for receive that the terminal sends, and the table statement of building at least carries to be stored
The data volume size of data, the table name of database table to be created, the row name of each row that the database table to be created includes
And data type；
Second determining module, for according to the data volume size, it is determined that the database for storing the data to be stored；
4th sending module, for if it is determined that be data warehouse, then create described to be created in the data warehouse
Database table；
5th sending module, for if it is determined that being storage engines, then one mark of storage engines of selection, deposits according to described
Store up the mark of engine, send it is described build table statement to the storage engines, make the storage engines establishment number to be created
According to storehouse table.
14. devices as claimed in claim 13, it is characterised in that second determining module includes：
Second determining unit, if being more than predetermined threshold value for the data volume size, it is determined that described to be stored for storing
Data database be data warehouse；
3rd determining unit, if being less than or equal to the predetermined threshold value for the data volume size, it is determined that for storing
The database of the data to be stored is storage engines.
15. device as described in claim 12 or 13, it is characterised in that described device also includes：
3rd determining module, for determining described depositing in the data type of each row included from the database table to be created
The data type that storage engine is not supported；
Modular converter, for it is described build table statement in by the data type conversion determined be the storage engines support
Data type.
16. devices as claimed in claim 12, it is characterised in that described device also includes：
Memory module, for the table name of database table to be created and the mark of storage engines for building table statement carrying to be deposited
Store up in the table name of database table with the corresponding relation of the mark of storage engines.
17. a kind of servers that data are processed, it is characterised in that the server includes：
One or more processors；With
Memory；
The memory storage has one or more programs, and one or more of programs are configured to by one or more of
Computing device, one or more of program bags are containing for carrying out such as claim 1 to 8 any one claim methods described
Instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310756095.5A CN104090901B (en) | 2013-12-31 | 2013-12-31 | A kind of method that data are processed, device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310756095.5A CN104090901B (en) | 2013-12-31 | 2013-12-31 | A kind of method that data are processed, device and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104090901A CN104090901A (en) | 2014-10-08 |
| CN104090901B true CN104090901B (en) | 2017-06-13 |
Family
ID=51638617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310756095.5A Active CN104090901B (en) | 2013-12-31 | 2013-12-31 | A kind of method that data are processed, device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104090901B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107077511A (en) * | 2015-03-28 | 2017-08-18 | 华为技术有限公司 | The apparatus and method that User Defined variable-size label is created on the record in RDBMS |
| CN106202084A (en) * | 2015-04-30 | 2016-12-07 | 阿里巴巴集团控股有限公司 | Date storage method and data storage device |
| TWI607331B (en) * | 2015-09-23 | 2017-12-01 | 財團法人工業技術研究院 | Method and device for analyzing data |
| CN105930506A (en) * | 2016-05-10 | 2016-09-07 | 乐视控股（北京）有限公司 | Object retrieval method and object storage system |
| CN106294581A (en) * | 2016-07-29 | 2017-01-04 | 苏州商信宝信息科技有限公司 | A kind of based on wages variation tendency exhibiting method during searching enterprise |
| CN108228606A (en) * | 2016-12-14 | 2018-06-29 | 北京国双科技有限公司 | The wiring method and device of data |
| CN108268517B (en) * | 2016-12-30 | 2021-11-26 | 希姆通信息技术（上海）有限公司 | Method and system for managing labels in database |
| CN107463693B (en) * | 2017-08-11 | 2020-05-01 | 深圳乐信软件技术有限公司 | Data processing method, device, terminal and computer readable storage medium |
| CN108280230A (en) * | 2018-02-27 | 2018-07-13 | 北京中关村科金技术有限公司 | A kind of method, apparatus, equipment and the storage medium of analysis data |
| CN109542953B (en) * | 2018-11-29 | 2021-01-05 | 成都四方伟业软件股份有限公司 | Presto-based data processing method and device |
| CN111143399B (en) * | 2019-12-19 | 2024-04-02 | 微民保险代理有限公司 | Data processing method, device, storage medium and computer equipment |
| CN111858644B (en) * | 2020-07-08 | 2022-11-18 | 联思智云(北京)科技有限公司 | Method, device and system for data fusion and query |
| US10963438B1 (en) | 2020-11-17 | 2021-03-30 | Coupang Corp. | Systems and methods for database query efficiency improvement |
| CN114817338A (en) * | 2022-06-28 | 2022-07-29 | 杭州湖畔网络技术有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7007029B1 (en) * | 1999-01-15 | 2006-02-28 | Metaedge Corporation | System for visualizing information in a data warehousing environment |
| CN102214236A (en) * | 2011-06-30 | 2011-10-12 | 北京新媒传信科技有限公司 | Method and system for processing mass data |
| CN102915373A (en) * | 2012-11-06 | 2013-02-06 | 无锡江南计算技术研究所 | Data storage method and device |
| CN103268336A (en) * | 2013-05-13 | 2013-08-28 | 刘峰 | Fast data and big data combined data processing method and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7792817B2 (en) * | 2005-04-19 | 2010-09-07 | International Business Machines Corporation | System and method for managing complex relationships over distributed heterogeneous data sources |
| US8356053B2 (en) * | 2005-10-20 | 2013-01-15 | Oracle International Corporation | Managing relationships between resources stored within a repository |
-
2013
- 2013-12-31 CN CN201310756095.5A patent/CN104090901B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7007029B1 (en) * | 1999-01-15 | 2006-02-28 | Metaedge Corporation | System for visualizing information in a data warehousing environment |
| CN102214236A (en) * | 2011-06-30 | 2011-10-12 | 北京新媒传信科技有限公司 | Method and system for processing mass data |
| CN102915373A (en) * | 2012-11-06 | 2013-02-06 | 无锡江南计算技术研究所 | Data storage method and device |
| CN103268336A (en) * | 2013-05-13 | 2013-08-28 | 刘峰 | Fast data and big data combined data processing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104090901A (en) | 2014-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104090901B (en) | A kind of method that data are processed, device and server | |
| CN104685498B (en) | The hardware implementation mode of polymerization/division operation：Hash table method | |
| CN106227800B (en) | Storage method and management system for highly-associated big data | |
| CN103902653B (en) | A kind of method and apparatus for building data warehouse table genetic connection figure | |
| CN102054000B (en) | Data querying method, device and system | |
| CN106708996A (en) | Method and system for full text search of relational database | |
| CN103440288A (en) | Big data storage method and device | |
| CN104111936B (en) | Data query method and system | |
| CN101566986A (en) | Method and device for processing data in online business processing | |
| US10846293B1 (en) | Factual query pattern learning | |
| CN104731859A (en) | Data processing method and device | |
| CN102902763B (en) | The method of association, retrieving information process data and process information task and device | |
| CN104268295A (en) | Data query method and device | |
| CN101202966A (en) | Method, system and apparatus for collecting user information | |
| CN104731945A (en) | Full-text searching method and device based on HBase | |
| Mahmood et al. | FAST: frequency-aware indexing for spatio-textual data streams | |
| CN102831122A (en) | Data storage method, inquiring method and inquiring device for workflow table | |
| CN102609421A (en) | Data query method and device | |
| CN104133891B (en) | A kind of storage method of the massive structured data based on relevant database | |
| CN103823846A (en) | Method for storing and querying big data on basis of graph theories | |
| CN103902705B (en) | Metadata-based cross-mechanism cloud digital content integration system and metadata-based cross-mechanism cloud digital content integration method | |
| CN104376015A (en) | Method and device for processing nodes in relational network | |
| CN109783484A (en) | The construction method and system of the data service platform of knowledge based map | |
| CN107636655A (en) | Data are provided in real time to service（DaaS）System and method | |
| CN103559307A (en) | Caching method and device for query |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |