CN106528898A - Method and device for converting data of non-relational database into relational database - Google Patents
Method and device for converting data of non-relational database into relational database Download PDFInfo
- Publication number
- CN106528898A CN106528898A CN201710005450.3A CN201710005450A CN106528898A CN 106528898 A CN106528898 A CN 106528898A CN 201710005450 A CN201710005450 A CN 201710005450A CN 106528898 A CN106528898 A CN 106528898A
- Authority
- CN
- China
- Prior art keywords
- data
- relational database
- database
- information
- configuration information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and device for converting data of a non-relational database into a relational database. The method comprises the following steps of: obtaining data from the non-relational database; screening the data through configuration information, so that first data is obtained; performing data conversion of the first data through a pre-determined rule, so that second data is obtained; and establishing database connection, and storing the second data in the relational database. By means of the method for converting the data of the non-relational database into the relational database disclosed by the invention, the data of the non-relational database can be converted into the relational database automatically and effectively; and the manpower and the time cost are reduced.
Description
Technical field
The present invention relates to data of non relational database is turned by database information process field in particular to one kind
Change to the method and device of relevant database.
Background technology
Relational database, the data base being built upon on the basis of relational model, by means of mathematical concepts such as algebras of sets and
The data that method is come in processing data storehouse.Relation mould is used in the various contacts between various entities and entity in real world
Type is representing.Normal data query language SQL is exactly a kind of language based on relational database, and this language is performed to relation number
According to the retrieval and operation of data in storehouse.Relational model constrains three by relational data structure, relational operation set, relation integraity
It is grouped into.With the rise of the Internet web2.0 websites, traditional relational database is dealing with web2.0 websites, particularly super large
The pure dynamic websites of web2.0 of the SNS types of scale and high concurrent have seemed unable to do what one wishes, expose
Problem, and the data base of non-relational is then due to having obtained very fast development the characteristics of itself.The product of NoSQL data bases
It is an especially big data application difficult problem in order to solve the challenge that large-scale data set multiple data species is brought to be born with.
Non-relational database feature is to need not move through SQL layer parsing, and readwrite performance is high；Stored based on key-value pair, extension
Property is strong.The use of non-relational database is first-selected for those have the system of data access pressure.But it is after data landing, non-
There is substantially deficiency in the analytic statisticss of relevant database, this is analyzed statistics in being accomplished by being transformed into relevant database.
Currently without ripe product support from non-relational database to relevant database data conversion.If specifically needing
Ask, need to write three subprograms to complete task：1, coding extracts data of non relational database.2, write journey
Non-relational database and relevant database are done field mapping by sequence.3, the data for having mapped are write relationship type by coding
Data base.Once demand is changed, modification program is needed.When there are new data to need conversion, need to repeat three above step,
So not only need to put into substantial amounts of manpower and materials, also there are many repeated works.
Accordingly, it would be desirable to a kind of new method and device that data of non relational database is transformed into relevant database.
Disclosed in the background section, above- mentioned information is only used for strengthening the understanding of the background to the present invention, therefore it
Can include not constituting the information to prior art known to persons of ordinary skill in the art.
The content of the invention
In view of this, the present invention provide it is a kind of by data of non relational database be transformed into relevant database method and
Device, can be transformed into automatically, efficiently the data of non-relational database in relevant database, save manpower and time
Cost.
Other characteristics and advantage of the present invention will be apparent from by detailed description below, or partially by the present invention
Practice and acquistion.
According to an aspect of the invention, it is proposed that a kind of for the data of non-relational database are transformed into relational data
The method in storehouse, the method include：
Data are obtained by non-relational database；
Data are screened by configuration information, to obtain the first data；
Data conversion is carried out to the first data by pre-defined rule, to obtain the second data；And
By the second data storage in relevant database.
In a kind of exemplary embodiment of the disclosure, also include：
Configuration information is loaded in internal memory.
In a kind of exemplary embodiment of the disclosure, also include：
Configuration information is loaded in internal memory by javaIO technologies.
In a kind of exemplary embodiment of the disclosure, configuration information includes：
The connection pool information of non-relational database, the connection pool information of relevant database, field map information and
Configuration screening conditions.
In a kind of exemplary embodiment of the disclosure, note of the field map information in internal memory for key-value forms
Volume table.
In a kind of exemplary embodiment of the disclosure, data are obtained by non-relational database, including：
By obtaining link in the connection pool of non-relational database；And
By connecting the data for obtaining non-relational database.
In a kind of exemplary embodiment of the disclosure, according to configuration information batch extracted data.
In a kind of exemplary embodiment of the disclosure, data conversion is carried out to the first data by pre-defined rule, to obtain
The second data are obtained, including：
Obtain the field information in registration table；
According to key values, desired value to be changed is obtained；
Data conversion is carried out to the first data according to desired value, to obtain the second data.
In a kind of exemplary embodiment of the disclosure, by the second data storage in relevant database, including：
The connection of opening relationships type data base；
The second Data Data is stored in relevant database by connection.
In a kind of exemplary embodiment of the disclosure, by connection by the batch storage of the second Data Data to relationship type number
According in storehouse.
In a kind of exemplary embodiment of the disclosure, the connection pool information of non-relational database, including：
Mongodb connection pool information.
In a kind of exemplary embodiment of the disclosure, the connection pool information of relevant database, including：
Mysql connection pool information.
In a kind of exemplary embodiment of the disclosure, according to paging batch extracted data.
According to an aspect of the invention, it is proposed that a kind of for the data of non-relational database are transformed into relational data
The device in storehouse, the device include：
Data acquisition module, for obtaining data by non-relational database；
Data screening module, for being screened to data by configuration information, to obtain the first data；
Data conversion module, for carrying out data conversion by pre-defined rule to the first data, to obtain the second data；With
And
Data storage module, for setting up data base's connection, by the second data storage in relevant database.
In a kind of exemplary embodiment of the disclosure, also include：
Internal memory load-on module, for configuration information is loaded in internal memory.
The method and device that data of non relational database is transformed into relevant database of the invention, can be by
The data of non-relational database are transformed in relevant database automatically, efficiently, save manpower and time cost.
It should be appreciated that the general description of the above and detailed description hereinafter are only exemplary, this can not be limited
Invention.
Description of the drawings
Its example embodiment is described in detail by referring to accompanying drawing, above and other target of the present invention, feature and advantage will
Become more fully apparent.Drawings discussed below is only some embodiments of the present invention, for the ordinary skill of this area
For personnel, on the premise of not paying creative work, can be with according to these other accompanying drawings of accompanying drawings acquisition.
Fig. 1 is that data of non relational database is transformed into relational data by the one kind according to an exemplary embodiment
The flow chart of the method in storehouse.
Fig. 2 is that data of non relational database is transformed into relationship type number by the one kind for implementing to exemplify according to another exemplary
According to the flow chart of the method in storehouse.
Fig. 3 is that data of non relational database is transformed into relationship type number by the one kind for implementing to exemplify according to another exemplary
According to the flow chart of the method in storehouse.
Fig. 4 is that data of non relational database is transformed into relational data by the one kind according to an exemplary embodiment
The block diagram of the device in storehouse.
Specific embodiment
Example embodiment is described more fully with referring now to accompanying drawing.However, example embodiment can be real in a variety of forms
Apply, and be not understood as limited to embodiment set forth herein；Conversely, thesing embodiments are provided so that the present invention will be comprehensively and complete
It is whole, and the design of example embodiment is comprehensively conveyed to into those skilled in the art.In figure, identical reference is represented
Same or similar part, thus repetition thereof will be omitted.
Additionally, described feature, structure or characteristic can in any suitable manner with reference in one or more enforcements
In example.In the following description, there is provided many details are fully understood to embodiments of the invention so as to be given.However,
It will be appreciated by persons skilled in the art that it is one or more during technical scheme can be put into practice without specific detail,
Or can be using other methods, constituent element, device, step etc..In other cases, it is not shown in detail or describes known side
Method, device, realization operate to avoid fuzzy each aspect of the present invention.
Block diagram shown in accompanying drawing is only functional entity, not necessarily must be corresponding with physically separate entity.
I.e., it is possible to realize these functional entitys using software form, or realize in one or more hardware modules or integrated circuit
These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.
Flow chart shown in accompanying drawing is merely illustrative, it is not necessary to including all of content and operation/step,
It is not required to perform by described order.For example, some operation/steps can also be decomposed, and some operation/steps can be closed
And or part merge, therefore the actual order for performing is possible to be changed according to practical situation.
It should be understood that although various assemblies may be described using term first, second, third, etc. herein, these groups
Part should not be limited by these terms.These terms are to distinguish a component with another component.Therefore, first group for being discussed herein below
Part can be described as teaching of second component without departing from disclosure concept.As used herein, term " and/or " include it is associated
The all combinations for listing any one and one or more in project.
It will be understood by those skilled in the art that accompanying drawing is the schematic diagram of example embodiment, the module or flow process in accompanying drawing
Not necessarily implement the present invention necessary, therefore cannot be used for limiting the scope of the invention.
Fig. 1 is that data of non relational database is transformed into relational data by the one kind according to an exemplary embodiment
The flow chart of the method in storehouse.
As shown in figure 1, in S102, obtaining data by non-relational database.
As described above, the purpose of the embodiment of the present invention is, by the data in non-relational database, to be transformed into relationship type
In data base, relevant database refers to and employs relational model to organize the data base of data.In simple terms, relational model
What is referred to is exactly two-dimensional table model, and relevant database be exactly by bivariate table and its between constituted one of contact
Data are organized.The concept commonly used in relational model：Relation：A bivariate table is can be understood as, each relation is closed with one
It is name, is exactly the table name generally said.Tuple：The a line in bivariate table is can be understood as, Jing is commonly referred to as record in data base.
Attribute：The string in bivariate table is can be understood as, Jing is commonly referred to as field in data base.Domain：The span of attribute, also
It is the value restriction of certain string in data base.Keyword：One group can frequently referred to be led in data base with the attribute of unique mark tuple
Key, is made up of one or more row.
Non-relational database is used for referring to those non-relationals, distributed, and does not typically ensure to follow ACID former
Data-storage system then.Non-relational database proposes another kind of theory, for example, is stored with key-value pair, and structure is not fixed,
Each tuple can have different field, and each tuple can increase some key-value pairs of oneself as needed, thus
It is not intended to be limited to fixed structure, it is possible to reduce the expense in some times and space.Make in this way, user can be according to need
Go to add the field for oneself needing, so, in order to obtain the different information of user, it is not necessary in as relevant database,
Inquiry is associated to multilist.Only need just complete inquiry according to the corresponding value of id taking-ups.But non-relational database
Due to little constraint, it is this for the inquiry of field attribute value situation that he can not provide the where provided as SQL.
And it is difficult to embody the integrity of design.He is only suitable for storing some relatively simple data, for needs carry out more complicated looking into
The data of inquiry, it is more particularly suitable that SQL database shows.According to the principle of relevant database and non-relational database, simply
Data transfer, it is impossible to realize the data storage between above two data base.Carrying out the data processing between two data bases
When, the data of non-relational database are obtained first.
In S104, data are screened by configuration information, to obtain the first data.In the present embodiment, configure
Information may be, for example, user configured screening conditions, by screening conditions, the data in non-relational database be screened,
And then obtain the first data.In embodiments of the present invention, for the data screening of non-relational data may be, for example, batch screening.
The screening operation of mass data is processed simultaneously.
In S106, data conversion is carried out to the first data by pre-defined rule, to obtain the second data.By screening
To after the first data, data conversion is carried out to the first data, for example can obtain field map information, according to field information
First data with desired value to be converted, are carried out data conversion, obtain the second data by key values.
In S108, by the second data storage in relevant database.Second data storage is in relevant database
When may be, for example, that bulk storage is alternatively stored one by one, the present invention is not limited.
The method that data of non relational database is transformed into relevant database of the invention, by configuration information
Data in non-relational database are screened, the data after screening are carried out by data conversion by predetermined rule then
Mode, the data of non-relational database can automatically, be efficiently transformed in relevant database, save manpower and when
Between cost.
It will be clearly understood that the present disclosure describe how being formed and using particular example, but the principle of the present invention to be not limited to
Any details of these examples.Conversely, the teaching based on present disclosure, these principles can be applied to many other
Embodiment.
In a kind of exemplary embodiment of the disclosure, also include：Configuration information is loaded into by javaIO technologies interior
In depositing.JavaIO is the core library of Java, there is provided comprehensive I/O interface.Including：File read-write, standard device output etc..
In Java, IO is to carry out input and output based on stream, and all data are serialized write output stream, or read from inlet flow
Enter.
In a kind of exemplary embodiment of the disclosure, configuration information includes：The connection pool information of non-relational database,
The connection pool information of relevant database, field map information and configuration screening conditions.Database connection pool is responsible for distributing, is managed
Reason and release data base's connection, it allows one existing data base of application program reuse to connect, rather than builds again again
It is vertical one；The releasing idling time exceed maximum idle time data base connection come avoid because do not discharge data base connection and
The data base's connection for causing is omitted.
In a kind of exemplary embodiment of the disclosure, note of the field map information in internal memory for key-value forms
Volume table.When so-called key-value is exactly every time one data of storage, it is that storage is indexed according to Key.key-value
Distributed memory system inquiry velocity is fast, storage data volume is big, support high concurrent, is especially suitable for being inquired about by major key, but
The condition query of complexity can not be carried out.
In a kind of exemplary embodiment of the disclosure, data are obtained by non-relational database, including：By non-relational
Link is obtained in the connection pool of data base；And by connecting the data for obtaining non-relational database.For a simple number
Quote according to storehouse, for not being very frequent to the access of data base.At this moment just simply when needing to access data base just can create
A connection is built, after being finished, it is shut off, do so will not also bring the expense in what obvious performance.But for one
Complicated data base quotes, and frequently sets up, closes connection, can greatly lower the performance of system, because making for connection
With the bottleneck into systematic function.For shared resource, for example a database connection pool can be set up, there is provided connection distributes, makes
With strategy.The ultimate principle of database connection pool is internally to safeguard a number of data base's connection in object pool, and externally
Exposure data base's connection is obtained and return method.Such as：External users can obtain connection by getConnection methods, make
Returned with will be connected by releaseConnection methods after finishing again.
In a kind of exemplary embodiment of the disclosure, according to configuration information batch extracted data.In batch extracted data
When can for example, in order to ensure stablizing for data base, setting data amount extracts threshold value, the data volume for extracting every time the numerical value with
Under, can also for example, by the data pick-up work time-division processing of data base, it is to avoid data base's pressure, the present invention are not limited.
Fig. 2 is that data of non relational database is transformed into relationship type number by the one kind for implementing to exemplify according to another exemplary
According to the flow chart of the method in storehouse.In Fig. 2, content is the exemplary description to S106 in Fig. 1.
As shown in figure 1, in S202, obtaining the field information in registration table.As described above, including field map information
It is the registration table of key-value forms in depositing.The field information in the registration table of key-value forms for example, can be obtained.
In S204, according to key values, desired value to be changed is obtained.For example according to key values, can obtain to be changed
Desired value value.In the present embodiment, can for example, by corresponding relation file set in advance, according to the key on file with
Value corresponding relations obtain target value value, also can for example, and according in prior art, the function in routine processes is logical in real time
Cross key values and obtain value values.The present invention is not limited.
In S206, data conversion is carried out to the first data according to desired value, to obtain the second data.
The method that data of non relational database is transformed into relevant database of the invention, by key-
Data in non-relational database are carried out Mapping and Converting with the data in relevant database by the relation between value
Mode, when can carry out data conversion between non-relational data and relevant database, can be real by simple configuration
It is existing, considerably increase work efficiency and also reduce job costs.
In a kind of exemplary embodiment of the disclosure, by the second data storage in relevant database, including：Set up
The connection of relevant database；The second Data Data is stored in relevant database by connection.As described above, connect
Pond is that the data base complicated for quotes, and when frequently setting up, closing connection, does not lower the mode of the performance of system.
In a kind of exemplary embodiment of the disclosure, by connection by the batch storage of the second Data Data to relationship type number
According in storehouse.
In a kind of exemplary embodiment of the disclosure, the connection pool information of non-relational database, including：mongodb
Connection pool information.Mongodb is a product between relational database and non-relational database, is non-relational database
Central function is most abundant, is most like relational database.The data structure that he supports is very loose, is the bson lattice of similar json
Formula, therefore more complicated data type can be stored.The characteristics of Mongo is maximum is that the query language that he supports is very powerful,
Its grammer is somewhat similarly to OO query language, can almost realize the exhausted big portion of similarity relation data base list table inquiry
Divide function, but also support index to be set up to data
In a kind of exemplary embodiment of the disclosure, the connection pool information of relevant database, including：Mysql connects
Pond information.Mysql is a Relational DBMS, and mysql is a kind of associated data base management system, associated data
Storehouse is saved the data in different tables, rather than all data are placed in a big warehouse, which adds speed simultaneously
Improve motility.The sql like language used by mysql is the most frequently used standardized language for accessing data base.
In a kind of exemplary embodiment of the disclosure, according to paging batch extracted data.In example in real time of the invention, point
The data included in data base for example can be carried out paging according to predetermined threshold, in current page per treatment by page extracted data
Data.
The method that data of non relational database is transformed into relevant database of the invention, is extracted by paging
The mode for carrying out data conversion again of data, can mitigate the pressure of data base's real-time processing data.
Fig. 3 is that data of non relational database is transformed into relationship type number by the one kind for implementing to exemplify according to another exemplary
According to the flow chart of the method in storehouse.
As described in Figure 3, user configuration information is loaded on startup.Whether inquiry has screening conditions, and the screening conditions can example
Such as it is, from user configured screening conditions, if screening conditions, then to carry out data screening using the condition, the screening conditions
For example can be obtained by configuration information.Can also for example, if no screening conditions, the implied termss logarithm that can be carried using system
According to being screened.When the data of non-relational database mongodb storage also for example, can be obtained, chain is obtained in connection pool first
Connect, extract the data for meeting condition according to user configured screening conditions batch.Data are filtered out, when carrying out data conversion, can
For example, field information in registration table is read, according to key values, obtains desired value value to be changed, data are changed.Number
Connect according to the data base for after conversion, setting up mysql, by the batch data for having mapped storage to relevant database mysql
In in the present embodiment, can for example, as described above, in order to avoid the unstable situation of data base, can be by the book in data base
Paging conversion is carried out, during paging change data, the data that batch is extracted every time can be the data of current page, process in process
After end, continue the data whether inquiry there are other pending, for example user can be detected whether by the mobile acquisition of vernier
The data of next page, and then continue executing with data transformation operations.
The method that data of non relational database is transformed into relevant database of the invention, by non-relational number
Seamless fusion is carried out according to extraction, relationship map, relevant database storage.Whole process to user be it is transparent, it is friendly.
User need to only configure the conversion row of needs, and click on converse routine.Data automatically, efficiently can be changed by instrument.
It will be appreciated by those skilled in the art that realizing that all or part of step of above-described embodiment is implemented as being performed by CPU
Computer program.When the computer program is performed by CPU, the above-mentioned work(limited by the said method of present invention offer is performed
Energy.Described program can be stored in a kind of computer-readable recording medium, and the storage medium can be read only memory, magnetic
Disk or CD etc..
Further, it should be noted that above-mentioned accompanying drawing is only the place included by method according to an exemplary embodiment of the present invention
That what is managed schematically illustrates, rather than limits purpose.It can be readily appreciated that above-mentioned shown in the drawings process is not intended that or limits at these
The time sequencing of reason.In addition, being also easy to understand, these process for example either synchronously or asynchronously can be performed in multiple modules.
It is following for apparatus of the present invention embodiment, can be used for performing the inventive method embodiment.For apparatus of the present invention reality
The details not disclosed in applying example, refer to the inventive method embodiment.
Fig. 4 is that data of non relational database is transformed into relational data by the one kind according to an exemplary embodiment
The block diagram of the device in storehouse.
Data acquisition module 402 is for obtaining data by non-relational database.
Data screening module 404 is used for screening data by configuration information, to obtain the first data.
Data conversion module 406 is used for carrying out data conversion by pre-defined rule to the first data, to obtain the second data.
And
Data storage module 408 is used for setting up data base's connection, by the second data storage in relevant database.
The device that data of non relational database is transformed into relevant database of the invention, by configuration information
Data in non-relational database are screened, the data after screening are carried out by data conversion by predetermined rule then
Mode, the data of non-relational database can automatically, be efficiently transformed in relevant database, save manpower and when
Between cost.
In a kind of exemplary embodiment of the disclosure, also include：Internal memory load-on module is for configuration information is loaded into
In internal memory.
It will be appreciated by those skilled in the art that above-mentioned each module can be distributed in device according to the description of embodiment, also may be used
To carry out in one or more devices of respective change uniquely different from the present embodiment.The module of above-described embodiment can be merged into
One module, it is also possible to be further split into multiple submodule.
The description of the embodiment by more than, those skilled in the art is it can be readily appreciated that example embodiment described herein
Can be realized by software, it is also possible to realize by way of software is with reference to necessary hardware.Therefore, according to present invention enforcement
The technical scheme of example can be embodied in the form of software product, and the software product can be stored in non-volatile memories
In medium (can be CD-ROM, USB flash disk, portable hard drive etc.) or on network, including some instructions so that a computing device (can
Being personal computer, server, mobile terminal or network equipment etc.) perform method according to embodiments of the present invention.
Detailed description by more than, those skilled in the art is it can be readily appreciated that according to embodiments of the present invention by non-pass
It is that type database data is transformed into the method and device of relevant database and has one or more of the following advantages.
According to some embodiments, the method that data of non relational database is transformed into relevant database of the present invention,
The data in non-relational database are screened by configuration information, then by predetermined rule by the data after screening
The mode of data conversion is carried out, the data of non-relational database can be transformed in relevant database automatically, efficiently,
Save manpower and time cost.
According to other embodiments, the side that data of non relational database is transformed into relevant database of the present invention
Method, by the relation between key-value, the data in non-relational database is carried out with the data in relevant database
The mode of Mapping and Converting, when can carry out data conversion between non-relational data and relevant database, by simple
Configuration is capable of achieving, and considerably increases work efficiency and also reduces job costs.
According to other embodiments, the side that data of non relational database is transformed into relevant database of the present invention
Non-relational data pick-up, relationship map, relevant database storage are carried out seamless fusion by method.Whole process is to user
Be it is transparent, it is friendly.User need to only configure the conversion row of needs, and click on converse routine.Instrument automatically, efficiently can be incited somebody to action
Data are changed.
The exemplary embodiment of the present invention is particularly shown and described more than.It should be appreciated that the invention is not restricted to
Detailed construction described herein, set-up mode or implementation method；On the contrary, it is intended to cover be included in claims
Various modifications and equivalence setting in spirit and scope.
Additionally, structure, ratio, size shown by this specification Figure of description etc., only to coordinate description institute
Disclosure, for skilled in the art realises that with reading, be not limited to the enforceable qualificationss of the disclosure, therefore
Do not have technical essential meaning, the modification of any structure, the change of proportionate relationship or the adjustment of size are not affecting the disclosure
Under the technique effect that can be generated and achieved purpose, still all should fall obtain and can cover in the technology contents disclosed in the disclosure
In the range of.Meanwhile, in this specification it is cited such as " on ", " first ", the term of " second " and " " etc., be also only and be easy to
Narration understands, and be not used to limit the disclosure enforceable scope, and its relativeness is altered or modified, without substantive change
Under technology contents, when being also considered as enforceable category of the invention.
Claims (15)
1. a kind of method for the data of non-relational database are transformed into relevant database, it is characterised in that include：
Data are obtained by non-relational database；
The data are screened by configuration information, to obtain the first data；
Data conversion is carried out to first data by pre-defined rule, to obtain the second data；And
By second data storage in relevant database.
2. the method for claim 1, it is characterised in that also include：
The configuration information is loaded in internal memory.
3. method as claimed in claim 2, it is characterised in that also include：
The configuration information is loaded in internal memory by javaIO technologies.
4. method as claimed in claim 2, it is characterised in that the configuration information includes：
The connection pool information of the non-relational database, the connection pool information of the relevant database, field map information
And configuration screening conditions.
5. method as claimed in claim 4, it is characterised in that the field map information is key- in the internal memory
The registration table of value forms.
6. method as claimed in claim 4, it is characterised in that described that data are obtained by non-relational database, including：
By obtaining link in the connection pool of the non-relational database；And
The data of the non-relational database are obtained by the connection.
7. method as claimed in claim 4, described the data to be screened by configuration information, to obtain the first number
According to, it is characterised in that according to the configuration information batch extracted data.
8. method as claimed in claim 5, it is characterised in that described that data are carried out to first data by pre-defined rule
Conversion, to obtain the second data, including：
Obtain the field information in the registration table；
According to key values, desired value to be changed is obtained；
Data conversion is carried out to first data according to the desired value, to obtain second data.
9. method as claimed in claim 5, it is characterised in that it is described by second data storage to relevant database
In, including：
Set up the connection of the relevant database；
Second Data Data is stored in relevant database by the connection.
10. method as claimed in claim 9, it is characterised in that second Data Data batch is deposited by the connection
Store up in relevant database.
11. methods as claimed in claim 4, it is characterised in that the connection pool information of the non-relational database, including：
Mongodb connection pool information.
12. methods as claimed in claim 4, it is characterised in that the connection pool information of the relevant database, including：
Mysql connection pool information.
13. methods as claimed in claim 7, it is described the data to be screened by configuration information, to obtain the first number
According to, it is characterised in that according to paging batch extracted data.
A kind of 14. devices for the data of non-relational database to be transformed into relevant database, it is characterised in that bag
Include：
Data acquisition module, for obtaining data by non-relational database；
Data screening module, for being screened to the data by configuration information, to obtain the first data；
Data conversion module, for carrying out data conversion by pre-defined rule to first data, to obtain the second data；With
And
Data storage module, for by second data storage in relevant database.
15. devices as claimed in claim 14, it is characterised in that also include：
Internal memory load-on module, for the configuration information is loaded in internal memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710005450.3A CN106528898A (en) | 2017-01-04 | 2017-01-04 | Method and device for converting data of non-relational database into relational database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710005450.3A CN106528898A (en) | 2017-01-04 | 2017-01-04 | Method and device for converting data of non-relational database into relational database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106528898A true CN106528898A (en) | 2017-03-22 |
Family
ID=58336915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710005450.3A Pending CN106528898A (en) | 2017-01-04 | 2017-01-04 | Method and device for converting data of non-relational database into relational database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106528898A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108776678A (en) * | 2018-05-29 | 2018-11-09 | 阿里巴巴集团控股有限公司 | Index creation method and device based on mobile terminal NoSQL databases |
| CN109213820A (en) * | 2018-08-30 | 2019-01-15 | 成都索贝数码科技股份有限公司 | Method for realizing fusion use of multiple types of databases |
| CN109871383A (en) * | 2019-02-15 | 2019-06-11 | 苏州麦迪斯顿医疗科技股份有限公司 | A kind of data processing method, device, storage medium and electronic equipment |
| CN110032571A (en) * | 2019-04-18 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Business flow processing method, apparatus, storage medium and calculating equipment |
| CN110727726A (en) * | 2019-09-30 | 2020-01-24 | 武汉达梦数据库有限公司 | Method and system for extracting data from document type database to relational database |
| CN110750563A (en) * | 2018-07-20 | 2020-02-04 | 北京京东尚科信息技术有限公司 | Multi-model data processing method, system, device, electronic equipment and storage medium |
| CN111506642A (en) * | 2020-06-12 | 2020-08-07 | 夸克云智科技(深圳)有限公司 | Data conversion method based on basic data type |
| CN112487075A (en) * | 2020-12-29 | 2021-03-12 | 中科院计算技术研究所大数据研究院 | Operator for integrating data conversion of relational database and non-relational database |
| WO2024001029A1 (en) * | 2022-06-29 | 2024-01-04 | 蚂蚁区块链科技(上海)有限公司 | Method and apparatus for maintaining blockchain data, electronic device, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530327A (en) * | 2013-09-25 | 2014-01-22 | 清华大学深圳研究生院 | Method for migrating data from non-relational database to relational database |
| CN103810275A (en) * | 2014-02-13 | 2014-05-21 | 清华大学 | Method and device for data interaction between non-relation type database and relation type database |
| CN105335412A (en) * | 2014-07-31 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Method and device for data conversion and data migration |
| CN105930354A (en) * | 2016-04-08 | 2016-09-07 | 四川师范大学 | Storage model conversion method and device |
-
2017
- 2017-01-04 CN CN201710005450.3A patent/CN106528898A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530327A (en) * | 2013-09-25 | 2014-01-22 | 清华大学深圳研究生院 | Method for migrating data from non-relational database to relational database |
| CN103810275A (en) * | 2014-02-13 | 2014-05-21 | 清华大学 | Method and device for data interaction between non-relation type database and relation type database |
| CN105335412A (en) * | 2014-07-31 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Method and device for data conversion and data migration |
| CN105930354A (en) * | 2016-04-08 | 2016-09-07 | 四川师范大学 | Storage model conversion method and device |
Non-Patent Citations (1)
| Title |
|---|
| 张丽: "《java web 应用详解》", 31 January 2015, 北京邮电大学出版社 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108776678A (en) * | 2018-05-29 | 2018-11-09 | 阿里巴巴集团控股有限公司 | Index creation method and device based on mobile terminal NoSQL databases |
| CN110750563A (en) * | 2018-07-20 | 2020-02-04 | 北京京东尚科信息技术有限公司 | Multi-model data processing method, system, device, electronic equipment and storage medium |
| CN109213820A (en) * | 2018-08-30 | 2019-01-15 | 成都索贝数码科技股份有限公司 | Method for realizing fusion use of multiple types of databases |
| CN109213820B (en) * | 2018-08-30 | 2021-10-22 | 成都索贝数码科技股份有限公司 | Method for realizing fusion use of multiple types of databases |
| CN109871383A (en) * | 2019-02-15 | 2019-06-11 | 苏州麦迪斯顿医疗科技股份有限公司 | A kind of data processing method, device, storage medium and electronic equipment |
| CN110032571A (en) * | 2019-04-18 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Business flow processing method, apparatus, storage medium and calculating equipment |
| CN110032571B (en) * | 2019-04-18 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Business process processing method and device, storage medium and computing equipment |
| CN110727726A (en) * | 2019-09-30 | 2020-01-24 | 武汉达梦数据库有限公司 | Method and system for extracting data from document type database to relational database |
| CN111506642A (en) * | 2020-06-12 | 2020-08-07 | 夸克云智科技(深圳)有限公司 | Data conversion method based on basic data type |
| CN112487075A (en) * | 2020-12-29 | 2021-03-12 | 中科院计算技术研究所大数据研究院 | Operator for integrating data conversion of relational database and non-relational database |
| CN112487075B (en) * | 2020-12-29 | 2021-08-31 | 中科院计算技术研究所大数据研究院 | Method for integrating relational database data conversion operators and non-relational database data conversion operators |
| WO2024001029A1 (en) * | 2022-06-29 | 2024-01-04 | 蚂蚁区块链科技(上海)有限公司 | Method and apparatus for maintaining blockchain data, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106528898A (en) | Method and device for converting data of non-relational database into relational database | |
| Khasawneh et al. | Sql, newsql, and nosql databases: A comparative survey | |
| JP5298117B2 (en) | Data merging in distributed computing | |
| US9256665B2 (en) | Creation of inverted index system, and data processing method and apparatus | |
| US11941034B2 (en) | Conversational database analysis | |
| CN104572895B (en) | MPP databases and Hadoop company-datas interoperability methods, instrument and implementation method | |
| CN104504001B (en) | Towards the vernier building method of magnanimity distributed relational database | |
| CN107391502B (en) | Time interval data query method and device and index construction method and device | |
| Li et al. | An integration approach of hybrid databases based on SQL in cloud computing environment | |
| CN107491487A (en) | A kind of full-text database framework and bitmap index establishment, data query method, server and medium | |
| Sivaraman et al. | High performance and fault tolerant distributed file system for big data storage and processing using hadoop | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| US11507555B2 (en) | Multi-layered key-value storage | |
| WO2021179722A1 (en) | Sql statement parsing method and system, and computer device and storage medium | |
| Wang et al. | Distributed storage and index of vector spatial data based on HBase | |
| DE112011101200T5 (en) | Column-oriented memory representations of data records | |
| CN107301214A (en) | Data migration method, device and terminal device in HIVE | |
| CN112214453B (en) | Large-scale industrial data compression storage method, system and medium | |
| Khan et al. | Predictive performance comparison analysis of relational & NoSQL graph databases | |
| US11809468B2 (en) | Phrase indexing | |
| Feuerlicht | Database Trends and Directions: Current Challenges and Opportunities. | |
| CN109829003A (en) | Database backup method and device | |
| US20210109974A1 (en) | Query Execution On Compressed In-Memory Data | |
| US10877998B2 (en) | Highly atomized segmented and interrogatable data systems (HASIDS) | |
| McClean et al. | A comparison of mapreduce and parallel database management systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170322 |