CN109815047A - A kind of method and relevant apparatus of data processing - Google Patents
A kind of method and relevant apparatus of data processing Download PDFInfo
- Publication number
- CN109815047A CN109815047A CN201910007413.5A CN201910007413A CN109815047A CN 109815047 A CN109815047 A CN 109815047A CN 201910007413 A CN201910007413 A CN 201910007413A CN 109815047 A CN109815047 A CN 109815047A
- Authority
- CN
- China
- Prior art keywords
- copy data
- data
- copy
- memory node
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application involves virtualization data migrations, disclose the method and relevant apparatus of a kind of data processing, it include: to obtain N number of copy data from N number of memory node, wherein, n-th of copy data in N number of copy data is obtained from n-th of memory node in N number of memory node, N is positive integer, and 0 < n≤N and n are integer；N number of copy data is verified；When verification passes through, N number of copy data is migrated to default node.Implement the embodiment of the present invention, is conducive to data inconsistence problems caused by avoiding the silence mistake of primary copy.
Description
Technical field
The present invention relates to field of computer technology more particularly to the methods and relevant apparatus of a kind of data processing.
Background technique
With the development and progress of informationization technology, more and more informationization industries also develop therewith.Thus, at this
Large-scale data can be generated in development process, usually, enterprise can store that data in distributed memory system, with
Convenient for solving the needs that traditional network store system is not able to satisfy Mass storage application.It is distributed to optimize memory space
When storage system is from memory node request data, data are generally read from primary copy, then will be rarely employed or unused data
Move to secondary storage system.
Currently, typically carrying out data check after Data Migration in Data Migration, not can solve in data
In transition process, in fact it could happen that data inconsistence problems caused by the silence mistake of primary copy.
Summary of the invention
The embodiment of the invention discloses a kind of method of data processing and relevant apparatus, be conducive to the silence for avoiding primary copy
Data inconsistence problems caused by mistake.
First aspect of the embodiment of the present invention provides a kind of method of data processing, comprising:
N number of copy data is obtained from N number of memory node, wherein n-th of memory node from N number of memory node
N-th of copy data in N number of copy data is obtained, N is positive integer, and 0 < n≤N and n are integer；
N number of copy data is verified；
When verification passes through, N number of copy data is migrated to default node.
Second aspect of the embodiment of the present invention provides a kind of data processing equipment, comprising:
Module is obtained, for obtaining N number of copy data from N number of memory node, wherein from N number of memory node
N-th of memory node obtains n-th of copy data in N number of copy data, and N is positive integer, and 0 < n≤N and n are integer；
Correction verification module, for being verified to N number of copy data；
Transferring module, for when verification passes through, N number of copy data to be migrated to default node.
The third aspect of the embodiment of the present invention provides a kind of electronic equipment of data processing, including processor, memory, logical
Believe interface and one or more programs, wherein one or more of programs are stored in the memory, and are matched
It sets and is executed by the processor, to execute the instruction included the steps that in a kind of method of data processing of any one.
Fourth aspect of the embodiment of the present invention provides a kind of computer storage medium, and the computer readable storage medium is used
In storage computer program, the storage computer program is executed by the processor, to realize a kind of including data of any one
The method of processing.
As can be seen that distributed memory system obtains N number of copy data from N number of memory node in above-mentioned technical proposal, then
N number of copy data is verified, when verification passes through, when verification passes through, N number of copy data is migrated to default section
Point.By first verifying, then the mode of copy data is migrated, avoids data inconsistence problems caused by the silence mistake of primary copy.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Wherein:
Fig. 1-a is a kind of flow diagram of the method for data processing provided in an embodiment of the present invention；
Fig. 1-b is that a kind of distributed memory system provided in an embodiment of the present invention from N number of memory node obtains N number of number of copies
According to schematic diagram；
Fig. 2-a is the flow diagram of the method for another data processing provided in an embodiment of the present invention；
Fig. 2-b is that a kind of distributed memory system provided in an embodiment of the present invention is stored from n-th in N number of memory node
Node obtains the schematic diagram of n-th of copy data in N number of copy data；
Fig. 3 is the flow diagram of the method for another data processing provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of data processing equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Term " includes " in description and claims of this specification and above-mentioned attached drawing and " having " and they appoint
What is deformed, it is intended that is covered and non-exclusive is included.Such as contain the process, method, system, production of a series of steps or units
Product or equipment are not limited to listed step or unit, but optionally further comprising the step of not listing or unit, or
Optionally further comprising the other step or units intrinsic for these process, methods, product or equipment.
One embodiment of the method for data processing of the present invention.Wherein, a kind of method of data processing, comprising:
Firstly, Fig. 1-a is a kind of stream of the method for data processing that one embodiment of the present of invention provides referring to Fig. 1-a
Journey schematic diagram.Wherein, as shown in Fig. 1-a, a kind of method for data processing that one embodiment of the present of invention provides may include:
101, distributed memory system obtains N number of copy data from N number of memory node.
Wherein, n-th obtained in N number of copy data from n-th of memory node in N number of memory node is secondary
Notebook data, N are positive integer, and 0 < n≤N and n are integer.
Distributed memory system is that data dispersion is stored in more independent equipment, using expansible system knot
Structure shares storage load using more storage servers, positions storage information using location server.
Optionally, memory node is more independent equipment, wherein more independent equipment for example may include more clothes
Business device.
Wherein, N for example can be 1,2,3,4,10 or other positive integers.
Wherein, n for example can be 1,2,3,4,10 or other positive integers.
Referring to Fig. 1-b, Fig. 1-b is a kind of distributed memory system for providing of one embodiment of the present of invention from N number of storage
Node obtains the schematic diagram of N number of copy data.
102, distributed memory system verifies N number of copy data.
103, when verification passes through, distributed memory system migrates N number of copy data to default node.
- a referring to fig. 2, Fig. 2-a are the process of the method for another data processing that another embodiment of the invention provides
Schematic diagram.Wherein, as shown in Fig. 2-a, the method for another data processing that another embodiment of the invention provides be can wrap
It includes:
201, distributed memory system obtains N number of copy data from N number of memory node.
Wherein, n-th obtained in N number of copy data from n-th of memory node in N number of memory node is secondary
Notebook data, N are positive integer, and 0 < n≤N and n are integer.
Distributed memory system is that data dispersion is stored in more independent equipment, using expansible system knot
Structure shares storage load using more storage servers, positions storage information using location server.
Optionally, memory node is more independent equipment, wherein more independent equipment for example may include more clothes
Business device.
Wherein, N for example can be 1,2,3,4,10 or other positive integers.
Wherein, n for example can be 1,2,3,4,10 or other positive integers.
Optionally, described to obtain N number of number of copies from N number of memory node in a kind of possible embodiment of the invention
According to, comprising:
Receive the reading message that client is sent, wherein the reading message carries N number of IP address and N number of copy data
Mark, N number of memory node and N number of IP address correspond, N number of copy data and N number of copy data
Mark corresponds；
Read the N number of IP address and N number of copy data mark in the reading message；
According to N number of IP address, N number of copy data request message is sent to N number of memory node, wherein the N
A memory node and N number of copy data request message correspond, and N number of copy data request message carries the N
A copy data mark, N number of copy data request message are used to indicate N number of memory node and save from N number of storage
It is searched in the database of point and identifies matched N number of copy data with N number of copy data, N number of copy data is asked
N-th of copy data request message in message is asked to carry n-th of copy data mark in N number of copy data mark,
N-th of copy data request message is used to indicate n-th of memory node from the database of n-th of memory node
Middle lookup identifies matched n-th of copy data with n-th of copy data；
N number of memory node is received to disappear for N number of copy data response that N number of copy data request message is sent
Breath, wherein the N number of copy data response message carries N number of copy data, N number of copy data request message with
N number of copy data response message corresponds, n-th of copy data response in N number of copy data response message
Message carries n-th of copy data；
Read N number of copy data in N number of copy data response message.
- b referring to fig. 2, Fig. 2-b are a kind of distributed memory system for providing of one embodiment of the present of invention from N number of storage
N-th of memory node in node obtains the schematic diagram of n-th of copy data in N number of copy data.
202, distributed memory system carries out completeness check to N number of copy data.
Optionally, described to N number of copy data in a first aspect, in a kind of possible embodiment of the invention
Carry out completeness check, comprising:
S1: N number of copy data is divided into I group, wherein I is positive integer；
S2: the integer that initial value is 1 is set by i；
S3: whether detection i is less than or equal to I, if so, S4 is thened follow the steps, if it is not, thening follow the steps S6；
S4: completeness check model is called to carry out completeness check copy data in organizing since i-th group in I group；
S5: (i+1) is assigned to i, returns to step S3；
S6: copy data in described group is terminated the completeness check model is called to carry out completeness check.
Wherein, I for example can be 1,2,3,4,10 or other positive integers.
Wherein, i for example can be 1,2,3,4,10 or other positive integers.
Wherein, completeness check model includes MD5 algorithm or other are used for the algorithm of checking data integrity.
Wherein, MD5 Message Digest 5 (English: MD5Message-Digest Algorithm), one kind is widely used
Cryptographic Hash Function, can produce out the hashed value (hash value) of one 128 (16 byte), for ensure information pass
It is finished whole consistent.
Optionally, described by N number of copy based in a first aspect, in a kind of possible embodiment of the invention
Data are divided into I group, comprising:
S1: whether the remainder X for calculating N/I is equal to zero, if so, step S2 is executed, if it is not, executing step S3, wherein X is
Natural number；
S2: N number of copy data is divided into I group；
S3: being divided into I group for (N-X) a copy data, and X copy data is divided into one group, to obtain K group, wherein
K is equal to (I+1), and K is positive integer；
S4: K is assigned to I.
For example, 10 N, I 3, then the remainder X of N/I is 1, and 9 copy datas are then averaged by distributed memory system
It is divided into 3 groups, remaining 1 copy data is divided into one group.
Optionally, described by N number of copy based in a first aspect, in a kind of possible embodiment of the invention
Data are divided into I group, comprising:
Random algorithm is called to be randomly divided into I group N number of copy data.
203, when completeness check passes through, distributed memory system to each numerical value in N number of copy data into
Row verification.
204, when verification passes through, distributed memory system migrates N number of copy data to default node.
Optionally, described when verification passes through in a first aspect, in a kind of possible embodiment of the invention, by institute
N number of copy data is stated to migrate to default node, comprising:
If each numerical value in N number of copy data verifies unanimously, by any pair in N number of copy data
Notebook data is migrated to the default node；
If having the numerical check of M copy data consistent in N number of copy data, and when N is odd number, ((N-1)/
2)≤M < N, when N is even number, (N/2)≤M < N then migrates any copy data in the M copy data to institute
State default node, wherein M positive integer；
If having the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-
1)/2), when N is even number, J < (N/2) then abandons migrating, wherein J is natural number.
Wherein, M for example can be 1,2,3,4,10 or other positive integers.
Wherein, J for example can be 0,1,2,3,4,10 or other integers.
Optionally, based in a first aspect, in a kind of possible embodiment of the invention, when J is greater than 0, if described
There is the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-1)/2), when N is
When even number, J < (N/2), then after abandoning migration, comprising:
Verification request message is sent to server-side, wherein the verification request message carries the J copy data, institute
It states verification request message and is used to indicate the server-side according to default first verification rule to each of described J copy data
Copy data is verified, to obtain meeting Q copy data of the default first verification rule, wherein Q is positive integer, Q
≤J；
Receive the verification response message that the server-side is sent for the verification request message, wherein the verification is rung
Message is answered to carry the Q copy data；
Read the Q copy data in the verification response message；
Any copy data in the Q copy data is migrated to the default node.
Wherein, Q for example can be 1,2,3,4,10 or other positive integers.
Referring to Fig. 3, Fig. 3 is the process signal of the method for another data processing that another embodiment of the invention provides
Figure.Wherein, as shown in figure 3, the method for another data processing that another embodiment of the invention provides may include:
301, distributed memory system obtains N number of copy data from N number of memory node.
Wherein, n-th obtained in N number of copy data from n-th of memory node in N number of memory node is secondary
Notebook data, N are positive integer, and 0 < n≤N and n are integer.
Distributed memory system is that data dispersion is stored in more independent equipment, using expansible system knot
Structure shares storage load using more storage servers, positions storage information using location server.
Optionally, memory node is more independent equipment, wherein more independent equipment for example may include more clothes
Business device.
Wherein, N for example can be 1,2,3,4,10 or other positive integers.
Wherein, n for example can be 1,2,3,4,10 or other positive integers.
Optionally, described to obtain N number of number of copies from N number of memory node in a kind of possible embodiment of the invention
According to, comprising:
Receive the reading message that client is sent, wherein the reading message carries N number of IP address and N number of copy data
Mark, N number of memory node and N number of IP address correspond, N number of copy data and N number of copy data
Mark corresponds；
Read the N number of IP address and N number of copy data mark in the reading message；
According to N number of IP address, N number of copy data request message is sent to N number of memory node, wherein the N
A memory node and N number of copy data request message correspond, and N number of copy data request message carries the N
A copy data mark, N number of copy data request message are used to indicate N number of memory node and save from N number of storage
It is searched in the database of point and identifies matched N number of copy data with N number of copy data, N number of copy data is asked
N-th of copy data request message in message is asked to carry n-th of copy data mark in N number of copy data mark,
N-th of copy data request message is used to indicate n-th of memory node from the database of n-th of memory node
Middle lookup identifies matched n-th of copy data with n-th of copy data；
N number of memory node is received to disappear for N number of copy data response that N number of copy data request message is sent
Breath, wherein the N number of copy data response message carries N number of copy data, N number of copy data request message with
N number of copy data response message corresponds, n-th of copy data response in N number of copy data response message
Message carries n-th of copy data；
Read N number of copy data in N number of copy data response message.
302, N number of copy data is divided into I group by distributed memory system.
Wherein, I is positive integer.
Wherein, I for example can be 1,2,3,4,10 or other positive integers.
303, i is set the integer that initial value is 1 by distributed memory system.
Wherein, i for example can be 1,2,3,4,10 or other positive integers.
304, whether distributed memory system detection i is less than or equal to I, if so, S4 is thened follow the steps, if it is not, then executing
Step S6.
Optionally, step S4 is 305.
Optionally, step S6 is 307.
305, distributed memory system calls completeness check model to copy data in organizing since i-th group in I group
Carry out completeness check.
Wherein, completeness check model includes MD5 algorithm or other are used for the algorithm of checking data integrity.
Wherein, MD5 Message Digest 5 (English: MD5Message-Digest Algorithm), one kind is widely used
Cryptographic Hash Function, can produce out the hashed value (hash value) of one 128 (16 byte), for ensure information pass
It is finished whole consistent.
306, (i+1) is assigned to i by distributed memory system, returns to step S3.
Optionally, step S3 is 304.
307, distributed memory system terminates copy data in described group the completeness check model is called to carry out
Whole property verification.
Optionally, described that N number of copy data is divided into I group in a kind of possible embodiment of the invention, packet
It includes:
S1: whether the remainder X for calculating N/I is equal to zero, if so, step S2 is executed, if it is not, executing step S3, wherein X is
Natural number；
S2: N number of copy data is divided into I group；
S3: being divided into I group for (N-X) a copy data, and X copy data is divided into one group, to obtain K group, wherein
K is equal to (I+1), and K is positive integer；
S4: K is assigned to I.
For example, 10 N, I 3, then the remainder X of N/I is 1, and 9 copy datas are then averaged by distributed memory system
It is divided into 3 groups, remaining 1 copy data is divided into one group.
Optionally, described by N number of copy based in a first aspect, in a kind of possible embodiment of the invention
Data are divided into I group, comprising:
Random algorithm is called to be randomly divided into I group N number of copy data.
308, when completeness check passes through, distributed memory system to each numerical value in N number of copy data into
Row verification.
309, when verification passes through, distributed memory system migrates N number of copy data.
Optionally, described when verification passes through in a first aspect, in a kind of possible embodiment of the invention, by institute
N number of copy data is stated to migrate to default node, comprising:
If each numerical value in N number of copy data verifies unanimously, by any pair in N number of copy data
Notebook data is migrated to the default node；
If having the numerical check of M copy data consistent in N number of copy data, and when N is odd number, ((N-1)/
2)≤M < N, when N is even number, (N/2)≤M < N then migrates any copy data in the M copy data to institute
State default node, wherein M positive integer；
If having the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-
1)/2), when N is even number, J < (N/2) then abandons migrating, wherein J is natural number.
Wherein, M for example can be 1,2,3,4,10 or other positive integers.
Wherein, J for example can be 0,1,2,3,4,10 or other integers.
Optionally, based in a first aspect, in a kind of possible embodiment of the invention, when J is greater than 0, if described
There is the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-1)/2), when N is
When even number, J < (N/2), then after abandoning migration, comprising:
Verification request message is sent to server-side, wherein the verification request message carries the J copy data, institute
It states verification request message and is used to indicate the server-side according to default first verification rule to each of described J copy data
Copy data is verified, to obtain meeting Q copy data of the default first verification rule, wherein Q is positive integer, Q
≤J；
Receive the verification response message that the server-side is sent for the verification request message, wherein the verification is rung
Message is answered to carry the Q copy data；
Read the Q copy data in the verification response message；
Any copy data in the Q copy data is migrated to the default node.
Wherein, Q for example can be 1,2,3,4,10 or other positive integers.
Optionally, based in a first aspect, in a kind of possible embodiment of the invention, when J is equal to 0, if described
There is the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-1)/2), when N is
When even number, J < (N/2), then after abandoning migration, comprising:
Request message is verified again to server-side transmission, wherein the request message of verification again carries described N number of
Copy data, the request message of verification again are used to indicate the server-side according to default second verification rule to described N number of
Each copy data in copy data is verified, to obtain meeting L copy data of the default second verification rule,
And then N number of copy data is replaced to obtain N number of replacement pair according to any copy data in the L copy data
Notebook data, wherein L is positive integer, L≤N；
Receive the verification response message again that the server-side is sent for the request message of verification again, wherein institute
It states verification response message again and carries N number of replacement copy data；
Read the N number of replacement copy data verified in response message again；
Any copy data in N number of replacement copy data is migrated to the default node.
Wherein, L for example can be 1,2,3,4,10 or other positive integers.
Wherein, the verification permission for presetting the second verification rule is higher than default first verification rule.
Optionally, based in a first aspect, in a kind of possible embodiment of the invention, further includes:
According to N number of IP address, N number of copy data is sent to N number of memory node and deletes message, wherein the N
A memory node and N number of copy data are deleted message and are corresponded, and N number of copy data deletes message and carries the N
A copy data mark, N number of copy data delete message and are used to indicate N number of memory node from N number of storage section
It is deleted in the database of point and identifies matched N number of copy data with N number of copy data, N number of copy data is deleted
Message is deleted except n-th of copy data in message and carries n-th of copy data mark, and n-th of copy data is deleted
It is deleted from the database of n-th of memory node and n-th of pair except message is used to indicate n-th of memory node
Notebook data identifies matched n-th of copy data.
It can be seen that above-mentioned technical proposal, it is N number of by being deleted from the database in N number of memory node after migration
Copy data has saved memory space.
Firstly, referring to fig. 4, Fig. 4 is a kind of schematic diagram for data processing equipment that one embodiment of the present of invention provides.Its
In, as shown in figure 4, a kind of data processing equipment 400 that one embodiment of the present of invention provides may include:
Module 401 is obtained, for obtaining N number of copy data from N number of memory node.Wherein, from N number of memory node
N-th of memory node obtain n-th of copy data in N number of copy data, N is positive integer, and 0 < n≤N and n are whole
Number.
Distributed memory system is that data dispersion is stored in more independent equipment, using expansible system knot
Structure shares storage load using more storage servers, positions storage information using location server.
Optionally, memory node is more independent equipment, wherein more independent equipment for example may include more clothes
Business device.
Wherein, N for example can be 1,2,3,4,10 or other positive integers.
Wherein, n for example can be 1,2,3,4,10 or other positive integers.
Correction verification module 402, for being verified to N number of copy data.
Transferring module 403, for when verification passes through, N number of copy data to be migrated to default node.
The embodiment of the present invention also provides a kind of electronic equipment of data processing, including processor, memory, communication interface
And one or more programs, wherein one or more of programs are stored in the memory, and are configured by institute
Processor execution is stated, to execute the instruction included the steps that in a kind of method of data processing of any one.
The embodiment of the present invention also provides a kind of computer storage medium, and the computer readable storage medium is based on storing
Calculation machine program, the storage computer program are executed by the processor, to realize a kind of including side of data processing of any one
Method.
It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because
According to the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know
It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules is not necessarily of the invention
It is necessary.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed device, it can be by another way
It realizes.For example, the apparatus embodiments described above are merely exemplary, such as the division of the module, it is only a kind of
Logical function partition, there may be another division manner in actual implementation, such as multiple module or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be through some interfaces, the indirect coupling or communication connection of device or module,
It can be electrical or other forms.
The module as illustrated by the separation member may or may not be physically separated, aobvious as module
The component shown may or may not be physical module, it can and it is in one place, or may be distributed over multiple
On network module.Some or all of the modules therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in a processing module
It is that modules physically exist alone, can also be integrated in two or more modules in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.
If the integrated module is realized in the form of software function module and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including several message are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although referring to before
Stating embodiment, invention is explained in detail, those skilled in the art should understand that: it still can be to preceding
Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features；And these
It modifies or replaces, the range for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (10)
1. a kind of method of data processing characterized by comprising
N number of copy data is obtained from N number of memory node, wherein is obtained from n-th of memory node in N number of memory node
N-th of copy data in N number of copy data, N are positive integer, and 0 < n≤N and n are integer；
N number of copy data is verified；
When verification passes through, N number of copy data is migrated to default node.
2. the method according to claim 1, wherein described obtain N number of copy data, packet from N number of memory node
It includes:
Receiving the reading message that client is sent, wherein the reading message carries N number of IP address and N number of copy data mark,
N number of memory node and N number of IP address correspond, N number of copy data and N number of copy data mark one
One is corresponding；
Read the N number of IP address and N number of copy data mark in the reading message；
According to N number of IP address, N number of copy data request message is sent to N number of memory node, wherein described N number of to deposit
It stores up node and N number of copy data request message corresponds, N number of copy data request message carries N number of pair
Notebook data mark, N number of copy data request message are used to indicate N number of memory node from N number of memory node
It is searched in database and identifies matched N number of copy data with N number of copy data, N number of copy data request disappears
N-th of copy data request message in breath carries n-th of copy data mark in N number of copy data mark, described
N-th of copy data request message is used to indicate n-th of memory node and looks into from the database of n-th of memory node
It looks for and identifies matched n-th of copy data with n-th of copy data；
N number of copy data response message that N number of memory node is sent for N number of copy data request message is received,
Wherein, the N number of copy data response message carries N number of copy data, N number of copy data request message with it is described
N number of copy data response message corresponds, n-th of copy data response message in N number of copy data response message
Carry n-th of copy data；
Read N number of copy data in N number of copy data response message.
3. the method according to claim 1, wherein described verify N number of copy data, comprising:
Completeness check is carried out to N number of copy data；
When completeness check passes through, each numerical value in N number of copy data is verified.
4. according to the method described in claim 3, it is characterized in that, described carry out completeness check to the N number of copy data,
Include:
S1: N number of copy data is divided into I group, wherein I is positive integer；
S2: the integer that initial value is 1 is set by i；
S3: whether detection i is less than or equal to I, if so, S4 is thened follow the steps, if it is not, thening follow the steps S6；
S4: completeness check model is called to carry out completeness check copy data in organizing since i-th group in I group；
S5: (i+1) is assigned to i, returns to step S3；
S6: copy data in described group is terminated the completeness check model is called to carry out completeness check.
5. according to the method described in claim 4, it is characterized in that, described be divided into I group for N number of copy data, comprising:
S1: whether the remainder X for calculating N/I is equal to zero, if so, step S2 is executed, if it is not, executing step S3, wherein X is nature
Number；
S2: N number of copy data is divided into I group；
S3: being divided into I group for (N-X) a copy data, and X copy data is divided into one group, to obtain K group, wherein K etc.
In (I+1), K be positive integer；
S4: K is assigned to I.
6. N number of copy data is moved the method according to claim 1, wherein described when verification passes through
Move to default node, comprising:
If each numerical value in N number of copy data verifies unanimously, by any number of copies in N number of copy data
According to migrating to the default node；
If having the numerical check of M copy data consistent in N number of copy data, and when N is odd number, ((N-1)/2)≤M
< N, when N is even number, (N/2)≤M < N then migrates any copy data in the M copy data to described default
Node, wherein M positive integer；
If having the numerical check of J copy data consistent in N number of copy data, and when N is odd number, J < ((N-1)/
2), when N is even number, J < (N/2) then abandons migrating, wherein J is natural number.
7. according to the method described in claim 6, it is characterized in that, J be greater than 0 when, if having in N number of copy data
The numerical check of J copy data is consistent, and when N is odd number, J < ((N-1)/2), when N is even number, J < (N/2), then
After abandoning migration, comprising:
Verification request message is sent to server-side, wherein the verification request message carries the J copy data, the school
It tests request message and is used to indicate the server-side according to default first verification rule to each copy in the J copy data
Data are verified, to obtain meeting Q copy data of the default first verification rule, wherein Q is positive integer, Q≤J；
Receive the verification response message that the server-side is sent for the verification request message, wherein the verification response disappears
Breath carries the Q copy data；
Read the Q copy data in the verification response message；
Any copy data in the Q copy data is migrated to the default node.
8. a kind of data processing equipment characterized by comprising
Module is obtained, for obtaining N number of copy data from N number of memory node, wherein n-th from N number of memory node
Memory node obtains n-th of copy data in N number of copy data, and N is positive integer, and 0 < n≤N and n are integer；
Correction verification module, for being verified to N number of copy data；
Transferring module, for when verification passes through, N number of copy data to be migrated to default node.
9. a kind of electronic equipment of data processing, which is characterized in that including processor, memory, communication interface and one or
Multiple programs, wherein one or more of programs are stored in the memory, and are configured to be held by the processor
Row requires the instruction of the step in any one of 1-7 method with perform claim.
10. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium is for storing computer
Program, the storage computer program are executed by the processor, to realize the described in any item methods of claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007413.5A CN109815047B (en) | 2019-01-04 | 2019-01-04 | Data processing method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007413.5A CN109815047B (en) | 2019-01-04 | 2019-01-04 | Data processing method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109815047A true CN109815047A (en) | 2019-05-28 |
| CN109815047B CN109815047B (en) | 2023-03-14 |
Family
ID=66603919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910007413.5A Active CN109815047B (en) | 2019-01-04 | 2019-01-04 | Data processing method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109815047B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111339061A (en) * | 2020-02-12 | 2020-06-26 | 杭州涂鸦信息技术有限公司 | Data migration method and system of distributed database |
| CN112363675A (en) * | 2020-11-18 | 2021-02-12 | 苏州元核云技术有限公司 | Control method and system based on distributed storage system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060069941A1 (en) * | 2004-09-14 | 2006-03-30 | Seagate Technology Llc | Embedded system with reduced susceptibility to single event upset effects |
| CN103294787A (en) * | 2013-05-21 | 2013-09-11 | 成都市欧冠信息技术有限责任公司 | Multi-copy storage method and multi-copy storage system for distributed database system |
| CN103927285A (en) * | 2014-04-23 | 2014-07-16 | 上海乐耘电气技术有限公司 | High-reliability data transmission method for two-channel serial buses |
| CN106488392A (en) * | 2016-09-23 | 2017-03-08 | 北京奇虎科技有限公司 | The method and device of data transfer |
-
2019
- 2019-01-04 CN CN201910007413.5A patent/CN109815047B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060069941A1 (en) * | 2004-09-14 | 2006-03-30 | Seagate Technology Llc | Embedded system with reduced susceptibility to single event upset effects |
| CN103294787A (en) * | 2013-05-21 | 2013-09-11 | 成都市欧冠信息技术有限责任公司 | Multi-copy storage method and multi-copy storage system for distributed database system |
| CN103927285A (en) * | 2014-04-23 | 2014-07-16 | 上海乐耘电气技术有限公司 | High-reliability data transmission method for two-channel serial buses |
| CN106488392A (en) * | 2016-09-23 | 2017-03-08 | 北京奇虎科技有限公司 | The method and device of data transfer |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111339061A (en) * | 2020-02-12 | 2020-06-26 | 杭州涂鸦信息技术有限公司 | Data migration method and system of distributed database |
| CN111339061B (en) * | 2020-02-12 | 2023-09-26 | 杭州涂鸦信息技术有限公司 | Data migration method and system for distributed database |
| CN112363675A (en) * | 2020-11-18 | 2021-02-12 | 苏州元核云技术有限公司 | Control method and system based on distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109815047B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110442652B (en) | Cross-chain data processing method and device based on block chain | |
| CN107577427B (en) | data migration method, device and storage medium for blockchain system | |
| US12010213B2 (en) | Pruning entries in tamper-evident data stores | |
| EP3652885B1 (en) | Secure token passing via blockchains | |
| CN113726751B (en) | Weight management method, device and equipment in block chain type account book | |
| CN115668194A (en) | Cross-partition calls in a partition's tamper-resistant data store | |
| CN104346365A (en) | Device and method for determining specific service associated logs | |
| CN107291750A (en) | A kind of data migration method and device | |
| CN110413595A (en) | A kind of data migration method and relevant apparatus applied to distributed data base | |
| CN107580032B (en) | Data processing method, device and equipment | |
| CN110069346A (en) | Resource share method, device, electronic equipment between multi-process | |
| US11204860B2 (en) | Methods and apparatuses for generating smart contract test case | |
| CN114281793A (en) | Data verification method, device and system | |
| CA3030250A1 (en) | File storage method and apparatus for blockchain network | |
| CN107102908A (en) | Data verification method, data fault tolerance method and device | |
| WO2022048359A1 (en) | Data archiving method and apparatus, and electronic device and storage medium | |
| CN110442502A (en) | A kind of method, apparatus buried a little, equipment and storage medium | |
| CN109815047A (en) | A kind of method and relevant apparatus of data processing | |
| CN109491962A (en) | A kind of file directory tree management method and relevant apparatus | |
| CN109597903A (en) | Image file processing apparatus and method, document storage system and storage medium | |
| CN112040015B (en) | Consensus node list generation method and related device | |
| CN107111495A (en) | Apparatus and method for virtual and calling interface method | |
| CN111428114A (en) | Index creating method and device for Elasticissearch search engine | |
| CN110362294A (en) | Development task executes method, apparatus, electronic equipment and storage medium | |
| CN103176843B (en) | The file migration method and apparatus of MapReduce distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |