CN107102918A - A kind of data processing method and device - Google Patents
A kind of data processing method and device Download PDFInfo
- Publication number
- CN107102918A CN107102918A CN201710386160.8A CN201710386160A CN107102918A CN 107102918 A CN107102918 A CN 107102918A CN 201710386160 A CN201710386160 A CN 201710386160A CN 107102918 A CN107102918 A CN 107102918A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- affiliated
- memory cell
- data flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 230000006835 compression Effects 0.000 claims abstract description 18
- 238000007906 compression Methods 0.000 claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 11
- 238000007405 data analysis Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2056—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
- G06F11/2064—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring while ensuring consistency
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2056—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
- G06F11/2069—Management of state, configuration or failover
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a kind of data processing method and device, method includes：When the first data data flow reach forwarding after, judge this node whether be first data data flow destination node；The destination node includes the cooperative nodes of the affiliated node of the first memory cell and the affiliated node of the first memory cell；Wherein, first memory cell is the writing unit of first data；If not, determining the destination node of the data flow of first data, the stream compression of first data is sent to the destination node, the data flow of first data is sent to cache layer by the destination node；If it is, the data flow of first data is sent into cache layer, by this invention ensures that the uniformity of data.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of data processing method and device.
Background technology
Most of existing storage system is binode storage system, when a node breaks down, can be used another
One node comes data cached, so as to ensure being normally carried out for business.
In recent years, in order to further improve the security of data buffer storage, on the basis of binode storage system, it is proposed that
Multinode storage system, multinode storage system still continues to use binodal point mode to carry out the caching of data.Specifically, for a certain
When data are cached, the data flow of the data can be forwarded in any node, asked so as to the inconsistent technology of data occur
Topic.
The content of the invention
In view of this, the present invention provides a kind of data processing method and device, inconsistent to solve data in the prior art
Technical problem.
To achieve the above object, the present invention provides following technical scheme：
A kind of data processing method, including：
When the first data data flow reach forwarding after, judge this node whether be first data data flow
Destination node；The destination node includes the cooperation of the affiliated node of the first memory cell and the affiliated node of the first memory cell
Node；Wherein, first memory cell is the writing unit of first data；
If not, the destination node of the data flow of first data is determined, by the forwarding data flow of first data
To the destination node, the data flow of first data is sent to cache layer by the destination node；
If it is, the data flow of first data is sent into cache layer.
It is preferred that, it is described judge this node whether be first data the destination node of data flow before, in addition to：
Whether judge this node is node in the affiliated node group of the first memory cell；
If it is, triggering is described to judge the step of whether this node is the destination node of data flow of first data；
If not, the step of destination node of triggering determination first data flow.
It is preferred that, the data flow by first data is sent to after cache layer, in addition to：
When this node is the affiliated node of first memory cell, by first data image to the affiliated section
The cooperative nodes of point；
When cooperative nodes of this node for the affiliated node of first memory cell, first data image is arrived
The preposition node of this node；
Wherein, the preposition node of described node is the affiliated node of first memory cell.
It is preferred that, the destination node for determining first data flow includes：
It is mesh that the big node of data-handling capacity is chosen from the cooperative nodes of the affiliated node and the affiliated node
Node.
It is preferred that, the selection data-handling capacity from the cooperative nodes of the affiliated node and the affiliated node is big
Node be purpose node, including：
The last node for carrying out data processing is determined from the cooperative nodes of the affiliated node and the affiliated node；
It is determined that except another node of the last node for carrying out data processing is purpose node.
A kind of data processing equipment, including：
First judging unit, for whether after the data flow of the first data reaches forwarding, it to be described to judge this node
The destination node of the data flow of first data；The destination node includes the affiliated node and the first storage list of the first memory cell
The cooperative nodes of the affiliated node of member；Wherein, first memory cell is the writing unit of first data；
Retransmission unit being determined, during for not being the destination node of data flow of first data when this node, institute is determined
The destination node of the data flow of the first data is stated, the stream compression of first data destination node is sent to, by described
The data flow of first data is sent to cache layer by destination node；
First transmitting element, during for being the destination node of data flow of first data when this node, by described
The data flow of one data is sent to cache layer.
It is preferred that, in addition to：
Second judging unit, for whether after the data flow of the first data reaches forwarding, it to be first to judge this node
Node in the affiliated node group of memory cell；
First trigger element, for when this node is the node in the affiliated node group of the first memory cell, triggering to be described
First judging unit perform judge this node whether be first data data flow destination node operation；
Second trigger element, for when this node is not the node in the affiliated node group of the first memory cell, triggering institute
State the operation for determining that retransmission unit performs the destination node for determining first data flow.
It is preferred that, in addition to：
First mirror image unit, for when this node is the affiliated node of first memory cell, described first to be counted
According to the cooperative nodes for being mirrored to the affiliated node；
Second mirror image unit, for when this node for first memory cell affiliated node cooperative nodes when, will
Preposition node of first data image to this node；
Wherein, the preposition node of described node is the affiliated node of first memory cell.
It is preferred that, the determination retransmission unit includes：
First determining module, during for not being the destination node of data flow of first data when this node, from described
It is purpose node that the big node of data-handling capacity is chosen in the cooperative nodes of affiliated node and the affiliated node；
First forwarding module, for the stream compression of first data to be sent into the destination node, by the purpose
The data flow of first data is sent to cache layer by node.
It is preferred that, first determining module is specifically for the cooperative nodes from the affiliated node and the affiliated node
It is middle to determine the last node for carrying out data processing, so that it is determined that except another section of the last node for carrying out data processing
Point is purpose node.
Understand that compared with prior art, the present disclosure provides a kind of data processing side via above-mentioned technical scheme
Method, specifically, when the first data data flow reach forwarding after, judge this node whether be the first data data flow mesh
Node, the cooperative nodes of the affiliated node of destination node including the first memory cell and the affiliated node of the first memory cell；
Wherein, the first memory cell be first data writing unit；If it is not, then determine the data flow of the first data
Destination node, destination node is sent to by the stream compression of the first data, is finally sent out the data flow of the first data by destination node
It is sent to cache layer.As can be seen here, the data flow of the first data in the application can be forwarded in destination node, delayed by destination node
It is stored to cache layer, it is ensured that the uniformity of data.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is a kind of structural representation of multinode storage system disclosed by the invention；
Fig. 2 is a kind of schematic flow sheet of data processing method disclosed in the embodiment of the present invention one；
Fig. 3 is a kind of schematic flow sheet of data processing method disclosed in the embodiment of the present invention two；
Fig. 4 is a kind of schematic flow sheet of data processing method disclosed in the embodiment of the present invention three；
Fig. 5 is a kind of structural representation of data processing equipment disclosed in the embodiment of the present invention four；
Fig. 6 is a kind of structural representation of data processing equipment disclosed in the embodiment of the present invention five；
Fig. 7 is a kind of structural representation of data processing equipment disclosed in the embodiment of the present invention six.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
Embodiment one
The present invention implements one and discloses a kind of data processing method, and the data processing method is applied to multinode storage system
In, specifically, Fig. 1 shows a kind of structural representation of multinode storage system disclosed by the invention, as shown in figure 1, client
End Host is connected with N number of node, and N number of node constitutes a node group, wherein, a node is previous node adjacent thereto
Cooperative nodes, be the preposition node of latter node adjacent thereto.That is, being folk prescription between two adjacent nodes
Cooperative relationship, is not cooperative relationship each other.That is, Node1 is Node0 cooperative nodes, the preposition node for being Node2；Node2
For Node1 cooperative nodes, the preposition node ... for being Node3 is by that analogy.
It should be noted that, multinode storage system includes at least one node group, and Fig. 1 is with client one node group of correspondence
Exemplified by, certain client can correspond to multiple node groups, and the present invention is not limited this.And each node group includes at least two
Individual node.
Fig. 2 shows a kind of schematic flow sheet of data processing method, as shown in Fig. 2 this method comprises the following steps：
Step 201：When the first data data flow reach forwarding after, judge this node whether be the first data data
The destination node of stream；If it is not, into step 202；If so, into step 203；
This node is referred to as receiving the node of the data flow of the first data in its forwarding.
Wherein, destination node includes the cooperation of the affiliated node of the first memory cell and the affiliated node of the first memory cell
Node.
First memory cell is the writing unit of the first data, specifically, the first memory cell is LUN (Logical
Unit Number, LUN), a LUN can be made up of one piece of disk, can also be made up of several pieces of disks, can be with
Segment space on one piece of disk is constituted etc..
The affiliated node of first memory cell is the node for possessing first memory cell, and the node is the first memory cell
Specified node, for by need write the first memory cell data be written in the first memory cell.First memory cell
Affiliated node cooperative nodes be the node adjacent with the affiliated node of the first memory cell, by being stored previously for multinode
The description of system is understood, is closed between the affiliated node of the first memory cell and the cooperative nodes of affiliated node for one direction cooperation
System, the affiliated node of the first memory cell is the preposition node of the cooperative nodes of the affiliated node of the first memory cell.
After the data flow for the first data that client is issued reaches the forwarding of a certain node, the node can judge this section
Point whether be the first data data flow destination node.
Specifically, the node index of this node that is stored with multinode storage system on each node and all nodes
Message bit pattern, therefore, the message bit pattern of node index and all nodes based on this node can interpolate that out whether is this node
For the destination node of the data flow of the first data.For example, message bit pattern is the integer value of 32, i.e., for node 0-31.This section
The node index of point is N, then, the first storage received on node N is represented if the nth position 1 of message bit pattern
The data flow of unit needs forwarding, i.e. node N is not the destination node of the data flow of the first data, if the N of message bit pattern
To set to 0, representing the data flow of the first memory cell received on node N need not forward, i.e. node N is the first data
The destination node of data flow.That is, except the first memory cell belonging to the data flow of the first data in message bit pattern
Outside the position of affiliated node and the position 0 of cooperative nodes, remaining position puts 1.
Step 202：The destination node of the data flow of the first data is determined, the stream compression of the first data is sent to purpose section
Point, is sent the data flow of the first data to cache layer by destination node；
, then can be from the institute of the first memory cell when it is determined that this node is not the destination node of the data flow of the first data
One node of selection is as destination node in the cooperative nodes for the affiliated node for belonging to node and the first memory cell, so that by first
The stream compression of data is sent to the forwarding of destination node, is sent the data flow of the first data to destination node by destination node
Cache layer.
Step 203：The data flow of first data is sent to cache layer.
When it is determined that this node is the destination node of the data flow of the first data, then directly by this node by the first data
Data flow is sent to the cache layer of this node.It should be noted that, may also have between the forwarding and cache layer of first node
There are other data analysis layers, still, the data flow of first data is ultimately sent in cache layer, i.e., by the first data
Data flow is sent to cache layer and is not necessarily referring to be directly issued to cache layer by forwarding, and may refer to as by forwarding
It is issued to other data analysis layers and cache layer is ultimately routed to after data processing.
In the present embodiment, whether after the data flow of the first data reaches forwarding, it is the first data to judge this node
Data flow destination node, the affiliated node of destination node including the first memory cell and the affiliated node of the first memory cell
Cooperative nodes；Wherein, the first memory cell be first data writing unit；If it is not, then determining the first number
According to data flow destination node, the stream compression of the first data is sent to destination node, finally by destination node by first number
According to data flow be sent to cache layer.As can be seen here, the data flow of the first data in the application can be forwarded in destination node,
Cache layer is cached to by destination node, it is ensured that the uniformity of data.
Embodiment two
The embodiment of the present invention two also discloses a kind of data processing method, as shown in figure 3, this method comprises the following steps：
Step 301：Whether after the data flow of the first data reaches forwarding, it is the first memory cell institute to judge this node
Belong to the node in node group；If so, into step 302；If it is not, into step 303；
First memory cell is the writing unit of the first data, with the affiliated node for possessing first memory cell.
In multinode storage system, each node belongs to a node group, accordingly, the affiliated node group of the first memory cell be with
The node group of the affiliated node of first memory cell.
Specifically, the node index of this node that is stored with multinode storage system on each node and all nodes
Message bit pattern, therefore, the message bit pattern of node index and all nodes based on this node can interpolate that out whether is this node
For the node in the affiliated node group of the first memory cell.
Step 302：Judge this node whether be first data data flow destination node；If it is not, into step
303, if so, into step 304；
Step 303：The destination node of the data flow of first data is determined, by the forwarding data flow of first data
To the destination node, the data flow of first data is sent to cache layer by the destination node；
As a kind of concrete implementation mode, determining the destination node of first data flow includes：From affiliated node and
It is purpose node that the big node of data-handling capacity is chosen in the cooperative nodes of affiliated node.
Specifically, for the purpose of choosing the big node of data-handling capacity from the cooperative nodes of affiliated node and affiliated node
Node, including：The last node for carrying out data processing is determined from the cooperative nodes of affiliated node and affiliated node, so that really
Fixed another node except the node of last data processing is purpose node.
Wherein, in the affiliated node of the first memory cell and the cooperative nodes of affiliated node, determine that the last time enters line number
According to the node of processing, if it is last carry out data processing node be the first memory cell affiliated node, then then determine the
The cooperative nodes of the affiliated node of one memory cell are purpose node；If it is the first storage that the last time, which carries out the node of data processing,
The cooperative nodes of the affiliated node of unit, then the affiliated node for then determining the first memory cell is purpose node.
Step 304：The data flow of first data is sent to cache layer.
In the present embodiment, whether after the data flow of the first data reaches forwarding, it is the first storage to judge this node
Node in the affiliated node group of unit, if this node is the node in the affiliated node group of the first memory cell, then then need to judge
This node whether be first data data flow destination node, if this node is not the affiliated node of the first memory cell
Node in group, then then can directly determine the destination node of the data flow of the first data, is incited somebody to action so as to realize by destination node
The cache layer that the data flow of first data is sent to, it is ensured that the uniformity of data.
Embodiment three
The embodiment of the present invention three also discloses a kind of data processing method, as shown in figure 4, this method comprises the following steps：
Step 401：When the first data data flow reach forwarding after, judge this node whether be the first data data
The destination node of stream；If it is not, into step 402；If so, into step 403；
Wherein, destination node includes the cooperation of the affiliated node of the first memory cell and the affiliated node of the first memory cell
Node.
First memory cell is the writing unit of the first data.
Step 402：The destination node of the data flow of the first data is determined, the stream compression of the first data is sent to purpose section
Point, is sent the data flow of the first data to cache layer by destination node；
, then can be from the institute of the first memory cell when it is determined that this node is not the destination node of the data flow of the first data
One node of selection is as destination node in the cooperative nodes for the affiliated node for belonging to node and the first memory cell, so that by first
The stream compression of data is sent to the forwarding of destination node, is sent the data flow of the first data to destination node by destination node
Cache layer.
Step 403：The data flow of first data is sent to cache layer；
Step 404：When this node is the affiliated node of the first memory cell, by the first data image to affiliated node
Cooperative nodes；
Step 405：When cooperative nodes of this node for the affiliated node of the first memory cell, the first data image is arrived
The preposition node of this node.
Wherein, the preposition node of this node is the affiliated node of the first memory cell.The affiliated node of first memory cell
It is for two adjacent nodes, and the affiliated node of the first memory cell with the cooperative nodes of the affiliated node of the first memory cell
The preposition node of the cooperative nodes of the affiliated node of first memory cell, when this node is the affiliated node of the first memory cell
During cooperative nodes, after the preposition node of the first data image to this node, then deposited equivalent to by the first data image to first
The affiliated node of storage unit.
In the present embodiment, after data flow reaches cache layer, when the affiliated section that this node is first memory cell
During point, by the cooperative nodes of first data image to the affiliated node, when this node is first memory cell
During the cooperative nodes of affiliated node, by the preposition node of first data image to this node, so as to ensure when a node
During failure, it can be carried out by another node come data cached.Also, deposited because the first data are stored in first respectively
In the affiliated node of storage unit and the cooperative nodes of affiliated node, therefore, it is possible to ensure the affiliated section by the first memory cell
The data that point is issued to the first memory cell are newest data.
Corresponding with a kind of above-mentioned data processing method, present invention also offers a kind of data processing equipment, the device should
For in multinode storage system.Specifically, being described in detail by following examples：
Example IV
The embodiment of the present invention four discloses a kind of data processing equipment, as shown in figure 5, the device includes：First judges single
First 501, retransmission unit 502, the first transmitting element 503 are determined；Wherein：
First judging unit 501, for whether after the data flow of the first data reaches forwarding, it to be institute to judge this node
State the destination node of the data flow of the first data；
Destination node includes the cooperative nodes of the affiliated node of the first memory cell and the affiliated node of the first memory cell；
Wherein, the first memory cell is the writing unit of first data.
The affiliated node of first memory cell is the node for possessing first memory cell, and the node is the first memory cell
Specified node, for by need write the first memory cell data be written in the first memory cell.First memory cell
Affiliated node cooperative nodes be the node adjacent with the affiliated node of the first memory cell, by being stored previously for multinode
The description of system is understood, is closed between the affiliated node of the first memory cell and the cooperative nodes of affiliated node for one direction cooperation
System, the affiliated node of the first memory cell is the preposition node of the cooperative nodes of the affiliated node of the first memory cell.
First judging unit 501 is used for the forwarding of a certain node of data flow arrival of the first data issued when client
Afterwards, judge this node whether be the first data data flow destination node.Specifically, each node in multinode storage system
On be stored with the information related to the first memory cell, the packet contains affiliated node and the first storage of the first memory cell
The cooperative nodes of the affiliated node of unit, thus the first judging unit based on the information be capable of determining that this node whether for the purpose of
Node.
Retransmission unit 502 is determined, during for not being the destination node of data flow of first data when this node, it is determined that
The destination node of the data flow of first data, is sent to the destination node, by institute by the stream compression of first data
State destination node and the data flow of first data is sent to cache layer；
When it is determined that this node is not the destination node of the data flow of the first data, determine that retransmission unit then can be from first
A node is selected to be used as purpose section in the cooperative nodes of the affiliated node of memory cell and the affiliated node of the first memory cell
Point, so that the stream compression of the first data to be sent to the forwarding of destination node, by destination node by the data flow of the first data
Send to the cache layer of destination node.
Specifically, as a kind of implementation, determining that retransmission unit 502 includes：First determining module and the first forwarding mould
Block.Wherein, the first determining module, during for not being the destination node of data flow of first data when this node, from described
It is purpose node that the big node of data-handling capacity is chosen in the cooperative nodes of affiliated node and the affiliated node；First forwarding
Module, for the stream compression of first data to be sent into the destination node, is counted by the destination node by described first
According to data flow be sent to cache layer.
Wherein, the first determining module from the cooperative nodes of the affiliated node and the affiliated node specifically for determining
Last time carries out the node of data processing, so that it is determined that except another node of the last node for carrying out data processing is mesh
Node.If that is, affiliated node of the last node for carrying out data processing for the first memory cell, then first is true
Cover half block then determines that the cooperative nodes of the affiliated node of the first memory cell are purpose node；If the last time carries out data processing
Node is the cooperative nodes of the affiliated node of the first memory cell, then the first determining module then determines the institute of the first memory cell
Category node is purpose node.
First transmitting element 503, will be described during for being the destination node of data flow of first data when this node
The data flow of first data is sent to cache layer.
In the present embodiment, whether after the data flow of the first data reaches forwarding, it is the first data to judge this node
Data flow destination node, the affiliated node of destination node including the first memory cell and the affiliated node of the first memory cell
Cooperative nodes；Wherein, the first memory cell be first data writing unit；If it is not, then determining the first number
According to data flow destination node, the stream compression of the first data is sent to destination node, finally by destination node by first number
According to data flow be sent to cache layer.As can be seen here, the data flow of the first data in the application can be forwarded in destination node,
Cache layer is cached to by destination node, it is ensured that the uniformity of data.
Embodiment five
The embodiment of the present invention five discloses a kind of data processing equipment, as shown in fig. 6, the device includes：Second judges single
First 601, first trigger element 602, the second trigger element 603, the first judging unit 604, determine that retransmission unit 605, first is sent out
Send unit 606；Wherein：
Second judging unit 601, for whether after the data flow of the first data reaches forwarding, it to be the to judge this node
Node in the affiliated node group of one memory cell；
First memory cell is the writing unit of the first data, with the affiliated node for possessing first memory cell.
In multinode storage system, each node belongs to a node group, accordingly, the affiliated node group of the first memory cell be with
The node group of the affiliated node of first memory cell.
Specifically, the information related to the first memory cell that is stored with multinode storage system on each node, the letter
Breath includes the cooperative nodes of the affiliated node of the first memory cell and the affiliated node of the first memory cell, also includes first
The affiliated node group identification of memory cell；The mark for its affiliated node group that is also stored with each node in multinode storage system.
Therefore, the second judging unit can be by judging the mark of the affiliated node group of this node and the affiliated node group mark of the first memory cell
Know and whether unanimously determine whether this node is node in the affiliated node group of the first memory cell.
First trigger element 602, for when this node is the node in the affiliated node group of the first memory cell, triggering the
One judging unit 604 perform judge this node whether be first data data flow destination node operation；
Second trigger element 603, for when this node is not the node in the affiliated node group of the first memory cell, triggering
Determine that retransmission unit 605 performs the operation for the destination node for determining the first data flow；
First judging unit 604, for judge this node whether be the first data data flow destination node；
Destination node includes the cooperative nodes of the affiliated node of the first memory cell and the affiliated node of the first memory cell；
Wherein, the first memory cell is the writing unit of the first data.
Retransmission unit 605 being determined, during for not being the destination node of data flow of the first data when this node, first is determined
The destination node of the data flow of data, destination node is sent to by the stream compression of the first data, by destination node by the first data
Data flow be sent to cache layer；
First transmitting element 606, during for being the destination node of data flow of the first data when this node, by the first data
Data flow be sent to cache layer.
In the present embodiment, whether after the data flow of the first data reaches forwarding, it is first to deposit first to judge this node
Node in the affiliated node group of storage unit, if this node is the node in the affiliated node group of the first memory cell, then then sentence again
Break this node whether be first data data flow destination node, if this node is not section belonging to the first memory cell
Node in point group, then directly determine the destination node of the data flow of the first data, is counted so as to realize by destination node by first
According to the cache layer that is sent to of data flow, it is ensured that the uniformity of data.
Embodiment six
The embodiment of the present invention six discloses a kind of data processing equipment, as shown in fig. 7, the device includes：First judges single
First 701, retransmission unit 702, the first transmitting element 703, the first mirror image unit 704 and the second mirror image unit 705 are determined；Its
In：
First judging unit 701, for whether after the data flow of the first data reaches forwarding, it to be institute to judge this node
State the destination node of the data flow of the first data；
Destination node includes the cooperative nodes of the affiliated node of the first memory cell and the affiliated node of the first memory cell；
Wherein, first memory cell is the writing unit of the first data.
Retransmission unit 702 being determined, during for not being the destination node of data flow of the first data when this node, first is determined
The destination node of the data flow of data, destination node is sent to by the stream compression of the first data, by destination node by described first
The data flow of data is sent to cache layer；
First transmitting element 703, during for being the destination node of data flow of the first data when this node, by the first data
Data flow be sent to cache layer；
First mirror image unit 704, for when this node is the affiliated node of the first memory cell, by the first data image
To the cooperative nodes of the affiliated node；
Second mirror image unit 705, for when cooperative nodes of this node for the affiliated node of the first memory cell, by the
Preposition node of one data image to this node；
Wherein, the preposition node of this node is the affiliated node of first memory cell.First memory cell it is affiliated
The cooperative nodes of node and the affiliated node of the first memory cell are two adjacent nodes, and the affiliated section of the first memory cell
Point is the preposition node of the cooperative nodes of the affiliated node of the first memory cell, when the affiliated section that this node is the first memory cell
During the cooperative nodes of point, after the preposition node of the first data image to this node, then equivalent to by the first data image to the
The affiliated node of one memory cell.
In the present embodiment, after data flow reaches cache layer, when the affiliated section that this node is first memory cell
During point, by the cooperative nodes of first data image to the affiliated node, when this node is first memory cell
During the cooperative nodes of affiliated node, by the preposition node of first data image to this node, so as to ensure when a node
During failure, it can be carried out by another node come data cached.Also, deposited because the first data are stored in first respectively
In the affiliated node of storage unit and the cooperative nodes of affiliated node, therefore, it is possible to ensure the affiliated section by the first memory cell
O'clock by the data of the first data distributing to the first memory cell be newest data.
The embodiment of each in this specification is described by the way of progressive, and what each embodiment was stressed is and other
Between the difference of embodiment, each embodiment identical similar portion mutually referring to.For device disclosed in embodiment
For, because it is corresponded to the method disclosed in Example, so description is fairly simple, related part is said referring to method part
It is bright.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or using the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The most wide scope caused.
Claims (10)
1. a kind of data processing method, it is characterised in that including：
When the first data data flow reach forwarding after, judge this node whether be first data data flow purpose
Node；The destination node includes the cooperation section of the affiliated node of the first memory cell and the affiliated node of the first memory cell
Point；Wherein, first memory cell is the writing unit of first data；
If not, determining the destination node of the data flow of first data, the stream compression of first data is sent to institute
Destination node is stated, the data flow of first data is sent to cache layer by the destination node；
If it is, the data flow of first data is sent into cache layer.
2. according to the method described in claim 1, it is characterised in that it is described judge this node whether be first data number
Before the destination node of stream, in addition to：
Whether judge this node is node in the affiliated node group of the first memory cell；
If it is, triggering is described to judge the step of whether this node is the destination node of data flow of first data；
If not, the step of destination node of triggering determination first data flow.
3. according to the method described in claim 1, it is characterised in that the data flow by first data is sent to caching
After layer, in addition to：
When this node is the affiliated node of first memory cell, by first data image to the affiliated node
Cooperative nodes；
When cooperative nodes of this node for the affiliated node of first memory cell, by first data image to this section
The preposition node of point；
Wherein, the preposition node of described node is the affiliated node of first memory cell.
4. method according to claim 1, it is characterised in that the destination node of determination first data flow includes：
Section for the purpose of the big node of data-handling capacity is chosen from the cooperative nodes of the affiliated node and the affiliated node
Point.
5. method according to claim 4, it is characterised in that the conjunction from the affiliated node and the affiliated node
It is purpose node to make to choose the big node of data-handling capacity in node, including：
The last node for carrying out data processing is determined from the cooperative nodes of the affiliated node and the affiliated node；
It is determined that except another node of the last node for carrying out data processing is purpose node.
6. a kind of data processing equipment, it is characterised in that including：
First judging unit, for whether after the data flow of the first data reaches forwarding, it to be described first to judge this node
The destination node of the data flow of data；Affiliated node and first memory cell of the destination node including the first memory cell
The cooperative nodes of affiliated node；Wherein, first memory cell is the writing unit of first data；
Retransmission unit being determined, during for not being the destination node of data flow of first data when this node, described is determined
The destination node of the data flow of one data, is sent to the destination node, by the purpose by the stream compression of first data
The data flow of first data is sent to cache layer by node；
First transmitting element, during for being the destination node of data flow of first data when this node, described first is counted
According to data flow be sent to cache layer.
7. device according to claim 6, it is characterised in that also include：
Second judging unit, for whether after the data flow of the first data reaches forwarding, it to be the first storage to judge this node
Node in the affiliated node group of unit；
First trigger element, for when this node is the node in the affiliated node group of the first memory cell, triggering described first
Judging unit perform judge this node whether be first data data flow destination node operation；
Second trigger element, it is described true for when this node is not the node in the affiliated node group of the first memory cell, triggering
Determine the operation that retransmission unit performs the destination node for determining first data flow.
8. device according to claim 6, it is characterised in that also include：
First mirror image unit, for when this node is the affiliated node of first memory cell, by the first data mirror
As the cooperative nodes to the affiliated node；
Second mirror image unit, will be described for when cooperative nodes of this node for the affiliated node of first memory cell
Preposition node of first data image to this node；
Wherein, the preposition node of described node is the affiliated node of first memory cell.
9. device according to claim 6, it is characterised in that the determination retransmission unit includes：
First determining module, during for not being the destination node of data flow of first data when this node, from described affiliated
It is purpose node that the big node of data-handling capacity is chosen in the cooperative nodes of node and the affiliated node；
First forwarding module, for the stream compression of first data to be sent into the destination node, by the destination node
The data flow of first data is sent to cache layer.
10. device according to claim 9, it is characterised in that first determining module is specifically for from described affiliated
The last node for carrying out data processing is determined in the cooperative nodes of node and the affiliated node, so that it is determined that except described upper one
Another node of the secondary node for carrying out data processing is purpose node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710386160.8A CN107102918B (en) | 2017-05-26 | 2017-05-26 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710386160.8A CN107102918B (en) | 2017-05-26 | 2017-05-26 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107102918A true CN107102918A (en) | 2017-08-29 |
| CN107102918B CN107102918B (en) | 2020-06-16 |
Family
ID=59670072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710386160.8A Active CN107102918B (en) | 2017-05-26 | 2017-05-26 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107102918B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109460187A (en) * | 2018-11-09 | 2019-03-12 | 郑州云海信息技术有限公司 | A kind of qcow2 file data consistency verification method and verifying terminal |
| CN110968453A (en) * | 2018-09-28 | 2020-04-07 | 杭州海康威视***技术有限公司 | Data storage method and device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101562543A (en) * | 2009-05-25 | 2009-10-21 | 阿里巴巴集团控股有限公司 | Cache data processing method and processing system and device thereof |
| CN102577278A (en) * | 2009-10-19 | 2012-07-11 | 国际商业机器公司 | Dynamic resource allocation for distributed cluster storage network |
| CN102594698A (en) * | 2012-03-12 | 2012-07-18 | 中国人民解放军总参谋部第六十三研究所 | DTN asynchronous routing algorithm based on node position projection |
| CN104008111A (en) * | 2013-02-27 | 2014-08-27 | 深圳市腾讯计算机***有限公司 | Data storage management method and device |
| US9069681B1 (en) * | 2013-03-20 | 2015-06-30 | Google Inc. | Real-time log joining on a continuous stream of events that are approximately ordered |
| WO2016012320A1 (en) * | 2014-07-22 | 2016-01-28 | Alcatel Lucent | Signaling for transmission of coherent data flow within packet-switched network |
| CN105577547A (en) * | 2016-01-18 | 2016-05-11 | 重庆邮电大学 | Multi-Qos based route selection method in mobile self-organizing network |
| CN105721398A (en) * | 2014-12-04 | 2016-06-29 | 深圳市金蝶中间件有限公司 | Data access method and device |
| CN106156328A (en) * | 2016-07-06 | 2016-11-23 | 中国银行股份有限公司 | A kind of bank's running log data monitoring method and system |
-
2017
- 2017-05-26 CN CN201710386160.8A patent/CN107102918B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101562543A (en) * | 2009-05-25 | 2009-10-21 | 阿里巴巴集团控股有限公司 | Cache data processing method and processing system and device thereof |
| CN102577278A (en) * | 2009-10-19 | 2012-07-11 | 国际商业机器公司 | Dynamic resource allocation for distributed cluster storage network |
| CN102594698A (en) * | 2012-03-12 | 2012-07-18 | 中国人民解放军总参谋部第六十三研究所 | DTN asynchronous routing algorithm based on node position projection |
| CN104008111A (en) * | 2013-02-27 | 2014-08-27 | 深圳市腾讯计算机***有限公司 | Data storage management method and device |
| US9069681B1 (en) * | 2013-03-20 | 2015-06-30 | Google Inc. | Real-time log joining on a continuous stream of events that are approximately ordered |
| WO2016012320A1 (en) * | 2014-07-22 | 2016-01-28 | Alcatel Lucent | Signaling for transmission of coherent data flow within packet-switched network |
| CN105721398A (en) * | 2014-12-04 | 2016-06-29 | 深圳市金蝶中间件有限公司 | Data access method and device |
| CN105577547A (en) * | 2016-01-18 | 2016-05-11 | 重庆邮电大学 | Multi-Qos based route selection method in mobile self-organizing network |
| CN106156328A (en) * | 2016-07-06 | 2016-11-23 | 中国银行股份有限公司 | A kind of bank's running log data monitoring method and system |
Non-Patent Citations (1)
| Title |
|---|
| 田秀霞等: "输入/输出流", 《C++程序设计教程》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110968453A (en) * | 2018-09-28 | 2020-04-07 | 杭州海康威视***技术有限公司 | Data storage method and device |
| CN109460187A (en) * | 2018-11-09 | 2019-03-12 | 郑州云海信息技术有限公司 | A kind of qcow2 file data consistency verification method and verifying terminal |
| CN109460187B (en) * | 2018-11-09 | 2022-05-03 | 郑州云海信息技术有限公司 | Verification method and verification terminal for data consistency of qcow2 file |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107102918B (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7478154B2 (en) | Storage system with link selection control | |
| JP4291664B2 (en) | Storage apparatus and system having communication buffer reservation function | |
| US8185663B2 (en) | Mirroring storage interface | |
| US7797393B2 (en) | Shared storage network system and a method for operating a shared storage network system | |
| CN103890729B (en) | computerized method, storage server and management server | |
| US6848007B1 (en) | System for mapping addresses of SCSI devices between plurality of SANs that can dynamically map SCSI device addresses across a SAN extender | |
| TWI307035B (en) | Method and system for backing up remote mirror data on internet | |
| CN103338243B (en) | The data cached update method and system of Web node | |
| US20060095705A1 (en) | Systems and methods for data storage management | |
| US7779368B2 (en) | Systems and methods for presenting managed data | |
| US7987154B2 (en) | System, a method and a device for updating a data set through a communication network | |
| US20060010299A1 (en) | Systems and methods to avoid deadlock and guarantee mirror consistency during online mirror synchronization and verification | |
| JP2002526821A (en) | Coordination of persistent status information between multiple file servers | |
| CN106446159B (en) | A kind of method of storage file, the first virtual machine and name node | |
| JP2008526109A (en) | Method and apparatus for network packet capture distributed storage system | |
| US20030023808A1 (en) | Method and system for maintaining data coherency in a dual input/output adapter utilizing clustered adapters | |
| WO2006026708A2 (en) | Multi-chassis, multi-path storage solutions in storage area networks | |
| CN108696581A (en) | Caching method, device, computer equipment and the storage medium of distributed information | |
| JPH05102972A (en) | Method and apparatus for selective message insertion | |
| US7092982B2 (en) | Storage system with method of minimizing redundant transmissions over a plurality of communication paths | |
| US7107355B2 (en) | High availability lightweight directory access protocol service | |
| CN107102918A (en) | A kind of data processing method and device | |
| CN108897644A (en) | A kind of dual controller fault handling method and system | |
| CN110471906A (en) | Database switching method, device and equipment | |
| WO2001035188A2 (en) | Method and system for mapping addressing of scsi devices between storage area networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200520 Address after: 215100 No. 1 Guanpu Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province Applicant after: SUZHOU LANGCHAO INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 450018 Henan province Zheng Dong New District of Zhengzhou City Xinyi Road No. 278 16 floor room 1601 Applicant before: ZHENGZHOU YUNHAI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |