CN115277735B - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115277735B CN115277735B CN202210857497.3A CN202210857497A CN115277735B CN 115277735 B CN115277735 B CN 115277735B CN 202210857497 A CN202210857497 A CN 202210857497A CN 115277735 B CN115277735 B CN 115277735B
- Authority
- CN
- China
- Prior art keywords
- node
- node set
- target
- service data
- request message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 53
- 230000004044 response Effects 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims description 29
- 238000013500 data storage Methods 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 13
- 238000004590 computer program Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to a data processing method and device, an electronic device and a storage medium, wherein the method comprises the following steps: and acquiring a target request message, wherein the target request message is used for requesting to process target service data, and assembling the data stored by each node in the target node set into the target service data in response to the target request message, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set. The method and the device solve the technical problems that the storage capacity of a single node is limited and a large amount of data storage is difficult to meet in the related technology.
Description
Technical Field
The present disclosure relates to the field of computers, and in particular, to a method and apparatus for processing data, an electronic device, and a storage medium.
Background
In the related art, as business progresses, more and more data is stored in an intermediate container (such as a ZooKeeper), but the container has a maximum capacity limit, and a large amount of data affects the performance of the intermediate container. As in the ZooKeeper, the default capacity of the node is 1M, but it is not recommended to store too much data, which would affect the overall performance of the ZooKeeper, and for the case of storing large capacity data, the actual data is stored in NFS (network file system ) or HDFS (distributed file system, hadoop Distributed File System) file systems by the link where the actual data is stored at the node.
However, the data stored in the ZooKeeper node has strong consistency, and the third party storage structure is used, so that the consistency between the ZooKeeper and the third party storage structure needs to be additionally ensured, and the third party storage needs to have the same read-write access capability as the ZooKeeper.
Therefore, the related art has a technical problem that a single node has a limited storage capacity, and a large amount of data is difficult to store.
BRIEF SUMMARY OF THE PRESENT DISCLOSURE
The disclosure provides a data processing method and device, electronic equipment and a storage medium, so as to at least solve the technical problems that a single node in the related art has limited storage capacity and is difficult to meet the technical problem of mass data storage. The technical scheme of the present disclosure is as follows:
according to a first aspect of an embodiment of the present disclosure, there is provided a data processing method, including: acquiring a target request message, wherein the target request message is used for requesting to process target service data; and responding to the target request message, assembling the data stored by each node in the target node set into the target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes: responding to the target request message, and acquiring node configuration information, wherein the node configuration information is used for determining a node set for storing the target service data; and determining the target node set according to the node configuration information, and assembling the data stored by each node in the target node set into the target service data.
In an exemplary embodiment, the determining, in response to the target request message, the target node set according to the node configuration information includes: under the condition that the target request message indicates that the target service data is requested to be read, a first node set is obtained according to the node configuration information, wherein the first node set comprises nodes in a working state at the current moment; determining the first node set as the target node set under the condition that the first node set is acquired; and if the first node set is not acquired, determining a second node set as the target node set, wherein the second node set comprises a node which is predetermined before the current moment and indicated by the node configuration information.
In an exemplary embodiment, after the first node set is acquired, the method further includes: comparing whether the first version number of the first node set is consistent with the second version number in the node configuration information; determining the first node set as the target node set if the first version number and the second version number are consistent; and in the case that the first version number and the second version number are inconsistent, determining the second node set as the target node set.
In an exemplary embodiment, the method further comprises: acquiring node update indication information, wherein the node update indication information is used for modifying the node number of a target node set in the node configuration information; and determining the number of nodes in the target node set according to the node update indication information.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes: under the condition that the target request message indicates that the target service data is requested to be updated, a third node set is obtained according to the node configuration information, wherein the third node set is a mirror image node set corresponding to a first node set at the current moment, and the first node set comprises nodes in a working state at the current moment; writing the service data to be updated into the third node set; updating the third node set to the first node set under the condition that the service data to be updated is written into the third node set, and reconfiguring the third node set according to the updated first node set; and assembling the data stored by each node in the updated first node set into the updated target service data.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes: under the condition that the target request message indicates that the target service data is requested to be updated, a fourth node set is obtained according to the node configuration information, wherein the fourth node set is a mirror image node set corresponding to a fifth node set at the current moment, the fifth node set comprises nodes in a working state at the current moment, and the nodes in the fifth node set are nodes predetermined before the current moment; writing the service data to be updated into the fourth node set; updating the fourth node set to the fifth node set under the condition that the service data to be updated is written into the fourth node set, and reconfiguring the fourth node set according to the updated fifth node set; and assembling the data stored by each node in the updated fifth node set into the updated target service data.
In an exemplary embodiment, after the obtaining the target request message, the method further includes: under the condition that the target request message carries a distributed lock, responding to the target request message, and splicing data stored by each node in a target node set into the target service data; and after the data stored by each node in the target node set are spliced into the target service data, releasing the distributed lock carried by the target request message.
According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus, including: the acquisition module is used for acquiring a target request message, wherein the target request message is used for requesting to process target service data; and the assembling module is used for responding to the target request message and assembling the data stored by each node in the target node set into the target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
In an exemplary embodiment, the assembly module includes: a first obtaining unit, configured to obtain node configuration information in response to the target request message, where the node configuration information is used to determine a node set storing the target service data; the first assembling unit is used for determining the target node set according to the node configuration information and assembling the data stored by each node in the target node set into the target service data.
In an exemplary embodiment, the first splice unit includes: the first acquisition subunit is configured to acquire a first node set according to the node configuration information when the target request message indicates that the target service data is requested to be read, where the first node set includes a node in a working state at the current moment; a first determining subunit, configured to determine, when the first node set is acquired, the first node set as the target node set; and a second determining subunit, configured to determine a second node set as the target node set if the first node set is not acquired, where the second node set includes a node predetermined before the current time indicated by the node configuration information.
In an exemplary embodiment, the first splice unit further includes: a comparing subunit, configured to compare whether a first version number of the first node set is consistent with a second version number in the node configuration information after the first node set is acquired; a third determining subunit, configured to determine, when the first version number and the second version number are identical, the first node set as the target node set; and a fourth determining subunit, configured to determine the second node set as the target node set if the first version number and the second version number are inconsistent.
In an exemplary embodiment, the first splice unit further includes: a second obtaining subunit, configured to obtain node update indication information, where the node update indication information is used to modify a node number of a target node set in the node configuration information; and the adjusting subunit is used for adjusting the number of the nodes in the target node set according to the node updating indication information.
In an exemplary embodiment, the assembly module includes: a second obtaining unit, configured to obtain a third node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the third node set is a mirror image node set corresponding to a first node set at a current time, and the first node set includes a node in a working state at the current time; a first writing unit, configured to write service data to be updated into the third node set; a first updating unit, configured to update the third node set to the first node set, and reconfigure the third node set according to the updated first node set, when the service data to be updated has been written into the third node set; and the second splicing unit is used for splicing the data stored by each node in the updated first node set into the updated target service data.
In an exemplary embodiment, the assembly module includes: a third obtaining unit, configured to obtain a fourth node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the fourth node set is a mirror image node set corresponding to a fifth node set at a current time, the fifth node set includes a node in a working state at the current time, and a node in the fifth node set is a node predetermined before the current time; a second writing unit, configured to write service data to be updated into the fourth node set; a second updating unit, configured to update the fourth node set to the fifth node set, and reconfigure the fourth node set according to the updated fifth node set, when the service data to be updated has been written into the fourth node set; and the third splicing unit is used for splicing the data stored by each node in the updated fifth node set into the updated target service data.
In an exemplary embodiment, the apparatus further comprises: the processing module is used for assembling the data stored by each node in the target node set into the target service data in response to the target request message under the condition that the target request message is determined to carry the distributed lock after the target request message is acquired; and the releasing module is used for releasing the distributed lock carried by the target request message after the data stored by each node in the target node set are spliced into the target service data.
According to yet another aspect of the embodiments of the present disclosure, there is provided an electronic device for processing data, including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement a method of processing data as described above.
According to yet another aspect of the embodiments of the present disclosure, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to perform the above-described data processing method when run.
According to yet another aspect of embodiments of the present disclosure, there is also provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the processing method of the data as above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: the method comprises the steps of obtaining a target request message, wherein the target request message is used for requesting to process target service data; and in response to the target request message, assembling the data stored by each node in the target node set into target service data, wherein each node in the target node set stores part of the target service data, the storage space occupied by the target service data is larger than the storage space of any node in the target node set, determining the target node set by utilizing the target request message, and assembling the data stored by each node into the target service data, so that the storage space occupied by the target service data is larger than the storage space of any node in the target node set, the utilization ratio of the node storage space is improved, and the technical problems that the storage capacity of a single node is limited and a large amount of data is difficult to store in the related art are solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.
FIG. 1 is a schematic illustration of an application environment for a method of processing data, according to an exemplary embodiment;
FIG. 2 is a flow chart illustrating a method of processing data according to an exemplary embodiment;
FIG. 3 is an application scenario diagram illustrating a method of processing data according to an exemplary embodiment;
FIG. 4 is a schematic diagram of node configuration information illustrating a method of processing data according to an example embodiment;
FIG. 5 is a flow chart illustrating a method of processing data according to an exemplary embodiment;
FIG. 6 is a flowchart illustrating another method of processing data according to an example embodiment;
FIG. 7 is a flowchart illustrating yet another method of processing data according to an exemplary embodiment;
FIG. 8 is a schematic diagram illustrating a method of processing data according to an example embodiment;
FIG. 9 is a block diagram of a data processing apparatus according to an exemplary embodiment;
fig. 10 is a block diagram of an electronic device, according to an example embodiment.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
First, partial terms or terminology appearing in the course of describing the embodiments of the application are applicable to the following explanation:
The ZooKeeper (distributed application coordination service software) is distributed, open source code distributed application coordination service, is software for providing consistency service for distributed application, and provides functions including: configuration maintenance, domain name service, distributed synchronization, group service and the like, the goal of the ZooKeeper is to package complex and error-prone key services, and provide a simple and easy-to-use interface and a system with high performance and stable function for users.
The invention is illustrated below with reference to examples:
according to an aspect of the embodiment of the present invention, there is provided a data processing method, and optionally, in this embodiment, the data processing method may be applied to a hardware environment configured by the server 101 and the terminal device 103 as shown in fig. 1. As shown in fig. 1, a server 101 is connected to a terminal 103 through a network, and may be used to provide services to a terminal device or a client installed on the terminal device, where the client may be a video client, an instant messaging client, a browser client, an educational client, a game client, etc. The database 105 may be provided on or independent of the server for providing data storage services for the server 101, such as an image data storage server, which may include, but is not limited to: a wired network, a wireless network, wherein the wired network comprises: local area networks, metropolitan area networks, and wide area networks, the wireless network comprising: bluetooth, WIFI and other wireless communication networks, the terminal device 103 may be a terminal configured with an application program, and may include, but is not limited to, at least one of the following: the mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a notebook computer, a tablet computer, a palm computer, an MID (Mobile Internet Devices, mobile internet device), a PAD, a desktop computer, a smart television, etc., where the server may be a single server, a server cluster formed by a plurality of servers, or a cloud server, and the application program 107 using the data processing method is displayed through a terminal device 103 or a client installed on the server 101.
As shown in fig. 1, the above data processing method may be implemented in the terminal device 103 by the following steps:
s1, acquiring a target request message on a terminal device 103, wherein the target request message is used for requesting to process target service data;
s2, in response to the target request message, assembling data stored by each node in the target node set into target service data on the terminal equipment 103, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
Alternatively, in the present embodiment, the above-described data processing method may be implemented on the server 101 in a similar manner, or may be implemented asynchronously by the server 101 and the terminal device 103.
The above is merely an example, and the present embodiment is not particularly limited.
Optionally, as an optional implementation manner, as shown in fig. 2, the method for processing the data includes:
s202, acquiring a target request message, wherein the target request message is used for requesting to process target service data;
s204, in response to the target request message, assembling the data stored by each node in the target node set into target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
Alternatively, in this embodiment, the target request message may include, but is not limited to, a read signal (read signal) used in a ZooKeeper, where the request message is used to request processing of target service data determined according to a current service requirement.
Alternatively, in the present embodiment, the above-described data processing method may be applied to fields including, but not limited to, cloud computing, and specifically, may include, but not limited to, cloud storage.
Among them, cloud computing (cloud computing) is a computing mode that distributes computing tasks over a resource pool formed by a large number of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed.
As a basic capability provider of cloud computing, a cloud computing resource pool (cloud platform for short, generally referred to as IaaS (Infrastructure as a Service, infrastructure as a service) platform) is established, in which multiple types of virtual resources are deployed for external clients to select for use.
According to the logic function division, a PaaS (Platform as a Service ) layer can be deployed on an IaaS (Infrastructure as a Service ) layer, and a SaaS (Software as aService, software as a service) layer can be deployed above the PaaS layer, or the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, web container, etc. SaaS is a wide variety of business software such as web portals, sms mass senders, etc. Generally, saaS and PaaS are upper layers relative to IaaS.
Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside.
At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage position information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage position information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of Independent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.
Optionally, in this embodiment, the nodes in the target node set may include, but are not limited to, znode nodes (i.e., data nodes in a ZooKeeper), where each node in the target node set stores a part of target service data, which may be understood that, by storing a part of target service data in each node in the node set, respectively, and then assembling each part of target service data into complete target service data by a splicing manner, for subsequent use, where a storage space occupied by the target service data is larger than a storage space occupied by any node in the target node set may be understood that a storage space occupied by the target service data cannot be stored in any single node in the target node set, and at least two nodes need to be used for storage, and matching with subsequent data splicing to implement use of the target service data.
Fig. 3 is an application scenario schematic diagram of a data processing method according to an exemplary embodiment, and as shown in fig. 3, the application scenario schematic diagram includes a plurality of Znode nodes, where the Znode nodes may be temporary nodes or persistent nodes, the plurality of Znode nodes are nodes in the target node set, each Znode node is configured to store a portion of target service data, and when a target request message is acquired, the portion of service data stored in the plurality of Znode nodes is spliced to obtain the target service data, and in a ZooKeeper, the nodes are called Znode and Znode nodes are classified into two types: 1. persistent node, 2, temporary node.
The "/Java,/Java/zk,/Java/mq" shown in fig. 3 are all paths for acquiring the Znode node, that is, the above paths allow the storage of the partial target service data in the corresponding node.
The foregoing is merely an example, and embodiments of the present application are not limited in any way.
Through the embodiment, a target request message is acquired, wherein the target request message is used for requesting to process target service data; and in response to the target request message, assembling the data stored by each node in the target node set into target service data, wherein each node in the target node set stores part of the target service data, the storage space occupied by the target service data is larger than the storage space of any node in the target node set, determining the target node set by utilizing the target request message, and assembling the data stored by each node into the target service data, so that the storage space occupied by the target service data is larger than the storage space of any node in the target node set, the utilization ratio of the node storage space is improved, and the technical problems that the storage capacity of a single node is limited and a large amount of data is difficult to store in the related art are solved.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes:
responding to the target request message, and acquiring node configuration information, wherein the node configuration information is used for determining a node set for storing the target service data;
and determining the target node set according to the node configuration information, and assembling the data stored by each node in the target node set into the target service data.
Alternatively, in this embodiment, the node configuration information may include, but is not limited to, information for indicating the order of magnitude of the service data, information for indicating a node currently operating, information of a candidate node, information of a spam node, and information of a spam candidate node.
For example, fig. 4 is a schematic diagram of node configuration information of a data processing method according to an exemplary embodiment, and as shown in fig. 4, includes information of "+size", "+wordzknodebist", "+candidatezknodebist", "+backup work zknodebist", etc. to indicate information of the order of magnitude of the above-mentioned service data, information indicating a node currently in operation, information of a candidate node, information of a bottom-of-pocket node, and information of a bottom-of-pocket candidate node, etc.
The foregoing is merely an example, and embodiments of the present application are not limited in any way.
According to the method and the device for determining the node configuration information according to the target request message, the node set to be used can be determined according to the node configuration information, the utilization rate of the node storage space is improved, and the technical problem that the storage capacity of a single node is limited and a large amount of data is difficult to store in the related technology is solved.
In an exemplary embodiment, the determining, in response to the target request message, the target node set according to the node configuration information includes:
under the condition that the target request message indicates that the target service data is requested to be read, a first node set is obtained according to the node configuration information, wherein the first node set comprises nodes in a working state at the current moment;
determining the first node set as the target node set under the condition that the first node set is acquired;
and if the first node set is not acquired, determining a second node set as the target node set, wherein the second node set comprises a node which is predetermined before the current moment and indicated by the node configuration information.
Optionally, in this embodiment, the target request message indicates that the request for reading the target service data may be understood that the target request message is a read signal, where the read signal is used to read the target service data, after receiving the read signal, node configuration information may be read first, a first node set in a working state at the current moment is obtained from the node configuration information first, when the first node set is obtained successfully, the part of the target service data is read from the first node set, and finally, the read service data is assembled into complete target service data for downstream use.
Optionally, in this embodiment, when the first node set in the working state at the current moment is first obtained from the node configuration information, when the first node set fails to be obtained, a second node set is obtained, where the second node set is a bottom node set indicated by the node configuration information, the node set is a node set predetermined before the current moment, service data stored in the second node set is the same as service data stored in the first node set that is working currently, at this time, the partial target service data are respectively read from the second node set, and finally, the read service data are assembled into complete target service data for downstream use.
For example, fig. 5 is a schematic diagram illustrating a method for processing data according to an exemplary embodiment, as shown in fig. 5, may include, but is not limited to, the following steps:
s1, acquiring a read signal (corresponding to the target request message);
s2, zk configuration information reading (corresponding to the node configuration information);
s3, acquiring a zk node set (corresponding to the first node set) which is working;
s4, under the condition that the acquisition is successful, reading the data in the node set, and assembling the data into complete data;
s5, under the condition of acquisition failure, acquiring a spam zk node set (corresponding to the second node set), reading data in the spam zk node set, and assembling the data into complete data.
The above is merely an example, and the present embodiment is not limited in any way.
In an exemplary embodiment, after the first set of nodes is acquired, the method further includes:
comparing whether the first version number of the first node set is consistent with the second version number in the node configuration information;
determining the first node set as the target node set if the first version number and the second version number are consistent;
And in the case that the first version number and the second version number are inconsistent, determining the second node set as the target node set.
Optionally, in this embodiment, comparing whether the version number of the node set is consistent with the version number recorded in the node configuration information according to the acquired first node set that is working may include, but is not limited to:
1) If the data in each node are consistent, reading the data in each node, and finally assembling the data into complete data for business use;
2) If the nodes are inconsistent, acquiring the bottom zk nodes, and assembling the nodes into complete data for business use.
In an exemplary embodiment, the method further comprises:
acquiring node update indication information, wherein the node update indication information is used for modifying the node number of a target node set in the node configuration information;
and adjusting the number of nodes in the target node set according to the node update indication information.
Optionally, in this embodiment, the node update indication information may include, but is not limited to, indicating the number of nodes of the target node set in the modified node configuration information, where a preset number of extended nodes are preconfigured, and when the load of the target node set at the current time meets a preset condition, the node update indication information is obtained to indicate a client of the listening node to modify the node configuration information, and adjust the number of nodes currently working.
According to the embodiment, the pre-configured expansion nodes are adopted, the architecture realized by the plurality of nodes of the ZooKeeper is utilized, the horizontal expansion and dynamic expansion of the number of the nodes can be supported, and the horizontal expansion can be realized only by modifying the number of the nodes of the target node set configured in the node configuration information, so that the limit of the storage capacity of a single node is solved.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes:
under the condition that the target request message indicates that the target service data is requested to be updated, a third node set is obtained according to the node configuration information, wherein the third node set is a mirror image node set corresponding to a first node set at the current moment, and the first node set comprises nodes in a working state at the current moment;
writing the service data to be updated into the third node set;
updating the third node set to the first node set under the condition that the service data to be updated is written into the third node set, and reconfiguring the third node set according to the updated first node set;
And assembling the data stored by each node in the updated first node set into the updated target service data.
Optionally, in this embodiment, the target request message may include, but is not limited to, a concurrent update signal, and the third node set may be understood as a preconfigured candidate node set, where the node configuration information records the candidate node set, where the candidate node set is a mirror node set of the first node set, and a copy-on-write (copy-on-write) manner is used to ensure that, when a new node set does not entirely replace the first node set that is in operation, the read configuration is the same version of data, so that situations that read data is unavailable and data versions are inconsistent are avoided. The copyOnWrite idea is realized by redundancy of a candidate node set (corresponding to the third node set), and when writing data, the configuration information of the rewritten node is written successfully on the candidate node set, so that when writing the data of the candidate node set, the reading and writing of normal working data are not affected, and the data formed by each data node set is ensured to be a snapshot of complete data.
For example, fig. 6 is a schematic diagram illustrating another data processing method according to an exemplary embodiment, as shown in fig. 6, which may include, but is not limited to, the following steps:
s1, acquiring an update signal (corresponding to the target request message);
s2, obtaining signals of a redis lock through a redis distributed lock module;
s3, zk configuration information reading (corresponding to the node configuration information);
s4, acquiring a candidate zk node set (corresponding to the third node set);
s5, under the condition that the integration work of the candidate zk nodes is obtained, filling the candidate zk nodes with data to be filled;
s6, updating node configuration information, changing the candidate zk node set into a working zk node set, and changing the working zk node set into the candidate zk node set;
s7, releasing the redislock under the condition that the acquisition of the candidate zk node set fails or the updating of the node configuration information is completed.
The above is merely an example, and the present embodiment is not limited in any way.
In an exemplary embodiment, the assembling, in response to the target request message, data stored by each node in the target node set into the target service data includes:
under the condition that the target request message indicates that the target service data is requested to be updated, a fourth node set is obtained according to the node configuration information, wherein the fourth node set is a mirror image node set corresponding to a fifth node set at the current moment, the fifth node set comprises nodes in a working state at the current moment, and the nodes in the fifth node set are nodes predetermined before the current moment;
Writing the service data to be updated into the fourth node set;
updating the fourth node set to the fifth node set under the condition that the service data to be updated is written into the fourth node set, and reconfiguring the fourth node set according to the updated fifth node set;
and assembling the data stored by each node in the updated fifth node set into the updated target service data.
Optionally, in this embodiment, the fifth node set is a node set predetermined before the current time, which may be understood as a spam node set, that is, a node storing the target service data currently being used at a preset time, so as to maintain the storage of the normal data in the case that the first node set cannot be used.
Optionally, in this embodiment, the fourth node set is a mirror image set of the fifth node set, that is, when the fifth node set performs service processing, a backup is redundant, which is used as a candidate node set of the fifth node set.
For example, fig. 7 is a schematic diagram illustrating yet another data processing method according to an exemplary embodiment, as shown in fig. 7, may include, but is not limited to, the following steps:
S1, acquiring a spam task signal (a spam task corresponding to the target request message);
s2, obtaining signals of a redis lock through a redis distributed lock module;
s3, zk configuration information reading (corresponding to the node configuration information);
s4, acquiring a candidate spam zk node set (corresponding to the fourth node set);
s5, under the condition that the candidate spam zk node set is successfully acquired, filling the candidate spam zk nodes with data to be filled;
s6, updating node configuration information, changing the candidate bottom zk node set into a working bottom zk node set, and changing the working bottom zk node set into a candidate bottom zk node set;
s7, releasing the redislock under the condition that the acquisition of the candidate bottom zk node set fails or the updating of the node configuration information is completed.
In an exemplary embodiment, after the obtaining the target request message, the method further includes:
under the condition that the target request message carries a distributed lock, responding to the target request message, and splicing data stored by each node in a target node set into the target service data;
and after the data stored by each node in the target node set are spliced into the target service data, releasing the distributed lock carried by the target request message.
FIG. 8 is a schematic diagram illustrating a method of processing data according to an exemplary embodiment, as shown in FIG. 8, a Redis distributed lock includes:
the setNx syntax of the timeout time is used. setNx (set if not exit) is that the value is set only when the redis key is not present, and only one can be set successfully when a value is set simultaneously by multiple threads, thus ensuring the uniqueness of the lock. And the timeout time is set, so that the redis lock can be released under the abnormal condition that the timeout is not released, and the robustness of the redis lock is optimized.
Fig. 9 is a block diagram of a data processing apparatus according to an exemplary embodiment. Referring to fig. 8, the apparatus includes an acquisition module 902, and a splicing module 904.
The acquiring module 902 is configured to acquire a target request message, where the target request message is used for requesting to process target service data;
the assembling module 904 is configured to assemble, in response to the target request message, data stored by each node in a target node set into the target service data, where each node in the target node set stores a portion of the target service data, and a storage space occupied by the target service data is greater than a storage space of any node in the target node set.
In an exemplary embodiment, the assembly module includes:
a first obtaining unit, configured to obtain node configuration information in response to the target request message, where the node configuration information is used to determine a node set storing the target service data;
the first assembling unit is used for determining the target node set according to the node configuration information and assembling the data stored by each node in the target node set into the target service data.
In an exemplary embodiment, the first splice unit includes:
the first acquisition subunit is configured to acquire a first node set according to the node configuration information when the target request message indicates that the target service data is requested to be read, where the first node set includes a node in a working state at the current moment;
a first determining subunit, configured to determine, when the first node set is acquired, the first node set as the target node set;
and a second determining subunit, configured to determine a second node set as the target node set if the first node set is not acquired, where the second node set includes a node predetermined before the current time indicated by the node configuration information.
In an exemplary embodiment, the first splice unit further includes:
a comparing subunit, configured to compare whether a first version number of the first node set is consistent with a second version number in the node configuration information after the first node set is acquired;
a third determining subunit, configured to determine, when the first version number and the second version number are identical, the first node set as the target node set;
and a fourth determining subunit, configured to determine the second node set as the target node set if the first version number and the second version number are inconsistent.
In an exemplary embodiment, the first splice unit further includes:
a second obtaining subunit, configured to obtain node update indication information, where the node update indication information is used to modify a node number of a target node set in the node configuration information;
and the adjusting subunit is used for adjusting the number of the nodes in the target node set according to the node updating indication information.
In an exemplary embodiment, the assembly module includes:
a second obtaining unit, configured to obtain a third node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the third node set is a mirror image node set corresponding to a first node set at a current time, and the first node set includes a node in a working state at the current time;
A first writing unit, configured to write service data to be updated into the third node set;
a first updating unit, configured to update the third node set to the first node set, and reconfigure the third node set according to the updated first node set, when the service data to be updated has been written into the third node set;
and the second splicing unit is used for splicing the data stored by each node in the updated first node set into the updated target service data.
In an exemplary embodiment, the assembly module includes:
a third obtaining unit, configured to obtain a fourth node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the fourth node set is a mirror image node set corresponding to a fifth node set at a current time, the fifth node set includes a node in a working state at the current time, and a node in the fifth node set is a node predetermined before the current time;
a second writing unit, configured to write service data to be updated into the fourth node set;
A second updating unit, configured to update the fourth node set to the fifth node set, and reconfigure the fourth node set according to the updated fifth node set, when the service data to be updated has been written into the fourth node set;
and the third splicing unit is used for splicing the data stored by each node in the updated fifth node set into the updated target service data.
In an exemplary embodiment, the apparatus further comprises:
the processing module is used for assembling the data stored by each node in the target node set into the target service data in response to the target request message under the condition that the target request message is determined to carry the distributed lock after the target request message is acquired;
and the releasing module is used for releasing the distributed lock carried by the target request message after the data stored by each node in the target node set are spliced into the target service data.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
According to still another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the above-mentioned data processing method, where the electronic device may be a terminal device or a server as shown in fig. 1. The present embodiment is described taking the electronic device as a server as an example. As shown in fig. 10, the electronic device comprises a memory 1002 and a processor 1004, the memory 1002 having stored therein a computer program, the processor 1004 being arranged to perform the steps of any of the method embodiments described above by means of the computer program.
Alternatively, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of the computer network.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s1, acquiring a target request message, wherein the target request message is used for requesting to process target service data;
s2, in response to the target request message, assembling the data stored by each node in the target node set into target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
Optionally, in this embodiment, the above processor may be further configured to execute the following steps by a computer program:
s1, acquiring a target request message, wherein the target request message is used for requesting to process target service data;
s2, in response to the target request message, assembling the data stored by each node in the target node set into target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set.
Alternatively, as will be appreciated by those skilled in the art, the structure shown in fig. 10 is merely illustrative, and the electronic device may be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palmtop computer, a mobile internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig. 10 is not limited to the structure of the electronic device and the electronic apparatus described above. For example, the electronics may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.
The memory 1002 may be configured to store software programs and modules, such as program instructions/modules corresponding to the data processing methods and apparatuses in the embodiments of the present invention, and the processor 1004 executes the software programs and modules stored in the memory 1002 to perform various functional applications and data processing, that is, implement the data processing methods described above. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory 1002 may further include memory located remotely from the processor 1004, which may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 1002 may be used for storing information such as, but not limited to, historical user data. As an example, as shown in fig. 10, the memory 1002 may include, but is not limited to, an acquisition module 902 and a splicing module 904 in a processing device including the data. In addition, other module units in the data processing apparatus may be included, but are not limited to, and are not described in detail in this example.
Optionally, the transmission device 1006 is configured to receive or transmit data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission means 1006 includes a network adapter (Network Interface Controller, NIC) that can be connected to other network devices and routers via a network cable to communicate with the internet or a local area network. In one example, the transmission device 1006 is a Radio Frequency (RF) module for communicating with the internet wirelessly.
In addition, the electronic device further includes: a display 1008 for displaying the target service data; and a connection bus 1010 for connecting the respective module parts in the above-described electronic apparatus.
In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the plurality of nodes through a network communication. Among them, the nodes may form a Peer-To-Peer (P2P) network, and any type of computing device, such as a server, a terminal, etc., may become a node in the blockchain system by joining the Peer-To-Peer network.
In an exemplary embodiment, a computer program product is also provided, comprising a computer program/instruction which, when executed by a processor, implements the above-described information transmission method.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (16)
1. A method of processing data, comprising:
acquiring a target request message, wherein the target request message is used for requesting to process target service data;
Responding to the target request message, assembling the data stored by each node in a target node set into the target service data, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set;
the responding to the target request message assembles the data stored by each node in the target node set into the target service data, and comprises the following steps: under the condition that the target request message indicates that the target service data is requested to be updated, a third node set is obtained according to node configuration information, wherein the third node set is a mirror image node set corresponding to a first node set at the current moment, the node configuration information records candidate node sets of which the third node set is the first node set, and the first node set comprises nodes in a working state at the current moment; writing the service data to be updated into the third node set; updating the node configuration information under the condition that the service data to be updated is written into the third node set, updating the third node set into the first node set, and reconfiguring the third node set according to the updated first node set; and assembling the data stored by each node in the updated first node set into the updated target service data.
2. The method according to claim 1, wherein assembling the data stored by each node in the target node set into the target service data in response to the target request message comprises:
responding to the target request message, and acquiring node configuration information, wherein the node configuration information is used for determining a node set for storing the target service data;
and determining the target node set according to the node configuration information, and assembling the data stored by each node in the target node set into the target service data.
3. The method of claim 2, wherein said determining said set of target nodes from said node configuration information in response to said target request message comprises:
under the condition that the target request message indicates that the target service data is requested to be read, a first node set is obtained according to the node configuration information, wherein the first node set comprises nodes in a working state at the current moment;
determining the first node set as the target node set under the condition that the first node set is acquired;
and if the first node set is not acquired, determining a second node set as the target node set, wherein the second node set comprises a node which is predetermined before the current moment and indicated by the node configuration information.
4. The method of claim 3, wherein after the first set of nodes is acquired, the method further comprises:
comparing whether the first version number of the first node set is consistent with the second version number in the node configuration information;
determining the first node set as the target node set if the first version number and the second version number are consistent;
and in the case that the first version number and the second version number are inconsistent, determining the second node set as the target node set.
5. The method according to claim 2, wherein the method further comprises:
acquiring node update indication information, wherein the node update indication information is used for modifying the node number of a target node set in the node configuration information;
and determining the number of nodes in the target node set according to the node update indication information.
6. The method according to claim 1, wherein assembling the data stored by each node in the target node set into the target service data in response to the target request message comprises:
Under the condition that the target request message indicates that the target service data is requested to be updated, a fourth node set is obtained according to the node configuration information, wherein the fourth node set is a mirror image node set corresponding to a fifth node set at the current moment, the fifth node set comprises nodes in a working state at the current moment, and the nodes in the fifth node set are nodes predetermined before the current moment;
writing the service data to be updated into the fourth node set;
updating the fourth node set to the fifth node set under the condition that the service data to be updated is written into the fourth node set, and reconfiguring the fourth node set according to the updated fifth node set;
and assembling the data stored by each node in the updated fifth node set into the updated target service data.
7. The method according to any one of claims 1 to 6, wherein after the obtaining the target request message, the method further comprises:
under the condition that the target request message carries a distributed lock, responding to the target request message, and splicing data stored by each node in a target node set into the target service data;
And after the data stored by each node in the target node set are spliced into the target service data, releasing the distributed lock carried by the target request message.
8. A data processing apparatus, comprising:
the acquisition module is used for acquiring a target request message, wherein the target request message is used for requesting to process target service data;
the splicing module is used for splicing the data stored by each node in the target node set into the target service data in response to the target request message, wherein each node in the target node set stores part of the target service data, and the storage space occupied by the target service data is larger than the storage space of any node in the target node set;
a second obtaining unit, configured to obtain a third node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the third node set is a mirror image node set corresponding to a first node set at a current time, and the first node set includes a node in a working state at the current time; a first writing unit, configured to write service data to be updated into the third node set; a first updating unit, configured to update the third node set to the first node set, and reconfigure the third node set according to the updated first node set, when the service data to be updated has been written into the third node set; and the second splicing unit is used for splicing the data stored by each node in the updated first node set into the updated target service data.
9. The apparatus of claim 8, wherein the make-up module comprises:
a first obtaining unit, configured to obtain node configuration information in response to the target request message, where the node configuration information is used to determine a node set storing the target service data;
the first assembling unit is used for determining the target node set according to the node configuration information and assembling the data stored by each node in the target node set into the target service data.
10. The apparatus of claim 9, wherein the first splicing unit comprises:
the first acquisition subunit is configured to acquire a first node set according to the node configuration information when the target request message indicates that the target service data is requested to be read, where the first node set includes a node in a working state at the current moment;
a first determining subunit, configured to determine, when the first node set is acquired, the first node set as the target node set;
and a second determining subunit, configured to determine a second node set as the target node set if the first node set is not acquired, where the second node set includes a node predetermined before the current time indicated by the node configuration information.
11. The apparatus of claim 10, wherein the first splicing unit further comprises:
a comparing subunit, configured to compare whether a first version number of the first node set is consistent with a second version number in the node configuration information after the first node set is acquired;
a third determining subunit, configured to determine, when the first version number and the second version number are identical, the first node set as the target node set;
and a fourth determining subunit, configured to determine the second node set as the target node set if the first version number and the second version number are inconsistent.
12. The apparatus of claim 9, wherein the first splicing unit further comprises:
a second obtaining subunit, configured to obtain node update indication information, where the node update indication information is used to modify a node number of a target node set in the node configuration information;
and the adjusting subunit is used for adjusting the number of the nodes in the target node set according to the node updating indication information.
13. The apparatus of claim 8, wherein the make-up module comprises:
A third obtaining unit, configured to obtain a fourth node set according to the node configuration information when the target request message indicates that updating of the target service data is requested, where the fourth node set is a mirror image node set corresponding to a fifth node set at a current time, the fifth node set includes a node in a working state at the current time, and a node in the fifth node set is a node predetermined before the current time;
a second writing unit, configured to write service data to be updated into the fourth node set;
a second updating unit, configured to update the fourth node set to the fifth node set, and reconfigure the fourth node set according to the updated fifth node set, when the service data to be updated has been written into the fourth node set;
and the third splicing unit is used for splicing the data stored by each node in the updated fifth node set into the updated target service data.
14. The apparatus according to any one of claims 8 to 13, further comprising:
the processing module is used for assembling the data stored by each node in the target node set into the target service data in response to the target request message under the condition that the target request message is determined to carry the distributed lock after the target request message is acquired;
And the releasing module is used for releasing the distributed lock carried by the target request message after the data stored by each node in the target node set are spliced into the target service data.
15. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of processing data as claimed in any one of claims 1 to 7.
16. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of processing data according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210857497.3A CN115277735B (en) | 2022-07-20 | 2022-07-20 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210857497.3A CN115277735B (en) | 2022-07-20 | 2022-07-20 | Data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115277735A CN115277735A (en) | 2022-11-01 |
| CN115277735B true CN115277735B (en) | 2023-11-28 |
Family
ID=83766655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210857497.3A Active CN115277735B (en) | 2022-07-20 | 2022-07-20 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115277735B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110474963A (en) * | 2019-07-17 | 2019-11-19 | 中国平安人寿保险股份有限公司 | A kind of resource access method based on zookeeper, system, medium and electronic equipment |
| CN111143382A (en) * | 2019-12-13 | 2020-05-12 | 新华三大数据技术有限公司 | Data processing method, system and computer readable storage medium |
| CN111212123A (en) * | 2019-12-26 | 2020-05-29 | 天津中科曙光存储科技有限公司 | ZooKeeper-based Lunmap management method |
| CN111737018A (en) * | 2020-08-26 | 2020-10-02 | 腾讯科技（深圳）有限公司 | ZooKeeper configuration file storage processing method, device, equipment and medium thereof |
| CN112162846A (en) * | 2020-11-27 | 2021-01-01 | 腾讯科技（深圳）有限公司 | Transaction processing method, device and computer readable storage medium |
| CN113297166A (en) * | 2020-07-27 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing system, method and device |
| CN113448938A (en) * | 2021-07-20 | 2021-09-28 | 恒安嘉新（北京）科技股份公司 | Data processing method and device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8108612B2 (en) * | 2009-05-15 | 2012-01-31 | Microsoft Corporation | Location updates for a distributed data store |
| WO2015100627A1 (en) * | 2013-12-31 | 2015-07-09 | 华为技术有限公司 | Data processing method and device in distributed file storage system |
-
2022
- 2022-07-20 CN CN202210857497.3A patent/CN115277735B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110474963A (en) * | 2019-07-17 | 2019-11-19 | 中国平安人寿保险股份有限公司 | A kind of resource access method based on zookeeper, system, medium and electronic equipment |
| CN111143382A (en) * | 2019-12-13 | 2020-05-12 | 新华三大数据技术有限公司 | Data processing method, system and computer readable storage medium |
| CN111212123A (en) * | 2019-12-26 | 2020-05-29 | 天津中科曙光存储科技有限公司 | ZooKeeper-based Lunmap management method |
| CN113297166A (en) * | 2020-07-27 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing system, method and device |
| CN111737018A (en) * | 2020-08-26 | 2020-10-02 | 腾讯科技（深圳）有限公司 | ZooKeeper configuration file storage processing method, device, equipment and medium thereof |
| CN112162846A (en) * | 2020-11-27 | 2021-01-01 | 腾讯科技（深圳）有限公司 | Transaction processing method, device and computer readable storage medium |
| CN113448938A (en) * | 2021-07-20 | 2021-09-28 | 恒安嘉新（北京）科技股份公司 | Data processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115277735A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9667750B2 (en) | Client-initiated leader election in distributed client-server systems | |
| US9413604B2 (en) | Instance host configuration | |
| CN102088490B (en) | Data storage method, device and system | |
| US20180020077A1 (en) | Live migration of containers based on geo-location | |
| US8966025B2 (en) | Instance configuration on remote platforms | |
| CN113783922A (en) | Load balancing method, system and device | |
| US20220083229A1 (en) | Data storage system with configurable durability | |
| CN107172214B (en) | Service node discovery method and device with load balancing function | |
| CN109151028A (en) | A kind of distributed memory system disaster recovery method and device | |
| CN110413685A (en) | Database service switching method, device, readable storage medium storing program for executing and computer equipment | |
| US8543680B2 (en) | Migrating device management between object managers | |
| CN113553178A (en) | Task processing method and device and electronic equipment | |
| CN111641521A (en) | Ambari-based method for deploying big data clusters on cloud | |
| CN111431730A (en) | Service processing method, system, computer equipment and readable medium | |
| CN111510480A (en) | Request sending method and device and first server | |
| CN109254873B (en) | Data backup method, related device and system | |
| US10684801B1 (en) | Data storage system with separate interfaces for bulk data ingestion and data access | |
| CN111338834A (en) | Data storage method and device | |
| CN115277735B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110333824B (en) | Capacity expansion method and device for storage system | |
| CN107491551B (en) | Method and device for realizing client access of distributed file system | |
| US11336723B1 (en) | Replicating data volume updates from clients accessing the data volume across fault tolerance zones | |
| CN108733805B (en) | File interaction method, system, computer equipment and storage medium | |
| CN113157392A (en) | High-availability method and equipment for mirror image warehouse | |
| EP3387533B1 (en) | Disaster recovery of cloud resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |