CN111506601A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN111506601A CN111506601A CN202010246306.0A CN202010246306A CN111506601A CN 111506601 A CN111506601 A CN 111506601A CN 202010246306 A CN202010246306 A CN 202010246306A CN 111506601 A CN111506601 A CN 111506601A
- Authority
- CN
- China
- Prior art keywords
- interface
- command
- rbd
- instruction
- standard function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computer And Data Communications (AREA)
- Stored Programmes (AREA)
Abstract
The application relates to the technical field of data storage, in particular to a data processing method and a data processing device, which are applied to an RBD client, wherein the RBD client at least comprises a command agent module, a packaged standard function and a command interface library, and the method comprises the following steps: the command agent module receives an interface calling instruction issued by an application system; the command agent module determines a first interface required to be called by an application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the standard function and command interface library encapsulated based on the first interface, wherein the standard function and command interface library encapsulated is developed by self-definition based on an application scene of an RBD client; and the command agent module calls the second interface to send a corresponding command message to the backend storage cluster.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a data processing method and apparatus.
Background
Ceph is an open-source software-defined storage that, after years of development, is now supported and widely used by numerous cloud computing vendors. Both RedHat and OpenStack may be integrated with Ceph to support backend storage of virtual machine images.
RBD (RADOS Block device) is a Block storage component outward by Ceph, and is one of three large storage service components (blocks, files and objects). The RBD is a block storage back-end interface of services such as virtual machines, containers, mirror images, cloud disks and the like in IaaS cloud computing environments such as Openstack, kubernets and the like, and is a de facto block device interface standard in cloud computing.
The open-source RBD interface can be conveniently in butt joint with the cloud application, the cloud application does not need to modify codes, and the block storage service can be used by directly using the open-source RBD interface. At present, the open-source RBD interface is usually used directly, but there are some problems in using the open-source RBD interface: because the metadata of the RBD is stored at the rear end of the ceph cluster and is stored together with the data, when a large number of RBD volumes exist in the storage system, if the system IO is busy, the query operation on the RBD is very slow, reaches the time delay of several seconds and tens of seconds, even leads to query timeout, and seriously influences the user experience and use; the open source RBD interface is a product that has not been subjected to a complete productization test, and thus, directly using the open source RBD interface may have a reliability problem for a production environment with insufficient reliability.
Disclosure of Invention
The embodiment of the application provides a data processing method and device, which are used for solving the problems of low query management efficiency of an open source RBD client and low user experience degree in the prior art.
The embodiment of the application provides the following specific technical scheme:
in a first aspect, the present application provides a data processing method, which is applied to an RBD client, where the RBD client at least includes a command agent module, and an encapsulated standard function and command interface library, where the method includes:
the command agent module receives an interface calling instruction issued by an application system;
the command agent module determines a first interface required to be called by an application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the standard function and command interface library encapsulated based on the first interface, wherein the standard function and command interface library encapsulated is developed by self-definition based on an application scene of an RBD client;
and the command agent module calls the second interface to send a corresponding command message to the backend storage cluster.
Optionally, the RBD client further includes a data caching module, and the method further includes:
and the command agent module stores the operation record related to the interface calling instruction to the data caching module.
Optionally, the data caching module backs up metadata of each RBD interface; the method further comprises the following steps:
the command agent module receives a query instruction issued by an application system;
judging whether the query instruction is an RBD metadata query instruction or not;
and if so, the command agent module acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
Optionally, the data caching module is a relational database.
Optionally, the RBD client further includes a standard function and a command interface library, where the interface call instruction is an Rsetful interface class instruction, a command line call instruction or a standard function call class instruction;
if the interface calling instruction is an Rsetful interface class instruction, the step of determining, by the command agent module, a first interface to be called by an application system based on the interface calling instruction, and determining, based on the first interface, an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library includes:
the command agent module determines a target Rsetful interface required to be called by an application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library;
if the interface calling instruction is a command line calling or standard function calling instruction, the step of determining, by the command proxy module, a first interface to be called by an application system based on the interface calling instruction, and determining, based on the first interface, a packaged second interface corresponding to the first interface from the packaged standard function and command interface library includes:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
In a second aspect, the present application provides a data processing apparatus, which is applied to an RBD client, where the RBD client includes at least a command agent module, and an encapsulated standard function and command interface library, where the apparatus includes:
the receiving unit is used for receiving an interface calling instruction issued by an application system;
the determining unit is used for determining a first interface required to be called by an application system based on the interface calling instruction, and determining a second packaged interface corresponding to the first interface from the standard function and command interface library packaged based on the first interface, wherein the standard function and command interface library packaged is developed based on the application scene customization of the RBD client;
and the sending unit is used for calling the second interface to send the corresponding command message to the backend storage cluster.
Optionally, the RBD client further includes a data caching module, the apparatus further includes a caching unit,
and the cache unit is used for storing the operation record related to the interface calling instruction to the data cache module.
Optionally, the data caching module backs up metadata of each RBD interface; the device also comprises a judging unit and a query unit,
the receiving unit is also used for receiving a query instruction issued by an application system;
the judging unit is used for judging whether the query instruction is an RBD metadata query instruction or not;
and if so, the query unit acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
Optionally, the data caching module is a relational database.
Optionally, the RBD client further includes a standard function and a command interface library, where the interface call instruction is an Rsetful interface class instruction, a command line call instruction or a standard function call class instruction;
if the interface call instruction is an Rsetful interface class instruction, the command agent module determines a first interface to be called by an application system based on the interface call instruction, and determines an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library based on the first interface, where the determining unit is specifically configured to:
the command agent module determines a target Rsetful interface required to be called by an application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library;
if the interface call instruction is a command line call instruction or a standard function call instruction, the command proxy module determines, based on the interface call instruction, a first interface that needs to be called by an application system, and determines, based on the first interface, an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library, where the determining unit is specifically configured to:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
In a third aspect, the present application provides a data processing apparatus comprising:
a memory for storing program instructions;
and a processor, configured to call the program instructions stored in the memory, and execute any one of the methods according to the first aspect according to the obtained program.
In a fourth aspect, the present application provides a computer storage medium having stored thereon computer-executable instructions for causing a computer to perform the method of any of the first aspects.
The beneficial effect of this application is as follows:
to sum up, the data processing method provided by the present application is applied to an RBD client, where the RBD client at least includes a command agent module, and a standard function and a command interface library after being encapsulated, and the method includes: the command agent module receives an interface calling instruction issued by an application system; the command agent module determines a first interface required to be called by an application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the standard function and command interface library encapsulated based on the first interface, wherein the standard function and command interface library encapsulated is developed by self-definition based on an application scene of an RBD client; and the command agent module calls the second interface to send a corresponding command message to the backend storage cluster.
By adopting the data processing method provided by the application, the encapsulated RBD interface which is self-defined and developed according to the current application scene is established, the mapping relation between the native RBD interface and the encapsulated RBD interface is established, namely, a certain native RBD interface is called, namely, the encapsulated RBD interface corresponding to the certain native RBD interface can be called according to the mapping relation. The packaged RBD interface is an interface developed according to the current application scene, is more suitable for the application of the current application scene, can avoid the instability problem of an open source RBD client (interface), improves the query management efficiency of the RBD client, and improves the user experience.
Furthermore, the RBD client provided by the application adds a mapping relationship between the Restful interface and the encapsulated RBD interface, so that the RBD client can process a Restful interface call instruction sent by an application system, and selects the encapsulated RDN interface corresponding to the Restful interface for service processing.
Drawings
Fig. 1 is a schematic structural diagram of a distributed system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 3 is a schematic view illustrating a processing flow of an instruction of an RBD client according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
First, the term "and" in the embodiment of the present application is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
When the present application refers to the ordinal numbers "first", "second", "third" or "fourth", etc., it should be understood that this is done for differentiation only, unless it is clear from the context that the order is actually expressed.
The scheme of the present application will be described in detail by specific examples, but the present application is not limited to the following examples.
The present application provides a data processing method, which is applied to an RBD Client, where the RBD Client is a software package deployed on a Client (e.g., a VM), and exemplarily, as shown in fig. 2, is a system architecture diagram of a distributed storage system provided in an embodiment of the present application, the distributed storage system is a backend storage system (e.g., each storage node) of a multi-Client service system (i.e., an application service system), the multi-Client service system at least includes Email service, Database service and VDI service, each VM may access the backend storage system through a local Client (e.g., an RBD Client), a Hypervisor may also access the backend storage system through the local Client, and the backend storage system provides storage services for the VM through a volume.
The following will describe the method embodiment of the present application in detail by using a specific embodiment, and exemplarily refer to fig. 2, the data processing method provided by the present application is applied to an RBD client, where the RBD client at least includes a command agent module, an encapsulated standard function and a command interface library, and the detailed flow of the data processing method is as follows:
step 200: and the command agent module receives an interface calling instruction issued by the application system.
In the embodiment of the application, the interface call instruction issued by the application system may be an Rsetful interface instruction, a command line call instruction or a standard function call instruction; when receiving an interface calling instruction issued by an application system, the command agent module processes the interface calling instruction, determines the type of the interface calling instruction and determines the specific interface information to be called by the application system.
Step 210: and the command agent module determines a first interface required to be called by the application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the encapsulated standard function and command interface library based on the first interface.
In the embodiment of the application, the packaged standard function and command interface library are developed by self-definition based on an application scene of an RBD client. The command agent module uses different processing logics according to the type of the interface calling instruction issued by the application system.
For example, if the command agent module determines that the interface call instruction is an Rsetful interface class instruction, the step of determining, by the command agent module, a first interface to be called by an application system based on the interface call instruction, and determining, based on the first interface, a packaged second interface corresponding to the first interface from the packaged standard function and command interface library includes:
and the command agent module determines a target Rsetful interface required to be called by the application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library.
That is to say, in the embodiment of the present application, the packaged standard function and command interface library is newly added, functions and commands in the packaged standard function and command interface library can be used for accessing the back-end distributed storage system, and the processing logic of the client is newly added with the processing logic relationship between each Restful interface and each standard function and command in the packaged standard function and command library. For example, the Restful interface 1 corresponds to the standard function 1, that is, when the command proxy module determines that the target Rsetful interface that the application system needs to call is the Restful interface 1, it determines that the standard function 1 needs to be called to access the back-end distributed storage system directly according to the processing logic relationship between the Restful interface 1 and the standard function 1.
For another example, if the interface call instruction is a command line call or a standard function call instruction, the step of determining, by the command agent module, a first interface to be called by the application system based on the interface call instruction, and determining, based on the first interface, a packaged second interface corresponding to the first interface from the packaged standard function and command interface library includes:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
That is to say, in the embodiment of the present application, the processing logic of the client is further added with the processing logic relationship between each native RBD interface and each standard function and each command in the encapsulated standard function and command library. For example, the native RBD interface 1 corresponds to the standard function 1, that is, when the command agent module determines that the target native RBD interface that the application system needs to call is the native RBD interface 1, it determines that the standard function 1 needs to be called to access the backend distributed storage system directly according to the processing logic relationship between the native RBD interface 1 and the standard function 1.
It should be noted that, in the embodiment of the present application, besides a native RBD interface (i.e., an existing RBD interface of an open-source RBD client), a packaged RBD interface that is developed by self-defining according to a current application scenario is also included, and a mapping relationship between the native RBD interface and the packaged RBD interface is established, that is, a certain native RBD interface is called, that is, the packaged RBD interface corresponding to the certain native RBD interface is called according to the mapping relationship. The packaged RBD interface is an interface developed according to the current application scene, is more suitable for the application of the current application scene, and can avoid the instability problem of an open-source RBD client (interface). Furthermore, the RBD client provided by the application adds a mapping relationship between the Restful interface and the encapsulated RBD interface, so that the RBD client can process a Restful interface call instruction sent by an application system, and selects the encapsulated RDN interface corresponding to the Restful interface for service processing.
Step 220: and the command agent module calls the second interface to send a corresponding command message to the back-end storage cluster.
In the embodiment of the application, the command agent module calls the encapsulated RBD interface, sends the processing command to the back-end storage cluster, the back-end storage cluster performs data processing according to the received processing command and returns the processing result to the command agent module, and the command agent module returns the received processing result to the application system to complete the data processing process.
Further, in this embodiment of the present application, the RBD client may further include a data caching module, and then the data processing method may further include the following steps:
and the command agent module stores the operation record related to the interface calling instruction to the data caching module.
Furthermore, in this embodiment of the present application, the data caching module backs up metadata of each RBD interface, and then the data processing method may further include the following steps:
step 1: the command agent module receives a query instruction issued by an application system;
step 2: the command agent module judges whether the query instruction is an RBD metadata query instruction;
and step 3: and if the command agent module judges that the query instruction is an RBD metadata query instruction, the command agent module acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
That is, the speed of the operations of sorting, searching and selecting the RBD attribute information is accelerated by the mode of setting the cache module in the RBD client and the mode of backing up the RBD metadata in the cache module.
In practical application, an application system calls an RBD interface to issue a query command, the interface is switched to a command agent module, the command agent module is switched to a logic processing layer after processing, and the logic processing layer judges whether the query command needs to be issued to a rear-end storage cluster or not. If not, the data content needing to be inquired is directly obtained from the cache module and then returned to the application system. And if the query needs to be carried out on the back-end storage cluster, sending a command to the back-end storage cluster for query and returning the command to the application system.
Furthermore, in this embodiment of the application, the cache module is a relational database. That is, the RBD client may be provided with a relational database for storing RBD metadata, and attribute information such as the latest metadata of each RBD interface may be maintained in the relational database. Then, when the subsequent application system queries the metadata of any RBD interface, the metadata of any RBD interface can be directly obtained from the relational database and returned to the application system.
Further, in this embodiment of the application, the RBD client may further include a value-added function module (for example, a disaster recovery module is set for a current application scenario), and then the command agent module may further determine whether a Restful interface or an RBD interface called by the application system is an interface corresponding to a newly added value-added function, for example, an interface corresponding to a disaster recovery function, and if so, notify the value-added function module, so that the value-added function module calls a target interface (a newly added RBD interface associated with the value-added function module) corresponding to the value-added function module after being encapsulated, and completes a corresponding value-added service.
The foregoing embodiment is described in detail with reference to a specific application scenario, and for example, with reference to fig. 3, a schematic view of an internal processing flow of an RBD client according to an embodiment of the present application is provided. The interface call instruction issued by the application system can be divided into a Restful interface call instruction sent by a foreground page, an RBD command call instruction sent by a background script and an RBD interface call instruction sent by an internal code.
The Restful interface calling instruction sent by the foreground page can be directly sent to the command proxy module, the command proxy module can judge whether the Restful interface is an interface associated with a value-added function, if so, the Restful interface calling instruction is sent to the value-added function module, the value-added function module determines a packaged RBD interface associated with the Restful interface according to the mapping relation between the Restful interface and a newly-added RBD interface (the newly-added RBD interface after packaging and renaming) and executes the related value-added function, otherwise, the packaged RBD interface associated with the Restful interface is determined directly according to the mapping relation between the Restful interface and the newly-added RBD interface, and related business operation is executed.
Aiming at an RBD command calling instruction sent by a background script, the RBD command is required to be adopted to call a corresponding RBD interface and send the RBD interface to a command agent module, a mapping relation between the RBD command and a native RBD interface is required to be determined, the native RBD interface required to be called is then sent to the command agent module, the command agent module can judge whether the native RBD interface is an interface associated with a value-added function, if so, the native RBD interface is sent to the value-added function module, the value-added function module determines a packaged RBD interface associated with the native RBD interface according to the mapping relation between the native RBD interface and a newly-added RBD interface (the newly-added RBD interface after package and name-change), and executes the related value-added function, otherwise, the packaged RBD interface associated with the native RBD interface is determined directly according to the mapping relation between the native RBD interface and the newly-added RBD interface, and executing relevant business operation.
The method comprises the steps that a native RBD interface required to be called by an application system is directly determined according to an RBD interface calling instruction sent by an internal code, and then the native RBD interface is sent to a command agent module, the command agent module can judge whether the native RBD interface is an interface associated with a value-added function, if yes, the native RBD interface is sent to the value-added function module, the value-added function module determines an encapsulated RBD interface associated with the native RBD interface according to the mapping relation between the native RBD interface and a newly-added RBD interface (the newly-added RBD interface is encapsulated and renamed) and executes a related value-added function, otherwise, the encapsulated RBD interface associated with the native RBD interface is directly determined according to the mapping relation between the native RBD interface and the newly-added RBD interface, and related service operation is executed.
Further, the command agent module may store the operation record in a local relational database when processing the business operation associated with any interface. And the relational database also stores the metadata of each RBD interface in a backup way, so that when a metadata query instruction of any RBD interface sent by the application system is received, the metadata of the corresponding RBD interface can be directly obtained from the local relational database and returned to the application system.
Based on the foregoing embodiments, referring to fig. 4, a data processing apparatus provided in an embodiment of the present application is applied to an RBD client, where the RBD client at least includes a command agent module, and an encapsulated standard function and command interface library, and the apparatus includes:
a receiving unit 40, configured to receive an interface call instruction issued by an application system;
a determining unit 41, configured to determine, based on the interface call instruction, a first interface that needs to be called by an application system, and determine, based on the first interface, a second encapsulated interface corresponding to the first interface from the standard function and command interface library after encapsulation, where the standard function and command interface library after encapsulation is developed in a user-defined manner based on an application scenario of an RBD client;
a sending unit 42, configured to invoke the second interface to send a corresponding command message to the backend storage cluster.
Optionally, the RBD client further includes a data caching module, the apparatus further includes a caching unit,
and the cache unit is used for storing the operation record related to the interface calling instruction to the data cache module.
Optionally, the data caching module backs up metadata of each RBD interface; the device also comprises a judging unit and a query unit,
the receiving unit is also used for receiving a query instruction issued by an application system;
the judging unit is used for judging whether the query instruction is an RBD metadata query instruction or not;
and if so, the query unit acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
Optionally, the data caching module is a relational database.
Optionally, the RBD client further includes a standard function and a command interface library, where the interface call instruction is an Rsetful interface class instruction, a command line call instruction or a standard function call class instruction;
if the interface call instruction is an Rsetful interface class instruction, the command agent module determines a first interface to be called by an application system based on the interface call instruction, and determines an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library based on the first interface, where the determining unit is specifically configured to:
the command agent module determines a target Rsetful interface required to be called by an application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library;
if the interface call instruction is a command line call instruction or a standard function call instruction, the command proxy module determines, based on the interface call instruction, a first interface that needs to be called by an application system, and determines, based on the first interface, an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library, where the determining unit is specifically configured to:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
Further, referring to fig. 5, the present application also provides a data processing apparatus comprising a memory 50 and a processor 51, wherein,
a memory 50 for storing program instructions;
and a processor 51, configured to call the program instructions stored in the memory 50, and execute any one of the above method embodiments according to the obtained program.
Still further, the present application provides a computer storage medium having computer-executable instructions stored thereon for causing a computer to perform any of the above-described method embodiments.
To sum up, the data processing method provided by the present application is applied to an RBD client, where the RBD client at least includes a command agent module, and a standard function and a command interface library after being encapsulated, and the method includes: the command agent module receives an interface calling instruction issued by an application system; the command agent module determines a first interface required to be called by an application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the standard function and command interface library encapsulated based on the first interface; and the command agent module calls the second interface to send a corresponding command message to the backend storage cluster.
By adopting the data processing method provided by the application, the encapsulated RBD interface which is self-defined and developed according to the current application scene is established, the mapping relation between the native RBD interface and the encapsulated RBD interface is established, namely, a certain native RBD interface is called, namely, the encapsulated RBD interface corresponding to the certain native RBD interface can be called according to the mapping relation. The packaged RBD interface is an interface developed according to the current application scene, is more suitable for the application of the current application scene, can avoid the instability problem of an open source RBD client (interface), improves the query management efficiency of the RBD client, and improves the user experience.
Furthermore, the RBD client provided by the application adds a mapping relationship between the Restful interface and the encapsulated RBD interface, so that the RBD client can process a Restful interface call instruction sent by an application system, and selects the encapsulated RDN interface corresponding to the Restful interface for service processing.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to encompass such modifications and variations.
Claims (10)
1. A data processing method is applied to an RBD client, wherein the RBD client at least comprises a command agent module, an encapsulated standard function and a command interface library, and the method comprises the following steps:
the command agent module receives an interface calling instruction issued by an application system;
the command agent module determines a first interface required to be called by an application system based on the interface calling instruction, and determines a second encapsulated interface corresponding to the first interface from the standard function and command interface library encapsulated based on the first interface, wherein the standard function and command interface library encapsulated is developed by self-definition based on an application scene of an RBD client;
and the command agent module calls the second interface to send a corresponding command message to the backend storage cluster.
2. The method of claim 1, wherein the RBD client further comprises a data caching module, the method further comprising:
and the command agent module stores the operation record related to the interface calling instruction to the data caching module.
3. The method of claim 2, wherein the data caching module backs up metadata for each RBD interface; the method further comprises the following steps:
the command agent module receives a query instruction issued by an application system;
judging whether the query instruction is an RBD metadata query instruction or not;
and if so, the command agent module acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
4. The method of claim 2 or 3, wherein the data caching module is a relational database.
5. The method of claim 1, wherein the RBD client further comprises a standard function and command interface library, the interface call instruction being a Rsetful interface class instruction, a command line call, or a standard function call class instruction;
if the interface calling instruction is an Rsetful interface class instruction, the step of determining, by the command agent module, a first interface to be called by an application system based on the interface calling instruction, and determining, based on the first interface, an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library includes:
the command agent module determines a target Rsetful interface required to be called by an application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library;
if the interface calling instruction is a command line calling or standard function calling instruction, the step of determining, by the command proxy module, a first interface to be called by an application system based on the interface calling instruction, and determining, based on the first interface, a packaged second interface corresponding to the first interface from the packaged standard function and command interface library includes:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
6. A data processing device is applied to an RBD client, the RBD client at least comprises a command agent module and an encapsulated standard function and command interface library, wherein the device comprises:
the receiving unit is used for receiving an interface calling instruction issued by an application system;
the determining unit is used for determining a first interface required to be called by an application system based on the interface calling instruction, and determining a second packaged interface corresponding to the first interface from the standard function and command interface library packaged based on the first interface, wherein the standard function and command interface library packaged is developed based on the application scene customization of the RBD client;
and the sending unit is used for calling the second interface to send the corresponding command message to the backend storage cluster.
7. The apparatus of claim 6, wherein the RBD client further comprises a data caching module, the apparatus further comprising a caching unit,
and the cache unit is used for storing the operation record related to the interface calling instruction to the data cache module.
8. The apparatus of claim 7, wherein the data caching module backs up metadata for each RBD interface; the device also comprises a judging unit and a query unit,
the receiving unit is also used for receiving a query instruction issued by an application system;
the judging unit is used for judging whether the query instruction is an RBD metadata query instruction or not;
and if so, the query unit acquires corresponding target RBD metadata from the data cache module and returns the target RBD metadata to an application system.
9. The apparatus of claim 7 or 8, wherein the data caching module is a relational database.
10. The apparatus of claim 6, wherein the RBD client further comprises a standard function and command interface library, the interface call instruction being a Rsetful interface class instruction, a command line call, or a standard function call class instruction;
if the interface call instruction is an Rsetful interface class instruction, the command agent module determines a first interface to be called by an application system based on the interface call instruction, and determines an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library based on the first interface, where the determining unit is specifically configured to:
the command agent module determines a target Rsetful interface required to be called by an application system based on the interface calling instruction, and directly determines a packaged RBD interface corresponding to the target Rsetful interface from the packaged standard function and command interface library;
if the interface call instruction is a command line call instruction or a standard function call instruction, the command proxy module determines, based on the interface call instruction, a first interface that needs to be called by an application system, and determines, based on the first interface, an encapsulated second interface corresponding to the first interface from the encapsulated standard function and command interface library, where the determining unit is specifically configured to:
and the command agent module determines a target RBD interface required to be called by the application system from the standard function and command interface library based on the interface calling instruction, and determines a packaged RBD interface corresponding to the target RBD interface from the packaged standard function and command interface library.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010246306.0A CN111506601B (en) | 2020-03-31 | 2020-03-31 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010246306.0A CN111506601B (en) | 2020-03-31 | 2020-03-31 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111506601A true CN111506601A (en) | 2020-08-07 |
| CN111506601B CN111506601B (en) | 2022-08-30 |
Family
ID=71875946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010246306.0A Active CN111506601B (en) | 2020-03-31 | 2020-03-31 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111506601B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113419673A (en) * | 2021-06-05 | 2021-09-21 | 济南浪潮数据技术有限公司 | Method, system, medium and device for RBD access storage cluster |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7340639B1 (en) * | 2004-01-08 | 2008-03-04 | Network Appliance, Inc. | System and method for proxying data access commands in a clustered storage system |
| CN103294525A (en) * | 2012-02-27 | 2013-09-11 | 国际商业机器公司 | Method and system for inquiring database with user defined function |
| WO2016197341A1 (en) * | 2015-06-10 | 2016-12-15 | Intel Corporation | Webgl application analyzer |
| CN107368300A (en) * | 2017-06-26 | 2017-11-21 | 北京天元创新科技有限公司 | A kind of data aggregation system and method based on MapReduce |
| CN107665216A (en) * | 2016-07-29 | 2018-02-06 | 银联数据服务有限公司 | A kind of data bank access method and middleware |
| CN109710340A (en) * | 2019-01-18 | 2019-05-03 | 北京顺丰同城科技有限公司 | Software tool call method and device, software tool packaging method and device |
| CN110297840A (en) * | 2019-05-22 | 2019-10-01 | 平安银行股份有限公司 | Data processing method, device, equipment and the storage medium of rule-based engine |
| CN110457345A (en) * | 2019-07-05 | 2019-11-15 | 平安证券股份有限公司 | Database read method, device and computer readable storage medium |
-
2020
- 2020-03-31 CN CN202010246306.0A patent/CN111506601B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7340639B1 (en) * | 2004-01-08 | 2008-03-04 | Network Appliance, Inc. | System and method for proxying data access commands in a clustered storage system |
| CN103294525A (en) * | 2012-02-27 | 2013-09-11 | 国际商业机器公司 | Method and system for inquiring database with user defined function |
| WO2016197341A1 (en) * | 2015-06-10 | 2016-12-15 | Intel Corporation | Webgl application analyzer |
| CN107665216A (en) * | 2016-07-29 | 2018-02-06 | 银联数据服务有限公司 | A kind of data bank access method and middleware |
| CN107368300A (en) * | 2017-06-26 | 2017-11-21 | 北京天元创新科技有限公司 | A kind of data aggregation system and method based on MapReduce |
| CN109710340A (en) * | 2019-01-18 | 2019-05-03 | 北京顺丰同城科技有限公司 | Software tool call method and device, software tool packaging method and device |
| CN110297840A (en) * | 2019-05-22 | 2019-10-01 | 平安银行股份有限公司 | Data processing method, device, equipment and the storage medium of rule-based engine |
| CN110457345A (en) * | 2019-07-05 | 2019-11-15 | 平安证券股份有限公司 | Database read method, device and computer readable storage medium |
Non-Patent Citations (6)
| Title |
|---|
| MARIO DI MAURO等: "Availability Analysis of IP Multimedia Subsystem in Cloud Environments", 《2019 4TH INTERNATIONAL CONFERENCE ON SYSTEM RELIABILITY AND SAFETY (ICSRS)》 * |
| 刘顺潮: "基于中间件的传统CAD软件协同设计与实现", 《电子科技》 * |
| 张俊: "基于Ceph的网络存储***", 《科技经济导刊》 * |
| 李敏: "分布式数据库安全代理的研究与实现", 《现代电子技术》 * |
| 王乐等: "Linux下的DDR DIMM总线接口设备检测方法", 《计算机工程》 * |
| 肖万斌等: "基于ACE Streams框架的舰船综合电子仿真***设计", 《火力与指挥控制》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113419673A (en) * | 2021-06-05 | 2021-09-21 | 济南浪潮数据技术有限公司 | Method, system, medium and device for RBD access storage cluster |
| CN113419673B (en) * | 2021-06-05 | 2022-05-27 | 济南浪潮数据技术有限公司 | Method, system, medium and device for RBD access storage cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111506601B (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160077863A1 (en) | Reliable and scalable image transfer for data centers with low connectivity using redundancy detection | |
| CN110247984B (en) | Service processing method, device and storage medium | |
| CN112929401B (en) | Registration method and device | |
| US8813076B2 (en) | Virtual machine updates | |
| CN109254825A (en) | A kind of method and device for receiving pipe virtual hard disk | |
| CN107800730B (en) | Capacity expansion method and device for virtual disk | |
| CN111124286A (en) | Libcloud-based multi-cloud management implementation method | |
| US12026066B2 (en) | Backing up data for a namespace assigned to a tenant | |
| US10990286B1 (en) | Parallel upgrade of nodes in a storage system | |
| CN107566470B (en) | Method and device for managing virtual machine in cloud data system | |
| CN111506601B (en) | Data processing method and device | |
| CN111459403A (en) | Storage hardware management method and device | |
| US20150220517A1 (en) | Efficient conflict resolution among stateless processes | |
| CN104517067A (en) | Method, device and system for data access | |
| US20130246347A1 (en) | Database file groups | |
| US9703788B1 (en) | Distributed metadata in a high performance computing environment | |
| CN106528876B (en) | The information processing method and distributed information processing system of distributed system | |
| CN110471744B (en) | Password modification method, device, equipment and computer readable storage medium | |
| CN105247618A (en) | Method and device for hard disk drive to execute application code | |
| CN105095021A (en) | Method and device for system recovery | |
| CN114840307A (en) | Container loading method, device, equipment and storage medium | |
| CN114416689A (en) | Data migration method and device, computer equipment and storage medium | |
| CN115113798A (en) | Data migration method, system and equipment applied to distributed storage | |
| US20200143001A1 (en) | Indexing structure with size bucket indexes | |
| WO2016122841A1 (en) | Exposing storage entity consistency capability status |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |