Disclosure of Invention
The application provides a cloud storage data direct transmission management method, a cloud transmission management platform and a cloud storage node direct transmission management system, which are used for solving the technical problem that in the prior art, in a client and cloud storage node direct transmission mode, a user needs to finish data transmission operation through clients of different operators respectively, and the operation in a transmission process is complex.
The first aspect of the present application provides a cloud storage data direct transmission management method, which is applied to a cloud transmission management platform and includes:
responding to a received data transmission request message, verifying storage authority of a user according to user information and target cloud storage operator information contained in the data transmission request message and combining service relation information of the user and a cloud storage operator which are stored in the cloud transmission management platform locally, and generating verification signature information after the verification is passed, wherein the data transmission request message is a message which is generated by a client and sent to the cloud transmission management platform in response to a data transmission operation instruction of the user;
feeding back the verification signature information, a target cloud storage node address and a target cloud storage protocol to the client, so that the client uploads data to the target cloud storage node in a cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol, wherein the cloud storage pool comprises cloud storage nodes of a plurality of cloud storage operators, the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage information generation rule protocol of the target cloud storage operator.
Preferably, the obtaining mode of the target cloud storage node address specifically includes:
determining cloud storage nodes of the target cloud storage operator in the cloud storage pool according to the target cloud storage operator information;
according to the IP distribution configuration information and the distribution configuration weight of the cloud storage nodes, determining target cloud storage nodes and target cloud storage node addresses corresponding to the target cloud storage nodes from the cloud storage nodes of the target cloud storage operator.
A second aspect of the present application provides a cloud transmission management platform, including:
the data transmission request message response unit is used for responding to the received data transmission request message, verifying the storage authority of the user according to the user information and the target cloud storage operator information contained in the data transmission request message and by combining the service relation information of the user and the cloud storage operator which are stored in the cloud transmission management platform locally, and generating verification signature information after the verification is passed, wherein the data transmission request message is a message which is generated by the client and sent to the cloud transmission management platform in response to the data transmission operation instruction of the user;
the data transmission request message feedback unit is configured to feed back the verification signature information, a target cloud storage node address and a target cloud storage protocol to the client, so that the client uploads data to the target cloud storage node in a cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol, where the cloud storage pool includes cloud storage nodes of multiple cloud storage operators, the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage information generation rule protocol of the target cloud storage operator.
Preferably, the method further comprises the following steps: a target cloud storage node address obtaining unit, configured to:
determining cloud storage nodes of the target cloud storage operator in the cloud storage pool according to the target cloud storage operator information;
according to the IP distribution configuration information and the distribution configuration weight of the cloud storage nodes, determining target cloud storage nodes and target cloud storage node addresses corresponding to the target cloud storage nodes from the cloud storage nodes of the target cloud storage operator.
A third aspect of the present application provides a cloud storage data direct transmission system, including: the cloud transmission management platform comprises a client, a cloud storage pool and the cloud transmission management platform provided by the second aspect of the application;
the client is used for responding to a data transmission operation instruction of a user, generating a data transmission request message and sending the data transmission request message to the cloud transmission management platform;
the cloud transmission management platform is used for responding to the received data transmission request message, combining service relationship information of the user and the cloud storage operator which are stored in the cloud transmission management platform locally according to the user information and the target cloud storage operator information which are contained in the data transmission request message, the storage authority of the user is verified, and after the verification is passed, verification signature information is generated and fed back to the client together with the target cloud storage node address and the target cloud storage protocol, causing the client to upload data to a target cloud storage node in the cloud storage pool according to the verification signature information, the target cloud storage node address, and the target cloud storage protocol, the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage rule protocol of the target cloud storage operator.
Preferably, the cloud transmission management platform is further configured to:
receiving file information fed back by the target cloud storage node, and sending the file information to a service database, wherein the file information is obtained by extracting data information of the data based on the received data by the target cloud storage node.
Preferably, the method further comprises the following steps: a file information caching unit;
the file information caching unit is used for storing the file information fed back by the target cloud storage node;
the cloud transmission management platform is further configured to:
and obtaining the file information from the file information cache unit, and sending the file information to a service database, wherein the file information is obtained by extracting data information of the data based on the received data by the target cloud storage node.
According to the technical scheme, the embodiment of the application has the following advantages:
the scheme provided by the application responds to a received data transmission request message through the cloud transmission management platform, carries out storage authority verification on a user according to user information and target cloud storage operator information contained in the data transmission request message and by combining service relation information of the user and the cloud storage operator locally stored in the cloud transmission management platform, and after the verification is passed, feeds back verification signature information, a target cloud storage node address and a target cloud storage protocol to a client so that the client uploads data to the target cloud storage node in a cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol, and under a mode that the client directly transmits to the cloud storage node, uniform data transmission operation is completed on different cloud storage operators, and the problem that the prior art is in the mode that the client directly transmits to the cloud storage node is solved, the technical problem of complex operation in the transmission process exists.
Detailed Description
The embodiment of the application provides a cloud storage data direct transmission management method, a cloud transmission management platform and a cloud storage node direct transmission management system, which are used for solving the technical problem that in the prior art, in a client and cloud storage node direct transmission mode, a user needs to finish data transmission operation through clients of different operators respectively, and the operation in a transmission process is complex.
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
A first embodiment of the present application provides a cloud storage data direct transfer system, including: the system comprises a client, a cloud storage pool and a cloud transmission management platform;
the client is used for responding to a data transmission operation instruction of a user, generating a data transmission request message and sending the data transmission request message to the cloud transmission management platform;
the cloud transmission management platform is used for responding to the received data transmission request message, verifying storage authority of a user according to user information and target cloud storage operator information contained in the data transmission request message and combining service relation information of the user and the cloud storage operator stored locally in the cloud transmission management platform, generating verification signature information after the verification passes, and feeding the verification signature information, a target cloud storage node address and a target cloud storage protocol back to the client, so that the client uploads data to the target cloud storage node in the cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol, wherein the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage rule protocol of the target cloud storage operator.
As shown in fig. 1, the cloud storage data direct transmission system provided in this embodiment mainly includes: the system comprises a client, a cloud storage pool and a cloud transmission management platform, wherein the client is a terminal device or an application program product operated by a user, such as various cloud storage devices or applications such as an intelligent device, a network disk, a social application and a monitoring device, and is used for responding to a data transmission operation instruction of the user, generating a data transmission request message and sending the data transmission request message to the cloud transmission management platform.
The cloud transmission management platform is used for responding to the received data transmission request message, verifying the storage authority of the user according to the user information and the target cloud storage operator information contained in the data transmission request message and combining the service relation information of the user and the cloud storage operator stored locally in the cloud transmission management platform, generating verification signature information after the verification is passed, and feeding the verification signature information, the target cloud storage node address and the target cloud storage protocol back to the client, so that the client uploads data to the target cloud storage node in the cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol.
The cloud storage operator mentioned in this embodiment includes, but is not limited to: the cloud storage system comprises an Ali cloud storage system, a space wing cloud storage system, a Huashi cloud storage system and the like, wherein a target cloud storage operator is one or more cloud storage operators selected from request messages submitted by users, a target cloud storage node is a cloud storage node of the target cloud storage operator, and a target cloud storage protocol is a storage rule protocol of the target cloud storage operator.
And then, the cloud storage pool is responsible for storing the received data in the corresponding target cloud storage node, and the data direct transmission process between the client and the cloud storage pool is completed.
As shown in fig. 2, the cloud storage application data direct transmission of the present embodiment is mainly divided into the following parts:
(1) the client application requests an address distribution service to perform identity verification and routing strategy selection.
(2) The address distribution service returns one or more target storage pool addresses and protocol rules.
(3) The client carries the protocol rule to request the target storage pool to upload, and the storage pool receives data after checking the rule.
More specifically, the present invention is to provide a novel,
(1) the client carries an authorized appKey request address distribution service gateway.
(2) And the service gateway inquires the authorization micro-service, the authorization micro-service inquires an authorization table according to the appKey, the authority information corresponding to the current appKey is returned, and the gateway verifies whether the gateway is accessible.
(3) And the gateway forwards the request to a back-end uploading micro-service, and the uploading service requests the shunting micro-service to obtain the storage pool id after carrying out a plurality of logic processes according to the client parameters.
(4) And the shunting microservice selects one storage pool to return according to the configured weight proportion according to the service binding table of the inquiry user and the cloud storage operator and the ip shunting table.
(5) The uploading service carries a storage pool id, a file directory, a speed limit value and the like returned by the shunting service to request the storage pool capacity micro-service, and the storage pool capacity service inquires a storage pool table through the id to obtain storage pool information and generates an uploading address and a rule return.
(6) And the uploading service encapsulates the uploading address and the rule of the storage pool and returns the uploading address and the rule to the client.
(7) The client carries the protocol rule to request the target storage pool to upload, the storage pool check rule is passed, then the client data stream is received, the upload result is returned, and the upload is completed.
In the embodiment, a method for directly transmitting the cloud storage resource pool by the client is adopted, a service provider only needs to provide the address of the cloud storage server and the uploading rule after calculation, and the client directly interacts with the cloud storage server after getting the address and the rule to complete data transmission. By using the method, the clients are uploaded independently without mutual influence, and the transmission speed is greatly improved. A service provider can support massive data uploading by maintaining a set of lightweight and low-cost address distribution service, core data leakage is avoided, system safety is improved, cost is reduced, unified nano-tubes such as Array cloud storage, sky wing cloud storage, Huacheng cloud storage and the like of various storage pools are realized, and different adaptations are made according to the types of the storage pools to create a unified mixed storage platform.
Further, the cloud transmission management platform is further configured to:
and receiving file information fed back by the target cloud storage node, and sending the file information to a service database, wherein the file information is obtained by extracting data information of data based on the received data by the target cloud storage node.
It should be noted that after the process of uploading file data to the cloud storage pool by the client is completed, a step of submitting file information to the service system by the client and warehousing the file information is also provided, however, the link is increased due to the fact that the client submits the file alone, and when a problem exists in the network or the system, the file is lost, and the service is affected. Therefore, in the scheme of this embodiment, a mode that the cloud transmission management platform submits the files is adopted, the cloud transmission management platform receives the file information fed back by the cloud storage pool and submits the file information to the service database in a unified manner, the client does not need to submit the files separately again, the system interaction complexity is simplified by automatic submission through the cloud transmission management platform, the process connection avoids that each step starts from the client side, the link length is reduced, and the network stability of the service system is utilized.
Further, the system further comprises: a file information caching unit;
the file information caching unit is used for storing file information fed back by the target cloud storage node;
the cloud transmission management platform is further used for:
and acquiring file information from the file information cache unit, and sending the file information to a service database, wherein the file information is obtained by extracting data information of data based on the received data by the target cloud storage node.
It should be noted that, the system provided in the embodiment of the present application may further include: the file information caching unit can be a message queue built by Kafka, rocktMQ and the like, and is used for storing file information fed back by a target cloud storage node, and when the system architecture comprises the file information caching unit, the cloud transmission management platform can be used for: the file information is obtained from the file information caching unit and sent to the service database, so that when problems of network abnormity, overhigh system load and the like occur, the file information fed back by the cloud storage pool can not be lost through the characteristics of asynchronization, persistence, peak clipping and valley filling of the message queue, and the availability and uploading success rate of the system are greatly improved.
The above is a detailed description of an embodiment of the cloud storage data direct transmission system provided by the present application, and the following is a detailed description of an embodiment of the cloud storage data direct transmission management method provided by the present application.
Referring to fig. 3, a first aspect of the present application provides a cloud storage data direct transmission management method, where the method is applied to a cloud transmission management platform mentioned in the first embodiment, and includes:
step 101, responding to the received data transmission request message, according to user information and target cloud storage operator information contained in the data transmission request message, and combining service relationship information of a user and a cloud storage operator stored locally on a cloud transmission management platform, performing storage authority verification on the user, and when the verification passes, generating verification signature information.
The data transmission request message is a message which is generated by the client and sent to the cloud transmission management platform when the client responds to a data transmission operation instruction of a user.
And 102, feeding back the verification signature information, the target cloud storage node address and the target cloud storage protocol to the client, so that the client uploads data to the target cloud storage node in the cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol.
The cloud storage pool comprises cloud storage nodes of a plurality of cloud storage operators, the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage information generation rule protocol of the target cloud storage operator.
Further, the obtaining mode of the target cloud storage node address specifically includes:
determining cloud storage nodes of a target cloud storage operator in a cloud storage pool according to the target cloud storage operator information;
according to the IP distribution configuration information and the distribution configuration weight of the cloud storage nodes, determining target cloud storage nodes and target cloud storage node addresses corresponding to the target cloud storage nodes from the cloud storage nodes of the target cloud storage operator.
The above is a detailed description of an embodiment of a cloud storage data direct transmission management method provided by the present application, and the following is a detailed description of an embodiment of a cloud transmission management platform provided by the present application.
Referring to fig. 4, a third embodiment of the present application provides a cloud transmission management platform, including:
a data transmission request message responding unit 201, configured to respond to a received data transmission request message, perform storage authority verification on a user according to user information and target cloud storage operator information included in the data transmission request message and by combining service relationship information between the user and a cloud storage operator stored locally in a cloud transmission management platform, and generate verification signature information after the verification passes, where the data transmission request message is a message generated by a client in response to a data transmission operation instruction of the user and sent to the cloud transmission management platform by the client;
the data transmission request message feedback unit 202 is configured to feed back the verification signature information, the target cloud storage node address and the target cloud storage protocol to the client, so that the client uploads data to the target cloud storage node in the cloud storage pool according to the verification signature information, the target cloud storage node address and the target cloud storage protocol, where the cloud storage pool includes cloud storage nodes of multiple cloud storage operators, the target cloud storage node is a cloud storage node of the target cloud storage operator, and the target cloud storage protocol is a storage information generation rule protocol of the target cloud storage operator.
Preferably, the method further comprises the following steps: the target cloud storage node address acquisition unit is used for:
determining cloud storage nodes of a target cloud storage operator in a cloud storage pool according to the target cloud storage operator information;
according to the IP distribution configuration information and the distribution configuration weight of the cloud storage nodes, determining target cloud storage nodes and target cloud storage node addresses corresponding to the target cloud storage nodes from the cloud storage nodes of the target cloud storage operator.
In the several embodiments provided in the present application, it should be understood that the disclosed terminal, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.