CN107092686A - A kind of file management method and device based on cloud storage platform - Google Patents
A kind of file management method and device based on cloud storage platform Download PDFInfo
- Publication number
- CN107092686A CN107092686A CN201710269824.2A CN201710269824A CN107092686A CN 107092686 A CN107092686 A CN 107092686A CN 201710269824 A CN201710269824 A CN 201710269824A CN 107092686 A CN107092686 A CN 107092686A
- Authority
- CN
- China
- Prior art keywords
- shared pool
- user
- file
- service
- cloud storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a kind of file management method and device based on cloud storage platform, and this method is applied to cloud storage platform, and the cloud storage platform includes the resource group being made up of some memory node equipment；Storage resource in the resource group is divided into some shared pools；Wherein, the different types of storage file of different shared pools correspondence；This method may include：Receive the service request for obtaining file destination that operation system is issued；Wherein, the User Catalog of the file destination is carried in the service request, the User Catalog is encoded including shared pool；The User Catalog is parsed, the shared pool coding is obtained；The shared pool for the shared pool codes match searched and got；The file destination is searched in the shared pool found, and the file destination is returned to the operation system to carry out corresponding business processing.The cloud storage method provided using the application, document retrieval can be improved efficiency.
Description
Technical field
The application is related to computer communication field, more particularly to a kind of cloud storage technology.
Background technology
Cloud storage can refer to by functions such as cluster application, network technology or distributed file systems, by network
Substantial amounts of different types of memory node cluster tool gets up to cooperate, common externally to provide data storage and Operational Visit work(
The system of energy.
Usual cloud storage can provide the interface of data storage and Operational Visit for service-user, and service-user is connect by calling
Mouthful, the preservation and access of service scripts can be achieved.Because the mixing that magnanimity is preserved in the memory node equipment of cloud storage is stored
Service scripts, such as picture file and video file mixing storage, therefore needed for how fast and effeciently searching service-user
Service scripts just turns into industry urgent problem to be solved.
The content of the invention
In view of this, the application provides a kind of file management method and device based on cloud storage platform, to improve text
Part retrieval ground efficiency.
Specifically, the application is achieved by the following technical solution：
According to the first aspect of the application there is provided a kind of file management method based on cloud storage platform, methods described should
For cloud storage platform, the cloud storage platform includes the resource group being made up of some memory node equipment；In the resource group
Storage resource be divided into some shared pools；Wherein, the different types of storage file of different shared pools correspondence；Methods described
Including：
Receive the service request for obtaining file destination that operation system is issued；Wherein, carried in the service request
The User Catalog of the file destination, the User Catalog is encoded including shared pool；
The User Catalog is parsed, the shared pool coding is obtained；
The shared pool for the shared pool codes match searched and got；
The file destination is searched in the shared pool found, and the file destination is returned into the operation system
To carry out corresponding business processing.
According to the second aspect of the application there is provided a kind of document management apparatus based on cloud storage platform, described device should
For cloud storage platform, the cloud storage platform includes the resource group being made up of some memory node equipment；In the resource group
Storage resource be divided into some shared pools；Wherein, the different types of storage file of different shared pools correspondence；Described device
Including：
Receiving unit, for receiving the service request for obtaining file destination that operation system is issued；Wherein, the industry
The User Catalog of the file destination is carried in business request, the User Catalog is encoded including shared pool；
Resolution unit, for parsing the User Catalog, obtains the shared pool coding；
Searching unit, for the shared pool for the shared pool codes match searched and got；
Returning unit, is returned for searching the file destination in the shared pool found, and by the file destination
The operation system is returned to carry out corresponding business processing.
The application proposes a kind of file management method based on cloud storage platform, by by different memory node equipment institutes group
Into resource group in storage resource, be divided into some shared pools for different file types.In retrieval, cloud storage platform
The service request for obtaining file destination that operation system is issued can be received.Wherein, the target is carried in the service request
The User Catalog of file, the User Catalog is encoded including shared pool.Cloud storage platform can parse the User Catalog, obtain this and share
Pond is encoded, and searches the shared pool of the shared pool codes match with getting.Cloud storage platform can be described shared what is found
The file destination is searched in pond, and the file destination is returned to the operation system to carry out corresponding business processing.
Because cloud storage platform can be based on shared pool, classification storage is carried out to service scripts, in retrieval, cloud storage platform
Filename is no longer based on to match each file, but by the way of two grades are searched, first based on being total in User Catalog
Pond coding is enjoyed, the shared pool where the file type identical file with the file destination is searched, then based on filename, from lookup
To shared pool in match this document, therefore with substantially increasing document retrieval efficiency, reduce retrieval bring memory node
The live load of equipment.
Brief description of the drawings
Fig. 1 is a kind of signal of the related cloud storage platform shown in the exemplary embodiment of the application one for document retrieval
Figure；
Fig. 2 is a kind of network of file management method based on cloud storage platform shown in the exemplary embodiment of the application one
Organization Chart；
Fig. 3 is a kind of flow chart of document retrieval based on cloud storage platform shown in the exemplary embodiment of the application one；
Fig. 4 is a kind of schematic diagram of shared pool shown in the exemplary embodiment of the application one；
Fig. 5 is a kind of schematic diagram of service scripts retrieval shown in the exemplary embodiment of the application one；
Fig. 6 is a kind of schematic diagram of occupied resource capacity acquiring method shown in the exemplary embodiment of the application one；
Fig. 7 is a kind of hardware structure diagram of cloud storage device place equipment shown in the exemplary embodiment of the application one；
Fig. 8 is a kind of block diagram of cloud storage device shown in the exemplary embodiment of the application one.
Embodiment
Here exemplary embodiment will be illustrated in detail, its example is illustrated in the accompanying drawings.Following description is related to
During accompanying drawing, unless otherwise indicated, the same numbers in different accompanying drawings represent same or analogous key element.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended
The example of the consistent apparatus and method of some aspects be described in detail in claims, the application.
It is the purpose only merely for description specific embodiment in term used in this application, and is not intended to be limiting the application.
" one kind ", " described " and "the" of singulative used in the application and appended claims are also intended to including majority
Form, unless context clearly shows that other implications.It is also understood that term "and/or" used herein refers to and wrapped
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the application
A little information should not necessarily be limited by these terms.These terms are only used for same type of information being distinguished from each other out.For example, not departing from
In the case of the application scope, the first information can also be referred to as the second information, similarly, and the second information can also be referred to as
One information.Depending on linguistic context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determining ".
Cloud storage can refer to by functions such as cluster application, network technology or distributed file systems, by network
Substantial amounts of different types of memory node cluster tool gets up to cooperate, common externally to provide data storage and Operational Visit work(
The system of energy.
Usual cloud storage can provide the interface of data storage and Operational Visit for service-user, and service-user is connect by calling
Mouthful, the preservation and access of service scripts can be achieved.
In the related art file management technology based on cloud storage platform, the file that cloud storage is stored is typically that mixing is protected
Deposit, such as mixing of picture file, video file or other kinds of file is preserved.
The storage mode of storage is mixed based on this files in different types, can be by as shown in Figure 1 in document retrieval
Retrieval mode is retrieved.Fig. 1 is that a kind of related cloud storage platform shown in the exemplary embodiment of the application one is directed to file
The schematic diagram of retrieval.
File path (such as usrdir001/20150314/ that cloud storage platform can be issued from service-user
S094000E.ts filename (S094000E.ts) is found in), then by institute in file name and each memory node equipment
Each file of storage is matched, and after file destination corresponding with file name is matched, the file destination is returned
Service-user.
But, on the one hand, the file mixing storage of this different file types, in document retrieval, it is necessary to different sections
Each file on point device is retrieved, and can just find the service scripts required for service-user.For example, service-user institute
The file needed is picture file A, and because the mixing of files in different types is stored, cloud storage platform requires to look up each memory node
Each file in equipment, whether picture file or video file, this considerably increases the workload of document retrieval, are reduced
The efficiency of document retrieval.
On the other hand, stored due to different types of file being mixed so that it is difficult to enter to the file of different file types
The disaster tolerance processing of row different stage.
In addition, picture file is small documents, the file of different file types is carried out mixing and deposited by big file during video file
Storage so that large and small file mixing storage, when obtaining the occupied resource capacity of memory node equipment, it is necessary to each file
It is scanned respectively, calculates the size of the shared resource of each file, the occupied resource that can just calculate the storage device holds
Amount.Due to needing that each file is scanned, the size of resource capacity shared by each file is calculated, is obtained so as to greatly reduce
The efficiency of occupied resource capacity is taken, while increasing the live load of equipment.
The application proposes a kind of file management method based on cloud storage platform, by by different memory node equipment institutes group
Into resource group in storage resource, be divided into some shared pools for different file types.In retrieval, cloud storage platform
The service request for obtaining file destination that operation system is issued can be received.Wherein, the target is carried in the service request
The User Catalog of file, the User Catalog is encoded including shared pool.Cloud storage platform can parse the User Catalog, obtain this and share
Pond is encoded, and searches the shared pool of the shared pool codes match with getting.Cloud storage platform can be in the shared pool found
The file destination is searched, and the file destination is returned to the operation system to carry out corresponding business processing.
On the one hand, because cloud storage platform can be based on shared pool, classification storage is carried out to service scripts, in retrieval, cloud
Storage platform, first based on the shared pool coding in User Catalog, can be searched and the file destination by the way of 2-level search
Shared pool where file type identical file, then based on filename, this document is matched from the shared pool found, without
It is that each file is matched, therefore with substantially increasing document retrieval efficiency, reduce retrieval and bring memory node to set
Standby live load.
On the other hand, due to that in cloud storage, shared pool can be based on, different types of file is stored respectively.
When obtaining occupied resource capacity, to picture file, i.e. small documents, the capacity of each small documents can be no longer scanned, but can
Directly calculate the data slot number taken of the shared pool for picture, you can obtain the capacity of the resource in the shared pool
Occupancy, substantially increases the efficiency of occupied resource capacity.
The third aspect, due to the file of different file types being stored respectively, it is possible to set up for not identical text
The disaster tolerance rank of the file of part type.
Referring to Fig. 2, Fig. 2 is a kind of file management side based on cloud storage platform shown in the exemplary embodiment of the application one
The network architecture diagram of method.The network architecture includes cloud storage platform, operation system and headend equipment.
Wherein, above-mentioned cloud storage platform, can provide interface, operation system is by calling cloud storage platform for operation system
Interface can realize the storage and access of service scripts.Cloud storage platform may include several cloud storage node devices, memory node
Equipment can constitute resource group.In the embodiment of the present application, in order to which different types of file is preserved respectively, memory node is set
Resource in the standby resource group constituted is divided into several shared pools, each different types of storage text of shared pool correspondence
Part.
Above-mentioned operation system, can be visual operation system software, can be flat to cloud storage according to different business demands
Service scripts on platform such as conducts interviews and stored at the operation.In field of video monitoring, above-mentioned operation system can be video monitoring
Platform etc., the video monitoring platform has to be managed to front-end camera, provides the user the functions such as video playback.Here,
Simply operation system is exemplarily illustrated, it not carried out specifically defined.
The hardware device for carrying above-mentioned operation system can be server, server cluster etc., here without specifically
Limit.
Above-mentioned headend equipment, refers to the equipment for carrying out the operations such as Data Collection.It is above-mentioned in the scene of field of video monitoring
Headend equipment can be front-end camera, such as IPC.Under live scene, above-mentioned headend equipment can be the end of live client
End equipment etc..Here, simply headend equipment is exemplarily illustrated, it not carried out specifically defined.
Under above-mentioned network construction, when service scripts is stored, User Catalog can be passed through operation system by cloud storage platform
Send to headend equipment, headend equipment is based on its owning user catalogue, service scripts is write to specified by the User Catalog
In the resource of the cloud storage platform, and generate file path information.
When service scripts is retrieved, cloud platform, can be based on text after the file path information that operation system is issued is received
Part routing information is locally carrying out the lookup of service scripts, and the service scripts found can be back into operation system.
Referring to Fig. 3, Fig. 3 is a kind of document retrieval stream based on cloud storage platform shown in the exemplary embodiment of the application one
Cheng Tu.This method is applied to cloud storage platform, it may include step 301 to step 303.
Step 301：Receive the service request for obtaining file destination that operation system is issued；Wherein, the business please
The middle User Catalog for carrying the file destination is sought, the User Catalog is encoded including shared pool；
Step 302：The User Catalog is parsed, the shared pool coding is obtained；
Step 303：The shared pool for the shared pool codes match searched and got；
Step 304：The file destination is searched in the shared pool found, and file destination return is described
Operation system is to carry out corresponding business processing.
It should be noted that the method for the cloud storage that the application is proposed can be applied in the scene with any progress cloud storage,
Such as video monitoring scene, live scene, electric business shopping scene, the cloud storage method that the application is proposed is not answered herein
Scene carries out specifically defined.
Herein, in order to more preferably introduce the cloud storage method that the application is proposed, below by taking video monitoring scene as an example, and from
The configuration of cloud storage platform, three aspects of the storage of service scripts and the retrieval of service scripts propose cloud storage side to the application
Method is described in detail.
1) configuration of cloud storage platform
The configuration of shared pool is introduced first.
In the embodiment of the present application, some memory node equipment in cloud storage platform can constitute resource group.The resource group
In resource can be divided into several resource blocks, several resource blocks marked off can constitute shared pool.Furthermore, it is understood that
Shared pool is the set of several resource blocks.
Configuration when, cloud storage administrative staff can on cloud storage platform typing shared pool information, typing shared pool letter
After breath, shared pool ID can be generated, such as 001, shared pool ID is unique.Then, cloud storage administrative staff can create shared pool.In reality
Now, cloud storage platform can import several resource blocks in shared pool, set up reflecting for several resource blocks and shared pool ID
Penetrate relation, shared pool of the generation with some resource blocks.
It should be noted that above-mentioned resource group can be made up of a memory node equipment, it can also be set by many memory nodes
Standby composition, herein, the composition form progress not to resource group are specifically defined.
The mapping relations of above-mentioned shared pool and resource block can be indicated in a variety of forms, such as can be after resource block ID
Increase shared pool coding, to set up the mapping relations of resource block and shared pool.
For example, above-mentioned shared pool is encoded to 0010, it is assumed that the resource block ID of resource block corresponding with the shared pool is respectively
Res001, res002, res003 etc..After shared pool and the resource block set up mapping relations, it can increase after resource block ID
Shared pool coding is to represent the corresponding relation of resource block and shared pool, and for example increasing the resource block ID after shared pool coding can be
Res001_0010, res002_0010, res003_0010 etc..
Certainly, the mapping relations of above-mentioned shared pool and resource block can be also indicated by mapping table.Here this is to money
Source block and the mapping relations of shared pool carry out exemplary explanation, it are not carried out specifically defined.For example, as shown in figure 4, figure
A kind of schematic diagram of shared pool shown in the exemplary embodiment of 4 the application one.Memory node equipment a, memory node b and storage section
Point c can constitute resource group, and Res represents the resource block in the resource group.Resource in resource group is divided into multiple resource blocks, should
Multiple resource blocks constitute the shared pool for different storage file types.Certainly, Fig. 4 is resource group, and the one of shared pool etc.
Kind exemplarily illustrate, herein not to shared pool, resource group etc. carries out specifically defined.
Secondly, the configuration of service-user is introduced.
Administrative staff can on cloud storage platform configuration service user user profile, in the user profile of the service-user
It may include the title of service-user ID, service-user, the target shared pool distributed for the service-user distributes for the service-user
Resource capacity size, the file type of the file stored by the service-user etc..
Cloud storage platform can also be by the service-user configuration distributing to operation system.Operation system is receiving business use
Family can record the user profile of service-user therein, record the corresponding relation of the service-user and affiliated shared pool with postponing
Deng.
Finally introduce the generation of shared pool coding.
Cloud storage can the user profile based on the service-user, find for the service-user distribute target shared pool.
Cloud storage platform can be according to the file type of the file stored by the service-user and the shared pool ID of the target shared pool, generation
The shared pool coding of the target shared pool.
For example, it is assumed that target shared pool ID is 001, bit identification position can be increased after target shared pool ID, can be passed through
The value of flag represents the file type of the service-user stored file.It is assumed that 0 represents stored by the service-user
File is video file, and 1 represents that the file stored by the service-user is picture file etc..
When cloud storage platform is reading the above-mentioned service-user file to be stored for video file, and it is the business
When the shared pool ID for the target shared pool that user divides is 001, the shared pool coding of cloud storage platform generation can be 0010.
It should be noted that the generating mode for the shared pool coding that the application is proposed is not merely generation shared pool coding,
But carry the file type that the shared pool wants storage file, the life encoded by this shared pool in shared pool coding
Into method, shared pool and the corresponding relation of the file type of its stored file are realized.
The general commonly used method for realizing shared pool and the corresponding relation of the file type of its stored file may include
Set up shared pool and the mapping table of file type.But shared pool and the corresponding relation of file type are set up by mapping table,
, it is necessary to which first looking into mapping table determines shared pool corresponding with file to be retrieved, due to the step for also needing to carry out looking into mapping table during retrieval
Suddenly, particularly with this retrieval scene with mass data of cloud storage platform, the system that can greatly increase cloud storage platform is born
Lotus.
And the application is that the correspondence for realizing shared pool and file type is encoded by shared pool, file to be retrieved is being retrieved
When, the step of without carrying out mapping table search again, the mark of file type to be retrieved is identified in directly being encoded by shared pool
Position identification shared pool corresponding with the file to be retrieved, greatly reduces the load of system, improves effectiveness of retrieval.
2) storage of service scripts
After the service-user on-line message that cloud storage platform receives operation system transmission, cloud storage platform can be look for
The target shared pool of target service user distribution, and can the shared pool based on the target shared pool encode, it is shared for the target
Pond creates one or more corresponding User Catalog.
For example, when the above-mentioned service-user file type to be stored is picture file, above-mentioned target shared pool is encoded to
When 0011, cloud storage platform can be encoded according to the ID and shared pool of service-user, and a use is created for the target shared pool
Family catalogue.
For example, the form of the User Catalog can be the 1179861310 (hash that ID progress hash operations are obtained
Value) _ 0011 (shared pool coding)
When the above-mentioned service-user file type to be stored is video file, above-mentioned target shared pool is encoded to
When 0010, the ID that cloud storage platform can be according to service-user, the multiple equipment of multiple headend equipments corresponding with the user
ID and shared pool coding, multiple User Catalogs are created for the target shared pool.
For example, the form of the User Catalog can be HTS_HC121_51_003 (video camera ID) #825060717 (users
ID carries out the hashed value that hash operations are obtained) _ 0010 (shared pool coding).
After generation User Catalog, cloud storage platform can send the User Catalog generated for the target shared pool to business
System, to be sent the User Catalog to its corresponding front-end camera of service-user by operation system.
For example, when above-mentioned User Catalog is the User Catalog for picture file, cloud storage platform can be by user's mesh
Record is sent to operation system, and operation system can send this User Catalog to all front ends corresponding with the service-user
Video camera.
When above-mentioned User Catalog is the User Catalog for video file, cloud storage platform can be by under multiple User Catalogs
Be sent to operation system, with by operation system based on the front-end camera id information carried in User Catalog, by the plurality of user's mesh
Record is issued to front-end camera corresponding with front-end camera ID respectively.
After front-end camera receives the User Catalog, when carrying out service scripts storage, the User Catalog can be based on,
Service scripts is sent to shared pool corresponding with the User Catalog and stored, and generates the routing information of service scripts.Should
It may include User Catalog in the routing information of service scripts, the information such as filename.Merely just to the routing information of service scripts
Exemplarily illustrated, it is not carried out specifically defined.
For example, for picture file, the routing information of above-mentioned service scripts can be 1179861310_0011/
20150314/S094000E.ts, wherein 1179861310_0011 is User Catalog, and 20150314 be temporal information,
S094000E.ts is filename.
For video file, the routing information of above-mentioned service scripts can be HTS_HC121_51_003#
825060717_0010/20150314/S094000E.ts.Wherein, HTS_HC121_51_003#825060717_0010 is use
Family catalogue, 20150314 be temporal information, and S094000E.ts is filename.
After above-mentioned service scripts routing information is generated, the routing information of the service scripts can be back to by cloud storage platform
Operation system or front-end camera.
3) retrieval of service scripts
In the embodiment of the present application, when service scripts is retrieved, due to cloud storage platform can using two grades search by the way of,
First based on the shared pool coding in User Catalog, shared pool corresponding with the file of the service scripts to be retrieved is searched, then be based on
Filename, matches this document from the shared pool found, rather than each file is matched, therefore substantially increases text
Part retrieval ground efficiency, reduces the live load that retrieval brings memory node equipment.
When realizing, after the service request for obtaining file destination that operation system is issued is received, cloud storage is put down
Platform can obtain the routing information of the file destination carried in the service request.Then, cloud storage platform can enter to the routing information
Row parsing, obtains the information such as User Catalog and filename in the routing information.
Cloud storage platform can be parsed to the User Catalog for the file destination, obtain shared in the User Catalog
Pond is encoded.
Cloud storage platform can be searched and the corresponding shared pool of shared pool coding based on the shared pool coding got.
Find and after the corresponding shared pool of shared pool coding, cloud storage platform can be searched and the text of the file destination in the shared pool
The corresponding file destination of part name.
When realizing, find with after the corresponding shared pool of shared pool coding, cloud storage platform can based on shared pool with
The mapping relations of resource block, search resource block corresponding with the shared pool, and searched and the target in the resource block found
The corresponding file destination of filename of file.
For example, still increasing shared pool coding after resource block ID with above-mentioned, to represent that the mapping of resource block and shared pool is closed
Exemplified by system.
When searching resource block corresponding with the shared pool, the shared pool coding that cloud storage platform can be based on the shared pool,
Search the resource block for carrying shared pool coding.After the resource block for carrying shared pool coding is found, cloud storage platform
File destination corresponding with the filename of the file destination can be searched in the resource block.
After file destination is found, the file destination is can be read in cloud storage platform, and the file destination is returned into business
System, correspondingly business processing is carried out to be based on the file destination by operation system.
For example, with reference to Fig. 5, Fig. 5 is a kind of schematic diagram of service scripts retrieval shown in the exemplary embodiment of the application one.
Assuming that the routing information of the file destination carried in above-mentioned service request is HTS_HC121_51_003#
825060717_0010/20150314/S094000E.ts.Cloud storage platform can parse the routing information, obtain the routing information
In User Catalog, such as HTS_HC121_51_003#825060717_0010, the information such as filename S094000E.ts.
Then, cloud storage platform can parse the User Catalog, obtain the shared pool coding carried in the User Catalog, such as
0010.Cloud storage platform can be searched and the corresponding shared pool of shared pool coding based on shared pool coding.
After the shared pool as shown in Fig. 5 (a) is found, cloud storage platform can the filename based on the file destination,
The file destination matched with file name is searched in the resource of the shared pool, and the file destination found is returned into business system
System, to carry out corresponding business processing by operation system.
In addition, in the embodiment of the present application, due to using shared pool mechanism so that different types of file can be protected independently
Deposit.Therefore, administrative staff can configure different priority, the service scripts stored by the shared pool of different priorities for shared pool
Disaster tolerance rank it is different, cloud storage platform can the priority based on shared pool, the service scripts stored to it carries out not at the same level
Other disaster-tolerant backup.
For example, for traffic police user, picture file is more important than video file, now the traffic police user can will be directed to
The shared pool to store picture file in picture file back up in one or more memory node equipment, with carry out
Disaster tolerance processing.
In the embodiment of the present application, due to by shared pool mechanism by the carry out classification preservation of files in different types so that
Big file and small documents can be stored independently., can be with when calculating the occupied resource capacity of memory node equipment
Respectively occupied resource capacity is obtained according to the different computational methods of big file and small documents.
In the embodiment of the present application, it is proposed that the implementation of the corresponding unique User Catalog of corresponding with picture file shared pool
Mode.
On the one hand, due to shared pool one user of correspondence, each layer of catalogue can be no longer scanned, each layer of catalogue is obtained
Under file shared by the size of resource capacity obtain the occupancy capacity of whole equipment, due to reducing directory scan quantity,
So as to substantially increase the efficiency for the occupied resource capacity for obtaining memory node equipment.
On the other hand, for small documents, such as file of picture file, generally normal size is carrying out small documents
During preservation, cloud storage platform the small documents can be preserved to the small documents size identical fixed capacity size data fragment
In, in scanning, it is not necessary to which each picture file is scanned, but the number of data slot can be directly taken by calculating
The size of data slot is measured and is multiplied by, you can count the size of occupied resource capacity, so as to substantially increase the small text of acquisition
The speed of the occupied resource capacity of part, therefore the occupied resource of the storage device of be stored with big file and small documents can be improved
The speed of procurement of reserve capacity, reduces the live load of memory node device scan occupied resource capacity.
For example, as shown in fig. 6, Fig. 6 is a kind of occupied resource procurement of reserve capacity shown in the exemplary embodiment of the application one
The schematic diagram of method.
It is assumed that T represents catalogue quota capacity, U catalogue occupied resource capacity.It is assumed to be being used for for service-user distribution
The quota capacity of the shared pool of picture file storage is 1T, has two storage devices under the shared pool, then set in every storage
Need to distribute the resource that a size is 512G respectively on standby, data slot slice present on each resource, is 512*
1024/64=8192 slice.
For picture file, picture file can be stored on slice.Hold obtaining using for the User Catalog
Amount, it is assumed that used in scanning to Res1 on 500 slice, Res2 using 610 slice, then U=T-64* (500+
610)。
In addition, in the embodiment of the present application, it is proposed that the unique User Catalog of each shared pool correspondence.Swept due to reducing
The capacity of resource shared by the service scripts under every first class catalogue is retouched, so substantially increasing the acquisition speed of occupied resource capacity
Rate.
The application proposes a kind of method of the file management based on cloud storage platform, by by different memory node equipment institutes
Storage resource in the resource group of composition, is divided into some shared pools for different file types.In retrieval, cloud storage is put down
Platform can receive the service request for obtaining file destination that operation system is issued.Wherein, the mesh is carried in the service request
The User Catalog of file is marked, the User Catalog is encoded including shared pool.Cloud storage platform can parse the User Catalog, obtain institute
Shared pool coding is stated, and searches the shared pool of the shared pool codes match with getting.Cloud storage platform can found
The shared pool in search the file destination, and the file destination is returned to the operation system to carry out corresponding business
Processing.
On the one hand, because cloud storage platform can be based on shared pool, classification storage is carried out to service scripts, in retrieval, cloud
Storage platform, first based on the shared pool coding in User Catalog, can be searched and the file destination by the way of two grades are searched
Shared pool where file type identical file, then based on filename, this document is matched from the shared pool found, without
It is that each file is matched, therefore with substantially increasing document retrieval efficiency, reduce retrieval and bring memory node to set
Standby live load.
On the other hand, due to that in cloud storage, shared pool can be based on, different types of file is stored respectively.
When obtaining occupied resource capacity, to picture file, i.e. small documents, the capacity of each small documents can be no longer scanned, but can
Directly calculate the data slot number taken of the shared pool for picture, you can obtain the capacity of the resource in the shared pool
Occupancy, substantially increases the efficiency of occupied resource capacity.
The third aspect, due to the file of different file types being stored respectively, it is possible to set up for not identical text
The disaster tolerance rank of the file of part type.
Embodiment with the foregoing file management method based on cloud storage platform is corresponding, and present invention also provides based on cloud
The embodiment of the document management apparatus of storage platform.
The embodiment of document management apparatus of the application based on cloud storage platform can be applied on cloud storage platform.Device
Embodiment can be realized by software, can also be realized by way of hardware or software and hardware combining.Exemplified by implemented in software,
As the device on a logical meaning, being will be corresponding in nonvolatile memory by the processor of cloud storage platform where it
Computer program instructions read in internal memory what operation was formed.For hardware view, as shown in fig. 7, being based on for the application
A kind of hardware structure diagram of cloud storage platform where the document management apparatus of cloud storage platform, except the processor shown in Fig. 7, interior
Deposit, outside network outgoing interface and nonvolatile memory, the cloud storage platform in embodiment where device is generally according to the cloud
The actual functional capability of storage platform, can also include other hardware, this is repeated no more.
Fig. 8 is refer to, Fig. 8 is a kind of file management based on cloud storage platform shown in the exemplary embodiment of the application one
The block diagram of device.The cloud storage device is applied to cloud storage platform, and the cloud storage platform includes being set by some memory nodes
The resource group of standby composition；Storage resource in the resource group is divided into some shared pools；Wherein, different shared pools correspondence
Different types of storage file；Described device may include：Receiving unit 810, resolution unit 820, searching unit 830 and return are single
Member 840.
Wherein, above-mentioned receiving unit 810, please for receiving the business for obtaining file destination that operation system is issued
Ask；Wherein, the User Catalog of the file destination is carried in the service request, the User Catalog is encoded including shared pool；
Resolution unit 820, for parsing the User Catalog, obtains the shared pool coding；
Searching unit 830, for the shared pool for the shared pool codes match searched and got；
Returning unit 840, for searching the file destination in the shared pool found, and by the file destination
The operation system is returned to carry out corresponding business processing.
In a kind of optional implementation, it is single that the document management apparatus based on cloud storage platform may also include generation
Member 850, the user profile of pre-configured service-user for obtaining；Wherein, the user profile includes the service-user institute
The file type for the file to be stored and the target shared pool distributed for the service-user, and to be deposited based on the service-user
The shared pool ID of the file type of the file of storage and the target shared pool distributed for the service-user, generates the target and shares
The shared pool coding in pond.
In another optional implementation, the document management apparatus based on cloud storage platform may also include transmission
Unit 860, the target shared pool for when arbitrary target service-user is reached the standard grade, looking for target service user distribution；Base
It is that the target shared pool creates corresponding User Catalog in target shared pool coding；By the User Catalog send to
Headend equipment corresponding with the service-user, to be based on the User Catalog by the headend equipment, by file type with being somebody's turn to do
The corresponding service scripts of User Catalog is stored to the target shared pool.
In another optional implementation, the shared pool has different priority, and shared pool priority is different,
The corresponding disaster tolerance rank of the shared pool is different.
In another optional implementation, the unique User Catalog of each shared pool correspondence；Or, at least with picture
The unique User Catalog of the corresponding each shared pool correspondence of file.
The function of unit and the implementation process of effect specifically refer to correspondence step in the above method in said apparatus
Implementation process, will not be repeated here.
For device embodiment, because it corresponds essentially to embodiment of the method, so related part is real referring to method
Apply the part explanation of example.Device embodiment described above is only schematical, wherein described be used as separating component
The unit of explanation can be or may not be physically separate, and the part shown as unit can be or can also
It is not physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to reality
Selection some or all of module therein is needed to realize the purpose of application scheme.Those of ordinary skill in the art are not paying
In the case of going out creative work, you can to understand and implement.
The preferred embodiment of the application is the foregoing is only, not to limit the application, all essences in the application
God is with principle, and any modification, equivalent substitution and improvements done etc. should be included within the scope of the application protection.
Claims (10)
1. a kind of file management method based on cloud storage platform, it is characterised in that methods described is applied to cloud storage platform, institute
State the resource group that cloud storage platform includes being made up of some memory node equipment；Storage resource in the resource group is divided into
Some shared pools；Wherein, the different types of storage file of different shared pools correspondence；Methods described includes：
Receive the service request for obtaining file destination that operation system is issued；Wherein, carry described in the service request
The User Catalog of file destination, the User Catalog is encoded including shared pool；
The User Catalog is parsed, the shared pool coding is obtained；
The shared pool for the shared pool codes match searched and got；
The file destination is searched in the shared pool found, and the file destination is returned into the operation system.
2. according to the method described in claim 1, it is characterised in that methods described also includes：
Obtain the user profile of pre-configured service-user；Wherein, the user profile to be stored including the service-user
File file type and for the service-user distribute target shared pool；
File type and the target for service-user distribution based on the service-user file to be stored are shared
The shared pool ID in pond, generates the shared pool coding of the target shared pool.
3. method according to claim 2, it is characterised in that methods described also includes：
When arbitrary target service-user is reached the standard grade, the target shared pool of target service user distribution is look for；
It is that the target shared pool creates corresponding User Catalog based on target shared pool coding；
The User Catalog is sent to headend equipment corresponding with the service-user, it is described to be based on by the headend equipment
User Catalog, file type service scripts corresponding with the User Catalog is stored to the target shared pool.
4. according to the method described in claim 1, it is characterised in that the shared pool has different priority, shared pool is excellent
First level is different, and the corresponding disaster tolerance rank of the shared pool is different.
5. according to the method described in claim 1, it is characterised in that the unique User Catalog of each shared pool correspondence；Or, extremely
The corresponding unique User Catalog of few each shared pool corresponding with picture file.
6. a kind of document management apparatus based on cloud storage platform, it is characterised in that described device is applied to cloud storage platform, institute
State the resource group that cloud storage platform includes being made up of some memory node equipment；Storage resource in the resource group is divided into
Some shared pools；Wherein, the different types of storage file of different shared pools correspondence；Described device includes：
Receiving unit, for receiving the service request for obtaining file destination that operation system is issued；Wherein, the business please
The middle User Catalog for carrying the file destination is sought, the User Catalog is encoded including shared pool；
Resolution unit, for parsing the User Catalog, obtains the shared pool coding；
Searching unit, for the shared pool for the shared pool codes match searched and got；
Returning unit, institute is returned for searching the file destination in the shared pool found, and by the file destination
Operation system is stated to carry out corresponding business processing.
7. device according to claim 6, it is characterised in that described device also includes：
Generation unit, the user profile of pre-configured service-user for obtaining；Wherein, the user profile includes the business
The file type of user's file to be stored and the target shared pool distributed for the service-user, and based on the service-user
The shared pool ID of the file type for the file to be stored and the target shared pool distributed for the service-user, generates the mesh
Mark the shared pool coding of shared pool.
8. device according to claim 7, it is characterised in that described device also includes：
Transmitting element, the target for when arbitrary target service-user is reached the standard grade, looking for target service user distribution is shared
Pond；It is that the target shared pool creates corresponding User Catalog based on target shared pool coding；The User Catalog is sent out
Headend equipment corresponding with the service-user is delivered to, to be based on the User Catalog by the headend equipment, by file type
Service scripts corresponding with the User Catalog is stored to the target shared pool.
9. device according to claim 6, it is characterised in that the shared pool has different priority, shared pool is excellent
First level is different, and the corresponding disaster tolerance rank of the shared pool is different.
10. device according to claim 6, it is characterised in that the unique User Catalog of each shared pool correspondence；Or,
The corresponding unique User Catalog of each shared pool at least corresponding with picture file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710269824.2A CN107092686B (en) | 2017-04-24 | 2017-04-24 | File management method and device based on cloud storage platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710269824.2A CN107092686B (en) | 2017-04-24 | 2017-04-24 | File management method and device based on cloud storage platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107092686A true CN107092686A (en) | 2017-08-25 |
| CN107092686B CN107092686B (en) | 2020-04-10 |
Family
ID=59637697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710269824.2A Active CN107092686B (en) | 2017-04-24 | 2017-04-24 | File management method and device based on cloud storage platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107092686B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107943423A (en) * | 2017-12-07 | 2018-04-20 | 郑州云海信息技术有限公司 | The management method and computer-readable recording medium of storage resource in cloud system |
| CN109241053A (en) * | 2018-07-27 | 2019-01-18 | 阿里巴巴集团控股有限公司 | A kind of mark code distributing method, device and server |
| CN109669790A (en) * | 2018-12-17 | 2019-04-23 | 北京锐安科技有限公司 | Data sharing method, device, shared platform and storage medium based on cloud platform |
| CN111125475A (en) * | 2019-12-26 | 2020-05-08 | 深圳前海环融联易信息科技服务有限公司 | Shared image management method and device, computer equipment and storage medium |
| CN111881305A (en) * | 2020-06-23 | 2020-11-03 | 上海博泰悦臻电子设备制造有限公司 | Scanning method and related equipment |
| CN112084250A (en) * | 2020-09-15 | 2020-12-15 | 深圳市宝能投资集团有限公司 | Data storage method, data query method and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101354726A (en) * | 2008-09-17 | 2009-01-28 | 中国科学院计算技术研究所 | Method for managing memory metadata of cluster file system |
| CN103823760A (en) * | 2014-03-03 | 2014-05-28 | 山西科泰微技术有限公司 | High-efficiency data storage and analysis method |
| CN104660690A (en) * | 2015-02-06 | 2015-05-27 | 中国农业大学 | Cloud video service monitoring system |
| CN104991742A (en) * | 2015-07-01 | 2015-10-21 | 北京奇虎科技有限公司 | Cloud resource classifying and storing method and apparatus |
| CN105005611A (en) * | 2015-07-10 | 2015-10-28 | 中国海洋大学 | File management system and file management method |
| CN105204782A (en) * | 2015-10-13 | 2015-12-30 | 中国联合网络通信集团有限公司 | Data storage achieving method and device |
| CN105224392A (en) * | 2015-10-13 | 2016-01-06 | 中国联合网络通信集团有限公司 | A kind of virtual computing resource quota management method and platform |
-
2017
- 2017-04-24 CN CN201710269824.2A patent/CN107092686B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101354726A (en) * | 2008-09-17 | 2009-01-28 | 中国科学院计算技术研究所 | Method for managing memory metadata of cluster file system |
| CN103823760A (en) * | 2014-03-03 | 2014-05-28 | 山西科泰微技术有限公司 | High-efficiency data storage and analysis method |
| CN104660690A (en) * | 2015-02-06 | 2015-05-27 | 中国农业大学 | Cloud video service monitoring system |
| CN104991742A (en) * | 2015-07-01 | 2015-10-21 | 北京奇虎科技有限公司 | Cloud resource classifying and storing method and apparatus |
| CN105005611A (en) * | 2015-07-10 | 2015-10-28 | 中国海洋大学 | File management system and file management method |
| CN105204782A (en) * | 2015-10-13 | 2015-12-30 | 中国联合网络通信集团有限公司 | Data storage achieving method and device |
| CN105224392A (en) * | 2015-10-13 | 2016-01-06 | 中国联合网络通信集团有限公司 | A kind of virtual computing resource quota management method and platform |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107943423A (en) * | 2017-12-07 | 2018-04-20 | 郑州云海信息技术有限公司 | The management method and computer-readable recording medium of storage resource in cloud system |
| CN109241053A (en) * | 2018-07-27 | 2019-01-18 | 阿里巴巴集团控股有限公司 | A kind of mark code distributing method, device and server |
| CN109241053B (en) * | 2018-07-27 | 2022-08-23 | 创新先进技术有限公司 | Identification code allocation method, device and server |
| CN109669790A (en) * | 2018-12-17 | 2019-04-23 | 北京锐安科技有限公司 | Data sharing method, device, shared platform and storage medium based on cloud platform |
| CN111125475A (en) * | 2019-12-26 | 2020-05-08 | 深圳前海环融联易信息科技服务有限公司 | Shared image management method and device, computer equipment and storage medium |
| CN111881305A (en) * | 2020-06-23 | 2020-11-03 | 上海博泰悦臻电子设备制造有限公司 | Scanning method and related equipment |
| CN111881305B (en) * | 2020-06-23 | 2024-03-01 | 博泰车联网科技(上海)股份有限公司 | Scanning method and related equipment |
| CN112084250A (en) * | 2020-09-15 | 2020-12-15 | 深圳市宝能投资集团有限公司 | Data storage method, data query method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107092686B (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107092686A (en) | A kind of file management method and device based on cloud storage platform | |
| CN101449559B (en) | Distributed storage | |
| US6484198B1 (en) | Method and device for automated transfer and maintenance of internet based information | |
| AU757667B2 (en) | Access to content addressable data over a network | |
| CN111736775B (en) | Multi-source storage method, device, computer system and storage medium | |
| US6742023B1 (en) | Use-sensitive distribution of data files between users | |
| US7506157B2 (en) | Access to content addressable data over a network | |
| CN100499596C (en) | Instant messaging object store | |
| CN102067617B (en) | Methods and apparatus for self-organized caching in a content delivery network | |
| US6363375B1 (en) | Classification tree based information retrieval scheme | |
| CN100346324C (en) | Analysis method for user's request | |
| CN107835437B (en) | Dispatching method based on more cache servers and device | |
| CN102726031A (en) | Content processing method, device and system | |
| CN102571720A (en) | Method and device for processing heterogeneous information contents | |
| WO2002078276A1 (en) | A method for transmission of classified and prioritised information | |
| CN103858124A (en) | Method for data maintenance | |
| WO2007081149A1 (en) | Svc file data sharing method and svc file thereof | |
| CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
| US20100306687A1 (en) | System and method for avatar management | |
| CN109005433B (en) | A kind of video cloud service platform architecture and implementation method | |
| US20020078461A1 (en) | Incasting for downloading files on distributed networks | |
| CN107276912B (en) | Memory, message processing method and distributed storage system | |
| US20060075063A1 (en) | Method and apparatus for providing data storage in peer-to peer networks | |
| CN107122136A (en) | A kind of capacity acquiring method and device | |
| CN101322383B (en) | Method and peer network for ascertaining the peer network originating station for a file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221104 Address after: 250101 4th floor, building 5, zone A2, Hanyu Jingu, Shunhua South Road, high tech Zone, Jinan City, Shandong Province Patentee after: Jinan Yushi Intelligent Technology Co.,Ltd. Address before: 310051 Zhejiang Jiangling Hangzhou Road, Binjiang District, Jiangling, 88, No. 10 South Block 1-11. Patentee before: ZHEJIANG UNIVIEW TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |