CN109828828A - Object method for uploading, device, electronic equipment and machine readable storage medium - Google Patents
Object method for uploading, device, electronic equipment and machine readable storage medium Download PDFInfo
- Publication number
- CN109828828A CN109828828A CN201910062436.6A CN201910062436A CN109828828A CN 109828828 A CN109828828 A CN 109828828A CN 201910062436 A CN201910062436 A CN 201910062436A CN 109828828 A CN109828828 A CN 109828828A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- task
- index
- bucket
- fragments
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
Object method for uploading, device, electronic equipment and the machine readable storage medium of the application offer object storage system.In this application, object to be uploaded is obtained, task is segmented for the Object Creation, obtains the task identification of the segmentation task；The object is segmented in the segmentation task by preset strategy, obtains segmentation object, and distribute corresponding number-of-fragments for the segmentation object, wherein the number-of-fragments are used to indicate the sequence of the segmentation object in the object；Based on the cryptographic Hash for including at least the task identification and the number-of-fragments, the segmentation object is uploaded to the bucket, realize is more than that the object of 5GB uploads in object storage system, take full advantage of multiple storage medium capacity, in addition, by concurrently uploading, realizes transfer efficiency and greatly improve.
Description
Technical field
This application involves object method for uploading, device, the electronics of technical field of memory more particularly to object storage system to set
Standby and machine readable storage medium.
Background technique
With the fast development of Internet application, PB grades even EB grades of mass data storage becomes particularly important.Object
For storage system as a kind of novel distributed memory system, object is the primary entity in object storage system, passes through offer
Object-based access interface can store any type of data, such as: picture, video, audio, text etc..Object storage system
System efficiently solves the problems such as conventional store sharing capability is limited, scalability is poor.
Summary of the invention
The application provides a kind of object method for uploading of object storage system, and the object storage system includes at least bucket,
Wherein, the bucket is the container for saving the object, which is characterized in that the described method includes:
The object is obtained, task is segmented for the Object Creation, obtains the task identification of the segmentation task, wherein
The task identification is for being segmented task described in unique identification；
The object is segmented in the segmentation task by preset strategy, segmentation object is obtained, and is described point
Section object distributes corresponding number-of-fragments, wherein the number-of-fragments are used to indicate the segmentation object in the object
Sequentially；
Based on the cryptographic Hash for including at least the task identification and the number-of-fragments, the segmentation object is uploaded to institute
State bucket.
Optionally, it is described the segmentation object is uploaded to the bucket while, further includes:
Create or update the index of the object, wherein the index is used to indicate the segmentation object in the bucket
Storage location.
Optionally, the index includes at least indexing head, index information, wherein it is described right that the indexing head includes at least
The object name of elephant, the bucket and the task identification, the index information include at least the number-of-fragments.
Optionally, when the designated capabilities range of the object to be downloaded, further includes:
Based on the index and the designated capabilities range, the corresponding number-of-fragments of the object designated capabilities range are obtained
And its indicated segmentation object；
The indicated segmentation object of the designated capabilities range corresponding segments number of the object is locally downloading.
Optionally, when to be uploaded with the object object name when duplicate new object, by the segmentation of the new object
While object is uploaded to the bucket, further includes:
The index for creating the new object deletes the index and the corresponding segmentation object of the index of the object.
The application also proposes that a kind of object of object storage system uploads device, and the object storage system includes at least
Bucket, wherein the bucket is container for saving the object, which is characterized in that described device includes:
Creation module is segmented task for the Object Creation, obtains the segmentation task for obtaining the object
Task identification, wherein the task identification is for being segmented task described in unique identification；
The object is segmented in the segmentation task by preset strategy by the creation module, obtains segmentation pair
As, and corresponding number-of-fragments are distributed for the segmentation object, wherein the number-of-fragments are used to indicate the segmentation object and exist
Sequence in the object；
Uploading module, for that described will divide based on the cryptographic Hash for including at least the task identification and the number-of-fragments
Section object is uploaded to the bucket.
Optionally, it is described the segmentation object is uploaded to the bucket while, further includes:
The creation module is further, for creating or updating the index of the object, wherein the index is used to indicate
Storage location of the segmentation object in the bucket.
Optionally, the index includes at least indexing head, index information, wherein it is described right that the indexing head includes at least
The object name of elephant, the bucket and the task identification, the index information include at least the number-of-fragments.
Optionally, when the designated capabilities range of the object to be downloaded, further includes:
The uploading module is further, based on the index and the designated capabilities range, obtains the specified appearance of the object
Measure the corresponding number-of-fragments of range and its indicated segmentation object；
Download module, the segmentation object indicated for the designated capabilities range corresponding segments number by the object are downloaded
To local.
Optionally, when to be uploaded with the object object name when duplicate new object, by the segmentation of the new object
While object is uploaded to the bucket, further includes:
The creation module is further, creates the index of the new object, deletes the index and the rope of the object
Draw corresponding segmentation object.
The application also provides a kind of electronic equipment, including communication interface, processor, memory and bus, and the communication connects
Pass through bus between mouth, the processor and the memory to be connected with each other；
Machine readable instructions are stored in the memory, the processor is executed by calling the machine readable instructions
Above-mentioned method.
The application also provides a kind of machine readable storage medium, and the machine readable storage medium is stored with machine readable finger
It enables, the machine readable instructions realize the above method when being called and being executed by processor.
By above embodiments, object storage system includes at least bucket, wherein the bucket is for uploading the object
Container obtains object to be uploaded, is segmented task for the Object Creation, obtains the task identification of the segmentation task,
Described in task identification for being segmented task described in unique identification；By the object in the segmentation task by preset strategy into
Row segmentation obtains segmentation object, and distributes corresponding number-of-fragments for the segmentation object, wherein the number-of-fragments are used for
Indicate the sequence of the segmentation object in the object；Based on the Kazakhstan for including at least the task identification and the number-of-fragments
Uncommon value, is uploaded to the bucket for the segmentation object, realizes the object more than 5GB and upload in object storage system, sufficiently
Multiple storage medium capacity are utilized, in addition, realizing transfer efficiency by concurrently uploading and greatly improving.
Detailed description of the invention
Fig. 1 is a kind of flow chart of the object method for uploading for object storage system that an exemplary embodiment provides.
Fig. 2 is a kind of object download flow chart for object storage system that an exemplary embodiment provides.
Fig. 3 is a kind of block diagram of the upload device of the object for object storage system that an exemplary embodiment provides.
Fig. 4 is the hardware structure diagram for a kind of electronic equipment that an exemplary embodiment provides.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the application.
It is only to be not intended to be limiting the application merely for for the purpose of describing particular embodiments in term used in this application.
It is also intended in the application and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the application
A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from
In the case where the application range, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination ".
In order to make those skilled in the art more fully understand the technical solution in the embodiment of the present application, first to the application reality
A relevant technologies for referent storage system are applied, are briefly described.
In some scenes, existing object storage system is only supported on the non-more segmentation interfaces of the single PUT or POST passed through
Pass the object for being not more than 5GB.If the non-more segmentation interfaces of the single PUT or POST that user needs to pass through, upload super more than one
The object of 5GB is crossed into object storage system, then cannot achieve.
Based on this, the application proposes that a kind of object of object storage system uploads scheme, and the program is stored applied to object
The management node of system, the management node obtain it is to be uploaded be more than 5GB object, for the Object Creation segmentation appoint
Business, obtains the task identification of the segmentation task, wherein the task identification is for being segmented task described in unique identification；It will be described
Object is segmented in the segmentation task by preset strategy, obtains segmentation object, and correspond to for segmentation object distribution
Number-of-fragments, wherein the number-of-fragments are used to indicate the sequence of the segmentation object in the object；Based at least wrapping
The segmentation object is uploaded to the bucket, realized more than 5GB by the cryptographic Hash for including the task identification and the number-of-fragments
Object upload in object storage system, multiple storage medium capacity are taken full advantage of, in addition, realizing by concurrently uploading
Upper transfer efficiency greatly improves.
The application is described below by specific embodiment and in conjunction with specific application scenarios.
Referring to FIG. 1, Fig. 1 is a kind of object method for uploading for object storage system that one embodiment of the application provides, answer
For the management node of the object storage system, subsequent descriptions abbreviation management node；The object storage system includes at least
Bucket, wherein the bucket is the container for saving the object, and the above method executes following steps:
Step 102 obtains the object, is segmented task for the Object Creation, obtains the task of the segmentation task
Mark, wherein the task identification is for being segmented task described in unique identification.
The bucket is based at least one storage medium, and storage medium here refers to Jie of the actual storage of object
Matter, such as: the storage medium can be on one or more disks or one or more RAID or one or more RAID
Book etc., wherein the disk can be IDE (Integrated Drive Electronics) mechanical hard disk or SATA
(Serial Advanced Technology Attachment) mechanical hard disk or SAS (Serial Attached SCSI) machine
Tool hard disk or SCSI (Small Computer System Interface) mechanical hard disk or SATA solid state hard disk or SAS solid-state
Hard disk, any kind of storage dish (such as CD, DVD) or flash memory or nonvolatile memory or similar storage medium,
Perhaps their combination or based on it is above-mentioned may multiple and different disk types or multiple disks or their combination creation
RAID, in this application without being particularly limited to.Here object refers to the object in the object storage system,
The capacity of the object can be more than preset threshold 5GB, subsequent referred to as blob, such as: the blob can be one section and hold
Continuous audio and video data streams.In addition, application scheme can also be less than the object of 5GB with compatible processing, subsequent abbreviation is less than 5GB's
Object is plain objects.In order to keep the compatibility with original object storage system plain objects, for the blob for being more than 5GB,
The attribute for extending the object, increasing an attribute is Is_Larget_Object, if an object is blob, Is_
Larget_Object is 1, and otherwise Is_Larget_Object is 0.
Specifically, management node creates a bucket B in the object storage system in advance, wherein B is for preservation pair
The container of elephant, a bucket can be used to save an object, can be used for saving multiple objects；Management node obtains one big
Object L creates segmentation task for the blob, obtains the task identification U of the segmentation task, and wherein U is according to certain random
Algorithm generates, and the event of B is this time uploaded to for unique identification L.Such as: it may be deposited for the corresponding blob of the same name
It in the scene repeatedly uploaded, uploads for the first time, creates segmentation task for the blob, obtain the task identification of the segmentation task
U1；Second of upload creates segmentation task for the blob, obtains the task identification U2 of the segmentation task, wherein U2 and U1
Value it is not identical.For another example: the task identification for multiple and different corresponding segmentation tasks of blob L3, L4 is U3,
U4, then the value of U3 and U4 is not identical.
The object is segmented in the segmentation task by preset strategy by step 104, obtains segmentation object, and
Corresponding number-of-fragments are distributed for the segmentation object, wherein the number-of-fragments are used to indicate the segmentation object described
Sequence in object.
Here task identification U unique identification of the segmentation task as the segmentation as described in above-mentioned steps 102, this object for being
Refer to the blob as described in above-mentioned steps 102, is also possible to plain objects.Here by preset strategy carry out segmentation refer to
Management node is directed to the blob, is sized and is divided into multiple segmentations, such as: single split capacity C is not less than 5MB
And C is not more than 5GB, the single split size may be the same or different, be not especially limited in this application.
Specifically, such as: it is example that management node, which is mutually all 1GB according to single split size, is 6TB by above-mentioned capacity
Blob obtains 6144 segmentations after being split, wherein the size of each segmentation is 1GB, management node obtains above-mentioned 6144
The number-of-fragments of segmentation, such as: the consecution natural number that the number-of-fragments are from 1 to 6144, the number-of-fragments are used to indicate
The sequence of the segmentation object in the object.
Step 106, based on the cryptographic Hash for including at least the task identification and the number-of-fragments, by the segmentation object
It is uploaded to the bucket.
Specifically, continue by taking above step as an example, management node obtains the segmentation task identification U as described in step 102, such as
6144 segmentation objects described in step 104 and corresponding 6144 number-of-fragments.Management node is directed to multiple segmentation objects
It is uploaded respectively, specific upload can be sequence, be also possible to concurrent.In a kind of preferred mode, management node
Multiple segmentation objects can concurrently be uploaded into the corresponding bucket of the object.When each segmentation object uploads, management section
Object name of the point based on the blob L, the corresponding bucket name of bucket B, the task identification U, be each segmentation in advance minute
The cryptographic Hash for four parameters of number-of-fragments matched calculates the corresponding key of each segmentation object, wherein the key is in blob
The key assignments index of unique one segmentation object of instruction.Since the number-of-fragments of each segmentation are different, then each segmentation object
Corresponding key is different, by inquiring the corresponding of the corresponding key of each segmentation object some storage medium corresponding with bucket B
Each segmentation object is uploaded to different storage mediums by relationship, so as to realize not by some storage medium capacity limit,
Multiple storage medium volume spaces of the corresponding object storage system of bucket B are utilized to greatest extent, and due to be uploaded
Multiple segmentation objects of blob be dispersed on multiple storage mediums, may be implemented concurrently to upload, improve the upload of object
Efficiency.
The corresponding multiple segmentation objects of the blob L are uploaded to the bucket in above-mentioned management node and correspond to different storages
While medium, management node creates or updates the index of the object, wherein the index is used to indicate the segmentation object
In the storage location of the bucket；The index includes at least indexing head, index information, wherein the indexing head includes at least institute
Object name, the bucket and the task identification of object are stated, the index information includes at least the number-of-fragments.
Multiple segmentation objects of above-mentioned blob do not execute union operation after uploading to the bucket.The merging behaviour
Work is that in the prior art, after usually multiple segmentation objects upload, multiple segmentation objects can be merged into one again by management node
Blob.Due to needing to merge into a blob, need successively to copy original segmentation to blob correspondence by number-of-fragments
The parking space of bucket, that is, need double memory space；Therefore be easy to appear and be limited to storage medium capacity impact, upload multiple points
Section object success, but merge blob and can not necessarily succeed, and merge blob operation and can influence segmentation object than relatively time-consuming
Upper transfer efficiency.And in the application, management node saves the overhead for merging segmentation object and saving by only generating index
The memory space of half, thus more time saving save space.
In some scenes, when to be uploaded with the object object name when duplicate new object, management node is by institute
While stating the segmentation object of new object and be uploaded to the bucket, the index of the new object is created, the index of the object is deleted
And the corresponding segmentation object of the index.
Specifically, such as: management node by the corresponding segmentation object of blob L be uploaded to corresponding bucket after, greatly
The task identification of the corresponding segmentation task of object L is U, as the duplicate new object L1 to be uploaded with the object name of blob L,
The task identification of the corresponding segmentation task of new object L1 is U1, and management node obtains the attribute Is_Larget_ of new object L1 first
Object, and check that the value for being Is_Larget_Object shows a new object L1 if Is_Larget_Object is 1
For blob；If otherwise Is_Larget_Object is 0, show that a new object L1 is a plain objects.Secondly, carrying out
Judgement, compared with flow processing, covers old blob L of the same name, i.e., first deletes of the same name old if new object L1 is plain objects
All segmentation objects of blob L and its manipulative indexing, it is rear that new object L1 is written；If new object L1 is blob, due to two
The task identification that object L, L1 respectively correspond segmentation task is U and U1, and wherein U and U1 is different, then management node uploads new object
When L1, the corresponding segmentation object of L1 will not cover the corresponding segmentation object of L；Finally, management node is generating new object L1 index
Meanwhile need to delete object L of the same name index and its corresponding all segmentation objects.
Under other scene, management node after the segmentation object of blob is uploaded to corresponding bucket, when to
When downloading the designated capabilities range of the object, management node executes example as shown in Figure 2, executes following steps:
Step 202 is based on the index and the designated capabilities range, and it is corresponding to obtain the object designated capabilities range
Number-of-fragments and its indicated segmentation object.
Specifically, continuing to continue to illustrate with example described in step 104, the blob L of 6TB presses mono- fragment size of 1GB,
Multiple segmentation objects 1,2,3......6143,6144 are divided into, altogether the segmentation object of 6144 1G sizes.Exemplary one
In kind embodiment, if storing equipment when the partial data of the 1G of blob to be downloaded to 100G range and being based on institute
Designated capabilities range, i.e. 1G to 100G are stated, is obtained 1 corresponding to the corresponding number-of-fragments of the above-mentioned 100G capacity of the blob
To 100 segmentation object, management node first obtains the index of blob as described in step 106, wherein the index is for referring to
Show the segmentation object in the storage location of the bucket；The index includes at least indexing head, index information, wherein the rope
Leader includes at least object name, the bucket and the task identification of the object, and the index information includes at least described point
Segment number obtains the segmentation object in the storage location of the bucket by the number-of-fragments 1,2,3.......100.
The indicated segmentation object of the designated capabilities range corresponding segments number of the object is downloaded to this by step 204
Ground.
Specifically, in a kind of exemplary embodiment, when the 1G of blob to be downloaded is to the part of 100G range
When data, management node is based on the designated capabilities range, i.e. 1G to 100G obtains the above-mentioned 100G capacity of the blob
1 to 100 segmentation object corresponding to corresponding number-of-fragments, by the number-of-fragments 1,2, the corresponding segmentation of 3.......100
Object download is to locally.In exemplary another embodiment, if when the range of capacity to be downloaded of not specified blob,
It stores equipment default and downloads all segmentations, management node can obtain all segmentation objects by index, download all segmentations.
So far, process shown in FIG. 1 is completed, can be seen that the management section of object storage system by process shown in FIG. 1
Point obtains object to be uploaded, is segmented task for the Object Creation, obtains the task identification of the segmentation task, wherein institute
Task identification is stated for being segmented task described in unique identification；The object is divided in the segmentation task by preset strategy
Section obtains segmentation object, and distributes corresponding number-of-fragments for the segmentation object, wherein the number-of-fragments are used to indicate
The sequence of the segmentation object in the object；Based on the Hash for including at least the task identification and the number-of-fragments
Value, is uploaded to the bucket for the segmentation object.
Using the above embodiments of the present application, by the cryptographic Hash based on the task identification and the number-of-fragments, by institute
It states segmentation object and is uploaded to the bucket, on the one hand realize the object more than 5GB and upload in object storage system, make full use of
On the other hand multiple storage medium capacity by concurrently uploading, realize transfer efficiency and greatly improve.In addition, more than 5GB
Object after segmentation uploads to the object storage system, and nonjoinder is saved by only generating the index of the object
Merge expense, more saves time and memory space.
Fig. 3 is that a kind of object for object storage system that one exemplary embodiment of the application provides uploads the block diagram of device.
It is corresponding with above method embodiment, present invention also provides a kind of object of object storage system upload device embodiment,
The object storage system includes at least bucket, wherein the bucket is the container for saving the object, is please referred to shown in Fig. 3
A kind of object of object storage system of example uploads device 30, and described device includes:
Creation module 301 is segmented task for the Object Creation for obtaining the object, obtains the segmentation and appoints
The task identification of business, wherein the task identification is for being segmented task described in unique identification；
The object is segmented in the segmentation task by preset strategy, is segmented by the creation module 301
Object, and corresponding number-of-fragments are distributed for the segmentation object, wherein the number-of-fragments are used to indicate the segmentation object
Sequence in the object；
Uploading module 302 will be described for based on the cryptographic Hash for including at least the task identification and the number-of-fragments
Segmentation object is uploaded to the bucket.
In the present embodiment, it is described the segmentation object is uploaded to the bucket while, further includes:
The creation module 301 is further, for creating or updating the index of the object, wherein the index is used for
Indicate the segmentation object in the storage location of the bucket.
In the present embodiment, the index includes at least indexing head, index information, wherein the indexing head includes at least
The object name of the object, the bucket and the task identification, the index information include at least the number-of-fragments.
In the present embodiment, when the designated capabilities range of the object to be downloaded, further includes:
The uploading module 302 further, based on the index and the designated capabilities range, it is specified to obtain the object
The corresponding number-of-fragments of range of capacity and its indicated segmentation object；
Download module 303, the segmentation object indicated for the designated capabilities range corresponding segments number by the object
It is locally downloading.
In the present embodiment, when to be uploaded with the object object name when duplicate new object, by the new object
Segmentation object while be uploaded to the bucket, further includes:
The creation module 301 further, creates the index of the new object, deletes the index of the object and described
Index corresponding segmentation object.
For device embodiment, since it corresponds essentially to embodiment of the method, so related place is referring to method reality
Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separation unit
The module of explanation may or may not be physically separated, and the component shown as module can be or can also be with
It is not physical module, it can it is in one place, or may be distributed on multiple network modules.It can be according to actual
The purpose for needing to select some or all of the modules therein to realize application scheme.Those of ordinary skill in the art are not paying
Out in the case where creative work, it can understand and implement.
System, device, module or the module that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.A kind of typically to realize that equipment is computer, the concrete form of computer can
To be personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
In device, navigation equipment, E-mail receiver/send equipment, game console, tablet computer, wearable device or these equipment
The combination of any several equipment.
The embodiment that the object of the object storage system of the application uploads device can apply electronic equipment shown in Fig. 4
On.Installation practice can also be realized by software realization by way of hardware or software and hardware combining.With software reality
It is that machine readable storage is situated between by the processor of electronic equipment where it as the device on a logical meaning for existing
Corresponding computer program instructions run the machine-executable instruction of formation in matter after reading.For hardware view, such as Fig. 4 institute
Show, is a kind of hardware structure diagram of electronic equipment where the object of the object storage system of the application uploads device, in addition to Fig. 4 institute
Except the processor, communication interface, bus and the machine readable storage medium that show, the electronic equipment in embodiment where device is logical
Often according to the actual functional capability of the electronic equipment, it can also include other hardware, this is repeated no more.
Accordingly, it the embodiment of the present application also provides the hardware configuration of a kind of electronic equipment of Fig. 3 shown device, refers to
Fig. 4, Fig. 4 are the hardware structural diagram of a kind of electronic equipment provided by the embodiments of the present application.The equipment includes: communication interface
401, processor 402, machine readable storage medium 403 and bus 404；Wherein, communication interface 401, processor 402, machine can
It reads storage medium 403 and mutual communication is completed by bus 404.Wherein, communication interface 401, for carrying out network communication.Place
Reason device 402 can be a central processing unit (CPU), and processor 402 can execute to be stored in machine readable storage medium 403
Machine readable instructions, to realize process as described above.
Machine readable storage medium 403 referred to herein can be any electronics, magnetism, optics or other physical stores
Device may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium may is that easily
Lose memory, nonvolatile memory or similar storage medium.Specifically, machine readable storage medium 403 can be RAM
(Radom Access Memory, random access memory), flash memory, memory driver (such as hard disk drive), solid state hard disk,
Any kind of storage dish (such as CD, DVD) perhaps similar storage medium or their combination.
So far, hardware configuration description shown in Fig. 4 is completed.
In addition, the embodiment of the present application also provides a kind of machine readable storage medium including machine-executable instruction, example
Such as the machine-readable readable storage medium storing program for executing 403 in Fig. 4, the machine-executable instruction can be by data processing equipment
Device 402 is managed to execute to realize data processing method described above.
The function of each unit and the realization process of effect are specifically detailed in the above method and correspond to step in above-mentioned apparatus
Realization process, details are not described herein.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the application
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the application, these modifications, purposes or
Person's adaptive change follows the general principle of the application and including the undocumented common knowledge in the art of the application
Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the application are by following
Claim is pointed out.
It should be understood that the application is not limited to the precise structure that has been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.Scope of the present application is only limited by the accompanying claims.
The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.
Claims (12)
1. a kind of object method for uploading of object storage system, the object storage system includes at least bucket, wherein the bucket is
For saving the container of the object, which is characterized in that the described method includes:
The object is obtained, task is segmented for the Object Creation, obtains the task identification of the segmentation task, wherein described
Task identification is for being segmented task described in unique identification；
The object is segmented in the segmentation task by preset strategy, segmentation object is obtained, and is the segmentation pair
As distributing corresponding number-of-fragments, wherein the number-of-fragments are used to indicate the sequence of the segmentation object in the object；
Based on the cryptographic Hash for including at least the task identification and the number-of-fragments, the segmentation object is uploaded to described
Bucket.
2. the method according to claim 1, wherein the segmentation object is uploaded to the same of the bucket described
When, further includes:
Create or update the index of the object, wherein the index is used to indicate the segmentation object in the storage of the bucket
Position.
3. according to the method described in claim 2, it is characterized in that, it is described index include at least indexing head, index information,
In, the indexing head includes at least object name, the bucket and the task identification of the object, and the index information at least wraps
Include the number-of-fragments.
4. according to the method described in claim 3, it is characterized in that, being gone back when the designated capabilities range of the object to be downloaded
Include:
Based on it is described index and the designated capabilities range, obtain the corresponding number-of-fragments of the object designated capabilities range and its
Indicated segmentation object；
The indicated segmentation object of the designated capabilities range corresponding segments number of the object is locally downloading.
5. according to the method described in claim 3, it is characterized in that, when to be uploaded duplicate new right with the object name of the object
As when, while the segmentation object of the new object is uploaded to the bucket, further includes:
The index for creating the new object deletes the index and the corresponding segmentation object of the index of the object.
6. the object of object storage system a kind of uploads device, the object storage system includes at least bucket, wherein the bucket is
For saving the container of the object, which is characterized in that described device includes:
Creation module is segmented task for the Object Creation, obtains the task of the segmentation task for obtaining the object
Mark, wherein the task identification is for being segmented task described in unique identification；
The object is segmented in the segmentation task by preset strategy by the creation module, obtains segmentation object, and
Corresponding number-of-fragments are distributed for the segmentation object, wherein the number-of-fragments are used to indicate the segmentation object described
Sequence in object；
Uploading module, for based on the cryptographic Hash for including at least the task identification and the number-of-fragments, by the segmentation pair
As being uploaded to the bucket.
7. device according to claim 6, which is characterized in that the segmentation object is uploaded to the same of the bucket described
When, further includes:
The creation module is further, for creating or updating the index of the object, wherein the index is used to indicate described
Storage location of the segmentation object in the bucket.
8. device according to claim 7, which is characterized in that the index includes at least indexing head, index information,
In, the indexing head includes at least object name, the bucket and the task identification of the object, and the index information at least wraps
Include the number-of-fragments.
9. device according to claim 8, which is characterized in that when the designated capabilities range of the object to be downloaded, also
Include:
The uploading module is further, based on the index and the designated capabilities range, obtains the object designated capabilities model
Enclose corresponding number-of-fragments and its indicated segmentation object；
Download module, the segmentation object indicated for the designated capabilities range corresponding segments number by the object download to this
Ground.
10. device according to claim 8, which is characterized in that when to be uploaded duplicate new with object object name
When object, while the segmentation object of the new object is uploaded to the bucket, further includes:
The creation module is further, creates the index of the new object, deletes the index and the index pair of the object
The segmentation object answered.
11. a kind of electronic equipment, which is characterized in that including communication interface, processor, memory and bus, the communication interface,
It is connected with each other between the processor and the memory by bus；
Machine readable instructions are stored in the memory, the processor is executed by calling the machine readable instructions as weighed
Benefit requires 1 to 5 described in any item methods.
12. a kind of machine readable storage medium, which is characterized in that the machine readable storage medium is stored with machine readable finger
It enables, the machine readable instructions realize method described in any one of claim 1 to 5 when being called and being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910062436.6A CN109828828A (en) | 2019-01-23 | 2019-01-23 | Object method for uploading, device, electronic equipment and machine readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910062436.6A CN109828828A (en) | 2019-01-23 | 2019-01-23 | Object method for uploading, device, electronic equipment and machine readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109828828A true CN109828828A (en) | 2019-05-31 |
Family
ID=66861939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910062436.6A Pending CN109828828A (en) | 2019-01-23 | 2019-01-23 | Object method for uploading, device, electronic equipment and machine readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109828828A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111147235A (en) * | 2019-12-23 | 2020-05-12 | 杭州宏杉科技股份有限公司 | Object access method and device, electronic equipment and machine-readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105187551A (en) * | 2015-09-29 | 2015-12-23 | 成都四象联创科技有限公司 | Distributed computing method based on cloud platform |
| US20180034892A1 (en) * | 2016-07-28 | 2018-02-01 | Caringo, Inc. | Multi-part upload |
| CN108228799A (en) * | 2017-12-29 | 2018-06-29 | 北京奇虎科技有限公司 | The storage method and device of object indexing information |
| CN109067733A (en) * | 2018-07-27 | 2018-12-21 | 成都华为技术有限公司 | The method and apparatus for sending data, and the method and apparatus for receiving data |
-
2019
- 2019-01-23 CN CN201910062436.6A patent/CN109828828A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105187551A (en) * | 2015-09-29 | 2015-12-23 | 成都四象联创科技有限公司 | Distributed computing method based on cloud platform |
| US20180034892A1 (en) * | 2016-07-28 | 2018-02-01 | Caringo, Inc. | Multi-part upload |
| CN108228799A (en) * | 2017-12-29 | 2018-06-29 | 北京奇虎科技有限公司 | The storage method and device of object indexing information |
| CN109067733A (en) * | 2018-07-27 | 2018-12-21 | 成都华为技术有限公司 | The method and apparatus for sending data, and the method and apparatus for receiving data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111147235A (en) * | 2019-12-23 | 2020-05-12 | 杭州宏杉科技股份有限公司 | Object access method and device, electronic equipment and machine-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10209910B2 (en) | Copy-redirect on write | |
| US10872017B2 (en) | Restoring a file system object | |
| CN106484906B (en) | Distributed object storage system flash-back method and device | |
| CN106201771B (en) | Data-storage system and data read-write method | |
| CN104050216B (en) | For customizing the file system manager of resource allocation | |
| US10042714B2 (en) | Point-in-time copy on write for golden image | |
| CN104881466B (en) | The processing of data fragmentation and the delet method of garbage files and device | |
| US8825653B1 (en) | Characterizing and modeling virtual synthetic backup workloads | |
| CN108319654A (en) | Computing system, cold and hot data separation method and device, computer readable storage medium | |
| US20150324388A1 (en) | Dataset replica migration | |
| US11308223B2 (en) | Blockchain-based file handling | |
| CN113209633A (en) | Mirror image processing method and device, electronic equipment and readable storage medium | |
| CN107451138A (en) | A kind of distributed file system storage method and system | |
| CN107402950A (en) | Divide the document handling method and device of table based on point storehouse | |
| CN110968554A (en) | Block chain storage method, storage system and storage medium based on file chain blocks | |
| CN109471843A (en) | A kind of metadata cache method, system and relevant apparatus | |
| US11669404B2 (en) | Efficient access-based reallocation of backup data within an object storage | |
| CN107391544A (en) | Processing method, device, equipment and the computer storage media of column data storage | |
| CN110413376A (en) | A kind of method, equipment and the storage medium of Virtual Machine Manager USB device | |
| JP2023501640A (en) | POINT CLOUD PROCESSING METHOD, COMPUTER SYSTEM, PROGRAM AND COMPUTER-READABLE STORAGE MEDIUM | |
| JP6406283B2 (en) | Storage apparatus and storage method | |
| CN107798063A (en) | Snap processing method and snapshot processing unit | |
| CN107408239B (en) | Architecture for managing mass data in communication application through multiple mailboxes | |
| O'reilly | Network storage: Tools and technologies for storing your company’s data | |
| CN110399248A (en) | A kind of image file creation, acquisition methods, device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |