CN108733515A - Dispatching method, file backup method, device and the storage medium of file backup - Google Patents
Dispatching method, file backup method, device and the storage medium of file backup Download PDFInfo
- Publication number
- CN108733515A CN108733515A CN201810509163.0A CN201810509163A CN108733515A CN 108733515 A CN108733515 A CN 108733515A CN 201810509163 A CN201810509163 A CN 201810509163A CN 108733515 A CN108733515 A CN 108733515A
- Authority
- CN
- China
- Prior art keywords
- file
- backup
- storage device
- backup tasks
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of dispatching method of file backup, file backup method, device and storage mediums, belong to network technique field.The method is a full backup process for file, the storage device backed up is first determined by controlling equipment, again based on each process in backup, constantly update the Status of Backups of file, and based on the message transmitted by each storage device, come finally determine backup completion, by the above process, be capable of safeguard file backup success rate.
Description
Technical field
The present invention relates to network technique field, more particularly to a kind of dispatching method of file backup, file backup method, dress
It sets and storage medium.
Background technology
With the development of internet industry, need the file resource stored more and more, such as picture, audio, video etc.
Deng file is lost after disk failures in order to prevent, these files need to support more parts of backups, these backups to need to be stored in difference
Physics computer room, or storage on different physical machines, could realize High Availabitity truly.
Currently, existing file backup method is：The upper transmitting file of user is to the first storage device, when the first storage device is complete
It after the storage of pairs of file, then returns and is stored successfully to user, first time storage device respectively sets this document in the second storage
It is backed up in standby and third storage device, backup both sides do not need common clock, that is, user does not know this in real time
One backup procedure, is balance complementary relationship between the first, second and third storage device, and the first, second and third storage device can synchronize mutually
Backup, the first, second and third storage device can be three areas, can also be regional three equipment etc., to realize text
The backup storage of part is on different physical machines.
In the implementation of the present invention, the inventor finds that the existing technology has at least the following problems：
Existing file backup method is very fragile, and the first storage device is by the mistake of file backup to second and third storage device
Cheng Zhong is only to have carried out backing up this action, it can however not the success rate of safeguard file backup.
Invention content
An embodiment of the present invention provides a kind of dispatching method of file backup, file backup method, device and storage medium,
The technical issues of capable of solving ensure the success rate of file backup in the related technology.The technical solution is as follows：
On the one hand, a kind of dispatching method of file backup is provided, the method includes：
The metamessage for receiving the file of the first storage device transmission, is set as to be determined by the Status of Backups of the file and deposits
Equipment state is stored up, the metamessage of the file includes file name, file size and file store path；
According to the metamessage of the file, determine that at least one second storage device, at least one second storage are set
It is ready for use on and the file is backed up；
Metamessage based on the file and at least one second storage device, generate at least one backup tasks,
The Status of Backups of the file is updated to state to be backed up, the backup tasks are used to indicate the backup tasks corresponding
Two storage devices obtain the backup file of the file based on the file store path；
At least one backup tasks are sent respectively at least one second storage device；
When the receiving at least one second storage device of the task completes message, by the Status of Backups of the file
It is updated to backup completion status.
On the one hand, a kind of file backup method is provided, the method includes：
The backup tasks that controlling equipment is sent are received, the backup tasks are used to indicate the second storage device and are based on backup times
File store path in business obtains the backup file of file；
According to the file store path in the backup tasks, the first storage device is accessed, first storage device is
It is stored with the storage device of the file；
From first storage device, the backup file of the file is obtained.
On the one hand, a kind of dispatching device of file backup is provided, described device includes：
Receiving module, the metamessage of the file for receiving the transmission of the first storage device, the metamessage of the file include
File name, file size and file store path；
State update module, for setting the Status of Backups of the file to storage device status to be determined；
Determining module, for according to the metamessage of the file, determining at least one second storage device, described at least one
A second storage device is for backing up the file；
Generation module is used for the metamessage based on the file and at least one second storage device, generates at least
One backup tasks；
The state update module is additionally operable to the Status of Backups of the file being updated to state to be backed up, and the backup is appointed
Business is used to indicate corresponding second storage device of the backup tasks and obtains the standby of the file based on the file store path
Part file；
Sending module is set at least one backup tasks to be sent respectively at least one second storage
It is standby；
The state update module, for completing message when the receiving at least one second storage device of the task
When, the Status of Backups of the file is updated to backup completion status.
On the one hand, a kind of file backup device is provided, described device includes：
Receiving module, the backup tasks for receiving controlling equipment transmission, the backup tasks are used to indicate the second storage
Equipment obtains the backup file of file based on the file store path in backup tasks；
Access modules, for according to the file store path in the backup tasks, accessing the first storage device, described the
One storage device is to be stored with the storage device of the file；
Acquisition module, for from first storage device, obtaining the backup file of the file.
On the one hand, a kind of computer readable storage medium is provided, at least one instruction is stored in the storage medium,
Described instruction is loaded by processor and is executed to realize the operation as performed by above-mentioned file backup method.
The advantageous effect that technical solution provided in an embodiment of the present invention is brought is：
An embodiment of the present invention provides a full backup processes for file, and by controlling equipment, first determination is backed up
Storage device, then based on each process in backup, constantly update the Status of Backups of file, and be based on each storage device institute
The message of transmission by the above process, is capable of the success rate of safeguard file backup finally to determine the completion of backup.
Description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is a kind of structural schematic diagram of file backup system provided in an embodiment of the present invention；
Fig. 2 is a kind of flow chart of the dispatching method of file backup provided in an embodiment of the present invention；
Fig. 3 is a kind of flow chart of file backup method provided in an embodiment of the present invention；
Fig. 4 is a kind of flow chart of file backup method provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of the dispatching device of file backup provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of file backup device provided in an embodiment of the present invention；
Fig. 7 is a kind of structural schematic diagram of server provided in an embodiment of the present invention.
Specific implementation mode
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
Fig. 1 is a kind of structural schematic diagram of file backup system provided in an embodiment of the present invention, referring to Fig. 1, the system packet
It includes：Multiple storage device Zone and at least one controlling equipment GS.
(1) storage device Zone is used for storage file；
This document can be stored in bottom by user by transmitting file in client to storage device Zone, storage device Zone
Layer file storage engines in, and to controlling equipment send file metamessage.
Specifically, a storage device Zone includes upload/download function upload/download, message queue mq, disappears
Cease receive capabilities sync.sub and storage engines file system.Wherein upload/download for file upload or under
It carries；Mq is message queue function, for being lined up to the message received, to wait for subsequent processing；Sync.sub is message
Processing function is received, the task for subscribing to message queue mq in real time simultaneously carries out subsequent processing；File system are for storing text
Part.Concrete function based on storage device, above-mentioned file upload procedure may include：User is by client to storage device
Transmitting file in upload/download systems in Zone, when storage device detects the uploading instructions to file, by file
It stores to the file System storage engines of bottom, then, file name, file size and file are stored road by storage device
The metamessage of the files such as diameter reports to controlling equipment.
Further, which can also be performed the backup tasks of file, and storage device Zone receives file
After backup tasks, the metamessage of file to be backed up can be obtained from file backup task, by accessing in file meta-information
Store path obtains the backup file of file and is stored in the file storage engines of bottom, and sends file to controlling equipment
Backup success notification.
Concrete function based on storage device, above-mentioned file backup process may include：Message in storage device Zone
Queue mq receives the backup tasks of file, and the metamessage of file to be backed up can be obtained from file backup task, and message connects
The information that function sync.sub subscribes to message queue mq in real time is received, can know the backup tasks, and obtained from backup tasks
The metamessage of file to be backed up, storage device Zone can access depositing in file meta-information by download system download
Path is stored up, to obtain the backup file of file and be stored in the file storage engines of bottom, which may be used ground
The form of location link exists.
Further, in backup procedure, storage device can also carry out backup file the verification of data consistency：It deposits
Storage equipment Zone can carry out characteristic value calculating to the file data received, obtain the characteristic value of this document data, and should
Characteristic value is compared with the characteristic value that backup file carries, if two values are identical, illustrates to take in storage device Zone
To file data be complete errorless, then file storage is arrived by the uploading system upload in storage device Zone
File System storage engines, then back up success.After backup file successfully stores, backup success notification is sent to scheduling system.
(2) controlling equipment GS, for all storage devices to be managed and dispatched；
Controlling equipment receives the metamessage of the file of storage device transmission, and the storage that determination will carry out file backup is set
It is standby, it generates file backup task and is sent to the storage device that will carry out file backup, controlling equipment is in file backup process
In, it can be constantly according to the progress of backup, to update file backup state, and the controlling equipment can also periodically detect file backup
State, until file backup success.
Specifically, GS includes scheduling system scheduler, database D B and detection function sync.monitor, wherein
Scheduler has scheduling feature, a management platform is equivalent to, for being managed scheduling to fileinfo；DB is data
Library, metamessage and file backup state for recording file etc.；Sync.monitor has file backup status detection function,
For detecting the file backup state being recorded in database D B.
It should be noted that further including file backup system Sync file in the storage system, this document standby system can
To be run on a hardware device with controlling equipment GS, can also be separately operable in different hardware equipment, the present invention is implemented
Example does not limit this.Specifically, this document standby system Sync file include sync.mq and sync.pub, wherein
Sync.mq is message queue, and subsequent processing is carried out for receiving message, and according to the storage sequencing in message queue；
Sync.pub is that message launches function, and the information for subscribing to sync.mq in real time simultaneously carries out follow-up work.
Fig. 2 is a kind of flow chart of the dispatching method of file backup provided in an embodiment of the present invention, the tune of this document backup
Degree method can be applied in the controlling equipment in above-mentioned Fig. 1, and referring to Fig. 2, which specifically includes：
201, the metamessage for receiving the file that the first storage device is sent, is set as to be determined by the Status of Backups of this document
The metamessage of storage device status, this document includes file name, file size and file store path.
202, according to the metamessage of this document, at least one second storage device is determined, which sets
It is ready for use on and this document is backed up.
203, based on the metamessage of this document and at least one second storage device, at least one backup tasks are generated,
The Status of Backups of this document is updated to state to be backed up, which is used to indicate corresponding second storage of the backup tasks
Equipment obtains the backup file of this document based on this document store path.
204, at least one backup tasks are sent respectively at least one second storage device.
205, when the receiving at least one second storage device of the task completes message, by the Status of Backups of this document
It is updated to backup completion status.
In some embodiments, after which is sent at least one second storage device by this, this method
Further include：
Every preset time period detects the state of the backup tasks；
When the state of at least one backup tasks is not backup completion status, at least one backup is re-executed respectively
The transmission process of task.
In some embodiments, the metamessage according to this document, determines that at least one second storage device includes：
According to the distance between each storage device and first storage device in storage system, obtains and set apart from nearest
It is standby to be used as the second storage device；Or,
According to the load of each storage device in storage system, obtains the minimum equipment of load and set as the second storage
It is standby.
In some embodiments, after the metamessage for receiving the file that the first storage device is sent, this method further includes：
File, which is sent, to first storage device uploads success notification.
In some embodiments, metamessage that should be based on this document and at least one second storage device, generate at least
After one backup tasks, this method further includes：
At least one backup tasks are respectively sent in message queue, which launches function by message and order
It reads；
Function is launched after getting at least one of message queue backup tasks by the message, is executed this extremely
The step of few backup tasks are sent respectively at least one second storage device.
Fig. 3 is a kind of flow chart of file backup method provided in an embodiment of the present invention, and the method for this document backup can be with
Applied in the storage device in above-mentioned Fig. 1, referring to Fig. 3, which specifically includes：
301, the backup tasks that controlling equipment is sent are received, which is used to indicate the second storage device and is based on backup
File store path in task obtains the backup file of file.
302, according to the file store path in the backup tasks, the first storage device is accessed, which is
It is stored with the storage device of this document.
303, from first storage device, the backup file of this document is obtained.
In some embodiments, this document store path is web page interlinkage form.
In some embodiments, should be from first storage device, after the backup file for obtaining this document, this method is also
Including：
When receiving the new backup tasks of controlling equipment transmission, detect whether to exist identical with the backup tasks
The task of not completing then deletes new backup tasks if there is unfinished task identical with the backup tasks.
In some embodiments, after the backup tasks that the reception controlling equipment is sent, this method further includes：
The backup tasks are received by message queue, which is subscribed to by message sink function；
By the message sink function after the backup tasks in getting the message queue, executes and set from first storage
In standby, the step of obtaining the backup file of this document.
Fig. 4 is a kind of flow chart of file backup method provided in an embodiment of the present invention, and referring to Fig. 4, the embodiment is specific
Including：
400, the first storage device sends the metamessage of file to controlling equipment.
In embodiments of the present invention, the metamessage for the file that the first storage device is sent to controlling equipment is from user to the
The file that one storage device uploads.
Wherein, the metamessage of this document includes file name, file size and file store path, certainly, this document
Metamessage can also include other information, such as the uplink time of file, file type etc., and the embodiment of the present invention does not do this
It limits.
Structure based on Fig. 1, above-mentioned steps 400 can have following specific implementation process：First storage device by upload/
The metamessage of stored file is sent to the scheduling system scheduler of controlling equipment GS by download system.
401, when controlling equipment receives the metamessage of the file of the first storage device transmission, to first storage device
It sends file and uploads success notification.
In embodiments of the present invention, when controlling equipment receives the metamessage of the file of the first storage device transmission, i.e.,
Indicate that file uploads the success of the first storage device, this file upload success notification is sent to first storage and set by controlling equipment
Standby, for informing that user file uploads successfully, the backup procedure user of subsequent file is without knowing.
Structure based on Fig. 1, above-mentioned steps 401 can have following specific implementation process：Controlling equipment GS is by dispatching system
System scheduler receives the metamessage of the files such as file name, file size and file store path, dispatches system
Scheduler sends file to the first equipment and uploads success notification.
402, controlling equipment sets the Status of Backups of this document to storage device status to be determined.
In order to avoid losing because of situations such as file caused by system storage error, controlling equipment can be standby by file
Part is at least one storage device, and therefore, when receiving the metamessage of any file, controlling equipment can be that this document triggers
Subsequent file backup process, when starting backup, controlling equipment can be that this document safeguards its Status of Backups, to monitor backup
Progress.
The Status of Backups of file may include：Storage device status to be determined, state to be backed up, backup completion status.Tool
Body, specific introduce is made to the Status of Backups of file below：
(1) when the Status of Backups of file is storage device status to be determined, indicate that controlling equipment has just received first and deposited
The metamessage of this document that equipment is sent is stored up, there are no determinations to backup to by this document for which storage device.
(2) when the Status of Backups of file is state to be backed up, metamessage of the controlling equipment based on this document and really is indicated
The storage device of the fixed carry out file backup, generates file backup task.
(3) when the Status of Backups of file is backup completion status, indicate that this document backs up successfully, controlling equipment receives
The message that the file backup task that the storage device of above-mentioned determination is sent is completed.
It should be noted that controlling equipment receive the first storage device transmission file metamessage after, by this article
The metamessage of part is stored in the database of controlling equipment, and the database is for providing the services such as file polling to the user.
Structure based on Fig. 1, above-mentioned steps 402 can have following specific implementation process：Controlling equipment GS is by dispatching system
System scheduler receives the metamessage of the files such as file name, file size and file store path, and the member of this document is believed
Breath is recorded in database D B, is recorded as the Status of Backups of this document " storage device status to be determined " in DB.
It should be noted that above-described embodiment is first to first storage device transmission file upload success notification, then into
It is illustrated for the implementation sequence of the update step of row such as step 402.In other embodiments, the implementation sequence of controlling equipment
The update step of step 402 can also be first carried out, then executes the transmission process in step 401, alternatively, will be deposited receiving first
When storing up the metamessage for the file that equipment is sent, synchronizes and be updated and transmission process, the embodiment of the present invention do not limit herein.
403, controlling equipment determines at least one second storage device according to the metamessage of this document.
It is at least one therefore, it is necessary to be chosen from multiple storage device due to there is multiple storage devices in storage system
Storage device for backing up this document that is to say the second storage device.
In some embodiments, which may include following any realization methods：
A kind of realization method, according to the distance between each storage device in storage system and first storage device,
It obtains apart from nearest equipment as the second storage device.
Specifically, the geographical location based on each storage device in storage system obtains each storage device and is deposited with first
The distance between equipment is stored up, the distance between each storage device and the first storage device are compared, determine the distance
Minimum storage device is based on the realization method, the backup procedure speed of file is faster as the second storage device.
A kind of realization method, according to the load of each storage device in storage system, obtain the minimum equipment of load
As the second storage device.
Based on the total memory capacity of each storage device in the storage system and size of storage file, obtains each storage
The residual storage capacity of each storage device is carried out size comparison by the residual storage capacity of equipment, determines that the residue stores
The equipment of capacity minimum is based on the realization method as the second storage device, can efficiently utilize the storage of each storage device
Space avoids the waste to space.
It should be noted that the process of at least one second storage device of the determination is not limited to implementation above mode, for example,
It is also based on the residual storage capacity of each storage device in the size and storage system of this document, determines residual storage capacity
More than the storage device of the size of this document, as the second storage device.
In addition, in practical applications, in order to improve the efficiency and accuracy of file backup, the second storage of above-mentioned determination is set
It is standby with the first storage device as possible not in the same physics computer room.
Structure based on Fig. 1, above-mentioned steps 403 can have following specific implementation process：Scheduling system in controlling equipment GS
The scheduler that unites issues the task of a storage device to be determined to message queue sync.mq, due to being queue form, so
Message in queue needs to wait in line subsequent processing, and message launches function sync.pub and subscribes to message queue sync.mq in real time
Information, at this time send out existing file alternate device to be determined task, after getting this task, the scheduling into controlling equipment GS
System scheduler inquires which storage device this document should backup to, and scheduler is according to the plan configured in database D B
Slightly, it e.g. to backup to that distance is close or the small storage device of residual storage capacity, finally determine that the equipment to be backed up is the
Two storage devices.
404, controlling equipment is generated at least one standby based on the metamessage of this document and at least one second storage device
Part task, state to be backed up is updated to by the Status of Backups of this document.
In embodiments of the present invention, which is used to indicate corresponding second storage of the backup tasks and sets
The standby backup file for going to obtain file, the generating process of this backup tasks include：Obtain the metamessage and at least one of this document
The facility information of at least one second storage device is determined as task and sends mesh by the facility information of a second storage device
File store path in the metamessage of this document is generated the address link of file by mark, by the address link of this document, text
The information such as part size form the backup tasks.
In addition, after controlling equipment generates at least one backup tasks, controlling equipment is by the corresponding this document of the backup tasks
Status of Backups be updated to state to be backed up so that controlling equipment knows that this document has been in backup procedure.
Structure based on Fig. 1, above-mentioned steps 404 can have following specific implementation process：Scheduling system in controlling equipment GS
System facility informations of the scheduler by the metamessage and the second storage device of the file recorded in database D B, generates file
Backup tasks, while the Status of Backups of this document being recorded as " state to be backed up " in database D B.
405, at least one backup tasks are respectively sent in message queue by controlling equipment, and the message queue is by message
Function is launched to subscribe to.
The message of the controlling equipment is launched function and can be subscribed to message queue, so as to know message team in real time
At least one backup tasks increased newly in row, are issued respectively so as to obtain at least one backup tasks in time.When
So, which, which can refer to message dispensing function, can detect and obtain the message of the head of the queue positioned at message queue, to ensure to press
Backup tasks are executed according to the principle of first in, first out, that is to say, when backup tasks are sent in message queue, if message team
Further include that the message that function obtains is not launched by message in row, then needs to be waited in line according to the processing progress of message, until certainly
Body is moved to head of the queue.
Structure based on Fig. 1, above-mentioned steps 405 can have following specific implementation process：Controlling equipment GS is by backup tasks
The message queue sync.mq being sent in file backup system Sync file, message, which is launched function sync.pub and subscribed in real time, to disappear
Cease the information in queue sync.mq.
406, the message in controlling equipment launches at least one backup of function in getting the message queue respectively
After task, which is sent respectively at least one second storage device.
Above-mentioned steps 405 to 406 are that at least one backup tasks are sent respectively to described at least one second to deposit
The specific implementation for storing up equipment, in order to enable message team may be used in the progress that the backup of file can be orderly, controlling equipment
Row and message launch function to carry out issuing for backup tasks.
Structure based on Fig. 1, above-mentioned steps 406 can have following specific implementation process：Controlling equipment GS is thrown by message
After playing function sync.pub gets the backup tasks in message queue sync.mq, message launches function sync.pub, and this is standby
Part task is delivered in the message queue mq in the second storage device, carries out queuing processing, while sync.pub will be recorded
Metamessage to the All Files in database D B is sent to message queue mq.
407, the message queue in the second storage device receives the backup tasks, which is ordered by message sink function
It reads.
In practical applications, when which receives backup tasks, there may be following two situations：In the message queue
Existing task identical with the backup tasks；Or, task identical with the backup tasks is not present in the message queue.Specifically
Ground is introduced for above-mentioned two situations：
After the message queue receives the backup tasks, message sink function is according to the backup tasks received, detection
Existing backup tasks in the message queue, when existing backup tasks include the backup tasks, then in description messages queue
In the presence of task identical with the backup tasks, which can will be positioned at the backup tasks of the message queue tail of the queue
It deletes.Do not include the backup tasks when having in backup tasks, continues to handle according to the task order in queue, until this
Backup tasks are moved to head of the queue in message queue, can just be handled.Based on the above process, avoids identical file backup and appoint
Business is repeatedly processed through vaporization, condensation and absorption, and therefore, also avoids to increase unnecessary burden to storage device and controlling equipment.
Wherein, when being detected to backup tasks, the comparison of the metamessage of file can be carried out, when two backup tasks
Including file metamessage it is identical when, then two backup tasks be identical copy task, if it is different, then this two are standby
Part task is different backup tasks.Certainly, specific detection process can also be compared using other information, for example, backup is appointed
Whether the characteristic value that can include file in business, then can identical with feature based value, to determine whether backup tasks are identical.
Structure based on Fig. 1, above-mentioned steps 407 can have following specific implementation process：Message in second storage device
Queue mq receives the backup tasks, and message sink function sync.sub subscribes to message queue mq in real time, when sync.sub is detected
When backup tasks are the existing backup tasks in message queue mq, sync.sub deletes the backup tasks.
408, the message sink function in the second storage device is after the backup tasks in getting the message queue, according to
File store path in the backup tasks accesses the first storage device.
Wherein, address link form may be used in this document store path, which is linked by the address,
The first storage device is accessed, obtains the backup file of file.Certainly, if controlling equipment only adds text in backup tasks
Part store path, then the second storage device, can be with extraction document store path, and by this document when receiving the backup tasks
Store path is converted to address link form, to carry out subsequent backup procedure.
Structure based on Fig. 1, above-mentioned steps 408 can have following specific implementation process：Message in second storage device
After receive capabilities sync.sub gets the backup tasks in message queue mq, function sync.pub is launched according to message and is launched
To the metamessage of the file of message queue mq, the first storage device is accessed.
409, the second storage device obtains the backup file of this document from first storage device.
Second storage device replicates file during accessing to the first storage device, and will replicate
Obtained file data is stored to storage device, to obtain the backup file of file.
In embodiments of the present invention, which can carry out data consistency to the backup file got
Verification, specific checking procedure includes：Second storage device can carry out characteristic value calculating to the backup file got,
The characteristic value of the backup file is obtained, and this feature value is compared with the characteristic value that file to be backed up carries, if two
Characteristic value is identical, then illustrate that the backup file obtained in second storage device is identical as the file corresponding to backup tasks,
The backup file is stored to storage engines by the uploading system in the second storage device, then backs up success.If two spies
Value indicative differs, then illustrate that the backup file obtained in second storage device is different from the file corresponding to backup tasks,
Backup file can then be reacquired.
Structure based on Fig. 1, above-mentioned steps 409 can have following specific implementation process：Second storage device passes through download
System download accesses the store path in file meta-information, to obtain the backup file of file and be stored in the text of bottom
In part storage engines file system.
410, it when the second storage device gets the backup file of this document, sends task to controlling equipment and completes message.
Structure based on Fig. 1, above-mentioned steps 410 can have following specific implementation process：Second storage device will back up text
After part is stored in file storage engines file system, the scheduling system scheduler into controlling equipment GS sends task
Completion notice.
411, when controlling equipment receives the task completion message of at least one second storage device, by the standby of this document
Part state is updated to backup completion status.
Above-mentioned 400 to 411 the step of is a full backup process for file, first determines that progress is standby by controlling equipment
The storage device of part, then based on each process in backup, the Status of Backups of file is constantly updated, and be based on each storage device
Transmitted message by the above process, is capable of the success rate of safeguard file backup finally to determine the completion of backup.
In practical applications, it is also possible to the case where task execution failure of the second storage device occur, for example, the second storage
There is the case where accessing failure in equipment when accessing to first storage device；Or, any storage device suddenly lead by power-off
Suspension of task etc. is caused, is not repeated one by one herein.In the case, since the equipment for participating in backup could not be completed to back up,
The Status of Backups of this document in controlling equipment will not be updated to backup completion status, still maintain state to be backed up.To understand
The case where certainly above-mentioned task execution failure being likely to occur, controlling equipment every preset time period detects the standby of the backup tasks
Part state is re-executed when the controlling equipment detects that the state of the backup tasks is not backup completion status by the backup
Task is sent at least one second storage device and subsequent process, until the controlling equipment detects the standby of the backup tasks
Part state is backup success status, then the step of stopping the backup tasks being sent at least one second storage device.It is logical
This is crossed to be periodically detected Status of Backups, can to avoid due to backup caused by some emergency situations it is unsuccessful, into one
Step has ensured the success rate of backup.
It should be noted that controlling equipment receives the completion message of the corresponding backup tasks of all second storage devices
Afterwards, the Status of Backups of this document is just updated to backup completion status.If controlling equipment is not received by the task of this document
Message is completed, and is only to have received the task that the second storage device of part is sent to complete message, then the Status of Backups of this document
Remain as state to be backed up.
Structure based on Fig. 1, above-mentioned steps 411 can have following specific implementation process：Scheduling system in controlling equipment GS
After system scheduler receives task completion notice, the Status of Backups of this document is recorded as " backing up successfully in database D B
State ", while passing through the standby of the file in the detection function sync.monitor timing Test databases DB in controlling equipment GS
Part state delivers the backup tasks for the file that Status of Backups is " storage device status to be determined " and " state to be backed up " again
Into message queue sync.mq, this link is repeated, until the Status of Backups of the file in DB is " backup success status ".
An embodiment of the present invention provides a full backup processes for file, and by controlling equipment, first determination is backed up
Storage device, then based on each process in backup, constantly update the Status of Backups of file, and be based on each storage device institute
The message of transmission by the above process, is capable of the success rate of safeguard file backup finally to determine the completion of backup.Further
Ground, by being periodically detected to Status of Backups, can to avoid due to caused by some emergency situations backup it is unsuccessful, into
One step has ensured the success rate of backup.
Fig. 5 is a kind of structural schematic diagram of the dispatching device of file backup provided in an embodiment of the present invention.It, should referring to Fig. 5
Device includes：Receiving module 501, state update module 502, determining module 503, generation module 504 and sending module 505.
Receiving module 501, the metamessage of the file for receiving the transmission of the first storage device, the metamessage of this document include
File name, file size and file store path；
State update module 502, for setting the Status of Backups of this document to storage device status to be determined；
Determining module 503 determines at least one second storage device, this at least one for the metamessage according to this document
A second storage device is for backing up this document；
Generation module 504 generates at least one for metamessage and at least one second storage device based on this document
A backup tasks；
The state update module 502 is additionally operable to the Status of Backups of this document being updated to state to be backed up, the backup tasks
It is used to indicate the backup file that corresponding second storage device of the backup tasks obtains this document based on this document store path；
Sending module 505, at least one backup tasks to be sent respectively at least one second storage device；
The state updates mould 502, is additionally operable to when the receiving at least one second storage device of the task completes message,
The Status of Backups of this document is updated to backup completion status.
In some embodiments, which further includes：
Detection module is used for every preset time period, detects the state of the backup tasks；
The sending module 505 is additionally operable to when the state of at least one backup tasks is not backup completion status, respectively
Re-execute the transmission process of at least one backup tasks.
In some embodiments, which is used for：
According to the distance between each storage device and first storage device in storage system, obtains and set apart from nearest
It is standby to be used as the second storage device；Or,
According to the load of each storage device in storage system, obtains the minimum equipment of load and set as the second storage
It is standby.
In some embodiments, the sending module 505 is additionally operable to send file to first storage device and upload successfully to lead to
Know.
In some embodiments, which further includes：
Subscribing module, at least one backup tasks to be respectively sent in message queue, which is disappeared
Breath is launched function and is subscribed to；
The sending module 505 launches function to get at least one of the message queue standby for passing through the message
After part task, the step of at least one backup tasks are sent respectively at least one second storage device is executed.
It should be noted that：Above-described embodiment provide file backup dispatching device in schedule file backups, only with
The division progress of above-mentioned each function module, can be as needed and by above-mentioned function distribution by not for example, in practical application
Same function module is completed, i.e., the internal structure of controlling equipment is divided into different function modules, described above to complete
All or part of function.In addition, the dispatching method of the dispatching device and file backup for the file backup that above-described embodiment provides
Embodiment belongs to same design, and specific implementation process refers to embodiment of the method, and which is not described herein again.
Fig. 6 is a kind of structural schematic diagram of file backup device provided in an embodiment of the present invention.Referring to Fig. 6, the device packet
It includes：Receiving module 601, access modules 602 and acquisition module 603.
Receiving module 601, the backup tasks for receiving controlling equipment transmission, the backup tasks are used to indicate the second storage
Equipment obtains the backup file of file based on the file store path in backup tasks；
Access modules 602, for according to the file store path in the backup tasks, accessing the first storage device, this
One storage device is to be stored with the storage device of this document；
Acquisition module 603, for from first storage device, obtaining the backup file of this document.
In some embodiments, the file store path in the receiving module and access modules is web page interlinkage form.
In some embodiments, which further includes：
Removing module, for when receiving the new backup tasks of controlling equipment transmission, detecting whether exist and this
The identical unfinished task of backup tasks is then deleted new backup and is appointed if there is unfinished task identical with the backup tasks
Business.
In some embodiments, which further includes：
Subscribing module receives the backup tasks for passing through message queue, which is subscribed to by message sink function；
The acquisition module 603, for by the message sink function after the backup tasks in getting the message queue,
The step of executing from first storage device, obtaining the backup file of this document.
It should be noted that：The file backup device that above-described embodiment provides is in backup file, only with above-mentioned each function
The division progress of module, can be as needed and by above-mentioned function distribution by different function moulds for example, in practical application
Block is completed, i.e., the internal structure of the second storage device is divided into different function modules, with complete it is described above whole or
Person's partial function.In addition, the file backup device that above-described embodiment provides belongs to same design with file backup method embodiment,
Its specific implementation process refers to embodiment of the method, and which is not described herein again.
Fig. 7 is a kind of structural schematic diagram of computer equipment provided in an embodiment of the present invention, which can be because
Configuration or performance are different and generate bigger difference, may include one or more processors (central
Processing units, CPU) 701 and one or more memory 702, wherein it is stored in the memory 702
There are at least one instruction, at least one instruction to be loaded by the processor 701 and executed to realize that above-mentioned each method is implemented
The method that example provides.Certainly, which can also have wired or wireless network interface, keyboard and input and output to connect
The components such as mouth, to carry out input and output, which can also include other components for realizing functions of the equipments,
This is not repeated.
In the exemplary embodiment, a kind of computer readable storage medium, such as the memory including instruction are additionally provided,
Above-metioned instruction can by the processor in computer equipment execute with complete the file backup in above-described embodiment dispatching method or
File backup method.For example, the computer readable storage medium can be ROM, random access memory (RAM), CD-ROM, magnetic
Band, floppy disk and optical data storage devices etc..
One of ordinary skill in the art will appreciate that realizing that all or part of step of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, which can be stored in a kind of computer-readable storage
In medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.
Claims (20)
1. a kind of dispatching method of file backup, which is characterized in that the controlling equipment being applied in storage system, the method packet
It includes：
The metamessage for receiving the file of the first storage device transmission sets the Status of Backups of the file to storage to be determined and sets
Standby state, the metamessage of the file includes file name, file size and file store path；
According to the metamessage of the file, determine that at least one second storage device, at least one second storage device are used
It is backed up in the file；
Metamessage based on the file and at least one second storage device, generate at least one backup tasks, by institute
The Status of Backups for stating file is updated to state to be backed up, and the backup tasks are used to indicate the backup tasks corresponding second and deposit
Storage equipment obtains the backup file of the file based on the file store path；
At least one backup tasks are sent respectively at least one second storage device；
When the receiving at least one second storage device of the task completes message, the Status of Backups of the file is updated
For backup completion status.
2. according to the method described in claim 1, it is characterized in that, described be sent respectively at least one backup tasks
After at least one second storage device, the method further includes：
Every preset time period detects the state of the backup tasks；
When the state of at least one backup tasks is not backup completion status, at least one backup is re-executed respectively and is appointed
The transmission process of business.
3. according to the method described in claim 1, it is characterized in that, the metamessage according to the file, determines at least one
A second storage device includes：
According to the distance between each storage device in storage system and first storage device, obtain apart from nearest equipment
As the second storage device；Or,
According to the load of each storage device in storage system, the minimum equipment of load is obtained as the second storage device.
4. according to the method described in claim 1, it is characterized in that, the member letter for receiving the file that the first storage device is sent
After breath, the method further includes：
File, which is sent, to first storage device uploads success notification.
5. according to the method described in claim 1, it is characterized in that, the metamessage and described at least one based on the file
A second storage device, after generating at least one backup tasks, the method further includes：
At least one backup tasks are respectively sent in message queue, the message queue is launched function by message and ordered
It reads；
Function is launched after getting at least one of message queue backup tasks by the message, and execution will be described
The step of at least one backup tasks are sent respectively at least one second storage device.
6. a kind of file backup method, which is characterized in that the second storage device being applied in storage system, the method packet
It includes：
The backup tasks that controlling equipment is sent are received, the backup tasks are used to indicate the second storage device and are based in backup tasks
File store path obtain file backup file；
According to the file store path in the backup tasks, the first storage device is accessed, first storage device is storage
There is the storage device of the file；
From first storage device, the backup file of the file is obtained.
7. according to the method described in claim 6, it is characterized in that, the file store path is web page interlinkage form.
8. according to the method described in claim 6, it is characterized in that, described from first storage device, the text is obtained
After the backup file of part, the method further includes：
When receiving the new backup tasks that the controlling equipment is sent, detect whether to exist identical with the backup tasks
The task of not completing then deletes new backup tasks if there is unfinished task identical with the backup tasks.
9. according to the method described in claim 6, it is characterized in that, described receive the backup times that the controlling equipment is sent
After business, the method further includes：
The backup tasks are received by message queue, the message queue is subscribed to by message sink function；
By the message sink function after the backup tasks in getting the message queue, execute from first storage
In equipment, the step of obtaining the backup file of the file.
10. a kind of dispatching device of file backup, which is characterized in that described device includes：
Receiving module, the metamessage of the file for receiving the transmission of the first storage device, the metamessage of the file includes file
Title, file size and file store path；
State update module, for setting the Status of Backups of the file to storage device status to be determined,；
Determining module, for according to the metamessage of the file, determining at least one second storage device, described at least one the
Two storage devices are for backing up the file；
Generation module is used for the metamessage based on the file and at least one second storage device, generates at least one
Backup tasks；
The state update module is additionally operable to the Status of Backups of the file being updated to state to be backed up, and the backup tasks are used
The backup text of the file is obtained based on the file store path in corresponding second storage device of the instruction backup tasks
Part；
Sending module, at least one backup tasks to be sent respectively at least one second storage device；
The state update module is additionally operable to when the receiving at least one second storage device of the task completes message,
The Status of Backups of the file is updated to backup completion status.
11. device according to claim 10, which is characterized in that described device further includes：
Detection module is used for every preset time period, detects the state of the backup tasks；
The sending module is additionally operable to when the state of an at least backup tasks is not backup completion status, respectively again
Execute the transmission process of at least one backup tasks.
12. device according to claim 10, which is characterized in that the determining module is used for：
According to the distance between each storage device in storage system and first storage device, obtain apart from nearest equipment
As the second storage device；Or,
According to the load of each storage device in storage system, the minimum equipment of load is obtained as the second storage device.
13. device according to claim 10, which is characterized in that the sending module is additionally operable to set to first storage
Preparation send file to upload success notification.
14. device according to claim 10, which is characterized in that described device further includes：
Subscribing module, at least one backup tasks to be respectively sent in message queue, the message queue is disappeared
Breath is launched function and is subscribed to；
The sending module is getting the backup of at least one of described message queue for launching function by the message
After task, the step of at least one backup tasks are sent respectively at least one second storage device is executed.
15. a kind of file backup device, which is characterized in that described device includes：
Receiving module, the backup tasks for receiving controlling equipment transmission, the backup tasks are used to indicate the second storage device
The backup file of file is obtained based on the file store path in backup tasks；
Access modules, for according to the file store path in the backup tasks, accessing the first storage device, described first deposits
It is to be stored with the storage device of the file to store up equipment；
Acquisition module, for from first storage device, obtaining the backup file of the file.
16. device according to claim 15, which is characterized in that the file store path is web page interlinkage form.
17. device according to claim 15, which is characterized in that described device further includes：
Removing module, for when receiving the new backup tasks that the controlling equipment is sent, detect whether to exist with it is described
The identical unfinished task of backup tasks then deletes new backup if there is unfinished task identical with the backup tasks
Task.
18. device according to claim 15, which is characterized in that described device further includes：
Subscribing module receives the backup tasks for passing through message queue, and the message queue is subscribed to by message sink function；
The acquisition module, for by the message sink function after the backup tasks in getting the message queue,
The step of executing from first storage device, obtaining the backup file of the file.
19. a kind of computer equipment, which is characterized in that the computer equipment includes processor and memory, the memory
In be stored at least one instruction, described instruction is loaded by the processor and is executed to realize as claim 1 to right is wanted
The operation performed by the dispatching method of 5 any one of them file backups is asked, or, such as any one of claim 6 to claim 9
Operation performed by the file backup method.
20. a kind of computer readable storage medium, which is characterized in that be stored at least one instruction, institute in the storage medium
Instruction is stated to be loaded by processor and executed to realize the tune such as claim 1 to claim 5 any one of them file backup
Operation performed by degree method, or, as performed by claim 6 to claim 9 any one of them file backup method
Operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810509163.0A CN108733515A (en) | 2018-05-24 | 2018-05-24 | Dispatching method, file backup method, device and the storage medium of file backup |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810509163.0A CN108733515A (en) | 2018-05-24 | 2018-05-24 | Dispatching method, file backup method, device and the storage medium of file backup |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108733515A true CN108733515A (en) | 2018-11-02 |
Family
ID=63935240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810509163.0A Pending CN108733515A (en) | 2018-05-24 | 2018-05-24 | Dispatching method, file backup method, device and the storage medium of file backup |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108733515A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN110806953A (en) * | 2019-11-07 | 2020-02-18 | 中国联合网络通信集团有限公司 | Backup method and device |
| CN112306758A (en) * | 2020-12-24 | 2021-02-02 | 深圳市科力锐科技有限公司 | Backup success rate prediction method, device, equipment and storage medium |
| CN112882866A (en) * | 2021-02-24 | 2021-06-01 | 上海泰宇信息技术股份有限公司 | Backup method suitable for massive files |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050044163A1 (en) * | 2003-08-04 | 2005-02-24 | Manabu Kitamura | Computer system |
| CN101706805A (en) * | 2009-10-30 | 2010-05-12 | 中国科学院计算技术研究所 | Method and system for storing object |
| CN103782279A (en) * | 2011-09-07 | 2014-05-07 | 澳乐伽公司 | File management system and file management method |
| CN104331343A (en) * | 2014-11-06 | 2015-02-04 | 深圳市中兴移动通信有限公司 | File backup method and system |
| CN105872425A (en) * | 2016-04-29 | 2016-08-17 | 乐视控股（北京）有限公司 | Storing method and device for video file |
| CN107122262A (en) * | 2017-04-21 | 2017-09-01 | 杭州狮说教育科技有限公司 | Database intelligent backup system based on agent |
-
2018
- 2018-05-24 CN CN201810509163.0A patent/CN108733515A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050044163A1 (en) * | 2003-08-04 | 2005-02-24 | Manabu Kitamura | Computer system |
| CN101706805A (en) * | 2009-10-30 | 2010-05-12 | 中国科学院计算技术研究所 | Method and system for storing object |
| CN103782279A (en) * | 2011-09-07 | 2014-05-07 | 澳乐伽公司 | File management system and file management method |
| CN104331343A (en) * | 2014-11-06 | 2015-02-04 | 深圳市中兴移动通信有限公司 | File backup method and system |
| CN105872425A (en) * | 2016-04-29 | 2016-08-17 | 乐视控股（北京）有限公司 | Storing method and device for video file |
| CN107122262A (en) * | 2017-04-21 | 2017-09-01 | 杭州狮说教育科技有限公司 | Database intelligent backup system based on agent |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN110806953A (en) * | 2019-11-07 | 2020-02-18 | 中国联合网络通信集团有限公司 | Backup method and device |
| CN112306758A (en) * | 2020-12-24 | 2021-02-02 | 深圳市科力锐科技有限公司 | Backup success rate prediction method, device, equipment and storage medium |
| CN112882866A (en) * | 2021-02-24 | 2021-06-01 | 上海泰宇信息技术股份有限公司 | Backup method suitable for massive files |
| CN112882866B (en) * | 2021-02-24 | 2023-12-15 | 上海泰宇信息技术股份有限公司 | Backup method suitable for mass files |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10917457B2 (en) | Command processing in distributed computing systems | |
| US11210185B2 (en) | Method and system for data recovery in a data system | |
| US10210115B2 (en) | System for handling event messages for file collaboration | |
| CN108733515A (en) | Dispatching method, file backup method, device and the storage medium of file backup | |
| JP5988621B2 (en) | Scalability of high-load business processes | |
| US7827199B2 (en) | Centralized configuration data management for distributed clients | |
| US9798639B2 (en) | Failover system and method replicating client message to backup server from primary server | |
| CN111338773A (en) | Distributed timed task scheduling method, scheduling system and server cluster | |
| CN109714409A (en) | A kind of management method and system of message | |
| CN110737510B (en) | Block device management system | |
| JP6067714B2 (en) | Scale-out system that acquires event data | |
| US20220171791A1 (en) | Replicating large statements with low latency | |
| US20170235600A1 (en) | System and method for running application processes | |
| CN111488373B (en) | Method and system for processing request | |
| CN113791876A (en) | System, method and apparatus for processing tasks | |
| CN112825525A (en) | Method and apparatus for processing transactions | |
| CN110457327A (en) | A kind of data processing method, device and storage medium | |
| CN111163117B (en) | Zookeeper-based peer-to-peer scheduling method and device | |
| US20180063242A1 (en) | Method and apparatus for operating infrastructure layer in cloud computing architecture | |
| CN112783613B (en) | Method and device for scheduling units | |
| CN110677497A (en) | Network medium distribution method and device | |
| JP2010244117A (en) | Replication system, master server, replica server, replication method, and program | |
| US20230409535A1 (en) | Techniques for resource utilization in replication pipeline processing | |
| US12001404B2 (en) | Techniques for replication checkpointing during disaster recovery | |
| CN109460288B (en) | Transaction processing method, management server, transaction processing system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |
|
| RJ01 | Rejection of invention patent application after publication |