helpstoreBackup - Support: sr #108716, Resuming interrupted backups

 
 

sr #108716: Resuming interrupted backups

Submitted by:  Vianney Stroebel <viybel>
Submitted on:  Wed 07 Jan 2015 02:55:22 PM UTC  
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Operating System:  None

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon 12 Jan 2015 04:26:13 PM UTC, comment #4: 

StoreBackup's documentation is indeed very complete and this is one of the things that makes it great.

I think it's not only useful in order to use the software but it's also a great handbook on how to do backup right in general.

Very few (if any) other open source backup software have so much professional insight put into it AND documented.

Vianney Stroebel <viybel>
Sat 10 Jan 2015 11:47:19 AM UTC, comment #3: 

Have a look at section "Getting Started".
http://www.nongnu.org/storebackup/en/node6.html

"This can take some time depending on how much data is in your home directory, as storeBackup.pl will compress your files. It will use all cores of your system for this. Because of these compressions, the first backup is very slow."

And also, in section "Performance" there's a table which shows a dramatical increase in speed in the second backup, 49:20 vs. 02:25.

But the hint about adding data is missing.

One of the problems with documentation is, that it's about 100 pages and not everybody has the time to read everything.

Heinz-Josef Claes <hjclaes>
Project Administrator
Fri 09 Jan 2015 02:29:25 PM UTC, comment #2: 

Your solution makes sense and I already thought about it.

You might want to point out in the documentation that the first backup will probably take a long time (for the benefit of faster backups later) and mention this workaround of "start small and add up gradually" (however obvious it might seem, it isn't a usual practice in the software backup world where the common process is more "set up, launch first backup, verify it and forget").

Vianney Stroebel <viybel>
Thu 08 Jan 2015 10:24:23 AM UTC, comment #1: 

At the moment, there are several other ideas in the queue.

Resuming interrupted backups is difficult, because storeBackup does not use (and need) a centralized daemon. This makes in generally easier to use (and to maintain!), but has the disadvantage you described. On the other hands, you can have a good data integrity without the need to run a full backup ever again (even if you change mount points, copy to another filesystem,...). This is because you can use the combination of checking old backups via checksums and running replication(s).

But I think in your case it may be more a documentation issue.
If you have problems running the very first backup for a (very) long time, you can simply cut your data to backup into small pieces and add new data (directories) from day to day. This can be done via
symlinks (followLinks), exceptDirs, includeDirs, rules, series or any combination: Start small and add data until everything is complete. Then delete the first uncomplete backups.

Heinz-Josef Claes <hjclaes>
Project Administrator
Wed 07 Jan 2015 02:55:22 PM UTC, original submission:  

My computer has 600,000 files (80 GB) and the first backup to a USB disk took 5 hours to complete.

Before I had a complete backup, several backups were interrupted (for various reasons: computer went to sleep, usb was disconnected, some errors crashed storeBackup...) From what I understand, storeBackup does not reuse any work it has done during incomplete backups. So at each new attempt, everything has to be done all over again (including file checksumming and compression). Overall, it took me three days to achieve a successful first backup, with was quite stressful because I had no other backup of my data during these three days and I had no idea how long it was going to take (or wether storeBackup was a good solution for me).

This situation may not be uncommon in real life: when your previous backup system has failed you, it is very urgent to find and implement a new one (and that's when you try new software like storeBackup). Another common case is when you have only one backup disk and you need to erase it (that is lose your previous backups) before starting over with storeBackup: until your first backup succeed, you have no backup of your data whatsoever and no idea how long it's going to take before you manage to have one.

I had a similar experience a year ago trying to backup a server over a network.

Most other backup systems (at least commercial ones and those based on synchronisation + versioning) allow you to resume backups.

Is there any way (or any plan) to have storeBackup reuse at least some of the work done during incomplete backups? (preferably all of it). It would help a lot for first backups, which currently require multiple attempts spanning several days until, at last, one of them succeed.

It's especially important nowadays when computers (especially developer's computers) often contain huge amount of files and data (numerous large libraries in several programming languages, large databases and VM files, numerous and large videos, etc.)

Vianney Stroebel <viybel>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hjclaes (Posted a comment)
  • -email is unavailable- added by viybel (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.7