mainstoreBackup - Support: sr #108282, Error when replicating to samba

 
 

sr #108282: Error when replicating to samba

Submitter:  None
Submitted:  Sun 14 Apr 2013 04:09:33 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 16 Apr 2013 08:49:26 AM UTC, comment #4: 

That's fine.

I just changed it in the code for the next version.

(The second replacement was depending in some changes I already made for the next version.)

I've  closed this request.

Regards, Heinz-Josef

Heinz-Josef Claes <hjclaes>
Group administrator
Mon 15 Apr 2013 09:03:57 PM UTC, comment #3: 

Thanks a lot, it works now.

I could only do the first replacement, since my fileDir.pl did not have the comment in brackets as you wrote.

Anonymous
Mon 15 Apr 2013 06:48:49 AM UTC, comment #2: 

Hi,

in lib/fileDir.pl, can you replace in subrouting 'copyDir:

 
    my $cp = forkProc->new('-exec' => 'cp',
   '-param' =>
   ['-a', "$from", "$to"],
   '-outRandom' => $tmp,
   '-prLog' => $prLog);

with


    my $cp = forkProc->new('-exec' => 'cp',
   '-param' =>
   ['-r', "$from", "$to"],
   '-outRandom' => $tmp,
   '-prLog' => $prLog);

and

$prLog->print('-kind' => 'E',
      '-str' =>
      ["copying (with 'cp -a') of <$from> to <$to> reported:",
       @$out]);

with

$prLog->print('-kind' => 'E',
      '-str' =>
      ["copying (with 'cp -r') of <$from> to <$to> reported:",
       @$out]);

?

---
explanation:

I never tested or planned to use replication with samba shares - but I hope it works with the change above. Please report if it's working with the fix, so I can make that permanently.

To copy data from the delta cache, storeBackupUpdateBackup.pl uses 'cp -a' which generates all those error messages because it cannot change permissions on you samba share. Copying those information is (in principle) the same as to run a backup with option --lateLinks. And when you make a backup with --lateLinks, no permissions etc. are set in the backups. So '-a' is not necessary at that point of replication (this means 'cp -r' can be used instead of 'cp -a'). StoreBackupUpdateBackup.pl sets the permissions of the files in the backups at the very end of its run - and it doesn't check if this was successful, so it will not generate any error messages if this is not possible. Therefore it can be run on samba shares or eg. locally as a user without any problems.

The reason for this behavior is, that storeBackup does not depend on the permissions of the files in the backup - it's a 'nice to have', but not necessary for restoring data. Permissions when restoring data are set by storeBackupRecover.pl from the meta data stored in each individual backup. And that's necessary, because due to hard linking and de-duplication, it sometimes is impossible to map diverging permissions to one file with multiple hard links. (By that way, you can also restore hard links in / from the source correctly.)

Regards, Heinz-Josef

Heinz-Josef Claes <hjclaes>
Group administrator
Sun 14 Apr 2013 04:35:25 PM UTC, comment #1: 

I also checked that if I just use the samba share as master backup (no replication) using --lateLinks it works without problems. I don't even need to use --ignorePerms.

Balazs

Anonymous
Sun 14 Apr 2013 04:09:33 PM UTC, original submission:  

Hi Heinz-Josef,

I find storeBackup great, thanks for sharing it!

I have a problem replicating a backup on a samba share. I simplified the problem to this scenario:

1. Local source directory: /test/source
Contains a single text file:
-rw-rw-r-- 1 balazs balazs 36 Apr 14 17:15 fileToBackup.txt

2. Local master backup: /test/masterBackup
Contains only config file /test/masterBackup/storeBackupBaseTree.conf:
____________________
backupTreeName='Master Backup'
backupType=master
seriesToDistribute='test'
deltaCache=/test/deltaCache
____________________

3. Local delta cache: /test/deltaCache
Contains only config file /test/deltaCache/deltaCache.conf:
____________________
backupCopy0='Test Backup Copy' 'test'
____________________

4. Remote backup copy directory in mounted directory: /mnt/samba/backupCopy
Contains only config file /mnt/samba/backupCopy/storeBackupBaseTree.conf:
____________________
backupTreeName='Test Backup Copy'
backupType=copy
seriesToDistribute='test'
deltaCache=/test/deltaCache
____________________

5. storeBackup.pl -s /test/source -b /test/masterBackup -S test --lateLinks

Runs normally. (Also tried with additional --ignorePerms)

6. storeBackupUpdateBackup.pl -b /test/masterBackup

Runs normally.

7. storeBackupUpdateBackup.pl -b /mnt/samba/backupCopy

Produces the following output:
____________________
BEGIN     2013.04.14 17:27:45 24065 checking references and backup copying in </mnt/samba/backupCopy>
VERSION   2013.04.14 17:27:45 24065 storeBackupUpdateBackup.pl, 3.3
INFO      2013.04.14 17:27:45 24065 creating lock file </tmp/storeBackup.lock>
INFO      2013.04.14 17:27:45 24065 reading </mnt/samba/backupCopy/storeBackupBaseTree.conf>
INFO      2013.04.14 17:27:45 24065 reading </test/deltaCache/deltaCache.conf>
INFO      2013.04.14 17:27:45 24065 copying </test/deltaCache/test/2013.04.14_17.24.25> to </mnt/samba/backupCopy/test>
ERROR     2013.04.14 17:27:45 24065 copying of </test/deltaCache/test/2013.04.14_17.24.25> to </mnt/samba/backupCopy/test> reported:
ERROR     2013.04.14 17:27:45 24065 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.14_17.24.25/.md5CheckSums.info': Input/output error
ERROR     2013.04.14 17:27:45 24065 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.14_17.24.25/.storeBackupLinks/linkFile.bz2': Input/output error
ERROR     2013.04.14 17:27:45 24065 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.14_17.24.25/.md5CheckSums.bz2': Input/output error
ERROR     2013.04.14 17:27:45 24065 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.14_17.24.25/fileToBackup': Input/output error
ERROR     2013.04.14 17:27:45 24065 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.14_17.24.25/.md5BlockCheckSums.bz2': Input/output error
____________________

And quits without even producing a line with "END".

In my originial scenario when I was backing up my documents I got thousands of error messages like the above and also of type
____________________
ERROR     2013.04.14 12:26:36  3507 cp: clearing permissions for `/mnt/samba/backupCopy/test/2013.04.13_14.23.33/<filename>': Input/output error
ERROR     2013.04.14 12:26:36  3507 cp: preserving permissions for `/mnt/samba/backupCopy/test/2013.04.13_14.23.33/<filename>': Invalid argument
____________________

What could be wrong?

Thanks for your help,

Balazs

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hjclaes (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-04-16 hjclaes StatusIn Progress Done
    2013-04-15 hjclaes StatusNone In Progress

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code