mainstoreBackup - Support: sr #109396, Source code respository; Cannot...

 
 

sr #109396: Source code respository; Cannot set hard links when original directory has perms 644

Submitter:  None
Submitted:  Mon 09 Oct 2017 07:07:26 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 01 Nov 2017 02:46:58 PM UTC, comment #9: 

Nice to see we found an option that fulfils your needs.



Just to add point 2 below (which is totally corrupted):

2. (more important) It is impossible to restore individual files in the sourceDir without the information stored in the .md2CheckSum file. If you have two times the same file in the backup, one with the ownership of user1 and the other with the ownership of user2, storeBackup will store the file physically once with two hardlinks. Because there is only one inode, one set of ownership (and permissions) cat be stored: user1 OR user2.
So you need the information gone (because of the hardlink (deduplication)) somewhere else.
Another need to store additional meta data is restoring of hard links. Create directory "s" with the following contents:

$ cd s
$ cp /bin/ls .
$ cp ls ls2
$ ln ls ls1
$ ls -l
insgesamt 384
-rwxr-xr-x 2 hjc hjc 129696 Nov  1 15:36 ls
-rwxr-xr-x 2 hjc hjc 129696 Nov  1 15:36 ls1
-rwxr-xr-x 1 hjc hjc 129696 Nov  1 15:36 ls2

It's two files with three hard links.
In the backup, you can see one file (compressed) with three hard links. If you restore (with storeBackupRecover), you will get the same as in the source (s) directory. This is possible because of additional meta data only.

 Heinz-Josef Claes <hjclaes>
Group administrator
Tue 31 Oct 2017 06:02:01 PM UTC, comment #8: 


> Option "ignorePerms" of storeBackup.pl should solve your problem.


Yes, this addresses my use case perfectly.

For whatever reason, I never noticed this option before.

> I hope we now got it :-)


Yes.  Thank you.

And thanks for your work on this software, including taking the time to answer questions like this.  It is much appreciated.

<mfreemon>
Fri 27 Oct 2017 07:11:42 PM UTC, comment #7: 

Hi,

Option "ignorePerms" of storeBackup.pl should solve your problem. Please search all occurrences of "ignorePerms" in the pdf-documentation to get an explanation of that option.



just to explain: storeBackup normally sets the permission in the same way as in the sourceDir to allow user1 to see his file and not the files of user2. This also means that directories are created with 0700 permissions so no other user than the creator of the backup can read anything. This is the reason why you see direcotories with 0700 when using ignorePrems.

So, there is a need to (normally) set the file permissions correctly. It stores the permissions in .md5CheckSum additionally for three reasons:
1. changing the permissions in the backup by accident should not affect a restrore
2. (more important) It impossible to restore individual files in the sourceDir withou the information stored in .md2ChckSum file.
3. storeBackupUpdateBackup needs the information (in the actual implmenetation)

------

I hope we now got it :-)

Heinz-Josef Claes <hjclaes>
Group administrator
Tue 24 Oct 2017 05:33:30 PM UTC, comment #6: 

Thanks for responding.

I do understand that running things as root can avoid security problems like this, but I would respectfully suggest that is not the right answer for this situation.  Let me explain.

My use case is straightforward.  I want to run storeBackup (as root) on a source machine, with a destination on a remote filesystem, with lateLinks.  I'm using sshfs (not NFS) for the remote filesystem.  The ssh authentication/communication is as an ordinary regular linux user ("user1").  The only thing the backend storage server sees is activity by "user1".  Everything in the backup tree is owned by "user1".  storeBackupUpdateBackup runs as the regular user ("user1") on the backend storage server.  And this worked fine, until this "funny" directory popped up.

I don't want to run storeBackup as root on the storage machine, nor should it be necessary.  Indeed, we could easily envision situations where "user1" cannot run things as root on the backend storage server.

The suggestion of using "--exceptDirs" is obviously inconsistent with backing up the directories/files.  I also don't feel like the right answer is to manually modify directories/files like this after they have been installed by yum/dnf/apt.  Although this "644" directory is unusual, it is not invalid or illegal, so the backup system should be able to handle it.

What about the following as a possible solution:

We know that storebackup doesn't actually care about the permissions on the filesystem because that information is stored in .md5CheckSums.bz2.

Why not just set the permissions for all directories in the target tree to 755?

 <mfreemon>
Sun 22 Oct 2017 02:36:29 PM UTC, comment #5: 

Hi, thanks for the explanation and sorry for the late response. I think I now understand what's going wrong.

First of all, storeBackup does nothing "magic" or any "tricks" about permissioning. It simply runs as the user you start it.

---------------

If you're root, you can do whatever you want. Example:
root@fuhjc:/tmp/a# mkdir sub
root@fuhjc:/tmp/a# echo hello > sub/x
root@fuhjc:/tmp/a# chmod 000 sub
root@fuhjc:/tmp/a# cat sub/x
hello
root@fuhjc:/tmp/a# chmod 644 sub
root@fuhjc:/tmp/a# cat sub/x
hello

So if you run storeBackup as root, you can run your backup without any issues about permissions.

---------------

If you run storeBackup (or storeBackupUpdateBackup) with non root permissions, you can whatever you can do as a "normal" user:

root@fuhjc:/tmp/a# chown -R hjc.hjc sub
root@fuhjc:/tmp/a# exit
hjc@fuhjc:/tmp/a$ id
uid=1049(hjc) gid=1049(hjc) Gruppen=1049(hjc),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),115(lpadmin),130(sambashare),134(libvirtd
hjc@fuhjc:/tmp/a$ ls -l
insgesamt 0
drw-r--r-- 2 hjc hjc 60 Okt 22 16:03 sub
hjc@fuhjc:/tmp/a$ cat sub/x
cat: sub/x: no permission
hjc@fuhjc:/tmp/a$ cp sub/x /tmp/x
cp: call of stat not possible for 'sub/x' : no permissions

### I translated the error message from German to English, so the may be different on genuine English systems.

And that's what is the case of the error message in your example. The following lines are copied from you example:

81527 4 drw-r--r-- 2 user1 user1 4096 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub
29475 4 -rw-r--r-- 1 user1 user1 6 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub/file2

ERROR 2017.10.13 17:26:02 13165 could not link/copy /tmp/a/b/default/2017.10.13_17.24.36/../2017.10.13_17.07.22/sub/file2 /tmp/a/b/default/2017.10.13_17.24.36/sub/file2

And because the file could not be copied, there's a second error message that it could not be accessed when setting the file permissions. (It's simply not there.)
ERROR 2017.10.13 17:26:02 13165 cannot access </tmp/a/b/default/2017.10.13_17.24.36/sub/file2>

---------------

So if you want to get rid the of problem, you have to run storeBackupUpdate.pl as root (with option no_root_squash if you use NFS).
If you want to run storeBackupUpdateBackup.pl as non-root, you may use option "--exceptDirs sub" when running storeBackup.pl. Or change the permissions of that "funny" file before starting the Backup.

Hope I got it this time :-)
Heinz-Josef

 Heinz-Josef Claes <hjclaes>
Group administrator
Fri 13 Oct 2017 10:50:16 PM UTC, comment #4: 

<tt>
I think there are some differences between our two scenarios.

  • You have the backup tree owned by root and processes running as root
  • You stopped short of a second UpdateBackup run that would have tried to create hard links


Here is maybe a clearer description of my scenario:

Source tree is (note that directory s/sub is 644):

root@myhost:/tmp/a# find s -ls
    81521      4 drwxrwxr-x   3 root     root         4096 Oct 13 17:01 s
    81522      4 drw-r--r--   2 root     root         4096 Oct 13 17:01 s/sub
    13262      4 -rw-r--r--   1 root     root           11 Oct 13 17:01 s/sub/file2
    13077      4 -rw-rw-r--   1 root     root            6 Oct 13 17:01 s/2
    13170      4 -rw-rw-r--   1 root     root            6 Oct 13 17:01 s/file1

as root, run:

root@myhost:/tmp/a# storeBackup.pl -s s -b b --lateLinks --lateCompress

result is:

root@myhost:/tmp/a# find b -ls
    81523      4 drwxrwxr-x   3 root     root         4096 Oct 13 17:07 b
    81524      4 drwx------   3 root     root         4096 Oct 13 17:07 b/default
    81525      4 drwxr-xr-x   4 root     root         4096 Oct 13 17:07 b/default/2017.10.13_17.07.22
    81526      4 drwx------   2 root     root         4096 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks
    13505      4 -rw-------   1 root     root          179 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/linkFile.bz2
    13504      0 -rw-------   1 root     root            0 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/backup.Finished
    13265      4 -rw-------   1 root     root          974 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.info
    81527      4 drwx------   2 root     root         4096 Oct 13 17:07 b/default/2017.10.13_17.07.22/sub
    29475      4 -rw-------   1 root     root            6 Oct 13 17:07 b/default/2017.10.13_17.07.22/sub/file2
    29477      4 -rw-------   1 root     root            6 Oct 13 17:07 b/default/2017.10.13_17.07.22/2
    13497      4 -rw-r--r--   1 root     root          270 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.bz2

However, my backup directory is on a remote filesystem, which changes the ownership, so it really looks like this:

root@myhost:/tmp/a# find b -ls
    81523      4 drwxrwxr-x   3 user1    user1        4096 Oct 13 17:07 b
    81524      4 drwx------   3 user1    user1        4096 Oct 13 17:07 b/default
    81525      4 drwxr-xr-x   4 user1    user1        4096 Oct 13 17:07 b/default/2017.10.13_17.07.22
    81526      4 drwx------   2 user1    user1        4096 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks
    13505      4 -rw-------   1 user1    user1         179 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/linkFile.bz2
    13504      0 -rw-------   1 user1    user1           0 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/backup.Finished
    13265      4 -rw-------   1 user1    user1         974 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.info
    81527      4 drwx------   2 user1    user1        4096 Oct 13 17:07 b/default/2017.10.13_17.07.22/sub
    29475      4 -rw-------   1 user1    user1           6 Oct 13 17:07 b/default/2017.10.13_17.07.22/sub/file2
    29477      4 -rw-------   1 user1    user1           6 Oct 13 17:07 b/default/2017.10.13_17.07.22/2
    13497      4 -rw-r--r--   1 user1    user1         270 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.bz2

as user1, run:

user1@myhost:/tmp/a$ storeBackupUpdateBackup.pl -b b

result is:

root@myhost:/tmp/a# find b -ls
    81523      4 drwxrwxr-x   3 user1    user1        4096 Oct 13 17:07 b
    81524      4 drwx------   3 user1    user1        4096 Oct 13 17:07 b/default
    81525      4 drwxr-xr-x   4 user1    user1        4096 Oct 13 17:17 b/default/2017.10.13_17.07.22
    81526      4 drwx------   2 user1    user1        4096 Oct 13 17:17 b/default/2017.10.13_17.07.22/.storeBackupLinks
    13504      0 -rw-------   1 user1    user1           0 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/backup.Finished
    13265      4 -rw-------   1 user1    user1         974 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.info
    81527      4 drw-r--r--   2 user1    user1        4096 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub
    29475      4 -rw-r--r--   1 user1    user1           6 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub/file2
    29477      4 -rw-rw-r--   2 user1    user1           6 Oct 13 17:01 b/default/2017.10.13_17.07.22/2
    29477      4 -rw-rw-r--   2 user1    user1           6 Oct 13 17:01 b/default/2017.10.13_17.07.22/file1
    13497      4 -rw-r--r--   1 user1    user1         270 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.bz2

This result "looks" ok, but we already have a problem.

user1@myhost:/tmp/a$ cd b/default/2017.10.13_17.07.22/sub/
-su: cd: b/default/2017.10.13_17.07.22/sub/: Permission denied
user1@myhost:/tmp/a$ cat b/default/2017.10.13_17.07.22/sub/file2
cat: b/default/2017.10.13_17.07.22/sub/file2: Permission denied

The "cd" error is expected, based on the permissions bits.  The second one surprised me personally.  I expected that to work, so there is obviously something I don't quite understand going on there.

However, storeBackup has not yet encountered a problem.  Not yet.

I run a second backup.

as root, run:

root@myhost:/tmp/a# storeBackup.pl -s s -b b --lateLinks --lateCompress

result is:

root@myhost:/tmp/a# find b -ls
    81523      4 drwxrwxr-x   3 user1    user1        4096 Oct 13 17:07 b
    81524      4 drwx------   4 user1    user1        4096 Oct 13 17:24 b/default
    81525      4 drwxr-xr-x   4 user1    user1        4096 Oct 13 17:17 b/default/2017.10.13_17.07.22
    81526      4 drwx------   2 user1    user1        4096 Oct 13 17:24 b/default/2017.10.13_17.07.22/.storeBackupLinks
    29481      4 -rw-------   1 user1    user1          23 Oct 13 17:24 b/default/2017.10.13_17.07.22/.storeBackupLinks/linkFrom0
    13504      0 -rw-------   1 user1    user1           0 Oct 13 17:07 b/default/2017.10.13_17.07.22/.storeBackupLinks/backup.Finished
    13265      4 -rw-------   1 user1    user1         974 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.info
    81527      4 drw-r--r--   2 user1    user1        4096 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub
    29475      4 -rw-r--r--   1 user1    user1           6 Oct 13 17:03 b/default/2017.10.13_17.07.22/sub/file2
    29477      4 -rw-rw-r--   2 user1    user1           6 Oct 13 17:01 b/default/2017.10.13_17.07.22/2
    29477      4 -rw-rw-r--   2 user1    user1           6 Oct 13 17:01 b/default/2017.10.13_17.07.22/file1
    13497      4 -rw-r--r--   1 user1    user1         270 Oct 13 17:07 b/default/2017.10.13_17.07.22/.md5CheckSums.bz2
    81528      4 drwxr-xr-x   3 user1    user1        4096 Oct 13 17:24 b/default/2017.10.13_17.24.36
    81529      4 drwx------   2 user1    user1        4096 Oct 13 17:24 b/default/2017.10.13_17.24.36/.storeBackupLinks
    29480      4 -rw-------   1 user1    user1          23 Oct 13 17:24 b/default/2017.10.13_17.24.36/.storeBackupLinks/linkTo
    29479      4 -rw-------   1 user1    user1         255 Oct 13 17:24 b/default/2017.10.13_17.24.36/.storeBackupLinks/linkFile.bz2
    29478      0 -rw-------   1 user1    user1           0 Oct 13 17:24 b/default/2017.10.13_17.24.36/.storeBackupLinks/backup.Finished
    29474      4 -rw-------   1 user1    user1         974 Oct 13 17:24 b/default/2017.10.13_17.24.36/.md5CheckSums.info
    29476      4 -rw-r--r--   1 user1    user1         268 Oct 13 17:24 b/default/2017.10.13_17.24.36/.md5CheckSums.bz2

and then the update backup:

as user1, run:

user1@myhost:/tmp/a$ storeBackupUpdateBackup.pl -b b

BEGIN     2017.10.13 17:26:02 13165 checking references and backup copying in b
VERSION   2017.10.13 17:26:02 13165 storeBackupUpdateBackup.pl, 3.5
INFO      2017.10.13 17:26:02 13165 creating lock file </tmp/storeBackup.lock>
INFO      2017.10.13 17:26:02 13165 scanning directory b for existing backups
INFO      2017.10.13 17:26:02 13165 scanning directory b/default for existing backups
STATISTIC 2017.10.13 17:26:02 13165 found 1 backup series, 2 backups, 0 renamed backups
INFO      2017.10.13 17:26:02 13165 consistency check finished successfully
INFO      2017.10.13 17:26:02 13165 listing references:
INFO      2017.10.13 17:26:02 13165   /tmp/a/b/default/2017.10.13_17.24.36
INFO      2017.10.13 17:26:02 13165     -> /tmp/a/b/default/2017.10.13_17.07.22
INFO      2017.10.13 17:26:02 13165 (1/1) updating </tmp/a/b/default/2017.10.13_17.24.36>
INFO      2017.10.13 17:26:02 13165 phase 1: mkdir, symlink and compressing files
STATISTIC 2017.10.13 17:26:02 13165 created 1 directories
STATISTIC 2017.10.13 17:26:02 13165 created 0 symbolic links
STATISTIC 2017.10.13 17:26:02 13165 compressed 0 files
STATISTIC 2017.10.13 17:26:02 13165 used 0.0  instead of 0.0  (0 <- 0 ; 0.0%)
INFO      2017.10.13 17:26:02 13165 phase 2: setting hard links
ERROR     2017.10.13 17:26:02 13165 could not link/copy /tmp/a/b/default/2017.10.13_17.24.36/../2017.10.13_17.07.22/sub/file2 /tmp/a/b/default/2017.10.13_17.24.36/sub/file2
STATISTIC 2017.10.13 17:26:02 13165 linked 2 files
INFO      2017.10.13 17:26:02 13165 phase 3: setting file permissions
ERROR     2017.10.13 17:26:02 13165 cannot access </tmp/a/b/default/2017.10.13_17.24.36/sub/file2>
STATISTIC 2017.10.13 17:26:02 13165 set permissions for 2 files
INFO      2017.10.13 17:26:02 13165 phase 4: setting directory permissions
STATISTIC 2017.10.13 17:26:02 13165 set permissions for 1 directories
INFO      2017.10.13 17:26:02 13165 scanning directory b for existing backups
INFO      2017.10.13 17:26:02 13165 scanning directory b/default for existing backups
STATISTIC 2017.10.13 17:26:02 13165 found 1 backup series, 2 backups, 0 renamed backups
INFO      2017.10.13 17:26:02 13165 consistency check finished successfully
INFO      2017.10.13 17:26:02 13165 found no references to backups from lateLinks that need storeBackupUpdateBackup run
STATISTIC 2017.10.13 17:26:02 13165                       duration = 1s
STATISTIC 2017.10.13 17:26:02 13165  [sec] |      user|    system
STATISTIC 2017.10.13 17:26:02 13165 -------+----------+----------
STATISTIC 2017.10.13 17:26:02 13165 process|      0.08|      0.00
STATISTIC 2017.10.13 17:26:02 13165 childs |      0.05|      0.00
STATISTIC 2017.10.13 17:26:02 13165 -------+----------+----------
STATISTIC 2017.10.13 17:26:02 13165 sum    |      0.13|      0.00 => 0.13
INFO      2017.10.13 17:26:02 13165 syncing ...
END       2017.10.13 17:26:02 13165 checking references and copying in b

Note the error.
</tt>

 <mfreemon>
Fri 13 Oct 2017 12:23:23 PM UTC, comment #3: 

Hi,
to be honest, I don't understand what's happening :-/

I've done the following:
hjc@umpf:/tmp$ rm -r a
hjc@umpf:/tmp$ mkdir -p a/s/sub a/b
hjc@umpf:/tmp$ cd a
hjc@umpf:/tmp/a$ touch s/file1 s/file2 s/sub/file3
hjc@umpf:/tmp/a$ sudo -s
root@umpf:/tmp/a# chmod -R 644 s/sub
root@umpf:/tmp/a# chown -R root.root s/sub
root@umpf:/tmp/a# find s -ls
  2363531      0 drwxrwxr-x   3 hjc      hjc           100 Okt 13 14:01 s
  2365066      0 -rw-rw-r--   1 hjc      hjc             0 Okt 13 14:01 s/file2
  2365065      0 -rw-rw-r--   1 hjc      hjc             0 Okt 13 14:01 s/file1
  2363532      0 drw-r--r--   2 root     root           60 Okt 13 14:01 s/sub
  2365067      0 -rw-r--r--   1 root     root            0 Okt 13 14:01 s/sub/file3
root@umpf:/tmp/a# /opt/storeBackup/bin/storeBackup.pl -s s -b b --lateLinks
root@umpf:/tmp/a# find b -ls
  2363533      0 drwxrwxr-x   3 hjc      hjc            60 Okt 13 14:03 b
  2380667      0 drwx------   3 root     root           60 Okt 13 14:03 b/default
  2383411      0 drwxr-xr-x   3 root     root          120 Okt 13 14:03 b/default/2017.10.13_14.03.51
  2383441      0 -rw-------   1 root     root            0 Okt 13 14:03 b/default/2017.10.13_14.03.51/file2
  2384132      4 -rw-r--r--   1 root     root          244 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.bz2
  2383413      4 -rw-------   1 root     root          979 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.info
  2383412      0 drwx------   2 root     root           80 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks
  2383443      0 -rw-r--r--   1 root     root            0 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks/backup.Finished
  2380691      4 -rw-------   1 root     root          201 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks/linkFile.bz2

-> one of the files is created in the backup,
   the others are listed in linkFile.bz2 because of option lateLinks

root@umpf:/tmp/a# bzegrep -v '^#' b/default/2017.10.13_14.03.51/.storeBackupLinks/linkFile.bz2
dir sub
link d41d8cd98f00b204e9800998ecf8427e
./file2
file1
link d41d8cd98f00b204e9800998ecf8427e
./file2
sub/file3

-> all the files belong to user root. Now I start updateBackup as "normal" user:

root@umpf:/tmp/a# exit
hjc@umpf:/tmp/a$ /opt/storeBackup/bin/storeBackupUpdateBackup.pl -b b
BEGIN     2017.10.13 14:07:19 29184 checking references and backup copying in <_B_>
VERSION   2017.10.13 14:07:19 29184 storeBackupUpdateBackup.pl, 3.5.1 pre
INFO      2017.10.13 14:07:19 29184 creating lock file </tmp/storeBackup.lock>
INFO      2017.10.13 14:07:19 29184 scanning directory <_B_> for existing backups
INFO      2017.10.13 14:07:19 29184 scanning directory <_B_/default> for existing backups
STATISTIC 2017.10.13 14:07:19 29184 found 1 backup series, 0 backups, 0 renamed backups
INFO      2017.10.13 14:07:19 29184 consistency check finished successfully
INFO      2017.10.13 14:07:19 29184 found no references to backups from lateLinks that need storeBackupUpdateBackup run
INFO      2017.10.13 14:07:19 29184 everything is updated, nothing to do

hjc@umpf:/tmp/a$ find b -ls
  2363533      0 drwxrwxr-x   3 hjc      hjc            60 Okt 13 14:03 b
  2380667      0 drwx------   3 root     root           60 Okt 13 14:03 b/default
find: ‘b/default’: no permissions

-> back to user root

hjc@umpf:/tmp/a$ sudo -s
root@umpf:/tmp/a# find b -ls
  2363533      0 drwxrwxr-x   3 hjc      hjc            60 Okt 13 14:03 b
  2380667      0 drwx------   3 root     root           60 Okt 13 14:03 b/default
  2383411      0 drwxr-xr-x   3 root     root          120 Okt 13 14:03 b/default/2017.10.13_14.03.51
  2383441      0 -rw-------   1 root     root            0 Okt 13 14:03 b/default/2017.10.13_14.03.51/file2
  2384132      4 -rw-r--r--   1 root     root          244 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.bz2
  2383413      4 -rw-------   1 root     root          979 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.info
  2383412      0 drwx------   2 root     root           80 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks
  2383443      0 -rw-r--r--   1 root     root            0 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks/backup.Finished
  2380691      4 -rw-------   1 root     root          201 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks/linkFile.bz2
/opt/storeBackup/bin/storeBackupUpdateBackup.pl -b b

root@umpf:/tmp/a# find b -ls
  2363533      0 drwxrwxr-x   3 hjc      hjc            60 Okt 13 14:03 b
  2380667      0 drwx------   3 root     root           60 Okt 13 14:03 b/default
  2383411      0 drwxr-xr-x   4 root     root          160 Okt 13 14:09 b/default/2017.10.13_14.03.51
  2383441      0 -rw-r--r--   3 root     root            0 Okt 13 14:01 b/default/2017.10.13_14.03.51/file1
  2415562      0 drw-r--r--   2 root     root           60 Okt 13 14:01 b/default/2017.10.13_14.03.51/sub
  2383441      0 -rw-r--r--   3 root     root            0 Okt 13 14:01 b/default/2017.10.13_14.03.51/sub/file3
  2383441      0 -rw-r--r--   3 root     root            0 Okt 13 14:01 b/default/2017.10.13_14.03.51/file2
  2384132      4 -rw-r--r--   1 root     root          244 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.bz2
  2383413      4 -rw-------   1 root     root          979 Okt 13 14:03 b/default/2017.10.13_14.03.51/.md5CheckSums.info
  2383412      0 drwx------   2 root     root           60 Okt 13 14:09 b/default/2017.10.13_14.03.51/.storeBackupLinks
  2383443      0 -rw-r--r--   1 root     root            0 Okt 13 14:03 b/default/2017.10.13_14.03.51/.storeBackupLinks/backup.Finished


-> Why can you run storeBackupUpdateBackup without being root?
   Did I miss something?

Heinz-Josef

 Heinz-Josef Claes <hjclaes>
Group administrator
Wed 11 Oct 2017 05:50:40 PM UTC, comment #2: 

Hello.  Thanks for the quick response.

A specific example of this problem occurs on Fedora 25, when package pyxattr is installed.

That package creates the following items in the filesystem:

directory  /usr/share/doc/pyxattr         root root 644
file       /usr/share/doc/pyxattr/NEWS    root root 644
file       /usr/share/doc/pyxattr/README  root root 644

I use lateLinks and lateCompress.

During the first backup, everything works fine (both the storeBackup.pl and storeBackupUpdateBackup.pl).

storeBackup.pl runs as root.

Note that all files in the backup tree have a uid/gid of a single regular linux user (not root), which I will refer to as "user1".

storeBackupUpdateBackup.pl runs as "user1".

The items above are created in the backup tree as follows:

directory  /usr/share/doc/pyxattr             user1 user1 644
file       /usr/share/doc/pyxattr/NEWS.bz2    user1 user1 644
file       /usr/share/doc/pyxattr/README.bz2  user1 user1 644

The problem occurs starting on the second backup run.

storeBackup.pl runs fine.

During storeBackupUpdateBackup.pl on the server, I get these errors when it tries to hard link those two files:

"ERROR ... could not link/copy /.../usr/share/doc/pyxattr/NEWS.bz2 /.../usr/share/pyxattr/NEWS.bz2"
"ERROR ... could not link/copy /.../usr/share/doc/pyxattr/README.bz2 /.../usr/share/pyxattr/README.bz2"

The directory is created for the new backup as follows:

directory  /usr/share/doc/pyxattr             user1 user1 644

The two hard links are of course not created.

These errors occur on all subsequent backups.

When I change the directory permissions (for usr/share/doc/pyxattr) in the backup tree to 755, everything works fine.

The filesystem for the server is zfs, although I don't think that matters.  I did some simple command line tests with ext4 and found the same behavior.

Thanks for your assistance.

Anonymous
Wed 11 Oct 2017 10:16:28 AM UTC, comment #1: 

I tried to reproduce your problem and got:
- if I use a non-root user, I cannot read the contents of the 644 directory in sourceDir.
- if I'm using root, everything works fine.

This is expected behavior. So - if I understand you well, you got a problem depending on the read problem described above!?

Sorry for the question - I'm not sure about the origin your problem.

A source code repository is not publicly available - since some years [ :-( ] my time available for storeBackup is very limited so I cannot handle an open repository. I began to implement some new features but didn't had the time to finish them up to now :-(

Thanks for your patience
Heinz-Josef

 Heinz-Josef Claes <hjclaes>
Group administrator
Mon 09 Oct 2017 07:07:26 PM UTC, original submission:  

This is excellent software.  Thank you.  Like an earlier thread, I'm surprised this is not more popular.

When using lateLinks, the server-side UpdateBackup fails to create hard links when the original directory containing the file to be hard linked has permissions 644.

You have mentioned in these threads that you have fixed several issues since v3.5 was release in April 2014.

Has this problem been identified previously and fixed?

I am happy to dig further and submit a patch, but I have been unable to find your source code repository with your latest commits.  It is available publically?

Thank you.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mfreemon (Posted a comment)
  • -email is unavailable- added by hjclaes (Posted a comment)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-8eb0.
    Corresponding source code