bugstoreBackup - Bugs: bug #53204, Option to limit the number of...

 
 

bug #53204: Option to limit the number of hard-links

Submitter:  None
Submitted:  Thu 22 Feb 2018 12:15:37 PM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 04 Mar 2018 12:43:45 PM UTC, comment #4: 

Hi,

I added option maxHardLinks to linkToDirs.pl, storeBackup.pl and storeBackupUpdateBackup.pl.
Use option --help to see how this works - no other documentation up to now.

It seems to work for me (in several tests). Please send me a direct email so I can copy the version to you. Would be nice if you could test it.

Heinz-Josef

 Heinz-Josef Claes <hjclaes>
Group administrator
Sun 25 Feb 2018 03:43:33 PM UTC, comment #3: 

I think the option "maxHardLinks" has to be implemented in the following scripts:
storeBackup.pl storeBackupUpdateBackup.pl linkToDirs.pl

I'll try to implement it in the next time and will report here so you can get a test version via email.

Regards, Heinz-Josef

 Heinz-Josef Claes <hjclaes>
Group administrator
Sun 25 Feb 2018 02:46:10 PM UTC, comment #2: 

I tried to implement such a limit but failed due to too less knowledge about perl and the storeBackup framework.

However, I want to share my ideas on how to limit the number of hardlinks to a file:
Currently, the number of maximum allowed links is checked by the return value of perl's link function, which only returns false if the filesystem limit is hit.
The idea: replace all calls to perl's link with a call to a newly written function with a check for the number of links to source file. This check can be done via stat. The new function for linking should return an error if the number of already existing links to the source-file exceeds a limit (given by an option, maybe default at 90% of filesystem's limit).
However, I spot two issues here which needs to be taken into account:
    *) Symlinks should never be linked by an option. Also use lstat for symlinks.
    *) stat was most likely already called for the source-file within somewhere of storeBackup Thus, reuse it to reduce number of disk accesses, or check every time (the number of links might already have increased since last check).

 Anonymous
Fri 23 Feb 2018 11:40:57 AM UTC, comment #1: 

+1 (could run into the same problem) but I think it's not stable because if the file is linked multiple times in the tree to replicate with rsync we have to get this undetermined number of free hardlinks (e.g. a lot for empty files) hmm

lopiuh

 lopiuh <lopiuh>
Thu 22 Feb 2018 12:15:37 PM UTC, original submission:  

First of all, thanks for this wonderful backup tool.
I'm using it extensively and wonder, if a new option could be integrated to limit the number of hard-links on a file.

Background:
I want to replicate some selected backups with hard-links to another folder (located on the same filesystem but outside storeBackup's directory structure) to generate a folder-tree suitable for my offsite-backup-tool without having to have the data twice (one in storeBackups folder-tree and once in the folder tree of the offsite-backup-tool).
For this, I would like to use rsync*). However, as storeBackup already maxes out the hard-link limit for some files (especially empty files), rsync fails with "Too many links (31)".

As you mention in FAQ 9, hard-linking empty files is a good idea and storeBackup itself can handle the limited number hard-links for ext filesystems (in contrast to other tools like rsync).
However, an option to limit the number of hard-links in order to have headroom for using tools like rsync would be very nice.

*) I'm aware of linksToDest.pl which I could also use for the task of copying a with hard-links. However, in my test setup linkToDirs.pl needs 90 minutes to replicate the desired backups whereas rsync was done in 2 minutes.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hjclaes (Posted a comment)
  • -email is unavailable- added by lopiuh (Posted a comment)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Copyright © 2025 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-04e1.
    Corresponding source code