mainstoreBackup - Support: sr #107096, preCommand runs after exceptDirs...


sr #107096: preCommand runs after exceptDirs matches.

Submitter:  None
Submitted:  Sat 31 Oct 2009 08:26:42 PM UTC
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup

Jump to the original submission

Sun 22 Nov 2009 07:06:23 AM UTC, comment #6: 

I do think your idea would probably be the easiest to understand.  I'm not sure if there would be an advantage to both.  At least I can't think of any at the moment.

When I mentioned the popularity, I was actually thinking about other Linux command line alternatives such as rsnapshot, rdiff-backup, and even backuppc which all seem to be more popular.  Thanks again.

Marc <blklodge>
Fri 20 Nov 2009 09:32:11 AM UTC, comment #5: 

But in your way we also need two options for only one choice. Also that way I must explain the difference.
I think this is quiet the same `luxuriousness' than having two independent options.

Naturally, it will not be more difficult to set it up with two pre commands in a simple way: -s <something> -b <somethingelse>

Perhaps I should implement both - even with only one option (like now) I have to explain when it's called.

BTW, I think it's not more popular because it has no GUI and is not DOS compatible ;-) ;-)
To be serious: It's too complicated for the mass of the people: They need to understand what's a hard link, a checksum, how to use a text editor - not to mention the complicated parts, like rules or the 'blocked' stuff.
Most people simply do not understand that one copy is not a backup.

Heinz-Josef Claes <hjclaes>
Group administrator
Fri 20 Nov 2009 02:13:37 AM UTC, comment #4: 

I agree that is doesn't make sense to have multiple pre command options as I also think that will get confusing.  This is a minor enhancement as all this can be avoided with a wrapper script, but the ability to use the single cron script and just have one configuration for each backup in /etc/storebackup without anything additional is a "luxury", but also a nice one.  One of the great strengths of this tool is how easy it is to set up.

One idea could be one new parameter that specifies when the pre command script is executed.  This would make it fairly easy to document also.  For Example:

--precommandExecTime / precommandExecTime
    Determines when the precommand is executed.  Possible values are:
    1 - Execute immediately after lock is checked
    2 - Execute after directory checks
Letters could be used etc., but that's just a rough idea that should be easy to implement.  I'm sure there is a better name for that parameter as well.

BTW, I forgot to mention this is already an excellent backup tool that exceeds most other options.  That along with the superb documentation makes me wonder why it isn't more popular.  Thanks for you hard work!

Marc <blklodge>
Thu 19 Nov 2009 08:10:34 AM UTC, comment #3: 

So there are at least two typical situations:

1. pre command before anything of the backup runs
2. pre command after all checks (before the real copying etc. starts) are done

The first one can be implemented easily with a little shell script. You just have to make sure to write some qualified messages to the log file and to stop if something goes wrong. In this case you do not have the `luxury' of automated redirection of stdout, stderr and qualification as WARNING, ERROR, timestand, pid etc.

The second one can be called from storeBackup only.

I also think it does not make sense to implement both (but I'm not sure), because this would lead to even more confusion about too many options - or do you think it is a good idea to have both? (not a difficult task to implement this, except to use descriptive names ;-) )

Heinz-Josef Claes <hjclaes>
Group administrator
Thu 19 Nov 2009 06:10:48 AM UTC, comment #2: 

No worries on the reply time. 
Looks like I wasn't even logged in when I submitted the issue.

As far as the order of commands, I think it would depend on what you want to do in the precommand.  I use the precommand to take an LVM snapshot of a volume and mount it.  This is what actually gets backed up.  Since the source volume is not mounted when the pattern matches are checked, I get the "path or pattern of exceptDir does not exist" and "no directory resulting from pattern" errors.  By setting contExceptDirsErr=yes in the config, the backup still works, so it's not a huge issue, but you loose the ability to do those checks.

I'm sure there are many cases where you may want those checks to happen first if your precommand is executed, so there may not be an easy answer.

Marc <blklodge>
Wed 18 Nov 2009 12:20:40 PM UTC, comment #1: 

Sorry for the late response :-(

You're right, I'll change it in the next release.

But I prefer the order:
1. Check the lock
2. Process include/exclude matches and other checks.
3. Execute preCommand

because step 2 also can result in an exit of the program.

Do you agree?

Heinz-Josef Claes <hjclaes>
Group administrator
Sat 31 Oct 2009 08:26:42 PM UTC, original submission:  

Currently, the exceptDirs matches (as well as directory checking and probably others) run before the lock file is checked.  Therefore, the preCommand script runs after the exceptDirs matching. 

For example, I am trying to create and mount LVM snapshots ina preCommand script which is what will be backed up.  However, since the exceptDirs does not find any matches as the source directory is empty/unmounted, it still backs up all my excluded directories.

Wouldn't it make more sense to process items in this order:
1. Check the lock
2. Execute preCommand
3. Process include/exclude matches and other checks.

This would allow the preCommand to be used to prepare everything for the backup first.  With the current setup, the cron-storebackup job can't be utilized, and storeBackup is unaware of failures run outside of it's context.



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by blklodge (Posted a comment)
  • -email is unavailable- added by hjclaes (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.


    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-05-31 hjclaes StatusNone Done

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code