mainstoreBackup - Support: sr #107586, Blocked Files MD5sums will not be...


sr #107586: Blocked Files MD5sums will not be saved if original file was changed during backup

Submitter:  None
Submitted:  Thu 03 Feb 2011 03:11:57 PM UTC
Category:  None Priority:  5 - Normal
Severity:  4 - Important Status:  Done
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup

Wed 09 Feb 2011 04:10:39 PM UTC, comment #1: 


storeBackup simply does the following for blocked files:

It simply checks timestamps and size of a file to determine if there are any changes.

If there are no changes in size or timestamps, it will generate the directory for that blocked file and make hard links to the contents of the old one.

If there are changes in size or timestamps. it will open (perl function sysopen) the file and read blocks (with sysread) of the specified size until the end of the file. If sysread does not deliver data any more (read #bytes == 0), it stops.
(It calculates md5 sums of these blocks, etc.)

So, I've now idea from your description, what can go wrong (beside corrupt data because the vm is still running). storeBackup doesn't care about changes to the file.

In function
sub calcBlockMD5Sums
in there are many print statements. Can you uncomment them, run storeBackup again and send me the result?

The error messages you included do not help, they simply say that some variables for statistical use were not initialized, so something really has gone wrong.

Please write directly to my email address and append files / output as a separate file so I will not have to fight with stupid line feeds.

Thanks, HJC

Heinz-Josef Claes <hjclaes>
Group administrator
Thu 03 Feb 2011 03:11:57 PM UTC, original submission:  


My Case:

Ubuntu 10.10 x64 running a VM with VMWare Workstation 7.1.3

Its .vmdk is blocked by a rule.

However, I found that the hard link counter doesn't increase as I thought. I reduced the block sizes, but the problem persisted.

After running with debug I found that it is a problem that the VM is still running (I know that the backup will be probably worse than crash level within VM OS view).

It seems that if the a/ctime of the vmdk changes during the blocking/backup process, storeBackup won't save the MD5 checksums to the MD5 file at all.

In the second this happens, storeBackup skips that file and STDOUTs:

Not a HASH reference at /usr/bin/storeBackup line 3600.
Use of uninitialized value $size in addition (+) at /usr/bin/storeBackup line 7089.
Use of uninitialized value in addition (+) at /usr/bin/storeBackup line 7131.
Use of uninitialized value $statNoBlocks in addition (+) at /usr/bin/storeBackup line 6671.

Is there anything I can do in this regard? For me, this sounds like a bug. The file gets backupped though, but not hardlinked.

Or do I need to freeze/suspend VMware Workstation before somehow?


This is the used backup spec (no config file):

/usr/bin/nice /usr/bin/ionice -c 3 /usr/bin/storeBackup \
 --sourceDir / \
 -e data \
 -e tmp \
 -e proc \
 -e sys \
 -e dev \
 --backupDir /data/backup-pdbxe100/ \
 -S pdbxe100 \
 --linkSymlinks \
 --keepFirstOfWeek  180d2h \
 --keepFirstOfMonth 720d2h \
 --keepMaxNumber 500 \
 --minCompressSize 16384 \
 --logFile /data/backup-pdbxe100/storeBackup.log \
 --checkBlocksSuffix '\.vmdk' \
 --checkBlocksSuffix '\.pst' \
 --checkBlocksBS 128k



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by hjclaes (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.


    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-05-31 hjclaes StatusNone Done

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code