mainstoreBackup - Support: sr #108147, Error handling filenames...

 
 

sr #108147: Error handling filenames conatining a "\n"

Submitter:  None
Submitted:  Sun 30 Sep 2012 08:12:55 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  2 - Minor Status:  Need Info
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 06 Dec 2012 03:40:29 PM UTC, comment #6: 

Hi,

it works for me.

I started the following script to generate the file:

#! /usr/bin/perl
open(FILE, '>', "C:\n-anynametest.txt");
print FILE "abcdefghijklmnopqrstuvw" x 4000, "\n";
close(FILE);

The result is:
$ ls -lb
insgesamt 100
-rw-rw-r-- 1 hjc hjc 92001 Dez  6 16:28 C:\n-anynametest.txt

The run of storeBackup.pl results in:
$ ls -lb 2012.12.06_16.30.42/
insgesamt 12
-rw-rw-r-- 1 hjc hjc 126 Dez  6 16:30 C:\n-anynametest.txt.bz2


Can you repeat your test with version 3.3? You're still using 3.2! See how to "install" (just unpack a tar file) the version from here in the documentation of 3.3.

I've used Kubuntu 12.10 and storeBackup 3.3 (and the not yet ready next version) for my test.

Can you double check if you can repeat the problem with the actual version?
If yes, please send me an email (hjclaes@web.de) with a tar file of your source directory (with the "\n"-files).

btw, it's not a bad idea at all to use version 3.3, see http://www.nongnu.org/storebackup/en/node8.html

Best Regards, Heinz-Josef

Heinz-Josef Claes <hjclaes>
Group administrator
Wed 05 Dec 2012 11:30:44 PM UTC, comment #5: 

The attachment

Sala AlbumDiFamiglia # storeBackup -f /etc/storebackup.d/configFile
BEGIN     2012.12.06 00:20:04  9741 backing up directory </media/System/0Backup> to </DataBackup/Sala>
VERSION   2012.12.06 00:20:04  9741 storeBackup.pl, 3.2, build 361
INFO      2012.12.06 00:20:04  9741 setting ARG_MAX to 63488 (Linux)
INFO      2012.12.06 00:20:04  9741 comprRule = $size > 1024 and not $file =~ /.*\Z/i
INFO      2012.12.06 00:20:04  9741 scanning directory </DataBackup> for existing backups
INFO      2012.12.06 00:20:04  9741 scanning directory </DataBackup/Sala-Storico> for existing backups
INFO      2012.12.06 00:20:04  9741 scanning directory </DataBackup/Sala> for existing backups
INFO      2012.12.06 00:20:04  9741 scanning directory </DataBackup/lost+found> for existing backups
STATISTIC 2012.12.06 00:20:04  9741 found 4 backup series, 4 backups, 0 renamed backups
INFO      2012.12.06 00:20:04  9741 consistency check finished successfully
INFO      2012.12.06 00:20:04  9741 found no references to other backups
INFO      2012.12.06 00:20:04  9741 creating lock file </tmp/storeBackup.lock>
INFO      2012.12.06 00:20:04  9741 analysis of old Backups in </DataBackup/Sala>:
INFO      2012.12.06 00:20:04  9741    Wed 2012.12.05_23.43.03 (0d0h): keepDuplicate(7d)
INFO      2012.12.06 00:20:04  9741    Wed 2012.12.05_23.51.46 (0d0h): keepDuplicate(7d)
INFO      2012.12.06 00:20:04  9741    Wed 2012.12.05_23.55.30 (0d0h): keepMinNumber2, keepWeekDays(32d)
INFO      2012.12.06 00:20:04  9741    Thu 2012.12.06_00.20.04 (0d0h): keepLastOfMonth(a60d), keepMinNumber1, keepWeekDays(32d)
INFO      2012.12.06 00:20:04  9741 start reading /DataBackup/Sala/2012.12.05_23.55.30/.md5CheckSums.bz2
INFO      2012.12.06 00:20:04  9741 finished reading /DataBackup/Sala/2012.12.05_23.55.30/.md5CheckSums.bz2 (1 entries)
INFO      2012.12.06 00:20:04  9741 start reading /DataBackup/Sala-Storico/2012.12.04_21.08.10/.md5CheckSums.bz2
INFO      2012.12.06 00:21:17  9741 finished reading /DataBackup/Sala-Storico/2012.12.04_21.08.10/.md5CheckSums.bz2 (1693627 entries)
INFO      2012.12.06 00:21:17  9741 1450921 entries in dbm files
INFO      2012.12.06 00:21:17  9741 0 entries in dbm block files
Unsuccessful lstat on filename containing newline at /usr/bin/storeBackup line 5444.
Unsuccessful stat on filename containing newline at /usr/bin/storeBackup line 5444.
WARNING   2012.12.06 00:21:17  9741 </media/System/0Backup/C:\
ppdf32Log\debuglog.txt> removed during backup
Unsuccessful lstat on filename containing newline at /usr/bin/storeBackup line 5444.
Unsuccessful stat on filename containing newline at /usr/bin/storeBackup line 5444.
WARNING   2012.12.06 00:21:17  9741 </media/System/0Backup/C:\
-anyname\test (copia).txt> removed during backup
STATISTIC 2012.12.06 00:21:17  9741  [sec] |      user|    system
STATISTIC 2012.12.06 00:21:17  9741 -------+----------+----------
STATISTIC 2012.12.06 00:21:17  9741 process|     67.18|      1.01
STATISTIC 2012.12.06 00:21:17  9741 childs |     22.04|      0.50
STATISTIC 2012.12.06 00:21:17  9741 -------+----------+----------
STATISTIC 2012.12.06 00:21:17  9741 sum    |     89.22|      1.51 => 90.73 (1m31s)
STATISTIC 2012.12.06 00:21:17  9741                    directories = 0
STATISTIC 2012.12.06 00:21:17  9741                          files = 1
STATISTIC 2012.12.06 00:21:17  9741                 symbolic links = 0
STATISTIC 2012.12.06 00:21:17  9741                     late links = 0
STATISTIC 2012.12.06 00:21:17  9741                    named pipes = 0
STATISTIC 2012.12.06 00:21:17  9741                        sockets = 0
STATISTIC 2012.12.06 00:21:17  9741                  block devices = 0
STATISTIC 2012.12.06 00:21:17  9741              character devices = 0
STATISTIC 2012.12.06 00:21:17  9741      new internal linked files = 0
STATISTIC 2012.12.06 00:21:17  9741               old linked files = 1
STATISTIC 2012.12.06 00:21:17  9741                unchanged files = 1
STATISTIC 2012.12.06 00:21:17  9741                   copied files = 0
STATISTIC 2012.12.06 00:21:17  9741               compressed files = 0
STATISTIC 2012.12.06 00:21:17  9741                  blocked files = 0
STATISTIC 2012.12.06 00:21:17  9741    excluded files because rule = 0
STATISTIC 2012.12.06 00:21:17  9741    included files because rule = 0
STATISTIC 2012.12.06 00:21:17  9741         max size of copy queue = 0
STATISTIC 2012.12.06 00:21:17  9741  max size of compression queue = 0
STATISTIC 2012.12.06 00:21:17  9741            calculated md5 sums = 2
STATISTIC 2012.12.06 00:21:17  9741                    forks total = 1
STATISTIC 2012.12.06 00:21:17  9741                      forks md5 = 1
STATISTIC 2012.12.06 00:21:17  9741                     forks copy = 0
STATISTIC 2012.12.06 00:21:17  9741                    forks bzip2 = 0
STATISTIC 2012.12.06 00:21:17  9741                  sum of source = 145k (148304)
STATISTIC 2012.12.06 00:21:17  9741              sum of target all = 290k (296608)
STATISTIC 2012.12.06 00:21:17  9741              sum of target all = 200.00%
STATISTIC 2012.12.06 00:21:17  9741              sum of target new = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741              sum of target new = 0.00%
STATISTIC 2012.12.06 00:21:17  9741             sum of md5ed files = 674k (690262)
STATISTIC 2012.12.06 00:21:17  9741             sum of md5ed files = 465.44%
STATISTIC 2012.12.06 00:21:17  9741     sum internal linked (copy) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741    sum internal linked (compr) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741          sum old linked (copy) = 145k (148304)
STATISTIC 2012.12.06 00:21:17  9741         sum old linked (compr) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741           sum unchanged (copy) = 145k (148304)
STATISTIC 2012.12.06 00:21:17  9741          sum unchanged (compr) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741                 sum new (copy) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741                sum new (compr) = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741     sum new (compr), orig size = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741                 sum new / orig = 0.00%
STATISTIC 2012.12.06 00:21:17  9741       size of md5CheckSum file = 216  (216)
STATISTIC 2012.12.06 00:21:17  9741     size of temporary db files = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741            deleted old backups = 0
STATISTIC 2012.12.06 00:21:17  9741            deleted directories = 0
STATISTIC 2012.12.06 00:21:17  9741                  deleted files = 0
STATISTIC 2012.12.06 00:21:17  9741           (only) removed links = 0
STATISTIC 2012.12.06 00:21:17  9741 freed space in old directories = 0.0  (0)
STATISTIC 2012.12.06 00:21:17  9741       add. used space in files = 216  (216)
STATISTIC 2012.12.06 00:21:17  9741                backup duration = 1m13s
STATISTIC 2012.12.06 00:21:17  9741 over all files/sec (real time) = 0.01
STATISTIC 2012.12.06 00:21:17  9741  over all files/sec (CPU time) = 0.01
STATISTIC 2012.12.06 00:21:17  9741                      CPU usage = 124.29%
INFO      2012.12.06 00:21:17  9741 removing lock file </tmp/storeBackup.lock>
WARNING   2012.12.06 00:21:17  9741 -- 2 WARNINGS OCCURED DURING THE BACKUP! --
INFO      2012.12.06 00:21:17  9741 syncing ...
END       2012.12.06 00:21:18  9741 backing up directory </media/System/0Backup> to </DataBackup/Sala/2012.12.06_00.20.04>



Sala AlbumDiFamiglia # ls -1l /media/System/0Backup/
totale 828
-rw-r--r-- 1 andrea users 148304 ago 21 13:17 C:\anyname\test.txt
-rw-r--r-- 1 root   root  148304 ago 21 13:17 C:\n-anyname\test (copia).txt
-rw-r--r-- 1 andrea users 541958 ott  7 18:38 C:\nppdf32Log\debuglog.txt



Sala AlbumDiFamiglia # ls -1l /DataBackup/Sala/MostRecentBackup/
totale 148
-rw-r--r-- 3 andrea users 148304 ago 21 13:17 C:\anyname\test.txt



Sala AlbumDiFamiglia #

Luca <ldms01>
Wed 05 Dec 2012 11:28:30 PM UTC, comment #4: 

I have attached the output of the test that I did.

The files that contain \ n are not copied.

I think it's a problem with perl because I saw that these files can not be renamed using "rename"

(file #27045)

Luca <ldms01>
Wed 05 Dec 2012 11:55:08 AM UTC, comment #3: 

Hi,

can you write down the exact filename and the size of that file?

Do you want to compress or to copy that file?

This makes it easier for me to reproduce and to solve it.

Thanks, Heinz-Josef

Heinz-Josef Claes <hjclaes>
Group administrator
Tue 04 Dec 2012 10:30:13 PM UTC, comment #2: 

perhaps I am wrong, but the problem is not the message but rather the fact that the file is not copied. (At least in my tests). PERL can not open the file? (I confess I do not know).

ah, storeBackup is a great work. Bye. Luca

Luca <ldms01>
Wed 03 Oct 2012 06:58:28 AM UTC, comment #1: 

Yes, I know.

This is a perl feature - I do not know a working solution to suppress this message.

Let's assume the affected file is named 'xxx'. Then - at that point - storeBackup.pl checks if a file called 'xxx.bz2' (depends on your compression suffix) exists in sourceDir (because this means 'xxx' cannot be compressed). So in most cases this really is an unsuccessful stat. It's done with this statement:

-e "$dir/$file$postfix"

I replaced "-e" this with something like:

my $STAT = (stat())[2];
$STAT = S_ISREG($STAT) if $STAT;

this also doesn't help to suppress the message.

Using open or sysopen to check existence of that file also leads to the same message generated by perl.

Maybe anybody knows a solution (except forking or eval, that's too slow) to this problem?

Heinz-Josef Claes <hjclaes>
Group administrator
Sun 30 Sep 2012 08:12:55 AM UTC, original submission:  

storeBackup handles filename containing a "\n" in a wrong way:

The file:
-rw-r--r-- 1 xyz users 15484 Jul 25 16:03 /home/xyz/C:\nppdf32Log\debuglog.txt

...causes the following warning during the backup reported via the standard output:
WARNING   2012.09.30 02:06:14 19220 </home/xyz/C:\\nppdf32Log\debuglog.txt> has \n in the file name
WARNING   2012.09.30 02:06:14 19220 </srv/files/home/ado/C:\
ppdf32Log\debuglog.txt> removed during backup

...causes the following warning during the backup reported via the error output:
nsuccessful lstat on filename containing newline at /usr/local/bin/storeBackup.pl line 5666.
Unsuccessful stat on filename containing newline at /usr/local/bin/storeBackup.pl line 5666.

Version:
storeBackup-3.2.1

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #27045:  Allegato added by ldms01 (8KiB - application/octet-stream - Test results)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ldms01 (Posted a comment)
  • -email is unavailable- added by hjclaes (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-12-15 hjclaes StatusNone Need Info
        SummaryError handling filenames conatining a &quot;\\n&quot; Error handling filenames conatining a "\n"
    2012-12-06 hjclaes SummaryError handling filenames conatining a &quot;\\n&quot; Error handling filenames conatining a "\n"
    2012-12-05 ldms01 Attached File- Added Allegato, #27045
    2012-12-05 hjclaes SummaryError handling filenames conatining a &quot;\\n&quot; Error handling filenames conatining a "\n"
    2012-10-03 hjclaes Severity4 - Important 2 - Minor
        SummaryError handling filenames conatining a &quot;\\n&quot; Error handling filenames conatining a "\n"
    2012-10-03 hjclaes SummaryError handling filenames conatining a &quot;\\n&quot; Error handling filenames conatining a "\n"

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code