bugGNU Wget - Bugs: bug #36180, Wget mirroring (and recursive...

 
 

bug #36180: Wget mirroring (and recursive download) functions are not working on some FTP servers

Submitter:  Bob Bib <bobbib>
Submitted:  Sat 14 Apr 2012 11:41:51 PM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  1.13.4 Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 11 Apr 2014 10:37:49 PM UTC, comment #5: 

The quick fix mentioned in #4 has been committed for 1.15:

http://git.savannah.gnu.org/cgit/wget.git/commit/?id=bba67890f381a0fd3222fc5763b742d1520746c3

So this bug can be closed.

Sooner or later someone will open a new one, because this makes the parsing even more dubious not only with respect to internationalization, but also if some file is owned for example by a user named "jan" .

~mk

Anonymous
Thu 03 May 2012 09:43:10 PM UTC, comment #4: 

Missing this: "running on MS Windows"
might have led me on the wrong track.  Permissions don't come into play, because first of all wget fails to parse the listing.  If these


drw-rw-rw- 1 ftp ftp                0 jan 12 13:14 directory10
drw-rw-rw- 1 ftp ftp                0 may 16 17:18 directory11



DIRECTORY; perms 666;
Skipping.


are verbatim snippets from your listing and log files, respectively, then wget cannot find the abbreviated month name because it comes in all lowercase, while wget is looking for "Jan", "Feb", "Mar", ...


Use the month-name token as the "anchor" (the place where the
position wrt the file name is "known").  ...

This tactic is quite dubious when it comes to
internationalization issues (non-English month names), but it
works for now.  */


As a quick fix you might want to change

if (!strcmp (tok, months[i]))


to

if (!strcasecmp (tok, months[i]))


at or near line 206 in ftp-ls.c and rebuild wget.

Of course this makes the parsing even more dubious not only with respect to internationalization, but also if some file is owned for example by a user named "jan".

~mk

Anonymous
Thu 03 May 2012 03:52:47 PM UTC, comment #3: 


> yes, it's an ugly misconfigured server


What OS is it running on?  Or do you log in as root?

> should I download a single file from it with Wget


Not my intention. Nevertheless a good idea.

> or do something else


Yes, play around with standard ftp from the command line, something like

ftp 192.168.1.1
user user
pass password
dir
dir directory10
get directory10/file1 file1
cd directory10
dir
get file2

and post the transcript of this session.

~mk

Anonymous
Thu 03 May 2012 01:29:50 PM UTC, comment #2: 

mk,

  1. yes, it's an ugly misconfigured server, but I don't administrate it;
  2. nevertheless, some programs like LFTP and FileZilla seem to get successfully through it;
  3. what log do you need to see? should I download a single file from it with Wget or do something else (i.e. do some recursive download with some other program)?
Bob Bib <bobbib>
Thu 03 May 2012 10:27:46 AM UTC, comment #1: 

Don't blame wget!


drw-rw-rw- 1 ftp ftp                0 jan 12 13:14 directory10
drw-rw-rw- 1 ftp ftp                0 may 16 17:18 directory11


The directories don't have the x-flags set, so nobody has search permission, and nothing can be downloaded.  Bob, you better fix your ftp server and/or your directory permissions.

I propose to close this bug request unless the original poster provides a transcript of a successful manual ftp session.

~mk

Anonymous
Sat 14 Apr 2012 11:41:51 PM UTC, original submission:  

Wget FTP mirroring (and recursive download) functions are not working on some private FTP server (with user & password authetication, 2 connections per user limit, powered by Xlight 3.7 running on MS Windows).

Sample output of "wget -d -m ftp://user:password@192.168.1.1" is given in 'wget_mirror_log11.log' attachment (username, password and IP address are replaced for privacy reasons).
"wget -d -r ftp://user:password@192.168.1.1" results in the same problem, except an additional string appears: "Removed `192.168.1.1/.listing'."

The contents of '.listing' file look like that (a bunch of folders):

drw-rw-rw- 1 ftp ftp                0 jan 12 13:14 directory10
drw-rw-rw- 1 ftp ftp                0 may 16 17:18 directory11


Bob Bib <bobbib>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #25644:  wget_mirror_log11.log added by bobbib (2KiB - text/x-log)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bobbib (Submitted the item)
  •  

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-04-14 bobbib Attached File- Added wget_mirror_log11.log, #25644

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code