maindavfs2 - Support: sr #109127, davfs2 cant seem to show files...

 
 

sr #109127: davfs2 cant seem to show files over certain number of files in directory

Submitter:  None
Submitted:  Tue 30 Aug 2016 01:38:12 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  In Progress
Privacy:  Public Assigned to:  _71007
Originator Email:  -email is unavailable- Open/Closed:  Closed
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 05 Sep 2016 01:42:51 PM UTC, comment #3: 

The type of your server software and the WebDAV module would be useful. Many davfs2 bugs turn out to be server bugs.

Just to be sure I tested with about 100 files in one directory without any problems. There are also people that use davfs2 with more than 1000 files in one directory.

There is something else that causes the problem and we have to do some debugging. This will produce a lot of debug messages in your log files. To keep it as simple as possible please do as follows:

  • Check that your log facility does not restrict the number of log message for a given time. If of your client system uses systemd then options RateLimitInterval and RateLimitBurst in /etc/systemd/journal.conf would do this.


  • Prepare a directory on your server with just enough files to trigger the error. Test wether the error really happens.


  • When the davfs2 file system is not mounted add options "debug most" and "debug httpbody" to your davfs2.conf file.


  • Mount the file system.


  • Issue just one single ls command for the directory with too many files. Don't use bash completion for this and do anything else otherwise the amount of log messages may explode.


  • Unmount the file system.


Depending on your system the log messages from mount.davfs will be in /var/log/syslog or /var/log/daemon.log or /var/log/debug or ....

The log messages don't contain any credentials or passwords but they will contain the server name and of course a lot of file names. You may send this messages as an attachment to this support tracker or - if you think there is something confidential in the logs - to my private email address werner.baumann@onlinehome.de. Using gzip is ok.

It would be useful to also include the messages from the webserver's access log.

Werner

- <_71007>
Mon 05 Sep 2016 01:53:15 AM UTC, comment #2: 

Thanks for your reply.
Sorry, yes a bit more information.
I'm using a couple of linux machines to mount the remote webdav folder to, and having the same results on both (so possibly a remote machine issue)? Anyway, details:

Computer 1:
OS: Debian Wheezy 7.11 (installed on old iBook G4)
Free memory: ~280MB (not much I know)
davfs2 version: 1.4.6 (from apt-get)

Computer 2:
OS: Ubuntu 16.04 (installed on lenovo thinkpad)
Free memory: ~3.5GB
davfs2 version: 1.5.2 (from apt-get)

Webdav server is located on some national instruments CompactRIO hardware. Available memory on these two devices I've had issues with are ~200MB and ~600MB (two happening independently). I can mount and see all files from this hardware using windows fine (mounting network drives),which leads me to think it's a davfs2 issue, and not server-side.
When there's about 75~77 folders/files in a directory, it stops working.
When this happens, yes, mount.davfs is a running process.
when I don't "see anything" a ls command doesn't return an error, just returns without error and an empty list of files (I just get a lost+found directory (which is empty when I enter into it).
Cant see any error messages on both server and client side re. webdav/davfs (i.e. a grep search of 'dav' doesn't return anything other than me calling it to mount on my local machine(s) in any logs).

Thanks for your time.


Anonymous
Fri 02 Sep 2016 07:58:25 AM UTC, comment #1: 

davfs2 has no built-in restriction on the number of files in your file system or in a single directory. It may be restricted by the amount of working memory available. davfs2 needs about 200 - 300 bytes of working meemory for every file or directory.

Some more information would be useful:

  • your operating system
  • version of davfs2, from what source did you get it
  • kind of the WebDAV-server. If it is a public service, which one
  • available memory
  • approximate maximum number of files before the error happens
  • when this happens: is mount.davfs still running
  • how exactly does ls not "see anything". Does ls block, does it return with an error message (which one), does it return without error and an empty list of files
  • are there any error messages form mount.davfs or other error messages concerning the file system in your log files
  • if you have access to the WebDAV-server: any error messages in the server log


Werner

- <_71007>
Tue 30 Aug 2016 01:38:12 AM UTC, original submission:  

I've been mounting a webdav folder hourly on cron and using rsync to pull any new files across. I've noticed that once the directory contents get too many files (or folders in my case) gets too large, my system doesn't seem to see anything.
I can mount on a windows system and move some files off the webdav server, and once I have done that, the directory contents are now 'seeable' again (using just a ls command).
Is there some setting to change to be able to have davfs2 allow me to see more than X files?

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-12-26 _71007 Open/ClosedOpen Closed
    2016-09-02 _71007 StatusNone In Progress
        Assigned toNone _71007

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code