helpdavfs2 - Support: sr #109127, davfs2 cant seem to show files...

 
 

sr #109127: davfs2 cant seem to show files over certain number of files in directory

Submitted by:  None
Submitted on:  Tue Aug 30 01:38:12 2016  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: In Progress
Privacy: PublicAssigned to: Werner Baumann <wbaumann>
Originator Email: -unavailable-Open/Closed: Open
Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon Sep 5 13:42:51 2016, comment #3:

The type of your server software and the WebDAV module would be useful. Many davfs2 bugs turn out to be server bugs.

Just to be sure I tested with about 100 files in one directory without any problems. There are also people that use davfs2 with more than 1000 files in one directory.

There is something else that causes the problem and we have to do some debugging. This will produce a lot of debug messages in your log files. To keep it as simple as possible please do as follows:

  • Check that your log facility does not restrict the number of log message for a given time. If of your client system uses systemd then options RateLimitInterval and RateLimitBurst in /etc/systemd/journal.conf would do this.
  • Prepare a directory on your server with just enough files to trigger the error. Test wether the error really happens.
  • When the davfs2 file system is not mounted add options "debug most" and "debug httpbody" to your davfs2.conf file.
  • Mount the file system.
  • Issue just one single ls command for the directory with too many files. Don't use bash completion for this and do anything else otherwise the amount of log messages may explode.
  • Unmount the file system.

Depending on your system the log messages from mount.davfs will be in /var/log/syslog or /var/log/daemon.log or /var/log/debug or ....

The log messages don't contain any credentials or passwords but they will contain the server name and of course a lot of file names. You may send this messages as an attachment to this support tracker or - if you think there is something confidential in the logs - to my private email address werner.baumann@onlinehome.de. Using gzip is ok.

It would be useful to also include the messages from the webserver's access log.

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Mon Sep 5 01:53:15 2016, comment #2:

Thanks for your reply.
Sorry, yes a bit more information.
I'm using a couple of linux machines to mount the remote webdav folder to, and having the same results on both (so possibly a remote machine issue)? Anyway, details:

Computer 1:
OS: Debian Wheezy 7.11 (installed on old iBook G4)
Free memory: ~280MB (not much I know)
davfs2 version: 1.4.6 (from apt-get)

Computer 2:
OS: Ubuntu 16.04 (installed on lenovo thinkpad)
Free memory: ~3.5GB
davfs2 version: 1.5.2 (from apt-get)

Webdav server is located on some national instruments CompactRIO hardware. Available memory on these two devices I've had issues with are ~200MB and ~600MB (two happening independently). I can mount and see all files from this hardware using windows fine (mounting network drives),which leads me to think it's a davfs2 issue, and not server-side.
When there's about 75~77 folders/files in a directory, it stops working.
When this happens, yes, mount.davfs is a running process.
when I don't "see anything" a ls command doesn't return an error, just returns without error and an empty list of files (I just get a lost+found directory (which is empty when I enter into it).
Cant see any error messages on both server and client side re. webdav/davfs (i.e. a grep search of 'dav' doesn't return anything other than me calling it to mount on my local machine(s) in any logs).

Thanks for your time.

Anonymous
Fri Sep 2 07:58:25 2016, comment #1:

davfs2 has no built-in restriction on the number of files in your file system or in a single directory. It may be restricted by the amount of working memory available. davfs2 needs about 200 - 300 bytes of working meemory for every file or directory.

Some more information would be useful:

  • your operating system
  • version of davfs2, from what source did you get it
  • kind of the WebDAV-server. If it is a public service, which one
  • available memory
  • approximate maximum number of files before the error happens
  • when this happens: is mount.davfs still running
  • how exactly does ls not "see anything". Does ls block, does it return with an error message (which one), does it return without error and an empty list of files
  • are there any error messages form mount.davfs or other error messages concerning the file system in your log files
  • if you have access to the WebDAV-server: any error messages in the server log

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue Aug 30 01:38:12 2016, original submission:

I've been mounting a webdav folder hourly on cron and using rsync to pull any new files across. I've noticed that once the directory contents get too many files (or folders in my case) gets too large, my system doesn't seem to see anything.
I can mount on a windows system and move some files off the webdav server, and once I have done that, the directory contents are now 'seeable' again (using just a ls command).
Is there some setting to change to be able to have davfs2 allow me to see more than X files?

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by wbaumann (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri Sep 2 07:58:25 2016wbaumannStatusNone=>In Progress
      Assigned toNone=>wbaumann

    Back to the top


    Powered by Savane 3.1-cleanup1