helpdavfs2 - Support: sr #108158, davfs2 memory consumption...

 
 

sr #108158: davfs2 memory consumption increases over time?

Submitted by:  None
Submitted on:  Fri 12 Oct 2012 06:29:38 PM UTC  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: Done
Privacy: PublicAssigned to: Werner Baumann <wbaumann>
Originator Email: -unavailable-Open/Closed: Open
Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 20 Apr 2014 05:36:23 PM UTC, comment #14:

The patch is now added to the CVS sources branch MAIN and branch rel-1-4-6.

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 01 Mar 2013 08:23:52 AM UTC, comment #13:

Like I suspected this periodically cleaning of metadata keeps the performance good at least with our use. Naturally cleaning too often adds some extra work but it's negligible and nothing when compared with the worse case scenarios which are avoided by this. And one can tweak the frequency with file_refresh parameter.

So I'd say go ahead and merge this into the next release!

- Panu

Panu Outinen <poutinen>
Fri 22 Feb 2013 03:07:07 PM UTC, comment #12:

Good news! So far no errors. Ran our collector app couple of times, 20000 files / 1.7GB worth of data each round.

I will test further later on and especially does it help performance-wise us...

- Panu

Panu Outinen <poutinen>
Thu 21 Feb 2013 06:31:13 PM UTC, comment #11:

Sorry,

here is the patch.

Werner

(file #27497)

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Wed 20 Feb 2013 10:05:54 PM UTC, comment #10:

Did you attach the new patch or ?

Panu Outinen <poutinen>
Wed 20 Feb 2013 08:28:30 PM UTC, comment #9:

Hello Panu,

here is another try.

The problem was that a directory may be open for quite some time and the kernel will use inode information after the directory has been closed. It should only do this for 1 second (that's the live time of inode information set by davfs2).

The old patch took only the time when the directory has been refreshed to calculate whether a file node can be deleted. The new patch also takes the time when the directory has been closed (atime) into account.

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Mon 18 Feb 2013 01:36:14 PM UTC, comment #8:

Hmm... it runs longer but still into problems. I'll send the debug log file to you separately.

- Panu

Panu Outinen <poutinen>
Sun 17 Feb 2013 03:11:49 PM UTC, comment #7:

Here is another patch.
I removed option 'minimize_dirs' as it seems to be useless.
It should no longer remove file nodes that belong to an open directory.
Please have another try.

Werner

(file #27478)

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Wed 13 Feb 2013 08:42:02 PM UTC, comment #6:

Hello Panu,

thanks for the log file.

I think I could identify the bug. I'm removing nodes while the directory they belong to is still open for reading.

I will probably come up with a new patch next weekend.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue 12 Feb 2013 07:17:50 PM UTC, comment #5:

Please send it by direct email.
-unavailable-

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue 12 Feb 2013 03:58:57 PM UTC, comment #4:

Hi Werner!

I patched the source code of version 1.4.7 (even though the name in the diff file implied version 1.4.6). Compiled ok and I tested against a similar setup we are running daily. Like you said 'minimize_dirs 1' can only be used if one never set current directory inside the davfs2 file system. Just using find command of Linux is enough to trigger this problem (find seem to move it's cwd while running, easily seen with lsof).

But I also ran into problems with running with just 'minimize_mem 1'. Most of the time it works as expected (as seen from the debug log). But sometimes it triggers a situation where a file/files cannot be copied. And these files vary in different runs.

I'm using davfs2.conf file with just these set:
ask_auth 0
use_locks 0
gui_optimize 1
minimize_mem 1
debug most

I have a debug log file with one file having this behavior (stopped our script after noticing this).
This debug file is huge 43MB, should I gzip and attach here or email to you directly ?

- Panu

Panu Outinen <poutinen>
Sun 10 Feb 2013 03:30:12 PM UTC, comment #3:

Hello Panu,

when testing my old patch I noticed a major problem: if the working directory is set to be in the davfs2 file system the kernel will cache the inode number of PWD. davfs2 does not know of this and will remove the node which will cause file system errors.

Because of this I changed the patch and you now have two new configuration options:

  • 'minimize_mem 1' will tell davfs2 to remove unused file nodes, but it will not remove directory nodes. Note that local file times may change unexpectetly because the local times stored in the node may differ from the Last-Modified time stored on the server. You should not use this with applications that depend an file times (like backup programs).
  • additionally setting 'minimize_dirs 1' will tell davfs2 to also remove unused directory nodes. You may use this when your file systems has a lot of directories and you are sure that you never cd into the davfs2 file system.

Attached is a patch.

Please report about your experience so I can decide whether to include it in the next release or not.

Cheers
Werner

(file #27439)

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Thu 07 Feb 2013 08:42:42 AM UTC, comment #2:

Hi Werner!

Back in 2008 I notified about the davfs2 cache growing problem and you got it fixed back then. We've been using it ever since (currently version 1.4.7). And it is under heavy use (nightly packaging scripts with 10000's files going through).

But we are just using it as direct read-only file system access of Windows clients to svn repository through combination of davsf2, tmpfs, autofs & samba. The only problem we've encountered is that over time it slows down if automounter is not given the opportunity to cleanup! This doesn't happen with the automatic scripts itself but people leaving just some executables running. The memory consumption is not really issue for us but my educated guess is that cleaning metadata regularly would help this speed issue too.

So I would be willing to give this patch a test run (next week)!

- Panu

Panu Outinen <poutinen>
Sun 14 Oct 2012 08:53:19 AM UTC, comment #1:

This is a design issue of davfs2 which is not well suited for machines with a small amount of memory.

Details:
davfs2 stores meta data of all ever requested resources in a tree of nodes in the working memory. It only removes nodes that get invalid. When davfs2 is running for some time it might finally store meta information about all resources in memory. This might get rather big if the server holds many resources.

Possible solution:
About two years ago, on request of a user, I added code that will regularly scan the node structure and remove nodes that have not been used for some time. Unfortunately I got no response from the requester of this feature and was not able to seriously test it myself. So I removed that code. (See https://savannah.nongnu.org/support/?107437)

If you are interested to test this feature (and report your experience) I could create a patch for you (or a patched source package). If your tests are successful I would include it as an option in the next release.

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 12 Oct 2012 06:29:38 PM UTC, original submission:

10 minutes after a reboot I see mount.davfs2 consuming 1.8% of memory.

After ~18hrs uptime I see mount.davfs2 consuming 13.5% of memory.
After ~21hrs uptime I see mount.davfs2 consuming 15% of memory.

Running:
Linux RASPBERRY 3.2.27+ #160 PREEMPT Mon Sep 17 23:18:42 BST 2012 armv6l GNU/Linux

davfs2 1.4.7 <http://savannah.nongnu.org/projects/davfs2>

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #26754:  davfs2.conf added by None (2kB - application/octet-stream - davfs2.conf file)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by poutinen (Posted a comment)
  • -unavailable- added by wbaumann (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 9 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 20 Apr 2014 05:36:23 PM UTCwbaumannStatusIn Progress=>Done
    Thu 21 Feb 2013 06:31:13 PM UTCwbaumannAttached File-=>Added davfs2-1.4.7-min-mem-1.diff, #27497
    Wed 20 Feb 2013 08:23:55 PM UTCwbaumannAttached File#27478=>Removed
    Sun 17 Feb 2013 03:11:49 PM UTCwbaumannAttached File-=>Added davfs2-1.4.7-min-mem.diff, #27478
    Sun 17 Feb 2013 03:05:12 PM UTCwbaumannAttached File#27439=>Removed
    Sun 10 Feb 2013 03:30:12 PM UTCwbaumannAttached File-=>Added davfs2-1.4.6-min-mem.diff, #27439
    Sun 14 Oct 2012 08:53:19 AM UTCwbaumannStatusNone=>In Progress
      Assigned toNone=>wbaumann
    Fri 12 Oct 2012 06:29:38 PM UTCNoneAttached File-=>Added davfs2.conf, #26754

    Back to the top


    Powered by Savane 3.1-cleanup