maindavfs2 - Support: sr #110209, Really, really want 100% complete...

 
 

sr #110209: Really, really want 100% complete local caching

Submitter:  Brad Barnett <bbarnett>
Submitted:  Tue 17 Mar 2020 11:15:52 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  In Progress
Privacy:  Public Assigned to:  _71007
Open/Closed:  Open Operating System:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 24 Mar 2020 04:50:52 PM UTC, comment #3: 

To get more information about what is going on I ask you to do this:

  • set option "debug most"
  • remove your cache directory for this mount or use a different mountpoint to make sure davfs2 starts with an empty cache
  • make sure that no program tries to access the davfs2 file system, especially don't run any graphical file manager
  • mount the file system
  • issue one single command to copy one file from the davfs2 file system to some local file system. Do not use auto completion or ls. Type the full filename to copy (this is to keep the size of the logfile small).
  • unmount the davfs2 file system.
  • create a file that contains all the log messages from davfs2 and the output of ls -l on your cache directory.


After some time again mount the file system (but keep the cache directory unchanged), copy the same file, unmount and create a file with log messages and the output of ls -l.

Attach both files to this item.

The HTTP-messages in the log files together with the directory listing of the cache directory should show why davfs2 downloads the file a second time.

Werner

- <_71007>
Fri 20 Mar 2020 12:43:51 PM UTC, comment #2: 

I set my cache size, umounted/remounted:

$ grep -i cache_size .davfs2/davfs2.conf
cache_size      5000000         # MiByte

But, as soon as I umounted... traveled.. and remounted, the cache seemed to be flushed.  More investigation shows this:

Find a file in cache:

username@hmm:~$ ls .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-FocswZ -l
-rw------- 1 username username 3923394 Jan 14 18:06 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-FocswZ

Access/view file with image program.  Delay/bandwidth shows that the file is downloaded...

I now look at atime, and also notice a new copy of the file:

username@hmm:~$ ls -l --time=atime .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-*
-rw------- 1 username username 3923394 Mar 18 08:13 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-FocswZ
-rw------- 1 username username 3923394 Mar 20 05:33 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-Vp4bPg

So, it didn't access the cached file at all.

username@hmm:~$ ls -l --time=ctime .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-*
-rw------- 1 username username 3923394 Mar 18 08:13 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-FocswZ
-rw------- 1 username username 3923394 Mar 20 05:33 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-Vp4bPg

mtime:

username@hmm:~$ ls -l  .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-*
-rw------- 1 username username 3923394 Jan 14 18:06 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-FocswZ
-rw------- 1 username username 3923394 Jan 14 18:06 .davfs2/cache/remoteurl.host-remote.php-dav-files-username+home-username-DAVFS2+username/IMG_20200114_1539037.jpg-Vp4bPg

What I don't get, is other instances (like the nextcloud Android client), can tell when files have been updated.

Is there any way I can help debug what's up here?  Any suggestions on checking to see if something is non-compliant in nextcloud's webdav implementation?

Any davfs2 output I can provide to help here?

Thanks


Brad Barnett <bbarnett>
Wed 18 Mar 2020 11:28:06 AM UTC, comment #1: 

Just set option cache_size high enough. Your WebDAV server must of course comply to the specs. It must not send wrong information about the modification date and the etag of the file.

"I know, from many of the issues/reports I've read, that the thought in mind is 'save space!'"
This was never the intention of davfs2. It is an error of some users of davfs2. Please see the README file.

Werner

- <_71007>
Tue 17 Mar 2020 11:15:52 PM UTC, original submission:  

I know that davfs2 was developed with one usage case in mind.

However, I ask that other usage cases are taken into account.  Mine?

I want it all.

I have a nextcloud server, with webdav.  I have an android phone, using the nextcloud client, and another with a webdav client.

And, I have my desktop and laptop.

And disk space is completely irrelevant in my case.  I mean, I take photos with my Android device, and I just want them all on my desktop.  Always.  Without re-downloading ever.

I want a perpetual, 'forever' cloned dir.  And I want to save a file, anywhere, and have the file constantly on my device.

I know, from many of the issues/reports I've read, that the thought in mind is 'save space!', and that there are concerns about using all the space on a device.

Yet, all of my devices have 100s of gigs free.  If they ever get near full, I could free space.  And my webdav is a mere 20G.

And yup.. I want that 20G taking up space on every davfs2.

Can you add options to 'never expire cache, ever'?  And, '100TB is just fine as a cache size' or some such?

Because I really, really am not concerned about saving space.

I know this may not be the main usage case, but these days, with android devices + nextcloud and such, I bet lots of people would like this...


Brad Barnett <bbarnett>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by bbarnett (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-03-24 _71007 StatusDone In Progress
    2020-03-18 _71007 StatusNone Done
        Assigned toNone _71007

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.13-6755.
    Corresponding source code