maindavfs2 - Support: sr #108710, cache folder vs. davfs2.conf

 
 

sr #108710: cache folder vs. davfs2.conf

Submitter:  None
Submitted:  Mon 29 Dec 2014 07:23:58 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  In Progress
Privacy:  Public Assigned to:  _71007
Originator Email:  -email is unavailable- Open/Closed:  Closed
Operating System:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 31 Dec 2014 05:12:37 PM UTC, comment #3: 

1. davfs2 moved away from sourceforge more than 5 years ago. Everthing on sourceforge is out of date.

2. Due to the nature of the WebDAV-protocol davfs2 must store a local copy of all open files. And any file that has been changed will stay there until it has been successfully uploaded to the server. This is independent of option cache_size. So /home/backup just doubles what davfs2 already does.

3. And now the most important point: Your cache has changed dramatically when compared to your last post. There are a lot of files in the lost+found directory, which means that there is something seriously wrong with your connection to the server.

The files in lost+found are there because davfs2 tried to upload them to the server and this failed. It failed in a way that davfs2 must assume the problem is permanent and it will never be able to upload that file. Probably the server returned some error code to indacte this.

So the first thing to do is to check what is wrong with the connection to the server. Did you ever test whether your files really are on the server (you may use cadaver for this)?

My suggestion:

  • Save all files from the cache directory that you might need.


  • Remove the cache directory for this mount.


  • Set option "debug most" in your davfs2.conf file (and reset delay_upload to the default of 10 seconds).


  • mount the file system.


  • copy just one smal file into the davfs2 file system and wait 20 seconds.


  • umount the file system.


  • Check whether the file is on the server (you can use cadaver for this. Or you can remove the davfs2 cache and then mount again to look whether the file is still there).


  • If the file was successfully uploaded then try again with a lrge file.


  • If uploading fails, search your log files for entries from mount.davfs and send them.


It might happen that both tests succeed because the error only happens intermittendly. In this case I recommend to set option "debug http" (and remove option "debug most") to reduce the amount of log messages. Then you should run some of your regular backups and always look at the lost+found directory. As soon as any file appears in lost+found the error happend and the log files should show what went wrong.

Werner

- <_71007>
Wed 31 Dec 2014 03:49:46 PM UTC, comment #2: 

The Problem in my opinion:
dafvs2cache is bigger than defined in davfs2.conf.

In past I used the default configuration an the cache growed up to 200 GB.

I made some changes (cache_size      550), but my davfs-cache is more than 1,8 GB.

I used the long time (200), because I read your post here:

"When unmounting: if you want to make sure that the cache is resized before unmounting, you should wait for delay_upload seconds, to give davfs2 a chance."
https://sourceforge.net/p/dav/discussion/82589/thread/a5518230/#15ff/5a5f

Degrease the delay-upload to 20 didnt resolve the Problem.

I use this procedere
1. dump to /home/backup
2. mount davfs
3. rsync
4. umount davfs

to have EVER a backup.
If I would dump directly into davfs and my internet connection are broken, I will have NO backup. In my case I have at least a local backup.

I attached a file with the index, the davfs.conf and the ls-output from /var/cache/davfs2.
You will see, that the cachesize is a lot more then defined in the config file.

(file #32740)

Anonymous
Tue 30 Dec 2014 09:28:14 AM UTC, comment #1: 

I can't see the problem.

The only files in the cache of davfs2 are the latest versions written by rsync. Because you set delay-upload to 200 seconds they will not be uploaded (and therefore stay in the cache) untill you run umount. Now davfs2 will upload the files and terminate. The files will be removed from the cache the next time you mount the davfs2 file system.

What is the purpose of that additional stage: first writing to /home/backup and then using rsync to copy from /home/backup into the davfs2 file system? Why does your buckup script not directly write into the davfs2 file system and avoid the potential problems with file times that may occur when you use rsync on a davfs2 file system?

If it is the size of the files that concerns you: davfs2 has no influence on this. That are the files created by the backup script.

Werner

- <_71007>
Mon 29 Dec 2014 07:23:58 PM UTC, original submission:  

I#m using Debian 7.0x with

apt-cache show davfs2
Package: davfs2
Version: 1.4.6-1.1+wheezy1
...
Architecture: amd64
Depends: libc6 (>= 2.3), libneon27-gnutls, debconf (>= 0.5) | debconf-2.0, adduser

and my davfs-cache will not be deleting. i'm using davfs2 to rsync a lokal backup-folder with online-storage (HiDrive, the free version). The lokal automysqlbackup-script dump some mysqldatabases to /home/backup, then HiDrive will be mounted with davfs2 to /mnt/backup/... and rsync is running.
Then unmounting /mnt/webdav using umount.davfs2. (it is talking to me :-): waiting till the cache is syncron...
...
...
or something else)
only changes above, rest is #

root@ptLWL01:~# cat /etc/davfs2/davfs2.conf
cache_size 50 # MiByte
delay_upload 200

root@ptLWL01:~# ls -a /var/cache/davfs2/letzter3.webdav.hidrive.strato.com+mnt-webdav.letzter3+root
.daily_ptlwl_2014-12-10_00h34m_Wednesday.sql.bz2.1tqtfM-EtpJT1 .ERRORS_localhost-479520886.log.4ka0gC-VAhn28
index
.status_daily_2014-12-10_00h34m_Wednesday.txt.bz2.8zB7mI-l53lSr
.daily_ptlwl_2014-12-10_00h34m_Wednesday.sql.bz2.0VrBcG-gmTECn .fullschema_daily_2014-12-10_00h34m_Wednesday.sql.bz2.r4cBTG-IsdTXo
.localhost-477323664.log.5kvPYb-HX6GER
root@ptLWL01:~#

You can see in attached index, that .daily_ptlwl_2014-12-10_00h34m_Wednesday.sql.bz2 is over 500 MB and .daily_ptlwl_2014-12-10_00h34m_Wednesday.sql.bz2 is over 120 MB.

automysqlbackup is rotating the backups, so every day my davfs-cache is growing + 600 MB.

Michael

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #32740:  Inhalt_Index_davfs2-conf.txt added by None (34KiB - text/plain)
file #32728:  index.txt added by None (9KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-04-15 _71007 Open/ClosedOpen Closed
    2014-12-31 None Attached File- Added Inhalt_Index_davfs2-conf.txt, #32740
    2014-12-30 _71007 StatusNone In Progress
        Assigned toNone _71007
    2014-12-29 None Attached File- Added index.txt, #32728

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code