maindavfs2 - Support: sr #108636, "sync" operations

 
 

sr #108636: "sync" operations

Submitter:  Dale Worley <worley>
Submitted:  Tue 19 Aug 2014 06:27:34 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  _71007
Open/Closed:  Closed Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 17 Sep 2014 08:20:33 PM UTC, comment #9: 

Thinking about this a little more:

To verify that everything has been uploaded successfully,

1. Do umount.davfs, wait till it returns.
2. Mount the file system again.
3. Check whether .../lost+found is empty.  If it is empty, davfs2 thought it could upload everything; in any case, anything that can be read from the file system must also be present on the server, and with identical contents.

That sequence has the advantage that it uses only public interfaces to davfs2 (as opposed to looking in the index file, which is not documented).

Dale Worley <worley>
Tue 16 Sep 2014 04:09:48 PM UTC, comment #8: 

OK, that makes sense.  For the time being, I'll use umount.davfs in the few situations where I need the waiting.  In the long run, Fedora will distribute the new version of davfs2.

When I use umount.davfs, I get the expected message:

# umount.davfs ~worley/box.com
umount.davfs: waiting while mount.davfs (pid 13412) synchronizes the cache ... OK
#

Which means that one can force synchronization by doing a proper umount, and then looking to see if the index reveals anything in lost+found.

Dale Worley <worley>
Tue 16 Sep 2014 06:36:42 AM UTC, comment #7: 

Your system does not have a real /etc/mtab file but /etc/mtab is a symbolic link to /proc/mounts. While this has some advantages it also has some disadvantages:

  • /proc/mounts shows the kernel files system fuse, but does not show the type of the files system in userspace that is connected to fuse. Because of this umount can't call the proper umount helper.
  • /proc/mount does not show the option "user=..". Because of this you have to use the less secure mount option "users" instead of "user".


To solve these problems util-linux introduced the /var/run/mount/utab file. davfs2 supports it since version 1.5.0.

You have three options to get around the problem:

  • install davfs2 version 1.5.2 (recommended)

or

  • remove the symbolic link /etc/mtab and replace it by a regular file (this file should start as a copy of /proc/mounts)

or

  • call the umount helper /sbin/umount.davfs directly to unmount davfs2 file systems.


Werner

- <_71007>
Mon 15 Sep 2014 08:27:18 PM UTC, comment #6: 

When I umount the file system, umount returns more-or-less immediately, but the mount.davfs process continues for a while.

If the filesystem is idle, mount.davfs process continues for less than 15 seconds.  But if I do this sequence:

$ cp ~/not-replicated/time-warp/.git/objects/pack/pack-03ca3d543f032b96746c79d5ecee6c48808ad6a7.pack ~/box.com/bigfile ;  # a 100 MB file
 date ;
 umount ~/box.com ;
 while true ;
   do ll /var/run/mount.davfs/common-home-worley-box.com.pid ;
   sleep 15 ;
 done

mount.davfs continues for about 2 min 30 sec, which is reasonable, given the bandwidth to the server.

What mount shows is:

fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
https://dav.box.com/dav on /common/home/worley/box.com type fuse (rw,nosuid,nodev,noexec,relatime,user_id=1000,group_id=1000,allow_other,max_read=31744)

"davfs" is not present.

I see something resembling this reported in http://savannah.nongnu.org/support/?108638

The file /var/run/utab does not exist.

(BTW, I am running davfs2-1.4.7-3.fc19.x86_64 under Fedora 19.)

I am mounting the filesystem as a non-root user, if that makes any difference.  The fstab line is:

https://dav.box.com/dav         /home/worley/box.com davfs rw,users,noauto,uid=worley,gid=worley,dir_mode=770,file_mode=660  0 0

Dale Worley <worley>
Sun 14 Sep 2014 04:53:02 PM UTC, comment #5: 

If you umount a davfs2 file system with umount the umount command will not return until the daemon terminated. If you don't see the message "Waiting while ..." then there is something wrong.
How do you umount a davfs2 file system?
Does mount show either file system tiype 'davfs' or 'helper=davfs'?

If you set cache_size 0 then davfs2 will delete all files from the cache after uploading them to the server. So theoretically an empty cache (except the index file) will indicate that all files are saved to the server.
But note: when davfs2 is unmounted it will give each file only one upload attempt (it can't delay unmounting infinitely). In you case of a highly unreliable server connection it will probably not upload all files to the server and the cache will not be empty. The only thing you can do (except using another server):

  • stop using the file system, but do not unmount
  • check lost+foud; copy files that are in lost and found to the file system again and then delete them from lost+found
  • wait until the cache is empty and only then umount.


davfs2 uses HTTP Etags if available. As long as your server creates etags correctly davfs2 will always serve the latest version of a file. When you open a file it will always use etag to check whether there is a newer version of the file on the server.

Werner

- <_71007>
Thu 11 Sep 2014 07:10:02 PM UTC, comment #4: 

In regard to mounting, there is a timing problem.  With most
filesystems, by the time the umount command returns, all of the cached
data has been written to the device, and the device can be safely
disconnected.

Clearly, this behavior is not always possible with davfs, because
writing data to the server may take quite a while.  However, the data
is still safely stored, as it is in the cache, and if the host stops
and is restarted, davfs can continue writing the data from the cache
to the server.

It appears to me that when a davfs file system is unmounted, the
mount.davfs process continues running, presumably finishing up writing
to the server.  But unlike any other filesystem, you can't always
mount the file system again immediately, you have to wait until the
"old" mount.davfs is finished.  This is undesirable, because the mount
behavior is different from other filesystems, and it results in the
mount call failing in an unexpected way.

Ideally, if the user re-mounted the filesystem, the mount.davfs
process would just continue from where it left off.  I suspect this
may be hard to implement, but in terms of what the user sees, it is
exactly what you want.

If that isn't possible, it would help if there was a defined way to
wait for mount.davfs to finish.  This could be like the check "Is the
cache empty?", and would be done before the umount to ensure that
umount would cause mount.davfs to terminate immediately.
Alternatively, there could be a defined way to know "Attempting to
mount will fail because the old mount.davfs is still working."  Is the
existence of /var/run/mount.davfs/xxxxxx.pid that test?

I expect that this isn't a very common problem, but I ran into it
because I thought that one way to ensure davfs cleared its cache was
to unmount the filesystem and mount it again.

Dale Worley <worley>
Thu 11 Sep 2014 07:09:38 PM UTC, comment #3: 

As you say, given that WebDAV cannot write files in segments,
implementing an fsync operation by writing the currently-written data to
the server would be quite inefficient.  And it also means that there's
no good way to "pace" the writing of data to the server in any unit
less than a whole file.

I think that strictly speaking, davfs satisfies the semantics of
fsync by ensuring that the data that has already been written by the
application is written to the cache directory (along with whatever
metadata is needed to describe it), because the data is "safely on
disk" -- if the host crashes, when davfs is restarted, it can recover
the data from its cache and write it to the server.

The functional problem I have is that I am copying files to a WebDAV
server as a backup method, so I want to know when the data has been
written to the server, in case the host's disk fails.  fsync is not a
way to ensure this.  What would be a good method?  The functionality I
am looking for does not need to be part of the standard file system
interface.  Is there a proper way to test that all of the cached data
has been written to the server?  I can imagine that it might not be
difficult to write an auxiliary program that tests for this or waits
for this.

I suppose there are special-purpose programs that transfer files using
WebDAV, but davfs is such a wonderful tool because it integrates
WebDAV access into any application.  In this instance, I'm using
rsync to copy a file tree.  One benefit of combining rsync with davfs
is that if the transfer process fails, I can simply restart rsync (and
perhaps davfs) and rsync has all the logic to determine which files
have already been transferred.

Ideally, it would be possible to "clear the cache", to ensure that any
future attempts to read files would read their contents anew from the
server.  That allows the application to do a "read back verification"
of the data on the server.  From what I have seen of the log messages
and the ~/.davfs2 directory, it appears that the cache is cleared
automatically; once a file has been closed for a while and written to
the server, it is deleted from the cache.  (Is this the time set by
file_refresh?)

Even if that is not possible, being able to ensure that file metadata
(especially length) is being read from the server would be valuable.
Am I correct that dir_refresh controls how long metadata is cached by
mount.davfs?

Then again, perhaps HTTP cache validators are reliable enough that if
davfs and the server agree on the validator, then davfs knows that
the server has the same version of the file that it does.  This goes
beyond my knowledge of HTTP.

Dale Worley <worley>
Sun 24 Aug 2014 04:55:45 PM UTC, comment #2: 

In WebDAV the HTTP-method PUT is used to transfer files from the client to the server. This method only can transfer complete resources (files). Incremental upload is not defined in the specification (different from ranged requests in GET). There may be servers that allow ranged PUT, but I don't know them and most servers don't.

1. If davfs2 would transfer the file to the Webserver on fsync this would be a horrible waste of bandwith and time (e.g. 1 GB and four calls to fsync would mean: 1. transfer 250 MB, 2. transfer 500 MB, 3. transfer 750 MB and last transfer 1 GB. This sums up to 2.5 GB instead of 1 GB). I won't do this. Because of this davfs2 syncs to the cache and only transfers a file to the server after the file has been closed.

2. As I understand you want davfs2 to transfer files to the server while your application is still creating the file. HTTP connections usually have timeouts. If the next chunk of data is not send within this timeout the server will close the connection and discard all data already transfered. So this could only work if the application can guarantee to write the data without delay. But there is nothing in the file system interface to make this guarantee. Applications may intermittendly write to the file with arbitrary delays that may be hours or days. So a WebDAV file system can't do that.

3. and 4. As a file system davfs2 is restricted by the file system interface. This functions would be possible by either davfs2 specific fcntl functions or some means like dbus. But this is far beyond whalt I am able to do in the near future.

What I don't understand:

  • if you unmount a file system you will have to wait until the unmounting has finished before you can mount again. With davfs2 the umount command will not return before the daemon terminated. With a caching file system and an internet connection this may take some time. But why unmount to immediately mount again?


  • "... so that any later filesystem requests have to fetch the data from the server". Why would you want that. davfs2 uses HTTP cache validators. So when you open a file it will be the most recent version that is on the server. But when the cache holds exactly the same version as the server why want you to fetch it from the server?


Hint:
If you want to transfer files to the server imediately you can use cadaver. This requires that the file already exists.
To transfer while the application is still creating the file you should include WebDAV functionality into the application.

Werner

- <_71007>
Tue 19 Aug 2014 07:28:16 PM UTC, comment #1: 

Also:

4. Is there a way to flush the cache, so that any later filesystem requests have to fetch the data from the server?

Dale Worley <worley>
Tue 19 Aug 2014 06:27:34 PM UTC, original submission:  

I am using davfs2 to transfer some large files to a server (box.com).  In order to ensure that this works correctly, there are a number of "sync" operations that I'd like to be able to perform.  Does davfs2 support these, and if so, how can I ensure that they operate like expected?

1. Being able to fsync() a file.  That is, fsync() on a file should not return until the davfs2 daemon has written the data to the server and received confirmation that the data has been stored on the server.

2. Pacing of transfer.  The application program should not get too far ahead of the Webdav transfer process; writes should stall if there is too much data that hasn't been transfered yet.  This goal is less important and (it seems to me) may conflict with other performance goals of davfs2.  I believe that having fsync() work properly would suffice for the application's needs.

3. Monitoring of cached writes.  Is there a way to inquire as to the amount of cached data that has not yet been written to the server?  Is there a way to "wait until all cached data has been written"?  The last question is important because "umount" will appear to unmount the filesystem immediately, even if the daemon is still alive in the background writing data.  And if the daemon is still alive, the filesystem cannot be mounted again!


Dale Worley <worley>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by worley (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-04-15 _71007 Open/ClosedOpen Closed
    2014-08-24 _71007 StatusNone Done
        Assigned toNone _71007

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code