helpdavfs2 - Support: sr #107064, Filesystem operations locked while...

 
 

sr #107064: Filesystem operations locked while uploading

Submitted by:  None
Submitted on:  Fri 16 Oct 2009 02:41:37 PM UTC  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: Postponed
Privacy: PublicAssigned to: Werner Baumann <wbaumann>
Originator Email: -unavailable-Open/Closed: Open
Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 29 Jul 2012 08:19:49 PM UTC, comment #9:

Hello Martin,

sorry for the delay. Due to being sick I'm not able to concentrate as necessary.

I think I understand the overall design and agree with it. Though there surely is a lot of space for optimization I would agree to delay this until the basic version is running.

I think I found two problems that should be fixed:

  • function ensure_not_in_transfer_directory seems to only scan the direct childs of the directory, but it should scan the whole subtree
  • while the ensure_not_in_transfer functions wait the bg_thread might remove the node and the node pointer becomes invalid. So after waiting the node-pointer has to be revalidated (e.g. by using node_exists).

Some other ideas:

  • instead of using sleep(10) I would prefer to wait on a condition variable.
  • in the old version tidy_cache was called from dav_fuse because it depends on the select call. With the bg_thread this dependency is gone and I would prefer to start the bg_thread either from main or from cache.c.

I hope I will be able to do more serious work soon
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Wed 25 Jul 2012 07:23:22 PM UTC, comment #8:

Hi Robert,

in principle you are right. But actually I'm not sure if it is possible to stop a transfer, I guess it must be but I don't know.

What I'm sure is that just blocking is for now the simplest solution, apart from returning with an error, and I'd rather do one simple step after another.

It is still possible to implement something more clever later :-)

Bye,

Martin

Martin Wache <mw55>
Wed 25 Jul 2012 07:12:32 PM UTC, comment #7:

Hi,

as my C-knowledge is limited, nevertheless I would like to comment on the locking question.

In case there is a request on a file, which is currently uploaded, the transfer could be stopped and started again once the file is closed. This is closer to the behaviour of dropboxd, because given the file is changed, why should you upload the old version?

robert

Robert <crobe>
Wed 25 Jul 2012 07:09:30 PM UTC, comment #6:

Hi Werner,

no need to hurry, there are still some improvements possible anyway. I just wanted to hear your general opinion on this.

About coda, it should not be a big problem do adapt coda, I just didn't look at it. But I don't mind to get rid of it, especially if that means it is possible to simplify code.

About the AVM copyright, I'm not entirely sure. I think there are some small parts left, but they could easily be rewritten if you mind. However, if you don't mind just add them, after all they inspired the whole thing and released their code under gpl.

Thanks,

Martin

Martin Wache <mw55>
Wed 25 Jul 2012 01:28:11 PM UTC, comment #5:

Hello Martin,

thanks a lot for the work.

I will nedd about two days to understand (I never wrote a multithreated C-program) and to comment.

Concerning coda:
For quite some time I intend to remove coda-support. That would allow to remove some restrictions required by coda. fuse is part of the Linux-kernel for many years now and I don't think anybody is still using coda.

I was never sure what to put in separate threads. I tended to think of all HTTP-requests. With uploading by tidy_cache beeing the main blocker I believe it is a good idea to start with this background threat.

copyright:
Is there any of AVM's code left in your patch that would require to add AVM to the copyright-holders (additional to you)?

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue 24 Jul 2012 07:46:56 PM UTC, comment #4:

Hi,

I started porting the multithreaded file transfer from AVM's patched davfs2 to the CVS. First I wanted to port everything, but now I'm not so sure if AVM's solution is the best.

Now I have created a minimal version, which has just one background thread in which dav_tidy_cache() is called and the dirty files are uploaded.
As in AVM's version there is one lock to guard everything, it is only released during sleeping, and in case of the background thread during transfer of the files.
One problem is what to do if there is any change requested on a file that is currently in transfer, AVM usually returns errors, but I decided to block on those operations. This is closer to the original behaviour of davfs2.

I guess the patch still needs some testing, I think I broke coda (I didn't even try to compile it), but I would like to hear your opinion on it :-)

Bye,
Martin

(file #26252)

Martin Wache <mw55>
Sat 17 Oct 2009 03:34:16 PM UTC, comment #3:

Hi,
thanks for reply, at least its not pebkac :)

Anonymous
Sat 17 Oct 2009 12:05:23 PM UTC, comment #2:

No, it is not necessary, it is a shortcoming of davfs2.
davfs2 is still single threaded, so every action will block anything else.
Though there is some work based on version 1.3.x by AVM to make davfs2 multithreaded, I have not time enough to port this to the current version. I also have the feeling that making davfs2 multi threaded requires a serious redisign of many parts of davfs2.
I can't make any promises when this problem will be solved.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 16 Oct 2009 02:45:09 PM UTC, comment #1:

Sorry, I forgot the version, I'm using the archlinux package davfs2 1.4.1-1, kernel 2.6.30-ARCH and the fuse kernel fs.

Anonymous
Fri 16 Oct 2009 02:41:37 PM UTC, original submission:

Hi,

first davfs2 really rocks.

My problem is, that while davfs2 uploads data to the WebDAV storage, the davfs2 filesystem is completely locked, operations as 'ls' and 'df' don't work and even worse, these operations wait until the current file upload is complete.
I saw this when KDE4 checks ( I don't know exactly what it does, it shows up in strace ) all via fstab mounted filesystems and the whole desktop stalls when waiting for davfs2 while it uploads a file. This makes uploading bigger files with small bandwidth impossible.
I hope this describes the problem.

Is this really neccessary? I'm not familiar with Linux filesystem interna, is there maybe some Option to prevent this?

Thanks,

robert

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #26252:  bg_uploads.diff added by mw55 (33kB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by crobe (Posted a comment)
  • -unavailable- added by mw55 (Updated the item)
  • -unavailable- added by wbaumann (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 24 Jul 2012 07:46:56 PM UTCmw55Attached File-=>Added bg_uploads.diff, #26252
    Sun 18 Oct 2009 07:17:45 PM UTCwbaumannStatusNone=>Postponed
    Sat 17 Oct 2009 12:05:23 PM UTCwbaumannAssigned toNone=>wbaumann

    Back to the top


    Powered by Savane 3.1-cleanup