maindavfs2 - Support: sr #107907, davfs2 and Oracle Beehive...

 
 

sr #107907: davfs2 and Oracle Beehive Workspaces WebDAV (Patch included)

Submitter:  None
Submitted:  Mon 12 Dec 2011 12:08:08 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  4 - Important Status:  Done
Privacy:  Public Assigned to:  _71007
Originator Email:  -email is unavailable- Open/Closed:  Closed
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 18 Apr 2014 08:15:59 PM UTC, comment #8: 

The problem with cookies is fixed in the CVS sources branches MAIN and rel-1-4-6.

Werner

- <_71007>
Wed 28 Dec 2011 10:56:17 AM UTC, comment #7: 

Thanks for sending the cookies.

There is another request for support of more than one cookie (davfs only supports one cookie) and there is also a new Internet Proposed Standard, that tries to summarize how cookies are really used.

I don't know if I will be able to implement better support for cookies. But if I do davfs2 will have to parse the cookie headers on its own (Neon does not support cookies). To do this I will need a closer look at the new Proposed Standard as well as on as many real cookie headers as possible.

Unfortunately the cookies I got from you are already processed by some program. I would need the unprocessed headers to look for all the problems that may arise, like use of quotation, delimiters and the like.

Could you send the unprocessed headers by direct email? You can get them with option 'debug http' or 'debug most'.

Happy new year or einen guten Rutsch
Werner

- <_71007>
Thu 22 Dec 2011 11:23:53 PM UTC, comment #6: 

Hi Werner,

first of all, sorry for my late reply.

I can confirm that your two patches as attached to comment #5 (file #24621, file #24622) replacing my proposed patch also work fine to access my main Oracle Beehive WebDAV server.

And yes, you were indeed right that I introduced a severe bug to turn off rfc_quota on any error (including connection timeout)... ;-)


Regarding the cookie issue:

This issue only applies to the other WebDAV resource I am also accessing, it seems to establish two cookies both named JSESSIONID containing just a simple log string (an encrypted Java EE session ID), but for different contexts: one for a context named "/content", the other for a context named "/teamcollab".

For accessing the files via WebDAV, it will then need the "/content" cookie to be sent ("real" resource context path starts looks like content/dav/Oracle/XXX/...).

I am not completely sure whether it is this fact which makes your original cookie code fail, but the code from

http://savannah.nongnu.org/support/?107764

definitely seemed to work for me while the original did not.

If you need more details, I will also be able to send the full cookie information via direct mail to you (I don't want to disclose particular server name and cookie content...).


Many thanks anyway for the two official patches that now make Beehive WebDAV work properly with davfs2 - I will also try and advertise the fact that davfs2 now works fine for accessing their WebDAV with the Beehive team within Oracle...

Also, when I have more time, I may again look into the log details for the other resource without the cookie patch...

Merry Christmas and Happy Holidays!

Best regards,

Andreas

Anonymous
Sun 18 Dec 2011 04:15:28 PM UTC, comment #5: 

Quota:
So detecting support for WebDAV-quota works as expected, but detecting support for USERINFO fails due to a strange return code send by the Oracle server.
I created a patch (two files appended) to catch all errors on USERINFO except temporary ones. Your patch only works with the Oracle server but fails when the server returns the proper status code of 501.
Is there a reason why you turn off rfc_quota on any error (including connection timeout). The original code only turns it off when the server indicates that it is not support. This seems to work for the Oracle server too.

I also added an conditional to dav_quota to return immediately if quota are not supported.
I would prefer not to change anything in cache.h. The small penalty for calling dav_quota even if it is not supported is not worth introducing another global variable.
Concerning dav_fuse:
dav_fuse should never fail to return stat. But I agree that if this ever happens (due to a change in cache.d) ENOSYS would be the better error code.

Cookies:
I intend to redesign cookie support some day. But the state of standardization of cookies is a well known mess. So it would be usefull for me to see as many real world cookies as possible (cookies used with WevDAV only). Unfortunately your debug-logs don't show any cookies at all. Does the server really use cookies and is support for cookies required?
Anyway, if the server sends cookies could you please send me one to see why my parsing code does not work with it and how to improve parsing.

Werner

(file #24621, file #24622)

- <_71007>
Wed 14 Dec 2011 03:14:07 PM UTC, comment #4: 

Hi again,

so here is my "davfs2-1.4.6-patch-v3.patch" which uses davfs2 default content for the statfs result.

If I did correctly locate your defaults, this causes a display like "12.7 GB of 25.4 GB free" (see attached PNG), and this also causes the WebDAV filesystem to reappear in "df -kh" output (although with the above fictitious numbers...).

I have also fixed a small memory leak in my first patch where I forgot to call ne_propfind_destroy in one else branch...

I hope the patch is now useful for you as a starting point - for me, it works like a charm! ;-)

Best regards,

Andreas


file #24595)

Anonymous
Wed 14 Dec 2011 02:17:09 PM UTC, comment #3: 

Hi Werner,

many thanks for looking into this issue! :-)

I have included one more log (with credentials replaced by XXX), this time gzipped due to its size, which clearly shows that the issue is not the "unconventional" response to PROPFIND, but davfs2 currently not interpreting a 302 response to USERINFO as meaning "quota is not supported".

The log clearly shows that davfs2 only makes one PROPFIND attempt to retrieve the quota properties, which fails and will be properly detected as failed. All subsequent requests are USERINFO attempts which will be answered with 302 by Oracle Beehive WebDAV.

Many thanks for your pointers to the standard handling already included in davfs2 for the "quota not supported" scenario.

I will start looking into this later this afternoon and send an updated patch which uses/calls into your standard approach for this scenario.

Best regards,

Andreas


(file #24593)

Anonymous
Tue 13 Dec 2011 09:21:46 PM UTC, comment #2: 

I didn't have time to look at all details, so this is a partial response.

davfs2 already detects whether a sever supports WebDAV-quota or USERINFO and will stop requests if not supported. It is in webdav.c. But why does it not work with this server?

The response of the server to the PROPFIND for quota information seems syntactically correct but is at the same time rather strange. It starts with an empty prop-element and status 200 OK and only after that shows the prop element for the quota-available-bytes with status Not Found (that is what davfs2 is looking for). Maybe the neon-library treats it as a bug.

Could you additionally add option 'debug xml' to show how it is parsed by neon?

The USERINFO-method seems to be an nonstandard method invented by my provider. davfs2 would expect the server to respond with some 5xx status code for method not supported. Instead the oracle server sends a redirect 302 Not Found. I think I will be able to include this case.

stat-structure
In case davfs2 can't get any information about available space it sets the values in stat neither to -1 nor to 0, but to some fake size that looks reasonable for most cases. (This is done in cache.c, dav_init_cache). This is to avoid aplications from failing because of unreasonalble values in stat. (As I remember a graphical file browser refused to create files unless it got information from stat that there is enough space available.)

Cheers
Werner

- <_71007>
Tue 13 Dec 2011 08:30:46 PM UTC, comment #1: 

Hi again,

I'd like to slightly change my proposed patch after running into some tools which cannot properly handle a directory size of -1 and as a result show a free space of several terabytes and other interesting things.

Also, I now have found the following (hopefully) reliable statement in the Linux manpage of statfs:

"Fields  that  are  undefined for a particular file system are set to 0."

So please change the attached patch before applying and replace all occurrences of "-1" by "0"...

Best regards,

Andreas

Anonymous
Mon 12 Dec 2011 12:08:08 AM UTC, original submission:  

Hi Werner,

I have used the current davfs2 1.4.6 to mount my company's WebDAV resources ("Oracle Beehive Workspace Web DAV") and noticed that davfs2 seemed to be much (!!!) slower than any other WebDAV client I had tried before to do the same.

I then switched on debug options and noticed the issue show in the attached file debug.log:

Function dav_statfs in cache.c does not properly handle the case when a WebDAV server decides not to support the properties "quota-available-bytes" and "quota-used-bytes" (such as Oracle Beehive - don't ask me for what reason...).

Rather, davfs2 enters an endless loop and sends multiple USERINFO requests per second via dav_statfs to the WebDAV resource, which will make things even worse because Beehive WebDAV then reacts by sending a HTML error page telling that the server is being overloaded (denial of service attack) from my client address... :-(

Luckily, the fix is not too hard: method dav_statfs must be made to properly detect that calling dav_quota is not supported, and set all struct dav_stat fs_stat sizes to a fixed value of -1.

By doing so, I followed good(?) practice of Apple's WebDAV file system implementation:

http://opensource.apple.com/source/webdavfs/webdavfs-148/webdav_fs.kextproj/webdav_fs.kmodproj/webdav.h

After having done so, davfs2 got back on par with any other WebDAV implementations in terms of access speed to my Beehive Workspaces! :-)

I have attached my patch - which also includes the patch for issue

http://savannah.nongnu.org/support/?107764

(because I needed that one as well to access another Oracle-internal resource which required a cookie) - and hope that you will it useful and integrate it into the official version as time permits.

Many thanks & best regards,

Andreas

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #24594:  statfs_result.png added by None (57KiB - image/png)
file #24595:  davfs2-1.4.6-patch-v3.patch added by None (8KiB - application/octet-stream)
file #24593:  debug.log.gz added by None (2KiB - application/x-gzip)
file #24581:  debug.log added by None (10KiB - application/octet-stream - Debug Log showing the issue and diff patch fixing it :-) [with username/password replaced by XXX])
file #24573:  davfs2-1.4.6.patch added by None (8KiB - application/octet-stream - Debug Log showing the issue and diff patch fixing it :-))

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-04-22 _71007 Open/ClosedOpen Closed
    2014-04-18 _71007 StatusIn Progress Done
    2011-12-18 _71007 Attached File- Added davfs2-1.4.6-dav_fuse.c-quota.diff, #24621
        Attached File- Added davfs2-1.4.6-webdav.c-quota.diff, #24622
    2011-12-14 None Attached File- Added statfs_result.png, #24594
        Attached File- Added davfs2-1.4.6-patch-v3.patch, #24595
    2011-12-14 None Attached File- Added debug.log.gz, #24593
    2011-12-13 _71007 StatusNone In Progress
        Assigned toNone _71007
    2011-12-13 None Attached File- Added debug.log, #24581
    2011-12-12 _71007 Attached File#24572 Removed
    2011-12-12 None Attached File- Added debug.log, #24572
        Attached File- Added davfs2-1.4.6.patch, #24573

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code