helpdavfs2 - Support: sr #107114, Can not work with URLs with spaces...

 
 

sr #107114: Can not work with URLs with spaces in it...

Submitted by:  None
Submitted on:  Mon 09 Nov 2009 05:56:26 PM UTC  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: Ready For Test
Privacy: PublicAssigned to: Werner Baumann <wbaumann>
Originator Email: -unavailable-Open/Closed: Open
Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 13 Apr 2011 03:01:54 PM UTC, comment #12:

I can confirm that this patch works for me, too. Thank you! Do we believe that the neon developer(s) will include this?

Werner, I believe you can close #107651 as a duplicate of this issue.

Derek Atkins <warlord>
Sun 10 Apr 2011 08:38:12 AM UTC, comment #11:

Thanks for the patch. It will work fine with davfs2 without breaking something else.

It might be a good idea to offer this patch to the Neon developer. He will better know whether this will break something else and if not he might include it in Neon.
The Neon mailing list:
http://lists.manyfish.co.uk/mailman/listinfo/neon/

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Mon 04 Apr 2011 03:41:41 PM UTC, comment #10:

Here is a patch which worksforme(tm). I don't know what else it breaks however.

diff --git a/src/ne_uri.c b/src/ne_uri.c
index 8d86c31..a930b9c 100644
--- a/src/ne_uri.c
+++ b/src/ne_uri.c
@@ -96,7 +96,7 @@ static const unsigned int uri_chars[256] = {
/* 0xXX x0 x2 x4 x6 x8 xA xC xE */
/* 0x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* 1x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
-/* 2x */ OT, SD, OT, GD, SD, PC, SD, SD, SD, SD, SD, PS, SD, DS, DT, FS,
+/* 2x */ AL, SD, OT, GD, SD, PC, SD, SD, SD, SD, SD, PS, SD, DS, DT, FS,
/* 3x */ DG, DG, DG, DG, DG, DG, DG, DG, DG, DG, CL, SD, OT, SD, OT, QU,
/* 4x */ AT, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL,
/* 5x */ AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, GD, OT, GD, OT, US,
@@ -475,7 +475,7 @@ char ne_path_unescape(const char uri)

/* CH must be an unsigned char; evaluates to 1 if CH should be
* percent-encoded. */
-#define path_escape_ch(ch) (uri_lookup(ch) & URI_ESCAPE)
+#define path_escape_ch(ch) ((ch == ' ') | (uri_lookup(ch) & URI_ESCAPE))

char ne_path_escape(const char path)
{

Arpad Magosanyi <magwas>
Sun 12 Sep 2010 04:24:15 PM UTC, comment #9:

Thanks for going into the trouble of raisning this issue with Microsoft. I always hoped somebody will do it some day. Though Microsoft's reaction shows that chances for a bugfix are not too good.

As you already presented the traces, showing space characters within an URI, as well as RFC 4918 you gave them everthing they would need to see the error. The only thing I can do in addition is to point at the relevant parts of the standard documents.

RFC 3986 defines the syntax of URIs. A summary is in Appendix A. Collected ABNF for URI. An excerpt of the part concerning this case looks like this:

path-absolute = "/" [ segment-nz *( "/" segment ) ]
segment = *pchar
pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
pct-encoded = "%" HEXDIG HEXDIG
unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="

The space character is neither in the "unreserved" set of characters nor in the set of "sub-delims". It is not allowed in the path segment of an URI. But the percent-encoding of the space character, %20, is allowed.

RFC 4918, 8.3.1 Example - Correct URL Handling, makes it - in my opinion - absolutely clear.

Is there any chance to make Microsofts verbatim response, or parts of it, public on this list or somewhere else?

Workaround in davfs2:
While I don't like workarounds for bugs that could easily fixed I am not completely dogmatic about this. There are already some workaraounds in davfs2.
When I looked at this case some time ago I came to the conclusion that I would have to replace part of the Neon library which seems too much trouble. I will have another look at this within the next months. Maybe an easier fix would be to use a patched version of Neon. Would this be an alternative for you? I think I would be able to create the necessary patch.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 10 Sep 2010 10:40:13 AM UTC, comment #8:

I have also encountered this problem with SharePoint not correctly percent-encoding URI responses, and have gathered various Ethereal traces from SharePoint and also IIS WebDAV responses. I have sent the information to Microsoft in the form of a support ticket with them, but they claim that this is not their problem to solve, which is infuriating...

I raised the ticket through our SharePoint reseller, and they have asked for any further information and evidence from the creators of the DavFS2 project, so I am hoping I can push this back to Microsoft some more in the hope of getting some sort of fix from them. I have also emailed the editor of the WebDAV RFC 4918 and I am waiting for a response to see if they will comment.

I understand why you would not want to make changes to the DavFS2 project to accommodate a Microsoft bug; do you have any further information that I could pass to Microsoft to support my ticket?

Thanks,
Clive

Clive Lawrence <clivel>
Thu 06 May 2010 07:24:35 PM UTC, comment #7:

They use another WebDAV-library. I already explained why davfs2 can not do this kind of work around and I have no intentions to switch to another library, rewriting half of davfs2, just to work around an intentional Microsoft incompatibility.

When sending another proposal how davfs2 should fix this Microsoft bug, please include information about

  • your bug reports sent to Microsoft
  • Microsoft's response

Yes, it makes me angry when you silently accept Microsoft's intentional violations of standards and then expect free software to solve the problem.

davfs2 does not work with SharePoint because Microsoft does not want you to use free software. How you to react on this is your responsibility.

And if for some reason you really have to use that broken Microsoft server, why don't you avoid the error condition by not using file names with spaces?

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Wed 05 May 2010 10:57:09 PM UTC, comment #6:

Don't know how Cyberduck for mac does this, but it does..
http://trac.cyberduck.ch/ticket/2223

Anonymous
Thu 11 Mar 2010 08:42:30 PM UTC, comment #5:

Sharepoint is non-free software. The only one who can fix this bug in Sharepoint is the owner of the software, i.e. Microsoft.
If you need to use this broken software avoid the failure condition; don't use file names with spaces, as Sharepoint can't handle them.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Thu 11 Mar 2010 10:07:29 AM UTC, comment #4:

It appears there is no sulution or workaround yet? Facing the same issue, spaced folders on Sharepoint not visible through davfs (Ubuntu).

Frans <gjs>
Wed 11 Nov 2009 05:44:29 PM UTC, comment #3:

Searching the web I found this:

Some users of Sharepoint had problems because Sharepoint exposed the percent-encoded file names to applications that expect the unescaped form of the file name.

Microsoft supplied a Hotfix (http://support.microsoft.com/?scid=kb%3Ben-us%3B939592&x=9&y=12) for this.

Maybe it was too hot and removed percent-encoding from HTTP-traffic as well, where it is required.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue 10 Nov 2009 08:12:24 PM UTC, comment #2:

Thanks for the log file.

It is the same Sharepoint bug. Sharepoint sends href-elements like
"<D:href>http://sharepoint.local.de/Freigegebene Dokumente/LINUX/Knoten/Linux_Knotenliste_wegen_Performancereport"
with spaces not percent encoded.

This is either a configuration issue or it is an intended incompatibility, as IIS does percent-encode spaces.

There is another issue with the PROPFIND response, but it may as well be an error in the debug output of davfs2:
"<D:getlastmodifiedexecutable/>"
davfs2 did not request a property of this name, it requested
"<getlastmodified xmlns="DAV:"/>" and "<executable xmlns="http://apache.org/dav/props/"/>".

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Mon 09 Nov 2009 06:58:07 PM UTC, comment #1:

Hello Robert,

I got a similar report about Sharepoint (unfortuneately it was not posted to this forum but to my email adress, so you can't see it).

The problem seems to be a protocol violation by Sharepoint. In response to a PROPATCH request WebDAV servers send an XML-body containing href-elements. These href-elements enclose the URI of the resource. According to the spec space characters in an URI must be percent encoded like '/ein%20test/'. Sharepoint instead sends a space character without encoding. As this is a protocol violation the Neon library will stop processing of this element. (see also http://www.greenbytes.org/tech/webdav/rfc4918.html#url-handling).

A workaround in davfs2 would be a lot of work and additional code as this is handled by the Neon library and I would have to duplicate a lot of library code.

What you can do:
1. Make sure it is this bug. You need option 'debug httpbody' to see the XML-response to PROPFIND. Check wether spaces in href are percent encoded or not. (You may as well catch the HTTP-traffic with Wireshark to exclude any changes by Neon or davfs2.)
2. If spaces are not percent encoded: please contact the server administrator to see if there is any configuration option with Sharepoint to change to the correct behaviour. If there is please report it here for other Sharepoint users.
3. If there is no way to force Sharepoint to compliant behaviour please file a but report to Microsoft. I see no way to work this around in davfs2. It would mean about 1000 lines of additional code.

Cheers
Werner

P.S.: Thanks for the debug log. It shows that at least NTLM-authentication is working now.

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Mon 09 Nov 2009 05:56:26 PM UTC, original submission:

Good evening!

First of all here is my env:

SLES10 SP2
Linux 2.6.16.60-0.39.3-vmi #1 SMP
davfs2-1.4.3
neon-0.29

debug most
allow_cookie 1
precheck 1
use_locks 0
use_expect100 1
if_match_bug 1

I have some trouble to work with Sharepoint folders with spaces in the name. Mounting is no problem. The problem is if I create a document or folder after mounting it disappears after seconds. I can not see the newly created folders or documents anymore. If I have a look inside Sharepoint then I can see the new items. Very strange. I have tested exact the same things with a URL without spaces and all is working fine. It seems to be a little bug with the name handling. It is very important for me to be able to work with this folder.

# mount.davfs http://sharepoint/ein\ test /sharepoint
# mount
http://sharepoint/ein test on /sharepoint type davfs (rw,nosuid,nodev,_netdev)

The mount is working no problem, but...

#cd /sharepoint
#ls
lost+found
# mkdir dir
#ls
lost+found dir
#ls
lost+found

dir is disappeared after seconds! But in sharepoint the directory exists. I have attached some lines from my messages. Maybe it helps you to find the problem faster.

Many thanks in advance!

Cheers!
Robert

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #23697:  neon-fix-davfs-spaces-issue.patch added by erikj (1kB - text/x-patch - I'm attaching the patch that worked for me, due to formatting being lost in the one in the post. Just trying to save the next guy from having to reformat.)
file #19020:  messages.txt added by None (91kB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by erikj (Updated the item)
  • -unavailable- added by warlord (Posted a comment)
  • -unavailable- added by magwas (Posted a comment)
  • -unavailable- added by clivel (Posted a comment)
  • -unavailable- added by gjs (Posted a comment)
  • -unavailable- added by wbaumann (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 25 Jul 2011 08:02:09 PM UTCerikjAttached File-=>Added neon-fix-davfs-spaces-issue.patch, #23697
    Sun 10 Apr 2011 08:32:33 AM UTCwbaumannStatusWont Do=>Ready For Test
    Thu 06 May 2010 07:24:35 PM UTCwbaumannStatusDone=>Wont Do
    Mon 11 Jan 2010 08:14:55 PM UTCwbaumannStatusIn Progress=>Done
    Mon 09 Nov 2009 06:58:07 PM UTCwbaumannSeverity5 - Blocker=>3 - Normal
      StatusNone=>In Progress
      Assigned toNone=>wbaumann
    Mon 09 Nov 2009 05:56:26 PM UTCNoneAttached File-=>Added messages.txt, #19020

    Back to the top


    Powered by Savane 3.1-cleanup