helpdavfs2 - Support: sr #107402, 302 Found / Skydrive

 
 

sr #107402: 302 Found / Skydrive

Submitted by:  ze10diez <ze10diez>
Submitted on:  Sun 13 Jun 2010 08:49:58 PM UTC  
Votes:  2  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: Need Info
Privacy: PublicAssigned to: Werner Baumann <wbaumann>
Open/Closed: OpenOperating System: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 23 Mar 2014 06:48:17 PM UTC, comment #17:

Ther is now experimental support for redirects.
Please see https://savannah.nongnu.org/bugs/?35428 for details.

This item has got mixed up.

  • problems with authentication and single sign on
  • redirects for servers that have their content distributed on several pools.

Please use this item only for authentication problems (Skydrive).

For the other problem use bug 35428.
The new experimental redirect feature is intended to help with pools of servers. It will definitely not help with the authentication problems.

Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Wed 19 Feb 2014 07:55:33 PM UTC, comment #16:

Any news on this?

Real Name <l29ah>
Sat 04 Feb 2012 09:09:08 PM UTC, comment #15:

Hey Werner.

Good to read. Keep us tuned, I'll happily try it out when you have something.

Guess you're right with the issues with respect to the RFCs and clients/servers...

I'll follow this bug.

Cheers,
Chris.

Christoph Anton Mitterer <calestyo>
Wed 01 Feb 2012 03:40:28 PM UTC, comment #14:

Hello Chris,

thanks for the information.

From what I know now I will try to do this (but only after handling with some other issues):

Implement temporary redirects (302 and 307) for GET and HEAD.
I have created a bug report for this (bug #35428).

I think this should make it workable with reasonable effort and without running in unsolved problems (like PUT).

The information of Paul beeing very helpful I can't restrain from remarking ont his:
"The simple answer is read the corresponding RFCs: RFC 4918, RFC 2616 and maybe RFC 4437"
That is - unfortunately - too simple. These RFCs are full of optional features and without knowing what is really used by servers and useful for your demand you will create bloatware with 90% of useless features and you will probably introduce a lot of bugs to implement what is not used anyway. Besides servers may use nonstandard features and they will have bugs too. So it is essential to know what servers are really doing. Besides, implementing only 20% of these RFCs would be far beyond what I would be able to do. I already spend a substantial part of my development time on working around non standard server behaviour (let alone the shortcomings of that poorly designed RFC 4918).

I have still some questions open and will try to direct them directly at the dCache maintainers.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Tue 31 Jan 2012 11:31:37 PM UTC, comment #13:

Hey Werner.

I'm not a WebDAV expert at all... but I asked a friend of the dCache developers (credits to Paul Millar) who gave me some answers:
------------------------------------------------------
The simple answer is read the corresponding RFCs: RFC 4918, RFC 2616 and maybe RFC 4437.

Here are some answers to the specific questions at the top of the above Savannah ticket:

> A Redirect in response to GET seems not too difficult;
> just repeat the request to 'Location'.


Correct; this should work fine and solves a number of issues for us.

> But this is WebDAV - where to send PUT?


We (dCache.org) would like clients to support redirect on PUT, too; however, this is likely to be more of an uphill struggle.

dCache current behaviour is to proxy/tunnel the incoming data to the corresponding pool node; so dCache will work "OK" with clients that don't support redirect for PUT.

The issue is that, although this is functional, it doesn't scale. If lots of clients are trying to write data then you will need lots of doors.

RFC 2616 defines behaviour if the server returns HTTP code 307 to a PUT request. The client should honour the redirect; however, as-is, this would involve sending the data twice: once to the door (only to receive a 307) and once to the pool.

> - will PROPFIND be redirected too.


dCache doesn't do this (so we don't care), but it is technically possible. RFC 4437 seems to go into this issue in some detail.

> - has davfs2 to remember a different server for every
> resource or should it just direct every request to the
> initial server and repeat it in case of a redirect?


This is basically just a question about caching. Read RFC 2616 for details, but the short answer is client may cache redirections for the duration specified by the server (in HTTP headers).

> (This might be expensive for PUT.)


Yes, but there's an extension / work-around / proposal to remove the additional cost for PUT.
-----------------------------------------------------------

So much for Paul's answers. I've opened a ticket at dCache.org, with respect to dafs2 / dCache-webdav interaction.
Unfortunately the ticket system there is not publicly visible, but you can add questions to it, by sending mail to -unavailable- with the subject being "#6991 WebDAV guru needed". I guess they will give you some friendly help, especially when it comes to interaction questions with dCache's WebDAV server.

My own "answers" perhaps:
- as Paul already said,.. if redirects would be supported for PUT, too, that would be great.

> has davfs2 to remember a different server for every
> resource or should it just direct every request to the
> initial server and repeat it in case of a redirect?

As Paul said, at most to the extent of what cahing allows... the storage system behind might have already moved the data to some other physical host.
So I guess (really just a guess) dCache would always return "0" for the caching time.

>And there are read-only repositories. How is the indendet
>use case here?
>Will davfs2 just be used to get a local copy, in which
>case plain HTTP/HTML might be the better solution.

I'm not sure how far davfs / FUSE use the kernels filesystem cache... but this could be one use case.
Though I haven't thought about the implications of filesystem caching... I mean somehow you'd have to update the status of cached files (whether they still exist, have been changed, etc.)

>Or is mapping into a local file system required, e.g.
>because applications want to access the resources through
>the file system interface?

This is IMHO definitely also a strong reason!

Cheers,
Chris

Christoph Anton Mitterer <calestyo>
Mon 30 Jan 2012 07:43:25 PM UTC, comment #12:

There are different request for Redirect support and with most of them it is not clear to me what is required.

- there are redirects to some outhentication server with unknown authentication protocol (maybe HTNL and JavaScript based). Without substantial information about it I won't do anything.

- there are redirects caused by distributed repositories. Let us sort out these:

A Redirect in response to GET seems not too difficult; just repeat the request to 'Location'.
But this is WebDAV.
- where to send PUT?
- will PROPFIND be redirected too.
- has davfs2 to remember a different server for every resource or should it just direct every request to the initial server and repeat it in case of a redirect? (This might be expensive for PUT.)

And there are read-only repositories. How is the indendet use case here? Will davfs2 just be used to get a local copy, in which case plain HTTP/HTML might be the better solution. Or is mapping into a local file system required, e.g. because applications want to access the resources through the file system interface?

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Thu 26 Jan 2012 10:32:35 PM UTC, comment #11:

Oh and perhaps one can retitle this bug, as it's definitely not just Skydrive related :)

Christoph Anton Mitterer <calestyo>
Thu 26 Jan 2012 10:31:41 PM UTC, comment #10:

Hi.

What's the status on this? Not supporting the redirect family of statuses is a critical blocker for many WebDAV enabled servers.

Especially for mass storages systems (like dCache, which is extensively used in the high energy physics community around the world, by CERN and all other big LHC partners with severl ten-thousands of users) it's absolutely reasonable to use 302.
These systems typically have a storage name for a file, e.g. something like:
http://webdav-door.example.org/this/is/my/human-readable/path
but the file lays one of many hundrets of disk pools, e.g.:
http://pool42.example.org/someHexadecimalFileID

As these files may move around for optimisation and as there may be many replicas of the same file on different pools, 302 is not only an option, but mandatory.

I doubt that following these redirects would be so difficult to implement and the benefit for all users would be quite large.

Cheers,
Chris.

Christoph Anton Mitterer <calestyo>
Sun 16 Oct 2011 10:17:16 AM UTC, comment #9:

Hello Samuel,

sorry to disappoint you. I'm not really working on this.

I would like to add support for Single Sign On which might also need some redirect support. But it is hard to find good documentation about this. And most probably there are many incompatible ways of Single Sign On. With me not having access to any of this (and no interest too) I can't see how to do this.

Another thing are general redirects. I did not include them in davfs2 because I can't see how this would work with read and write access and I'm not sure about security implications.

With read-only access as in your case it might be different. I could imagine to have an option to enable redirects for read operations (GET, PROPFIND). But I can't see me doing this in the near future.

Maybe you could create a patch.
As the pool to which clients are redirected probably depends on the location of the client there might be other ways to assign the nearest pool to the clients.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 30 Sep 2011 02:20:02 PM UTC, comment #8:

(Well, sorry, read the thread in the wrong direction. See now that there is work ongoing already. Never mind.)

Samuel Lampa <samuellampa>
Fri 30 Sep 2011 02:17:42 PM UTC, comment #7:

We are running into the same problem, trying to mount the national storage for Swedish High Performing Computing Centers, which uses redirects for direct access to pools (located across the country) for reading.

Would highly appreciate it davfs2 would support redirects! Otherwise we have to go from 30 to 1 Gbit/s bandwidth, when reading from one single node :-o

Cheers // Samuel

Samuel Lampa <samuellampa>
Sun 04 Sep 2011 02:48:45 PM UTC, comment #6:

I did some research on this -- the Passport authentication is somewhat tricky. If you want to add support for it, have a look at
http://skydrivesimpleviewer.codeplex.com/

There is full C# source available for Skydrive client that does proper authentication routine.

Lauri

Anonymous
Fri 24 Sep 2010 08:02:27 PM UTC, comment #5:

What happens:
When you try to connect you are redirected to a login-server. This login-server should probably provide some cookie for authentication and then redirect you to the WebDAV-server again.
To get this working the details of the communication with the login-server would be important. With Microsoft having no interest in providing free webspace for GNU/Linux-clients there is not much change to get these details.

I tried to connect to the login-server using gnutls-cli. What I got:
- 2 cookies
- an error message complaining about Javascript beeing disabled
- a load of Javascript.
So the login-server seems to require Javascript support. davfs2 will not do this.

What you may try:
Connect and authenticate using the web interface.
Get the URL of the webdav server from the browser (it may be different from the original one).
Try to get the cookies from your browser.
Configure davfs2 for this: use Option 'add_header' to add the cookies (header name is 'Cookie').
Try to mount again.
There will be a lot of trials needed:
- which cookies to use (both, only one, which)
- which URL (the URL for the web interface may be different from that for WebDAV)

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Thu 23 Sep 2010 08:51:05 PM UTC, comment #4:

I am having the same issue. After running the following and using the debugs mentioned below I pulled the following out.

mount.davfs "https://gggk8a.docs.live.net/a5ded7389baaaaaa/stuff" /tmp/skydrive

Header Name: [location], Value: https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1285274014&rver=6.1.6206.0&wp=MBI&wreply=https:%2F%2Fgggk8a.docs.live.net:447%2Fa5ded7389baaaaaa%2Fstuff%2F&lc=1111&id=222222

Using the wreply url I get a connection timed message.

mount.davfs "https://gggk8a.docs.live.net:447/a5ded7389baaaaaa/stuff/&lc=1111&id=222222" /tmp/skydrive

mount.davfs: Request ends, status 0 class 0xx, error line:#012Could not connect to server: Connection timed out

I have tried different sub-folders and without a sub-folder with the same results. It does not matter if I put in my ID and password or an incorrect ID and password. I still get the same timeout message.

Is there anything else I can try? Has anyone been able to get this working?

dpkg -l davfs2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-=================================-=================================-==================================================================================
ii davfs2 1.4.6-1 mount a WebDAV resource as a regular file system

mike <mikeones>
Sun 08 Aug 2010 08:56:52 AM UTC, comment #3:

Hello Guido,

I doubt that the SkyDrive problem and the SharePoint problem are the same.
Therefore I created a new thread at
https://savannah.nongnu.org/support/index.php?107453

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Fri 06 Aug 2010 01:32:47 PM UTC, comment #2:

same with SharePoint...

http://msdn.microsoft.com/en-us/library/cc250199%28PROT.10%29.aspx

SEE: Section 2.2.8

+ + +

I even tryied to recompile forcing a IE6 user agent string, but still got a 302...

Guido Serra <zeph>
Tue 15 Jun 2010 06:28:25 PM UTC, comment #1:

With "302 found" the server redirects you to another URL to access the requested resource. But - according to the spec - it also means that this new URL is only temporarily valid.
The same response where you got the "302 found" status code from must contain a "Location" header with the new URL.

This is quite uncommon behaviour for a WebDAV server that offers "remote disk space" to its customers. Redirects are not supported by davfs2. davfs2 expects you to know the correct URL of the root directory of the davfs2 file system in advance. It also expects that all files in your file system have URLs that are descendents of this root URL. And in no case it will connect to another server.

As you got the response code from the server you should be able to get the Location header too. You may try to use the URL from the Location header when mounting the file system, instead of the URL you use now.

Cheers
Werner

Werner Baumann <wbaumann>
Project AdministratorIn charge of this item.
Sun 13 Jun 2010 08:49:58 PM UTC, original submission:

I get "302 found" when I try to connect skydrive webdav address generated by "skydrive simple viewer".
Please help.

ze10diez <ze10diez>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by l29ah (Posted a comment)
  • -unavailable- added by neve
  • -unavailable- added by gmerino
  • -unavailable- added by calestyo (Posted a comment)
  • -unavailable- added by calestyo (Voted in favor of this item)
  • -unavailable- added by samuellampa (Posted a comment)
  • -unavailable- added by mikeones (Posted a comment)
  • -unavailable- added by zeph (Posted a comment)
  • -unavailable- added by wbaumann (Posted a comment)
  • -unavailable- added by ze10diez (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 2 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 11 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 06 Aug 2012 09:43:32 AM UTCneveCarbon-Copy-=>Added neve
    Fri 02 Mar 2012 02:45:00 PM UTCgmerinoCarbon-Copy-=>Added gmerino
    Mon 30 Jan 2012 07:43:25 PM UTCwbaumannStatusPostponed=>Need Info
    Thu 26 Jan 2012 10:31:41 PM UTCcalestyoCarbon-Copy-=>Added calestyo
    Sun 16 Oct 2011 10:17:16 AM UTCwbaumannStatusIn Progress=>Postponed
    Fri 24 Sep 2010 08:02:27 PM UTCwbaumannStatusNeed Info=>In Progress
      Open/ClosedClosed=>Open
    Sat 28 Aug 2010 09:14:54 AM UTCwbaumannOpen/ClosedOpen=>Closed
      StatusIn Progress=>Need Info
    Tue 15 Jun 2010 06:28:25 PM UTCwbaumannStatusNone=>In Progress
      Assigned toNone=>wbaumann

    Back to the top


    Powered by Savane 3.1-cleanup