bugdavfs2 - Bugs: bug #27191, can davfs support HTTP compression?

 
 

bug #27191: can davfs support HTTP compression?

Submitter:  Jason Haar <jhaar>
Submitted:  Thu 06 Aug 2009 07:43:08 PM UTC
Votes: 100
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Wont Fix
Privacy:  Public Assigned to:  _71007
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 13 Mar 2014 04:48:27 PM UTC, comment #8: 

No, but there is an easy way to test by sending Accept-Encoding:

  • add option 'add_header Accept-Encoding gzip' to davfs2.conf (no ":")
  • add option 'debug HTTP' to davfs2.conf
  • mount the file system and open a file for reading that is not in the cache.

If your server supports content-encoding then this will probably fail. But one of your log files will show the HTTP-headers. Look for header "Content-Encoding: gzip".

Finally remove these options from davfs2.conf.

Werner

- <_71007>
Thu 13 Mar 2014 07:28:52 AM UTC, comment #7: 

Thanks for the hint. I followed your discussion about TE and Content Encoding, and I'm still a bit confused..
I'm looking forward to see the change in a release version. Is there some easy way to see if my WebDav-Server supports Content Encoding without sending a request with Accept-Encoding: gzip ?

Massimo <massimo_b>
Wed 12 Mar 2014 05:12:40 PM UTC, comment #6: 

There is now a solution for downloads using content-encoding (see bug #41355). I will not care about TE-compression until there is evidence that some important WebDAV-servers implemented it.
But anyway this would even then better be done in the Neon library.

Werner

- <_71007>
Mon 25 Feb 2013 08:28:39 PM UTC, comment #5: 

Sorry, there are no news and it does not look like I could do this in the near future.

What are the problems:

  • As far as I can see I would be forced to use a lower level of the Neon interface which means more code to write myself. But I already lack time to do more pressing changes.


  • I am not sure it is worth the effort. The question is: does any reasonable number of Web-servers support Transfer encoding at all? Does Apache? My impression is that this is not the case but would be grateful for more reliable information about this.


Please note:
There is some confusion about but an important difference between Transfer-Encoding gzip (header TE) and Content-Encoding gzip (headers Accept-Encoding and Content-Encoding).
If it is Content-Encoding gzip davfs2 must not decompress it. davfs2, as a file system, has to treat it as any other regular file of any other MIME-Type (browsers may behave differently).
Only Transfer-Encoding gzip would be useful for davfs2.

Do you know whether Apache supports Transfer-Encoding, and if so whether any WebDAV-service supports it?

Werner

- <_71007>
Wed 20 Feb 2013 09:58:46 AM UTC, comment #4: 

Are there any news about that issue? Using http compression could save a huge amount of traffic on WANs. What is the main effort to implement it using zlib to http if the server supports it?

Some background: I'm using davfs2 with unison to have a centralized synchronisation of various document folders with some ssl secured WebDAV from my provider. Since the data is mostly binary and I don't need history I choosed this approach instead of some SCM like git.
fastcheck=default using inode changes only works on Unix filesystems, but not on vfat and I guess not on davfs2. So I force it to fastcheck=true "[quote from manuals].. to use file creation times as 'pseudo inode numbers' when scanning replicas for updates, instead of reading the full contents of every file...". I also have the davfs2 cache as big as the whole WebDAV space. This seems to work with davfs2 and improves speed, but compression would still help with the many uncompressed files.

Best regards,
Massimo

Massimo <massimo_b>
Sun 18 Oct 2009 07:15:50 PM UTC, comment #3: 

I am sorry to tell that implementing HTTP transport compression requires more work than I expected as it is not supported by the neon library.
So I had to delay work on this and I can't tell when I will have time enough to dig into the matter.

Cheers
Werner

- <_71007>
Thu 06 Aug 2009 07:43:08 PM UTC, comment #2: 

This item has been reassigned from the project davfs2 support tracker to your tracker.

The original report is still available at support #106963

Following are the information included in the original report:

[field #0]                  Item ID: 106963
[field #1]                 Group ID: 10199
[field #2]              Open/Closed: Open
[field #3]                 Severity: 1 - Wish
[field #4]                  Privacy: Public
[field #8]                         : Unknown support Field Display Type
[field #9]                 Category: None
[field #10]             Submitted by: jhaar
[field #11]              Assigned to: wbaumann
[field #12]             Submitted on: Do 06 Aug 2009 09:05:43 GMT
[field #13]                  Summary: can davfs support HTTP compression?
[field #14]      Original Submission: Hi there

I can see with a sniffer that when I use davfs as a filesystem, it doesn't take advantage of Apache's ability to compress HTTP content. ie opening a large file causes the large file to be "GET"/got uncompressed.

I was wondering if that is an oversight, or is there something funky about DAV and compression that don't allow them to play well together?

On a WAN, compress could make a BIG difference

Thanks

Jason
[field #16]               Item Group: None
[field #17]                   Status: None
[field #18]        Component Version: None
[field #19]         Operating System: GNU/Linux
[field #20]          Reproducibility: None
[field #21]               Size (loc): None
[field #22]            Fixed Release: None
[field #23]          Planned Release: None
[field #24]                   Effort: 0.00
[field #28]                 Priority: 5 - Normal
[field #31]         Percent Complete: 0%
[field #33]                  Release: None
[field #58]     Custom Select Box #1: None
[field #59]     Custom Select Box #2: None
[field #60]     Custom Select Box #3: None
[field #61]     Custom Select Box #4: None
[field #62]     Custom Select Box #5: None
[field #63]     Custom Select Box #6: None
[field #64]     Custom Select Box #7: None
[field #65]     Custom Select Box #8: None
[field #66]     Custom Select Box #9: None
[field #67]    Custom Select Box #10: None

- <_71007>
Thu 06 Aug 2009 07:41:52 PM UTC, comment #1: 

Hello Jason,

thanks for the hint.
It is an oversight. From all I know there is nothing in DAV that would prevent it. I agree with you on its importance for WAN and in my opinion davfs2 should aim mainly on WAN connections.
From my current schedule in real life I can't promise more than a prominent place in my TODO list. For the moment I will move this item to the bug tracker.

Cheers
Werner

- <_71007>
Thu 06 Aug 2009 07:43:08 PM UTC, original submission:  

Hi there

I can see with a sniffer that when I use davfs as a filesystem, it doesn't take advantage of Apache's ability to compress HTTP content. ie opening a large file causes the large file to be "GET"/got uncompressed.

I was wondering if that is an oversight, or is there something funky about DAV and compression that don't allow them to play well together?

On a WAN, compress could make a BIG difference

Thanks

Jason

Jason Haar <jhaar>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by massimo_b (Posted a comment)
  • -email is unavailable- added by massimo_b (Voted in favor of this item)
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by jhaar (Submitted the item)
  •  

    There are 100 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-04-22 _71007 Open/ClosedOpen Closed
    2014-03-12 _71007 StatusPostponed Wont Fix
    2013-02-20 massimo_b Carbon-Copy- Added massimo_b
    2009-10-18 _71007 Severity1 - Wish 3 - Normal
        StatusNone Postponed
        Assigned toNone _71007
    2009-08-06 _71007 Reassign itemdavfs2, sr #106963 davfs2, bug #27191

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code