maindavfs2 - Support: sr #106878, IIS6 requres MIME type

 
 

sr #106878: IIS6 requres MIME type

Submitter:  Hans-Jürgen Oertwig <juergen_998>
Submitted:  Fri 05 Jun 2009 09:01:05 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  4 - Important Status:  Postponed
Privacy:  Public Assigned to:  _71007
Open/Closed:  Closed Operating System:  Microsoft Windows
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 08 Jun 2009 06:34:02 PM UTC, comment #3: 

Hello Hans-Juergen,

agreed.
Though there is indeed some information about IIS in the FAQ; but it's not easy to find there, I don't maintain the FAQ anymore and I don't like FAQs.
I always wanted to and sometime I will do a proper GNU-style documentation and there will be a structered section about known server specific problems.
Meanwhile I hope to not lose any of this information that is mainly based on user feedback.

Cheers
Werner

- <_71007>
Mon 08 Jun 2009 04:31:43 PM UTC, comment #2: 

Hello Werner,

Thank you for the detailed reply, it really is a messy subject! My proposal is to add a hint regarding mime types and IIS into the manual or readme. This should save others a lot of time investigating this in the future.

Cheers

Hans-Juergen

Hans-Jürgen Oertwig <juergen_998>
Sat 06 Jun 2009 11:52:22 AM UTC, comment #1: 

Hello Hans-Juergen,

you are touching a very specialized field of chaos theory, also known as file-type, content-type, MIME-type or file-extension. I'm not an expert in this field but will try to give an overview from the point of view of an interested layman and then try to explain how this influences the behaviour of IIS-WebDAV. If you haven't got anything better to do you might look at http://lists.w3.org/Archives/Public/ietf-http-wg/2009JanMar/0290.html to see what experts are able to do, especially when commercial interests of the advertising industry are involved.

Overview (the little I know):
Once upon a time applications started to use the a dot and some characters at the end of a file name to mark the type of a file or the content included in this file. This is also known as file extension. There is and never was a standard for this though some of these extension are quite commonly used. But there are many cases where different applications will use the same extensions for different content, or different extensions for the same content.
Then came MIME-Types, first used in email. They are much more verbose and there is a central registry so they should be unique. But up to this operating systems offered no way to store this kind of meta data along with the file contents. Nowadays most have some way to to this (e.g. extended file attributes), but as far as I can see, this is not standardized and not commonly used.
The state of art seems to be this:
Operating systems or applications maintain a registry for MIME-types. Usually this is just a mapping from file extensions to MIME-types. As there is no standard for file extensions this mapping is just as unreliable. You probably have already seen mislabeled files in your graphical file explorer, like source code labeled as screen saver and the like.

And here is HTTP and WebDAV:
HTTP makes it a SHOULD requirement for webservers to send a Content-Type header containing the MIME-type. Webservers usually get this from the above mentioned registry, so some mislabeling will occur. Webbrowser will then mistrust the Content-Type header and guess the MIME-type by their own means, failing just as often.
In WebDAV the requirements of HTTP must be met too. But there is no requirement to announce the MIME-type when uploading a file from the client to the server. WebDAV allows the client to set the MIME-type by sending a PROPPATCH request. I tried to implement this in davfs2, the result was frustrating. Apache as well as IIS would accept the request, but will then send two different MIME-types: one type in the Content-Type header - derived from the file extension, another one on PROPFIND - the one defined by the client. So I stopped work on this nonsense.

How is this related to your experience with IIS:
I haven't access to an IIS at the moment; my last tests were about two years ago, but it seems to be unchanged.
IIS allows uploading of any kind of file, independent of the file extension or any MIME-type declaration. But when someone tries to download the file it tries to get the MIME-type (according the SHOULD-requirement of HTTP). It gets this from the file extension using it's MIME-type registry (or that of the OS). If the extension is not registered it sends 404 NOT FOUND, pretending there is no such file. This is intended behaviour and the only way for you to change this, is to register your MIME-types, or in case of WebDAV register a wildcard MIME-type because you can't know all the file extensions clients might use.
NOTE 1: there is no way for clients to influence this, it is solely determined by the internal working of IIS.
NOTE 2: to register a wildcard extension is not a dirty workaround. IIS policy is to only server registered file extensions. So you have to register all the file extension you want to be served. If you want IIS to serve any file extension you have to say so; any is the wildcard. If IIS logs a 404 error for a registered file extension this is a bug or some misconfiguration and you should contact microsoft.

In the case of davfs2 it gets worse: davfs2 does caching and therefore needs some meta-data for cache validation, mostly it needs the Etag. Servers do not submit the Etag in their response to an upload (they are allowed to, but they don't). As davfs2 needs the Etag it does an HEAD request immediately after the upload. IIS will answer 404 NOT FOUND because of unregistered file extension and davfs2 must assume the upload failed. That's how your files end up in lost+found.

Cheers
Werner

P.S.:
I doubt that this kind of MIME-type restriction done by IIS enhances security. Faking the file extension is not a problem at all. And current browsers tend to ignore the MIME-type according to their policy:
- showing a blank page because of bad MIME-type is not acceptable
- showing some cool animation while executing malware in the background is an exiting Web-experience.

- <_71007>
Fri 05 Jun 2009 09:01:05 AM UTC, original submission:  

Hello,

I wanted to transfer a batch of backup slices created with DAR to a local Windows 2003 server with IIS6. But several files just appeared in the lost+found directory and were not transferred. The IIS server log showed 404 errors with subcode 0 50. According to
<a href="http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0f4ac79a-dc2b-4a5f-89c1-d57266aa6ffe.mspx?mfr=true">http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0f4ac79a-dc2b-4a5f-89c1-d57266aa6ffe.mspx?mfr=true</a>
this means "MIME map policy prevents this request."

I found nothing about this in the davfs documentation or in any discussion groups. I got the backup working by adding the .dar-extension to the IIS metabase as mime type application/octet-stream. But this seems to be dirty workaround because the 404 errors still appear. They only disappear if a wildcard extension is added which disables all mime type security. Furthermore it is quite a problem if only files are transferred properly with an extension known in the IIS metabase.

Version info: Davfs2, Version 1.3.3 with Neon 0.28 in Debian 4.0

Thanks

Hans-Juergen

Hans-Jürgen Oertwig <juergen_998>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by juergen_998 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-12-26 _71007 Open/ClosedOpen Closed
    2009-10-18 _71007 StatusNone Postponed
    2009-06-06 _71007 Assigned toNone _71007

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code