Although, you should consider ranged requests a feature request.  Without that, we will still get timouts on the first download of large files.

davfs2 gets different etags on PROPFIND and GET.

At start davfs2 does a PROPFIND for the root directory:
Request:
line  124: PROPFIND / HTTP/1.1
Response:
line  148: HTTP/1.0 207 Multi-Status
line  561: <D:href>/README</D:href>
line  564: <lp1:getetag>"1694001-3f-4931d671b0000"</lp1:getetag>

The file is not in the davfs2-cache and davfs2 does an unconditional GET:
Request:
line 2646: GET /README HTTP/1.1
Response:
line 2655: HTTP/1.0 200 OK
line 2670: ETag: "3f-4931d671b0000"
The file is cached by davfs2 with etag "3f-4931d671b0000".

The file is in the cache and davfs2 does a conditional GET:
Request:
line 3093: GET /README HTTP/1.1
line 3099: If-None-Match: "3f-4931d671b0000"
Response:
line 3104: HTTP/1.0 304 Not Modified
line 3109: ETag: "3f-4931d671b0000"
davfs2 serves the file system call from its cache as the file is not modified on the server. The file stays in the cache.

In reaction to a ls davfs2 does a PROPFIND:
Request:
line 3509: PROPFIND / HTTP/1.1
Response:
line 3533: HTTP/1.0 207 Multi-Status
line 3947: <D:href>/README</D:href>
line 3950: <lp1:getetag>"1694001-3f-4931d671b0000"</lp1:getetag>
davfs2 will remove the cache file because the etag has changed.

The next GET is again unconditional:
Request:
line 4703: GET /README HTTP/1.1
Response:
line 4713: HTTP/1.0 200 OK
line 4728: ETag: "3f-4931d671b0000"

The reason for removing the cached file is that davfs2 gets etag "3f-4931d671b0000" on GET, but etag "1694001-3f-4931d671b0000" on PROPFIND. What is strange is that the etag in GET is a truncated version of the etag in PROPFIND. The etag in PROPFIND is from the origin server. Maybe the proxy is truncating the etag. But it may be a bug in the origin server as well.

Anyway, etags in PROPFIND and in GET must be identical according to the spec. davfs2 has no workaround to handle mismatching etags as reliable etags are essential in distributed authoring.

I hope the owner of the cache server will be able to locate the bug and to fix it. Maybe the above information will be helpful for this.

Cheers
Werner

Thanks for the log file and yes, it looks like a problem with the proxy.

There is a small problem with the log file. The request lines (e.g. line 518: "Sending request headers:" GET and some headers are missing) is truncated. But it is not a big problem. From the context the request method is obvious. I don't know whether this is a problem with davfs2/neon (but I never saw this before) or with the log file.

Now what is going on.

When README is opened the first time davfs2 sends a GET request (line 518). The response is 200 OK meaning the file is fetched from the server. When it is opened for the second time (line 737) the response is 304 Not Modified and davfs2 takes the file from the davfs2 cache. This looks pretty well.

But the response also contains some headers from the proxy that indicate the reason for the trouble:

• Warning: 110 squid/3.1.11 "Response is stale"
• Warning: 111 squid/3.1.11 "Revalidation failed"
• X-Cache: HIT from mirror-cache2-1.mtv.corp.google.com

The proxy served a stale version from its cache because it could not validate it or fetch the latest version from the origin server.

When an ls is done davfs2 will do a PROPFIND request (line 949). The response is 207 Multi-Status which is OK. But the proxy message is now

• X-Cache: MISS from mirror-cache2-1.mtv.corp.google.com

This response is served from the origin server. The response contains meta-data of the files in the directory, including the etag. Most probably the meta-data from the origin server do not match the ones from the stale response from the proxy and davfs2 removes the stale files from the cache.

To be absolutely sure about this one had to look at the XML-body of the PROPFIND-response which is not included in 'debug most'. You could add option 'debug httpbody' or fetch the http-traffic with Wireshark.

But from all I can see the problem is in that "Revalidation failed"-message. The proxy should not serve stale files.

Cheers
Werner

"Caching:
With some more testing, what seems to be happening is that every file in the cache gets invalidated after dir_refresh seconds. Also, simply running ls, or doing a shell glob of files in a directory invalidates that cache, forcing a full re-download on next read. Is this the expected behavior? I don't understand why reading a directory needs to invalidate the cache of a file in that directory. I could raise the dir_refresh to something ridiculously high, and avoid directory reads, but this would have other consequences as well."
This is not what davfs2 usually does. There is something wrong with cache validation. Maybe the server does not send etags and/or Last_modified headers or changes them without reason.
Running davfs2 with debugging enabled would probably show what is going wrong.

Cheers
Werner

Caching:
When a file is requested that is already in the cache davfs2 will do a a conditional GET (header If-Not-Modified) and the file will only be downloaded again when it has changed on the server. Assuming your cache is big enough and the value in option "cache-size" is big enough too. When files are redownloaded without need there is probably something wrong with cache validation. Please set option "debug most" and send the debug messages to see what is going wrong in the communication with the server.

Timeouts:
I assume they occur when the tftp client tries to download a file that is not in the cache. The reason is that davfs2 will first download the complete file before the tftp server can use it. Partial requests (ranged request) would help here but are not implemented in davfs2. AVM's FritzBox uses a patched version of davfs2 that uses ranged requests. But I had not time to look at it more closely and implement it in davfs2.

Better ways to do it:
I am not too familiar with other tools. There are a lot of tools to send and fetch data from a server (scp, rsync, sftp, wget). The problem is to connect these tools with the tftp server.

Cheers
Werner