maindavfs2 - Support: sr #110234, Unable to write certain files to...

 
 

sr #110234: Unable to write certain files to server

Submitter:  Hubert Krause <hukrause>
Submitted:  Thu 07 May 2020 02:23:46 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  In Progress
Privacy:  Public Assigned to:  _71007
Open/Closed:  Open Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 18 Jul 2020 10:53:02 AM UTC, comment #4: 

Hello,

I've opend a bug in the Debian bugtracking system.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=965211

Hubert

 Hubert Krause <hukrause>
Sun 28 Jun 2020 08:14:17 AM UTC, comment #3: 

"I'am totally confused... Should I raise a bug in the apache packet?"

Yes, please do. It looks like the xxx.ifo-file has a special purpose on the server. When it is overwritten by your file the bug is triggered. But anyway: it is a server bug.

Werner

- <_71007>
Tue 09 Jun 2020 02:21:05 PM UTC, comment #2: 

Hello Werner,

sorry for the delay. It is indeed the webdav server which seems to be confused. The files with Problems are allways with the same file ending and as such are pushed allways the same way to the server. It is allways a file with <name>.csv.info and <name>.csv. One the client machine the following is happen (with the example of my first article:

cd somewhere/on/the/local/filesystem
rm -rf /mnt/webdav/Java11/Ubuntu_16.04_LTS/bench-output
mkdir /mnt/webdav/Java11/Ubuntu_16.04_LTS/bench-output
cp -r bench-output/* /mnt/webdav/Java11/Ubuntu_16.04_LTS/bench-output

The rm -rf leads to directory traversel. The read of the Directory Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ will be answered correctly with only <name>.csv.info files. These files will be deleted. After that the folder will be deleted.

With the cp -r the folder is created again (lfooGridProviderBenchmark/ ) and than all files in this folder are locked (<name>.csv.info, <name>.csv). After locking the file <name>.csv.info is written (with PUT) and then davfs try to write  <name>.csv and got an 412.

If one go to the webdav server with a browser, it is possible to fetch the file <name>.csv without error although the file is not there. The file content is the same as the file <name>.csv.info.

From that point on, it is impossible to write the file <name>.csv.

It become possible again to copy such a file, if one delete the <name>.csv.info file.


172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ HTTP/1.1" 207 2836
172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "DELETE /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv.info HTTP
/1.1" 204 -
172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "DELETE /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers.csv.info HTTP/1.1" 204 -
172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "DELETE /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/disableDistributedLock.csv.info HTTP/1.1" 204 -
172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "DELETE /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv.info HTTP/1.1" 204 -
172.16.xx.yy - [09/Jun/2020:01:04:42 +0200] "DELETE /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ HTTP/1.1" 204 -
......
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "MKCOL /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ HTTP/1.1" 201 370
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv.info HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/disableDistributedLock.csv.info HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers.csv.info HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv.info HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/disableDistributedLock.csv HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers.csv HTTP/1.1" 200 426
172.16.xx.yy - [09/Jun/2020:01:04:44 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv HTTP/1.1" 200 426
......
172.16.xx.yy - [09/Jun/2020:01:05:21 +0200] "PUT /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv.info HTTP/1.1" 201 388
172.16.xx.yy - [09/Jun/2020:01:05:21 +0200] "HEAD /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv.info HTTP/1.1" 200 -
172.16.xx.yy - [09/Jun/2020:01:05:21 +0200] "UNLOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv.info HTTP/1.1" 204 -
......
172.16.xx.yy - [09/Jun/2020:01:05:21 +0200] "PUT /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv HTTP/1.1" 412 348
172.16.xx.yy - [09/Jun/2020:01:05:21 +0200] "UNLOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/oneConsumer.csv HTTP/1.1" 400 341


We use mod_dav on the apache2 on our webdav server (2.4.38-3+deb10u3):


ls -l /etc/apache2/mods-enabled/
total 0
lrwxrwxrwx 1 root root 36 Nov 25  2019 access_compat.load -> ../mods-available/access_compat.load
lrwxrwxrwx 1 root root 28 Nov 25  2019 alias.conf -> ../mods-available/alias.conf
lrwxrwxrwx 1 root root 28 Nov 25  2019 alias.load -> ../mods-available/alias.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 auth_basic.load -> ../mods-available/auth_basic.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 authn_core.load -> ../mods-available/authn_core.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 authn_file.load -> ../mods-available/authn_file.load
lrwxrwxrwx 1 root root 34 Nov 26  2019 authnz_ldap.load -> ../mods-available/authnz_ldap.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 authz_core.load -> ../mods-available/authz_core.load
lrwxrwxrwx 1 root root 38 Nov 26  2019 authz_groupfile.load -> ../mods-available/authz_groupfile.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 authz_host.load -> ../mods-available/authz_host.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 authz_user.load -> ../mods-available/authz_user.load
lrwxrwxrwx 1 root root 32 Nov 25  2019 autoindex.conf -> ../mods-available/autoindex.conf
lrwxrwxrwx 1 root root 32 Nov 25  2019 autoindex.load -> ../mods-available/autoindex.load
lrwxrwxrwx 1 root root 29 Nov 25  2019 dav_fs.conf -> ../mods-available/dav_fs.conf
lrwxrwxrwx 1 root root 29 Nov 25  2019 dav_fs.load -> ../mods-available/dav_fs.load
lrwxrwxrwx 1 root root 26 Nov 25  2019 dav.load -> ../mods-available/dav.load
lrwxrwxrwx 1 root root 30 Nov 25  2019 deflate.conf -> ../mods-available/deflate.conf
lrwxrwxrwx 1 root root 30 Nov 25  2019 deflate.load -> ../mods-available/deflate.load
lrwxrwxrwx 1 root root 26 Nov 25  2019 dir.conf -> ../mods-available/dir.conf
lrwxrwxrwx 1 root root 26 Nov 25  2019 dir.load -> ../mods-available/dir.load
lrwxrwxrwx 1 root root 26 Nov 25  2019 env.load -> ../mods-available/env.load
lrwxrwxrwx 1 root root 29 Nov 25  2019 filter.load -> ../mods-available/filter.load
lrwxrwxrwx 1 root root 27 Nov 26  2019 ldap.conf -> ../mods-available/ldap.conf
lrwxrwxrwx 1 root root 27 Nov 26  2019 ldap.load -> ../mods-available/ldap.load
lrwxrwxrwx 1 root root 27 Nov 25  2019 mime.conf -> ../mods-available/mime.conf
lrwxrwxrwx 1 root root 27 Nov 25  2019 mime.load -> ../mods-available/mime.load
lrwxrwxrwx 1 root root 32 Nov 25  2019 mpm_event.conf -> ../mods-available/mpm_event.conf
lrwxrwxrwx 1 root root 32 Nov 25  2019 mpm_event.load -> ../mods-available/mpm_event.load
lrwxrwxrwx 1 root root 34 Nov 25  2019 negotiation.conf -> ../mods-available/negotiation.conf
lrwxrwxrwx 1 root root 34 Nov 25  2019 negotiation.load -> ../mods-available/negotiation.load
lrwxrwxrwx 1 root root 33 Nov 25  2019 reqtimeout.conf -> ../mods-available/reqtimeout.conf
lrwxrwxrwx 1 root root 33 Nov 25  2019 reqtimeout.load -> ../mods-available/reqtimeout.load
lrwxrwxrwx 1 root root 31 Nov 25  2019 setenvif.conf -> ../mods-available/setenvif.conf
lrwxrwxrwx 1 root root 31 Nov 25  2019 setenvif.load -> ../mods-available/setenvif.load
lrwxrwxrwx 1 root root 36 Nov 25  2019 socache_shmcb.load -> ../mods-available/socache_shmcb.load
lrwxrwxrwx 1 root root 26 Nov 25  2019 ssl.conf -> ../mods-available/ssl.conf
lrwxrwxrwx 1 root root 26 Nov 25  2019 ssl.load -> ../mods-available/ssl.load
lrwxrwxrwx 1 root root 29 Nov 25  2019 status.conf -> ../mods-available/status.conf
lrwxrwxrwx 1 root root 29 Nov 25  2019 status.load -> ../mods-available/status.load


And our server configuration (anonymized) is attached

I'am totally confused... Should I raise a bug in the apache packet?

Best regards,

Hubert

(file #49233)

 Hubert Krause <hukrause>
Sun 17 May 2020 08:59:31 AM UTC, comment #1: 

So what happens with file
Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv

At line 583 of the log file davfs2 sends a PROPFIND-request for directory Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/
The request is successful but file two_consumers_disableDistributedLock.csv
does not exist.

In line 644 davfs2 sends a lock request for file two_consumers_disableDistributedLock.csv
This is successful. Statuscode is 200 OK.This means: the name is locked, but no file is created (otherwise status had to be 201 Created).

At line 697 davfs2 send a PUT-request for that file with additional header "If-None-Match: *", so the request must only succeed if the file does not yet exist. Status code is "412 Precondition Failed". This means: the file already exists.

Why. There are different possible reasons:

  • the server has some kind of if_match_bug (Note: WebDAV is not necessarily handled by Apache or the mod_dav module of Apache. It looks like WebDAV is done by some other module but the headers don't tell which one).
  • the WebDAV-module has a bug in the implementation of LOCK. Maybe it creates an empty file (which is allowed) but responds with the wrong status code (200 instead of 201).
  • some other user created a file with that name.


You may try options "if_match_bug 1" and in a second try additionally "precheck 0".
You may also check if there is something common with files that are rejected, e.g. only new files, only files with an underscore in the name, or ...

Werner

- <_71007>
Thu 07 May 2020 02:23:46 PM UTC, original submission:  

Hello,

we use one webdav server to which a few clients are connected. Each client writes into its on folder exclusively. The clients writes in single nightly jobs to the webdav server.
While most read write operations work, some files has permanent problems to write.

The environment is:

  • davfs2 1.5.2
  • ubuntu 1604
  • Apache/2.4.38 (Debian) as webdav server


We connect via https, authentication is via ip-address and ssl client certificate.

I've done some testing with one of such a file. Steps with timestamp:

  • umount the davfs2 fs: 2020-05-07 15:14:02
  • do rm -rf /var/cache/davfs2
  • mount: 2020-05-07 15:14:08
  • copy some small file to the destination: 2020-05-07 15:14:13
  • wait approx 15 seconds
  • run test if file exists: 2020-05-07 15:14:30


The accesslog of the webdavserver:


172.16.xx.yy - [07/May/2020:15:14:08 +0200] "OPTIONS / HTTP/1.1" 200 -
172.16.xx.yy - [07/May/2020:15:14:08 +0200] "PROPFIND / HTTP/1.1" 207 3463
172.16.xx.yy - [07/May/2020:15:14:08 +0200] "PROPFIND / HTTP/1.1" 207 307
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "PROPFIND /Java11/ HTTP/1.1" 207 5234
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/ HTTP/1.1" 207 7174
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/bench-output/ HTTP/1.1" 207 7468
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/ HTTP/1.1" 207 26130
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ HTTP/1.1" 207 3367
172.16.xx.yy - [07/May/2020:15:14:13 +0200] "LOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv HTTP/1.1" 200 426
172.16.xx.yy - [07/May/2020:15:14:23 +0200] "PUT /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv HTTP/1.1" 412 348
172.16.xx.yy - [07/May/2020:15:14:23 +0200] "UNLOCK /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/two_consumers_disableDistributedLock.csv HTTP/1.1" 204 -
172.16.xx.yy - [07/May/2020:15:14:30 +0200] "PROPFIND /Java11/Ubuntu_16.04_LTS/bench-output/benchmarks/lfooGridProviderBenchmark/ HTTP/1.1" 207 3367


The debuglog of davfs2 is attached. Both logfiles are somehow pseudonomized.

The http code 412 looks like an old error where if_match_bug comes in, but the apache version is recent enough I think. Could someone help me please?

Best regards,

Hubert


Hubert Krause <hukrause>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49233:  webdav.conf added by hukrause (3KiB - application/octet-stream)
file #49028:  debug20200507151400.log added by hukrause (68KiB - text/x-log)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by _71007 (Posted a comment)
  • -email is unavailable- added by hukrause (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-06-09 hukrause Attached File- Added webdav.conf, #49233
    2020-05-17 _71007 StatusNone In Progress
        Assigned toNone _71007
    2020-05-07 hukrause Attached File- Added debug20200507151400.log, #49028

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.13-397b.
    Corresponding source code