bugConcurrent Versions System - Bugs: bug #14840, zlib.c implementation in 1.12.12+...

 
 

bug #14840: zlib.c implementation in 1.12.12+ still broken

Submitted by:  Rahul Bhargava <b_rahul>
Submitted on:  Sat 22 Oct 2005 01:29:45 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: None
Open/Closed: ClosedRelease: 1.12.13
Fixed Release: NoneFixed Feature Release: 1.12.14

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 28 Oct 2005 03:09:04 PM UTC, comment #8:

I've committed a fix and regression tests for this.

Derek Robert Price <dprice>
Project Administrator
Thu 27 Oct 2005 07:48:05 PM UTC, comment #7:

Hello Derek -

It works!

Your fix is right on. When I looked at the 1.12.13 zlib.c code that line looked suspicious. My testcase (add.c from cvs src tree) is working fine with your fix. Tried all the zlib compression levels (0-9)and saw no problem with it.

Regarding testcase script -

As I noted in the bug report, I see this problem from a Java client. To test this in a scripted fashion, you could download WANdisco CVS replicator for 1 node and run the normal command "cvs -z6 ci -m change add.c" via it. That's the closest you can come to scripting it. We will be happy to provide you with a license. Other not so manual ways would be to download CVS clients like SmartCVS (foundation version is free) or JetBrains' IdeaJ IDE with CVS plugin. We internally have our stress tests that will catch issues like these, but we test only after a release from CVS project is out. So it will be good to have a testcase pre-release. Let me know if we can help with setting up a test scenario.

Best Regards,
Rahul Bhargava,

Rahul Bhargava <b_rahul>
Thu 27 Oct 2005 06:14:35 PM UTC, comment #6:

That last submission was from me. Thought I was already logged in.

Derek Robert Price <dprice>
Project Administrator
Thu 27 Oct 2005 06:13:12 PM UTC, comment #5:

Does the attached patch solve your issue? If so, I'll commit it.

It is interesting that this problem didn't show up in nightly testing. If you could submit a short CVS session in shell script that reproduces the problem prior to applying this patch, that would be useful to prevent this problem from recurring. For example:

cvs -z6 co emptyproject
cd emptyproject
cat <<EOF >>newfile
some content
some more content
EOF
cvs -z6 add newfile
cvs -z6 ci -m 'Force a hang with compression bug.'

Anonymous
Wed 26 Oct 2005 03:43:38 PM UTC, comment #4:

Thanks for re-opening this. Given the problems with src/zlib.c in 1.12.13, could we revert back to the zlib.c in 1.12.12 ? That way while the zlib.c bug is being resolved we will have a feature version that does not have zlib incompatability issue ?

The zlib version can stay at 1.2.3, that way we get the security patches made to zlib 1.2.3 and maintain compatability with other CVS clients.

Regards,
Rahul Bhargava
WANdisco,Inc
http://www.wandisco.com

Rahul Bhargava <b_rahul>
Mon 24 Oct 2005 12:23:14 AM UTC, comment #3:

BTW we also copied the src/zlib.c from 1.12.12 into 1.12.13 tree and re-compiled the cvs binary. With that we are able to use 1.12.13 with zlib 1.2.3 without any problems!

So the bug seems to be in the changes made to zlib.c since 1.12.12.

Rahul Bhargava <b_rahul>
Sun 23 Oct 2005 11:48:18 PM UTC, comment #2:

Further investigation revealed the following -

With cvs 1.12.13 neither external nor bundled zlib 1.2.3 made any difference. In either case the CVS server process is hung in select as it is trying to read more data than what client expects to send as part of the compressed stream.

If you look at the attached trace files, you will see:

1) The ethereal capture shows client sending 8311 bytes to server. The exact same bytes stream from client to a cvs 1.12.12 server (with external zlib 1.2.3) encounters no problem whatsoever, as the server is able to decompress fine. With a cvs 1.12.13 server, the server is trying to read 1 byte, beyond 8311.

====
smp2 /tmp/cvs-serv24028# grep "read(0" /tmp/trace
read(0, "BEGIN AUTH REQUEST\n/tmp/cvsroot\n"..., 4096) = 65
read(0, "Root /tmp/cvsroot\nValid-response"..., 4096) = 396
read(0, "UseUnchanged\nGzip-stream 6\nx\234r\316\317"..., 3700) = 55
read(0, "\304\\{s\333\306\265\377\233\374\24ktl\221\22IIN\322$R"..., 4096) = 4096
read(0, "\356=c\266\23\2118L;\262\332,\"\247\224\226\34\376\323"..., 4096) = 3699

smp2 /tmp/cvs-serv24028# bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
65+396+55+4096+3699
8311
======

If you grep through the strace file, you can see the server's byte accouting is off causing it to get stuck in select. In short the bug seems to be with reading compressed stream off the STDIN not a zlib issue. Please reopen the bug so that we can track it.

Regards,
Rahul Bhargava

Rahul Bhargava <b_rahul>
Sun 23 Oct 2005 06:26:33 PM UTC, comment #1:

You're a little late -- zlib 1.2.3 was imported into the 1.12 tree about a month ago.

Larry Jones <scjones>
Project Administrator
Sat 22 Oct 2005 01:29:45 AM UTC, original submission:

The cvs1.12.12+ tree's zlib.c has compatability problems with
several Java based cvs clients - SmartCVS, WANdisco, JetBrains Ideaj etc. This manifests via cvs commands will hang for some files. See the bundled test file (add.c from cvs src dir)

We have tested with cvs1.11 tree right upto 1.11.21 and has no problems interoperating with the cvs server's zlib implementation.

To further confirm the issue we tried with all the zlib levels and the incompatability remains.

We recompiled with cvs 1.12.12 src with the --with-external-zlib and dynamically linked the default libz.so on Gentoo and Red Hat Linux distros (1.1.4) and that works like a charm. While running configure we noticed :

checking selected ZLIB... external
configure: WARNING: Package ZLIB is more recent than requested external version
(1.2.2 > 1.1.4). configure with the --without-external-zlib
option to select the more recent version.
checking that ZLIB library works... yes

Seems like CVS is using 1.2.2, so we downloaded the latest zlib versions (1.2.3) from http://www.zlib.net/ and tried it with the cvs 1.12.12 src. Again no problem, all works like a charm.

This seems to suggest the 1.2.2 version bundled is broken. It may be a good idea to move to 1.2.3 on the cvs1.12. tree.

Also attached is a testcase tar ball. It includes:
- Ethereal sniff of the client - server traffic when the zlib hang happened
- the src file (add.c) that caused the problem
- The cvs server tmp dir with the partially read (by the cvs pserver process) add.c file
- The repository RCS file
- An strace capture of the cvs pserver process when it got hung sucking in the zlib6 compressed add.c file

Hope this provides enough information to debug 1.2.2 zlib, it might be just simpler to migrate to newer zlib (1.2.3)

Regards,
Rahul Bhargava
WANdisco,Inc

Rahul Bhargava <b_rahul>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #2792:  zlib-read-compressed.diff added by None (634B - application/octet-stream - Possible fix.)
file #2816:  strace-etheral.tbz2 added by b_rahul (13KiB - application/octet-stream)
file #2828:  bug-cvs-zlib.tgz added by b_rahul (25KiB - application/x-compressed - testcase file with description in the bug report)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follow 14 latest changes.

Date Changed By Updated Field Previous Value => Replaced By
Fri 28 Oct 2005 03:09:04 PM UTCdpriceStatusNone=>Fixed
  Open/ClosedOpen=>Closed
  Release=>1.12.13
  Fixed Feature ReleaseNone=>1.12.14
Thu 27 Oct 2005 06:13:12 PM UTCNoneAttached File-=>Added zlib-read-compressed.diff, #3068
Wed 26 Oct 2005 03:13:37 AM UTCscjonesStatusFixed=>None
  Open/ClosedClosed=>Open
  Fixed Feature Release1.12.13=>None
Sun 23 Oct 2005 11:48:18 PM UTCb_rahulAttached File-=>Added strace-etheral.tbz2, #3056
Sun 23 Oct 2005 06:26:33 PM UTCscjonesStatusNone=>Fixed
  Open/ClosedOpen=>Closed
  Fixed Feature ReleaseNone=>1.12.13
Sat 22 Oct 2005 01:29:45 AM UTCb_rahulAttached File-=>Added bug-cvs-zlib.tgz, #3052
  Carbon-Copy-=>Added -unavailable-

Back to the top


Powered by Savane 3.1-cleanup1