bugConcurrent Versions System - Bugs: bug #14840, zlib.c implementation in 1.12.12+...

 
 

bug #14840: zlib.c implementation in 1.12.12+ still broken

Submitter:  Rahul Bhargava <b_rahul>
Submitted:  Sat 22 Oct 2005 01:29:45 AM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Fixed
Privacy:  Public Assigned to:  None
Open/Closed:  Closed Release:  1.12.13
Fixed Release:  None Fixed Feature Release:  1.12.14
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 28 Oct 2005 03:09:04 PM UTC, comment #8: 

I've committed a fix and regression tests for this.

Derek Robert Price <dprice>
Group administrator
Thu 27 Oct 2005 07:48:05 PM UTC, comment #7: 

Hello Derek -

It works!

Your fix is right on. When I looked at the 1.12.13 zlib.c code that line looked suspicious. My testcase (add.c from cvs src tree) is working fine with your fix. Tried all the zlib compression levels (0-9)and saw no problem with it.

Regarding testcase script -

As I noted in the bug report, I see this problem from a Java client. To test this in a scripted fashion, you could download WANdisco CVS replicator for 1 node and run the normal command "cvs -z6 ci -m change add.c" via it.  That's the closest you can come to scripting it. We will be happy to provide you with a license. Other not so manual ways would be to download CVS clients like SmartCVS (foundation version is free) or JetBrains' IdeaJ IDE with CVS plugin. We internally have our stress tests that will catch issues like these, but we test only after a release from CVS project is out. So it will be good to have a testcase pre-release. Let me know if we can help with setting up a test scenario.

Best Regards,
Rahul Bhargava,


Rahul Bhargava <b_rahul>
Thu 27 Oct 2005 06:14:35 PM UTC, comment #6: 

That last submission was from me.  Thought I was already logged in.

Derek Robert Price <dprice>
Group administrator
Thu 27 Oct 2005 06:13:12 PM UTC, comment #5: 

Does the attached patch solve your issue?  If so, I'll commit it.

It is interesting that this problem didn't show up in nightly testing.  If you could submit a short CVS session in shell script that reproduces the problem prior to applying this patch, that would be useful to prevent this problem from recurring.  For example:

   cvs -z6 co emptyproject
   cd emptyproject
   cat <<EOF >>newfile
   some content
   some more content
   EOF
   cvs -z6 add newfile
   cvs -z6 ci -m 'Force a hang with compression bug.'

Anonymous
Wed 26 Oct 2005 03:43:38 PM UTC, comment #4: 

Thanks for re-opening this. Given the problems with src/zlib.c in 1.12.13, could we revert back to the zlib.c in 1.12.12 ? That way while the zlib.c bug is being resolved we will have a feature version that does not have zlib incompatability issue ?

The zlib version can stay at 1.2.3, that way we get the security patches made to zlib 1.2.3 and maintain compatability with other CVS clients.

Regards,
Rahul Bhargava
WANdisco,Inc
http://www.wandisco.com

Rahul Bhargava <b_rahul>
Mon 24 Oct 2005 12:23:14 AM UTC, comment #3: 

BTW we also copied the src/zlib.c from 1.12.12 into 1.12.13 tree and re-compiled the cvs binary. With that we are able to use 1.12.13 with zlib 1.2.3 without any problems!

So the bug seems to be in the changes made to zlib.c since 1.12.12.

Rahul Bhargava <b_rahul>
Sun 23 Oct 2005 11:48:18 PM UTC, comment #2: 

Further investigation revealed the following -

With cvs 1.12.13 neither external nor bundled zlib 1.2.3 made any difference. In either case the CVS server process is hung in select  as it is trying to read more data than what client expects to send as part of the compressed stream.

If you look at the attached trace files, you will see:

1) The ethereal capture shows client sending 8311 bytes to server. The exact same bytes stream from client to a cvs 1.12.12 server (with external zlib 1.2.3) encounters no problem whatsoever, as the server is able to decompress fine. With a cvs 1.12.13 server, the server is trying to read 1 byte, beyond 8311.

====
smp2 /tmp/cvs-serv24028# grep "read(0" /tmp/trace
read(0, "BEGIN AUTH REQUEST\n/tmp/cvsroot\n"..., 4096) = 65
read(0, "Root /tmp/cvsroot\nValid-response"..., 4096) = 396
read(0, "UseUnchanged\nGzip-stream 6\nx\234r\316\317"..., 3700) = 55
read(0, "\304\\{s\333\306\265\377\233\374\24ktl\221\22IIN\322$R"..., 4096) = 4096
read(0, "\356=c\266\23\2118L;\262\332,\"\247\224\226\34\376\323"..., 4096) = 3699

smp2 /tmp/cvs-serv24028# bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
65+396+55+4096+3699
8311
======

If you grep through the strace file, you can see the server's byte accouting is off causing it to get stuck in select. In short the bug seems to be with reading compressed stream off the STDIN not a zlib issue. Please reopen the bug so that we can track it.

Regards,
Rahul Bhargava


Rahul Bhargava <b_rahul>
Sun 23 Oct 2005 06:26:33 PM UTC, comment #1: 

You're a little late -- zlib 1.2.3 was imported into the 1.12 tree about a month ago.

Larry Jones <scjones>
Group administrator
Sat 22 Oct 2005 01:29:45 AM UTC, original submission:  

The cvs1.12.12+ tree's zlib.c has compatability problems with
several Java based cvs clients - SmartCVS, WANdisco, JetBrains Ideaj etc. This manifests via cvs commands will hang for some files. See the bundled test file (add.c from cvs src dir)

We have tested with cvs1.11 tree right upto 1.11.21 and has no problems interoperating with the cvs server's zlib implementation.

To further confirm the issue we tried with all the zlib levels and the incompatability remains.

We recompiled with cvs 1.12.12 src with the --with-external-zlib and dynamically linked the default libz.so on Gentoo and Red Hat Linux distros (1.1.4) and that works like a charm. While running configure  we noticed :

checking selected ZLIB... external
configure: WARNING: Package ZLIB is more recent than requested external version
           (1.2.2 > 1.1.4).  configure with the --without-external-zlib
           option to select the more recent version.
checking that ZLIB library works... yes


Seems like CVS is using 1.2.2, so we downloaded the latest zlib versions (1.2.3) from  http://www.zlib.net/ and tried it with the cvs 1.12.12 src. Again no problem, all works like a charm.


This seems to suggest the 1.2.2 version bundled is broken. It may be a good idea to move to 1.2.3 on the cvs1.12. tree.

Also attached is a testcase tar ball. It includes:
- Ethereal sniff of the client - server traffic when the zlib hang happened
- the src file (add.c) that caused the problem
- The cvs server tmp dir with the partially read (by the cvs pserver process) add.c file
- The repository RCS file
- An strace capture of the cvs pserver process when it got hung sucking in the zlib6 compressed add.c file

Hope this provides enough information to debug 1.2.2 zlib, it might be just simpler to migrate to newer zlib (1.2.3)

Regards,
Rahul Bhargava
WANdisco,Inc

Rahul Bhargava <b_rahul>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #2792:  zlib-read-compressed.diff added by None (634B - application/octet-stream - Possible fix.)
file #2816:  strace-etheral.tbz2 added by b_rahul (13KiB - application/octet-stream)
file #2828:  bug-cvs-zlib.tgz added by b_rahul (25KiB - application/x-compressed - testcase file with description in the bug report)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only logged-in users can vote.

 

Follow 14 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2005-10-28 dprice StatusNone Fixed
    Open/ClosedOpen Closed
    Release 1.12.13
    Fixed Feature ReleaseNone 1.12.14
2005-10-27 None Attached File- Added zlib-read-compressed.diff, #3068
2005-10-26 scjones StatusFixed None
    Open/ClosedClosed Open
    Fixed Feature Release1.12.13 None
2005-10-23 b_rahul Attached File- Added strace-etheral.tbz2, #3056
2005-10-23 scjones StatusNone Fixed
    Open/ClosedOpen Closed
    Fixed Feature ReleaseNone 1.12.13
2005-10-22 b_rahul Attached File- Added bug-cvs-zlib.tgz, #3052
    Carbon-Copy- Added -email is unavailable-

Back to the top

Powered by Savane 3.13-cf05.
Corresponding source code