Thu 18 Sep 2008 09:40:47 PM UTC, comment #7:
I was told about a tool called "strace", so I was playing around with this a bit and found something interesting.
There was this other system I tested with:
Ubuntu Linux, 2.6.24-19 kernel
cvs 1.12.13 and openssh 4.7p1
I could not reproduce the bug on that system before BUT:
When the machine is on high load (like compiling cvs), the bug also occurs at this machine.
I did a strace while idle (strace_ubuntu_idle)
and one with load, when the bug occurs (strace_ubuntu_load)
characteristic lines in idle:
select(6, [5], NULL, NULL, NULL) = 1 (in [5])
--- SIGINT (Interrupt) @ 0 (0) ---
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [INT], 8) = 0
write(2, "cvs [update aborted]: received i"..., 48) = 48
close(1) = 0
exit_group(1) = ?
while the first mention line has an "ERESTARTNOHAND" error befor the sigint:
select(6, [5], NULL, NULL, NULL) = ? ERESTARTNOHAND (To be restarted)
--- SIGINT (Interrupt) @ 0 (0) ---
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [INT], 8) = 0
write(2, "cvs [update aborted]: received i"..., 48) = 48
close(1) = 0
exit_group(1) = ?
I also did a strace of my system, which is still:
Arch Linux, kernel 2.26.5, on a PIII800Mhz
cvs 1.11.23
last part of strace_arch_piii800:
read(5, 0xb7f6e000, 4096) = ? ERESTARTSYS (To be restarted)
--- SIGINT (Interrupt) @ 0 (0) ---
write(2, "cvs", 3) = 3
write(2, " ", 1) = 1
write(2, "[", 1) = 1
write(2, "update", 6) = 6
write(2, " aborted]", 9) = 9
write(2, ": ", 2) = 2
write(2, "received ", 9) = 9
write(2, "interrupt", 9) = 9
write(2, " signal", 7) = 7
write(2, "\n", 1) = 1
exit_group(1) = ?
So I don't see much in the strace, but I suspect that this is a timeout issue, which occurs when not much computing power is available. Like it is the case while on 100% usage or just having an PIII 800 MHz?
I attached the 2 mentioned straces.
(file #16521, file #16522, file #16523)
|
Mon 15 Sep 2008 08:26:06 PM UTC, comment #5:
I checked ps. I can find entries for ssh when I run "cvs update", but they vanish right after I interrupt.
I also checked an ubuntu machine with cvs 1.12.13 and there was no problem with echo. However, this is not about the version. Installing cvs 1.12.13 on my system didn't help.
So this is something somewhat system dependent.
I don't know if it helps or not, but:
(I am testing with a local test repo now)
$ cvs -t update
-> main loop with CVSROOT=:ext:localhost:/home/jonnyjd/cvsroot
-> Starting server: ssh localhost cvs server
jonnyjd@localhost's password: cvs [update aborted]: received interrupt signal
-> Lock_Cleanup()
cvs [update aborted]: received interrupt signal
-> Lock_Cleanup()
$gdb --args cvs update
...
0xb7fd4424 in __kernel_vsyscall ()
cvs [update aborted]: received interrupt signal
(gdb) bt
#0 0xb7fd4424 in __kernel_vsyscall ()
#1 0xb7c66073 in __read_nocancel () from /lib/libc.so.6
#2 0xb7c0f8de in _IO_new_file_underflow () from /lib/libc.so.6
#3 0xb7c1128b in _IO_default_uflow_internal () from /lib/libc.so.6
#4 0xb7c12698 in __uflow () from /lib/libc.so.6
#5 0xb7c0c1bf in getc () from /lib/libc.so.6
#6 0x080503ea in stdio_buffer_input ()
#7 0x0805095f in buf_read_line ()
#8 0x0805406d in read_line ()
#9 0x08054528 in get_server_responses ()
#10 0x080594b4 in start_server ()
#11 0x080a0df3 in update ()
#12 0x08076a4b in main ()
gdb catches the interrupt and the echo problem doesn't occur, while running with gdb, by the way.
I compiled a package from source myself, so I wouldn't have a problem testing a patched version with some debug info or similar on my system.
I am not really experienced with interrupts/signals, though.
|
Mon 15 Sep 2008 06:41:25 PM UTC, comment #4:
The enter-thing might make sense, since CVS as the parent process gets the first chance to handle the interrupt and its interrupt handler exits immediately, as opposed to SSH's, which I presume allows the handling of ^C as input.
Anyhow, you didn't tell me if SSH is still running in the background after you interrupt CVS? (`ps -eaf |grep ssh') My current hypothesis is that somehow the SSH interrupt handler isn't getting called on your system and that on other systems we are relying on either automatic pass-through of the SIGINT after CVS's handler completes or automatic delivery of SIGKILL to children when the parent process dies. Then SSH's handler can do the correct terminal cleanup.
If the SSH process started by your CVS is still running, then that would verify this hypothesis. Of course, your SSH could still die without being given a chance to process any signals. Not sure how to test that, except to install a new cleanup handler that delivers a SIGKILL to the child and then waits for it (with some sort of timeout) before dying.
|
Mon 15 Sep 2008 05:17:17 PM UTC, comment #3:
Since you told me to check careful, I installed ssh2 from ssh.com:
When I connect ssh-only, I have to press enter after my ^C, but the echo problem doesn't occur.
When I set CVS_RSH=ssh2 I get a different password prompt (so I am sure it is really using ssh2) but otherwise the same problem. I might note, that I cvs interrupts right after STRG+C, different to having to press enter again, when I run ssh2-only.
So I am quite sure it is not a problem on ssh/ssh2 side, but of course, I can't be sure that it isn't something else on my system. Some console library cvs is using, i.e.
I have heimdal 1.2, but this shouldn't be affected, since ssh2 is not using this, refering to my package dependencies and ldd on ssh2 (no libheim). Anything else to check?
|