bugmonotone - Bugs: bug #28799, mtn pull hangs with invalid...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #28799: mtn pull hangs with invalid --keydir option

Submitter:  Thomas Keller <tommyd>
Submitted:  Wed 03 Feb 2010 12:49:43 PM UTC
   
 
Category:  performance Severity:  2 - Minor
Item Group:  incorrect behavior Status:  Fixed
Privacy:  Public Assigned to:  zzagorski
Open/Closed:  Closed
mtn version --full: 

mtn-0.46, 0.47

Jump to the original submission

Tue 11 May 2010 09:40:42 PM UTC, comment #6: 

committed revision 068f28e6a2b6d700daec031853de0b925cf415f6

(no test as this is only sporadic performance issue)

Zbigniew Zagórski <zzagorski>
Group Member
Fri 30 Apr 2010 06:55:04 AM UTC, comment #5: 

I haven't tried it out myself, but it looks ok. So I'm ok if you commit it - if you just fix these little whitespace issues ;)

Thomas Keller <tommyd>
Group administrator
Thu 29 Apr 2010 10:24:36 PM UTC, comment #4: 

Fix in attachment:

1. read_packets is fast O(n) when reading big files
2. monotone warns about bad keys files in key store (bad key files is one who has no packets inside).

If no one objects i will commit this tommorow.

(file #20376)

Zbigniew Zagórski <zzagorski>
Group Member
Thu 29 Apr 2010 06:14:03 PM UTC, comment #3: 

Agree -> This is not a hangup but monotone is just incredibly slow at reading big packets (is this only here?).

(It just happened that I had big files in my home folder)

--
$ ls -l /home/zbigg/bar/
total 984
-rw-r--r-- 1 zbigg zbigg 1000000 2010-04-29 14:08 one_megabyte_file
$ time mtn --keydir /home/zbigg/bar ls keys
mtn: no keys found

real    0m52.541s
user    0m52.450s
sys     0m0.080s
|i686| zbigg@home:/home/zbigg/bar
--

This is in read_packets(packets.cc) bug which tries to read packet in 255 byte chunks and suffers heavily with "Shlemiel the painter's algorithm" problem (constantly searches buffer for [end] string). Probably some Optimization is needed here. (Wandering if this doesn't affect overall monotone performance)

Zbigniew Zagórski <zzagorski>
Group Member
Thu 29 Apr 2010 05:24:01 PM UTC, comment #2: 

I can't replicate with 0.47:

$ time mtn --keydir $HOME ls keys
mtn: no keys found

real    0m1.712s
user    0m1.704s
sys     0m0.004s

It does seem to take a long time though, especially if I feed it a directory with many files. (It took over 40 seconds when given a dir with 280 object files in it). But it eventually finishes.

Jack Lloyd <randombit>
Group Member
Thu 29 Apr 2010 05:10:08 PM UTC, comment #1: 


Still in 0.47
Additional test:

  mtn --keydir $HOME ls keys

also hangs in same place. Stacktrace is same:

#0  0xb7b77350 in std::string::rfind () from /usr/lib/libstdc++.so.6
#1  0xb7b773c4 in std::string::rfind () from /usr/lib/libstdc++.so.6
#2  0x08201ef6 in read_packets (in=@0xbf91d08c, cons=@0xbf91d154) at packet.cc:385
#3  0x081ed998 in key_store_state::maybe_read_key_dir (this=0x855f060) at key_store.cc:241
#4  0x081f3cc3 in key_store::get_key_ids (this=0xbf91d3a8, priv=@0xbf91d21c) at key_store.cc:248
#

Zbigniew Zagórski <zzagorski>
Group Member
Wed 03 Feb 2010 12:49:43 PM UTC, original submission:  

If mtn pull is given an invalid --keydir option (ie. the dir exists, but contains other files than key files), monotone hangs in packet.cc:385 (endpos = accum.rfind(end);)

Thomas Keller <tommyd>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #20376:  fast_read_packets.patch added by zzagorski (2KiB - application/octet-stream - fix read_packets to be fast, adds warning about bad key files)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by randombit (Posted a comment)
  • -email is unavailable- added by zzagorski (Posted a comment)
  • -email is unavailable- added by tommyd (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-05-11 zzagorski StatusNone Fixed
        Open/ClosedAccepted Closed
    2010-04-29 zzagorski Attached File- Added fast_read_packets.patch, #20376
        Severity3 - Normal 2 - Minor
        Assigned toNone zzagorski
    2010-04-29 zzagorski Categorynetworking performance
        mtn version --fullmtn-0.46 mtn-0.46, 0.47
    2010-04-29 zzagorski Open/ClosedOpen Accepted

    Back to the top

    Powered by Savane 3.13-aa77.
    Corresponding source code