bugmonotone - Bugs: bug #28799, mtn pull hangs with invalid...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #28799: mtn pull hangs with invalid --keydir option

Submitted by:  Thomas Keller <tommyd>
Submitted on:  Wed 03 Feb 2010 12:49:43 PM UTC  
 
Category: performanceSeverity: 2 - Minor
Item Group: incorrect behaviorStatus: Fixed
Privacy: PublicAssigned to: Zbigniew Zagórski <zzagorski>
Open/Closed: Closed
mtn version --full: 

mtn-0.46, 0.47

(Jump to the original submission Jump to the original submission)

Tue 11 May 2010 09:40:42 PM UTC, comment #6:

committed revision 068f28e6a2b6d700daec031853de0b925cf415f6

(no test as this is only sporadic performance issue)

Zbigniew Zagórski <zzagorski>
Project MemberIn charge of this item.
Fri 30 Apr 2010 06:55:04 AM UTC, comment #5:

I haven't tried it out myself, but it looks ok. So I'm ok if you commit it - if you just fix these little whitespace issues ;)

Thomas Keller <tommyd>
Project Administrator
Thu 29 Apr 2010 10:24:36 PM UTC, comment #4:

Fix in attachment:

1. read_packets is fast O(n) when reading big files
2. monotone warns about bad keys files in key store (bad key files is one who has no packets inside).

If no one objects i will commit this tommorow.

(file #20376)

Zbigniew Zagórski <zzagorski>
Project MemberIn charge of this item.
Thu 29 Apr 2010 06:14:03 PM UTC, comment #3:

Agree -> This is not a hangup but monotone is just incredibly slow at reading big packets (is this only here?).

(It just happened that I had big files in my home folder)

--
$ ls -l /home/zbigg/bar/
total 984
-rw-r--r-- 1 zbigg zbigg 1000000 2010-04-29 14:08 one_megabyte_file
$ time mtn --keydir /home/zbigg/bar ls keys
mtn: no keys found

real 0m52.541s
user 0m52.450s
sys 0m0.080s
|i686| zbigg@home:/home/zbigg/bar
--

This is in read_packets(packets.cc) bug which tries to read packet in 255 byte chunks and suffers heavily with "Shlemiel the painter's algorithm" problem (constantly searches buffer for [end] string). Probably some Optimization is needed here. (Wandering if this doesn't affect overall monotone performance)

Zbigniew Zagórski <zzagorski>
Project MemberIn charge of this item.
Thu 29 Apr 2010 05:24:01 PM UTC, comment #2:

I can't replicate with 0.47:

$ time mtn --keydir $HOME ls keys
mtn: no keys found

real 0m1.712s
user 0m1.704s
sys 0m0.004s

It does seem to take a long time though, especially if I feed it a directory with many files. (It took over 40 seconds when given a dir with 280 object files in it). But it eventually finishes.

Jack Lloyd <randombit>
Project Member
Thu 29 Apr 2010 05:10:08 PM UTC, comment #1:

Still in 0.47
Additional test:

mtn --keydir $HOME ls keys

also hangs in same place. Stacktrace is same:

#0 0xb7b77350 in std::string::rfind () from /usr/lib/libstdc++.so.6
#1 0xb7b773c4 in std::string::rfind () from /usr/lib/libstdc++.so.6
#2 0x08201ef6 in read_packets (in=@0xbf91d08c, cons=@0xbf91d154) at packet.cc:385
#3 0x081ed998 in key_store_state::maybe_read_key_dir (this=0x855f060) at key_store.cc:241
#4 0x081f3cc3 in key_store::get_key_ids (this=0xbf91d3a8, priv=@0xbf91d21c) at key_store.cc:248
#

Zbigniew Zagórski <zzagorski>
Project MemberIn charge of this item.
Wed 03 Feb 2010 12:49:43 PM UTC, original submission:

If mtn pull is given an invalid --keydir option (ie. the dir exists, but contains other files than key files), monotone hangs in packet.cc:385 (endpos = accum.rfind(end);)

Thomas Keller <tommyd>
Project Administrator

 

Attached Files
file #20376:  fast_read_packets.patch added by zzagorski (2KiB - application/octet-stream - fix read_packets to be fast, adds warning about bad key files)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by randombit (Posted a comment)
  • -unavailable- added by zzagorski (Posted a comment)
  • -unavailable- added by tommyd (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 11 May 2010 09:40:42 PM UTCzzagorskiStatusNone=>Fixed
      Open/ClosedAccepted=>Closed
    Thu 29 Apr 2010 10:24:36 PM UTCzzagorskiAttached File-=>Added fast_read_packets.patch, #20376
      Severity3 - Normal=>2 - Minor
      Assigned toNone=>zzagorski
    Thu 29 Apr 2010 06:14:03 PM UTCzzagorskiCategorynetworking=>performance
      mtn version --fullmtn-0.46=>mtn-0.46, 0.47
    Thu 29 Apr 2010 05:10:08 PM UTCzzagorskiOpen/ClosedOpen=>Accepted

    Back to the top


    Powered by Savane 3.1-cleanup1