mainSavannah Administration - Support: sr #109651, Git checking is slow - DNS timeout?

 
 

sr #109651: Git checking is slow - DNS timeout?

Submitter:  Ole Tange <tange>
Submitted:  Mon 04 Mar 2019 11:08:59 PM UTC
   
 
Category:  Source code repositories - developer access Priority:  5 - Normal
Severity:  2 - Minor Status:  Done
Privacy:  Public Assigned to:  None
Operating System:  GNU/Linux Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 08 Jan 2021 01:27:35 AM UTC, comment #10: 

Boo that the root cause of the problem was never determined.  Yay that the problem seems not to be there anymore!

Bob Proulx <rwp>
Site Administrator
Thu 07 Jan 2021 07:26:03 PM UTC, comment #9: 

I can no longer reproduce the error.

Let us close the issue.

Ole Tange <tange>
Mon 27 May 2019 08:04:08 PM UTC, comment #8: 

It is pretty clear it is not a DNS problem in my end.

The attached logs from ssh and tcpdump (run at the same time) show that the connection is established immediately.



(file #46981, file #46982)

Ole Tange <tange>
Sun 21 Apr 2019 01:21:36 PM UTC, comment #7: 

So the problem also exists in ssh:

stdout ssh -vvvv 209.51.188.201 echo | timestamp --delta > ssh-vvvv-log.txt

It is actually faster for me to run `git` through `torsocks`:

    $ time git pull
    real    0m32.518s
    user    0m0.060s
    sys     0m0.036s

    $ time torsocks git pull
    real    0m2.865s
    user    0m0.057s
    sys     0m0.043s

And I can live with this being a bit sluggish.

Again it brings us back to something on the server's SSH config really does not like my IP-address.



(file #46806)

Ole Tange <tange>
Sun 31 Mar 2019 06:49:39 PM UTC, comment #6: 

Yay for LibrePlanet, vacation travel, catching a cold, getting better. :-) All taking some time.  But not like the time it is taking for your git push to complete. :-(

Looking at your strace it unfortunately only shows me where at three different occurrences there looks to be a 10 second timeout in a select() call.

select(4, [3], NULL, NULL, NULL) = 1 (in [3])

I couldn't tell where that select() call was coming from.

It is good that you have two different cases, one fast and one slow, to compare.  However I think that since you are the only one able to observe your slow case that the burden will be yours to try to debug why it is the slow case.

I would look at DNS on your side.  Look at /etc/nsswitch.conf for the hosts line.  Likely there is some other modules installed there.  If so then I suggest trying a debug case with them reduced to the known good set here.

hosts: files dns

Also I would verify that /etc/resolv.conf has a good configuration.  Because if there are several nameservers listed there then the first one may be timing out and falling back to a secondary one.  That may be accounting for the timeouts you are seeing.  Again creating a debug case that avoids the dns lookup by setting a temporary IP in /etc/hosts or otherwise ensuring a safe test case may help debug the problem.


Bob Proulx <rwp>
Site Administrator
Sun 17 Mar 2019 11:02:09 PM UTC, comment #5: 

Great. Not DNS problem then.

Just tested if it gave a difference through if I use internet connection through the phone (5.33.14.209).

  5.33.14.209$ time git pull
  Already up to date.

  real    0m8.525s
  user    0m0.083s
  sys     0m0.030s

  89.23.224.57$ time git pull
  Already up to date.

  real    0m31.837s
  user    0m0.064s
  sys     0m0.019s

The same result whether I used wireless connection or wired.

So it does seem it has to do with the internet connection.

See attached strace.txt.

Ole Tange <tange>
Tue 12 Mar 2019 03:48:09 AM UTC, comment #4: 

FWIW, on vcs0 the git frontend system:

root@vcs0:~# time host 89.23.224.57
57.224.23.89.in-addr.arpa domain name pointer 3224057-cl69.boa.fiberby.dk.

real    0m0.027s
user    0m0.006s
sys     0m0.022s

Ensuring that the DNS cache is cleared and then:

root@vcs0:~# time dig -x 89.23.224.57 +short
3224057-cl69.boa.fiberby.dk.

real    0m0.880s
user    0m0.012s
sys     0m0.012s

Bob Proulx <rwp>
Site Administrator
Tue 12 Mar 2019 03:40:43 AM UTC, comment #3: 

Am I missing seeing an attached file?

Bob Proulx <rwp>
Site Administrator
Sun 10 Mar 2019 08:02:19 PM UTC, comment #2: 

IPv4 only.

Here is an "strace -tt -ff git pull" anonymized (strings replaced).

Ole Tange <tange>
Wed 06 Mar 2019 06:13:15 PM UTC, comment #1: 

I cannot reproduce this long timeout at this time.  All of my times are in the range between 5 and 15 seconds with most between 5-7 seconds.  I am testing from Colorado to Boston.

I happen to be on an IPv4 only network.  Are you using IPv4 or IPv6 networking to reach git.savannah.gnu.org ?  (IPv6 should be working too.  But just trying to narrow down things somewhat.)

Bob Proulx <rwp>
Site Administrator
Mon 04 Mar 2019 11:08:59 PM UTC, original submission:  

$ time git push

real    0m31.823s
user    0m0.036s
sys     0m0.011s

Since it does complete it smell like reverse lookup failing - maybe the primary nameserver used by git.sv.gnu.org is down?

I am sitting on: 89.23.224.57 which has in-addr.arpa: 3224057-cl69.boa.fiberby.dk.

Ole Tange <tange>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #46981:  ssh.log added by tange (16KiB - text/x-log)
file #46982:  tcpdump.log added by tange (8KiB - text/x-log)
file #46806:  ssh-vvvv-log.txt added by tange (15KiB - text/plain)
file #46568:  strace.txt added by tange (331KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rwp (Posted a comment)
  • -email is unavailable- added by tange (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-01-08 rwp StatusNone Done
        Open/ClosedOpen Closed
    2019-05-27 tange Attached File- Added ssh.log, #46981
        Attached File- Added tcpdump.log, #46982
    2019-04-21 tange Attached File- Added ssh-vvvv-log.txt, #46806
    2019-03-17 tange Attached File- Added strace.txt, #46568

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code