mainSavannah Administration - Support: sr #110546, git clone...

 
 

sr #110546: git clone https://git.savannah.gnu.org/... fails with error "server certificate verification failed"

Submitter:  Bruno Haible <haible>
Submitted:  Sat 02 Oct 2021 05:07:01 PM UTC
   
 
Category:  Source code repositories - anonymous access Priority:  5 - Normal
Severity:  4 - Important Status:  Done
Privacy:  Public Assigned to:  rwp
Operating System:  GNU/Linux Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 04 Oct 2021 01:08:50 AM UTC, comment #11: 


> the fix is simply to install security patch updates as per standard best practice.


Absolutely. On systems where the administrator can either

  • install only security patches, or
  • install all package updates, without compromising the purpose of the system,

this is the recommended action.

Bruno Haible <haible>
Mon 04 Oct 2021 12:19:29 AM UTC, comment #10: 

On the freshly installed pristine original 2016 Ubuntu 16.04 system I found that upgrading to a current ca-certificates package plus removing the DST Root CA X3 certificate from the trust anchor was sufficient.

apt-get install ca-certificates
sed --in-place '/DST_Root_CA_X3.crt/s/^/!/' /etc/ca-certificates.conf
update-ca-certificates

And then everything works.

Bob Proulx <rwp>
Site Administrator
Mon 04 Oct 2021 12:11:06 AM UTC, comment #9: 

That is an impressive tabulation of systems! However you included fixes for only three.  And for at least several of those that I spot-check the fix is simply to install security patch updates as per standard best practice.  I imagine that most of those would be fixed simply by installing the latest official security patches for each system.  For one example I am routinely using Debian 9 and Debian 10 systems and cannot reproduce the problem on maintained systems.

And then for example I just now as a test installed an ISO of ubuntu-16.04.4-desktop-i386.iso from 2018 which I had available and by 2018 it had already been patched for this issue.  And having been patched I could not reproduce the problem.

Being curious I then downloaded and installed a pristine Ubuntu 16.04-server (no point releases, the original release) from 2016 and did reproduce the problem at that vintage.  Without exhaustively going through the releases this says that the issue was fixed before 2018.

I am as much of a fan of having a large matrix of systems and versions as anyone.  But any system that interacts with the hostile Internet will find that the hostile Internet is always changing.  Whether we like it or not we find it impossible to freeze those parts that communicate and interact.  Not because the local end has changed but because the remote end of the communicating pair has changed.  Sometimes whether we like it or not we simply must upgrade the local part of our client in order to communicate.

As to the other two pages you asked to have hints placed for documentation for the /tls page I obsoleted it entirely since it was truly so old it was harmfully misinformed.  I redirected that page to the other one.  The other one was originally just a few lines.  I expanded it with a first pass description of everything I know about things now and various hints.

https://savannah.gnu.org/maintenance/SavannahTLSInfo/

Bob Proulx <rwp>
Site Administrator
Sun 03 Oct 2021 09:45:10 PM UTC, comment #8: 

More info about specific operating systems / distributions (in their original state, NOT fully patched):

Affected:

Debian 9.3
Debian 9.6
Debian 10.7
Ubuntu 16.04
Ubuntu 18.04  (fix: sudo apt install ca-certificates)
Ubuntu 20.04  (fix: sudo apt install ca-certificates)
ElementaryOS 5.0  (fix: sudo apt install ca-certificates)
Fedora 13
CentOS 7.3
Guix 1.0
Guix 1.1
macOS 10.13.4
NetBSD 8.0
NetBSD 9.0
Solaris 11.3
Solaris 11.4
Cygwin 2.9.0

Not affected:

Ubuntu 20.10
Fedora 27
Fedora 29
Fedora 30
Fedora 31
Fedora 32
Fedora 33
CentOS 7.8
CentOS 8.0
Manjaro 17
OpenSUSE 15.2
Alpine Linux 3.9
macOS 11.3
FreeBSD 12.0
FreeBSD 13.0
OpenBSD 6.8
OpenBSD 6.9
Solaris 11 OmniOS 2020
AIX 7.1

Bruno Haible <haible>
Sun 03 Oct 2021 11:38:10 AM UTC, comment #7: 


> I am surprised that you found Ubuntu 16.04 and Ubuntu 18.04 affected since as I understood it those were still under LTS support and I presume would have updates available to them.


I tested Ubuntu 16.04 and Ubuntu 18.04 in their original state, not fully patched.
1) because the Docker images for these OSes, used for continuous integration builds, are not fully patched.
2) because also some desktop users prefer to not fully patch their systems.

The simple commands mentioned in https://stackoverflow.com/questions/21181231/

sudo apt-get install -y apt-transport-https ca-certificates
sudo update-ca-certificates

work on Ubuntu 18.04; they upgrade the package 'ca-certificates' to version 20210119~18.04.2, and this fixes the issue.

However, on Ubuntu 16.04 in its original state they don't help.

Bruno Haible <haible>
Sun 03 Oct 2021 04:59:43 AM UTC, comment #6: 

Your reference to https://savannah.gnu.org/tls/ is interesting since I was completely unaware of that page before, it is being maintained outside the wiki page area, and it is completely outdated now.

I guess a first step is to obsolete that page entirely.

Bob Proulx <rwp>
Site Administrator
Sun 03 Oct 2021 02:08:15 AM UTC, comment #5: 

As you have deduced this is a client side problem.  It is one that has perhaps caught a large number of people operating older client systems unawares however.  I'll post something to the web UI front page.  And then go look at the pages you referenced and add hints there too.

I am surprised that you found Ubuntu 16.04 and Ubuntu 18.04 affected since as I understood it those were still under LTS support and I presume would have updates available to them.  Debian has posted patches for their LTS suites.  I looked at Debian 9 "Stretch", which is now N-2 releases old, and found it has been LTS patched.

References to Debian patches.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961889
https://lists.debian.org/debian-lts/2021/09/msg00008.html


Bob Proulx <rwp>
Site Administrator
Sat 02 Oct 2021 05:22:54 PM UTC, comment #4: 

It seems to be a client-side problem, per https://docs.certifytheweb.com/docs/kb/kb-202109-letsencrypt/ which says "If your site is working for most devices but not for some, the problem is with their trust store".

Nonetheless, maybe a hint could be given in https://savannah.gnu.org/maintenance/SavannahTLSInfo/ or https://savannah.gnu.org/tls/ ?

Bruno Haible <haible>
Sat 02 Oct 2021 05:16:36 PM UTC, comment #3: 

Regarding client-side fixes:

https://stackoverflow.com/questions/21181231/ gives a number of workarounds, but some of them have negative security implications. For example,

$ GIT_SSL_NO_VERIFY=1 git clone https://git.savannah.gnu.org/git/gnulib.git

works, but disable SSL/TLS checking and thus opens the door to DNS spoofing and man-in-the-middle attacks.

Bruno Haible <haible>
Sat 02 Oct 2021 05:11:41 PM UTC, comment #2: 

Inspecting the SSL certificates of git.savannah.gnu.org through https://www.ssllabs.com/ssltest/ reveals that the problem is that a root certificate "DST Root CA X3" expired on 2021-09-30.

This web page https://docs.certifytheweb.com/docs/kb/kb-202109-letsencrypt/ says:
"From Sept 30th 2021 Let's Encrypts previous root certificate DST Root CA X3 (and it's R3 intermediate) will expire. It has been replaced by their ISRG Root X1 certificate (and replacement R3 intermediate)."

"In some cases, the expiry of the root (and its related expiring R3 intermediate certificate) may causes certificates to be considered untrusted or invalid. To fix this you need to make your server use (serve) the correct chain.

In other cases, the issue may be with the client computer."

Bruno Haible <haible>
Sat 02 Oct 2021 05:08:05 PM UTC, comment #1: 

It depends on the operating system.

Affected: Ubuntu 16.04 and 18.04.

Not affected: Ubuntu 20.04

Bruno Haible <haible>
Sat 02 Oct 2021 05:07:01 PM UTC, original submission:  

Since 2021-10-01, cloning repositories from git.savannah.gnu.org fails on some operating systems. Example:

$ git clone https://git.savannah.gnu.org/git/gnulib.git
Cloning into 'gnulib'...
fatal: unable to access 'https://git.savannah.gnu.org/git/gnulib.git': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none


Bruno Haible <haible>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rwp (Posted a comment)
  • -email is unavailable- added by haible (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-10-04 rwp StatusNone Done
        Assigned toNone rwp
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-0159.
    Corresponding source code