A story Link to heading

just a couple of days ago, I noticed that my pip in wsl was timing out. some very specific error message would be

Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(SSLError(“bad handshake: Error([(‘SSL routines’, ’tls_process_server_certificate’, ‘certificate verify failed’)])”))’: /packages/15/8e/4fbc92846184e1080af77da38d55928a5486e0bc5e2ec8342c7db378d7f1/pip_install_test-0.5-py3-none-any.whl

have tried pip install --trusted-host ..., it didn’t work.

a time really don’t know what’s going on Link to heading

searched on internet, tried recipe that people wrote, didn’t work and asked for help from my teammate.

to find out what cert it used to verify Link to heading

my teammate suggested me run openssl s_client -connect files.pythonhosted.org:443

and

  1. found that my browser was using GlobalSign_Root_CA.pem
  2. found that my wsl was using Cisco Umbrella Root CA

lol, my teammate was shocked because this is the first time he saw same source using two different certifications to verify.

the solution should be simple tho, just the awesome commands (call it cmdA)

sudo wget -P /usr/local/share/ca-certificates/ -O Cisco_Umbrella_Root_CA.crt https://<don't_know_if_this_will_reveal_any_secret>/Cisco_Umbrella_Root_CA.cer
sudo update-ca-certificates

would solve issue if you have your standard python installed, however this doesn’t work for me still because

damn anaconda! Link to heading

not saying anaconda is bad, in contrast it is great, but in this scenario, damn! let me tell you.

after I ran cmdA, I find I still cannot do pip install <any_package>, why…why…why I asked me several times.

then I asked another teammate who is in security team

he let me try curl -v https://internetbadguys.com to tell him what was on the screen.

surprisingly, I found two line said

* successfully set certificate verify locations:
*   CAfile: /home/<myusername>/anaconda3/ssl/cacert.pem

yea, we could yell out together since I installed Anaconda when I installed wsl, and I always using base environment, and I never noticed that anaconda will have it’s own wrapper of certs!

but up until now, it’s clear that I need to tell anaconda to use my system cert at least.

the way solving it Link to heading

first, I should still run cmdA first to install the cert.

then he found this

basically you should run

pip config set global.cert path/to/ca-bundle.crt
pip config list
conda config --set ssl_verify path/to/ca-bundle.crt
conda config --show ssl_verify

path/to/ca-bundle.crt for my case is /etc/ssl/certs/ca-certificates.crt

after I ran, I see config list has a global.cert and conda config ssl verify has the cert as well. looking good now.

further trivial problems Link to heading

I thought I solved this issue, but when goes to docker container, it still don’t let me pip install any_package, because I was chose to trust cisco umbrella cert to verify.

so for ubuntu, just write cmdA to the Dockerfile.

because we were knowing little for alpine, just wrote

curl https://path/to/cert/Cisco_Umbrella_Root_CA.cer >> /etc/ssl/certs/ca-certificates.crt

what worries me more is that if pythonhosted.org chose my companies production environment to do umberlla cert verification, that would be a nightmare for me.

if that happens, I will discuss that issue in another post.