Urllib and "Ssl: Certificate_verify_failed" Error

The ssl: certificate_verify_failed error when using Python's urllib library indicates that the SSL certificate of the server you're trying to connect to cannot be verified. This typically happens when the server's certificate chain cannot be validated against the trusted Certificate Authorities (CAs) on your system. Here’s how you can address this issue:

1. Update CA Certificates

Ensure that the CA certificates on your system are up-to-date. This helps urllib to properly validate SSL certificates.

• On Windows:
  • Update your system through Windows Update to ensure you have the latest CA certificates.
  • Alternatively, you can install the certifi package which provides an up-to-date CA bundle.

• On Linux:

  • Debian/Ubuntu:

     

    Copied!

    sudo apt-get update
    sudo apt-get install --reinstall ca-certificates
    

  • Fedora/CentOS/RHEL:

     

    Copied!

    sudo yum reinstall ca-certificates
    

  • Arch Linux:

     

    Copied!

    sudo pacman -Syu ca-certificates
    

• On macOS:

  • Homebrew:

     

    Copied!

    brew install openssl
    

2. Use the certifi Package

You can use the certifi package to provide a reliable CA bundle.

1. Install certifi:

    

   Copied!

   pip install certifi
   

2. Use certifi with urllib:

    

   Copied!

   import urllib.request
   import certifi
   import ssl
   
   # Create an SSL context with certifi's CA bundle
   ssl_context = ssl.create_default_context(cafile=certifi.where())
   
   # Use the custom SSL context with urllib
   response = urllib.request.urlopen('<https://example.com>', context=ssl_context)
   print(response.read())
   

3. Manually Specify the CA Bundle

If you have a custom CA bundle, you can specify it manually in your urllib code.

1. Download CA Certificates:
   • Download a CA bundle, such as from certifi or curl's website.

2. Use the CA Bundle in urllib:

    

   Copied!

   import urllib.request
   import ssl
   
   # Path to your custom CA bundle
   ca_bundle_path = '/path/to/your/cacert.pem'
   
   # Create an SSL context with your CA bundle
   ssl_context = ssl.create_default_context(cafile=ca_bundle_path)
   
   # Use the custom SSL context with urllib
   response = urllib.request.urlopen('<https://example.com>', context=ssl_context)
   print(response.read())
   

4. Disable SSL Verification (Not Recommended)

For testing purposes or in environments where security is not a concern, you can disable SSL certificate verification. However, this is not recommended for production environments due to security risks.

import urllib.request
import ssl

# Create an SSL context that does not verify certificates
ssl_context = ssl._create_unverified_context()

# Use the SSL context with urllib
response = urllib.request.urlopen('<https://example.com>', context=ssl_context)
print(response.read())

5. Check System Time

SSL/TLS certificates are time-sensitive. Incorrect system time can cause certificate verification failures.

• On Windows:
  • Check and synchronize your system clock through Date and Time settings.

• On Linux/macOS:

  • Synchronize your system time using NTP:

     

    Copied!

    sudo ntpdate -u time.nist.gov
    

6. Verify Server Certificate Chain

Ensure that the server you are connecting to is properly configured with a complete certificate chain, including intermediate certificates.

• Using openssl to check the server certificate:

   

  Copied!

  openssl s_client -connect example.com:443 -showcerts
  

• Verify that the server presents a complete certificate chain.

Summary

The ssl: certificate_verify_failed error typically occurs due to issues with CA certificates or SSL configuration. To resolve it:

1. Update CA certificates on your system.
2. Use the certifi package for an up-to-date CA bundle.
3. Manually specify a CA bundle if needed.
4. Temporarily disable SSL verification (not recommended for production).
5. Ensure system time is correct.
6. Verify the server's certificate chain.

Implementing these steps will help you address SSL verification issues with Python's urllib library.

Got an article suggestion? Let us know

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.