Configuring Python Requests to Trust Self-Signed SSL Certificates: Methods and Best Practices

Fundamentals of SSL Certificate Verification

In HTTPS communication, SSL/TLS certificates are used to verify server identity and establish encrypted connections. When using self-signed certificates, since they are not issued by trusted Certificate Authorities (CAs), Python Requests library by default rejects connection establishment and throws requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed error.

Core Solution: verify Parameter Configuration

Requests library provides flexible SSL certificate verification mechanisms, primarily controlled through the verify parameter. Here are three main configuration approaches:

Method 1: Specify Certificate File Path

This is the most direct and recommended method, by passing the path of the self-signed certificate to the verify parameter:

import requests

data = {'foo': 'bar'}
url = 'https://foo.com/bar'

# Specify self-signed certificate path
r = requests.post(url, data=data, verify='/path/to/public_key.pem')

This method explicitly specifies the trusted certificate source, ensuring security while avoiding the risks of completely disabling certificate verification.

Method 2: Environment Variable Configuration

For scenarios requiring global certificate trust configuration, the REQUESTS_CA_BUNDLE environment variable can be set:

# Set in command line
export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem
python script.py

# Permanently add to bash configuration file
echo "export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem" >> ~/.bash_profile
source ~/.bash_profile

This approach is suitable when multiple scripts need to use the same certificate trust configuration.

Method 3: Custom CA Certificate Bundle

Requests library supports using custom CA certificate bundle files:

requests.get(url, verify=path_to_bundle_file)

The certificate bundle file can contain multiple trusted CA certificates, providing more flexible trust management.

Self-Signed Certificate Generation and Configuration

Understanding the self-signed certificate generation process helps in better SSL connection configuration. Here's a complete example of generating self-signed certificates using cryptography library:

from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
import datetime
import ipaddress

def create_self_signed_cert(cert_file, key_file, ip_address):
    # Generate RSA private key
    key = rsa.generate_private_key(public_exponent=65537, key_size=4096)
    
    # Set certificate subject and issuer
    subject = issuer = x509.Name([
        x509.NameAttribute(NameOID.COMMON_NAME, ip_address),
        x509.NameAttribute(NameOID.ORGANIZATION_NAME, "Some Name"),
    ])
    
    # Build certificate
    cert_builder = x509.CertificateBuilder().subject_name(
        subject
    ).issuer_name(
        issuer
    ).public_key(
        key.public_key()
    ).serial_number(
        x509.random_serial_number()
    ).not_valid_before(
        datetime.datetime.utcnow()
    ).not_valid_after(
        datetime.datetime.utcnow() + datetime.timedelta(days=365)
    )
    
    # Add extension fields
    cert_builder = cert_builder.add_extension(
        x509.SubjectAlternativeName([
            x509.DNSName(ip_address),
            x509.IPAddress(ipaddress.ip_address(ip_address))
        ]),
        critical=False
    )
    
    cert_builder = cert_builder.add_extension(
        x509.BasicConstraints(ca=True, path_length=None),
        critical=True
    )
    
    # Sign and save certificate
    cert = cert_builder.sign(private_key=key, algorithm=hashes.SHA512())
    
    with open(cert_file, "wb") as f:
        f.write(cert.public_bytes(serialization.Encoding.PEM))
    
    with open(key_file, "wb") as f:
        f.write(key.private_bytes(
            encoding=serialization.Encoding.PEM,
            format=serialization.PrivateFormat.TraditionalOpenSSL,
            encryption_algorithm=serialization.NoEncryption()
        ))

# Usage example
create_self_signed_cert("selfsigned.crt", "private.key", "192.168.1.172")

Practical Application Scenarios and Best Practices

In development environments, self-signed certificates are commonly used in the following scenarios:

Local Development Server

When building local HTTPS servers using frameworks like Flask, self-signed certificates can be used for testing:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello_world():
    return "<p>Hello, World!</p>"

# Start server with SSL
app.run(host="192.168.1.172", port=10330, ssl_context=("selfsigned.crt", "private.key"))

Client Request Configuration

Clients need to use the generated certificates for secure connections:

import requests

files = {"file": b"example"}
response = requests.post("https://192.168.1.172:10330/tor", 
                        files=files, 
                        verify="selfsigned.crt")

Security Considerations and Precautions

When using self-signed certificates, the following security aspects should be considered:

Avoid using verify=False: Completely disabling certificate verification exposes to man-in-the-middle attacks and should only be used temporarily in testing environments.

Certificate validity management: Self-signed certificates require regular updates to ensure renewal before expiration.

Private key protection: Private key files must be strictly protected to avoid security risks from leakage.

Performance Optimization Recommendations

For high-concurrency scenarios, consider the following optimization measures:

Session reuse: Use requests.Session() to reuse SSL connections and reduce handshake overhead.

Certificate caching: Cache verified certificates in memory to avoid repeated file read operations.

Connection pool configuration: Properly configure connection pool size to balance resource usage and performance requirements.

Conclusion

By properly configuring the verify parameter in Requests library, developers can securely use self-signed SSL certificates for HTTPS communication. The recommended approach is to prioritize specifying certificate file paths, which ensures security while providing clear trust configuration. Combined with proper certificate generation and management practices, reliable secure communication can be achieved in both development and production environments.