Complete guide to using proxies with Python's requests library in 2026. Covers HTTP, HTTPS, SOCKS5, authenticated proxies, rotating proxy pools, session-level proxies, retry logic, concurrent requests, and common errors — with working code examples for every setup.
The Python requests library is the most widely used HTTP client in any language — over 300 million downloads per month on PyPI. It's also the foundation of nearly every scraping, automation, and data-collection script ever written.
If you're hitting rate limits, IP bans, or geo-blocks while using requests, proxies are the fix. This guide covers every proxy configuration you'll need: HTTP, HTTPS, SOCKS5, authenticated proxies, rotating proxies, and session-level proxy handling — with working Python code for each.
You need requests installed. For SOCKS5 proxy support you also need PySocks:
Pass a proxies dict to any requests call. The key is the scheme (http or https) and the value is the proxy URL:
The proxy dict is scheme-keyed: the http key controls plain HTTP requests, the https key controls HTTPS requests. You can route them through different proxies or use the same one for both.
For HTTPS targets the proxy itself can still be HTTP (the CONNECT tunnel method handles the TLS handshake end-to-end):
If the proxy presents a self-signed certificate, pass verify=False to skip TLS verification (not recommended in production):
Mobile and residential proxies typically support SOCKSSOCKS5 proxies are more flexible — they handle any protocol (HTTP, HTTPS, raw TCP) without needing to understand the traffic:
Use socks5h:// instead of socks5:// to delegate DNS resolution to the proxy server (prevents DNS leaks and is preferred for anonymity):
> socks5 vs socks5h: socks5 resolves DNS locally then tunnels the IP. socks5h sends the hostname to the proxy and lets it resolve — this prevents your local DNS from leaking the target domain.
Most commercial proxy services require authentication. Put the credentials directly in the URL:
For SOCKS5 with auth:
If you make many requests from the same script, use a Session to set the proxy once instead of passing it on every call:
Sessions also preserve cookies across requests, which is critical for scraping authenticated pages.
Rotating proxies are the most important pattern for any serious scraping or automation. Each request (or every N requests) uses a different IP, preventing rate-limit bans:
Production scrapers need retry logic. If one proxy fails, automatically fall back to another:
Some scrapers need to maintain the same IP for a full workflow — logging in, navigating, taking action. Use a sticky session proxy or a single session object that pins to one proxy:
If you prefer not to hardcode proxies, requests automatically picks up the standard HTTPPROXY, HTTPSPROXY, and NOPROXY` environment variables:
Then your script needs no proxy config at all:
To disable environment-variable proxies for a specific request:
For high-throughput scraping, use concurrent.futures with a separate proxy per thread:
> Thread safety: Each requests.Session is not thread-safe. Either use plain requests.get() in each thread, or create a new Session per thread.
| Error | Cause | Fix | |---|---|---| | ProxyError: Cannot connect to proxy | Wrong proxy host/port, proxy is down | Try another proxy from the pool | | SSLError: CERTIFICATEVERIFYFAILED | Proxy intercepts TLS and presents its own cert | Pass verify=False or add proxy CA to trust store | | ConnectionError: Remote end closed connection | Target site blocked the proxy IP | Rotate to a new IP | | HTTPError: 407 Proxy Authentication Required | Missing or wrong credentials in proxy URL | Check username:password in proxy URL | | Timeout | Proxy or target too slow | Increase timeout, try a different proxy | | ConnectionError: SOCKSHTTPSConnectionPool | requests[socks] not installed | pip install requests[socks] |
| Proxy Type | Use Case | Detection Risk | |---|---|---| | Datacenter | High-speed scraping, non-aggressive targets | High — most sites block AWS/GCP ASNs | | Residential | General scraping, e-commerce, social media | Medium — real IPs but shared pools | | Mobile (4G/5G) | AI APIs, social platforms, aggressive anti-bot | Very low — carrier IPs treated like real users |
For scripts hitting rate-limited APIs (ChatGPT, Google, LinkedIn) or sites with aggressive bot detection, 4G/5G mobile proxies are the only type that consistently avoids bans. Carrier IPs share NAT with millions of real users — anti-abuse systems cannot block them without collateral damage.
How to Use Proxies with Playwright — browser automation with rotating proxies How to Use Proxies with Puppeteer — Node.js headless Chrome proxy setup Proxy Rotation Best Practices — rotation strategies, backoff, and pool management