Async HTTP client for Python. Run hundreds of concurrent proxied requests with asyncio and aiohttp through KnoxProxy.
Install aiohttp and the SOCKS connector.
pip install aiohttp aiohttp-socksSet your KnoxProxy credentials.
PROXY_URL = "http://USER:PASS@gw.knoxproxy.com:7000"Use aiohttp.ClientSession with the proxy parameter.
import aiohttp, asyncio
async def fetch():
async with aiohttp.ClientSession() as session:
async with session.get("https://httpbin.org/ip",
proxy="http://USER:PASS@gw.knoxproxy.com:7000") as resp:
print(await resp.json())
asyncio.run(fetch())Target countries via username flags.
proxy = "http://USER-country-de:PASS@gw.knoxproxy.com:7000"Use asyncio.gather for parallel requests.
async def fetch_many(urls):
async with aiohttp.ClientSession() as session:
tasks = [session.get(url, proxy=PROXY_URL) for url in urls]
return await asyncio.gather(*tasks)Verify your exit IP.
asyncio.run(fetch())"""KnoxProxy + aiohttp -- async concurrent scraping."""
import aiohttp
import asyncio
PROXY = "http://USER:PASS@gw.knoxproxy.com:7000"
async def fetch(session, url):
async with session.get(url, proxy=PROXY, timeout=aiohttp.ClientTimeout(total=30)) as resp:
return await resp.text()
async def main():
urls = [f"https://httpbin.org/ip" for _ in range(10)]
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, u) for u in urls])
for r in results:
print(r)
asyncio.run(main())Default: new IP per request (each connection through the gateway). For sticky sessions, use -session-{id} in the username.
| Problem | Fix |
|---|---|
| ClientProxyConnectionError | Verify URL format: http://USER:PASS@gw.knoxproxy.com:7000. Check network connectivity. |
| TimeoutError | Increase timeout: aiohttp.ClientTimeout(total=60). |
| ServerDisconnectedError | Add retry logic with exponential backoff. Check if the target rate-limits. |
USER-country-de-city-berlin-session-profile07Order matters -- geo flags before the session flag. The session name is free text; use the profile ID so the mapping is self-documenting. Password stays as issued; no flags belong there. HTTP on :7000, SOCKS5 on :7001, same credentials.
For concurrent workloads, aiohttp is faster than requests through KnoxProxy because it runs many requests in parallel on one thread using asyncio, while requests is synchronous and needs threading or multiprocessing to achieve the same parallelism. KnoxProxy itself has no connection limit, though aiohttp defaults to 100 connections per host unless you raise it with TCPConnector(limit=500).
KnoxProxy imposes no limit on concurrent connections, but aiohttp itself defaults to only 100 connections per host, which caps your effective throughput unless you change it. Pass a custom TCPConnector with limit=500 (or higher) to aiohttp.ClientSession to raise that cap, and asyncio.gather lets you run all of those connections at once.
Yes, aiohttp supports SOCKS5, though not out of the box. Install the aiohttp-socks package, then build a connector with ProxyConnector.from_url("socks5://USER:PASS@gw.knoxproxy.com:7001") and pass it to your ClientSession. Port 7001 is the KnoxProxy SOCKS5 gateway, separate from the HTTP proxy port 7000, and the same -session-{id} username suffix still gives you a sticky IP.
Wrap each request in a try/except block that catches aiohttp.ClientError, which covers connection failures, timeouts, and disconnects from the proxy or target site. On failure, retry with asyncio.sleep() between attempts, ideally with exponential backoff, since KnoxProxy or the target may just be briefly overloaded rather than permanently unreachable.
Rotating residential proxies -- 5 minutes setup, instant activation, 14-day money-back guarantee.