Async HTTP client for Python. Run hundreds of concurrent proxied requests with asyncio and aiohttp through KnoxProxy.
Install aiohttp and the SOCKS connector.
pip install aiohttp aiohttp-socksSet your KnoxProxy credentials.
PROXY_URL = "http://USER:PASS@gw.knoxproxy.com:7000"Use aiohttp.ClientSession with the proxy parameter.
import aiohttp, asyncio
async def fetch():
async with aiohttp.ClientSession() as session:
async with session.get("https://httpbin.org/ip",
proxy="http://USER:PASS@gw.knoxproxy.com:7000") as resp:
print(await resp.json())
asyncio.run(fetch())Target countries via username flags.
proxy = "http://USER-country-de:PASS@gw.knoxproxy.com:7000"Use asyncio.gather for parallel requests.
async def fetch_many(urls):
async with aiohttp.ClientSession() as session:
tasks = [session.get(url, proxy=PROXY_URL) for url in urls]
return await asyncio.gather(*tasks)Verify your exit IP.
asyncio.run(fetch())"""KnoxProxy + aiohttp -- async concurrent scraping."""
import aiohttp
import asyncio
PROXY = "http://USER:PASS@gw.knoxproxy.com:7000"
async def fetch(session, url):
async with session.get(url, proxy=PROXY, timeout=aiohttp.ClientTimeout(total=30)) as resp:
return await resp.text()
async def main():
urls = [f"https://httpbin.org/ip" for _ in range(10)]
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, u) for u in urls])
for r in results:
print(r)
asyncio.run(main())Default: new IP per request (each connection through the gateway). For sticky sessions, use -session-{id} in the username.
| Problem | Fix |
|---|---|
| ClientProxyConnectionError | Verify URL format: http://USER:PASS@gw.knoxproxy.com:7000. Check network connectivity. |
| TimeoutError | Increase timeout: aiohttp.ClientTimeout(total=60). |
| ServerDisconnectedError | Add retry logic with exponential backoff. Check if the target rate-limits. |
USER-country-de-city-berlin-session-profile07Order matters -- geo flags before the session flag. The session name is free text; use the profile ID so the mapping is self-documenting. Password stays as issued; no flags belong there. HTTP on :7000, SOCKS5 on :7001, same credentials.
For concurrent workloads, yes. aiohttp runs many requests in parallel on a single thread via asyncio. requests is synchronous and needs threading or multiprocessing for concurrency.
KnoxProxy has no connection limit. aiohttp defaults to 100 connections per host. Increase with TCPConnector(limit=500) for higher concurrency.
Yes, install aiohttp-socks and use ProxyConnector.from_url("socks5://USER:PASS@gw.knoxproxy.com:7001").
Wrap requests in try/except for aiohttp.ClientError. Implement retry with asyncio.sleep() between attempts.
Free trial on rotating residential -- 5 minutes setup, no credit card.