The essential points from this guide -- each one is explained in detail below.
Track success rate, latency, and block rate per proxy to identify underperforming IPs.
Implement cooldown windows that temporarily remove blocked proxies from the active pool.
Use geo-targeted rotation to match proxy exit locations with target site expectations.
Backconnect gateways automate rotation but custom pool management enables finer control.
Monitor rotation effectiveness with dashboards that show success rate trends over time.
Effective rotation requires knowing which proxies are working well and which are degraded. Implement a scoring system that tracks recent success rate, average latency, and consecutive failures for each proxy in your pool. Use these scores to weight proxy selection toward better-performing IPs.
from dataclasses import dataclass, field
from time import time
@dataclass
class ProxyHealth:
address: str
successes: int = 0
failures: int = 0
total_latency: float = 0.0
last_used: float = 0.0
cooldown_until: float = 0.0
@property
def success_rate(self) -> float:
total = self.successes + self.failures
return self.successes / total if total > 0 else 1.0
@property
def avg_latency(self) -> float:
return self.total_latency / self.successes if self.successes > 0 else float('inf')
@property
def is_available(self) -> bool:
return time() > self.cooldown_until
def record_success(self, latency: float):
self.successes += 1
self.total_latency += latency
self.last_used = time()
def record_failure(self):
self.failures += 1
if self.failures >= 3:
self.cooldown_until = time() + 300 # 5 min cooldownWith a backconnect gateway like KnoxProxy, health checking happens server-side -- the gateway removes bad IPs from the pool automatically. Direct pool management is needed when you run your own proxy list or need custom scoring logic.
When a proxy gets blocked (403, 429, or CAPTCHA response), putting it on cooldown prevents wasting requests on a known-bad IP. The cooldown duration should match the target site's block expiry -- typically 5-30 minutes for soft blocks, and 24+ hours for hard blocks.
A tiered cooldown system works best: first block triggers a 5-minute cooldown, second block within an hour triggers 30 minutes, and a third block quarantines the proxy for 24 hours. After the cooldown expires, move the proxy back to the active pool with a reduced score so it is selected less frequently until it proves reliable again.
For backconnect gateways, cooldown management is less critical because the gateway's pool is large enough (90.4M+ IPs for KnoxProxy) that you rarely see the same exit IP twice. However, if your gateway username encodes a sticky session, the session itself can get blocked, and you should rotate to a new session ID rather than retrying the same one.
Some scraping tasks require exit IPs from specific countries or cities -- for example, monitoring localized search results, verifying geo-restricted content, or scraping region-specific pricing. Geo-targeted rotation assigns proxies based on the target site's expected user location.
class GeoProxyRouter:
def __init__(self, gateway='gw.knoxproxy.com:7000'):
self.gateway = gateway
def get_proxy(self, country: str, city: str = '') -> str:
user_parts = ['USER']
user_parts.append(f'country-{country}')
if city:
user_parts.append(f'city-{city}')
username = '-'.join(user_parts)
return f'http://{username}:PASS@{self.gateway}'
router = GeoProxyRouter()
us_proxy = router.get_proxy('us', 'new_york') # US/New York exit
uk_proxy = router.get_proxy('gb', 'london') # UK/London exit
jp_proxy = router.get_proxy('jp') # Japan, any cityKnoxProxy supports country and city targeting through username parameters. The gateway selects an IP matching your geo criteria from its residential pool. Note that city-level targeting has a smaller available pool, so expect slightly higher latency as the gateway finds a matching IP.
Without monitoring, you cannot tell whether your rotation strategy is working. Track these metrics over time: overall success rate (target >95%), unique IPs used per hour, average response time, block rate by target domain, and bandwidth consumption. Log each request's proxy, status code, and latency for post-hoc analysis.
Build a simple dashboard or periodic report that shows trends in these metrics. A sudden drop in success rate often indicates the target site updated its anti-bot system, not that your proxies failed. A gradual decline suggests IP reputation degradation in your pool.
For production scrapers, set up alerts for: success rate dropping below 90%, average latency exceeding 10 seconds, or more than 20% of requests returning CAPTCHA challenges. These thresholds help you catch issues before they waste significant proxy bandwidth and time.
Ready to put this into practice? Browse Rotating Proxies
KnoxProxy Research Team · Technical Content
Network engineers and proxy infrastructure specialists with 10+ years in anti-bot systems, web scraping, and IP routing.
90.4M+ ethically sourced residential IPs across 195 countries. Start free -- no credit card required.