A status code is a three-digit number a server sends back with every HTTP response to describe the result of the request. It tells the client whether the request succeeded, failed, or needs further action.
Status codes are grouped by their first digit. Codes starting with 2, like 200, mean success, while codes starting with 3 signal a redirect to another URL. Codes starting with 4 point to a client-side problem, such as 404 for a missing page or 429 for too many requests, and codes starting with 5 mean the server itself had an error. A scraper reads this code after every request to decide whether to save the data, retry, or stop.
Handle it deliberately in production scrapers -- most breakage traces back to skipping this step.
USER-country-de-session-task01Add this string to your scraper's proxy credentials and every request in the job shares one exit IP, which keeps status code-related behavior consistent across the run. Change "task01" per worker to isolate parallel scrapes.
Isolate the logic for this step so every scraper in the project shares one tested implementation.
Sites change layouts and behavior over time -- recheck this part of the scraper on a schedule, not just at launch.
This works best over residential or ISP IPs, so the target sees ordinary browsing rather than clustered datacenter traffic.
Capture what actually failed so a broken selector or a new status code surfaces instead of getting masked by automatic retries.
A scraper checks for a 200 status code before parsing a page, and treats a 403 or 429 code as a sign to slow down or switch proxies.
Status codes tell a scraper exactly what happened with a request, which is essential for building reliable retry and error-handling logic. Watching for codes like 429 or 403 also helps a scraper catch rate limiting or blocks early, before they turn into a full IP ban.
A 429 status code means "Too Many Requests," which signals that the client has exceeded the rate limit set by the server. Scrapers usually respond by slowing down or switching to a different proxy IP address.
Not always. A 403 means "Forbidden" and can result from a missing permission, a blocked IP address, or a failed bot check, so it is worth checking the response body for more detail before assuming a full ban.
Ready to put this into practice? Read the Docs
Start a free trial and test with real targets -- no credit card, no sales call.