Business directories and public registries contain millions of company profiles with contact information. Collecting this data at scale requires proxies that avoid rate limits while respecting the boundary between public business data and personal information.
Use rotating residential proxies for directory scraping. Business directories block datacenter IPs and rate-limit single-IP access. Residential rotation gives each request a fresh identity, maintaining access across large-scale directory crawls.
| Expected success | 98%+ on major directories |
| Rotation | Per request for directory pages |
| Geo targeting | Country-level for regional directories |
| Cost fit | ~$2.10/GB residential PAYG |
import requests
proxy = "http://USER:PASS@gw.knoxproxy.com:7000"categories = ["plumber", "electrician", "dentist"]cities = ["chicago", "houston", "phoenix"]
for cat in categories: for city in cities: page = 1 while page <= 10: r = requests.get( f"https://directory.example/search?q={cat}&loc={city}&p={page}", proxies={"https": proxy}) businesses = parse_listings(r.text) save_contacts(businesses) # Public name, phone, address only page += 1Lead generation collects publicly listed business contact information from directories. Collect only publicly displayed business data -- company name, business phone, business address. Do not scrape personal email addresses, personal phone numbers, or private profiles. Comply with CAN-SPAM, GDPR, and local data protection regulations.
Business directories do not just block scrapers -- they throttle them. After 50-100 requests from one IP, response times slow from 200ms to 5-10 seconds. After a few hundred, you get CAPTCHAs or empty results. Residential rotation resets the counter on every request.
The only reliable way to see what a real user sees is to become one.
Scheduler, proxy fetch, parser, store -- the proxy is one line in the fetch step. Everything else is pipeline you already run.
Extract company name, business phone, business address, and business email only. Personal contact information is off-limits under GDPR and CAN-SPAM.
Directories paginate results. Crawl all pages for each category-location combination to build complete datasets, not partial samples.
The same business appears in multiple directories. Deduplicate by phone number or address before storing to keep your database clean.
Failed fetches are never billed, so your effective cost tracks the success rate you actually observe.
Collecting publicly listed business contact information (company name, business phone, business address) is generally legal. Personal data collection has stricter requirements under GDPR and CAN-SPAM. Consult local regulations.
Residential proxies work on major business directories. Each directory has its own rate limits and terms -- start with low concurrency and increase gradually.
Extract only business-level fields: company name, business phone, business address, and published business email. Skip personal profiles, personal emails, and personal phone numbers.
GDPR applies when you collect personal data of EU residents. Publicly listed business contact data has a legitimate-interest basis, but personal data requires consent. When in doubt, consult a data protection advisor.
Free trial on rotating residential -- city targeting included, no credit card.