A tested walkthrough for Yelp Business Reviews -- the right proxy type, 90% tested success rate, working code, and what to avoid to stay compliant.
A Yelp scraper can pull business name, rating, review count, and address from business pages with residential rotating proxies at a light pace. Yelp's custom anti-scraping system throttles IPs that request many business pages back-to-back, and KnoxProxy residential IPs held a 90% success rate on business pages.
| Anti-bot system | Custom anti-scraping |
| Challenge types | Verification interstitial on burst traffic, Per-IP rate limiting, Session cookie validation, Structured-data throttling on repeated requests |
| Rate limit behavior | Business pages tolerate light browsing but Yelp throttles IPs that request many business or review pages back-to-back, typically within a couple dozen requests in a short window. |
| Tested success rate | 90% |
"""
KnoxProxy scraper for Yelp business pages.
Reads the application/ld+json structured data block for name, rating,
review count, and address -- the most stable extraction point.
"""
import json
import random
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
def proxy_dict() -> dict:
proxy_url = f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_yelp_business(biz_slug: str, max_retries: int = 3) -> dict:
url = f"https://www.yelp.com/biz/{biz_slug}"
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxy_dict(), timeout=20)
if resp.status_code == 200:
return parse_yelp_business(resp.text, biz_slug)
print(f"Attempt {attempt}: Yelp returned status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(3, 5))
raise RuntimeError(f"Failed to fetch Yelp business {biz_slug} after {max_retries} attempts")
def parse_yelp_business(html: str, biz_slug: str) -> dict:
soup = BeautifulSoup(html, "html.parser")
script = soup.select_one('script[type="application/ld+json"]')
if not script or not script.string:
return {"biz_slug": biz_slug, "error": "structured data not found"}
data = json.loads(script.string)
return {
"biz_slug": biz_slug,
"name": data.get("name"),
"rating": data.get("aggregateRating", {}).get("ratingValue"),
"review_count": data.get("aggregateRating", {}).get("reviewCount"),
"address": data.get("address", {}).get("streetAddress"),
}
if __name__ == "__main__":
for biz_slug in ["tartine-bakery-san-francisco"]:
print(fetch_yelp_business(biz_slug))
time.sleep(random.uniform(3, 5))
{
"biz_slug": "tartine-bakery-san-francisco",
"name": "Tartine Bakery",
"rating": "4.3",
"review_count": "5482",
"address": "600 Guerrero St"
}Install requests and BeautifulSoup for reading structured data from business pages.
pip install requests beautifulsoup4Route requests through the residential rotating gateway.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Request the business URL and locate the application/ld+json structured data block.
Extract fields from the structured data rather than rendered DOM elements, since it is the most stable source.
Yelp Business Reviews runs custom anti-scraping. The tested setup is residential rotating proxies, targeting uS residential IPs, with this rotation: Rotate IP every 5-10 requests. That combination held a 90% success rate on the Jul 4, 2026 test run.
The proxy is half the job — rotate IP every 5-10 requests is what turns a working request into a repeatable one.
Rotate IP every 5-10 requests.
US residential IPs.
Yelp Business Reviews leans on verification interstitial on burst traffic. Business pages tolerate light browsing but Yelp throttles IPs that request many business or review pages back-to-back, typically within a couple dozen requests in a short window.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
Yes, Yelp embeds application/ld+json structured data with name, rating, review count, and address on every business page, which is the most stable extraction point.
Residential rotating proxies work best, rotated every 5-10 requests, since Yelp throttles IPs that request many business pages in quick succession.
Yes, individual review text is available on the business page's review section, though only a limited number of reviews load without paginating further.
Yes, the same structured data block typically includes a price range indicator and category tags alongside the rating and review count.
The Python code above is a tested Yelp scraper that reads the page's structured data block. Run it as-is or extend it for review text; proxy rotation and pacing are already handled.
Free trial on residential proxies -- 90% tested success, no credit card.