A tested walkthrough for Yelp Business Reviews -- the right proxy type, 90% tested success rate, working code, and what to avoid to stay compliant.
A Yelp scraper can pull business name, rating, review count, and address from business pages with residential rotating proxies at a light pace. Yelp's custom anti-scraping system throttles IPs that request many business pages back-to-back, and KnoxProxy residential IPs held a 90% success rate on business pages.
| Anti-bot system | Custom anti-scraping |
| Challenge types | Verification interstitial on burst traffic, Per-IP rate limiting, Session cookie validation, Structured-data throttling on repeated requests |
| Rate limit behavior | Business pages tolerate light browsing but Yelp throttles IPs that request many business or review pages back-to-back, typically within a couple dozen requests in a short window. |
| Tested success rate | 90% |
"""
KnoxProxy scraper for Yelp business pages.
Reads the application/ld+json structured data block for name, rating,
review count, and address -- the most stable extraction point.
"""
import json
import random
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
def proxy_dict() -> dict:
proxy_url = f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_yelp_business(biz_slug: str, max_retries: int = 3) -> dict:
url = f"https://www.yelp.com/biz/{biz_slug}"
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxy_dict(), timeout=20)
if resp.status_code == 200:
return parse_yelp_business(resp.text, biz_slug)
print(f"Attempt {attempt}: Yelp returned status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(3, 5))
raise RuntimeError(f"Failed to fetch Yelp business {biz_slug} after {max_retries} attempts")
def parse_yelp_business(html: str, biz_slug: str) -> dict:
soup = BeautifulSoup(html, "html.parser")
script = soup.select_one('script[type="application/ld+json"]')
if not script or not script.string:
return {"biz_slug": biz_slug, "error": "structured data not found"}
data = json.loads(script.string)
return {
"biz_slug": biz_slug,
"name": data.get("name"),
"rating": data.get("aggregateRating", {}).get("ratingValue"),
"review_count": data.get("aggregateRating", {}).get("reviewCount"),
"address": data.get("address", {}).get("streetAddress"),
}
if __name__ == "__main__":
for biz_slug in ["tartine-bakery-san-francisco"]:
print(fetch_yelp_business(biz_slug))
time.sleep(random.uniform(3, 5))
{
"biz_slug": "tartine-bakery-san-francisco",
"name": "Tartine Bakery",
"rating": "4.3",
"review_count": "5482",
"address": "600 Guerrero St"
}Install requests and BeautifulSoup for reading structured data from business pages.
pip install requests beautifulsoup4Route requests through the residential rotating gateway.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Request the business URL and locate the application/ld+json structured data block.
Extract fields from the structured data rather than rendered DOM elements, since it is the most stable source.
Yelp Business Reviews runs custom anti-scraping. The tested setup is residential rotating proxies, targeting uS residential IPs, with this rotation: Rotate IP every 5-10 requests. That combination held a 90% success rate on the Jul 4, 2026 test run.
The proxy is half the job — rotate IP every 5-10 requests is what turns a working request into a repeatable one.
Rotate IP every 5-10 requests.
US residential IPs.
Yelp Business Reviews leans on verification interstitial on burst traffic. Business pages tolerate light browsing but Yelp throttles IPs that request many business or review pages back-to-back, typically within a couple dozen requests in a short window.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
Yes, Yelp embeds application/ld+json structured data with name, rating, review count, and address on every business page, which is the most stable extraction point. Reading straight from that JSON block, rather than rendered DOM elements, is also what let KnoxProxy hold a 90% success rate on business pages during testing.
Residential rotating proxies work best, rotated every 5-10 requests, since Yelp throttles IPs that request many business pages in quick succession. Pacing requests to roughly one every 3-5 seconds on top of that rotation held KnoxProxy's proxies to a 90% success rate on business pages in testing.
Yes, individual review text is available on the business page's review section, though only a limited number of reviews load without paginating further. The structured data block covers name, rating, and review count reliably, but pulling full review text means parsing the rendered review section directly instead.
Yes, the same structured data block typically includes a price range indicator and category tags alongside the rating and review count. Reading these from the application/ld+json block instead of rendered HTML keeps the extraction stable even when Yelp changes its page layout or CSS class names.
The Python code above is a tested Yelp scraper that reads the page's structured data block. Run it as-is or extend it for review text; proxy rotation and pacing are already handled, the setup that held a 90% success rate on business pages during the most recent test.
residential proxies -- 90% tested success, instant activation, 14-day money-back guarantee.