A tested walkthrough for Best Buy Electronics -- the right proxy type, 87% tested success rate, working code, and what to avoid to stay compliant.
A Best Buy scraper can reach SKU pages with residential rotating proxies, though the official Products API offers a more durable path at higher volume. Best Buy runs Akamai Bot Manager, which throttles direct HTML scraping after roughly 10-15 requests per minute from one IP, and KnoxProxy residential IPs held an 87% success rate on SKU pages.
| Anti-bot system | Akamai Bot Manager |
| Challenge types | Akamai sensor_data JavaScript challenge, 403 Access Denied for flagged sessions, Per-IP rate throttling, SKU-based authentication on the official Products API |
| Rate limit behavior | Direct HTML scraping is throttled aggressively after roughly 10-15 requests per minute from one IP; Best Buy's official Products API, available with a free developer key, is a more reliable path for higher volume but is separately rate-limited per key. |
| Tested success rate | 87% |
"""
KnoxProxy scraper for Best Buy SKU pages.
Extracts title, price, and fulfillment state from a public product page.
"""
import random
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
def proxy_dict() -> dict:
proxy_url = f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_bestbuy_sku(sku: str, max_retries: int = 3) -> dict:
url = f"https://www.bestbuy.com/site/sku/{sku}.p"
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxy_dict(), timeout=20)
if resp.status_code == 200:
return parse_bestbuy_sku(resp.text, sku)
print(f"Attempt {attempt}: Best Buy returned status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(3, 6))
raise RuntimeError(f"Failed to fetch Best Buy SKU {sku} after {max_retries} attempts")
def parse_bestbuy_sku(html: str, sku: str) -> dict:
soup = BeautifulSoup(html, "html.parser")
title_el = soup.select_one(".sku-title h1")
price_el = soup.select_one(".priceView-customer-price span")
cart_button = soup.select_one(".fulfillment-add-to-cart-button button")
return {
"sku": sku,
"title": title_el.get_text(strip=True) if title_el else None,
"price": price_el.get_text(strip=True) if price_el else None,
"addable_to_cart": bool(cart_button and "Add to Cart" in cart_button.get_text()),
}
if __name__ == "__main__":
for sku in ["6525421"]:
print(fetch_bestbuy_sku(sku))
time.sleep(random.uniform(3, 6))
{
"sku": "6525421",
"title": "Apple - AirPods Pro (2nd generation) with MagSafe Case (USB-C)",
"price": "$189.99",
"addable_to_cart": true
}Install requests and BeautifulSoup for SKU page scraping.
pip install requests beautifulsoup4Route SKU page requests through the residential rotating gateway.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Request the product URL, which includes the numeric SKU, with realistic browser headers.
Extract fields Best Buy uses for its buy box, and treat a missing add-to-cart button as an out-of-stock signal.
Best Buy Electronics runs akamai Bot Manager. The tested setup is residential rotating proxies, targeting uS residential IPs, with this rotation: Rotate IP on every request. That combination held a 87% success rate on the Jun 29, 2026 test run.
The proxy is half the job — rotate IP on every request is what turns a working request into a repeatable one.
Rotate IP on every request.
US residential IPs.
Best Buy Electronics leans on akamai sensor_data JavaScript challenge. Direct HTML scraping is throttled aggressively after roughly 10-15 requests per minute from one IP; Best Buy's official Products API, available with a free developer key, is a more reliable path for higher volume but is separately rate-limited per key.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
Yes, the Best Buy Products API is free with a developer key and covers pricing, availability, and specifications, making it a more durable option than HTML scraping for sustained or high-volume use. It is rate-limited per developer key rather than per IP, so it scales past the point where direct SKU page scraping starts triggering Akamai throttling.
Residential rotating proxies work best for direct SKU page scraping since Akamai Bot Manager throttles datacenter IPs and non-browser traffic patterns quickly. Rotate the IP on every request and keep volume under 10-15 requests per minute; above that, the free official Products API is a more durable path since it is rate-limited per key instead of per IP.
The add-to-cart button is replaced with alternate fulfillment messaging, such as "Sold Out" or store-pickup-only text, when an item is unavailable for standard shipping. The scraper above checks the fulfillment-add-to-cart-button element and returns addable_to_cart as false whenever that button text does not literally contain the phrase Add to Cart.
Best Buy can show store-specific pricing and availability based on a selected ZIP code or store, so results without a location context reflect the default online price. The scraper above requests the plain SKU URL with no location parameter, so it always returns that same national online price rather than a store-adjusted figure.
The Python code above is a tested Best Buy scraper for SKU pages that held an 87% success rate in the last test cycle. Run it as-is or adapt it to call the official Products API instead for higher volume; proxy rotation and retry logic are already wired in either way.
residential proxies -- 87% tested success, instant activation, 14-day money-back guarantee.