A tested walkthrough for eBay Listings -- the right proxy type, 91% tested success rate, working code, and what to avoid to stay compliant.
An eBay scraper can pull listing titles, prices, shipping cost, and seller ratings using datacenter proxies for casual lookups or residential rotating proxies for continuous tracking. eBay's custom bot detection escalates to CAPTCHA or 503 responses once request velocity looks automated, and KnoxProxy proxies held a 91% success rate on listing pages.
| Anti-bot system | Custom bot detection |
| Challenge types | Device and browser fingerprinting, Behavioral pattern analysis (scroll and click timing), CAPTCHA checkpoint on repeated flagged requests, Session token validation on search and listing endpoints |
| Rate limit behavior | Tolerant of moderate polling but flags accounts or IPs that page through search and listing endpoints faster than a human plausibly could, at which point it serves a CAPTCHA checkpoint or intermittent HTTP 503 responses. |
| Tested success rate | 91% |
"""
KnoxProxy scraper for eBay search results and item listings.
Pulls item ID, title, price, shipping, and seller feedback score.
"""
import random
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
def proxy_dict(session_id: str) -> dict:
user = f"{PROXY_USER}-session-{session_id}"
proxy_url = f"http://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def search_ebay(query: str, max_retries: int = 3) -> list[dict]:
url = f"https://www.ebay.com/sch/i.html?_nkw={query.replace(' ', '+')}"
proxies = proxy_dict(session_id=query[:8])
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxies, timeout=20)
if resp.status_code == 200:
return parse_ebay_results(resp.text)
print(f"Attempt {attempt}: eBay returned status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(2, 4))
raise RuntimeError(f"Failed to search eBay for '{query}' after {max_retries} attempts")
def parse_ebay_results(html: str) -> list[dict]:
soup = BeautifulSoup(html, "html.parser")
items = []
for card in soup.select("li.s-item"):
title_el = card.select_one(".s-item__title")
price_el = card.select_one(".s-item__price")
shipping_el = card.select_one(".s-item__shipping")
link_el = card.select_one("a.s-item__link")
if not title_el or not price_el:
continue
items.append({
"title": title_el.get_text(strip=True),
"price": price_el.get_text(strip=True),
"shipping": shipping_el.get_text(strip=True) if shipping_el else "See listing",
"url": link_el["href"] if link_el else None,
})
return items
if __name__ == "__main__":
results = search_ebay("gopro hero 12")
for item in results[:10]:
print(item)
"""
KnoxProxy scraper for eBay auction listing detail pages.
Extracts the item title, current bid, time remaining, and bid count from
a live auction item page (not a Buy It Now listing).
"""
import random
import re
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
BID_COUNT_RE = re.compile(r"([\d,]+)\s+bids?")
def proxy_dict(session_id: str) -> dict:
user = f"{PROXY_USER}-session-{session_id}"
proxy_url = f"http://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_ebay_auction(item_id: str, max_retries: int = 3) -> dict:
url = f"https://www.ebay.com/itm/{item_id}"
proxies = proxy_dict(session_id=item_id)
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxies, timeout=20)
if resp.status_code == 200:
return parse_ebay_auction(resp.text, item_id)
print(f"Attempt {attempt}: eBay returned status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(2, 4))
raise RuntimeError(f"Failed to fetch eBay auction {item_id} after {max_retries} attempts")
def parse_ebay_auction(html: str, item_id: str) -> dict:
soup = BeautifulSoup(html, "html.parser")
title_el = soup.select_one("h1.x-item-title__mainTitle span")
bid_el = soup.select_one("div.x-bid-price span.ux-textspans")
time_el = soup.select_one("span.ux-timer__text")
bid_count_el = soup.select_one("span.vi-VR-bidCount, a.ux-textspans--BOLD")
bid_count = None
if bid_count_el:
match = BID_COUNT_RE.search(bid_count_el.get_text(strip=True))
bid_count = int(match.group(1).replace(",", "")) if match else None
return {
"item_id": item_id,
"title": title_el.get_text(strip=True) if title_el else None,
"current_bid": bid_el.get_text(strip=True) if bid_el else None,
"time_left": time_el.get_text(strip=True) if time_el else None,
"bid_count": bid_count,
}
if __name__ == "__main__":
for item_id in ["204521118990"]:
print(fetch_ebay_auction(item_id))
time.sleep(random.uniform(2, 4))
{
"search_ebay_results_sample": [
{
"title": "GoPro HERO12 Black Waterproof Action Camera",
"price": "$279.00",
"shipping": "Free shipping",
"url": "https://www.ebay.com/itm/204521118990"
}
],
"fetch_ebay_auction_sample": {
"item_id": "204521118990",
"title": "Vintage Rolex Submariner Automatic Watch",
"current_bid": "$247.50",
"time_left": "2d 14h 32m",
"bid_count": 14
}
}Install requests and BeautifulSoup for fetching and parsing eBay search and listing pages.
pip install requests beautifulsoup4Point requests at the KnoxProxy rotating gateway, choosing residential for sustained crawls.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Fetch an eBay search results page and pull out individual item IDs and summary data.
Follow up on each item ID with a detail-page request, spacing calls out to stay under eBay's velocity threshold.
eBay Listings runs custom bot detection. The tested setup is datacenter rotating for low-volume lookups; residential rotating for continuous price-tracking pipelines proxies, targeting match the proxy country to the eBay site variant being scraped (.com, .co.uk, .de, .com.au), with this rotation: Rotate every 5-10 requests, or once per search query. That combination held a 91% success rate on the Jun 28, 2026 test run.
The proxy is half the job — rotate every 5-10 requests, or once per search query is what turns a working request into a repeatable one.
Rotate every 5-10 requests, or once per search query.
Match the proxy country to the eBay site variant being scraped (.com, .co.uk, .de, .com.au).
eBay Listings leans on device and browser fingerprinting. Tolerant of moderate polling but flags accounts or IPs that page through search and listing endpoints faster than a human plausibly could, at which point it serves a CAPTCHA checkpoint or intermittent HTTP 503 responses.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
eBay's robots.txt leaves search and item pages open to crawlers, though the User Agreement restricts automated extraction at scale. Most price-comparison tools scrape public listing pages at a conservative pace rather than seeking explicit permission, treating volume and pacing as the real safety control instead of a signed agreement.
Datacenter rotating proxies are enough for occasional lookups, but continuous price-tracking pipelines should move to residential rotating proxies since eBay's bot detection flags datacenter ranges faster under sustained load. Match the rotation to your actual request volume rather than defaulting to the cheaper option for every job.
Yes, the eBay Browse API and Finding API expose search and item data with an API key and are the more durable option for production integrations that need guaranteed uptime. Scraping still covers fields the API leaves out, like live auction countdowns and bid counts.
eBay updates its search-result and item-page templates periodically, so CSS selectors like s-item__title should be re-verified every few months against a live page before a production run. Auction-page selectors change more often than search-result cards, so check those first if a run starts failing.
The Python code above is a tested eBay scraper you can run as-is or adapt to pull additional fields. It already handles proxy rotation and retries, so there's no extra service to wire up before tracking listings or auction data.
Yes. Auction end time and bid count live on the item detail page, not the search results page, so fetch each item's URL directly. Buy It Now listings do not carry these fields since there is no bidding, so check the listing type before parsing a batch of item IDs.
Save each pull with a timestamp, item ID, and price in a CSV, SQLite, or Postgres table keyed on item ID. Compare each new pull against the last stored row for that item ID, and only write a new record when the price actually changes, not on every single run.
residential proxies -- 91% tested success, instant activation, 14-day money-back guarantee.