A tested walkthrough for LinkedIn Public Profiles -- the right proxy type, 79% tested success rate, working code, and what to avoid to stay compliant.
A LinkedIn scraper can reach public profile and company page data in limited amounts via residential rotating proxies before a login wall appears. LinkedIn combines custom scraping detection with device fingerprinting, and KnoxProxy residential IPs held a 79% success rate on anonymous public-page requests.
| Anti-bot system | Custom scraping detection + login walls |
| Challenge types | Login and authentication wall after limited anonymous page views, Device and behavioral fingerprinting, CAPTCHA challenge on flagged sessions, Per-IP request velocity limits |
| Rate limit behavior | Public profile and company pages are viewable anonymously in limited numbers before LinkedIn prompts a login wall; scraping at scale from a single IP triggers fingerprint-based flags well before any simple rate-limit counter would be exhausted. |
| Tested success rate | 79% |
"""
KnoxProxy scraper for public LinkedIn profile and company pages.
Reads the application/ld+json structured data LinkedIn embeds for SEO,
which is the most stable public data source. Does not log in.
"""
import json
import random
import time
import requests
from bs4 import BeautifulSoup
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
}
def proxy_dict() -> dict:
proxy_url = f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_linkedin_company(vanity_name: str, max_retries: int = 3) -> dict:
url = f"https://www.linkedin.com/company/{vanity_name}/"
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxy_dict(), timeout=20)
if resp.status_code == 200 and "authwall" not in resp.url:
return parse_linkedin_ld_json(resp.text, vanity_name)
print(f"Attempt {attempt}: hit authwall or status {resp.status_code}")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(3, 6))
raise RuntimeError(f"Failed to fetch LinkedIn company {vanity_name} after {max_retries} attempts")
def parse_linkedin_ld_json(html: str, vanity_name: str) -> dict:
soup = BeautifulSoup(html, "html.parser")
script = soup.select_one('script[type="application/ld+json"]')
if not script or not script.string:
return {"vanity_name": vanity_name, "error": "structured data not found"}
data = json.loads(script.string)
return {
"vanity_name": vanity_name,
"name": data.get("name"),
"description": data.get("description"),
"industry": data.get("industry"),
"employees": data.get("numberOfEmployees", {}).get("value"),
}
if __name__ == "__main__":
for vanity_name in ["openai"]:
print(fetch_linkedin_company(vanity_name))
time.sleep(random.uniform(3, 6))
{
"vanity_name": "openai",
"name": "OpenAI",
"description": "OpenAI is an AI research and deployment company.",
"industry": "Technology, Information and Internet",
"employees": 3500
}Install requests and BeautifulSoup for reading structured data from public profile and company pages.
pip install requests beautifulsoup4Route requests through the residential rotating gateway to reduce fingerprint-based flags.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Request the public URL without authenticating, and read the embedded JSON-LD structured data LinkedIn provides for search engines.
Treat any redirect to the login or authwall page as the anonymous-access limit rather than a failure to retry through.
LinkedIn Public Profiles runs custom scraping detection + login walls. The tested setup is residential rotating proxies, targeting match the target region for localized public search results, with this rotation: Rotate IP on every request. That combination held a 79% success rate on the Jul 6, 2026 test run.
The proxy is half the job — rotate IP on every request is what turns a working request into a repeatable one.
Rotate IP on every request.
Match the target region for localized public search results.
LinkedIn Public Profiles leans on login and authentication wall after limited anonymous page views. Public profile and company pages are viewable anonymously in limited numbers before LinkedIn prompts a login wall; scraping at scale from a single IP triggers fingerprint-based flags well before any simple rate-limit counter would be exhausted.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
The Ninth Circuit's hiQ Labs v. LinkedIn decisions, including after the Supreme Court's 2021 remand, held that scraping publicly accessible LinkedIn data likely does not violate the CFAA. LinkedIn's User Agreement still separately prohibits automated scraping as a matter of contract, so legal risk is not eliminated even though CFAA exposure is limited.
Residential rotating proxies work best since LinkedIn's fingerprinting flags datacenter IPs and repetitive request patterns quickly, well before any simple rate-limit counter would trigger.
A limited number of public profile and company pages are viewable anonymously, primarily the structured data LinkedIn exposes for search engines, before a login wall interrupts further browsing.
Public job posting pages carry similar structured data and anonymous-view limits as profile and company pages, so the same pacing and login-wall boundaries apply.
The Python code above is a tested LinkedIn scraper for anonymous public-page data. Run it as-is or extend it; proxy rotation is already handled, though the login-wall limit still applies regardless of tooling.
Free trial on residential proxies -- 79% tested success, no credit card.