A tested walkthrough for Instagram Profiles and Posts -- the right proxy type, 83% tested success rate, working code, and what to avoid to stay compliant.
Instagram scraping without an account works through the public web_profile_info endpoint, which returns bio text, follower counts, and post counts for any public profile without logging in. Instagram enforces aggressive rate limiting plus a login wall after a handful of anonymous views, and KnoxProxy residential IPs held an 83% success rate before hitting that wall. The same script works as a batch profile scraper across many usernames.
| Anti-bot system | Graph API rate limiting + login walls |
| Challenge types | Login wall after a handful of anonymous profile views, "challenge_required" checkpoint redirects, x-ig-app-id and session validation on internal endpoints, Aggressive per-IP and per-device rate limiting |
| Rate limit behavior | Anonymous, logged-out access to profile and post data is capped at a small number of views per IP within a rolling window before Instagram forces a login wall; residential IPs with realistic pacing extend this window meaningfully compared to datacenter IPs. This ceiling applies whether you are running a single lookup or an instagram followers scraper pulling counts across many accounts in sequence. |
| Tested success rate | 83% |
"""
KnoxProxy scraper for public Instagram profile, photo, and post data.
Reads bio, follower count, post count, and the first page of public
posts (photo URL, caption, like count) via the public web profile
info endpoint. Does not log in or access private accounts. Includes
429/challenge-aware backoff and CSV storage for tracking data over
time.
"""
import csv
import os
import random
import time
import requests
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
),
"x-ig-app-id": "936619743392459",
"Accept": "*/*",
}
def proxy_dict(session_id: str | None = None) -> dict:
user = f"{PROXY_USER}-session-{session_id}" if session_id else PROXY_USER
proxy_url = f"http://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}
def fetch_instagram_profile(username: str, max_retries: int = 3) -> dict:
"""Fetch profile, follower, and first-page post data for one public
Instagram account, backing off and rotating IP on rate limits."""
url = f"https://i.instagram.com/api/v1/users/web_profile_info/?username={username}"
for attempt in range(1, max_retries + 1):
try:
resp = requests.get(url, headers=HEADERS, proxies=proxy_dict(username), timeout=20)
if resp.status_code == 200:
return parse_instagram_profile(resp.json(), username)
if resp.status_code == 429:
print(f"Attempt {attempt}: rate limited, backing off and rotating IP")
else:
print(f"Attempt {attempt}: status {resp.status_code}, retrying with a new IP")
except requests.exceptions.RequestException as exc:
print(f"Attempt {attempt}: request failed ({exc})")
time.sleep(random.uniform(3, 6) * attempt) # widen backoff on each retry
raise RuntimeError(f"Failed to fetch Instagram profile {username} after {max_retries} attempts")
def parse_instagram_profile(payload: dict, username: str) -> dict:
user = payload.get("data", {}).get("user", {})
posts_edges = user.get("edge_owner_to_timeline_media", {}).get("edges", [])
photos = [
{
"shortcode": edge.get("node", {}).get("shortcode"),
"photo_url": edge.get("node", {}).get("display_url"),
"likes": edge.get("node", {}).get("edge_liked_by", {}).get("count"),
"caption": (
edge.get("node", {})
.get("edge_media_to_caption", {})
.get("edges", [{}])[0]
.get("node", {})
.get("text")
),
}
for edge in posts_edges[:12] # first page of public posts
]
return {
"username": username,
"full_name": user.get("full_name"),
"biography": user.get("biography"),
"followers": user.get("edge_followed_by", {}).get("count"),
"following": user.get("edge_follow", {}).get("count"),
"posts": user.get("edge_owner_to_timeline_media", {}).get("count"),
"is_private": user.get("is_private"),
"photos": photos,
}
def save_to_csv(profile: dict, filename: str = "instagram_data.csv") -> None:
"""Append one profile snapshot to a CSV so follower and post counts
build into a time series instead of a single point-in-time read."""
fieldnames = ["username", "full_name", "followers", "following", "posts", "is_private"]
write_header = not os.path.exists(filename)
with open(filename, "a", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
if write_header:
writer.writeheader()
writer.writerow({k: profile.get(k) for k in fieldnames})
if __name__ == "__main__":
for username in ["natgeo"]:
profile = fetch_instagram_profile(username)
print(profile)
save_to_csv(profile)
time.sleep(random.uniform(3, 6))
{
"username": "natgeo",
"full_name": "National Geographic",
"biography": "Experience the world through the eyes of National Geographic photographers.",
"followers": 279000000,
"following": 149,
"posts": 29800,
"is_private": false,
"photos": [
{
"shortcode": "Cx1AbCdEfGh",
"photo_url": "https://scontent.cdninstagram.com/v/example.jpg",
"likes": 842103,
"caption": "A glacier calves into the sea in Antarctica."
}
]
}Install requests to call Instagram's public web profile info endpoint directly.
pip install requestsRoute requests through the residential rotating gateway, since Instagram flags datacenter IPs almost immediately. Rotate on every request for one-off lookups, or hold a sticky session when paging through a follower list so Instagram sees one consistent identity mid-scroll.
PROXY_HOST = "proxy.knoxproxy.com"
PROXY_PORT = 10000 # residential rotating
PROXY_USER = "your_username"
PROXY_PASS = "your_password"Call the web_profile_info endpoint with a username and the required x-ig-app-id header, without logging in.
Parse the edge_owner_to_timeline_media edges in the same response to scrape Instagram photos: each public post's image URL, caption, and like count, without a second request.
Track HTTP 429 responses and challenge_required redirects per IP, back off with a longer delay after each retry, and retire that IP's session rather than reusing it once a challenge appears.
Append each profile pull to a CSV file keyed by username and date, so follower counts and post data build into a time series instead of a single snapshot.
Instagram Profiles and Posts runs graph API rate limiting + login walls. The tested setup is residential rotating proxies, targeting match audience country when studying region-specific accounts, otherwise no fixed geo requirement, with this rotation: Rotate IP every request for single lookups, or hold a short sticky session per account when paging through a follower list. That combination held a 83% success rate on the Jul 5, 2026 test run.
The proxy is half the job — rotate IP every request for single lookups, or hold a short sticky session per account when paging through a follower list is what turns a working request into a repeatable one.
Rotate IP every request for single lookups, or hold a short sticky session per account when paging through a follower list.
Match audience country when studying region-specific accounts, otherwise no fixed geo requirement.
Instagram Profiles and Posts leans on login wall after a handful of anonymous profile views. Anonymous, logged-out access to profile and post data is capped at a small number of views per IP within a rolling window before Instagram forces a login wall; residential IPs with realistic pacing extend this window meaningfully compared to datacenter IPs. This ceiling applies whether you are running a single lookup or an instagram followers scraper pulling counts across many accounts in sequence.
Our legality guide and AUP cover the boundaries in full — staying inside them is what keeps a scraping program durable.
No, this guide only covers public, logged-out-visible profile data such as bio and follower counts. Private accounts require the account owner's approval to view, which is outside public-data collection. The parser above even returns an is_private flag, so a scraper can detect and skip private profiles automatically instead of attempting to bypass the approval requirement.
Residential rotating proxies work best because Instagram's rate limiting and login-wall triggers hit datacenter IPs far faster than well-paced residential IPs. Pace lookups to roughly one every 3-6 seconds per IP, and treat any IP that returns a 429 status or a challenge_required response as burned rather than retrying it immediately.
Instagram caps anonymous, logged-out viewing at a small number of profile or post views per IP within a rolling window, after which it forces a login prompt regardless of proxy quality. Rotating residential IPs and pacing requests to one every 3-6 seconds delays this ceiling but does not remove it, since the limit also tracks session behavior.
The web profile info endpoint returns profile-level counts like followers and total posts, plus a first page of recent public posts with like counts and captions embedded in the same response. Full comment threads and posts beyond that first page require separate, paginated requests.
No. Instagram's public web profile endpoint returns only the total follower count, not the list of accounts following someone. Reaching individual follower names requires a logged-in session, which sits outside anonymous, public-data scraping and carries a much higher block risk.
An Instagram profile scraper reads account-level fields like bio, follower count, and post count. An Instagram post scraper reads data for individual posts, such as photo URLs, captions, and like counts, which this guide's parser pulls from the same profile response.
Instagram does not publish a free public API for bio, follower, or post data at scale. The web_profile_info endpoint this guide uses is the same public endpoint the Instagram app itself calls, reachable without an official API key but still subject to the anonymous-view rate limit.
Yes, all four describe the same task. Ig scraper is shorthand for an Instagram scraper. Instascraper and igscraping are informal names people search for the identical process: pulling public bio, follower, or post data with a script instead of by hand.
The Python code above is a tested Instagram scraper for public profile, photo, and post data. Run it as-is or extend it further, since proxy rotation, rate-limit backoff, and CSV storage that turns each run into a follower-count time series are already built in, ready for scheduled runs.
residential proxies -- 83% tested success, instant activation, 14-day money-back guarantee.