Collect posts, profiles and pages with Apify scrapers

The agent does not write a scraper of its own, it runs a ready-made actor from the Apify Store: an actor has a name like apify~instagram-hashtag-scraper and an input — JSON with hashtags, accounts and a result limit. Short jobs go synchronously: a single request waits up to 300 seconds and returns the collected items in the response. Long ones go asynchronously: the agent starts a run, polls its status and, once it reaches SUCCEEDED, pulls the dataset by defaultDatasetId.

A run goes through the statuses READY → RUNNING → SUCCEEDED, FAILED or ABORTED, and the agent reads not only the result but the log: it shows whether the scraper hit a site's protection, returned an empty result or died on a bad input. A stuck run it aborts so that it stops burning compute units, and instead of polling the status forever it can register a webhook on completion and take the data once it fires.

Through Apify the agent reads public data — the same thing a logged-out visitor sees on the page: posts, Reels and videos by hashtag, feeds and profiles of accounts, search results, cards from directories and maps. Publishing a post, leaving a like or messaging someone else's account through Apify is not possible: the platform only collects. The one thing that does change is your own Apify account: every run creates a run and a dataset and spends plan credits.

Usage is counted both by the platform and by the actor itself. The platform part is in compute units: 1 CU is 1 GB of memory for one hour, costing $0.2 per CU on the free plan and Starter, while the free plan gives $5 of usage a month with no card, 25 concurrent runs and up to 16 GB of memory per run. Some Store actors charge their own price per result on top, so a large collection is sized by its result count up front rather than launched blind. The dataset is pulled right after the run: on the free plan only the 10 most recent runs are kept, and only for 4 months.