Runs on JavaScript

JavaScript is the language of the web. Although there are JavaScript tools like puppeteer and cheerio, there was no universal framework that would enable large-scale high-performance web scraping and crawling of any website. Until now!

Automates any web workflow

Run headless Chrome or Selenium, manage lists and queues of URLs to crawl, run crawlers in parallel at maximum system capacity. Handle storage and export of results and rotate proxies.

Works locally and in the cloud

Apify SDK can be used stand-alone in your Node.js projects or it can run as a serverless microservice on the Apify Cloud. Get started with Apify Cloud

Easy crawling

There are three main classes that you can use to start crawling the web in no time. Need to crawl plain HTML? Use the blazing fastCheerioCrawler.
For complex websites that use React, Vue or other front-end javascript libraries and require JavaScript execution, spawn a headless browser with PuppeteerCrawler.
To control all aspects of your crawling, just use the bare bones BasicCrawler