Kickstarter Search Scraper

jaroslavhejlek/kickstarter-search

Missing Kickstarter API? Need fresh Kickstarter news or list of best and finished projects? Try this new wrapper for Kickstarter search, which allows you to configure search filters and get the list of items from Kickstarter search page.

Actor - Kickstarter search

Kickstarter open API

Because Kickstarter public API does not provide structured output of search results, you can get list of Kickstarter news and projects with this app (actor).

Planned features

Unit tests

Compare current run with previous one and output only new projects

Kickstarter open source

You can manage search results in any languague (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Kickstarter Search Actor.
The code of this Kickstarter search actor is also open source, so you can create your own solution if you need.

INPUT

Input of this actor should be JSON containing filter specification. Allowed filters are:

Important considerations

Location - If you provide location to the Kickstarter search actor and it's set a string, it will run another actor to find
ID associated with the location. Once the location is found, it will be outputed in the console. Next time if you provide
the numeric ID instead of the location, it will not do the lookup again.

This option requires access to Apify Proxy, if you do not have access to it, then the actor will fail.
You can avoid this by going to Kickstarter, looking up anything and selecting the location you want. Then in addressbar of your browser you should see that the address now contains "woe_id" parameter. If you copy it's value into the location field of this actor. It will use it instead and you will not need to use proxy.

maxResults - Kickstarter will return a maximum of 200 pages, which is 2400 results. To get more results then this limit run multiple instances of this actor with more specific search terms.

datasetName - If you provide name of a named dataset then all everytime you run this Kickistarter search actor, it will clear the dataset and write new data into it. You can use this option for example if you want to use named dataset as RSS feed or if you are creating an API.

Run & Console output

During the run, the actor will output messages letting the you know what is going on.

If you provide incorrect input the Kickstarter search actor, it will immediately stop with Failure state and output an explanation of
what is wrong.

Once input is properly parsed the actor will load first page of results and output information about the
number of found results and how many results are remaining.

Every 10th page load will output prediction of how long it will take for the run to finish.

Dataset items

During the run, the actor is storing results into dataset, each project is a separate item in the dataset and it's
structure looks like this: