Mar 15, 2013

If you have files that cannot be easily shared from iTunes to your iPad device, and have access to Python,
simply open a command prompt or terminal in the directory you want to share and enter the command:

python -m SimpleHTTPServer 8000

You can substitute 8000 with any other free port.
Run ipconfig , or ifconfig if you are in Linux to find your local ip e.g. 192.168.1.2

From your iPad, open safari and go to
http://192.168.1.2:8000
to access and download the files to your iPad.

Mar 11, 2013

The Goal

To automatically perform keyword based searches at one of kickasstorrents categories, scrap relevant data that match our keywords and category, download the .torrent file and push it to transmission torrent client for auto downloading .
Setup a cron job to repeat the search at intervals, scraping and downloading torrents automatically.

Kickass Spider

Next we define the Spider, responsible for scraping data and storing TorrentItem information.
We instantiate it with two arguments, category and keywords. Create a new file kickass_spider.py in directory kickass/kickass/spiders:

The spider, simply parses the first page of torrents for a given category sorted by age - most recent first.
Then extracts torrent information and if a keyword matches a torrent title is added to a list of TorrentItems to be later processed by the pipeline defined in the next step.
The URL for a given category sorted buy time looks like this:http://kat.ph/usearch/category%3Abooks/?field=time_add&sorder=desc

Torrent Pipeline

All TorrentItems that were scrapped by the spider by matching the keyword list are passed to this pipeline for further processing. In our case, the pipeline will be responsible for downloading the actual torrent files and invoking transmission torrent client. Edit the file pipelines.py in directory kickass/kickass:

Next, we must declare the new pipeline in kickass/kickass/settings.py configuration file. Add the following entry:

ITEM_PIPELINES = ['kickass.pipelines.TorrentPipeline']

CURLing for the Torrent

The pipeline gets the URL path from the scrapped TorrentItem and calls script curl_torrent.sh.
The script follows the URL and the redirection to get the real filename of the torrent and donwloads it. Then, it runs transmission to start the download.
Place the script under your kickass/ directory.

Considerations

Finally, is recommended to modify the setting.py file under kickass/kickass directory to tune the spiders behavior and adjust logging. The following settings introduce a download delay of 5 seconds per request and limit concurrent requests to 1. That is to prevent hammering. Here is the complete file:

This is my first attempt at doing anything with python so i guess some things could be done more efficiently.
I am still experimenting with the language, and coming from a heavy Java background
i can confess that i am fascinated.
Also, i am pretty sure that spawning a new process with curl to fetch the torrent is not the most optimal way to do it.

Feel free to directly checkout the code at Github, and point out improvement / corrections.
I would very much appreciate that.