NarenOnIt

Sunday, 29 March 2015

Blog post after a lonnng time. So I like to watch movies and series a lot. I started following so many series that it became difficult to keep track of what series is aired when and what episode am I on. It was almost like monday one, tuesday another, wednesday third one. And after that search and download it from somewhere( ;) torrents). So for a long time I had this idea that there should be some application that automatically takes care of everything from keeping track of all my series (which episode is aired when and download the next aired episode from torrents to my dropbox and notify me after that). But didn't find time to build one. But finally I was able to put something together.

So it all started when I came across the popcorn app, which is used for streaming movies and series from torrents. So I just got curious about how it works, and I came the torrent stream library, that the popcorn app used. It basically takes a torrent and generates a stream out of it, and you can write it to a file, stream it on a video player. So I thought if I could upload the stream to some cloud storage. Now dropbox provides a chunked upload feature that can take chunks of file, so essentially you could break down a very large file into chunks and upload it, or resume a partial download using this feature. So I took the chunks in stream provided by library and uploaded chunks to dropbox. It seems even google drive provides an api for it. But for now I have it only for dropbox. As I am open sourcing it, someone might be interested in extending it to other cloud storage services. So now I had the portion that uploaded a torrent to dropbox. Now as this library is in nodejs. But I am not so proficient in JS. So for the other part I used ruby. So I wrote a service that essentially monitors all series that you specify in configs and download them to dropbox using the nodejs script. It uses the tv maze api, to get the next episode for each series that it needs to download. So it essentially runs like a cron, every 2 hours it gets an episode of a series from api checks if that episode is out for download, if not it records it in a DbStore. (I used redis as my db store.) and sleeps. Next time when it runs it checks if the next episode in my db store is available. Now if it is then it downloads the file to dropbox, and then updates the next episode to download in db store.

Once both scripts were done, I thought of deploying it to my (free) heroku instance. However the challenge with that is how do I trigger it at specific times in a day. It does provide something called a worker dyno, but I don't have much idea if heroku provides a free plugin for scheduling scripts. So I came up with this. I run a web server (I used sinatra), and have a continuous background thread that actually runs the above task, then sleeps for x hours and then comes back, so essentially behaving like a scheduled task. And using this technique I could also add a route that simple dumps the logs of the task to monitor if everything went well. Now there was one more small issue with heroku. Heroku shuts down your app if there is inactivity for some time( I think 15 minutes.) But now I cannot afford my task shutting down as I don't have any way to track if it shutdown in between a download. So to keep it up all the time, I used a service (pingdom) that sends a ping to the app every 5 minutes. It is essentially used to track the uptime of a service. Ironically, I used it to keep my app up all the time.

I have open sourced it on github. You could check it out, and help me improve it. And use it at your own risk. It can be deployed in both modes as a webserver on heroku or a cron on some other cloud instance. Although to use it you need to create a app on your dropbox, authorize it and get a access token. And add the access token to dnode.js. It is the nodejs script that is actually used for downloading the torrent to dropbox. If you want yourself to be notified you could also create a mailgun account and create a key and add it to constants.yml that is used to send emails to your specified email. Otherwise you could comment the mail sending line. You also need a redis node on cloud, which is used as a db store. I used redislabs service which provides a 25 MB node for free. And to add the series you want to monitor you could add it to the list in constants.yml.

Sunday, 5 January 2014

Recently, I came across this gem called terminal-notifier, it allows you to send Mac OS X User Notifications. It also comes with a command-line tool. I was thinking of some interesting use-case for it. Then an idea struck, how about an application that notifies when I receive a mail contains specific text, or from some specific user.
Now I'll give a little background about how and why this app. In our office on any special occasion, people get chocolates or sweets and put it in cafeteria, and then mail in a common group to let people know. As it happens, I don't have the habit to check my mails very frequently, and often miss out on important mails at right time. So how about an script that checks for mails containing certain words or from a specific user.
So the task was simple check for the new emails and if they contain a match for the given criteria in my case it was if the mail subject or the body contains words sweets or chocolates then send a User Notification. To get the emails I used the gmail gem. Now this gem needs your username and password. But storing them as plaintext in the script is not smart from security point. So I was looking for a simple and reliable option to store and retrieve my username and password. So I used OSX keychain tool to store the sensitive information. To retrieve the email and password I used the commandline tool "security". Initially to store the username and password in OSX Keychain use,

Moving forward once I get the username and password, I logged into the gmail account by,

gmail = Gmail.new(username, password)

After logging in I searched through the unread mails for the search queries in both subject and body of mail. Once I find the mail containing the search term I fire a notification. Now as this gem didn't have a sound notification I used the system sounds file and played it using the afplay command as,

and then marked the mail as read so that it didn't come up when next time the cron runs. Finally to continuously scan through my inbox I set up this script as a cron that will run every minute,
*/1 * * * * ruby gmail_notifier.rb

Wednesday, 26 December 2012

I am basically a java developer and I recently started learning ruby and was amazed at the ease in which you can develop in it and elegance of the language. The easiest way to learn any programming language is to develop simple applications in it that use various features of language. So when going through the various features I came across Reflection and Metaprogramming in Ruby. A very powerful feature in ruby. It amazing how the eval function allows one to write and execute ruby code dynamically. So while thinking of an application using this feature I came up with the following application.
It is a simple No-SQL key-value db that stores data in hash in ruby. The first line declares a hash. Then the following part of script reads its own code and uses eval function to execute it declare the hash. Then depending upon the function called GET/SET it either retrieves the value associated with the key or sets a new key/value in hash. Then it simply stores the new hash in the source code.
Here is the source code for the ruby script:

Thursday, 13 December 2012

Today my friend introduced me to this cute little but addictive game named Little Alchemy.
So playing around with it for few minutes and looking at the time it took to find new elements I wondered how much time it would take to find all. Not having the patience to play whole game to determine all elements I thought why not do it the hacker style. So popped open the Chrome Web Inspector and looked around in the network tab to see what all data is sent by littlealchemy web page. I noticed it stores all data in application cache and retrieves from there. Looking around the js files for something interesting I struck gold when I found the logic in alchemy.js file. It seemed to make ajax call to 2 files /base/names.json and /base/base.json. Then I looked into these files and found that the mapping of all the elements was stored in base.json in array forms and all the names of elements were stored in names.json. After finding this it was a piece of cake to hack together a javascript code to display the combinations for all elements. So then I opened the javascript console and put together this piece of code to print the combinations for all elements and dumped it to a html file. You can check the output here.

Monday, 20 August 2012

You must have used the Google's Did you mean feature. It corrects any mistakes in your query automatically. It acts as a spell checker. Best part is it also handles proper nouns like Delhi, Sachin Tendulkar, or any place names or any movie title or series name which is not possible using any regular spell checker. So here is a trick to use Google's Did you mean feature in your own apps.
I wrote a small java code to demonstrate it. The function didYouMean takes as input the query and makes a request to Google search and parses the HTML response to get the suggestion (if any). Its written in java and uses the Jsoup library for parsing the HTML content. But understanding the logic it can be converted to any language.
If you take a look at the HTML response from Google you will find that the div tag with id "topstuff" contains the Did you mean suggestion by google if any. We simply extract that element and the parse it to plain text.Then we remove any extra characters in result and extract only the actual suggestion.
You can find the code for it on github.

Thursday, 16 August 2012

Few days back, some hardware issue with my hard drive left it irrepairable and all my data was lost including softwares, movies, music. (Thank god I use github to host all my projects).

Now loss of movies and softwares is no big deal. But finding one's song collection is difficult as one has a particular taste for music. But good for me all that music is stored in my IPod. But as we all know Apple doesn't allow one to copy music files from IPod to computer. So what I did was look around some way to get my music from IPod to PC. I opened the IPod in USB storage mode and looked at where the files are stored. I found all the files but they were stored with random (unrecognizable) names in it. So for starters I copied all those files to my PC. Then I looked for a way to get the files some understandable names. But naming around 1000 songs listening to each one is not the way a geek would do it. So then I wrote this java app that scanned through all mp3 files and read the ID3 tags to get the song title and renamed all songs. Cool so it scanned all files and did the task in few minutes. Then I thought why not modify it a little and let it scan through your entire computer and look for mp3 files, rename them using song title and store them in a single directory where the songs are categorized in subdirectories using the album name or artist name. In this way all the duplicates of a single mp3 would also be removed.

So this Mp3Manager is a console utility in java that takes as input the name of root directory to store all your music files and type of categorization to use to store the music files in directories(album name or artist). It is available on github

Read the readme.md to know how to use it. Let me know if you like it or if you would like any modifications to it.

Sunday, 22 July 2012

Past couple of days I had this idea in mind of implementing a autocomplete that uses fuzzy matching. For fuzzy matching of a partial string with all the strings in dictionary it uses the Levenshtein distance. Now finding the Levenshtein distance of given string with each string in dictionary is very inefficient. So I used the idea described here. Converted it to java and modified it to fit the needs.

It is available in 2 modes. First is the Http server mode in which you run the standalone server and call it directly from your ajax script.It returns data in json format with decreasing scores. For this you need to configure it by providing the file name containing the words for suggestion. To use it run the server and then use this url to get the results http://server-ip/autoc?word=word_for_autocomplete&tf=max_typos_allowed

Specify the word_for_autocomplete and the max_typos_allowed parameter.

The second is you can embed it directly in your application by first creating trie and then calling the getResult() method in CHandler class. See the source code for better understanding. A sample main method is given which uses the second method.