tag:blogger.com,1999:blog-36650646962287061042018-01-29T18:33:52.716+05:30KodeClutzunstoppable force hitting an immovable object... or something like that anyway.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.comBlogger288125tag:blogger.com,1999:blog-3665064696228706104.post-67862202408227658112016-01-01T10:45:00.000+05:302016-01-01T10:45:02.135+05:30Writing a simple database in rust, part 1<a href="http://birrell.org/andrew/papers/024-DatabasesPaper-SOSP.pdf">A Simple and Efficient Implementation for Small Databases</a> is an old paper that I've wanted to implement for a long time. It describes a simple application that embodies a lot of "systems" knowledge at the same time:<br /><ul><li>Using the filesystem as a persistence and synchronization mechanism.</li><li>Using a write-ahead log as a source of truth.</li><li>Creating crash tolerant (or crash-only) systems.</li></ul>While ideally you'd want to write this in C or C++, mucking around with strings and files isn't a very pleasant experience unless you are willing to pull in some third party libraries. Fortunately, in 2015 there is a langauge that is great for systems programs which ships with good libraries and offers several high-level constructs that make writing systems programs a pleasure. I've been <a href="https://bitbucket.org/nikhilm/rust-diet">playing around</a> with Rust and it seems like a good tool for the job.<br />The paper describes a key-value store with granular locking, write-ahead logs and snapshotting (log compaction). It then describes a Chubby/Zookeeper precursor built using the system. My aim is only to implement the former (for now). I will also try to add some ideas from more modern systems, like immutability, if I have the time and inclination.<br />The aim is to start from a very naive implementation that works and then make it more idiomatic, or have better names, or add performance improvements.<br />This first post covers creating the in-memory key-value store and the write-ahead log without compaction. Along the way we'll take a look at the design choices and how Rust helps to write safer programs.<br /><h2 id="the-in-memory-key-value-store">The in-memory key-value store</h2><a href="https://bitbucket.org/nikhilm/rust-simpledb/src/ab1d4f39d4950ec515c36f3fa86f7c40e0d3aa5e/src/lib.rs?fileviewer=file-view-default">Code</a><br />The in-memory KV store is really easy to implement in Rust. A built-in HashMap and generics makes it very similar to C++ or Java. Ignore the extraneous imports and the <code>impl Drop for SimpleDB...</code>, they are remnants of some experiments. The Path and I/O related operations follow in the next section.<br />In true low-level goodness, our database will operate only on arrays of bytes. There are no length restrictions on the size of either keys or values. A <code>Vec&lt;u8&gt;</code> captures this.<br />Our library exposes only one public struct, <code>SimpleDB</code>.<br /><br /><pre><code class="lang-rust">type SimpleCollection = HashMap&lt;Vec&lt;u8&gt;, Vec&lt;u8&gt;&gt;;<br />type Records = Arc&lt;RwLock&lt;SimpleCollection&gt;&gt;;<br /><br />pub struct SimpleDB {<br /> records: Records<br />}<br /></code></pre><br />We do want users to be able to use the same database from multiple threads. Rust's thread-safety is a diamond-in-the-rough for those new to the language. It demands a lot of thinking when designing the program (and grappling with weird errors sometimes), but when everything is shaken out, it makes sense and you get strong guarantees that there aren't data races waiting to happen.<br />We are going to skimp on per-key locking for now and lock the entire database for any operation. Inefficient, but simple. We will also only use a read-write lock unlike the read-update-exclusive lock used in the paper, simply because it already exists in the standard library.<br />When I started hacking on this, I wondered why locks and mutexes needed to be wrapped in <code>Arc</code> when they were supposed to be thread-safe on their own. It is because the lock itself is separate from the data it guards. Locks and mutexes provide safe internal mutability on the data they guard. They do so by leveraging the Rust ownership rules so that only one thread can ever acquire the contents for mutation. But the lock instance itself is also shared between threads. The rust compiler requires all objects to only be owned by <i>one</i> other object. The language provides various escape hatches that internally dip into <code>unsafe</code> code. The intention is that those hatches are vetted for safety while requiring higher level code to conform to strict requirements. For an object (in this case, the lock) to safely be 'owned' by multiple threads, it is wrapped in an atomic ref-counting pointer that will ensure that the lock itself is only deleted once, when all threads have released it. These are the sort of mistakes easy to make in other systems languages that Rust catches at compile time.<br />We also make sure that <code>SimpleDB</code> can be used on multiple threads:<br /><br /><pre><code class="lang-rust">unsafe impl Sync for SimpleDB {}<br />unsafe impl Send for SimpleDB {}<br /></code></pre><br />There is actually no need to make this explicit. The compiler can infer that all the members of <code>SimpleDB</code> are themselves thread-safe, so <code>SimpleDB</code> is too.<br />How Rust captures thread-safety via traits is fascinating, but too big to cover here in it's entirety. I recommend reading the <a href="http://doc.rust-lang.org/book/concurrency.html">concurrency</a> chapter in the book.<br />We expose a <i>static</i> "constructor" <code>open()</code> that does some other things and then <code>create()</code>s and returns an empty database.<br /><br /><pre><code class="lang-rust">let db = SimpleDB {<br /> // ...,<br /> records: Arc::new(RwLock::new(SimpleCollection::new()))<br />};<br />Ok(db)<br /></code></pre><br />The <code>get()</code> method acquires the underlying HashMap in read-only mode and returns a value if it exists. Rust's <code>Option&lt;&gt;</code> struct captures the possibility of emptyness and provides several "Promise-like" combinators to make decisions based on empty or non-empty.<br /><br /><pre><code class="lang-rust">pub fn get&lt;S: Into&lt;Vec&lt;u8&gt;&gt;&gt;(&amp;self, key: S) -&gt; Option&lt;Vec&lt;u8&gt;&gt; {<br /> self.records.read().ok().and_then(|guard| guard.get(&amp;key.into()).map(|val| val.clone()))<br />}<br /></code></pre><br />The lock's <code>read()</code> getter may fail if the lock is poisoned, so it returns a <code>Result</code>. RwLocks get poisoned if a writer panics while holding the lock. I found this idea fascinating, but we will ignore it for now and use some conversions to get access to the lock's guard. As long as the guard object stays alive, the lock is read-locked. <code>guard</code> also acts like a smart pointer (by implementing <code>Deref</code>) so we can call <code>get()</code> <br />The <code>put</code> and <code>delete</code> operations are similar. We acquire a write lock and insert or remove keys from the hash table.<br />In all these operations, we alias <code>io::Error</code> to our library's exported Error type. My error message <code>Boo</code> isn't very useful either. We'll leave this as it is for a while, and look at maintaining a Write-Ahead Log next.<br /><h2 id="adding-some-disk-operations">Adding some disk operations</h2><a href="https://bitbucket.org/nikhilm/rust-simpledb/src/5d1264cff6295e5f6d01f2d3a203667f25f51119/src/lib.rs?fileviewer=file-view-default">This is the revision of code</a> used for the following section.<br />A write-ahead log maintains a record of mutations on disk. The change is only reflected in memory after it is written to disk. This allows recovery to the last known consistent point from the application's point of view in case of a crash. Production ready databases allow tuning how often the write ahead log is fsynced to save on disk I/O cost vs. possibility of losing data. We do no such thing, every mutation will be fsynced.<br />Users of our library will pass a directory name to the <code>SimpleDB</code> constructor and all our I/O operations will be performed on files within this directory.<br /><pre><code class="lang-rust">let mut db = try!(SimpleDB::open("data-dir"))<br /></code></pre>Notice how we use Rust's traits to accept anything that is willing to convert to a reference to a Path. The <code>Path</code> type is unsized and meant to be used as a pointer. The <code>PathBuf</code> is sized, owned and mutable. The <code>AsRef&lt;Path&gt;</code> allows users to pass a PathBuf, a string or any custom type willing to deref to a Path. We then try to create the directory. Once that is done, we create an instance of the <code>Log</code>. The <code>Log</code> constructor creates a writable, append-only <code>operations.log</code>.<br />The lack of a <code>null</code> value in Rust leads to a pattern of creating <i>fully initialized</i> objects whenever possible. In C++ I'd leave several fields as <code>nullptr</code> in the constructor, then have setters. Anybody using those fields would have to check for non-null to avoid crashes. This often meant forgetting to check something and leading to lower quality software. In Rust you are either forced to have a <code>Option&lt;&gt;</code> member, which the compiler will force you to deal with at all callsites, or to use a <a href="https://aturon.github.io/ownership/builders.html">builder pattern</a> to set up even a complicated object as fully initialized. While more cumbersome when writing, it leads to safer code and nicer APIs.<br />Who should own the log? Since the log should also be protected by the mutex and we will operate on the log and hashtable together, it makes sense to put it into the <code>Mutex</code>. This makes it <i>compile-time</i> impossible to acquire the log without acquiring the mutex.<br />The <code>get</code> operation remains unchanged. <code>put</code> and <code>delete</code> now call the relevant methods on <code>Log</code> before making the change to themselves.<br />We'll use Rust's rich enums to represent log operations. I realized that this was overkill while writing this post, but changesets are immutable, so I'll simplify the code later.<br /><br /><pre><code class="lang-rust">enum LogOperation {<br /> Put(Vec&lt;u8&gt;, Vec&lt;u8&gt;),<br /> Delete(Vec&lt;u8&gt;),<br />}<br /></code></pre><br />The <code>Log</code> operations delegate to the <code>append</code> method which actually writes bytes to the file. The log format is simple. Each list of bytes is prefixed with it's length as a 8-byte, big-endian number. We use a <code>p</code> for a put and <code>d</code>for delete. So a sample log would look like (each character represents 1 byte):<br /><br /><pre><code>00000001p00000005hello00000006world!00000001d00000005hello<br /></code></pre><br />This allows fast sequential reading of the file. We will use the <a href="https://crates.io/crates/byteorder">byteorder</a> crate to perform the u64 conversion.<br /><br /><pre><code class="lang-rust">fn append(&amp;mut self, op: LogOperation) -&gt; io::Result&lt;()&gt; {<br /> let mut bytes = Vec::new();<br /> match op {<br /> LogOperation::Put(ref key, ref value) =&gt; {<br /> bytes.append(&amp;mut encode_vec(&amp;['p' as u8]));<br /> bytes.append(&amp;mut encode_vec(key));<br /> bytes.append(&amp;mut encode_vec(value));<br /> },<br /> LogOperation::Delete(ref key) =&gt; {<br /> bytes.append(&amp;mut encode_vec(&amp;['d' as u8]));<br /> bytes.append(&amp;mut encode_vec(key));<br /> }<br /> }<br /> self.file.write_all(&amp;bytes).and_then(|unused| self.file.sync_data())<br />}<br /></code></pre><br />We write out the bytes, then use <code>sync_data()</code> to force writes to disk on systems which support it. This gives us a key value store that can save data to disk. The problem is, a new run of the program doesn't load the data back from disk. We'll get to that in the next post.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-74658564858446903252015-07-29T11:54:00.000+05:302015-07-29T20:24:02.604+05:30Banner PeakNote: Image titles can be viewed by hovering over the image.<br />My interest in Banner Peak and Mt. Ritter began when I encountered them while hiking the John Muir Trail in August 2014. Immortalized by Ansel Adams, the 2 mountains framed by Garnet Lake and Thousand Island Lake are well known and frequently climbed.<br /><img title="A picture I took of Banner and Ritter from Garnet Lake in July 2014" src="https://www.dropbox.com/sc/afc1mhxcmdwxgzv/AACUddI_iHuyP7_cdaulXlGya?raw=1" width="100%" /><br />I snagged permits for Ediza Lake several months ago, with an intention to climb both Ritter and Banner via their scrambling routes from the Ritter/Banner saddle.<br /> <iframe width="100%" height="500px" src="http://caltopo.com/m/3A15"></iframe> The hardest problem in High Sierra climbing is balancing a technology job in Silicon Valley at the same time. Getting to the eastern Sierra takes forever from the Bay Area. I hate to drive that distance twice over a weekend and it prevents deeper exploration of backcountry peaks. Fortunately, things had worked out so that Banner Peak was the first in a series of adventures as I drove up along the west coast from the Bay to Whistler, BC. I left the Bay Area on Friday night and drove into Hodgdon Meadows campground at midnight. The rest of the crew was already there. After a decent sleep under pines, we packed up and drove to the Mammoth Lakes Visitor Center to pick up the permit. It was a ridiculously hot day and it seemed like the uphill wasn't going to be fun. On the other hand, I was carrying <a href="http://opticus.com/product/altice-chullo-2/">new glacier glasses</a> and was excited to try them on!<br />The Mammoth Ski Area and Devil's Postpile were very crowded and we had to wait for 2 shuttles before we could get on. With several pointy things on our packs, people gave us a wide berth. Taking the shuttle brought back not so fond memories of the JMT, when we had emerged out of Ansel Adams wilderness after a 17-mile day followed by a <b>boring</b> 14-mile day. Fortunately the River Trail is much nicer than the JMT as it winds its way up the Middle Fork San Joaquin. <br />After all these shenanigans to get to the trailhead, we started hiking at 12:54pm. The hot weather had led to typical Sierra afternoon clouds and there was occasional thunder as we began the hike. Fortunately the winds were carrying the clouds away from us. After about 2 miles the trail heads west to follow Shadow Creek and begins the 700ft ascent to the lake outlet. Shadow Creek was in full flow, creating remarkable waterfalls down quality granite.<br /><img title="Electrical activity" src="https://www.dropbox.com/sc/ej363mrifjrixzm/AACBkuj6mjXe3T4l_zDLTbcoa?raw=1" width="100%" /><br /><br /><img title="Shadow Creek waterfall" src="https://www.dropbox.com/sc/yyvfm1vdj4fzwhq/AADnxedTAwSRLnZHxP-LGkKca?raw=1" width="100%" /><br />Although Banner and Ritter require crampons and ice axe, they are not very technical. The best footwear is mid-weight hikers which are solid enough to accept strap-on crampons. Unfortunately I don't have something like that, so I wore trail runners and paid the penalty of carrying 3 season mountaineering boots. Still, I'd have hated to do the 8mi approach in mountaineering boots in such hot weather and I paid the price willingly.<br />We stopped for lunch near the Shadow Lake inlet and enjoyed Jamie's summer sausage. Earl was having some trouble keeping up and the mosquitoes got to work on us until he caught up. Here I made a mistake and started heading south on the JMT, fortunately realizing within a few hundred feet when the trail started downhill. After following the JMT for a short stretch, with the picture of the missing hiker still present at the post, we branched off to the Ediza Lake trail. This is great hiking with several cascades and tree cover cooling things down. Views of the Ritter range peek through under brilliant blue skies. Yellow glacier glasses also tend to pop blues, making it even nicer!<br /><img title="Ritter from Shadow Lake" src="https://www.dropbox.com/sc/8cewnz9m98fydcw/AAC3764FDZstUXDHgpKOgdCua?raw=1" width="100%" /><br /><br /><img title="Ritter from Ediza Lake" src="https://www.dropbox.com/sc/q9g4s1b661infji/AAAjWKNALU71tMfPHjyaJ6GRa" width="100%" /><br />We reached Ediza Lake around 4pm and debated if we should cross the shorter talus or the longer trail. The trail won and we made it to the west side quickly. There were already several tents pitched here. At this point we still had several hours of daylight and most of us were feeling good. We decided to put in another 2miles and 1000ft of effort to go higher up the valley leading to Ritter and Banner. This allowed us to<br />1) Be above the treeline with nicer views and more solitude. 2) Get a good idea of conditions on the route. 3) Not have to deal with finding the way up the valley in the dark the next morning. 4) Shave some elevation off the next day which was going to be much longer.<br />Our efforts were rewarded with an incredible waterfall!<br /><img title="Cascade on the way up the valley" src="https://www.dropbox.com/sc/2vh84ebz8oic6bz/AADP-sAmdruClZVyEMz5bfnEa?raw=1" width="100%" /><br /><br /><img title="Incredible backcountry waterfall" src="https://www.dropbox.com/sc/50x4ug25de46ft8/AABK1Pw7tBgfmtAcFpZtQ6CGa?raw=1" width="100%" /><br />Over all, I highly recommend the higher elevation campsite. No mosquitoes, solitude and great views. Around 10200 feet there are some reasonably flat spots about 200feet east of the small creek. This is right next to a series of big, flat boulders that shape the creek. After a ramen and scrambled eggs and bacon dinner, we prepped the packs for the next day and spent some time enjoying the absolutely stunning scenery. A huge cell had developed further down the valley and the setting sun was throwing up every shade of orange on it. I also spent some time walking Jamie through ice-axe and crampon fundamentals, since she was the only one in the group with no prior experience. Ed had decided not to attempt the summit so he wouldn't make his knee any worse.<br /><img title="Looking back down the valley" src="https://www.dropbox.com/sc/iyx2dkkkrfcze8s/AADDVqNF8hewz75m6vE2m_jMa?raw=1" width="100%" /><br /><br /><img title="Giant cloud" src="https://www.dropbox.com/sc/g25am5xgqszttju/AACvT3v5pST98XxDO0ocEqJka?raw=1" width="100%" /><br /><br /><img title="Looking at the route from the campsite" src="https://www.dropbox.com/sc/76kfsszzceh40ha/AADaCXECWu_9jMDzm1QH4opRa?raw=1" width="100%" /><br />The next day we started moving at 5:10am. Already the weather was very mellow. In fact it hadn't even reached freezing at night. Within a quarter mile it was time to put on crampons to deal with the hard snowfields lower down the valley. Arun found out that his rental crampons were not going to fit his shoes and had to bow out too. Earl would turn back after some time as he was having trouble keeping up again.<br /><img title="Heading up the lower, temporary snowfields" src="https://www.dropbox.com/sc/c6tr6p15ye4mnc9/AADujPDHgzB7nyo9MZHwToGna?raw=1" width="100%" /><br /><br /><img title="Iceberg Lake comes into view" src="https://www.dropbox.com/sc/22fn2wb07nmocgk/AADxKEu5Cp5YN62_YF-FZy4xa?raw=1" width="100%" /><br />Conditions were good and the rest of us - Dawn, Jamie, Michal and I - made good progress to the true snowfield. There was some snow melting already and slush was starting to show up in places. As you gain the snowfield you are treated to the first views of remote Iceberg Lake. The Minarets Loop has long been on my list and I should get to it soon. I'd like to reach a fitness level that would let me do it in 2 days while still enjoying it before I attempt it.<br />The wind picks up as soon as you are on the snowfield and once you enter Banner's shadow, things get really chilly. The couloir leading up to the the saddle is intimidating, but can't be more than 50 degrees at the steepest. Stay to the right so avoid occasional rockfall. We gained the saddle some time after 7. There was still plenty of snow on Ritter's North face and the saddle. The glacier on the west led directly to frozen Lake Catherine. There was no one in the mountains yet except us.<br /><img title="Couloir to the saddle" src="https://www.dropbox.com/sc/mltpwy4fi29hnyg/AADnZi5P7C2E4M6aWDdmQRXZa?raw=1" width="100%" /><br /><br /><img title="Ritter-Banner saddle" src="https://www.dropbox.com/sc/pbw4x91ovm7tnz7/AADYFWfj8LkDZj8JxzsmQ3E0a?raw=1" width="100%" /><br /><br /><img title="View of Ritter's North face from the saddle" src="https://www.dropbox.com/sc/1s3sf3vwt1igmmx/AADgeG3CtRTHgpDJgMB4EJqKa?raw=1" width="100%" /><br />We left our crampons at the saddle and started up the talus slog that is Banner's west face. It is unstable, tricky, and this early in the season had dangerously icy blocks as we went higher up. There were cairns marking the way occasionally, but we'd often have to take detours to avoid the ice. Fortunately the route finding is straightforward. Stay low and head towards the west ridge. It is important though to not actually get to the ridge. With some occasional class 3 climbing we were able to get close to the summit fairly quickly, but then had a decision to make. Should we get to the west ridge or was there a direct way to the summit?<br />I decided to head for the ridge and that culminated in two truly scary moments. One was mantling onto a icy rock with a fair amount of exposure. That allowed me to gain the ridge, but as I found out, that was too far. Although you can ride the ridge to get near the summit (close enough that you can hear and see your team which has stumbled upon the correct path while you've been trying to kill yourself!), there is a gendarme that blocks the way and I had to do another bit of tricky traversing (grumble, mountaineering boots, grumble) to get back on route. After an unnerving few minutes, I was finally on the summit at 9:15am.<br />The views from Banner Peak are astounding. Remote landscapes across the Sierra and Mono county merge with familiar landmarks. Thousand Island and Garnet Lakes are just a steep drop away, Donohue pass is up north, Mono Lake twinkles farther out and Iceberg Lake stands alone among the Minarets. To the west, Ansel Adams wilderness stretches out. Down south the peaks of the High Sierra go on forever and while I'm no great peak identifier yet, just knowing that Mt. Brewer and Whitney and others were out there adds a sense of grandeur to the adventure.<br /><img title="Looking towards Yosemite" src="https://www.dropbox.com/sc/h113g3u33gcml11/AABzDiSJbQZgbJVCYttUdIz7a?raw=1" width="100%" /><br /><br /><img title="Lake Catherine and the west side Glacier" src="https://www.dropbox.com/sc/rtvu0p0hwr46rr6/AADAP1-4lYO7bifQnqWEW5tya?raw=1" width="100%" /><br /><br /><img title="Thousand Island and Garnet Lakes" src="https://www.dropbox.com/sc/jtdm4zwezw5wrrf/AADen3p-LelCp6RQGEvMlQiIa?raw=1" width="100%" /><br /><br /><img title="Nydiver, Ediza and Iceberg Lakes" src="https://www.dropbox.com/sc/u09wl0inviuugux/AAA8LcpeyL_cj7jXdYOMWvh9a?raw=1" width="100%" /><br /><br /><img title="Mt. Ritter from Banner's summit" src="https://www.dropbox.com/sc/peyslduosst3xqx/AADGaePOgx6r9jOhzDRzVYtqa?raw=1" width="100%" /><br />The elephant in the room is of course Mt. Ritter. It was painfull obvious that we wouldn't be able to climb it. Going up Banner had been slow and descending would be even slower due to the unstable rock. Facing over 13mi of hiking from the summit to the car, the prospect of hiking up another 1000ft of unstable rock was unappealing even if we had had the time.<br />After signing the register Michal headed down while the rest of us enjoyed the summit a little longer.<br />The way down was uneventful except for the occasional pile of rocks someone would send tumbling down. If you are experienced enough in the Sierra to attempt Banner, you should already be aware of spacing people out and staying out of fall lines. Don't forget a helmet! Back at the saddle we noticed a trio halfway up Ritter. An enterprising lone skier made it to the saddle from Lake Catherine and skied down. We put the crampons back on and downclimbed the couloir. This was not fun at all. The snow was sticky and unstable and the going was slow. Slush and dangerous potholing on the lower snowfield was followed by a talus descent until we finally arrived back to camp around 12:30.<br />Ed and Ankur had hiked on ahead so they could take it slow, while Earl was waiting for us and left as we arrived. Everyone was exhausted, but there was enough adrenalin to keep things in motion and we packed up and started hiking out by 1pm.<br />The descent was brutal on the knees and took forever. We took the talus side of Ediza Lake, crossed the outlet via a series of very delicate steps, and trundled quickly down the rest of the trail. I felt fairly empty inside, so imagine my surprise when a hiker going up remarked that I looked really fresh. The more adventures you do the more abuse your body seems to withstand, but the mind doesn't always calibrate itself correctly, especially when the weather is bad or the stomach is rumbling.<br /><img title="Crossing the Ediza Lake outlet required some good balance" src="https://www.dropbox.com/sc/1oubo38pn4ozam4/AADyg0rF6PnGG_vO2hP6rd22a?raw=1" width="100%" /><br />Tackling the 300 feet of uphill specially reserved for the very end of the trail we reached Agnew meadows after 11hr and 39min of being on the move for the most part. After a shuttle dance we got to the cars around 6pm.<br />After a quick dinner at Pita Pit, everyone else left to drive back to the Bay Area. I was heading to Reno, for the first of several nights not spent in my own bed. I stopped near Mono Lake to catch sunset, before driving the rest of the way along beautiful 395 (the second time within a month) and collapsed at the Nugget around 10:30.<br /><img title="Sunset at Mono Lake" src="https://www.dropbox.com/sc/ybzxe3ly3sspyto/AAD_tnoTuGq6s_hhQqfH4Qjca?raw=1" width="100%" /><br /><h4>Hiking stats according to Suunto Core (includes breaks)</h4>Trip date: June 13-14, 2015<br /><ul><li>June 13 - start 12:54. Car to camp: 6hr 15min 36sec</li><li>Peak elevation: 3044m</li><li>Ascent: 617m, avg 6m/min</li><li>Descent: 115m 5m/min</li></ul><ul><li>June 14 - start 5:10. Camp to summit to car: 11hr 38min 58sec</li><li>Peak elevation: 3871m</li><li>Ascent: 954m avg 7m/min</li><li>Descent: 1457m 9m/min</li></ul> Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com2tag:blogger.com,1999:blog-3665064696228706104.post-81501406098535250842015-02-11T23:05:00.002+05:302015-02-11T23:05:47.145+05:30Getting Vidyo running on Archlinux & Plasma 5I recently upgraded to Plasma 5.2 based upon <a href="https://kver.wordpress.com/2015/01/22/plasma-5-2-the-quintissential-breakdown/">this glowing review</a>. While the transition was smooth for the most part (apart from minor KWin issues), Vidyo started to segfault. The Plasma system tray no longer supports older system tray protocols. I'm not aware of the details, but VidyoDesktop would complain about not being able to set a system tray icon and segfault. The fix is to install the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">sni-qt</span> package from the extra repository.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com3tag:blogger.com,1999:blog-3665064696228706104.post-44396677344599918962014-07-27T07:10:00.001+05:302014-07-27T07:12:18.601+05:30ServiceWorkers in Firefox Update: July 26, 2014<b>(I will be on vacation July 27 to August 18 and unable to reply to any comments. Please see the end of the post for other ways to ask questions and raise issues.)</b><br />It’s been over 2 months since my last post, so here is an update. But first, <br />a link to a latest build (and this time it won’t expire!). For instructions on <br />enabling all the APIs, see <a href="http://blog.nikhilism.com/2014/05/serviceworker-implementation-status-in-firefox.html">the earlier post</a>.<br /><br /><h3><a href="http://nikhilism.com/serviceworker-build-20140726/">Download builds</a></h3><div class="se-section-delimiter"></div><h3 id="registration-lifecycle">Registration lifecycle</h3>The patches related to ServiceWorker registration have landed in Nightly <br />builds! unregister() still doesn’t work in Nightly (but does in the build <br />above), since <a href="https://bugzil.la/1011268">Bug 1011268</a> is waiting on review.<br />The <a href="https://bugzil.la/1043701">state mechanism</a> is not available. But the bug is easy to fix and <br />I encourage interested Gecko contributors (existing and new) to give it a shot.<br />Also, the ServiceWorker specification <a href="https://bugzil.la/1043004">changed</a> just a few days ago, <br />so Firefox still has the older API with everything on ServiceWorkerContainer. <br />This bug is another easy to fix bug.<br /><h3 id="fetch">Fetch</h3>Ben Kelly has been hard at work implementing Headers and some of them have <br />landed in Nightly. Unfortunately that isn’t of much use right now since the <br />Request and Response objects are very primitive and do not handle Headers. <br />We do have a spec updated <a href="http://fetch.spec.whatwg.org/">Fetch</a> API, with <br />Request, Response and fetch() primitives. What works and what doesn’t?<br /><ol><li>Request and Response objects are available and the fetch event will hand <br />your ServiceWorker a Request, and you can return it a Response and this will <br />work! Only the Response(“string body”) form is implemented. You can of <br />course create an instance and set the status, but that’s about it.</li><li>fetch() does not work on ServiceWorkers! In documents, only the fetch(URL) <br />form works.</li><li>One of our interns, Catalin Badea has taken over implementing Fetch while <br />I’m on vacation, so I’m hoping to publish a more functional API once I’m <br />back.</li></ol><h3 id="postmessagegetserviced">postMessage()/getServiced()</h3>Catalin has done a great job of implementing these, and they are <a href="https://bugzil.la/982726">waiting for <br />review</a>. Unfortunately I was unable to integrate his patches into the <br />build above, but he can probably post an updated build himself.<br /><div class="se-section-delimiter"></div><h3 id="push">Push</h3>Another of our interns, Tyler Smith, has implemented the <a href="http://lists.w3.org/Archives/Public/public-webapps/2014AprJun/0223.html">new Push <br />API</a>! This is available for use on navigator.pushRegistrationManager <br />and your ServiceWorker will receive the Push notification.<br /><div class="se-section-delimiter"></div><h3 id="cachefetchstore">Cache/FetchStore</h3>Nothing available yet.<br /><div class="se-section-delimiter"></div><h3 id="persistence">Persistence</h3>Currently neither ServiceWorker registrations, nor scripts are persisted or available offline. Andrea Marchesini is working on the <a href="http://bugzil.la/984050">former</a>, and will be back from vacation next week to finish it off. Offline script caching is currently <a href="http://bugzil.la/931249">unassigned</a>. It is fairly hairy, but we think we know how to do it. Progress on this should happen within the next few weeks.<br /><h3 id="documentation">Documentation</h3>Chris Mills has started working on <a href="https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorker_API">MDN pages</a> about ServiceWorkers.<br /><div class="se-section-delimiter"></div><h3 id="contributing-to-serviceworkers">Contributing to ServiceWorkers</h3>As you can see, while Firefox is not in a situation yet to implement <br />full-circle offline apps, we are making progress. There are several employees <br />and two superb interns working on this. We are always looking for more <br />contributors. Here are various things you can do:<br />The ServiceWorker <a href="http://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html">specification</a> is meant to solve your needs. Yes, it <br />is hard to figure out what can be improved without actually trying it out, but <br />I really encourage you to step in there, ask questions and <a href="https://github.com/slightlyoff/ServiceWorker/issues">file <br />issues</a> to improve the specification before it becomes immortal.<br />Improve Service Worker documentation on MDN. The ServiceWorker spec introduces <br />several new concepts and APIs, and the better documented we have them, the <br />faster web developers can use them. <a href="https://developer.mozilla.org/en-US/docs/MDN/Getting_started">Start <br />here</a>.<br />There are several Gecko implementation bugs, here ordered in approximately <br />increasing difficulty:<br /><ul><li><a href="https://bugzil.la/1040924">1040924</a> - Fix and re-enable the serviceworker tests on non-Windows.</li><li><a href="https://bugzil.la/1043711">1043711</a> - Ensure ServiceWorkerManager::Register() can always <br />extract a host from the URL.</li><li><a href="https://bugzil.la/1041335">1041335</a> - Add mozilla::services Getter for <br />nsIServiceWorkerManager.</li><li><a href="https://bugzil.la/982728">982728</a> - Implement ServiceWorkerGlobalScope update() and <br />unregister().</li><li><a href="https://bugzil.la/1041340">1041340</a> - ServiceWorkers: Implement [[HandleDocumentUnload]].</li><li><a href="https://bugzil.la/1043004">1043004</a> - Update ServiceWorkerContainer API to spec.</li><li><a href="https://bugzil.la/931243">931243</a> - Sync XMLHttpRequest should be disabled on ServiceWorkers.</li><li><a href="https://bugzil.la/1003991">1003991</a> - Disable https:// only load for ServiceWorkers when Developer Tools are open.</li><li><a href="https://bugzilla.mozilla.org/buglist.cgi?quicksearch=ServiceWorker&amp;list_id=10835234">Full list</a></li></ul>Don’t hesistate to ask for help on the <code>#content</code> channel on <code>irc.mozilla.org</code>.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-79035785647630110642014-05-16T23:32:00.001+05:302014-06-27T01:39:52.587+05:30ServiceWorker implementation status in Firefox<br />ServiceWorkers are a new web platform feature that allows sites to use JavaScript to provide a better offline experience, and eventually various other 'background service' capabilities. Jake Archibald <a href="http://jakearchibald.com/2014/service-worker-first-draft/">has a good article</a> on his blog explaining them further.<br /><br />I've been working on the Gecko implementation for a while now, helped by several other Mozillians and some of the first patches are now beginning to land on Nightly. Interest in trying out a build has been high within both Mozilla and outside. I've been procrastinating about a detailed hacks article, but I thought I'd at least share the build and what Firefox supports for now and what it doesn't.<br /><br /><ol><li><a href="https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/try-builds/nsm.nikhil@gmail.com-298797979eb9">Download the build</a></li><li>Run it using a clean profile.</li><li>Go to about:config and set "dom.serviceWorkers.enabled" to true.</li></ol><br /><br />Supported:<br /><ul><li>Registration, installation, activation, unregistration. Update() is not according to the spec, the scripts themselves are not cached offline, so don't actually try to make offline apps yet.</li><li>Intercepting navigation and fetch events and replying using a simple SameOriginResponse object.</li><li>Intercepting Push API events.</li></ul>Not supported:<br /><ul><li>Cache API</li><li>fetch() - You'll have to use XHR in the worker.</li><li>ClientLists and postMessage() to talk to windows</li><li>Persistence - ServiceWorker registrations are currently not stored across restarts.</li><li>Anything that throws an error :)</li><li>No devtools support right now. </li></ul><br />SameOriginResponse is a relic of an earlier version of the spec. Similarly the Request object received from the fetch event is not exactly up to speed with the spec.<br /><br />I have two sample applications on Github to try out the build. Note that ServiceWorkers require HTTPS connections. For development, please create a new boolean preference in about:config - "dom.serviceWorkers.testing.enabled" and set it to true. This will disable the HTTPS requirement.<br /><br /><a href="https://nikhilm.github.io/simplepush-test-app/">Push API Test</a> - Remember to run the Firefox build from the command line since this demo prints stuff to the terminal. Click the register button to sign up for Push notifications. To perform the push, copy the unique URL and run<br /><br /><pre class="code"><code><br />curl -vX PUT 'URL'</code></pre><br />The ServiceWorker will be spun up and a 'push' notification delivered.<br /><br /><a href="https://nikhilm.github.io/serviceworker-experiment">Simple Network Interception</a> - This example will load an iframe with some content. Refresh the page and the iframe's content should change and the browser should show an alert dialog. These changes are done by the ServiceWorkers (there are 2, one for the current scope and one for the sub/ scope) intercepting the requests for both "fakescript.js" and the iframe and injecting their own responses.<br /><br /><h3>Contributing</h3>We could always use help with the specification, implementation and testing.<br /><br />The <a href="http://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html">specification</a> is improved by <a href="https://github.com/slightlyoff/ServiceWorker/issues">filing issues</a>.<br /><br />The tracking bug for the Gecko implementation is <a href="https://bugzil.la/903441">Bug 903441</a>. There is a <a href="http://hg.mozilla.org/users/nsm.nikhil_gmail.com/service-workers2/">mercurial patch queue</a> for patches that have not landed on trunk.<br /><br />I (nsm)&nbsp;am around on the #content channel in irc.mozilla.org if you have questions.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com1tag:blogger.com,1999:blog-3665064696228706104.post-68710542038351426792014-05-15T06:42:00.001+05:302014-05-15T21:02:32.672+05:30Trip Report: Canyonlands Backpacking and Packrafting<p><em>Lacking in pictures because even in 2014, embedding images from alternate sources in a blog post is ridiculous UX fail!</em></p> <p>In March (19-25), Raj, Michaela, Roxana and I went on a 6 day trip to Utah. The itinerary was to fly into Salt Lake City, drive to Moab, do a 4 day backpacking and packrafting trip in Canyonlands NP, spend a day in Arches NP and head back. It was one of the best trips I’ve done and a splendid (and early!) start to this year’s trips.</p> <p>The backpacking trip was based on one covered in <a href="http://www.backpacker.com/adventure-travel-north-america/destinations/17869">Backpacker magazine</a> but modified to fit our schedule and capabilities. We’d start from the Elephant Hill TH in the Needles District, hike down to the Colorado at Spanish Bottom and cross it by packraft. Then hike up to the Doll House, hike north to the Harvest Scene, head east to Water Canyon, then go off the established trail to follow Water Canyon to the Green River. From there we’d raft down to Spanish Bottom and hike out. It was some 36 miles of hiking and 8 miles rafting over 4 days.</p> <iframe src="http://caltopo.com/map?id=0P52" height="500px" width="500px"></iframe> <p>We used Alpacka rafts rented from <a href="http://packraftrentals.com/">Packraft Rentals</a> at $150 for the raft, paddles and PFDs.</p> <p>Having visited Moab before, Michaela was our “guide”. The Moab Brewery for dinner was a very good recommendation! With the final civilized meal consumed, we drove 2 more hours to the Canyonlands Needles district. All the walk-in campsites in the park were taken (2 people actually refused to share a campsite with us, which was very rude considering how much space there was per site!), so we drove back 20 minutes to the privately run campsite called Needles Outpost ($20/night).</p> <p>It turned out to be a very cold night, making it hard to leave the sleeping bag. The reward for getting out was the incredible views of canyon country, the first time we saw it after driving in in the dark. The Needles Outpost itself is in a fairly large plateau so that you can look far away to the Needles rising beautifully in the west, while also seeing the pinnacles of Indian Creek and more distant areas in the east. While I missed sunrise, reds, browns and yellows were still shimmering and changing in the early morning Sun.</p> <p>At the ranger station we found out that the established campsites throughout the park are only for car campers, and backpackers have to practice dispersed camping. While not a deal breaker, it was surprising, since the website doesn’t mention this. We also spoke to the ranger at Hans Flat over the telephone who told us that there were no reliable sources of water except the Colorado and a few springs near the Harvest Scene, so we tanked up with 4 liters each.</p> <p><strong>NOTE</strong>: WAG bags are a good idea in Canyonlands since the climate is too dry for waste to decompose quickly. Please use them! They are <em>required</em> by law when camping near rivers.</p> <p>We eventually set out at 11 from the car. Boy were the packs heavy with several days of food, 4 liters of water and 4kg of raft! The journey from the Elephant Hill TH to the river is relatively uneventful, though very beautiful. The trail follows the 4 wheel drive road to Devil’s Lane, where it goes westward into the sheer wall ahead. Go into the clump of trees to climb onto the wall and follow it to get on the trail for Cyclone Canyon and Lower Red Lake.</p> <p><img src="https://farm6.staticflickr.com/5325/13912660510_af9e216e9f_n.jpg" alt="" title="Looking back to the road toward's Devil's Kitchen. Devil's Lane runs from left to right"></p> <p>The last 1.5 miles of the Needles hike are very different from the beginning. The canyons become taller and narrower, and the rock is now a dark red. Here it seems like the original trail has been closed due to rockfall, because the National Geographic map is not congruent with the new trail which climbs along the southern canyon walls before beginning a steep descent into Lens Canyon. Then it’s a mile of walking on sand, making your own path along the dry river bed to Spanish Bottom. We reached the bottom around 5pm. The Needles side of Spanish Bottom is a expansive beach and while everyone wanted to relax I refused to let them do so. The N-S orientation of the river at this point and the 1400ft high cliff walls on the western side means that sunset comes early. So at 5:30 we began inflating our rafts. While doing so, a group of real motorized rafts went down the Colorado as a guided rafting trip. Raj’s raft had developed a puncture and needed a second inflation after patching it up. Frankly, it was too much work to get across 30-40m of calm water. My thoughts about the rafts were to improve only a little over the next few days.</p> <p><a href="https://www.flickr.com/photos/code_kreator/14096043861"><img src="https://farm3.staticflickr.com/2927/14096043861_3786a763c9_n.jpg" alt="DSC00297" height="240" width="320"></a></p> <p>I should point out one constant nagging feeling I had the whole trip. This whole place is <strong>made for climbing</strong>. Everywhere, the sandstone has straight cracks running through it, towers that rise above the canyons and even the occasional boulder at a canyon bottom is very inviting. It is isolated though, and I’d imagine lugging equipment and supplies would be as bad as hauling our rafts. No wonder climbers stick to Indian Creek.</p> <p>Slightly downstream, where the Colorado begins to turn east, there are noticeable steps on the western bank and an obvious campsite. The steps are also in an area of exceptionally still water so there isn’t any fear of being swept downstream while trying to land. The 30ft length of cord I’d brought along proved immensely useful this entire trip. Hooking it to a rock allowed me to clamber out of the raft without worrying about my boat and pack, and it was our anchor for the last night along the river. The Green and the Grand (<a href="http://en.wikipedia.org/wiki/Colorado_River#Naming_controversy">now called the Colorado</a>) carry tons of silt down their course and that and the sandy banks means that transitioning back to land is a muddy affair.</p> <p>Once all 4 boats had been pulled in and no one had been lost to the river we setup camp. The rangers had advised us that Colorado water was far too silty to use filters, but with a Sawyer Mini and carrying the backflow syringe, I didn’t face any problems. Using the cord tied to a Smartwater bottle, and using my foot to hold the bottle down allowed retrieving water without falling into the river. Tomorrow would be a long day with no water again till the very end, and so I filled 7 liters to last through the night and day.</p> <p>At this point Roxana couldn’t find her tent and we figured at best it was left behind in the car, and at worst lost, but that she’d be spending the next 3 nights without one. Which isn’t a bad deal at all when the weather is great. For tonight she’d be sleeping over her inflated packraft, her fancy solar powered lamp on a stake next to her.</p> <p>Although we were tired and dirty, the twin annoyances of the desert - dryness and gritty sand in everything - had not yet begun to gnaw on us. We went to bed in high spirits.</p> <p>Day 2: The first night in the wilderness was quite warm, with temperatures probably hovering around 5C. This was to be true all 3 nights and I can’t fathom why that first night at Needles Outpost had been so cold.</p> <p>We woke up around 7 (the alarm was for 6, but I was tired and it was still dark outside!) and were ready to roll by 9:30. Packing the rafts had taken up at least 30 minutes of our time. Today seemed like it would be the day of most effort. We had to carry the rafts and plenty of water and most of our food up 1400ft and 2 miles followed by 3 miles on a very gentle uphill to Chimney Rock. As we started up the southwestern side of Spanish Bottom, we encountered a group of day hikers from the guided trip we’d seen last night. They had camped before the start of the Cataract Canyon rapids and were on a quick jaunt to the Doll House before (I guess) hitting the rapids. Compared to them, we looked like pack mules! The load on our backs was - our standard pack, the middle halves of the oars sticking out of the side pockets and the paddles in the outer pocket of the pack. The PFD was hooked onto the oars, and then the raft lashed to the bottom or top of the pack. With my relatively fragile pack, I also carried my water in the front in a daypack to avoid overloading it.</p> <p>Nevertheless we completed the steepest of the ascent in a swift 45 minutes (considering our loads) and were at the Doll House within the hour. Here we could not find the restrooms we hoped for. Carrying waste around is nobody’s idea of fun, so I was disappointed at having to use my WAG bag a day before I would’ve hoped to. Other than that quibble, the Doll House is an incredible view of stacked boulders and cylindrical rock formations surrounding a huge meadow of prickly pear and other desert shrubs. If you look northeast from certain points you can see the distant La Sals, their snow covered peaks a stark contrast to the desert around us for miles in every direction. We encountered the 4WD road that heads northwest to Chimney Rock. Once it leaves the Doll House this road is singularly uninteresting within the immediate surroundings. Standing Rock on the west and Chimney Rock in the north beckon, but canyon country is farther to the east and each canyon is too small, yet the sheer number too big for the mind to really comprehend it.</p> <p>The 4WD road diverges from Chimney Rock the way mountain trails do, before finally turning back to meet it just as you are starting to despair from the weight. Chimney Rock to me wasn’t very interesting. It was impressive that the spire had formed, but now it just sat there through dry millenia, acting as a waypoint in the confusing mass of canyons and mesas. There were several caches of water left here by other parties. We found a nice cave and stashed our rafts and associated implements there, then sat at the Chimney Rock Trailhead and had lunch. It was now around noon.</p> <p>A full stomach and a lighter pack are a huge boost to morale and speed, and the fact that things literally went downhill from here really served us well. From Chimney Rock, 3 trails head out. One goes east to take the long way back to the Doll House, and two go north. Contrary to what the National Geographic map says, both trails do not start west of Chimney Rock. Rather, the westward trail is on the west of the rock and the eastward trail is on the east side. We followed the one on the west. Head a little further north and true canyon country immediately opens up. While the views had previously been dominated by red rock, here the tops of the canyons were all yellow while the sides were darker brown and it looked spectacular. The Chocolate Drops are quite obvious and Maze Overlook ascends behind them. The trail immediately goes around one small canyon and descends into the next. Descending the canyon is a very interesting experience. All the Canyonlands trails try to be as direct as possible, which meant we were either descending steep but smooth sandstone walls by the magic of rubber soles, or going down big ‘steps’ where our packs were a hindrance. Cairns mark the descent, but there are plenty of other rocks scattered around, so be careful! During the descent we crossed a few pools of water, but not expecting to find them, we had all the water we needed. Once at the bottom, there begins 3 miles of walking on the sandy river beds of long dry rivers. I have no idea whether these rivers are still active, but there at the bottom it sure didn’t seem like any substantial flow had occurred on any human timescale. About half way in, the trail joins up with the one coming from Lizard Rock. While we were sure that we were at the right canyon, we could not find the actual trail. Our destination of course was north, and that trail was easy to find. Following the meandering river is an exercise in patience, and it seems I wasn’t the only one growing irritable about that. The trail cuts through the banks at nearly every turn, and this compacted sand was almost a pleasure to walk on compared to the loose, deep sand in the river bed! Be careful though, as prickly pears and other spiky plants might stab your feet. Larger juniper and cottonwood trees lined this and several other river banks in this part of the park.</p> <p>Continuing on, we were vexed when Harvest Scene didn’t show up where it was supposed to. The map was wrong again, Harvest Scene is not on the east of the trail but actually on the wall to the ‘left’ (northwest). I must say that I was expecting some sort of NPS sign or similar, but there was none, and the scene, while beautiful, is at eye level and small. I had had the notion that it would be something majestic, since everyone seemed to talk about it, and in that sense I was disappointed. The paintings themselves are incredible though, and almost alien in there depictions.</p> <p>The ranger had said that camping was prohibited 1 mile around Maze Overlook, and that Harvest Scene would be a good place to camp (not near it obviously), but we wanted to press on to where the trail hits a wall and becomes northwest-southeast, the left side going to Maze Overlook and Horse Canyon, the right side going back towards Chimney Rock. There is a ‘Spring’ marked on the map here and confirmed by the ranger (Cynthia?) as flowing. We reached this intersection around 5pm. I come from the wetter areas of the world and my usual stomping grounds are the Sierras, so I thought a spring would be emerging from the rock and trickling as a little waterfall of clear water. Instead, the spring was just a muddy trickle, that was lost underground a few hundred feet east. Fortunately, heading a few hundred feet west, the water had collected in a small ditch. It is likely that in late spring this water will be stagnant and possibly smelly (unless it dries before that), but right now it was cool and clear and I was okay with drinking it.</p> <p>This ditch was just to the west of the trail intersections where the next canyon on the west starts, seperated from the Harvest Scene trail canyon by a small spur. In this canyon, ascend just a few feet up the river bed (south) and a fairly clear path leads uphill (westward) to a rock platform. This platform is a great campsite. It’s neither high nor low, fairly close to water and on hard ground. The rock can sleep upto 3 people (don’t move the kitchen platform in the center! :)) and two more can fit on the soil in the clearing. There is a wall to the southwest, but there are great views of the southwest-northeast canyon and the effects of sunset and sunrise are gorgeous. In <em>Beyond the Hundredth Meridian</em>, Wallace Stagner quotes someone (too bad I can’t find it right now), about how distances play tricks in the Plateau Province, so that the air itself seems thicker and things seem more crisp, and I had to agree.</p> <p>While in the morning, and even until I reached the Harvest Scene, I had been enthusiastic about a quick hike up to Maze Overlook, we were all exhausted by this point and I too chose to stick to camp chores. I swear I poured out a half a pound of sand from my shoes and socks! Meshy, low-cut trail runners and no gaithers are not good desert wear. I should try sandals next time.</p> <p>Roxana and me shared the tent today as she did not have the raft and other items as insulation.</p> <p>Day 3: We set off at 7:30 today. Today we would follow the eastern fork of the trail back to Chimney Rock. We continued for 2 miles down the canyon system we had been in since yesterday. Here the trail is easy to miss among the sandy bottom and for a while the cairns are far apart. It is better to use the more detailed Maze district map. The major landmarks are a few islands where the trail proceeds to their left. Eventually the trail steepens and involves some scrambling as it rises out of the canyon. There is one class 3 (easy, but exposed and tipsy when wearing a pack) move to gain a ledge, then head left to climb over a barrier. From here, bound up the cairned trail and you eventually emerge on a mesa that will lead south to Chimney Rock.</p> <p>This part of the trail is easily the most enjoyable of the trip. Minimum elevation change and great slickrock make for easy walking, allowing you to enjoy the views of the canyons on both sides, the Chocolate Drops and the rest of Maze country spread out in the west and the La Sals gracing their presence in the east. It is possible to make really good time here and we were at Chimney Rock a little before noon.</p> <p>Take a lazy break here, as the next part of the journey is equally inspiring and terrifying.</p> <p>After lunch, we retrieved our caches, bringing the packs back to their “stupid heavy” weight and headed east on the trail that goes back to the Doll House. The trail descends into a slickrock canyon that is ridiculously fun to descend. Teetering masses of stone serve as stairs in certain parts, while other parts require some scrambling, but the footing is sound and the journey is quick. Our luck of cloudy mornings had held out today as well, but now the sun had broken through and it was hot at the bottom of the canyon. Continue to follow the well marked trail along the canyon. The Shot Canyon stream had pockets of water when we were there, and in a wet year may even be flowing. Here the trail again cuts through meadows rather than follow the meander and it is a good idea to keep an eye out for cairns. At this point my raft was making a big deal of trying to fall of my pack.</p> <p>Eventually the trail will turn southeast (it has been heading northeast from Chimney Rock down Shot Canyon). This turn takes longer to arrive than it looks on the map, but again, the trail is well cairned and you should avoid worrying. Here you’ll climb up several hundred feet, only to descend back down the other side into Water Canyon. On the way down, the trail enters a narrow hanging canyon. When it exits in a couple of yards, it is to a sheer drop into Water Canyon. This is a beautiful spot to spend some time. Water Canyon is huge, and green, and your seat is in a coveted place. Once satisfied with the view, turn right to follow the ledge, which descends and takes a U-turn before descending to the canyon floor.</p> <p>True to it’s name, Water Canyon has a fairly large stream and water is assured. At the bottom, <em>leave the trail</em> and head east, following the course of the stream. Although not appearing on any maps, there is an obvious trail down to the Green. Obvious does not mean easy though. About 1.5 miles before the intersection with the Green the stream careens off the cliff. </p> <p>Here the trail goes right and takes an adventurous path down over extremely rotten rock. This sketchy trail should be approached with extreme caution. There are several places where it narrows to have only enough place for one foot. Gravel and sand abound to make footing unsure. Talus fields make up whatever is left. Remember to avoid hiking below someone’s fall line as it is very easy to kick of rockfall. The only consolation is that cairns mark the way.</p> <p>Set aside at least an hour for this half mile, 500 ft descent to the canyon floor. Almost at the bottom of this adventure, the stream collects into a very inviting pool. Unfortunately, short on time and having no beta of what lay ahead, I forced the others to abandon this luxury, but you may want to make use of it.</p> <p>Once at the bottom, follow the stream until it starts flowing over solid slickrock and the course of the Green is visible. About half a mile from the intersection the stream will enter another pool. Here the route is confusing. One way would be to enter this waist deep pool and wade through it, but you do not know what the other side holds, and at least in March the water is cold. But head up and look to left and you will see a cairn marking a trail following the cliff.</p> <p>A use trail also goes to the right (south) of the stream. <strong>DO NOT</strong> take this. We did and ended up at a dead end near the Green resulting in a significant detour to get back on track. After reaching this dead end I realistically considered descending the cliff at a very prominent ledge system, but was shot down by the others :)</p> <p>The trail on the north side is fairly straightforward, although it can be daunting at the end of an exhausting day. Soon you will reach the Green River. Such a large body of water, flowing calmly through huge canyons was a wondrous sight after 2 days in the dry desert. This is famous Stillwater Canyon, ahead of which lie the rapids of Cataract Canyon. The trail does not immediately descend to the river bank; Tamarisk trees have completely taken over this delta and made the approach impassable. Instead, the trail goes north along the Green for a few hundred feet and then descends to a sandy shore. Do not take the first faint turn. The actual trail has a small area that would make a good campsite right by the river.</p> <p>We had originally hoped to reach Spanish Bottom today, finishing all 8 miles of the river journey. The unexpected slow descent followed by going on the wrong trail put us really behind schedule though and it was 6:30pm by the time we hit the water. So our aim was to keep going while there was enough light and find a campsite in one of many small canyons along the Green.</p> <p>Not having a rain jacket, and not wanting to use my down jacket on a river, meant that I was wearing only two thin layers. The wind was blowing and the river was cold, so that I was quickly shivering.</p> <p>This was my first major bit of rafting, since on day 1 we had simply crossed the Colorado laterally. While I appreciate the stability of a raft, it’s thin bottom is not very insulating, and it really lacks the fun manuevarability of a kayak. The large bottom also means that whether stuck on a sandbar or when launching off, it is pretty much impossible to shimmy to get the thing to move. Strapping the pack horizontally (perpendicular to the raft and on top of the (gunwales?)) is a better position as it allows your legs to be placed inside the raft. I didn’t do this on day 3 and got uncomfortable quickly.</p> <p>Having a closed cell foam pad is great since you can sit on it and further increase insulation from the water.</p> <p>Stillwater Canyon is easy to go through and you can easily spin around and take in the surroundings as you let the river carry you at 1-2mph. Just make sure to do some sandbar spotting before as there are several of these on the stretch to Spanish Bottom.</p> <p>One time something that was possibly an otter swam by us, and we saw a few ducks, but otherwise it seemed like the 4 of us were the only living things on the river. We did paddle frequently since we wanted to cover more distance, finishing about 4 miles that day in 1hr 15min. Here (see map), a sandy beach large enough for several packrafts allows you to pull in. A well defined trail leads up this to a ledge that is probably not meant for as many people as the rafts. This ledge is wide enough for 2 people at the entrance, but narrows to a space comfortable for 1 person and their stuff the rest of the way. Pitching a tent is not a good idea since it just takes up too much space (Michaela and Raj insisted). The wind was low and it wasn’t so cold once we were out of the water, so I was happy to cowboy camp. The ledge does continue around to face the Green, and while an excellent spot to sit down, I would not trust myself to not roll of into the Green. More intrepid sleepers will find that it is easy to fit 7-8 people here, based on how cosy or daring they are.</p> <p>Other than the ledge, the rest of the small canyon has trees and a meadow. There was no real place to hang the food, so that I just slept with it that night. I didn’t expect any larger animals to come down into this canyon that must see human habitation only infrequently. Rats I’d just have to live with. (They ate Roxana’s food and left turds in her pot, mine seemed to be untouched).</p> <p>Day 4: Day 4 was an early start again. I have to commend the group on managing to stick to our departures regularly, the small number of people definitely helped. With the orientation, direct sunlight was going to be hard to come by for a while, so that the campsite was quite cold at first light. The advantage of sleeping with my food was that I could stay in my bag, eat, change and pack while staying warm. Only once I was ready to go did I get out of it :)</p> <p>We had tied all the rafts together and looped the rope around a rock - just in case - and so they were all there waiting for us. We pushed off around 8am and soon reached the confluence, where Green and Colorado meet to form a river that will cross nearly half the continent before draining into the Gulf of California. It is a surprisingly calm confluence, and the rising Sun definitely raised our spirits after 3 days in less than ideal conditions.</p> <p>Along the entire route of Stillwater Canyon red and yellow walls rise majestically, as if trying to make up for the indifference of the desert by the rugged beauty. Unless you are as dumb as a brick, Spanish Bottom is easy to identify due to the large beach and familiar surroundings from 3 days ago, but just in case, there is a sign board on the eastern bank a mile or so before it, warning that the Cataract Canyon rapids are coming up and packrafters with no prior experience better get out at the big beach up ahead. We pulled ashore a little before 10 and spent 90 minutes resting, waiting for the rafts to dry a bit, cleaning them up and filling up our water.</p> <p>The final part of the journey reverses the first day’s route and I will not go into the details. It is an arduous climb in the beginning, and with the excitement of the upcoming trip gone, so does all energy and optimism. Just slog through it.</p> <p>Just before the trail intersects Devil’s Lane Raj seriously thought we were lost since it was taking forever, and I was starting to have my doubts too, inspite of a correct compass bearing and vaguely familiar terrain.</p> <p>We were very happy then to come across a family of day hikers. To see other humans after 2 days, ones that were clean and happy, really put a spring in our step. They also assured us we were on the right track and indeed we quickly emerged into the wide canyon.</p> <p>We encountered a group of 3 teenagers who were heading to Spanish Bottom and they didn’t have a map! I hope they made it through fine.</p> <p>The table at Devil’s Kitchen was a welcome place to sit on, but after eating through all our remaining food (those sausages were delicious!) we all returned to the ground, the primal living of the last few days still dominating our civilized selves!</p> <p>From Devil’s Kitchen, which we reached around 2:30 and left at 3:30, it took us another 2.5 hours to finish the last 3.5 miles of 4WD track. We encountered two Texan 4WDs and their drivers on the way, who must have surely thought we were idiots to be walking.</p> <p>As much as the Needles district is stunning, towards the end of the trip I was a mission to finish it and ignore everything around me that isn’t helping in that task. I maintained an unnaturally fast pace to reach civilization as soon as I can. It is funny how easy it is to ignore the calls of civilization out in the wilderness, but once you know it’s near, how hard it is to keep away.</p> <p>I reached the Elephant Hill TH at 6pm and we spent another 45 minutes while everyone arrived and cleaned up. Then the long drive to Moab began with a quick stop at Newspaper Rock and occasional gawking at the climbers still on the cracks at Indian Creek.</p> <p>“Thanksgiving Dinner” at the Moab Grill was heaven, even though CNN just wouldn’t give up on MH370(?) in the background. We then headed next door to the Big Horn Lodge and washed away the grime and sand. Hot showers are the second best thing invented after restaurants!</p> <p>Arches:</p> <p>Breakfast with green chile at the Moab Diner was great. We were still ravenous and went through a skillet each. This hunger was to haunt us the whole day, supplemented by pita chips, chocolates, yogurt, strawberries, big honking sandwiches and sugary candy that we munched on pretty much the whole time. After washing and shipping the rafts we grabbed the food at CityMarket and drove to Arches. We explored Arches like normal people do, avoiding getting out of the car, driving to observation points, rolling down the window and taking pictures. We still managed to get out 3 times, once to view Delicate Arch from the upper viewpoint, once to get to Sand Dune Arch (beautiful and easy scrambled to the top) and Broken Arch, and once to reach Double Arches (again some fun scrambling). We had lunch (the BHS) at a nice picnic spot near Broken Arch. After some wandering through the souvenir store, we began the long drive to Salt Lake City around 4pm. We didn’t make the most of Arches, but I think I’ll be coming back to Moab. That place is really happening!</p> <p>A huge Chinese takeout dinner finally assuaged my hunger pangs and we slept for a few hours before catching the early flights to familiar, relatively humid, California.</p><div class="se-section-delimiter"></div> <h3 id="gear-thoughts">Gear thoughts</h3> <p>This year, I’ve acquired some new gear in to cut down weight for several upcoming trips. Below are thoughts on the gear that I used for the first time on this trip.</p> <p><a href="http://www.mountainlaureldesigns.com/shop/product_info.php?products_id=174"><strong>MLD Cricket tarp</strong></a> - I’ve had the MLD Cricket for a while, but this was the first extended trip I’ve taken it on. I can’t comment on weather protection since the weather was calm the whole time, but it’s ease of pitching is great! I usually keep the center pole at 135cm and the door pole at 110cm. This is a good height to be able to enter without touching my knees to the ground. It is also surprisingly warm for being open on one side. One day it even acted as a comfortable 2 person tent. In this configuration, the center pole has to be put up vertical and not at an angle. The second person does end up being a little outside the tarp coverage, so it wouldn’t suffice in case of rain. I think if you were comfortable getting really cosy, then the 2 poles could be arranged in a V-configuration with space to squeeze in between.</p> <p>The <a href="http://gossamergear.com/shelters/shelter-accessories/polycryo-ground-cloth-medium.html">groundsheet</a> i use with it also proved very durable for it’s weight and thickness (or lack of it).</p> <p><a href="http://www.zpacks.com/backpacks/arc_blast.shtml"><strong>Zpacks Arc Blast</strong></a> - This was my first time with my new, <em>orange</em> pack. It is the 60L variant with 1 hip belt pocket, hiking pole carriers, ice axe loops and bungee cord over the mesh. I was very impressed with it’s durability and load carrying capacity.</p> <p>In spite of several scrambles on slickrock, it didn’t suffer any ruptures. There is a small hole near the top where I think something from the rafting gear may have pricked it, but it’s small enough to not bother even taping. It is mightly stained though. The material is nowhere as tough as standard packs, so I did ensure that on really bad descents I would take the pack off and lower it.</p> <p>Most impressive was that it didn’t completely collapse from carrying 40lbs when it’s only rated to 30. I kept the frame straight to not load it. I have a feeling the stitching at the top of the shoulder straps was stressed, but it held up and I’ve had no problem in subsequent trips. I did not suffer any undue shoulder pain or hotspots, and the wide shoulder and hip belts really helped.</p> <p>The 1L side pocket is very useful since it is bigger than most backpack hip pockets. I could fit the day’s snacks, knife, compass, headlamp and camera in there. I might get another pocket though to keep things seperated a little and avoid having one hip so heavy :)</p> <p>The rolltop is secure and easy to use and allows great control over pack volume, so that it can become as small as the stuff you are actually carrying.</p> <p><a href="http://www.zpacks.com/quilts/sleepingbag.shtml"><strong>Zpacks 10F quilt</strong></a> - I replaced all the <em>big 3</em> for this season since I’ve a major trip later this year and each of them has performed wonderfully on this trip. My 10F quilt is the 6’, regular size with 900fp down.</p> <p>I am a side sleeper and I turn a lot, so I never bothered getting the zipper below me. On the first day when it went below freezing, that part did let in some cold since it’s not insulated. but I simply compromised by snuggling deeper into the bag and cinching it around my head. This immediately made it comfortable.</p> <p>On the other 3 nights, I didn’t once have to zip it up completely. In fact using the Ghost Whisperer down jacket for the torso and the bag for the legs was enough. My legs remained warmed all the nights, even with just using my empty backpack as ground insulation. (I was carrying the torso length part of the Z-lite sol to save on weight and volume.)</p> <p><a href="http://sawyer.com/products/sawyer-mini-filter/"><strong>Sawyer Mini</strong></a> - Performed splendidly. The backflush syringe took care of Colorado silt. What I love about the Mini is not having to filter all water at the source. I carried a 2L camelbak as the clean reservoir, and a 2L Platypus as the dirty reservoir and squeeze bottle. This allowed me to carry 4L but only expend effort into filtering the dirty 2L when required. The ability to fill the camelbak without taking it out of the pack is a big winner as is drinking straight from the dirty bottle with the inline attachment.</p> <p>One design decision that was disappointing is that the washer on the dirty end is easy to lose while shaking the filter out. Mine flew into the trees on Day 3. Since this renders the filter ineffective, I’d expect it to be affixed much more tightly. I was also disappointed that Sawyer did not get back to me about what size of washer to use to replace it, despite several social media posts and and email. Eventually, I got one from Lowe’s that required some scissor work, and was a tight squeeze, but at least it won’t be flying off ever again.</p> <p>I also carried a Smartwater bottle to add some more capacity in the desert and allow easily filling the Platypus.</p> <p>This is my one and only experience with Packrafts, so I can’t comment on them, but I really wish manufacturers find a way to reduce the packed volume. We had to spend several minutes and lots of energy trying to get them small and even then they wouldn’t fit in the backpack with the other stuff, leading to a unstable carry on the outside. Also maybe a teflon coating so the sand would just slide off later? :)</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-10025041046958096232013-05-15T15:11:00.002+05:302013-05-15T15:15:38.148+05:30Your Jabber ID as your Persona identity<p><em>(This is NOT an official Mozilla project and does not in any way reflect the views of my employer.)</em></p> <p><a href='https://login.persona.org/about'>Mozilla Persona</a> is a way for users to use their e-mail ID as their identity on the web. While cool, it will only really take off when existing services that people use become Identity Providers. XMPP (Jabber) is a widely deployed IM protocol whose IDs look like e-mail and it is a secure, federated system in alignment with Persona&#8217;s goals. I thought it would be really cool if I could log in to Persona enabled sites using Jabber IDs. I&#8217;d like to announce <a href='https://github.com/nikhilm/browserid-xmpp'>browserid-xmpp</a> which does just that.</p> <p>It should work with any XMPP server that supports components and BOSH. That said I have only tested it on my VPS (with Prosody, ejabberd and Openfire), so any issues and pull requests are welcome, as is a quick comment if you deploy it on your server. You&#8217;ll also need a relatively sophisticated web server like Apache or nginx to serve the browserid file with the right Content-Type. <a href='https://checkmyidp.org'>CheckMyIdP</a> is a great way to check if everything is setup properly.</p> <p>browserid-xmpp is two things. The first is a XMPP component that can plug into any XMPP server and answer a certificate signing query. This is a fork of the &#8220;official&#8221; <a href='https://github.com/mozilla/browserid-certifier'>browserid-certifier</a> with an Jabber-RPC front-end rather than a web service.</p> <p>The second is the set of provisioning and sign in pages that can be re-used by any domain. The authentication is handled as a two stage process using BOSH. This was my first experience with BOSH and it is ridiculously cool how it works and supports session hand-off to another page, without which this would not be possible. On the sign in page, an XMPP stream is established and authentication is done using standard XMPP authentication. The established BOSH stream has a session ID and every message sent has an incrementing request ID. On successful sign in, the sign in page sticks these two, along with the JID into <code>sessionStorage</code>. The provisioning page reads these out and &#8216;attaches&#8217; to the existing BOSH stream. Due to the unpredictable nature of the SID and RID, there is a reasonable guarantee that someone who attached to the stream successfully knew about the stream before. The provisioning page then makes a Jabber-RPC call over the same stream to the XMPP component. This call is performed on behalf of the JID and a certificate is sent back to the browser. You are now signed in!</p> <p>P.S. I&#8217;d like to thank Cory Benfield for an <a href='http://lukasa.co.uk/2013/04/Writing_A_Persona_Identity_Provider/'>excellent guide</a> to writing an IdP.</p> <p>P.P.S. This post was published right before a 12-hour plane ride, so I&#8217;ll be back for tech support in a while.</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com1tag:blogger.com,1999:blog-3665064696228706104.post-34535025201122646112013-04-26T22:12:00.000+05:302013-04-26T22:16:48.676+05:30Push notifications for the Open WebI’m excited to announce a new WebAPI that Mozilla has been working on for the past few months – <a href="https://wiki.mozilla.org/WebAPI/SimplePush">Push Notifications</a> for Web applications. Push Notifications let web applications be notified that something has changed on the server and that the application should refresh its data. For example, a calendar application can use Push Notifications such that whenever a new event is added on the server, the calendar application gets started in the background. The calendar app would then add the event to its local agenda and shut down, all without the user’s intervention. When the (pleasantly surprised) user looks at his phone, it has a current copy of his agenda.<br />Thanks to Push Notifications, Web applications are freed from repeatedly polling for updates, leading to a better experience for everybody. Users get better battery life and more responsive applications. Developers don’t have to re-implement polling logic in every application they write. Mobile devices benefit from intelligent scheduling of notifications to reduce network usage and further improve battery life.<br />There were several challenges involved in making a push notification system for the open Web:<br /><ol><li>Developers shouldn’t have to pre-register their applications with a push notification provider. Native app ecosystems require this; for example, you have to get a token from Google to use Cloud Messaging for Android. But for Web apps, which can be self-hosted instead of being distributed by an app store, such restrictions would only limit the audience. In addition, anybody should be free to run their own push server for their devices. Mozilla can run a server for Firefox users. Carriers can run their own servers, on which they can use cellular infrastructure to wake up Firefox OS devices. In fact, an individual user can even run their own push server if they choose.<br /> </li><li>Users shouldn’t have to log in to a third party to use push capabilities. Note: Users may still need to log in to the app so the app knows what information to push to the device. For example an email app isn’t very useful without logging in.<br /> </li><li>Protect the user’s privacy. This means that the push server should not receive any private or user-identifying information that is specific to the application. In keeping with this, the push server does not receive any application-specific data. It can only act as a “shoulder tap” or carry a number specifying a “version” or similar identifier.<br /> </li><li>Make Push Notifications an application wakeup API and not a “user notification” API. The act of displaying a badge or popup notification is not covered by Push Notifications. This decoupling means that applications can use push for all types of tasks, from dealing with IM updates to syncing information across devices without interrupting the user. Applications that do need to notify the user can do so by using the Notifications API with Push.<br /> </li></ol>We believe we now have a system that can satisfy the goals while still being secure and scalable.<br /><h3 id="how_do_i_make_my_web_application_push_enabled">How do I make my Web application push enabled?</h3>It is very simple for web applications to use Push Notifications. In fact, in only about <a href="https://gist.github.com/anonymous/5305425">100 lines of code</a>, I modified Gaia to support “Push To Install”. Once you’ve logged in using Persona on your phone, you can install an application to your phone from any browser on any device, just by pushing it to the phone.<br />The basic flow for Web application developers is:<br /><ol><li>Your application calls <code>navigator.push.register()</code> to acquire a unique URL, called a <i>push endpoint</i>. This endpoint usually refers to a push server. Your app can call <code>register()</code> multiple times for different uses. For example, an email application could have one <i>push endpoint</i> for every account it tracks.<br /> </li><li>Your application uses something like XMLHttpRequest to notify your application server of this <i>push endpoint</i>. The application server is maintained by the application developer. For example, if you write a microblogging application, you will have a server where posts are aggregated. You would then associate the push endpoint with the particular user of your application.<br /> </li><li>When your application server believes something interesting has happened that your application should be notified about, it makes an HTTP PUT request to the <i>push endpoint</i>.<br /> </li><li>The push server and the user agent (Web browser/Web runtime/Firefox OS device) communicate and deliver a notification to your application. If your application is not running, it will be started in the background and receive the notification. The notification is delivered using <a href="http://runtime.sysapps.org/#system-messages">System Messages</a>.<br /> </li><li>When it receives a notification, your application should connect to the application server and download the newest data. Since Push Notifications is a signaling system, it does not carry data in the notification.<br /> </li></ol>I’ll do a blog post with a detailed Push Notifications tutorial soon.<br /><h3 id="status">Status</h3>On March 28, 2013 the first pieces of code to enable Push Notifications <a href="https://hg.mozilla.org/mozilla-central/rev/2aaf82b852e7">landed on mozilla-central</a> and are enabled for Firefox OS builds. Set the pref <code>services.push.serverURL</code> to <code>wss://www.simple-push.com:9999</code> and the APIs should be available. You can also <a href="https://github.com/dougt/go-push-server">run your own server</a>.<br />The protocol is unlikely to change, but client bugs are still being fixed. As such, this is an <b>experimental</b> API. <a href="https://wiki.mozilla.org/WebAPI/SimplePush">Push Notifications</a> also <b>obsoletes</b> the older <a href="https://wiki.mozilla.org/WebAPI/PushAPI">Push Notifications API</a>.<br />Push Notifications is only available to Web applications at this point. There is agreement that web pages loaded in tabs should also be able to use the API, but various technical issues need to be resolved before this can work properly. This is the next major issue the team will address.Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com24tag:blogger.com,1999:blog-3665064696228706104.post-87122233044875076072012-11-15T10:09:00.000+05:302012-11-15T10:11:21.931+05:30A new adventure<p><i>(This is a long overdue post)</i>.</p><p>I remember when I used to have homework, then for a few years in college I had no work. Last month, after a 4 month vacation, I began ‘grown-up life’. Unlike other giant leaps, mine didn’t start with a small step. It started with a 13000km trip to Silicon Valley. </p><p>I am very happy to have started a new (and <i>first!</i>) job at Mozilla. After last year’s incredible internship, it feels just like home. I have joined the Platform team and begun working on some upcoming APIs which I’ll blog about as they shape up. Being able to work with extremely talented people and to work on a class of problems that very few engineers in the world get to toy with is very satisfying. It does come with its share of problems, like having to get to grips with a decade old, sparsely documented code-base, but damn, when that code executes correctly after hundreds of iterations, there are so many snacks to celebrate with :) But seriously, joining Mozilla at a time when Web architecture and APIs are undergoing massive upgrades, and we get ready to launch a mobile operating system, is very exciting. Oh and employees get to <s>be guinea pigs</s> test it. </p><p>Couple this with managing your whole life for the first time. No longer do parents or the dorm automatically pay electricity bills or ensure your Internet connection is working. House hunting, paying rents, moving, buying furniture, managing finances, cooking, everything has been a first. Smooth for the most part, but far too salty sometimes. </p><p>Couple this with the constant technical, cultural and recreational melange that is the Bay Area and the regret about not having enough hours in the day and it’s overwhelming. In a month I’ve discovered rock climbing, seen a trapeze act, won second prize in a hackathon and gone to various meetups. Every day when I <s>snuggle into my comforter-sleeping bag combo (you see, I still haven’t bought a mattress)</s> get into bed, I’m exhausted, but every day is also a very satisfying adventure. I can’t wait to have more of them. </p><p>P. S. The libuv book has suffered due to this, but I’ll get back to it soon. </p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-24204127066225485422012-11-09T03:02:00.000+05:302012-11-09T03:02:05.209+05:30A poor man's Notational Velocity on Linux<p>I use <a href="http://notational.net">Notational Velocity</a> on my Mac all the time. It holds all my notes, lists and any other snippet of text. I love the interface and simplicity, and most of all I love the simple use of text files in Dropbox as a store. This way I can access my notes anywhere, without needing NV to be installed. I also love the global key binding feature so that I can quickly raise it with Cmd+Shift+N. </p><p>At work (more on this soon) I started using a Thinkpad x230 running Archlinux. But I sorely missed NV. I experimented with <a href="https://github.com/cpbotha/nvpy">nvpy</a>, but it didn't cut it for me at all. The tkinter UI looks bad in a Qt/GTK desktop, notes are saved in JSON by default, and the text file option is a sort of hack which stores the title in the first line, messing up the notes in NV. So rather than write my own version, I got an almost as nice, and definitely more powerful NV equivalent in Linux.</p><p>I am going to assume you use a standard desktop environment like KDE or that your window manager is EWMH compatible. You'll need:</p><ul><li>To know how to define custom global shortcuts to run a command. For KDE this is <em>System Settings -&gt; Shortcuts and Gestures -&gt; Edit -&gt; New -&gt; Global Shortcut -&gt; Command/URL</em></li><li><a href="http://www.vim.org">gvim</a></li><li><a href="http://tomas.styblo.name/wmctrl/">wmctrl</a> (available in Arch community repo).</li></ul><p>Create a new shortcut which should launch the following command string</p><pre><code>gvim --remote-silent +':lcd %:p:h | :au FocusLost * :wa' \<br />'/home/nikhil/Dropbox/Notational Data' &amp;&amp; wmctrl -a 'GVIM'</code></pre><p>You should edit the path to point to your Dropbox/NV directory. Now whenever you press the global shortcut combination you should see gvim with a list of all files (notes). Press Enter on a file to open it.</p><p>We use <code>remote-silent</code> to make sure that gvim uses an existing window if it is already open. The <code>:lcd %:p:h</code> option sets vim's current working directory to the NV directory. This will be useful later. We use the autocommand FocusLost to save the file whenever the gvim window loses focus (simulating NV's autosave feature). Finally <code>wmctrl</code> raises the window to the top by matching the string to the title. If you use gvim on a regular basis (I use terminal vim) and have other windows open, you'll have to tweak this.</p><p>So this setup is completely like NV, except for one divergence. Whereas NV searchs the note title and content together, our system will treat it as two flows. To search note titles/file names use <code>/</code> when in the main view. As part of my standard vim plugin set I have <a href="https://github.com/kien/ctrlp.vim">ctrlp</a>and <a href="https://github.com/mileszs/ack.vim">ack.vim</a><sup>*</sup> which will serve us well here. To always have access to note titles use ctrlp. I map it to <code>sf</code> so that I get quick fuzzy find. Similarly to search note contents I map <code>sd</code> to trigger ack.vim. This is where setting vim's current directory is important. Both plugins will use it as the base search directory.</p><p>This NV approximation is fast and works almost as well as the original, although without a slick interface. But nice fonts and a good vim colour scheme come pretty close.</p><hr /><p>* You'll need <a href="http://betterthangrep.com">ack</a> installed to use ack.vim. <code>ack</code> does not include text files by default. Put <code>--text</code> in <code>~/.ackrc</code> to do so.</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com3tag:blogger.com,1999:blog-3665064696228706104.post-27528499368758485702012-08-25T18:15:00.001+05:302012-08-25T18:15:20.726+05:30Automatic Github Pages generation from Sphinx documentation<p><a href='http://sphinx.pocoo.org'>Sphinx</a> is a very common documentation tool which gobbles up ReStructuredText and other free-form markup formats and outputs great HTML, PDF and other formats. It is meant for reference manuals and API documentation due to its good integration with source code (especially Python). The <a href='http://nikhilm.github.com/uvbook/'>libuv book</a> is written using Sphinx so that it looks so good with minimum effort.</p> <p>Sphinx uses <code>make</code> to generate the HTML, which is great. The only problem is that deploying this to Github Pages requires multiple commands to switch branches to <code>gh-pages</code>, pull in the source text, then cleanup the working copy and switch back to <code>master</code>. This is boring after about one time, so I automated it, and I think other projects can benefit from it as well. Once you follow the instructions, running:</p> <pre><code>make gh-pages</code></pre> <p>will take the latest commit, switch to the <code>gh-pages</code> branch, generate HTML, push it to Github, then clean everything up and switch back to <code>master</code>.</p> <p><strong>NOTE: You need to commit or revert any working copy modifications before running this.</strong></p> <h3 id='onetime_commands'>One-time commands</h3> <p>These steps only need to be run the first time when you want to generate Github Pages. First setup the branch to have no parents. <strong>Let me stress again the importance of making sure all changes are committed! Otherwise they&#8217;ll be lost</strong>.</p> <pre><code>$ cd repo<br />$ git checkout --orphan gh-pages<br />$ git rm -rf .<br />$ echo &quot;First commit&quot; &gt; index.html<br />$ git add .<br />$ git commit -m &quot;Just to create the branch.&quot;<br />$ git push origin gh-pages</code></pre> <p>Now the gh-pages branch is setup. We can start generating the actual pages instead of the current <code>index.html</code>.</p> <h3 id='set_the_source_files'>Set the source files</h3> <p>Edit the Sphinx <code>Makefile</code>. Add a variable <code>GH_PAGES_SOURCES</code>. This should be a list of the files/directories that contain the documentation sources. This will usually be only <code>source</code> which contains the Sphinx reST docs, but if you are embedding external code or images, those directories have to be listed as well. In addition the <code>Makefile</code> <strong>has</strong> to be in the list. For the libuv book it is:</p> <pre><code>GH_PAGES_SOURCES = source code libuv Makefile</code></pre> <h3 id='add_the_target'>Add the target</h3> <p>Create a target <code>gh-pages</code> with the following commands (remember to use TABs in Makefiles):</p> <pre><code>gh-pages:<br /> git checkout gh-pages<br /> rm -rf build _sources _static<br /> git checkout master $(GH_PAGES_SOURCES)<br /> git reset HEAD<br /> make html<br /> mv -fv build/html/* ./<br /> rm -rf $(GH_PAGES_SOURCES) build<br /> git add -A<br /> git ci -m &quot;Generated gh-pages for `git log master -1 --pretty=short --abbrev-commit`&quot; &amp;&amp; git push origin gh-pages ; git checkout master</code></pre> <p>Here is how it goes. The checkout simply switches branches. Then we remove all the old data to prevent any rebuilding artifacts. Since the <code>gh-pages</code> branch won&#8217;t have any of your original data, but only the HTML output, we need to pull the sources from the <code>master</code> branch. Then we generate the HTML. We move these from the <code>build</code> folder to the top level. Then we remove all the sources and the now empty <code>build</code> folder. We stage all the changes. Finally the last line generates a commit message for the <code>gh-pages</code> changes which is the first line of the latest commit on <code>master</code> and pushes to Github. The reason the <code>git checkout master</code> command is on the same line and semi-colon separated; if the push were to fail for some reason (network error, DAG inconsistency), I want to be returned to master in my working copy. If you don&#8217;t want this to happen, feel free to move it to a new line on its own.</p> <h3 id='done'>Done!</h3> <p>You can now get back to your main task - writing great documentation. Whenever you have an urge to show it to the world, simply run <code>make gh-pages</code> and your latest documentation is served fresh!</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com5tag:blogger.com,1999:blog-3665064696228706104.post-65183201885075807662012-08-17T16:16:00.001+05:302012-08-17T16:16:22.280+05:30Installing node.js on the Raspberry Pi Archlinux<p>My Raspberry Pi arrived a few weeks ago and I had some problems with getting node.js to build on it. This post is specifically about building <a href='http://www.nodejs.org'>node.js</a> <em>from source</em> on <em><a href='http://www.archlinuxarm.org'>Archlinux</a></em>.</p> <p>All activity and commands in this tutorial are run <em>on the Raspberry Pi</em>, either via SSH or physical keyboard. It is possible to cross-compile on your laptop/desktop for ARM, but I preferred not to in this case.</p> <h3 id='prerequisites'>Prerequisites</h3> <p>Have the <code>base-devel</code> group installed, you should have a working <code>gcc</code> and friends, along with <code>openssl</code> and <code>zlib</code>. Also install <code>python2-virtualenv</code> and <code>git-core</code>.</p> <h3 id='get_the_source'>Get the source</h3> <p>I usually stay on node bleeding edge so:</p> <pre><code>$ git clone git://github.com/joyent/node</code></pre> <h3 id='use_python2'>Use Python2</h3> <p>Since Archlinux uses python3 by default, the configure and build scripts screw up. You can edit the individual files, but I find it easier to just use <a href='http://www.virtualenv.org'>virtualenv</a>, which will setup the shell environment to use python2 and its libraries.</p> <pre><code>$ virtualenv2 env<br />$ source env/bin/activate</code></pre> <p>Here <code>env</code> is the name of the directory virtualenv will use to store scripts. You can use some other name. Sourcing the <code>activate</code> script will set up the environment so that <code>python</code> will actually be <code>python2</code>.</p> <h3 id='patch_the_source'>Patch the source</h3> <p>Geoff Flarity has created a <a href='https://github.com/gflarity/node_pi'>patch</a> to tweak the build parameters a bit. You can either follow the instructions to apply the patch, or just comment the 12 <code>vfp3:on...</code> lines yourself, which is what I did. Also set the environment variables to build for ARM as in the README.</p> <pre><code>export GYP_DEFINES=&quot;armv7=0&quot;<br />export CCFLAGS=&#39;-march=armv6&#39;<br />export CXXFLAGS=&#39;-march=armv6&#39;`</code></pre> <p><strong>NOTE: All commands should be run in the same shell that has the virtualenv environment and the above variables set.</strong></p> <h3 id='use_system_zlib_and_openssl'>Use system zlib and openssl</h3> <p>Attempting to compile the included openssl failed on my Archlinux setup with some ARM assembly errors. Using the installed libs will also reduce the compilation time. So</p> <pre><code>$ ./configure --shared-openssl\<br />--shared-openssl-includes=/usr/include/openssl\<br />--shared-openssl-libpath=/usr/lib\<br />--shared-zlib\<br />--shared-zlib-includes=/usr/include\<br />--shared-zlib-libpath=/usr/lib</code></pre> <h3 id='build'>Build</h3> <pre><code>$ make<br />$ su -c &#39;make install&#39;</code></pre> <p>The build will take some time (30 minutes to an hour), so be patient. Once it is done you will have a fully working node runtime on your Raspberry Pi. Use it to power your home automation control server or whatever else. I use it to experiment with my <a href='https://github.com/nikhilm/kademlia'>DHT implementation</a>.</p> <p>The use of node.js to write low-level systems services or networking code over traditional languages like C/C++ is very interesting, because it provides a safer, garbage-collected runtime, with efficient I/O and a less verbose language. Inexpensive devices like the Raspberry Pi allow experimenting with multiple devices or peer-to-peer configurations rather than being bound to only localhost testing for those on a tight budget. So get a Pi and have fun.</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-26983261177669248162012-07-13T02:54:00.001+05:302012-07-13T11:02:34.887+05:30Europe2012: Helsinki<p>I landed in Helsinki on the 26th of June from a connecting flight via Frankfurt. As you&#8217;ll see in the rest of the journey, food is a <strong>very integral</strong> part of my life, and I can&#8217;t resist describing airplane food either. So in Lufthansa, on the BOM-FRA flight, they served this decent croissant and omelette with spinach and chicken for &#8216;breakfast&#8217;. Fruits in planes are always shit, and this was no exception.</p> <p>This was my first time at Frankfurt as a transit point. They make you wander a lot to get to passport control. For a second I thought passport control wasn&#8217;t going to happen at port of entry. Then the immigration officer gave me another scare by being very scrutinizing, checking all my credit/debit cards and my dad&#8217;s permission letter. Yay! for having all my documents in order.</p> <p>Finally being let through, the FRA-HEL flight was again Lufthansa. I love day flights for their take-offs and landings where you get amazing views of the land. This was no different with the plane descending as it left the shores of the north sea near Germany and then approached Helsinki-Vantaa International Airport.</p><a href='http://www.flickr.com/photos/code_kreator/7558107522/' title='DSC02709 by Nikhil Marathe, on Flickr'><img alt='DSC02709' height='375' src='http://farm8.staticflickr.com/7107/7558107522_66293a7f87.jpg' width='500' /></a><p>In flight meal was a delicious &#8216;American club sandwich&#8217; which was bacon and chicken with mayo in rye bread. I&#8217;ve had rye bread every time I&#8217;ve been to Europe since 2010, and I love that stuff. It has great texture, a sweetness that develops in the mouth and complements meat very well.</p> <p>HEL is located quite some distance from the main city as it serves the greater Helsinki region. The first thing I do when I land in a European city is to buy a pass for public transport for the number of days I am going to be staying. A citizen of Mumbai, I love public transport and passes turn out to be very cheap all over Europe. So just blindly buy one if you plan to explore cities. In this case the machine didn&#8217;t have change so I bought a one day pass then bought another one the next day. Passes printed by a machine are on paper. You can buy travel cards or seasonal passes from major terminals, tourist points and most conveniently from R-kioski general stores. These green cards are very useful and easy to use. They are available in single region (Helsinki only), or 2 and 3 region varieties, although the price difference isn&#8217;t much. Overall, I ended up paying EUR 36 for 4 days of unlimited travel in 3 regions. If you think that is a lot, consider this. The pass allows travel on the entire HRT network, bus, tram, metro and rail. In addition, the Suomenlinna ferry operated by HRT is included. In EUR 36 I roamed about the city, visited a major tourist attraction and went all the way to Nuuksio national park in Espoo! But I&#8217;m getting ahead of myself. Back to the airport. The regional bus 615 leaves every 15 minutes and will take you to Helsinki Central Station from where transport lines go all over the city.</p> <p>Bicycles are common in Helsinki and bike lanes exist but aren&#8217;t sharply delineated, so make sure you don&#8217;t treat a bike path as the pavement. While riding bikes on the pavement is frowned upon in San Francisco, it is fairly common in Europe to ride on the pavement when a bike path does not exist. It also isn&#8217;t so much of a hindrance with the super-wide pavements that are so common in Europe.</p> <p>I stayed in Hostel Erottajanpuisto, which is on Uudenmankatu (katu is Finnish for street). Take the 3B tram from Helsinki Central Station and get down at Frederikinkatu or Iso Roobertinkatu and walk to the hostel. It is a nice, clean place. There are 4 toilets and 4 showers on the hostel floor. My room was a 6-bed dorm. Everything was clean and the staff were very helpful. Prices were EUR 27 + 30 + 30 due to the European Athletics Championships which began from June 27th.</p><a href='http://www.flickr.com/photos/code_kreator/7558108870/' title='DSC02716 by Nikhil Marathe, on Flickr'><img alt='DSC02716' height='375' src='http://farm8.staticflickr.com/7258/7558108870_8e85ae7001.jpg' width='500' /></a><p>That evening I was pretty tired and just wandered the city. First I went to Verkkokauppa, which is one of Europe&#8217;s largest electronics store. I didn&#8217;t find much there that I could afford :P, so I just went upstairs to the viewing gallery on the 7th storey which gives great views of Helsinki, especially the West harbour. It also has a real MIG-21 plane on display.</p><a href='http://www.flickr.com/photos/code_kreator/7558120326/' title='DSC02725 by Nikhil Marathe, on Flickr'><img alt='DSC02725' height='375' src='http://farm9.staticflickr.com/8150/7558120326_7a9f8da90f.jpg' width='500' /></a><a href='http://www.flickr.com/photos/code_kreator/7558123298/' title='DSC02726 by Nikhil Marathe, on Flickr'><img alt='DSC02726' height='375' src='http://farm8.staticflickr.com/7250/7558123298_5711dafe47.jpg' width='500' /></a><a href='http://www.flickr.com/photos/code_kreator/7558125890/' title='DSC02728 by Nikhil Marathe, on Flickr'><img alt='DSC02728' height='375' src='http://farm8.staticflickr.com/7258/7558125890_80cb99175c.jpg' width='500' /></a><p>From there I came back to the city centre, and wandered down Esplanadi, which is the cities main promenade. Halfway down Esplanadi is a cafe and the area across it has a stage for performances. A seniors band was performing that day though I only stopped for one song. At the end of Esplanadi is Kauppatori, an open air market which would be the source of culinary delights. Sadly I arrived late (it closes at 18:00) and missed it the first day. Slightly disappointed, I wandered around Keskuskatu which is the central square. The area around Helsinki Central Station is full of high-end shops and departmental stores and is fun to walk about. I ended up eating at some place called Chilly&#8217;s (salmon and fries with salad) near the station (I can&#8217;t find it on Google Maps now) when I got really hungry and couldn&#8217;t find something nice. Tired from the flying and walking, I came back to the room and crashed. Well after the daily routine of backing up the photos.</p><a href='http://www.flickr.com/photos/code_kreator/7558126954/' title='DSC02732 by Nikhil Marathe, on Flickr'><img alt='DSC02732' height='375' src='http://farm9.staticflickr.com/8150/7558126954_7d16daf205.jpg' width='500' /></a><p>On day 2, I was all eager, it was time to go to Suomenlinna! But before that I needed breakfast and a 3-day pass. R-kioski sells both. I had a nice chicken and bacon sandwich and mint-flavoured coffee sitting on a bench at South Harbour.</p><a href='http://www.flickr.com/photos/code_kreator/7558129996/' title='DSC02735 by Nikhil Marathe, on Flickr'><img alt='DSC02735' height='375' src='http://farm9.staticflickr.com/8432/7558129996_a1cfaf021c.jpg' width='500' /></a><p>It isn&#8217;t everyday that you get to have such a beautiful view in the morning. Now the day pass (You can get multiple days too). The HRT has a green colour card for tickets. The card is coded with information, and gray readers on all modes of transport do their job. Each reader has a quadrant, with 0/L, 1, 2, 3 marking each quadrant. Now, the numbers represent how many regions you want a ticket to be valid in. If you are a resident who has a balance on the card you have to do some button pressing for zone selection. But with a day pass we don&#8217;t need to bother with that. If you bought a 3-day, 3-region pass, then you can go anywhere in 3 days. All you&#8217;ve to do is hold the card to the reader when you get in. If everything is fine, the green light will switch on, accompanied by a beep. If things are not right, well I guess you&#8217;ll have to find out. Its very simple, so don&#8217;t get confused by the numbers. But make sure you aren&#8217;t travelling in a zone where the ticket is invalid. If you just want to see Helsinki, a 1-region pass is enough, it covers Suomenlinna too. But it does not cover the airport, so you should buy a separate ticket for that. I bought a 3-region pass as I would be visiting Nuuksio.</p> <p>Suomenlinna, literally the &#8216;Castle of Finland&#8217;, is a island to 2km to the south of Helsinki, part of the large Finnish archipelago. The Suomenlinna ferry leaves from South Harbour every 20 minutes. On the way there are several smaller islands on which lucky people have houses.</p><a href='http://www.flickr.com/photos/code_kreator/7558135456/' title='DSC02746 by Nikhil Marathe, on Flickr'><img alt='DSC02746' height='375' src='http://farm8.staticflickr.com/7277/7558135456_a3bd2a2fa6.jpg' width='500' /></a><p>You can see the attractive but short skyline of Helsinki as you leave the mainland and be amazed by Scandinavian cleanliness and efficiency. On Suomenlinna there are a few cafes, a few points of interest and the fort at the end. On the way to the visitor centre at the middle of the island you&#8217;ll already pass the church and its gardens. The blue path on the map (get one from the Visitor&#8217;s centre, or follow the blue sign boards) is the &#8216;suggested path&#8217;, but I wandered of in 2 directions not suggested. If you take a right as soon as you cross the bridge near the visitor&#8217;s centre you&#8217;ll reach a building in which they are building a wooden gunboat using traditional tools and techniques. It&#8217;s not open yet but you can go in through the door, just don&#8217;t creep out the workers by taking too many pictures. Continue on and you&#8217;ll reach the marina, which has a very nice collection of small boats, and the church in the background. On a sunny day it is a very nice sight. Which is why the Finns have put a cafe there as well, because in the summer you can enjoy that sight all day. Return to the blue path, but stick to the right after leaving the tomb, and take the route between points 6 and 22 on the <a href='http://www.suomenlinna.fi/files/1280/EN_Suomenlinna2010.pdf'>map</a> (the little gate in the wide building) and go towards 4. Here you&#8217;ll see the cannon emplacements, ducks swimming in the pools and a small (about the size of a big truck) artificial beach tucked away. I was lucky to also spot a lone paraglider riding the nice updrafts generated by the sea breeze.</p><a href='http://www.flickr.com/photos/code_kreator/7558136552/' title='DSC02787 by Nikhil Marathe, on Flickr'><img alt='DSC02787' height='375' src='http://farm9.staticflickr.com/8433/7558136552_243c37749c.jpg' width='500' /></a><a href='http://www.flickr.com/photos/code_kreator/7558137184/' title='DSC02838 by Nikhil Marathe, on Flickr'><img alt='DSC02838' height='375' src='http://farm9.staticflickr.com/8430/7558137184_e562b4c28a.jpg' width='500' /></a><p>By this time you&#8217;ll have walked a lot and must be pretty hungry. But, with a budget of EUR 20 for a day we aren&#8217;t going to be dining at any of these sit down cafes. So I just went to the restroom (probably shouldn&#8217;t be going into <em>this much</em> detail), then sat for a while at the King&#8217;s gate.</p><a href='http://www.flickr.com/photos/code_kreator/7558137522/' title='DSC02876 by Nikhil Marathe, on Flickr'><img alt='DSC02876' height='375' src='http://farm9.staticflickr.com/8289/7558137522_b01692ebb4.jpg' width='500' /></a><p>There is some very nice fort design at work near the Gate to ambush ships. On the way back you can explore the little park, or not. All the good planning of Suomenlinna was due to its architect, Augustin Ehrensvärd, who cared about his soldiers. You&#8217;ll learn this if you watch the short movie screened in the visitor centre (English at 12:30). That and the museum together will set you back EUR 6.50. There is also Wi-Fi in the visitors centre and some nice souvenirs though I don&#8217;t buy any. This is because most souvenirs are just Made in China these days.</p> <p>So&#8230; keeping the idealogy aside, I&#8217;ve managed to give a run down of the place in one paragraph! What do I do now!? Well, the bid to save money does not stop me from being able to pass up lemon-liquorice icecream. So I had one at the shop near the church and got a huge scoop for 2.80. Now I could hold on till I made it to the mainland.</p><a href='http://www.flickr.com/photos/code_kreator/7558138288/' title='DSC02915 by Nikhil Marathe, on Flickr'><img alt='DSC02915' height='375' src='http://farm8.staticflickr.com/7264/7558138288_c3500b9761.jpg' width='500' /></a><p>A note about liquorice. Liquorice is like, well, something. You either love it or hate it. I first had it in 2001 and I&#8217;ve loved it. In 2010 I was unfortunate to try salmiakki, since then I stick to the less salty variants :) The Finns (call it Lakritsi) and the Dutch (call it Drop) love liquorice and you can find it in every decent general store. Buy some, try some. My favourite is the plain black unflavoured, slightly sweetened cylinders. I got mine in some yellow packet (400g for EUR 3 or so), but there are a lot of brands and flavours.</p> <p>While you were thinking of liquorice, I finished my icecream and the ferry was back to South Harbour. Behold the wonderland that is Kauppatori.</p><a href='http://www.flickr.com/photos/code_kreator/7558135126/' title='DSC02737 by Nikhil Marathe, on Flickr'><img alt='DSC02737' height='375' src='http://farm8.staticflickr.com/7124/7558135126_e93b64a766.jpg' width='500' /></a><p>Kauppatori is an open market similar to most European cities, with local food stalls. In Finland that means Salmon, Vendaces, Calamari, Reindeer sausages and Moose sausages. Kauppatori was to become my lunch spot all three days. On the first day I had salmon with blue cheese, with a side of vendaces, calamari and veggies (EUR 15).</p><a href='http://www.flickr.com/photos/code_kreator/7558138928/' title='DSC02923 by Nikhil Marathe, on Flickr'><img alt='DSC02923' height='375' src='http://farm9.staticflickr.com/8286/7558138928_ff6e847ca6.jpg' width='500' /></a><p>I thought that was expensive, but it was so huge that I kept half of it and had it for dinner. I also got half a litre of cherries for EUR 3. They were the best cherries I&#8217;ve ever eaten, much bigger than the cherries in India. They were also extremely sweet and I had great fun munching on them while walking the streets.</p> <p>After lunch I went to Linnanmaki amusement park. Going to a amusement park is not something you may do in a foreign country, except perhaps Disneyland. But, India has no decent amusement parks, and this one looked good. To reach there take the 3B tram to Alppila stop and walk about 500m in the direction the tram will go. A day pass is EUR 37 and gets you on all the rides as many times as you want. I just went to the 6-7 most extreme rides. Rain prevented me from going on the best one - Ukko - so I paid EUR 5 to extend my pass to the next day. Completely bushed by now, it was time to take the 3B back to my room.</p> <p>On day 3 it was time for something completely off the tourist map. On the outskirts of Espoo is Nuuksio National Park. It has some nice trails through the wilderness of conifer forests and Finnish lakes. With a 3 region pass, you can go all the way to Nuuksio for &#8216;free&#8217;. You can take the S, U, L or E trains to Espoo then take bus number <a href='http://aikataulut.hsl.fi/linjat/en/e85.html'>85</a>. Get down at Nuuksionpaa or Kattila (last stop). Once in the countryside the stops don&#8217;t have their names written on them, so it&#8217;s best to just get down at Kattila. From there you can see the trail marked. There are two, red and blue. Red is about 4km and goes to the main park entrance at Haukkalammentie. The blue one is 8km and I don&#8217;t know where it ends up because I didn&#8217;t take it. I had fun traipsing through the park for about 2 hours.</p><a href='http://www.flickr.com/photos/code_kreator/7558140350/' title='DSC02978 by Nikhil Marathe, on Flickr'><img alt='DSC02978' height='375' src='http://farm9.staticflickr.com/8027/7558140350_3a8b218516.jpg' width='500' /></a><a href='http://www.flickr.com/photos/code_kreator/7558139762/' title='DSC02948 by Nikhil Marathe, on Flickr'><img alt='DSC02948' height='375' src='http://farm8.staticflickr.com/7115/7558139762_b28782d050.jpg' width='500' /></a><p>Finland is a pretty flat country so there aren&#8217;t any hikes, just walking. There&#8217;s silver birch, spruce, alder, aspen and lime trees and two lakes on the trail. Three-fourth of the way through, you&#8217;ll come to a fork, one going left and the other right over a wooden bridge. The signs here have fallen into the lake and the trail is unmarked for a while. So take a left to stay on the red trail. At Haukkalamppi there is a info cabin with details about the indigenous wildlife and vegetation if you are so inclined. There is also drinking water and, surprisingly, a gift shop. Walk 2 km along the paved road to reach Nuuksionpaa from where you can catch the 85 back.</p> <p>From Helsinki Central Station it was back to Kauppatori. This time I tried the sausage mix (EUR 8).</p><a href='http://www.flickr.com/photos/code_kreator/7558087716/' title='2012-06-28-010 by Nikhil Marathe, on Flickr'><img alt='2012-06-28-010' height='500' src='http://farm9.staticflickr.com/8009/7558087716_87a9493518.jpg' width='375' /></a><p>I particularly loved the reindeer, which is dense and similar to beef. This was topped off by fresh blueberries (EUR 5). From there I went to Linnanmaki to ride the best roller coaster and try the other rides again. The Ukko takes you up five storeys, then sends you straight down vertically, to go all the way up the opposite face, then go back down in reverse and up five storeys again (in reverse!) then down again and up again where the hydraulics kick in and slowly lower you to ground level. It is super and there are correspondingly large lines, but it is well worth the wait and the rain.</p> <p>The final day was a bit tempered. I had seen everything I wanted to see (or so I thought) so I just wandered around Kauppatori, Kauppahalle (the gourmet product hall south of Kauppatori), Stockmann and co. Picked up some gooseberry jam, some Fazer chocolate and a pack of plain liqourice. I accidentally wandered onto Sofiakatu and Helsinki&#8217;s main square.</p><a href='http://www.flickr.com/photos/code_kreator/7558086156/' title='DSC03026 by Nikhil Marathe, on Flickr'><img alt='DSC03026' height='375' src='http://farm8.staticflickr.com/7262/7558086156_23cc1081dc.jpg' width='500' /></a><p>Just before that is the museum which had a Finnish films exhibition. It was free so I saw some of it then took some photos of the main square. It was surprising that I forgot about this, of course there is nothing very interesting, they are just nice buildings. Then I went into Asematunneli complex but there wasn&#8217;t much there and I left soon. After this I was really but had a while to go for my ferry, so I just spent some time sitting on a bench at Esplanadi, nibbling on chocolate, had reindeer meatball lunch and then went back to the hostel, took the bags and walked to Makasiiniterminaali where the Linda Line runs to Tallinn in 1.5 hours. Ferries run regularly, with fares being in the range of EUR 25-45 depending on season, day and time. My ferry on a Friday at 14:00 was EUR 44. Book online since on the spot booking has a EUR 5 surcharge.</p><a href='http://www.flickr.com/photos/code_kreator/7558104098/' title='2012-06-29-019 by Nikhil Marathe, on Flickr'><img alt='2012-06-29-019' height='500' src='http://farm9.staticflickr.com/8168/7558104098_a747c76237.jpg' width='375' /></a><p>Helsinki had been amazing, it was the first stop on my tour and Scandinavia was well worth it! Next stop Tallinn, Estonia to attend two days of Akademy 2012&#8230;</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-21442872444112902822012-04-18T12:40:00.000+05:302012-04-18T12:40:28.172+05:30Lessons in 4 years<p><em>(This article was written as a piece of advice in the final edition of Entelechy for the year, now that I will be graduating)</em></p> <blockquote><p>Never give advice — a wise man won&#8217;t need it, a fool won&#8217;t heed it.</p></blockquote> <p>So rather think of what follows as a crystallization of my opinions, which may serve you well. The last four years of my life have been a period of discovery and extreme change. In that I&#8217;ve either realised or read and agreed with certain qualities that will prove indispensible.</p> <p>The first is <strong>self control</strong>. In college, nothing else matters! If you want to get things done, if you want to <em>stay on the right path</em> (subjectively of course), then nothing else will matter more than self control. Not your intelligence, your wealth, your environment or your friends. Only your will-power. You and I are living in a world where <a href='http://paulgraham.com/addiction.html'>addiction</a> is easier than ever to fall prey to. Information bombards us from everywhere, a plethora of online services crave for our attention, and in the physical world, chatter, video, liquids and powders demand our attention. These distractions will stop you from giving your time to the things that matter the most to you. Those who can stick to their course will fare better.</p> <p>If you are the sort of person who compulsively says yes to everything, learn to say NO. If you are a doer, people will ask for your help for the most trivial of things, or sometimes for the most important things. Decide if you have the time to devote to it, don&#8217;t immediately leave what you are already doing. If it is not worth it, learn to say a firm, direct, NO. Having your attention over forty different things means that all of them will remain incomplete.</p> <p>The second thing that will prove invaluable is <strong>self learning</strong>. You are in an information technology college, and being autodidacts is most relevant to us than to <em>anyone</em> else. Information Technology moves far too fast, much faster than academia, than law, than social conventions or the speed of light. An inability to teach yourself new things will quickly put you out of the competition.</p> <p>Being able to learn things on your own also opens a universe of experiences, each of which can be used to replace the boredom that leads to addictions. Self learning is not just watching video tutorials of the Internet, it means being able to evaluate yourself, setting new challenges and goals and keeping yourself on the path to achieve them.</p> <p>Learning on your own will also help guide you towards your <a href='http://blog.figuringshitout.com/nov-5th-day-23-three-types-of-passion/'>passion(s)</a>. Because anything on which you are willing to spend time without anyone else telling you too, is clearly something you love doing. Choose a job you love, and you will never have to work a day in your life. <sup id='fnref:1'><a href='#fn:1' rel='footnote'>1</a></sup></p> <p>Stay <strong>fit</strong>. You come to college and forget all about play. You spend evenings either poring over books, or playing computer games or wasting time lounging about. In these actions you are setting the pattern for how you will live your life. You might think, &#8216;I&#8217;ll just eat less&#8217; or &#8216;I don&#8217;t mind getting a bit fat&#8217;, but few things are as joyful as sweating it out on lush green grass, day after day. Revel not only in the improvement of your mind, but in the miracle of your body. It may not get you a job, but it will ensure that when you walk into office, heads turn. Finally, like everything else, don&#8217;t over do it. Learn to listen to your body and your body will listen to you.</p> <p>Have a relentless pursuit to <strong>rise above the mediocrity</strong> that is encouraged by our social system (&#8216;our&#8217; here applies all over the world, I&#8217;m not dissing on India). Are you scared of what you&#8217;ve to do to achieve your dream? Start small, a little step here, another one there. Find the right level of challenge. But always <a href='http://zenpencils.com/comic/36-bruce-lee-there-are-no-limits/'>aim higher</a> than you know you can go. Have a bucket list, and an ideas list. Both lists will start to fill up like mad and you will never be able to tick all of them off. You are far too small, and the world too big to be able to see it all, but just by breaking out of your comfort zone, you&#8217;ll have seen more of it than most people can imagine.</p> <p>Finally, go and <strong>Create</strong>!</p> <blockquote><p>If your daily life seems poor, do not blame it; blame yourself, tell yourself that you are not poet enough to call forth its riches; for to the creator there is no poverty and no poor indifferent place</p> <pre><code> -- Rainer Maria Rilke</code></pre></blockquote> <p>There is nothing more anti-human that the lazy consumerism that is prevalent today. I do not mean you should write a book, or create software or write a song. The smallest creations make a big difference to your happiness. True happiness can be found in a football move created on the ground, or the smile created on another face by your actions. Let your creations, and not your tastes, be what define you. Tastes only narrow and exclude people. So create. <sup id='fnref:2'><a href='#fn:2' rel='footnote'>2</a></sup></p> <p>I&#8217;ll leave you with <a href='http://mitadmissions.org/blogs/entry/50_things'>50 more things</a>.</p><div class='footnotes'><hr /><ol><li id='fn:1'><p>Confucius said that</p><a href='#fnref:1' rev='footnote'>&#8617;</a></li><li id='fn:2'><p>Paraphrased from why the lucky stiff</p><a href='#fnref:2' rev='footnote'>&#8617;</a></li></ol></div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com5tag:blogger.com,1999:blog-3665064696228706104.post-87895339781434830052012-04-16T02:10:00.000+05:302012-04-16T10:42:35.473+05:30Demystifying JSCrushSome of you may have seen Philip Buchanan’s award winning. <a href="http://js1k.com/2012-love/demo/1243">Autumn Evening</a> entry for <a href="http://js1k.com/">JS1K</a> 2012 Love. While skimming the source I saw that he had used <a href="http://twitter.com/aivopaas">Aivo Paas</a>’s <a href="http://www.iteral.com/jscrush/">JSCrush</a> to compress the code. The JSCrush website is intriguing with the source code minified and then passed through JSCrush itself (so it can be submitted to JS1K). The JSCrush version used in the page, in &lt;script&gt; tags is the JSCrushed version. As a weekend project I tried to ‘reverse engineer’ JSCrush and understand it. It took me about 4 hours. What follows is a walk-through of the process.<br />JSCrush is a very interesting JavaScript program, liberally abusing <code>eval()</code>, global variables and insane levels of nesting to achieve a sort of compression.<br />Remember that your browser’s web development tools are indispensible for activities like this. I made extensive use of Firebug.<br /><br><h2 id="the_deobfuscation"><strong>The de-obfuscation</strong></h2>I chose to start with the compressed version in the &lt;script&gt; tag rather than the plain text in the upper text field.<br /> <script src="https://gist.github.com/2393861.js?file=jscrush-original.js"> </script> The syntax is clearly wrong for JavaScript, and it is all stuck in a string assigned to <code>_</code>. The part at the end is interesting though (properly formatted).<br /> <script src="https://gist.github.com/2393861.js?file=jscrush-end.js"> </script> For every character in <code>$</code> it is splitting <code>_</code> on the character, using <code>with</code> to make the resulting array the scope. Then joining the pieces using the last piece and reassigning to <code>_</code>. For example:<br /><pre><code>_ = "HelloRWorldRCrushedRAB"<br />$='R'<br /><br />var temp = _.split($) // =&gt; temp = ['Hello', 'World', 'Crushed', 'AB']<br />var last = temp.pop() // =&gt; last = 'AB',<br /> // temp = ['Hello', 'World', 'Crushed']<br />_ = temp.join(last) // =&gt; _ = 'HelloABWorldABCrushed'</code></pre>Remember this step, it is key to how JSCrush works. These steps are repeated for every character in <code>$</code>, after which the ‘decompressed’ output is the minified source code:<br /> <script src="https://gist.github.com/2393861.js?file=jscrush-minified.js"> </script> You can see this for yourself by putting a <code>console.log(_)</code> just before the <code>eval(_)</code>.<br />Now we’ve a fair idea of how JSCrush is doing decompression. Compressed scripts are stored in <code>_</code>, decompressed using the loop and then executed using <code>eval()</code>.<br />The next thing I did was to un-minify the source (manually):<br /> <script src="https://gist.github.com/2393861.js?file=jscrush-reformat.js"> </script> One change I’ve made is the call to <code>setTimeout()</code>. I’ve converted it to a function to make it easier to read, and directly used the script tags innerHTML, since I had the decompressed source in the tag. The JSCrush code generates the textareas and button as part of it’s run and assumes <code>body.children[9]</code> to be the &lt;script&gt; tag with the JSCrush compressed source. Hence it replaces the eval call with the program source itself so that the inner <code>eval()</code> call in <code>setTimeout()</code> extracts the decompressed source and puts it as the value of the first textarea. It then calls <code>L()</code>, the JSCrush <i>crushing function</i> to compress the original code back, so that you get the compressed version of JSCrush in the lower text field. Mind-boggling.<br />The <code>setTimeout()</code> without a time simply causes the code to be executed after the script has finished evaluating completely.<br /><br><h2 id="understanding"><strong>Understanding</strong></h2>Now that we’ve decompressed code, it still has the scars of minification – single letter variable names and no comments. Time to start reading the code. Line 1 is just setting up the HTML for the user. Lines 2-4 is the first interesting piece. The array <code>Q</code> is being populated with all the ASCII characters, in reverse order! The characters <code>\n</code>, <code>\r</code>, <code>\\</code>, <code>'</code> and <code>"</code> are excluded, as are <code>\0</code> and DEL, so that Q has 121 characters. Rather than using a readable <code>if</code> statement, Aivo is using the fact that <code>&amp;&amp;</code> is <a href="http://en.wikipedia.org/wiki/Short-circuit_evaluation">‘short-circuiting’</a> in JavaScript. Much space saving here.<br />Next we come to the definition of <code>L</code>. Line 12 just removes blank lines, whitespace and single line comments (except those following code). It also escapes backslashes so that the code is ready to be put into a string later. This is assigned to the letters <code>i</code> and <code>s</code>. Be warned from here, in the goal of smaller size, variables frequently change their meaning to promote reuse. <code>s</code> is always going to point to the code, but <code>i</code> is used as a counter all over the place.<br />Next, <code>B</code> is half the length of the program, m is the empty string. Line 15 is where it starts getting interesting. The pattern:<br /><pre><code>encodeURI(string).replace(/%../g, 'i')</code></pre>occurs thrice in the code. Its task is to get the <i>byte length</i> of the string rather than the <i>number of characters</i> that <code>string.length</code> gives. In ASCII there is no difference, but if there are Unicode characters, they may occupy 2-4 bytes. <code>encodeURI</code> will replace each byte with a ‘%xx’ code with <code>xx</code> being the hexadecimal byte value. Replacing this with the single letter ‘i’ will get us one ‘i’ for every byte, so that the length of the resulting string is the <i>byte length</i>. This was one of the many clever tricks present in the JSCrush code. They might be well known, but this is the first time I came across it.<br />The initialization in the for loop is only to save a byte, it does not affect the loop itself in any way. Similarly the <code>m = c + m</code> call can be moved to the end of the loop body. This construct will generate the decompression sequence contained in <code>$</code>. This for loop is then actually an infinite while loop.<br />Line 43 is again a trade-off of readability for size. Here it is in a cleaner form:<br /><pre><code>c = 0<br />i = 121<br />while (!c &amp;&amp; i) {<br /> if (! (~s.indexOf(Q[i])))<br /> c = Q[i]<br /> --i;<br />}</code></pre><code>~</code> is binary NOT. If Q[i] is not found in the source, then <code>indexOf</code> will return -1, <code>NOT -1</code> = 0 and <code>!0 === true</code>, so that this code is actually saying:<br /><pre><code>For every character Qi in Q in reverse:<br /> If the source does NOT have the character in it:<br /> c = Qi</code></pre>Or, c is set to an ASCII character that is not present in the program. Initially it will be ASCII 1, then perhaps 2 and so on. This ‘c’ is now the character that will be used to join the pieces obtained in Lines 20-32. This is one <b>round</b>. When all the characters have been used up, compression stops (Line 18).<br />Lines 12-32 basically try to find long, repetitive strings that can be replaced with a single character, to get the best compression. JSCrush follows a brute-force approach to find these segments. With single variable names, the code is a mess, so here is a cleaned up version which makes things much clearer:<br /> <script src="https://gist.github.com/2393861.js?file=jscrush-compress-pseudo.js"> </script> Lines 9-28 try to find segments which repeat atleast twice in the code. Longer segments will give better compression, so we try all of them. For segments of length 1, we try every character in the string, for segments of length 2, we try every pair and so on. If it repeats we keep track of the segment count.<br />The <code>segmentLengthBound = longestSegmentLength</code> (<code>B=Z</code>) bit is interesting, and it took me some thinking to figure it out. It relies on the following facts:<br /><ul><li>The longest segment in the current source is <code>longestSegmentLength</code>.</li><li>Splitting by something, and then joining by a character <i>not in the source</i> will not lead to creation of longer segments.</li></ul>So we can restrict segmentLength of the next round to <code>segmentLengthBound</code>.<br />Lines 32-41 choose the best segment to substitute in this round. The expression <code>(R=o[i])*j-R-j-1</code> may seem cryptic, until you look a little later in the code where the split and join is done, and you remember how JSCrush works. <code>R * j</code> is the number of bytes we will remove by replacing this segment. But to join the split, we’ll need one character for every repetition, followed by one character to separate the segment suffix itself. The conditional asks if this leads to actual, and better, compression than what we already know of. If no such segment was found, we are done compressing. Otherwise we split by the segment, join the pieces by the <i>join</i> character and tack on the segment at the end. One round done!<br />Once multiple rounds have been done, the script is compressed, and only some trivial things remain. The value of <code>B</code> is now changed to store the quotes (<b>double</b> or <b>single</b>), based on which are fewer. Since the compressed program is stored in a string, using the quotes that appear less times means less ’\’ to substitute, each of which costs a byte. We then prepare the boilerplate, setting <code>_</code> to the now compressed source, setting <code>$</code> to the decompression sequence <code>m</code> and adding the evaluation code. The savings accomplished are announced too.<br />One trick I picked up in the code is forcing a certain digit view precision.<br /><pre><code>i/S * 100</code></pre>would give a float percentage with many digits after the floating point. Instead multiplying by 1000, gets us two digits in the integral positions, bitwise OR-ing with 0 casts to an integer, losing the floating point digits, then dividing by 100 gets us the two digits we want.<br /><br /><h2 id="tldr"><strong>Summary</strong></h2>JSCrush works by:<br /><ol><li>Finding the first unused ASCII character to act as the <i>join</i></li><li>Finding the substring of the program text that gives the best space savings if its repetitions are all replaced by the ASCII character from 1.</li><li>Splitting the <i>source</i> on 2 and joining the pieces using 1, tacking on 2 to the end. This string replaces the original source.</li><li>Repeating 1, 2 and 3 until no more savings are possible or we’re all out of ASCII.</li><li>Wrapping the compressed source into a string, then using the list of <i>join</i> ASCII characters to unroll the string.</li><li>Unrolling is performed by splitting on every ASCII character used in 3, extracting the original repeated substring 2 from the split and joining the parts.</li></ol>I hope this (long) post was interesting and educational. If you have feedback, a comment would be great.<br />Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com6tag:blogger.com,1999:blog-3665064696228706104.post-9143701811597572622012-04-06T20:12:00.002+05:302012-04-06T20:12:33.254+05:30Playing with Go<p>With Go 1 being released, I&#8217;ve been playing with the language once again. As a long weekend hack, I created a clone of the literate programming <a href='http://jashkenas.github.com/docco/'>Docco</a> tool in Go &#8211; quite naturally called <a href='http://nikhilm.github.com/gocco/'>Gocco</a>.</p> <p>This blog post chronicles my feelings about the language, and some rough spots I got stuck in.</p> <p>I started out with a <em>direct translation</em> of the original CoffeeScript to Go. Go is remarkably unlike C and more like high level languages. Much of the translation was automatic and is not very different from the CoffeeScript source, except the pervasive use of <code>[]byte</code> rather than <code>string</code>.</p> <p>What I loved about Go was the ability to directly import and install packages from various hosting services like GitHub, Google Code and Bitbucket. This combined with the <code>go</code> command makes package management a breeze. What I would really like though is for Go to switch to default <em>local-to-global</em> search path semantics, like <a href='http://www.mikealrogers.com/posts/nodemodules-in-git.html'>npm</a>. You can do it using <code>GOPATH</code>, but making it the default would be better. I am also not sure how you can specify which version of your dependencies to install.</p> <p>The inbuilt templating support is also a great tool, I didn&#8217;t have to hunt around for a templating library, and the syntax was very clean and similar to Jinja2 or Mustache. There seems to be some bug in the <code>ParseFiles</code> method though, as it kept complaining that the template was incomplete or empty, but when I manually read the same file into a string and called <code>Parse</code>, it worked.</p> <p>The tooling support is excellent, with <code>gofmt</code> and <code>go</code> ensuring everything is &#8216;standardised&#8217; and makes others code much easier to read.</p> <p>Coming to the slightly wonky/undefined/undocumented/(may actually be my fault) parts&#8230;</p> <p>I still haven&#8217;t really figured out Go&#8217;s package system. The <code>go/build</code> package says it allows introspection over packages, like figuring out the installation path (Similar to Python&#8217;s <code>__file__</code>). But I couldn&#8217;t get this to work with my code, especially when it was not installed. I wanted it so I could keep the HTML template and CSS in separate files, and use the package path to figure out where they were kept. I finally got very frustrated and just stuck them into a go file as multi-line strings.</p> <p>Syntactically, multiple return values are a good way to signal errors without exception handling, there <em>really</em> should be a way to ignore the second, error, parameter. Not being able to do that, makes composing functions impossible. Similarly the unused variables and imports errors can get annoying while developing, when there is lots of stub code. There is a <a href='http://weekly.golang.org/doc/go_faq.html#unused_variables_and_imports'>workaround</a> but still! :)</p> <p>Still, it was a lot of fun using Go. I originally wanted to implement Docco in C, but the lacking standard library and types support was scaring me. Now I will reach for Go when it comes to systems programming.</p> <p><em>A thanks to Russ Ross, for an <a href='https://github.com/russross/blackfriday'>excellent Markdown implementation</a>.</em></p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-18729770263184523482012-04-02T03:42:00.001+05:302012-04-02T03:42:13.755+05:30Great Indie-an Acts<p><em>(This was originally published in Entelechy Edition 34, March 2012)</em></p> <p>With Synapse a few weeks old, the constant Raghu Dixit songs are starting to fade away from the hostel corridors. Except when we hear bands live, or are aware of an upcoming concert, Indian artists aren&#8217;t given much ear time. Bollywood dominates too much. But a growing independent music scene is flourishing in India, mainly due to rising economic levels and more people willing to pursue their dreams. Here are some great Indian artists that I&#8217;ve recently liked. Vishal Shah and <a href='http://www.indianmusicrevolution.com'>Indian Music Revolution</a> are instrumental in introducing some of them.</p> <h2 id='advaita'>Advaita</h2> <p>I first heard of Advaita and Swarathma in &#8221;<a href='http://thebigm.co.in/wordpress/tag/hindi-hein-hum/'>Hindi Hein Hum</a>&#8221;. Advaita term themselves as a &#8216;Rock/Eclectic/Organic Fusion&#8217; band from Delhi and are similar to The Raghu Dixit project. Finding their music is extremely hard, both legally and illegally because they haven&#8217;t launched their music on Flyte, and everywhere else is too expensive. I&#8217;ve been resorting to streaming from Spotify, but finally caved in and ordered one of their CDs from Flipkart (low-tech maybe, but also the cheapest). Not much is known about them, but they are an octet (geeky?) who had their breakthrough when John Leckie selected them (and Swarathma) as one of the four Indian bands in the India Soundpad project.</p> <p>Their debut album is <em>Grounded in Space</em>, released in 2009. <em>Ghir Ghir</em> is an amazing number on this album. Advaita makes very good use of some Indian classical vocals and raagas, while injecting western instruments and sound structures with the tabla and sarangi. In fact, <em>Ghir Ghir</em> has parts which wouldn&#8217;t sound out of place coming from <em>Dream Theater</em>.</p> <p>Perhaps their best song is <em>Mo Funk</em> from their latest album <em>The Silent Sea</em> (2012). Vocals (by Ujwal Nagar) and tones that would make my (stuck-up) school music teacher very happy, layered on a background score that is like trance or house music make this 6 minute piece a must-listen. 4 minutes into the song, western vocalist and acoustic guitarist Chayan Adhikari takes over and culminates the &#8216;Funk&#8217; part of the song.</p> <p>Other songs to start with are <em>Tremor</em> (SS) and <em>Drops of Earth</em> (GiS).</p> <h2 id='swarathma'>Swarathma</h2> <p>I think I&#8217;ve already introduced Swarathma in the previous section :) Swarathma is a 6-member ensemble from Bangalore (out of which 2 crack PJs). They&#8217;ve featured on The Dewarists Episode 7 with Shubha Mudgal.</p> <p>One of their SoundPad project compositions &#8211; <em>Yeshu, Allah aur Krishna</em> &#8211; goes &#8220;Sant Kabir aye dharti pe, &#8230;, jo socha tha woh reh gaya sapna &#8230;&#8221; and seems to reflect the band&#8217;s philosophy. Techies, B-school grads and other achievers, pursuing their passion and making their dreams come true. Their only album is self-titled and released in 2009.</p> <p>Leader <em>Jana Kahan Hai Mujhe</em> is about &#8220;choices we face at each step in our lives, and we are frequently at a loss when called upon to make a decision&#8221;. In contrast to Advaita, Swarathma has restrained instruments, a slow, low chord guiding the vocals, and a splash of tabla here and there.</p> <p>My other favourite is <em>Ee Bhoomi</em>. Kannada must be a happy language indeed, to produce gems like this one and Raghu Dixit&#8217;s <em>Lokada Kalaji</em>. This upbeat song describes the transformation of the Earth to Paradise (&#8230; <em>bhoomi swarga</em> &#8230;) as you let Swarathma wash over you.</p> <h2 id='peter_cat_recording_co'>Peter Cat Recording Co.</h2> <p>In a case of last but not the least, <a href='http://petercatrecordingco.com/'>PCRC</a> is the band I&#8217;ve actually been listening to the longest. They remind me of the 60s and 70s when Rock was being born and could be happy and everyday, not having to force melancholy or abstract ideas to be appealing. I&#8217;ve been a fan before they had a released album and were tracks on SoundCloud, and these guys doubled that by distributing both their albums ( <em>Sinema</em> and <em>Wall of Want</em>) free for download. The New Delhi quartet describes their music as &#8220;Gypsy Jazz to Ballroom Waltz to Midnight Moonlight car chase music&#8221;.</p> <p><em>Sinema</em> is their older and better album. All the songs have a grainy distortion throughout as if playing from vinyl, although whether this adds to their charm or not is a personal opinion. To quote <a href='http://helterskelter.in/2011/12/cd-review-sinema/'>Helter Skelter</a>:</p> <blockquote><p>That it screams ‘SEX SEX SEX’ right under the ‘Free Download’ link for their album Sinema on the Peter Cat Recording Co. web site should be argument enough for you to go ahead and download it.</p></blockquote> <p><em>The Clown on the 22nd floor</em> sets the mood for the album, with old hindi movie clips tacked on to the end. The album crafts a series of love affairs that end badly. Suryakant Sawhney&#8217;s rounded drawl adds warmth to the songs, while clever lyrics mingle girls and philosophy. Coming from a band obsessed with humour and subtlety closer <em>Tokyo Vijaya</em>&#8217;s lyrics just go &#8216;AAAaaaaaaaaaaaaaaaaaaaaaaaaaaaa&#8217; against a dense instrumentation of drums and dragged out guitars ending the story inconclusively. PCRC is a band that aims big and delivers funny.</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com0tag:blogger.com,1999:blog-3665064696228706104.post-88001352778578763642012-04-01T10:23:00.000+05:302012-04-16T10:43:03.689+05:30Apache Cassandra: Iterate over all columns in a row<p>Recently I have been using Cassandra for one of my projects, and one of the needs is to iterate over all columns of a row. Each column represents an individual data, of type identified by row id, and keeps changing. So I can&#8217;t simply use a set of known column names. Using the <code>setRange</code> call on a <code>SliceQuery</code> and setting a large <code>count</code> is also not an option, since Cassandra will try to load the entire set of columns into memory. Instead I&#8217;ve written this iterator which takes a query on which row key and column family has been set, and will load columns as they are requested. By default it loads a 100 columns at a time. You could make it take the count as a parameter and all, but this works for me for now.</p><script src="https://gist.github.com/2270971.js?file=AllColumnsIterator.java"> </script><p>The one &#8216;problem&#8217; with this is the removal of the last column to ensure that there are no duplicates, but still having a start point for the next query. This is because each column is independent, so you cannot ask a column who it&#8217;s next neighbour is and start the next query from there. If anybody has a tip to make it more elegant, I&#8217;d love to hear it.</p>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com2tag:blogger.com,1999:blog-3665064696228706104.post-57333652874938097032012-03-20T14:22:00.001+05:302012-03-20T14:22:45.835+05:30I'm scared for the Internet<i>(This article was written for Entelechy Edition 33 (February 2012). The news is slightly old, but posted here so I have a public permalink to the article) </i><br /><br />When the Internet began over 30 odd years ago, it was an ideal of democracy. Born in universities, where only meritrocracy ruled, it was used by hackers whose ideals were very egalitarian. In its very protocols, the Internet encodes equality. No piece of data is considered more <i>important</i> or more <i>dangerous</i> than any other.<br /><br />Even as the World Wide Web exploded some 20 years ago, it retained the ideals of the Internet, although in some sense these very ideals led to corners of the web which were just evil.<br /><br />Of course, as the Internet went global and required physical infrastructure and government permissions, what you could and could not do became much more controlled. Child pornography was plain illegal, but freedom of speech was not, often because much of the core architecture and servers were based in democratic countries who were benefiting massively from the Web.<br /><br />Of course the web came along and made piracy much much easier. As Paul Tassi <a href="http://www.forbes.com/sites/insertcoin/2012/02/03/you-will-never-kill-piracy-and-piracy-will-never-kill-you/">states in Forbes</a>, piracy is a 4 step process, while any conventional means of media consumption from the Old Big Media Houses is way more inconvenient. With laws differing across borders, delayed releases, abominations like DeCSS and so on, it was far easier to use a general purpose computer to circumvate all these measures.<br /><br />In the last few years however, piracy and freedom of speech has come back to bite us. Rather, it has come to haunt the old bastions of power, governments, religions and media publishers. Faced with a series of attacks, the Web is now caught between the devil and the deep sea. On one side is the battle for intellectual property. On the other hand are repressive governments (even so-called democracies) where certain factions wish to curtail the one medium that is impossible to truly shutdown.<br /><br />There are also the silos of Facebook, Google and a thousand walled App Gardens that are eating mouthfuls of our data and keeping it behind closed doors. They aren’t the worst part though.<br /><br />I’m not scared that the Internet will die, but I’m scared that it could lose its essence. The essence of freedom, of equality, of opportunity and communication that has brought hope to many and improved countless lives immeasurably. If we the common people keep lying down as our freedoms are taken from us, we will lose the one chance to truly step into something new, to fully embrace human potential, and go back to watching the shiny rainbows on Blu-ray discs whose only use is as coasters. It is not as if we are powerless, we just need to be educated. After all in the case of SOPA/PIPA, mass protests were effective in sending the bill for reconsideration.<br /><br />It is no secret that politics is financed by capitalists. Much of this money flows in from media conglomerates and thus politicians are puppets when the MPAA or RIAA decide to take out their battle axes on pirates. What the old media does not realize is that the Internet has levelled the playing field. It is now possible for anyone to publish high quality content. Rather than focusing on making it cheap and easy to spread their content, so that most users will be willing to pay for it legally, they continue to impose draconian laws on sharing, copying and accessing their media. The solution – piracy. When technologists like Netflix, Hulu and Amazon try to make it ever easier, they crawl back into their shells. A perfect example is HBO’s <a href="http://theoatmeal.com/comics/game_of_thrones">Game of Thrones</a>.<br /><br />There are fundamental problems with new bills that are tabled to <i>deal</i> with piracy. These laws are often framed in secret in a nexus of politics and old media houses, most notably ACTA (the Anti-Counterfeiting Trade Agreement), rather than in public scrutiny where rights groups and the general public can see what is happening. It was only as ACTA started to get ratified around the world that people realized what is happening. The result – plenty of EU countries have seen protests that have led to ACTA being put on hold. What is laughable is that the United States actually <a href="http://news.cnet.com/8301-13578_3-10195547-38.html?tag=mncol;txt">rejected Right to Information requests</a> saying that it could be a <i>threat to national security</i>. The second problem is that all copyright laws have always focused far too much on tightening copyright regulations, and what it means for something to be copied, what requires royalties and what requires permission. Copyright was initially a way to <i>give creators sufficient returns for their works</i>, not <i>how can I get the maximum money out of this</i>. By tightening the noose, it is getting harder for artists to re-use other’s creativity to create even better works. In addition every anti-piracy effort has put costs on innocent third-parties, taken huge cuts from taxpayers money, and pushed technology that had non-infringing uses out of the market. For example, BitTorrent is a fascinating technology with huge potential for better Internet services, yet it has achieved negative connotations due to its use for piracy. In the aggregate, they reflect a disproportionate focus on the interests of a handful of large companies. It’s hard to think of a single example during this twenty-year period of copyright restrictions being repealed, relaxed, or any in any meaningful way liberalized. Finally, there is a great paradox between the Western world’s constant demands for the right to free speech on the Internet and the principles embodied in stronger copyright laws. For example, SOPA’s <i>feature</i> of allowing the shutdown of arbitrary websites without judicial hearing, ACTA’s removal of safe harbour protections and the outlawing of circumvention technology leads to the internet quickly toeing the line with new regulations. Ironically, even as SOPA seeks to outlaw technology like <a href="http://torproject.org/">Tor</a>, the US Department of Defense actively funds its development to help activists in repressive regimes like Iran.<br /><br />The Internet has now reached its prime, e-commerce is commonplace, startups for media distribution are blossoming all over, our identities are now better known by Facebook and <a href="http://blogoscoped.com/archive/2007-09-17-n72.html">Google</a> than by our Governments. Power and opinions flow over wires, free of the meddling of <i>higher-ups</i>. This is leading to politicians and other old bastions of power (cable TV, telephony) feeling lost, and so they are taking concrete steps to clamp down on what they consider a menace. They then repackage it and sell it to the public as ‘stealing’ or ‘content that can cause public unrest’. Caught between these two forces, the Internet could become a nanny state in the next few years.<br /><br />Over 20 years after an international agreement that deregulated the Internet and led to a meteoric success story of capitalism and free markets above all else, the United Nations plans to establish “international control over the Internet”. In December 2012, Russia, China and others will push for:<br /><ul><li>allowing ISPs to charge ‘international’ fees for Internet traffic - this is just absurd. By its very nature the Internet is not supposed to have ‘boundaries’. It goes completely against <a href="http://en.wikipedia.org/wiki/Net_neutrality">Net Neutrality</a></li><li>Subsume under governmental bodies many of the tasks of the Internet Engineering Task Force, the Internet Society, the Internet Corporation for Assigned Names and Numbers and others. These bodies currently operate solely on merit, technical competency and democratic votes, and as private bodies are free from governmental interference.</li></ul>among others. Censorship is emerging stronger and stronger. Censorship across the world has been well covered, and I will not go into the details. Two things are relevant though. One is #IdiotSibal’s attempts at getting social networks to take down ‘offending’ content. It seems in our bid to compete with China, we’ve decided to one-man up them in censorship too. Shivam Vij has very correctly stated in <a href="http://indiatoday.intoday.in/story/kapil-sibal-for-monitoring-offensive-content-on-internet/1/163107.html">Kapil Sibal doesn’t understand the Internet</a>,<br /><blockquote>So Sibal and Tharoor think social media can cause riots, but it hasn’t actually done so yet. Now that Sibal and Tharoor are telling us there’s stuff out there that could make us kill each other, some of us will go looking for it out of curiosity and…</blockquote>and<br /><blockquote>In neighbouring Pakistan, every Tom, Dick and Harry with complaints of online hate speech approaches the Lahore High Court. In India, Kapil Sibal wants to be the high court.<br />He wants to be judge, jury and executioner. And he wants to do it silently so we don’t get to know.</blockquote>Google’s <a href="http://www.google.com/transparencyreport/governmentrequests/IN/">Transparency report</a> on India clearly identifies politicians lack of ability to accept criticism.<br /><blockquote>In addition, we received a request from a local law enforcement agency to remove 236 communities and profiles from orkut that were critical of a local politician.</blockquote>In the private sector, Reliance Communications has taken it upon itself to be the moral and economic guardian by obtaining ‘John Doe’ orders from a local court to ban file sharing websites in the days around a movie release. In sheer violation of Indian law which states that only the Department of Information Technology may request censorship, they then went and blocked websites country wide. It was as if SOPA was already passed in India. A John Doe order is the type of insanity that you think can only happen in movies until some lawyer actually dreams it up. Multiple, unknown, offenders can be acted against. Interestingly while Reliance stated that they were within law, no actual complaint was recieved. The court order and ban was based on speculation. I sent a Minority Report Precrime in action.<br /><br />In retrospect there are 3 things (amongst others) that threaten the future of the Internet as a ‘commons’: Net neutrality, draconian copyright-laws and censorship. There is only one thing stopping that from happening - <b>YOU</b>.<br /><br />Further reading:<br /><ul><li><a href="http://www.free-culture.cc/">Free Culture: How Big Media Uses Technology and the Law to Lock Down Culture and Control Creativity</a> by Lawrence Lessig</li><li><a href="http://www.theatlantic.com/technology/archive/2012/02/if-not-orwell-then-huxley-the-battle-for-control-of-the-internet/252792/">If Not Orwell, Then Huxley: The Battle for Control of the Internet</a></li><li><a href="http://www.theatlantic.com/technology/archive/2012/02/enough-already-the-sopa-debate-ignores-how-much-copyright-protection-we-already-have/252742/">Enough, Already: The SOPA Debate Ignores How Much Copyright Protection We Already Have</a></li><li><a href="http://online.wsj.com/article/SB10001424052970204792404577229074023195322.html">The U.N. Threat to Internet Freedom</a></li></ul>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com1Gandhinagar, Gujarat, India23.1876281 72.6271016-5.5924409 32.1974141 51.9676971 113.0567891tag:blogger.com,1999:blog-3665064696228706104.post-16205297833631186162012-03-10T11:47:00.001+05:302012-03-10T11:47:05.960+05:30The TT Incident<div class='posterous_autopost'><p>This incident is about those moments of pure magic that sometimes happen when two individuals have light bulbs go on in their head at the same instance based on a series of earlier shared experiences and context. No, not the soul mate sort. To convey the expression requires introducing you, the reader, to the back story. In essence this post is about me and <a href="http://experimentingwithidiotism.blogspot.com">Naman&rsquo;s</a> library adventures. It is a set of confessions which I hope will not get us kicked out of the library now that there is a change of guard.</p> <p>Naman&rsquo;s animosity with the library began sometime in the fourth year after Mr. K&rsquo;s incompetency started getting on his nerves. Then, quite justifiably, he got banned from using the internet in the library for 5 days for stealing Mr K&rsquo;s IP address for one full day and causing his work to stop. Although Naman claims he did not know whose IP it was he didn&rsquo;t mind the happy co-incidence. That was the prick in the balloon! I have always been somewhat annoyed with the library&rsquo;s arbitrary rules, how the staff is a stickler for them even under special circumstances and how easy it actually is to circumvent them. This was also, then, an experiment in how far we could go without being caught, although all these are just excuses to deny the fact that we wanted some childish fun and narcissistic attention. Our library violations begin with this SMS from Naman to me, dated August 31, 2011, 11am:</p> <blockquote class="posterous_medium_quote"> <p>Who is the most fucking awesome? Bet #1. Who can get the most outrageous food items in library till they get caught. In other terms, who can smuggle food inside rc the longest before getting caught.</p> </blockquote> <p>It didn&rsquo;t take too long for Naman to start. He arrived a few days later into the library, all smiles, pockets bulging. Out of one packet same a sandwich. Mr. V. wandered in and set a crumpled plastic cup in front of Naman. I have to admit that I wasn&rsquo;t too impressed yet. But then out of the second pocket came a plastic bag with tea, and he poured it into the cup and coolly had his breakfast in the library. In all his smugness he went and publicised his achievement :) Not to be outdone, I convinced certain people to go eat pizza with me that very night (this is a lie). When Naman wanted his delivered to him, I took it seriously and delivered it in the library. One Margherita eaten right under the watchful eyes of Dr. K.</p> <p>Score TFA 1 - 0 library. (We are called Team Fucking Awesome because we were at that point inspired by Julien Smith&rsquo;s <a href="http://inoveryourhead.net/the-short-and-sweet-guide-to-being-fucking-awesome/">The Short and Sweet Guide to Being Fucking Awesome</a> and took it very seriously)</p> <p>In time our food escapades fizzled out and we returned to the standard supply of biscuits, chocolates and chips to keep our hands busy while our minds (sometimes) churned over (a bit of) academic knowledge. Our dream still remains incomplete though: <em>cook a packet of instant noodles in the library and consume it there</em>.</p> <p>Another day, a group of third year girls walked into the library after having celebrated someone&rsquo;s birthday. They came armed with those annoying birthday trumpets (sometimes sweetly called a <em>bhoppu</em>). We had a hoot blowing raspberries loudly in the library, where silence is a law of nature. Hardly content with this, we stepped outside and (amongst raucous laughter) recorded this sample:</p> <p><iframe src="http://www.youtube.com/embed/C6loprHZ6I8" frameborder="0" height="315" width="560"> </iframe></p> <p>Then we went back to the library and forgot about it. While leaving we ensured this was played loudly and completely, but acted as if we didn&rsquo;t know what was happening. Thus we left, taking a bit of everyone&rsquo;s <a href="http://blog.nikhilism.com/2011/11/high-five-experiment.html">stress</a> with us.</p> <p>TFA 2 - 0 library.</p> <p>Every one of these incidents have been during an exam, when the library is filled with students studying for their exams. I am not sure how it reflects on our character that we perform these acts when we should be hitting the books or letting others study in peace. We like to think we&rsquo;re just injecting some fun into the otherwise serious atmosphere.</p> <p>Towards the beginning of the final semester &ndash; Feb 7, 2012, 8:20pm to be precise &ndash; I had come for dinner and was going directly to the library. Naman had been playing table-tennis and directly arrived for dinner. He was using my rackets and ball, so he handed them over hoping I would lug them along everywhere I went. After a bit of discussion over who should keep them, he made the (rational) point that I should carry them and put them in my bag.</p> <p>There was a flash moment, the kind of epiphany that was experienced by Einstein or Newton, when all their years of thinking collapsed into one beautiful result instantly. I had table-tennis rackets and a ball, I was going to the library&hellip; I had to just utter &lsquo;Hey&rsquo; and it clicked instantly in Naman&rsquo;s head too. We high five-d each other and let out a huge laugh, reveling in how the perfect next library prank had formed. The whole caf&eacute; was staring at us as we started in jubilation. The idea was set up, time for execution.</p> <p>The equipment was rolled in, the surveillance was set up and the stunt done that very night. Too bad the library was filled mostly with geeks at that time. Hardly anyone raised their heads to pay us any attention.</p> <p><iframe src="http://www.youtube.com/embed/NZISyb5oiAE" frameborder="0" height="315" width="420"> </iframe></p> <p>TFA 3 - 0 library.</p> <p style="font-size: 10px;"> <a href="http://posterous.com">Posted via email</a> from <a href="http://bombayrain.posterous.com/the-tt-incident">nikhil's posterous</a> </p> </div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com5tag:blogger.com,1999:blog-3665064696228706104.post-88019443178936740652012-01-29T12:08:00.001+05:302012-01-29T12:08:38.164+05:30The IT Crowd?<div class='posterous_autopost'><p><em>(This article was originally published in Entelechy, edition 32, Jan 2012. It is being published here in full with some annotations.)</em></p> <p>It has continued to surprise me over the last 3.5 years how few information technology students actually bother to use the innovations of information technology to improve their productivity in any manner. More importantly, they are usually unaware of the products themselves. Recently the issue was brought to the fore when <a href="http://sevobs.wordpress.com/about/">Skish Champi</a> pointed out that <em>Zimbra Collaboration Suite</em> had great calendar integration (and I agree), and we as a college are still struggling around with sending meeting emails and reminders.</p> <p>We are all experts at using <a href="http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FDirect_Connect_%28file_sharing%29">DC++</a> to share files over our network. Yet, when it comes to sending the same files over the Internet you still stick to <em>(gasp)</em> e-mail. If you send me a 50Mb file over email in 2012, I&rsquo;m going to knock on your door with a gun in my hand. Use <a href="http://www.dropbox.com">Dropbox</a> or a thousand other such services. You only need to upload once, your multiple devices can continuously sync the files, individual folders can be shared with individual people &mdash; this is great for working on projects and such. In addition Dropbox keeps old versions around. Heck, using the Public folder you can even host a complete static website without paying a paisa! Once your file is on Dropbox you can go trigger happy with the public link and send the link in the e-mail instead. Less load for the e-mail servers, and the link can easily be shared via any other medium too.</p> <p>Similarly, if you are in charge of planning a lot of events (looks at the committees and clubs and faculty), create a new Calendar event in Zimbra and send it to all batches. That way a student simply has to accept the invitation, and he will also occasionally get reminders. Synced with a desktop Calendar application, you can even have your computer play sounds or show messages even when webmail is not open. Now you have no reason to forget or be late for a meeting.</p> <p>If you are working on a software project, and your way to &lsquo;work as a team&rsquo; on the code is to ship around newer versions of the files to everyone via e-mail, your project is already dead. Why? What happens when you suddenly need an older version? Or one feature from Amrita and the other one from Rajni? Spend your time copy-pasting? Let a <a href="http://mercurial.selenic.com">version</a> <a href="http://www.git-scm.org">control</a> <a href="http://subversion.tigris.org">system</a> do it for you!</p> <p>I&rsquo;m going to mangle <a href="http://www.faqs.org/docs/artu/ch01s06.html">The Unix Philosophy</a> slightly to suit my purposes:</p> <ul> <li>Use software that does one thing, and does it well.</li> <li>Use software that does not lock your data in, or modify it so no other software can use it. A hyperlink should always be available.</li> <li>A software which allows its output to be the input for another program, is usually better than one that doesn&rsquo;t. An API can do wonders.</li> <li>Understand your work flow. Don&rsquo;t hesitate to throw away the clumsy parts.</li> <li>Use tools in preference to manual labour, even if you have to detour to build the tools.</li> </ul> <p>So why do we continue to use legacy technology, mouse around and generally not maximise our use of the computer?</p> <p>One reason is of course <em>inertia</em>. Our biological brains are always trying to survive and if one thing works we aren&rsquo;t willing to go improve. Another is a lack of curiosity about wanting to hear about new technology.</p> <p>But it is also an <em>attitude</em> problem. Technology continues to be the poor step-daughter. If someone shows you how to skin potatoes faster, you&rsquo;ll quickly adopt the technique. But as soon as the metaphor moves onto the computer, there is immediate unwillingness. From the very beginning of our computer education we&rsquo;re asked to treat the computer as some magic box, and restricted from doing anything outside of that <em>&lsquo;education&rsquo;</em>. <em>&ldquo;Usse haat mat lagana nahi to kharab ho jayega.&rdquo;</em> is what your parents/teacher says. (English: Don&rsquo;t touch that, you&rsquo;ll break it). We carry over this belief that computers are frail creatures even when we become more responsible. The second reason is that the internals of computers continue to be treated as unknowns for much of the population. Most people who drive a car have a qualitative idea of how an internal combustion engine works. They also know how to change a tire. But the same people don&rsquo;t know how to add RAM or the basics of a processor. Worldwide, <em>computer literacy</em> focuses on office suites and the like. Even when you do eventually study the internals, they remains something from the textbook, so that whenever your C program is malfunctioning you don&rsquo;t once bother to think in terms of how that program is being interpreted in a certain way, and how that can help solve the problem. This disconnect is alarming, I&rsquo;ve seen computer science professors being unaware of basic computer features. The funniest example I can think of this is when Windows users keep right clicking on the desktop and hitting <em>Refresh</em> when things aren&rsquo;t moving.</p> <p>The media is also responsible. World news is quick to highlight the policies and laws that will affect say, retail, or corruption or some industry. Technology media however is focused on <em>product reviews</em> and the <em>next revolutionary technology</em> (which is usually some old concept re-hashed), and sidelines <em>actual problems</em> which will impact privacy, ownership and other fundamental rights in the digital age, so that the notion of <em>computing</em> in itself is never given enough attention by the general public. If car manufacturers specified which brand of fuel you could use in their cars, there would be a hue and cry over anti-competitive behaviour and not giving choice to people. Yet mobile phone carriers fleece people every day with locked, underpowered cell phones. The <a href="http://boingboing.net/2012/01/10/lockdown.html">war on computation</a> keeps going the wrong way.</p> <p>I also believe, that just as in mathematics, computers require significantly more complex mental models to be manipulated in the mind. A car engine is heat and metal and chemicals and can directly be observed doing something. The levels of abstraction between the electrons racing through the wires and the point and click metaphor of daily interaction are numerous in comparison. As a user you do not of course need to have any inkling of them, but even at the software layer, a computer desktop is much more <em>congested</em> and much less <em>tangible</em>. Most normal people seem to keep missing subtle user interface clues that convey meaning. To those with the mental ability to hold these models and think rationally and logically (the only way in which the computer can think), the paradigms are much clearer.</p> <p>But as ICT students you are expected to have that mental ability. The <strong>first</strong> thing to do is to lose fear of the computer. Modern operating systems and applications are robust enough so that they won&rsquo;t go down just due to clicking in the wrong place. Experiment with preferences, try buttons which have only icons, many a feature can be discovered this way. <strong>Second</strong>, internalise the knowledge as much as you can by always trying out new things yourself, until they become second nature. <strong>Third</strong>, think through what you are doing rather than just clicking as your friend told you to. <strong>Fourth</strong>, remember that like in all engineering disciplines, <em>convention</em> is implicitly followed in computing as well. So concepts you learn in one application (say drag and drop) can be generalized and applied everywhere. <strong>Fifth</strong>, start thinking of computing tasks as recipes. In cooking, you apply a series of <em>tools</em>, to transform various <em>inputs</em> (ingredients) to the final outcome. The output of the knife becomes the input of the frying pan. On the other hand, most software normal people use tends to be monolithic, one tool that will do all the steps. Good software on the other hand has separate knives and separate frying pans and allows a great deal of flexibility. The Unix command line is the most pervasive and powerful example. <strong>Sixth</strong>, keep your eyes open for new ways of better using technology. Blogs like <a href="http://www.lifehacker.com">lifehacker</a> are an excellent source of such tips. <strong>Seventh</strong>, pay attention when governments and companies try to cripple your right to compute and the right to information.</p> <p>With a little active effort, you streamline your computing experience, so that you can devote complete attention to the fantastic things you create.</p> <p style="font-size: 10px;"> <a href="http://posterous.com">Posted via email</a> from <a href="http://bombayrain.posterous.com/the-it-crowd">nikhil's posterous</a> </p> </div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com6tag:blogger.com,1999:blog-3665064696228706104.post-90454705437195334652012-01-09T17:18:00.001+05:302012-01-09T17:27:17.370+05:30Toggling Proxy settings quickly with Pentadactyl and Firefox<div class="posterous_autopost">If you use the excellent <a href="http://dactyl.sf.net/">Pentadactyl</a> plugin for <a href="http://www.firefox.com/">Firefox</a> to get vim super-powers to the browser, here is a quick and painless way to toggle between a direct connection to the internet or using the default proxy settings. Add this to your <code>~/.pentadactylrc</code><br /><div class="CodeRay"><div class="code"><pre>command proxy -nargs=1 :set! network.proxy.type=&lt;args&gt;<br />nmap up :proxy 1&lt;CR&gt;<br />nmap np :proxy 0&lt;CR&gt;</pre></div></div>Now pressing <code>up</code> (<b>U</b>se <b>P</b>roxy) will enable <i>Manual Proxy Settings</i> while pressing <code>np</code> (<b>N</b>o <b>P</b>roxy) will use <i>Direct Connection</i>. <code>network.proxy.type</code> can take <a href="http://kb.mozillazine.org/Network.proxy.type">other values</a> which might be suited to your setup. You can change the key bindings too.<br /><div style="font-size: 10px;"><a href="http://posterous.com/">Posted via email</a> from <a href="http://bombayrain.posterous.com/toggling-proxy-settings-quickly-with-pentadac">nikhil's posterous</a> </div></div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com4tag:blogger.com,1999:blog-3665064696228706104.post-78569941297379190442011-12-22T23:37:00.001+05:302012-01-13T09:01:33.373+05:30Tic-Tac-Toe with Redis and Backbone.jsThis post is a technical overview of my <a href="http://tictactoe.nikhilism.com/">Tic-Tac-Toe</a> implementation. It is a <b>zero server-side logic</b>, <b>pub-sub</b> based, <b>realtime</b>, multiplayer game which uses <a href="http://redis.io/">Redis</a> and <a href="http://documentcloud.github.com/backbone/">Backbone</a> as the key enablers of real-time and moving logic to the client side, respectively. Now that all the buzz-words are out of the way, the source code is on <a href="https://github.com/nikhilm/tictactoe">Github</a>. There are rough edges with reliable communication, security holes do exist, but for the most part it works well.<br />I would specially like to thank <a href="http://twitter.com/yowgi">Nicolas Favre-Félix</a> for <a href="http://webd.is/">Webdis</a> without which this would’ve been impossible.<br /><h3> Basic functioning</h3>On visiting the <a href="http://tictactoe.nikhilism.com/">home page</a> the user has an option to either join a game, or play with a friend. The join game option pairs the player with another player who also wants to play (if you are the only one online, you’ll have to wait). If you play with a friend you get a link you can send him/her so you can play together.<br />The players are asymmetric in the sense that there is a ‘hoster’ and a ‘joiner’, whose roles I’ll get into in a bit.<br /><h3> Technology Stack</h3><a href="http://tictactoe.nikhilism.com/">Tic-Tac-Toe</a> game me the opportunity to experiment with a ton of technologies/products I hadn’t handled before. The stack goes like this.<br /><ul><li>A <a href="http://linode.com/">Linode</a> VM hosts the service</li><li><a href="http://redis.io/">Redis</a> powers base Pub/Sub</li><li><a href="http://webd.is/">Webdis</a> provides a REST API to <a href="http://redis.io/">Redis</a> so that the clients can directly talk to it</li><li><a href="http://nginx.org/">nginx</a> serves static files</li><li><a href="http://haproxy.1wt.eu/">haproxy</a> routes requests to nginx or Webdis depending on the path</li><li><a href="http://documentcloud.github.com/backbone/">Backbone</a> is used for MVC</li><li><a href="http://raphaeljs.com/">Raphaël</a> is used to draw the grid and pieces using SVG</li><li><a href="http://jquery.com/">jQuery</a> is used for AJAX and the <a href="http://trentrichardson.com/Impromptu/index.php">impromptu</a> is used for modal dialogs</li><li><a href="http://documentcloud.github.com/underscore/">underscore</a> is a utility belt</li><li><a href="https://github.com/LiosK/UUID.js">UUID.js</a> for generating UUIDs</li></ul>Yep, thats a lot of stuff for something so simple, so I’ve tried to rationalize it below :P<br /><h3> Clients</h3>All clients are identified by a UUID. The ‘host’ player’s UUID is used as the name of the Redis Pub/Sub channel. The ‘joiner’ will also subscribe to this channel and publish to it. This is why the ‘host’, ‘join’ bifurcation, so that a common channel can be used for communication. All messages are JSON objects.<br />All communication is initiated by the ‘joiner’. The host player keeps waiting. The joiner starts the game, after which both players keep sending each other the moves made by the humans playing. After each move, both sides check for a win/lose/draw. Again, the joiner sends a <i>gameover</i> message when it detects somebody has won (or it’s a draw). The host then <i>verifies</i> that this is actually true, and responds back with it’s own <i>gameover</i> message. Then both parties change their UI accordingly. This causes the slight delay between the actual win and the notification. <br /><a href="http://www.flickr.com/photos/code_kreator/6554180495/" title="WSD for Tic-Tac-Toe by Nikhil Marathe, on Flickr"><img alt="WSD for Tic-Tac-Toe" height="386" src="http://farm8.staticflickr.com/7011/6554180495_52df260d2e.jpg" width="340" /></a><br /><h3> Server side</h3>The first question is <i>‘why both nginx and haproxy?’</i> I could’ve run Webdis and nginx on two different ports since Webdis supports <a href="http://en.wikipedia.org/wiki/Cross-Origin_Resource_Sharing">CORS</a>, but <a href="http://www.opera.com/">Opera</a> does not support it. Running just nginx web-facing and then proxying to Webdis based on path is also not possible since nginx does not currently support HTTP chunked replies, while haproxy does. Chunked replies are used by Webdis to relay Pub/Sub messages.<br /><h4> Implementing ‘Join Game’</h4>To allow two people to be paired, a Redis list is maintained. When a client clicks Join Game it attempts to LPOP a UUID off the list. If none are found (no other players), it RPUSHes itself onto the list.<br /><h3> Client side</h3>The client side has all the logic and so is more interesting. I won’t go deep (you can <a href="https://github.com/nikhilm/tictactoe">read the source</a>), but will discuss the key areas.<br />Once a game has been initiated, both clients open a XMLHttpRequest to the subscriber channel. They watch for <b>readystatechange</b> events and try to parse the messages. This is the <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/tictactoe.js#L4"><b>Subscriber</b></a> which <a href="http://documentcloud.github.com/backbone/#Events-trigger">fires events</a> when it receives valid messages.<br />The <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/tictactoe.js#L29"><b>Publisher</b></a> component is used to send messages, while <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/tictactoe.js#L43"><b>Webdis</b></a> is used for other Redis commands (currently only LPOP and RPUSH).<br />The <b>GameRouter</b> is the <i>controller</i>, sets up models and views, watches for <i>gameover</i> and beginning a host or join. HTML5 pushState or hashes are used to offer <i>permalinks</i> for games. <code>/host/UUID</code> is for hosting and <code>/play/UUID</code> is the joiner. Backbone’s <a href="http://documentcloud.github.com/backbone/#Router">Router</a> and <a href="http://documentcloud.github.com/backbone/#History">History</a> is used to cleanly handle this. One feature I wish there was, is a way to query what the current route is via Backbone instead of mucking with <code>window.location</code>.<br />The <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/models.js#L1"><b>GridModel</b></a> drives the game once it begins, triggering <i>gameover</i> events and checking the grid after every move, also handling the turns. <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/views.js#L81"><b>GridView</b></a> handles rendering the SVG grid and pieces based on GridModel and processes clicks while <a href="https://github.com/nikhilm/tictactoe/blob/master/lib/views.js#L162"><b>HUDView</b></a> notifies the user of his turn, piece and game state. Events are an integral part of this entire setup and Backbone makes it very simple and modularized.<br />One thing that stands out in my use of Backbone is the absence of <a href="http://documentcloud.github.com/backbone/#Sync">Sync</a>. I’ve not used it, although it integrates well with Backbone. The game model didn’t seem suited to it, being more <i>event-based</i> rather than the model reflecting the game state. <a href="http://documentcloud.github.com/backbone/#Collection">Collection</a> is also not used since there is only one grid.<br /><h3> Issues</h3>The lack of any server side code does lead to certain issues. None of them are serious when it comes to Tic-Tac-Toe but some affect the user experience and others are security issues.<br /><h4> Usability</h4>On the usability front, the current code is very <i>fickle</i>. The joiner only tries to initiate the connection the first time it starts. If the host fails to respond for some reason (e.g. network connectivity) then both parties will keep waiting. If a joiner refreshes his page mid-way through a game, the game will restart for both parties (a good idea if you are a joiner and you are losing \:P). If the host refreshes the page, the joiner has to refresh after him.<br />The slight lag to decide win/lose/draw was probably unnecessary. Rather than verifying both sides are on the same page, the notification could’ve been shown directly since this is just a game. The UX would’ve benefited.<br /><h4> Security</h4>Webdis sports a HTTP interface to Redis, but there is no real authentication support, which means the Redis instance hosting Tic-Tac-Toe is also effectively a public domain database (although the command set is restricted). Ideally this should run only in a trusted intranet. Ideally Webdis itself would be capable of serving files and deliver something like a nonce which it would then use to ensure that only its own connections are allowed to relay messages. Similarly nothing is stopping someone from grabbing the UUID and then playing future moves using a bot of some kind to always play optimally. In the case of Tic-Tac-Toe this is just a minor prick, but for actual client-side MVC applications this requires fixing.<br /><h3> Conclusion</h3>Tic-Tac-Toe demonstrates that with a data structure server and HTTP interface to it, web applications where all logic is client-side are possible. Using the server as a Pub/Sub relay also allows near real-time performance, with WebSockets or SPDY possibly leading to better performance. Security policies still mean that true peer-to-peer isn’t possible. Authenticity and authorization also remain to be solved. On trusted intranets, such applications could be used for non-critical tasks. Meanwhile, games like this at least can be safely implemented and played as long as you have no sore losers :).<br /><div style="font-size: 10px;"><a href="http://posterous.com/">Posted via email</a> from <a href="http://bombayrain.posterous.com/tic-tac-toe-with-redis-and-backbonejs">nikhil's posterous</a> </div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com2tag:blogger.com,1999:blog-3665064696228706104.post-78630729347698501792011-12-13T17:10:00.001+05:302011-12-13T17:10:53.592+05:30Why Indian students should attend college<div class='posterous_autopost'><p>In recent months, the number of <a href="http://www.hnsearch.com/search#request/submissions&amp;q=title%3A%28drop+out%29&amp;sortby=create_ts+desc&amp;start=0">posts extolling dropping out</a> of college, or of people recounting their experiences (mostly positive, probably because the negatives won&rsquo;t share) has increased substantially on <a href="http://news.ycombinator.com">Hacker News</a> (I believe Steve Jobs effects on humanity extend here too). Meanwhile the <a href="http://yuvi.in/blog/story-average-indian-techie/">The Story of Average Indian &lsquo;Techie&rsquo;</a>, <a href="http://aarvay.in/2011/12/07/whats-your-gpa.html">What&rsquo;s your GPA?</a> and <a href="http://nainomics.blogspot.com/2011/11/welcome-to-indias-higher-education.html">other</a> posts bring to the fore some things I do agree with:</p> <ul> <li>Most computer science curricula are outdated or just poor quality</li> <li>The majority of students are in it for the money</li> <li>The professors are almost always bad</li> </ul> <p>In fact I didn&rsquo;t particularly like most of my CS courses either. There were a few gems like System Software and Computer Networks at DA-IICT, but the rest were totally out of sync with the real world. So if you are a precocious hacker should you drop out of college in India? (Assuming your Indian parents will let you do that!)</p> <h3><strong>NO!</strong></h3> <p>Try your utmost to get into a good college with <em>good infrastructure</em>. Here is why you would want to do so. Not only is the infrastructure itself important, it also attracts the smartest people. Do well in college <em>while</em> improving your own skills and knowledge.</p> <p>My reasons are based on personal experience, and in ways document some of my shortcomings too :P</p> <h3>Like minded people</h3> <p>Unlike the dense technological concentration of Silicon Valley, India doesn&rsquo;t have technological hotspots. Even in Bangalore, very few people are <strong>passionate</strong> about technology and are hacking on open source software or launching startups (while this is pretty high in India terms, it is nothing in Bangalore terms). FOSS talent is instead concentrated in the <a href="http://google-opensource.blogspot.com/2011/10/whos-being-schooled.html">students of engineering colleges</a>. (I focus on FOSS because it&rsquo;s a good way to filter out passionate people.) You will get to discuss problems, hack on code and be inspired by these people. A concentration of geeks also leads to geek events like hackathons. During various college fests there will be programming contests and so on. You&rsquo;ll get to have fun. Finally there will be a lot of smart people doing things other than computer science. But they will be equally as passionate as you are, they will be liberal and forward looking and it will be a pleasure to interact with them. Oh and please don&rsquo;t think of every person you approach as a potential future employer, employee or general networking and increasing contacts kind of person. Sometimes you (and certain people in the Valley too) just need friends. Face it, do you want to spend the next few years talking to your mom about why REST-ful APIs are the bomb or why <a href="http://xkcd.com/221/">this</a> is funny?</p> <h3>Facilities</h3> <p>High-speed Internet access in India is still not too common, but colleges will usually have a great LAN setup, a lease line to the Internet and generally good connectivity. Use these to experiment with your peer-to-peer applications, host websites, or write the next great DDoS program (I&rsquo;m not advocating this). That said they also may have ridiculously bureaucratic system administrators, censorship and the like. You just have to deal with in (and in some cases, circumvent).</p> <p>Your college will also have a certain &lsquo;relic from the past&rsquo; which is far more useful than any of our modern day, 140 characters technology. The library. Specific technology education is always best done via Internet, but general concepts and deep theory is still found in books. Use it well, you will regret the day you leave college and books will have to be purchased. (To overseas readers &ndash; there is hardly a public library system in India.) Oh, and do remember the fiction section.</p> <h3>Motivation and Persistence</h3> <p>When you are working on personal projects it&rsquo;s very easy to give up or change tracks. You also tend to focus only on the things you like. College courses will force you to persist at subjects you don&rsquo;t like, and keep you onto one thing for 3-4 months. Valuable lessons when your first commercial project is 90% done and you don&rsquo;t want to polish it up because <em>node.js</em> just came along and is much more fun to play with. Do a great and challenging final project and end your education on a high note.</p> <h3>Find some other interests</h3> <p>Your life isn&rsquo;t going to be just about CAP theorems, cache invalidation and naming your projects. You <em>will</em> have to interact with society. Take a <a href="http://blog.nikhilism.com/2011/10/have-some-humanity.html">humanities course</a>. Learn to loosen up a bit &mdash; travel, listen to music and play sports. Waste time with friends once in a while. Don&rsquo;t burn out before you&rsquo;ve even started. Your whole life before 25 should not be spent being a workaholic. Sometimes I think the Valley propagates Minimum Viable Product, pitching to VCs and beer and pizza far too much. You don&rsquo;t want to end up like <a href="http://www.wallchan.com/images/sandbox/42353-fat-nerd.png">this</a>. You might also want to try some of <a href="http://apps.ycombinator.com/item?id=1866231">these things</a>.</p> <p>So if you were thinking of dropping out, just give it a second thought. If you are still convinced you should drop out, <strong><a href="http://derekneighbors.com/2009/01/just-fucking-do-it/">do it</a></strong>. But please let me know at <a href="mailto:nsm.nikhil@gmail.com">nsm.nikhil@gmail.com</a> why you did so.</p> <p style="font-size: 10px;"> <a href="http://posterous.com">Posted via email</a> from <a href="http://bombayrain.posterous.com/why-indian-students-should-attend-college">nikhil's posterous</a> </p> </div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com7tag:blogger.com,1999:blog-3665064696228706104.post-6518887884779671512011-11-28T11:46:00.001+05:302011-11-28T11:46:48.512+05:30A High-Five Experiment<div class='posterous_autopost'><p>Final exam time is always a time of stress for students. When you have 6 exams in 6 days, the time just before the exam is one of general apprehension and attempts at mentally revising all the key points. Everybody notices that in the glum looks students have walking into the exam hall.</p> <p>Fortunately, the fourth year in DA-IICT is pretty light, and due to a multitude of projects, me and <a href="http://experimentingwithidiotism.blogspot.com">Naman</a> had only a couple of papers. Inspired by Charlie Todd&rsquo;s <a href="http://www.ted.com/talks/charlie_todd_the_shared_experience_of_absurdity.html">TED video</a> about <a href="http://improveverywhere.com/">Improv Everywhere</a> we decided to inject some cheer into the poor souls of the other batches.</p> <p>Four signs went up on Thursday, Nov 24, along the long, curved staircase leading to the hall.</p> <p><img src="http://farm8.staticflickr.com/7172/6416695261_63ddf92cbd.jpg" alt="Somebody" /> <img src="http://farm7.staticflickr.com/6215/6416697199_9c53d67cf8.jpg" alt="wants to" /> <img src="http://farm7.staticflickr.com/6220/6416698209_50d43373a0.jpg" alt="give you" /> <img src="http://farm7.staticflickr.com/6106/6416699405_0f0afdd1cb.jpg" alt="a high five" /></p> <p>They were nicely spaced out so only one could be seen at a time, and we were standing at the end of it, right at the entry doors.</p> <p>As the students started coming in and the high-fives started flying, accompanied by words of encouragement (and the occasional &ldquo;Heap sort is O(nlgn)&rdquo;), it was very satisfying to see those faces light up in the happiness of an unexpected surprise and lose some of the pressure.</p> <p>So go out and spread some cheer! You can freely use and modify the above images for a noble cause :) There will of course always be people who are unable to take in why someone would do something like this, they&rsquo;ll walk past you without an upward glance. Others will eye you with suspicion even as they raise a half-hearted hand. Grumpy administrators will tell you you aren&rsquo;t supposed to be in the examination area if you don&rsquo;t have an examination. But on that day I saw the most introverted student give a resounding high-five back and walk with a spring in his step. It was worth it.</p> <p style="font-size: 10px;"> <a href="http://posterous.com">Posted via email</a> from <a href="http://bombayrain.posterous.com/a-high-five-experiment">nikhil's posterous</a> </p> </div>Nikhilhttp://www.blogger.com/profile/02751486022348919717noreply@blogger.com4