tag:blogger.com,1999:blog-10339334131278323102018-12-15T04:24:20.619-08:00An opinion on...Software, Climbing, Art, Music, Comedy, Travel, Motorcycles, Bicycles, ChickensJon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.comBlogger354125tag:blogger.com,1999:blog-1033933413127832310.post-70123667829996010872014-03-06T09:18:00.004-08:002014-03-06T09:18:54.727-08:00I miss Steve JobsThis morning, my dad sent me this screenshot from his several months old Macbook Pro... sigh.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-cgXRoeEief4/Uxitr2oh_8I/AAAAAAAABqA/Rp2miKbJVFM/s1600/2014-03-06+08.16.13.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-cgXRoeEief4/Uxitr2oh_8I/AAAAAAAABqA/Rp2miKbJVFM/s1600/2014-03-06+08.16.13.jpg" height="480" width="640" /></a></div><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/0OsPchID6lw" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2014/03/i-miss-steve-jobs.htmltag:blogger.com,1999:blog-1033933413127832310.post-12980219578312425312014-01-17T20:03:00.000-08:002014-01-17T20:03:09.332-08:00How to get to the Marieta islands<div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-oiGNSUsBMkw/Utn4BMGTdFI/AAAAAAAABok/PVsTwg3jNjg/s1600/The+Hidden+Beach+in+the+Marieta+Islands.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-oiGNSUsBMkw/Utn4BMGTdFI/AAAAAAAABok/PVsTwg3jNjg/s1600/The+Hidden+Beach+in+the+Marieta+Islands.jpg" height="400" width="282" /></a></div><br />On our recent trip to <a href="http://en.wikipedia.org/wiki/Sayulita" target="_blank">Sayulita</a>, our friends recommended we check out the <a href="http://en.wikipedia.org/wiki/Marieta_Islands" target="_blank">Marieta islands</a>&nbsp;for the amazing hidden beach that you have to swim into. &nbsp;Unfortunately, we were there during a busy time of year and all of the chartered boats companies in Sayulita were sold out. &nbsp;It is actually a good thing because it turns out they tend to oversell their boats and pack people into them. Also, Sayulita and <a href="http://en.wikipedia.org/wiki/Puerto_Vallarta" target="_blank">Puerto Vallarta</a> are pretty far from the islands. That's a whole day trip right there. &nbsp;Asking Google for directions only just came up with these commercial companies, we thought we weren't going to get to go.<br /><br />So, instead of playing that game, we drove to the small town of <a href="http://en.wikipedia.org/wiki/Punta_Mita" target="_blank">Punta Mita</a>, which is the closest town to the islands. &nbsp;We had a good lunch at one of the local places and then started walking down the main strip. &nbsp;Within 5 minutes someone approached us and offered to take us out on a private tour for a decent rate. &nbsp;I'm sure the price was somewhat inflated for the gringos, but whatever, we had the whole boat to ourselves and it wasn't going to take all day.<br /><br />On the way out to the islands, we saw a bunch of humpback whales... the guy who offered the boat up gave a money back guarantee that we'd see whales. I guess we were paying for sure now.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-YlO9WPKO_50/Utn7rDeI25I/AAAAAAAABow/LMhzlIUGfNM/s1600/1545018_10152596480244816_730537435_n.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-YlO9WPKO_50/Utn7rDeI25I/AAAAAAAABow/LMhzlIUGfNM/s1600/1545018_10152596480244816_730537435_n.jpg" height="300" width="400" /></a></div><br />Once we were at the island, they anchored the boat, we jumped in the water swam through the cave to the beach in and we hung out for a while on the sand and people watched. &nbsp;After we swam back to the boat, we drove around the entire island, saw another cave (no beach) and got sprayed by swells hitting the rocks.<br /><br />Unfortunately, we didn't see any more whales on the way back as the day was pretty much over. &nbsp;That said, it was an excellent adventure and I'm really glad how it all worked out.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/dXCWW7Elk-c" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com1http://lookfirst.com/2014/01/how-to-get-to-marieta-islands.htmltag:blogger.com,1999:blog-1033933413127832310.post-67099441355044827302014-01-14T09:44:00.000-08:002014-01-14T09:46:07.299-08:00After all these years, I still write Java code...<a href="http://www.theserverside.com/news/thread.tss?thread_id=78441">I wish I had another hand so I could give Scala three thumbs down!</a><br /><br />"New languages are indubitably exciting to learn and play with, and everyone is interested in improving upon what we already have. Sometimes it appears that languages like Scala or Clojure or Ceylon are the fix that the Java ecosystem needs to improve productivity and obtain that linear scalability that simply doesn't exist with traditionally written Java applications. <b>But the fact is, Java, despite some misgivings, is a well thought out language that is both powerful and consistent, making it easy to learn, and more importantly, easy to maintain.</b> Sure, new systems will appear that will try to knock the crown off the Java language, but for now, the want-to-be emperors of the JVM are increasingly being shown to be wearing no clothes."<br /><br /><br />Pretty much exactly how I've felt all along.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/KPKLzrSL0bo" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com1http://lookfirst.com/2014/01/after-all-these-years-i-still-write.htmltag:blogger.com,1999:blog-1033933413127832310.post-6700215842126491192014-01-01T15:04:00.002-08:002017-05-11T10:26:27.802-07:00Rental Car InsuranceI recently took a trip to Puerto Vallarta, Mexico with my wife and rented a car for our 8 day stay. I had done the research in the past with regards to insurance and knew that as long as I use my VISA card, <a href="http://usa.visa.com/personal/cards/benefits/bft_dmg_waiver_personal.html" rel="nofollow" target="_blank">I should decline any extra coverage that the rental car agency offers</a>. It turns out that in Mexico, you need at least one basic level of coverage called SLI or SAI insurance. This covers you against damage to someone else. This ran us an extra $115 on our bill. If we had gone for their full inclusive insurance, it would have been an additional $300! Way more than we paid to rent the car.<br /><br />I've also <a href="http://www.sfgate.com/mexico/mexicomix/article/Renting-a-car-in-Mexico-What-you-need-to-know-3787891.php" rel="nofollow" target="_blank">read articles like this one</a> that claim "Declining to buy the insurance (some of which is mandatory, anyway) is foolhardy to the extreme, but buying the full package without knowing what you're buying is only slightly less so." The article must be out of date or I got bad information from the person at the rental counter, but the SLI/SAI insurance was mandatory.<br /><br />Well, to make a long story shorter, our brand new rental car got a nice big scratch on the side of it...<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-hQZb4hfXWOg/UsRgbQfLKhI/AAAAAAAABoM/QuTl8dr_E9k/s1600/IMG_2565.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://1.bp.blogspot.com/-hQZb4hfXWOg/UsRgbQfLKhI/AAAAAAAABoM/QuTl8dr_E9k/s1600/IMG_2565.jpg" width="320" /></a></div><br />In the US, this would cost probably $1000-1500 to fix. This had me worried that we would run into all sorts of trouble at the rental car company, so on our last day, we left a bit early to take care of things. I kept thinking, maybe we should have gotten the all inclusive insurance.<br /><br />When we arrived, they noticed the scratch immediately, of course. They were very nice about it and simply asked for a copy of the original rental agreement through the 3rd party company that we got the car from. They asked me to write up an accident report detailing what happened. This was a simple sentence. Then, they told me the price for the damage... only about $89! I didn't argue it. I've opened a ticket with VISA and I expect they will pay it after some period of time.<br /><br />This got me thinking... the rental car place must have their own insurance which covers mishaps like this. Why in the world would anyone go with the all inclusive insurance for $300+ when simple damage can be paid for relatively cheaply by the rental car company itself. Sure, there is probably a risk of total loss of the car, but that is super rare. Even still, VISA would cover it under their own insurance.<br /><br />So, unless you don't have VISA coverage, don't fret not getting the extra insurance. I'm sure others have worse stories, but this one turned out pretty well for me.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/v4a2CFjxRKg" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com1http://lookfirst.com/2014/01/rental-car-insurance.htmltag:blogger.com,1999:blog-1033933413127832310.post-11817040898926145122012-10-24T14:52:00.002-07:002012-10-25T12:50:39.604-07:00How to determine DKIM key length?I read <a href="http://www.wired.com/threatlevel/2012/10/dkim-vulnerability-widespread/" rel="nofollow" target="_blank">this article in Wired</a> about a researcher who figured out that Google was using a weak (512-bit) key for its implementation of DKIM. It turns out <a href="http://blog.jgc.org/2010/06/facebooks-dkim-rsa-key-should-be.html" rel="nofollow" target="_blank">this is old news</a> as someone did the same thing to Facebook.<br /><br />This got me wondering if the keys for our emailing service are long enough. But, how to easily determine the length of the key? Turns out it is kind of convoluted so I decided to repeat the info here for my own benefit when I forget about this later.<br /><br />Take your public DKIM key (probably from your DNS TXT record). It looks like this one from Google:<br /><br /><pre>20120113._domainkey.google.com. 86400 IN TXT "k=rsa\; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp5kQ31/aZDreQqR9/ikNe00ywRvZBFHod6dja+Xdui4C1y8SVrkUMQQLOO49UA+ROm4evxAru5nGPbSl7WJzyGLl0z8Lt+qjGSa3+qxf4ZhDQ2chLS+2g0Nnzi6coUpF8r" "juvuWHWXnzpvLxE5TQdfgp8yziNWUqCXG/LBbgeGqCIpaQjlaA6GtPbJbh0jl1NcQLqrOmc2Kj2urNJAW+UPehVGzHal3bCtnNz55sajugRps1rO8lYdPamQjLEJhwaEg6/E50m58BVVdK3KHvQzrQBwfvm99mHLALJqkFHnhyKARLQf8tQMy8wVtIwY2vOUwwJxt3e0KcIX6NtnjSSwIDAQAB"</pre><br />Save the p= part of the TXT record into a file&nbsp;(google.key)&nbsp;that is line wrapped to around 78 columns <i>(yes, it needs to be line wrapped or the openssl command used below breaks)</i>. Google seems to store their key in two parts so I removed the " " that is embedded in the middle of the blob above. You will also need the BEGIN/END PK block.<br /><br /><pre>-----BEGIN PUBLIC KEY-----<br />MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp5kQ31/aZDreQqR9/<br />ikNe00ywRvZBFHod6dja+Xdui4C1y8SVrkUMQQLOO49UA+ROm4evxAru5nGPbSl7WJzyGLl0z8Lt+<br />qjGSa3+qxf4ZhDQ2chLS+2g0Nnzi6coUpF8r<br />juvuWHWXnzpvLxE5TQdfgp8yziNWUqCXG/<br />LBbgeGqCIpaQjlaA6GtPbJbh0jl1NcQLqrOmc2Kj2urNJAW+<br />UPehVGzHal3bCtnNz55sajugRps1rO8lYdPamQjLEJhwaEg6/<br />E50m58BVVdK3KHvQzrQBwfvm99mHLALJqkFHnhyKARLQf8tQMy8wVtIwY2vOUwwJxt3e0KcIX6Ntn<br />jSSwIDAQAB<br />-----END PUBLIC KEY-----<br /></pre><br />Then run <code>openssl rsa -noout -text -pubin &lt; google.key</code><br /><br />Which then outputs this chunk with your answer: <br /><br /><pre>Modulus (2048 bit):<br /> 00:a7:99:10:df:5f:da:64:3a:de:42:a4:7d:fe:29:<br /> 0d:7b:4d:32:c1:1b:d9:04:51:e8:77:a7:63:6b:e5:<br /> dd:ba:2e:02:d7:2f:12:56:b9:14:31:04:0b:38:ee:<br /> 3d:50:0f:91:3a:6e:1e:bf:10:2b:bb:99:c6:3d:b4:<br /> a5:ed:62:73:c8:62:e5:d3:3f:0b:b7:ea:a3:19:26:<br /> b7:fa:ac:5f:e1:98:43:43:67:21:2d:2f:b6:83:43:<br /> 67:ce:2e:9c:a1:4a:45:f2:b8:ee:be:e5:87:59:79:<br /> f3:a6:f2:f1:13:94:d0:75:f8:29:f3:2c:e2:35:65:<br /> 2a:09:71:bf:2c:16:e0:78:6a:82:22:96:90:8e:56:<br /> 80:e8:6b:4f:6c:96:e1:d2:39:75:35:c4:0b:aa:b3:<br /> a6:73:62:a3:da:ea:cd:24:05:be:50:f7:a1:54:6c:<br /> c7:6a:5d:db:0a:d9:cd:cf:9e:6c:6a:3b:a0:46:9b:<br /> 35:ac:ef:25:61:d3:da:99:08:cb:10:98:70:68:48:<br /> 3a:fc:4e:74:9b:9f:01:55:57:4a:dc:a1:ef:43:3a:<br /> d0:07:07:ef:9b:df:66:1c:b0:0b:26:a9:05:1e:78:<br /> 72:28:04:4b:41:ff:2d:40:cc:bc:c1:5b:48:c1:8d:<br /> af:39:4c:30:27:1b:77:7b:42:9c:21:7e:8d:b6:78:<br /> d2:4b<br />Exponent: 65537 (0x10001)<br /></pre><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/kTUL3_NLq9Q" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com4http://lookfirst.com/2012/10/how-to-determine-dkim-key-length.htmltag:blogger.com,1999:blog-1033933413127832310.post-9233240186304949142012-05-09T19:03:00.000-07:002012-09-03T11:21:22.216-07:00Scrape webpages with node.jsI recently had the task of scraping data from a website so I choose to use <a href="http://nodejs.org/">node.js</a> in order to get a bit more experience with it. After checking out a few different options for scraping, I finally settled on the&nbsp;<a href="https://github.com/chriso/node.io">node.io</a> project which provided the most robust handling and configuration features that I could find.<br /><br />The hard part about scraping data from websites is coming up with ways to quickly and reliably pick out pieces from the document object model (DOM). These days, I spend a lot of time using the <a href="http://api.jquery.com/category/selectors/">jQuery selector syntax</a>&nbsp;to develop <a href="https://www.voo.st/">my site</a>&nbsp;which means that ideally I'd find a solution that can download a webpage and then provide me with jQuery-like functions and selectors to pick out pieces from the DOM. For this purpose, node.io uses a project called <a href="https://github.com/harryf/node-soupselect">node-soupselect</a> by default, but I found the selector syntax to be lacking. Thus, I layered another project called&nbsp;<a href="https://github.com/MatthewMueller/cheerio">cheerio</a> on top. Whatever you do, don't use <a href="https://github.com/tmpvar/jsdom">jsdom</a>&nbsp;as it is too slow and very strict in its processing of html.<br /><br />Node.io stands out from the rest of the projects because it&nbsp;applies a 'jobs' approach to scraping. This is something that I used in&nbsp;<a href="https://github.com/lookfirst/jmxtrans">another project of mine</a> and it worked out really well. In other words, you write a 'job' which gets executed and if there is a failure during the run of the job, you have control over what you do next (skip, fail, retry).<br /><br />In order to develop and debug your code, you shouldn't continually hit the website that you are scraping from. What I do is download the page I want to scrape and then run a local webserver to serve up the file. I found the python webserver the easiest to use as it is just one simple command, <span style="font-family: 'Courier New', Courier, monospace;">python -m SimpleHTTPServer 8000</span> which will serve up files from whichever directory you run that command from.<br /><br />For debugging code, I recommend setting up the excellent <a href="https://github.com/dannycoates/node-inspector">node-inspector</a>&nbsp;which will allow you to setup breakpoints in your code and step through to inspect objects as necessary, just like you do with web page JavaScript development. This becomes invaluable with JavaScript because the lack of types makes it hard to know what properties objects have. For logging output to the console so that I could keep track of the execution, I ended up with the <a href="https://github.com/igo/nlogger">nlogger</a> project which I wasn't super happy with, but worked well enough for this project.<br /><br />Writing your first job is easy and if you are a <a href="http://coffeescript.org/">CoffeeScript</a> (CS) fan, node.io will automatically compile your CS files for you. &nbsp;If you aren't a CS fan, I apologize as my example is in CS. This simple job @get's a page and selects the &lt;title&gt; element:<br /><br /><pre class="prettyprint">nodeio = require 'node.io'<br />cheerio = require 'cheerio'<br />log = require('nlogger').logger(module)<br /><br />count = 0<br />class InputScraper extends nodeio.JobClass<br /> input: [476,1184]<br /> run: (inputId) -&gt;<br /> @get "http://localhost:8000/#{inputId}.html", (err, data, headers) =&gt;<br /> @exit(err) if err?<br /> try<br /> log.info('Started: {}', inputId)<br /> $ = cheerio.load(data)<br /> @emit($('title').text())<br /> log.info("(#{count}) Finished inputId: #{inputId}")<br /> count++<br /> catch error<br /> log.error('Error: {} : {}', inputId, error.stack)<br /> @skip()<br /> output: (data) -&gt;<br /> console.log(data)<br />@class = InputScraper<br />@job = new InputScraper({spoof:true, max: 1})<br /><br />nodeio.start(@job)</pre><br />The 'run' function is called for each piece of input data in the array (476, 1184). @get() grabs the html page data. On success, @get() executes the callback function which loads the data into cheerio and @emit()'s the title. When the 'run' function is complete, node.io calls output which logs the @emit() data to the console.<br /><br />In my code, the line above the @emit(), I have another class function which I pass the $ cheerio object into which handles all of my parsing and the result is an object that I pass into @emit(). This allows me to re-use the InputScraper boilerplate to parse all sorts of different pages.<br /><br />I also run things directly with node 'nodeio.start(@job)' so that I can use the node-inspector more easily. This means that I also end up actually compiling the CS myself using my answer on <a href="http://stackoverflow.com/a/7507987/253773">StackOverflow</a>.<br /><br />Obviously, this is a pretty simple example, but it should get you up and running with the node.io framework quickly. The speed of this is fairly impressive, on my desktop and home network, I was able to crawl and extract data from about 2000 webpages using max: 20 in about 1.5 minutes. Most of the time is spent downloading the pages and the parsing only takes a few milliseconds.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/or9S5NPZoCM" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com1http://lookfirst.com/2012/05/scrape-webpages-with-nodejs.htmltag:blogger.com,1999:blog-1033933413127832310.post-14347600794880807322012-03-29T11:51:00.000-07:002012-09-03T11:32:21.143-07:00The first comprehensive review of 22 online athletic event registration services. The Good, the Bad, and the Ugly.<div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;">Before we decided to build Voost, we spent a lot of time studying the myriad registration services already available online. There are a staggeringly large number of choices with wildly differing levels of sophistication. Comparing them is incredibly difficult, in no small part due to the fact that many of these websites seem to go out of their way to hide critical information like fees and disbursement schedules.</span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;"><br /></span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;">Over a hundred hours of tedious research, creating accounts, combing through documentation and FAQs, setting up test events, going through registration processes, calculating fees, etc. While we certainly have biases, we have tried to present the information as objectively as possible - this isn't a cheap marketing gimmick designed to show green checkboxes for us and red Xes for our competitors. We freely admit that there are features other services have that Voost does not (yet!) - and this information is reflected on the table.</span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; text-align: left;"><br /></span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;">The table is not complete. We have left fields blank where we just couldn't figure out the answers (and believe me, we tried). Despite our best efforts, there may also be errors - again, some websites seem to deliberately befuddle attempts at objective comparison.</span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;"><br /></span></div><div class="separator" style="clear: both; text-align: -webkit-auto;"><span style="font-family: inherit; line-height: 22px;"><a href="https://www.voo.st/compare" target="_blank">Click the image to view the whole table.</a></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://www.voo.st/compare" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;" target="_blank"><span style="color: black; font-family: inherit;"><img border="0" height="300" src="http://2.bp.blogspot.com/-1sHM12AdrTY/T3Srk25zcLI/AAAAAAAAAtY/6QQTZu0EMBk/s400/compare.jpg" width="400" /></span></a></div><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/3Qyzyx3r5Hg" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2012/03/first-comprehensive-review-of-22-online.htmltag:blogger.com,1999:blog-1033933413127832310.post-43949389621323765542012-01-25T01:03:00.000-08:002012-01-25T01:03:49.171-08:00GitHire spam again...<div class="tr_bq">Just keeping this around for posterity since they deleted the HN posting and maybe others will want to see the truth if they are googling around for why you are getting spam from these guys.</div><br /><a href="http://news.ycombinator.com/item?id=3508655">http://news.ycombinator.com/item?id=3508655</a><br /><br />I called them out for being a the spammers that they are, and I got a rather odd response:<br /><blockquote>Hi LatchKey,<br />I'm really sorry that we sent you that email. We just launched a little over a week ago with this crazy idea, and were extremely surprised at how quickly we were overwhelmed with orders.<br /><b>We made a bad judgement call</b> in sending some emails to people asking if anyone is interested in jobs.<br />If it makes you feel any better, you can see that we aren't finding very many talented engineers, and we will likely need to refund a lot of money in a few days.<br />We are honestly trying to be a great service for software developers and employers. We need feedback from people like yourself to learn how we can be the best service possible to reshape the hiring industry.<br />We actually sent you an email, but never heard back. Please let us know if you're interested in continuing this discussion further on or off of a public message board.<br />Thanks for keeping us honest.</blockquote>The HN thread goes on with a lot of people pointing out their own issues with GitHire and I have my response to the above quote here:&nbsp;<a href="http://news.ycombinator.com/item?id=3508750">http://news.ycombinator.com/item?id=3508750</a><br /><br />Sorry, but I just don't have any tolerance for spammers or people trying to profit off my profile without my permission.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/eutMn7hPL0A" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2012/01/githire-spam-again.htmltag:blogger.com,1999:blog-1033933413127832310.post-77837124720397953492012-01-22T13:17:00.000-08:002012-01-22T13:18:05.273-08:00Scalable System Architecture ComedyI was reading <a href="http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html" target="_blank">Scaling a PHP MySQL Web Application</a>, which is a technical document published on the Oracle website. As I was scrolling down the page, I saw the typical Load balancing <i>Figure 1</i> that you always see in any PHP/MySql web application.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-xYdl3k6hk2g/Txx4aEk2LzI/AAAAAAAAAsI/A_IUq2irAyA/s1600/355130.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="http://1.bp.blogspot.com/-xYdl3k6hk2g/Txx4aEk2LzI/AAAAAAAAAsI/A_IUq2irAyA/s320/355130.png" width="320" /></a></div><br />But then, as the article goes on, it gets more entertaining. It goes on to <i>Figure 3</i>, showing Multiple MySQL Slaves, which is now 4 machines.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-3Qw_t6IMem8/Txx4zFDbe_I/AAAAAAAAAsQ/CeJpp5n68tc/s1600/355132.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="301" src="http://3.bp.blogspot.com/-3Qw_t6IMem8/Txx4zFDbe_I/AAAAAAAAAsQ/CeJpp5n68tc/s320/355132.png" width="320" /></a></div>But wait, there's more. Now you need a dedicated database slave for each Web server, so the picture expands to even more lines and arrows in <i>Figure 4</i>. A total of 8 machines.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/--zf8v6ZelQk/Txx5FBYM-SI/AAAAAAAAAsY/APdQxMRsmqU/s1600/355133.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="208" src="http://3.bp.blogspot.com/--zf8v6ZelQk/Txx5FBYM-SI/AAAAAAAAAsY/APdQxMRsmqU/s320/355133.png" width="320" /></a></div><br />As you keep scrolling, you get to <i>Figure 5</i>. A real gem of an image. Arrows in every direction. Arrows jumping through other arrows. 8 machines, but a completely incomprehensible image.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-f9bs_Fss-2E/Txx5ZTRniUI/AAAAAAAAAsg/RBNoA-Xl5Fk/s1600/355134.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="http://3.bp.blogspot.com/-f9bs_Fss-2E/Txx5ZTRniUI/AAAAAAAAAsg/RBNoA-Xl5Fk/s320/355134.png" width="320" /></a></div><br />Ok, now we've randomized the connections between all the web servers and database slaves.<br /><br />Could you imagine one of these machines going down or throwing errors and trying to figure out which one it is or how it connects to the other machines?<br /><br />We all know that as systems grow, they get more complex. That said, if you draw an incomprehensible picture of your architecture, it is a clear sign that you are doing it wrong.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/rhDIG2lFI4s" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com2http://lookfirst.com/2012/01/scalable-system-architecture-comedy.htmltag:blogger.com,1999:blog-1033933413127832310.post-63269568214375348012012-01-16T23:12:00.000-08:002012-01-16T23:12:59.554-08:00GitHire.com is a spammerI've kept up with all the recent HackerNew articles on <a href="http://githire.com/">GitHire</a>. They seemed like a rather interesting service because I believe that hosting your projects on a site like GitHub, is the best kind of resume a software engineer can have.<br /><br />That is, until they just spammed me with a whole list of completely random and unrelated jobs. I could understand it if I signed up to their site and requested spam (aka: LinkedIn), but I'm definitely not interested as I'm in the middle of starting my own company!<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://img.skitch.com/20120117-e6dgdrbg242wkw9thpc4a6ykr6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://img.skitch.com/20120117-e6dgdrbg242wkw9thpc4a6ykr6.jpg" width="318" /></a></div><br />After checking out their site, I realized they've also got a <a href="http://githire.com/profiles/lookfirst">profile up for me</a>&nbsp;that I had no hand in creating, nor desire having. Apologies if that link stops working, hopefully they will remove my information soon. Maybe I should be pissed that I'm only in their Top 50%? ;-)<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://img.skitch.com/20120117-q9dc6ey5p1g2bp869sapqjhdrb.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="267" src="https://img.skitch.com/20120117-q9dc6ey5p1g2bp869sapqjhdrb.jpg" width="320" /></a></div><br />Since I wanted to remove myself, I clicked the "Opt out of Githire" link, which then takes me to a page on Github to authorize their application?!?<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://img.skitch.com/20120117-bhpx8pk8i31nq5h3wh3m4qj2iw.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="236" src="https://img.skitch.com/20120117-bhpx8pk8i31nq5h3wh3m4qj2iw.jpg" width="320" /></a></div><br />Hell no, I'm not going to authorize your application, just so I can opt out of your website. That is wrong on so many levels.<br /><br />Anyway, I cc'd support@github on my response to 'Steve', so hopefully they will be going away soon. I can't see how GitHub is allowing this company to exist, when they so clearly violate their terms of service policy.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/5MwAWUhCSkk" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com4http://lookfirst.com/2012/01/githirecom-is-spammer.htmltag:blogger.com,1999:blog-1033933413127832310.post-24708362768264810892012-01-06T16:17:00.000-08:002012-01-23T13:00:59.299-08:00Voost Logo<div class="separator" style="clear: both; text-align: center;"><a href="http://voo.st/" imageanchor="1" style="margin-left: 1em; margin-right: 1em;" target="_blank"><img border="0" src="http://1.bp.blogspot.com/-LEczkdSiYNs/TweO2wE_59I/AAAAAAAAAr8/nxAAg9ivoW4/s1600/voost-logo-medium.png" /></a></div><br /><div><br /></div><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/flZDLl5IzQc" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com1http://lookfirst.com/2012/01/voost-logo.htmltag:blogger.com,1999:blog-1033933413127832310.post-14013654484453501502012-01-05T02:57:00.000-08:002012-01-05T03:13:51.813-08:00Going on 6 months now...Here is a bit of a status update for the new year:<br /><br />We've been working full steam ahead on <a href="http://www.voo.st/">Voost</a>,&nbsp;going on 6 months now. In that timeframe, Jeff and I have accomplished an impressive amount of work for just two people. I've been putting in 10-15 hour days, seven days a week, of solid coding and Jeff has been doing the same. When he or I are out of town, we sit on Skype all day (and night) long working through any issues we have and bouncing ideas off each other. It has been a hugely productive cooperative development effort.<br /><br />I've become a much better UX/UI designer than when I started. It had been years since I had worked on this side of things and it has been a lot of fun picking it back up. I've also become an absolute expert in CoffeeScript, JQuery, Less and all of the other hot front end technologies that are out there. On the back end, we've integrated with <a href="https://browserid.org/">BrowserID</a> for secure sign in as an option to Facebook Connect. We've also switched to <a href="http://objectify-appengine.googlecode.com/">Objectify 4</a> which is the most advanced way to interact with the Google AppEngine database backend.<br /><br />The sad face news is we have nothing public to show for all of this hard work quite yet.&nbsp;I could go on with a list of reasons, but they aren't really worth going over in detail. Suffice it to say, we just aren't ready to launch. I'd say that we are about 85-90% of the way there. Hopefully not more than about a month or so. For a few of my friends, what we have is enough and they are pressuring me to just put something out there, even if it is incomplete or buggy.&nbsp;I'm pushing back on them.<br /><br />While I realize the cycling season is quickly picking up, I'm not in a huge hurry. Thankfully, after years of penny pinching, I have enough savings left to last me until we do launch. I really want to do this right. I want all of my cards on the table. I want people to wonder why nobody has done a site this good before. As cheesy as it sounds, I expect something close to perfection, even if it isn't absolutely feature complete. I think of how the original iPhone disrupted the cell phone market. We went from the clam shells and keyboards to touch screens overnight. It may sound silly, but I'm passionate about doing something like that with the event registration market.<br /><br />Even without all of the features that other companies have, our application is light years more advanced than any other registration product out there. I know this because I've seen their systems, analyzed everything wrong with them and spent the time to come up with a vastly better designed user experience. This takes a lot of hard work and this will be a huge differentiator in the marketplace for us. I'm very proud of that fact. It will be very expensive and nearly impossible for our competitors to hire enough engineering talent to catch up with us.<br /><br />A question I get a lot is: do you have any customers? Well, we don't. Not yet. I'm ok with that because I do have enough contacts and relationships to get the word out there to promoters. I think that people also really want this product, so when we launch, it will almost sell itself.&nbsp;I can't tell you how many times I've heard 'I hate XYZ's excessive fees!' and 'This XYZ registration site is so difficult to use!'<br /><br />Besides, the cycling community, our initial target audience, is very small and I don't want to really start pressuring promoters to try out a system that isn't launched yet. I sure wouldn't trust anyone who doesn't have a live product. On the flip side, if I was a competitor, I'd be really scared of us right now. We are going to be <i>very</i> hungry for customers and it will be that much harder to compete when we have a better product <i>with</i> better pricing.<br /><br />A bit of good news is that we are close to having a great company logo. We put a bounty up on one of those crowd sourced websites full of designers and got a number of excellent designs, out of over a hundred submissions. We are in the process of choosing the final one over the next couple of days. I look forward to announcing it.<br /><br />Thanks for listening. Thanks to all my friends and family for the encouragement and advice. Thanks to my wife for putting up with me working all the time. Thanks to everyone who has offered to help. Expect another update soon. This is going to be a lot of fun!<br /><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/nyjpw_9YupU" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com2http://lookfirst.com/2012/01/going-on-6-months-now.htmltag:blogger.com,1999:blog-1033933413127832310.post-64871677640290348562011-12-18T13:19:00.000-08:002014-06-27T11:14:07.224-07:00Don't use the jQuery .data() method. Use .attr() instead.I just discovered the hard way that the jQuery .data() method is horribly broken. By design, it attempts to convert whatever you put into it into a native type.<br /><br />I've got a template where I'm generating a button with a data-key element:<br /><br /><code>&lt;button id="fooButton" data-key="1.4000"&gt;Click me to edit&lt;/button&gt;</code><br /><br /><a href="http://jsfiddle.net/KwjvA/">http://jsfiddle.net/KwjvA/</a><br /><br />It looks like a float where one could assume that 1.4000 === 1.4, but what I really want here is the string 1.4000. I certainly don't want my input to be modified. One suggestion I found in the issue tracker is to put single quotes around the field (ie: data-key="'1.4000'"). That seems rather absurd as well.<br /><br />The only reason why I'm warning about this here is that I've seen a bunch of libraries using .data() to store stuff in elements in the DOM.&nbsp;I think it is a really bad idea to have a method called .data() where you expect to be able to store something in it and be able to get back out exactly what you put into it.<br /><br />The recommended alternative is to use .attr(). The problem with this is that while it achieves the same effect, it is actually much different functionality from .data(). .data() stores information within an internal cache within jQuery, while .attr() calls element.setAttribute().<br /><br />I read through several bug reports on the jQuery website, where people are also confused by this behavior and all of them get closed with a wontfix. I see this as a terrible choice. Yuck.<br /><br />Update: Here is a bug I just filed, hopefully that explains things better to the people who seem to be having a hard time understanding what I'm talking about:&nbsp;<a href="http://bugs.jquery.com/ticket/11060">http://bugs.jquery.com/ticket/11060</a><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/ha7vYGduUIQ" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com66http://lookfirst.com/2011/12/dont-use-jquery-data-method-use-attr.htmltag:blogger.com,1999:blog-1033933413127832310.post-48162248926566797362011-12-08T16:13:00.001-08:002012-09-03T11:34:45.504-07:00Optimizing Web Application JavaScript DeliveryFor <a href="http://voo.st/">my new company</a>, I had the following design goals for my heavy use of JavaScript (JS):<br /><ul><li>I'm using <a href="http://jashkenas.github.com/coffee-script/">CoffeeScript</a>&nbsp;(CS), so I need to have my <a href="http://stackoverflow.com/questions/6645640/integrating-coffeescript-with-eclipse/7507987#7507987">IDE automatically compile</a>&nbsp;CS to JS when I save. That way the whole Change file, Save, Reload the browser process works cleanly.</li><li>Be able to split the files up depending on which page is loaded so that only the JS which is needed for the page is sent to the client.</li><li>Be able to differentiate JS files to be loaded between different states such as logged in, logged out and both. That way, once someone is already logged in, the JS which controls the login and forgot password dialog does not get served again. The flip side is that JS for logged in pages is not served up to anonymous users.</li><li>In development mode, have everything un-minimized, but in production mode, automatically minimize everything.</li><li>Run all of the JS through the <a href="http://code.google.com/closure/compiler/">Closure compiler</a> regardless of dev/prod so that I know that things that work in dev also work in prod.</li><li>Limit the number of <code>&lt;script&gt;</code> tags to the bare minimum. Ideally, 2-3 for .js files served from my site and not directly off of a CDN. Fewer loads means less network traffic.</li><li>Be able to transparently support new code / application versions so that when I upgrade the application, the browsers dump their cached copies of my files.</li><li>No dependencies on external xml, json, property or other configuration file formats to implement the goals above. Everything should be configured by someone editing the html templates.</li></ul>In order to accomplish the goals above, I first looked at a bunch of solutions, but they all failed in various ways. So, I started on my own and went through various iterations before I came up with the ideal solution which I think is pretty unique and easy.<br /><div><br /></div><div>Let's start off by talking about one of the tools I'm using. <a href="http://labjs.com/">LabJS</a> enables me to load JS only when I need it. As part of the 'master' template which contains the skin for all pages, at the very bottom before the <code>&lt;/body&gt;</code> element I have something that looks like this:<br /><br /></div><pre class="prettyprint">&lt;script src="/js/LAB.min.js"&gt;&lt;/script&gt;<br />&lt;script&gt;<br /> var country = "${country}";<br /> var fbAppId = "${fb.APP_ID}";<br /><br /> var js = '${tool.jsbuilder(<br /> me != null,<br /> 'json2:both',<br /> 'handlebars.1.0.0.beta.master:both',<br /> 'bootstrap-twipsy:both',<br /> 'bootstrap-popover:both',<br /> 'gen/global/page:both',<br /> 'gen/global/common:both',<br /> 'gen/global/search:both',<br /> 'gen/modal/loginDialog:out', // ! logged in<br /> 'gen/global/loggedInMenu:in', // logged in<br /> 'gen/global/master:both'<br /> )}';<br /><br /> var lab = $LAB<br /> .script("//ajax.googleapis.com/ajax/libs/jquery/1.7/jquery.min.js")<br /> .wait()<br /> .script("//ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/jquery-ui.js")<br /> .script("//connect.facebook.net/en_US/all.js")<br /> .script("//apis.google.com/js/plusone.js")<br /> .script("//platform.twitter.com/widgets.js")<br /> .wait()<br /> .script(js)<br /> ;<br /><br /> // Variable "pagecode" should be a function that takes a LAB and does any page-specific loading<br /> if (typeof pagecode === 'function')<br /> pagecode(lab);<br />&lt;/script&gt;<br /></pre><div><br />Since I'm using <a href="http://code.google.com/p/cambridge/">Cambridge Templates</a> with <a href="http://commons.apache.org/jexl/">JEXL</a> to process things first, the ${tool.jsbuilder(...)} section runs some Java code which does a lot of the magic during the rendering portion of the page. The first argument is a boolean to indicate whether or not I'm logged in. 'me' is an object in the context and if it is null I'm not logged in. The rest of the arguments are String[]. The method signature looks like this:<br /><br /><div class="p1"><pre class="prettyprint"> public String jsbuilder(<span class="s1">boolean</span> loggedIn, String[] files);</pre></div><div class="p1"><br /></div><div class="p1">What happens in that method is that it will parse the array of Strings, and based on a setting of 'in' for logged in, 'out' for logged out, or 'both' for either logged in or out, it will compare that to the loggedIn boolean and either load the appropriate JS file or not. The files are then loaded into memory, in order, and sent through the Closure compiler to minify the code.</div><div class="p1"><br /></div><div class="p1">The output from Closure is then cached in a global HashMap which is never cleared out. (Note: for languages that don't really persist memory between requests, like PHP, you can store this data in something like memcached).<br /><br />The key of the Map is generated from a md5 hash of the list of filenames combined together + application version. The hash looks something like this: <code>be712950814b2ccc6b92ff5c3</code>. This hash is the <code>String</code> that is returned from the <code>jsbuilder</code> method. By using the names + application version, that ensures a new hash will be generated each time the application is upgraded.<br /><br />In dev mode, the Map isn't used at all. The code is generated for each request, which ensures that my changes get immediately reflected in the browser. In production, the Map is first checked for the key and if it exists, the key is immediately returned from the <code>jsbuilder</code> method.</div><div class="p1"><br /></div><div class="p1">The final rendered page looks something like this to the web browser:</div><div class="p1"><br /></div><div class="p1"></div><pre class="prettyprint"> var js = '/js2/be712950814b2ccc6b92ff5c3.js';<br /></pre><div><br /></div><div>When the LabJS code executes in the browser and loads my script with the line <code>.script(js)</code>, there is a Servlet listening for requests to <code>/js2/*.js</code> and it looks up the key from the url in the Map and returns the appropriate JS data. This servlet can also set the correct browser cache headers depending on dev or prod.</div><div><br /></div><div>As you can see, 10+ separate files have been combined and minified into a single file which makes the requests more efficient. All without configuration files or a crazy syntax that only a backend developer can understand.<br /><br />If I wanted to split the JS files into more loads so that the browser can take advantage of concurrent loading, I could do that as well by just creating more calls to <code>jsbuilder</code>. That is effectively what is happening in the <code>pagecode</code> section near the end of the <code>&lt;/script&gt;</code> element above. The body template which is loaded into the master template by Cambridge, optionally has a JS function defined called <code>pagecode</code>. When it executes, it calls <code>lab.script()</code> again with similar output from the <code>jsbuilder</code> tool. This allows me to split up my code so that there is global code as well as page specific code.</div><br />Enjoy.</div><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/y6cZtjXAPYs" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/12/optimizing-your-javascript-delivery.htmltag:blogger.com,1999:blog-1033933413127832310.post-76871998332130450862011-12-07T10:41:00.001-08:002011-12-07T11:07:08.872-08:00Github PagesI'm a huge fan of Github. Pull requests are the best innovation since the idea behind open source was created.<br /><br />For one of my projects hosted on Google Code, I was recently asked to move it to Github so that someone could submit pull requests more easily. I happily complied because of how much I love Github.<br /><br />As part of this move, I decided to finally explore Github Pages in order to publish the nicely formatted documentation of my project. I was thinking they'd be as great as pull requests, and after an hour of reading the documentation and installing everything, I was terribly disappointed.<br /><br />The main issue is that it uses a site generation tool called <a href="https://github.com/mojombo/jekyll">Jekyll</a>. While this tool is generally ok, it has a quite a few major failings as a product for Github.<br /><ul><li>It is clearly a product of Not-Invented-Here syndrome. How many static website generators does this world need? Why did you feel the need to invent yet another one? Hell, I don't even want a static website generator, I just want to write some documentation.</li><li>I don't want 50 different options for generating content. Just give me Markdown. I don't even want 2 different types of Markdown. Just give me the one that works best, as default.</li><li>By default, it comes with nothing to help you design a site full of great looking documentation. Even just a default template setup would suffice. Some people have tried to create some helper projects, but they all have terrible UI and they all seem somewhat&nbsp;abandoned.</li><li>It requires me to become an expert in this tool. I have to install a bunch of random software, learn configuration files, learn a specific file layout. All I want to do is write some documentation that looks nice!</li><li>It was basically created for publishing blogs. Why is this being promoted as GH Pages? It seems rather absurd for a source code repository to provide a tool for publishing blogs, but not a tool for publishing great documentation.</li></ul><div>Back over on Google Code, I just create some wiki pages, link them together with a table of contents (also a wiki page) and point at a specific url. Everything just works and looks great.</div><div><br /></div><div>Github, please fix this!</div><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/_bIVNG7hSMc" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com4http://lookfirst.com/2011/12/github-pages.htmltag:blogger.com,1999:blog-1033933413127832310.post-78747505103353182362011-12-01T21:28:00.001-08:002011-12-01T21:59:56.747-08:00Social buttonsToday, I finally got around to implementing those social Like buttons that you see on all of the websites. Personally, I never click them, but it is clear lots of other people do so I'm going with the bandwagon.<br /><br />They look something like this:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-o140kEK2c5w/TthiM2vdCNI/AAAAAAAAArs/P6lv7ZNMc2E/s1600/Screen+Shot+2011-12-01+at+9.29.04+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-o140kEK2c5w/TthiM2vdCNI/AAAAAAAAArs/P6lv7ZNMc2E/s1600/Screen+Shot+2011-12-01+at+9.29.04+PM.png" /></a></div><br />The top ones are Facebook, then Twitter, then Google+.<br /><br />Styling the first two rows of buttons with CSS is simple. They have <i>class="fb-like"</i> and <i>class="twitter-share-button"</i>. I can move them around on my page and place them exactly where I want them quite easily.<br /><br />What does Google have you ask? Nothing. Zip. Zilch. Instead, it has&nbsp;<i>id="___plusone_0"</i> which means that it is somewhat useless as a general CSS selector.<br /><br />I know this is nit picky on such a small issue, but an oversight like this seems hard to fathom. I've google'd around a bit and it really makes me wonder, how come no other site designers have brought this up?<br /><br />I was hoping that a work around for this would be to just write it as a div instead of as the <i>&lt;g:plusone&gt;</i> element. But it turns out that div loses the class attribute when it is re-written by their JavaScript.<br /><br /><pre class="prettyprint" style="background-color: white; border-bottom-color: rgb(187, 187, 187); border-bottom-style: solid; border-bottom-width: 1px; border-color: initial; border-image: initial; border-left-color: rgb(187, 187, 187); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(187, 187, 187); border-right-style: solid; border-right-width: 1px; border-style: initial; border-top-color: rgb(187, 187, 187); border-top-style: solid; border-top-width: 1px; color: #333333; font-family: inherit; font-size: 13px; font: normal normal normal 1em/normal 'Droid Sans Mono', monospace; line-height: 1.5; margin-bottom: 1.5em; margin-top: 1.5em; overflow-x: auto; overflow-y: auto; padding-bottom: 6px; padding-left: 10px; padding-right: 10px; padding-top: 6px; vertical-align: baseline;"><span class="tag" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #000088; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">&lt;div</span><span class="pln" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: black; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;"> </span><span class="atn" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #660066; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">class</span><span class="pun" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #666600; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">=</span><span class="atv" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #008800; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">"g-plusone"</span><span class="pln" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: black; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;"> </span><span class="atn" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #660066; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">data-size</span><span class="pun" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #666600; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">=</span><span class="atv" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #008800; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">"tall"</span><span class="pln" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: black; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;"> ... </span><span class="tag" style="border-bottom-width: 0px; border-color: initial; border-image: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; color: #000088; font-family: inherit; font-size: 13px; font-style: inherit; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; vertical-align: baseline;">&gt;&lt;/div&gt;</span></pre><br />In the end, I just put my own div around the element, but that seems like such a kludge when the other services seem to do this correctly.<br /><br />Maybe I'm reading too far into this because it is such a little thing, but it really makes me feel like Google doesn't understand the needs of site developers like the other Social players do.<br /><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/mIopubupmIs" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com3http://lookfirst.com/2011/12/social-buttons.htmltag:blogger.com,1999:blog-1033933413127832310.post-85102604891572973822011-11-17T09:35:00.001-08:002011-11-17T10:00:56.978-08:00Contributing to Open SourceI've been working on various open source projects since around 1993. Long before I even really thought of it as open source. It just seemed natural to me to make the fixes I needed and contribute them back. It was always a bit of a challenge to figure out how to get my fixes to the developers. Obviously, they don't know me, so they aren't going to just let me write the files directly. So, I end up sending patches via email or some other means.<br /><br />Over the years, the process for contributing to projects has gotten easier. Even more recently, it has grown by leaps and bounds thanks to <a href="http://github.com/">Github</a>.<br /><br />Case in point. I've been using the <a href="http://twitter.github.com/bootstrap/">twitter bootstrap project</a>&nbsp;for parts of the design of my new company&nbsp;<a href="http://voo.st/">Voost</a>. I like the project a lot. Like millions of other projects, it is hosted on <a href="https://github.com/twitter/bootstrap">github</a>.<br /><br />Yesterday, I noticed a small bit of documentation was missing, so I forked the project by clicking a button on the website, created a branch to work on (git checkout -b docadditions), edited the documentation, committed and&nbsp;published&nbsp;my changes and then created a pull request which tells the developers of bootstrap that I have something to contribute:<br /><br /><a href="https://github.com/twitter/bootstrap/pull/647">https://github.com/twitter/bootstrap/pull/647</a><br /><br />Mark, one of the developers, who I've never met in my life, was able to take my contributions and combine them with his code by simply clicking a button on the website. Yes, it was that easy.<br /><br />I also had an enhancement request... so I created an issue... <br /><br /><a href="https://github.com/twitter/bootstrap/issues/646">https://github.com/twitter/bootstrap/issues/646</a><br /><br />It was resolved in a few hours with just a small bit of effort. I can then merge his changes into my local fork of the project with a couple easy commands. We stay in perfect sync together.<br /><br />Bam. That is how collaborative development should work. <br /><br />As a comparison, in the past, I've done a huge amount of work for the <a href="http://apache.org/">Apache Software Foundation</a>. They have a great open source license, and a huge following. But, they don't use github.<br /><br />With the ASF, it feels like 1993 again. For each project I want to contribute to, it feels like I'm making a lifetime commitment to that project.<br /><br />I have to go to the project website and navigate around&nbsp;to figure out how to join a&nbsp;mailing list. This takes several contextual steps in an email client. I need make sure to setup a mail filter to&nbsp;deal with a potentially insane amount of email that I really don't care about. Then, I email a patch to&nbsp;the list (or put it up on <a href="http://gist.github.com/">gist</a>&nbsp;/&nbsp;<a href="http://pastebin.com/">pastebin</a>)... and I hope maybe one of&nbsp;the developers might be watching my carefully crafted subject line. Chances are that nobody would respond or the email&nbsp;would get lost, so I'd have to keep nagging people because everyone is busy...<br /><div><br /></div><div>I don't really contribute to the ASF nearly as much anymore.</div><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/WFlw9YJmo1c" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com13http://lookfirst.com/2011/11/contributing-to-open-source.htmltag:blogger.com,1999:blog-1033933413127832310.post-74681451154683188792011-11-03T16:51:00.000-07:002011-11-03T17:07:38.593-07:00Month 3 - It is official now.It is official, Jeff and I have our own company now. <a href="http://voo.st/">Voost LLC</a>. You can sign up on the website to be notified when we launch.<br /><br />The focus of the company is on sporting event registrations. As a road bike racer who started his 'career' sending checks in the mail to promoters, I've often wondered why there isn't a great solution for handling registrations to events. (Especially when you'd get to an event, but your check didn't.)<br /><br />Many companies have sprung up to do this online, but they suffer from high fees, websites that look like they were designed in the early 1990's by people with no design skills and general poor execution. Promoters have to jump through hoops to create events and get their money. Participants are missing out on social features like communicating about the event, ride share / hotels and simple things like the ability to see their (and others) results over time. How many times have you had registration open and the site just melted down as 6k people all tried to register at the same time?<br /><br />So, it turns out that there is 10+ million athletes in the United States who cross a finish line at an endurance event every year. Voost is setup to do it right and disrupt the entire industry with a well designed solution. We've done our homework and we have a clear path of execution. Now, it is just a matter of time and effort to put our designs into code.<br /><br />Things are still rolling along with long days of coding and adding features. It is now possible to sign up for an event and go through the entire purchase process. It sounds simple, but there was a huge amount of plumbing that needed to happen first. We are currently working on the event editor which allows promoters to easily setup their event entirely online in no time at all.<br /><br />We've done a huge amount of work and I'm really proud of it. On a technical note, we've recently switched from <a href="http://sass-lang.com/">Sass</a> to <a href="http://lesscss.org/">Less</a>. I like the syntax of Less better (especially the mixin's) and it was pretty simple to do.<br /><br />We are hoping for a soft launch around December and pick up steam as we head into the new year. Wish us luck.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/MbSwbEuL2xM" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/11/month-3-it-is-official-now.htmltag:blogger.com,1999:blog-1033933413127832310.post-55039531437800175352011-10-07T22:34:00.000-07:002011-10-07T22:34:13.427-07:00Two months of work...I realize that I've lapsed in posting! I've had my head down and full concentration is in effect. The days have been long, with 12+ hour near daily sessions of coding away. We've made an astonishing amount of progress in such a short time. The framework is built and things are starting to come together. Features are appearing all over the place.<br /><br />A few days ago, we got some major news. One of our direct competitors has gone out of business. On one hand, it worries me that they couldn't survive. On the other hand, I know that a big part of their failure is due to the fact that they needed to contract out their website development to another company. This means that every time they needed to add a feature or fix a bug on the site, they had to pay someone to do this work for them. This is a huge advantage that we have over all of the existing companies. Both Jeff and I are the ones who really know how to write code. Our only expense is our own time spend on this project. This will also allow us to offer our superior service at a much lower price than anyone else.<br /><br />Thanks to a discussion with very smart friend of mine, we were able to take advantage of the other companies failure and craft a well worded email to a large group of people who we look forward to doing business with in the future. We asked them for their ideas on what they would like to see from us. Nothing beats listening to the people you want to do business with and letting them have a positive influence over what you are creating. We got a lot of amazing feedback and being a small agile company,&nbsp;have already made the appropriate adjustments.<br /><br />All of the other sites who currently have a business still have a major advantage over us. They have a shipping product and we don't. Of course, this also means that we can study them in minute detail to find their weaknesses and exploit them to our advantage. I feel that we have been very good at this. We've spent a huge amount of time designing around every one else's failures. Our user experience is unique in its beauty and simplicity. It is hard to beat that.<br /><br />Interesting times... looking forward to our beta release... now back to coding...<br /><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/q8QrFlo1hNg" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/10/two-months-of-work.htmltag:blogger.com,1999:blog-1033933413127832310.post-4853066979897884462011-09-03T01:10:00.000-07:002011-09-03T08:35:23.416-07:00After a Month of DevelopmentIt has been about a month since I came up with my new Stealthy Startup idea and we've gotten so much done already. Never mind the fact that I'm <i>working </i>and<i> coding</i> like a mad man. The integration of the <a href="http://lookfirst.com/2011/08/tools-for-new-generation.htmlhttp://lookfirst.com/2011/08/tools-for-new-generation.html">whole infrastructure</a> is complete and I'm coding up application features as quickly as I can. This is a huge project and will take a lot of effort to do it right. That said, it really is a lot of fun to develop on this platform and the site is very web 3.0 dynamic.<br /><br />We've switched around a few JavaScript libraries. We replaced <a href="http://requirejs.org/">RequireJS</a> with <a href="http://labjs.com/">LabJS</a>. The documentation for RequireJS is pretty, but when you really dig into it, it is more like fluff. We got what we wanted out of LabJS though.<br /><br />For textarea's we've integrated the <a href="http://code.google.com/p/pagedown/">Markdown editor</a> from <a href="http://stackoverflow.com/">Stackoverflow</a>. That was actually quite fun to do. Thanks for making that available.<br /><br />I spent way too much time on implementing file upload. I started to write my own, but got overwhelmed with the cross browser issues. I ended up using <a href="http://www.plupload.com/">Plupload</a>, which isn't great, but&nbsp;everything else out there is a pile of junk. The lack of good documentation made it a royal pain to integrate Plupload, but once I dug through the source code and figured out how it works, I was able to make it do what I want.<br /><br /><a href="http://jashkenas.github.com/coffee-script/">CoffeeScript</a> has been a godsend. The guy who came up with it should get a Nobel prize for brilliance. If you are writing any JavaScript at all, you should immediately stop what you are doing and switch your entire environment to CS. I'm writing a metric ton of CS and I just can't even begin to explain how much easier my life has been.<br /><br />There is a lot of libraries for dealing with html forms in a MVC way, but in the end, I've found that they require as much code as just writing it yourself. CoffeeScript's scoping and class system makes it easy to contain the logic.<br /><br />Using <a href="https://github.com/">Github</a> is also great. I've got two private repo's setup now and it is so nice to be able to get emails and diff's without having to setup and manage my own server. Plain and simple: Fuck Subversion. I know those are harsh words, but <a href="http://www.youtube.com/watch?v=4XpnKHJAok8">as others have said</a>, setting CVS as your goal to beat was a really bad idea. They are almost done with version 1.7 and basic merging is still a <a href="http://lookfirst.com/2011/04/subversion-mistake.html">pile of shit</a>. Everything is always going to be a catchup game to git, so you might as well just use git. I'm still in trial mode with <a href="http://www.git-tower.com/">Tower</a> and this is definitely an application worth paying for.<br /><br />I know everyone is complaining about <a href="http://code.google.com/appengine/">Google App Engine</a> suddenly costing them more than $0. The reality of the situation is that nothing else compares to GAE. You know what is worth an unlimited amount of money? Not having to do IT. I'm an expert sysadmin and I never want to be woken up in the middle of the night ever again because some server is acting badly. I'm more than happy to pay for the&nbsp;privilege&nbsp;of having that be someone else's problem. I really don't care if you can set something up on <a href="http://www.linode.com/">Linode</a> or <a href="http://aws.amazon.com/">AWS</a> for less money. It just isn't worth the headache when one of your $10/month micro instances decides to randomly disappear.<br /><br />I still can't tell you what I'm working on, but we should have something to show soon.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/UoKzXUa2pgY" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/09/after-month-of-development.htmltag:blogger.com,1999:blog-1033933413127832310.post-65356161827284753082011-08-09T12:51:00.000-07:002011-08-11T23:16:04.258-07:00Tools for a new generationWe've been hard at work on the Stealthy Startup. A few days of 12+ hour coding sessions and things are really starting to come together. We have a lot more work to do to implement the application logic, but at least we have settled on what we feel is a great basis for building a next generation web application.<br /><br />One of the goals of this application is to do as little framework code as possible. While we are more than technically capable, we also don't want to have to hire an IT / operations department or run servers ourselves. There just isn't a need to do that any longer and this will allow us to focus on coding features and running the business. It is simply amazing that the services and frameworks that have been created in the last 6-7 years exist and are as high quality as they are.<br /><br />While everyone seems to be zigging towards Rubyland, I still feel that staying the course with Java is the right choice. You just can't beat a strongly typed language when it comes to building an application. Especially from scratch. The need to be able to re-factor code on a whim prevents bad decisions and mistakes from propagating long term. On top of that, the JVM is simply a faster product. Scala is another approach, but I'm afraid I'm just not fond of the language syntax. I'm super efficient at writing bug free Java, so why would I want to slow my development time down? Anyway, this isn't intended as a language war posting...<br /><br />People have been asking me what tools we are using for our startup, so here is the list so far:<br /><ul><li><a href="http://code.google.com/appengine/">Google App Engine</a></li><li><a href="http://code.google.com/p/google-guice/">Guice</a></li><li><a href="http://code.google.com/p/objectify-appengine/">Objectify-AppEngine</a></li><li><a href="http://www.jboss.org/resteasy">RestEasy</a></li><li><a href="http://code.google.com/p/htmleasy/">HtmlEasy</a></li><li><a href="http://code.google.com/p/cambridge/">Cambridge Template Engine</a></li><li><a href="http://projectlombok.org/">Lombok</a></li><li><a href="http://code.google.com/p/guava-libraries/">Guava</a></li><li><a href="http://commons.apache.org/">Apache Commons</a></li><li><a href="http://jashkenas.github.com/coffee-script/">CoffeeScript</a></li><li><a href="http://requirejs.org/">RequireJS</a></li><li><a href="http://jquery.com/">JQuery</a></li><li><a href="http://sass-lang.com/">Sass Scss</a></li><li><a href="https://github.com/">GitHub</a></li><li><a href="http://eclipse.org/">Eclipse</a></li><li><a href="http://www.zeroturnaround.com/">JRebel</a></li></ul><div>Google App Engine and the Datastore enables us to not have to run servers or manage databases. It also means that as our traffic increases, we can scale automatically, without even having to think about it or code up solutions on EC2. This is because we are running on Google's own architecture and we can trust them to deal with these issues in a timely fashion. We also know that we won't have to wear a pager if the system goes down and we won't have to hire a team in India to help us when things go south at 2am.</div><div><br /></div><div>Cambridge Template Engine is one of the more unique and somewhat 'beta' projects that we are using, so it is a bit of a risk, but not that bad as it seems to work pretty well as it stands right now. That said, the language it uses enables us to write extremely flexible html code. Take a look at this&nbsp;<a href="http://blog.erdincyilmazel.com/2011/02/10/comparing-cambridge-template-engine-with-jsp-velocity-freemarker-and-play-framework-templates/">blog posting</a>&nbsp;for a comparison between various engines.</div><div><br />Lombok is one of the coolest extensions to Java ever invented. It prevents you from having to write all of that silly boilerplate code that the Rubytards make fun of the Javatards for. It also allows you to annotate your classes with a simple @Slf4j annotation to enable a 'log' object to be used. Unfortunately, it only works with Eclipse, so if you are in a mixed IDE environment, you are out of luck. That said, Eclipse is the best free IDE out there, so there isn't much reason to not use it.<br /><br /></div><div>These days, using Rest on the server side is pretty much the best way to deal with things. You aren't returning HTML, you are returning data and you use JavaScript to process that data. Both Resteasy and Htmleasy make this simple to implement.<br /><br />For dependency injection, Guice is the way to go. While I like some parts of Spring, it doesn't suffer from the jarfest/xmlfest that Spring comes with and is just as fast and powerful.</div><div><br /></div><div>RequireJS is great because it allows us to separate our CoffeeScript/JS into small components that we can include on pages that need it. Effectively giving us an 'import' statement.<br /><br />You don't really want to use JavaScript anymore either. Once you check out CoffeeScript, you won't go back. People who are against using CS are out of their minds. Take this example for using JQuery to do an Ajax call to flip some toggles(), bring effects to the page and post some data to a server:<br /><br /></div><pre class="brush: js">$ -&gt;gt; $('#nameSave').click -&amp;amp;gt;<br /> $('#nameEdit img').show()<br /> $('#nameSave').attr("disabled", "disabled")<br /> $.ajax<br /> type: "POST"<br /> url: "/account/name"<br /> data:<br /> firstName: $("input[name='firstName']").text()<br /> lastName: $("input[name='lastName']").text()<br /> success: -&gt;<br /> $('.confirmation').show()<br /> $('.error').hide()<br /> $('#nameEdit img').fadeOut(1500)<br /> $('#nameSave').attr("disabled", "")<br /> error: -&gt;<br /> $('.confirmation').hide()<br /> $('.error').fadeIn().delay(2000).fadeOut()<br /> $('#nameEdit img').fadeOut(1500)<br /> $('#nameSave').delay(2000).attr("disabled", "")<br /></pre><br />That generates this JavaScript:<br /><br /><pre class="brush: js">(function() {<br /> $(function() {<br /> return $('#nameSave').click(function() {<br /> $('#nameEdit img').show();<br /> $('#nameSave').attr("disabled", "disabled");<br /> return $.ajax({<br /> type: "POST",<br /> url: "/account/name",<br /> data: {<br /> firstName: $("input[name='firstName']").text(),<br /> lastName: $("input[name='lastName']").text()<br /> },<br /> success: function() {<br /> $('.confirmation').show();<br /> $('.error').hide();<br /> $('#nameEdit img').fadeOut(1500);<br /> return $('#nameSave').attr("disabled", "");<br /> },<br /> error: function() {<br /> $('.confirmation').hide();<br /> $('.error').fadeIn().delay(2000).fadeOut();<br /> $('#nameEdit img').fadeOut(1500);<br /> return $('#nameSave').delay(2000).attr("disabled", "");<br /> }<br /> });<br /> });<br /> });<br />}).call(this);<br /></pre><br />I know they are both kind of difficult to read (that is the nature of these languages), but which one do you think is easier to write?&nbsp;I've integrated it with Eclipse with a Builder so that as soon as you save a .coffee file, it gets 'compiled' into js. That JS is then watched with another Builder which runs is through the RequireJS optimizer which runs UglyJS or Closure Compiler on the output.<br /><div><br /></div>Sass scss is wonderful because it gives us greater control over our CSS. Another Eclipse Builder is used to automatically generate the CSS files as well.<br /><br />We are using GitHub for all of our source code and wiki documents. At $7/month it is the perfect environment for doing development. Their new Mac GitHub native app is pretty useful as well. I just wish you could do a sync without having to also push.<br /><br />JRebel allows us to write Java code, click save and then reload in the browser to see the changes immediately. No need to restart Jetty/Tomcat/JBoss any longer.<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/IL2UcB4YNhA" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com2http://lookfirst.com/2011/08/tools-for-new-generation.htmltag:blogger.com,1999:blog-1033933413127832310.post-41808233574840957852011-08-05T10:10:00.000-07:002011-08-05T10:10:47.367-07:00A busy week... A new company...You'd think that being recently jobsingle would have allowed me to slow down for a few weeks and catch my breath. Well, turns out that is not the case. I just can't sit still. My mind has been buzzing for new possibilities. As a result of this focus, a really great business idea that satisfies all of my requirements for success has presented itself to me. After talking it through with a few people who also like the idea, I know it is a GO.<br /><br />Thus, my buddy Jeff and I are in the process of creating a stealthy startup that is going to provide a useful service to a whole lot of people, in a market that is sorely lacking in technology and sophistication. Even better is that it starts with an area I know quite well and absolutely love... bicycles. We have an exciting name, a great domain, a solid plan, a clear set of requirements and are busy setting up the infrastructure to execute upon.<br /><br />I love the idea of working for myself again. Why didn't I think of this sooner? ;-)<br /><br /><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/Zn2jzeAWHug" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/08/busy-week-new-company.htmltag:blogger.com,1999:blog-1033933413127832310.post-48822355926994225572011-07-30T13:19:00.000-07:002011-07-30T13:19:36.544-07:00Jobsingle and seeking!I'm happily jobsingle and I'm looking&nbsp;for the next perfect job.<br /><br />I'd like to find a stealthy startup that expects huge amounts of traffic and has a clear business model that won't go bust when this bubble bursts. I'm interested in the backend technology and helping solve scaling solutions based on my wide experience working in that area for the last 5 years.<br /><br /><a href="http://linkedin.com/in/lookfirst">http://linkedin.com/in/lookfirst</a><br /><br />cheers,<br /><br />jon<img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/lu-sucTFkAE" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/07/jobsingle-and-seeking.htmltag:blogger.com,1999:blog-1033933413127832310.post-39585742260485899552011-07-30T12:05:00.000-07:002011-07-30T12:05:11.048-07:00Why machine images don’t workI just read a really good blog posting from RightScale on <a href="http://blog.rightscale.com/2010/03/22/rightscale-servertemplates-explained/">"Why machine images don’t work."</a><div><br /></div><div>After investigating the way that <a href="http://aws.amazon.com/amis">AMI's</a> are built and seeing how utterly difficult it was to build one, I've been&nbsp;trying to put to words my feelings about it. This article does a great job of describing it with four simple statements:<br /><ul><li>Images are too monolithic.</li><li>Images are opaque.</li><li>Images are too big.</li><li>Images are too static.</li></ul></div><div>I have one additional thing that I would like to add:</div><div><ul><li>Images are difficult to upgrade.</li></ul></div><div>I quickly came to the conclusion that building Debian packages that can be quickly installed on any machine is a far better way to go. Not only are they easily to create, but you can integrate them into your continuous integration system so that every time someone commits code, a new package is built and added to a central repository. Updating the code on your machines in all of your environments is as simple as <i>'aptitude update; aptitude safe-upgrade'</i>.</div><div><br /></div><div>I also think that using tools like Fabric, Puppet and Chef (FPC) just add another layer of unnecessary complexity which completely fails the <a href="http://en.wikipedia.org/wiki/KISS_principle">KISS</a> principle. You can do everything that FPC can do in a single Debian or break it out into multiple ones depending on how you want to setup the deployment&nbsp;hierarchy. Why install some other complicated piece of software (and all of its dependencies) with its own domain specific language when you can just write relatively simple bash shell scripts?</div><div><br /></div><div>With my deployments, I like to set things up so that there is a 'base' Debian package which gets installed called project-init. It is responsible for creating users, accepting the JDK license agreement, setting the timezone of the machine and any other low level OS settings that can be used on all machines.</div><div><br /></div><div>From there, everything gets layered on top of that base package. Some packages will be optional depending on which environment they get put into. For example, you probably don't want or need to install <a href="https://github.com/lookfirst/clarity">Clarity</a> on your production servers. If you need packages for specific environments (dev, staging, prod), you can use <a href="http://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual">Debian Virtual packages</a> to create 'aliases' for which the system you want installed.</div><div><br /></div><div>In the end, I know this system works really well. I've done it for one of the most complicated systems one can imagine with 25+ different packages for all of the components that needed to be installed.</div><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/awVze_WFFt4" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com0http://lookfirst.com/2011/07/why-machine-images-dont-work.htmltag:blogger.com,1999:blog-1033933413127832310.post-16898689645839571742011-07-27T14:47:00.001-07:002011-07-27T16:51:38.459-07:00Easy Install HBase/Hadoop in Pseudo Distributed Mode<span class="Apple-style-span" style="font-size: x-large;">Introduction</span><br /><br />This documentation should get you up and running with a full pseudo distributed Hadoop/HBase installation in an Ubuntu VM quickly. I use Ubuntu because the Debian Package Management (apt) is by far the best way to install software on a machine. It is possible to also use this on regular hardware as well.<br /><br />The reason why you will need this is because much of the existing documentation is spread around quite a few different locations. Thus, I've already done the work of digging this information out so that you don't have to.<br /><br />This documentation is intended to be read and used from top to bottom. Before you do an initial install, I suggest you read through it once first.<br /><br /><span class="Apple-style-span" style="font-size: large;">Reference Manuals</span><br /><ul><li><a href="https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation">https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation</a></li><li><a href="https://ccp.cloudera.com/display/CDHDOC/CDH3+Deployment+in+Pseudo-Distributed+Mode">https://ccp.cloudera.com/display/CDHDOC/CDH3+Deployment+in+Pseudo-Distributed+Mode</a></li><li><a href="https://ccp.cloudera.com/display/CDHDOC/ZooKeeper+Installation">https://ccp.cloudera.com/display/CDHDOC/ZooKeeper+Installation</a></li><li><a href="https://ccp.cloudera.com/display/CDHDOC/HBase+Installation">https://ccp.cloudera.com/display/CDHDOC/HBase+Installation</a></li><li><a href="http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#PseudoDistributed">http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#PseudoDistributed</a></li><li><a href="http://hbase.apache.org/book.html">http://hbase.apache.org/book.html</a></li><li><a href="http://hbase.apache.org/pseudo-distributed.html">http://hbase.apache.org/pseudo-distributed.html</a></li></ul><br /><span class="Apple-style-span" style="font-size: large;">Create the virtual machine</span><br /><br />The first thing that you will want to do is download a copy of the <a href="http://www.ubuntu.com/download/server/download">Ubuntu Server 10.04</a> 64bit ISO image. This version is the current Long Term Support (LTS) version. These instructions may work with a newer version, but I'm suggesting the LTS because that is what I test with and also what your operations team will most likely want to install into production. Once you have the ISO, create a new virtual machine using your favorite VM manager (I like vmware fusion on my Mac).<br /><br /><span class="Apple-style-span" style="font-size: large;">Unix Box Setup</span><br /><br />Once you have logged into the box, we need to setup some resources...<br /><br /><pre class="brush: shell">echo "deb http://archive.canonical.com/ lucid partner" &gt; /etc/apt/sources.list.d/partner.list<br />echo "deb http://archive.cloudera.com/debian lucid-cdh3 contrib" &gt;&gt; /etc/apt/sources.list.d/cloudera.list<br />echo "deb-src http://archive.cloudera.com/debian lucid-cdh3 contrib" &gt;&gt; /etc/apt/sources.list.d/cloudera.list<br />echo "sun-java6-bin shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections<br />echo "hdfs - nofile 32768" &gt;&gt; /etc/security/limits.conf<br />echo "hbase - nofile 32768" &gt;&gt; /etc/security/limits.conf<br />echo "hdfs soft/hard nproc 32000" &gt;&gt; /etc/security/limits.conf<br />echo "hbase soft/hard nproc 32000" &gt;&gt; /etc/security/limits.conf<br />echo "session required pam_limits.so" &gt;&gt; /etc/pam.d/common-session<br /><br />aptitude install curl wget<br />curl -s http://archive.cloudera.com/debian/archive.key | sudo apt-key add -<br />aptitude update<br />aptitude install openssh-server ntp<br />aptitude install sun-java6-jdk<br />aptitude safe-upgrade<br />reboot now<br /></pre><br />You can now use ifconfig -a to find out the IP address of the virtual machine and log into it via ssh. You will want to execute most of the commands below as root.<br /><br /><span class="Apple-style-span" style="font-size: large;">LZO Compression</span><br /><br />This setup provides LZO compression for your data in HBase which greatly reduces the amount of data which is stored on disk. Sadly, LZO is under the GPL license, so it can't be distributed with Apache. Therefore, I'm providing a nice debian that I got ahold of for you to use. On your vm:<br /><br /><pre class="brush: shell">wget "https://github.com/lookfirst/fileshare/blob/master/Cloudera-hadoop-lzo_20110510102012.2bd0d5b-1_amd64.deb?raw=true"<br />dpkg -i Cloudera-hadoop-lzo_20110510102012.2bd0d5b-1_amd64.deb<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">Hadoop / HDFS</span><br /><br />Install some packages:<br /><pre class="brush: shell">apt-get install hadoop-0.20<br />apt-get install hadoop-0.20-namenode hadoop-0.20-datanode hadoop-0.20-jobtracker hadoop-0.20-tasktracker<br />apt-get install hadoop-0.20-conf-pseudo<br /></pre><br />Edit some files:<br /><br /><b>/etc/hadoop/conf/hdfs-site.xml</b><br /><pre class="brush: xml">&lt;property&gt;<br /> &lt;name&gt;dfs.datanode.max.xcievers&lt;/name&gt;<br /> &lt;value&gt;4096&lt;/value&gt;<br />&lt;/property&gt;<br /></pre><b>/etc/hadoop/conf/core-site.xml</b><br /><pre class="brush: xml">&lt;property&gt;<br /> &lt;name&gt;io.compression.codecs&lt;/name&gt;<br /> &lt;value&gt;org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec&lt;/value&gt;<br />&lt;/property&gt;<br /> <br />&lt;property&gt;<br /> &lt;name&gt;io.compression.codec.lzo.class&lt;/name&gt;<br /> &lt;value&gt;com.hadoop.compression.lzo.LzoCodec&lt;/value&gt;<br />&lt;/property&gt;<br /></pre><b>/etc/hadoop/conf/mapred-site.xml</b><br /><pre class="brush: xml">&lt;property&gt;<br /> &lt;name&gt;mapred.compress.map.output&lt;/name&gt;<br /> &lt;value&gt;true&lt;/value&gt;<br /> &lt;/property&gt;<br /> <br /> &lt;property&gt;<br /> &lt;name&gt;mapred.map.output.compression.codec&lt;/name&gt;<br /> &lt;value&gt;com.hadoop.compression.lzo.LzoCodec&lt;/value&gt;<br /> &lt;/property&gt;<br /> <br /> &lt;property&gt;<br /> &lt;name&gt;mapred.child.ulimit&lt;/name&gt;<br /> &lt;value&gt;1835008&lt;/value&gt;<br /> &lt;/property&gt;<br /> <br /> &lt;property&gt;<br /> &lt;name&gt;mapred.tasktracker.map.tasks.maximum&lt;/name&gt;<br /> &lt;value&gt;2&lt;/value&gt;<br /> &lt;/property&gt;<br /><br /> &lt;property&gt;<br /> &lt;name&gt;mapred.tasktracker.reduce.tasks.maximum&lt;/name&gt;<br /> &lt;value&gt;2&lt;/value&gt;<br /> &lt;/property&gt;<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">ZooKeeper</span><br /><pre class="brush: shell">apt-get install hadoop-zookeeper-server<br /></pre><b>/etc/zookeeper/zoo.cfg</b><br /><pre class="brush: shell">Change localhost to 127.0.0.1<br /> Add: maxClientCnxns=0<br /></pre><pre class="brush: shell">service hadoop-zookeeper-server restart<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">HDFS/HBase Setup</span><br /><br />Make an /hbase folder in hdfs<br /><pre class="brush: shell">sudo -u hdfs hadoop fs -mkdir /hbase<br />sudo -u hdfs hadoop fs -chown hbase /hbase<br />NOTE: If you want to delete an existing hbase folder, first stop hbase!<br />sudo -u hdfs hadoop fs -rmr -skipTrash /hbase<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">HBase Installation</span><br /><pre class="brush: shell">apt-get install hadoop-hbase<br />apt-get install hadoop-hbase-master<br /></pre><br /><b>/etc/hbase/conf/hbase-site.xml</b><br /><pre class="brush: xml">&lt;property&gt;<br /> &lt;name&gt;hbase.cluster.distributed&lt;/name&gt;<br /> &lt;value&gt;true&lt;/value&gt;<br />&lt;/property&gt;<br />&lt;property&gt;<br /> &lt;name&gt;hbase.rootdir&lt;/name&gt;<br /> &lt;value&gt;hdfs://localhost/hbase&lt;/value&gt;<br />&lt;/property&gt;<br /></pre><br /><b>/etc/hbase/conf/hbase-env.sh</b><br /><pre class="brush: shell">export HBASE_CLASSPATH=`ls /usr/lib/hadoop/lib/cloudera-hadoop-lzo-*.jar`<br />export HBASE_MANAGES_ZK=false<br />export HBASE_LIBRARY_PATH=/usr/lib/hadoop/lib/native/Linux-amd64-64<br /></pre><br /><b>/etc/hadoop/conf/hadoop-env.sh</b><br /><pre class="brush: shell">export HADOOP_CLASSPATH="$HADOOP_CLASSPATH":`hbase classpath`<br /></pre><br />Now, restart the master and start the region server:<br /><pre class="brush: shell">service hadoop-hbase-master restart<br />apt-get install hadoop-hbase-regionserver<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">Starting/Stopping everything</span><br /><br /><b>Start</b><br /><ul><li>service hadoop-zookeeper-server start</li><li>for service in /etc/init.d/hadoop-0.20-*; do sudo $service start; done</li><li>service hadoop-hbase-master start</li><li>service hadoop-hbase-regionserver start</li></ul><b>Stop</b><br /><ul><li>service hadoop-hbase-regionserver stop</li><li>service hadoop-hbase-master stop</li><li>for service in /etc/init.d/hadoop-0.20-*; do sudo $service stop; done</li><li>service hadoop-zookeeper-server stop</li></ul><br /><span class="Apple-style-span" style="font-size: large;">Hbase Shell</span><br /><pre class="brush: shell">su - hbase<br />hbase shell<br /></pre><br /><span class="Apple-style-span" style="font-size: large;">Ports</span><br /><br />To ensure that everything is working correctly, visit your VM's ip address with these ports on the end of a http url.<br /><ul><li>HDFS: 50070</li><li>JobTracker: 50030</li><li>TaskTracker: 50060</li><li>Hbase Master: 60010</li><li>Hbase RegionServer: 60030</li></ul><img src="http://feeds.feedburner.com/~r/AnOpinionOn/~4/lF5iNsUVpLc" height="1" width="1" alt=""/>Jon Stevenshttps://plus.google.com/111536601970188216986noreply@blogger.com13http://lookfirst.com/2011/07/easy-install-hbasehadoop-in-pseudo.html