Jekyll2019-02-07T05:59:30+00:00https://kevinslin.com/feed.xmlThence ConsultingWritings on AWS, Technology and PhilosophyKevin S Linkevin@cloud.thence.ioBenchmarking Lambda’s Idle Timeout Before A Cold Start2019-02-05T00:00:00+00:002019-02-05T00:00:00+00:00https://kevinslin.com/aws/lambda_cold_start_idle<figure class="">
<img src="/assets/images/20190204-header.jpg" alt="Warming from a cold start" />
</figure>
<h2 id="history-of-compute">History of Compute</h2>
<p>In the not so distant past, businesses that wanted to build internet services needed to provision their own hardware. That meant buying a rack and a bunch of servers and stashing them in a cool part of the building and hope that no one would trip over the wires.</p>
<p>As businesses grew, so did their IT infrastructure which meant that they sometimes had to expand into data centers to handle all that new traffic. Besides costing an obscene amount of money, this also involved much complexity, especially as traffic spiked, machines failed and disks died.</p>
<p>Then came the advent of cloud computing which promised to free businesses from all that complexity for a less obscene amount of money. Instead of ordering hardware ahead of time, a server of your choice was just an API call away. This meant that companies could pay only for the traffic they have today but also have the capacity to handle spikes and growth.</p>
<p>Cloud computing drastically slashed the capital and hardware necessary for businesses to scale up their IT. But under the hood, it was still running workloads on VMs in the same way that people have been doing since dot com. But all this changed in a big way November of 2014.</p>
<h2 id="aws-lambda">AWS Lambda</h2>
<p>AWS introduced <a href="https://aws.amazon.com/lambda/">Lambda</a> during re:invent of 2014 and presented it as a new paradigm for thinking about code. Instead of worrying about servers, upload the application code and lambda would take care of the rest.</p>
<p>Lambda promised instant deployments and <em>infinite</em> scale with the benefit of costing nothing when it wasn’t invoked. But as with any technology, there are tradeoffs for having nice things and with Lambda, one of those tradeoffs is cold starts.</p>
<h2 id="cold-starts">Cold Starts</h2>
<p>Lambda executes code in <a href="https://www.docker.com/resources/what-container">containers</a>. During the first invocation of your function, Lambda will instantiate the container containing your application code. This initial initialization cost is known as a <strong>cold start</strong>.</p>
<p>The penalty of a cold start varies by language, framework, memory allocation and whether you’re running your Lambda inside a VPC (just don’t). Typically, it can be anywhere from ~200ms (python) to over 5s (C#). For a more detailed analysis on the breakdown, you can read the following <a href="https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76">post</a>.</p>
<p>Subsequent calls to Lambda will reuse the initial container. Lambda will keep the container <strong>hot</strong> as long as it continues to be invoked within an <em>idle timeout</em> interval. The value of this <em>idle timeout</em> is not published by AWS and the subject of this post.</p>
<p>Before I begin, I have to mention that I’m not the first person to write about this topic. I personally like the <a href="https://read.acloud.guru/how-long-does-aws-lambda-keep-your-idle-functions-around-before-a-cold-start-bf715d3b810">analysis</a> done by AWS Hero Yan Cui. By executing Lambdas of various memory sizes with monotonically increasing idle wait times between each invocation, Cui was able to derive an upper bound for idle timeout of various Lambda sizes. An image of his results is posted below.</p>
<figure class="">
<img src="https://cdn-images-1.medium.com/max/1600/1*NjZamQ0Utn2nUudiBm12lg.png" alt="Lambda cold start timeouts" />
<figcaption>Cui’s Idle Timeout Numbers
</figcaption>
</figure>
<p>Based on the above numbers, it seemed that Lambda containers were able to idle for as much as an hour before being reaped. Whether Lambda’s would actually idle for that long seemed to depend on Lambda size and VM utilization.</p>
<p>Cui published his post in 2017 and Lambda has made extensive changes to the platform in that time, including increasing the memory limit of Lambda to 3GB. I was curious to see how <em>idle timeout</em> changed in the interim and run the test for the bigger Lambda memory sizes.</p>
<p>For the test, I re-used Yan Cui’s <a href="https://github.com/theburningmonk/lambda-when-will-i-coldstart">code</a> and setup - this involves creating a series of lambdas using the <a href="https://serverless.com">serverless framework</a> and orchestrating the tests using <a href="https://aws.amazon.com/step-functions/">step functions</a>. We start off with an idle timeout hard coded at 10min and use step functions to continuously invoke the lambda function to see if we get a cold start. If not, we increase the timeout, wait and try again. We repeat this process until we get 10 cold starts and use the maximum timeout to determine an upper bound on the idle timeout.</p>
<p>The following test was conducted in us-west-2 at 2019-01-29</p>
<h2 id="results-us-west-2">Results (us-west-2)</h2>
<p>And now for the results. Drum roll… 🥁🥁🥁</p>
<figure class="">
<img src="/assets/images/20190204-us_west_2_lambda_upper.png" alt="Lambda us-west-2 Idle Timeout before cold start" />
<figcaption>Lambda Max Idle Timeout Before Cold Start
</figcaption>
</figure>
<p>So this turned out not be the most exciting graph in the world and also different from what I was expecting. 26min was not just the max idle timeout but the idle timeout we got in every execution of Lambda across all memory sizes.</p>
<figure class="">
<img src="/assets/images/20190204-us_west_2_lambda_desc.jpg" alt="Lambda us-west-2 Idle Timeout Description" />
<figcaption>All Lambda Idle Timeouts
</figcaption>
</figure>
<p>To read the above table, know that the first column is the Lambda memory size and the other columns show the statistical properties of the idle timeout in minutes.</p>
<p>The idle timeout of 26min is actually half the max idle timeout from when Cui did his <a href="https://read.acloud.guru/how-long-does-aws-lambda-keep-your-idle-functions-around-before-a-cold-start-bf715d3b810">benchmark</a> in 2017. This is also 5x more than what tools like the <a href="https://github.com/FidelLimited/serverless-plugin-warmup">serverless-plugin-warmup</a> will ping your lambda at (default is set to 5min). Something to note is that these results only show the <em>upper bound</em> of the idle timeout and it could very well be the case that the lambdas did timeout after 5minutes though the consistency of the results would suggest otherwise.</p>
<h2 id="but-wait">But Wait…</h2>
<p>At this point, I was ready to make dinner and call it a day but there was one more thing I wanted to do before wrapping up the experiment. Something to always consider when doing a benchmark on AWS is that it is always region dependent. Each AWS region is designed to be <a href="https://docs.aws.amazon.com/aws-technical-content/latest/aws-overview/global-infrastructure.html">completely isolated</a> from other Amazon regions - this means physical proximity, hardware, power supply, etc. The instance type availability differs from region to region and so can the code that is deployed within each region. us-east-1 is AWS’s oldest region and has the widest possible skew in terms of instance availability and workloads. I wanted to find out if the observed Lambda idle timeouts were also consistent here so I repeated the above experiment in us-east-1.</p>
<p>The following test was conducted in us-east-1 at 2019-02-01</p>
<h2 id="results-us-east-1">Results (us-east-1)</h2>
<figure class="">
<img src="/assets/images/20190204-us_east_1_lambda_upper.png" alt="Lambda us-east-1 Idle Timeout before cold start" />
<figcaption>Lambda Max Idle Timeout Before Cold Start
</figcaption>
</figure>
<p>So this is a more interesting graph. Note that there is a general trend of the idle timeout decreasing with higher memory allocations.</p>
<figure class="">
<img src="/assets/images/20190204-us_east_1_lambda_desc.jpg" alt="Lambda us-east-1 Idle Timeout Description" />
<figcaption>All Lambda Idle Timeouts
</figcaption>
</figure>
<p>Some immediate observations:</p>
<ul>
<li>2624MB lambda shows the same behavior as us-west-2 lambda, having a consistent idle timeout of 26min</li>
<li>all other lambdas have higher idle timeouts, all the way up to 65min</li>
<li>initial idle timeout still starts at 26min with two exceptions, 31min for 256MB and 22min for 1856MB</li>
<li>1024MB lambda is only Lambda under 1536MB to have under 60min of idle timeout</li>
</ul>
<p>Now that we know what the max idle timeout is, what about idle timeouts less than that? That is to say, before we hit the upper bound in idle timeout, what were the other idle timeout’s that Lambda hit along the way?</p>
<figure class="">
<img src="/assets/images/20190204-us_east_1_lambda_facet.png" alt="Lambda us-east-1 Idle Timeout Before Hitting Uppper Bound" />
<figcaption>Anyone ask for more graphs?
</figcaption>
</figure>
<p>There’s a lot going on here but some observations:</p>
<ul>
<li>there is much less variance among bigger Lambdas - after 1792MB, most of the idling is done at the max idle timeout (which ranges from 26-54min)</li>
<li>smaller Lambdas (under 1792MB) have a much higher variance when it comes to idle timeout lengths but tend to idle at high timeouts (~1h)</li>
</ul>
<h2 id="takeaways">Takeaways</h2>
<p>So takeaways from this experiment?</p>
<ul>
<li>Lambdas in us-west-2 have a consistent idle timeout of 26min for all lambda sizes</li>
<li>Lambdas in us-east-1 have varying idle timeouts, ranging from 22min - 65min</li>
<li>Lambdas in us-east-1 with smaller memory sizes (under 1536MB) have the highest upper idle timeouts</li>
<li>idle timeout is different between different AWS regions and you should not assume that benchmarks done in one region stay consistent across regions</li>
<li>any number that is not officially announced by AWS is subject to change</li>
</ul>
<p>Finally,I would be remiss at this point not to add the disclaimer that these benchmarks were done to satisfy my own curiosity and you shouldn’t base your production worloads off these results.</p>
<p>That being said, if you’re curious to run through any of the above scenarios, you can find the source code to run this experiment on <a href="https://github.com/kevinslin/lambda-when-will-i-coldstart">github</a></p>
<p>Some ideas for further tests:</p>
<ul>
<li>Idle timeout of Lambda using Lambda Layers</li>
<li>Idle timeout of Lambda run at increasing concurrencies</li>
<li>Idle timeout of Lambda in VPC</li>
<li>Idle timeout of Lambda that make use of their scratch space</li>
<li>Idle timeout of Lambda run during region peak hours (looking at you <a href="https://qz.com/india/989659/netflix-nflx-peak-viewing-hours-are-very-different-in-india-and-argentina-compared-to-the-us-and-europe/">Netflix</a>)</li>
</ul>
<p>Hope you found this article useful and may the Lambda be (warm) with you!</p>Kevin S Linkevin@cloud.thence.ioAn investigation of how long a Lambda can idle before a cold start. Benchmarks includes Lambdas from 128MB all the way up to 3008MB across multiple regions.Giving Third Parties Limited Access to your AWS Account2019-01-17T22:37:58+00:002019-01-17T22:37:58+00:00https://kevinslin.com/aws/aws_account_access_policies<figure class="">
<img src="/assets/images/20190121-access_lock.jpg" alt="AWS Access Control represented by lock picture" />
</figure>
<p>Whenever I’m working with a new client, one of the first things we talk about is access. Most clients have the majority of their business running on top of AWS and don’t feel comfortable (nor would I recommend) that they give a consultant complete access on day one. That being said, said consultant does need minimal access in order to advise, whether this is on security, architecture, cost, or any number of reasons. This is when we turn to IAM.</p>
<h1 id="iam-and-managed-policies">IAM and Managed Policies</h1>
<p><a href="https://aws.amazon.com/iam/">IAM</a> (Identity Access Management) is one of the longest running and most comprehensive service in AWS (and arguably the most feature rich IAM service of any cloud provider). With IAM, users can specify precisely the level of access any given entity will have to their AWS resources. You can narrow down permissions to single actions in specific services (eg. Bob can only call <code class="highlighter-rouge">s3:GetLifecycleConfiguration</code> on bucket <code class="highlighter-rouge">Foo</code>) as well as use wildcards to specify a whole range of permissions (eg. Bob can call <code class="highlighter-rouge">s3:List*</code> actions on all buckets starting with <code class="highlighter-rouge">Foo*</code>). IAM is an incredibly flexible and powerful way of only providing exactly the permissions that are actually necessary but comes at the cost of being verbose and tricky to get right. AWS is aware of this issue and offers <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies">managed policies</a> that deal with some common use cases. The advantage of using managed policies is that they are implemented and managed by AWS security engineers and are regularly updated to include the latest services and features.</p>
<p>Today, we’re going to talk about three of these managed policies which I recommend my clients use when giving third parties access. They are as follows:</p>
<ul>
<li>ViewOnlyAccess: This policy grants permissions to view resources and basic metadata across all AWS services.
<ul>
<li>great for initial audit since it allows the user to see all aws resources and metadata but not the actual business and customer data</li>
<li>eg: will allow user to see EC2 instances, RDS instance, S3 buckets and objects (only the names but not the contents, configs), etc</li>
</ul>
</li>
<li>SecurityAudit: This policy grants access to read security configuration metadata. It is useful for software that audits the configuration of an AWS account.
<ul>
<li>great for security assessments - very similar to <code class="highlighter-rouge">ViewOnlyAccess</code> but grants additional access to configuration/policy while limiting certain view operations</li>
<li>comparison: <code class="highlighter-rouge">ViewOnlyAccess</code> allow users to perform <code class="highlighter-rouge">glacier:List*</code> operations but no <code class="highlighter-rouge">glacier:Get*</code> operations whereas <code class="highlighter-rouge">SecurityAudit</code> allows <code class="highlighter-rouge">glacier:ListVault</code> and limited <code class="highlighter-rouge">GET</code> operations like <code class="highlighter-rouge">glacier:GetVaultAccessPolicy</code>. Similar trade-offs are made throughout other AWS services</li>
</ul>
</li>
<li>ReadOnlyAccess: This policy provides read-only access to AWS services and resources.
<ul>
<li>great for a deeper audit - useful for troubleshooting issues because users can read data out of aws resources like S3 and Dynamo</li>
<li>comparison: all the permissions of <code class="highlighter-rouge">ViewOnlyAccess</code> with additional permissions to read resource contents as well as the metadata</li>
<li>note that anyone with this policy can <strong>READ AND DOWNLOAD DATA from S3, Dynamo and your other aws services</strong> - for most cases, <code class="highlighter-rouge">ViewOnlyAccess</code> is sufficient</li>
</ul>
</li>
</ul>
<p>While the above mentioned security policies are updated to include the latest AWS services, the rate that AWS releases changes means that these even these policies can be out of date. As of this writing, here are some shortcomings I found:</p>
<ul>
<li>ViewOnlyAccess:
<ul>
<li>missing permissions for GuardDuty, Alexa4Business, Cloud9, Comprehend, etc</li>
</ul>
</li>
<li>SecurityAudit:
<ul>
<li>missing permissions for Inspector, Lex, Lightsail, Polly, Simple Work Flow, WorkDocs, etc</li>
<li>a lot of the missing permissions here center around high level services like Lex and WorkDocs so that might be why they’re excluded but it’s surprising that Inspector isn’t in here as it is a security assessment service</li>
</ul>
</li>
<li>ReadOnlyAccess:
<ul>
<li>missing permissions for AppMesh, Chime, License Manager, MediaConnect, Quicksight, Resource Access Manager, RoboMaker, SecurityHub, AWSMarketPlace, etc</li>
<li>most of the missing permissions are with recently launched services (eg. RoboMaker, AppMesh), but the AWSMarketPlace is surprising since <code class="highlighter-rouge">ViewOnlyAccess</code> has the <code class="highlighter-rouge">aws-marketplace:ViewSubscriptions</code> permission which makes me suspect that this is an oversight</li>
</ul>
</li>
</ul>
<p>The takeaway here is that these managed policies are a good starting point to give a third party access limited access to your AWS account. They do miss whitelisting a service here and there but it is much better to whitelist a few services after the fact then realize you’ve given too many permissions and blacklist.</p>
<p>Now that we’ve gone over a few managed policies, below are instructions on how to apply them to your own AWS Account. Note that the instructions are for applying the <code class="highlighter-rouge">ViewOnlyAccess</code> policy but the same steps apply for the other managed policies.</p>
<h1 id="granting-viewonlyaccess-to-your-aws-account---console-instructions">Granting ViewOnlyAccess to your AWS Account - Console Instructions</h1>
<h2 id="creating-an-aws-user">Creating an AWS User</h2>
<ul>
<li>Navigate to the user tab under IAM by going to <a href="https://console.aws.amazon.com/iam/home#/users">https://console.aws.amazon.com/iam/home#/users</a> and click <code class="highlighter-rouge">Add User</code></li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-1-add_user.jpg" alt="Add User to IAM console" />
</figure>
<ul>
<li>Set a name for the user and enable both Programatic and Console Access. Then click to <code class="highlighter-rouge">Next: Permissions</code></li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-2-user_details.jpg" alt="Enable AWS console access" />
</figure>
<ul>
<li>Click <code class="highlighter-rouge">Attach existing policy directly</code> and look for <code class="highlighter-rouge">ViewOnlyAccessPolicy</code>. Select the policy before clicking <code class="highlighter-rouge">Next: Tags</code></li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-3-add_existing_policy.jpg" alt="Attach existing policies directly" />
</figure>
<figure class="">
<img src="/assets/images/20190121-aws_audit-4-add_policy.jpg" alt="ViewOnlyAccess Policy" />
</figure>
<ul>
<li>Optionally, add a tag to the project for tracking purposes. Click <code class="highlighter-rouge">Next </code> when done.</li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-5-optional_add_tag.jpg" alt="Add IAM Tags" />
</figure>
<ul>
<li>Make sure that everything looks right and click <code class="highlighter-rouge">Create User</code></li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-6-create_user.jpg" alt="Review managed policies" />
</figure>
<ul>
<li>Download the csv and send an email to the person you are granting access to with the IAM login link. Send the csv with the credentials along with the email or separately in a different channel of your choice.</li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-7-download_email.jpg" alt="" />
</figure>
<figure class="">
<img src="/assets/images/20190121-aws_audit-8-email.jpg" alt="Access Key Id, Secret Access Key and Password screen" />
</figure>
<h2 id="deleting-an-aws-user">Deleting an AWS User</h2>
<ul>
<li>After the audit is done, its important to delete the credentials to limit further access (optionally, you can also choose to disable the keys and password but this guide will cover deleting the credential). To do so, go back to the IAM user page <a href="https://console.aws.amazon.com/iam/home#/users">here</a>. Select the user you wish to delete and click <code class="highlighter-rouge">Delete User</code>. Click <code class="highlighter-rouge">Yes</code> on the ensuing confirmation box.</li>
</ul>
<figure class="">
<img src="/assets/images/20190121-aws_audit-9-delete_user.jpg" alt="Send email with Sign-in URL to Audit Only User" />
</figure>
<p>And you’re done. Hope this article was informative and feel free to reach out or comment if you have any questions or feedback for this post.</p>
<h1 id="granting-viewonlyaccess-to-your-aws-account---cli-instructions">Granting ViewOnlyAccess to your AWS Account - CLI Instructions</h1>
<p>If you want to automate the above process, you can do so by running the following series of AWS CLI commands</p>
<h2 id="creating-an-aws-user-1">Creating an AWS User</h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aws iam create-user --user-name "ViewOnlyUser" --path "/"
aws iam attach-user-policy --policy-arn "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" --user-name "ViewOnlyUser"
aws iam attach-user-policy --policy-arn "arn:aws:iam::aws:policy/IAMUserChangePassword" --user-name "ViewOnlyUser"
# do this if you want to enable programatic access for user. this cmd outputs AccessKey &amp; SecretAccessKey. make sure to store this in a safe place
aws iam create-access-key --user-name "ViewOnlyUser" &gt; /tmp/aws_credentials.json
# do this if you want user to have console access - note that we use openssl to generate the password. you can use your own means of password generation
password=`openssl rand -base64 32`
aws iam create-login-profile --user-name "ViewOnlyUser" --password-reset-required --password $password
# get signin link - note, this uses the jq cli (https://stedolan.github.io/jq/) to parse the json output
signinlink=`aws iam get-user --user-name "ViewOnlyUser" | jq .User.Arn | awk -F : '{print "https://"$5".signin.aws.amazon.com/console"}'`
# use a secure channel to send the following output to a third party
cat /tmp/aws_credentials.json
echo $password
echo $signinlink
</code></pre></div></div>
<h2 id="deleting-an-aws-user-1">Deleting an AWS User</h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aws iam delete-login-profile --user-name ViewOnlyUser
aws iam detach-user-policy --user-name ViewOnlyUser --policy-arn "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess"
# this requires the access key of IAM user - note, this uses the jq cli (https://stedolan.github.io/jq/) to parse the json output
aws iam delete-access-key --user-name "ViewOnlyUser" --access-key-id=`aws iam list-access-keys --user-name "ViewOnlyUser" | jq ".AccessKeyMetadata[0].AccessKeyId"`
aws iam delete-user --user-name ViewOnlyUser
</code></pre></div></div>Kevin S Linkevin@cloud.thence.ioWhenever I'm working with a new client, one of the first things we talk about is access. Most clients have the majority of their business running on top of AWS and don't feel comfortable (nor would I recommend) that they give a consultant complete access on day one. That being said, said consultant does need minimal access in order to advise, whether this is on security, architecture, cost, or any number of reasons. This is when we turn to IAM.Luck and Chance2018-05-13T00:00:00+00:002018-05-13T00:00:00+00:00https://kevinslin.com/thoughts/chance<p>Looking back at my life, something that stands out like a garden gnome in a basketball lineup is that most of the significant moments of my life seemed to have happened by chance.</p>
<p>The joining of the cross country team in middle school because the coach promised cookies leading to the discovery of a life long passion. Drawing political cartoons on a SAT test and being offered a job at the natural history museum. The chance meeting of a person in the college cafeteria that would become my closest friend.</p>
<p>Ask anyone that knows me well and they’ll probably tell you that I’m not the type that leaves things to chance. Instead of chance, I have daily goals, monthly reviews and yearly bucket lists. And even though I leave certain gaps in my schedule for entropy, most of my days are structured around a common set of routines and activities, all of which are tailored to achieve pre-defined goals. So how is it then that despite all this, the things in life that I’m most grateful for tend to be things that I never planned for?</p>
<p>So I have a theory about this.</p>
<p>Every day of every moment, there’s an endless amount of things that might happen. You cross the street. You might get hit by a car. You might bump into someone. A seagull might drop a doozie on you. You might change your mind about crossing the street and pick up a call from your mother. We can list examples all day. Whatever you end up doing in that moment, now all your subsequent moments are changed. Explore this tree of possibilities even a few layers deep and you’ll find the number of potential realities quickly approach infinity. Not everything will lead somewhere and most won’t, but live long enough and statistically, there’s an almost certain chance that one of these moments will. Now compare this to the handful of goals that you or others might have picked out for you. What are the chances that you actually achieve those goals before a chance moment changes your life?</p>
<p>Another assumption is that achieving those goals will bring some sort of happiness. It is a well studied phenomenon that people are bad at predicting what will make them happy. This is for all sorts of reasons, chiefly because of how we are wired in a physiological sense to always want more and because we don’t understand what we want. As a result, when we set goals with the idea that their achievement will bring out a state of happiness, there’s a good chance that the achievement of said goals might not actually bring about the expected feelings of joy. And if this is the case, it might then make more sense to pick randomly and be open to a full range of results than pick deliberately and unconsciously bias towards the sort of ends that don’t bring about lasting happiness.</p>
<p>In improv, there’s this idea of walking on stage with a plan but being ready to drop that plan at a moment’s notice if the opportunity for something better comes along. There’s a lesson to be had in there. A lesson from improv. Now what are the chances of that?</p>Kevin S Linkevin@cloud.thence.ioLooking back at my life, something that stands out like a garden gnome in a basketball lineup is that most of the significant moments of my life seemed to have happened by chance.Why I Write2018-03-10T00:00:00+00:002018-03-10T00:00:00+00:00https://kevinslin.com/thoughts/why_i_write<p>I enjoy the art of creation. Maybe enjoy is too light of a word for what I’m trying to describe. I feel compelled by. I have a need to. I cannot not create.</p>
<p>One of the mediums I do this is through writing. I write about all sorts of things. Philosophical discussions I’ve had either with myself or other people. My feelings regarding recent events. Some new insight at work. Frustration at some poorly designed programming interface. The weather.</p>
<p>I write these things in half formed sentences and rough notes. Sometimes I will string them together into a longer more coherent narrative. Sometimes I will even finish this narrative. At this point, I might or might not publish. More often than not, I don’t.</p>
<p>By the point I get to publishing, I will find a reason not to. Usually, its because I’m afraid of being judged. Because I don’t think I have anything useful to contribute to the discussion. Because there is enough noise in the internets and I would just be making it worse. And so I don’t publish.</p>
<p>Two months ago, I read a Leonardo da Vinci biography by Walter Isaacson. I love biographies and I’m a fan of how Walter Isaacson writes them.</p>
<p>There’s a great many things I learned from this book. Like how Leonardo was a great procrastinator and that for all the works we know him by, there were even more that were never finished, never published or abandoned half way through. I learned that Leonardo was a bastard son, a master of hydraulics and that he liked men. Most importantly, I learned that Leonardo was selfish.</p>
<p>Leonardo didn’t cut up human cadavers because he wanted to advance the state of medical knowledge, he didn’t paint images of Jesus because of religious piety and he didn’t finish work so the world could enjoy them. Rather, he did all of this chiefly to satisfy his own calling.</p>
<p>Leonardo was compulsively curious and he desired to know how nature worked. Its not to say he didn’t care about the world at large but it is to emphasize that he was driven to do things primarily out of personal callings.</p>
<p>Some historians look at the unfinished works of Leonardo and lament at all the inventions that the world had to rediscover because Leonardo didn’t publish his work. I don’t think Leonardo would care very much at all at what those historians thought.</p>
<p>Doing things primarily for personal reasons. This is often taught as something selfish, especially in eastern cultures. Being selfish is bad. There’s a term used a lot in my family growing up - “ke qi”. One should be “ke qi” in the presence of others. One way of translating this is to be respectful of others. The implied subtext is that you should respect others before self.</p>
<p>I don’t agree with this. All the reasons why I don’t agree with this might be the topic of another entry.</p>
<p>Suffice to say that my deeply held belief today is that the most important thing you can do (and indeed must do) is to respect yourself. Dare I use a phrase from the Woodstock era - to love yourself. This is not to say that we should be narcissistic, arrogant or completely self absorbed. Instead, it means to have build a foundation of self respect that ones actions can be said to truly come from one’s self (as much as any action can be said to come from a self) instead of a need to fulfill expectations or gain external validation.</p>
<p>Even if after all this, if your prerogative is still putting other first, that’s fine. It will no longer be an obligation. It will be an action taken in adherence to deeply personal values.</p>
<p>I think of the airplane analogy - in case of de-pressurization, secure your own oxygen mask before helping others. Lack of oxygen to the brain leads to inability to think. This leads to a distorted view of reality. Instead of helping, you might end up injuring others instead.</p>
<p>To bring this back to writing - a chief reason I didn’t publish more is because I was too concerned about what everyone else might think. I didn’t think I had anything sufficiently insightful or novel to say.</p>
<p>But this is not why I write - that is, for the sake of other people. I’m writing firstly for myself. I write because the words need to come out. Because it brings coherence to many loose strands of thought. Because I want to. Because I need to. It would be nice if others found use in these writings but ultimately, that is secondary.</p>
<p>This is all to say that the entries in this blog might not be unique in a world of plus seven billion tweeting/posting/blogging people but they are uniquely my own. And that’s okay.</p>Kevin S Linkevin@cloud.thence.ioI enjoy the art of creation. Maybe enjoy is too light of a word for what I’m trying to describe. I feel compelled by. I have a need to. I cannot not create.Abstractions - Part12017-06-20T00:00:00+00:002017-06-20T00:00:00+00:00https://kevinslin.com/thoughts/abstractions_part_1<p>I think about abstractions a bunch. It comes with the territory of writing code for a living. Software is, after all, just a series of abstractions, translating the written word into instructions for a machine to follow. Examples of software abstractions include type systems, objects and memory models just to name a few.</p>
<p>An abstraction is a set of generalized rules and concepts that target a particular domain.
Abstractions are useful because they give us power over the domain that they describe. The whole field of <a href="https://en.wikipedia.org/wiki/Euclidean_geometry">Euclidean geometry</a> was given form by just five axioms that Euclid laid out in his book, the “Elements”. Similarly, all general purpose computers today can be wholly derived from the abstract <a href="https://en.wikipedia.org/wiki/Turing_machine">Turing machine</a> as conceptualized by Alan Turing less than a century ago.</p>
<p>Abstractions are everywhere. Take music. Fundamentally, music is simply sound waves propagating through air at different frequencies and amplitudes. But unless you are someone building an automatic speech recognition system, you won’t think of music in this way. Instead, we use words like “mellow” or “rad” to describe the bands we like. The more musically refined might use words like “key” or “timbre”.</p>
<p>If you look around at anything, then all you see are abstractions. The world is a concept generated by our brains when making sense of the light hitting the sensors in our corneas. It converts the primitives of light at different frequencies and turns them into color, edges and figures.</p>
<p>I went to yellow stone a year ago with a friend and we camped out for a week. My favorite part of that trip was watching the stars at night. It simply boggled my mind to look up at the night sky filled with countless pinpricks of light, each one a star so far that even light had to travel for millions of years to reach us. My mind couldn’t comprehend the vastness of the universe - it was too much.</p>
<p>Joseph Stalin is reputed to have said that “the death of one person is a tragedy; the death of one million is a statistic”. He then instituted policies that drove up many such “statistics”. It is a well studied phenomenon that people tend to become less compassionate as <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4062481/">numbers of victims increase</a>. In one such experiment, University of Oregon professor Paul Slovic ran a study on how people responded to tragedies. In one experiment, Slovic asked volunteers in one group if they would help raise money to save eight children who were dying of cancer. Slovic asked another group how much they would raise to save a single cancer patient. Slovic found that people were willing to raise more money for the single cancer patient then the eight. The findings suggested that our ability feel sympathy for people is limited.</p>
<p>If we extrapolate from this result, it seems reasonable to think that our ability to hold anything would be limited given that our minds are of a finite size (this presumes a physical basis for consciousness). British anthropologist Robin Dunbar first made the observation between a primate’s brain size and the size of its average social group. By extrapolating out to humans, Dunbar proposed that 150 relationships pushed the cognitive limit of the number of relationships a human brain could comfortably maintain. This number is famously known as Dunbar’s number and is also mentioned in Malcom Gladwell’s book, “The Tipping Point”. In it, Gladwell described the company, W. L. Gore and Associates, that through trial and error discovered that a building with more than 150 people working together would result in social problems and so limited new buildings to 150 people.</p>
<p>Similarly to Dunbar’s number, I suspect that there exists a similar cognitive limit for what people can comprehend without resorting to abstractions. An abstraction is tripped in the same manner a circuit breaker would be - when the forces acting upon the mind become too much for the medium to safely process. At this point, abstractions compress away the immense detail into smaller representations of themselves. The thing with compression algorithms though, is that there are always trade-offs and it usually comes at the expense of fidelity.</p>
<p>When programming in higher level programming languages, the ease of writing code is only matched in the inverse by the difficulty of reasoning about the performance of said code. The following is a mistake I commonly see in people who begin programming in python. What’s wrong with the following code?</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>res = ""
for s in list_of_strings:
res += s
</code></pre></div></div>
<p>This code takes a list of strings and concatenates all the elements into one long string. Because python strings are immutable, every loop will create a new string with all elements of the string thus far. This would be the equivalent of writing a book and on each additional word, rewriting all the words that have come before. Let’s take a look at the proper way to write the above code.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>res = "".join(list_of_strings)
</code></pre></div></div>
<p>This code only creates one string to achieve the same result. The mistake I just described is easy to make because to realize the cost requires knowing about the implementation details of the language at lower levels - the very thing a higher level language “protects” you from in the first place. And so while abstractions are helpful, there lie dragons.</p>
<p>It is dangerous to hold firm to an abstraction without questioning its origins as such actions give birth, not just to performance issues when coding, but also to bigotry and prejudice. Racism is an example of an abstraction taken too far, when an entire class of people is reduced to a collection of stereotypes. Other abstractions to add to this list include climate change denial and terrorism. In each case, believers have formed these abstractions with fundamentally wrong models of the underlying reality and it has led to disastrous outcomes for society.</p>
<p>There is a Heisenberg like uncertainty that happens when we abstract - as we gain more expressive power over the collection, we lose detail on specifics. Despite these trade-offs, the mind must make abstractions for the same reasons we need to build circuit breakers into our electrical systems - to prevent a meltdown. This is especially true today if one’s mind is to be sane in the endless cacophony of global information, news and social media. This also makes it especially important to question assumptions and consider things from first principles. Abstractions are useful but we should not be slaves to it. While we think in the abstract, we live in reality. Reality as we know it is constantly changing and so we must change our abstractions of it in turn. In abstract this seems easy to do. In reality, this is always a different.</p>Kevin S Linkevin@cloud.thence.ioI think about abstractions a bunch. It comes with the territory of writing code for a living. Software is, after all, just a series of abstractions, translating the written word into instructions for a machine to follow. Examples of software abstractions include type systems, objects and memory models just to name a few.Peloton2017-05-31T00:00:00+00:002017-05-31T00:00:00+00:00https://kevinslin.com/thoughts/pelotone<p>Over the weekend, I competed in <a href="http://skitosea.com/about">Ski To Sea</a>, a seven leg relay where each leg features a different sport. The relay starts off with downhill skiing off Mt Baker and ends with a kayak race into Bellingham, Washington. I did the road bike leg, a 41 mile course that starts in the middle of Mt. Baker Highway and ends in the city of Everson, Washington.</p>
<p>Drafting was allowed in the bike leg which meant bikers could form pelotons. The word comes from French and originally meant “platoon”. In the world of road biking, a peloton refers to a group of riders riding together. The reason you want to form a peloton is because having other riders in front will reduce your headwind - this makes a tremendous difference when cycling as being in the middle of a group can reduce drag by <a href="https://www.ncbi.nlm.nih.gov/pubmed/2318782">as much as 40%</a>.</p>
<p>It can be tricky finding the right peloton - if the group is too slow then you’ll lose time but if the group is too fast then you’ll burn out. Ideally, you want to find a peloton that’s just at the limit of being too fast for you - a peloton where you are the slowest member in the group.</p>
<p>It wasn’t until 20 miles into my race that I found my peloton - a group of a dozen bikers whose pace I synced with. I stuck with this group for the remainder of the race. Our numbers would change as we’d lost people on hills and gained them on windy straightaways. The core of the peloton stayed consistent - me, a girl in an orange jersey, a guy with the number 100 race bib and another guy with an iron man bike jersey. We took turns taking the wind and attacking climbs. We encouraged each other to keep pushing.</p>
<p>Even though we were all competing, we worked to keep the group together for most of the race. It was in everyone’s interest to keep a big peloton so that no individual had to take the wind for too long of a stretch. I didn’t end up breaking away until two miles to the finish. I ended up finishing the 41 miles in 1h and 54min; my average pace was 23mph, 8mph faster than my usual speed(a huge difference)! I high-fived the other members of my peloton and then gorged myself on orange slices and bagels at the food tent.</p>
<p>The peloton is a great example of the whole being greater than the sum of its parts. None of us individually could have finished in the time that we did but together, we did what none of us alone could.</p>
<p>I like the peloton because it’s an example of a group that benefits each of its members and one where everyone’s incentives are aligned. This isn’t the case with most groups. As coined by Fred Brooks in the <a href="https://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959">Mythical Man-Month</a>, one of the most widely read project management books for software engineers, “adding manpower to a late software project makes it later”.</p>
<p>This is because there’s a lot of time spent getting people caught up to speed in a big project. The more people working on a project, the more coordination needs to take place and the more work needs to be done simply to coordinate the work.</p>
<p>This is all to say that finding the right people and creating the right situation where groups can thrive is not a trivial problem.</p>
<p>It works out in the peloton for a few reasons.</p>
<p>One, the communication overhead is low. Each rider follows simple heuristics, like birds in a flock. For example, every rider stays a few inches behind the rear wheel of the previous rider to maintain formation.</p>
<p>Two, there’s a natural selection that takes place among the riders in a peloton. This means that the slowest rider is still able to lead a non-zero percentage of the time and can still keep up with the fastest member when drafting. Anyone that can’t clear that bar is dropped.</p>
<p>Third, everyone has a common purpose - making it to the finish in the fastest time possible.</p>
<p>I find the peloton a useful mental model when joining groups in other areas of life (eg. work groups, friend groups, romantic entanglements, etc.) Are the conditions ripe for this group to come out ahead? By coming together, can all of us be better off? Life is not a zero sum game. The whole can be bigger than the sum of its parts.</p>
<p>No man is an island. So peloton.</p>Kevin S Linkevin@cloud.thence.ioOver the weekend, I competed in Ski To Sea, a seven leg relay where each leg features a different sport. The relay starts off with downhill skiing off Mt Baker and ends with a kayak race into Bellingham, Washington. I did the road bike leg, a 41 mile course that starts in the middle of Mt. Baker Highway and ends in the city of Everson, Washington.Reflections on Asia2017-01-13T00:00:00+00:002017-01-13T00:00:00+00:00https://kevinslin.com/thoughts/thoughts_on_asia<p>Today is my last day in Japan and first day back in the states after half a month of travelling. I’ve made it a tradition to take a trip at the end of each year as it helps clear my head and helps me prepare for the next one.</p>
<p>Last year, I spent a month split between Nepal and India. This year, it was between Taiwan and Japan.</p>
<p>This was my second time to Taiwan. My first came a few weeks after I graduated from university and decided to do a four month backpacking trip. When I first arrived at Taiwan, I knew very little about what it would be like or anyone who lived there. When I left the country two weeks later, I found it to be one of my favorite places on this earth and left with many happy memories and newly made friends.</p>
<p>So when I arrived at Taoyuan airport this time, I was both excited and wary. To be back in Taiwan was exhilarating but I was afraid I had build up too much expectations of what this trip would be like. Much like how a first kiss is never repeated a second time, I learned from past experience that it’s foolish coming in with preconceptions to things.</p>
<p>I had come with Tuling this time, a former roommate and close friend. We stayed the first night in the apartment of a semi-retired Taiwanese couple. Their daughter had moved out and so they let us sleep in her room which was stuffed to the brim with plush animals and books. That first night we quickly slept, unbelievably wary from the arduous task of sitting all day.</p>
<p>The next day, Jack, the father and owner of the apartment we stayed at, took us to a nearby 7-11 to get a SIM card. 7-11 is a unique phenomenon in Taiwan that has few western equivalents. In America, 7-11 is that small convenient shop that sells iffy slurpees. In Taiwan, 7-11 is equal parts post office, coffee shop, restaurant, phone carrier, ticket counter and hangout spot. They are as common as Starbucks, convenient as online shopping and as necessary as pants. My Taiwanese friends joke that the country would fall apart without 7-11.</p>
<p>That first day, we took the train to Jiufen (九份), an abandoned mining town cut into the face of a cliff facing the East China Sea. Today, it is a popular tourist attraction due to the stunning scenery and multitude of food shops that have replaced minerals as the top export of this place.</p>
<p>Most of the shops have English translations next to foodstuff but this was not always the case. While I’m comfortable speaking Mandarin, I moved out of China when I was four and my literacy remains at about that level. Worse yet, what little I do know is in simplified script but in Taiwan, Mandarin is written in traditional. Simplified Chinese was established in mainland China in the 50’s to promote literacy among the population since simplified typically had fewer strokes for the same character. While adopted in the mainland, Taiwan has stuck firm with traditional which has the consequence of making all the characters look frustratingly familiar but slightly out of reach.</p>
<p>At night, we met with another close friend of mine from university who now lives in Taipei. She took us on a trip to the north side of the island, to the BaYan (八煙) natural hot springs. This was an one hour drive followed by a twenty minute hike at night. We smelled the springs before we saw them, the odor of rotten eggs signalled our eminent arrival.</p>
<p>The hot springs were located in a relatively flat part of the forest where a series of natural depressions have caused different wells of water to pool. A little further ahead was a waterfall where one could go to cool down (I ended up being the only one to do this and was rewarded with a cold for days afterwards). We stayed at the springs for a couple hours before heading back to the city, smelling simply delightful.</p>
<p>The next day, our host family took me and Tuling to a wine museum. There, while sampling some of the local brews, we were interviewed by a local film crew after they found out we were American. They asked us to compare Taiwanese wine with western wine. Though they probably could not have found two less qualified people to comment about the wine, they nevertheless were satisfied with our callow assessment that Taiwanese wine seemed sweeter than the American counterpart. The store owner gave us a box of biscuits as thanks for the interview and we stayed a bit longer to try out more samples and talk to the bartender. He was interested in what parts of America we hailed from and our lives there.</p>
<p>Something that I’ve noticed about Taiwan is that many people here have an adoration for America, kind of like the ardour a Seattleite might feel about the Seahawks. I struck up a conversation with an old man during a hike last time I was here. When I told him I was from America, he proudly told me that his son was studying in Chicago and sung praise about the western lifestyle. The old man had never been to America but he was convinced that it was better in every way.</p>
<p>Having lived in the former and travelled around the latter, I say its not that black and white. There are a lot of things that I like better in Taiwan, especially the way they’ve managed to mix democracy with capitalism and social cohesion.</p>
<p>One example of this can be seen in the life of the elderly. In America, it’s common for older folk to be relegated to homes and kept out of sight. In Taiwan, I see them doing tai chi in the city center, strolling through the parks, shopping with their grandchildren and in general, being a fixture of everyday life.</p>
<p>Older people play an active part in the everyday. In society, they are given room and respect and in the family, it is common and even expected for children to house their parents and take care of them as they get older. Filial duty is a strong concept in Chinese culture and Confucianism teaches us to respect our family and our community. This focus on community is prevalent in Taiwan and throughout the east.</p>
<p>Having left China when I was four and living in America for most of my life, I find this culture both endearing and stifling.</p>
<p>The west celebrates rugged individualism and pulling oneself up by one’s own bootstraps. I might not own a single pair of boots but I am a fan of the idea but am concerned that we take it too far. In modern times, the political discussion has turned from what is best for the country to the “us vs them” mentality. There is growing disparity between the rich and the poor, compromise is a bad word and our healthcare is the most dysfunctional of any first world country (despite us spending more on it than any other nation).</p>
<p>Basically, there’s room for improvement (that was a strong message in our most recent election).</p>
<p>After spending a week in Taiwan, we flew to Japan early in the morning. The flight was mercifully short (3h) compared to the one that brought us to Taiwan (14h) and we arrived in Tokyo before noon.</p>
<p>Tokyo is huge - a city over 13 million people living in a vast metropolis that seems to go on forever. Despite the great population density, the sidewalks are as clean as the counters in an apple store. I always assumed that dirt and litter came part and parcel with living in a dense area but Tokyo is proof that this doesn’t have to be. What’s interesting is that this cleanliness is achieved despite the city having an aversion to public thrash cans, of which I found practically none outside of metro stations. I lived in cities in Canada that had drives to clean the streets from time to time. Despite putting out thrash cans and recycling boxes everywhere, the city would always soon revert to its littered self. In Tokyo, everything was neat and tidy despite the great inconvenience of throwing out thrash. I suppose it shows that if the underlying values are there, then the details will take care of themselves (conversely, if the fundamentals aren’t sound, than no bandage solution will fix the problem).</p>
<p>When boarding the subways, I observed the famous Japanese efficiencies as I never waited more than a few minutes for a train to come. The subways have an overhead panel that shows the arrival time of the next train to the minute and in my experience, I have never seen it be off.</p>
<p>Being on the metro was a little unnerving because of the silence. Despite the trains being packed, I could hear myself breathe. People talk in hushed whispers and there are signs everywhere telling you to set your phone to silence and to not talk on the phone when in the metro.</p>
<p>Everywhere we went, people were very polite. The phrase “arigato gozaimasu” started being filtered out in my head as I’ve heard it so much. It means “thank you very much” and is doled out in heavy servings by all the Japanese we talked to.</p>
<p>In Japan, we decided to follow the popular tourist migration route that goes from Tokyo to Osaka and ends in Kyoto. We travelled between cities using the “Shinkansen” (bullet train), my new favorite form of transport, first pioneered in Japan, which travels up to 200 mph (initially I thought “Shinkansen” was a train line and marvelled how a single line seemed to run everywhere).</p>
<p>Something to note about traveling in Japan, unlike most Asian countries, is that it’s expensive. Food and transportation prices are comparable to that of America. I converted $400 USD into yen at the beginning of the trip and six days later had almost nothing left.</p>
<p>I used the remainder of my money buying food stuff at the airport on the way back. Theres an unwritten rule somewhere that one must buy strawberry flavored kitkats when travelling to Japan. To be safe, I also got wasabi, cheese cake and some sort of chocolate almond mix flavored kitkats.</p>
<p>After three movies, intermittent naps and two hours of an audio book, I caught a glimpse of the space needle from the window seat of the hump of the Boeing 747 which I had been prisoner of the past 13 hours. My heart skipped a beat and I felt a warm fuzzy sensation in my chest. Seattle feels like home to me now and I realized that I missed it.</p>
<p>Sometimes we must journey outward to appreciate what we have. One of my goals for this year is to show greater gratitude for what I have. I think this comes from my eastern heritage as all my other goals involve being able to have more. This is mostly in the form of time to pursue the interests that I care about - like online idea pollution (blogging). How successful I will be at this remains to be seen.</p>Kevin S Linkevin@cloud.thence.ioToday is my last day in Japan and first day back in the states after half a month of travelling. I’ve made it a tradition to take a trip at the end of each year as it helps clear my head and helps me prepare for the next one.Some Thoughts on VR2016-08-31T00:00:00+00:002016-08-31T00:00:00+00:00https://kevinslin.com/thoughts/vr<p>VR champagne. So sweet that I can taste it. At least that’s what I tell my colleague as we make a bet about the future of VR.</p>
<p>At stake is a bottle of champagne of the winner’s choosing, in 5 years time. My prediction is that VR would be mainstream within this timeframe, where “mainstream” meant 10% of the American population using a VR device 1h, on average, every day. My colleague is not a believer.</p>
<p>VR (virtual reality), in case you haven’t heard or gone through the initial (motion sickness inducing) demos, is a computer generated 3D environment that a person can interact with. The primary interface today is a head mounted display. The target market at launch has been gaming though there’s been much buzz about its potential impact in verticals like work and education.</p>
<p>Despite the potential, VR today is a niche industry reserved mostly for gamers with CUDA cores to burn or people willing to strap cardboard to their faces. VR tomorrow is anyone’s champagne.</p>
<p>I think it comes down to a communication problem.</p>
<p>This is an universal theme for almost any system involving people. It’s a common factor in everything from breakups to software development. I’ve been in design meetings where six people end up walking out thinking that we’ve agreed on six different outcomes.</p>
<p>People have been around for two hundred thousand years. You think that’d be long enough figure this communications thing out but turns out it’s not a trivial problem.</p>
<p>There are approximately 100 billion neurons in the human brain. Each neuron can fire between 5-50 times a second with neighboring neurons which can number in the thousands. Hand waving a little like Queen Elizabeth in a royal parade, this means that hypothetically, the brain can produce over 100 trillion bits of information a second (that’s 11.369 terabytes/s). Consider also that this information can be combined and mixed with a lifetime of memories and it becomes a wonder that we can understand each other at all.</p>
<p>We do have some help in the matter today. In addition to words, we have pictures, gifs, emojis, movies, vines, tweets, snaps, doodles, and even face swaps. While these different techniques have increased the information density of transmitted content, its still like trying to stream 4K television through a dial up connection. And at the end of the day, no matter how high fidelity the signal, the content is still subject to interpretation on the receiving end.</p>
<p>This is true every time we receive a message. Don’t (solely) blame the personalization algorithms of big software companies for filtering your content because our minds are the masters of this art. Our personal biases color every moment we experience, every message we receive. There’s probably no better example of this right now then the current American presidential elections. Here you have two parties that cannot receive any viewpoints counter to their own.</p>
<p>I think VR can help us not just with increasing the information density of transmitted messages but also with the way we receive messages - it can help us see from another point of view.</p>
<p>They say that you shouldn’t judge a man (or woman) unless you walk 1 mile in their shoes. With VR, we can put on a headset and do just that (just watch the cord). Even in today’s first generation low resolution models, VR can create scenes so real that grown men will freeze when asked to jump down a single step of stairs. The virtual chasm projected at their eyeballs becomes as real as the train that sent crowds of people running from their seats at the premier of the first silent black and white movie.</p>
<p>Instead of oncoming trains, VR could paint disparate viewpoints. Instead of sending a message in isolation and trying to justify with cold facts and numbers, one could build VR content that would make the receiver understand the message on a visceral level. A massacre in the Congo on paper is just a statistic quickly forgotten - a VR simulation that puts you into the middle of the carnage and the countless lives wrecked in the process is much harder to ignore.</p>
<p>Today, there are existing tools that help you see from other points of view. Sites on the internet will show you images in grayscale to simulate color blindness. Others simulate dyslexia by jumbling characters around. What VR could do is generalize this by completely immersing the user in a near perfect simulation. And while we will probably never fully comprehend the mind of another person(or even our own for that matter), with VR we can get so close that we’ll be “virtually there”.</p>
<p>We live in a world rife with technological marvels. While the ability to query all of human knowledge in one’s pocket has undeniably done much good, the lure of an endless stream of distractions has also made it much easier to become isolated from the world at large. There’s a very real fear that the onset of VR will hasten this trend and lead people to abandon the real world altogether for a virtual counterfeit.</p>
<p>On the flip side, there’s a real potential that VR could help increase our humanity. It could help us craft scenes that would make it easier to understand other people. And in the end, isn’t that something we all want? To be fully understood by another and for them, with that understanding, to accept and perhaps even love us for who we are. I’ll toast to that - once I get my champagne.</p>Kevin S Linkevin@cloud.thence.ioVR champagne. So sweet that I can taste it. At least that’s what I tell my colleague as we make a bet about the future of VR.What Losing My Luggage in Nepal Taught Me About Decision Making2016-01-03T00:00:00+00:002016-01-03T00:00:00+00:00https://kevinslin.com/thoughts/blog/what_losing_my_luggage_in_nepal_taught_me_about_decision_making<p>You know that feeling you get when you’re in a Nepalese airport and realize
that all your luggage is gone?</p>
<p>If so, my sincere condolences. Otherwise, let me fill you in.</p>
<p>A few weeks ago, I was in Nepal leaving the Kathmandu airport with only
the clothes on my back. This was not according to plan. The plan was to
complete the Annapurna circuit, a 200km trek across the Himalayas, and to
do that I was going to need a lot more equipment.</p>
<p>Before I continue, first a brief recap of events that led to this moment.
I arrived in Kathmandu via a flight from China where I was visiting
friends and family. I left China via the Beijing Capital International
Airport. When I left, I carried a 65L Osprey backpack filled to the brim
with new REI trekking equipment which I had purchased only the week prior.</p>
<p>I didn’t know this at the time, but my 09:15 flight out of the country had
been changed to a flight leaving 1h earlier. Around midnight the previous
night, an email was sent by travelocity informing me of this change. This
email was swallowed by the great firewall of China which rendered Gmail
unusable in the country. And so when I went to the airport that morning,
I didn’t understand why the China Eastern Airline clerk couldn’t find my
name on the flight. After some digging on travelocity, I found that my new
flight left at 08:30. Ticketing closed 40min before departure and the time
was 08:04. I would not be able to get on this rescheduled flight.</p>
<p>I made a quick dash to the ticketing counter and was able to rebook my
original flight leaving at 09:15. Since my flight to Kathmandu had
a layover in Kunming China and because I had “changed” the first leg of my
flight, the airline clerk explained that I would need to retrieve my
checked bag at Kunming and check it in again to my final decision.</p>
<p>This wasn’t a problem in in theory since I had a 2h layover in Kunming. In
practice, the flight was over an hour late in landing. I knew that I would
not be able to get my ticket if I waited for my luggage so I made the
decision to run to the arrival gate, foregoing my luggage. I reached the
counter exactly 40minutes before departure and the airline clerk grumpily
printed my ticket. He told me the plane would finish boarding in 10
minutes and that if I got on the flight, I would not be able to retrieve
my luggage. I asked the clerk if there was a later flight but was told the
next flight would be leaving three days later. My choices were to leave
now and have the airline ship my luggage on Monday or stay in Kunming for
three days and leave Monday with my luggage. I choose the former.</p>
<p>And so I arrived in Kathmandu feeling naked as I looked around at the
people around me with carts of luggage and backpacks. I met up with my
friend from college who was doing the trek with me and explained the
situation. Neither one of us were pleased but we made the best of the
situation, using the extra days to travel around Kathmandu and make day
trips (on the rooftops of overcrowded buses) to surrounding cities.</p>
<p>I told my friend that I had already mentally accepted my pack as lost. In
fact, I made a list of all the things I would need if/when my luggage
didn’t arrive. I also mentally kicked myself for not only losing a bag
with brand new trekking equipment but also for leaving ~$600 USD in the
bag. All in all, it was around $1500 of money and equipment.</p>
<p>On the days leading up to the date when the luggage was supposed to arrive, my
friend took every opportunity to remind me that it probably wouldn’t come.
While I was the one who originally said that I didn’t expect to see my
luggage, the constant reminder struck a nerve and led to our first
dispute.</p>
<p>I told my friend to back off. That his constant bringing up of the issue
wasn’t helping. That as much as I tried to brush it off, a part of me was
deeply upset at losing my pack. To this point, my friend replied replied
with his own laundry list of complaints. He was upset as well. The <em>stunt</em>
with my luggage was costing us valuable trekking time when we were already
under a tight timeline to begin with. My friend also criticized my
decision making. That I shouldn’t have left my bag behind like that. He
said that he wasn’t surprised that something like this happened.</p>
<p>We proceeded to walk in silence for some time, processing what the other
had just said. The criticism about my decision making struck another nerve</p>
<ul>
<li>this was not the first time this point had been brought up.</li>
</ul>
<p>It’s true that at one point in my life, I developed a (well earned)
reputation for not thinking things through. I made impulsive decisions,
climbed buildings and went on impromptu trips with little forethought. One
example is a 200 mile bike ride from Houston to Austin over a long weekend
in college. The trip was made with the same friend that I was currently in
Nepal with. We made the trip without food or water and only one road bike
between the two of us (and one regular bike) which we took turns riding.
We brought one bike pump but never bothered to check if the pump valve
matched the bikes (it didn’t). We didn’t know where we would stay the
first night and ended up in the middle of nowhere lying in the back of
someone’s suburban. We didn’t realize how painful it would be to ride
a bike 200 miles over bad roads and no bike shorts and I had trouble
sitting for the week afterwards. It was a miracle that we made it out of
Houston.</p>
<p>I used to justify my behavior by claiming to be a big picture sort of
person. I didn’t want to be caught in the details. As I mention in a <a href="http://kevinslin.com/blog/2013/12/10/noticing-the-trees-among-the-forest.html">previous blog
entry</a>,
I believed details are decoys to a meaningful life. I believed that as
long as I had the right goals, I would be able to work out the details
along the way.</p>
<p>A wise man is someone who is able to change their believes given new
evidence - while I can’t claim to be wise, I did change my mind about
details. My major focus for the past two years has been to do a better job
focusing on the details and making better decisions. But as this luggage
situation showed - I still had a long way to go.</p>
<p>I could argue that this wasn’t my fault. That the great firewall blocked
the notification of my rescheduled flight. That the flight that I ended up
on was 2h late. That there were no other flights going to Kathmandu for
three days. That I left my money in the bag because I didn’t have time to
take it out in the heat of the moment. But I knew I wasn’t being honest
with myself. I’ve flown enough to know that almost all my flights get
rescheduled. I could have booked tickets from China using a different
email account. I shouldn’t have kept money in a checked in bag in the
first place. I could have done much more.</p>
<p>So instead of arguing, I conceded the point to my friend and we settled
our dispute. My bag never showed up. I ended up buying all my gear at
a local trekking shop. Since I was getting all my equipment from one
place, I was able to bargain for a discount. One consolation was that
trekking equipment in Nepal costs a fraction of what it did in America,
and I purchased everything (pack, sleeping bag, shoes, clothing, etc) for
under $200.</p>
<p>As for the actual trek, we managed to complete it in its entirely within
the two weeks. We even had time for a 3 day side trip to Tilicho lake
(claimed by Nepal to be both the highest and biggest lake but at 4919m and
4.8 square km, it was neither). Life was simple within the mountains</p>
<ul>
<li>every day our schedule consisted of eating, trekking, journaling and
sleeping. The simplicity of our daily routine allowed us to shift focus to
other parts of our lives.</li>
</ul>
<p>For me, it was a reflection of my decision making process. For years now,
I’ve been keeping a journal of all the mistakes that I’ve made (of which
there are many). I’ve drawn many lessons from these journals and used the
trek to distill these learnings into something more concise I could use
for future decisions. Through this process, I came up with the following
checklist.</p>
<ol>
<li>Important? What is the impact? Is there a higher priority problem that needs to be worked on? Does doing this align with my goals and values? Are we solving the real underlying problem or just addressing symptoms?</li>
<li>Biases? Emotional bias? Sunk cost bias? Think like Spock.</li>
<li>Assumptions? What do I believe to be true and are these truths valid?</li>
<li>Analyze. What are the details of the current situation? Put on Sherlock hat and really understand the situation and precursors that led to it.</li>
<li>Contextualize. What is “normal” for this situation? Is this normal? Have I encountered something in the past that was like this?</li>
<li>Plan. Think of next moves. Explore different branches. Draw out pros and cons of following each branch. What are the effects and consequences of these actions?</li>
<li>Prepare. What actions can I take now? Isolate and tackle long poll tasks.</li>
<li>Contingencies. Plan for things not going according to plan. Do plans take into account edge cases? What about the worst case scenario? Have backup plans.</li>
<li>Salvage. If a move doesn’t go your way, is there still something positive you can get out of it?</li>
<li>Improve. If a move does go your way, does it make sense to settle for the original goal or should I set sights higher?</li>
<li>Clean up. Are there loose threads that I need to be resolved? Outstanding issues? Reap what you sow.</li>
<li>Verify. Make sure problem really has been solved.</li>
<li>Review. Are there any lessons I can take away from this?</li>
</ol>
<p>This checklist sums up all the lessons I have learned (so far) about problem solving. While I’ve always been aware of these lessons, it is still easy for me to skip over specific steps (2 and 3 are my Achilles heel). By explicitly listing out my decision making process and consulting this list when faced with a problem, I plan to make better choices. After all, it has been proven that the humble <a href="http://amzn.to/1SBFstu">checklist</a> is still one of our best tools when it comes to mastering complicated processes.</p>
<p>As a bonus, when we came back to Kathmandu, I received notice that my
luggage arrived. The bad news is that inside my pack was a bag of fruit my
cousin had gifted me in Beijing before I left. After two weeks of sitting
around and being squeezed by other pieces of luggage , it had gone bad and
juices oozed from the once fruit-like lumps onto the other articles inside
my pack. I was afraid that the airport would book me for unleashing some
toxin inside the premises. Luckily this scenario did not play out and
instead I walked away from the experience with both my backpack and
a better understanding of my decision making process. I hope it will also
be helpful to readers but if you only take away one lesson, remember this</p>
<ul>
<li>never check in fruit with your luggage (it’s a rotten idea).</li>
</ul>Kevin S Linkevin@cloud.thence.ioYou know that feeling you get when you’re in a Nepalese airport and realize that all your luggage is gone?Why I Choose Vim2015-09-25T00:00:00+00:002015-09-25T00:00:00+00:00https://kevinslin.com/thoughts/why_I_choose_vim<p>Helen of Troy might have had a face that launched a thousand ships but the
Trojan war pales in comparison to the <a href="https://en.wikipedia.org/wiki/Editor_war">flame
wars</a> developer wage today over
their favorite editor.</p>
<p>As a developer, a lot of time is spent inside an editor writing code.
Editors have changed much in the past fifty years, evolving from simple
<a href="https://en.wikipedia.org/wiki/Line_editor">line editors</a> like qed to rich
<a href="https://en.wikipedia.org/wiki/Integrated_development_environment">Integrated development
environments</a>
(IDE). IDE’s bear as much resemblance to simple text editors as iPhone’s do
to flip phones.</p>
<p>The accumulation of features was spurred not just by better technology but
also by the increasing complexity of both programming languages and
programs written in said languages. The Java programming language is so
verbose that writing it outside an IDE (or at least with auto completion)
classifies as masochism of the first order. And as far as programs go,
whereas the original version of Unix(1.0) was written with ~15,000 lines
of code (LOC), the windows vista operating system is over 50 million LOC.</p>
<p>Today’s technologies are many orders of magnitude more complex than what
came before - the editor is where developers go to battle this complexity.
Like a Nascar race car, every setting needs to be tuned to a fine hairs
precision. Spending time in an editor is like spending time
with a loved one - it requires commitment and deep understanding of the
other to make it work.</p>
<p>This writer has chosen <a href="https://github.com/vim/vim">Vim</a> as his editor.</p>
<p>Vim was created by Bram Moolenaar during the start of the 90’s as a major
upgrade to the vi editor (Vim stands for vi improved). Vim is a simple
line based editor that is available by default on all Unix based operating
systems and works on all platforms. It is a modal text editor infamous for
its steep learning curve.</p>
<p>In typical text editors, one can just start typing and have key presses
map directly to the characters appearing on screen. In Vim, one needs to
first enter “insert mode” before this happens. Vim by default starts in
normal mode where each key press maps instead to a command. There is
a third mode known as visual mode which applies commands to highlighted
blocks of text.</p>
<p>Vim is extremely powerful once you get your head around its modal editing
modes (and endless key mappings) but the real icing of this metaphorical
cake is the inexhaustible number of ways it can be extended. This is done
via Vimscript, a custom scripting language that ships with Vim which gives
developers control of every aspect of the editor, from changing the
background color to custom commands that activate only if the current word
under the cursor matches a specific regex. The way I like to think of
Vimscript is as a meta programming language to the coding process.</p>
<p>Meta programming is writing a program that is able to treat other programs
as data, giving the developer the power to change the very laws of said
programming language. This is a very powerful feature (just ask a lisper);
in the real world, you can think of this as being able to change the laws
of physics at whim. Want to explore other galaxies? Simply enable the
“faster-than-speed-of-light” travel option and it’s done. Popular web
frameworks like rails and django make extensive use of meta programming to
radically simplify the web development process.</p>
<p>Coming back to Vimscript, if you think of an editor as the world in which
code is crafted, Vimscript is the means to change the laws of said world.
And that is precisely the case today as Vim plugins which enable fuzzy
file search, multiple cursors and intelligent auto completion bring this
over two decade old editor on par with the newest IDEs of the day.</p>
<p>If we step out one abstraction level and see editors like Vim as a tool to
extend human capabilities, Vimscript becomes a means of continual tool
refinement. You can create an incredibly powerful feedback loop when you
can continuously improve the tools that you use and in fact, use the very
tool you’re refining to make the next version of said tool.</p>
<p>And at the end of the day, building better tools is not just a fun past
Time but something essential to the continual evolution of the human race.
Humans are completely dependent on the tools we make. This was true in the
past when we worked in groups to take down large predators with spears and
knifes and it’s true today where every aspect of our lives, from banks to
hospitals, is managed by the tools we have build.</p>
<p>Better tools are the rising tide that raises all ships - by ship I mean
the collective capabilities of the human race. The popular proverb “a bad
workman always blames his tools” is true to an extent but I would argue
that a workman who doesn’t invest in their tooling might deserve the
blame. The world today is moving in a trend of ever increasing complexity.
Gone are the days of using electrical tape and solder to fix household
electronics; these devices have evolved from simple one way radios
to smart phones that require specialized screws just to pop open the
casing and electron microscopes to see the circuits. The only way we can
reign in this complexity is to do what we’ve been doing since the dawn of
our species - create better tools. My tool of choice in this eternal
struggle is Vim.</p>Kevin S Linkevin@cloud.thence.ioHelen of Troy might have had a face that launched a thousand ships but the Trojan war pales in comparison to the flame wars developer wage today over their favorite editor.