Derek's Personal Website on Derek's Personal Websitehttps://derekchiang.com/
Recent content in Derek's Personal Website on Derek's Personal WebsiteHugo -- gohugo.ioen-usSat, 21 Oct 2017 00:00:00 +0000Reusable and type-safe options for Go APIshttps://derekchiang.com/posts/reusable-and-type-safe-options-for-go-apis/
Sat, 21 Oct 2017 00:00:00 +0000https://derekchiang.com/posts/reusable-and-type-safe-options-for-go-apis/
<p><a href="https://frasco.io/reusable-and-type-safe-options-for-go-apis-6b51d431df5d"><em>Japanese translation</em></a></p>
<hr />
<h2 id="background">Background</h2>
<p>In this blog post, I would like to describe an extension to the popular &ldquo;functional options&rdquo; pattern that has been described by people like <a href="https://commandcenter.blogspot.com/2014/01/self-referential-functions-and-design.html">Rob Pike</a> and <a href="https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis">Dave Cheney</a>. I recommend reading one of these articles if you are unfamiliar with this pattern, as it&rsquo;s very useful in practice.</p>
<h2 id="the-problem">The problem</h2>
<p>To see the limitations of the pattern, consider the <a href="https://godoc.org/github.com/coreos/etcd/clientv3">etcd v3 client</a>. Specifically, let&rsquo;s look at the <a href="https://godoc.org/github.com/coreos/etcd/clientv3#KV"><code>KV</code></a> interface which exposes APIs for putting and getting key-value pairs. Here&rsquo;s the <code>Get</code> API for instance:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#a6e22e">Get</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">key</span> <span style="color:#66d9ef">string</span>, <span style="color:#a6e22e">opts</span> <span style="color:#f92672">...</span><span style="color:#a6e22e">OpOption</span>) (<span style="color:#f92672">*</span><span style="color:#a6e22e">GetResponse</span>, <span style="color:#66d9ef">error</span>)</code></pre></div>
<p>Here, <code>opts</code> is a list of functional options. To use this API, you write code like this:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#a6e22e">resp</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">kvc</span>.<span style="color:#a6e22e">Get</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#e6db74">&#34;sample_key&#34;</span>, <span style="color:#a6e22e">WithPrefix</span>(), <span style="color:#a6e22e">WithRev</span>(<span style="color:#a6e22e">sample_rev</span>))</code></pre></div>
<p>Here we are passing two options <code>WithPrefix</code> and <code>WithRev</code> to the API. Note how both options are functions and therefore can take arbitrary arguments themselves.</p>
<p>The problem with this approach though is that you can pass any option with the type <code>OpOption</code> to the API, whether the option actually makes sense or not. For instance, we could pass <a href="https://godoc.org/github.com/coreos/etcd/clientv3#WithLease"><code>WithLease</code></a> to <code>Get</code>, even though the option only applies to <code>Put</code>, as described in the documentation.</p>
<p>Therefore, we say that the options are not &ldquo;type safe&rdquo;, in that passing the wrong options is an error detectable only at run time, not compile time.</p>
<h2 id="the-wrong-solution">The wrong solution</h2>
<p>It&rsquo;s tempting to solve this problem by defining separate option types for different APIs. For instance, we could have a <code>GetOption</code> type which only <code>Get</code> accepts, and a <code>PutOption</code> type which only <code>Put</code> accepts, so on and so forth.</p>
<p>The problem with this approach is that different APIs might take the same options. Since Golang does not support <a href="https://en.wikipedia.org/wiki/Function_overloading">function overloading</a>, you&rsquo;d have to define separate instances of the same option, one for each API, like this:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">WithPrefixForGet</span>() <span style="color:#a6e22e">GetOption</span> { <span style="color:#f92672">...</span> }
<span style="color:#66d9ef">func</span> <span style="color:#a6e22e">WithPrefixForDelete</span>() <span style="color:#a6e22e">DeleteOption</span> { <span style="color:#f92672">...</span> }
<span style="color:#66d9ef">func</span> <span style="color:#a6e22e">WithPrefixForWatch</span>() <span style="color:#a6e22e">WatchOption</span> { <span style="color:#f92672">...</span> }</code></pre></div>
<p>Which is clearly less than ideal for both the developer and the user. You could also define the options in different packages, one for each API, but that&rsquo;s even more cumbersome.</p>
<h2 id="the-solution">The solution</h2>
<p><a href="https://gist.github.com/derekchiang/c9709eace4b588353ec9df32d845ac9c">I&rsquo;ve put up an example here</a>. For simplicity we only support two APIs (<code>Get</code> and <code>Delete</code>) and two options (<code>WithPrefix</code> and <code>WithRev</code>).</p>
<p>We start by defining the APIs:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Get</span>(<span style="color:#a6e22e">key</span> <span style="color:#66d9ef">string</span>, <span style="color:#a6e22e">ops</span> <span style="color:#f92672">...</span><span style="color:#a6e22e">GetOption</span>)
<span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Delete</span>(<span style="color:#a6e22e">key</span> <span style="color:#66d9ef">string</span>, <span style="color:#a6e22e">ops</span> <span style="color:#f92672">...</span><span style="color:#a6e22e">DeleteOption</span>)</code></pre></div>
<p>Then, we define one interface per API:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#66d9ef">type</span> <span style="color:#a6e22e">GetOption</span> <span style="color:#66d9ef">interface</span> {
<span style="color:#a6e22e">SetGetOption</span>(<span style="color:#f92672">*</span><span style="color:#a6e22e">getOptions</span>)
}
<span style="color:#66d9ef">type</span> <span style="color:#a6e22e">DeleteOption</span> <span style="color:#66d9ef">interface</span> {
<span style="color:#a6e22e">SetDeleteOption</span>(<span style="color:#f92672">*</span><span style="color:#a6e22e">deleteOptions</span>)
}</code></pre></div>
<p>Finally, we define one function per option.</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#75715e">// WithPrefix can be used with Get and Delete
</span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">WithPrefix</span>() <span style="color:#66d9ef">interface</span> {
<span style="color:#a6e22e">GetOption</span>
<span style="color:#a6e22e">DeleteOption</span>
} {
<span style="color:#75715e">// See the link above for the implementation
</span><span style="color:#75715e"></span>}
<span style="color:#75715e">// WithRev can be used only with Get
</span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">WithRev</span>(<span style="color:#a6e22e">rev</span> <span style="color:#66d9ef">int64</span>) <span style="color:#66d9ef">interface</span> {
<span style="color:#a6e22e">GetOption</span>
} {
<span style="color:#75715e">// See the link above for the implementation
</span><span style="color:#75715e"></span>}</code></pre></div>
<p>One interesting thing to note here is that the functions return <em>anonymous interfaces</em> that embed the <code>*Option</code> interfaces. This has the benefit that the code becomes self-documenting (and thus <code>godoc</code>-friendly), in that you can look at the type signature of an option and instantly know which APIs it can be used with.</p>
<p>Now let&rsquo;s see if the options are in fact reusable and type-safe. Since <code>WithPrefix()</code> implements both <code>GetOption</code> and <code>DeleteOption</code>, the following code works with no issues:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#a6e22e">Get</span>(<span style="color:#e6db74">&#34;sample_key&#34;</span>, <span style="color:#a6e22e">WithPrefix</span>())
<span style="color:#a6e22e">Delete</span>(<span style="color:#e6db74">&#34;sample_key&#34;</span>, <span style="color:#a6e22e">WithPrefix</span>())</code></pre></div>
<p>In contrast, if we use <code>WithRev</code> with <code>Delete</code>, we get a compile error:</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#a6e22e">Delete</span>(<span style="color:#e6db74">&#34;sample_key&#34;</span>, <span style="color:#a6e22e">WithRev</span>(<span style="color:#ae81ff">1</span>))</code></pre></div>
<pre><code>./main.go:88:30: cannot use WithRev(1) (type interface { SetGetOption(*getOptions) }) as type DeleteOption in argument to Delete:
interface { SetGetOption(*getOptions) } does not implement DeleteOption (missing SetDeleteOption method)
</code></pre>
<p>Which tells us that <code>WithRev</code> is not a <code>DeleteOption</code>!</p>
<h2 id="summary">Summary</h2>
<p>To summarize, I have described a pattern for defining options that are:</p>
<ul>
<li>Reusable, in that the same option can be used in multiple APIs.</li>
<li>Type-safe, in that we get a compile-time error if we pass the wrong option.</li>
</ul>
<hr />
<p><em>I would like to thank <a href="https://twitter.com/jdoliner">JD</a> for discussing this pattern with me.</em></p>
<p><em>Thanks to @ar1819 and @natefinch from <a href="https://www.reddit.com/r/golang">r/golang</a> for suggesting the use of anonymous interfaces.</em></p>
<p><em>Thanks to Ren Sakamoto for translating the article into Japanese.</em></p>