tag:blogger.com,1999:blog-66289850225318661932017-08-18T03:07:34.673+01:00The OldWood Thingblog = reinterpret_cast&lt;&gt;(life);Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.comBlogger262125tag:blogger.com,1999:blog-6628985022531866193.post-91162284781017310252017-06-15T19:34:00.001+01:002017-06-15T19:34:02.630+01:00Refactoring – Before or After?<p><font face="Trebuchet MS">I recently worked on a codebase where I had a new feature to implement but found myself struggling to understand the existing structure. Despite paring a considerable amount I realised that without other people to easily guide me I still got lost trying to find where I needed to make the change. I felt like I was walking through a familiar wood but the exact route eluded me without my usual guides.</font></p> <p><font face="Trebuchet MS">I reverted the changes I had made and proposed that now might be a good point to do a little reorganisation. The response was met with a brief and light-hearted game of “Ken Beck Quote Tennis” - some suggested we do the refactoring before the feature whilst others preferred after. I felt there was a somewhat superficial conflict here that I hadn’t really noticed before and wondered what the drivers might be to taking one approach over the other.</font></p> <p><strong><font face="Trebuchet MS">Refactor After</font></strong></p> <p><font face="Trebuchet MS">If you’re into Test Driven Development (TDD) then you’ll have the mantra “</font><a href="http://www.jamesshore.com/Blog/Red-Green-Refactor.html"><font face="Trebuchet MS">Red, Green, Refactor</font></a><font face="Trebuchet MS">” firmly lodged in your psyche. When practicing TDD you first write the test, then make it pass, and finally finish up by refactoring the code to remove duplication or otherwise simplify it. Ken Beck’s </font><a href="https://www.amazon.co.uk/d/cka/Test-Driven-Development-Addison-Wesley-Signature-Kent-Beck/0321146530"><font face="Trebuchet MS">Test Driven Development: By Example</font></a><font face="Trebuchet MS"> is probably the de facto read for adopting this practice.</font></p> <p><font face="Trebuchet MS">The approach here can be seen as one where the refactoring comes after you have the functionality working. From a value perspective most of it comes from having the functionality itself – the refactoring step is an investment in the codebase to allow future value to be added more easily later.</font></p> <p><font face="Trebuchet MS">Just after adding a feature is the point where you’ve probably learned the most about the problem at hand and so ensuring the design best represents your current understanding is a worthwhile aid to future comprehension.</font></p> <p><strong><font face="Trebuchet MS">Refactor Before</font></strong></p> <p><font face="Trebuchet MS">Another saying from Kent Beck that I’m particularly fond of is “</font><a href="https://twitter.com/KentBeck/status/250733358307500032"><font face="Trebuchet MS">make the change easy, then make the easy change</font></a><font face="Trebuchet MS">” [1]. Here he is alluding to a dose of refactoring up-front to mould the codebase into a shape that is more amenable to allowing you to add the feature you really want.</font></p> <p><font face="Trebuchet MS">At this point we are not adding anything new but are leaning on all the existing tests, and maybe improving them too, to ensure that we make no functional changes. The value here is about reducing the risk of the new feature by showing that the codebase can safely evolve towards supporting it. More importantly It also gives the earliest visibility to others about the new direction the code will take [2].</font></p> <p><font face="Trebuchet MS">We know the least amount about what it will take to implement the new feature at this point but we also have a working product that we can leverage to see how it’s likely to be impacted.</font></p> <p><strong><font face="Trebuchet MS">Refactor Before, During &amp; After</font></strong></p> <p><font face="Trebuchet MS">Taken at face value it might appear to be contradictory about when the best time to refactor is. Of course this is really a straw man argument as the best time is in fact “all the time” – we should <em>continually</em> keep the code in good shape [3].</font></p> <p><font face="Trebuchet MS">That said the act of refactoring should not occur within a vacuum, it should be driven by a need to make a more valuable change. If the code never needed to change we wouldn’t be doing it in the first place and this should be borne in mind when working on a large codebase where there might be a temptation to refactor purely for the sake of it. Seeing stories or tasks go on the backlog which solely amount to a refactoring are a smell and should be heavily scrutinised.</font></p> <p><strong><font face="Trebuchet MS">Emergent Design</font></strong></p> <p><font face="Trebuchet MS">That said, there are no absolutes and whilst I would view any isolated refactoring task with suspicion, that is effectively what I was proposing back at the beginning of this post. One of the side-effects of emergent design is that you can get yourself into quite a state before a cohesive design finally emerges.</font></p> <p><font face="Trebuchet MS">Whilst on paper we had a number of potential designs all vying for a place in the architecture we had gone with the simplest possible thing for as long as possible in the hope that more complex features would arrive on the backlog and we would then have the forces we needed to evaluate one design over another.</font></p> <p><font face="Trebuchet MS">Hence the refactoring decision became one between digging ourselves into an even deeper hole first, and then refactoring heavily once we had made the functional change, or doing some up-front preparation to solidify some of the emerging concepts first. There is the potential for waste if you go too far down the up-front route but if you’ve been watching how the design and feature list have been emerging over time it’s likely you already know where you are heading when the time comes to put the design into action.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] I tend to elide the warning from the original quote about the first part potentially being hard when saying it out loud because the audience is usually well aware of that :o).</font></p> <p><font face="Trebuchet MS">[2] See “<a href="http://chrisoldwood.blogspot.co.uk/2017/01/the-cost-of-long-lived-feature-branches.html">The Cost of Long-Lived Feature Branches</a>” for a cautionary tale about storing up changes.</font></p> <p><font face="Trebuchet MS">[3] See “</font><a href="http://chrisoldwood.blogspot.co.uk/2014/11/relentless-refactoring.html"><font face="Trebuchet MS">Relentless Refactoring</font></a><font face="Trebuchet MS">” for the changes in attitude towards this practice.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-15450500067113874522017-06-12T19:44:00.001+01:002017-06-12T19:44:45.429+01:00Stack Overflow With Custom JsonConverter<p><font face="Trebuchet MS">[<em>There is a <a href="https://gist.github.com/chrisoldwood/b604d69543a5fe5896a94409058c7a95">Gist on GitHub</a> that contains a minimal working example and summary of this post.</em>]</font></p> <p><font face="Trebuchet MS">We recently needed to change our data model so that what was originally a list of one type, became a list of objects of different types with a common base, i.e. our JSON deserialization now needed to deal with polymorphic types.</font></p> <p><font face="Trebuchet MS">Naturally we googled the problem to see what support, if any, <a href="http://www.newtonsoft.com/json">Newtonsoft’s JSON.Net</a> had. Although it has some built-in support, like many built-in solutions it stores fully qualified type names which we didn’t want in our JSON, we just wanted simple technology-agnostic type names like “<font face="Courier New">cat</font>” or “<font face="Courier New">dog</font>” that we would be happy to map manually somewhere in our code. We didn’t want to write <em>all</em> the deserialization logic manually, but was happy to give the library a leg-up with the mapping of types.</font></p> <p><strong><font face="Trebuchet MS">JsonConverter</font></strong></p> <p><font face="Trebuchet MS">Our searching quickly led to the following question on Stack Overflow: “</font><a href="https://stackoverflow.com/questions/19307752/deserializing-polymorphic-json-classes-without-type-information-using-json-net"><font face="Trebuchet MS">Deserializing polymorphic json classes without type information using json.net</font></a><font face="Trebuchet MS">”. The lack of type information mentioned in the question meant the <em>exact</em> .Net type (i.e. name, assembly, version, etc.), and so the answer describes how to do it where you can infer the resulting type from one or more attributes in the data itself. In our case it was a field unsurprisingly called “<font face="Courier New">type</font>” that held a simplified name as described earlier.</font></p> <p><font face="Trebuchet MS">The crux of the solution involves creating a <font face="Courier New">JsonConverter</font> and implementing the two methods <font face="Courier New">CanConvert</font> and <font face="Courier New">ReadJson</font>. If we follow that Stack Overflow post’s top answer we end up with an implementation something like this:</font></p> <p><font face="Courier New">public class CustomJsonConverter : JsonConverter <br />{ <br />&#160; public override bool CanConvert(Type objectType) <br />&#160; { <br />&#160;&#160;&#160; return typeof(BaseType). <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; IsAssignableFrom(objectType); <br />&#160; } <br /> <br />&#160; public override object ReadJson(JsonReader reader, <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Type objectType, object existingValue, <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; JsonSerializer serializer) <br />&#160; { <br />&#160;&#160;&#160; JObject item = JObject.Load(reader); <br /> <br />&#160;&#160;&#160; if (item.Value&lt;string&gt;(“type”) == “Derived”) <br />&#160;&#160;&#160; { <br />&#160;&#160;&#160;&#160;&#160; return item.ToObject&lt;DerivedType&gt;(); <br />&#160;&#160;&#160; } <br />&#160;&#160;&#160; else <br />&#160;&#160;&#160; . . . <br />&#160; } <br />}</font></p> <p><font face="Trebuchet MS">This all made perfect sense and even agreed with a couple of other blog posts on the topic we unearthed. However when we plugged it in we ended up with an infinite loop in the <font face="Courier New">ReadJson</font> method that resulted in a <font face="Courier New">StackOverflowException</font>. Doing some more googling and checking the Newtonsoft JSON.Net documentation didn’t point out our “obvious” mistake and so we resorted to the time honoured technique of fumbling around with the code to see if we could get <em>this</em> (seemingly promising) solution working.</font></p> <p><strong><font face="Trebuchet MS">A Blind Alley</font></strong></p> <p><font face="Trebuchet MS">One avenue that <em>appeared</em> to fix the problem was manually adding the <font face="Courier New">JsonConverter</font> to the list of Converters in the <font face="Courier New">JsonSerializerSettings</font> object instead of using the <font face="Courier New">[JsonConverter]</font> attribute on the base class. We went back and forth with some unit tests to prove that this was indeed the solution and even committed this fix to our codebase.</font></p> <p><font face="Trebuchet MS">However I was never really satisfied with this outcome and so decided to write this incident up. I started to work through the simplest possible example to illustrate the behaviour but when I came to repro it I found that neither approach worked – attribute or serializer settings - I always got into an infinite loop.</font></p> <p><font face="Trebuchet MS">Hence I questioned our original diagnosis and continued to see if there was a more satisfactory answer.</font></p> <p><strong><font face="Trebuchet MS">ToObject vs Populate</font></strong></p> <p><font face="Trebuchet MS">I went back and re-read the various hits we got with those additional keywords (recursion, infinite loop and stack overflow) to see if we’d missed something along the way. The two main candidates were “</font><a href="https://stackoverflow.com/questions/29124126/polymorphic-json-deserialization-failing-using-json-net"><font face="Trebuchet MS">Polymorphic JSON Deserialization failing using Json.Net</font></a><font face="Trebuchet MS">” and “</font><a href="https://stackoverflow.com/questions/25404202/custom-inheritance-jsonconverter-fails-when-jsonconverterattribute-is-used"><font face="Trebuchet MS">Custom inheritance JsonConverter fails when JsonConverterAttribute is used</font></a><font face="Trebuchet MS">”. Neither of these explicitly references the answer we initially found and what might be wrong with it – they give a different answer to a slightly different question.</font></p> <p><font face="Trebuchet MS">However in these answers they suggest de-serializing the object in a different way, instead of using <font face="Courier New">ToObject&lt;DerivedType&gt;()</font> to do <em>all</em> the heavy lifting, they suggest creating the uninitialized object yourself and then using <font face="Courier New">Populate()</font> to fill in the details, like this:</font></p> <p><font face="Courier New">{ <br />&#160; JObject item = JObject.Load(reader); <br /> <br />&#160; if (item.Value&lt;string&gt;(“type”) == “Derived”) <br />&#160; { <br />&#160;&#160;&#160; var @object = new DerivedType(); <br />&#160;&#160;&#160; serializer.Populate(item.CreateReader(), @object); <br />&#160;&#160;&#160; return @object; <br />&#160; } <br />&#160; else <br />&#160;&#160;&#160; . . . <br />}</font></p> <p><font face="Trebuchet MS">Plugging this approach into my minimal example worked, and for both the converter techniques too: attribute and serializer settings.</font></p> <p><strong><font face="Trebuchet MS">Unanswered Questions</font></strong></p> <p><font face="Trebuchet MS">So I’ve found another technique that works, which is great, but I still lack closure around the whole affair. For example, how come the answer in the the original Stack Overflow question “</font><a href="https://stackoverflow.com/questions/19307752/deserializing-polymorphic-json-classes-without-type-information-using-json-net"><font face="Trebuchet MS">Deserializing polymorphic json classes</font></a><font face="Trebuchet MS">” didn’t work for us? That answer has plenty of up-votes and so should be considered pretty reliable. Has there been a change to Newtonsoft’s JSON.Net library that has somehow caused this answer to now break for others? Is there a new bug that we’ve literally only just discovered (we’re using v10)? Why don’t the JSON.Net docs warn against this if it really is an issue, or are we looking in the wrong part of the docs?</font></p> <p><font face="Trebuchet MS">As described right at the beginning I’ve published <a href="https://gist.github.com/chrisoldwood/b604d69543a5fe5896a94409058c7a95">a Gist</a> with my minimal example and added <a href="https://stackoverflow.com/questions/19307752/deserializing-polymorphic-json-classes-without-type-information-using-json-net#comment75717489_19308474">a comment</a> to the Stack Overflow answer with that link so that anyone else on the same journey has some other pieces of the jigsaw to work with. Perhaps over time my comment will also acquire up-votes to help indicate that it’s not so cut-and-dried. Or maybe someone who knows the right answer will spot it and point out where we went wrong.</font></p> <p><font face="Trebuchet MS">Ultimately though this is probably a case of not seeing the wood for the trees. It’s so easy when you’re trying to solve one problem to get lost in the accidental complexity and not take a step back. Answers on Stack Overflow generally carry a large degree of gravitas, but they should not be assumed to be infallible. All documentation can go out of date even if there are (seemingly) many eyes watching over it.</font></p> <p><font face="Trebuchet MS">When your mind-set is one that always assumes the bugs are of your own making, unless the evidence is overwhelming, then those times when you might actually not be entirely at fault seem to feel all the more embarrassing when you realise the answer was probably there all along but you discounted it too early because your train of thought was elsewhere.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-48928363131942948542017-05-23T21:00:00.001+01:002017-05-23T21:00:12.598+01:00Are Refactoring Tools Less Effective Overall?<p><font face="Trebuchet MS">Prior to the addition of automatic refactoring tools to modern IDEs refactoring was essentially a manual affair. You would make a code change, hit build, and then fix all the compiler errors (at least for statically typed languages). This technique is commonly known as “leaning on the compiler”. Naturally the operation could be fraught with danger if you were far too ambitious about the change, but knowing when you could lean on the compiler was part of the art of refactoring safely back then.</font></p> <p><strong><font face="Trebuchet MS">A Hypothesis</font></strong></p> <p><font face="Trebuchet MS">Having lived through both eras (manual and automatic) and paired with developers far more skilled with the automatic approach I’ve come up with a totally non-scientific hypothesis that suggests automatic refactoring tools are actually less effective than the manual approach, overall.</font></p> <p><font face="Trebuchet MS">I guess the basis of this hypothesis pretty much hinges on what I mean by “effective”. Here I’m suggesting that automatic tools help you easily refactor to a <em>local</em> minima but not to a <em>global</em> minima [1]; consequently the codebase <em>as a whole</em> ends up in a less coherent state.</font></p> <p><strong><font face="Trebuchet MS">Shallow vs Deep Refactoring</font></strong></p> <p><font face="Trebuchet MS">The goal of an automatic refactoring tool appears to be to not break your code – it will only allow you to use it to perform a simple refactoring that can be done safely, i.e. if the tool can’t fix up <em>all</em> the code it can see [2] it won’t allow you to do it in the first place. The consequence of this is that the tool constantly limits you to taking very small steps. Watching someone refactor with a tool can sometimes seem tortuous as they may need to use so many little refactoring steps to get the code into the desired state because you cannot make the leaps you want in one go unless you switch to manual mode.</font></p> <p><font face="Trebuchet MS">This by itself isn’t a bad thing, after all making a safe change is clearly A Good Thing. No, where I see the problem is that by fixing up all the call sites automatically you don’t get to see the wider effects of the refactoring you’re attempting.</font></p> <p><font face="Trebuchet MS">For example the reason you’d choose to rename a class or method is because the existing one is no longer appropriate. This is probably because you’re learned something new about the problem domain. However that class or method does not exist in a vacuum, it has dependencies in the guise of variable names and related types. It’s entirely likely that some of these may now be inappropriate too, however you won’t easily see them because the tool has likely hidden them from you.</font></p> <p><font face="Trebuchet MS">Hence one of the “benefits” of the old manual refactoring approach was that as you visited each broken call site you got to reflect on your change in the context of where it’s used. This often led to further refactorings as you began to comprehend the full nature of what you had just discovered.</font></p> <p><strong><font face="Trebuchet MS">Blue or Red Pill?</font></strong></p> <p><font face="Trebuchet MS">Of course what I’ve just described could easily be interpreted as the kind of “black hole” that many, myself included, would see as an unbounded unit of work. It’s one of those nasty rabbit holes where you enter and, before you know it, you’re burrowing close to the Earth’s core and have edited nearly every file in the entire workspace.</font></p> <p><font face="Trebuchet MS">Yes, like any change, it takes discipline to stick to the scope of the original problem. </font><font face="Trebuchet MS">Just because you keep unearthing more and more code that no longer appears to fit the new model it does not mean you have to tackle it right now. Noticing the disparity is the first step towards fixing it.</font></p> <p><strong><font face="Trebuchet MS">Commit Review</font></strong></p> <p><font face="Trebuchet MS">It’s not entirely true that you won’t see the entire outcome of the refactoring – at the very least the impact will be visible when you review the complete change before committing. (For a fairly comprehensive list of the things I go through at the point I commit see my <a href="https://accu.org/index.php/journal">C Vu</a> article “<a href="http://www.chrisoldwood.com/articles/in-the-toolbox-commit-checklist.html">Commit Checklist</a>”.)</font></p> <p><font face="Trebuchet MS">This assumes of course that you do a thorough review of your commits before pushing them. However by this point, just as writing tests after the fact are considerably less attractive, so is finishing off any refactoring; perhaps even more so because the code is not broken per-se, it just might not be the best way of representing the solution.</font></p> <p><font face="Trebuchet MS">It’s all too easy to justify the reasons why it’s okay to go ahead and push the change as-is because there are more important things to do. Even if you think you’re aware of technical debt it often takes a fresh pair of eyes to see how you’re living in a codebase riddled with inconsistencies that make it hard to see it’s true structure. One is then never quite sure without reviewing the commit logs what is the legacy and what is the new direction.</font></p> <p><strong><font face="Trebuchet MS">Blinded by Tools</font></strong></p> <p><font face="Trebuchet MS">Clearly this is not the fault of the tool or their vendors. What they offer now is far more favourable than not having them at all. However once again we need to be reminded that we should not be slaves to our tools but that we are the masters. This is a common theme which is regularly echoed in the software development community and something I myself tackled in the past with “<a href="http://chrisoldwood.blogspot.co.uk/2012/02/dont-let-your-tools-pwn-you.html">Don’t Let Your Tools Pwn You</a>”.</font></p> <p><font face="Trebuchet MS"><a href="http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule">The Boy Scout Rule</a> (popularised by <a href="https://en.wikipedia.org/wiki/Robert_Cecil_Martin">Uncle Bob</a>) says that we should always leave the camp site cleaner than we found it. While picking up a handful of somebody else’s rubbish and putting it in the bin might meet the goal in a literal sense, it’s no good if the site is acquiring rubbish faster than it’s being collected.</font></p> <p><font face="Trebuchet MS">Refactoring is a technique for improving the quality of a software design in a piecewise fashion; just be careful you don’t spend so long on your hands and knees cleaning small areas that you fail to spot the resulting detritus building up around you.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] I wasn’t sure whether to say minima or maxima but I felt that refactoring was about lowering entropy in some way so went with the reduction metaphor.</font></p> <p><font face="Trebuchet MS">[2] Clearly there are limits around published APIs which it just has to ignore.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com1tag:blogger.com,1999:blog-6628985022531866193.post-31854182512402572822017-05-18T20:02:00.001+01:002017-05-18T20:02:36.833+01:00Excel-style DDE Requests<p><font face="Trebuchet MS">Despite being over 2 decades old Microsoft’s <a href="https://en.wikipedia.org/wiki/Dynamic_Data_Exchange">Dynamic Data Exchange</a> (DDE) in Windows still seems to be in use for Windows IPC by a not insignificant number of companies. At least, if the frequency of DDE questions in my inbox is anything to go by [1][2].</font></p> <p><font face="Trebuchet MS">Earlier this year I got a question from someone who was trying to use my <em><a href="http://www.chrisoldwood.com/win32.htm#ddecommand">DDE Command</a></em> tool (a command line tool for querying DDE servers) to get data out of the <a href="https://www.metatrader4.com">MetaTrader</a> 4 platform. Finance is the area I first came across DDE in anger and it still seems to be a popular choice there even to this day.</font></p> <p><strong><font face="Trebuchet MS">Curious Behaviour</font></strong></p> <p><font face="Trebuchet MS">The problem was that when they used the <font face="Courier New">ddecmd</font> “<font face="Courier New">request</font>” verb to send an <font face="Courier New">XTYP_REQUEST</font> message to the MetaTrader 4 DDE Server (<font face="Courier New">MT4</font>) for a symbol they always got an immediate result of “<font face="Courier New">N/A</font>”. As a workaround they tried using the “<font face="Courier New">advise</font>” verb, which sends an <font face="Courier New">XTYP_ADVSTART</font>, to listen for updates for a short period instead. This worked for symbols which changed frequently but missed those that didn’t change during the interval. Plus this was a dirty hack as they had to find a way to send a <font face="Courier New">CTRL+C</font> to my tool to stop it after this short interval.</font></p> <p><font face="Trebuchet MS">Clearly the MetaTrader DDE server couldn’t be <em>this</em> broken, and the proof was that it worked fine with Microsoft Excel – the other stalwart of the finance industry. Hence the question posed to me was why Excel appeared to work, but sending a request from my tool didn’t, i.e. was there a bug in my tool?</font></p> <p><strong><font face="Trebuchet MS">Reproducing the Problem</font></strong></p> <p><font face="Trebuchet MS">Given the popularity of the MetaTrader platform and Microsoft Excel the application of Occam’s Razor would suggest a bug in my tool was clearly the most likely answer, so I investigated…</font></p> <p><font face="Trebuchet MS">Luckily MetaTrader 4 is a free download and they will even give you a demo account to play with which is super welcome for people like me who only want to use the platform to fix interop problems in their own tools and don’t actually want to use it to trade.</font></p> <p><font face="Trebuchet MS">I quickly reproduced the problem by sending a DDE <font face="Courier New">request</font> for a common symbol:</font></p> <p><font face="Courier New">&gt; DDECmd.exe request -s MT4 -t QUOTE -i COPPER <br />N\A</font></p> <p><font face="Trebuchet MS">And then I used the DDE <font face="Courier New">advise</font> command to see it working for background updates:</font></p> <font face="Courier New">&gt; DDECmd.exe advise -s MT4 -t QUOTE -i COPPER <br />2.5525 2.5590 <br />2.5520 2.5590 <br />. . .&#160; <br /></font> <p><font face="Trebuchet MS">I also tried it in Excel too to see that it was successfully managing to request the current value, even for slow ticking symbols.</font></p> <p><strong><font face="Trebuchet MS">How Excel Requests Data Via DDE</font></strong></p> <p><font face="Trebuchet MS">My DDE Command tool has a nice feature where it can also act as a DDE server and logs the different requests sent to it. This was originally added by me to help diagnose problems in my <em>own</em> DDE client code but it’s also been useful to see how other DDE clients behave.</font></p> <p><font face="Trebuchet MS">As you can see below, when Excel opens a DDE link (<font face="Courier New">=TEST|TEST!X</font>) it actually sends a number of different <font face="Courier New">XTYP_ADVSTART</font> messages as it tries to find the highest fidelity format to receive the data in:</font></p> <p><font face="Courier New">&gt; DDECmd.exe listen –s TEST –t TEST <br />XTYP_CONNECT: 'TEST', 'TEST' <br />XTYP_CONNECT_CONFIRM: 'TEST', 'TEST' <br />XTYP_ADVSTART: 'T…', 'T…', 'StdDocumentName', '49157' <br />XTYP_ADVSTART: 'TEST', 'TEST', 'X', '50018' <br />. . . <br />XTYP_REQUEST: 'TEST', 'TEST', 'X', '50018'</font></p> <p><font face="Trebuchet MS">After it manages to set-up the initial advise loop it then goes on to send a one-off <font face="Courier New">XTYP_REQUEST</font> to retrieve the initial value. So, apart from the funky data formats it asks for, there is nothing unusual about the DDE request Excel seems to make.</font></p> <p><strong><font face="Trebuchet MS">Advise Before Request</font></strong></p> <p><font face="Trebuchet MS">And then it dawned on me, what if the MetaTrader DDE server <em>required</em> an advise loop to be established on a symbol before you’re allowed to request it? Sure enough, I hacked a bit of code into my <font face="Courier New">request</font> command to start an advise loop first and the subsequent DDE request succeeded.</font></p> <p><font face="Trebuchet MS">I don’t know if this is a bug in the MetaTrader 4 DDE server or the intended behaviour. I suspect the fact that it works with Excel covers the vast majority of users so maybe it’s never been a priority to support one-off data requests. The various other financial DDE servers I coded against circa 2000 never exhibited this kind of requirement – you could make one-off requests for data with a standalone <font face="Courier New">XTYP_REQUEST</font> message.</font></p> <p><strong><font face="Trebuchet MS">The New Fetch Command</font></strong></p> <p><font face="Trebuchet MS">The original intent of my DDE Command tool was to provide a tool that allows each <font face="Courier New">XTYP_*</font> message to be sent to a DDE server in isolation, mostly for testing purposes. As such the tools’ verbs pretty much have a one-to-one correspondence with the DDE messages you might send yourself.</font></p> <p><font face="Trebuchet MS">To allow people to use my tool against the MetaTrader 4 platform to snapshot data would therefore mean making some kind of small change. I did consider adding various special switches to the existing <font face="Courier New">request</font> and <font face="Courier New">advise</font> verbs, either to force an advise first or to force a request if no immediate update was received but that seemed to go against the ethos a bit.</font></p> <p><font face="Trebuchet MS">In the end I decided to add a <em>new</em> verb called “<font face="Courier New">fetch</font>” which acts just like “<font face="Courier New">request</font>”, but starts an advise loop for every item first, then sends a request message for the latest value, thereby directly mimicking Excel.</font></p> <p><font face="Courier New">&gt; DDECmd.exe fetch -s MT4 -t QUOTE -i COPPER -i SILVER <br />COPPER|2.6075 2.6145 <br />SILVER|16.771 16.821</font></p> <p><font face="Trebuchet MS">Hey presto it now works!</font></p> <p><font face="Trebuchet MS"><em>This feature was released in DDE Command v1.6.</em></font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] This is a bit of artistic licence :o), they are not a daily occurrence but once every couple of months wouldn’t be far off. So yes, “<a href="http://chrisoldwood.blogspot.co.uk/2009/10/dde-is-still-alive-kicking.html">DDE Is Still Alive &amp; Kicking</a>”.</font></p> <p><font face="Trebuchet MS">[2] Most recently it seems quite a few people are beginning to discover that Microsoft <a href="https://en.wikipedia.org/wiki/List_of_features_removed_in_Windows_Vista#Networking">dropped NetDDE support</a> way back in Windows Vista.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-36995868941260674262017-05-16T19:41:00.001+01:002017-05-16T19:41:00.076+01:00My Dislike of GOPATH<p><font face="Trebuchet MS">[<em>This post was written in response to <a href="https://twitter.com/mattetti/status/839729336139730946">a tweet</a> from <a href="https://twitter.com/mattetti">Matt Aimonetti</a> which asked “did you ever try #golang? If not, can you tell me why? If yes, what was the first blocker/annoyance you encountered?”</em>]</font></p> <p><font face="Trebuchet MS">I’m a dabbler in Go [1], by which I mean I know enough to be considered dangerous but not enough to be proficient. I’ve done a number of katas and even paired on some simple tools my team has built in Go. There is so much to like about it that I’ve had cause to prefer looking for 3rd party tools written in it in the faint hope that I might at some point be able to contribute one day. But, every time I pull the source code I end up wasting so much time trying to get the thing to build because Go has its own opinions about how the source is laid out and tools are built that don’t match the way I (or most other tools I’ve used) work.</font></p> <p><strong><font face="Trebuchet MS">Hello, World!</font></strong></p> <p><font face="Trebuchet MS">Go is really easy to get into, on Windows it’s as simple as:</font></p> <p><font face="Courier New">&gt; choco install golang</font></p> <p><font face="Trebuchet MS">This installs the compiler and standard libraries and you’re all ready to get started. The obligatory first program is as simple as doing:</font></p> <p><font face="Courier New">&gt; pushd \Dev <br />&gt; mkdir HelloWorld <br />&gt; notepad HelloWorld.go <br />&gt; go build HelloWorld.go <br />&gt; HelloWorld</font></p> <p><font face="Trebuchet MS">So far, so good. It’s pretty much the same as if you were writing in any other compiled language – create folder, create source file, build code, run it.</font></p> <p><font face="Trebuchet MS">Along the way you may have run into some complaint about the variable <font face="Courier New">GOPATH</font> not being set. It’s easily fixed by simply doing:</font></p> <p><font face="Courier New">&gt; set GOPATH=%CD%</font></p> <p><font face="Trebuchet MS">You might have bothered to read up on all the brouhaha, got side-tracked and discovered that it’s easy to silence the complaint without any loss of functionality by setting it to point to <em>anywhere</em>. After all the goal early on is just to get your first program built and running not get bogged down in tooling esoterica.</font></p> <p><font face="Trebuchet MS">When <em>I</em> reached this point I did a few katas and used a locally installed copy of the excellent multi-language exercise tool <a href="http://cyber-dojo.org/">cyber-dojo.org</a> to have a play with the test framework and some of the built-in libraries. Using a tool like cyber-dojo meant that <font face="Courier New">GOPATH</font> problems didn’t rear their ugly head again as it was already handled by the tool and the katas only needed standard library stuff.</font></p> <p><font face="Trebuchet MS">The first <em>non</em>-kata program my team wrote in Go (which I paired on) was a simple HTTP smoke test tool that also just lives in the same repo as the service it tests. Once again their was nary a whiff of <font face="Courier New">GOPATH</font> issues here either – still simple.</font></p> <p><strong><font face="Trebuchet MS">Git Client, But not Quite</font></strong></p> <p><font face="Trebuchet MS">The problems eventually started for me when I tried to download a 3rd party tool off the internet and build it [2]. Normally, getting the source code for a tool from an online repository, like GitHub, and then building it is as simple as:</font></p> <p><font face="Courier New">&gt; git clone https://…/tool.git <br />&gt; pushd tool <br />&gt; build</font></p> <p><font face="Trebuchet MS">Even if there is no Windows build script, with the little you know about Go at this point you’d hope it to be something like this:</font></p> <p><font face="Courier New">&gt; go build</font></p> <p><font face="Trebuchet MS">What usually happens now is that you start getting funny errors about not being able to find some code which you can readily deduce is some missing package dependency.</font></p> <p><font face="Trebuchet MS">This is the point where you start to haemorrhage time as you seek in vain to fix the missing package dependency without realising that the <em>real</em> mistake you made was right back at the beginning when you used “<font face="Courier New">git clone</font>” instead of “<font face="Courier New">go get</font>”. Not only that but you also forgot that you should have been doing all this inside the “<font face="Courier New">%GOPATH%\src</font>” folder and not in some arbitrary <font face="Courier New">TEMP</font> folder you’d created just to play around in.</font></p> <p><font face="Trebuchet MS">The goal was likely to just build &amp; run some 3rd party tool in isolation but that’s not the way Go wants you to see the world.</font></p> <p><strong><font face="Trebuchet MS">The Folder as a Sandbox</font></strong></p> <p><font face="Trebuchet MS">The most basic form of isolation, and therefore version control, in software development is the humble file-system folder [3]. If you want to monkey with something on the side just make a copy of it in another folder and you know your original is safe. This style of isolation (along with other less favourable forms) is something I’ve written about in depth before in my </font><a href="https://accu.org/"><font face="Trebuchet MS">C Vu</font></a><font face="Trebuchet MS"> <em><a href="http://chrisoldwood.blogspot.co.uk/2016/11/in-toolbox-season-two.html">In the Toolbox</a></em> column, see “</font><a href="http://www.chrisoldwood.com/articles/in-the-toolbox-the-developers-sandbox.html"><font face="Trebuchet MS">The Developer’s Sandbox</font></a><font face="Trebuchet MS">”.</font></p> <p><font face="Trebuchet MS">Unfortunately for me this is how I hope (expect) all tools to work out of the box. Interestingly Go is a highly opinionated language (which is a good thing in many cases) that wants you to do <em>all</em> your coding under one folder, identified by the <font face="Courier New">GOPATH</font> variable. <a href="https://golang.org/doc/code.html#Overview">The rationale</a> for this is that it reduces friction from versioning problems and helps ensure everyone, and everything, is always using a consistent set of dependencies – ideally the latest.</font></p> <p><strong><font face="Trebuchet MS">Tool User, Not Developer</font></strong></p> <p><font face="Trebuchet MS">That policy makes sense for Google’s developers working on <em>their</em> company’s tools, but I’m not a Google developer, I’m a just a user of the tool. My goal is to be able to build and run the tool. If there happens to be a simple bug that I can fix, then great, I’d like to do that, but what I do not have the time for is getting bogged down in library versioning issues because the language believes everyone, everywhere should be singing from the same hymn sheet. I’m more used to a world where dependencies move forward at a pace dictated by the author not by the toolchain itself. Things can still move quickly without having to continually live on the bleeding edge.</font></p> <p><strong><font face="Trebuchet MS">Improvements</font></strong></p> <p><font face="Trebuchet MS">As a Go outsider I can see that the situation is definitely improving. The use of <a href="http://stackoverflow.com/questions/35999046/vendoring-in-go-1-6">a vendor subtree</a> to house a snapshot of the dependencies in source form makes life much simpler for people like me who just want to use the tool hassle free and dabble occasionally by fixing things here and there.</font></p> <p align="left"><font face="Trebuchet MS">In the early days when I first ran into this problem I naturally assumed it was my problem and that I just needed to set the <font face="Courier New">GOPATH</font> variable to the root of the repo I had just cloned. I soon learned that this was a fools errand as the repo also has to buy into this and structure their source code accordingly. However a variant of this has got some traction with <a href="https://getgb.io/">the gb tool</a> which has (IMHO) got <a href="https://getgb.io/rationale/">the right idea</a> about isolation but sadly is not the sanctioned approach and so you’re dicing with the potential for future impedance mismatches. Ironically to build and install this tool requires <font face="Courier New">GOPATH</font> to still be working the proper way.</font></p> <p><font face="Trebuchet MS">The latest version of Go (1.8) will assume a default location for <font face="Courier New">GOPATH</font> (a “<font face="Courier New">go</font>” folder under your profile) if it’s not set but that does not fix the fundamental issue for me which is that you need to understand that any code you pull down may be somewhere unrelated on your file-system if you don’t understand how all this works.</font></p> <p><strong><font face="Trebuchet MS">Embracing GOPATH</font></strong></p> <p><font face="Trebuchet MS">Ultimately if I am going to properly embrace Go as a language, and I would like to do more, I know that I need to stop fighting the philosophy and just “get with the programme”. This is hard when you have a couple of decades of inertia to overcome but I’m sure I will eventually. It’s happened enough times now that I know what the warnings signs are and what I need to Google to do it “the Go way”.</font></p> <p><font face="Trebuchet MS">Okay, so I don’t like or agree with <em>all</em> of (that I know) the choices the Go language has taken but then I’m always aware of the popular quote by </font><a href="https://en.wikiquote.org/wiki/Bjarne_Stroustrup"><font face="Trebuchet MS">Bjarne Stroustrup</font></a><font face="Trebuchet MS">:</font></p> <blockquote> <p><font face="Trebuchet MS">“There are only two kinds of languages: the ones people complain about and the ones nobody uses.”</font></p> </blockquote> <p><font face="Trebuchet MS">This post is but one tiny data point which covers one of the very few complaints I have about Go, and even then it’s just really a bit of early friction that occurs at the start of the journey if you’re not a regular. Still, better that than yet another language nobody uses.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] Or golang if you want to appease the SEO crowd :o).</font></p> <p><font face="Trebuchet MS">[2] It was </font><a href="https://github.com/masterzen/winrm-cli"><font face="Trebuchet MS">winrm-cli</font></a><font face="Trebuchet MS"> as I was trying to put together a bug report for Terraform and there was something going wrong with running a Windows script remotely via WinRM.</font></p> <p><font face="Trebuchet MS">[3] Let’s put old fashioned technologies like COM to one side for the moment and assume we have learned from past mistakes.</font></p> <p><font face="Trebuchet MS"></font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-39208924283753477802017-03-01T20:14:00.001+00:002017-03-01T20:14:00.792+00:00Manual Mutation Testing<p><font face="Trebuchet MS">One of the problems when making code changes is knowing whether there is good test coverage around the area you’re going to touch. In theory, if a rigorous test-first approach is taken no production code should be written without first being backed by a failing test. Of course we all know the old adage about how theory turns out in practice [1]. Even so, just because a test has been written, you don’t know what the quality of it and any related ones are.</font></p> <p><strong><font face="Trebuchet MS">Mutation Testing</font></strong></p> <p><font face="Trebuchet MS">The practice of <a href="https://en.wikipedia.org/wiki/Mutation_testing">mutation testing</a> is one way to answer the perennial question: how do you test the tests? How do you know if the tests which have been written adequately cover the behaviours the code should exhibit? Alternatively, as a described recently in “<a href="http://chrisoldwood.blogspot.co.uk/2016/11/overly-prescriptive-tests.html">Overly Prescriptive Tests</a>”, are the tests too brittle because they require too exacting a behaviour?</font></p> <p><font face="Trebuchet MS">There are tools out there which will perform mutation testing automatically that you can include as part of your build pipeline. However I tend to use them in a more manual way to help me verify the tests around the small area of functionality I’m currently concerned with [2].</font></p> <p><font face="Trebuchet MS">The principle is actually very simple, you just tweak the production code in a small way that would likely mimic a real change and you see what tests fail. If no tests fail at all then you probably have a gap in your spec that needs filling.</font></p> <p><font face="Trebuchet MS">Naturally the changes you make on the production code should be sensible and functional in behaviour; there’s no point in randomly setting a reference to null if that scenario is impossible to achieve through the normal course of events. What we’re aiming for here is the simulation of an accidental breaking change by a developer. By tweaking the boundaries of any logic we can also check that our edge cases have adequate coverage too.</font></p> <p><font face="Trebuchet MS">There is of course the possibility that this will also unearth some dead code paths too, or at least lead you to further simplify the production code to achieve the same expected behaviour.</font></p> <p><strong><font face="Trebuchet MS">Example</font></strong></p> <p><font face="Trebuchet MS">Imagine you’re working on a service and you spy some code that appears to format a DateTime value using the default formatter. You have a hunch this might be wrong but there is no obvious unit test for the formatting behaviour. It’s possible the value is observed and checked in an integration or acceptance test elsewhere but you can’t obviously [3] find one.</font></p> <p><font face="Trebuchet MS">Naturally if you break the production code a corresponding test should break. But how badly do you break it? If you go <em>too</em> far all your tests might fail because you broke something fundamental, so you need to do it in varying degrees and observe what happens at each step.</font></p> <p><font face="Trebuchet MS">If you tweak the date format, say, from the US to the UK format nothing may happen. That might be because the tests use a value like 1st January which is 01/01 in both schemes. Changing from a local time format to an ISO format may provoke something new to fail. If the test date is particularly well chosen and loosely verified this could well still be inside whatever specification was chosen.</font></p> <p><font face="Trebuchet MS">Moving away from a purely numeric form to a more natural, wordy one should change the length and value composition even further. If we reach this point and no tests have failed it’s a good chance nothing will. We can then try an empty string, nonsense strings and even a null string reference to see if someone only cares that some arbitrary value is provided.</font></p> <p><font face="Trebuchet MS">But what if after all that effort still no lights start flashing and the klaxon continues to remain silent?</font></p> <p><strong><font face="Trebuchet MS">What Does a Test Pass or Fail Really Mean?</font></strong></p> <p><font face="Trebuchet MS">In the ideal scenario as you slowly make more and more severe changes you would eventually hope for one or maybe a couple of tests to start failing. When you inspect them it should be obvious from their name and structure what was being expected, and why. If the test name and assertion clearly specifies that some arbitrary value is required then its probably intentional. Of course It may still be undesirable for other reasons [4] but the test might express its intent well (to document and to verify).</font></p> <p><font face="Trebuchet MS">If we only make a very small change and a lot of tests go red we’ve probably got some brittle tests that are highly dependent on some unrelated behaviour, or are duplicating behaviours already expressed (probably better) elsewhere.</font></p> <p><font face="Trebuchet MS">If the tests stay green this does not necessary mean we’re still operating within the expected behaviour. It’s entirely possible that the behaviour has been left completely unspecified because it was overlooked or forgotten about. It might be that not enough was known at the time and the author expected someone else to “fill in the blanks” at a later date. Or maybe the author just didn’t think a test was needed because the intent was <em>so</em> obvious.</font></p> <p><strong><font face="Trebuchet MS">Plugging the Gaps</font></strong></p> <p><font face="Trebuchet MS">Depending on the testing culture in the team and your own appetite for well defined executable specifications you may find mutation testing leaves you with more questions than you are willing to take on. You only have so much time and so need to find a way to plug any holes in the most effective way you can. Ideally you’ll follow the <a href="http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule">Boy Scout Rule</a> and at least leave the codebase in a better state than you found it, even if that isn’t entirely to your own satisfaction.</font></p> <p><font face="Trebuchet MS">The main thing I get out of using mutation testing is a better understanding of what it means to write good tests. Seeing how breaks are detected and reasoned about from the resulting evidence gives you a different perspective on how to express your intent. My tests definitely aren’t perfect but by purposefully breaking code up front you get a better feel for how to write less brittle tests than you might by using TDD alone.</font></p> <p><font face="Trebuchet MS">With TDD you are the author of both the tests and production code and so are highly familiar with both from the start. Making a change to existing code by starting with mutation testing gives you a better orientation of where the existing tests are and how they perform before you write your own first new failing test.</font></p> <p><font face="Trebuchet MS"><strong>Refactoring Tests</strong></font></p> <p><font face="Trebuchet MS">Refactoring is about changing the code without changing the behaviour. This can also apply to tests too in which case mutation testing can provide the technique for which you start by creating failing production code that you “fix” when the test is changed and the bar goes green again. You can then commit the refactored tests before starting on the change you originally intended to make.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] “In theory there is no difference between theory and practice; in practice there is.” –- <a href="https://en.wikiquote.org/wiki/Jan_L._A._van_de_Snepscheut">Jan L. A. van de Snepscheut</a>.</font></p> <p><font face="Trebuchet MS">[2] Just as with techniques like static code analysis you really need to adopt this from the beginning if you want to keep the noise level down and avoid exploring too large a rabbit hole.</font></p> <p><font face="Trebuchet MS">[3] How you organise your tests is a subject in its own right but suffice to say that it’s usually easier to find a unit test than an acceptance test that depends on any given behaviour.</font></p> <p><font face="Trebuchet MS">[4] The author may have misunderstood the requirement or the requirement was never clear originally and so the the behaviour was left loosely specified in the short term.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-51376148766133607092017-02-28T20:54:00.001+00:002017-02-28T20:54:59.576+00:00Automate Only What You Need To<p><font face="Trebuchet MS">The meme tells us to “automate all the things” and it’s a noble cause which has sprung up as a backlash against the ridiculous amount of manual work we’ve often had to do in the past. However in our endeavour to embrace the meme we should not go overboard and lose sight of what we’re automating and why.</font></p> <p><strong><font face="Trebuchet MS">The Value</font></strong></p> <p><font face="Trebuchet MS">The main reason we tend to automate things is to save ourselves time (and by extension, money) by leveraging tools that can perform tasks quicker than we can, but also with more determinism and reliability, thereby saving even more time. For example, pasting a complex set of steps off a wiki page into a command prompt to perform a task is slower than an interpreter running a script and is fraught with danger as we might screw up at various points along the way and so end up not doing exactly what we’d intended. Ultimately computers are good at boring repetitive tasks whilst we humans are not.</font></p> <p><font face="Trebuchet MS">However if we only do this operation once every six months and there are too many potential points of failure we might spend far longer trying to automate it than it actually takes to do carefully, manually. It’s a classic trade-off and like most things in IT there are some <a href="https://xkcd.com">XKCD</a>’s for that – “<a href="https://xkcd.com/1319/">Automation</a>” and “<a href="https://xkcd.com/1205/">Is It Worth the Time</a>”. They make sobering reading when you’re trying to work out how much time you <em>might</em> save automating something and therefore also gives a good indication of the maximum amount of time you should spend on achieving that.</font></p> <p><strong><font face="Trebuchet MS">Orchestration First, Actor Later</font></strong></p> <p><font face="Trebuchet MS">Where I think the meme starts to break down is when we get this balance wrong and begin to lose sight of where the real value is, thereby wasting time trying to automate not only all the steps but also wire it into some job scheduling system (e.g. CI server) so that once in a blue moon we can push a button and the whole task from start to finish is executed for us without further intervention.</font></p> <p><font face="Trebuchet MS">The dream suggests at that point we can go off and do something else more valuable instead. Whilst this notion of autonomy is idyllic it can also come with a considerable extra up-front cost and any shortcuts are likely to buy us false security (i.e. it silently fails and we lose time investigating downstream failures instead).</font></p> <p><font face="Trebuchet MS">For example there are many crude command prompt one-liners I’ve written in the past to pick up common mistakes that are trivial for me to run because they’ve been written to automate the expensive bit, not the entire problem. I often rely on my own visual system to filter out the noise and compensate for the impurities within the process. Removing these wrinkles is often where the proverbial “<a href="https://en.wikipedia.org/wiki/Ninety-ninety_rule">last 10% that takes 90% of the time</a>” goes [1].</font></p> <p><font face="Trebuchet MS">It’s all too easy to get seduced by the meme and believe that no automation task is truly complete until it’s <em>fully</em> automated.</font></p> <p><strong><font face="Trebuchet MS">An Example</font></strong></p> <p><font face="Trebuchet MS">In .Net when you publish shared libraries as NuGet packages you have a <font face="Courier New">.nuspec</font> file which lists the package dependencies. The library <font face="Courier New">.csproj</font> build file also has project dependencies for use with compilation. However these two sets of dependencies should be kept in sync [2].</font></p> <p><font face="Trebuchet MS">Initially with only a couple of NuGet packages it was easy to do manually as I knew it was unlikely to change. However once the monolithic library got split it up the dependencies started to grow and manually comparing the relevant sections got harder and more laborious.</font></p> <p><font face="Trebuchet MS">Given the text based nature of the two files (XML) it was pretty easy to write a simple shell one-liner to <font face="Courier New">grep</font> the values from the two sets of relevant XML tags, dump them in a file, and then use diff to show a side-by-side comparison. Then it just needed wrapping in a <font face="Courier New">for</font> loop to traverse the solution workspace.</font></p> <p><font face="Trebuchet MS">Because the one-liner was mine I got to take various shortcuts like hardcoding the input path and temporary files along with “knowing” that a certain project was always misreported. At this point a previously manual process has largely been automated and as long as I run it regularly will catch any mistakes.</font></p> <p><font face="Trebuchet MS">Of course it’s nice to share things like this so that others can take advantage after I’m gone, and it might be even better if the process can be added as a build step so that it’s caught the moment the problem surfaces rather than later in response to a more obscure issue. Now things begin to get tricky and we start to see diminishing returns.</font></p> <p><font face="Trebuchet MS">First, the <a href="https://github.com/bmatzelle/gow/wiki">Gnu on Windows</a> (GoW) toolset I used isn’t standard on Windows so now I need to make the one-liner portable or make everyone else match my tooling choice [3]. I also need to fix the hard coded paths and start adding a bit of error handling. I also need to find a way to remove the noise caused by the one “awkward” project.</font></p> <p><font face="Trebuchet MS">None of this is onerous, but this all takes time and whilst I’m doing it I’m <em>not</em> doing something (probably) more valuable. The majority of the value was in being able to scale out this safety check, there is (probably) far less value in making it portable and making it run reliably as part of an automated build. This is because essentially it only needs to be run whenever the project dependencies change and that was incredibly rare once the initial split was done. Additionally the risk of not finding an impedance mismatch was small and should be caught by other automated aspects of the development process, i.e. the deployment and test suite.</font></p> <p><strong><font face="Trebuchet MS">Knowing When to Automate More</font></strong></p> <p><font face="Trebuchet MS">This scenario of cobbling something together and then finding you need to do it more often is the bread and butter of build &amp; deployment pipelines. You often start out with a bunch of hacked together scripts which do <em>just enough</em> to allow the team to bootstrap itself in to an initial fluid state of delivery. This is commonly referred to as <a href="http://alistair.cockburn.us/Walking+skeleton">a walking skeleton</a> because it forms the basis for the entire development process.</font></p> <p><font face="Trebuchet MS">The point of starting with the walking skeleton rather than just diving headlong into features is to try and tackle some of the problems that historically got left until it was too late, such as packaging and deployment. In the modern era of continuous delivery we look to deliver a thin slice of functionality quickly and then build upon it piecemeal.</font></p> <p><font face="Trebuchet MS">However it’s all too easy to get bogged down early on in a project and spend lots of time just getting the build pipeline up and running and have nothing <em>functional</em> to show for it. This has always made me feel a little uncomfortable as it feels as though we should be able to get away with far less than perhaps we think we need to.</font></p> <p><font face="Trebuchet MS">In “<a href="http://chrisoldwood.blogspot.co.uk/2014/10/building-pipeline-process-led-or.html">Building the Pipeline - Process Led or Automation Led</a>” and my even earlier post “<a href="http://chrisoldwood.blogspot.co.uk/2013/01/layered-builds.html">Layered Builds</a>” I’ve tried to promote a more organic approach that focuses on what I think really matters most which is a consistent and extensible approach. In essence we focus first on producing a simple, repeatable process that can be used <em>locally</em> to enable the application skeleton to safely evolve and then balance the need for automating this further along with the other features. If quality or speed of delivery drops and more automation looks to be the answer then it can be added with the knowledge that it’s being done for deliberate reasons, rather than because we’ve got carried away gold plating the build system based on what other people think it should do (i.e. a <a href="https://en.wikipedia.org/wiki/Cargo_cult">cargo cult</a> mentality).</font></p> <p><strong><font face="Trebuchet MS">Technical Risk</font></strong></p> <p><font face="Trebuchet MS">The one caveat to being leaner about your automation is that you may (accidentally) put off addressing one or more technical risks because you don’t perceive them as risks. This leads us back to why the meme exists in the first place – failing to address certain aspects of software delivery until it’s too late. If there is a technical concern, address it, but only to the extent that the risk is understood, you may not need to do anything about it now.</font></p> <p><font face="Trebuchet MS">With a team of juniors there is likely to be far more unknowns [4] than with a team of experienced programmers, therefore the set of perceived risks will be higher. Whilst you might not know the most elegant approach to solving a problem, knowing <em>an</em> approach already reduces the risk because you know that you can trade technical debt in the short term for something else more valuable if necessary.</font></p> <p><strong><font face="Trebuchet MS">Everything is Negotiable</font></strong></p> <p><font face="Trebuchet MS">The thing I like most about an agile development process is that every trade-off gets put front-and-centre, <em>everything</em> is now negotiable [5]. Every task now comes with an implicit question: is this the most valuable thing we could be doing?</font></p> <p><font face="Trebuchet MS">Whilst manually building a private cloud for your <em>production</em> system using a UI is almost certainly not the most scalable approach, neither is starting day one of a project by diving into, say, Terraform when you don’t even know what you’re supposed to be building. There is nothing wrong with starting off manually, you just need to be diligent and ensure that your decision to only automate “<em>enough</em> of the things” is always working in your favour.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] See “<a href="http://chrisoldwood.blogspot.co.uk/2016/05/the-curse-of-ntlm-based-http-proxies.html">The Curse of NTLM Based HTTP Proxies</a>”.</font></p> <p><font face="Trebuchet MS">[2] I’m not aware of Visual Studio doing this yet although there may now be extensions and tools written by others I’m not aware of.</font></p> <p><font face="Trebuchet MS">[3] Yes, the Unix command line tools should be ubiquitous and maybe finally they will be with Bash on Windows.</font></p> <p><font face="Trebuchet MS">[4] See “<a href="http://chrisoldwood.blogspot.co.uk/2010/04/turning-unconscious-incompetence-to.html">Turning Unconscious Incompetence to Conscious Incompetence</a>”.</font></p> <p><font face="Trebuchet MS">[5] See “<a href="http://chrisoldwood.blogspot.co.uk/2016/08/estimating-is-liberating.html">Estimating is Liberating</a>”.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-58112483250763590512017-02-28T20:24:00.001+00:002017-02-28T20:24:44.937+00:00LINQ: Did You Mean First(), or Really Single()?<p><em><font face="Trebuchet MS">TL;DR: if you see someone using the LINQ method <font face="Courier New">First()</font> without a comparator it’s probably a bug and they should have used <font face="Courier New">Single()</font>.</font></em></p> <p><font face="Trebuchet MS">I often see code where the author “knows” that a sequence (i.e. an <font face="Courier New">Enumerable&lt;T&gt;</font>) will result in just one element and so they use the LINQ method <font face="Courier New">First()</font> to retrieve the value, e.g.</font></p> <p><font face="Courier New">var value = sequence.First();</font></p> <p><font face="Trebuchet MS">However there is also the <font face="Courier New">Single()</font> method which could be used to achieve a <em>similar</em> outcome:</font></p> <p><font face="Courier New">var value = sequence.Single();</font></p> <p><font face="Trebuchet MS">So what’s the difference and why do I think it’s probably a bug if you used <font face="Courier New">First</font>?</font></p> <p><font face="Trebuchet MS">Both <font face="Courier New">First</font> and <font face="Courier New">Single</font> have the same semantics for the case where the the sequence is empty (they throw) and similarly when the sequence contains only a single element (they return it). The difference however is when the sequence contains more than one element – <font face="Courier New">First</font> discards the extra values and <font face="Courier New">Single</font> will throw an exception.</font></p> <p><font face="Trebuchet MS">If you’re used to SQL it’s the difference between using “<font face="Courier New">top</font>” to filter and trying extract a single scalar value from a subquery:</font></p> <p><font face="Courier New">select top 1 x as [value] from . . .</font></p> <p><font face="Trebuchet MS">and</font></p> <p><font face="Courier New">select a, (select x from . . .) as [value] from . . .</font></p> <p><font face="Trebuchet MS">(The latter tends to complain loudly if the result set from the subquery is not just a single scalar value or <font face="Courier New">null</font>.)</font></p> <p><font face="Trebuchet MS">While you might argue that in the face of a single-value sequence both methods could be interchangeable, to me they say different things with <font face="Courier New">Single</font> begin the only “correct” choice.</font></p> <p><font face="Trebuchet MS">Seeing <font face="Courier New">First</font> says to me that the author knows the sequence <em>might</em> contain multiple values and they have expressed an ordering which ensures the right value will remain after the others have been consciously discarded.</font></p> <p><font face="Trebuchet MS">Whereas <font face="Courier New">Single</font> suggests to me that the author knows this sequence contains one (and only one) element and that any other number of elements is wrong.</font></p> <p><font face="Trebuchet MS">Hence another big clue that the use of <font face="Courier New">First</font> is probably incorrect is the absence of a comparator function used to order the sequence. Obviously it’s no guarantee as the sequence might be being returned from a remote service or function which will do the sorting instead but I’d generally expect to see the two used together or some other clue (method or variable name, or parameter) nearby which defines the order.</font></p> <p><font face="Trebuchet MS">The consequence of getting this wrong is that you don’t detect a break in your expectations (a multi-element sequence). If you’re lucky it will just be a test that starts failing for a strange reason, which is where I mostly see this problem showing up. If you’re unlucky then it will silently fail and you’ll be using the wrong data which will only manifest itself somewhere further down the road where it’s harder to trace back.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-52623627650668556452017-01-30T23:11:00.001+00:002017-01-30T23:11:45.857+00:00Dumbing Down Code<p><font face="Trebuchet MS">A conversation with a colleague, which was originally sparked off by “</font><a href="https://www.codeproject.com/Articles/1083348/Csharp-BAD-PRACTICES-Learn-how-to-make-a-good-code"><font face="Trebuchet MS">C# BAD PRACTICES: Learn how to make a good code by bad example</font></a><font face="Trebuchet MS">” (an article about writing clear code), reminded me about a recent change I had just made. When I reflected back I began to consider whether I had just “dumbed down” the code instead of keeping my original <em>possibly</em> simpler version. I also couldn’t decide whether the reason I had changed it was because I was using an old idiom which might now be unfamiliar or because I just didn’t think it was worth putting the burden on the reader.</font></p> <p><strong><font face="Trebuchet MS">Going Loopy</font></strong></p> <p><font face="Trebuchet MS">The change I made was to add a simple retry loop to some integration test clean-up code that would occasionally fail on a laggy VM. The loop just needed to retry a small piece of code a few times with a brief pause in between each iteration.</font></p> <p><font face="Trebuchet MS">My gut instinct was to write the loop like this:</font></p> <p><font face="Courier New">int attempt = 5; <br /> <br />while (attempt-- &gt; 0) <br />{ <br />&#160; . . . <br />}</font></p> <p><font face="Trebuchet MS">Usually when I need to write a “<font face="Courier New">for</font>” loop these days, I don’t. By far the most common use case is to iterate over a sequence and so I’d use LINQ in C# or, say, a pipeline (or <font face="Courier New">foreach</font>) in PowerShell. If I did need to manually index an array (which is rare) I’d probably go straight for a classic <font face="Courier New">for</font> loop.</font></p> <p><font face="Trebuchet MS">After I wrote and tested the code I took a step back just to review what I’d written and realised I felt uncomfortable with the while loop. Yes this was only test code but I don’t treat it as a second class citizen, it still gets the same TLC as production code. So what did I not like?</font></p> <ul> <li><font face="Trebuchet MS">While loops are much rarer than for loops these days.</font></li> <li><font face="Trebuchet MS">The use of the pre and post-decrement operators in a conditional expression is also uncommon.</font></li> <li><font face="Trebuchet MS">The loop counts down instead of up.</font></li> </ul> <p><font face="Trebuchet MS">Of these the middle one – the use of the post-decrement operator in a conditional expression – was the most concerning. Used by themselves (in pre or post form) to just mutate a variable, such as in the final clause of a <font face="Courier New">for</font> loop [1], seems trivial to comprehend, but once you include it in a conditional statement the complexity goes up.</font></p> <p><font face="Trebuchet MS">Hence there is no difference between <font face="Courier New">--attempt</font> and <font face="Courier New">attempt--</font> when used in a simple arithmetic adjustment, but when used in a conditional expression it matters whether the value is decremented before or after the comparison is made. If you get it wrong the loop executes one less time than you expect (or vice-versa) [2].</font></p> <p><font face="Trebuchet MS">Which leads me to my real concern – how easy is it to reason about how many times the loop executes? Depending on the placement of the decrement operator it might be one less than the number “attempt” is initialised with at the beginning.</font></p> <p><font face="Trebuchet MS">Of course you can also place the “<font face="Courier New">while</font>” comparison at the end of the block which means it would execute at least once, so subconsciously this might also cause you to question the number of iterations, at least momentarily.</font></p> <p><font face="Trebuchet MS">Ultimately <em>I know</em> I’ve written the loop so that the magic number “5” which is used to initialise the <font face="Courier New">attempt</font> counter represents the maximum number of iterations, but will a reader trust me to have done the same? I think they’ll err on the side of caution and work it out for themselves.</font></p> <p><strong><font face="Trebuchet MS">Alternatives</font></strong></p> <p><font face="Trebuchet MS">The solution I went with in the end was this:</font></p> <p><font face="Courier New">const int maxAttempts = 5; <br /> <br />for (int attempt = 0; attempt != maxAttempts; <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ++attempt) <br />{ <br />&#160; . . . <br />}</font></p> <p><font face="Trebuchet MS">Now clearly this is more verbose, but not massively more verbose than my original “<font face="Courier New">while</font>” loop. However the question is whether (to quote Sir Tony Hoare [3]) it obviously contains less bugs that the original. Being au fait with both forms it’s hard for me to decide so I’m trying to guess that the reader would prefer the latter.</font></p> <p><font face="Trebuchet MS">Given that we don’t care what the absolute value of “<font face="Courier New">attempt</font>” is, only that we execute the loop N times, I did consider some other approaches, at least momentarily. Both of these examples use methods from the <font face="Courier New">Enumerable</font> class.</font></p> <p><font face="Trebuchet MS">The first generates a sequence of 5 numbers starting from the “arbitrary” number 1:</font></p> <p><font face="Courier New">const int maxAttempts = 5; <br /> <br />foreach (var _ in Enumerable.Range(1, maxAttempts)) <br />{ <br />&#160; . . . <br />}</font></p> <p><font face="Trebuchet MS">The second also generates a sequence of 5 numbers but this time by repeating the “arbitrary” number 0:</font></p> <font face="Courier New">const int maxAttempts = 5; <br /> <br />foreach (var _ in Enumerable.Repeat(0, maxAttempts)) <br />{ <br />&#160; . . . <br />}</font> <p><font face="Trebuchet MS">In both cases I have dealt with the superfluous naming of the loop variable by simply calling it “<font face="Courier New">_</font>”.</font></p> <p><font face="Trebuchet MS">I discounted both these ideas purely on the grounds that they’re overkill. It just feels wrong to be bringing so much machinery into play just to execute a loop a fixed number of times. Maybe my brain is addled from too much assembly language programming in my early years but seemingly unnecessary waste is still a hard habit to shake.</font></p> <p><font face="Trebuchet MS">As an aside there are plenty of C# extension methods out there which people have written to try and reduce this further so you only need write, say, “<font face="Courier New">5.Times()</font>” or “<font face="Courier New">0.To(5)</font>” but they still feel like syntactic sugar just for the sake of it.</font></p> <p><strong><font face="Trebuchet MS">Past Attempts</font></strong></p> <p><font face="Trebuchet MS">This is not the first time I’ve questioned whether it’s possible to write code that’s perhaps considered too clever. Way back in 2012 I wrote “</font><a href="http://chrisoldwood.blogspot.co.uk/2012/09/can-code-be-too-simple.html"><font face="Trebuchet MS">Can Code Be Too Simple?</font></a><font face="Trebuchet MS">” which looked at some C++ code I had encountered 10 years ago and which first got me thinking seriously about the subject.</font></p> <p><font face="Trebuchet MS">What separates the audience back then and the one now is the experience level of the programmers who will likely tackle this codebase. A couple of years later in “</font><a href="http://chrisoldwood.blogspot.co.uk/2014/10/will-your-successor-be-superstar.html"><font face="Trebuchet MS">Will Your Successor Be a Superstar Programmer?</font></a><font face="Trebuchet MS">” I questioned whether you write code for people of your own calibre or the (inevitable) application support team who have the unenviable task of having to be experts at two disciplines – support <em>and</em> software development. As organisations move towards development teams owning their services this issue is diminishing.</font></p> <p><font face="Trebuchet MS">My previous musings were also driven by my perception of the code other people wrote, whereas this time I’m reflecting solely on my own actions. In particular I’m now beginning to wonder if my approach is in fact patronising rather than aiding? Have I gone too far this time and should I give my successors far more credit? Or doesn’t it matter as long as we just don’t write “really weird” code?</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] In C++ iterators can be implemented as simple pointers or complex objects (e.g. STL container iterators in debug builds) and so you tend to be aware of the difference because of the performance impacts it can have.</font></p> <p><font face="Trebuchet MS">[2] I was originally going to write <font face="Courier New">while (attempt-- != 0)</font>, again because in C++ you normally iterate from “<font face="Courier New">begin</font>” to “<font face="Courier New">!= end</font>”, but many devs seem to be overly defensive and favour using the <font face="Courier New">&gt;</font> and <font face="Courier New">&lt;</font> comparison operators instead.</font></p> <p><font face="Trebuchet MS">[3] “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.” – <a href="https://en.wikipedia.org/wiki/Charles_Antony_Richard_Hoare">Sir Tony Hoare</a>.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-21000391897999190102017-01-26T20:24:00.001+00:002017-01-26T20:24:51.408+00:00Journey Code<p><font face="Trebuchet MS">In the move from a Waterfall approach to a more Agile way of working we need to learn to be more comfortable with our software being in a less well defined state. It’s not any less correct per-se, it’s just that we may choose to prioritise our work differently now so that we tackle the higher value features first.</font></p> <p><font face="Trebuchet MS"><strong>Destination Unknown</strong></font></p> <p><font face="Trebuchet MS">A consequence of this approach is that conversations between, say, traditional architects (who are talking about the system as they expect it to end up) and those developing it (who know it’s only somewhere between started and one possible future), need interpretation. Our job as programmers is to break these big features down into much smaller units that can be (ideally) independently scoped and prioritised. Being agile is about adapting to changing requirements and that’s impossible if every feature needs to designed, delivered and tested in it’s entirely.</font></p> <p><font face="Trebuchet MS">In an attempt to bridge these two perspectives I’ve somewhat turned into a broken record by frequently saying “but that’s the destination, we only need to start the journey” [1]. This is very much a statement to remind our (often old and habitual) selves that we don’t build software like that anymore.</font></p> <p><font face="Trebuchet MS"><strong>Build the Right Thing</strong></font></p> <p><font face="Trebuchet MS">The knock on effect is that any given feature is likely only partially implemented, especially in the early days when what we are still trying to explore what it is we’re actually trying to build in the first place. For example, data validation is feature we want in the product before going live, but we can probably make do with only touching on the subject lightly whilst we explore what data even needs validating in the first place.</font></p> <p><font face="Trebuchet MS">This partially implemented feature has started to go by the name “journey code”. While you can argue that all code is essentially malleable as the system grows from its birth to its eventual demise, what we are really trying to convey here is code which cannot really be described as having been “designed”. As such it does not indicate in any meaningful way the thoughts and intended direction of the original author – they literally did the simplest possible thing to make the acceptance test pass and that’s all.</font></p> <p><font face="Trebuchet MS"><strong>Destination in Sight</strong></font></p> <p><font face="Trebuchet MS">When we finally come to play out the more detailed aspects of the feature that becomes the time at which we intend to replace what we did along the journey with where we think the destination really is. Journey code does not even have to have been written in response to the feature itself, it may just be some infrastructure code required to bootstrap exploring a different one, such as logging or persistence.</font></p> <p><font face="Trebuchet MS">What it does mean is that collaboration is even more essential when the story is eventually picked up for fleshing out to ensure that we are not wasting time trying to fit in with some design that never existed in the first place. Unless you are someone who happily ignores whatever code anyone else ever writes anyway, and they do exist, you will no doubt spend at least some time trying to understand what’s gone before you. Hence being able to just say “journey code” has become a nice shorthand for “it wasn’t designed so feel free to chuck it away and do the right thing now we know what really needs doing”.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] I covered a different side-effect of this waterfall/agile impedance mismatch in “<a href="http://chrisoldwood.blogspot.co.uk/2016/10/confusion-over-waste.html">Confusion Over Waste</a>”.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-45058731384201837582017-01-24T22:20:00.001+00:002017-01-24T22:20:57.165+00:00Weak and Strong Idempotency<p><font face="Trebuchet MS">In my recent post “</font><a href="http://chrisoldwood.blogspot.co.uk/2016/10/put-vs-post-and-idempotency.html"><font face="Trebuchet MS">PUT vs POST and Idempotency</font></a><font face="Trebuchet MS">” I touched on the different <em>degrees</em> of idempotency. In retrospect I think it’s better to describe our idempotency guarantee as weak or strong. Either way this post looks at why you might choose one over the other.</font></p> <p><strong><font face="Trebuchet MS">Weak Idempotency</font></strong></p> <p><font face="Trebuchet MS">At one end of the spectrum we can choose to classify our unique operations with a single scalar value such as a large number, UUID or complex string value (with other embedded values like the date &amp; time). In this scenario we assume that a simple value represents the identity of the operation in much the same way that a reference (or pointer) is the unique address of a value in languages like C# and Java (or C++).</font></p> <p><font face="Trebuchet MS">In SQL terms your write with an idempotency check might look something like this:</font></p> <p><font face="Courier New">insert into Operations values(...) <br />where not exists <br />( <br />&#160; select 1 <br />&#160; from Operations <br />&#160; where OperationId &lt;&gt; @operationId <br />)</font></p> <p><font face="Trebuchet MS">For a document oriented database like MongoDB where you might store everything in a single object to achieve atomic updates you could do an equivalent <a href="https://docs.mongodb.com/manual/reference/method/db.collection.update/">find-and-modify-where</a> style query. If the data model requires multiple writes you might still be able to order them carefully to achieve the desired effect (See “<a href="http://chrisoldwood.blogspot.co.uk/2015/12/observable-state-versus-persisted-state.html">Observable State versus Persisted State</a>”).</font></p> <p><strong><font face="Trebuchet MS">Strong Idempotency</font></strong></p> <p><font face="Trebuchet MS">In contrast, at the other end, we define an idempotent operation based not only on some unique operation identifier but also the arguments to the operation itself, i.e. the request body in HTTP. In this instance we are not willing to accept that a replay can be distinguished solely by it’s handle but also wish to ensure the actual details match too, just like a “deep” value equality comparison.</font></p> <p><font face="Trebuchet MS">In SQL terms we potentially now have two steps, one to attempt the write and, if a conflict occurs, a subsequent read to validate it:</font></p> <font face="Trebuchet MS"><font face="Courier New">insert into Operations values(...) <br />where not exists <br />( <br />&#160; select 1 <br />&#160; from Operations <br />&#160; where OperationId &lt;&gt; @operationId <br />) <br /> <br />select * <br />from Operations <br />where OperationId = @operationId <br /></font> <br /></font> <p><font face="Trebuchet MS">If replays are rare (and I’d expect them to be) you should find this adds only a little extra overhead but also allows you to report broken requests properly.</font></p> <p><strong><font face="Trebuchet MS">Identity and Equality</font></strong></p> <p><font face="Trebuchet MS">Although it might seem like these weak and strong guarantees are directly analogous to the concept of identity and equality in normal programming there is a subtle difference.</font></p> <p><font face="Trebuchet MS">Whilst in the weak case we are happy for the unique operation identifier to act as our sole definition of equality, in the strong guarantee <em>both</em> the identifier and the input values [1] take part in the comparison. In a programming language we tend to choose either the reference based comparison or the value based approach but not both [2].</font></p> <p><strong><font face="Trebuchet MS">Failure Modes</font></strong></p> <p><font face="Trebuchet MS">In theory it should be adequate for us to take the simpler approach, and as the provider of a service it’s almost certainly slightly cheaper for us to do that. However the downside is that it makes it harder to diagnose missed updates caused by the service erroneously treating them as replays. But that’s just an edge case right?</font></p> <p><font face="Trebuchet MS">During development when your clients are coding against your API for the first time they may make a simple mistake and not correctly generate unique requests. This is not quite as uncommon as you may think.</font></p> <p><font face="Trebuchet MS">Another potential source of mistaken uniqueness can come from transforming requests, e.g. from a message queue. If you’re integrating with a legacy system what you may think is unique might in fact turn out to be an artefact of the small data set you sampled and it could turn out only to be <em>mostly</em> unique [3].</font></p> <p><font face="Trebuchet MS">Answering these kinds of support queries (“what happened to my request?“) can easily chip away at a team’s time. The more effort you put into making it harder for consumers to incorrectly use your API the less time you’ll need to support them.</font></p> <p><strong><font face="Trebuchet MS">Debug Switch</font></strong></p> <p><font face="Trebuchet MS">Although the need to verify a replay <em>should</em> be pretty infrequent you could easily wrap the validation logic in a feature switch so that development and test environments provide better diagnostics whilst production has less work to do. But you should measure before considering dropping such a low-cost feature.</font></p> <p><strong><font face="Trebuchet MS">Monitoring</font></strong></p> <p><font face="Trebuchet MS">What will help both the service and it’s consumers is if you can afford to capture when replays do occur so that the monitoring can alert if there appears to be a sudden rise in them. For example a client may go into an failure state that keeps replaying the same message over and over again. Alternatively the rise might suggest some configuration problem where upstream traffic is being duplicated. From a functional point of view the entire system could be behaving correctly but wasting resources in the process that could lead to a more catastrophic failure state.</font></p> <p><font face="Trebuchet MS">Although a client should never need to behave differently from a functional perspective when a replay occurs, the flip side to the service tracking the event is for the client to report it as well. They could do this if, say, the return code changes in the replay case. In HTTP for example there is <font face="Courier New">200 OK</font> and <font face="Courier New">201 Created</font> which could be used to tell the client what happened.</font></p> <p><font face="Trebuchet MS"><strong>Middle Ground</strong></font></p> <p><font face="Trebuchet MS">As I described in my previous post there can also be points between the two extremes. One of the problems with modern document oriented databases is that they have limits on the size of a single document. Depending on the anticipated depth of history this may exceed the practical document limit and therefore require some compromises to be made.</font></p> <p><font face="Trebuchet MS">If no false replays can be tolerated than perhaps the model needs splitting across documents which removes the ability to do simple atomic updates. If the history grows slowly and data retention is negotiable then maybe the fine details of much older operations can be discarded to make room for the new.</font></p> <p><font face="Trebuchet MS"><strong>Brave New World</strong></font></p> <p><font face="Trebuchet MS">The new era of databases continues to bring interesting challenges and trade offs due to their differing goals from a traditional big iron RDBMS. Pushing back on some requirements (e.g. “<a href="http://chrisoldwood.blogspot.co.uk/2015/12/dont-be-afraid-to-throw-away-data.html">Don’t Be Afraid to Throw Away Data</a>”) is essential if we are to balance their constraints with correctness.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] You may store server derived properties as well, like a “created at” timestamp. These are of course ignored in any comparison.</font></p> <p><font face="Trebuchet MS">[2] Of course breaking an invariant like the relationship between <font face="Courier New">Equals()</font> and <font face="Courier New">GetHashCode()</font> is a popular mistake which can make objects <a href="http://stackoverflow.com/questions/371328/why-is-it-important-to-override-gethashcode-when-equals-method-is-overridden">appear to go missing</a> in maps &amp; sets.</font></p> <p><font face="Trebuchet MS">[3] I know of one organisation where many people mistakenly believed their customer identifiers were unique, but we discovered they were only truly unique if you took their legacy back-office sources into account too.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-72201581016702338462017-01-17T20:53:00.001+00:002017-01-17T20:53:01.520+00:00The Cost of Long-Lived Feature Branches<p><font face="Trebuchet MS">Many moons ago I was working at large financial organisation on one of their back office systems. The ever increasing growth of the business meant that our system, whilst mostly distributed, was beginning to creak under the strain. I had already spent a month tracking down some out-of-memory problems in the monolithic orchestration service [1] and a corporate programme to reduce hardware meant we needed to move to a 3rd party compute platform to save costs by sharing hardware.</font></p> <p><strong><font face="Trebuchet MS">Branch Per <strong>Project</strong></font></strong></p> <p><font face="Trebuchet MS">The system was developed by a team (both on-shore and off-shore) numbering around 50 and the branching strategy was based around the many ongoing projects, each of which typically lasted many months. Any BAU work got done on the tip of whatever the last release branch was.</font></p> <p><font face="Trebuchet MS">While this allowed the team to hack around to their heart’s content without bumping into any other projects it also meant merge problems where highly likely when the time came. Most of the file merges were trivial (i.e. automatic) but there were more than a few awkward manual ones. Luckily this was also in a time <em>before</em> “</font><a href="http://chrisoldwood.blogspot.co.uk/2014/11/relentless-refactoring.html"><font face="Trebuchet MS">relentless refactoring</font></a><font face="Trebuchet MS">” too so the changes tended to be more surgical in nature.</font></p> <p><strong><font face="Trebuchet MS">Breaking up the Monolith</font></strong></p> <p><font face="Trebuchet MS">Naturally any project that involved taking one large essential service apart by splitting it into smaller, more distributable components was viewed as being very risky. Even though I’d managed to help get the UAT environment into a state where parallel running meant regressions were usually picked up, the project was still held at arms length like the others. The system had a history of delays in delivering, which was unsurprising given the size of the projects, and so naturally we would be tarred with the same brush.</font></p> <p><font face="Trebuchet MS">After spending a bit of time working out architecturally what pieces we needed and how we were going to break them out we then set about splitting the monolith up. The service really had two clearly distinct roles and some common infrastructure code which could be shared. Some of the orchestration logic that monitored outside systems could also be split out and instead of communicating in-process could just as easily spawn other processes to do the heavy lifting.</font></p> <p><strong><font face="Trebuchet MS">Decomposition Approach</font></strong></p> <p><font face="Trebuchet MS">The use of an enterprise-grade version control system which allows you to keep your changes isolated means you have the luxury of being able to take the engine to pieces, rebuild it differently and then deliver the new version. This was the essence of the project, so why not do that? As long as at the end you don’t have any pieces left over this probably appears to be the most efficient way to do it, and therefore was the method chosen by some of the team.</font></p> <p><font face="Trebuchet MS">An alternative approach, and the one I was more familiar with, was to extract components from the monolith and push them “down” the architecture so they turn into library components. This forces you to create abstractions and decouple the internals. You can then wire them back into both the old and new processes to help verify early on that you’ve not broken anything while you also test out your new ideas. This probably appears less efficient as you will be fixing up code you know you’ll eventually delete when the project is finally delivered.</font></p> <p><font face="Trebuchet MS">Of course the former approach is somewhat predicated on things never changing during the life of the project…</font></p> <p><strong><font face="Trebuchet MS">Man the Pumps!</font></strong></p> <p><font face="Trebuchet MS">Like all good stories we never got to the end of our project as a financial crisis hit which, through various business reorganisations, meant we had to drop what we were doing and make immediate plans to remediate the current version of the system. My protestations that the project we were currently doing would be <em>the answer</em> if we could just finish it, or even just pull in parts of it, were met with rejection.</font></p> <p><font face="Trebuchet MS">So we just had to drop months of work (i.e. leave it on the branch in stasis) and look for some lower hanging fruit to solve the impending performance problems that would be caused by a potential three times increase in volumes.</font></p> <p><font face="Trebuchet MS">When the knee jerk reaction began to subside the project remained shelved for the foreseeable future as a whole host of other requirements came flooding in. There was an increase in data volume but there were now other uncertainties around the existence of the entire system itself which meant it never got resurrected in the subsequent year, or the year after so I’m informed. This of course was still on our current custom platform and therefore no cost savings could be realised from the project work either.</font></p> <p><strong><font face="Trebuchet MS">Epilogue</font></strong></p> <p><font face="Trebuchet MS">This project was a real eye opener for me around how software is delivered on large legacy systems. Having come from a background where we delivered our systems in small increments as much out of a lack of tooling (a VCS product with no real branching support) as a need to work out what the users really wanted, it felt criminal to just waste <em>all</em> that effort.</font></p> <p><font face="Trebuchet MS">In particular I had tried hard to encourage my teammates to keep the code changes in as shippable state as possible. This wasn’t out of any particular foresight I might have had about the impending economic downturn but just out of the discomfort that comes from trying to change too much in one go.</font></p> <p><font face="Trebuchet MS">Ultimately if we had made each small refactoring on the branch next being delivered (ideally the trunk [2]) when the project was frozen we would already have been reaping the benefits from the work already done. Then, with most of the work safely delivered to production, the decision to finish it off becomes easier as the risk has largely been mitigated by that point. Even if a short hiatus was required for other concerns, picking the final work up later is still far easier or could itself even be broken down into smaller deliverable chunks.</font></p> <p><font face="Trebuchet MS">That said, it’s easy for us developers to criticise the actions of project managers when they don’t go our way. Given the system’s history and delivery record the decision was perfectly understandable and I know I wouldn’t want to be the one making them under those conditions of high uncertainty both inside and outside the business.</font></p> <p><font face="Trebuchet MS">Looking back it seems somewhat ridiculous to think that a team would split up and go off in different directions on different branches for many months with little real coordination and just hope that when the time comes we’d merge the changes [3], fix the niggles and release it. But that’s exactly how some larger teams did (and probably even still do) work.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] The basis of this work was written up in “</font><a href="http://www.chrisoldwood.com/articles/utilising-more-than-4gb.html"><font face="Trebuchet MS">Utilising More Than 4GB of Memory in a 32-bit Windows Process</font></a><font face="Trebuchet MS">”.</font></p> <p><font face="Trebuchet MS">[2] See “</font><a href="http://www.chrisoldwood.com/articles/branching-strategies.html"><font face="Trebuchet MS">Branching Strategies</font></a><font face="Trebuchet MS">”.</font></p> <p><font face="Trebuchet MS">[3] One developer on one project spend most of their time just trying to resolve the bugs that kept showing up due to semantic merge problems. They had no automated tests either to reduce the feedback loop and a build &amp; deployment to the test environment took in the order of hours.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-86808631195239440612016-12-12T22:36:00.001+00:002016-12-12T22:36:29.414+00:00Surprising Defaults – HttpClient ExpectContinue<p><font face="Trebuchet MS">One of the things you quickly discover when moving from building services on-premise to “the cloud” is quite how many more bits of wire and kit suddenly sit between you and your consumer. Performance-wise this already elongated network path can then be further compounded when the framework you’re using invokes unintuitive behaviour by default [1].</font></p> <p><strong><font face="Trebuchet MS">The Symptoms</font></strong></p> <p><font face="Trebuchet MS">The system was a new REST API built in C# on the .Net framework (4.6) and hosted in the cloud with AWS. This AWS endpoint was then further fronted by Akamai for various reasons. The initial consumer was an on-premise adaptor (also written in C#) which itself had to go through an enterprise grade web proxy to reach the outside world.</font></p> <p><font face="Trebuchet MS">Naturally monitoring was added in fairly early on so that we could start to get a feel for how much added latency moving to the cloud would bring. Our first order approximation to instrumentation allowed us to tell how long the HTTP requests took to handle along with a breakdown of the major functions, e.g. database queries and 3rd party requests. Outside the service we had some remote monitoring too that could tell us the performance from a more customer-like position.</font></p> <p><font face="Trebuchet MS">When we integrated with the 3rd party service some poor performance stats caused us to look closer into our metrics. The vast majority of big delays were outside our control, but it also raised some other questions as the numbers didn’t quite add up. We had expected the following simple formula to account for virtually all the time:</font></p> <p><font face="Trebuchet MS">HTTP Request Time ~= 3rd Party Time + Database Time</font></p> <p><font face="Trebuchet MS">However we were seeing a 300 ms discrepancy in many (but not <em>all</em>) cases. It was not our immediate concern as there was bigger fish to fry but some extra instrumentation was added to the OWIN pipeline and we did a couple of quick local profile runs to look out for anything obviously out of place. The finger seemed to point to time lost somewhere in the Nancy part of the pipeline, but that didn’t entirely make sense at the time so it was mentally filed away and we moved on.</font></p> <p><strong><font face="Trebuchet MS">Serendipity Strikes</font></strong></p> <p><font face="Trebuchet MS">Whilst talking to the 3rd party about our performance woes with their service they came back to us and asked if we could stop sending them a “<font face="Courier New">Expect: 100-Continue</font>” header in our HTTP requests.</font></p> <p><font face="Trebuchet MS">This wasn’t something anyone in the team was aware of and as far as we could see from the various RFCs and blog posts it was something “naturally occurring” on the internet. We also didn’t know if it was us adding it or one of the many proxies in between us and them.</font></p> <p><font face="Trebuchet MS">We discovered how to turn it off, and did, but it made little difference to the performance problems we had with them, which were in the order of seconds, not milliseconds. Feeling uncomfortable about blindly switching settings off without really understanding them we reverted the change.</font></p> <p><font face="Trebuchet MS">The mention of this header also cropped up when we started investigating some errors we were getting from Akamai that seemed to be more related to a disparity in <a href="https://community.akamai.com/community/web-performance/blog/2016/09/07/how-to-setup-aws-origin-to-work-with-akamai">idle connection timeouts</a>.</font></p> <p><font face="Trebuchet MS">Eventually, as we learned more about this mysterious header someone in the team put two-and-two together and realised this was possibly where our missing time was going too.</font></p> <p><strong><font face="Trebuchet MS">The Cause</font></strong></p> <p><font face="Trebuchet MS">Our REST API uses <font face="Courier New">PUT</font> requests to add resources and it appears that the default behaviour of the .Net <font face="Courier New">HttpClient</font> class is to enable the sending of this “<font face="Courier New">Expect: 100-Continue</font>” header for those types of requests. Its purpose is to tell the server that the headers have been sent but that it will delay sending the body until it receives a <font face="Courier New">100-Continue</font> style response. At that point the client sends the body, the server can then process the entire request and the response is handled by the client as per normal.</font></p> <p><font face="Trebuchet MS">Yes, that’s right, it splits the request up so that it takes two round trips instead of one!</font></p> <p><font face="Trebuchet MS">Now you can probably begin to understand why our request handling time appeared elongated and why it also appeared to be consumed somewhere within the Nancy framework. The request processing is started and handled by the OWN middleware as that only depends on the headers, it then enters Nancy which finds a handler, and so requests the body in the background (asynchronously). When it finally arrives the whole request is then passed to our Nancy handler just as if it had been sent all as a single chunk.</font></p> <p><strong><font face="Trebuchet MS">The Cure</font></strong></p> <p><font face="Trebuchet MS">When you google this problem with relation to .Net you’ll see that there are a couple of options here. We were slightly nervous about choosing the nuclear option (<a href="http://stackoverflow.com/questions/14063327/how-to-disable-the-expect-100-continue-header-in-httpwebrequest-for-a-single">setting it globally on the <font face="Courier New">ServicePointManager</font></a>) and instead added an extra line into our <font face="Courier New">HttpClient</font> factory so that it was localised:</font></p> <p><font face="Courier New">var client = new HttpClient(...); <br />... <br />client.DefaultRequestHeaders.ExpectContinue = false;</font></p> <p><font face="Trebuchet MS">We re-deployed our services, checked our logs to ensure the header was no longer being sent, and then checked the various metrics to see if the time was now all accounted for, and it was.</font></p> <p><strong><font face="Trebuchet MS">Epilogue</font></strong></p> <p><font face="Trebuchet MS">In hindsight this all seems fairly obvious, at least, once you know what this header is supposed to do, and yet none of the people in my team (who are all pretty smart) joined up the dots right away. When something like this goes astray I like to try and make sense of why we didn’t pick it up as quickly as perhaps we should have.</font></p> <p><font face="Trebuchet MS">In the beginning there were so many new things for the team to grasp. The difference in behaviour between our remote monitoring and on-premise adaptor was assumed to be one of infrastructure especially when we had already battled the on-premise web proxy a few times [2]. We saw so many other headers in our requests that we never added so why would we assume this one was any different (given <em>none</em> of us had run across it before)?</font></p> <p><font face="Trebuchet MS">Given the popularity and maturity of the Nancy framework we surmised that no one would use it if there was the kind of performance problems we were seeing, so once again were confused as to how the time could appear to be lost inside it. Although we were all aware of what the <font face="Courier New">async</font>/<font face="Courier New">await</font> construct does none of us had really spent any serious time trying to track down performance anomalies in code that used it so liberally and so once again we had difficulties understanding perhaps what the tool was really telling us.</font></p> <p><font face="Trebuchet MS">Ultimately though the default behaviour just seems so utterly wrong that none of use could imagine the out-of-the-box settings would cause the <font face="Courier New">HttpClient</font> to behave this way. By choosing this default we are in essence optimising <font face="Courier New">PUT</font> requests for the scenario where the body does not need sending, which we all felt is definitely the exception not the norm. Aside from large file uploads or massive write contention we were struggling to come up with a plausible use case.</font></p> <p><font face="Trebuchet MS">I don’t know what forces caused this decision to be made as I clearly wasn’t there and I can’t find any obvious sources that might explain it either. The internet and HTTP has evolved so much over the years that it’s possible this behaviour provides the best compatibility with web servers out-of-the-box. </font><font face="Trebuchet MS">My own HTTP experience only covers the last few years along with few more around the turn of the millennium, but my colleagues easily cover the decades I’m missing so I don’t feel I’m missing anything obvious.</font></p> <p><font face="Trebuchet MS">Hopefully some kind soul will use the comments section to link to the rationale so we can all get a little closure on the issue.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] Violating </font><a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment"><font face="Trebuchet MS">The Principle of Least Astonishment</font></a><font face="Trebuchet MS"> for configuration settings was something I covered more generally before in “</font><a href="http://chrisoldwood.blogspot.co.uk/2012/11/sensible-defaults.html"><font face="Trebuchet MS">Sensible Defaults</font></a><font face="Trebuchet MS">”.</font></p> <p><font face="Trebuchet MS">[2] See “</font><a href="http://chrisoldwood.blogspot.co.uk/2016/05/the-curse-of-ntlm-based-http-proxies.html"><font face="Trebuchet MS">The Curse of NTLM Based HTTP Proxies</font></a><font face="Trebuchet MS">”.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-69648250330824212642016-11-17T20:48:00.001+00:002016-11-17T20:48:11.964+00:00Overly Prescriptive Tests<p><font face="Trebuchet MS">In my recent post “<a href="http://chrisoldwood.blogspot.co.uk/2016/11/tautologies-in-tests.html">Tautologies in Tests</a>” I adapted one of Einstein’s apocryphal sayings and suggested that tests should be “as precise as possible, but not too precise”. But what did I mean by that? How can you be <em>too</em> precise, in fact isn’t that the point?</font></p> <p><strong><font face="Trebuchet MS">Mocking</font></strong></p> <p><font face="Trebuchet MS">One way is to be overly specific when tracking the interactions with mocks. It’s very easy when using a mocking framework to go overboard with your expectations, just because you can. My personal preference (detailed before in “<a href="http://chrisoldwood.blogspot.co.uk/2011/04/mock-to-test-outcome-not-implementation.html">Mock To Test the Outcome, Not the Implementation</a>”) is to keep the details of any interactions loose, but be specific about the <em>outcomes</em>. In other words what matters most is (usually) the observable behaviour, not necessarily how it’s achieved.</font></p> <p><font face="Trebuchet MS">For example, rather than set-up detailed instructions on a mock that cover all the expected parameters and call counts I’ll mostly use simple hand-crafted mocks [1] where the method maps to a delegate where I’ll capture only the salient details. Then in the assertions at the end I verify whatever I need to in the same style as the rest of the test. Usually though the canned response is test case specific and so rarely needs any actual logic.</font></p> <p><font face="Trebuchet MS">In essence what I’m creating some people prefer to call stubs as they reserve the term “mocks” for more meatier test fakes that record interactions for you. I’d argue that using the more complex form of mock is largely unnecessary and will hurt in the long run. To date (anecdotally speaking) I’ve wasted too much time “fixing” broken tests that overused mocks by specifying every little detail and were never written to give the implementation room to manoeuvre, e.g. during refactoring. In fact an automated refactoring tool is mandatory on code like this because the methods are referenced in so many tests it would take forever to fix-up manually.</font></p> <p><font face="Trebuchet MS">I often feel that some of the interactions with dependencies I’ve seen in the past have felt analogous to testing private methods. Another of my previous posts that was inspired by mocking hell is “<a href="http://chrisoldwood.blogspot.co.uk/2014/11/dont-pass-factories-pass-workers.html">Don’t Pass Factories, Pass Workers</a>”. Naturally there is a fine line here and maybe I’ve just not seen enough of it done well to appreciate how this particular tool can be used effectively.</font></p> <p><font face="Trebuchet MS"><strong>White-Box Testing</strong>&#160;</font></p> <p><font face="Trebuchet MS">The other form of overly specific test I’ve seen comes from what I believe is relying too much on a white-box testing approach so that the tests express the output <em>exactly</em>.</font></p> <p><font face="Trebuchet MS">The problem with example based tests is that they are often taken literally, which I guess is kind of the point, but as software engineers we should try and see passed the rigid examples and verify the underlying behaviour instead, which is what we’re really after.</font></p> <p><font face="Trebuchet MS">For example, consider a pool of numbers [2] up to some predefined limit, say, 10. A naïve approach to the problem might test the pool by asserting a very specific sequence, i.e. the starting one:</font></p> <p><font face="Courier New">[Test] <br />public void returns_sequence_up_to_limit() <br />{ <br />&#160; var pool = new NumberPool(10); <br />&#160; var expected = new[] { 1, 2, 3, ... , 9, 10 }; <br /> <br />&#160; for (var number in expected) <br />&#160;&#160;&#160; Assert.That(pool.Acquire(), Is.EqualTo(number)); <br />}</font></p> <p><font face="Trebuchet MS">From a white-box testing approach we can look inside the <font face="Courier New">NumberPool</font> and probably see that it’s initially generating numbers using the <font face="Courier New">++</font> operator. The implementation might eagerly generate that sequence in the constructor, add them to the end of a queue, and then divvy out the front of the queue.</font></p> <p><font face="Trebuchet MS">From a “programmer’s test” point of view (aka unit test) it does indeed verify that, if my expectation is that the implementation should return the exact sequence 1..10, then it will. But how useful is that for the maintainer of this code? I’d argue that we’ve over-specified the way this unit should be allowed to behave.</font></p> <p><font face="Trebuchet MS"><strong>Verify Behaviours</strong></font></p> <p><font face="Trebuchet MS">And that, I think, lies at that heart of the problem. For tests to be truly effective they should not describe <em>exactly</em> what they do, but should describe how they need to behave. Going back to our example above the <font face="Courier New">NumberPool</font> class does not need to return the exact sequence 1..10, it needs to satisfy some looser constraints, such as not returning a duplicate value (until re-acquired), and limiting the range of numbers to between 1 and 10.</font></p> <p><font face="Courier New">[Test] <br />public void sequence_will_be_unique() <br />{ <br />&#160; var pool = new NumberPool(10); <br />&#160; var sequence = new List&lt;int&gt;(); <br /> <br />&#160; for (var i in Enumerable.Range(1, 10)) <br />&#160;&#160;&#160; sequence.Add(pool.Acquire()); <br /> <br />&#160; Assert.That(sequence.Distinct().Count(), <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Is.EqualTo(10));&#160; <br />}</font></p> <p><font face="Courier New">[Test] <br />public void sequence_only_contains_one_to_limit() <br />{ <br />&#160; var pool = new NumberPool(10); <br />&#160; var sequence = new List&lt;int&gt;(); <br /> <br />&#160; for (var i in Enumerable.Range(1, 10)) <br />&#160;&#160;&#160; sequence.Add(pool.Acquire()); <br /> <br />&#160; Assert.That(sequence.Where(n =&gt; (n &lt; 1) || (n &gt; 10)), <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Is.Empty); <br />}</font></p> <p><font face="Trebuchet MS">With these two tests we are free to change the implementation to generate a random sequence in the constructor instead if we wanted, and they would still pass, because it conforms to the looser, albeit still well defined, behaviour. (It may have unpredictable performance characteristics but that is a different matter.)</font></p> <p><font face="Trebuchet MS">Once again we are beginning to enter the realm of <a href="http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html">property based testing</a> which forces us to think harder about what behaviours our code exhibits rather than what it should do in one single scenario.</font></p> <p><font face="Trebuchet MS">This does not mean there is no place for tests that take a specific set of inputs and validate the result against a known set of outputs. </font><font face="Trebuchet MS">On the contrary they are an excellent starting point for thinking about what <em>the real test</em> should do. They are also </font><font face="Trebuchet MS">important in scenarios where you need some smoke tests that “kick the tyres” or you are naturally handling a very specific scenario.</font></p> <p><font face="Trebuchet MS"><strong>Indicative Inputs</strong></font></p> <p><font face="Trebuchet MS">Sometimes we don’t intend to make our test look specific but it just turns out that way to the future reader. For example in our <font face="Courier New">NumberPool</font> tests above what is the significance of the number “10”? Hopefully in this example it is fairly obvious that it is an arbitrary value as the test names only talk about “a limit”. But what about a test for code that handles, say, an HTTP error?</font></p> <p><font face="Courier New">[Test] <br />public void client_throws_when_service_unavailable() <br />{ <br />&#160; using (FakeServer.Returns(InternalServerError)) <br />&#160; { <br />&#160;&#160;&#160; var client = new RestClient(. . .); <br /> <br />&#160;&#160;&#160; Assert.That(client.SendRequest(. . .), <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Throws.InstanceOf&lt;RequestException&gt;()); <br />&#160; } <br />}</font></p> <p><font face="Trebuchet MS">In this test we have a mock (nay stub) HTTP server that will return a non-<font face="Courier New">2XX</font> style result code. Now, what is the significance of the <font face="Courier New">InternalServerError</font> result code returned by the stub? Is it a specific result code we’re handling here, or an indicative one in the <font face="Courier New">5XX</font> range? The test name uses the term “service unavailable” which maps to the more specific HTTP code <font face="Courier New">503</font>, so is this in fact a bug in the code or test?</font></p> <p><font face="Trebuchet MS">Unless the original author is around to ask (and even remembers) we don’t know. We can surmise what they probably meant by inspecting the production code and seeing how it processes the result code (e.g. a direct comparison or a range based one). From there we might choose to see how we can avoid the ambiguity by refactoring the test. In the case where <font face="Courier New">InternalServerError</font> is merely indicative we can use a suitably named constant instead, e.g.</font></p> <p><font face="Courier New">[Test] <br />public void throws_when_service_returns_5xx_code() <br />{ <br />&#160; const int CodeIn5xxRange = InternalServerError; <br /> <br />&#160; using (FakeServer.Returns(CodeIn5xxRange)) <br />&#160; { <br />&#160;&#160;&#160; var client = new RestClient(. . .); <br /> <br />&#160;&#160;&#160; Assert.That(client.SendRequest(. . .), <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Throws.InstanceOf&lt;RequestException&gt;()); <br />&#160; } <br />}</font></p> <p><font face="Trebuchet MS">A clue that there is a disconnect is when the language used in the test name isn’t correctly reflected in the test body itself. So if the name isn’t specific then nor should the test be, but also vice-versa, if the name is specific then expect the test to be. A corollary to this is that if your test name is vague don’t surprised when the test itself turns out equally vague.</font></p> <p><font face="Trebuchet MS"><strong>Effective Tests</strong></font></p> <p><font face="Trebuchet MS">For a suite of tests to be truly effective you need them to remain quietly in the background until you change the code in a way that raises your awareness around some behaviour you didn’t anticipate. The fact that you didn’t anticipate it means that you’ll be relying heavily on the test rather than the code you just changed to make sense of the original intended behaviour.</font></p> <p><font face="Trebuchet MS">When it comes under the spotlight (fails) a test needs to convince you that it was well thought out and worthy of your consideration. To be effective a guard dog has to learn the difference between friend and foe and when we write tests we need to learn how to leave enough room for safe manoeuvring without forgetting to bark loudly when we exceed our remit.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] When you keep your interfaces simple and focused this is pretty easy given how much a modern IDE can generate for you when using a statically typed language.</font></p> <p><font face="Trebuchet MS">[2] This example comes from a real one where the numbers where identifiers used to distinguish compute engines in a grid.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com1tag:blogger.com,1999:blog-6628985022531866193.post-56578819730646438672016-11-15T19:01:00.001+00:002016-11-15T19:01:25.696+00:00In The Toolbox – Season Two<p><font face="Trebuchet MS">As I pointed out in my blog post that collates <a href="http://chrisoldwood.blogspot.co.uk/2014/08/in-toolbox-season-one.html">Season One</a> of my <em>In The Toolbox</em> <a href="https://accu.org/index.php/journal">C Vu</a> column I never intended to write more than a couple of introductory articles before handing it over for others to share their experiences. Yet now, three years later, I’m still plugging away at it and Season Three is already in the making with a couple of episodes already under my belt.</font></p> <p><font face="Trebuchet MS">Just as before I also strongly advise you to become a member of the <a href="https://accu.org">ACCU</a> so you can get this, plus loads of much better content, which may or may not be published online by their respective authors. As I write this post it’s still only <a href="https://accu.org/index.php/membership">a measly £45 per year</a> and is one of the last remaining printed journals about programming.</font></p> <p><font face="Trebuchet MS">Anyway, here are links and summaries for episodes 7 through 12.</font></p> <p><font face="Trebuchet MS">7: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-feature-tracking.html">Feature Tracking</a></font></p> <p><font face="Trebuchet MS">We have so many ideas for our products but only so many hours in the day to develop them. Sometimes all it needs is a simple text file in the repo, whilst bigger projects seem to demand an enterprise-grade solution like JIRA.</font></p> <p><font face="Trebuchet MS">8: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-taming-the-inbox.html">Taming the Inbox</a></font></p> <p><font face="Trebuchet MS">Email is still the predominant means of [a]synchronous communication for many organisations and the barrage of messages need to be triaged if we stand any hope of separating the wheat from the chaff.</font></p> <p><font face="Trebuchet MS">9: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-the-developers-sandbox.html">The Developer’s Sandbox</a></font></p> <p><font face="Trebuchet MS">As programmers we need a safe environment in which to write and test our code, free from the distractions going on around us. When running the tests it should not be at the mercy of other developers running tests at the same time as us; first and foremost we start in isolation, if we can.</font></p> <p><font face="Trebuchet MS">10: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-dictionary-and-thesaurus.html">Dictionary &amp; Thesaurus</a></font></p> <p><font face="Trebuchet MS">One of the hardest problems in computer science is naming and yet two of the oldest tools used to solve this problem often lay dormant on the programmer’s bookshelf.</font></p> <p><font face="Trebuchet MS">11: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-finding-text.html">Finding Text</a></font></p> <p><font face="Trebuchet MS">It’s a simple question: how do you find a piece of text? And yet there is a dizzying array of choices available that depend heavily on what’s accessible at the time and where and how that elusive text is stored.</font></p> <p><font face="Trebuchet MS">12: <a href="http://www.chrisoldwood.com/articles/in-the-toolbox-whiteboards.html">Whiteboards</a></font></p> <p><font face="Trebuchet MS">In the move to go digital the humble whiteboard has been pushed aside, which is disappointing as it’s still probably the best design tool available. It also has many other uses than drawing pictures of boxes, drums and cylinders.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-34515564922465007792016-11-14T20:47:00.001+00:002016-11-14T20:47:58.256+00:00Automated Integration Testing with TIBCO<p><font face="Trebuchet MS">In the past few years I’ve worked on a few projects where TIBCO has been the message queuing product of choice within the company. Naturally being a test-oriented kind of guy I’ve used unit and component tests for much of the donkey work, but initially had to shy away from writing any automated integration tests due to the inherent difficulties of getting the system into a known state in isolation.</font></p> <p><strong><font face="Trebuchet MS">Organisational Barriers</font></strong></p> <p><font face="Trebuchet MS">For any automated integration tests to run reliably we need to control the <em>whole</em> environment, which ideally is our development workstations but also our CI build environment (see “<a href="http://www.chrisoldwood.com/articles/in-the-toolbox-the-developers-sandbox.html">The Developer’s Sandbox</a>”). The main barriers to this with a commercial product like TIBCO are often technological, but also more often than not, organisational too.</font></p> <p><font face="Trebuchet MS">In my experience middleware like this tends to be proprietary, very expensive, and owned within the organisation by a dedicated team. They will configure the staging and production queues and manage the fault-tolerant servers, which is probably what you’d expect as you near production. A more modern DevOps friendly company would recognise the need to allow teams to test <em>internally</em> first and would help them get access to the product and tools so they can build their test scaffolding that provides the initial feedback loop.</font></p> <p><font face="Trebuchet MS">Hence just being given the client access libraries to the product is not enough, we need a way to bring up and tear down the service endpoint, in isolation, so that we can test connectivity and failover scenarios and message interoperability. We also need to be able develop and test our logic around poisoned messages and dead-letter queues. And all this needs to be automatable so that as we develop and refactor we can be sure that we’ve not broken anything; manually testing this stuff is not just not scalable in a shared test environment at the pace modern software is developed.</font></p> <p><font face="Trebuchet MS">That said, the <a href="https://docs.tibco.com/products/tibco-enterprise-message-service-6-3-0">TIBCO EMS SDK</a> I’ve been working with (v6.3.0) has all the parts I needed to do this stuff, albeit with some workarounds to avoid needing to run the tests with administrator rights which we’ll look into later.</font></p> <p><font face="Trebuchet MS">The only other thorny issue is licensing. You would hope that software product companies would do their utmost to get developers on their side and make it easy for them to build and test their wares, but it is often hard to get clarity around how the product can be used outside of the final production environment. For example trying to find out if the TIBCO service can be run on a developer’s workstation or in a cloud hosted VM solely for the purposes of running some automated tests has been a somewhat arduous task.</font></p> <p><font face="Trebuchet MS">This may not be solely the fault of the underlying product company, although the old fashioned licensing agreements often do little to distinguish production and <em>modern</em> development use [1]. No, the real difficulty is finding the right person within the client’s company to talk to about such matters. Unless <em>they</em> are au fait with the role modern automated integrated testing takes place in the development process you will struggle to convince them your intended use is in the interests of the 3rd party product, not stealing revenue from them.</font></p> <p><font face="Trebuchet MS">Okay, time to step down from the soap box and focus on the problems we <em>can</em> solve…</font></p> <p><strong><font face="Trebuchet MS">Hosting TIBEMSD as a Windows Service</font></strong></p> <p><font face="Trebuchet MS">From an automated testing perspective what we need access to is the <font face="Courier New">TIBEMSD.EXE</font> console application. This provides us with one or more TIBCO message queues that we can host on our local machine. Owning thing process means we can therefore create, publish to and delete queues on demand and therefore tightly control the environment.</font></p> <p><font face="Trebuchet MS">If you only want to do basic integration testing around the sending and receiving of messages you can configure it as a Windows service and just leave it running in the background. Then your tests can just rely on it always being there like a local database or the file-system. The build machine can be configured this way too.</font></p> <p><font face="Trebuchet MS">Unfortunately because it’s a console application and not written to be hosted as a service (at least v6.3 isn’t), you need to use a shim like <font face="Courier New"><a href="https://support.microsoft.com/en-gb/kb/137890">SRVANY.EXE</a></font> from the Windows 2003 Resource Kit or something more modern like <a href="http://nssm.cc/">NSSM</a>. These tools act as an adaptor to the console application so that the <a href="https://en.wikipedia.org/wiki/Service_Control_Manager">Windows SCM</a> can control them.</font></p> <p><font face="Trebuchet MS">One thing to be careful of when running <font face="Courier New">TIBEMSD</font> in this way is that it will stick its data files in the CWD (Current Working Directory), which for a service is <font face="Courier New">%SystemRoot%\System32</font>, unless you configure the shim to change it. Putting them in a separate folder makes them a little more obvious and easier to delete when having a clear out [2].</font></p> <p><strong><font face="Trebuchet MS">Running TIBEMSD On Demand</font></strong></p> <p><font face="Trebuchet MS">Running the TIBCO server as a service makes certain kinds of tests easier to write as you don’t have to worry about starting and stopping it, unless that’s exactly the kinds of test you want to write.</font></p> <p><font face="Trebuchet MS">I’ve found it’s all too easy when adding new code or during a refactoring to accidentally break the service so that it doesn’t behave as intended when the network goes up and down, especially when you’re trying to handle poisoned messages.</font></p> <p><font face="Trebuchet MS">Hence I prefer to have the <font face="Courier New">TIBEMSD.EXE</font> binary included in the source code repository, in a known place so that it can be started and stopped on demand to verify the connectivity side is working properly. For those classes of integration tests where you just need it to be running you can add it to your fixture-level setup and even keep it running across fixtures to ensure the tests running at an adequate pace.</font></p> <p><font face="Trebuchet MS">If, like me, you don’t run as an Administrator all the time (or use elevated command prompts by default) you will find that <font face="Courier New">TIBEMSD</font> doesn’t run out-of-the-box in this way. Fortunately it’s easy to overcome these two issues and run in a LUA (Limited User Account).</font></p> <p><strong><font face="Trebuchet MS">Only Bind to the Localhost</font></strong></p> <p><font face="Trebuchet MS">One of the problems is that by default the server will try and listen for remote connections from anywhere which means it wants a hole in the firewall for its default port. This of course means you’ll get <em>that</em> firewall popup dialog which is annoying when trying to automate stuff. Whilst you could grant it permission with a one-off <font face="Courier New">NETSH ADVFIREWALL</font> command I prefer components in test mode to not need any special configuration if at all possible.</font></p> <p><font face="Trebuchet MS">Windows will allow sockets that only listen for connections from the local host to avoid generating the annoying firewall popup dialog (and this was finally extended <a href="http://stackoverflow.com/questions/4019466/httplistener-access-denied#">to include HTTP</a> too). However we need to tell the TIBCO server to do just that, which we can achieve by creating a trivial configuration file (e.g. <font face="Courier New">localhost.conf</font>) with the following entry:</font></p> <p><font face="Courier New">listen=tcp://127.0.0.1:7222</font></p> <p><font face="Trebuchet MS">Now we just need to start it with the <font face="Courier New">–conf</font> switch:</font></p> <p><font face="Courier New">&gt; tibemsd.exe -config localhost.conf</font></p> <p><strong><font face="Trebuchet MS">Suppressing the Need For Elevation</font></strong></p> <p><font face="Trebuchet MS">So far so good but our other problem is that when you start <font face="Courier New">TIBEMSD</font> it wants you to elevate its permissions. I presume this is a legacy thing and there may be some feature that really needs it but so far in my automated tests I haven’t hit it.</font></p> <p><font face="Trebuchet MS">There are a number of ways to control elevation for legacy software that doesn’t have a manifest, like using an external one, but <font face="Courier New">TIBEMSD</font> does and that takes priority. Luckily for us there is a solution in the form of the <font face="Courier New">__COMPAT_LAYER</font> environment variable [3]. Setting this, either through a batch file or within our test code, supresses the need to elevate the server and it runs happily in the background as a normal user, e.g.</font></p> <p><font face="Courier New">&gt; set __COMPAT_LAYER=RunAsInvoker <br />&gt; tibemsd.exe -config localhost.conf</font></p> <p><strong><font face="Trebuchet MS">Spawning TIBEMSD From Within a Test</font></strong></p> <p><font face="Trebuchet MS">Once we know how to run <font face="Courier New">TIBEMSD</font> without it causing any popups we are in a position to do that from within an automated test running as any user (LUA), e.g. a developer or the build machine.</font></p> <p><font face="Trebuchet MS">In C#, the language where I have been doing this most recently, we can either hard-code a <em>relative</em> path [4] to where <font face="Courier New">TIBEMSD.EXE</font> resides within the repo, or read it from the test assembly’s <font face="Courier New">app.config</font> file to give us a little more flexibility.</font></p> <p><font face="Courier New">&lt;appSettings&gt; <br />&#160; &lt;add key=”tibemsd.exe” <br />&#160;&#160;&#160;&#160;&#160;&#160; value=”..\..\tools\TIBCO\tibemsd.exe” /&gt; <br />&#160; &lt;add key=”conf_file” <br />&#160;&#160;&#160;&#160;&#160;&#160; value=”..\..\tools\TIBCO\localhost.conf” /&gt; <br />&lt;/appSettings&gt;</font></p> <p><font face="Trebuchet MS">We can also add our special <font face="Courier New">.conf</font> file to the same folder and therefore find it in the same way. Whilst we could generate it on-the-fly it never changes so I see little point in doing this extra work.</font></p> <p><font face="Trebuchet MS">Something to be wary of if you’re using, say, NUnit to write your integration tests is that it (and ReSharper) can copy the test assemblies to a random location to aid in insuring your tests have no accidental dependencies. In this instance we do, and a rather large one at that, so we need the relative distance between where the test assemblies are built and run (Xxx<font face="Courier New">IntTests\bin\Debug</font>) and the <font face="Courier New">TIBEMSD.EXE</font> binary to remain fixed. Hence we need to disable this copying behaviour with the <font face="Courier New"><a href="http://stackoverflow.com/questions/26337677/how-to-disabled-shadow-copy-in-nunit-from-console-exe">/noshadow</a></font> switch (or “Tools | Unit Testing | Shadow-copy assemblies being tested” in ReSharper).</font></p> <p><font face="Trebuchet MS">Given that we know where our test assembly resides we can use <font face="Courier New">Assembly.GetExecutingAssembly()</font> to create a fully qualified path from the relative one like so:</font></p> <p><font face="Courier New">private static string GetExecutingFolder() <br />{ <br />&#160; var codebase = Assembly.GetExecutingAssembly() <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; .CodeBase; <br />&#160; var folder = Path.GetDirectoryName(codebase); <br />&#160; return new Uri(folder).LocalPath; <br />} <br />. . . <br />var thisFolder = GetExecutingFolder(); <br />var tibcoFolder = “..\..\tools\TIBCO”; <br />var serverPath = Path.Combine( <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; thisFolder, tibcoFolder, “tibemsd.exe”); <br />var configPath = Path.Combine( <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; thisFolder, tibcoFolder, “localhost.conf”);</font></p> <p><font face="Trebuchet MS">Now that we know where the binary and config lives we just need to stop the elevation by setting the right environment variable:</font></p> <p><font face="Courier New">Environment.SetEnvironmentVariable(&quot;__COMPAT_LAYER&quot;, &quot;RunAsInvoker&quot;);</font></p> <p><font face="Trebuchet MS">Finally we can start the <font face="Courier New">TIBEMSD.EXE</font> console application in the background (i.e. no distracting console window) using <font face="Courier New">Diagnostics.Process</font>:</font></p> <p><font face="Courier New">var process = new System.Diagnostics.Process <br />{ <br />&#160; StartInfo = new ProcessStartInfo(path, args) <br />&#160; { <br />&#160;&#160;&#160; UseShellExecute = false, <br />&#160;&#160;&#160; CreateNoWindow = true, <br />&#160; } <br />}; <br />process.Start();</font></p> <p><font face="Trebuchet MS">Stopping the daemon involves calling <font face="Courier New">Kill()</font>. There are more graceful ways of remotely <a href="http://stackoverflow.com/a/15281070/106119">stopping a console application</a> which you can try first, but <font face="Courier New">Kill()</font> is always the fall-back approach and of course the TIBCO server has been designed to survive such abuse.</font></p> <p><font face="Trebuchet MS">Naturally you can wrap this up with the Dispose pattern so that your test code can be self-contained:</font></p> <p><font face="Courier New">// Arrange <br />using (RunTibcoServer()) <br />{ <br />&#160; // Act <br />} <br /> <br />// Assert</font></p> <p><font face="Trebuchet MS">Or if you want to amortise the cost of starting it across your tests you can use the fixture-level set-up and tear down:</font></p> <p><font face="Courier New">private IDisposable _server; <br /> <br />[FixtureSetUp] <br />public void GivenMessageQueueIsAvailable() <br />{ <br />&#160; _server = RunTibcoServer(); <br />} <br /> <br />[FixtureTearDown] <br />public void StopMessageQueue() <br />{ <br />&#160; _server?.Dispose(); <br />&#160; _server = null; <br />}</font></p> <p><font face="Trebuchet MS">One final issue to be aware of, and it’s a common one with integration tests like this which start a process on demand, is that the server might still be running unintentionally across test runs. This can happen when you’re debugging a test and you kill the debugger whilst still inside the test body. The solution is to ensure that the server definitely isn’t already running before you spawn it, and that can be done by killing any existing instances of it:</font></p> <p><font face="Courier New">Process.GetProcessesByName(“tibemsd”) <br />&#160;&#160;&#160;&#160;&#160;&#160; .ForEach(p =&gt; p.Kill());</font></p> <p><font face="Trebuchet MS">Naturally this is a sledgehammer approach and assumes you aren’t using separate ports to run multiple disparate instances, or anything like that.</font></p> <p><strong><font face="Trebuchet MS">Other Gottchas</font></strong></p> <p><font face="Trebuchet MS">This gets us over the biggest hurdle, control of the server process, but there are a few other little things worth noting.</font></p> <p><font face="Trebuchet MS">Due to the asynchronous nature and potential for residual state I’ve found it’s better to drop and re-create any queues at the start of each test to flush them. I also use the <font face="Courier New">Assume.That</font> construct in the arrangement to make it doubly clear I expect the test to start with empty queues.</font></p> <p><font face="Trebuchet MS">Also if you’re writing tests that cover background connect and failover be aware that the TIBCO reconnection logic doesn’t trigger unless you have <em>multiple</em> servers configured. Luckily you can specify the same server twice, e.g.</font></p> <p><font face="Courier New">var connection= “tcp://localhost,tcp://localhost”;</font></p> <p><font face="Trebuchet MS">If you expect your server to shutdown gracefully, even in the face of having no connection to the queue, you might find that calling <font face="Courier New">Close()</font> on the session and/or connection blocks whilst it’s trying to reconnect (at least in EMS v6.3 it does). This might not be an expected production scenario, but it can hang your tests if something goes awry, hence I’ve used a slightly distasteful workaround where the call to <font face="Courier New">Close()</font> happens on a separate thread with a timeout:</font></p> <p><font face="Courier New">Task.Run(() =&gt; _connection.Close()).Wait(1000);</font></p> <p><strong><font face="Trebuchet MS">Conclusion</font></strong></p> <p><font face="Trebuchet MS">Writing automated integration tests against a middleware product like TIBCO is often an uphill battle that I suspect many don’t have the appetite or patience for. Whilst this post tackles the technical challenges, as <em>they</em> are at least surmountable, the somewhat harder problem of tackling the organisation is sadly still left as an exercise for the reader.</font></p> <p><font face="Trebuchet MS"></font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] The modern NoSQL database vendors appear to have a much simpler model – use it as much as you like outside production.</font></p> <p><font face="Trebuchet MS">[2] If the data files get really large because you leave test messages in them by accident they can cause your machine to really grind after a restart as the service goes through recovery.</font></p> <p><font face="Trebuchet MS">[3] </font><a href="https://blogs.msdn.microsoft.com/cjacks/2009/09/13/how-to-run-applications-manifested-as-highestavailable-with-a-logon-script-without-elevation-for-members-of-the-administrators-group/"><font face="Trebuchet MS">How to Run Applications Manifested as Highest Available With a Logon Script Without Elevation for Members of the Administrators Group</font></a></p> <p><font face="Trebuchet MS">[4] A relative path means the repo can then exist anywhere on the developer’s file-system and also means the code and tools are then always self-consistent across revisions.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-35208901371588869082016-11-01T23:21:00.001+00:002016-11-01T23:21:53.590+00:00Tautologies in Tests<p><font face="Trebuchet MS">Imagine you’re writing a test for a simple function like <font face="Courier New">abs()</font>. You would probably write something like this:</font></p> <p><font face="Courier New">[Test] <br />public void abs_returns_the_magnitude_of_the_value() <br />{ <br />&#160; Assert.That(Math.Abs(-1), Is.EqualTo(1)); <br />}</font></p> <p><font face="Trebuchet MS">It’s a simple function, we can calculate the expected output in our head and just plug the expectation (<font face="Courier New">+1</font>) directly in. But what if I said I’ve seen this kind of thing written:</font></p> <p><font face="Courier New">[Test] <br />public void abs_returns_the_magnitude_of_the_value() <br />{ <br />&#160; Assert.That(Math.Abs(-1), Is.EqualTo(Math.Abs(-1))); <br />}</font></p> <p><font face="Trebuchet MS">Of course in real life it’s not nearly as obvious as this, the data is lifted out into variables and there is more distance between the action and the way the expectation is derived:</font></p> <p><font face="Courier New">[Test] <br />public void abs_returns_the_magnitude_of_the_value() <br />{ <br />&#160; const int negativeValue = –1; <br /> <br />&#160; var expectedValue = Math.Abs(-1); <br /> <br />&#160; Assert.That(Math.Abs(negativeValue), <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Is.EqualTo(expectedValue)); <br />}</font></p> <p><font face="Trebuchet MS">I still doubt anyone would <em>actually</em> write this and a simple function like <font face="Courier New">abs()</font> is not what’s usually under test when this crops up. A more realistic scenario would need much more distance between the test and production code, say, a component-level test:</font></p> <p><font face="Courier New">[Test] <br />public void processed_message_contains_the_request_time() <br />{ <br />&#160; var requestTime = new DateTime(. . .); <br />&#160; var input = BuildTestMessage(requestTime, . . . ); <br />&#160; var expectedTime = Processor.FormatTime(requestTime); <br /> <br />&#160; var output = Processor.Process(input, . . .); <br /> <br />&#160; Assert.That(output.RequestTime, <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Is.EqualTo(expectedTime)); <br />}</font></p> <p><strong><font face="Trebuchet MS">What Does the Test Say?</font></strong></p> <p><font face="Trebuchet MS">If we mentally inline the derivation of the expected value what the test is saying is “When a message is processed the output contains a request time which is formatted by the processor”. This is essentially a tautology because the test is describing its behaviour in terms of the thing under test, it’s self-reinforcing [1].</font></p> <p><font face="Trebuchet MS">Applying the advice from Antoine de Saint-Exupéry [2] about perfection being achieved when there is nothing left take away, lets implement <font face="Courier New">FormatTime()</font> like this:</font></p> <p><font face="Courier New">public string FormatTime(DateTime value) <br />{ <br />&#160; return null; <br />}</font></p> <p><font face="Trebuchet MS">The test will still pass. I know this change is perverse and nobody would ever make <em>that</em> ridiculous a mistake, but the point is that the test is not really doing its job. Also as we rely more heavily on refactoring tools we have to work harder to verify that we have not silently broken a different test that was also inadvertently relying on some aspect of the original behaviour.</font></p> <p><strong><font face="Trebuchet MS">Good Duplication</font></strong></p> <p><font face="Trebuchet MS">We duplicate work in the test for a reason, as a cross-check that we’ve got it right. This “duplication” is often just performed mentally, e.g. formatting a string, but for a more complex behaviour could be done in code using an alternate algorithm [3]. In fact one of the advantages of a practice like TDD is that you <em>have</em> to work it out beforehand and therefore are not tempted to paste the output from running the test on the basis that you’re sure it’s <em>already</em> correct.</font></p> <p><font face="Trebuchet MS">If we had duplicated the work of deriving the output in the example above my little simplification would not have worked as the test would then have failed. Once again, adopting the TDD practice of starting with a failing test and transitioning to green by putting the right implementation in proves that the test will fail if the implementation changes unexpectedly.</font></p> <p><font face="Trebuchet MS">This is a sign to watch out for – if you’re not changing the key part of the implementation to make the test pass you might have overly-coupled the test and production code.</font></p> <p><strong><font face="Trebuchet MS">What is the Test Really Saying?</font></strong></p> <p><font face="Trebuchet MS">The problem with not being the person that wrote the test in the first place is that it may not be telling you what you think it is. For example the tautology may be there because what I just described is not what the author intended the reader to deduce.</font></p> <p><font face="Trebuchet MS">The test name only says that the output will contain the time value, the <em>formatting</em> of that value may well be the responsibility of another unit test somewhere else. This is a component level test after all and so I would need to drill into the tests further to see if that were true. A better approach might be to make the breaking change above and see what actually fails. Essentially I would be doing a manual form of Mutation Testing to verify the test coverage.</font></p> <p><font face="Trebuchet MS">Alternatively the author may be trying to avoid creating a brittle test which would fail if the formatting was tweaked and so decided the best way to do that would be to reuse the internal code. The question is whether the format matters or not (is it a published API?), and with no other test to specifically answer that question one has to work on an assumption.</font></p> <p><font face="Trebuchet MS">This is a noble cause (not writing brittle tests) but there is a balance between the test telling you about a fault in the code and it just being overly specific and annoying by failing on unimportant changes. Sometimes we just need to work a little harder to express the true specification in looser terms. For example maybe we only need to assert that a constituent part of the date is included, say, the year as that is usually the full 4 digits these days:</font></p> <p><font face="Courier New">Assert.That(output.RequestTime, <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; Is.StringContaining(“2010”));</font></p> <p><font face="Trebuchet MS">If we are careful about the values we choose we can ensure that multiple formats can still conform to a looser contract. For example <font face="Courier New">10:33:44</font> on <font face="Courier New">22/11/2016</font> contains no individual fields that could <em>naturally</em> be formatted in a way where a simple substring search could give a false positive (e.g. the hour being mistaken for the day of the month).</font></p> <p><strong><font face="Trebuchet MS">A Balancing Act</font></strong></p> <p><font face="Trebuchet MS">Like everything in software engineering there is a trade-off. Whilst we’d probably prefer to be working with a watertight specification that leaves as little room for ambiguity as possible, we often have details that are pretty loose. When that happens we have to decide how we want to trigger a review of this lack of clarity in the future. If we make the test overly restrictive it runs the risk of becoming brittle, whilst making it overly vague could allow breaking changes to go unnoticed until too late.</font></p> <p><font face="Trebuchet MS">Borrowing (apocryphally) from Einstein we should strive to make our tests as precise as possible, but not overly precise. In the process we need to ensure we do not accidentally reuse production code in the test such that we find ourselves defining the behaviour of it, with itself.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] I’ve looked at the self-reinforcing nature of unit tests before in “<a href="http://chrisoldwood.blogspot.co.uk/2016/01/man-cannot-live-by-unit-testing-alone.html">Man Cannot Live by Unit Testing Alone</a>”.</font></p> <p><font face="Trebuchet MS">[2] See “<a href="http://chrisoldwood.blogspot.co.uk/2014/11/my-favourite-quotes.html">My Favourite Quotes</a>” for some of the other programming related quotes I find particularly inspiring.</font></p> <p><font face="Trebuchet MS">[3] Often one that is slower as correctness generally takes centre stage over performance.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-48060639810887478602016-10-27T23:26:00.001+01:002016-10-27T23:26:23.741+01:00Unmatched REST Resources – 400, 404 or 405?<p><font face="Trebuchet MS">There is always a tension in programming between creating something that is hard to misuse but at the same time adheres to standards to try and leverage the <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">Principle of Least Surprise</a>. One area I personally struggle with this conflict is how to communicate to a client (of the software kind) that they have made a request for something which doesn’t currently exist, and almost certainly will <em>never</em> exist.</font></p> <p><font face="Trebuchet MS">As a general rule when someone requests a resource that doesn’t exist then you should return a <font face="Courier New">404</font> (Not Found). And this makes perfect sense when we’re in production and all the bugs have been ironed but during development when we’re still exploring the API it’s all too easy to make a silly mistake and not realise that it’s due to a bug in our code.</font></p> <p><strong><font face="Trebuchet MS">An Easy Mistake</font></strong></p> <p><font face="Trebuchet MS">Imagine you’re looking up all orders for a customer, you might design your API something like this:</font></p> <p><font face="Courier New">GET /orders/customer/12345</font></p> <p><font face="Trebuchet MS">For a starter you have the whole singular noun vs plural debate which means you’ll almost definitely try this by accident:</font></p> <p><font face="Courier New">GET /order/customer/12345</font></p> <p><font face="Trebuchet MS">or make the inverse mistake</font></p> <p><font face="Courier New">GET /orders/customers/12345</font></p> <p><font face="Trebuchet MS">By the standard HTTP rules you should return a <font face="Courier New">404</font> as the resource does not exist at that address. But does it actually help your fellow developers to stick to the letter of the law?</font></p> <p><strong><font face="Trebuchet MS">Frameworks</font></strong></p> <p><font face="Trebuchet MS">What makes this whole issue much thornier is that if you decide you want to do the right thing by your fellow programmers you will likely have to fight any web framework you’re using because they usually take the moral high ground and do what the standard says.</font></p> <p><font face="Trebuchet MS">What then ensues is a fight between the developer and framework as they try their hardest to coerce the framework to send all unmatched routes through to a handler that can return their preferred non-<font face="Courier New">404</font> choice.</font></p> <p><font face="Trebuchet MS">A colleague who is also up for the good fight recently tried to convince the <a href="http://nancyfx.org/">Nancy</a> .Net framework to match the equivalent of “<font face="Courier New">/.*</font>” (the lowest weighted expression) only to find they had to define one route for each possible list of segments, i.e. “<font face="Courier New">/.*</font>”, “<font face="Courier New">/.*/.*</font>”, “<font face="Courier New">/.*/.*/.*</font>”, etc. [1].</font></p> <p><font face="Trebuchet MS">Even then he still got some inconsistent behaviour. Frameworks also make it really easy to route based on value types which gives you a form of validation. For example if I know my customer ID is always an integer I could express my route like this:</font></p> <p><font face="Trebuchet MS">“<font face="Courier New">/orders/customer/{integer}</font>”</font></p> <p><font face="Trebuchet MS">That’s great for me but when someone using my API accidentally formats a URL wrong and puts the wrong type of value for the ID, say the customer’s name, they get a <font face="Courier New">404</font> because no route matches a non-integer ID. I think this is a validation error and should probably be a <font face="Courier New">400</font> (Bad Request) as it’s a client programmer bug, but the framework has caused it to surface in a way that’s no different to a completely invalid route.</font></p> <p><strong><font face="Trebuchet MS">Choice of Status Code</font></strong></p> <p><font face="Trebuchet MS">So, assuming we want to return something other than Not Found for what is clearly a mistake on the client’s part, what are our choices?</font></p> <p><font face="Trebuchet MS">In the debates I’ve seen on this <font face="Courier New">400</font> (Bad Request) seems like a popular choice as the request, while perhaps not technically malformed, is often synonymous with “client screwed up”. I also like <a href="http://parker0phil.com/">Phil Parker</a>’s suggestion of using <font face="Courier New">405</font> (Method Not Allowed) because it feels like less abuse of the <font face="Courier New">4XX</font> status codes and is also perhaps not as common as a <font face="Courier New">400</font> so shows up a bit more.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS"></font></p> <p><font face="Trebuchet MS">[1] According to <a href="http://stackoverflow.com/questions/12679553/nancyfx-catch-all-route">this StackOverflow post</a> it used to be possible, maybe our Google fu was letting us down.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-19317272752556553502016-10-27T18:57:00.001+01:002016-10-27T18:57:14.881+01:00PUT vs POST and Idempotency<p><font face="Trebuchet MS">In RESTful APIs there is often a question mark around the use of <font face="Courier New">PUT</font> versus <font face="Courier New">POST</font> when creating and updating resources. The canonical example is the blogging engine where we wish to add new posts and comments. The default choice appears to be <font face="Courier New">POST</font> which I guess is because we tend to shy away from that contentious discussion about whether <font face="Courier New">PUT</font> or <font face="Courier New">POST</font> is more suitable.</font></p> <p><font face="Trebuchet MS">As I’ve understood it, when the client can determine the address of the resource you can use <font face="Courier New">PUT</font> whereas when only server knows (i.e. generates it) then a <font face="Courier New">POST</font> is more appropriate. Hence you could say that the question boils down to whether or not the client can create a unique ID for the resource or has to leave that to the server.</font></p> <p><strong><font face="Trebuchet MS">POST</font></strong></p> <p><font face="Trebuchet MS">If we look at the blogging engine it’s probably easier on the client if the server just takes care of it:</font></p> <p><font face="Courier New">POST /blog/create <br />{ <br />&#160; title: “My New Blog Post” <br />}</font></p> <p><font face="Trebuchet MS">Here the server returns the URL for the freshly minted blog post. An obvious choice would probably be to generate a GUID for it and use that as the permanent ID:</font></p> <p><font face="Courier New">GET /blog/1234-1234-1234-1234 <br />{ <br />&#160; title: “My New Blog Post” <br />}</font></p> <p><strong><font face="Trebuchet MS">Idempotency</font></strong></p> <p><font face="Trebuchet MS">The problem with this approach is that we now have a problem if the request fails. We don’t know if the blog post was created because we don’t know its ID yet. If we retry the request, unless the server has a cache of the request, we’ll end up with a duplicate if the server completed on its side. A common solution is to include a client generated ID for the request that the server can use to detect when a request is being replayed:</font></p> <p><font face="Courier New">POST /blog/create <br />{ <br />&#160; requestId: “9876-9876-9876-9876” <br />&#160; title: “my new blog post” <br />}</font></p> <p><font face="Trebuchet MS">But wait, haven’t we just solved the client generated ID problem? If we need to come up with a unique ID for the request for the purposes of idempotency, why don’t we just <font face="Courier New">PUT</font> the resource with that ID in the first place?</font></p> <p><font face="Courier New">PUT /blog/9876-9876-9876-9876 <br />{ <br />&#160; title: “My New Blog Post” <br />}</font></p> <p><strong><font face="Trebuchet MS">Natural Keys</font></strong></p> <p><font face="Trebuchet MS">When the client is just a single machine and it’s creating the resource itself from scratch it has far more latitude in choosing the resource’s ID, but if you’re transforming data in a distributed system it can get a little more tricky.</font></p> <p><font face="Trebuchet MS">If your event comes in the form of an upstream message you cannot just use a GUID because when a message gets replayed (which it will, eventually) you’ll end up generating a duplicate as the request IDs will be different. Hence you need to look for something in the upstream message that can be used as a more natural key for the resource.</font></p> <p><font face="Trebuchet MS">Going back to our blogging engine example we already had one in the guise of the blog post’s title:</font></p> <font face="Courier New">PUT /blog/my-new-blog-post <br />{ <br />&#160; title: “My New Blog Post” <br />}</font> <p><font face="Trebuchet MS">Yes, if the title changes then the ID will no longer match, but that was only a convenience anyway. Hopefully there is enough data in the request itself to make an ID clash extreme</font><font face="Trebuchet MS">ly unlikely, e.g.</font></p> <font face="Courier New">PUT /blog/2016-10-01-chris-oldwood-my-new-blog-post <br />{ <br />&#160; author: “Chris Oldwood” <br />&#160; title: “My New Blog Post” <br />&#160; created: “2016-10-01T08:34:00” <br />}</font> <p><strong><font face="Trebuchet MS">Mutable Resources</font></strong></p> <p><font face="Trebuchet MS">If the resource is immutable then you only need to guard against duplicate attempts to create it, but if it’s mutable then you may already be considering using <font face="Courier New">PUT</font> to mutate it [1]. In this case you’ll probably already have come up with a versioning scheme that will allow you to detect concurrency conflicts from multiple sources, e.g. <a href="https://en.wikipedia.org/wiki/HTTP_ETag">ETag</a>. In this scenario the request will not contain an “original version” tag to compare against so it’s a conflict just the same as when one occurs when updating later.</font></p> <p><strong><font face="Trebuchet MS">Degrees of Idempotency</font></strong></p> <p><font face="Trebuchet MS">It might sound like an oxymoron to say that there are varying levels of idempotency, but there can be. Essentially we’re making the usual time/space trade-off as we balance the need to verify the details of a potential duplicate or erroneous request against the need to persist more state to allow the comparison to be made at an indeterminate point in the future. Also depending on whether you are doing a <font face="Courier New">POST</font> or a <font face="Courier New">PUT</font> means there are assumptions about what a duplicate can be.</font></p> <p><font face="Trebuchet MS">For example at the weakest end of the spectrum if you’re doing a <font face="Courier New">POST</font> with a simple unique request ID you might just say that if you’ve seen it before, irrespective of the content, then it’s a duplicate request. Also if you are happy to miss a replay, say, one month later, then you can even put a time-to-live (TTL) on the unique request ID to save space. Either way you’re treating idempotency very much as a temporal anomaly, which essentially is what it usually is, a fast or slow retry [2].</font></p> <p><font face="Trebuchet MS">If you’re looking for a little extra piece of mind you might decide to generate a hash of the salient headers and content too which would allow you to detect if your request ID might have been reused with different content, either through programmer naivety or an unexpected change in the system. This is not a duplicate but a bad request. Sadly textual formats like XML and JSON have many equally valid representations [3] so a straight text hash is imprecise, but that may well be the pragmatic choice.</font></p> <p><font face="Trebuchet MS">At the other end of the spectrum you might not be happy to silently discard false positives and so you need to persist enough about the request to be sure it really is a duplicate. This likely involves doing a proper comparison of the attributes in the request, which implies that you must still have them. If you save each change as an event rather than just mutating the state to provide an updated snapshot, then you’ll have what you need, but the extra storage is going to cost you. Of course there are other benefits to storing event streams but that’s a different story.</font></p> <p><strong><font face="Trebuchet MS">Just Pedantry?</font></strong></p> <p><font face="Trebuchet MS">Hence one advantage with using a <font face="Courier New">PUT</font> over a <font face="Courier New">POST</font> is that you push the responsibility onto the client to define what it means to be talking about the same resource. This is done by elevating the traditional unique request ID (or some other data) to become the permanent resource ID. Consequently idempotency starts to become implicit in the design rather than appearing more like an afterthought.</font></p> <p><font face="Trebuchet MS">In scenarios where requests come from a single source this is largely an academic exercise but when you have the potential for the same request to come via multiple sources (or be replayed at different times), then I think it’s useful to try and formalise the way we address resources to push the issue to the forefront.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] For partial updates you might choose to use the less well known <font face="Courier New"><a href="http://restcookbook.com/HTTP%20Methods/patch/">PATCH</a></font> verb.</font></p> <p><font face="Trebuchet MS">[2] See “<a href="http://chrisoldwood.blogspot.co.uk/2012/11/when-does-transient-failure-stop-being.html">When Does a Transient Failure Stop Being Transient</a>”. I’ve recently heard the term “<a href="https://twitter.com/viktorklang/status/789036133434978304">effectively once</a>” to describe this notion of compensating for the inability to actually <em>guarantee</em> “once only” delivery.</font></p> <p><font face="Trebuchet MS">[3] My current system receives XML messages in both compact and pretty printed versions. Why, who knows? Perhaps one of the upstream system nodes has a debugging flag left switched on?</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-19093357874218411822016-10-25T19:39:00.001+01:002016-10-25T19:39:46.337+01:00Every Software System is Bespoke<p><font face="Trebuchet MS">Contrary to what some business folk, project managers and programmers may believe, every software system we build is essentially bespoke. They may well think this LOB application, REST API or web site is very similar to many others that have already been built but that is almost certainly because the feature set and user interface have been dreamt up by copying what has already gone before rather than thinking about what they actually <em>need </em>this time around.</font></p> <p><font face="Trebuchet MS">It’s an easy trap to fall into, after all, isn’t the point of hiring experienced people because you want them to leverage all that knowledge they already have to quickly build your system? But what knowledge do they actually bring with them to each new product or project? Unless each one is only a couple of weeks long, which is almost non-existent in my experience [1], then there are way too many different variables to consider this new venture the same as any predecessor.</font></p> <p><font face="Trebuchet MS">Okay, so they may be “similar”, but not the same. At least, not unless you and your organisation have absolutely zero desire to learn anything new about the process and tools used to create software. And that’s essentially my point – the industry moves so incredibly fast that something, probably many things, change between each project. But even then that assumes that no learning occurs <em>during</em> the project itself and, once again, unless it’s only a few weeks long that is also highly unlikely to happen.</font></p> <p><font face="Trebuchet MS">In the last few years I’ve been mostly working in the enterprise arena on web services using the classic enterprise-grade offerings. The enterprise is generally renowned for its glacial pace and yet in that time the technology stack alone has moved along in leaps and bounds. For example that first API was in C# 4 / .Net 3.5 and now we’re looking at C# 6 / .Net Core with one eye looking at running on Linux machines too. Service hosting has changed from IIS / MVC to self-hosting / OWIN, with Nancy in there somewhere. The database too has switched from the relative safety of Oracle &amp; SQL Server to MongoDB &amp; Couchbase, and in one instance [2]has been a hybrid of the relational and document paradigms. Even the ancillary tooling like the VCS, CI product and testing &amp; mocking frameworks have all changed as well, either to a different product or have received a non-trivial upgrade.</font></p> <p><font face="Trebuchet MS">At the same time as the technology stack is evolving so too is the development process. The various organisations I’ve been working in recently have all undergone, or should I say are <em>still</em> undergoing, a major change to a more agile way of working. This in itself is not a one-time switch but a change in mind-set to one of continual learning and therefore by definition is subject to relentless change. Admittedly those changes become more gradual as the bigger problems are addressed but even so the way the teams around you change can still have an affect on your own ways of working – the DevOps message may have yet to reach the parts of the organisation you interact with.</font></p> <p><font face="Trebuchet MS">Even if the toolchain and process largely stay the same the way we apply the technology changes too as what was once “best practice” gets replaced by a new “best practice”, thereby making somewhat of a mockery of the whole notion. Maybe once before we were happy to return null references but now we wish to use an <font face="Courier New">Optional</font> type instead, or we realise the inappropriate nature of the Singleton pattern in highly testable codebase.</font></p> <p><font face="Trebuchet MS">With all the change going on around us you might rightly question what being “experienced” actually means in this industry, if we apparently can’t carry over much from one project to the next. Clearly this is a little extreme though as there is plenty we do carry over. In reality, although everything <em>eventually</em> does change, it does not <em>all</em> change at the same time. Hence at the beginning I said that no system is ever the same, it can be very similar, but will be far from identical.</font></p> <p><font face="Trebuchet MS">As experienced programmers what we carry over are the battle scars. What this should lead us to do is ask those questions that the less experienced know not what to ask in the first place, and often only discover themselves the hard way. We should never assume that just because we did anything one particular way before that <em>that</em> was the only way, or will even still be the best way in this new setting.</font></p> <p><font face="Trebuchet MS">It might be a good way to start out but we should always be looking for ways to improve it, or detect when the problem has diverged such that our first-order approximation is no longer fit for purpose. It’s all too easy to try and solve problems we’ve had in the past, the second time around, and then completely fail to notice we’re really solving a different problem, or at least one with enough differences that we should let go of the past instead. In the end you may be right and you actually converge on a similar design to before, congratulations, you <em>were</em> right this time; as long as you didn’t sacrifice delivering more important stuff just to satisfy your hunch.</font></p> <p><font face="Trebuchet MS">By all means bring and share your experiences on your next venture, but be careful you do not get blindsided by them. Only solve today’s problems with yesterday’s solutions if that really is the best thing to do. You might be surprised just how much has changed in the world of programming since then.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] I’ve spent longer than that just trying to fix a single bug before!</font></p> <p><font face="Trebuchet MS">[2] See “<a href="http://chrisoldwood.blogspot.co.uk/2014/09/deferring-database-choice.html">Deferring the Database Choice</a>” which also highlights the design process changes too.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-41302227559503085312016-10-21T22:59:00.001+01:002016-10-21T22:59:53.475+01:00When Mocks Became Production Services<p><font face="Trebuchet MS">We were a brand new team of 5 (PM + devs) tasked with building a calculation engine. The team was just one part of a larger programme that encompassed over a dozen projects in total. The intention was for those other teams to build some of the services that ours would depend on.</font></p> <p><font face="Trebuchet MS">Our development process was somewhat DSM-like in nature, i.e. iterative. We built a skeleton based around a command-line calculator and fleshed it out from there [1]. This skeleton naturally included vague interfaces for some of the services that we knew we’d need and that we believed would be fulfilled by some of the other teams.</font></p> <p><strong><font face="Trebuchet MS">Fleshing Out the Skeleton</font></strong></p> <p><font face="Trebuchet MS">Time marched on. Our calculator was now being parallelised and we were trying to build out the distributed nature of the system. Ideally we would like to have been integrating with the other teams long ago but the programme RAG status wasn’t good. Every other team apart from us was at “red” and therefore well behind schedule.</font></p> <p><font face="Trebuchet MS">To compensate for the lack of collaboration and integration with the other services we needed we resorted to building our own naïve mocks. We found other sources of the same data and built some noddy services that used the file-system in a dumb way to store and serve it up. We also added some simple steps to the overnight batch process to create a snapshot of the day’s data using these sources.</font></p> <p><strong><font face="Trebuchet MS">Programme Cuts</font></strong></p> <p><font face="Trebuchet MS">In the meantime we discovered that one of the services we were to depend on had now been cancelled and some initial testing with another gave serious doubts about its ability to deliver what we needed. Of course time was marching on and our release date was approaching fast. It was fast dawning on us that these simple test mocks we’d built may well have to become our production services.</font></p> <p><font face="Trebuchet MS">One blessing that came out of building the simple mocks so early on was that we now had quite a bit of experience on how they would behave in production. Hence we managed to shore things up a bit by adding some simple caches and removing some unnecessary memory copying and serialization. The one service left we still needed to invoke had found a more performant way for us to at least bulk extract a copy of the day’s data and so we retrofitted that into our batch preparation phase. (Ideally they’d serve it on demand but it just wasn’t there for the queries we needed.)</font></p> <p><strong><font face="Trebuchet MS">Release Day</font></strong></p> <p><font face="Trebuchet MS">The delivery date arrived. We were originally due to go live a week earlier but got pushed back by a week because an important data migration got bumped and so we were bumped too. Hence we would have delivered on time and, somewhat unusually, we were well under budget our PM said [2].&#160; </font></p> <p><font face="Trebuchet MS">So the mocks we had initially built just to keep the project moving along were now part of the production codebase. The naïve underlying persistence mechanism was now a production data store that needed high-availability and backing up.</font></p> <p><strong><font face="Trebuchet MS">The Price</font></strong></p> <p><font face="Trebuchet MS">Whilst the benefits of what we did (not that there was any other real choice in the end) were great, because we delivered a working system on time, there were a few problems due to the simplicity of the design.</font></p> <p><font face="Trebuchet MS">The first one was down to the fact that we stored each data object in its own file on the file-system and each day added over a hundred-thousand new files. Although we had partitioned the data to avoid the obvious 400K files-per-folder limit in NTFS we didn’t anticipate running out of inodes on the volume when it quickly migrated from a simple Windows server file share to a Unix style DFS. The calculation engine was also using the same share to persist checkpoint data and that added to the mess of small files. We limped along for some time through monitoring and zipping up old data [3].</font></p> <p><font face="Trebuchet MS">The other problem we hit was that using the file-system directly meant that the implementation details became exposed. Naturally we had carefully set ACLs on the folders to ensure that only the environment had write access and our special support group had read access. However one day I noticed by accident that someone had granted read access to another group and it then transpired that they were building something on top of our naïve store.</font></p> <p><font face="Trebuchet MS">Clearly we never intended this to happen and I’ve said more about this incident previously in “The File-System Is An Implementation Detail”. Suffice to say that an arms race then developed as we fought to remove access to everyone outside our team whilst others got wind of it [4]. I can’t remember whether it happened in the end or not but I had put a scheduled task together than would use CALCS to list the permissions and fail if there were any we didn’t expect.</font></p> <p><font face="Trebuchet MS">I guess we were a victim of our success. If you were happy with data from the previous COB, which many of the batch systems were, you could easily get it from us because the layout was obvious.</font></p> <p><strong><font face="Trebuchet MS">Epilogue</font></strong></p> <p><font face="Trebuchet MS">I have no idea whether the original versions of these services are still running to this day but I wouldn’t be surprised if they are. There was a spike around looking into a NoSQL database to alleviate the inode problem, but I suspect the ease with which the data store could be directly queried and manipulated would have created too much inertia.</font></p> <p><font face="Trebuchet MS">Am I glad we put what were essentially our mock services into production? Definitely. Given the choice between not delivering, delivering much later, and delivering on time with a less than perfect system that does what’s important – I’ll take the last one every time. In retrospect I wish we had delivered sooner and not waited for a load of other stuff we built as the MVP was probably far smaller.</font></p> <p><font face="Trebuchet MS">The main thing I learned out of the experience was a reminder not to be afraid of </font>d<font face="Trebuchet MS">oing the simplest thing that could work. If you get the architecture right each of the pieces can evolve to meet the ever changing requirements and data volumes [5].</font></p> <p><font face="Trebuchet MS">What </font><font face="Trebuchet MS">we did here fell under the <em>traditional</em> banner of Technical Debt – making a conscious decision to deliver a sub-optimal solution now so it can start delivering value sooner. It was the right call.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] Nowadays you’d probably look to include a slice through the build pipeline and deployment process up front too but we didn’t get any hardware until a couple of months in.</font></p> <p><font face="Trebuchet MS">[2] We didn’t build half of what we set out to, e.g. the “dashboard” was a PowerShell generated HTML page and the work queue involved doing non-blocking polling on a database table.</font></p> <p><font face="Trebuchet MS">[3] For regulatory reasons we needed to keep the <em>exact</em> inputs we had used and couldn’t guarantee on being able to retrieve them later from the various upstream sources.</font></p> <p><font face="Trebuchet MS">[4] Why was permission granted without questioning anyone in the team that owned and supported it? I never did find out, but apparently it wasn’t the first time it had happened.</font></p> <p><font face="Trebuchet MS">[5] Within reason of course. This system was unlikely to grow by more than an order of magnitude in the next few years.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-73102954203533729112016-10-20T21:20:00.001+01:002016-10-20T21:20:07.244+01:00Confusion Over Waste<p><font face="Trebuchet MS">When looking at the performance of our software we often have to consider both first-order and second-order effects. For example when profiling a native application where memory management is handled explicitly we can directly see the cost of allocations and deallocations because this all happens at the moment we make them. In contrast the world of garbage collected languages like C# exhibit different behaviour. The cost of memory allocations here are minimal because the algorithm is simple. However the deallocation story is far more complex, and it happens at a non-deterministic time later. </font></p> <p><font face="Trebuchet MS">A consequence of this different behaviour is that it is much harder to see the effects that localised memory churn is having on your application. For example I once worked on a C# data transformation tool where the performance was appalling. Profiling didn’t immediately reveal the problem but closer inspection showed that the garbage collector was running full tilt. Looking much closer at the hottest part of the code I realised it was spending all it’s time splitting strings and throwing them away. The memory allocations were cheap so there were no first-order effects, but the clean-up was really expensive and happened later and therefore appeared as a second-order effect which was harder to trace back.</font></p> <p><strong><font face="Trebuchet MS">Short Term Gains</font></strong></p> <p><font face="Trebuchet MS">We see the same kind of effects occurring during the development process too. They are often masked though by the mistaken belief that time is being saved, it is, but only in the short term. The problem is the second-order effects of such time saving is actually lost later, and when it’s more precious.</font></p> <p><font face="Trebuchet MS">This occurs because the near term activity is being seen as wasteful of a certain person’s time, on the premise that the activity is of low value (to them). But what is being missed is the second-order effects of doing that, such as the learning about the context, people and product involved. When crunch time comes that missed learning suddenly has to happen at the later time when potentially under time pressure or after money has already been spent; then you’re heading into </font><a href="https://en.wikipedia.org/wiki/Sunk_costs"><font face="Trebuchet MS">sunk costs</font></a><font face="Trebuchet MS"> territory.</font></p> <p><font face="Trebuchet MS">In essence what is being perceived as waste is the time spent in the short term, when the real waste is time lost in the future due to rework caused by the missed opportunity to learn sooner.</font></p> <p><strong><font face="Trebuchet MS">All Hail “Agile”</font></strong></p> <p><font face="Trebuchet MS">Putting this into more concrete terms consider a software development team where the developer’s time is assumed to be best spent designing and writing code. The project manager assumes that having conversations, perhaps with ops or parts of the business is of low value, from the developer’s perspective, and therefore decides it’s better if someone “less expensive” has it instead.</font></p> <p><font face="Trebuchet MS">Of course we’re all “agile” now and we don’t do that anymore. Or do we? I’ve worked in supposedly agile teams and this problem still manifests itself, maybe not quite to the same extent as before, but nonetheless it still happens and I believe it happens because we are confused about what the <em>real</em> waste is that we’re trying to avoid.</font></p> <p><font face="Trebuchet MS">Even in teams I’ve been in where we’ve tried to ensure this kind of problem is addressed, it’s only addressed locally, it’s still happening further up the food chain. For example a separate architecture team might be given the role of doing a spike around a piece of technology that a development team will be using. This work needs to happen inside the team so that those who will be developing and, more importantly, supporting the product will get the most exposure to it. Yes, there needs to be some governance around it, but the best people to know if it <em>even</em> solves their problem in the first place is the development team.</font></p> <p><font face="Trebuchet MS">Another manifestation of this is when two programme managers are fed highlights about potential changes on <em>their</em> side of the fence. If there is any conflict there could be a temptation to resolve it without going any lower. What this does is cut out the people that not only know most about the conflict, but are also the best placed to negotiate a way out. For example instead of trying to compensate for a potential breaking change with a temporary workaround, which pushes the product away from its eventual goal, see if the original change can be de-prioritised instead. If a system is built in very small increments it’s much easier to shuffle around the high priority items to accommodate what’s happening around the boundaries of the team.</font></p> <p><strong><font face="Trebuchet MS">Time for Reflection</font></strong></p> <p><font face="Trebuchet MS">How many times have you said, or heard someone else say, “if only you’d come to us earlier”. This happens because we try and cut people out of the loop in the hope that we’ll save time by resolving issues ourselves, but what we rarely do is reflect on whether we really did save time in the long run when the thread eventually started to unravel and the second-order effects kicked in.</font></p> <p><font face="Trebuchet MS">Hence, don’t just assume you can cut people out of the loop because you think you’re helping them out, you <em>might</em> not be. They might want to be included because they have something to learn or contribute over-and-above the task at hand. Autonomy is about choice, they might not always want it, but if you don’t provide it in the first place it can never be leveraged.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-9566493343868178022016-08-22T23:24:00.001+01:002016-08-22T23:24:24.833+01:00Sharing Code with Git Subtree<p><font face="Trebuchet MS">The codebase I currently work on is split into a number of repositories. For example the infrastructure and deployment scripts are in separate repos as are each service-style “component”.</font></p> <p><strong><font face="Trebuchet MS">Manual Syncing</font></strong></p> <p><font face="Trebuchet MS">To keep things moving along the team decided that the handful of bits of code that were shared between the two services could easily be managed by a spot of manual copying. By keeping the shared code in a separate namespace it was also partitioned off to help make it apparent that this code was at some point going to be elevated to a more formal “shared” status.</font></p> <p><font face="Trebuchet MS">This approach was clearly not sustainable but sufficed whilst the team was still working out <em>what</em> to build. Eventually we reached a point where we needed to bring the logging and monitoring stuff in-sync and I also wanted to share some other useful code like an <font face="Courier New">Optional&lt;T&gt;</font> type. It also became apparent that the shared code was missing quite a few unit tests as well.</font></p> <p><strong><font face="Trebuchet MS">Share Source or Binaries?</font></strong></p> <p><font face="Trebuchet MS">The gut reaction to such a problem in a language like C# would probably be to hive off the shared code into a separate repo and create another build pipeline for it that would result in publishing a package via a NuGet feed. And that is certainly what we expected to do. However the problem was where to publish the packages to as this was closed source. The organisation had its own license for an Enterprise-scale product but it wasn’t initially reachable from outside the premises where our codebase lay. Also there were some problems with getting NuGet to publish to it with an API key that seemed to lay with the way the product’s permissions were configured.</font></p> <p><font face="Trebuchet MS">Hence to keep the ball rolling we decided to share the code at the <em>source</em> level by pulling the shared repo into each component’s solution. There are two common ways of doing this with Git – <a href="http://stackoverflow.com/questions/31769820/differences-between-git-submodule-and-subtree">subtrees and submodules</a>.</font></p> <p><strong><font face="Trebuchet MS">Git Submodules</font></strong></p> <p><font face="Trebuchet MS">It seemed logical that we should adopt the more modern submodule approach as it felt easier to attach, update and detach later. It also appeared to have support in the Jenkins 1.x plugin for doing a recursive clone so we wouldn’t have to frig it with some manual Git voodoo.</font></p> <p><font face="Trebuchet MS">As always there is a difference between theory and practice. Whilst I suspect the submodule feature in the Jenkins plugin works great with publicly accessible open-source repos it’s not quite up to scratch when it comes to private repos that require credentials. After much gnashing of teeth trying to convince the Jenkins plugin to recursively clone the submodules, we conceded defeat assuming that we’re another victim of <a href="https://issues.jenkins-ci.org/browse/JENKINS-20941">JENKINS-20941</a>.</font></p> <p><strong><font face="Trebuchet MS">Git Subtree</font></strong></p> <p><font face="Trebuchet MS">Given that our long term goal was to move to publishing a NuGet feed we decided to try using a Git subtree instead so that we could at least move forward and share code. This turned out (initially) to be much simpler because for tooling like Jenkins it appears no different to a single repo.</font></p> <p><font face="Trebuchet MS">Our source tree looked (unsurprisingly) like this:</font></p> <p><font face="Courier New">&lt;solution&gt; <br />&#160; +- src <br />&#160;&#160;&#160;&#160; +- app <br />&#160;&#160;&#160;&#160; +- shared-lib <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- .csproj <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- *.cs</font></p> <p><font face="Trebuchet MS">All we needed to do was replace the <font face="Courier New">shared-lib</font> folder with the contents of the new <font face="Courier New">Shared</font> repository.</font></p> <p><font face="Trebuchet MS">First we needed to set up a Git remote. Just as the remote main branch of a cloned repo goes by the name <font face="Courier New">origin/master</font>, so we set up a remote for the Shared repository’s main branch:</font></p> <p><font face="Courier New">&gt; git remote add shared https://github/org/Shared.git</font></p> <p><font face="Trebuchet MS">Next we removed the old shared library folder:</font></p> <p><font face="Courier New">&gt; git rm src\shared-lib</font></p> <p><font face="Trebuchet MS">…and grafted the new one in from the remote branch:</font></p> <p><font face="Courier New">&gt; git subtree add --prefix src/shared shared master --squash</font></p> <p><font face="Trebuchet MS">This effectively takes the <font face="Courier New">shared/master</font> branch and links it further down the repo source tree to <font face="Courier New">src/shared</font> which is where we had it before.</font></p> <p><font face="Trebuchet MS">However the organisation of the new Shared repo is not <em>exactly</em> the same as the old <font face="Courier New">shared-lib</font> project folder. A single child project usually sits in it’s own folder, but a full-on repo has it’s <em>own</em> <font face="Courier New">src</font> folder and build scripts and so the source tree now looked like this:</font></p> <font face="Courier New">&lt;solution&gt; <br />&#160; +- src <br />&#160;&#160;&#160;&#160; +- app <br />&#160;&#160;&#160;&#160; +- shared <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- src <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- shared-lib <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- .csproj <br />&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; +- *.cs </font> <p><font face="Trebuchet MS">There is now two extra levels of indirection. First there is the <font face="Courier New">shared</font> folder which corresponds to the external repo, plus there is <em>that</em> repo’s <font face="Courier New">src</font> folder.</font></p> <p><font face="Trebuchet MS">At this point all that was left to do was to fix up the build, i.e. fix up the path to the <font face="Courier New">shared-lib</font> project in the Visual Studio solution file (<font face="Courier New">.sln</font>) and push the changes.</font></p> <p><font face="Trebuchet MS">We chose to use the <font face="Courier New">--squash</font> flag when creating the subtree as we weren’t interested in seeing the entire history of the shared library in the solution’s repository.</font></p> <p><strong><font face="Trebuchet MS">Updating the Subtree</font></strong></p> <p><font face="Trebuchet MS">Flowing changes from the parent repo down into the subtree of the child repo is as simple as a <font face="Courier New">fetch</font> &amp; <font face="Courier New">pull</font>:</font></p> <p><font face="Courier New">&gt; git fetch shared master <br />&gt; git subtree pull --prefix src/shared shared master --squash</font></p> <p><font face="Trebuchet MS">The latter command is almost the same as the one we used earlier but we <font face="Courier New">pull</font> rather than <font face="Courier New">add</font>. Once again we’re squashing the entire history as we’re not interested in it.</font></p> <p><strong><font face="Trebuchet MS">Pushing Changes Back </font></strong></p> <p><font face="Trebuchet MS">Naturally you might want to make a change in the subtree in the context of the entire solution and then push it back up to the parent repo. This is doable but involves using <font face="Courier New"><a href="http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/">git subtree push</a></font> to normalise the change back into the folder structure of the parent repo.</font></p> <p><font face="Trebuchet MS">Personally we decided just to make the changes test-first in the parent and always flow down to the child. In the few cases the child solution helped in debugging we decided to work on the fix in the child solution workspace and then simply manually copy the change over to the shared workspace and push it out through the normal route. It’s by no means optimal but a NuGet feed was always our end game so we tolerated the little bit of friction in the short term.</font></p> <p><strong><font face="Trebuchet MS">The End of the Road</font></strong></p> <p><font face="Trebuchet MS">If we were only sucking in libraries that had no external dependencies themselves (up to that point our small shared code only relied on the .Net BCL) we might have got away with this technique for longer. But in the end the need to pull in 3rd party dependencies via NuGet in the shared project pushed it over the edge.</font></p> <p><font face="Trebuchet MS">The problem is that NuGet packages are on a <em>per-solution</em> basis and the <font face="Courier New">&lt;HintPath&gt;</font> element in the project file assumes a relative path (essentially) from the solution file. When working in the real repo as part of the shared solution it was “<font face="Courier New">..\..\packages\Xxx</font>”, but when it’s part of the subtree based solution it needed to be two levels further up as “<font face="Courier New">..\..\..\..\packages\Xxx</font>”.</font></p> <p><font face="Trebuchet MS">Although I didn’t spend long looking I couldn’t find a simple way to easily overcome this problem and so we decided it was time to bite-the-bullet and fix the real issue which was publishing the shared library as a NuGet feed.</font></p> <p><strong><font face="Trebuchet MS">Partial Success</font></strong></p> <p><font face="Trebuchet MS">This clearly is not anything like what you’d call an <em>extensive</em> use of <font face="Courier New">git subtree</font> to share code, but it certainly gave me a feel for it can do and I think it was relatively painless. What caused us to abandon it was tooling specific (the relationship between the enclosing solution’s NuGet packages folder and the shared assembly project itself) and so a different toolchain may well fair much better if build configuration is only passed down from parent to subtree.</font></p> <p><font face="Trebuchet MS">I suspect the main force that might deter you from this technique is how much you know, or feel you need to know, about how git works. When you’re inside a tool like Visual Studio it’s very easy to make a change in the subtree folder and check it in and not necessarily realise you’re modifying what is essentially read-only code. When you next update the subtree things get sticky. Hence you really need to be diligent about your changes and pay extra attention when you commit to ensure you don’t accidentally include edits within the subtree (if you’re not planning on pushing back that way). Depending on how experienced your team are this kind of tip-toeing around the codebase might be just one more thing you’re not willing to take on.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com1tag:blogger.com,1999:blog-6628985022531866193.post-23798710200175871362016-08-22T22:45:00.001+01:002016-08-22T22:45:00.827+01:00Manually Forking Chunks of Open Source Code<p><font face="Trebuchet MS">Consuming open source projects is generally easy when you are just taking a package that pulls in source or binaries into your code “as is”. However on occasion we might find ourselves needing to customise part of it, or even borrow and adapt some of its code to either workaround a bug or implement our own feature.</font></p> <p><font face="Trebuchet MS">If you’re forking the entire repo and building it yourself then you are generally going to play by their rules as you’re aware that you’re playing in somebody else’s house. But when you clone just a small part of their code to create your own version then it might not seem like you have to continue honouring their style and choices, but you probably should. At least, if you want to take advantage of upstream fixes and improvements you should. If you’re just going to rip out the underlying logic it doesn’t really matter, but if what you’re doing is more like tweaking then a more surgical approach should be considered instead.</font></p> <p><strong><font face="Trebuchet MS">Log4Net Rolling File Appender</font></strong></p> <p><font face="Trebuchet MS">The driver for this post was having to take over maintenance of a codebase that used the <a href="https://logging.apache.org/log4net/">Log4Net</a> logging framework. The service’s shared libraries included a customised Log4Net appender that took the basic <a href="https://logging.apache.org/log4net/log4net-1.2.11/release/sdk/log4net.Appender.RollingFileAppender.html">rolling file appender</a> and then tweaked some of the date/time handling code so that it could support the finer-grained rolling log file behaviour they needed. This included keeping the original file extension and rolling more frequently than a day. They had also added some extra logic to support compressing the log files in the background too.</font></p> <p><font face="Trebuchet MS">When I joined the team the Log4Net project had moved on quite a bit and when I discovered the customised appender I thought I’d better check that it was still going to work when we upgraded to a more recent version. Naturally this involved diffing our customised version against the current Log4Net file appender.</font></p> <p><font face="Trebuchet MS">However to easily merge in any changes from the Log4Net codebase I would need to do a three-way diff. I needed the common ancestor version, my version and their version. Whilst I could fall back to a two-way diff (latest of theirs and mine) there were lots of overlapping changes around the date/time arithmetic which I suspected were noise as the Log4Net version appeared to now have what we needed.</font></p> <p><font face="Trebuchet MS">The first problem was working out what the common ancestor was. Going back through the history of our version I could see that the first version checked-in was already a highly modded version. They had also appeared to apply some of the ReSharper style refactorings which added a bit of extra noise into the mix.</font></p> <p><font face="Trebuchet MS">What I had hoped they would have done is started by checking in the exact version of the code they got from Log4Net and put in the check-in commit the Subversion revision number of the code so that I could see at what version they were going to fork it. After a few careful manual comparisons and some application of logic around commit timestamps I pinned down what I thought was the original version.</font></p> <p><font face="Trebuchet MS">From here I could then trace both sets of commit logs and work out what features had been added in the Log4Net side and what I then needed to pull over from our side which turned out to be very little in the end. The hardest part was working out if the two changes around the date rolling arithmetic were logically the same as I had no tests to back up the changes on our side.</font></p> <p><font face="Trebuchet MS">In the end I took the latest version of the code from the Log4Net codebase and manually folded in the compression changes to restore parity. Personally I didn’t like the way the compression behaviour was hacked in [1] but I wanted to get back to working code first and then refactor later. I tried to add some integration tests too at the same time but they have to be run separately as the granularity of the rollover was per-minute as a best case [2].</font></p> <p><font face="Trebuchet MS">Although the baseline Log4Net code didn’t match our coding style I felt it was more important to be able to rebase our changes over any new Log4Net version than to meet our coding guidelines. Naturally I made sure to include the relevant Log4Net Subversion revision numbers in my commits to make it clear what version provided the new baseline so that a future maintainer has a clear reference point to work from.</font></p> <p><font face="Trebuchet MS">In short if you are going to base some of your own code very closely on some open source stuff (or even internal shared code) make sure you’ve got the relevant commit details for the baseline version in your commit history. Also try and avoid changing too much unnecessarily in your forked version to make it easier to pull and rebase underlying changes in the future.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] What worried me was the potential “hidden” performance spikes that the compression could put on the owning process. I would prefer the log file compression to be a background activity that happens in slow time and is attributable to an entirely separate process that didn’t have tight per-request SLAs to meet.</font></p> <p><font face="Trebuchet MS">[2] I doubt there is much call for log files that roll every millisecond :o).</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0tag:blogger.com,1999:blog-6628985022531866193.post-6256787027522891762016-08-08T20:31:00.001+01:002016-08-08T20:31:10.682+01:00Estimating is Liberating<p><font face="Trebuchet MS">Right now after having just read the title you’re probably thinking I’ve gone mad. I mean, why would a developer actively promote the use of estimation? Surely I should be joining the ranks of the <a href="https://www.google.co.uk/?#q=NoEstimates">#NoEstimates</a> crowd and advocate the abolishment of estimating as a technique?</font></p> <p><font face="Trebuchet MS">Well, yes, and no. As plenty of other much wiser people than me have already pointed out, the #NoEstimates movement, like the #NoSql one before it, is not the black-and-white issue it first appears. However this blog post isn’t really about whether or not I think you should create estimates per-se but about some of the effects I’ve observed from teams performing it. Whether the means justifies the end (even in the short term) is for you to decide.</font></p> <p><strong><font face="Trebuchet MS">Establishing Trust</font></strong></p> <p><font face="Trebuchet MS">The first time I realised there was more to estimating than simply coming up with a figure that describes how long some piece of work will likely take was almost a decade ago. I was working as “just another developer” on a greenfield project that was using something approaching <a href="https://en.wikipedia.org/wiki/Dynamic_systems_development_method">DSDM</a> as a project management style.</font></p> <p><font face="Trebuchet MS">We had started off really well delivering a walking skeleton and fleshing it out bit-by-bit based on what the project manager and team thought was a suitable order. There were many technical risks to overcome, not least due to the cancellation of some services we were dependent on.</font></p> <p><font face="Trebuchet MS">After what seemed like a flying start things slowly went downhill. 6 months in the project manager (PM) and I had a quiet word [1] as they were concerned things seemed to be taking so much longer than earlier in the project. </font><font face="Trebuchet MS">I suggested that we consider going back to estimating our work. What I had noticed is that the problems we were encountering were really just delays caused by not really thinking through the problems. Hence every day the work would be finished “tomorrow” which naturally caused the project manager to start losing faith in his team.</font></p> <p><font face="Trebuchet MS">By forcing ourselves to have to come up with an idea of how long we would need to work on a feature we started breaking it down into much smaller chunks. Not only did this mean that we thought through more clearly what issues we might need to tackle but it also allowed us to trim any obvious fat and work in parallel where possible.</font></p> <p><font face="Trebuchet MS">The result was an increase in trust again in the team by the project manager, but also by extension the customer and PM therefore had less “awkward” conversations too [2].</font></p> <p><strong><font face="Trebuchet MS">Knowledge Sharing</font></strong></p> <p><font face="Trebuchet MS">The next moment where I began to see the positive effects of estimation was when joining a team that adopted Planning Poker as a way of estimating the work for a sprint.</font></p> <p><font face="Trebuchet MS">In the (not so) good old days, work was assigned to individuals and they were responsible for estimating and performing that work largely in isolation. Of course many of us would prefer to seek advice from others, but you were still essentially seen as responsible for it. As a corollary to that the number of people in the team who knew what was being worked on was therefore small. Even if you did have a whiff of what was happening you probably knew very little about the details unless you stuck your nose in [3].</font></p> <p><font face="Trebuchet MS">This team worked in a similar fashion, but by opening up the planning session to the whole team everyone now had a better idea of what was going on. So, even if they weren’t actively going to be working on a feature they still had some input into it.</font></p> <p><font face="Trebuchet MS">What the planning poker session did was bring everyone together so that they all felt included and therefore informed. Additionally by actively canvasing their opinion for an estimate on each and every feature their view was being taken into consideration. By providing an overly small or large estimate they would have a sure-fire means of having their opinion heard because the conversation tends to focus on understanding the outliers rather than the general consensus. It also allowed members of the team to proactively request involvement in something rather than finding out later it has already been given to someone else.</font></p> <p><font face="Trebuchet MS">I think the team started to behave more like a collective and less like a bunch of individuals after adopting this practice.</font></p> <p><strong><font face="Trebuchet MS">Transparency</font></strong></p> <p><font face="Trebuchet MS">More recently I was involved in some consultancy at a company where I got to be a pure observer in someone else’s agile transformation. The units of work they were scheduling tended to be measured in weeks-to-months rather than hours-to-days.</font></p> <p><font face="Trebuchet MS">I observed one planning meeting where someone had a task that was estimated at 4 weeks. I didn’t really know anything about their particular system but the task sounded pretty familiar and I was surprised that it was anything more than a week, especially as the developer was fairly experienced and it sounded like it was similar to work they had done before.</font></p> <p><font face="Trebuchet MS">I later asked them to explain how they came up with their estimate and it transpired that buried inside were huge contingencies. In fact a key part of the task involved understanding how an API that no one had used before worked. </font><font face="Trebuchet MS">In reality there was a known part for which a reasonably sound estimate could be given, but also a large part of it was unknown. Like many organisations it never acknowledged that aspects of software development are often unknown and, when faced with something we’ve never done before, we are still expected to be able to say how long it will take.</font></p> <p><font face="Trebuchet MS">Consequently I got them to break the task down into smaller parts and present those estimates instead. Most notable was an upfront piece around understanding the 3rd party API – a technical spike. This very consciously did not have an estimate attached and it allowed us to explain to the stakeholders what spikes are and how &amp; when to use them to explore the unknown.</font></p> <p><font face="Trebuchet MS">This openness with the business made both them and the delivery team more comfortable. The business were now more in the loop about the bigger risks and could also see how they were being handled. Consequently they also now had the ability to <a href="http://chrisoldwood.blogspot.co.uk/2015/11/dont-fail-fast-learn-cheaply.html">learn cheaply (fail faster)</a> by keeping the unknown work more tightly under control and avoid unexpected spiralling costs or delays.</font></p> <p><font face="Trebuchet MS">The benefit for the delivery team was the recognition from the business there is stuff we just don’t know how to do. For us this is hugely liberating because we can now lay our cards firmly on the table instead of hiding behind them. Instead of worrying about how much to pad our estimate to ensure we have enough contingency to cover all the stuff we definitely don’t know about, we can instead split off that work and play it out up front as a time-boxed exercise [4].Instead of being sorry that we are going to be late, again, we have the opportunity to be praised for saving the business money.</font></p> <p><strong><font face="Trebuchet MS">Training Wheels</font></strong></p> <p><font face="Trebuchet MS">What all of these tales have in common is that the end product – the actual estimate is of little importance to the team. The whole #NoEstimates movement has plenty to say on whether they are useful or not in the end, but the <em>by-product</em> of the process of <em>estimating</em> certainly has some use as a teaching aid.</font></p> <p><font face="Trebuchet MS">A mature (agile) team will already be able to break work down into smaller chunks, analyse the risks and prioritise it so that the most valuable is done first (or risks reduced). But an inexperienced team that has had little direct contact with its stakeholders may choose to go through this process as a way of gaining trust with the business.</font></p> <p><font face="Trebuchet MS">In the beginning both sides may be at odds, both believing that the other side doesn’t really care about what is important to them. Estimation <em>could</em> be used as a technique that allows the technical side to “show it’s workings” to the other side, just as an exam student proves to the examiner that they didn’t just stumble upon the answer through luck.</font></p> <p><font face="Trebuchet MS">As trust eventually grows and the joint understandings of “value” take shape, along with a display of continuous delivery of the business’s (ever changing) preferred features, the task of estimation falls away to leave those useful practices which always underpinned it. At this point the training wheels have come off and the team feels liberated from the tyranny of arbitrary deadlines.</font></p> <p><font face="Trebuchet MS">&#160;</font></p> <p><font face="Trebuchet MS">[1] This is how process improvement used to take place (if at all) before retrospectives came along.</font></p> <p><font face="Trebuchet MS">[2] Without direct involvement from the customer all communication was channelled through the project manager. Whilst well-meaning (to protect the team) this created more problems than it solved.</font></p> <p><font face="Trebuchet MS">[3] I guess I’m a “busy-body” because I have always enjoyed sticking my nose in and finding out what others are up to, mostly to see if they’d like my help.</font></p> <p><font face="Trebuchet MS">[4] The common alternative to missing your deadline and being held to it is to work longer hours and consequently lose morale. Either way the business eventually loses, not that they will always realise that.</font></p>Chris Oldwoodhttps://plus.google.com/114924737136553379980noreply@blogger.com0