Random thoughts about stuff that interests me

I’m inspired by Kent Beck‘s fantastic post on how Delay Chokes Innovation to write something that has niggled around at the back of my mind for over a decade. If you haven’t read Kent’s post, please do so as it provides the context for this one which is a ‘yes, and’ rather than a ‘yes, but’ response. It is also a poker story.

Poker

I hate tournament poker, largely because I’m crap at it. I don’t like the ‘winner takes all’ dynamic and the use of artificial escalation to force a result. I once played a tournament at the Luxor in Vegas and it was the most grindingly grim poker experience of my life.

I learned to play poker in an ever-evolving group working on a project away from home. Once we’d quickly exhausted the delights of the local town we turned to making our own entertainment and a poker school playing two or three nights a week was born. Sometimes there would be four players, sometimes twenty. There were a few regulars, of which I was one, and a number of people who drifted in and out. The standard of play was high and a couple from the group went on to be successful semi-pro tournament players.

I was one of three complete novices when we started with four or five others who had been playing for a few years. My first night was a shock as I lost hand after hand, quickly ripping through my €50 budget for the evening (they were kind to me and suggested I set a budget to avoid financial ruin on top of the inevitable humiliation). So I sat and watched, itching to buy back in, and resolved to learn how play properly.

I set myself an overall budget of €300: if I was still losing hand over fist after I’d spent that I’d give up on it. The second night of play I lost my €50 limit again but managed to last the whole night and from then on I started to lose less, then win a bit, then win a lot. I never quite spent my €300 ‘tuition fee’ and at the end of the project I had a drawer stuffed full of 5, 10 and 20 Euro bills. I learned an awful lot about bluffing, mirroring, psychological manipulation and risk management, not by reading books but by watching and responding to the actions of my friends/foes across the smokey, dimly lit table.

Pot Odds

Some poker players know the odds of every possible hand given the cards they hold and the cards on the table. I once played against someone in the final year of their PhD in game theory whose party trick (deployed before the game to intimidate us) was to play open hands reciting the odds of winning as he went. Very impressive*!

Less clever players like myself work on the concept of ‘outs’ rather than exact odds. An ‘out’ is a card that leads to a possibly winning hand and if you can identify all the outs you can do a rough approximation of odds: the greater the number of outs and the stronger the hands you end up with the better your chance of winning. ‘Possibly winning’ is important here: many published poker strategies focus on having the best possible hand (or something very close to it), bluffing hard, or else folding quickly to minimise losses and tournament poker tends to favour such strategies. School poker is different because it is played out over many nights and there tends to be more risk taking because the cost of failure (whether losing a hand or losing out over the course of a night) is lower: you can always come back later.

Both forms of poker work on the basis of ‘pot odds’ but these are more important in school poker than tournament. Pot odds is basically the ratio of your odds of winning to the amount of your money in the pot (the total of all the bets on this hand). Let’s say your chances of winning are 1 in 5 and your current stake is 10% of the total pot. A 1 in 5 chance of winning isn’t great, and most poker strategies will tell you to fold such a hand, but with a potential 9:1 return on your money it’s worth hanging around for a bit to see what happens. Conversely if your chances of winning are 1 in 5 but your share of the pot is 25% it’s likely time to get out.

It’s not as simple as that because factors like the number of players left in on the hand, the amount of money you and your opponents have left to bet with, the stage of the evening, your’s and others’ propensity for bluffing, etc. all contribute. But it stands as a basic heuristic: if the pot odds are with you stay in, if they’re against you get out**.

Innovation and Maximising the Value of Success

So what’s this got to do with innovation? As Kent says:

The cost of delay in choking innovation is insidious. From outside the company looks to be doing fine. Smaller wins times a larger market equals continued growth.

This is basically the tournament strategy of folding unless you’re going to bluff hard or have the best hand. The thing is that unlike in the movies those ‘nuts’ hands, where you’re dealt a pair of Aces and another one turns up on the flop, are very rare. You can wait for hours for something like that to turn up, and then try to eek out the betting only for everyone else to fold because they have dogshit and your moment of triumph nets you all of €15 with €5 of that being your own money. There is definitely a role for smaller wins times a larger market in a stable business but sometimes you need or want to do more.

Invisible are all the little roads not taken that could have turned out to be superhighways

Playing pot odds helps you look for those little roads that can turn out to be superhighways. If you do want or need step-change growth you have to find the opportunities that, even if more risky, have much more reward in return. It’s all about maximising the value of success rather than minimising the risks of failure.

If you’re a company owner or senior manager you need to encourage a ‘play big, win big’ attitude but know how to direct that to areas of innovation that have a chance of disproportionate reward, not to safe projects that will almost certainly bring a modest return and not to vanity projects that deliver all the buzzwords but little value. And sometimes it’s right to go ‘all in’ on an initiative because, as with school poker, there is always the chance to come back later.

Last Words

Those nights playing poker against, amongst many others, Darren, Darren, Phil, Johnny P, Johnny B, James, Brett and Klaus were amazing fun and a greater understanding of human psychology and risk management was just a happy by-product. If any of you happen to stumble across this, thank you.

Thanks too to Kent for a great post and also for this wonderful ‘separated at birth’ opportunity:

Woody Harrelson and Kent Beck. Or is that Kent Beck and Woody Harrelson?

*also completely hubristic. By revealing his key strength up front he told me all I needed to know to play him. Two hours later he was all out.

**basic heuristics don’t last any longer in a game of poker than any other rule or formulaic strategy. It’s the foundation for a strategy, not a strategy itself.

“We need some new development task tracking software. What is everyone using?” asked someone on a CTO email list I’m a member of. “Is Jira still the standard?”

Jira, Trello, LeanKit, Pivotal and their ilk have never been for me. Since I first started practicing XP mumble, mumble years ago it’s always been about cards, magnets and a wall. Above is the old Singletrack office with two separate boards for our two main product strands and a third for customer roll-out work.

“But what if you have a dispersed team with remote workers” a Trello fan will ask. Well, maybe if we were entirely dispersed I’d use something like Trello but I’m also a fan of co-located teams so even with several remote workers at any one time we have most people working in front of the wall and stick smartphone pictures onto Slack for those not in the office. Local people move cards on behalf of the remotes.

“What about reporting?” asks the Pivotal advocate. Who needs reporting when I can go stand in front of the wall for a few seconds, see where everything is, and ask devs questions about what they worked on and are working on now?

“What about losing cards and work not getting done” says the Jira guru. In those mumble, mumble years, even working with teams of 50-odd devs it’s never been a problem. Very occasionally a card goes missing but we realise that pretty quickly and write it afresh. And you can’t tell me tasks don’t get lost in Jira sometimes; in fact I hear that is more the norm than the exception.

However these objections miss the fundamental point. The real value in using cards, magnets and a wall is in the physicality of it. Cards get written collaboratively, not entered into a system. A card might get written once in a planning session then ripped up and re-written as two or three more cards: it’s such a low-friction process. We use different coloured cards for different types of work, badges to indicate different initiatives, coloured star stickers for priority work. Stuff get’s written on the front and on the back and related cards get clumped together.

And it all just sits in front of the whole team, and indeed the whole company, the whole time.

We had a developer in for interview the other day and she told us her current team had to schedule a daily reminder to make sure their Trello got updated otherwise they forgot to do so. The mind just boggles.

But that’s just the day-to-day mechanics. Last year I got to fulfil a long-held ambition and had a custom wall built to replace the old magnetic whiteboards when we moved into a new office:

In explaining what I wanted to the initially very skeptical designer, who was a big fan of Basecamp, I had to describe the purpose of the wall, not just the function. Here’s what I wrote for them:

“To me, it is not just a functional tool. It is where the challenge of what the technical team needs to deliver is quite literally writ large. You can see the cadence of releases in the ebb and flow of cards on the wall and the hither and thither of developers and QA as they go to move cards.

“It is easy to spot when a card gets stuck in the backlog or priority cards aren’t being progressed. I know when we have just reviewed all the work because the board is all neat and when we’ve got a good chunk of work done as it’s all messy again.

“It is a physical manifestation of what we’re tasked to achieve and we celebrate that rather than hide it away in an electronic tool like Basecamp or Trello.”

A week or so after this exchange with the designer one of our graduate developers had a real aha moment after a couple of months with us. “I think I get it”, they said “it’s not about “here’s the work” [mimes being given a card], it’s about “here’s the work” [waves their hands across the whole of the wall]”. Spot on; not the work you have to do as an individual or a pair but the work we have to do together as a team.

A couple of months ago someone on the same CTO mailing list asked if anyone had an example of an inspiring bit of office design and Jonathon Lister Parsons of PensionBee was kind enough to suggest our wall. Which was such a thrill as that is exactly what it is designed to be: an inspiration to the team.

Technical Details

The Singletrack Wall is constructed of plasterboard mounted on batons attached to the office wall underneath. Glued to this are sheets of 0.8mm ferrous metal (avoid magnetic paint or self-adhesive magnetic sheeting, the magnetism isn’t strong enough) which is covered with mildly textured, hard-wearing wallpaper. Strip lights at the top and bottom give the Wall a ‘glow’ and we have office lighting directly above it so the cards are never in shadow.

The wall is divided into sections using magnetic tape so it can easily be reconfigured. We have a variety of magnets as different people prefer different styles. Our Head of development likes multi-coloured ones, the lead on our analytics product stream only uses black ones (you can see those in the top section of the board) and I love these neat little extra-strong Neodymium numbers. This mixture does nothing for my OCD tendencies but, self organising team, what can I do?

Exciting news: we’re just about to move the Product Engineering team to their own office so I get to build a WHOLE NEW WALL. Any suggestions for improvement?

“But do you still code?” is a commonly asked question when I tell people I’m a ‘hands-on CTO’.

“Yes. Not every day, but most days.”

“Real code? Like production code, not just R&D?”

“Yes, production code.”

This is extremely important to me. In amongst all the other responsibilities I have as CTO I feel that being able to get hands-on with the code keeps me grounded in the reality of delivering our technology, it helps me relate closely with our developers, and it keeps me sane. Well, sane-ish.

Then someone asked me a question no one else has asked before:

“How do you do that?”

As their company grew they had less and less time to code and, when they tried, it just didn’t work. The codebase was so unfamiliar that they struggled to get started and if they got started they never had time to finish. Within just a few months they felt they no longer had the ‘right’ to change a system they had single-handedly built the earliest versions of.

It turned out the answer was simple: an XP process, specifically, TDD and Pairing, and a bit of commitment.

Last week we needed to make a very quick change to an important data processor. The change was subtle but complex so rather than asking the dev team to read the spec from the data consumer, or get a lecture on it from me, I paired in with one of the devs to write the tests and then the new functionality. Pairing on the tests was actually pretty quick which meant that if I had got dragged away to deal with something, someone else could have taken over on the functionality update. But, as it was, a couple of fun hours later we had all the changes deployed for final testing and then into production by the end of the day.

Recently I ran a new static code analysis tool that highlighted a specific set of code quality issues. I wanted to understand the cost and the benefit of addressing the highlighted issues before raising them with the dev team so I spent a few one hour sessions refactoring code to address one or two examples of each issue. TDD is more than just having tests but not for the first time I was thankful for the comprehensive and trustworthy test suite to prove I’d not screwed things up. Making these changes was valuable for me to evaluate the validity of the issues and served as a good set of exemplars for a team-wide initiative to address all the issues and add the tool to the build process.

TDD and Pairing are easy, they’re just how I work. Commitment is much harder but just a few hours a week is all I need to retain a very real connection with our technology and our development practice.

Some time in 1992, the inspirational Bruce Anderson told me I needed to learn Smalltalk. The web was in its infancy and with little information in the University of Essex library my only resources on this quest were a 720Kb 3.5″ disk with Smalltalk/V on it and a videotape. Being an impoverished student I first had to borrow a video recorder and then my introduction to a programming language still more advanced than most of what is out there began.

The video is only twenty minutes long but I had to watch it in two parts due to the headache induced by the lack of vertical sync between the video camera and screen. Bruce tells me the video was made in 1985 at Queen Mary College, London on a Tektronix 4400 series.

Many thanks to Bruce for giving me permission to publish this small but pivotal part of my programming history

I recently discovered that the best way to spend some time in Venice is to buy an all day ticket for the Vaporetto (ferry bus), find a seat at the front, and just take it all in. Over time I found my attention drawn to how the various users of waterways picked their way through the Canal Grande.

Elegant Gondole present tourists with a facsimile of romance and a personal tour of the city’s heart.

Luxurious Taxi d’Acqua provide both the fastest way of getting from A to B and a comfortable way of touring.

Utility barges bring wine and beer (and presumably other goods) into the city, and take the garbage away.

Lumbering Vaporetti move large numbers of tourists and workers en masse through the city.

And so on, all sharing the same small area of water.

Given the vast differences in speed, size and agility of all these craft you might expect a series of strict rules and controls to govern all this, but it appears not. There are no traffic lights or aquatic roundabouts, there is no requirement to stick to one side of the canal or other and no clear right-of-way precedence. What there is is a series of calls and signals that indicate the intentions of the pilots, taxi-drivers and gondoliers to each other. There appear to be a number of common-sense conventions based around the needs and abilities of each craft which are applied, or not, as each situation demands. And the men and women that navigate the canals on a daily basis are highly skilled at what they do.

It is a chaotic system in the physics or maths sense of the word: a complex system whose behaviour is so unpredictable as to appear random. But it is not the randomness of total mayhem, rather the collaborative choreography of a diverse group united in common cause: to service, directly or indirectly, the tourists who are the lifeblood of the city.

Returning to work I was struck by some parallels between the seemingly random and yet ultimately highly effective Venetian waterways and our development process at Singletrack. As a team we have lots of different demands on our time: core product development, support, customer-specific customisations, new customer roll-outs, support of sales efforts and so on. All of these move at different speeds and have different, sometimes competing, characteristics and constraints. Our development process is a mild variation of late-90s XP: a variant because we’re self organising and most of the team wasn’t doing development in the late 90s, and mild because those core practices and principles still work. Its very lightweight, everybody understands it, and its very effective.

I’ve recently begun to worry that this lack of evolution from how I learned to do things nearly two decades ago demonstrates an unwillingness or inability to change. To keep up with the state of the art. I spend time looking at new and different processes and techniques but somehow they fail to engage me and, other than the occasional cherry-picked idea, I go back to what I know.

I realise that it can’t be that Scrum or Kanban or Modern Agile are sham approaches that deliver nothing. It can’t be that NoEstimates and NoProjects have nothing worthwhile to say (even if they do seem to be determined to say it in a particularly negative way). MobProgramming must be good for some people somewhere. But I’ve found them to be limited in value to me.

What my Venetian experience helped me understand is what’s important is that you have a set a of calls, signals and conventions that allow you to collaboratively choreograph your movements, not what that those calls, signals and conventions actually are. The ones that are called XP work for me, maybe the ones called Kanban and MobProgramming work for you. What’s important is that your approach actively works towards the common cause of your organisation not that its the one that gets the most hashtag traffic on Twitter.

When I thought about writing this blog post I did a bit of research on Venice and its waterways. It turns out that what rules and restrictions are imposed had remained unchanged for nearly 200 years until the death of a tourist in 2013 (http://www.telegraph.co.uk/news/worldnews/europe/italy/10268403/Venice-to-control-boat-traffic-on-crowded-canals.html). The response to this tragedy wasn’t to impose a totally new regime of strict controls but simply to update the conventions slightly to reflect Venice in the modern era. Barges have had the hours they can enter the city limited, and illegal docks and piers have been removed, but the fundamentals of the collaborative choreography to suit the common cause remain intact.

So don’t look for the latest or the ‘best’. Look for the calls, signals and conventions that work for you. Find the collaborative choreography that marshals all the disparate people with their disparate needs and disparate talents towards your common cause. Make it work and then keep it working.

Like most enterprise software there is usually a point in our sales cycle where the prospect’s technical team want to do a bit of technical fact-finding or due diligence on us. As we grow one of the common internal complaints is Sales find it hard to secure time from our tech team to attend these prospect meetings. So I wrote a short guide to getting the best out of tech at Sales meetings which I realised is not particularly specific to Singletrack:

“Remember the ZX Spectrum?” the Guardian asked a few days ago. Remember it? That fragile black-and-grey box with the jaunty flash of rainbow colours and precariously attached ‘RAM Pack’ set me on a path that has come to define my entire life.

I remember hooking it up to the battered second-hand black and white portable that served as our family TV in the early 80s and loading in a copied game from tape. Brrrrr, chi. Brrrrr, CHEEEEE etc.

I remember getting it to print ‘Hello World’ 10 times using BASIC, and then modifying that to print ‘Hello Paul’.

I remember the hours of typing Hexadecimal from cheap-paper magazines smudged with sweaty fingerprints. “Is that an E or an F”. Like alchemists we toiled over our steaming cauldron of a computer trying to turn the base metal of those listings into the gold of a simple game. We even succeeded once or twice.

I remember the animated Valentine’s card hand-crafted in Z80A for my first girlfriend. Diagonal scrolling, jaunty tune, teenage poetry. She dumped me the next day for reasons I still haven’t fully grasped.