Author: polsonby

I’m a hands-on product person who loves working in startups and agile software teams to help define, design and build great web applications that people find useful and enjoy using. I’ve built several products aimed at online learning and employability, and am keenly interested in “tech for good” and building significant online products for ethical or social businesses.

Working on Hireist.com, a Rails 5 app which uses Minitest and Capybara, I ended up with some order dependent test failures that completely derailed things for a few days. In the end my brilliant son helped me to solve the problem and hopefully the key points will help someone else!

So, the app is a SaaS product and uses subdomains to separate tenant accounts. When the failure occurred, it was because the @current_account was unset in situations where it should have been and we ended up calling methods on nil. Those tests worked fine on their own, but during the whole test run, something was happening to make it mess up; some state or unsuspected dependency leaking from a previous test.

So I remembered Ryan Davis talking about this in his talk during RailsConf 2015, and how you could use minitest-bisect to do the leg work of trying to narrow down which other tests caused this dependency and failure. So, here were the steps I went through over the course of several days in order to get a test suite we could trust once again!

Step 1 – realise you have a problem!

This bit is easy. If your test suite runs green on one run, then throws up lots of red on a second when there have been no changes to the code, you have an order dependent test failure.

Step 2 – identify the test dependencies

Install minitest-bisect in your gem file in the development group and run your tests. Here’s the first gotcha, however. The readme and instructions are for a generic Ruby app; if you’re using Rails, you need to pass it the right file names to act on and finding the right incantation took a little googling!

Now you can pass the options displayed to the test runner and reproduce the subset of the problem.

Step 3- Identify the conditions which cause the failure

Steps 1 and 2 are, sadly, the easy bit. The tricky bit is then working out what is happening and what to do about it! But at least now you have a workable surface area to attack. In this case, just in case anyone else hits this slightly unusual edge case, our app uses the ActsAsTenant gem and requires a subdomain for particular areas to work properly, so we use a setup method with the host! method to set it before the tests that need this.

def setup
host! "verso.example.com"
end

But the problem was that if prior to those test files running, a feature test was run that used “visit xxx_path”, the host! method in later tests no longer had any effect because the URL had already been set.

feature "View home page" do
scenario "generic home page" do
visit root_path
page.must_have_content "Hireist"
end
scenario "home page for account" do
visit "http://verso.hireist.test"
page.must_have_content "Verso"
end
end

Step 4 – Work out the dependency and fix it!

This is where I hit a brick wall, and my son, a talented Ruby developer, stepped in to help. It took some time but eventually using byebug and pry we were able to step through the code and dig down into the framework to try to find out where the domain was being set and why it leaked across test instances. We found that when a feature test runs, it set the default_url_options to “test.local” and this got set in a class variable – hence the persistence across test runs.

Because our integration test had the relevant classes in its ancestry chain, it turned out we could access the options hash inside our tests and reset the URL options before our test runs, so that the host! method would always have the desired effect.

In our test_helper.rb:

def clean_url_environment
app.routes.default_url_options = {}
end

And in our setup method in the controller tests:

def setup
clean_url_environment
host! "verso.example.com"
end

This was a tough one to fix, and it took sustained effort and more than one pair of eyes to do it. But without the ability to bisect the test runs (thanks to Ryan Davis), we’d probably never have gotten to the root cause.

Sometimes we find ourselves working on stuff that is less than scintillating, maybe discussing the implementation details of yet another pointless cookie notification bar or whether it would be a bad thing to implement one of those things that highlights random words on your site like “gambling” and pops up ads if you accidentally mouse over them [yes! – Ed.]; at such times it’s sometimes tempting to dream you were instead working in Silicon Valley on some ground-breaking project like self-driving cars.

I have an antidote for that though; suppose you really were working on the software that guides a self-driving car. That would be awesome, right? Even the worst implementation of an AI-controlled vehicle is soon likely to be many times safer for the passengers and the world at large than even the best human driver, and there are really smart people working on the problems right now.

I’ve been reading this amazing essay from Paul Ford, “What is Code?”, and in it he explains why if those rockstar ninja developers exist, they’re probably going to have pretty high standards as to what problems they work on and are therefore probably not going to be sitting in the next cubicle in a typical dev team.

They’re not interviewing at your crappy company for your crappy job. They’re not going to come and rescue your website; they’re not going to make you an app that puts mustaches on photos; they’re not going to listen to you when you offer them the chance to build the next Facebook, because, if they exist, they are busy building the real Facebook.

Very astute, right? But it was this next sentence that made me glad I’m not one of them.

Sometimes they’re thinking about higher mathematics, or how to help a self-driving car manage the ethical choice between running over a squirrel and driving off a cliff. Or they’re riding their bikes, or getting really into pottery.

If you’re the one designing the anti-collision software, you’re the one who has to tell it how to decide in emergency situations. So the squirrel case is pretty easy (once you train it to reliably differentiate between a child in a furry hat and a squirrel, but that’s another matter) – bye bye Mr Squeaks, right?

But how would you suggest to your automaton that it handle the sort of situation that psychologists use to bother otherwise happy people; like, assuming that its brakes have failed and it’s hurtling down a hill, able to steer but not stop, and it has to choose the lesser of two evils.

Do you suggest it should stay on the road and aim, for instance, at the mother and a child in a pushchair on the pedestrian crossing (two people with their lives ahead of them), or steer into the bus queue of twenty older people instead?

But what if there are boy scouts in the bus queue too?

If you’re a rockstar programmer and your next question is, “how many boy scouts?” there may be a job for you in Mountain View. Tell them I suggested you give them a call.

Now, what colour would you like your cookie notification bar, Mr Smithers?

It’s funny how SO has become such a big part of our lives as developers.

It’s doubly fascinating for me, because I was there at the birth. Back in 2007 I followed Jeff Atwood and Joel Spolsky’s blogs eagerly; Spolsky, of course, had been an influential tech blogger for many years and reached massive fame for three things; firstly, his company Fog Creek’s popular bug tracker Fogbugz, which many dev teams used routinely. Secondly, Fog Creek’s innovative approach to programmer happiness, like ensuring that every developer had their own office with a door that closes! And thirdly, for the Joel Test, still widely used as a quick objective metric of the quality of a developer team and their processes.

And Atwood? Atwood was always a delightful read; opinionated and funny, and self-deprecating in a way I wish more developers were. His Coding Horror logo, used by permission from Steve McConnell of Code Complete fame, reminds everyone that there is bad code everywhere and much of it is our own; but now we know and try to do better!

So when these two towering figures in the community joined forces and announced they were going to collaborate on a project, of course my ears pricked up. The best thing was, they were going to podcast their weekly project update meetings, since Joel lived in New York and Jeff in California.

Jeff was doing the hands-on work, and for the first few months of the podcast it was hilarious how his estimating (“Oh, 12 to 13 weeks I expect”) style grated with Joel. Of course it took way longer than Jeff thought, but the best bit was seeing the vision unfold; the StackOverflow we see today is not that much changed from how it was after the first six months of work, and we got to experience the decision making process first hand, understanding exactly what lay behind every design decision.

And of course, they were very, very funny to listen to.

When Jeff sent out the first invitations to the Beta, I immediately signed onto the waiting list. And a couple of days later, I got my invite. It felt really special, being one of this elite group of developers from all different disciplines – all collaborating on something to benefit every developer for ever.

So I’m user 137 on StackOverflow. They’re up to over five million now and rising. In the beginning we didn’t have the rules for good questions we have now; we discovered the later rules out of how people used the product and in particular, through Jeff’s excellent judgement about what was beneficial and what was not. We found out pretty early on that questions that solicited lists or subjective judgements were harmful and now such questions are rightly discouraged as low-brow attempts at rep-whoring.

But in the earliest days, I had one goal in mind: to get editing rights, so I could help to improve the questions I saw that needed it and help to curate and preserve this vital body of developer knowledge. And that meant I needed 3,000 reputation, and I did everything I could to earn it while still asking questions that I thought were useful to people.

It took me a while to get there but in a few months I had achieved a reasonable level of editing power and I was content. Real work took me all my time and as thousands of developers began to compete to post the best, quickest answers my useful participations quickly tailed off.

But over the years, those questions from the early days still continue to earn me reputation despite my relative inactivity on the platform. My rep is over 13,000 and it’s all from questions that would never be allowed now. But I’m kind of glad they’re there – they’re part of the social anthropology of the web, and even if they aren’t good questions, every week they’re upvoted by people who don’t know any better but still value them.

So it’s kind of tough now being user 137 – I feel a bit of a fraud because my rep comes from early experiments and earnest rep-whoring of the most well-meaning sort. But I’d never give it up.

The more I learn about OO design (a journey which started in 2007, mind you!), the more I want to apply what I know about good design in my daily work. But it’s hard, being the sole developer building an MVP for a startup where every second counts. There’s an increasing cognitive dissonance between what I know I want to do, like unit testing, tdd, building small POROs for business logic and so on just like Sandi Metz and Avdi Grimm encourage us to do, and what Rails encourages through its file structure and what I end up doing in practice, which is just throwing another method into the User or CourseModule class and getting the feature out the door.

The conundrum is pretty simple; the Rails way doesn’t of itself lead you to write good code, it encourages you to think about everything within its structure, and doesn’t make obvious places for your own classes outside of Controllers and Models. I want to extract stuff out into single responsibility classes, but if I do that, what about Rails? Rails is the reason I’m using Ruby, after all; I want the productivity and programmer joy that comes from using it. But I also want to develop the craft of software design.

So I was overjoyed to come across this blog post by Vladimir Rybas on just this topic. Here he quickly outlines the problem and describes a few current approaches, most of which just don’t feel rails-y – and more importantly, mean you end up creating a completely custom file structure which makes it really hard to predict where stuff is going to be. That’s one of the joys of Rails, that pretty much everything has a home and that it’s consistent from Rails app to Rails app. It’s such a small but important thing to give up.

It’s his fourth one which hit the spot for me – using namespaces and subdirectories within the existing default Rails directory structure. Here’s a quick summary, but do read his post and his rationale, it’s great.

It’s been a few months now since I first came across Sandi Metz and the Poodr. Ever since then I’ve been hunting down her conference videos.

If you’ve not come across her and you are a programmer, you really should take some time to watch her stuff. She did the best video on testing I’ve ever seen.

But this is her first talk, at Gotham Ruby Conference in 2009, and for some reason I’d never come across it till now! I’m linking all this stuff up to make it easier to find and to remind me to keep plugging away at understanding and applying it.

So there I was the other day, adding a feature where users could invite their friends to join Fluency too. The UI is a textarea where the user is asked to enter the email addresses on separate lines, so that’s pretty simple to grab the emails from:
recipients = params[:contact_emails].split(/\s/)

But because the line separator is actually two characters (newline and carriage return) you end up with empty elements in the returned array:
["someone@example.com","","something@gmail.com"]

Enter Ruby; by passing a block to the array, it can test for empty strings and reject them:

and then use a similar technique by passing a block to the array of cleaned strings to verify that they are indeed emails. The snippet above is a simple test – ensuing that there is a dot and an @ sign with at least one character surrounding them in the right order.

Not a day goes by that I don’t find another thing in Rails that delights me and makes life easier. Yesterday it was finding that you can access the Rails application from the rails console by simply calling app:

This returns the entire app instance with all of its methods available to call directly. So helpers you can call in the controller context, you can call on the console as messages to app:

Posts navigation

Ian Anderson

I’m a hands-on product person who loves working in startups and agile software teams to help define, design and build great web applications that people find useful and enjoy using. I’ve built several products aimed at online learning and employability, and am keenly interested in “tech for good” and building significant online products for ethical or social businesses.