Category Archives: Organizing Systems

A place that I used to work ran periodic ‘Hackathons[1]’. After trying to describe them to various people it became clear that there were a number of different definitions of what a ‘Hackathon’ could (or should) be, so here’s my description, with some thoughts as to why structuring it this way might be a good idea.

Definition?:
What is a ‘Hackathon’? It’s a lot of different things to different people. Most definitions I’ve seen see it as an opportunity to spend a day (often 24 hours, or a weekend) building something that they wouldn’t normally build. The thing built is not necessarily a ‘thing’. It could be a website, and app, some other type of computer program, it could even be an organization. The important thing here is that whatever is created/built is taken from the concept stage to working prototype with at least some useful feature(s) by the end of the hackathon.

Purpose:
Why do you want to have a Hackathon? The ones that I’ve been involved with were an opportunity for people in a software organization to try something a little different for a day. Some reasons they did it:
– Learning a new skill or programming language by building something ‘real’
– ‘Scratching that itch’, solving some problem that they never quite get the time or priority to solve in their day-to-day
– Working with people that they don’t normally work with
– Building a full product (instead of working on a tiny piece of a huge system)
– Building a visualization tool
– Doing something totally different

Those involved generally seemed to greatly enjoy the experience, the chance to work on something different, to push themselves in a new and different direction, and perhaps the chance to receive the acclaim of their peers.

How did it work?:

There was a committee formed to organize the Hackathon. They were responsible for:
– Publicity
– Getting buy-in from management (this had already happened, so this part was relatively easy)
– Convincing judges to judge the competition (These were usually senior people in the organization who were not part of a ‘hack’)
– Organizing the various parts of the event
– The ‘pitch’
– The presentations
– The prizes
– The voting for the ‘Audience Choice’
– A/V and some method for telling presenters their time was up (we used a stop light)
– Finding sponsors for any ‘Sponsored Hacks’
– All of the various other small things required to run an event like this
– Buying the pizza for the party after the presentations

How often did they happen?
– The Hackathons happened once per quarter, generally in the middle of the week

A couple of days before:
– Run the ‘Pitch Session’
– Provide a place (usually a wiki page) for people to join groups following the pitches

The first day of the hackathon (The hackathon would run noon to noon, with presentations 3-4pm the following day):
– Start the hackathon, giving any support where necessary

The second day of the hackathon:
– Collect presentations, so they can be presented in a timely manner
– Collect the judges, so they can judge the competition
– Run the presentations
– Run the ‘Audience Choice’ vote
– Count the ‘Audience Choice’ votes
– Distract the people with pizza while the judges are deliberating
– Help the judges present prizes

Some more details:

A ‘Pitch Session’ is:
– Each person gets one minute to talk about their idea, in the hopes that they can attract a group of people to work on it with them. There was a rule that teams had to pitch something if they wanted to win ‘best hack’, to encourage them to participate in the pitches and include others

How are teams formed?
– Teams can be of any number of people, but we never saw a team of more than 12 people or fewer than 1 person[2].
– To encourage different parts of the organization to work together, each team would be awarded points for each group represented in the team. Including someone from ‘Customer Experience’ or ‘Marketing’ would be worth three points, while including people from Engineering (the expected default for a hackday) would be worth 1 point
– Teams, once formed are added to the hackday webpage, for posterity, and so people can coordinate (and so the organizers can coordinate collecting all of the presentations)

What happened during the 24-hour Hackathon period?
– During the 24 hours of ‘hacking’, people generally put their project work aside and work on their hacks. Some people were given more or less time to do so, depending on their particular management chain and the urgency of their specific work at hand. Of course, if a production issue cropped up in the middle of the day, that would have priority.

How did presentations work?
– Each team was given 3 minutes to present. Luckily, we had a traffic light that was repurposed to give an easily visible signal to presenters when they were running short of time.
– There was generally an audience, of the other people hacking, the judges, and whoever else wanted pizza later

How were they judged?
– ‘Best Pitch’ (a friend of mine routinely won this part, due to his uniquely hilarious presentation style)
– This was generally awarded based on presentation originality and style
– ‘Most Productizeable’ (Which hack was easiest to productize for customers, either internal or external?)
– ‘Best Hack’ (this is the ‘best overall’ category)
– Not necessarily the one that won any other category, but the best overall
– ‘Best Presentation’
– Similar to ‘Best Pitch’, but generally a higher level of polish (and humour) is expected
– ‘Best Sponsored Hack’
– This is like ‘Best Hack’, but restricted to the specific sponsored category
– Hacks would be graded on the criteria above by the judges, IIRC on a 1-10 scale. (There may have been other criteria which were then rolled up into the categories above. That would be up to any event organizers, should someone wish to take these instructions and run with them.)

How did ‘Sponsored Hack’ work?
– This was a later innovation after some people saw the power of the various hacks that had taken place.
– A person or persons within the company would put up money for prize(s) for the best hack that would fulfill certain criteria. There was one hack to link a system to a particular enterprise solution, and one hack to use a new internal API that had been developed
– There was generally only one ‘Sponsored Hack’ per hackathon, and it was a bidding contest to determine which one would be the official ‘Sponsored Hack
– We found that having super-clear criteria about what constituted a ‘successful hack’ was extra important when the prize money for ‘Sponsored Hack’ greatly outweighed the prize money for ‘Best Hack’

What were the prizes?
– Generally gift cards of some type, glory, and a trophy
– The gift cards would be in the $50-100 range for first place. The glory was the really important part.
– Part of the trophy process was the expectation that each group would modify the trophy in some way before presenting it to the next team at the next hackaton
– We had some issues with one of the sponsored hacks when the prize money reached into the hundreds of dollars, because we had not clearly defined ‘When is a hack good enough to be considered a successful hack?’, and the difficulty of the particular hack

What types of ‘hacks’ did you see?
– There were a number of visualizations of various parts of the system
– There were a number of creative front-end interfaces for various parts of the system
– There was a one-line fix to a bug, in a effort to win ‘most productizeable’ by already being in production
– There was a musical number
– Once, the entire team of interns worked together on a hack
– We had an issue with not being able to tell whether our non-bookable meeting rooms were occupied, so one group made some lights with door sensors to quickly communicate down the hall that a room was occupied or not
– And many others…

Pleaes drop me a line if you want to run one of these. They’re a lot of fun, and can really help people get to know others and build an ‘esprit do corps’ in an organization.

Recently, I made the decision to delegate[2] the running of most of our team meetings to my reports. Typically, we have daily five[3]-minute standups, bi-weekly Sprint Planning, the occasional brainstorm, and a weekly meeting with stakeholders.

We now do a rotation, with each member of the team (including the interns) running the daily standups and bi-weekly Sprint Planning meetings in turn.

I’ve had to learn some things about delegating, but that’s a story for another post.

For today, I wanted to talk about some of my observations about how to run (what I think is) a good retrospective.

1) Adjust any tickets which have changed status since the daily standup[4]
2) Close the Sprint
3) Retrospective
4) Choose a name for the new Sprint (Generally the most difficult[5] part)
5) Add items to the new sprint, prioritizing and estimating as we go[6]

It wasn’t until I watched others running a Retrospective that I understood a lot of the small things that I do that make a difference.

The (way I see) steps to a Retrospective are as follows:

1) The Facilitator draws the visualization[7] to start people thinking:

The facilitator writes something akin[8] to the following diagram on the board:

Went Well Not So Well
*---------------*----------------*
| | |
| | | In Our Control
| | |
*---------------*----------------*
| | |
| | | Not In Our Control
| | |
*---------------*----------------*

The point of this is to help the group think about and distinguish things that are in their direct control and out of their direct control. Many groups will come up with a definition for this after their first argument, often placing the line of ‘direct control’ at the edge of the group in the meeting/team in question. There are also many ways to write ‘Not So Well’, which I won’t get into, except to say that I prefer the hopeful phrasing. 🙂

Overall, you can think of your group process as bringing issues from ‘Not So Well/Not In Our Control’ to ‘In Our Control’, then ‘Went Well’, then noting new issues and starting the cycle again. We had one retrospective a while back where basically everything was ‘Went Well/In Our Control’, or ‘Not So Well/Not In Our Control’, so we ended up brainstorming ways to get things under our control, or at least influence, so we could fix them.

It’s not necessarily the worst thing in the world if most items are in the ‘Went Well/In Our Control’ category, but this may also mean that there are underlying issues that you may need to ferret out in your one-on-ones (You *are* having one-on-ones with your team, right?). One such issue came up recently in a one-on-one, and when it was brought up in the Retro., I hijacked the meeting for half an hour to have a specific brainstorm on that topic, to make sure it was covered in depth.

2) The facilitator asks the team to write things went well or did not go well on sticky notes and to put them on the board:

Each member of the team writes happenings from the past two weeks on sticky notes and posts them on the diagram. We use standard-sized sticky notes, and sharpies, so that the notes can be (mostly) read from across the room. My understanding of why this is done with sticky notes is so that people feel some measure of safety and anonymity, and therefore are more likely to express what they really think.

When running this part, I ask people to think along process lines. It’s helpful (and sometimes nice!) to talk about the fact that a particular project went well or poorly, but it’s often more helpful to talk about the parts of our process or other teams’ process that affected the outcome. Identifying process issues and fixing them is what really makes the difference here.

3) The facilitator clarifies each of the posts:

In some order, the facilitator goes through each (group of) post(s) and invites the author to clarify anything ambiguous, such that the whole group understands what each post means (often caused by my handwriting… :/ ). This should only take a few minutes. As they go, the facilitator will often do step 4). Along the way, some problems may be resolved simply by bringing them up, but some may require more thought/brainstorming. Those conversations are shelved until a later step by the facilitator.

As far as ordering, we generally talk about what went well first, probably for psychological safety reasons.

4) The facilitator de-duplicates the posts:

This is more of an art than a science. In some serious brainstorms, the facilitator will call a break at this stage, because it is non-trivial to get correct, while being quite important.

The goal is to group the posts into groups that are likely to have similar solutions. I don’t have good advice here, but it is often a good call for the facilitator to ask the group if two things go together, if they seem to sound similar. More insight on this will come with experience[9].

5) The facilitator decides whether to continue:

The facilitator decides whether any of the posts require a more in-depth conversation. This will likely have become obvious in step 3). You can always check in with the group, if you’re unsure.

6) The facilitator invites dot voting to determine the order for further discussion:

If there are multiple items to address, the facilitator invites the team to come up and ‘dot vote'[10] on which items they think are most important. A good number to choose seems to be 1/3 to 1/2 the number of items to be voted on. If people complain about the number of dots, you can change this, or remind/tell them that they can put multiple dots on items.

7) The facilitator counts the dots and orders the items in dot priority order from highest to lowest

Starting with the item which had the most dots:
a) Make sure everyone understands what the issue is. If this is really unclear, you can brainstorm a list of ideas to try to get to the root cause[11].
b) Brainstorm solutions. Some of these will have obvious actions and owners. For things which are not so obvious, you may want to dot vote again.
c) Make sure you clearly write down (and perhaps make into tickets) any actions which arise.

d) Continue until no items with dot votes are left, you run out of time, or your group becomes unengaged.

That’s it! You’ve now run your first Retrospective. Let me know what you think in the comments below!

[1]Wow, I write a *lot* about meetings…

[2]Stay tuned for a post about ‘Power to the Edge’!

[3]Our daily standups are trending to the long side these days, perhaps up to 10 minutes, but people seem to find them useful, specifically the conversations to solve problems that can more easily be initiated when you know people are already interrupted.

[4]It’s always nice when people update their tickets as they’re doing things, but it’s not necessary. We’ve had success with doing it in our daily standup.

[5]I am not joking. Try it with your team and see for yourself. I went to an auction at a gaming convention during my youth, and the item which went for the largest amount of money was a book to assist people in making character names. Making good names is *difficult*.

[6]’Just-in-time’ planning. This also feels like a separate topic for a post.

[7]This is a whole topic. I like this particular visualization, but you could see that many others could work just as well. The following [8] is another.

[8]There are a number of ways to draw this. It’s sometimes good to change it up, to help people think about things differently. Another popular diagram is a ‘speedboat’ diagram, with wind, anchors, obstacles, and goals as the four categories. It’s not an exact mapping to the above, which has it benefits and drawbacks.

[9]I should write about this, too.

[10]’Dot Voting’ is where you give each person a number of ‘dots’ that they can place on the things they’re voting on. In this context, we use it to choose which thing(s) to talk about next. Some people insist on only one dot from each person per topic, some are more flexible.

[11]’Getting to the root’ in a Brainstorm is a really interesting topic. It’s non-trivial, and deserves its own write-up.

A friend of mine recently posted a few questions about fb about life coaching. I felt that I had more to say than could be conveniently be expressed in a fb comment, so you’re getting it here.

The questions they asked were (mild editing for clarity):

0) ‘I wonder if now’s an opportune time for me to try it’
1) ‘General opinions, and beliefs about its effectiveness in various contexts’
2) ‘What can it do (for you, or more generally), and how does it do that?’
3) ‘How do I find someone really good and really compatible with me and my goals?’

0) ‘I wonder if now’s an opportune time for me to try it’

I’m a firm believer in the idea that the best time to do something is ‘now’. There’s a great story about a famous barbershop coach (Greg Lyne, I think). He was talking with the leadership of a chorus about him giving them some coaching, and they said something like “We’re looking for a five year plan to get better.” His response was “Why wait? Be good now!” Similar to Agile software development practices, you want to test your ideas and theories as soon as possible in as-close-to-real-world-situations, so that you can iterate on them. If you have an idea that might help you improve everything about you and your life, why would you wait to try it out, especially if it might take some time to ramp up?

1) ‘General opinions, and beliefs about its effectiveness in various contexts’

I feel like in our culture (and probably many others), it is considered a sign of weakness to ask for help. And yet, we do this every day. Every time that we exchange money for a good or a service, we are asking for help. You could learn to make your own shoes, or you could perform some task where you have more of a competitive advantage, receive money for that, and then exchange that money for shoes. By specializing[1] like that, we make our modern civilization possible. A Life Coach is a specialist in helping you achieve your potential, whatever that might be.

So, what is Life Coaching? I see it as:

– Helping you achieve clarity on your goals
– Helping you figure out how you are ‘getting in your own way'[2] of achieving those goals
– Helping you work through yourself to make progress and eventually achieve your goals

I’ve personally found it useful for ‘getting unstuck’ in job and career, for helping me unlock my love of writing, and for helping me set boundaries in various parts of my life. But I think the clarity it can bring is the key, and the foundation upon which all other things are built. If you know exactly what you want, and why, you can be so much more focused and effective.

2) ‘What can it do (for you, or more generally), and how does it do that?’

I think I’ve answered much of this under question 1), but I’ll go a little more into the ‘how’.

First, you want to figure out what your goals actually are. From my experience, this often includes some individuation and separation of your ‘social self’ (what others want from you) and your ‘essential self’ (what you actually want on the inside).

Next, you want to figure out how you are preventing yourself from doing these things. You may be plagued by self-doubts brought on by years of exposure to certain types of people, you may be a perfectionist who never starts anything because it will never be good enough, you may be spending all of your time trying to please others, and never taking any time for yourself. This seems to me like a very personal and individual process, involving a number of exercises designed to help you to better understand yourself and your interactions/experiences.

Then, now that you know your goals and how you’re preventing yourself from achieving them, you make plans and start to work towards these goals.

It’s an iterative and very personal process, but it can be tremendously helpful. As I mentioned above, I’ve personally found it useful for ‘getting unstuck’ in job and career, for helping me unlock my love of writing, and for helping me set boundaries in various parts of my life.

3) ‘How do I find someone really good and really compatible with me and my goals?’

Like finding a job or a life partner, good fit with a life coach is very important. I don’t have any easy rules to follow here. Ultimately, you’re at the mercy of your ability to judge people (and more importantly, how you feel around those people).

I would treat it as an interview, the type where you are interviewing them in the same way that they are interviewing you. See if the types of questions they’re asking might help you achieve clarity. See if they are expressing realistic expectations about what a life coach can and cannot do (you yourself need to be engaged, and it can take months). But perhaps most of all, see if you trust the person you’re talking to[3]. Do you feel comfortable talking with them[4]?

With all of the questions above, you can always just simply ask them of a prospective life coach, and see how they answer. You can glean a lot of how and whether they share your values, and how they will approach things. Read their website. Read any testimonials they may have.

If you trust your gut[5], and make sure it feels right, you should be okay.

20 years ago, I watched Contact in the theater with my family[1]. Tonight, I watched it again, with S.

To me, it held up well as a movie. All the characters were believable, and the science and the effects were well within the normal parameters of suspension of disbelief.

What struck me[2] was how hopeful a movie it was, that our better natures would win out, that our endless curiosity would take us places we’ve never imagined.

[Note that spoilers follow]

It’s always interesting the things you remember 20 years later. “Why not make two, at twice the price?” The destruction scene. The prime numbers sounding so ominously alien from the aether. The speaking through her father. The 18 hours of static[3].

Interestingly, I had remembered that 18 hours of static as being the vindication at the end of the movie, that she was not crazy, that something had indeed happened, but I had forgotten how much it was covered up.

The one (gaping) plot hole I had missed the first time around was the absence of study and testing before a human was sent through the machine. If you look at the history of the Apollo program, you see that it was preceded by Mercury and Gemini, with dozens of sequential missions, each testing new parts, to make sure that each part of the system and plan were well-enough understood to ensure successful missions. The idea that they would build a half-trillion-dollar system in Contact and not fully study it (especially if it’s generating strange EM radiation) before sending a human through it ‘strains credulity’. Even the EM it’s radiating would be a fantastic discovery for humans.

But I can understand how they would cut out things to make a move that was watchable, and which was able to spend its time focusing on the humans in the story.

The alternative view of events that the NSA directory was trying to convince people of at the end of the movie was reminiscent (for me) of the big con[4] at the end of ‘Watchmen’, albeit at the opposite end of the hope-fear axis.

Apparently, like Bladerunner, the ending was supposed to keep your doubt alive as to whether the events she experienced had actually happened. To me, it didn’t, as 18 hours of static (and whatever metallurgical data they could get from the sphere) would be enough to prove the story.

I laughed, I cried, I am full of hope. A new year dawns. Time to use that hope to build something meaningful, starting with some words.

[1]We immediately followed it with Men In Black. I’ll leave it to you to enjoy this juxtaposition.

[2]If you’d read or watched any Carl Sagan, this would probably not be surprising. “The sky calls to us. If we do not destroy ourselves, we will one day venture to the stars.”

[3]I had remembered it as 18 minutes.

[4]In ‘Contact’, it was posited that a billionaire had faked first contact to inspire humans to push themselves outwards. In ‘Watchmen’ (the graphic novel[5]), Adrian Veidt fakes an alien invasion to scare humans into working together against a common foe.

[5]’Watchmen’ the movie simplified the plot to have Doctor Manhattan be the scapegoat. this lead to a much tighter movie, but slightly less appropriate for my analogy, however much he played with space and time.

Having a say:
– Each vote should have the highest probability possible of changing the representation of the House of Commons

Quick:
– The public should know the results within hours of the polls closing.

Fair:
– Political parties should not be significantly inconvenienced by the electoral system for not having money.
– Any barriers to entry should be reasonable (number of candidates to be a registered party, number of votes to get deposits back, percentage of popular vote to qualify to get seats, etc…)
– The system should not unduly give power to very small groups (49/49/2 split, the 49 and 2 have equal power).
– The system should be ‘simple enough’ for people to understand. Currently, people vote for one person, one party with the same vote. A similar system being successfully used elsewhere in the world is a reasonable way to determine ‘simple enough’.

Representative:
– There are a number of ways to be representative:
– Geographically
– Representation of party by population
– Minority groups
– Diversity of opinions

Resistant to cheating:
– Secret ballot to reduce intimidation and coercion as factors
– Reasonable voter ID laws to increase voter turnout while keeping the risk of personation low.
– Distributed counting makes the current system quite resistant to cheating. One would have to mess with the voting tally computers in real-time to change this. The fact that there is an anonymous paper record of every vote cast in the ballot boxes is also an important check on this system.

As the Canadian government has (very likely) decided that whatever the parliamentary committee has decided will go to a referendum, I’m going to add one more criterion:

This is the current system in Canada. The country is divided up into ridings (currently 338) of approximately equal population (generally geographically larger ridings have less population per riding.

Advantages:
– Simple
– What we’re currently doing

Disadvantages:
– Vote splitting by riding (candidates can win a riding with less than 30% of the vote)
– Vote splitting across the country (a party can win a majority government with less than 40% of the popular vote)

Single Transferable Vote (STV) is used for elections in Ireland, Malta, much of Australia, and various other parts of the English-speaking world.

Basically, the country or region is divided into single-[2] or multi-member ridings. In each of these ridings, voters rank the candidates on their ballots. Each candidate who receives more votes than the number required to be elected is elected, and all of their ‘extra’ votes are passed on to other candidates proportionally. If there are no candidates who have the number of votes required to be elected, the candidate with the fewest number of votes is eliminated and their votes are redistributed as above. Example here.

Advantages:
– More proportional representation than First-past-the-post
– ‘wasted votes’ guaranteed to be less than (1/(# of seats per riding)*100%), or for example <33% for a riding with three possible elected candidates
Disadvantages:
- You have to have all of the votes in one place to count them
- Ridings must have many candidates per riding to reduce the number of 'wasted votes'
Supplementary Member or ‘Parallel Voting’:

Technically, Supplementary Member Voting or Parallel Voting is defined as combining any two (or more) voting systems in parallel. Most often, it is used to combine some proportionality with a First-past-the-post system. Voters would vote twice, once for their local riding, and once for a proportional slate of candidates. These votes would be separate, leading to the results being more proportional, but not fully proportional.

Advantages:
– More proportional than First-past-the-post

Disadvantages:
– Not really that proportional
– More complicated than First-past-the-post

Instant-runoff voting is used in various elections in Australia, India, Ireland, Papua New Guinea, and various local elections around the world, as well as by some political parties.

Similar to Single Transferable Voting, voters rank candidates in order on their ballot. If one candidate has a majority of the votes, that candidate is elected. If no candidates have a majority of the votes, candidates are eliminated and their votes are redistributed according the the voters’ preferences until one candidate receives a majority of the vote

Advantages:
– More votes count than in First-past-the-post, as no candidate can win without the plurality of the votes in a riding.
– ‘Vote splitting’ is much less of an issue, as parties or candidates who would normally ‘split’ votes would tend to be likely to be the second choice of those voters.

Disadvantages:
– You have to have all of the votes in one place to count them
– Up to half of the votes in each riding can be ‘wasted’

Mixed Member Proportional Voting is used in Germany and various sub regions of the United Kingdom. It was the subject of the failed Ontario referendum of 2007. In most implementations, voters have two votes. One vote for a local candidate, and one vote for a party. Local candidates are elected using a First-past-the-post system. There are an additional number of representatives elected to bring the results in line with the popular vote. These additional representatives are generally based on party lists, but some proposals have them selected on a more regional basis, to allow better regional representation.

Advantages:
– In most cases, as proportional as electoral systems get
– Includes a strong local representation element
– Should be easy to describe to the public

This tends to lead to voter disillusionment, as many voters (rightly) believe that their vote has no chance of influencing an election. The ‘Per Vote Subsidy‘ was one attempt to rectify this, by counting votes to fund political parties, so voters could feel that no matter where they were voting, their vote was doing something.

So, we want to change this system. What do we want out of a voting system?

At its most fundamental, the goal of a voting system is to provide a system for a peaceful transition of power. The way voting systems do this is by making people feel like they have a say in that transition of power.

At the same time, you want the system to be quick, fair, and resistant to cheating (as there are millions of lives and hundreds of billions of dollars at stake).

(I’m also assuming that we will continue to have a representative democracy, and the number of representatives will remain approximately the same. I’m also assuming that there will be political parties in whatever new system we come up with.)

So: having a say, quick, fair, representative, and resistant to cheating.

Having a say:
– Each vote should have the highest probability possible of changing the representation of the House of Commons

Quick:
– The public should know the results within hours of the polls closing.

Fair:
– Political parties should not be significantly inconvenienced by the electoral system for not having money.
– Any barriers to entry should be reasonable (number of candidates to be a registered party, number of votes to get deposits back, percentage of popular vote to qualify to get seats, etc…)
– The system should not unduly give power to very small groups (49/49/2 split, the 49 and 2 have equal power).
– The system should be ‘simple enough’ for people to understand. Currently, people vote for one person, one party with the same vote. A similar system being successfully used elsewhere in the world is a reasonable way to determine ‘simple enough’.

Representative:
– There are a number of ways to be representative:
– Geographically
– Representation of party by population
– Minority groups
– Diversity of opinions

Resistant to cheating:
– Secret ballot to reduce intimidation and coercion as factors
– Reasonable voter ID laws to increase voter turnout while keeping the risk of personation low.
– Distributed counting makes the current system quite resistant to cheating. One would have to mess with the voting tally computers in real-time to change this. The fact that there is an anonymous paper record of every vote cast in the ballot boxes is also an important check on this system.

Interestingly, the current system seems to do most of the above well, except for representative part (and the current voter ID laws).

Next time, we’ll look at a list of options to increase the representativeness, and see how they affect the rest of the criteria.

How do you design a self-driving car to appropriately value human life? Can you use a Facebook group to speed the development of philosophical discourse?

The ‘Trolley Problem‘ is a problem in ethics, first known to be described in its modern form in the early 1950s. Basically, it boils down to the question:

If you have a choice between action and inaction, where both will cause harm, but your action will harm fewer people, is it moral to perform that action?

Interestingly, people answer this question differently, based on how active the action of harm is, the ratio of people hurt between the choices of action and inaction, and other reasons.

The astute will notice that this type of decision problem is a very common one, the most obvious being in military applications, but also vaccines (and invasive health procedures in general), firebreaks, and perhaps the canonical example, automobile design and manufacturing.

This type of decision making has become even more important with the advent of self-driving cars:

Now, you can argue that this page is purely for entertainment, but I think there’s a lot more hidden there. There is a fomenting and interchange of ideas, much faster and more fluidly than at any time in history. The person who writes the next book[6] on the ethics of decision making could well be influenced by or be an avid user of a site such as this one.

They may have started with Rick-rolling, but image macros are helping the advancement of human knowledge. Stew on that one for a while.

“The creator might argue that his robot is an ‘individual’, capable of his own decisions, while the opposition would say that he (the creator) is responsible for the algorithm that led to the action. Imagine this happening – it would give birth to one of the greatest on-court debates ever.” From Patrice Leiteritz via Trolley Problem Memes

[1]If everyone cooperates, overall they will receive a better result, but if any one of them betrays the others, they get an even better result, but everyone else’s result is much worse. This theoretically leads everyone to betray everyone else, leading to everyone having a worse overall outcome.

[2]People also like the feeling of control.

[3]Check out the article. Apparently, the Sophists were the first (recorded) right-wing think tanks.

[4]My undergrad Philosophy 101 prof. made the argument that because philosophy was not useful for anything else, it must be inherently be useful (and that that was better).

Today, we’re going to look at types of coding and algorithm questions. As discussed before, these can be divided up into ‘Problem Solving’ and ‘Knowledge’ questions.

As mentioned before, ‘Knowledge’ questions are very close to ‘human glossary’ questions. ‘What is the Big-O order of QuickSort? Average case? Worst case?’.

But there are some questions which straddle the line between knowledge and problem solving, answers that few but an expert in that topic would be able to exactly recall, like ‘what exactly happens between when you type google.com into your browser and the page appears?’, or ‘compare and contrast various sorting algorithms’.

For those questions, you have to be as widely read as possible, they tend to select for those who are more naturally inquisitive for things outside their specific area of expertise.

Now, for coding questions. There seem to be a few different types, which I’ll try to separate out by data structure[1]:

Arrays and Strings – Any data structure where any element is addressable in O(1) time, where elements are allocated together in memory.

Linked Lists, Stacks, and Queues – Data structures in linear form, where elements far away from the origin are O(N) difficult to access.

Trees – Data structures arranged in a tree form, with a clear root and directionality. Often sorted.

Graphs – Data structures with nodes and edges, where the edges can connect the nodes in arbitrary ways. Home to at least the plurality of the known NP-Complete problems. Note that Graph problems are a superset of the above.

Search and Optimization – Problems where you’re trying to find an optimal (or close to optimal) solution in a large multidimensional tensor or vector field. Many in this category are easily mappable to Graph Theory questions, but many are not, such as 3-D Protein Structure Prediction. Most interviews would likely not ask questions in this category, at least not very complex ones.

Machine Learning and Statistics – Somewhat related to Search and Optimization, problems dealing with how one trains a computer to solve a problem. Likely to become more and more important.

Hashes – Data structures where space is traded for speed. Generally assumed to have 0(1) insertion and retrieval

In previous posts, I’ve talked about the most important types of interview questions:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Today, I’ll cover some other types of questions that are known to not have much predictive power, but people still ask, either as an ice breaker, or because they have other reasons for asking these questions.

‘Ice Breaker’ questions ask ‘tell me a story about yourself, to help relax you.’

The purpose of ‘Ice Breaker’ questions is to get the conversational flow started. My personal favourite is ‘tell me about the project you’re most proud of’, because it will help to relax the candidate, and has the dual purpose of showing what a candidate is like when they’re excited about something.

From the link, examples might include “What kind of animal would you like to be?” or “What color best describes you?[1]” The ostensible purpose is to try to get beyond pre-programmed/rehearsed answers, looking for original thoughts. (I tend to prefer the ‘tell me what you’re most proud of’ type of question, as if you’re trying to knock a person off their rehearsed interview game, if they’re nervous, that might torpedo them, and you’re torpedoing them based on their interview skills, rather than actual skills. Better to choose a topic they know, and explore the limits of their thinking there.)

‘Illegal’ questions ask ‘I want to discriminate against you, in some illegal way’

Which questions are illegal will vary by jurisdiction, but generally include questions about things such as gender, age, marital status, religion, etc… Larger and governmental organizations tend to be better at not asking such questions, whether because of visibility or lawsuits. Knowing how to answer such questions can be tricky, because of the power differential between interviewer and interviewee, but especially because the organizations asking such questions may be hiring from a labour pool with few options.

‘Brainteaser’ or ‘Fermi‘ questions ask ‘How many piano tuners are there in New York’?

Next time, we’ll go more in dept about specific types of technical questions. Stay tuned!

[1]My favourite story on this topic comes from the brainstorming exercise: “List all the things you could do with this brick.” People would come up with some small number of ideas (like <10) for how to use the brick. Then the facilitator would say something like: "List me all the ways that your wackiest friend could use this brick." Interestingly, this generally elicits many more ideas, as it removes some of the social opprobrium of being 'weird'.
[2]cf. The British Empire no longer uses the ‘Imperial’ system.

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Today, I want to talk about ‘Technical’ questions. This includes two types:

‘Problem Solving’ questions, where the interviewer asks a technical question, and expects you to go through some process to solve it, similar in some way to what one would do in a job in the field.

‘Knowledge’ questions, where the interviewer asks specific questions about your field of study or work. For a programming job, they might be about memory management or data structures, for HR, they might be about what is legal or accepted practice in the jurisdiction in question, etc…

(Note that these generally don’t include questions about a resume, which I would group under the ‘Behavioural’ umbrella, as the interviewee is expected to tell a story about them.)

So what is an interviewer looking for in these questions?

For both of these questions, the interviewer is looking for command of the subject matter and problem solving ability. There’s a whole smear of possible questions between these two extremes. (‘What is an array’ to ‘Design LinkedIn’.)

For basic knowledge questions, it would probably suffice to re-read a textbook, or read (and understand!) a glossary of the topics one would be interviewed in.

For ‘Problem Solving’ questions, answers are generally more involved.

Generally, the interviewee is given a problem statement:

“Write a program which counts from 1 to 100, and outputs ‘Fizz’ when the number is a multiple of 3 and ‘Buzz’ when the number is a multiple of 5.”

This problem statement may or may not be well defined, so it falls on the interviewee to ask questions until it is adequately defined:

“Does it also print the number when it is a multiple of 3 or 5?” “Is proper syntax required?” “What language?”

(This also makes sure that the interviewer and the interviewee are on the same page.)

I like to draw a large diagram, and/or write down my assumptions in the upper-left corner when doing problems like this. Makes things explicit, people can see what you’re thinking.

One of my best bosses described his best programmer as ‘having a reason for every single line of code’. Talking through one’s code as it’s being written can help with this.

So:

Write down assumptions
Draw a big diagram
State the overall algorithm
Write down the solution, while talking about it
Think about corner cases, run an example through in your head.

Next time, we’ll talk some other types of questions, the kinds that are known to be not as predictive, but that interviewers still ask anyways, for various reasons. Stay tuned!