Behind the Scenes

I'm Chris Saunders, one of Shopify's developers. I like to keep journal entries about the problems I run into while working on the various codebases within the company.

Recently we ran into a issue with authentication in one of our applications and as a result I ended up learning a bit about Rack middleware. I feel that the experience was worth sharing with the world at large so here's is a rough transcription of my entry. Enjoy!

As Shopify and our merchants have grown, the sheer amount of data we deal with has grown as well. With that in mind, we've been forced to transition towards ID columns (and ID references) from 32-bit to 64-bit to ensure that we can continue to issue unique IDs.

MySQL, Ruby on Rails, and most other technologies default ID columns to INT(11). While we're very excited that we're hitting this limitation (problems caused by massive growth are the best problems to have), it is going to require Shopify App and integration developers to make the same transition to ensure compatibility with Shopify in the future.

In short: anywhere you're dealing with an ID from Shopify (whether that's shop_ID, order_ID, or any other ID), you need to be prepared to deal with integers larger than 32-bit datatypes can provide.

The apps/API team here at Shopify recently started providing API support through our tag on Stack Overflow. This replaced our previous setup, which was a Google Group. I'd like to explain a little bit about the descision to go this route as well as the aftermath.

One question we get a fair bit from merchants is 'Can I trust the apps in the Shopify App Store?'. This is a valid concern, especially as a lot of apps are a) charging money and b) getting access to a lot of shop data.

We've always had a thorough vetting process in place for apps and I talk about it with developers all the time but we've never explained it for merchants. Until today that is. Here's a short video that whisks through the app review process to show you what happens before we have confidence to publish an app in the App Store. Enjoy!

Shopify Makes Fast Company’s “50 Most Innovative Companies” List

For democratizing and automating ecommerce tools. Shopify offers pre-made templates that allow people to quickly and easily set up an online store without needing to know how to code a website. Shopify creates tools and templates to power online storefronts. (Notable clients include Rovio, Angry Birds’ parent company, and GE.) Shopify has grown to almost 20,000 storefronts in 88 countries, which did a combined $275 million in online sales, up from $120 million in 2010. Up next: Making it as easy to buy sell to mobile customers.

Join Us, It’s Bliss!

If making the Fast Company list doesn’t impress you, maybe my earlier article about why Shopify’s a great place to work will. From the company’s success to interesting projects to the way we get stuff done to the cool gear that’s standard issue for Shopify employees (see the photo above; you get to pick between a MacBook Pro and MacBook Air these days), there are reasons aplenty to hitch your wagon to Shopify’s star.

Shopify is looking to hire a Software Engineer for our growing Applications Team. The Applications Team is responsible for building supported Shopify Applications for the Shopify App Store as well as 3rd party applications. If you are interested in working on challenging Ruby on Rails projects with a team of highly motivated and talented individuals then this position is for you.

Shopify is looking to hire a Software Engineer to maintain and extend our sophisticated SaaS billing platform servicing over 20,000 merchants. The Shopify billing system is a core piece of infrastructure that handles millions of dollars of financial transactions. If you are interested in working on challenging Ruby projects with a team of highly motivated and talented individuals then this position is for you.

You may have heard or learned from painful experience that job fairs are like this:

Unspace’s gatherings are a little more like this:

The two photos above were taken at an event that they threw called Technologic, which took the typical evening tech seminar on its ear. You can read more about it in my blog entry about that event.

If you’re looking for work that involves Ruby programming and you’re going to be in downtown Toronto on Friday, you should register to attend the Ruby Job Fair. It’ll be your chance to meet prospective Ruby employers and their representatives, which will include me – I’ll be there as the Shopify Guy. You won’t be able to miss me: I’ll be the one with the accordion…

The quick details about Ruby Job Fair:

Date: Friday, February 10, 2012

Time: 6:00 p.m. – 9:00 p.m.. Do not show up early. They’ve got work to do.

Why Work at Shopify: The Hard-Nosed, Pragmatic Business Reasons

Normally, I’d start with a description of Shopify’s hacker ethic, how it’s a great-yet-casual work environment, that everyone gets a MacBook Pro or MacBook Air as their work machine, that and how fun and rewarding it is to work there. That’s all true, but I’m sure every software development shop has a spiel along the same lines. So I’ll give you that spiel later. How ‘bout I answer the question that might be lingering somewhere in your mind: “Are you guys still going to be around a year from now, or are you going to crash and burn and leave me looking for work again?”

For starters, we’re in the ecommerce business, and business is good. How good? In the second quarter of 2011 – remember, that’s only April, May and June – ecommerce sales in the U.S. were $48 billion. And impressive as that figure may be, ecommerce is still less than 5% of all retail.

Why Work at Shopify: The “I Want to Work Someplace Cool” Reasons

Good work needs a good physical environment, and we’ve got that in spades. Check out our brand-new office. Here’s the reception desk, which is occupied by Laura, our gets-stuff-done-so-we-can-get-stuff-done person:

We believe that small, agile teams work best, so we’ve broken our space into offices just like the one below, and each team is free to set up and decorate their space as they see fit. They’re not normally this crowded; the photo below is from the party we had on Friday:

I’m in the developer advocate/evangelism group, and we went with this pop art wall covering in our zone:

Others on our team have some great illustration talents and put them to good use:

It’s the 8-bit paradise. I spent an afternoon working on API docs in the room, a nice quiet space where you can concentrate, after which you can reward yourself with classic 1980s console action!

This poster was created by our design team, a very talented bunch:

We’ve got a fine collection of vintage cartridges:

Ah, the Atari 2600. It takes me back to my wonderfully misspent youth:

Why Work at Shopify: The “I Want to Draw the Owl” Reasons

One of the reasons that Shopify is successful is that we’ve worked out some ways of doing things. We’re all about “drawing the owl”, and the way we do things is an in the service of drawing that owl. (Don’t worry, you’ll soon know what “drawing the owl means”.)

Shopifolks – that’s what I like to call people at Shopify – are self-starters. Once given a goal, they use their skills, knowledge and good judgement to do the work necessary to hit that goal. They get stuff done. They’re what Y Combinator’s Paul Graham calls “resourceful”.

Act like an owner. You don’t "just work here", you own a piece of a company and have a stake in its success. Work as if your livelihood, career and reputation were riding on it, because as an owner, it is! Be entrepreneurial and own your domain: if you have an idea and it lines up with the company’s goals, make that idea happen.

Know what to work on and what things to ship. While owners have the freedom to work on and ship whatever they like, they also work in the real world. 80% of what makes the company go is often achieved by doing the most important work first, which typically makes up 20% of the available tasks. Sometimes these tasks can be tedious and feel like drudgery, but if they’re what makes things happen for our customers and their customers, they’ve got to be done, and with the highest priority.

Done is better than perfect, or "the best" is the enemy of "the good".Perfectionism is a form of procrastination. It assumes that time is an infinite resource, that other tasks can wait while you add "just one more touch" and that "perfect" is attainable. You have to be able to make the call and say "done" at some point. A good feature that our customers use and enjoy is infinitely better than a perfect one that "will be available soon". As they say at Apple, "Real artists ship".

Have high standards. While done is better than perfect, good still remains better than bad.

It’s okay to fail; just fail gracefully. The only sure-fire way to not fail is to not do anything. Since we can’t do that and remain in business, never mind take the company to the heights we want to, we have to accept failure as part and parcel of trying. Sometimes we’ll make mistakes, other times we’ll do things right and still our best-laid plans won’t work because of circumstances outside our control. The trick is to learn from failure and make sure our failures aren’t fatal. As our CEO Tobi likes to say: "If I’m not failing every now and again, I’m not trying hard enough."

Communicate good news quickly, communicate bad news ever more so. The first part is easy: it takes no effort to tell the team your project is a success. It’s a good thing to do so; good news bolsters the team and success often breeds more success. However, a combination of pride and fear (and in some companies, a "cover your ass" culture) makes it difficult to tell the team that you’re having trouble or that something’s not working out. It’s best to tackle problems as soon as possible, while they’re still small and manageable, and the best way to do this is to communicate bad news as quickly as possible — remember, it’s okay to fail.

Understand and respect the makers’ and managers’ schedules. As Paul Graham wrote in his essay, Maker’s Schedule, Manager’s Schedule, makers and managers operate by different schedules. Managers’ days are determined by their appointment calendars, which divide the days into hours and even half-hours, and things like meetings fit into the manager’s schedule easily. Makers, on the other hand, do things in half-day or even full-days blocks, and things like meetings are disruptive. Some of the team operate on a maker’s schedule, other operate on a manager’s schedule, and many of us switch between the two, depending on what day it is and what tasks they have on that day. Know who operates on which schedule (and when), and understand and respect those schedules.

Operate lean and mean. We’re made up of multi-talented, capable, autonomous, ambitious go-getters, and that means we don’t have to operate like a big, lumbering beast. Unless the circumstances are unusual, there really should be 2 people maximum per deal or project. Meetings and calls should be kept to 30 minutes or less, not counting brainstorming or design pow-wows. And full-on meetings aren’t always necessary: you should be able to "just pop by" anyone’s office or desk or call them up on Skype.

Think You Can Work at Shopify? See Me at Ruby Job Fair.

If Shopify looks like the sort of place where you’d like to work, and if you think you’ve got the skills, enthusiasm and passion to work with us, come see at Ruby Job Fair. I’d be happy to answer all your questions and hook you up!

Pictured above is the team at Shopify to which I belong. It's the Apps Team, and while it may be small, it takes on the company's most ambitious projects: the Shopify App Store, Shopify Experts, Shopify Partners and Shopify Fund as well as the company's business development and developer advocacy efforts. It's our team's job to take the Shopify platform and see how far we can take it.

As a small team charged with a lot of responsibilities, we have to do things in a way that maximize the effect our actions have. Over the past year, we've worked out a number of ways of doing this, some gained from experience, others from experimentation. They've remained what's called "tacit knowledge" -- practiced by the team but not written down or formally codified in an operations manual -- until team leader Harley Finkelstein, our Chief Platform Officer, collected them into a set of slides.

The way we get things done boils down to the general principles listed below. Your team may not be like ours, but I'm sharing these principles because you might find at least some of them useful:

Act like an owner. You don't "just work here", you own a piece of a company and have a stake in its success. Work as if your livelihood, career and reputation were riding on it, because as an owner, it is! Be entrepreneurial and own your domain: if you have an idea and it lines up with the company's goals, make that idea happen.

Know what to work on and what things to ship. While owners have the freedom to work on and ship whatever they like, they also work in the real world. 80% of what makes the company go is often achieved by doing the most important work first, which typically makes up 20% of the available tasks. Sometimes these tasks can be tedious and feel like drudgery, but if they're what makes things happen for our customers and their customers, they've got to be done, and with the highest priority.

Done is better than perfect, or "the best" is the enemy of "the good". Perfectionism is a form of procrastination. It assumes that time is an infinite resource, that other tasks can wait while you add "just one more touch" and that "perfect" is attainable. You have to be able to make the call and say "done" at some point. A good feature that our customers use and enjoy is infinitely better than a perfect one that "will be available soon". As they say at Apple, "Real artists ship".

Have high standards. While done is better than perfect, good still remains better than bad.

It's okay to fail; just fail gracefully. The only sure-fire way to not fail is to not do anything. Since we can't do that and remain in business, never mind take the company to the heights we want to, we have to accept failure as part and parcel of trying. Sometimes we'll make mistakes, other times we'll do things right and still our best-laid plans won't work because of circumstances outside our control. The trick is to learn from failure and make sure our failures aren't fatal. As our CEO Tobi likes to say: "If I'm not failing every now and again, I'm not trying hard enough."

Communicate good news quickly, communicate bad news ever more so. The first part is easy: it takes no effort to tell the team your project is a success. It's a good thing to do so; good news bolsters the team and success often breeds more success. However, a combination of pride and fear (and in some companies, a "cover your ass" culture) makes it difficult to tell the team that you're having trouble or that something's not working out. It's best to tackle problems as soon as possible, while they're still small and manageable, and the best way to do this is to communicate bad news as quickly as possible -- remember, it's okay to fail.

Understand and respect the makers' and managers' schedules. As Paul Graham wrote in his essay, Maker's Schedule, Manager's Schedule, makers and managers operate by different schedules. Managers' days are determined by their appointment calendars, which divide the days into hours and even half-hours, and things like meetings fit into the manager's schedule easily. Makers, on the other hand, do things in half-day or even full-days blocks, and things like meetings are disruptive. Some of the team operate on a maker's schedule, other operate on a manager's schedule, and many of us switch between the two, depending on what day it is and what tasks they have on that day. Know who operates on which schedule (and when), and understand and respect those schedules.

Operate lean and mean. We're made up of multi-talented, capable, autonomous, ambitious go-getters, and that means we don't have to operate like a big, lumbering beast. Unless the circumstances are unusual, there really should be 2 people maximum per deal or project. Meetings and calls should be kept to 30 minutes or less, not counting brainstorming or design pow-wows. And full-on meetings aren't always necessary: you should be able to "just pop by" anyone's office or desk or call them up on Skype.

A little while back, we announced the Shopify Fund, a one million dollar pool of money set aside to stimulate the development of Shopify apps, applications that made use of the Shopify API to extend, enhance and automate Shopify shops. We asked developers to submit their app proposals and if their app was chosen, we’d give them somewhere in the neighborhood of five to ten thousand dollars to take a few weeks to work on their app idea full-time, complete it and put it into the Shopify App Store.

In the end, we received 143 app proposals – many of which were submitted on the deadline date, November 30th -- a considerable deal more than we’d expected. We’ve been spending the past couple of weeks deliberating over which apps should get funding in the Fund’s first round, in closed-room sessions not unlike the scene from 12 Angry Men shown above. We still have to have a few more discussions before we make our final choices, and we’ll announce which apps are getting funding in the new year.

If you didn’t get a chance to submit an app idea or if your app idea submission doesn’t get selected, don’t worry. This is just the first round, and we want to continue funding the development of apps through the coming months – both apps that you propose and apps that we have on our wishlist. The Fund will continue because:

We think it builds interest and excitement about the Shopify ecosystem. The number of responses we’ve received from the developers proposing apps seems to indicate this.

We want to make it possible for developers to have the time they need to build Shopify apps. By funding developers, we give them enough money so that they don’t have to take on any other clients and just work on an app full-time.

We want Shopify to be the ecommerce platform with the most capabilities. Shopify does a lot “out of the box”, and it does so much more when you extend it with apps. More apps means more capabilities and customizations, and we think that’s a good thing.

So keep an eye on this blog for announcements in the new year – not just about whose apps are being funded in the first round, but also for new chances for you to get funding to develop Shopify apps!

This is just a quick update to let you know that yes, we know that the Shopify developer documentation needs work. There’s a fair bit of information there, but it could stand some improvement. There’s some missing information, it could be organized better, there are parts of it that are confusing and there need to be examples in languages and frameworks other than Ruby and Rails.

This update is also here to let you know that we’re actively working on it, bit by bit, every day. As I write this, David Underwood and are are working on a wholesale reorganization of the developer sections of the wiki and clear writeups of all the API resources, including explanations of the parameters they expect and the attributes they return as well as how they relate to other resources and what effects they have on shops. We’re also working on more example code, in more languages.

If you’ve got comments, questions and suggestions about the docs or what we’re doing with them, please let us know -- feel free to leave a comment or drop me a line.