Humans are awesome at certain things, like closing enterprise sales deals, delivering rousing speeches, and comforting small children. For these tasks and lots of others, humans are far superior to code.

The fact that Google & Facebook solve their primary problems with code rather than people doesn’t just impact their metrics and margins, it shapes the elegance and efficiency of the solutions they provide. The subtle advantage of solving problems with code rather than humans is that it tends to make your core product better over time.

Crayon is small, but we dream big. We solve problems with code, not humans, because we want to serve millions of users as elegantly as possible. It’s just down-right hard to do that with too many humans in the mix, especially if you want to scale quickly. (And especially if you’re not Uber.)

The rest of this post is about a big problem we had and how we solved it with code.

The Problem: Some Web Pages Don’t Look So Hot

Crayon is a visual inspiration platform. We help marketers get great ideas so they can do better marketing.

We’re programmatically adding about half a million pages to our system every day. As I write this article, we’ve got about 13 million designs in the system, and we’ll have close to 100 million by the end of the year.

But we still had a problem in deeper categories: folks would be happily browsing, only to have the smooth inspirational vibe unceremoniously broken by a crappy looking page. How could we get the “best” designs to the top of the result set and the “least inspiring” designs to the bottom?

We Challenged Ourselves to Solve it With Code

Can code separate “inspiring” marketing designs from “crappy” ones?

Many said it couldn’t be done… but if they were right, would I be writing this article?

Here’s How We Solved It

Step 1: Pick Training Sets

We picked a set of 200 “inspiring” marketing designs and a set of 200 “uninteresting” marketing designs.

Yes, this is a bit of a fuzzy thing to do because it’s based on human judgment. But after running the services team at HubSpot for 5.5 years, and working closely with more than 8,000 professional marketers, I felt qualified to judge which marketing designs were likely to be interesting and instructive to other marketers. So sue me!

Step 2: Make Some Guesses About Why The Good Ones Are Good

This part is like picking a basketball team without being able to watch the candidates play basketball. What characteristics of the players would you look at? For instance, you might pick taller players or players wearing high-tops.

If your Html is all based on tables… you might not have a great marketing design (tweet)

If you’ve got lots of inline styles… you might not have a great marketing design (tweet)

If you don’t use media queries in your CSS… you might not have a great marketing design (tweet)

You get the idea.

Step 3. Test Your Guesses

We wrote some code to test each “guess” from step 2 against each design in the training sets from step 1. We were hoping to find things that were “true” for the “inspiring” pages, and “untrue” for the “uninteresting pages”.

We looked at 45 discrete “guesses”, and found 25 of them were predictive of “inspiring” marketing designs. Success!

Some of the best factors were things like:

Setting the viewport meta tag

Including Facebook Open Graph tags

Using the Bootstrap framework

Specifying Apple touch icons

Note that these factors don’t directly predict which pages are “inspiring”. Rather, these factors indicate that someone clueful created the page, which is directly predictive of a page being “inspiring”.

Some of the things we thought might work, but weren’t predictive at all include:

Whether a page uses JQuery doesn’t matter… it’s just too ubiquitous

The total text on the page doesn’t matter. It turns out there are equally as many crappy short pages as there are long pages (tweet)

The final step was some mathematical mojo based on how strong the signals were in step 2 to come up with an overall “inspiringness score” for each page.

The Results

Pages like the chatterbox.co homepage did very well:

Machine coding

While our friends at biomarkerstrategies.com didn’t fare as well:

Overall, here’s what we got:

How awesome is that?

The top 10% of Crayon search results just went from 50% “inspiring” to a whopping 93%!

We have 13 million designs in the system today, and reviewing those manually would take a 10 person team about 5 years. Ouch! But we’ll have 100 million designs by the end of the year, and that’s just the beginning. If we didn’t solve this problem with code, we’d be in serious trouble. And no humans were involved… except for the one human writing this article.

Your thoughts?

Does your business have a tough problem you plan to solve with code rather than humans?

Have you used machine learning to deliver elegant solutions to customers at scale?