It Only Works When Your Users Have Problems

Setapp is a floating boat in the open waters. Our team is a pack of weathered sailors trying to navigate it in the right direction with variable success. Seems to always be like that with new products in search of their fit.

We recently developed a messenger bot as an experiment to see if people would get more engaged when onboarding gets more personal. We decided the bot should be amusing, authentic, and useful. That’s a lot big words.

Why we believed we needed a chatbot

Setapp is a collection of Mac apps for an innumerable amount of tasks. It’s convenient, well-designed, and financially practical at $9.99/month, about the price of an avocado toast. Forget lattes, as millennials, we measure everything in avocado toasts.

I’ve been with Setapp since we launched 1.5 years ago. People love it, that is, the ones who figure out how to use it. I’ve seen hundreds of comments from those who cannot imagine their lives without our multi-purposed toolbox. And an equal number of comments asking how to find the apps they need; or if they should pay for every app individually; or how does this thing work anyway.

To address those questions we came up with an idea: what if we made an app-recommending bot that knows all about Setapp and can find you apps based on your preferences? Sounded like the solution.

How to make a Facebook messenger bot

First, you need a character. Luckily, at MacPaw (the company behind Setapp), we have two adorable cats that live in the office. A few years ago we joined forces with our talented illustrator, Alexander Ageev, and created Hoover and Fixel stickers that were immediately adopted as a part of the MacPaw brand and are currently known as MacPaws.

Fixel (on the left) is a bit too full of himself to appeal to a general audience, so Hoover won the public vote and became the furry face of our bot. We made him excited, goofy, helpful, and a little tongue-in-cheek. Once that was settled, we got down to writing.

We tried to cover every possible concern and question the user might have and to also provide actual, practical recommendations for trial users on which apps to install first. Sometimes, we went a little off-road.

This happens when you install Noizio and turn on Kitty Purring.

Hoover is one lovable little beast. Now, let’s turn to the technical side of the project.

Choosing a bot-writing tool: pros and cons of ManyChat

The easy way to test out our idea was to run it via Facebook and use a tool for bot creation, we employed ManyChat. After about a months’ work, I had a pretty clear idea of what was good and what wasn’t about it.

What’s good:

The flow builder. Creating the bot itself was a pleasure. I’d never done flowcharts with messages before but the tool is so simple. It’s like Sketch for chats. You just drag things around and they work. Beautiful.

Flow builder in ManyChat tool

2. Facebook integration. ManyChat launches as a Facebook messenger chat window that can be activated via link and requires user confirmation. It’s pretty straightforward—all you need is to comply with Facebook regulations for marketing bots. Getting set up doesn’t take long.

3. Built-in stats. You can see conversion rates right in the app — no need to attach extra tools. This includes delivery, opened messages, and clicked links.

What could be better:

The actual chat. We’ve seen numerous untimely messages, tons of duplicated answers and greetings, delays, hidden multiple choice options, and other malfunctions. This majorly affects user behavior and their trust level. Not to mention how authenticity and amusement suffer when a bot tells a joke twice or gets stuck on “Hello!”.

The test itself: mechanics and placement

We tried our bot on four different entry points: after signup, in-app, email, and Facebook ads. We expected a lot of traffic from the in-app cue (the cat appeared at the bottom of the Setapp desktop application) but we were miles away from reality—it was such a poor conversion channel.

The after-signup page wasn’t particularly high in bot-signups either and the email channel also proved average. Only Facebook ads never failed us. Despite a few stray Windows users who seemed to be thoroughly upset that there’s no Setapp for Windows—the audience was rather happy to cat-chat.

We launched a few campaigns featuring our AI assistant as a main benefit and collected enough customers to make conclusions. Here they are…

If you think you need a bot, think again

Chatbots are the buzz these days. Everyone wants one like parallax landings a few years ago or videos on the home page a few years before that. Although the main takeaway is: not all chatbots are created equal.

In our case, the flow between the bot and the app was too complicated. It required downloading apps, installing said apps, and figuring them out—all of that mostly without internal triggers (personal incentive) that are present in other use cases like booking tickets or buying shoes. We suggested new apps to users and they rarely felt the need to install them.

Such suggestions were offering users solutions which had no apparent problem.

Sure, we had successful cases of people proceeding to install three or more applications and engaging in long, heartfelt conversations with Hoover, but those were exceptions.

Chatbots can be of great use when your users have:

A clear, preferably singular problem a bot can solve.

Ability to easily use the tools the bot suggests.

Time and incentive to engage with the bot.

With Hoover, we missed on the first item. The problems were either too diverse, or there were none at all. We decided to keep the bot and to continue our experiment on other user interaction points. Let’s see how it goes.