It was about 9 o'clock in the morning, early Saturday, with the sun shining and my refrigerator totally empty.

I had to buy groceries. Luckily I live in San Francisco, a city filled with startups who can’t wait to outsource the minutiae of everyday life. At the tap of a button a complete stranger will come over to my house and live life on my behalf. It’s both disturbing and beautiful.

With so many choices it can be hard to know which one to pick. So instead of take a gamble, I decided to put them to the test. Three delivery services, head-to-head, in a race to purchase and deliver an identical order of groceries on a Saturday afternoon. Game on.

The Experiment

I started by compiling my grocery list. I took my last 12 trips to Safeway and built a comprehensive database of every item I had purchased. From that I crafted a standard basket - the order that I felt was representative of a normal trip. After a few adjustments to make sure I didn’t get too much food, I ended up with this.

Breakfast: Milk, Cereal, Yogurt, Donuts, Bacon

Produce: Strawberries, Avocado

Snacks: Crackers, Soda, Guacamole Mix

Dinner: Chicken Breast, Frozen Dinner, Wine

While this list may seem simple, it includes some hidden landmines:

Milk - I left a note to check expiration dates and pick up the freshest carton. I always pull my milk from the back to ensure the longest shelf life in my fridge

Guacamole mix - I asked for McCormicks, a specialty brand that’s out of the way and hard to find.

Avocado - I left a note to purchase a ripe avocado. You’d be surprised how many people don’t know how to tell if an avocado is ripe.

Strawberries - These can be very hit or miss. If you pick the package on top you could end up with a bunch of nasty fruit. A rational shopper will check for the best.

With my cart assembled it was time to order.

Ordering Process

When ordering on Postmates or Instacart, you start by choosing the store. You then go through a loop of picking a category, scrolling through the list to find your item, and then adding it to your cart.

The merchandising in these apps is terrible. Sometimes the item you want is right at the top of the list. Other times you have to scroll through 15 screens of junk to find what you want.

Search in these apps can be painful. Typing in a brand will bring up every single item under that brand name across the store. When you get in to store brands there’s no way you’ll find what you need.

The biggest value to Instacart over Postmates is the ability to schedule ahead and plan out substitutes. Instacart let’s you take any order and pick the day it will show up where Postmates restricts you to just “Get It Now”. You can also pick and choose what alternative items you want on Instacart in the event your favorites are out of stock.

Ordering on Exec was much simpler in comparison. Getting an Exec Errand is just a simple text box to fill out. My typical grocery store process involves my wife texting me a list and me going to the store so it was easy to just copy & paste the list in. Having simple text is great for fuzzy search. For example, I asked Exec for the cheapest bacon on sale and I ended up with a much higher quality product at the same price as the others. The big issue is specificity. For certain items we struggled with how to word our request such that it was perfectly clear what we wanted.

I took my standard cart and entered it into each app without triggering delivery, double checking each to make sure they were at parity.

Cost

Each service has a very different cost structure. No one service is the best at any pricepoint.

Postmates has a dynamic pricing model based on time, demand, distance, and order size. I paid $12 for delivery on this order, though I’ve seen the price be as low as $7.50. They also charge a flat 5% fee on top of the purchase price.

Exec has a model that’s purely labor based. The more time your Exec spends working, the more you pay. Their rate is $25/hour with a 45 minute minimum. On top of that, you pay 50¢/mile for travel and a 3% fee on any purchases.

Instacart‘s pricing is the most ambiguous. They charge a flat delivery fee based on order size: $7.99 for under $35 and $3.99 otherwise. Often on the weekends they will offer free delivery for orders over $60. Where it gets tricky is that you pay the retail price for grocery items plus an unspecified fee on top. So if you’re shopping at Safeway where discounting is rampant, it’s not clear just how much more you’re paying. For my order, the Instacart cost for any food item was ~20% over what I would pay at the checkout.

My Costs

Items

Food Cost

Fees

Delivery

Total

Postmates

13

$44.34

$2.22

$12.00

$58.56

Exec

14

$50.71

$19.50

$0

$70.21

Instacart

11

$33.71

$6.44

$3.99

$44.14

Cost By Order Size

This graph helps show how the costs change with order size.

Timeline

When noon on Saturday rolled around, I started the race. I triggered each app simultaneously to delivery my groceries as soon as possible. I kept a record of every call, text, and push notification I received and compiled them together. This timeline tracks all communication I received along the way.Each event is an SMS or Push Notification unless otherwise specified

Results

As the orders came in I went through each item and cataloged them along the following criteria

Did I get the item I wanted?

When items were unavailable, did I get an acceptable replacement?

Did they follow my special instructions?

Is the item of a quality I would I have purchased myself?

So how did they do? As expected, each service had no problem fulfilling the easy items on my list. Though, for about half of my grocery list, there were problems.

Exec

Exec was the first order to arrive with a total delivery time of 65 minutes. Exec makes no promises on delivery time and if they run over, it costs you directly. Of all three services, they were the only ones able to fill my 14 item order. Overall the order quality was top notch.

✔ The strawberries, milk, and guacamole mix came in perfect

✔ The donuts I wanted were out. He messaged me through the app and I told him a suitable replacement

~ The exec thought the frozen dinner I wanted was out and asked about a replacement. I realized my description of the item was not specific enough, as he bought the completely wrong thing.

✘ The avocado was not ripe. I was never asked about a replacement.

Postmates

Postmates was the second order to arrive with a total delivery time of 75 minutes. Postmates promises a 60 minute delivery, making them 15 minutes late. My postmate was the only competitor who was a woman, a detail that weighed heavily in their favor. She did a fantastic job on the produce, blowing away everyone on her handling of my avocado request.

✔ The strawberries and milk were great

✔ Text messaged me that there were no ripe avocados available in singles, but there were in a 3 pack. I told them to pick any suitable single avocado.

~ Text messaged me that the frozen chicken was unavailable, but substituted it with an unnecessarily expensive alternative.

✘ The guacamole mix was substituted with a cheaper variety without consulting me. This is the variety right next to the avocados in the store.

✘ Postmates did not have bakery items listed and did not have functionality for custom requests. Therefore I did not get any donuts.

Instacart

Instacart was the third order to arrive with a total delivery time of 111 minutes. They promise delivery within a 3 hour window, so this delivery was well within the normal range.

✘ The milk was not the freshest available, expiring 4 days earlier than the others

✘ The strawberries were of very poor quality as judged by my wife, the Chief Strawberry Eater

✘ Instacart reported that the Guacamole mix was sold out and did not supply a substitute

✘ The avocado was not ripe

~ I got a text message that some items had been substituted. The chicken had been swapped out for a larger size (1.5lb) and higher coset.

✘ Instacart does not deliver alcohol so I did not receive wine.

— Instacart handles custom requests but I failed to place one for the donuts.

Quality

Accuracy

Item

Postmates

Exec

Instacart

Postmates

Exec

Instacart

16oz Cinnamon Toast Crunch

✔

✔

✔

✔

✔

✔

1 Quart, 1% Lucerne Milk

✔

✔

✘

✔

✔

✘

1lb Strawberries

✔

✔

✘

✔

✔

✔

1x Avocado

✔

✔

✔

✔

✘

✘

McCormick Guacamole Mix

✔

✔

—

✘

✔

✘

2 Liter Coke Zero

✔

✔

✔

✔

✔

✔

Safeway Wheat Crackers

✔

✔

✔

✔

✔

✔

Eat Right Sesame Chicken Frozen Dinner

✔

✔

✔

✔

~

✔

Chobani Non-fat Strawberry

✔

✔

✔

✔

✔

✔

Chobani Non-fat Raspberry

✔

✔

✔

✔

✔

✔

1lb Chicken Breast, Boneless Skinless

✔

✔

✔

~

✔

~

1 Bottle Cupcake Sauvignon Blanc

✔

✔

—

✔

✔

✘

3x Boston Crème Bakery Donuts

—

✔

—

✘

✔

—

1 Package Bacon

✔

✔

✔

✔

✔

✔

13/13

14/14

9/11

11/14

12/14

8/13

Postmates

Exec

Instacart

Postmates

Exec

Instacart

Summary

Postmates was hands down the best value per dollar. With their inexpensive delivery fees and great accuracy, I feel like I had a great experience relative to the amount I paid. It’s hard to say whether this was due to the startup or because the competitor was a female and thus had more attention to detail.

Exec excelled at speed and accuracy. I was stunned by how fast they put my order together and how spot on the items were. My Exec communicated promptly; I never felt like I was out of the loop. However, the one thing they messed up on they got majorly wrong, partially due to me not being specific enough.

Instacart wins at convenience and price. My wife liked the fact that Instacart “got the job done without getting in your way”. They try to frontload all of the decision making so you won’t be bothered when the job is happening. And despite their cloudy pricing model they came out cheapest for small-to-medium shopping trips.

Going forward I’ll use Exec for all my same-day trips and in the event I need complicated shopping orders. I’ll continue to use Instacart for small, simple orders that I want to plan in advance.