Blog

Why we drastically changed a product that we and our customers initially loved - pioneering Content Automation for email.

At Kickdynamic we automate personalized and relevant email content. We currently do this for over 150 brands including ASOS, Asda, New Look and many more.

I’d like to talk about a product we predicted was going to be amazing, but initially wasn't, and how we quickly developed it into a market leader - and I'll explain the reasons why.

Sometime in early 2013, Gabri, Ricky and I had an idea for an incredible new product feature. If you've had feature ideas that you’re sure will be a hit, you know the excitement that comes with it!

The product feature was the ability to ‘scrape’ content directly from a site and add that content into an email, at the time the email is opened. OK, the concept of web scraping (also known as web cropping) is not new at all but, the idea to use it in email and in real-time, was.

Why were we so excited about it?

Site is normally the system of record for products - site = latest content

No need for tech team to get involved - which is very appealing to email marketers

Everything pointed to it being an easy setup

Content is live

Looks great in sales demos

Bosh, winning!

I was especially very excited about this, it aligned perfectly with our vision and would be a "perfect fit" for our customers. So we did some customer validation and built what we called ‘Web to Email’ (WTE).

But as we began rolling it out, we found some serious issues:

Sites are very heavy and have very different code. Some use code from the 90s, others very new - so overall compatibility became an issue.

The way that scraping hits a site can look like a unknown crawler so it gets initially blocked. This means the IT team has to get involved to stop the block.

IT teams don’t want site to be hit heavily so they may block it. There is a caching layer that helps this so not every email open is a site ‘hit' - but IT can still be nervous and block it.

If something in the code changes such as a CSS naming convention, the web scrape can stop working - so quick interventions are needed to ensure there are no missing images in email.

Sites are slower than what is needed for email - images need to be instant. If the site is slow then images can be slow to load especially on first open so they may be missing or broken.

Finally, and most importantly...

Web scraping/cropping like this does not allow for deep personalization.

Why?

It takes the first 5/10/15 etc. products from the page only and does not include all information about a product required for personalization; such as Category, Brand, Size, Stock level and more.

It also doesn’t allow you to swap products if out of stock - many retailers keep items on the site but out of stock - while they wait for new stock.

We found that to make the best use of web scraping, the customer had to set up a micro-site with the content that needed to be scraped - which required customisation and ongoing management.

All of these issues added up significantly and for us, it became a tricky, time-consuming feature to manage and setup, all of which defeats the point of building something that makes email marketers' jobs easier.

This just wasn’t good enough for us - we believe our customers deserve better...

Therefore, we took WTE and spent a lot of time researching technical alternatives - and we found there was a much better way...

Working with our customers on this, we discovered that feeds are the perfect solution. Although initially sourcing a feed can sometimes be a challenge, using a feed or API with our Content Automation tool was by far a better long-term solution to automate and personalize content.

So, we took our web scraping technology and developed into something that is much more powerful.

We pioneered Content Automation for email - a game changer that makes traditional web scraping, or cropping, obsolete.

How does this solution work?

Our customers use their .csv or xml feed or JSON API that contains all their products and content. They plug this feed into the Kickdynamic platform, build an image template, set personalization rules and simply add a tag into the email.

This method is far better and more effective because it's:

Easier to setup and manage ongoing - once the feed is ready, it only needs to be setup once

More reliable - feed doesn’t change or update so no incorrect images

More suitable for email opens - no slowness and therefore no missing or broken images

Most importantly:

The feed allows for greater personalization because it has more products and more information about those products such as the Category, Brand, Size, Stock level, Price, and much more. Using a frequently-updated product feed, our customers are personalizing emails on a huge scale, repeatedly, in every email they send.

This is a fully automated way to achieve deep personalization. It is setup once, with no customisation or ongoing management required.