Why Yashi uses Aerospike

Yashi, a video Demand Side Platform (DSP), differentiates itself from other DSPs by integrating location focused technology into their stack. With continued annual growth since 2007, Yashi has also found itself in Inc’s 5000 fastest growing companies. Yashi’s integration with majority of the video Ad exchanges leads to processing and analyzing over 40 billions opportunities that are sent their way. Their goal is to help those advertisers reach their target audience through the publishers that make their inventory available on all the exchanges.

This is done via a process called Real Time Bidding (RTB) where Yashi bids on behalf of their clients if the opportunity is right. The key metric here is the response time — the decision to bid or not must be made within 100 milliseconds. Beyond that it is considered a lost opportunity, which has direct impact on revenue.

Hear how Yashi analyzes more than 40 billion opportunities, and delivers hundreds of millions of video ads every month – with no downtime.

Yashi’s Infrastructure

This level of performance, scale, and real-time nature of billions of transactions flowing through the system at any given moment is powered by infrastructure that comprises of over 100 dedicated bare metal servers, proprietary RTB technology and other technologies such as NoSQL databases including Aerospike.

We also obviously use the Aerospike database which has been a huge lifesaver for us in certain user management and cookie-based things that we are trying to do

– Larry Nolan, CTO

The decision to use dedicated bare metal servers (hosted by SoftLayer) vs cloud solution (AWS to be more specific) was not made until they put AWS servers through serious stress and rigorous testing with the volume of data that they’d need to process in production environment. Based on these tests they realized that the only way for them to scale on AWS and achieve the expected throughput was to spin up more servers. This did not make business sense.

Everything we were doing through AWS was shared public and we were just getting crushed on prices, you know, when you’re pushing that kind of data.

– Larry Nolan, CTO

In comparison, SoftLayer was able to provide them with a private, unmetered and dedicated internal network.

Yashi on Speed, Scalability And Price Performance

One of the most important aspects of Ad Tech, and what Yashi excels at, is to find the right target audience for the advertisers. To do this well, third party data from multiple sources must be parsed through and synced with millions of users’ data flowing through the system. Once the users are synced is when most accurate determination of ‘who is looking for what and when’ can be made and presented to the advertisers. This level of precision in real-time presents a different kind of challenge because number of users online and the amount of data generated on the Internet both are growing rapidly. All of this data must be stored and constantly processed in such a way that it is instantly accessible for decision-making within 100 milliseconds.

To tackle this speed and scale problem, Yashi put about six database technologies including MongoDB, Redis and Aerospike to test. Among them they found Aerospike to be the best choice for storing and querying hundreds of gigabytes of data.

…we have to access everything in under a hundred milliseconds, so Aerospike really let us get that data out very quickly and scale wherever we are in the world.” So if we want to set up a data center in Europe or Australia or wherever we are, we’ll just pop up a data center and have our Aerospike sync itself up and we’re good to go and we have all that data out there.

– Andy McConaghie, VP of Systems Engineering

The two key things highlighted in these excerpts are:

Aerospike’s ability to provide RAM-equivalent performance when querying data stored on SSDs — effectively making the deployment very cost-effective since adding SSDs is less expensive than RAM

Aerospike’s Cross Data Center (XDR) replication which is easy to configure and works out of the box with minimal supervision.

It is also worth noting that Yashi does use Redis for small database deployments for data that does not need to be replicated across data centers and can live entirely in memory.

Summary

Yashi believed in its ground-breaking location focused technology and foresaw not just the amount of growth but also the rate at which they’d grow. This forward thinking allowed them to be smart about creating certain pieces of technology in-house and putting tremendous amount of time and effort into Research and Development in choosing other complementary technologies. This approach of building a complete technology stack has played a very crucial role in their overall success.