Looker flouts conventional wisdom in making data lakes accessible

Ever since business intelligence (BI) was invented, people have been trying to fix it.

It was supposed to put visual, easy-to-understand dashboards on the desk of everyman/woman, but that promise remained unfulfilled because of the limitations of those dashboards and the complexity of classic BI platforms. It wasn’t until the emergence of Tableau that at least the promise of ubiquitous dashboards (we now call them visualizations) finally materialized.

But Looker has a different take.

It starts with data lakes, which are supposed to put all the data in one place, and self-service, which threatens to undermine that by proliferating islands of visualizations, each with their own caches or extracts of the data. The question is, has self-service come at the cost of providing a single source of the truth from the data lake? It’s the latest chapter in the age-old saga of empowerment vs. control.

Looker claims it has an answer to that – it aims to offer the best of both worlds. How it accomplishes that is the interesting part: Looker’s approach has constantly tilted at windmills.

First, it straddles the self-service vs. centralization debate. Looker is not against self-service, but it’s against proliferating separate caches of the data. It offers a visual client side that allows business users to dynamically explore data on their own without being limited to what’s cached or in memory. The modeling tier allows that to happen, keeping the views and the data populating them consistent, so everybody is literally on the same page.

Then there’s Looker’s middleware-style architecture that may seem a bit retro (especially if you survived the SOA era) as it abstracts the view of the data from the underlying physical representation. And finally, in an era of open source languages, Looker has the audacity to introduce its own proprietary LookML SQL-based modeling language.

By conventional wisdom, Looker’s done everything wrong, but in the five years since its founding, the company has raised nearly $100 million in venture funding, and over the past year has doubled the client base to roughly 750.

Looker works by relying on the LookML modeling language to, in effect, wrap SQL with far richer metadata on what tables to use, how to join them, and how to calculate derived data. Much of this metadata is autogenerated by the tool itself from crawling target databases. By modeling, not only how the data is structured, but also how it is consumed, Looker can reduce or eliminate ETL jobs.