Running online grocery delivery on a national scale is complex, which is why REWE digital created a Google Cloud Platform architecture making development teams autonomous, ready to meet the challenge.

Google Cloud Results

Scales from two development teams to more than 48 teams with managed services

Synchronizing 48 independent teams on over 270 services

The word "Lebensmittelinformationsverordnung", or food information regulation, is more than just a mouthful for German food retailers. Abbreviated as LMIV, the EU regulation requires that labeling for food precisely matches what people find on their plates, so that everyone knows where food is from and what it contains. That's great for improving transparency and protecting consumers, but it's one of many regulations that make supply chains truly complicated, as Ansgar Brauner, Software Architect at REWE digital, explains:

"It really makes a difference if a tomato comes from the Netherlands or from Spain. That's something both customers and retailers need to know exactly. Even the same kind of milk from the same brand will have different article numbers if it is produced at separate farms. LMIV greatly increased the variety of products that a retailer has to track, handle, and document in detail."

In terms of challenges, LMIV is the tip of the iceberg for REWE Lieferservice, Germany's first and biggest large-scale grocery delivery service. Serving 75 German cities from eight specialized food fulfillment centers, REWE Lieferservice reaches most of the residents of Europe's most populous country. At that scale, grocery delivery is exceptionally complex. Everything from stock picking to vehicle availability must be synchronized, optimized, tracked, and planned. And because groceries perish, they must be kept at one of up to six temperature zones to ensure high-quality standards and to meet legal requirements, all the way from the warehouse shelf to a customer's apartment door.

"Each of our three main platforms for REWE Lieferservice – fulfillment, ecommerce, and big data – contains many different products, such as search or recommendations. We decided that for rapid and stable development, every product should be as independent as possible, with minimal dependence on other services."

—Ansgar Brauner, Software Architect, REWE digital

These complex logistics are made possible by the specialized fulfillment platform created by REWE digital, REWE Group's digital innovation team. "We needed an intermediate platform to coordinate our food fulfillment centers and the more than 150 REWE supermarkets where we offer click and collect services," says Ansgar. "Consolidating data from all of those locations is what enables planning routes for vehicles and the management of stock."

Looking to create the best delivery service possible for its customers, REWE digital designed a solution that could develop at speed using a microservices architecture on Google Cloud Platform (GCP).

"Each of our three main platforms for REWE Lieferservice – fulfillment, ecommerce, and big data – contains many different products, such as search or recommendations," says Ansgar. "We decided that for rapid and stable development, every product should be as independent as possible, with minimal dependence on other services."

Faster development with domain-driven design

In order to compete, major businesses need solutions that not only scale, but also evolve fast and continually with minimal disruption. For the team at REWE digital, it was clear that their fulfillment platform needed to address a wide range of related but distinct technical challenges.

"We had a close look at all the necessary logistics processes," says Ansgar. "We found all sorts of information coming into our food fulfillment center, including article data, orders, and advanced shipment notices from suppliers, as well as fresh food that must arrive with consumers well before its expiry date. Coming out, there was detailed stock information, value calculations, records of breakages, or products that have spoiled, and a process to substitute articles that are no longer available."

The team looked to divide the giant problem of large-scale grocery delivery into manageable pieces and adopted a domain-driven design approach. "Domain-driven design techniques established how these things connect to each other and where they can be separated," says Ansgar. "Through that, we found the points where we could establish clear boundaries between domains, ready to assign autonomous teams to individual problems, such as stock picking, quality control, and vehicle allocation."

"The low latency produced by Google Cloud Platform was really competitive and effective. We didn't see any disadvantages. Our microservices are located in different places, and we see no difference in latency if it runs in only one location or is split between on-premises and GCP. That's a real advantage."

—Markus Rebbert, IT Architect, REWE digital

REWE Group already had a proven logistics platform, but its classic, monolithic design was more suited to traditional logistics. For the fast-moving, data-driven world of ecommerce, a different approach was required. "For our fast-growing business, we have to be quick and innovate constantly. Small, independent teams building small, independent services are much faster than when everyone works on a monolithic platform," says Markus Rebbert, IT Architect at REWE digital. "Separating the source code means developers have less code to handle and less need to coordinate with other teams when they bring changes live. Our main strategic goal was to empower that dispersed organizational structure within a single, scalable, resilient platform. We started with 2 development teams, but we wanted to be ready to scale to more than 48 development teams, each as independent as possible, creating new products and services."

In 2016, the team began benchmarking hybrid and cloud solutions, looking at compliance and security, as well as the latencies of the solutions with three on-premises data centers used by REWE Group.

"The low latency produced by Google Cloud Platform was really competitive and effective," says Markus. "We didn't see any disadvantages. Our microservices are located in different places, and we see no difference in latency if it runs in only one location or is split between on-premises and GCP. That's a real advantage."

Making teams more autonomous

To empower small teams of developers, REWE digital uses GCP managed services wherever possible, such as Google Kubernetes Engine (GKE), Cloud SQL, BigQuery, Cloud Bigtable, and Cloud Dataproc. "Even services that are not yet offered as managed services can be run in GKE," says Markus. "And from there we can offer it to all the development teams."

REWE digital has more than 14 development teams dealing with specialized fulfillment topics, assisted by only four people tasked with operational issues such as building VPNs to on-premises locations, establishing software-defined networks, or build up and maintenance of Google Projects, GKE Clusters, and basic infrastructure like Apache Kafka.

"Where others would waste a lot of time coordinating things horizontally," says Markus. "We aim to verticalize responsibilities inside one team, so that, within boundaries, they can decide themselves which technologies to use and how to implement them."

"With GCP, software development teams can really act autonomously," says Ansgar. "They build services in their domain without relying on an operations department or other teams around them. They don't need to do things like request a new server, then wait six weeks for it. That autonomy fits well to the existing REWE digital philosophy and this combination is what makes the whole architecture so effective."

"With Google Cloud Platform, we can give development teams a playground to experiment in. We want to push decisions about implementation and features inside single development teams and GCP supports us well in enabling this autonomy, providing services that teams can use out of the box to build their product."

—Markus Rebbert, IT Architect, REWE digital

Europe's most advanced food delivery warehouse

In two years, the REWE Lieferservice project has expanded from 2 teams to 48 teams running more than 270 services. "Since implementing GCP, we've seen the number of deployments by every team increase," says Markus. "Things are accelerating, and we are working on KPIs to assess that progress."

The latest developments show rapid progress on multiple fronts. In September 2018, REWE Lieferservice opened a new, partly automated warehouse, now the most advanced fulfillment center in Europe for food delivery. Pick-up services, where customers order online then collect their items in-store, are being rolled out fast to cover most of the 3,500 REWE supermarkets in Germany. To keep products at the correct temperatures from warehouse to customer and optimize the use of refrigeration, REWE digital is investigating several innovative technologies partly using Cloud IoT core. Meanwhile, a migration of the ecommerce platform from on-premises to GCP is well underway.

"We don't detect significant differences in performance between running the platform on-premises or on GCP," says Ansgar. "That's great, as it means we can run services on both locations while the transition happens. The high-performing latencies connecting to Google Cloud help us a lot. Because we don't notice a difference in network speed, we neither have to move everything to the cloud at once, nor build it cloud-native. We can build it cloud-native at the pace the teams decide, with minimal disruption."

"With Google Cloud Platform, we can give development teams a playground to experiment in," says Markus. "We want to push decisions about implementation and features inside single development teams and GCP supports us well in enabling this autonomy, providing services that teams can use out of the box to build their product."

About REWE digital

Founded in 2014, REWE digital works to digitalize REWE Group, one of the leading retail cooperatives in Germany with revenues of over €57 billion.