CloudHop Ltd

AWS Case Study: Lenddo

Created by: Malik Amani

Modified on: Thu, 20 Sep, 2018 at 9:09 AM

About Lenddo

Lenddo enables businesses to simply and securely evaluate both the character and identity of potential customers before extending credit. Lenddo’s service helps individuals who have either no credit score or a low credit score due to lack of a credit history to access credit and services. The company also uses predictive algorithms and proprietary technology to provide verification of identity for financial and e-commerce transactions and job applications.

Founded in the Philippines in 2011, Lenddo has made tens of thousands of credit decisions around the world. “Our long-term goal is to improve lives,” says Richard Eldridge, Lenndo’s cofounder and chief operating officer. “By helping people access credit and other life-improving services, we are facilitating economic empowerment around the world.” Lenddo was named 2014 Technology Pioneer at the World Economic Forum in Davos, Switzerland.

The Challenge

Lenddo hosted its technology on Amazon Web Services (AWS) from its inception to take advantage of the flexibility and scalability of the AWS cloud. A typical Lenndo credit application has more than 12,000 data points, which can be used to predict creditworthiness. “We knew from the outset that we would need a huge amount of storage space with the ability to continually expand it, says Naveen Agnihotri, chief technology officer. “As a startup, the first thing you have to realize is the market is inherently unpredictable. The most important thing is flexibility—which is why we chose AWS. Furthermore, running on AWS gives us the ability to deploy our service in multiple regions so that we can reach isolated customers in emerging markets.”

Running on AWS helped Lenddo quickly deploy servers and environments to build and test ideas. “We were able to concentrate on developing our technology platform, building and testing out algorithms, and building the business. AWS helped us keep costs low because we only paid for what we used,” says Agnihotri.

The company grew much faster than anticipated and data storage quickly became an issue. Initially, Lenddo stored social data on MongoDB databases running on Amazon Elastic Compute Cloud (Amazon EC2) cr1.8xlarge instances. “By early 2014, social data points had taken up about 3.5 TB of space and continued to grow 10 times faster than member data,” says Agnihotri. “We kept moving to larger instance sizes, but we still couldn’t keep up with index size. As social data per applicant grew, our database and storage costs grew exponentially.” Lenddo needed to optimize its architecture on AWS to continue to grow while keeping costs low.

Why Amazon Web Services

In March 2014 Lenddo redesigned its architecture to include Amazon DynamoDB for analysis and Amazon Simple Storage Service (Amazon S3) for long-term storage of social data in numerous files types, including Apache Avro, Snappy, and chunked files. In the newschema , Lenddo temporarily caches newly received data in MongoDB databases and shifts the data to Amazon S3 at the end of each day.

Lenddo built its service-oriented architecture using a LAMP web service stack configuration, consisting of Linux, Apache HTTP servers, MySQL database server, and PHP. Developers code applications using PHP and Python programing languages. The new architecture on AWS contains numerous plugins to integrate with major online social networks. “We currently store and analyze about 113 million relationships between more than 120 million profiles,” says Eldridge.

Lenddo uses DynamoDB to store the data it needs to access routinely for queries and analytics. “AWS has been able to help us create a convenient way of organizing data by how we use it,” says Agnihotri. “Moving data from MongoDB to DynamoDB provided better scalability and ease of management without worrying about database maintenance or server upgrades.”

Lenddo currently uses about 120 Amazon EC2 instances with Auto Scaling to adjust the company’s computational and storage capacities in response to real-time usage. The company dedicates 20 Amazon EC2 instances to Amazon Elastic MapReduce (Amazon EMR) clusters running Apache Hadoop processes. “Every time we gain a new member, we use algorithms—such as Bayesian inference techniques, interior point methods, and graph variable methodology—to analyze their communications and generate a reliable credit score,” explains Agnihotri. “It can be as many as seven million groups and 15 million interests—the numbers are huge and we’re constantly refining the ways in which we analyze the information to better understand our customers. Using Amazon EMR, we can run routine calculations at great speed and complete evaluations in a matter of minutes.”

“The great thing about AWS for us is that we can spin up instances at any time,” says Agnihotri. “The scalability of the AWS cloud is second to none. While we predict that our storage and access requirements will remain stable for the next 10 iterations of growth, being on AWS means that we’ll be able to handle unanticipated usage spikes without worry. In the meantime, we’re saving significant amounts of money by using Amazon EMR clusters to run routine jobs.”

He adds, “There was a time when we considered the idea of hiring a third-party Hadoop implementation partner, but AWS services worked so well together for our purposes, we’ve never needed to look beyond them.”

“AWS provides the foundation for the resilient, strong, and flexible online infrastructure that we need while reducing our monthly IT spend by 40 percent,” says Eldridge. The new architecture on AWS helps Lenddo reduce costs while supporting continual growth without latency or storage issues. “Using AWS allows us to focus on our core business: building complex algorithms for identity verification and credit scoring,” says Eldridge. “Thanks to the computational power of AWS, we can deliver a quick, seamless, and reliable service to our customers.”