Hacking Education with the Elastic Stack

Today, October 5th, is World Teachers’ Day, and we wanted to celebrate it in our own geeky, Elastic way. We figured why not do something fun and interesting with a dataset related to education using Elasticsearch, Logstash, and Kibana to mark this special day and say thank you to teachers around the world. The DonorsChoose.org dataset immediately popped to mind.

DonorsChoose.org is an online charity founded in 2000 with the goal of directly connecting do-gooders that care about education with public schools and classrooms in need of resources. As it happens, the great folks at DonorsChoose.org also extend their “mission to connect” to their datasets. As a part of their Open Datasets initiative, they have provided about a decade’s worth of donations, resources, and projects data, as well as data APIs to data crunchers around the world for “hacking education” as they put it.

So, we downloaded the datasets and decided to have fun with them. In this blog, we will share some of our insights. But, be sure to check out the code from our 'examples' GitHub repo to join in on the fun and find your own insights.

The Big Picture

We decided to start exploring the big picture, and the numbers are impressive. Between 2000 and 2014, DonorsChoose.org helped connect around 1.4 million donors to 57,000 schools. This raised close to $255 million from 3.5 million donations that funded 648,000 projects.

Exponential Growth

Absolute numbers are great, but we love dynamic trends too. We wondered how things had evolved since the early days of DonorsChoose.org. If you thought the above absolute numbers were impressive, the graph below showing the trend over time is equally stellar. You can see a steady growth in total monthly donations (in $) over time. Oh, and by the way, the Y-axis in the chart below is in log scale!

Interestingly, we noticed that while the total monthly donation amount has steadily increased, the average donation value (indicated by the size of the dot above) has actually decreased – this is not surprising and can be expected as the initiative got wider adoption and more donors joined in. It is also perhaps influenced by the lack of a minimum donation policy, as DonorsChoose.org has adopted a “no amount is too small” philosophy. And that brings us to our next insight: Every dollar matters!

Every dollar matters!

A quick look at the amount raised shows that even small donations (< $10) have a large impact over time, and have raised close to $1.7 million. The subsequent donation bracket from $10 - $100 netted out a whopping $60 million.

This made us wonder how the typical donation amount varied by the donor’s location. So, we looked at the fraction of the donations in various dollar amount brackets by donor’s state. The typical distribution seems pretty consistent across most states. But, some states do stand out. For example, a greater fraction of donations from Oklahoma and Arkansas are in the > $500 bracket. By contrast, a larger percentage of donations received from Idaho and Alabama are in the sub $10 bracket.

Looking at where the donations were coming from prompted our next question: Where are the donations going to and which areas received the most aid?

Where are the donations going?

A quick geomap immediately highlights that most of the donations are going towards schools in big metropolitan areas: New York, Los Angeles, San Francisco, Philadelphia, Washington, D.C., Chicago, Indianapolis, etc. There’s a strong correlation between number of projects initiated in a city and the received aid.

But wait, there's more!

As you can tell, we could go on and on with our insights. But, we would love for you to join in on the fun. Grab the code from our examples GitHub repo and mine your own insights. We have even included a pre-built Kibana dashboard to help you get started. The dashboards provide a peek into many other interesting facets of the data, like how donations have varied over time by different facets of the projects. We explored factors such as the project primary focus areas (Literature, Math, Science, etc), requested resource type (technology, books, supplies, etc), grade-level, and poverty level. So, dig in and if you find something interesting, do Tweet us @elastic.

Finally, don’t forget to check out DonorsChoose.org. It’s a great platform that enables a noble cause – and as we showed you with data, every dollar makes a difference. And, there are many classrooms, teachers, and students that would love your help!