In this lab you will learn to use Cosmos DB and Azure Databricks to build real-time stream processing solution. You will use a pre-built application to read the Twitter data stream into Cosmos DB. You will then configure Azure Databricks to be able to read the change feed of your Cosmos DB collection and you will use Scala to process the data and visualize the data stream in a Databricks Notebook.

Exercises

In this exercise, you will connect to the environment that was setup for the lab.

In this exercise, you will setup your Azure Cosmos DB and Azure Databricks accounts. You will then use the Azure Portal to configure a database and a collection in Cosmos DB that you will stream data into. You will then create your Azure Databricks cluster and configure it to be able to connect to Cosmos DB.

In this exercise, you will setup your stream processing pipeline. You will start the TwitterServer application which will pull the Twitter feed and load it into Cosmos DB. You will then create a Databricks Notebook using Scala to access the Cosmos DB Change Feed allowing you to generate realtime metrics about the Twitter feed.