This is the fifth (and final) part in a series about how we can build a serverless workflow using Azure Durable Functions, but implement some of the activities in that workflow using containers with Azure Container Instances. Today we're finally ready to create our workflow with Durable Functions. The workflow has the following steps:

A HTTP triggered starter function starts off a new Durable Functions orchestration

The orchestrator function calls an activity function that uses the Azure .NET SDK to create the new ACI container group

It then waits for that ACI container group to finish exiting using a "sub-orchestration"

The sub-orchestrator function repeatedly calls an activity function that polls for the status of the ACI container group

The orchestrator then calls a final activity function that deletes the ACI container group

Starter Function

First of all, we need a function to start off our durable orchestration. Since this is just a proof of concept application, I'm using a HTTP triggered function that you can post an instance of my ContainerGroupDefinition class to. This means that the caller has complete freedom to ask for whatever container image they want and customize the ACI container settings. Obviously for the real-world scenarios I want to use this in, the "starter" function would have a more focused scope - e.g. it might simply let you submit the URI of a video file you want to be processed. But this starter function gives us the flexibility to try out different ideas with ACI container groups.

Here's a simplified version of my starter function. You can see that I return the durable functions "HTTP management payload" which contains the URLs needed to check on the progress of the Durable Functions orchestration, and to cancel it if necessary. This is really convenient for test purposes, but again, in a real-world application we may not expose low-level details like this to the end users of our workflow.

Orchestrator function

The starter function used StartNewAsync to call our orchestrator function, and passed it a ContainerGroupDefinition. The orchestrator function retrieves this input data (with GetInput), calls the AciCreateActivity activity function which creates our ACI container group, and then starts off a sub-orchestrator that will wait up to 30 minutes for that ACI container group to finish running. Finally, whether the container finished or not within the time limit, we call our third activity function that deletes the container group.

Wait for exit sub-orchestrator function

Let's look at the "wait for exit" sub-orchestrator function next. In an ideal world we wouldn't need this. I'd like to see Azure Container Instances automatically publishing events to Event Grid when a container instance or container group stops running. That way, we wouldn't need this sub-orchestrator, and could use WaitForExternalEvent (with a timeout) instead, with the AciMonitor function we looked at earlier passing on the Event Grid notification to the Durable Functions orchestration. But for now we are required to poll.

I've implemented the polling using an "eternal orchestration" pattern, where we use ContinueAsNew to loop back round and run the same orchestrator function again. This is because the underlying event-sourcing implementation of Durable Functions means that an orchestrator function should avoid looping many times like ours potentially will.

In this example, we start by calling an activity function that can get the status of our container group, and if that indicates that the first (and only in our example) container instance has terminated, that means the container group's work is complete and we can continue our workflow.

In a real-world application we'd also want to check the exit code of the container instance, and put in some exception handling in case we fail to retrieve the container group status for any reason.

Then we sleep for 30 seconds with a call to await ctx.CreateTimer, and so long as we've not been going longer than our maximum wait time, we'll loop back round with ContinueAsNew. Notice that the current time check needs to use DurableOrchestrationContextBase.CurrentUtcDateTime to function correctly. Orchestrator functions should never access DateTime.Now directly as it makes them non-deterministic.

Activity functions

Our three activity functions are extremely simple. They all just pass through to the methods on AciHelpers that we discussed in part 4. Here's the AciCreateActivity function. If you're wondering why we even need activity functions and couldn't call the AciHelpers directly from the orchestrator function, that's because the orchestrator function must not be used for long-running or non-deterministic tasks. So we have to do this from an activity function.

Testing the workflow

With our Durable Functions orchestration in place, we're finally ready to test this thing. For my test scenario, I'm going to upload a video to our Azure Storage File Share, and then ask for a container running FFMPEG with that file share attached to extract a thumbnail image.

First, let's upload a randomly selected video from the excellent Channel 9 website to our file share to use as the input file. Note that the PowerShell commands I'm showing here assume that we still have access to the various PowerShell variables we retrieved in part 2 of this series.

Now I'm going to create my ContainerGroupDefinition JSON to pass to our starter function. You can see that the main things I'm customizing are the container image, the command line (FFMEG), and the file share volume to mount.

This will return very quickly, as it's not waiting for the whole process to complete (or even for the ACI container group to be created). It has simply started off the Durable Functions orchestration.

Monitor orchestration progress

When you enable Durable Functions for an Azure Functions app, it exposes some additional APIs that can be used to query the status of orchestrations and cancel them. Since we returned these URLs from our starter function, we can use the statusQueryGetUri to request the current status of the Durable Functions orchestration. And we can use the terminatePostUri to cancel our workflow if we want. (Note that this won't cancel the sub-orchestrator, but you'll notice I created that with a predictable orchestration ID, so we could send a termination request through for that as well if we wanted).

And we can of course also use the Azure CLI to see if the container group has been created yet, and if so, whether it is still running or not by calling az container show. We can also use az storage file list to see if the thumbnail we generate from the container has appeared on our File Share yet.

Summary

This series has been quite a long journey, and it was more complicated than I hoped to get all this working. I certainly learned a lot in the process. But we achieved the objective: we've got an Azure Durable Functions orchestration that is able to make use of Azure Container Instances with Azure File Shares to implement tasks that could not easily be performed by the Function App itself. From my limited testing it seems very quick: the whole container can spin up and run to completion in under a minute, so costs should be very reasonable.

There are of course a lot of ways in which my sample app could be improved. I only supported customizing the specific features of ACI that I needed to modify for my demo. My orchestration relies on a sub-orchestrator to perform polling to wait for container exit when an event driven approach powered by Event Grid would be much nicer.

It might be possible to take this code and turn it into a generic helper or extension for Durable Functions to simplify the process of running a containerized "activity" in ACI. While I've been working on this series, I've actually come across several people who are attempting similar things, such as this great example from Anthony Chu) who uses PowerShell functions to create the containers (which is actually quite a bit simpler compared to using the Azure SDK)! So I'm certainly not alone in seeing the potential of combining ACI with Azure Functions.

About Mark Heath

I'm a Microsoft MVP and software developer based in Southampton, England, currently working as a Software Architect for NICE Systems. I create courses for Pluralsight and am the author of several open source libraries. I currently specialize in architecting Azure based systems and audio programming. You can find me on: