Caching

Description

Course Description

This course teaches you how to manage application and network services in the Azure ecosystem.

Course Objectives

By the end of this course, you'll have gained a firm understanding of the key components that comprise the Azure application and network services ecosystem. Ideally, you will achieve the following learning objectives:

How to use Active Directory.

How to understand networking strategies for Azure and communication services.

How to use Redis cache.

Intended Audience

This course is intended for individuals who wish to pursue the Azure 70-532 certification.

Prerequisites

You should have work experience with Azure and general cloud computing knowledge.

This Course Includes

1 hour and 10 minutes of high-definition video.

Expert-led instruction and exploration of important concepts surrounding Azure application and network services.

What You Will Learn

How to utilize Azure Active Directory.

How to implement Azure communication strategies.

How to take advantage of Redis caching.

Transcript

Hello and welcome back. In this section we'll be discussing the topic of designing and implementing a communication strategy using Azure. Here we'll cover the topic of Service Bus, we'll look at creating a Service Bus name space, queues, topics, and relays. We'll also cover notification hubs and event hubs as well as the subject of tiers and scaling.

Azure Service Bus is a hosted infrastructure service. It is a suite of multi-tenant services that provide various communication channels between applications. The key features of Service Bus include the following. Relays, these provide endpoints for synchronous messaging, bridging a network boundary, for example, communication from an external client and an on-premises service. Of the items we're looking at here, this is the only synchronous message pattern. Queues provide a messaging solution that does not require the receiver to be available. Messages can be placed in the queue and they're then picked up by the receiver when the receiver is ready to process the messages on the queue. Subscriptions and topics support the publisher and subscriber pattern or pub-sub as it's sometimes referred to, a pattern of broadcasted messages that can be picked up by more than one receiver known as the subscriber. Event hubs support high-volume messaging where the receiver or multiple receivers are able to read and re-read incoming messages asynchronously, while notification hubs support high-volume push notifications to mobile devices.

Let's go to the Azure portal now and see how we can create a name space. I'm here on the classic Azure portal. Let's start by creating a Service Bus name space. A Service Bus name space is a container for other Service Bus objects such as queues, topics, relays, et cetera. A name space allows you to group these objects into a single container and use multiple name spaces to separate resources when they have different management or scaling needs. We can create a name space by clicking on the Service Bus option on the left menu, and then selecting New Name Space. Here we'll create a new messaging type name space. We just need to provide the name, select the type, messaging, and the messaging tier, which we'll set to Standard for this demo. And we click the tick, and here we have our new name space. If we click on our name space, we are able to look at the individual resources in the name space, which we'll go into later.

Let's now have a look at managing credentials. We need to click on the Configure tab in our name space, which is at the top and the far right option, and this is where we can firstly define a shared access policy. Let's go ahead and create a typical policy for a message receiver. We'll call this policy myreceiver and we'll give it listen permissions. And then we click Save at the bottom of the screen. Having created our policy, we can now access or re-generate a primary and secondary key, the secret that a listener client must provide when communicating with the relay.

The three supported protocols include the Service Bus Messaging Protocol, HTTP, as well as the advanced messaging queuing protocol. SBMP is a common default for relay services, as well as the brokered messaging .Net client library. HTTP is a firewall-friendly cross platform option, though it is considered less efficient than the specialized protocols. Finally, the Advanced Messaging Queuing Protocol is the commonly recommended solution, as it's a modern cross platform protocol supported by a variety of clients and services.

Let's have a brief look at relay services. A typical use case for a relay is exposing an on-premise service to an external client. As per this diagram, the delay acts as a gateway for communication between the external applications and the clients and the on-premises service. The relay and on-premises application are configured for communication over the typical network boundaries, including things like firewalls. General requirements to setting up a relay service include creating the name space to host the service, defining the shared access policies to restrict access appropriately, followed by the service implementation itself. Using a .Net development environment as an example, a typical implementation may involve creating a service contract, a WCF service using relay bindings and finally creating the client that will invoke the service. It is of course possible to create a relay without WCF too.

It should be noted that relays are not created through the portal. Relays are created by registering them through the Azure Service Bus API. We won't be covering this in detail, but here is a minimal example of a WCF configuration used to register a tcpRelayBinding. Assuming we already have an existing contract and service, in this case the Cloud Academy service, we specify this service, the binding using netTcpRelayBinding as well as the Azure Service Bus address unique to our service. We also defined a sharedAccessSignature as our token provider and we've created an accessible relay once a WCF app is up and running. When using WCF and defining a relay endpoint, a variety of relay binding options are available. Without going into these in detail, the first three HTTP-based bindings in this list support two way messaging using SOAP or HTTP-based communication referred to as REST.

The next three TCP-based services also support two way messaging as well as one way messaging and event publishing to multiple receivers. As a note, TCP relays support to connection modes. The first is known as relayed, where traffic goes through the relay, and the second method is hybrid where the communication channel is created between the client and the server directly where such a connection is possible.

Service Bus queues provide what's called a brokered messaging service used to couple the sender and receiver both physically and in terms of availability, physically because messages are separated by separate dedicated service, and in terms of availability because the receiver does not need to be available and listening in order for the client to send messages to the queue. Queues provide an F-I-F-O, FIFO, or First-In, First-Out messaging, with each message being available for one receiver that removes the message from the queue. Several key properties define the behavior of a queue. This list is far from complete but highlights some of the more significant settings. Some but not all settings can be modified after the queue has been created. The queue's max size defines the message holding capacity of the queue and can be set to up to five gigabytes. The time to live defines the message expiry lifetime after which the message should be removed from the queue. The option to move expired messages to a dead letter queue can be enabled to save messages to this queue in the event that messages expire.

Another key option is the lock duration. This is the length of time that a message remains unavailable to other receivers when a receiver requests what's called a peek lock on a message. A peek lock message is a non destructive read, meaning that the receiver can read the message while making it inaccessible to other readers. It is usually followed by a message delete if the receiver is processing a message. The duration defaults to 30 seconds within a maximum of five minutes, and effectively defines the amount of time that the receiver has to process the message before it becomes available to other receivers again. Sessions allow messages to be grouped, meaning that their delivery is guaranteed to be in order. And lastly, partitioning allows messages to be split across multiple brokers, enabling greater scaling potential.

Let's go to Azure portal now and see how we can create a queue, some credentials, and see the connection strings. I'm here in the classic Azure portal, and I'm on the Dashboard page for our Service Bus. Let's create a new queue and have a look at configuring its credentials and fetching the connection strings so it can use 'em in an application. We click on our name space, and then at the top of the page we click on the Queue tab, and then from here we can click Create New Queue. Using the quick Create wizard, we simply need to provide a unique queue name, the region which is already populated for us, and the name space which we've already selected. So let's click Create and just create test-ca-queue Click Create New Queue. Once the queue is created, we can configure our shared access policies and get the connection strings. We need to go into the configure tab first of all, so let's click on our queue and then select Configure, and now we can set our shared access policy. So similar to the configuring the name space credentials, we just create a new policy here, and we'll call this one mysender, give it the send permission and then we can click Save. Now our policy is created. We can view and regenerate the secret keys. As you can see at the bottom, the buttons have become available for us to select.

Now let's go back to the Dashboard. Having created a shared access policy and associated keys, we can now view the connection string specific to our policy. Let's copy this connection string so that we can view it in a text editor and see the details. Here's the connection string. I've put it into a text editor and I've laid it out so that we can see the various components of that connection string. A noteworthy part of the connection string is that the transport type is not defined. In order to use AMQP, for example, we need to append this to the connection string like so. So we just set the transport type as AMQP and we're ready to go. We might now repeat the process if we need a receiver connection string and policy, or multiple sender policies if we want to petition our senders, for example. But for now, this will conclude the demo.

Let's now have a look at how we might send a message from a .Net application. Using the types available in the Microsoft Service Bus name space, the process is fairly straightforward. Having set up our queue name and connection containing our shared access key secret and selecting AMQP as our transport type for this example, we create a new instance of the messaging factory passing in our connection string as the only argument. Using our factory we create the queue client, passing in the queue name to specify the queue we wish to connect to. After that, it's just a matter of creating a brokered message which allows us to pass a string as an argument for the message content. Sending the message is then just a matter of calling send with our message on the queue client.

When creating a receiver client, we have two options when fetching messages from the queue. We can use the ReceiveAndDelete method, meaning that the message is removed from the queue as soon as the client retrieves it successfully regardless of whether the client processes the message successfully or not. Or we have the PeekLock method, which is the default. The PeekLock method allows a client to retrieve a message and make it unavailable to other queue readers until a configurable amount of time has passed, at which point the client must delete the message from the queue to stop it becoming visible and available again. The PeekLock method ensures that if the client fails to call delete for whatever reason, such as having failed to process the message or having crashed, the message isn't lost. The client needs to process the message and call delete to confirm the completion within the define timeout period.

Let's now have a look at a simple client queue reader. The setup is nearly identical to our message sending example. The only difference is in the connection string where we've specified a different SharedAccessKeyName, myReceiver in this instance. Having created our queue client, we need to call the receive method on the queue to read a message. In this example, we first check whether we've received a message by checking whether the message property is not null. We can't assume there was a message on the queue in the first place, but having received a message successfully, we print out a few message properties as well as the message body. This will effectively be our processing step.

This example highlights some of the key message properties. Being familiar with these, especially the delivery count, is key to the certification path. The MessageId is a user defined property that can be used by Service Bus to identify duplicate messages if duplicate message detection is enabled. The DeliveryCount is set by Azure and is incremented every time the message is delivered. This property can be used by a client code to take actions once the message has been delivered a set number of times, or can be used by Azure to identify messages that need to be sent to a dead letter queue once the delivery threshold has been reached. Lastly, we call the GetBody method and cast it to the expected type. Here, we're expecting a string, but the message body can be any serializable type. And finally, having processed the message, we call Complete to mark the message as processed and delete it from the queue. Should any errors occur, we call Abandon to abandon the message peek lock and not call delete, meaning the message will become available in the queue again with an incremented delivery count and that will complete the receiver example code session.

Unlike a queue, which provides a one-to-one message delivery mechanism, subscriptions and topics provide a one-to-many delivery service. When a message is published to a topic, it's copied to each subscription, configured as a listener for that topic, and receivers read these subscriptions to get the messages. Filters can also be defined to further restrict the messages which are delivered from a topic to each subscription. In comparison to queues, messages are published to a topic in the same way that they are published to a queue. However, messages are not read off the topics. They are read off the subscriptions. Reading a message off a subscription is identical to reading it off a queue. Given their similarity, both subscriptions and topics have similar properties to those we saw available when configuring a queue. This includes max size of topics, defining their message capacity, and message time to live for both topics and subscriptions, and dead letter queues for subscriptions, to name just but a few.

Let's go to Azure now and see how we can create topics and subscriptions. I'm here in the classic Azure portal. Let's now have a look at creating a topic, subscription, and the required shared access policies. To create a topic, we need to open our Service Bus name space. Then, we can click on the Topics tab at the top of the page, and then we can click Create New Topic. I'm gonna go ahead and click Custom Create. All we need to do is give it a topic name, and then for this demo we'll just leave the rest as normal.

We can specify the topic max size as well as the message time to live. We'll leave the defaults for duplicate detection and petitioning and click to create topic. Now that's created. Let's go into our topic. The topic dashboard should look familiar and provides monitoring and key information about a topic. Having created our topic, we can now create a corresponding subscription. So we click on the Subscription tab at the top. We'll first need to provide a unique subscription name. So let's click on this, give it a name. Click Next. We can now configure a variety of settings, most of which we covered when we looked at queue properties. We have the previously mentioned message time to live, expiry dead letter behavior, lock duration, sessions, and the maximum delivery count.

But one noteworthy option that we haven't discussed yet is batch operations, and we'll cover this shortly. You may have also noticed the second dead letter option that mentions filters, this option here. We'll also have a look at this in detail in a moment. Without going into further details here, let's leave these as defaults and then continue.

Having created a topic and subscription, the last step is to create the credentials for our sender and receiver. So we have to click on the Configure tab. And now we're gonna create mysender, which will have the send permission, and we'll also have myreceiver, which will of course have the listen option. And now we can click Save, and let's dismiss those. Having created our shared access policies and clicking Save, we now have access to our regenerate buttons so we can regenerate our primary and secondary keys.

Let's go back to the Dashboard, and we can view the connection string. Now we see we've got two connection strings here, one for the receiver and one for the sender. Having completed this now, we have a topic, subscription, and the credentials in use in our application.

Sending messages to a topic and receiving them from a subscription is very similar to the queue example we looked at a few slides ago. This example publishes a message to a topic. The code is nearly identical to the queue sending example with one key difference. Instead of calling CreateQueueClient we call CreateTopicClient to get the topic client that allows us to publish our message. This message receiving example is again nearly identical to the previous queue receiving example, only difference being that we call CreateSubscriptionClient instead of CreateQueueClient to get a subscription client. That allows you to receive a message from a subscription.

One of the feature options we saw earlier when creating a subscription was enabling batch processing. When this option is enabled, it allows our client to receive multiple messages at once. This can be desirable when processing large volumes of messages, as it reduces the number of round trips the receiver client has to make, and also reduces the load on the subscription or queue due to the reduced number of received calls that need to be handled. Instead calling Receive in the previous example, we can call ReceiveBatch, which will retrieve up to the maximum number of messages specified or fewer, depending on how many messages are pending on the queue.

Filtering messages is a key feature of the topic and subscriber model. A filter is a rule that restricts the messages arriving on the subscription. Filters can be created programmatically when creating a subscription as per this example. We first need to get a reference to our name space manager, then create an appropriate filter, in this case a SqlFilter, that allows us to create a filter from an SQL-like expression. And lastly, we create a subscription specifying the topic, subscription name, and the filter we just created.

And that concludes this section, but if you stay tuned, the next section that comes up will be talking about Event Hubs.

About the Author

Isaac has been using Microsoft Azure for several years now, working across the various aspects of the service for a variety of customers and systems. He’s a Microsoft MVP and a Microsoft Azure Insider, as well as a proponent of functional programming, in particular F#. As a software developer by trade, he’s a big fan of platform services that allow developers to focus on delivering business value.