Topics

Featured in Development

Understandability is the concept that a system should be presented so that an engineer can easily comprehend it. The more understandable a system is, the easier it will be for engineers to change it in a predictable and safe manner. A system is understandable if it meets the following criteria: complete, concise, clear, and organized.

Featured in Architecture & Design

Sonali Sharma and Shriya Arora describe how Netflix solved a complex join of two high-volume event streams using Flink. They also talk about managing out of order events and processing late arriving data, exploring keyed state for maintaining large state, fault tolerance of a stateful application, strategies for failure recovery, data validation batch vs streaming, and more.

Featured in Culture & Methods

Tim Cochran presents research gathered from ThoughtWorks' varied clients and projects, and shows some of the metrics their teams have identified as guides to creating the platform and the culture for high performing teams.

Microsoft Announces Public Preview of Event Grid on IoT Edge

Microsoft have announced the public preview of Event Grid on IoT Edge, which enables the adoption of event-driven architectures on systems created for disconnected or remote environments. This means that implementing event-driven scenarios using a publish/subscribe model with routing capabilities are now available on Internet of Things devices running anywhere.

Event Grid has been available on Azure since August 2017, and provides a managed service for routing events between applications. These events adhere to one of three schemas, the contents of which generally consist of an envelope used to determine where each message should be sent, and some form of application-specific payload.

EventGrid is the "out-of-the-box" schema, while CustomEvent allows an application to define its schema, and CloudEvents implements an industry standard for event-driven communication. Moreover, Event Grid performs communication using a push-push publish/subscribe mechanism as previously described on InfoQ by Steef-Jan Wiggers, Microsoft Azure MVP and Cloud Azure Technology Consultant.

The concept of Event Grid evolves around events emitted from a source (publisher), which can be an Azure service or a third party source that adheres to the event schema.The event publishers in Azure include IoT Hub, Storage, and the recently added Event Hubs. Subsequently, the events are sent to a topic in Event Grid, and each topic can be one or multiple subscribers (event handlers). A topic can be configured with the event publisher or can be a custom topic for custom events. Finally, the event handlers reactto the events and process them. The event handlers in Azure include Functions, WebHook, and Event Hubs.

Events received by the service are pushed out to one or more subscribers, using filtering to determine which subscriber is interested in which messages, while implementing mechanisms to ensure delivery, such as retries and dead-lettering.

IoT Edge is a managed service that implements IoT capabilities on devices running outside of Azure. Integration with IoT Hub enables communication between these devices and Azure. As such, IoT Edge provides options for working disconnected or with minimal data exchange, while still providing the opportunity to collect telemetry and take actions based on this on the device. It does this using Docker containers called modules, combined with a runtime that orchestrates the messages between these modules.

This newly introduced capability means that Event Grid is now available as a Docker container, which can run on IoT Edge. Consequently, it is now possible to send and receive events on these devices, and the service can interact with modules on the same device, other devices, or services in Azure. Additionally, there is complete feature parity between Event Grid on Azure and IoT Edge, meaning the work the same as well.

Currently, the Blob Storage module is the only out-of-the-box module having integrated interaction with Event Grid, allowing the service to react whenever a blob is placed in a container. However, other modules can communicate with the service as well, for which they need to use the HTTP endpoints exposed, as shown in the documentation.

To get started with these new capabilities, it is necessary first to have an IoT Edge device configured; however, this is also available as a Virtual Machine in the Azure Marketplace. Subsequently, set the desired modules in IoT Hub, which pushes the configuration to the device. The Docker images are then downloaded and installed, such as the module of Event Grid on Docker Hub, after which routing starts taking place. Several examples of this are available on the documentation, including local events, cloud exchange, and event forwarding.