IIOT and IOT combined in an Airport Use Case

One of the most interesting use cases for IoT application is around airports. Airports worldwide are truly the most challenging environments for IoT and IIoT in the future. Imagine the airport of the future. What does the IIoT and IoT vision for them look like?

Airports have an enormous array of devices that are part of the infrastructure such as the sensors, valves, security systems of all types plus a mixture of facility energy management, vehicles for transportation inside and around the airport. robot devices for baggage handling and other purposes, authorization systems for personnel access. In addition it is conceived airplanes themselves would be IoT devices. Like a car you could imagine an API for accessing information about the plane for use by airport, traffic control and security personnel. In fact many of these things already exist.

IoT and IIoT is about much more than the devices themselves. It is about the network effect of having many devices connected. Uber demonstrated a dramatic improvement was possible in an old business with simple application of some IoT ideas. It is a combination of the network effect and the orchestration of connected devices that can make our world better.

In addition to the highly secure and industrial operational uses of IIoT, the airport of the future we would expect to be a hospitable place for consumers to access services and information that would make our travel experience vastly easier, smoother, less error prone. Consumers would like to know information about plane arrival, baggage processing, find services in the airport or request services and have the service be able to contact the customer individually. We would expect over time the airport and airlines would learn to interact with our devices to make this future better.

The airports are thinking of a highly IoT future in which both customers and the airport itself would have many novel devices to facilitate operations, efficiency and convenience for the customer in the future. It is exciting to think of all the novel ways this might evolve and airports want to be ready for innovation and diversity in the future.

Here are 3 uses cases I am going to go in depth about I think you will find interesting both personally and from a technology perspective. It may provide fodder for your own ideas how to make travel easier.

1) Judy is planning on taking a flight today and she is having a bit of bad luck. Her friend Linda is picking her up.

2) A plane is arriving and needs some servicing before it can take off again.

3) Jonathon is having a medical emergency while at the airport.

I have purposely chosen use cases with more consumer focus although there are lots of interesting use cases around the operational aspects of the airport that are as complicated and involve many more devices. Possibly this could be a second blog on this topic.

Use Case 1: Judy is taking a flight – things don’t go well

Judy has an IoT watch with a silver band that has an airport app for her local airport. Linda, her friend in Paris that is picking her up has a similar watch with an antique brass band. The watch could just as easily be a phone or some other always connected device. Judy is taking an international flight today from LAX to Paris. Hours before the flight Judy is informed that weather is impacting the flights into and out of Paris. She is given the option to reschedule her flight.

Backstory: These events can be major hassles and costs for people, airports and airlines. The passenger should be given specific information as it is available of the possibility of cancellations, delays of their flight and recommendations to reschedule or delay if possible their flight. In many cases airlines will waive fees in such scenarios to reschedule considering the costs of handling inconvenienced passengers. Making the inconvenience and costs of these kinds of regular incidents less would be a very desirable goal. This implies that the APIs of the airport and the airlines are in sync so that each is aware of situations and conditions that affect each other and are up to date always and show the same information. Ideally this applies to all the vendors or other organizations who are involved in the airport whether for infrastructure vendors or outward customer servicing vendors.

The airport should have APIs that are accessible by applications on watches, phones, cars or other IoT devices that queries airport status, known flight status, delays, potential delays for individual flights as well as general airport conditions. Data can be fed to these services automatically from the IoT devices at the airport as well as airlines and other providers. If a security situation requires a person to leave the airport or avoid the airport this could also be provided so that applications on Judy or Linda‘s phone or watch and warn them of such situations. Today Judy receives a warning the airport she is going is going into has potential weather situation that may affect her flight. She is on alert to see if her flight will be cancelled. The flight is not cancelled before she leaves for the airport.

As the flight time comes closer Judy is automatically asked to check in if she hasn’t done so already.

If the flight destination requires special visas, or has other restrictions she will be informed automatically before she heads to the airport. The airport has also told her that congestion at the airport is high and knowing her current location it is able to tell her the latest she can leave from where she is right now to get to the airport in sufficient time.

Judy determines that she will take the chance and go to the airport. She leaves when the watch suggests giving her time to allow for the congestion at the airport. She also approves her friend Linda to get updates on her status and location.

As she approaches the airport she is automatically informed of which parking garages are full, which are available. If she has an electric car the system may be able to tell her what charging ports are open. The system will tell her which terminal her flight is leaving from and the status of the plane arriving to LAX.

All of these things require personalization of the app so Judy needs to tell the app her preferences and specific things she would like. She needs to tell if she is traveling by car or mass transit and many other things. She may walk slowly and want the app to give her extra time to get places. She may want an audible alarm in addition to a buzz on her wrist or phone.

When she parks her car and enters the airport her watch synchronizes with the airport and establishes a secure channel to her device and application on her device.

Knowing her airline for her first segment Judy’s IoT device informs her as she enters the terminal that she can use desks 112-120 to check in for her flight.

Her IoT device communicates with the airport to tell the airport her location and this is relayed as well to the friends or others she has designated to share this information with. When she gets to the checkin desk the system automatically brings up her flight record for the airline agent because the system has been tracking her. The agent informs her that the problem in the destination city has gotten better but that her flight may still be delayed.

Judy is issued electronic tickets to her airport watch app and her luggage is identified with her watch identity through attached NFC devices at the airport. Judy is authorized to go to the upper class waiting area as she may have to spend extra time at the airport.

She can use her phone or watch’s identity fingerprint reader to go directly through the security system without having her identity checked manually. She puts her bag through security and the airport knows she has gone through security successfully.

So far Judy’s path has been highly streamlined and made more efficient both for her, the airline and the airport in a difficult situation. There is enhanced security in knowing where Judy is and through enhanced biometric data.

As Judy is in the airport her watch app tells her of the status of the airplane and knowing her location and boarding priority it can estimate when she needs to head to the gate to board.

Judy has ordered several presents from duty-free to take with her to her friend. Her watch app is informed she has duty free items to pick up. When they are ready she is told that she needs to go to duty free to pick up her products and directed how to get there.

Similarly, she is led to her gate via whatever buses, trains or paths she needs to take. When Judy gets to the gate agent her ticket is located in her watch passbook as well as her boarding priority so that she can simply walk through the gate at the right time and onto the walkway to her plane without needing a gate agent to handle her ticket. If the plane requires an ID check before boarding she could simply use her biometric reader to bypass this.

During her traversal of the airport Judy’s IoT device has determined her location and sent that information to the airport app but there are times when a more precise location is required and a NFC capability or similar functionality is required to provide short distance identification such as at various interactive devices, gates or ticket counters.

Judy is on the plane so it seems the worst part of her journey may be over. She sees her luggage made it as well. Unfortunately, this is not Judy’s lucky day and before departure a physical problem with the plane is detected. Judy is informed the airline is preparing an alternate plane to take her. She must de-board the plane and proceed to a different gate. She is informed her seat will be slightly different because of the different configuration of the new plane. Judy gets off the plane and proceeds to the new gate.

She gets in her seat on the new plane similarly to the last time without having to carry a ticket since her ticket is electronic it knows her plane and gate have been reassigned.Judy left something on the first plane in her seat. Don’t tell me you haven’t done this.

Since the items have her identity on them the system knows where she is, the new plane and seat she is now located in. The attendant simply dispatches an automated courier robot which picks up the item and proceeds to deliver it to Judy on her new plane.

Judy is very happy she didn’t end up forgetting the duty free items. Whew!

The airport knows that Judy and her luggage are together on the same plane. However, when she boards the new plane her luggage cannot fit onto the smaller plane. The baggage system transfers her luggage automatically to the next flight to that destination.

Judy is informed before takeoff that her luggage will be delayed and will be scheduled onto the next flight in 6 hours to that destination. She is automatically registered as having missing luggage and is issued a kit of consumables to help her survive without her luggage.

Linda of course is aware of all this and texts Judy she regrets her baggage will be late and will be able to lend her some stuff too. Judy’s plane finally takes off. During the flight the special request she had made a week earlier from the airline web site for a Domaine Carneros La Reve Champagne was delivered to her. This made the flight a lot more enjoyable. She put on her virtual reality goggles and watched an engrossing version of an old movie called Avatar by James Cameron she had never seen.

To Judy’s relief the pilot was able to make up some time en-route but not enough to make it on-time.

Judy is able to see en-route through the planes IoT systems that are connected to the internet that her luggage did indeed make it onto the next flight. She is able to tell the baggage system her location at her destination to send the luggage to. This time as she exits the plane Judy doesn’t forget the duty free items. 🙂

In the meantime Linda is aware the plane is delayed. Her watch app tells her that given congestion on the streets and at the airport to arrive to pick up her friend she must leave 47 minutes prior to landing. This includes the time Judy would wait for baggage (in this case 0) and the traversal through passport control and customs at this time of day. The calculation is quite precise and Linda arrives at the airport knowing that Linda is out of customs and walking to door 7 within a few minutes of the expected time.

Judy pops out into the miserable weather but the happy arms of her friends embrace knowing exactly where each other are.

They drive out of the airport and the airport disengages and sends Judy a welcome message to her new city and any information that they might deem useful to her including traffic problems or other security information.

Use Case II: Flight 909 is a long haul flight from LAX to Paris. Flight 909 is sitting at the gate waiting for the automated baggage and other systems to finish loading the plane up.

The plane itself is a giant set of IoT devices. Entertainments system, baggage, GPS, food systems and of course the operation and controls of the plane itself. All of these systems have telemetry to report to the airline, airport and various service providers. When the plane landed the airport personnel and vendors already knew what consumables had been used during the last flight and what special needs there were for the new flight so that it could be stocked appropriately. The plane knew for instance that one of the passengers wanted a special champagne to be available for this flight.

In the final checkup of the plane by inspectors at LAX they determine that the plane has a defect which will not allow it to fly to the destination immediately. This is a rare occurrence because the plane normally detects these situations in flight and informs them prior to arrival at the destination. The decision is made that a different plane can be made available.

A special complex orchestration is initiated which automates much of the work needed to transfer to the new airplane including robots needed to move the luggage or help in the movement of the airplanes. Appropriate paperwork has to be filed with the tower and flight agencies. Flight plans updated. Passengers seats reassigned, The new plane is requisitioned. As soon as this decision is final all passengers on the flight are informed of the need to disembark and go to the new gate.

Some passengers who will miss connections due to the delay are automatically informed of alternative routes and times. They are automatically reassigned to new routes after the passenger agrees to the new routing. Passengers that will have to spend the night in this location are automatically told which hotels to go to and given directions. The hotel is booked for them and the room paid for. If they have special assistance needs an autonomous vehicle is dispatched to pick them up at the gate and transport them to the place they need to go.

During the transfer to the new smaller plane in this instance some luggage does not make it. The passengers with displaced luggage are informed and the luggage is immediately routed to the new plane by the automated baggage system robots when the appropriate time is reached.

The new plane is boarded and as the plane goes through its final manifest and route approval all of the process is automated to make the rerouting seamless and fast.

As the new plane leaves the gate all the IoT devices on the Tarmac are automatically reassigned to other duties. Each device on the tarmac and other areas of the operational area of the airport is instrumented so that it can be automatically fetched and dispatched where it is needed. Each device has a health status so that any abnormal behavior or service needed is handled in advance as much as possible.

Autonomous operation is a much easier job at an airport with very limited paths and destinations.

Every door and access has biometric reading to validate personnel are allowed in that area of the airport. If any device is commandeered and moved outside of a geo-fence it becomes de-activated and immediately all data is wiped from the device as needed and the system is informed.

In flight

The plane takes off and telemetry from the plane is transmitted constantly to various authorized consumers. The consumers may be airlines and airplane manufacturers, service vendors who need to understand the operation of the plane itself, detailed location information for air traffic control, less detailed information for other parties. Consumables are recorded as they are used. Most important any safety or mechanical trouble is reported instantly along with logs. Cockpits are instrumented to provide visual confirmation of the condition of the cockpit and can be remotely operated in case the pilots become unable to fly the plane.

En-route the airplane detects an asymmetrical weighting on one wing. A camera captures the image of a ape-like man who is tearing at the wing. Such things occur regularly so the pilot authorizes electrifying the wing causing the gremlin to jump off the planes wing. Soon after that the airplane detects a small condition in one engine needs to be looked at. 3 parts are involved and this information is relayed to suppliers so that the parts can be made available if needed the instant the plane lands at the gate in Paris.

Use Case III (3) : Jonathon is having a medical emergency while at the airport.

In Paris in terminal one a passenger is feeling ill. He has a condition called COPD which is a progressive disease. He has been issued a medical monitor armband which detects conditions when Jonathon may be in danger. As Jonathon is walking to his next flight he feels a light-headedness.

His monitor detects Jonathon’s blood has a spectrum that is indicating insufficient oxygen is being consumed, an acceleration in his heart rate, increased perspiration and with little physical movement it concludes something is wrong. The monitor broadcasts to closeby IoT devices that there is a person in distress. These signals are relayed to the airports emergency medical technicians who are dispatched to the exact location of Jonathon within the terminal within 2 minutes. An autonomous medical vehicle is also routed in case it is needed.

The technicians are able to help Jonathon with some simple medications which help him breathe better and insure if he does have a heart attack that his heart will have minimal damage. Within 20 minutes Jonathon is able to proceed on his journey after the EMT’s determine he is okay to travel.

What is needed at an airport to enable this kind of service and efficiency?

There are obviously requirements for hardware to support all these functions. This blog will not concern itself so much with specific hardware sources or choices but simply specify the requirements in terms of standards and software components, capabilities to facilitate the functionality described.

Categorization of IoT devices at the airport

An airport in this scenario will have devices that require high security and fast response times and many diverse devices for numerous other functions. It could easily be the case there are 10s of thousands of IIoT devices in and around the airport that have to be managed. This number of devices requires a large amount of automation to make it practical. There has to be well defined security protocols and standards as well as policies and rules for devices so that they can be managed in this highly complex environment.

It’s important to note that because of the low cost of IoT hardware, the ubiquity of standards and other technology that whether or not the airports want all the infrastructure and security made into IoT devices they will evolve rapidly to all become IoT devices.

An airport will need IoT devices which are designed for high security and high reliability operation. Some of the protocols such as CoAp are evolving rapidly to support more secure applications. In other cases hardwired connections or high security wifi connections will be used.

Let us establish some terms for different types of environments that devices need to operate within.

INFRASTRUCTURE: Devices related to HVAC, energy management, pumps, watering systems and anything that has to do with the basic operation of the buildings and environment.

OPERATIONS: Devices related to operating the airport including transportation vehicles, robots, baggage handling, conveyers. Also, will include in this category things like airplanes themselves.

SECURITY: Devices related to locks, authentication systems, entitlement control systems, security monitoring devices, cameras, detectors of dangerous situations. This might include cameras that can recognize possibly known people who are not supposed to be traveling, the security devices at security checkpoints.

SERVICE: Devices that are designed to provide services to customers. This might include monitors that display information or dispense information to consumers, beacons for instance, devices that help consumers with baggage or travel assistance wheelchairs and carts.

CUSTOMER: Devices that customers bring into the airport that need to access services within the airport.

It is expected that devices in the INFRASTRUCTURE and OPERATIONS category are always connected devices that have large battery power or directly connected to a power source and network connection that is physical and possibly replicated in some cases for reliability and security.

We would hope the same high level of service was possible for SECURITY devices but it is expected that this might include things like NFC or RFID devices. Security devices might need to be mobile and thus hard wired power and networking may not be possible in all cases.

These classes of devices in the first 3 categories will also have to have the ability to be managed by a device manager that can establish geo-fences, wipe devices contents and de-authorize if they are tampered with or accessed outside the geo-fence they are assigned to. Devices in these categories should have health APIs to tell of failure or imminent failure, battery loss or other concern. They should support a authentication mechanism and all data on the devices at rest as well as data in motion over whatever protocol the device uses must be encrypted. The certificates used by such encryption systems should be managed automatically so they can be revoked and re-issued automatically periodically.

Due to the large number of people coming and going from an airport at any time and the sheer number of devices with telemetry the amount of data captured by the systems and the number of transactions is truly a very large number. It is anticipated that the system for an average big city airport would have to handle potentially tens of thousands of messages / second possibly many more. This is roughly a billion messages / day and is quite reasonable for todays systems to handle.

Security

A comprehensive full featured Identity and Security manager component is required which will support all the following:

Authentication

OPENID, SAML2, Kerberos

Multi-factor authentication

credential mapping across different protocols

federation via OPENID, SAML2

account locking on failed user attempts

Account recovery with email and secret questions

bio-metric authentication

User/Group Management

LDAP, Active Directory or any database including Cassandra to support large user stores

SCIM support

Entitlement

OAUTH2

RBAC Role Based Access Control

Fine Grained Policies via XACML

Entitlement Management for APIs – REST or SOAP

Geofencing

preventing login outside defined geofences

Auditing

XDAS/JMX

logging integration with BAM and CEP for KPI’s and suspicious activity eventing

BigData

A bigdata infrasturcture is required because of the data flow requirements as well as the scale of data being collected. Each device will be polled frequently and that data will be logged both for use in security applications as well as for analysis to improve efficiency, discover loopholes, discover new automations and implement improvements in functionality quickly.

Support for standards such as Cassandra,

Bigdata

Collection

Must Collect to a common bigdata store for analysis

collect information on all API usage

collect information on data from devices or services that require polling

collect data from devices or services that publish themselves

easy to add new streams of unstructured or semi-structured data

discover new sources for data dynamically and collect data

support 10000s of messages / second easily

collect metadata from some services or devices where the raw data repository is elsewhere

collect data on the system itself, such as metrics produced, actions taken, events generated

Support Apache Thrift, HTTP, MQTT, JMS, SOAP, Kafka and Web services

ability to add GPS, other information to stream

ability to also send data to other loggers as needed

real-time analytics

non-programmers should be able to add new metrics, KPI’s or continuously calculated quantities

ability to create events based on exceeding limits, falling below limits, breaking a geofence, entering a geofence or other conditions

ability to aggregate data in an event from other events or data streams

ability to process rule based analytics in real time

visualization

easy to create dashboards of visualizations of any event or data in any stream

easy to create maps of events, devices, data on any event stream

easy to use tools like Google gadgets to create visualizations

ability to aggregate data from multiple sources including bigdata, conventional databases, file systems or other sources

batch analytics

ability to integrate with hadoop and open source batch big data analytics tools such as pentaho

manage batch analytics to perform

Management

ability to manage large clusters of cassandra or other big data storage databases automatically

ability to scale on increasing demand rapidly

Governance Registry / API/App/Web Store

A governance Registry and Enterprise Store Capability is needed to insure security, configuration consistency, managing the lifecycle of APIs, promote services to vendors, partners and the public. A governance registry provides lifecycle management as well as fault tolerance configuration and some security services but the typical governance registry doesn’t provide a friendly interface for the public, vendors or even to the internal development. An Enterprise Store designed to make it easy to find services, documentation, helpful hints to promote a community of users of services is required. This is an essential part of the Platform 3 message I talk about that is key to productivity and agility.

Some of the features required in these components:

gov Registry / Enterprise Store

Registry

Need to be able to support any type of asset as a governed enttity including APIs for services, APIs for devices, different types of mobile devices, mobile phones, keys for APIs, credentials for devices, certificates for services, GPS coordinates for fixed devices, geofencing zones

UI

easy to add devices and services

easy to find status of all entities

easy to find and view devices and services

must support APIs, Devices, APIs for devices and Apps

easy to find documentation about any asset

ability to create new asset classes

lifecycle creation for each class

easy to manage lifecycle of each classcertificate management for services and devices

Internet of Things (IoT)

The Internet of Things components have to do mainly with device management. In the past device management has been focused on cell phones. New device management capabilities must include the capability to manage devices of all types. The purpose of device management is to register devices, configure devices in a uniform way, detect anomalous behaviors, handle device upgrades, replacement, failures, theft, maintenance or even tampering.

In order to support the wide variety of devices in both the IIoT and IoT sphere it is necessary to support a Connected Device Management Framework which allows abstraction of basic functions and services associated with any device. An important aspect of device management in such a complex environment as an Airport is to group devices in intelligent ways that allows management and analysis of data contextually.

IoT

Types

IP type devices connected over wired interfaces

IP type devices connected over wireless interfaces Wifi

IP type devices connected over CoAp/Zigbee

NFC/RFID

Protocols

REST and SOAP

MQTT

Device Management

Device Profile

API registration

Owner Registration

GPS and GeoFencing

Beacon Security Profile

Supported Security

Health Status monitoring

Authentication

Entitlement / Authorization Profile

Data Logging

Device Wipe

Upgrade Status and Upgrade

Documentation

Applications

functional tagging

Groups

Groups are non-heirarchical list of connected devices that depend on each other and have a set of analytics or services that operate across the group. Groups can contain other groups.

Group Profile

Group APIs

Group Geofencing

Group Health

Group part of Groups, tags

Applications

Documentation

Group Data Logging

Framework

Connected Device Management Framework

Support for OMA

Support for LWM2M

Extensible Support for Devices that don’t work under these

Extensible Support for all the device management semantics above

Integration and Orchestration

A key capability of the IoT infrastructure proposed is the intelligent working of multiple services together to produce intelligent behavior. In the past individual devices were handled individually as their own function and not much thought was put into devices autonomously working together or information from multiple devices used to produce automation.

In order to make the Airport function efficiently with tens of thousands of devices and many services involving access to multiple devices it is necessary to be able to integrate all these devices so they can work together and to be able to establish rules, processes and simple workflows and dataflows between devices and people. As a result there is a need for all 3 types of orchestration tools that we have used before: Rules Engines, Integration Patterns in Enterprise Busses and Business Process Engines. These 3 patterns provide an orthogonal and complete set of functionalities to specify behaviors that are simple integration of information, distribution of information from different devices to all the parties needed and to provide rules for complex behaviors or business process logic that may involve humans and take longer than a microsecond to fully process.

orchestration

the full suite of enterprise integration patterns supported

JMS, AMQP, HTTP(S), Files,

support for visual process management scripting orchestration

support for business rules orchestration

support for ETL from wide variety of sources

APIs and Enterprise Integration Patterns

Scalability, Performance

Since this system must support high message rates and highly variable demand it is important that the system be able to scale as needed to provide services. In addition it is highly desirable to have automatic failure detection and service replacement. A cloud based architecture is well suited to this as has been shown by numerous companies such as Google, Yahoo, Twitter.

In order to support a cloud infrastructure a DevOps/PaaS strategy should be employed to automate the management, deployment of services and move, scale or upgrade them. PaaS platforms provide many features that take months and years to custom build and then are inflexible.

For example at an airport it may be possible to anticipate load at certain times of the day or based on flight arrival and departure times, events in an area that may increase or decrease base load. To keep response to services always efficient it should be possible to allocate instances to specific devices or regions.

Overall

Fault Tolerance

The entire system and every component must support active/active and active/passive FT

Disaster Recovery

Data must be replicated to alternate site and applications should be runnable in alternate environment through replicated governance registry contents

Scale

The system should support dynamic scaling allowing for peak period data flows and stress conditions 10 times the average flow

The system should support 1000s of messages / second at average flow

The system should be able to dynamically add instances of services if required to meet demand

Load Balancing

Hybrid Support

multiple clouds in different locations

Polyglot

Ability to support different development environments, different development tools and applications

Container Support

Support for Docker, Zen and other containers

Orchestration

Support for Kubernetes

Operations

Operational Management capabilities including performance monitoring

AutoScaling

The ability to scale a process based on numerous factors not just queue lengths

Summary

The modern Airport will be one of the most challenging environments for IoT and IIoT applications. The ability to provide efficient operation and security go hand in hand in this new world. Many of the new devices and capabilities will make customers lives dramatically better especially in stressful situations. People can be better informed and have more of the basic things handled automatically.

The purpose of all these “things” is to make our life easier and better. If the complexity overwhelms the system then the purpose of IoT is lost. The “things” enable intelligent behavior from the ability to sense and act. However, if each thing has to be managed individually we will spend our lives managing the things and not enjoying. So, the key is “intelligence.” The key to implementing intelligence is interoperability and orchestration. However, to know what to do that is intelligent requires BigData. We must be able to discover the patterns and the actions that will result in saved time and effort. We must figure out how to respond to security situations intelligently and to handle typical failures and events like weather or plane outages or worse in as automated and smooth a way as possible or the complexity of the system will overwhelm the people intended to manage and produce good results.

The technology exists today in open source to do a modern Airport like I have described. Nothing I suggested or described was beyond what we have today. It is simply a matter of the desire to build a better Airport and world for ourselves.