High-level overview of Azure IoT services (with .NET demo app)
When I started investigating Azure IoT services, I was a bit puzzles by all the service names and terms. What always helps me is a simple overview model and a simple code sample. Maybe this is useful for you too, so I hope it helps 🤓.
Azure IoT Hub — management and messaging
The main service you run into when working with Azure IoT is Azure IoT Hub. This service has many aspects, but the main ones are device management and the other is messaging. For communicating with the Azure IoT Hub instance an endpoint is provided.
Device management will have a list of registered devices that can make use of the IoT Hub instance. For the service and for each device you can setup security. The simples form of security (especially in demo’s) is symmetric keys, but for production scenario’s it’s advised to work with certificates.
The messaging part is about sending data from the device to IoT Hub or send data to the device. What kind of data? This could be telemetry from sensors attached to the device. Or it can be an instruction for the device (restart for instance).
There are some other interesting components in Azure IoT Hub, but we’ll discuss that later in this article.
But how does the device start communicating with Azure IoT Hub when it starts up for the very first time? For this purpose you can use the Azure Device Provisioning Service (DPS).
Azure Device Provisioning Service (DPS)
The Azure Device Provisioning Service, or in short DPS, is a simple service in essence that has a list of registered devices. When a device starts up, it can connect to DPS with it’s unique ID. DPS will then tell the device which IoT Hub service it should connect to.
There is a global endpoint that can be used to connect to a DPS instance together with the ID Scope of that DPS service. DPS can use various ways to determine which IoT Hub to return for a specific device. This can be done for instance by location (return the closest IoT Hub in the region). Or it can be done in a load-balancing scenario to enable support for millions of devices.
To make this work some initial work needs to be done before a device is shipped for use. The device needs to ‘know’ it’s unique ID, which could be the serial number for instance. The device needs to be setup with code to connect with DPS using that unique ID. In DPS the unique ID needs to be registered with the configuration how to determine which IoT Hub to use.
DPS is not always required in an IoT setup. Rule of thumb: if the IoT Hub used by devices can be changed you might need DPS. DPS makes it easier to add IoT Hub instances and move devices between them. So, when managing large scale deployments, DPS is your friend.
Azure IoT Hub — Device twins
Once a device is connected to Azure IoT Hub, it also has a device twin. This is actually just a JSON-structure with some data. This device twin can be used for two purposes: configuration settings that can be set on the Azure side and used by the device, but also reported settings that can be set on the device side.
So, device twins provide a way to have centralized configuration settings, but also a (high level, simple) way to see what the device is doing. When data is updated in the device twin, timestamps are automatically added together with an automatic version number.
Keep in mind that device twins have a maximum size and is meant to do the primary configuration of the device and/or to store basic state. Currently it’s 8 KB for tags section, and 32 KB for desired and reported properties sections eac h. For more details, see Azure Hub Limits.
Connecting Azure IoT Hub to other services
Once devices start sending messages to Azure IoT Hub you can see them, for instance with the Azure IoT Explorer, but the data is handled. You could store all message in Azure Blob Storage, but more common is to connect a service like Azure Event Hubs to receive all messages (high volume!) and make it available for handling and processing.
Azure IoT Hub has a built-in subscription model that is identical to Event Hubs. If you don’t want to pre-process data before making it available, this internal service could be used as well. For more information see the article Compare Azure IoT Hub to Azure Event Hubs.
One of the services that is often used in combination with Azure Event Hubs is Azure Stream Analytics. This service can be used to analyze all the messages coming from Events Hub and do smart things like notifications on values exceeding limits, compute averages, apply AI and more.
The general flow of DPS, Azure IoT Hub and Device Twins
Now we seen what these basic Azure IoT services and terms mean, let’s put it together in a general flow.
Step 1. Unique ID for device and register in DPS
First step is that you device needs an unique provisioning identity to register with DPS. This ID needs to be know/accessible on the device itself and needs to be part of the image that’s put on the device.
In DPS you now register this device with it’s provisioning identity. You will also give it a IoT Hub identity here, which can be the same name but can also be something else … as long as it is unique in that IoT Hub.
The configuration for a device in DPS can also have a standard configuration in the Device Twin. A device twin can contain Required settings that can only be change on the server side, but read on the client device. A device twin can also contain Reported settings that can be changed on the client side to report back configuration. In the device twin in IoT Hub are all kinds of information added for detailed tracking like timestamps and (generated) version numbers.
Step 2. Run code on the device to connect to DPS
The device needs to run code to connect to the global DPS endpoint and use the devices unique provisioning identity to register with DPS. You also need security measures in this connection like a symmetric key or (more secure) certificates.
DPS will look up the provisioning device ID and connect to the configured IoT Hub. If the device isn’t registered yet with that IoT Hub, it will be registered there with the given IoT Hub device id. The DPS service will now return the IoT Hub hostname and the IoT Hub identity to the application on the device. This enables the device now to connect to the IoT Hub with his identity.
Step 3. Connect to IoT Hub for configuration and/or sending telemetry (and more)
Now the application on the device knows the endpoint of the IoT Hub to connect to. It can read the Device Twin configuration if needed, and it can also report back as described.
The code can now also send and receive messages to or from the IoT Hub.
Demo: DPS, Azure IoT Hub and Device Twins in action
I’ve created a demo application with .NET using the Azure IoT SDK for C# to demonstrate the elements described above. You can find it in this repo mtirionMSFT/AzureIotDemo: .NET Console App demonstrating DPS, Iot Hub & Device Twin use (github.com)
The application has a simple flow:
- Connect to DPS (with given parameters)
- Connect to IoT Hub and retrieve device twin
- Send messages to IoT Hub in a loop (max messages configured in device twin)
- Wait for input to rerun step 3 or exit
Azure IoT Edge
Azure IoT Edge takes this setup to the next level. First, when a device is registered in DPS or Azure IoT Hub you can indicate if this is an IoT Edge device. That setting in itself will not do anything extra on the device. So, don’t expect runtimes to be deployed automatically that way.
Your device must have the IoT Edge runtime installed. The runtime takes care of installing and updating workloads, maintain security standards, ensure modules are running and manage communication between edge devices and/or Azure IoT Hub. The runtime has two services: the agent and the hub.
The agent takes care of instantiating modules, ensure that they run and report status back to the Azure IoT Hub.
The hub is actually a local proxy for Azure IoT Hub with (almost) the same interface. It has optimizations for communication back to Azure IoT Hub.
The modules are units of execution implemented as Docker compatible containers. The are modules in the Azure marketplace you can use, but you can also provide your own custom module. The module can then use the mechanics of the IoT Edge runtime. One of the things provided is the Module Device Twin, the configuration and status data for that module, that works like the device twin but in this case with a focus on the module. A module can also talk to other services, determine the protocol to be used, and more.
There are tutorials that take you through the process of provisioning an IoT Edge device and deploy modules. I’ll mention two here that helped me:
- Quickstart create an Azure IoT Edge device on Linux | Microsoft Docs
- Quickstart to create an Azure IoT Edge device on Windows | Microsoft Docs
If you are more interested in getting started with custom modules, have a look at this tutorial: Tutorial — Develop C# module for Linux using Azure IoT Edge | Microsoft Docs. There are tutorials there for C, Java, Node.js and Python as well.
Conclusion
This is my overview summary of the most important services and components of the Azure IoT Services. The beauty of these services is that the combination make it easier to scale to even millions of devices. So it’s build to use in a small environment, but also in the largest enterprise environments.
In most cases the devices on the edge will connect to sensors or machines that will give back the actual information you’re interested in. So they are just a logical endpoint that can be connected to much more endpoint devices.
In my opinion, you’ll probably need an extra UI layer to manage all the devices in a more semantical way for your solution. Imagine that you have millions of devices to manage, then you might want to group them in locations, buildings, floors and such. Or have a way to get to all devices that manage a particular type of sensor. It’s up to you to come up with a useful approach for your users.
So, the basics are covered with the Azure IoT services, but the real solution is up to you 😊
