Introduction
IoT devices require a central hub (in our case: the Azure IoT Hub) to upload their data to and to retrieve new configurations or firmware updates from. For some scenarios, this system must be able to handle millions of connections and large amounts of incoming data.
Such a system must be able to authenticate clients and should offer a way to block a device if it is misbehaving or has been stolen. Due to the expected amount of IoT devices, the registration of new devices and configuration management must be easy to automate.
To manage device configurations, it is desirable to have an asynchronous bidirectional communication channel between the IoT device and the hub where settings can be stored so the device can retrieve them on the next connect.
Azure IoT Hub basics
IoT Hub is a managed service running on Microsoft Azure which offers bidirectional messaging and device management for IoT devices. It is highly scalable and integrates well with other Azure components that can be used for message processing.
Transports and authentication
IoT devices can choose between different transports for their connecting to IoT Hub
- MQTT, MQTT over WebSockets
- AMQP, AMQP over WebSockets
- HTTPS
Authentication is done using a symmetric key or X.509 certificates which can be either self-signed or signed by a CA. There is also a way to authenticate devices using a TPM. This requires the Azure Device Provisioning Service which will be covered in a follow up post.
Any device registered to an IoT Hub can be blocked from connecting if it misbehaves or is no longer wanted. The block is not permanent and can be lifted at any time.
Device to cloud messaging
IoT devices can send messages with a size of up to 256 KB. The message format can be text or binary.
The messages are made available on an EventHub compatible endpoint of the IoT Hub by default where they are kept for up to seven days until they are consumed. Messages can also be routed to custom targets like Event Hubs, Azure Storage, Service Bus Queues or Service Bus Topics.
For data with a size of more than 256 KB, there is also a file upload feature which enables the IoT device to upload a file into a blob storage.
Cloud to device messaging
IoT Hub supports two different ways of cloud to device messaging.
Normal cloud to device messages can be up to 64 KB in size. If a message can not be delivered to the IoT device within 48 hours, it is discarded.
Direct methods are used to invoke a method on an IoT device from the cloud and receive the results as soon as the method call is finished. The payload size is up to 128 KB. If a device is not currently connected, the method call fails right away.
The Device Twin
The retention periods of the cloud to device messaging features described above makes them unsuitable for scenarios where a configuration change is requested by the cloud and must be applied when the client comes back online.
The device twin is a JSON document IoT Hub keeps for every registered device. The document can be updated from the cloud and by the IoT device.
{
"deviceId": "MyDemoDevice",
"etag": "AAAAAAAAAAM=",
"deviceEtag": "MTY1MTM4MzE5",
"status": "enabled",
"statusUpdateTime": "0001-01-01T00:00:00Z",
"connectionState": "Disconnected",
"lastActivityTime": "2020-02-27T14:30:07.854667Z",
"cloudToDeviceMessageCount": 0,
"authenticationType": "sas",
"x509Thumbprint": {
"primaryThumbprint": null,
"secondaryThumbprint": null
},
"version": 4,
"tags": {
"location": "Darmstadt"
},
"properties": {
"desired": {
"sendInterval": 300,
"$metadata": {
"$lastUpdated": "2020-04-29T10:31:34.9028815Z",
"$lastUpdatedVersion": 3,
"sendInterval": {
"$lastUpdated": "2020-04-29T10:31:34.9028815Z",
"$lastUpdatedVersion": 3
}
},
"$version": 3
},
"reported": {
"$metadata": {
"$lastUpdated": "2019-12-11T08:09:25.0658234Z"
},
"$version": 1
}
},
"capabilities": {
"iotEdge": false
}
}
The cloud side can write the desired
object to set configuration parameters for the IoT device, in this example the sendInterval
property. If the device is online while the changes are being applied, it will receive an update of the device twin. If the device is offline, it will receive the changed device twin on its next connect.
Only the IoT device can write the reported
object. Its main purpose is to report on the success when applying configuration or to report the current software and hardware version.
If a property is not included in the desired
or reported
object during a Device Twin update, the old value is kept. To remove a property, it must be explicitly set to null
.
The metadata
objects contain information on when and how often a property has been updated.
The desired
and reported
objects can each be up to 32 KB in payload size and be nested up to 10 levels deep
A device twin can have tags
which are only visible on the cloud side and can be used to organize devices in groups.
Device management
Tags and properties from the device twin can be used as filter criteria for device management. For example, one could create a configuration to set the desired
property sendInterval
to 600 for all devices on the IoT hub that have the location
tag set to “Darmstadt” and the firmwareVersion
in the reported
property set to “1.2.3”.
IoT Hub tiers
Azure offers different service tiers for IoT Hub. The Basic tier of IoT Hub is much cheaper than the Standard tier but doesn‘t support device twins, device management and cloud to device messages.
For evaluation purposes, there is also a Free tier of IoT Hub which has all the features from the Standard tier but only handles 8000 messages per day.
Upgrading from Basic to Standard is possible but there is no way to upgrade from Free to Standard. Keep this in mind before building a complex setup with several devices on a Free instance.
Service and device SDKs
Microsoft offers SDKs for several programming languages on several platforms. The SDKs consist of two parts. The service SDK is used for interaction with the IoT Hub service, for example device registration or device management. The device SDK is used to create the software for the IoT device and offers functionality for establishing a connection to IoT Hub, sending and receiving messages and reading and updating the device twin.
It is also possible to interact with IoT Hub without an SDK by calling the APIs or establishing an MQTT or AMQP connection directly, but using one of the SDKs is recommended.
IoT device example in C++
The following example shows how to build a simple Qt based telemetry sender application with device twin support using the Azure IoT SDK C. The SDK can be built with CMake, the default settings are sufficient.
Running this example requires a Free or Standard tier IoT Hub with at least one device with shared key authentication.
The full example code is hosted on github.
Walkthrough
TheIotHubClient
class uses the low level client from the Azure IoT Device SDK to communicate with IoT Hub. Sending messages and updating the reported
object in the device twin is done via method calls. Changes to the connection status, the success of sending messages, the device twin update success and changes to the desired
object of the device twin are exposed as Qt signals. #include <QCoreApplication>
#include <QDateTime>
#include <QDebug>
#include <QJsonDocument>
#include <QJsonObject>
#include <QTimer>
#include <azureiot/iothub.h>
#include <azureiot/iothub_device_client_ll.h>
#include <azureiot/iothub_client_options.h>
#include <azureiot/iothub_message.h>
#include <azureiot/iothubtransportamqp.h>
class IotHubClient : public QObject
{
Q_OBJECT
public:
IotHubClient(QObject *parent = nullptr);
~IotHubClient();
bool init(const QString &connectionString);
bool sendMessage(int id, const QByteArray &data);
bool updateDeviceTwin(const QJsonObject &reported);
bool connected() const;
signals:
void connectedChanged(bool connected);
void deviceTwinUpdated(int statusCode);
void messageStatusChanged(int id, bool success);
void desiredObjectChanged(QJsonObject desired);
private:
// Callbacks for the IoT Hub client
static void connectionStatusCallback(IOTHUB_CLIENT_CONNECTION_STATUS result,
IOTHUB_CLIENT_CONNECTION_STATUS_REASON reason,
void* context);
static void sendConfirmCallback(IOTHUB_CLIENT_CONFIRMATION_RESULT result, void* context);
static void deviceTwinCallback(DEVICE_TWIN_UPDATE_STATE updateState, const unsigned char* payload,
size_t size, void* context);
static void reportedStateCallback(int statusCode, void* context);
// Call the IOT SDK's DoWork function
void doWork();
void setConnected(bool connected);
struct MessageContext {
MessageContext(IotHubClient *client, int id, IOTHUB_MESSAGE_HANDLE message) {
this->client = client;
this->id = id;
this->message = message;
}
IotHubClient *client;
int id;
IOTHUB_MESSAGE_HANDLE message;
};
QTimer mDoWorkTimer;
IOTHUB_DEVICE_CLIENT_LL_HANDLE mDeviceHandle = nullptr;
bool mConnected = false;
};
IotHubClient::IotHubClient(QObject *parent)
: QObject(parent)
{
QObject::connect(&mDoWorkTimer, &QTimer::timeout, this, &IotHubClient::doWork);
mDoWorkTimer.setInterval(200);
}
IotHubClient::~IotHubClient()
{
mDoWorkTimer.stop();
if (mDeviceHandle)
IoTHubDeviceClient_LL_Destroy(mDeviceHandle);
IoTHub_Deinit();
}
void IotHubClient::doWork()
{
if (mDeviceHandle)
IoTHubDeviceClient_LL_DoWork(mDeviceHandle);
}
bool IotHubClient::connected() const
{
return mConnected;
}
void IotHubClient::setConnected(bool connected)
{
if (mConnected != connected) {
mConnected = connected;
emit connectedChanged(mConnected);
}
}
Initialization of the SDK and client setup
The SDK must be initialized by calling IoTHub_Init()
before it can be used. IoTHub_Deinit()
deinitializes the SDK and is called in the destructor.
There are different methods to create a client, depending on the authentication method used. In our example, we use a connection string with a shared key included and have to call IoTHubDeviceClient_LL_CreateFromConnectionString
(). If nullptr
is returned, this indicates that something is wrong with the connection string.
IoTHubDeviceClient_LL_SetConnectionStatusCallback()
and IoTHubDeviceClient_LL_SetDeviceTwinCallback()
add callbacks that are not specific to a certain operation of the client.
After the client has been initialized, the function IoTHubDeviceClient_LL_DoWork()
must be called as often as possible. We use the Qt event loop for this and set up a QTimer
which triggers every 200ms.
bool IotHubClient::init(const QString &connectionString)
{
if (mDeviceHandle) {
qDebug() << "Client is already initialized";
return false;
}
auto result = IoTHub_Init();
if (result != IOTHUB_CLIENT_OK) {
qWarning() << "IoTHub_Init failed with result" << result;
return false;
}
mDeviceHandle = IoTHubDeviceClient_LL_CreateFromConnectionString(connectionString.toUtf8().constData(),
AMQP_Protocol);
if (!mDeviceHandle) {
qWarning() << "Failed to create client from connection string";
return false;
}
result = IoTHubDeviceClient_LL_SetConnectionStatusCallback(mDeviceHandle, connectionStatusCallback, this);
if (result != IOTHUB_CLIENT_OK) {
qWarning() << "Failed to set connection status callback with result" << result;
return false;
}
result = IoTHubDeviceClient_LL_SetDeviceTwinCallback(mDeviceHandle, deviceTwinCallback, this);
if (result != IOTHUB_CLIENT_OK) {
qWarning() << "Failed to set device twin callback with result" << result;
return false;
}
mDoWorkTimer.start();
return true;
}
Connection status callback
ConnectionStatusCallback
is called on changes to the client’s connection status. It is called with a status and a reason, for example IOTHUB_CLIENT_CONNECTION_UNAUTHENTICATED
due to IOTHUB_CLIENT_CONNECTION_NO_PING_RESPONSE
.
In our example, the callback sets the mConnected
variable and emits the connectedChanged()
signal.
void IotHubClient::connectionStatusCallback(IOTHUB_CLIENT_CONNECTION_STATUS result,
IOTHUB_CLIENT_CONNECTION_STATUS_REASON reason,
void *context)
{
qDebug() << "Connection status changed to" << MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONNECTION_STATUS, result) <<
"with reason" << MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONNECTION_STATUS_REASON, reason);
const auto client = static_cast<IotHubClient *>(context);
client->setConnected(result == IOTHUB_CLIENT_CONNECTION_AUTHENTICATED);
}
Send confirm callback
SendConfirmCallback
is called if sending a message succeeds or fails. For example, IOTHUB_CLIENT_CONFIRMATION_BECAUSE_DESTROY
indicates that the message could not be sent before the client was destroyed.
Our implementation emits the messageStatusChanged()
signal which informs the connected slots about success or failure.
void IotHubClient::sendConfirmCallback(IOTHUB_CLIENT_CONFIRMATION_RESULT result, void *context)
{
const auto messageContext = static_cast<IotHubClient::MessageContext *>(context);
qDebug() << "Send confirm callback for message" << messageContext->id << "with" <<
MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONFIRMATION_RESULT, result);
emit messageContext->client->messageStatusChanged(messageContext->id, result == IOTHUB_CLIENT_CONFIRMATION_OK);
IoTHubMessage_Destroy(messageContext->message);
delete messageContext;
}
Device twin callback
DeviceTwinCallback
is called if the initial full device twin is received and on every subsequent update of the desired
object.
The example callback decodes the JSON string and emits the desiredObjectChanged()
signal with the new content of the desired
object.
void IotHubClient::deviceTwinCallback(DEVICE_TWIN_UPDATE_STATE updateState, const unsigned char *payload,
size_t size, void *context)
{
qDebug() << "Received" << (updateState == DEVICE_TWIN_UPDATE_PARTIAL ? "partial" : "full") << "device twin update";
const auto client = static_cast<IotHubClient *>(context);
const auto data = QByteArray::fromRawData(reinterpret_cast<const char *>(payload), size);
QJsonParseError parseError;
const auto document = QJsonDocument::fromJson(data, &parseError);
if (parseError.error) {
qWarning() << "Failed to parse JSON document:" << parseError.errorString();
return;
}
if (!document.isObject()) {
qWarning() << "JSON document is not an object";
return;
}
const auto deviceTwinObject = document.object();
qDebug() << "Device twin content:" << deviceTwinObject;
auto desired = deviceTwinObject;
// Partial updates contain the desired object without an enclosing object
if (updateState == DEVICE_TWIN_UPDATE_COMPLETE) {
const auto jsonIterator = deviceTwinObject.constFind(QLatin1String("desired"));
if (jsonIterator == deviceTwinObject.constEnd() || !jsonIterator->isObject()) {
qWarning() << "The desired property is missing or invalid";
return;
}
desired = jsonIterator->toObject();
}
emit client->desiredObjectChanged(desired);
}
Reported state callback
ReportedStateCallback
is called if a device twin update either succeeds or fails. The statusCode
parameter is the HTTP status code of the update operation which is emitted as payload of the deviceTwinUpdated()
signal. void IotHubClient::reportedStateCallback(int statusCode, void *context)
{
const auto client = static_cast<IotHubClient *>(context);
emit client->deviceTwinUpdated(statusCode);
}
Sending a message
The sendMessage()
method implements sending a message to IoT Hub.
The id can be chosen by the caller and is used in the messageStatusChanged()
signal when reporting on success or failure of a message sending operation.
For sending a message, we first have to create a message by calling IoTHubMessage_CreateFromByteArray()
which is then put into the client’s send queue using IoTHubDeviceClient_LL_SendEventAsync()
.
The MessageContext
struct is passed to the send confirm callback and contains the necessary meta information for the messageStatusChanged()
signal.
bool IotHubClient::sendMessage(int id, const QByteArray &data)
{
const auto message = IoTHubMessage_CreateFromByteArray(
reinterpret_cast<const unsigned char *>(data.constData()), data.size());
const auto context = new MessageContext(this, id, message);
const auto result = IoTHubDeviceClient_LL_SendEventAsync(mDeviceHandle, message, sendConfirmCallback, context);
if (result != IOTHUB_CLIENT_OK) {
IoTHubMessage_Destroy(message);
delete context;
}
return result == IOTHUB_CLIENT_OK;
}
Updating the device twin
Updating the reported
object of the device twin is implemented in updateDeviceTwin()
.
IoTHubDeviceClient_LL_SendReportedState()
takes a JSON object in text form and sends it to IoT Hub. The success of the operation is reported in the reported state callback.
bool IotHubClient::updateDeviceTwin(const QJsonObject &reported)
{
const auto reportedDocument = QJsonDocument(reported).toJson();
const auto result = IoTHubDeviceClient_LL_SendReportedState(
mDeviceHandle,
reinterpret_cast<const unsigned char *>(reportedDocument.constData()),
reportedDocument.size(), reportedStateCallback, this);
return result == IOTHUB_CLIENT_OK;
}
Using IotHubClient
The main()
function creates an IotHubClient
object and connects all signals to lambdas.
Most of the lambdas just print reports, the interesting work happens in the desiredObjectChanged()
handler. It extracts the sendInterval
property from the desired
object and starts a QTimer
which triggers sending a message with a JSON body. The successful application of the new send interval is confirmed to the cloud by setting the sendInterval
property in the device twin’s reported
object to the new value.
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
IotHubClient client(&a);
const auto connectionString = QLatin1String("PUT_YOUR_CONNECTION_STRING_HERE");
auto result = client.init(connectionString);
qDebug() << "Init:" << (result ? "successful" : "failed");
if (!result)
return 1;
QObject::connect(&client, &IotHubClient::connectedChanged, [](bool connected) {
qDebug() << "Client is now" << (connected ? "connected" : "disconnected");
});
QObject::connect(&client, &IotHubClient::deviceTwinUpdated, [](int statusCode) {
qDebug() << "Device twin update returned" << statusCode;
});
QObject::connect(&client, &IotHubClient::messageStatusChanged, [](int id, bool success) {
qDebug() << "Message" << id << (success ? "successfully sent" : "failed to send");
});
QTimer sendMessageTimer;
int messageId = 0;
QObject::connect(&sendMessageTimer, &QTimer::timeout, &client, [&]() {
qDebug() << "Message send timer triggered" << QDateTime::currentDateTime().toString(Qt::ISODate);
QJsonObject body;
body[QStringLiteral("messageNumber")] = ++messageId;
const auto data = QJsonDocument(body).toJson(QJsonDocument::JsonFormat::Compact);
qDebug() << "Send data:" << data;
client.sendMessage(messageId, data);
});
QObject::connect(&client, &IotHubClient::desiredObjectChanged, [&](const QJsonObject &desired) {
const auto sendIntervalKey = QLatin1String("sendInterval");
auto jsonIterator = desired.constFind(sendIntervalKey);
if (jsonIterator != desired.constEnd() && jsonIterator->isDouble() && jsonIterator->toInt(-1) > 0) {
const auto sendInterval = jsonIterator->toInt();
qDebug() << "Got send interval" << sendInterval << "from device twin";
QJsonObject reported;
reported[sendIntervalKey] = sendInterval;
client.updateDeviceTwin(reported);
// Initialize the send timer with the new interval
sendMessageTimer.start(sendInterval * 1000);
}
});
return a.exec();
}
#include "main.moc"
Building the example
If azure-iot-sdk-c is installed to/opt/azure
, the application can be built using Qt creator or qmake with the following qmake file QT -= gui
CONFIG += c++11 console
CONFIG -= app_bundle
# Change /opt/azure/ to the location where azure-iot-sdk-c is installed
QMAKE_CXXFLAGS += -I/opt/azure/include/
QMAKE_CXXFLAGS += -I/opt/azure/include/azureiot
LIBS += -L/opt/azure/lib
SOURCES += \
main.cpp
# Flags for the static version of azure-iot-sdk-c
LIBS += \
-liothub_client \
-liothub_client_amqp_transport \
-laziotsharedutil \
-liothub_service_client \
-lparson \
-lserializer \
-luamqp \
-lumock_c \
-lcurl \
-lssl \
-lcrypto \
-luuid
Testing the example
To make the example send messages, the sendInterval
property must be added to the device twin. This can be done on the Azure portal by opening the detailed information on the device in the IoT devices
explorer and then switching to the Device twin
tab.
Just extend the desired
object in the editor with the new property
"sendInterval": 60
and you should see a device twin update on the console output of our example which will then begin to send a message every 60 seconds.
Refreshing the device twin on the portal should show a new sendInterval property in the reported
property corresponding to the value we just have set. All changes to sendInterval
while the example is running will be applied directly and the interval of message sending will change.
As an exercise for the reader, this example could be extended to react to other device twin properties. If other IoT Hub features are required, the examples in the azure-iot-sdk-c repository are a good starting point.
Conclusion
Azure IoT Hub fulfills all requirements to a central hub for IoT devices that were identified in the introduction.
The different mechanisms for device to cloud and cloud to device messages as well as the device twin can support various use cases and the availability of the Free tier makes it easy to evaluate if IoT Hub is suitable for your special use case.
As we have seen from the example, implementing an IoT Hub client using the official SDK is not too complicated, a prototype can be implemented and tested in a matter of hours