Generic Structs in OPC UA
The OPC UA data type system is hierarchically organized and enables its users to model custom structured data types and enumerations to represent data in the optimal way.
OPC UA clients and servers mostly use the Binary Data Encoding defined in Part 6 of the OPC UA specification for data exchange. As its main design goal was a small size of the encoded data, it does not contain any information about data types and field names which means that a decoder must know the type and structure of the data it is going to decode.
The basic building blocks of the OPC UA type system are the so-called built-in types where explicit encoding rules are available in Part 6. They can be used to form structured data types following a set of rules:
- Structure fields are serialized in the order they appear in the structure
- Arrays are serialized as an Int32 containing the array length followed by all array elements
- Multi-dimensional arrays are prefixed by the array dimensions array serialized as previously described
- Structs derived from Union have their switch field serialized first, then the only member contained in the union (if any) is serialized
- Structs with optional fields start with a UInt32 which contains a bit mask. Each optional field is assigned one bit in the bit mask with the first optional field corresponding to the first bit and so on. An optional field is only serialized if the corresponding bit it the bit mask is set.
Due to the definitions above, the decoding must be done in a multi-step process. For example, the switch field or bit mask must be decoded to know which fields are really present in the serialized data and for array fields, the array length must be decoded to determine the number of array members to deserialize. So far the decoding logic had to be programmed manually based on external knowledge of the types involved (for example the XML based data type description).
Type Descriptions
Since OPC UA 1.04, the OPC UA node class DataType has the optional attribute DataTypeDescription which contains a StructureDefinition value for types derived from Structure or Union and an EnumDefinition value for types derived from Enumeration.
The StructureDefinition type contains all necessary information to decode a structured type:
- The default encoding node id which is also present in an extension object containing a serialized value of that type
- The node id of the direct super type
- An enumerated value describing if the structure is a Union or if it has optional fields
- A list of structure fields with name, description, data type, value rank, array dimensions and a flag indicating if the field is optional. While all enums are encoded as an Int32 and no additional information is required to deserialize them, the EnumDefinition type contains a list of all possible enum values together with their symbolic names.
A client that knows the set of explicit encoding rules mentioned above to decode all built-in types and how to interpret the DataTypeDescription attribute will be able to decode and encode all custom data types exposed by a server, even if it has never seen them before.
New Classes in Qt OPC UA
The dev branch of the Qt OPC UA module has been extended with six new classes to support generic type decoding and encoding:
-
- QOpcUaStructureDefinition and QOpcUaStructureField are returned for reading the DataTypeDescription attribute of a Structure DataType node
- QOpcUaEnumDefinition and QOpcUaEnumField are returned for reading the DataTypeDescription attribute of an Enumeration DataType node
- QOpcUaGenericStructHandler is responsible for traversing the data type hierarchy of a server and encoding and decoding of generic structs
- QOpcUaGenericStructValue is the data class which stores a decoded value generated by QOpcUaGenericStructHandler and can also be used to pass data to QOpcUaGenericStructHandler’s encoding method. It contains type information and a map of structure field names to values. For structures with optional fields and unions, only the fields contained in the serialized value are present in the map. For nested custom structs, the value is again of type QOpcUaGenericStructValue.
How to use QOpcUaGenericStructHandler
QOpcUaGenericStructHandler requires a connected client for initialization. It starts with the well-known node id of BaseDataType and traverses the entire data type hierarchy. The gathered information is then organized in internal data structures that represent the hierarchical relationships of all available data types and the necessary information to decode them.
In addition to the type descriptions gathered from the server, it is also possible to add custom type descriptions to enable encoding and decoding for servers that don’t expose the DataTypeDescription attribute.
The following example showcases all features of the generic struct handler.
Our project requires Qt Core and Qt OPC UA. The CMakeLists.txt finds them for us automatically and builds the demo executable:
project(GenericStructBlog LANGUAGES CXX)
set(CMAKE_AUTOMOC ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(Qt6 REQUIRED COMPONENTS Core OpcUa)
add_executable(GenericStructBlog
main.cpp
)
target_link_libraries(GenericStructBlog Qt6::Core Qt6::OpcUa) First, we create a test class which establishes a connection to the Qt OPC UA open62541 test server, initializes the generic struct handler and reads the value attributes of several test nodes with custom struct values. The structured types of these nodes were modelled to show interesting features like nested structs, optional fields and unions.
#include <QCoreApplication>
#include <QOpcUaClient>
#include <QOpcUaGenericStructHandler>
#include <QOpcUaProvider>
class GenericStructHandlerExample : public QObject {
Q_OBJECT
public:
bool start();
signals:
void done();
private:
void handleEndpointsRequestFinished(const QList<QOpcUaEndpointDescription> &endpoints,
QOpcUa::UaStatusCode statusCode, const QUrl &requestUrl);
void handleConnected();
void handleInitializeFinished(bool success);
void decoderDemo(const QList<QOpcUaReadResult> &results, QOpcUa::UaStatusCode serviceResult);
void encoderDemo();
void customTypeDescriptionDemo();
QScopedPointer<QOpcUaClient> m_client;
QScopedPointer<QOpcUaGenericStructHandler> m_handler;
};
#include <main.moc>
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
GenericStructHandlerExample handler;
QObject::connect(&handler, &GenericStructHandlerExample::done, &a, &QCoreApplication::quit);
handler.start();
return a.exec();
}
bool GenericStructHandlerExample::start()
{
m_client.reset(QOpcUaProvider().createClient("open62541"));
if (!m_client) {
emit done();
return false;
}
QObject::connect(m_client.get(), &QOpcUaClient::endpointsRequestFinished, this,
&GenericStructHandlerExample::handleEndpointsRequestFinished);
return m_client->requestEndpoints(QUrl(QStringLiteral("opc.tcp://127.0.0.1:43344")));
}
void GenericStructHandlerExample::handleEndpointsRequestFinished(const QList<QOpcUaEndpointDescription> &endpoints,
QOpcUa::UaStatusCode statusCode, const QUrl &requestUrl)
{
if (statusCode != QOpcUa::UaStatusCode::Good || endpoints.isEmpty()) {
emit done();
return;
}
QObject::connect(m_client.get(), &QOpcUaClient::connected,
this, &GenericStructHandlerExample::handleConnected);
m_client->connectToEndpoint(endpoints.first());
}
void GenericStructHandlerExample::handleConnected()
{
m_handler.reset(new QOpcUaGenericStructHandler(m_client.get()));
QObject::connect(m_handler.get(), &QOpcUaGenericStructHandler::initializeFinished,
this, &GenericStructHandlerExample::handleInitializeFinished);
m_handler->initialize();
}
void GenericStructHandlerExample::handleInitializeFinished(bool success)
{
if (!success) {
qWarning() << "Failed to initialize generic struct handler";
emit done();
return;
}
QObject::connect(m_client.get(), &QOpcUaClient::readNodeAttributesFinished,
this, &GenericStructHandlerExample::decoderDemo);
m_client->readNodeAttributes({
// Nested struct node on the test server
QOpcUaReadItem(QStringLiteral("ns=4;i=6009")),
// Union node on the test server with first member set
QOpcUaReadItem(QStringLiteral("ns=4;i=6011")),
// Union node on the test server with second member set
QOpcUaReadItem(QStringLiteral("ns=4;i=6003")),
// Struct node with optional field which is specified on the test server
QOpcUaReadItem(QStringLiteral("ns=4;i=6010")),
// Struct node with optional field which is not specified on the test server
QOpcUaReadItem(QStringLiteral("ns=4;i=6002"))
});
} Now we attempt to decode the binary-encoded extension objects we received from the server. Each extension object is passed to the QOpcUaGenericStructHandler::decode() method and the resulting QOpcUaGenericStructValue is printed to the terminal:
void GenericStructHandlerExample::decoderDemo(const QList<QOpcUaReadResult> &results, QOpcUa::UaStatusCode serviceResult)
{
if (serviceResult != QOpcUa::UaStatusCode::Good) {
qWarning() << "Failed to read";
emit done();
return;
}
qDebug() << "\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>";
qDebug() << "> Decode and print values read from the server <";
qDebug() << "<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<";
for (const auto &result : results) {
if (result.statusCode() != QOpcUa::UaStatusCode::Good) {
qWarning() << "Failed to read value for node" << result.nodeId() << result.statusCode();
continue;
}
// Get the extension object that was read from the server
auto extensionObject = result.value().value<QOpcUaExtensionObject>();
qDebug() << "\nRead encoded value of type" <<
m_handler->typeNameForBinaryEncodingId(extensionObject.encodingTypeId()) <<
"for node" << result.nodeId();
// Attempt to decode and check success
bool success = false;
auto decodedValue = m_handler->decode(extensionObject, success);
if (!success) {
qWarning() << "Failed to decode custom struct value for node" << result.nodeId();
emit done();
continue;
}
// Output via built-in debug stream operator
qDebug() << "Decoded value:" << decodedValue;
}
encoderDemo();
customTypeDescriptionDemo();
emit done();
}
The next method shows the usage of the QOpcUaGenericStructHandler::encode() method. We use the QOpcUaGenericStructHandler::createGenericStructValueForTypeId() helper to create a pre-filled QOpcUaGenericStructValue where only the fields and their values have to be inserted. After successfully encoding the values to binary-encoded extension objects, we decode these again to print the decoded data to the terminal.
void GenericStructHandlerExample::encoderDemo()
{
QList<QOpcUaGenericStructValue> values;
qDebug() << "\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>";
qDebug() << "> Demonstrate the encoding of generic struct values <";
qDebug() << "<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n";
// Add an instance of the optional field struct without the optional field specified
auto optionalFieldStruct = m_handler->createGenericStructValueForTypeId(QStringLiteral("ns=4;i=3006"));
optionalFieldStruct.fieldsRef()["MandatoryMember"] = 23.0;
values.push_back(optionalFieldStruct);
// Add an instance of the union struct with the second member specified
auto unionStruct = m_handler->createGenericStructValueForTypeId(QStringLiteral("ns=4;i=3005"));
auto innerStruct = m_handler->createGenericStructValueForTypeId(QStringLiteral("ns=4;i=3004"));
innerStruct.fieldsRef()["DoubleSubtypeMember"] = 42.0;
unionStruct.fieldsRef()["Member2"] = innerStruct;
values.push_back(unionStruct);
// Encode the values as extension object
// The resulting extension objects could be written into the value attribute
// of a node of the corresponding struct type on the server
QList<QOpcUaExtensionObject> encodedValues;
for (const auto &value : values) {
QOpcUaExtensionObject obj;
const auto success = m_handler->encode(value, obj);
if (!success) {
qWarning() << "Failed to encode generic struct value";
continue;
}
qDebug() << "Serialized data:" << obj.encodedBody();
encodedValues.push_back(obj);
}
// Decode and print result
for (const auto &extensionObject : encodedValues) {
qDebug() << "\nRead encoded value of type" <<
m_handler->typeNameForBinaryEncodingId(extensionObject.encodingTypeId());
// Attempt to decode and check success
bool success = false;
auto decodedValue = m_handler->decode(extensionObject, success);
if (!success) {
qWarning() << "Failed to decode custom struct value";
emit done();
continue;
}
// Output via built-in debug stream operator
qDebug() << "Decoded value:" << decodedValue;
}
} To finish our example, there is a demonstration on how to add manually-created QOpcUaStructureDefinition objects to the handler to enable decoding of types the server did not expose a structure definition for. For real world usage, the type id and default encoding id for such a type must correspond to the values from the server the client is connected to.
void GenericStructHandlerExample::customTypeDescriptionDemo()
{
qDebug() << "\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>";
qDebug() << "> Demonstrate the encoding and decoding with custom descriptions <";
qDebug() << "<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n";
// Invented node ids, this should correspond to the real ids on a server
const auto customTypeId = "ns=2;i=1234";
const auto customTypeEncodingId = "ns=2;i=1235";
QOpcUaStructureDefinition customDefinition;
customDefinition.setStructureType(QOpcUaStructureDefinition::StructureType::StructureWithOptionalFields);
customDefinition.setBaseDataType(QOpcUa::namespace0Id(QOpcUa::NodeIds::Namespace0::Structure));
customDefinition.setDefaultEncodingId(customTypeEncodingId);
QOpcUaStructureField mandatoryField;
mandatoryField.setDataType(QOpcUa::namespace0Id(QOpcUa::NodeIds::Namespace0::Int32));
mandatoryField.setValueRank(-1);
mandatoryField.setName("MyCustomMandatoryField");
QOpcUaStructureField optionalField;
optionalField.setDataType(QOpcUa::namespace0Id(QOpcUa::NodeIds::Namespace0::String));
optionalField.setValueRank(-1);
optionalField.setName("MyCustomOptionalField");
customDefinition.setFields({ mandatoryField, optionalField });
auto success = m_handler->addCustomStructureDefinition(customDefinition,
customTypeId,
"MyCustomTypeWithOptionalField");
if (!success) {
qWarning() << "Failed to add custom structure definition";
return;
}
auto value = m_handler->createGenericStructValueForTypeId(customTypeId);
value.fieldsRef()["MyCustomMandatoryField"] = 23;
value.fieldsRef()["MyCustomOptionalField"] = QStringLiteral("My test string");
QOpcUaExtensionObject obj;
success = m_handler->encode(value, obj);
if (!success) {
qWarning() << "Failed to encode custom structure";
return;
}
qDebug() << "Serialized data:" << obj.encodedBody();
const auto decodedValue = m_handler->decode(obj, success);
if (!success) {
qWarning() << "Failed to decode custom structure";
return;
}
qDebug() << "Decoded value:" << decodedValue;
} After successful initialization, the following output is produced by the example:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Decode and print values read from the server <
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Read encoded value of type "QtTestStructType" for node "ns=4;i=6009"
Decoded value: Struct QtTestStructType (QualifiedNameMember: QVariant(QOpcUaQualifiedName, QOpcUaQualifiedname(1, "TestName")) NestedStructMember: QVariant(QOpcUaGenericStructValue, Struct QtInnerTestStructType (DoubleSubtypeMember: QVariant(double, 42))) EnumMember: QVariant(int, 1) LocalizedTextMember: QVariant(QOpcUaLocalizedText, QOpcUaLocalizedText("en", "TestText")) Int64ArrayMember: QVariant(QList<qlonglong>, QList(9223372036854775807, 9223372036854775806, -9223372036854775808)) NestedStructArrayMember: QVariant(QList<QOpcUaGenericStructValue>, QList(Struct QtInnerTestStructType (DoubleSubtypeMember: QVariant(double, 23)), Struct QtInnerTestStructType (DoubleSubtypeMember: QVariant(double, 42)))) StringMember: QVariant(QString, TestString))
Read encoded value of type "QtTestUnionType" for node "ns=4;i=6011"
Decoded value: Union QtTestUnionType (Member1: QVariant(qlonglong, 42))
Read encoded value of type "QtTestUnionType" for node "ns=4;i=6003"
Decoded value: Union QtTestUnionType (Member2: QVariant(QOpcUaGenericStructValue, Struct QtInnerTestStructType (DoubleSubtypeMember: QVariant(double, 23))))
Read encoded value of type "QtStructWithOptionalFieldType" for node "ns=4;i=6010"
Decoded value: StructWithOptionalFields QtStructWithOptionalFieldType (MandatoryMember: QVariant(double, 42) OptionalMember: QVariant(double, 23))
Read encoded value of type "QtStructWithOptionalFieldType" for node "ns=4;i=6002"
Decoded value: StructWithOptionalFields QtStructWithOptionalFieldType (MandatoryMember: QVariant(double, 42))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Demonstrate the encoding of generic struct values <
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Serialized data: "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00""7@"
Serialized data: "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00""E@"
Read encoded value of type "QtStructWithOptionalFieldType"
Decoded value: StructWithOptionalFields QtStructWithOptionalFieldType (MandatoryMember: QVariant(double, 23))
Read encoded value of type "QtTestUnionType"
Decoded value: Union QtTestUnionType (Member2: QVariant(QOpcUaGenericStructValue, Struct QtInnerTestStructType (DoubleSubtypeMember: QVariant(double, 42))))
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Demonstrate the encoding and decoding with custom descriptions <
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Serialized data: "\x00\x00\x00\x00\x17\x00\x00\x00\x0E\x00\x00\x00My test string"
Decoded value: StructWithOptionalFields MyCustomTypeWithOptionalField (MyCustomMandatoryField: QVariant(int, 23) MyCustomOptionalField: QVariant(QString, My test string)) Conclusion
Previously, a developer had to look at a structure’s XML definition and implement the decoding and encoding by hand using the QOpcUaBinaryDataEncoding class while handling every aspect like switch fields for unions and bit masks for structures with optional fields manually.
The newly added classes make it much easier to interact with a server exposing custom structured type values because the QOpcUaGenericStructValue class is ideal for an explorative approach where getting to know the types involved can be as easy as printing a decoded value to the terminal and looking at the field names and values.
The new feature is currently only available on the Qt OPC UA dev branch. Until the feature freeze for Qt 6.7, which will take place in December, it will still possible to make API changes. Please let us know how you like the new API for generic structs!
