Common Settings for DataLoggers

Different pervasive service plugins exists for data logging (publishers) and data consuming (subscribers):

Name	Description	Publisher	Subscriber
HumanOS.PeSeL.AzureIoTClient	Access to Microsoft Azure cloud	X	X
HumanOS.PeSeL.CsvDataLogger	Logger to simple csv file	X	-
HumanOS.PeSeL.InfluxDbLogger	Logger to the influx database	X	-
HumanOS.PeSeL.MQTTClient	Access to a MQTT Broker	X	X
HumanOS.PeSeL.NodeSpaceDataLogger	Access to a NodeSpace commands	X	-
HumanOS.PeSeL.RabbitMqClient	Access to RabbitMessageQueue protocol	X	X
HumanOS.PeSeL.RestDataLogger	Logger to any web endpoint	X	-
HumanOS.PeSeL.SqlDataLogger	Logger to SQL Database	X	-

Plugin Configuration

The plugin configuration for publisher and subscriber contains various settings.

The next picture shows the class model of the configuration.

Each plugin is configured by a DataLoggerPluginConfiguration. This Configuration contains following Parameters:

Parameter	Description	Data Type
Disabled	Flag if the plugin is disabled. Set to false for plugin activation.	System.Boolean
Publishers	List of data publishers	TAbstractDataLoggerConfiguration
Subscribers	List of data subscribers	TAbstractDataPublisherConfiguration

DataConsumer Configuration (Subscriber)

Each subscriber instance represents a client subscribing data from a data source. Each subscriber has its own configuration.

Parameter	Description	Data Type
Id	Id of the data consumer	System.Guid
Name	Name of the data consumer. Used in log files	System.String
Connection	Connection string of the data consumer	System.String
NetworkTimeout	[OPT] Timeout in milliseconds [ms] of the network call when sending or receiving data from the platform.	System.Double
PayloadScriptFile	C# Script file to parse the subscribed data. Further information see the specific plugin documentation.	System.String
Properties	[OPT] array of key-value pairs containing custom specific data. Properties are accessible for payload processing.	[String, String]
SamplingRate	Time interval in seconds [s] in which data are polled.	System.Double
ServiceRule	The service rule enables and binds the client to the HumanOS® kernel.	TServiceRuleConfig

DataLogger Configuration (Publisher)

Each publisher instance represents a client publishing data to a destination (data sink). Each publisher has its own configuration.

Parameter	Description	Data Type
Id	Id of the publisher	System.Guid
Name	Name of the publisher. Used in log files	System.String
Connection	Connection string of the data logger	System.String
DataSets	Array of data sets see DataSet Config.
NetworkTimeout	[OPT] Timeout in milliseconds [ms] of the network call when sending or receiving data from the platform. Careful this impacts the buffering of data directly. Default is 1000ms	System.Double
PayloadDumpEnabled	[OPT] Flag if payload dump is enabled.	System.Boolean
PayloadDumpFileName	[OPT] Filename of the log file to dump the payload. Payload is logged to a text file when a send of the payload is successful	System.String
PayloadDumpRetentionTime	[OPT] Retention time in days [d] to keep the log files.	System.Int32
PayloadScriptFile	C# Script file to process datasets and generate the customized publish payload (message).	System.String
Properties	[OPT] array of key-value pairs containing custom specific data. Properties are accessible for payload processing.	[String, String]
SamplerType	Type of the data sampler.	EDataSamplerType
SamplingRate	Time interval in seconds [s] in which log data is sent. If set to 0, then only data is sent on changes (event-based logging)	System.Double
StartingSecond	Defines the a starting second, e.g. 10 will trigger the logging on the next 10 second mark	System.Int32
MinimalInterruptionTime	Minimum time which the logger must rest between sending	System.Double
ProcessOnlyValidData	If this flag is set, values are reported only if the data state is EDataState.Good	System.Boolean
ServiceRule	The service rule enables and binds the client to the HumanOS® kernel.	TServiceRuleConfig

Well known Properties

Here are some properties that are always available on a logger:

Parameter	Description	Data Type
LastTimeStamp	Timestamp of the last successful send	System.DateTime
Connected	Data logger is running and not buffering	System.Boolean
Function	Node indicator, always 'DataLogger'	System.String
SamplingRate	The sampling rate in which the data logger sends data	System.Double
Buffering	Indicates wether the logger is buffering	System.Boolean

Sampler Type

The following values are possible for the property SamplerType:

Value	Description
OnlyLastValue	Sends only the last value in the buffer. (last data received on interval end)
AllData	Sends all data recorded between two sampling cycles as time series. (Send all data received during interval)

Buffering

Whenever a data logger cannot send its data to the endpoint, it enters the buffering mode.

SamplingRate	SamplerType	Description
= 0ms	OnlyLastValue	In case of buffering the complete history is processed at once.
= 0ms	AllData	In case of alarm and event buffering, all alarms are processed
> 0ms	OnlyLastValue	Returns the history in sampling cycles, but only the last value within this cycle. If no values are present in current cycle, the last value of the prior cycle is returned. If history is empty, no data is returned.
> 0ms	AllData	Returns all history data within a sampling cycle. If no values are present in current cycle, no data is returned.

NOTE

For buffering, the history mode is required for data nodes and alarm event pools. See historization.

DataSet Configuration

The DataSet-element specifies the log entry data set. The data set contains fields.

info

The Dataset Name must be UNIQUE.

The dataset is specified by following elements (fields, more fields are possible):

Parameter	Description	Data Type
Name	Name of the field	System.String
NodeFilter	Node filter expression, e.g. node.hasProperty<string>("EnableMqtt", "1")	System.String
Type	Type of data provider. Possible values are: "AlarmEvent" (Alarm event objects), "DataNode" (data nodes)	EDataSetType
Tag	Tag data of the dataset	System.String
Properties	[OPT] array of key-value pairs containing custom specific data. Properties are accessible for payload processing.	[String, String]
Fields	Field data of the dataset. See Field Configuration.

DataField Configuration

Parameter	Description	Data Type
Name	Name of the field	System.String
NodeFilter	Node filter expression	System.String
DataType	Datatype	System.Type
Query	Expression selecting the data from a node. Data from a node is always expressed in the following format: item.{DataField}. See Available Query Information. **	System.String

Following objects can be used to query information:

Query	Value
item.GlobalId	Global node id
item.Node	Data node object
item.TimeStamp	Source timestamp
item.Value	Value
item.DataState	DataState
item.getValue<T>(name)	Gets the sub-value. Only available for TGenericEntity data types
logger.TimeStamp	Sampler timestamp, takes into account the historization

Service Rules

Service rules are used to bind services and devices. More information about Service Rules.

Important:

• Each client (publisher or subscriber) needs its own rule to bind and unbind nodes from devices. This allows to pipe each device to another Data Logger Client (broker, payload, etc.)

Understanding the Sampling Based Logger

If a sampling rate greater than zero is set, a sampling behavior applies. There are three main cases which can occur while sampling:

• Case 1: Sample duration not exceeded and minimal interruption can be applied
  • Each sample is executed as scheduled
• Case 2: Sample duration exceeded
  • Reschedules to the next possible execution based on the scheduling timeline
  • Will then add any missed data to the actual data
• Case 3: Minimal interruption time cannot be applied
  • Minimal interruption time is applied and the scheduler will then behave as described in case 2

Below is an example with a sample rate of 2s and a minimal interruption time of 1s: