The Property Set Approach

Yes

This article details an approach developed by Prachi Rath and Roy Clarke, refined by the EDC team in the December 2019's Remote Monitoring of Assets Reference Benchmark , and used to handle multi-property business rules in an Enterprise ThingWorx application.

Introduction

If there are logic rules which depend upon multiple properties, and each property receives its updates one at a time, then each property will need to have an identical subscription, because there is no way for any one subscription to know the most up-to-date values for the other properties. This inefficient approach would create redundancy and sizing constraints, reducing the capacity of the application to scale up to the Enterprise level. The Property Set Approach resolves this issue by sending in all property updates as one Info Table or JSON property (called the “property set”), which can then have a single subscription. The property set is assembled on the Edge when an update needs to be sent, and then the Platform dissects, processes, and stores the data within this property set as required by the business logic.

This approach also involves caching the last property value into a runtime variable so that it can be referenced within the business logic subscription without having to be retrieved from the database. This can significantly improve the runtime of the subscription, reducing the number of resources required to sustain the business logic and ensuring that any alerts or events resulting from the business logic occur as soon as possible. It also reduces the load on the database, ensuring that data ingestion can complete unhindered.

So, while there are many benefits to this approach, it is also more complicated. It tightly couples the development of the Edge and Platform code and increases the application complexity, making it slightly less easy to maintain the application in the long run. The property set also requires a little more bandwidth and a more stable internet connection between Edge devices and the Platform since there is more metadata in an Info Table property, and therefore every update is slightly larger than it would be otherwise. So this approach is only recommended when multi-property rules are a requirement of the application and a stable internet connection exists between the Edge and Platform.

Platform Implementation

I. Create an Info Table (or JSON) Property

This tutorial uses the out-of-the-box Data Shape called NamedVTQ for the Info Table property, which is defined on a Thing Template as a remote property. It is important that this is not marked as persistent or logged, as the purpose is to reduce the amount of database writes and reads required by the Platform. The Info Table property has the following property definition:

<PropertyDefinition aspect.dataChangeType="ALWAYS" aspect.dataShape="NamedVTQ" aspect.isPersistent="false" baseType="INFOTABLE" isLocalOnly="false" name="numberPropertySetAsInfotable"/>

II. Create a Data Change Event Subscription for the Info Table Property

The subscription has three parts:

Cache the last value for the property in a runtime variable
Start off the business rules processing, sending in the whole Info Table
Send the Info Table to be logged as individual local properties in the database

// First step caches the last Value, refer to the next step…

// Second step sets off the business rules processing with the Info Table

me.ScaleTestBusinessRuleForPropertySetAsInfotable({
      PropertySetAsInfotable: eventData.newValue.value
});

// Third step sends the Info Table as one property into a service which parses it into the 
// individual properties, updating both the runtime properties on the remote thing and the database

me.UpdatePropertyValues({
     values: eventData.newValue.value /* INFOTABLE */
});

III. Set-Up Caching

Each property which needs to be cached should be created on the Thing Template level and named in a similar way, say by placing the word “Last” at the end, such as “Property1” => “Property1Last”, “Property2” => “Property2Last”, etc. This property should NOT be logged or persistent, as the point of this is to store the most recent value in memory, removing any superfluous dependency on database queries in the process. Note that while storing the property in runtime memory makes it much more accessible, it also means that the property needs to be rewritten manually upon Platform restart. Additional code (not provided here) must be written to populate these properties from the database upon application start-up.

The following code should be placed in the data change event subscription (option 1 in the case where only a few properties need caching, or option 2 if every property value needs to be cached):

Option 1: Some but Not All Properties Need Caching

// Names of properties for which you want to cache the last value
var propertyNames = ['number1', 'number2'];

// Loop through the properties and cache their time if they are found in the property set
propertyNames.map(assignLast);

// This function can be split into two functions for Age and Last separately if need be
function assignLast(propertyName) {
    logger.debug("Looping for property -> "+ propertyName);
    var searchprop = new Object();
    searchprop.name = propertyName;
    property = eventData.newValue.value.Find(searchprop);
    if(property){
        logger.debug("Found Row. Name= " + property.name);
        
	 var lastPropertyName = propertyName+"Last";
	    
	 if(property.value) {
      // Set the cache property on me, this entity, to the current property value
             me[lastPropertyName] = me[propertyName];
        } 
    } else {
    	 logger.debug("Property Not Found in property set -> " + propertyName);
    }
}

Option 2: All Properties Need Caching

var rowCount = eventData.newValue.value.getRowCount();

for(var i=0; i<rowCount; i++){
    logger.warn("property name->" + eventData.newValue.value[i].name + "----- property new value->" + eventData.newValue.value[i].value.value);
    var propertyName = eventData.newValue.value[i].name;
    var lastPropertyName = propertyName+"Last";
    me[lastPropertyName] = me[propertyName];
    logger.warn("done last subscription, last property value for  lastPropertyName" + me[lastPropertyName]);
}

Useful Platform Code Snippets

I. Age Calculation

var date1 = new Date();
var date2 = me.GetPropertyTime({
     propertyName: propertyName /* STRING */
});

var result = millisToMinutesAndSeconds (dateDifference(date1, date2) );

// This function converts from an unintelligibly large number in milliseconds to something formatted in minutes and seconds
function millisToMinutesAndSeconds(millis) {
     var minutes = Math.floor(millis / 60000);
     var seconds = ((millis % 60000) / 1000).toFixed(0);
     return (seconds == 60 ? (minutes+1) + ":00" : minutes + ":" + (seconds < 10 ? "0" : "") + seconds);
}

II. Sort the Info Table by Time

var params = {
	sortColumn: "time" /* STRING */,
	t: me.propertySet/* INFOTABLE */,
	ascending: ascending /* BOOLEAN */
};
var result = Resources["InfoTableFunctions"].Sort(params);

III. Search the Info Table for a Property

var searchprop = new Object();
searchprop.name = propertyName;
property = PropertySetAsInfotable.Find(searchprop);
if(property === null){
      logger.info("Property Not Found -> " + propertyNumber1);
} else {
      logger.info("Found Row. Name= [" + property.name + "], value= " + property.value.value);
}

Edge Implementation

This example implementation uses the .NET Edge SDK to build a property set Info Table at the Edge.

I. Define the Data Shape

A standard Data Shape is used (NamedVTQ), but because this Data Shape is not exposed in the Edge SDK code, it has to be created manually.

// Data Shape definition for NamedVTQ
FieldDefinitionCollection namedVTQFields = new FieldDefinitionCollection();

namedVTQFields.addFieldDefinition(new FieldDefinition(CommonPropertyNames.PROP_NAME, BaseTypes.STRING));

namedVTQFields.addFieldDefinition(new FieldDefinition(CommonPropertyNames.PROP_VALUE, BaseTypes.VARIANT));

namedVTQFields.addFieldDefinition(new FieldDefinition(CommonPropertyNames.PROP_TIME, BaseTypes.DATETIME));

namedVTQFields.addFieldDefinition(new FieldDefinition(CommonPropertyNames.PROP_QUALITY, BaseTypes.STRING));
            
base.defineDataShapeDefinition("NamedVTQ", namedVTQFields);

II. Define the Info Table Property

The property defined should NOT be logged or persistent, and it can be read-only, since data is always pushed from the Edge and read from the server cache when accessed on the Platform. Note that the push type of the info table property MUST be set to "ALWAYS" (if set to "VALUE", the data change event will only fire if the number of rows changes).

// Property Set Definitions
[ThingworxPropertyDefinition(
        name = "DevicePropertySet", 
        description = "Alternative representation of properties as an Info Table for rules processing", 
        baseType = "INFOTABLE", 
        category = "Status", 
        aspects = new string[] {
            "isReadOnly:true",
            "isPersistent:false",
            "isLogged:false",
            "dataShape:NamedVTQ",
            "cacheTime:0",
            "pushType:ALWAYS" }
        )
]

III. Define a Property to Store the GOOD Quality Status

private static String QUALITY_STATUS_GOOD = QualityStatus.GOOD.name();

IV. Define Functions to Populate the Value Collections

An Info Table is really just made up of many Value Collections, where each Value Collection is considered a row. These services take in the name and value of a property and return a Value Collection object which can be added to the property set Info Table.

public ValueCollection createNumberValueCollection(String name, double value)
{
       ValueCollection vc = new ValueCollection();

       // Add quality and time entries to the Value Collection
       vc.SetStringValue(CommonPropertyNames.PROP_QUALITY, QUALITY_STATUS_GOOD);
       vc.SetDateTimeValue(CommonPropertyNames.PROP_TIME, new DatetimePrimitive(DateTime.UtcNow));

       vc.SetStringValue(CommonPropertyNames.PROP_NAME, name);
       vc.SetNumberValue(CommonPropertyNames.PROP_VALUE, value);
       return vc;
}

        
public ValueCollection createBooleanValueCollection(String name, Boolean value)
{
       ValueCollection vc = new ValueCollection();

       // Add quality and time entries to the Value Collection
       vc.SetStringValue(CommonPropertyNames.PROP_QUALITY, QUALITY_STATUS_GOOD);
       vc.SetDateTimeValue(CommonPropertyNames.PROP_TIME, new DatetimePrimitive(DateTime.UtcNow));

       vc.SetStringValue(CommonPropertyNames.PROP_NAME, name);
       vc.SetBooleanValue(CommonPropertyNames.PROP_VALUE, value);
       return vc;
}

V. Build the Property Set

Call this code from the processScanRequest method to build the property set.

// Create an instance of a new Info Table using the standard "NamedVTQ" Data Shape
InfoTable propertySet = new InfoTable(getDataShapeDefinition("NamedVTQ"));

// Set name/value for Temperature using convenience function
propertySet.addRow(createNumberValueCollection("Temperature", temperature));

// Set name/value for Pressure using convenience function
propertySet.addRow(createNumberValueCollection("Pressure", pressure));

// Set name/value for TotalFlow using convenience function
propertySet.addRow(createNumberValueCollection("TotalFlow", this._totalFlow));

// Set name/value for InletValve using convenience function
propertySet.addRow(createBooleanValueCollection("InletValve", inletValveStatus));

// Set name/value for FaultStatus using convenience function
propertySet.addRow(createBooleanValueCollection("FaultStatus", faultStatus));

// Set the property set Info Table property
base.setProperty("DevicePropertySet", propertySet);

VI. Update the subscribed properties

These two lines of code update the properties and events, actually sending the property set (containing all property updates) to the Platform.

base.updateSubscribedProperties(15000);
base.updateSubscribedEvents(60000);

Conclusion

Following these steps will enable the Edge to build a property set before sending any property updates to the Platform. The Platform can then rely on caching to process the business logic with no database dependency, which is faster and more efficient than any other approach. Finally the updates are still written to the database, so in the end, there is no functional difference between using a property set and binding each property individually. Please don't hesitate to comment here with any questions about this approach.

The Property Set Approach

Best Practices

EDC