Extension Package Development Questions

[jasong]

I am working on an Extension Package and had some questions. Rather than test all of these myself, perhaps someone knows? If not, I will test and post results here. Maybe this can be turned into a Q&A at some point.

I've numbered my questions so that readers and responders can quickly identify the question being answered.

(1) - Answered

If I build a package as version 1.0 and then again as 1.1, can I install 1.1 over top of 1.0 without uninstalling?
Am I protected from Installing 1.0 over top of 1.1? Boy I hope I get the right answer from you folks here. Having to uninstall first would be really bad news.

Assuming I can install v1.1 over v1.0...

(2)

Lets say v1.0 has a ThingTemplate in it called MyThingTemplate and it's defined purely as XML metadata.

Now v1.1 has created a new ThingPackage and defined MyThingTemplate using that new package, using the now very thin XML.

Can I install v1.1 over top of v1.0 in this case? I would guess yes but have not yet tried it.

(3) - Answered

If a ThingTemplate or DataShape or StyleDefinition or any Entity is marked as "isEditableExtensionObject == true", when you upgrade from v1.0 to v1.1, will that entity overwrite any changes that have been made? Or will it be skipped?

(4) - Answered

You can define properties (@ThingworxPropertyDefinition) in Java at the ThingPackage level or on the Template in the XML. Is there one advantage over the other?

(5) - Answered

initializeThing() - When does this run, ThingStart? Any other time? If not, should I use a standard Constructor to execute code on ThingStart?

(6) - Answered

The extension package examples show that you define a Property in Java and then include a private variable in your thing class. And then in initializeThing(), you grab that value from the Thing's persistence I guess and put it into the local private variable. But when you go to write this, don't you need to call this.setPropertyValue(...)?

And since you are only writing these values during initializeThing() , don't all of yoru services that you make that use those properties need to refresh those local copies to see if they have changed? And if so, why bother with the local class level private variable.

(7) - Answered

Can I make a local private variable that essentially acts as a hidden Property on the thing, knowing that it does not have persistence? Meaning, I can manipulate it from any of my java code from various services, but it won't be an actual visible thing property, and of course if the thing restarts, poof goes the value. Is that ok to do?

(8) - Answered

Can you define self-subscriptions on a ThingPackage in Java Code? Or only in the ThingTemplate XML metadata.

(9) - Answered

The guide say that ThingShapes do not need to extend Thing. But to get logging, I think it does.

protected static Logger aLog = LogUtilities.getInstance().getApplicationLogger(MYCLASSNAME.class);

Can I extend Thing and still use it as a ThingShape?

I tried this and did not see any errors but maybe this is not kosher. Is this okay? Or should I find a different way to make the logger work?

(10) - Answered

Can I write to ReadOnly properties from the Java side? That would be handy.

(11) - Answered

ThingShapes cannot have configuration tables: true or false?

(12) - Answered

Is there any way to code against another extension package? Do you just include the extension package's jar in Eclipse, careful not to include it in the build?

(13) - Answered

I seem to be able to, instead of extends Thing, use say, extends SchedulerThing. This seems to be ok but is this bad practice? If I wanted to make a SuperDuperSchedulerThing, this would work. But it would also work to simply have the ThingPackage extend Thing and then the ThingTemplate to use the ThingPackage I made with a BaseThingTemplate of SchedulerThing. Right? Either or? One way better than another?

(14) - Answered

In HelloWorldThing SDK example we see variables defined as Java primitives and not ThingWorx primitives.

e.g.

private String _stringProperty1 = "";

private Double _numberProperty1 = 0.0;

vs

private com.thingworx.types.primitives.StringPrimitive _stringProperty1 = "";

So, that's fine. I just want to know that this is the way to move forward with more complex ThingWorx types, such as JSON or Location.

private JSONObject _locationProperty = new JSONObject('"latitude":"0","longitude":"0","elevation":"0"}');

private string _jsonProperty = "";

and not

using primitives like

com.thingworx.types.primitives.JSONPrimitive

com.thingworx.types.primitives.LocationPrimitive

... and so on

The same question applies to Service inputs as well.

(15) - Answered

I need a good way to convert JSON Objects into POJOs. I see in the SDK that I have access to some of the Jackson json package but it appears to be missing a bunch of it, as all Jackson examples that I see out there reference objects that are not included in the SDK. I have service inputs that take JSON based input payloads and in the Java service they end up being JSONObjects, but that is not a very friendly tool, it'd be nice to be able to convert JSONObject to an actual POJO. Maybe I am missing something. I suppose I could deserialize the JSONObject (obj.toString()) but then I still need to deserialize it back into a POJO, and I'd like to use the Jackson libraries to do this, looks pretty simple but like I said, SDK does not seem to include the full Jackson package.

(16) - Solved

I cannot figure out how to execute a method on an extended Thing class from Java

To get an set properties, you grab the thing and then call the service to get or set

Thing myThing = ThingManager.getInstance().getEntityDirect("MyThingName");

myThing.setPropertyValue(...)

But I'd like to access public class variables or methods on my Thing. If I include a public bool or a public bool MyService() in my Thing's class, I still have no way of getting to it.

Additionally, I have no way of executing the methods I have defined as @ThingworxServiceDefinition

I thought casing might allow me to do this

MyThing myThing = (MyThing)ThingManager.getInstance().getEntityDirect("MyThingName");

but I get a cast exception. Thoughts?

(17) - Answered

My package is going to depend on several other extension packages. I suspect that some of these will cause errors upon package install if they are missing because they will be referenced as BaseThingTemplates or something. But others will only be referenced in code, and I am unsure if those missing packages will be caught. Is there any way to deal with that situation on package install? Like, some place I can place an Install Script to check for those and throw errors if they don't exist?

(This would also be helpful to have so that on install I could create some Things - see #18).

Otherwise I will just have to catch those errors and log and/or return errors to the user.. not terrible, but just wondering if there was a cleaner option.

(18) - Answered

I don't seem to be able to include actual Things in my metadata.xml and have them show up once the extension is installed.

Is this not allowed or perhaps did I do something dumb?

I am sure I will have more but that is all for now. Thanks for anyone who comes to help me out, I already appreciate it!

[jasong]

Some things I have learned

Changes to Resource service definitions require a refresh of composer to see, just like importing widgets. Or you can see changes immediately with the with REST navigation.

(10)

using this method

this.setPropertyValue("MyProperty", new StringPrimitive(myProperty));

where MyProperty is a Read only property, you get the expected error that the property is Read Only.

I only asked this because I really really struggle to understand any use case where you would use ReadOnly. If you can only set that value on the template for all instantiated Things... it's just a use case I cannot see.

(16)

this works: I am unsure why I was getting a cast error before, but as Saeed showed me, this is the way to do it. Works with getEntityDirect and getEntity

MyThing myThing = (MyThing)ThingManager.getInstance().getEntityDirect("MyThingName");

[asinclair]

(16) What if the Java code defines the Thing Template but not the Thing? Then how do you call a service on a Thing that inherits from the Thing Template? 

[danme]

I believe you will need to call processServiceRequest on your thing.

Assuming "myThing" is a reference to your thing:

String serviceName = "myservice";

ValueCollection paramValues = new ValueCollection();

paramValues.setValue("param1", new StringPrimitive("param1 value"));

myThing.processServiceRequest(serviceName, paramValues);

[jasong]

Andy, maybe I'm under-thinking this or misreading it, but I would guess you are new to ThWx and would benefit from reading up on the ThWx model.

A Thing is always defined by a ThingTemplate, and therefore, you cannot create a Thing in Java in the extension package. You create a Thing Template, and you can instantiate it as a Thing, and all of your properties and code that you designed at the Template level exist and execute at a thing.  It is not actually possible at this point to write a service that executes on a ThingTemplate, you just define them there. Same with shapes.

[asinclair]

Jason, thanks for the interest and helpful comments.You are right I have been working in the Java environment for only a few days converting behavior that is works in the Composer. I understand the inheritance relationships that are at play. What had me confused was I thought the only way you could execute a service in Java was to call the defining class member directly, which would require the Java has access to the call signature at compile time.

Dan supplied the missing piece for me ...

myThing.processServiceRequest(serviceName, params)

executes a forward reference in Java that is the equivalent of saying

me.serviceName(params);

in javasscript.

I had not seen one code sample of this until now - the samples in the Extension Developer Guide only access properties of a thing.

[jasong]

Ah got it.  I figured I was making bad assumptions!

[smanley]

(1) If I build a package as version 1.0 and then again as 1.1, can I install 1.1 over top of 1.0 without uninstalling?

Am I protected from Installing 1.0 over top of 1.1? Boy I hope I get the right answer from you folks here. Having to uninstall first would be really bad news.

Response- The new extension can be installed over the old one, but a tomcat restart is required to clear the old one out of memory and for the new changes to take effect.

[ibanham]

Hi

Answering the ones that I can...

(1) If I build a package as version 1.0 and then again as 1.1, can I install 1.1 over top of 1.0 without uninstalling?
Am I protected from Installing 1.0 over top of 1.1? Boy I hope I get the right answer from you folks here. Having to uninstall first would be really bad news.

As long as you're only updating the Property, Service and Event Definitions, just import over the existing extension and then re-start Tomcat. Do not attempt to re-name entities in the extension and then re-load it without removing the old version first. The Extension Developers Guide (downloadable from the Reference Documents section of the Support Site) is worth a read for best practices.

(5) initializeThing() - When does this run, ThingStart? Any other time? If not, should I use a standard Constructor to execute code on ThingStart?

This code runs when you create (or save) a Thing instance created from the ThingTemplate defined in the extension.

(11) ThingShapes cannot have configuration tables: true or false?

I would say true.

[danme]

I cant answer all of these, but here are the ones I can answer so far:

(5) initializeThing() gets executed whenever the thing is created OR saved.

(7) yes, a property will not show up on the paltform unless it is specifically defined, so you are free to use any variables you want in the java code that are not exposed. And your point about persistence is important when considering doing this.

(8) I believe subscriptions can only be defined in the XML

(12) I imagine for the most part this is bad practice. The java code in the background of extensions is meant to accomplish a specific goal and could change in future versions. If you want to interact with entities from other extensions, it is best to call the services through Thingworx rather than explicitly calling java functions. That being said, I imagine this would work if there was some use case where you need some background communitcation.

(13) I believe the difference here would be if you want to override a function. If you want to inherit functionality and add to it, then it is best to treat the template you want to extend as your baseThingTemplate. If however you want to override how an individual function(s) works, then you can extend it directly in the java code and change the functionality for this particular version of it.

[jasong]

Regarding 12, I wouldn't code against the 'lowercase' methods, just the ones public to everyone, the Uppercase methods.

If my package wants to work with a MailServer, then I still need to access the Java oterhwise I won't compile

e.g.

MailServer someMailServerThing = (MailServer)ThingManager.getInstance().getEntityDirect("MyMailServer");

I can't do that because during development I don't know what the "MailServer" class is.

I could do this:

Resource someMailServerThing = ThingManager.getInstance().getEntityDirect("MyMailServer");

But I don't get access to any of the methods, uppercase/"public" ones or not.

So my thought was to bring in the package's Java into my project and make sure it's not including it in the package that I am installing. I will try it sometime soon - still new to Java so I am fighting with the IDE a little bit

[danme]

Ah, well in that case you would want to make sure you included the jar for the class you are referencing. You can then cast the thing after getting it:

MailServerThing someMailServerThing = (MailServerThing) ThingManager.getInstance().getEntityDirect("MyMailServer");

This assumes that MailServerThing is the name of the class used to represent the mail server itself. Once the object is cast to the class you should be able to use it's services without the compiler complaining as long as the class is on your classpath. For any generic Thingworx template, you should be able to do this already when using the Extension SDK.

[thearonh]

(4) As a general rule of thumb, It is always preferred to use the Annotations (@ThingworxPropertyDefinition) if they are available.  I think that the origin of the XML version of defining these properties is there mostly because, when a Property on a Thing Template is created in composer and then exported, that is the xml that gets generated (because there isn't Java code to fall back on).

(12) and (17) - If there is an extension that you know is open to being extended (For example, one you have already created or one that includes a jar that you don't also want to include) there is a "dependsOn" attribute in the ExtensionPackage Node that will do the validation you are looking for.  This is where you can define a comma separated list of Extension names that need to be installed before that ExtensionPackage can be installed. If you try to install it out of order, you will receive an error message saying that you are missing a dependency and the import will fail.

Ex.

    <ExtensionPackages>

        <ExtensionPackage   description="my_description"

                                         minimumThingWorxVersion="6.0.0"

                                         name="Example_Extension"

                                         dependsOn="This_Extension_MUST_Exist"

                                         packageVersion="1.0"

                                         vendor="me" />

    </ExtensionPackages>

[jasong]

Just saw this post, confirming that you can indeed do subscriptions in Java

Extension : Thingworx Event Subscription

[thearonh]

(3) The isEditableExtensionObject flag essentially treats the Entity as if it is not an Extension Object when it comes to saving/overriding/editing etc.  I would strongly encourage that you set this flag to false when putting you Extension into production if you believe there will be iterations and multiple versions of your Extension (there almost always is).  However, if you will be subscribing to the pattern of importing your Extension with entities, editing them, exporting the new xml and pasting it into the metadata.xml for a new revision, then that flag can be a useful tool during the development process. This is, of course, only valuable in a few situations and If your Entity has associated Java code, I suggest that you do not do it this way and instead use the annotations to iterate your Entities.

The flag itself is ideally used in situations like extending and creating a custom Authenticator where you have input for properties that must be edited. For instance, the URL of the SSO web service that the Authenticator would communicate with - something that will be unique across different installs of the Extension.

[LeenaAgarwal]

We ended up setting isEditableExtensionObject=true on a mashup and packaged it in an extension and pushed it to our test and production systems some days back. Now, we need to make some changes in the mashup. I noticed the property is set but not needed, so removed it, made my changes and created an updated extension. When I imported it on my test server, the changes in mashup are not reflected. Other changes in other entities are incorporated correctly. Looks like entities with isEditableExtensionObject=true were skipped during extension import as you mentioned.

My question is, how to fix this now?

1. We are using ThingWorx 6.5.
2. Deleting the previous extension from production before importing latest extension is not an option for us.
3. We found a workaround - Open the mashup in composer, save it and then import the updated extension. All changes are reflected in this case. I am not sure what is happening behind the scene because of the explicit save in composer. Can someone explain this? Is this a reliable workaround?

[davel]

I'll have to look into it deeper, but I believe this is an OK workaround. What I think happens is that the save removes the isEditableExtension=true, so that on import the platform does not think it is editable, so overwrites it. Once you've successfully imported the new mashup, is it editable? I would assume not.

[LeenaAgarwal]

Since I have removed the property from the mashup in the updated extension, the mashup is no more editable directly from the composer.

It would help if you could look into it deeper and confirm. Thanks in advance!

[davel]

Sorry this took a while to respond (I'm not getting notices of responses for some reason). If you include the Mashup in the extension, then it will not be editable (unless marked as an editable extension object).

The basic issue is that adding isEditableExtensionObject=true to a Mashup makes it editable in Composer, but then it won't upgrade when you import a new version of the extension. Either an entity is editable, or upgradable, but it can't be both.

[davel]

(9) Logging should work OK with a ThingShape made from a POJO. It does not need to extend Thing. If you create a ThingShape from a class that extends Thing, it should work OK, but you'll be adding a lot of baggage from the Thing class that you don't need.

[jasong]

I am unsure how Services would work here. if you want to access Properties, in a normal Thing you could use

this.setPropertyValue

but unless you are extending a Thing, you don't have access to that service.

You also can't cast a Thing as a normal POJO, so you can't go get it via .getEntity() either.

As you can see, I have trouble understanding how this ThingShape development in Java is worthwhile for anything other than properties.

[ibanham]

Hi

You can use the following in ThingShape code to get the Thing instance:

Object me = ThreadLocalContext.getMeContext();

if (me instanceof Thing) {

Thing meThing = (Thing)me;

thingName = meThing.getName();

Regards

Ian

[jasong]

Right, ok. I will have to think about this one. It would not appear that I have access to any other services on the thing, like we would in Javascript, because we cannot cast to the specific Thing class. This might not be an issue for me - just trying to understand the implications.

[ckulak]

If you modeled this Thing in Java, you can cast it. ThingWorx instantiates those via reflection when it reads the Thing from the database, which implies the need for a default constructor. It silently falls back to instantiating a generic Thing if there were any exceptions during construction.

However, it is still not clear why you would like to do it, since you can operate on its public interface via ThingWorx own reflection mechanism (Thing.setPropertyValue, Thing.processServiceRequest, etc.)

[jasong]

Ah, Thing.processServiceRequest. I was unaware this was a legit way to use it, although I had seen it there. Bear in mind that there is very little information on developing via the platform SDK - that's why I have posted this as a single post with multiple questions contained to help those in the future. Thanks for the tip, this was helpful in multiple ways. I would urge someone, anyone, at PTC/ThWx to compile this information into the SDK dev guide, which is a good start but is still lacking in a lot of information.

Thanks!

[davel]

(12) You should be able to code against another extension jar - it would just be another dependent jar, like a logging jar or other utility jar. You would need create a dependency between the two extensions so that when you import the second one, it will work and not throw a class not found exception.

[davel]

(18) Including Things in your metadata.xml should work OK. Perhaps you have some issue with the thingTemplate the Thing points to.

[ckulak · Accepted solution]

Disclaimer: those are my own thoughts, not confirmed by R&D.

(6): Correct, local variables are dangerous and to me they only look useful in the following scenarios:

1. Caching for performance reasons (don't try this at home);

2. Storing transient data, e.g. JDBC Connection instance;

(14): According to (6) I wouldn't bother with local variables altogether.

(15): Of course you can include something like GSON, which is a 200 Kb JAR, but I would not recommend adding dependencies to your extension unless you really do A LOT of JSON (de)serialization. There's really not much difference between the two for a rather complex JSON:

String propertyName = json.getJSONObject("request").getJSONArray("properties").getJSONObject(0).getString("name");

vs

String propertyName = new GsonBuilder().create().fromJson(reader, MyEntity.class).getProperties().get(0).getName();

And for a simple one things get real ugly with the mapping:

String propertyName = json.getString("response");

vs

String propertyName = new GsonBuilder().create().fromJson(reader, MyEntity.class).getResponse();

Besides, you'll have to maintain the useless POJOs.

[jasong]

Just wanted to drop a line here and mentioned that... A TON OF STUFF HAS CHANGED from 6.5. to 6.6. Tons of now private classes, the whole method of getting Things or Resources is all different. Not noted in the release notes as per usual, and quite a few other things. A bunch of the questions now don't even apply anymore. Oye.

[sharmon]

You're right, Jason. To help others, I'll list a few I ran into updating a 6.5 extension to a 6.6. extension. This list is not exhaustive; it just contains the issues with which I dealt:

• Classes in the com.thingworx.system.managers package aren't public anymore. If you've used classes like ThingTemplateManager, DataShapeManager, ThingShapeManager, and others in the package to find entities, you'll need to update your code. Here's a code snippet with the old way, followed by the new way:

//Example - old method for finding a ThingShape
ThingShapeManager manager = ThingShapeManager.getInstance();
ThingShape myShape = manager.getEntity(shapeName);
//New way to find a ThingShape
ThingShape myShape = (ThingShape) EntityUtilities.findEntity(shapeName, ThingworxRelationshipTypes.ThingShape);

• The SetStringValue(...) method is no longer available on the ValueCollection object. Use put(...), instead. Here's some sample code:

ValueCollection parameterValues = new ValueCollection();
//Old way - use the SetStringValue(...) method
parameterValues.SetStringValue("name", paramName);
//New way - use put
parameterValues.put("name", new StringPrimitive(paramName));

• The FileRepositoryThing doesn't expose the fileExists(...) method anymore. This is easy to fix; just catch a FileNotFoundException:

//First, find a particular FileRepositoryThing without using a ThingManager
FileRepositoryThing myRepo =
  (FileRepositoryThing) EntityUtilities.findEntity("MyRepo", ThingworxRelationshipTypes.Thing);
//Old way - use the fileExists method
if(!myRepo.fileExists(filename){
  return "File \"" + filename + "\" does not exist";
}
//New way - Try to open the file for reading, catch exception if it doesn't exist.
try {
  myRepo.openFileForRead(filename);
} catch (FileNotFoundException fnfe){
   return "File \"" + filename + "\" does not exist";
} catch (Exception mysteryException){
  _logger.error(mysteryException.getLocalizedMessage());
}

• The com.thingworx.vocabularies.Vocabulary object is no longer available. My use case consisted of the following two requirements:
  • Look for a model tag vocabulary, and create it, it it doesn't exist. I did a, "rip and replace," style refactor, so the, "old way/new way," code comparison won't work here. I'll describe the old way - Use a TagManager object to return a Vocabulary object. Extract the tags from the vocabulary as an InfoTable. If the InfoTable has non-zero rows, it exists, and you do nothing. If zero rows, use EntityServices to create a new vocabulary.
  • Create a new model tag in the vocabulary. We formerly used to Vocabulary object to do that. Here's the new code, with some comments:

/*
  * STEP 1: See if the ModelTag Vocabulary exists by trying to create it.
  * If it already exists, swallow the exception.
  */
  try {
    EntityServices services = new EntityServices();
    services.CreateModelTagVocabulary(vocabularyName, null, null, true);
    } catch (InvalidRequestException invalidRequestException) {
      if (invalidRequestException.getMessage().contains("already exists")) {
      // No-op - bury the exception if the Vocabulary already exists.
      _logger.debug("No-op. ModelTag Vocabulary " + vocabularyName + " already exists.");
    } else {
      _logger.error("Unexpected error while creating Model Tag Vocabulary: "
      + invalidRequestException.getMessage());
    }
  } catch (Exception unexpectedException) {
    _logger.error("Unexpected error while creating Model Tag Vocabulary: " + unexpectedException.getMessage());
  }
  /*
  * STEP 2: Add the new tags to the Model Tag Vocabulary
  */
  try {
    // Create the tags
    TagLink taglink = TagLink.fromString(tagString);
    TagCollection tagsToAdd = new TagCollection();
    tagsToAdd.addTag(taglink);
    // Add them to the entity
    /* NOTE -  Here's the neat thing - the SDK no longer exposes Vocabulary, but it DOES
    *  expose RootEntity. The AddTags(...) method is on that parent object, so you can see
    *  it. I just follow my pattern of trying operations, and reacting to Exceptions. */
    RootEntity vocabulary = EntityUtilities.findEntity(vocabularyName,
    ThingworxRelationshipTypes.ModelTagVocabulary);
    if (vocabulary != null) {
      vocabulary.AddTags(tagsToAdd);
    } else {
      _logger.error("Problem adding term " + tagName + " to vocabulary " + vocabularyName);
    }
  } catch (Exception e2) {
    _logger.error("Unexpected error while adding tag " + tagName + " to Model Tag Vocabulary " + vocabularyName
    + ": " + e2.getMessage());
  }

• A small one - InfoTable doesn't expose the RowCount() method in the Extension SDK. I've used the, "lowercase function," getRowCount() as a replacement. It's not a best practice, but it solves my immediate issue.
• The LoadJSON() method is not directly exposed by the FileRepositoryThing class anymore. Here's how I solved the issue with the processServiceRequest(...) method:

//Old way
entityJson = myRepo.LoadJSON(filename).getJSONObject("thingTemplate");
/* New way - use the processServiceRequest method.
*
 * Note - The old method returned a JSONObject, so I was able to add an inline call
 * to getJSONObject to extract the JSON in which I was interested. processServiceRequest()
 * returns an InfoTable, so I need to do some extra processing. In its current state, it's 
 * kludgey. In the interest of sharing information, I post it here, but I would very much 
 * like to slim in down to something more elegant. For example, I could probably inline
 * the JSON-processing calls, but I'll leave that as an exercise for the reader.
 *
 * Replies with nicer ways to process InfoTables as JSON are welcome.
*/
//Create parameter collection to pass to service
ValueCollection serviceParams = new ValueCollection();
serviceParams.put("path", new StringPrimitive(filename));
//Use processServiceRequest to retrieve the JSON we need
InfoTable myJSONTable = myRepo.processServiceRequest("LoadJSON", serviceParams);
JSONObject allTheJSON = new InfoTableFunctions().ToJSON(myJSONTable);
JSONArray tempJSONArray = allTheJSON.getJSONArray("rows");
JSONObject firstRow = tempJSONArray.getJSONObject(0);
JSONObject content = firstRow.getJSONObject("Content");
entityJson = content.getJSONObject("thingTemplate");

• The AddTag(...) method is no longer exposed by TagCollection. See the answer about the Vocabulary object on how to solve that issue.