IoT Tips

Video Author:                     Christophe Morfin Original Post Date:            October 2, 2017 Applicable Releases:        ThingWorx Analytics 8.1   Description:​ In this video we will walk thru the installation steps of ThingWorx Analytics Server 8.1.  This covers the Native Linux installation though the steps will be similar for a docker installation on Windows or Linux.    

      Thingworx extensions are a great place to explore UI ideas and get that special feature you want.   Here is a quick primer on Widgets (Note: there is comprehensive documentation here which explores the complete development process ).The intention is not to explain every detail but just the most important points to get you started. I will explore more in additional posts. I also like images rather than lost of words to read. I have attached the simple Hello Word example as a start point and  I'm using Visual Code as my editor of choice.   The attached zip when unzipped will contain a folder called ui and metadata xml file. Within the ui folder there needs to be a folder that has the same name as the widget name. In this case its helloworld.   Metadata file - The 3 callouts are the most import. Package version: is the current version and each time a change is made the value needs to be updated. name: a unique name used through out the widget definition UIResources: The source locations for the widget definition. The UIResources files are used to define the widget in the ide (Composer) and runtime (Mashup). These 2 environments ide and runtime have matching pairs of css (cascading style sheets)  and a js (javascript) files.   The js files are where most of the work is done. There a number of functions used inside the javascript file but just to get things going we will focus on the renderHtml function. This is the function that will generate the HTML to be inserted in the widget location.   renderHtml(helloWorld.ide.js) In this very simple case the renderHtml in the runtime is the same as in the ide renderHtml (helloWorld.runtime.js)   Hopefully you can see that the HTML is pretty easy just some div and span tags with some code to get the Property called Salutation.   So we have the very basics and we are not worried to much about all the other things not mentioned. So to get the simple extension into Thingworx we use the Import -> Extensions menu option. The UI and metadata.xml file needs to be zipped up (as per attachment).  Below is a animated gif that shows how to import and use the widget   Very Quick Steps to import and use in mashup. Video Link : 2147   The next blog will explore functions and allow a user to click the label and display a random message. This will show how to use events   Widget Extensions Click Event

Licensing summary, installing 8.1: - Now instance specific license used which prevents license sharing and protects our intellectual property further - 2 paths for getting up and running with a license:          -Connected: platform-settings configuration only          -Disconnected: text file generated, self-serve creation from PTC support portal, license_capability_response.bin generated to be placed in platform folder   Connected / Disconnected mode LicensingSubsystem::GetLicenseState returns : UNINITIALIZED in disconnected mode LICENSE_EXISTS in connected mode   User Journey Refer to Licensing ThingWorx 8.1 and Later for specific instructions for generating a license for your instance   NOTE: Each instance needs a valid license file to be running.   Troubleshooting: - If any misconfiguration or connection failures occur, error messages will be thrown to Application log Example: unable to fetch license file with device id; Unable to connect to the PTC license server. Please make sure the LicensingConnectionSettings settings... - Connection attempt will occur upon ThingWorx startup. If connection can't be made, instance will be following disconnected scenario  -Make sure Pop Up Blockers are turned off     Platform Settings - Licensing License Connection Settings (in platform-settings.json - plain text sample) -new section for licensing connection strings: -user name - used to connect to ptc support portal -password - encrypted (recommended) or plain text password used to connect to ptc support portal   "LicensingConnectionSettings": { "username":"<PTC_support_portal_username>", "password":"<PTC_support_portal_password>" }    New Services on Licensing Subsystem - GetInstanceID - returns instance/device ID which is created at startup - WriteLicenseUsageData - writes encrypted license usage (same as system data table) to ThingworxStorage\reports\LicenseUsageReport folder   Instance ID The license file is now bound to the platform Instance ID aka Device ID - (unlike most other PTC products where the license is bound to the hardware : CPUID, MAC, ...) This Instance ID is generated during first startup and stored in the database Instance ID is accessible in composer with LicensingSubsystem::GetInstanceID   Q: What happens when license files are bad or missing? A: If there is an invalid license file, with 8.0 valid license.bin needs to be in the folder before starting up; tomcat will crash. In 8.1 no need for license file as long as connected (platform-settings.json), no need to take an extra step, dynamic connection.  In disconnected scenario, ThingWorx will run for 30 days in limited mode with monitoring mashup accessible to check logs.   Q: Where is the InstanceID stored ? A: It's stored in the database   Q: Will the instanceID change during updates (minor 8.1.0 to 8.1.1) A: Device id's don't change.   Q: What will happen in disconnected scenario if there is no valid license after 30 days? The application will not start anymore or user is not able to login? A:  It shuts down, so there is no more "limited" mode. However, a user can come along on day 55 (for example) and can drop in a valid license and start the web app to get it fully running.   Q: What happens if the customer has to reinstall the platform after the license has been fetched ? Is it possible to "return" the license ? A: Reinstalling the platform results in the generation of a new Device ID, therefore a new license file will need to be generated.  

A common issue that is seen when trying to deploy, design or scale up a ThingWorx application is performance.  Slow response, delayed data and the application stopping have all been seen when a performance problems either slowly grows or suddenly pops up.  There are some common themes that are seen when these occur typically around application model or design.  Here are a few of the common problems and some thoughts on what to do about them or how to avoid them. Service Execution This covers a wide range of possibilities and is most commonly seen when trying to scale an application.  Data access within a loop is one particular thing to avoid.  Accessing data from a Thing, other service or query may be fast when only testing it on 100 loops, but when the application grows and you have 1000 suddenly it's slow.  Access all data in one query and use that as an in memory reference.  Writing data to a data store (Stream, Datatable or ValueStream) then querying that same data in one service can cause problems as well.  Run the query first then use all the data you have in the service variables.   To troubleshoot service executions there are a few methods that can be used.  Some for will not be practical for a production system since it is not always advisable to change code without testing first. Used browser development tools to see the execution time of a service.  This is especially helpful when a mashup is slow to load or respond.  It will allow quickly identifying which of multiple services may be the issue. Addition of logging in a service.  Once a service is identified adding simple logging points in the service can narrow what code in the service cases the slow down (it may be another service call).  These logging statements show up in the script logs with time stamps ( you can also log the current time with the logging statements). Use the test button in Composer.  This is a simple on but if the service does not have many parameters (or has defaults) it's a fast and easy way to see how long a service takes to return,' When all else fails you can get thread dumps from the JVM.  ThingWorx Support created an extension that assists with this.  You can find it on the Marketplace with instructions on how to use it.  You can manually examine the output files or open a ticket with support to allow them to assist.  Just be careful of doing memory dumps, there are much larger, hard to analyse and take a lot of memory.  https://marketplace.thingworx.com/tools/thingworx-support-tools Queries ​These of course are services too but a specific type.  Accessing data in ThingWorx storage structures or from external sources seems fairly straight forward but can be tricky when dealing with large data sets.  When designing and dealing with internal platform storage refer to this guide as a baseline to decide where to store data...  Where Should I Store My Thingworx Data?   NEVER store historical data in infotable properties.  These are held in memory (even if they are persistent) and as they grow so will the JVM memory use until the application runs out of it.  We all know what happens then.  Finally one other note that has causes occasional confusion.  The setting on a query service or standard ThingWorx query service that limits the number of records returned.  This is how many records are returned to from the service at the end of processing, not how many are processed or loaded in memory.  That number may be much higher and could cause the same types of issues. Subscriptions and Events ​This is similar to service however there is an added element frequency.  Typical events are data change and timers/schedulers.  This again is often an issue only when scaling up the number of Things or amount of data that need to be referenced.  A general reference on timers and schedulers can be found here.  This also describes some of the event processing that takes place on the platform.  Timers and Schedulers - Best Practice For data change events be very cautions about adding these to very rapidly changing property values.  When a property is updating very quickly, for example two times each second, the subscription to that event must be able to complete in under 0.5 seconds to stay ahead of processing.  Again this may work for 5-10 Things with properties but will not work with 500 due to resources, speed and need to briefly lock the value to get an accurate current read.  In these cases any data processing should be done at the edge when possible (or in the originating system) and pushed to the platform in a separate property or service call.  This allows for more parallel processing since it is de-centralized. A good practice for allowing easier testing of these types of subscription code is to take all of the script/logic and move it to a service call.  Then pass any of the needed event data to parameters in the service.  This allows for easier debug since the event does not need to fire to make the logic execute.  In fact it can essentially be stand alone by the test button in Composer. Mashup Performance This​ one can be very tricky since additional browser elements and rendering can come into play. Sometimes service execution is the root of the issue and reviewed above, other times it is UI elements and design that cause slow down. The Repeater widget is a common culprit. The biggest thing to note here is that each repeater will need to render every element that is repeated and all of the data and formatting for each of those widgets in the repeated mashup. So any complex mashup that is repeated many times may become slow to load. You can minimize this to a degree based on the Load/Unload setting of the widget and when the slowness is more acceptable (when loading or when scrolling). When a mashup is launched from Composer it comes with some debugging tools built in to see errors and execution. Using these with browser debug tools can be very helpful. Scaling an Application When initially modeling an application scale must be considered from the start. It is a challenge (but not impossible) to modify an application after deployment or design to be very efficient. Many times new developers on the ThingWorx platform fall into what I call the .Net trap. Back when .Net was released one of the quote I recall hearing about it's inefficiencies was "memory is cheap". It was more cost efficient to purchase and install more memory than to take extra development time to optimize memory use. This was absolutely true for installed applications where all of the code was complied and stored on every system. Web based applications are not quite a forgiving since most processing and execution is done on the single central web server. Keep this in mind especially when creating Shapes, Templates and Subscriptions. While you may be writing one piece of code when this code is repeated on 1,000 Things they will all be in memory and all be executing this code in parallel. You can quickly see how competition for resources, locks on databases and clean access to in memory structures can slow everything down (and just think when there are 10,000 pieces of that same code!!). Two specific things around this must be stated again (though they were covered in the above sections). Data held in properties has fast access since it is in JVM memory. But this is held in memory for each individual Thing, so hold 5 MB of information in one Thing seems small, loading 10,000 Thing mean instant use of 50 GB of memory!! Next execution of a service. When 10 things are running a service execution takes 2 seconds. Slow but not too bad and may not be too noticeable in the UI. Now 10,000 Things competing for the same data structure and resources. I have seen execution time jump to 2 minutes or more. Aside from design the best thing you can do is TEST on a scaled up structure. If you will have 1,000 Things next year test your application early at that level of deployment to help identify any potential bottlenecks early. Never assume more memory will alleviate the issue. Also do NOT test scale on your development system. This introduces edits changes and other variables which can affect actual real world results. Have a QA system setup that mirrors a production environment and simulate data and execution load. Additional suggestions are welcome in comments and will likely update this as additional tool and platform updates change.

Ran into this recently thought I share an approach to getting a table with multi-column distinct yet retaining all the columns of the row. If you use Distinct, you get only the Columns you do Distinct on. This isn't very helpful if you want the 'latest' or the 'first occurrences'  of records in your table with a combination of fields being unique. For example I had Process, Part, Dimension and Point for which I had multiple value and date time entries, but I only wanted the latest entries. Following is how I solved it, if you have a better way please leave a comment! P.S.: for the query I used the awesome query builder available in the snippet section! --------------------------------------- var q1Result = Things["MyThing"].QueryStreamEntriesWithData({maxItems:99999, query:query1}); //Below creates a temporary measurement table to store the latest meaurement values var params = {                 infoTableName : "InfoTable",                 dataShapeName : "MyDatashape.DS" }; // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(MyDataShape.DS) var tempTable1 = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); // Extract only the latest measurements for the PART from the measurement result table 'q1Result' //The way we are going to reduce this to unique measurements is //1. records are in reverse order of date time //2. get distinct by Process Part Dim Point //3. Step through and match against distinct set //4. First match goes into final set //5. Upon match remove from distinct set //6. If no match then skip record //7. If no more distinct match records break loop var params = {                 t: q1Result /* INFOTABLE */,                 columns: 'ProcessID,PartID,Dimension,Point' /* STRING */ }; // result: INFOTABLE var distinctResult = Resources["InfoTableFunctions"].Distinct(params); for (var x = 0; x < q1Result.rows.length; x++) {     var query = {       "filters": {         "type": "AND",         "filters": [           {             "fieldName": "ProcessID",             "type": "EQ",             "value": q1Result.rows .ProcessID           },           {             "fieldName": "PartID",             "type": "EQ",             "value": q1Result.rows .PartID          },           {             "fieldName": "Dimension",             "type": "EQ",             "value": q1Result.rows .Dimension           },           {             "fieldName": "Point",             "type": "EQ",             "value": q1Result.rows .Point           }         ]       }     };   var params = {                 t: distinctResult /* INFOTABLE */,                 query: query /* QUERY */ }; // result: INFOTABLE var matchResult = Resources["InfoTableFunctions"].Query(params);     if (matchResult.rows.length == 1) {         tempTable1.AddRow(q1Result.rows );            var params = {             t: distinctResult /* INFOTABLE */,             query: query /* QUERY */         };         // result: INFOTABLE         var distinctResult = Resources["InfoTableFunctions"].DeleteQuery(params);         if (distinctResult.rows.length == 0) {                        break                    }            }    } //I now have a tempTable1 with the full rows and the 4 fields distinct result = tempTable1

This is a note/reminder in cross-reference to https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS267248&lang=en_US Java 9 is expected to be released by Oracle September 21 2017 and is not currently supported for ThingWorx, refer to ThingWorx System Requirement guides for support details: ThingWorx 8 ThingWorx 7 ThingWorx 6

New Generation Composer is available from ThingWorx 7.4 and later. Each subsequent release of ThingWorx will contain additional New Composer features/functionalities. This video is focused on the layout change and new features implemented from ThingWorx 7.4.     How to enable the new Composer? 1. In the top right-hand corner, click on the User Menu and select the Preferences option                Figure 1 2. Click the check box for Turn on New Composer Features. 3. Click Done. A New Composer link is added to the menu bar at the top of the Composer window.     Figure 2   4. Click the New Composer link to open a new tab for the New Composer view   Figure 3   What's the layout change in New Composer?     Three areas layout   Figure 4     Menu bar on the left (Area 1) Set Project Context to set default project name for new entities Two views: Recent and Browse Recent view will quickly locate the recent access entities Browse is almost the same to the old menu navigation bar Could be sizable or hidden, and the main idea here is to increase screen real-estate to allow bigger view/edit area Main area for listing and feeding entities in the middle (Area 2) It provides you a wider area to edit entities (author services, mashup builder, etc.) An extra area on the right for preview, properties/events editing, etc. (Area 3) It gave you an easier and handy way to get a glance of an entity’s basic information   Layout change in an entity’s editing page When you create a new Thing, you will find all the facets (general information, properties, services, events, subscriptions) of the Thing are listed in a dropdown list Figure 5       By doing so it will save more area for the feeding   Properties and Alerts Properties and Alerts are now listed separately (in two different Tabs) Figure 6 Figure 7  Properties and Alert are now edited on the right area of the page (See Figure 4 Area 3 Services and Subscription A bigger editor area Events Events are edited on right area of the page (See Figure 4 Area 3)   What's the main function/features change in New Composer   Industrial Connections The Industrial Connections entity allows you to connect with and configure industrial things in ThingWorx. With the “Discover” feature, you could easily bind the Industrial device’s (e.g., Kepware) Tags to a ThingWorx Entity Figure 8 From ThingWorx 8.0.0, Anomaly Alert type is supported An anomaly alert is only useful if you have configured Anomaly Detection to monitor a stream of data Anomaly metrics settings are allowed to be configured in the Alert edit page Figure 9 Subscriptions You could now manually remove a subscription permanently from the system in New Composer which is impossible in old Composer Services New Composer provided assistant scripting tools like static code analysis, String search or replace, etc Figure 10   How could I switch back to the Old Composer when editing an entity? It is so easy! As long as you have opened an ThingWorx Entity, you will notice there is a button “Edit in Composer”; it will lead you back to the old Composer, and all the editing that have been saved will be logged in the old Composer.     Video demonstration for the New Composer is also available now. Feel free to review from: New Composer Video

Yes, the ThingWorx Manufacturing apps can be imported as ThingWorx extensions into an existing ThingWorx Platform install (v8.0.1). ThingWorx Manufacturing Apps are imported as a ThingWorx extension for all instances other than the Express and Developer Edition licenses. Note: If you ran the Express installer, you cannot import ThingWorx Manufacturing Apps as a ThingWorx extension. Instructions for importing these are included with the download, or are also available here: Not authorized to view the specified document 3992

Follow these steps to restart the ThingWorx Server, then to verify that all install services are running: 1. Open the ThingWorx Server Properties configuration application. To locate this application in Windows Server edition operating systems, search for "Configure PTC Servelet Engine"; or the application can be launched from the Windows Explorer. The default install directory is: <ThingWorx install path>\ThingWorxManufacturingApps\PTC_Servlet_Engine\bin\thingworxServerw.exe 2. Use the controls in the General tab to Stop and then Start this service. 3. Now open the Windows Services console (services.msc) and verify these four services are installed and running:      a. ThingWatcherResult      b. ThingWatcherTraining      c. Thingworx Server      d. ThingworxPostqresql_Service-PostgreSQL_Server9.4

A license is not required to download and install KEPServerEX V6. After finishing the full install, all features of the KEPServerEX Configuration are available for setup and testing, including the ThingWorx Native Interface. When either a Client application makes a request of a Driver, or a Plug-in becomes activated, then a license check is performed. If a feature is not licensed, a two-hour demo countdown period will begin for that feature. There are System tags that can be monitored in KEPServerEX to determine which features are licensed, which are in a time-limited demo countdown period, and which have expired. To reset the countdown timer for an unlicensed feature, the KEPServerEX Runtime service can be stopped/restarted. See the links and steps below for more information. How do I download and install KEPServerEX?​ Monitor licensed, time-limited, and expired features in KEPServerEX:​ 1. Open the KEPServerEX Configuration window 2. The right-most icon in the Configuration icon toolbar is the white "QC" icon. Select this icon to launch a new OPC Quick Client window. The server project should automatically build 3. In the left pane, locate the _System folder 4. Within this folder locate the following three Tags, with the corresponding information displaying in the Value column: Tag (Item ID) Information displayed as a String in the Value column _System._ExpiredFeatures Any unlicensed driver or plug-in that has timed out _System._LicensedFeatures Drivers or plug-ins that are currently licensed for use with this install _System._TimeLimitedFeatures Any unlicensed driver or plug-in that is in demo mode, with the remaining demo timer countdown (in seconds) Note: The ThingWorx Native Interface does not require a KEPServerEX license. Restart the KEPServerEX Runtime service The KEPServerEX Runtime service (server_runtime.exe) runs in the background as a system service. Resetting this service will reset the demo countdown for any unlicensed feature. To reset the Runtime service: 1. Right-click on the Administration tool (the green EX icon in the Systray by the System Clock) 2. Select "Stop Runtime Service. 3. Either the user will be prompted to restart this service, or the same Administration tool menu can be used again to "Start Runtime Service"

The most common cause for this is a shortage of available RAM. It may be necessary to increase the available RAM or install additional cores. Please refer to the minimum install requirements listed here: What are the minimum Hardware and System requirements for installing the ThingWorx Manufacturing Apps?

Connecting Existing Things to ThingWorx Industrial Gateway for Anomaly Detection   In this Video you will learn how to :   - To bind a property of an existing entity to the KEPSserverEX Data Feed - To create an Alert on that property and monitor it's behavior   Updated Link for access to this video:  Connecting Existing Things to ThingWorx Industrial Gateway for Anomaly Detection

/* Define a DataShape used in an InfoTable Parameter for this service call */ twDataShape* sampleInfoTableAsParameterDs = twDataShape_Create(twDataShapeEntry_Create("ColumnA",NO_DESCRIPTION,TW_STRING)); twDataShape_AddEntry(sampleInfoTableAsParameterDs,twDataShapeEntry_Create("ColumnB",NO_DESCRIPTION,TW_NUMBER)); twDataShape_AddEntry(sampleInfoTableAsParameterDs,twDataShapeEntry_Create("ColumnC",NO_DESCRIPTION,TW_BOOLEAN)); twDataShape_SetName(sampleInfoTableAsParameterDs,"SampleInfoTableAsParameterDataShape");      /* Define Input Parameter that is an InfoTable of Shape SampleInfoTableAsParameterDataShape */ twDataShapeEntry* infoTableDsEntry = twDataShapeEntry_Create("itParam",NULL,TW_INFOTABLE); twDataShapeEntry_AddAspect(infoTableDsEntry, "dataShape", twPrimitive_CreateFromString("SampleInfoTableAsParameterDataShape", TRUE));    twDataShape* inputParametersDefinitionDs = twDataShape_Create(infoTableDsEntry);   /* Register remote function */ twApi_RegisterService(TW_THING, SERVICE_INTEGRATION_THINGNAME, "testMultiRowInfotable", NO_DESCRIPTION,   inputParametersDefinitionDs, TW_NOTHING, NULL, PlatformCallsServiceWithMultiRowInfoTableServiceImpl, NULL); /* Note that you will have to manually create the datashape in ThingWorx before attempting to add this remote service to your Thing. */

Starting with the 7.4 version of Thingworx, a license.bin file locked to the specific version of Thingworx is required in order to successfully start the Thingworx webapp. If something is wrong with the licensing, Tomcat will crash and will not show any information regarding the problem in its log files. The Catalina*.log file will look like this 13-Jun-2017 04:36:43.268 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Catalina 13-Jun-2017 04:36:43.268 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.13 13-Jun-2017 04:36:43.315 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\Thingworx.war 13-Jun-2017 04:36:56.080 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. instead of continuing on through the rest of the war files present on the server as it would if everything worked properly. 13-Jun-2017 04:37:20.001 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\Thingworx.war has finished in 36,684 ms 13-Jun-2017 04:37:20.006 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\docs 13-Jun-2017 04:37:20.113 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\docs has finished in 107 ms 13-Jun-2017 04:37:20.113 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\examples 13-Jun-2017 04:37:20.617 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\examples has finished in 504 ms 13-Jun-2017 04:37:20.618 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\host-manager 13-Jun-2017 04:37:20.661 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\host-manager has finished in 43 ms 13-Jun-2017 04:37:20.661 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\manager 13-Jun-2017 04:37:20.847 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\manager has finished in 186 ms 13-Jun-2017 04:37:20.847 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\ROOT 13-Jun-2017 04:37:20.866 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory C:\PTC\KinexForManufacturing\PTC_Servlet_Engine\webapps\ROOT has finished in 18 ms 13-Jun-2017 04:37:20.949 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-jsse-nio-443"] 13-Jun-2017 04:37:20.957 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"] 13-Jun-2017 04:37:20.958 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 37733 ms The error will actually be in the ThingworxStorage/logs/ApplicationLog.log file - something like this: 2017-06-14 10:00:19.057-0700 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Subsystem LicensingSubsystem is starting 2017-06-14 10:00:19.057-0700 [L: INFO] [O: c.t.s.s.Subsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Starting Subsystem [LicensingSubsystem] 2017-06-14 10:00:19.088-0700 [L: ERROR] [O: c.t.s.s.l.LicensingSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] ==================================================================== 2017-06-14 10:00:19.088-0700 [L: ERROR] [O: c.t.s.s.l.LicensingSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] C:\PTC\KinexForManufacturing\ThingworxPlatform\license.bin: license file does not exist! 2017-06-14 10:00:19.088-0700 [L: ERROR] [O: c.t.s.s.l.LicensingSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] ==================================================================== 2017-06-14 10:00:19.088-0700 [L: WARN] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Shutting down the Platform. Get your license installed properly and the problem should go away.

In this blog I will be inspecting the setup below, using the REST APIs exposed by the different components (log-analysis-free guaranteed).   The components are started in stages, and I will do some exploration between each stages : EMS only EMS + LSR 1 EMS + LSR 1 + LSR 2    The REST APIs   Edge MicroServer (EMS) REST API This API is very similar to the ThingWorx platform REST API, see REST APIs Supported by WS EMS for specificity. I will be monitoring the EMS using the LocalEms virtual Thing (EMS only) ThingWorx Platform REST API This is the well known ThingWorx Core REST API -  see REST API Core Concepts I will be monitoring the EMS, from the platform, using an EMSGateway thing. The EMSGateway exposes on the platform some of the LocalEms services (like GetEdgeThings). I'm also inspecting the remote properties / services / events exposed on the things using the RemoteThing::GetRemoteMetadata service. Lua Script Resource (LSR) REST API This API largely differs from the ones above, the API documentation is served by the LSR itself (e.g. http://localhost:8001/)   I will use it to list the scripts loaded by the LSR process.  Configuration   See above diagram - Platform is listening on port 8084, no encryption, EMS and LSRs on the same host...   Platform configuration : for each edge thing, a corresponding remote thing was manually created on the platform EMSGateway1 as a EMSGateway Thing Template EMSOnlyThing as a RemoteThingWithFileTransfer LUAThing2 as a RemoteThing LUAThing1 as a RemoteThing LUAOnlyThing as a RemoteThing   EMS configuration : /etc/config.json - listening on default port 8000   {     "ws_servers": [{             "host": "tws74neo",             "port": 8084         }     ],     "appKey": "xxxxxx-5417-4248-bc01-yyyyyyy",     "logger": {         "level": "TRACE"     },     "ws_connection": {         "encryption": "none"     },     "auto_bind": [{             "name": "EMSGateway1",             "gateway": true         }, {             "name": "EMSOnlyThing",             "gateway": false         }, {             "name": "LUAThing2",             "host": "localhost",             "port": 8002,             "gateway": false         }, {             "name": "LUAThing1",             "gateway": false         }     ],     "file": {         "virtual_dirs": [{                 "emsrepository": "E:\\ptc\\ThingWorx\\EMS-5-3-2\\repositories\\data"             }         ],         "staging_dir": "E:\\ptc\\ThingWorx\\EMS-5-3-2\\repositories\\staging"     } } LSR process (1) : /etc/config.lua - listening on default port 8001 (using the out of the box sample Lua scripts)   scripts.log_level = "INFO"   scripts.LUAThing1 = {     file = "thing.lua",     template = "example", }   scripts.sample = {   file = "sample.lua" } LSR process (2) : /etc/config2.lua - listening on port 8002 (using the out of the box sample Lua scripts) This LSR process is started with command "luaScriptResource.exe -cfg .\etc\config2.lua"   scripts.log_level = "INFO" scripts.script_resource_port = 8002 scripts.LUAThing2 = {     file = "thing.lua",     template = "example", }   scripts.LUAOnlyThing = {     file = "thing.lua",     template = "example", }   Stage 1 : EMS only     ThingWorx REST API   Request: Call the GetEdgeThings service on the EMSGateway1 thing POST  twx74neo:8084/Thingworx/Things/EMSGateway1/Services/GetEdgeThings Response: As expected, only the remote things flagged as auto_bind are listed   name host port path keepalive timeout proto user accept EMSGateway1   8001.0 / 60000.0 30000.0 http   application/json EMSOnlyThing   8001.0 / 60000.0 30000.0 http   application/json LUAThing1   8001.0 / 60000.0 30000.0 http   application/json LUAThing2 localhost 8002.0 / 60000.0 30000.0 http   application/json                                                               Request: Call the GetRemoteMetadata service on the LUAThing1 thing POST  twx74neo:8084/Thingworx/Things/LUAThing1/Services/GetRemoteMetadata Response: As expected, remote properties / services and events are not available since the LSR associated with this thing is off.   Unable to Invoke Service GetRemoteMetadata on LUAThing1 : null   EMS REST API   Request: Call the GetEdgeThings service on the LocalEms virtual thing POST  localhost:8000/Thingworx/Things/LocalEms/Services/GetEdgeThings Response: output is identical to the gateway thing on the platform   { "name": "EMSGateway1", "host": "", "port": 8001, "path": "/",  "keepalive": 60000, "timeout": 30000, "proto": "http", "user": "", "accept": "application/json" }, { "name": "EMSOnlyThing", "host": "", "port": 8001, "path": "/", "keepalive": 60000, "timeout": 30000, "proto": "http", "user": "", "accept": "application/json" }, { "name": "LUAThing1", "host": "", "port": 8001, "path": "/", "keepalive": 60000, "timeout": 30000, "proto": "http", "user": "", "accept": "application/json" }, { "name": "LUAThing2", "host": "localhost", "port": 8002, "path": "/", "keepalive": 60000, "timeout": 30000, "proto": "http", "user": "", "accept": "application/json"}            LSR REST API N/A - no LSR process started yet.   Stage 2 : EMS + LSR 1 (8001)     ThingWorx REST API   Request: Call the GetEdgeThings service on the EMSGateway1 thing POST  twx74neo:8084/Thingworx/Things/EMSGateway1/Services/GetEdgeThings Response: LUAThing1 is associated to an LUA script   name host port path keepalive timeout proto user accept EMSGateway1   8001.0 / 60000.0 30000.0 http   application/json EMSOnlyThing   8001.0 / 60000.0 30000.0 http   application/json LUAThing1 localhost 8001.0 /scripts/Thingworx 60000.0 15000.0 http   application/json LUAThing2 localhost 8002.0 / 60000.0 30000.0 http   application/json   Request: Call the GetRemoteMetadata service on the LUAThing1 thing POST  twx74neo:8084/Thingworx/Things/LUAThing1/Services/GetRemoteMetadata Response: Now that the Lua script for LUAThing1 is running, remote properties / services and events are available   {"isSystemObject":false,"propertyDefinitions":{"Script_Pushed_Datetime":{"sourceType":"ThingShape","aspects":{"isReadOnly":false,"dataChangeThreshold":0,"defaultValue":1495619610000,"isPersistent":false,"pushThreshold":0,"dataChangeType":"VALUE","cacheTime":0,"pushType":"ALWAYS"},"name":"Script_Pushed_Datetime","description":"","category":"","tags":[],"baseType":"DATETIME","ordinal":0},"Pushed_InMemory_Boolean":{"sourceType":"ThingShape","aspects":....   EMS REST API      LocalEms::GetEdgeThings returns same output as EMSGateway::GetEdgeThings   LSR REST API (port 8001)   Request: List all the scripts running in the first LSR GET  localhost:8001/scripts?format=text/html Response: We find our sample script and the script associated with LUAThing1 (the Thingworx script is part of the infrastructure and always there)   Name Status Result File LUAThing1 Running   E:\ptc\ThingWorx\EMS-5-3-2\etc\thingworx\scripts\thing.lua sample Running   sample.lua Thingworx Running   E:\ptc\ThingWorx\EMS-5-3-2\etc\thingworx\scripts\thingworx.lu   Stage 3 : EMS + LSR 1 (8001) + LSR 2 (8002)     ThingWorx REST API   Request: Call the GetEdgeThings service on the EMSGateway1 thing POST  twx74neo:8084/Thingworx/Things/EMSGateway1/Services/GetEdgeThings Response: LUAOnlyThing is now listed and LUAThing2 is associated with a LUA script   name host port path keepalive timeout proto user accept EMSGateway1   8001.0 / 60000.0 30000.0 http   application/json EMSOnlyThing   8001.0 / 60000.0 30000.0 http   application/json LUAOnlyThing localhost 8002.0 /scripts/Thingworx 60000.0 15000.0 http   application/json LUAThing1 localhost 8001.0 /scripts/Thingworx 60000.0 15000.0 http   application/json LUAThing2 localhost 8002.0 /scripts/Thingworx 60000.0 15000.0 http   application/json   EMS REST API      LocalEms::GetEdgeThings returns same output as  EMSGateway::GetEdgeThings   LSR REST API (port 8002)   Request: List all the scripts running in the second LSR GET  localhost:8002/scripts?format=text/html Response: Returns the status of all the scripts currently loaded   Name Status Result File LUAOnlyThing Running   .\etc\thingworx\scripts\thing.lua LUAThing2 Running   .\etc\thingworx\scripts\thing.lua Thingworx Running   .\etc\thingworx\scripts\thingworx.lua

This zip file contains my slides and buildable project from my Liveworx 207 developer chat presentation featuring the Thing-e Robot and raspberry pi zero integration with ThingWorx using the C SDK.

It usually happens that we need to copy a large file to ThingWorx server periodically, and what's worse, the big file is changing(like a log file). This sample give a simpler way to implement. The main idea in the sample is: 1. Lower the management burden from ThingWorx server and instead it put all the work in edge SDK side 2. Save network burden with only uploading the incremented file and append it to the older file on ThingWorx server   Java SDK version in this sample: 6.0.1-255

Disclaimer: example was provided by Hatcher Chad - chad@onfarmsystems.com   //   // For this example, we'll have an Math service   // which takes two numbers, and an operation.   // The result will be that operation performed on the two inputs.       //   // We either need an Application Key,   // or user credentials to perform the reads and writes.   // App keys are a little safer.   // In this demo, we'll store it on the Entity as a Property.   var appKey = me.appKey;       //   // The service name needs to be unique and not already in use.   var serviceName = "MyMath";       //   // What are the inputs to the service?   // We'll define them nicely here, but manipulate this object later.   var parameters = {   "op" : "STRING",   "x" : "NUMBER",   "y" : "NUMBER"   };       //   // What datatype does the service return?   // If it's an infotable,   // then you'll also have to specify the data shape   // as part of the resultType's aspect,   // but I won't demonstrate that here.   var output = "NUMBER";       //   // What is the actual service script?   // We'll define it here as an array of lines, and then join them together.   var serviceScript = [   "var result = (function() {",   " switch(op) {",   " case \"add\": return x + y;",   " case \"sub\": return x - y;",   " case \"mult\": return x * y;",   " case \"div\": return x / y;",   " default: return op in Math ? Math[op](x, y) : 0;",   " };",   "})();",   ].join("\n");       ////////       //   // Let's convert the friendly parameter definition   // into the structure that ThingWorx uses:   var parameterDefinitions = Object.keys(parameters).reduce(function(parameterDefinitions, parameterName, index) {   var parameterType = parameters[parameterName];   parameterDefinitions[parameterName] = {   "name": parameterName,   "aspects": {},   "description": "",   "baseType": parameterType,   "ordinal": index   };   return parameterDefinitions;   }, {});       //   // Now let's set up our service definition and implementation.   var definition = {   "isAllowOverride": false,   "isOpen": false,   "sourceType": "Unknown",   "parameterDefinitions": parameterDefinitions,   "name": serviceName,   "aspects": {   "isAsync": false   },   "isLocalOnly": false,   "description": "",   "isPrivate": false,   "sourceName": "",   "category": "",   "resultType": {   "name": "result",   "aspects": {},   "description": "",   "baseType": output,   "ordinal": 0   }   };       var implementation = {   "name": serviceName,   "description": "",   "handlerName": "Script",   "configurationTables": {   "Script": {   "isMultiRow": false,   "name": "Script",   "description": "Script",   "rows": [{   "code": serviceScript   }],   "ordinal": 0,   "dataShape": {   "fieldDefinitions": {   "code": {   "name": "code",   "aspects": {},   "description": "code",   "baseType": "STRING",   "ordinal": 0   }   }   }   }   }   };       ////////       //   // Here are the URLs we'll need in order to make updates.   // You can change the thing name ('ServiceModifier' here)   // to something else.   // If you use credentials instead of an app key,   // then you can remove the appKey parameter here,   // but you'll have to add the username and password   // to the two ContentLoaderFunctions calls.   var url = {   export : "http://127.0.0.1:8080/Thingworx/Things/ServiceModifier?Accept=application/json&appKey="+appKey,   import : "http://127.0.0.1:8080/Thingworx/Things/ServiceModifier?appKey="+appKey   };       //   // We can download the entity to modify as a JSON object.   // Older versions of ThingWorx might not support this.   var config = Resources.ContentLoaderFunctions.GetJSON({   url : url.export,   });       //   // We have to modify both the 'effectiveShape',   // as well as the 'thingShape'.   config.effectiveShape.serviceDefinitions[serviceName] = definition;   config.effectiveShape.serviceImplementations[serviceName] = implementation;       config.thingShape.serviceDefinitions[serviceName] = definition;   config.thingShape.serviceImplementations[serviceName] = implementation;       // Finally, we can push our updates back into ThingWorx.   Resources.ContentLoaderFunctions.PutText({   url : url.export,   content : JSON.stringify(config),   contentType : "application/json",   });       // The end.

Hello! I have just written a tutorial on how to set up Lua to be run from the command line. As many already know, there is no good way to debug Lua scripts as they are used in package deployment in Software Content Manager, and building such a debugger is a vast and difficult undertaking. As an alternative, running small portions of code in the command line to ensure they will work as expected is one way to verify the validity of the Lua syntax prior to attempting a deployment. Here are the steps to set up Lua as a command line tool:   Grab the GCC compiler called TDM-GCC Run the exe file and follow the install instructions Remember the install directory for this, as the attached install script will need to be configured in a later step Default location in the install file is "C:\Program Files\TDM-GCC\" Note: leaving the "Add to PATH" selected will allow you to compile C code on the command line as well by typing "gcc" (this is not required for this set-up) Download the Lua source code ​This comes as a ".tar.gz" file, which can be tricky to extract in Windows Download 7zip for freeware which can unzip this type of archive Extract the Lua source code and navigate to the top level directory which should just contain one folder named like "lua-x.x.x" where the x's refer to the version Download and extract the attached zip file containing the build file Copy the "build.cmd" file from this to the top level directory of the Lua source code Modify the configuration as needed: Version number default is 5.3.4 (parameter is called lua_version) GCC install path default is "C:\Program Files\TDM-GCC\bin" (parameter is called lua_build_dir) Double click the "build.cmd" file A console window will appear with installation details If you see the following, then it worked successfully: The "lua\" directory will be created in the same folder as the "build.cmd" file Copy the "lua\" directory to "C:\Program Files\" Open "Computer" > "System Properties" > "Advanced system settings" Click "Environment Variables" > "New..." Call the variable "LUA" and make the value "C:\Program Files\lua\bin" Find the "Path" variable and click "Edit..." At the end of what is already there (do NOT delete anything that is already there), add "%LUA%" (make sure there is a ";" between the previous path and this entry) Click "Ok" Open a new command prompt (has to be new to load the new path) and type "lua" to see if it works Example syntax test from Lua command line:   I hope this is helpful to people! Let me know if you have any questions!

1. Create a network and added all Entities that implement from a specific ThingShape in the network 2. Create a ThingShape mashup as below Note: Bind the Entity parameter to DynamicThingShapes_TracotrShape's service GetProperties input EntityName. Laso bind mashup RefreshRequested event to that service 3. Create a mashup named ContentShape, add Tree widget and ContainedMashp in it 4. Bind Service GetNetworkConnection's Selected Row(s) result and Selected RowsChanged event to ContainedMashup widget Note: Master can total replace ThingShape mashup. Suggest to use Master after ThingWorx 6.0

Embedded databases come with the installation of the ThingWorx Platform No additional installation or configuration is required for embedded databases Read about the various benefits and pitfalls of embedded versus external below Database Options H2 RDBMS (relational database management system), written in Java Has a small memory footprint Embedded into ThingWorx for easy installation Not as robust as other database options Not scalable in production environments (unless used alongside a separate, external database for stream, value stream, and other data) ​ See KCS Article CS243975 for further reading on the use of external databases Meant to be used for quick deployments and testing environments PostgreSQL ORDBMS (object-relational database management system), written in C PostgreSQL is the ThingWorx recommended database for production systems More Robust External database installed separately from ThingWorx Beneficial because external databases can be specifically configured for use in production, while embedded databases cannot Able to efficiently handle larger amounts of data and store more data without affecting ThingWorx system performance Greater Stability Recover from data corruptions more easily by accessing the database from an external application (separate from ThingWorx) using simple SQL statements Easier to back-up the database in case of issues (further reading in KCS Article CS246598) Less risky and simpler upgrade procedure, which occurs "in-place" Instead of exporting and importing data and entities, a simple schema update allows these to automatically persist into the new version If ThingworxStorage folder is accidentally deleted, entities and data are secure in the external database More Secure HA (High Availability) allows for multiple server instances at different locations in the network Assists in time of failover, i.e. if one server fails, the other can immediately take over Secures the data and prevents further data loss in the event of a failure Customizable security settings and complex password requirements Fewer security vulnerabilities than other databases Because Postgres is an external database, it can be harder to install Follow the steps in the installation guide closely See KCS Articles CS235937 and CS230085 for troubleshooting and help with installation and configuration Hana RDBMS (relational database management system) In-memory, column based data storage For more information on this database, please see the Getting Started with SAP HANA Guide Neo4J GDBMS (graph database management system), written in Java Data is not easily accessed by external applications, and CQL must be used instead of SQL, making recovery from corruptions very difficult Embedded database with limited configuration options Known to have issues with deadlocks Deprecated in version 7.0 (related KCS Article: CS228537) For full installation steps for H2 and PostgreSQL, see the ThingWorx Installation Guide