IoT Tips

Performance and memory issues in ThingWorx often appear at the Java Virtual Machine (JVM) level. We can frequently detect these issues by monitoring JVM garbage collection logs and thread dumps. In this first of a multi-part series, let's discuss how to analyze JVM garbage collection (GC) logs to detect memory issues in our application. What are GC logs? When enabled on the Apache server, the logs will show the memory usage over time and any memory allocation issues (See: How to enable GC logging in our KB for details on enabling this logging level).  Garbage Collection logs capture all memory cleanup operations performed by the JVM.  We can determine the proper memory configuration and whether we have memory issues by analyzing GC logs. Where do I find the GC logs? When configured as per our KB article, GC logs will be written to your Apache logs folder. Check both the gc.out and the gc.restart files for issues if present, especially if the server was restarted when experiencing problems. How do I read GC logs? There are a number of free tools for GC log analysis (e.g. http://gceasy.io, GCViewer). But GC logs can also be analyzed manually in a text editor for common issues. Reading GC files: Generally each memory cleanup operation is printed like this: Additional information on specific GC operations you might see are available from Oracle. A GC analysis tool will convert the text representation of the cleanups into a more easily readable graph: Three key scenarios indicating memory issues: Let's look at some common scenarios that might indicate a memory issue.  A case should be opened with Technical Support if any of the following are observed: Extremely long Full GC operations (30 seconds+) Full GCs occurring in a loop Memory leaks Full GCs: Full GCs are undesirable as they are a ‘stop-the-world’ operation. The longer the JVM pause, the more impact end users will see: All other active threads are stopped while the JVM attempts to make more memory available Full GC take considerable time (sometimes minutes) during which the application will appear unresponsive Example – This full GC takes 46 seconds during which time all other user activity would be stopped: 272646.952: [Full GC: 7683158K->4864182K(8482304K) 46.204661 secs] Full GC Loop: Full GCs occurring in quick succession are referred to as a Full GC loop If the JVM is unable to clean up any additional memory, the loop will potentially go on indefinitely ThingWorx will be unavailable to users while the loop continues Example – four consecutive garbage collections take nearly two minutes to resolve: 16121.092: [Full GC: 7341688K->4367406K(8482304K), 38.774491 secs] 16160.11: [Full GC: 4399944K->4350426K(8482304K), 24.273553 secs] 16184.461: [Full GC: 4350427K->4350426K(8482304K), 23.465647 secs] 16207.996: [Full GC: 4350431K->4350427K(8482304K), 21.933158 secs]      Example – the red triangles occurring at the end of the graph are a visual indication that we're in a full GC loop: Memory Leaks: Memory leaks occur in the application when an increasing number of objects are retained in memory and cannot be cleaned up, regardless of the type of garbage collection the JVM performs. When mapping the memory consumption over time, a memory leaks appears as ever-increasing memory usage that’s never reclaimed The server eventually runs out of memory, regardless of the upper bounds set           Example(memory is increasing steadily despite GCs until we'll eventually max out): What should we do if we see these issues occurring? Full GC loops and memory leaks require a heap dump to identify the root cause with high confidence The heap dump needs to be collected when the server is in a bad state Thread dumps should be collected at the same time A case should be opened with support to investigate available gc, stacktrace, or heap dump files

1. Add an Json parameter Example: { ​    "rows":[         {             "email":"example1@ptc.com"         },         {             "name":"Qaqa",             "email":"example2@ptc.com"         }     ] } 2. Create an Infotable with a DataShape usingCreateInfoTableFromDataShape(params) 3. Using a for loop, iterate through each Json object and add it to the Infotable usingInfoTableName.AddRow(YourRowObjectHere) Example: var params = {     infoTableName: "InfoTable",     dataShapeName : "jsontest" }; var infotabletest = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); for(var i=0; i<json.rows.length; i++) {     infotabletest.AddRow({name:json.rows.name,email:json.rows.email}); }

Want to do a REST call from ThingWorx Want to use REST to send request to External System. Want to get data from other system using REST Here is how you can do this.... ThingWorx has ContentLoaderFunctions API which provides services to load or post content to and from other web applications. One can issue an HTTP request using any of the allowed actions (GET, POST, PUT, DELETE). List of available ContentLoaderFunctions: Delete GetCookies GetJSON GetText GetXML LoadBinary LoadImage LoadJSON LoadMediaEntity LoadText LoadXML PostBinary PostImage PostJSON PostMultipart PostText PostXML PutBinary PutJSON PutText PutXML Example: Using LoadXML snippet in a custom service to retrieve an XML document from a specific URL Insert the LoadXML snippet into a custom service var params = {     proxyScheme: undefined /* STRING */,     headers: "{ 'header1':'value1','header2':'value2'}" /* JSON */,     ignoreSSLErrors: false /* BOOLEAN */,     useProxy: undefined /* BOOLEAN */,     proxyHost: undefined /* STRING */,       url: "http://some_url/sampleXMLDocument.xml" /* STRING */,     timeout: 30000 /* NUMBER */,     proxyPort: undefined /* INTEGER */,     password: "fakePassword" /* STRING */,     username: "Administrator"/* STRING */ }; var result = Resources["ContentLoaderFunctions"].LoadXML(params); The snippet above contains an example of how to format any headers in JSON that need to be passed in, the URL that points directly to some XML document, a password, username, timeout, and ignoreSSLErrors set to false When LoadXML is exercised it will retrieve the XML document, and this can then be parsed or handled however is necessary To see the XML document that is returned from this service the service can be called from a third-party client, such as Postman Note: If a proxy or username and password are required to connect to the URL, those parameter MUST be specified Using the PostXML snippet in a custom service to send content to another URL, in this example, another service in Composer Insert the PostXML snippet into a custom service var content = "<xml><tag1>NAME</tag1><tag2>AGE</tag2></xml>"; var params = {  url: "http://localhost/Thingworx/Things/thingName/Services/serviceName?postParameter=parameterName" /* STRING */, content: content /* STRING */, password: "admin" /* STRING */, username: "Administrator" /* STRING */ }; var result = Resources["ContentLoaderFunctions"].PostXML(params); When posting XML content to another ThingWorx service the postParameter header must be defined in the url parameter for the PostXML snippet  The postParameter header, in the url parameter, is set equal to the name of the input parameter for the service we are POSTing to Change the parameterName variable in the url to the name of the input parameter defined for the service The content parameter is set to the XML content that will be passed into the function or manually specified Note: When declaring namespace URLs in an element make sure that there is a white space in between each declaration ex: <root xmlns:h="http://www.w3.org/TR/html4/" xmlns:f="http://www.w3schools.com/furniture">

Timers and schedulers can be useful tool in a Thingworx application.  Their only purpose, of course, is to create events that can be used by the platform to perform and number of tasks.  These can range from, requesting data from an edge device, to doing calculations for alerts, to running archive functions for data.  Sounds like a simple enough process.  Then why do most platform performance issues seem to come from these two simple templates? It all has to do with how the event is subscribed to and how the platform needs to process events and subscriptions.  The tasks of handling MOST events and their related subscription logic is in the EventProcessingSubsystem.  You can see the metrics of this via the Monitoring -> Subsystems menu in Composer.  This will show you how many events have been processed and how many events are waiting in queue to be processed, along with some other settings.  You can often identify issues with Timers and Schedulers here, you will see the number of queued events climb and the number of processed events stagnate. But why!?  Shouldn't this multi-threaded processing take care of all of that.  Most times it can easily do this but when you suddenly flood it with transaction all trying to access the same resources and the same time it can grind to a halt. This typically occurs when you create a timer/scheduler and subscribe to it's event at a template level.  To illustrate this lets look at an example of what might occur.  In this scenario let's imagine we have 1,000 edge devices that we must pull data from.  We only need to get this information every 5 minutes.  When we retrieve it we must lookup some data mapping from a DataTable and store the data in a Stream.  At the 5 minute interval the timer fires it's event.  Suddenly all at once the EventProcessingSubsystem get 1000 events.  This by itself is not a problem, but it will concurrently try to process as many as it can to be efficient.  So we now have multiple transactions all trying to query a single DataTable all at once.  In order to read this table the database (no matter which back end persistence provider) will lock parts or all of the table (depending on the query).  As you can probably guess things begin to slow down because each transaction has the lock while many others are trying to acquire one.  This happens over and over until all 1,000 transactions are complete.  In the mean time we are also doing other commands in the subscription and writing Stream entries to the same database inside the same transactions.  Additionally remember all of these transactions and data they access must be held in memory while they are running.  You also will see a memory spike and depending on resource can run into a problem here as well. Regular events can easily be part of any use case, so how would that work!  The trick to know here comes in two parts.  First, any event a Thing raises can be subscribed to on that same Thing.  When you do this the subscription transaction does not go into the EventProcessingSubsystem.  It will execute on the threads already open in memory for that Thing.  So subscribing to a timer event on the Timer Thing that raised the event will not flood the subsystem. In the previous example, how would you go about polling all of these Things.  Simple, you take the exact logic you would have executed on the template subscription and move it to the timer subscription.  To keep the context of the Thing, use the GetImplimentingThings service for the template to retrieve the list of all 1,000 Things created based on it.  Then loop through these things and execute the logic.  This also means that all of the DataTable queries and logic will be executed sequentially so the database locking issue goes away as well.  Memory issues decrease also because the allocated memory for the quries is either reused or can be clean during garbage collection since the use of the variable that held the result is reallocated on each loop. Overall it is best not to use Timers and Schedulers whenever possible.  Use data triggered events, UI interactions or Rest API calls to initiate transactions whenever possible.  It lowers the overall risk of flooding the system with recourse demands, from processor, to memory, to threads, to database.  Sometimes, though, they are needed.  Follow the basic guides in logic here and things should run smoothly!

This Best Practices document should offer some guidelines and tips & tricks on how to work with Timers and Schedulers in ThingWorx. After exploring the configuration and creation of Timers and Schedulers via the UI or JavaScript Services, this document will also highlight some of the most common performance issues and troubleshooting techniques.   Timers and Schedulers can be used to run jobs or fire events on a regular basis. Both are implemented as Thing Templates in ThingWorx. New Timer and Scheduler Things can be created based on these Templates to introduce time based actions. Timers can be used to fire events in a certain interval, defined in the Timer's Update Rate (default is 60000 milliseconds = 1 minute). Schedulers can be used to run jobs based on a cron pattern (such as once a day or once an hour). Schedulers will also allow for a more detailed time based setup, e.g. based on seconds, hours, days of week or days months etc. Events fired by both Timers and Schedulers can be subscribed to with Subscriptions which can be utilized to execute custom service scripts, e.g. to generate "fake" or random demo data to update Remote Things in a test environment. In general subscriptions and scripts can be used to e.g. run regular maintenance tasks or periodically required functions (e.g. for data aggregation) For more information about setting up Timers and Schedulers it's recommended to also have a look at the following content:   How to set up and configure Timers How to set up and configure Schedulers How to create and configure Timers and Schedulers via JavaScript Services Events and Subscriptions for Timers and Schedulers   Example   The following example will illustrate on how to create a Timer Thing updating a Remote Thing using random values. To avoid any conflicts with permissions and visibility, use the Administrator user to create Things.   Remote Thing   Create a new Thing based on the Remote Thing Template, called myRemoteThing. Add two properties, numberA and numberB - both Integers and marked as persistent. Save myRemoteThing. Timer Thing   Create a new Thing based on the Timer Template, called myTimerThing. In the Configuration, change the Update Rate to 5000, to fire the Event every 5 seconds. User Context to Administrator. This will run the related services with the Administrator's user visibility and permissions. Save myTimerThing. Subscriptions   To update the myRemoteThing properties when the Timer Event fires, there are two options: Configure a Subscription on myRemoteThing and listen to Timer Events on the myTimerThing. Configure a Subscription on myTimerThing and listen to Timer Events on itself as a source. In this example, let's go with the first option and Edit myRemoteThing. Create a new Subscription pointing to myTimerThing as a Source. Select the Timer Event Note that if no source is selected, the Timer Event is not availabe, as myRemoteThing is based on the Remote Thing Template and not the Timer Template Enable the Subscription. In the Script area use the following code to assign two random numbers to the Thing's custom properties: me.numberA = Math.floor(Math.random() * 100); me.numberB = Math.floor(Math.random() * 100); Save myRemoteThing. Validation   The Subscription will be enabled and active on saving it. Switch to the myRemoteThing Properties Refreshing the Values will show updates with random numbers between 0 and 99 every 5 seconds (Timer Update Rate).   Performance considerations   Timers and Schedulers are handled via the Event Processing Subsystem. Metrics that impact current performance can be seen in Monitoring > Subsystems > Event Processing Implementing Timers and Schedulers on a Thing Template level might flood the system with services executions originating from Subscriptions to Timer / Scheduler triggered Events. Subscribing to another Thing's Events will be handled via the Event Processing Subsystem. Subscribing to an Event on the same Thing will not be handled via the Event Processing Subsystem, but rather execute on the already open in memory Thing. If Timers and Schedulers are not necessarily needed, the Services can be triggered e.g. via Data Change Events, UI Interactions etc. Recursion can be a hidden performance contributer where a Subscription to a certain Event executes a service, triggering another Event with recursive dependencies. Ensure there are no circular dependencies and service calls across Entities. If possible, reads for each and every action from disk should be avoided. Performance can be increased by storing relevant information in memory and using Streams or Datatables or for persistence. If possible, call other Services from within the Subscription instead of handling all code within the Subscription itself. For full details, see also Timers and Schedulers - Best Practice   How to identify and troubleshoot technical issues   Check the Event Processing Subsystem for any spikes in queued Events (tasks submitted) while the total number of tasks completed is not or only slowly increasing. For a historical overview, search the ApplicationLog for "Thingworx System Metrics" to get system metrics since the server has been (re-) started. In the ApplicationLog the message "Subsystem EventProcessingSubsystem is started" indicates that the Subsystem is indeed started and available. Use custom loggers in Services to get more context around errors and execution in the ScriptLog Custom Loggers can be used to identify if Events have fired and Subscriptions are actually triggered Example: logger.debug("myThing: executing subscribed service") For issues with Service execution, see also CS268218 Infinite loops in Services could render the server unresponsive and might flood the system with various Events To change the timing for a Timer, restarting the Thing is not enough. The Timer must be disabled and enabled at the desired start time. Schedulers will allow for a much more flexible timing and setting / changing execution times in advance. For further analysis it's recommended to generate Thread Dumps to get more information about the current state of Threads in the JVM. The ThingWorx Support Tools Extension can help in generating those. See also CS245547 for more information and usage.

Sometimes the following error is seen filling up the application log: "HTTP header: referrer". It does not cause noticeable issues, but does fill up the log. This article has been updated to reflect the workaround: https://www.ptc.com/en/support/article?n=CS223714 To clear the error, locate the validation.properties ​inside the ThingworxStorage\esapi directory. Then  change the values of both Validator.HTTPHeaderValue_cookie and Validator.HTTPHeaderValue_referer to: Validator.HTTPHeaderValue_cookie= ^.*$ Validator.HTTPHeaderValue_referer= ^.*$

This is a basic troubleshooting guide for ThingWorx. It goes over the importance, types and levels of logs, getting started on troubleshooting the Composer, Mashup and Remote Connectivity.     For full-sized viewing, click on the YouTube link in the player controls.   Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

Preface   In this blog post, we will discuss how to Start and Stop ThingWorx Analytics, as well as some other useful triaging/troubleshooting commands. This applies to all flavors of the native Linux installation of the Application.   In order to perform these steps, you will have to have sudo or ROOT access on the host machine; as you will have to execute a shell script and be able to view the outputs.   The example screenshots below were taken on a virtual CentOS 7 Server with a GUI as ROOT user.     Checking ThingWorx Analytics Server Application Status   1. Change directory to the installation destination of the ThingWorx Analytics (TWA) Application. In the screenshot below, the application is installed to the /opt/ThingWorxAnalyticsServer directory   2. In the install directory, there are a series of folders and files. You can use the ​ls​ command to see a list of files and folders in the installation directory.     a. You will need to go navigate one more level down into the ./ThingWorxAnalyticsServer/bin​ ​directory by using command ​cd ./bin​     b. As you can see above, we used the ​pwd​ command to verify that we are in the correct directory.   3. In the ./ThingWorxAnalyticsServer/bin directory, there should be three shell files: configure-apirouter.sh, configure-user.sh, and twas.sh     a. To run a status check of the application, use the command ./twas.sh status           i. This will provide a list of outputs, and a few warning messages. This is normal, see screenshot below:      b. You will have a series of services, which will have a green active (running) or red not active (stopped).           i. List of services: twas-results-ms.service - ThingWorx Analytics - Results Microservice twas-data-ms.service - ThingWorx Analytics - Data Microservice twas-analytics-ms.service - ThingWorx Analytics - Analytics Microservice twas-profiling-ms.service - ThingWorx Analytics - Profiling Microservice twas-clustering-ms.service - ThingWorx Analytics - Clustering Microservice twas-prediction-ms.service - ThingWorx Analytics - PredictionMicroservice twas-training-ms.service - ThingWorx Analytics - Training Microservice twas-validation-ms.service - ThingWorx Analytics - Validation Microservice twas-apirouter.service - ThingWorx Analytics - API Router twas-edge-ms.service - ThingWorx Analytics - Edge Microservice   Starting and Stopping ThingWorx Analytics   If you encounter any errors or stopped services in the above, a good solution would be to restart the TWA Server application.   There are two methods to restart the application, one being the restart ​command, the other would be using the stop​ and ​start​ commands.   Method 1 - Restart Command:   1. In the same ./ThingWorxAnalyticsServer/bin​ ​directory, run the following command: ./twas.sh restart     a. The output of a successful restart will look like the following: 2. The restart should only take a few seconds to complete   Method 2 - Stop / Start Commands:   1. In the same ./ThingWorxAnalyticsServer/bin​ ​directory, run the following command: ./twas.sh stop 2. After the application stops, run the following command: ./twas.sh start   Note: You can confirm the status of the TWA Server application by following the steps in the "Checking ThingWorx Analytics Server Application Status" section above.

Licensing summary, installing 8.1: - Now instance specific license used which prevents license sharing and protects our intellectual property further - 2 paths for getting up and running with a license:          -Connected: platform-settings configuration only          -Disconnected: text file generated, self-serve creation from PTC support portal, license_capability_response.bin generated to be placed in platform folder   Connected / Disconnected mode LicensingSubsystem::GetLicenseState returns : UNINITIALIZED in disconnected mode LICENSE_EXISTS in connected mode   User Journey Refer to Licensing ThingWorx 8.1 and Later for specific instructions for generating a license for your instance   NOTE: Each instance needs a valid license file to be running.   Troubleshooting: - If any misconfiguration or connection failures occur, error messages will be thrown to Application log Example: unable to fetch license file with device id; Unable to connect to the PTC license server. Please make sure the LicensingConnectionSettings settings... - Connection attempt will occur upon ThingWorx startup. If connection can't be made, instance will be following disconnected scenario  -Make sure Pop Up Blockers are turned off     Platform Settings - Licensing License Connection Settings (in platform-settings.json - plain text sample) -new section for licensing connection strings: -user name - used to connect to ptc support portal -password - encrypted (recommended) or plain text password used to connect to ptc support portal   "LicensingConnectionSettings": { "username":"<PTC_support_portal_username>", "password":"<PTC_support_portal_password>" }    New Services on Licensing Subsystem - GetInstanceID - returns instance/device ID which is created at startup - WriteLicenseUsageData - writes encrypted license usage (same as system data table) to ThingworxStorage\reports\LicenseUsageReport folder   Instance ID The license file is now bound to the platform Instance ID aka Device ID - (unlike most other PTC products where the license is bound to the hardware : CPUID, MAC, ...) This Instance ID is generated during first startup and stored in the database Instance ID is accessible in composer with LicensingSubsystem::GetInstanceID   Q: What happens when license files are bad or missing? A: If there is an invalid license file, with 8.0 valid license.bin needs to be in the folder before starting up; tomcat will crash. In 8.1 no need for license file as long as connected (platform-settings.json), no need to take an extra step, dynamic connection.  In disconnected scenario, ThingWorx will run for 30 days in limited mode with monitoring mashup accessible to check logs.   Q: Where is the InstanceID stored ? A: It's stored in the database   Q: Will the instanceID change during updates (minor 8.1.0 to 8.1.1) A: Device id's don't change.   Q: What will happen in disconnected scenario if there is no valid license after 30 days? The application will not start anymore or user is not able to login? A:  It shuts down, so there is no more "limited" mode. However, a user can come along on day 55 (for example) and can drop in a valid license and start the web app to get it fully running.   Q: What happens if the customer has to reinstall the platform after the license has been fetched ? Is it possible to "return" the license ? A: Reinstalling the platform results in the generation of a new Device ID, therefore a new license file will need to be generated.  

How to enable ThingWorx Performance Advisor Applies To ThingWorx 7.2+ Description How to enable ThingWorx Performance Advisor Resolution In the ThingWorx Composer, go to Systems > Subsystems, select the PlatformSubsystem and choose Configuration In the Metrics Reporting Service Configuration Select the "Enable Metrics Reporting" checkbox to activate Performance Advisor reporting Enter your current PTC credentials (username and password) for either the customer support portal or the developer portal After providing those details, use the Request button to request an Authorization Key. Customer Number and Name will be filled automatically and an Authorization Key is generated which allows the server identifying itself to the PTC environment. Those fields are read-only. ThingWorx is now ready to send Performance Advisor data and metrics to PTC Related FAQ - Performance Advisor for ThingWorx | PTC https://community.thingworx.com/community/developers/blog/2017/05/22/performance-advisor-for-thingworx-explore-configure…

A common issue that is seen when trying to deploy, design or scale up a ThingWorx application is performance.  Slow response, delayed data and the application stopping have all been seen when a performance problems either slowly grows or suddenly pops up.  There are some common themes that are seen when these occur typically around application model or design.  Here are a few of the common problems and some thoughts on what to do about them or how to avoid them. Service Execution This covers a wide range of possibilities and is most commonly seen when trying to scale an application.  Data access within a loop is one particular thing to avoid.  Accessing data from a Thing, other service or query may be fast when only testing it on 100 loops, but when the application grows and you have 1000 suddenly it's slow.  Access all data in one query and use that as an in memory reference.  Writing data to a data store (Stream, Datatable or ValueStream) then querying that same data in one service can cause problems as well.  Run the query first then use all the data you have in the service variables.   To troubleshoot service executions there are a few methods that can be used.  Some for will not be practical for a production system since it is not always advisable to change code without testing first. Used browser development tools to see the execution time of a service.  This is especially helpful when a mashup is slow to load or respond.  It will allow quickly identifying which of multiple services may be the issue. Addition of logging in a service.  Once a service is identified adding simple logging points in the service can narrow what code in the service cases the slow down (it may be another service call).  These logging statements show up in the script logs with time stamps ( you can also log the current time with the logging statements). Use the test button in Composer.  This is a simple on but if the service does not have many parameters (or has defaults) it's a fast and easy way to see how long a service takes to return,' When all else fails you can get thread dumps from the JVM.  ThingWorx Support created an extension that assists with this.  You can find it on the Marketplace with instructions on how to use it.  You can manually examine the output files or open a ticket with support to allow them to assist.  Just be careful of doing memory dumps, there are much larger, hard to analyse and take a lot of memory.  https://marketplace.thingworx.com/tools/thingworx-support-tools Queries ​These of course are services too but a specific type.  Accessing data in ThingWorx storage structures or from external sources seems fairly straight forward but can be tricky when dealing with large data sets.  When designing and dealing with internal platform storage refer to this guide as a baseline to decide where to store data...  Where Should I Store My Thingworx Data?   NEVER store historical data in infotable properties.  These are held in memory (even if they are persistent) and as they grow so will the JVM memory use until the application runs out of it.  We all know what happens then.  Finally one other note that has causes occasional confusion.  The setting on a query service or standard ThingWorx query service that limits the number of records returned.  This is how many records are returned to from the service at the end of processing, not how many are processed or loaded in memory.  That number may be much higher and could cause the same types of issues. Subscriptions and Events ​This is similar to service however there is an added element frequency.  Typical events are data change and timers/schedulers.  This again is often an issue only when scaling up the number of Things or amount of data that need to be referenced.  A general reference on timers and schedulers can be found here.  This also describes some of the event processing that takes place on the platform.  Timers and Schedulers - Best Practice For data change events be very cautions about adding these to very rapidly changing property values.  When a property is updating very quickly, for example two times each second, the subscription to that event must be able to complete in under 0.5 seconds to stay ahead of processing.  Again this may work for 5-10 Things with properties but will not work with 500 due to resources, speed and need to briefly lock the value to get an accurate current read.  In these cases any data processing should be done at the edge when possible (or in the originating system) and pushed to the platform in a separate property or service call.  This allows for more parallel processing since it is de-centralized. A good practice for allowing easier testing of these types of subscription code is to take all of the script/logic and move it to a service call.  Then pass any of the needed event data to parameters in the service.  This allows for easier debug since the event does not need to fire to make the logic execute.  In fact it can essentially be stand alone by the test button in Composer. Mashup Performance This​ one can be very tricky since additional browser elements and rendering can come into play. Sometimes service execution is the root of the issue and reviewed above, other times it is UI elements and design that cause slow down. The Repeater widget is a common culprit. The biggest thing to note here is that each repeater will need to render every element that is repeated and all of the data and formatting for each of those widgets in the repeated mashup. So any complex mashup that is repeated many times may become slow to load. You can minimize this to a degree based on the Load/Unload setting of the widget and when the slowness is more acceptable (when loading or when scrolling). When a mashup is launched from Composer it comes with some debugging tools built in to see errors and execution. Using these with browser debug tools can be very helpful. Scaling an Application When initially modeling an application scale must be considered from the start. It is a challenge (but not impossible) to modify an application after deployment or design to be very efficient. Many times new developers on the ThingWorx platform fall into what I call the .Net trap. Back when .Net was released one of the quote I recall hearing about it's inefficiencies was "memory is cheap". It was more cost efficient to purchase and install more memory than to take extra development time to optimize memory use. This was absolutely true for installed applications where all of the code was complied and stored on every system. Web based applications are not quite a forgiving since most processing and execution is done on the single central web server. Keep this in mind especially when creating Shapes, Templates and Subscriptions. While you may be writing one piece of code when this code is repeated on 1,000 Things they will all be in memory and all be executing this code in parallel. You can quickly see how competition for resources, locks on databases and clean access to in memory structures can slow everything down (and just think when there are 10,000 pieces of that same code!!). Two specific things around this must be stated again (though they were covered in the above sections). Data held in properties has fast access since it is in JVM memory. But this is held in memory for each individual Thing, so hold 5 MB of information in one Thing seems small, loading 10,000 Thing mean instant use of 50 GB of memory!! Next execution of a service. When 10 things are running a service execution takes 2 seconds. Slow but not too bad and may not be too noticeable in the UI. Now 10,000 Things competing for the same data structure and resources. I have seen execution time jump to 2 minutes or more. Aside from design the best thing you can do is TEST on a scaled up structure. If you will have 1,000 Things next year test your application early at that level of deployment to help identify any potential bottlenecks early. Never assume more memory will alleviate the issue. Also do NOT test scale on your development system. This introduces edits changes and other variables which can affect actual real world results. Have a QA system setup that mirrors a production environment and simulate data and execution load. Additional suggestions are welcome in comments and will likely update this as additional tool and platform updates change.

This blog is intended to help diagnose and fix the most common issues that may be encountered when working with ThingWatcher. It cannot be stressed strongly enough that you should be familiar with your data including the average time interval between data points, and the collection duration and certainty threshold you specified. Before you start troubleshooting ThingWatcher, check that result and training microservices is running. For testing result microservices open a web browser and paste result URL; http://<IP of microservices>:<Port of results microservices>/results/models (e.g., http://localhost:8096/results/models) For testing training microservices open a web browser and paste training URL; http://<IP of microservices>:<Port of training microservices>/training (e.g., http://localhost:8091/training) If you see either: {"values":[],"total":0,"next":null,"previous":null} or a list of training jobs in JSON format, this means the result and training microservice service is available. 1. Question. I haven't seen an anomaly but I believe that my 'property' is anomalous?         This can be caused by different reasons, here are the most common causes: The certainty is too high. If the certainty is too high ThingWatcher is conservative in its categorization of "true positives" and therefore may emit more "false negatives". Reducing the certainty will change this behavior but note that ThingWatcher may now categorize too many "false positives" as a result. In other words, ThingWatcher may detect the desired anomalies but also some non-anomalies. The 'property' is anomalous during training data collection. If ThingWatcher creates a predictive model from anomalous data, it may not be able to detect the desired anomalies during MONITORING because the data does not really appear to be anomalous. So ThingWatcher treats this pattern as 'normal'. Therefore, ensure that 'property' values are also non-anomalous during training. There are long time gaps during the monitoring state so ThingWatcher stays in Buffering and categorizes these data points as non-anomalous. 2. Question. ThingWatcher detects an anomaly but my 'property' is non-anomalous? The certainty might be too low. In this case, ThingWatcher reports anomalies when the incoming data pattern looks even slightly different from the expected data pattern. ThingWatcher might need more training data. If the 'property' data has a pattern that occurs over a long time span, ThingWatcher needs to collect multiple cycles of all these patterns in order to detect a true anomaly without emitting too many false positives. 3. Question. ThingWatcher is in FAILED State, why?     There are many possible reasons for a failed state, here are the most likely problems that can cause a failed state. ThingWatcher emits a FAILED ThingWatcher State because the training service has not been setup or is down. similarly, the result service is not available. NotemessageText=Unexpected exception. {Throwable=[ConnectException: Operation timed out}]]messageText=Unexpected exception. {Throwable=[ConnectException: Connection refused}]]. Note that ThingWatcher is still able to collect all training data and you will only begin to see these failed states after ThingWatcher tried to post the training request. ThingWatcher emits a FAILED ThingWatcher State because time gaps prevent the data collection for training.You will see this warning in the log messages : "A long time gap was detected in the data that is greater than the threshold of {n}". This means you have a long gap in the training data and ThingWatcher will recollect the data. If there are more than 3 recollections due to a long time gap, ThingWatcher transitions to a failed state and will not be able to recover. In this case you can either instruct ThingWatcher to retrain and try again or check the data source to make sure it does not have long gaps. 4. Question. Why does ThingWatcher remain in Buffering? There are many possible reasons for ThingWatcher to remain in Buffering, but the most likely issue is time gaps which cause ThingWatcher to remain stuck in Buffering. If the incoming data regularly contains long time gaps, you will notice that ThingWatcher keeps alternating between the monitoring and buffering states. You may need to provide better quality data i.e. more evenly spaced data. Source: Alex Meng, Specialist Software Engineer

An introduction to installing the ThingWorx platform. Information on the environment, prerequisites, and configuration steps when installing ThingWorx. Includes walkthroughs of installing with H2 and PostgreSQL databases, an introduction and demonstration of the Linux installation script, solutions to common installation problems and more.     For full-sized viewing, click on the YouTube link in the player controls.   Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

Error: Failed to load SQL Modules into database Cluster. This error is usually seen during initializing the database cluster phase (in the setup as shown below). To resolve this issue, follow below steps: Create a PostgreSQL data folder before you start the installation (c:\postgres-data) and give full control for the user. Select the newly created data directory during the setup. After the successful installation, you can follow the remaining procedure for configuring it with ThingWorx from the respective installation document.

Welcome to the ThingWorx Manufacturing Apps Community! The ThingWorx Manufacturing Apps are easy to deploy, pre-configured role-based starter apps that are built on PTC’s industry-leading IoT platform, ThingWorx. These Apps provide manufacturers with real-time visibility into operational information, improved decision making, accelerated time to value, and unmatched flexibility to drive factory performance.   This Community page is open to all users-- including licensed ThingWorx users, Express (“freemium”) users, or anyone interested in trying the Apps. Tech Support community advocates serve users on this site, and are here to answer your questions about downloading, installing, and configuring the ThingWorx Manufacturing Apps.     A. Sign up: ThingWorx Manufacturing Apps Community: PTC account credentials are needed to participate in the ThingWorx Community. If you have not yet registered a PTC eSupport account, start with the Basic Account Creation page.   Manufacturing Apps Web portal: Register a login for the ThingWorx Manufacturing Apps web portal, where you can download the free trial and navigate to the additional resources discussed below.     B. Download: Choose a download/packaging option to get started.   i. Express/Freemium Installer (best for users who are new to ThingWorx): If you want to quickly install ThingWorx Manufacturing Apps (including ThingWorx) use the following installer: Download the Express/Freemium Installer   ii. 30-day Developer Kit trial: To experience the capabilities of the ThingWorx Platform with the Manufacturing Apps and create your own Apps: Download the 30-day Developer Kit trial   iii. Import as a ThingWorx Extension (for users with a Manufacturing Apps entitlement-- including ThingWorx commercial customers, PTC employees, and PTC Partners): ThingWorx Manufacturing apps can be imported as ThingWorx extensions into an existing ThingWorx Platform install (v8.1.0). To locate the download, open the PTC Software Download Page and expand the following folders:   ThingWorx Platform | Release 8.x | ThingWorx Manufacturing Apps Extension | Most Recent Datacode     C. Learn After downloading the installer or extensions, begin with Installation and Configuration.   Follow the steps laid out in the ThingWorx Manufacturing Apps Setup and Configuration Guide 8.2   Find helpful getting-started guides and videos available within the 'Get Started' section of the ThingWorx Manufacturing Apps Portal.     D. Customize Once you have successfully downloaded, installed, and configured the Manufacturing Apps, begin to explore the deeper potential of the Apps and the ThingWorx Platform.   Follow along with the discussion and steps contained in the ThingWorx Manufacturing Apps and Service Apps Customization Guide  8.2   Also contained within the the 'Get Started' page of the ThingWorx Manufacturing Apps Portal, find the "Evolve and Expand" section, featuring: -Custom Plant Layout application -Custom Asset Advisor application -Global Plant View application -Thingworx Manufacturing Apps Technical Lab with Sigma Tile (Raspberry Pi application) -Configuring the Apps with demo data set and simulator -Additional Advanced Documentation     E. Get help / give feedback / interact Use the ThingWorx Manufacturing Apps Community page as a resource to find documentation, peruse past forum threads, or post a question to start a discussion! For advanced troubleshooting, licensed users are encouraged to submit support tickets to the PTC My eSupport portal.

The KEPServerEX ThingWorx Native Interface was originally released with KEPServerEX v5.21 to enable connectivity with v6.6 of the ThingWorx Platform. Compatibility with the ThingWorx NextGen Composer (v7.4) was implemented with the release of KEPServerEX v6.1, while still allowing support of older ThingWorx composer versions through the Native Interface "Legacy mode" setting.   When connecting with ThingWorx v7.4 and newer, an Industrial Gateway thing template is configured in the NextGen Composer. The instructions for connecting KEPServerEX v6.1 (and newer) with ThingWorx v7.4 (and newer) can be found in the ThingWorx Help Center here:   KEPServerEX / ThingWorx Industrial Connectivity Connection Example (Expand ThingWorx Model Definition and Composer > Industrial Connections > Industrial Connections Example)     When connecting with versions of ThingWorx that are older than v7.4-- or if the older version of ThingWorx Composer will be used-- it will necessary to download and import the KEPServerEX Extension from the ThingWorx marketplace. The "RemoteKEPServerEXThing" thing template will then be used to allow connection with KEPServerEX. Here is a link to the extension download: ThingWorx IoT Marketplace   To enable Legacy Mode in KEPServerEX (v6.1 or newer, when connecting with versions of ThingWorx that are older than v7.4): 1. Open the KEPServerEX Configuration window 2. In the top-left Project view pane, right-click on "Project" and select "Properties" 3. Select the ThingWorx Property Group 4. Change "Enable" to "Yes"; and change "Legacy Mode" to "Enable"     See the chart below for a version compatibility summary:   KEPServerEX version ThingWorx version v5.21 pre-7.4, using RemoteKEPServerEXThing v6.0 pre-7.4, using RemoteKEPServerEXThing v6.1 or higher w/ Legacy Mode enabled pre-7.4, using RemoteKEPServerEXThing v6.1 or higher w/ Legacy Mode disabled (default) v7.4 or higher, using Industrial Gateway

Error: Exception: JavaException: java.lang.Exception: Import Failed: License is not available for Product: null Feature: twx_things Possible root cause: editing web.xml without further restart of the platform. In result, the product name does not pick up and the license path drops while the composer is still running, Fix: Go to LicensingSubsystem -> Services and run the AcquireLicense service. IF the issue does not get resolved, please contact the Support team, attaching your license.bin to the ticket.

Twilio extends the ThingWorx functionality to send SMS and voice messages in variety of languages. Starting from scratch I'll cover the steps required to setup the Twilio extension and the basic account registration (trial version) and setup at Twilio.com Twilio allows registering with the trial account, which comes with $15 as initial account balance, something which is quite useful for testing the initial setup and testing of Twilio extension in conjunction to ThingWorx. Prerequisite 1. Sign up, if not already done, with Twilio.com for free trail account 2. Configure the account to setup a verified phone number to receive text and voice messages, free trial account allows setting up 1 verified phone number that can receive text and voice messages Note: See What's the difference between a verified phone number and a Twilio phone number? 3. Choose and configure a Twilio phone number capable of sending text and voice messages, free trial account allows setting up of 1 Twilio number which will be used in the Extension configuration in ThingWorx for sending the messages 4. Download Twilio Extension from ThingWorx Marketplace Note that Twilio phone number should be enabled with the send SMS or send voice message capability, depending on what your use case is. Failing to do so will lead to error while testing the sendSMS or sendVOICE service (two default services provided with the Twilio Extension when imported in ThingWorx) Signing up and setting up Twilio account 1. Sign up on Twilio.com with your details and emailID. When registering for the first time you'll be prompted for your personal phone number as part of the security requirement. Also with free trial account this will be the only phone number where you'll be able to send SMS or Voice message via Twilio Extension in ThingWorx 2. Next would be to setup the Twilio number with voice and text messaging, navigate to https://www.twilio.com/console/phone-numbers/getting-started 3. Do ensure that you setup the voice or text capabilities for the new Twilio number, failing to do so will lead to error in sending message via Twilio Extension in ThingWorx 4. Different Twilio Number have different capabilities, it's quite possible that you don't find the number with required capabilities i.e. Voice, SMS - in that scenario simply search for different number, one that has the capabilities you are looking for 5. With trial account only 1 Twilio number is avilable for setup 6. While setting up the Twilio number you will be required to provide a valid local address of your country The cost of setting up the number and further sending the SMS will only be deducted from the $15 made available when you signed up for the first time. Navigate to your account details on Twilio.com to make note of the following information which will be used when configuring the Twilio Extension in ThingWorx: a. Account SID b. Authentication ID c. Your Verified Phone Number (will be used for receiving the messages) d. Twilio Phone number created with required capabilities, e.g. that number is capable of sending text message Here's how it shows up once you are successfully registered and logged on to the twilio.com/console To check your Twilio Phone Number and the Verified Phone Number navigate to twilio.com/phone-numbers As it can be seen in the above screenshot, the number I have bought from Twilio is capable of sending Voice and SMS. With this all's set  and ready to configure the Twilio Extension in ThingWorx, which is pretty straight forward. Setup Twilio Extension in ThingWorx 1. Let's setup the extension by navigating to the ThingWorx Composer > Import/Export > Extensions > Import > browse to the location where you saved the Twilio Extension from ThingWorx Marketplace and click Import 2. Once imported successfully refresh the composer and navigate to Modeling > Things , to create a new Thing implementing the Twilio Template (imported with the extension) like so, in screenshot below Thing is named DemoTwilioThing 3. Navigate to the the Configuration section of that Thing, to provide the information we noted above from Twilio account Note: this demo thing is setup to send the text SMS to my verified phone number only, therefore the CallerID in the configuration above is the Twilio number I created after signing up on Twilio with text message capability. 4. Save the created the Thing 5. Finally, we can test with the 1 of the 2 default services provided with the Twilio Template i.e. SendSMSMessage service 6. While testing the SendSMSMessage service use your verified phone number in To for receiving the message Troubleshooting some setup related issues I'll try to cover some of the generic errors that came up while doing this whole setup with Twilio Extension Error making outgoing SMS: 404 / TwilioResponse 20404 Reason If the Twilio number is not enabled with SMS sending capabilities you may run into such an error when testing the service. Here's the full error stack Unable to Invoke Service SendSMSMessage on DemoTwilioThing : Error making outgoing SMS: 404 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>20404</Code><Message>The requested resource /2010-04-01/Accounts/<AccountSID>/SMS/Messages was not found</Message><MoreInfo>https://www.twilio.com/docs/errors/20404</MoreInfo><Status>404</Status></RestException></TwilioResponse> Resolution Navigate back to the Phone Number section on Twilio's website and as highlight in screenshot above check if the Twilio number is enabled with SMS capabilities. Error making outgoing SMS:400 / TwilioResponse 21608 Reason You may encounter following error when attempting to send SMS/Voice message, here's the full error detail Wrapped java.lang.Exception: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21608</Code><Message>The number <somePhoneNumber>is unverified. Trial accounts cannot send messages to unverified numbers; verify <somePhoneNumber>at twilio.com/user/account/phone-numbers/verified, or purchase a Twilio number to send messages to unverified numbers.</Message><MoreInfo>https://www.twilio.com/docs/errors/21608</MoreInfo><Status>400</Status></RestException></TwilioResponse> Cause: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21608</Code><Message>The number <somePhoneNUmber>is unverified. Trial accounts cannot send messages to unverified numbers; verify <somePhoneNumber> at twilio.com/user/account/phone-numbers/verified, or purchase a Twilio number to send messages to unverified numbers.</Message><MoreInfo>https://www.twilio.com/docs/errors/21608</MoreInfo><Status>400</Status></RestException></TwilioResponse> Resolution As the error points out clearly the number used in To section while testing the SendSMSMessage service didn't have verified number. With free trial account you can only use the registered verified phone number where SMS/Voice message can be sent. If you want to use different number an account upgrade is required. Error making outgoing SMS:400 / TwilioResponse 21606 Reason Following error is thrown while testing SendSMSMessage service with different Twilio number which is either not the same as the number you bought when setting up the trial account or it doesn have SMS sending capabiltiy. Here's the full error stack Wrapped java.lang.Exception: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21606</Code><Message>The From phone number <TwilioNumber>is not a valid, SMS-capable inbound phone number or short code for your account.</Message><MoreInfo>https://www.twilio.com/docs/errors/21606</MoreInfo><Status>400</Status></RestException></TwilioResponse> Cause: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21606</Code><Message>The From phone number <TwilioNumber> is not a valid, SMS-capable inbound phone number or short code for your account.</Message><MoreInfo>https://www.twilio.com/docs/errors/21606</MoreInfo><Status>400</Status></RestException></TwilioResponse> Resolution Check the configuration in the ThingWorx Composer for the Thing created out of Twilio Template whether or not the callerID is configured with correct Twilio account number

Merging InfoTables can seem to be an overwhelming task in ThingWorx. What even IS an InfoTable?? The short answer is that an InfoTable is a specifically structured JSON object. Merging InfoTables is therefore as easy as incrementing through each, adding the columns of each to another table, and then populating the new table with data. For InfoTables with the same DataShapes, the built in "Union" service can be used. Find this snippet under the InfoTableResources section. There is also a snippet for incrementing through DataShape fields if the DataShape is known. For InfoTables which don't have the same DataShapes or for which the DataShape is not known, things get a bit trickier. Thankfully, some new KCS documentation provides example code to step you through merging any number of tables together. I hope this documentation is helpful!

As per ThingWorx Documentation: Updating Properties Automatically in a Mashup A mashup using the GetProperties service can be configured to use websockets and receive updates to properties automatically. When creating or editing a mashup, you can configure the GetProperties service so that the properties are automatically updated by selecting the Automatically update values when able checkbox in the service properties panel. So, the feature to update Properties Automatically in a Mashup is limited to GetProperties service only. Following are the steps to invoke our own custom Service automatically when a property change: 1. Find all the Properties in your Thing for which the DataChange should trigger the custom service. 2. In mashup; add value display widget (or some other widgets) for each property in Step1. 3. Bind the properties from GetProperties service to these widget. 4. Set visible property of these widgets to false so that they don't show up at the RunTime. 5. Now bind the ServiceInvokeCompleted event of GetProperties Service to your custom Service. 6. Save and view Mashup. Result: When any of the Property from Step1 is changed; Custom Service will be invoked in our Mashup.

If the ThingWorx 7.4 installation with MSSQL db doesn't start with a license error on the splash screen, despite taking the necessary steps for specifying the license path, the error might be misleading and the problem is actually lying in the database connection. Going to THingworxStorage/logs and opening ApplicationLog.log might reveal the following (or similar ) errors: 2017-04-13 10:26:17.993-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Sending Post Start Notifications... 2017-04-13 10:26:17.999-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.001-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.004-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.005-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.007-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.008-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.008-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Thingworx Server Application...ON A few things to check here. First, how the database was set up. The standard expected port if 1433, however, if you choose a different port - ensure that the port is available. If it isn't - update the port setting in SQL configuration manager. Another possible root cause -- the database scripts did not run properly upon the setup. Currently, there is an active Jira PSPT-3587 for resolving the script issue (NOTE, this thread to be updated once the Jira is fixed). There are two options here, either to run the sql files manually (found in the "install" folder of the downloadable), or edit the bat scripts manually. It's recommended to edit the script files instead because running the sql commands allows more room for a mistake. The following line in the bat scripts needs to be edited to have the ".\" removed: sqlcmd.exe -S %server%\%serverinstance%,%port% -U %adminusername% -v loginname=%loginname% -v database=%database% -v thingworxusername=%thingworxusername% -v schema=%schema%  -i .\thingworx-database-setup.sql Note, that the scripts need to be run from the same directory ("/install")