IoT & Connectivity Tips

The DPM User Experience Written by Tori Firewind, IoT EDC Team   As discussed in a previous post, DPM is a tool designed to be beneficial at all levels of a company, from the operators monitoring automated data on production events from the factory machines themselves, to the production supervisors who need to establish, task out, and track machine maintenance and improvement measures. DPM also engages the continuous improvement and plant leadership, by providing a standardized way to monitor performance that ultimately rolls up to the executive level. The end users of DPM are therefore diverse both in how they access DPM, and how they make use of its various features.   One of the perks to building DPM on top of the ThingWorx Foundation is that many of the webpages (called “mashups”) within ThingWorx are already responsive, and any  which aren’t responsive OOTB can be modified and custom designed for different size viewing screens to ensure that if necessary, end users can access DPM   from a variety of locations and devices. Most of the time, end users will be accessing mashups from hard-wired dashboards mounted on the actual devices,    or from wireless laptops which have standard size screens with standard resolutions. For use cases involving phones or tablets, however, it may be necessary to see how DPM will perform across a variety of bandwidth and latency conditions. Often, cellular or satellite connection is a must to facilitate field team cooperation, and 5G networks often result in worsened performance.   So, to demonstrate the influence of bandwidth and latency on the responsiveness of DPM, the Production Dashboard was loaded in the Google Chrome browser repeatedly under varying conditions. This dashboard is the webpage most operators and field users would access to log event information and production details (so it is widely used by end users). This provides a sort of benchmark of the DPM solution, something which indicates what can be expected and tells us a few things about how DPM should be deployed and configured.   Latency was introduced by hosting the servers involved in the test in different regions (all Azure cloud hosted servers, one in US East, one US West, and one in Japan East). Bandwidth was introduced using a tool on the PC with either no bandwidth or 4 megabits/second.   Browser caching was turned on and off as well, to simulate the difference between new and return users; new users would not have the webpage cached, so their load times are expected to be longer. Tomcat compression was also configured in half of the runs to demonstrate the importance of compression for optimal performance.   Each of these 24 scenarios was then tested 10 times from each location, and the actual data can be found in the attached benchmark document (a working  solution benchmark, which is not designed to be referenced directly, as matters of infrastructure may influence the exact performance of the solution).  Even with bandwidth, every region sees better performance for return users versus new users, which may be important to note. However, because DPM field users most commonly access DPM often, the return user time is a better indicator of adoption, and those numbers look great in our simulations. Notice the top line which shows the very worst of mobile performance, what happens over networks with bandwidth when Tomcat Compression is not enabled. Load times vary only slightly for regular networks when Tomcat Compression is enabled, and they vastly improve performance across regions and on mobile networks, so it is highly recommended (instructions on how to enable are below).   Key Takeaways Latency and bandwidth impact DPM performance in exactly the way one would expect of a web application. While any DPM server can be accessed from any region, regions with more latency will experience delays proportional to the amount of latency. In the chart here, find the three regions represented three times by three different colors (different from the charts above): The three different shades of each color represent the different regions Green represents the optimal configuration settings (Tomcat compression enabled, caching turned on) for returning users with bandwidth limitations (i.e. mobile networks like 5G) Blue shows first-time page visitors with no bandwidth limitations Purple shows first-time visitors that do have bandwidth limitations The uncompressed first-time load for mobile users (those with bandwidth limitations imposed) within the same region is also given to demonstrate the importance of enabling Tomcat Compression (load times only get worse without compression the farther the region) Notice how the green series has lower load times across the board than the blue one, meaning that return users even with bandwidth limitations have better performance across every region than new users. Also notice how the gap is larger between lighter colors and darker colors, where the darker the color, the farther the region from the DPM servers. This indicates that network latency has a more significant influence on performance versus bandwidth, with only longer running transactions like file uploads seeing a significant performance hit when on a network with bandwidth limitations.  Find out how to enable tomcat compression  and review the full solution benchmark in the document attached.  

Abstract This article explores an approach on how to optimize ThingWorx thread usage during service execution. It addresses common scenario where the Event Processor executor thread pool can become saturated, which can lead to queuing and slow responsiveness to users. This article is a proposal for configuring and optimizing services, with an example. Please, use the post comments to provide feedback or suggestions. All used capabilities are 100% ThingWorx OOTB. In the following, the acronym SETO (Service execution thread optimization) will be used to refer to entities and their capabilities.   Introduction During the execution of a service, events trigger a subscription that runs in a new thread. In addition to the best practices for timers and schedulers, SETO uses this mechanism to parallelize the execution of a service across subdivisions of a population of Things. When using ThingWorx, clients often encounter issues related to the regular execution of services such as "get data" or "update KPIs".  These services, when executed frequently, can lead to several challenges: Single thread processing: Processing occurs only in one thread, might lead to long execution time. Thread Pool Saturation: When too many services are executed simultaneously, the thread pool can become saturated, slowing down the entire system. Queue Overflows: Thread pool saturation can lead to queue overflows, where new tasks cannot be processed in time. Task Loss: Overflows can result in the loss of important tasks, affecting service reliability. Unresponsive Server: Due to overload, the server can become unresponsive, impacting user experience and operational continuity. Data processing requirements often lead to decreasing execution time without the required consideration towards impact on stability and availability of the other required services.  This post provides explanations and examples of how to think differently about how to execute data processing quickly and reliably while also focusing on the systems stability and reliability.   Why Use Service Execution Thread Optimization? To address these issues, Service Execution Thread Optimization (SETO) is essential. It allows for effective management of resource saturation and its consequences. SETO enables you to control the percent or number of used threads and the safety inter-execution delay.   Selective Execution Management Prioritized processing involves sorting tasks from oldest to newest, focusing on older tasks to prevent them from falling too far behind. However, it is crucial not to limit execution to only old tasks but to find a balance to process all eligible tasks. If all things don’t execute the service before the inter-execution delay, SETO skips them. These remaining things will be prioritized for the next timer event. Selectively orchestrating work execution allows designing your data processing workflow so that it is coherent with your various use cases functional requirements as well as being tuned for performance and your provisioned resources. This example covers functional requirements which require data to be processed within a 1-hour period, and hence sorts the processing by oldest first, and allows for skipping execution windows if the system is busy at that time. Your application of this approach may leverage a more complex approach suited to your needs that could add sensitive assets that should be processed with more urgency depending on the type or location of an asset.  This is where multiple copies of the SETO entity would adapt according to your use cases, selection criteria, work sorting, timing, thread usage configurations, etc.   Applicability and Ease of Use SETO has been designed to avoid touching the existing model as much as possible. The only prerequisite is to have a Datetime property on each Thing to track the last execution of the service, one property per service.   How to use it? Configuration of Allocated Threads Per Service 1. Create an Thing from the PTCSC.SETO.WorkProcessManager_TT ThingTemplate. 2. Configure the AllocatedThreadsPerService Table:   TS or TT: The entity type that carries the service. Accepted values are “TT” for ThingTemplate and “TS” for ThingShape TS or TT name: The name of the entity that carries the service. Service name: The name of the service you want to streamline the execution. Safety timeout: A security time value that defines a hard stop before the next timer trigger, in seconds. Minimum of 5 seconds. Example: If my timer is set to 3 minutes and my safety timeout is set to 10, the service won’t be executed on remaining things at 2 minutes and 50 seconds. The safety timeout must be less than the timer update rate. Last update property name: The name of the datetime property on things used to store the last service execution timestamp. Update older than: The minimum number of minutes since the last update before the service is triggered on an object. If set to 0, all things will be taken in account anytime. Percent available thread: The percentage of available thread to use. Value constrained from 0 to 99. This value is truncated to select a round number. Example: 13 threads available, 60% requested → 13*0,6=7,8, so 7 threads will be used. Use number of threads: Way to the select the requested number of threads. TRUE, use percent of available threads / FALSE, use the number of threads value column. Number of threads: Maximum number of threads allowed to run in parallel to process the list of things. This number should be set to leave at least a few threads free. SETO let at least on thread free, so the number of request threads might be lower or equal to zero. The remaining thread might be used by other execution outside SETO. 3. Creating a Subscription in the new Thing.   Create a Subscription to link the Timer to the Service to execute.   Result example The following example demonstrates how logging is seamlessly integrated into SETO during the execution optimization process. Logging is part of the ServiceExecution_SUB subscription in PTCSC.SETO.WorkProcessManager_TT ThingTemplate.   Important Notes Appropriate entity selection, sorting, scheduling algorithms will depend on the specific use cases and workload. SETO uses QueryImplementingThingsOptimized for better performance, so indexing is encouraged. Processing needs to address resulting status and potentially needed retries as a part of the scheduling algorithm.  You'll want to add use case specific logic which ensures the entities with failing processing are only attempted a couple of times and that their error state is notified and addressed by a system administrator. Ex: just sorting by oldest last execution time will cause anomalous non-working entities to surface to the top and taking priority over all other processing (this is an anti-pattern to avoid, requiring a circuit breaker) SETO can only trigger Services without input parameters.  You could evolve the concept to leverage passing Service parameters through JSON property event data. In this article, a Timer is used to trigger a regular event. As best practice, it is better to use a Scheduler (regular as a Timer) Indeed, when server starts, all timers will start. Scheduler trigger depends on current datetime and not on sever startup.   Conclusion and provided file By optimizing Service execution, businesses can improve the performance and reliability of their applications built on the ThingWorx Platform. Proper configuration helps prevent common issues and ensures smooth service execution. Find attached to this article the SETO entities valid from version 9.3.   I've just put resource here as more generic as it is not only the thread pool that would become saturated in the given context.  It could be many other aspects downstream which are tightly coupled like the database.     NB: thanks to Greg EVA for sharing knowledge and review. 

Follow these steps to restart the ThingWorx Server, then to verify that all install services are running: 1. Open the ThingWorx Server Properties configuration application. To locate this application in Windows Server edition operating systems, search for "Configure PTC Servelet Engine"; or the application can be launched from the Windows Explorer. The default install directory is: <ThingWorx install path>\ThingWorxManufacturingApps\PTC_Servlet_Engine\bin\thingworxServerw.exe 2. Use the controls in the General tab to Stop and then Start this service. 3. Now open the Windows Services console (services.msc) and verify these four services are installed and running:      a. ThingWatcherResult      b. ThingWatcherTraining      c. Thingworx Server      d. ThingworxPostqresql_Service-PostgreSQL_Server9.4

Configuring Navigate search using Common Tailoring and configuring access permission on apps from role perspective.     For full-sized viewing, click on the YouTube link in the player controls.   Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

  Design Your Data Model Guide Part 3   Step 7: Prioritize     The first step in the design process is to use the Thing-Component Matrix to identify and prioritize groups of Components that are shared across multiple Things. These groups will be prioritized by number of shared Components, from highest to lowest, enabling is to break out the most commonly used groups of Components and package them into reusable pieces. Let’s examine our example Thing-Component Matrix to identify and prioritize groups. In the table below, we have done this and recognized that there are FOUR groups.   NOTE: Each item in our unique Thing-Component Matrix would also count as a group on its own. This can be dealt with almost separately from our process, though, because there is no overlap between different Things. The "Templates for Unique Components" and "Adding Components Directly to Things" sections in the Iterate step of this guide covers these "one-offs."     Step 8: Largest Group     The base building block we use most often is the Thing Template. To start the design process, the first step is to create a Thing Template for the largest Component group. Applying this to our Smart Factory scenario, we'll take the largest group ("Group 1") and turn it  into a Thing Template using the Entity Relationship Diagram schematic.   Since every item on our production line shares these Components, we will name this Thing Template Line Asset. Now, let's build this using our Entity Relationship Diagram.   The result is a ThingWorx Thing Template with five Properties, one Event, and one Subscription.     Step 9: Iterate     Once an initial base template has been created for the largest group, the rest of the groups can be added by selecting the appropriate entity type (Thing Template, Thing Shape, or directly-instantiated Thing). The following Entity Decision Flowchart explains which entity type is used in which scenario:   Now that we have established our "Line Asset" Thing Template for our largest group, the next step is to iterate through each of our remaining groups. Following the flowchart, we will identify what entity type it should be and add it to our design.   Group 2 - "System Connector"   The second group represents connectors into both of our internal business systems. We will call them System Connectors.   If we look at Group 2 versus our "Largest Group" Thing Template, we can see that there is no overlap between their Components. This represents the third branch of the Entity Decision Flowchart, which means we want to create a new Thing Template.   Following this rule, here is the resulting template:   Group 3 - "Hazardous Asset"   The third group represents line assets that require emergency shutdown capabilities because under certain conditions, the machinery can become dangerous. We will call these Hazardous Assets.   If we look at our two previous Thing Templates, we can see that there is full overlap of these Components with our previous largest group, the "Line Asset" Thing Template. This represents the fourth branch of the Entity Decision Flowchart, which means we want to create a CHILD Thing Template.   Following this rule, here is the resulting Thing Template:   Group 4 - "Inventory Manager"   The fourth group represents line assets that keep track of inventory count, to ensure the number of assembled-products is equal to the number of checked-products for quality.   If we look at our existing Thing Templates, we can see that there is some overlap of the Components in our "Hazardous Asset" and "Line Asset" Thing Templates. This represents the second branch of the Entity Decision Flowchart, which means we want to create a Thing Shape.   Following this rule, here is the resulting Thing Shape:   Templates for Unique Components   Now that we have handled all our shared component groups, we also want to look at the unique component groups. Since we have already established that each group in our Unique Thing-Component Matrix does not share its Components with other Things, we can create Child Thing Templates for these line assets.   NOTE: Refer to the finished design at the bottom to reference all of the inherited Thing Templates and Thing Shapes for these Child Thing Templates.   Adding Components Directly to Things   In many cases, we will have Components that only exist for a single Thing. This frequently occurs when there will be only one of something in a system. In our case, we will only need one "System Connector" for each of the Maintenance System and the Production Order System.   Instantiate Your Things   At this point, the Data Model is fully built. All we need to do now is instantiate the actual Things which represent our real-world machines and digital-connections.     Step 10: Validate     The final step of the model breakdown design process is validation through instantiation. This is the process by which we take our designed Thing Templates / Thing Shapes / Things and actually create them on the ThingWorx platform to ensure they meet all of our requirements. This is done by tracing back through the chain of inheritance for all the Things in the data model to ensure they contain all of the required Components from the Thing-Component Matrix. Once we have verified that each Thing contains all of its requirements, the data model is complete.   Using this technique on each of your Things, you can explicitly prove that all of the requirements have been met.   Step 11: Next Steps   Congratulations! You've successfully completed the Design Your Data Model Guide covering the first three steps of the proposed data model design strategy for ThingWorx. This guide has given you the basic tools to: Create user stories Identify endpoints in your system Break down your data model using an Entity Relationship Diagram Decide when to use Thing Templates vs. Thing Shapes vs. directly-instantiated Things     The next guide in the Design and Implement Data Models to Enable Predictive Analytics learning path is Data Model Implementation.

  Learn how to send and receive JSON and XML payloads to SOAP and REST services.   GUIDE CONCEPT   This project will introduce how to create services that can send and receive XML and JSON payloads.   Following the steps in this guide, you will develop services to handle, send, and receive XML and JSON.   We will teach you how to use the ThingWorx Platform to handle REST requests and send payload for external SOAP and REST services.   YOU'LL LEARN HOW TO   Efficiently handle, send, and receive XML and JSON in ThingWorx Make requests to external REST services Make requests to external SOAP services   NOTE:  The estimated time to complete all parts of this guide is 60 minutes.     Step 1: Completed Example   Download the attached JSONXMLEntities.zip from attached files and import it into ThingWorx Composer.   In this tutorial, you will learn how to make and handle requests to live external REST and SOAP services. The entities file provided contains the following files and entities as a completed version of the scenario that will be covered:   Name Description JSONRequestThing Thing with examples of making requests using JSON XMLRequestThing Thing with examples of making requests using XML JSONHandlerThing Thing with examples of handling JSON requests and returning organized data XMLHandlerThing Thing with examples of handling XML requests and returning organized data This guide will use services provided by Microsoft Azure. Create a free account to utilize Microsoft Azure Web API services. In order to utilize this web api, obtain a FREE API key from Microsoft Azure Portal using these Microsoft Azure instructions. You are able to use other Web APIs for this guide.   Follow the next steps in order to get started.     Step 2: Sending JSON Based Requests   Whether making a request to a public web service or sending data to a device running an application, ThingWorx is able to utilize different REST request methods (GET, POST, PUT, DELETE, etc) using the ContentLoaderFunctions Resource. This same resource provides services for XML payloads, but JSON is the default formatting for REST. See the "Sending XML Based Requests" section in this guide for working with XML. This guide will not explain REST or REST methods, only how to use these REST methods.   To get started, create a Thing using the below steps. You will create new services in this Thing to make REST requests and add a service to help with HTML encoding. NOTE: Examples of these services can be found in the JSONRequestThing entity, which is provided in the download. In ThingWorx Composer, click the + New at the top of the screen. Select Thing in the dropdown. Name the Thing JSONRequestExample and set the Base Thing Template to GenericThing. Click Save. Click the Services tab. Create a new service called EncodeQueryToHTML. Add the following Input: Name Base Type Required query String True   8. Add the following JavaScript to help encode the string and return an HTML friendly string. try { var result = ""; if(query) { result = query.replace(/&/g, "&").replace(/>/g, ">").replace(/</g, "<").replace(/"/g, "'"); } } catch(error) { logger.error("JSONRequestThing.EncodeQueryToHTML(): Error Encoding String - " + error.message); }   9. Click Save and Continue to save your changes.   GET Requests   For GET requests to a REST service, you will use the GetJSON service of the ContentLoaderFunctions Resource. This service takes parameters from the proxy information to how to handle SSL issues. All of the parameters are optional. Perform the following steps to create your ThingWorx service to make REST Get requests.   In the Services tab of the JSONRequestExample Thing, create a new service called JSONGetRequest. The service will have the below properties as Input. Name Base type Required url String True header JSON False   3. In the Snippets section, filter and search for GetJSON. 4. Once you've found GetJSON under the ContentLoaderFunctions section, add it to the editor. You'll see all of the possible parameters for the request. In this example, we'll only set the url and header values. 5. Update the code snippet to include the parameters. Use the below code as a reference. var params = { headers: header /* JSON */, url: url /* STRING */ }; // result: JSON var result = Resources["ContentLoaderFunctions"].GetJSON(params);   6. Set the Output as JSON. 7. Click Save and Continue to save your changes. This code will enable REST requests to be sent to an available web service. You will need to include the query string in the URL. In the provided sample, open the JSONRequestThing Thing and check out the JavaScript for the SearchMicrosoftBing service for a more complex example. Note that you are setting the subscription key needed for a Microsoft Azure request and how to encode raw string input into a query string. Check out the Microsoft Azure documentation of the varying query strings that can be used.     POST/PUT Requests POST and PUT requests are created similarly to GET requests. The methods in the ContentLoaderFunctions Resource is PostJSON and PutJSON. These services include a JSON based paramter titled content. This parameter will allow you to compose the body of the request.   Based on how the REST service is implemented, data will be in the content parameter or the headers parameter. See below for steps on creating a POST request. In the Services tab of the JSONRequestExample Thing, create a new service called JSONPostRequest. The service will have the below properties as parameters. Name Base Type Required url String True header JSON False body JSON False   3. In the Snippets section, filter and search for PostJSON. 4. Once you've found PostJSON under the ContentLoaderFunctions section, add it to the editor. 5. Update the parameter object to use the parameters for the service. Keep in mind, there are different ways this can be done. If the URL or the header will always be the same, then save these values as properties on the Thing and add parameters if the URL for this request will need them. 6. Test the new service with a running REST application or use both the GET and POST methods to setup your Azure Active Directory.   DELETE Requests   In ThingWorx, the DELETE method will be called similarly to the GET method. There will be a number of parameters that can be used and you will need to call the DELETE service of the ContentLoaderFunctions Resource. Follow to below instructions.   Duplicate the JSONGetRequest service you created earlier in order to get a head start on the new service. When prompted, title this new service JSONDeleteRequest and save. In the code area, use the snippet once again to add the delete service from ContentLoaderFunctions. Update the JavaScript code to utilize the delete service. Save, and you're done!   A simple test for calling this delete method via Azure, is using the REST service to delete an email template. The URI and parameters can be found in the Azure API.     Click here to view Part 2 of this guide.

    Step 9: Create Event and Subscription   In this section, you will create a mechanism to notify the user when inclement weather occurs. For example, whenever the weather indicates storm/snow/rain, an Event should be triggered. This event is then handled by a service called a Subscription.   For this tutorial, while updating weather information inside the service UpdateWeatherInfo,we need to fire an Event: BadWeather when the weather description indicates either storm/snow/rain. This Event is handled by a Subscription: HandleWeather, which is a service that gets called whenever the BadWeather Event is fired. The subscription service HandleWeather updates a property called AlertNotification.   Create Event   In this part of the lesson, we will create an Event: BadWeather that updates weather information inside the service UpdateWeatherInfo when the weather description indicates either storm/snow/rain.   Create a property AlertNotification with a baseType STRING using the Add Property feature of the plugin we discussed before. Right click inside your java file->ThingWorx Source->Add Event.   Set the name to BadWeather and set a name for the Data Shape to Weather. NOTE: This custom Data Shape has to be created in Composer. Importing Datashapes and other custom entities created in Composer into your extension will be discussed later in the tutorial. Our DataShape Weather includes one field called WeatherDescription with a STRING base type.  Click Finish. NOTE: This will create annotation for your EventDefinition.    Create Subscription   In this part of the lesson, we will create a Subscription: HandleWeather, which is a service that gets called whenever the BadWeather Event is fired. The subscription service HandleWeather updates a property called AlertNotification. Right click inside your java file ->ThingWorx Source->Add Subscription   Set the Event Name to BadWeather and Handler Service to HandleWeather. NOTE: This means that whenever the BadWeather event is fired, the HandleWeather service will be executed. Source is left blank if the event belongs to the same template. Click Finish. This creates annotation for subscription service and also creates a new service called HandleWeather.   Modify Service   In this part of the lesson, you'll ensure that when the properties are updated, BadWeather event is triggered if the description indicates rain/snow/thunderstorm. To do this, we will modify the UpdateWeatherInfo service.   After we have called the setPropertyValue method for the properties WeatherDescription and Temperature, we can check if the weather description contains snow/rain/thunderstorm. We will create an InfoTable from the Weather Datashape. InfoTables represent data sets that take the structure of the Datashape. Each row of an InfoTable can be passed as a ValueCollection to hold data within the table. When an event is fired, we need to send data along with it. This data will be passed as an InfoTable and it is then handled by the Subscription handling the Event.We use a ValueCollection to add the weatherDescription to the InfoTable. Then, this InfoTable is set as the Event Data.   Add the code snippet to UpdateWeatherInfo section at the end of the service after setting the properties- Temperature and WeatherDescription. /* fire event BadWeather */ if (description.contains("snow") || description.contains("rain") || description.contains("thunderstorm")) { ValueCollection v = new ValueCollection(); v.put("weatherDescription", new StringPrimitive(description)); InfoTable data = InfoTableInstanceFactory.createInfoTableFromDataShape("Weather"); data.addRow(v); _logger.info("Firing event"); ThingworxEvent event = new ThingworxEvent(); event.setEventName("BadWeather"); event.setEventData(data); this.dispatchEvent(event); } Inside the HandleWeather service, set the property AlertNotification. Ensure that you have created the property AlertNotification using the eclipse plugin. Add the following code snippet to the HandleWeather service. @ThingworxServiceDefinition(name = "HandleWeather", description = "Subscription handler", category = "", isAllowOverride = false, aspects = {"isAsync:false"}) public void HandleWeather( @ThingworxServiceParameter(name = "eventData", description = "", baseType = "INFOTABLE") InfoTable eventData, @ThingworxServiceParameter(name = "eventName", description = "", baseType = "STRING") String eventName, @ThingworxServiceParameter(name = "eventTime", description = "", baseType = "DATETIME") DateTime eventTime, @ThingworxServiceParameter(name = "source", description = "", baseType = "STRING") String source, @ThingworxServiceParameter(name = "sourceProperty", description = "", baseType = "STRING") String sourceProperty) throws Exception { _logger.trace("Entering Service: HandleWeather with: Source: \"\", Event: \"BadWeather\", Property: \"\""); this.setPropertyValue("AlertNotification", new StringPrimitive("Alert:"+eventData.getFirstRow().getStringValue("weatherDescription"))); _logger.trace("Exiting Service: HandleWeather"); } Now we have an event BadWeather fired every time weather description indicates storm/snow/rain and it is handled by HandleWeather service that sets the AlertNotification property to the InfoTable data passed by the event.     Step 10: Add Composer Entities In previous parts of this tutorial, we assumed we had a datashape Weather available with field weatherDescription as the Datashape of our event: BadWeather. In this part of the lesson, we'll create a DataShape. Go to ThingWorx Composer. Click the + button. In the dropdown, select Data Shape. Enter a name, for example: Weather. Add a Field Definition weatherDescription with baseType STRING. Click check mark in the top left, then Save.   Click the More drop-down, then click Export. Export the DataShape entity from Composer, it will download in your system as an xml file. Go back to Eclipse, right-click on your project ->Import..->ThingWorx->Entities. Click Next. Browse to the directory where the xml file was downloaded. Select the xml file and Click Finish.   NOTE: This adds the xml file to the Entities folder in your project. Following the same procedure, import other entities required for this Extension stored in the Entities folder of the download we provided for this training.   NOTE: You can uncheck the box for importing DataShapes_Weather.xml, if you already loaded your Datashape in the previous steps.     Click here to view Part 5 of this guide.

Troubleshooting platform issues is  generally done by using a layer approach, similar to a simplified OSI Model. From bottom to top, the following layers represent the areas to analyze during each step: 1. Physical (Server, power, wired connections): check the server status and condition, CPU and memory levels. 2. Software (Operating system, tomcat, java versions, compatibility, and configuration): refer to the compatibility matrix to ensure the requirements are met; verify Tomcat  java configuration. * Note: Tomcat manager, server status, conveniently provides this information in one place. 3. Network: ensure  proper connectivity, port availability, firewall  configuration, and additional security, if applicable. 4. Application. The main focus of this blog post will concentrate on the step 4. As the Thingworx application is driven by Tomcat, first available tools coming "out-of-the-box" is the built-in Tomcat manager app.  Clicking on the "Server Status" provides the information on the versions, memory usage, processes, times and thread counts. Keep in mind, the default Tomcat maximum thread number is 200. Some additional tools that could assist in troubleshooting java applications and gathering performance metrics are: Javamelody, new relic, profiler4j. These have to be obtained, installed, and configured separately. Javamelody: Free and lightweight monitoring tool which does not do any profiling, safe to use in production environments. It comes with a series of plug-ins including for Grails, Jenkins and Jira. New relic: Real-time Java application monitoring, features code deployment reports, transaction tracing across different tiers and the ability to create alerts. Subscription fee applies. Profiler4j: Profiler4J is a free open-source tool for profiling in Java. It is enabled by passing an argument at start-up with a path to the Profiler4J .jar file. It comes with several graphs and charts showing a call graph with method details, a call tree, a memory monitor, a class list and thread monitoring. From the application perspective, Thingworx composer provides a PlatformSubsystem and LoggingSubsystem: PlatformSubsystem contains such services as GetPerformanceMetrics, GetSummaryInformation, GetThingworxVersion, and more to provide fundamental information for any troubleshooting scenario. LoggingSubsystem contains the logs, log settings, and other monitoring values. List of recommended tools for troubleshooting all layers: Wireshark: monitors network traffic Jstack: monitors memory consumption of specific threads Dynatrace: system performance and web application performance jconsole: system or application performance ​​

This is a follow-up post on my initial document about Edge Microserver (EMS) and Lua Script Resource (LSR) security. While the first part deals with fundamentals on secure configurations, this second part will give some more practical tips and tricks on how to implement these security measurements.   For more information it's also recommended to read through the Setting Up Secure Communications for WS EMS and LSR chapter in the ThingWorx Help Center. See also Trust & Encryption Theory and Hands On for more information and examples - especially around the concept of the Chain of Trust, which will be an important factor for this post as well.   In this post I will only reference the High Security options for both, the EMS and the LSR. Note that all commands and directories are Linux based - Windows equivalents might slightly differ.   Note - some of the configuration options are color coded for easy recognition: LSR resources / EMS resources   Password Encryption   It's recommended to encrypt all passwords and keys, so that they are not stored as cleartext in the config.lua / config.json files.   And of course it's also recommended, to use a more meaningful password than what I use as an example - which also means: do not use any password I mentioned here for your systems, they might too easy to guess now 🙂   The luaScriptResource script can be used for encryption:   ./luaScriptResource -encrypt "pword123" ############ Encrypted String AES:A26fBYKHJq+eMu0Fm2FlDw== ############   The wsems script can be used for encryption:   ./wsems -encrypt "pword123" ############ Encrypted String AES:A26fBYKHJq+eMu0Fm2FlDw== ############   Note that the encryption for both scripts will result in the same encrypted string. This means, either the wsems or luaScriptResource scripts can be used to retrieve the same results.   The string to encrypt can be provided with or without quotation marks. It is however recommended to quote the string, especially when the string contains blanks or spaces. Otherwise unexpected results might occur as blanks will be considered as delimiter symbols.   LSR Configuration   In the config.lua there are two sections to be configured:   scripts.script_resource which deals with the configuration of the LSR itself scripts.rap which deals with the connection to the EMS   HTTP Server Authentication   HTTP Server Authentication will require a username and password for accessing the LSR REST API.     scripts.script_resource_authenticate = true scripts.script_resource_userid = "luauser" scripts.script_resource_password = "pword123"     The password should be encrypted (see above) and the configuration should then be updated to   scripts.script_resource_password = "AES:A26fBYKHJq+eMu0Fm2FlDw=="   HTTP Server TLS Configuration   Configuration   HTTP Server TLS configuration will enable TLS and https for secure and encrypted communication channels from and to the LSR. To enable TLS and https, the following configuration is required:     scripts.script_resource_ssl = true scripts.script_resource_certificate_chain = "/pathToLSR/lsrcertificate.pem" scripts.script_resource_private_key = "/pathToLSR/key.pem" scripts.script_resource_passphrase = "keyForLSR"     It's also encouraged to not use the default certificate, but custom certificates instead. To explicitly set this, the following configuration can be added:     scripts.script_resource_use_default_certificate = false     Certificates, keys and encryption   The passphrase for the private key should be encrypted (see above) and the configuration should then be updated to     scripts.script_resource_passphrase = "AES:A+Uv/xvRWENWUzourErTZQ=="     The private_key should be available as .pem file and starts and ends with the following lines:     -----BEGIN ENCRYPTED PRIVATE KEY----- -----END ENCRYPTED PRIVATE KEY-----     As it's highly recommended to encrypt the private_key, the LSR needs to know the password for how to encrypt and use the key. This is done via the passphrase configuration. Naturally the passphrase should be encrypted in the config.lua to not allow spoofing the actual cleartext passphrase.   The certificate_chain holds the Chain of Trust of the LSR Server Certificate in a .pem file. It holds multiple entries for the the Root, Intermediate and Server Specific certificate starting and ending with the following line for each individual certificate and Certificate Authority (CA):     -----BEGIN CERTIFICATE----- -----END CERTIFICATE-----     After configuring TLS and https, the LSR REST API has to be called via https://lsrserver:8001 (instead of http).   Connection to the EMS   Authentication   To secure the connection to the EMS, the LSR must know the certificates and authentication details for the EMS:     scripts.rap_server_authenticate = true scripts.rap_userid = "emsuser" scripts.rap_password = "AES:A26fBYKHJq+eMu0Fm2FlDw=="     Supply the authentication credentials as defined in the EMS's config.json - as for any other configuration the password can be used in cleartext or encrypted. It's recommended to encrypt it here as well.   HTTPS and TLS   Use the following configuration establish the https connection and using certificates     scripts.rap_ssl = true scripts.rap_cert_file = "/pathToLSR/emscertificate.pem" scripts.rap_deny_selfsigned = true scripts.rap_validate = true     This forces the certificate to be validated and also denies selfsigned certificates. In case selfsigned certificates are used, you might want to adjust above values.   The cert_file is the full Chain of Trust as configured in the EMS' config.json http_server.certificate options. It needs to match exactly, so that the LSR can actually verify and trust the connections from and to the EMS.   EMS Configuration   In the config.lua there are two sections to be configured:   http_server which enables the HTTP Server capabilities for the EMS certificates which holds all certificates that the EMS must verify in order to communicate with other servers (ThingWorx Platform, LSR)   HTTP Server Authentication and TLS Configuration   HTTP Server Authentication will require a username and password for accessing the EMS REST API. HTTP Server TLS configuration will enable TLS and https for secure and encrypted communication channels from and to the EMS.   To enable both the following configuration can be used:   "http_server": { "host": "<emsHostName>", "port": 8000, "ssl": true, "certificate": "/pathToEMS/emscertificate.pem", "private_key": "/pathToEMS/key.pem", "passphrase": "keyForEMS", "authenticate": true, "user": "emsuser", "password": "pword123" }   The passphrase as well as the password should be encrypted (see above) and the configuration should then be updated to   "passphrase": "AES:D6sgxAEwWWdD5ZCcDwq4eg==", "password": "AES:A26fBYKHJq+eMu0Fm2FlDw=="   See LSR configuration for comments on the certificate and the private_key. The same principals apply here. Note that the certificate must hold the full Chain of Trust in a .pem file for the server hosting the EMS.   After configuring TLS and https, the EMS REST API has to be called via https://emsserver:8000 (instead of http).   Certificates Configuration   The certificates configuration hold all certificates that the EMS will need to validate. If ThingWorx is configured for HTTPS and the ws_connection.encryption is set to "ssl" the Chain of Trust for the ThingWorx Platform Server Certificate must be present in the .pem file. If the LSR is configured for HTTPS the Chain of Trust for the LSR Server Certificate must be present in the .pem file.   "certificates": { "validate": true, "allow_self_signed": false, "cert_chain" : "/pathToEMS/listOfCertificates.pem" } The listOfCertificates.pem is basicially a copy of the lsrcertificate.pem with the added ThingWorx certificates and CAs.   Note that all certificates to be validated as well as their full Chain of Trust must be present in this one .pem file. Multiple files cannot be configured.   Binding to the LSR   When binding to the LSR via the auto_bind configuration, the following settings must be configured:   "auto_bind": [{ "name": "<ThingName>", "host": "<lsrHostName>", "port": 8001, "protocol": "https", "user": "luauser", "password": "AES:A26fBYKHJq+eMu0Fm2FlDw==" }]   This will ensure that the EMS connects to the LSR via https and proper authentication.   Tips   Do not use quotation marks (") as part of the strings to be encrypted. This could result in unexpected behavior when running the encryption script. Do not use a semicolon (:) as part of any username. Authentication tokens are passed from browsers as "username:password" and a semicolon in a username could result in unexpected authentication behavior leading to failed authentication requests. In the Server Specific certificates, the CN must match the actual server name and also must match the name of the http_server.host (EMS) or script_resource_host (LSR) In the .pem files first store Server Specific certificates, then all required Intermediate CAs and finally all required Root CAs - any other order could affect the consistency of the files and the certificate might not be fully readable by the scripts and processes. If the EMS is configured with certifcates, the LSR must connect via a secure channel as well and needs to be configured to do so. If the LSR is configured with certifcates, the EMS must connect via a secure channel as well and needs to be configured to do so. For testing REST API calls with resources that require encryptions and authentcation, see also How to run REST API calls with Postman on the Edge Microserver (EMS) and Lua Script Resource (LSR)   Export PEM data from KeyStore Explorer   To generate a .pem file I usually use the KeyStore Explorer for Windows - in which I have created my certificates and manage my keystores. In the keystore, select a certificate and view its details Each certificate and CA in the chain can be viewed: Root, Intermediate and Server Specific Select each certificate and CA and use the "PEM" button on the bottom of the interface to view the actual PEM content Copy to clipboard and paste into .pem file To generate a .pem file for the private key, Right-click the certificate > Export > Export Private Key Choose "PKCS #8" Check "Encrypted" and use the default algorithm; define an "Encryption Password"; check the "PEM" checkbox and export it as .pkcs8 file The .pkcs8 file can then be renamed and used as .pem file The password set during the export process will be the scripts.script_resource_passphrase (LSR) or the http_server.passphrase (EMS) After generating the .pem files I copy them over to my Linux systems where they will need 644 permissions (-rw-r--r--)

Background: Customer machines create files containing data, configuration, log data, etc. These files may contain critical information for service technicians about the asset and its data. How can you make sure you get this important information to the Cloud as soon as it’s available? Use File Watcher, a standard tool available with Firewall-Friendly Agents. You can configure File Watcher to monitor a directory or file specification and automatically upload the file(s) to the Axeda Cloud. By monitoring selected directories and files on the device, the File Watcher determines which files have changed or appeared and then upload those files to the Axeda Cloud Server. Sometimes the size of the file being written or copied into the file watch target is large enough that it takes time to complete. Without taking the right precautions, the file upload may try to start before the file creation process is completed. Recommendations: To ensure that large files don’t cause problems, we recommend using a “move” or “rename” operation instead of a file copy operation to place your file into a watched directory. Unlike a “copy” operation, using autonomous operations like move or rename will ensure that the Agent doesn’t detect a file is in the midst of growing and prematurely begin to upload that operation. In situations when it isn’t possible to use the autonomous “move” or “rename” operations, you can implement a delay. Setting a delay in the File Watcher configuration will prevent the Agent from sending files to the Axeda CloudServer before those files have transferred completely to the watched file directory. Finally, use file compression wisely. You need to balance the benefits from compressing files before sending with the potential adverse impact that compression will have on the Agent. Compressing very large files BEFORE sending to the Platform will affect the Agent; however, compressing smaller files can be beneficial. If the Agent’s computer is of lower power, the CPU may have to work overtime to compress huge files or to send huge uncompressed files. As a general rule, compressing files before sending is recommended. However, depending on your Agent and network setup, it may prove more beneficial to stream uncompressed files rather than compress first and then send. Need more information? For information about configuring and using file watchers, see the Axeda® Builder User’s Guide (PDF) or the online help in Axeda Builder.

This Expert Session reviews the concept of “Decision Trees” and the functionality that is available in ThingWorx Analytics Builder. First, you will learn how to create and upload a dataset in ThingWorx Analytics Builder.  After that, it shows you how to train a model and score on the model that was just generated. It then goes into detail on how the prediction learner "Decision Tree" operates and classifies inputs.     For full-sized viewing, click on the YouTube link in the player controls.   Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

In this video we go through the steps to install ThingWorx Analytics Server 8.0   Updated Link for access to this video:  ThingWorx Analytics Server 8.0

ThingWorx Manufacturing Tips & Tricks Webinar is a weekly opportunity to hear PTC Subject Matter Experts present on various topics related to the manufacturing space and applications.   Agenda for this week's recorded session - - Manufacturing Apps Overview - Christine Bahmer - PTC Sigma Tile IoT Starter Kit, Presented by Serge Romano    

ThingWorx Analytics Builder - Upload Data   This video walks you through how to upload data and shows the configuration settings. Please be aware that shown configuration settings page is different for version 8.1.   Updated Link for access to this video:  ThingWorx Analytics Builder: Upload Data

  The data in your industrial IoT application is only valuable if it tells a story. As a developer, you need to consider the logical connections between graphical elements and business data and determine how users want to consume the information. With the Mashup Builder in ThingWorx, you can rapidly create a custom visualization that displays data from your connected devices. These easy-to-configure widgets deliver real-time data functionality at your fingertips - streamlining, processing, and displaying valuable information for your application users.   In this video, you'll learn how to create immersive, interactive visualizations by utilizing dynamic charts and graphs in your GUI. ThingWorx technical engineer Jason Wyatt demonstrates how to: Create a UI with ThingWorx Composer and Mashup Builder Incorporate visual displays that highlight business data requirements Supply data to components in your Mashup leveraging pre-built widgets and services Implement the Time Series Chart, Pie Chart, Open Street Map, Location Picker, Auto Refresh, Textbox, and Button widgets Watch the recording above, and download this sample Mashup containing all the data and entities shared in the video.   Q&A   We didn’t have time to get to all of the questions during the live webcast, but we’ve answered them here on our blog. Have any additional questions? Please leave us a comment. THIS UI CAPABILITY CLEARLY IS USEFUL FOR PROTOTYPING. AT WHAT SCALE AND / OR COMPLEXITY DO YOU RECOMMEND USING TRADITIONAL PROGRAMMING METHODS? The ThingWorx platform does not require users to utilize the Mashup Builder to create a frontend for their application. Some PTC customers use ThingWorx for the backend and rely on a traditional HTML team to design a custom UI. In that scenario, ThingWorx provides the Edge connectivity and backend storage/organization/business-logic. But, that said, the Mashup Builder can be used to develop production-level sites, especially if you customize State and Style definitions and create a Master mashup. WHAT IS THE DIFFERENCE BETWEEN GETIMPLEMENTINGTHINGS VS. GETIMPLEMENTINGTHINGSWITHDATA VS. QUERYPROPERTYHISTORY? GetImplementingThings provides a list of all Things instantiated from a particular Thing Template. GetImplementingThingsWithData provides the current property values associated with the Things. QueryPropertyHistory provides historical Property values. WHAT ARE THE POSSIBLE SCENARIOS WHERE SERVICEINVOKECOMPLETED CAN BE TRIGGERED? ServiceInvokeCompleted is an Event that occurs when a Mashup Data Service completes. We recommend you use it in any situation where timing is important for proper application execution. For instance, in the application created for the webinar, I wanted QueryPropertyHistory to run after SetProperties was complete. Therefore, I used ServiceInvokeCompleted to trigger the second Service. IS THERE ANY WAY TO DISPLAY TWO INFO TABLE DATA IN SINGLE GRID? This isn't possible by default, but as a workaround you could make a custom Service to combine two InfoTables into one; and then you could apply the combined data to the Grid Widget. CAN I RESIZE MY WIDGETS DURING RUNTIME? You can change any Widget Property at runtime that accepts an incoming data-bind. You can tell which Widget Properties accept dynamic data by looking at the Property itself in the bottom-left section of the Mashup Builder. If the Property has a left arrow pointing at the Property name, then you can bind it to dynamic data and change it during runtime. For Widget Properties where this isn't possible (such as some Widgets' Width and Height), you can apply custom CSS. IS THERE AN UNDO FEATURE AVAILABLE ON MASHUP BUILDER? Currently, no, but I believe Undo is a feature request on the R&D radar. HOW DO YOU GET MASHUP INFORMATION UP TO THE MASTER MASHUP? You can pass information between two Mashups when one Mashup pushes a change down to a Thing, then the other Mashup may read that data from that same Thing. Additionally, Mashup Parameters and Session Variables can act like Global Variables that are accessible across multiple Mashups. DOES THE OPAQUE OR MAKING BG COLOR TRANSPARENT WORK? You may set certain Widgets' style in such a way that the Widget itself is visible, but the background of the Widget is transparent. CAN YOU SHOW AGAIN HOW YOU ADDED THE TEXTBOX TO THE PROPERTIES OF SERVICES? You can view the recording of the webinar to see how I made an invisible TextBox set the MaxItems Property of the QueryPropertyHistory Mashup Data Service. I used an invisible TextBox to get a static number, then applied that number to the MaxItems Parameter of QueryPropertyHistory in order to change the Service's functionality. WHAT IS THE PURPOSE FOR SETPROPERTIES? SetProperties is one of several Mashup Data Services that sends information from the Mashup to the backend. DO WE HAVE FILTERS ON THE PIE CHART FOR CHANGING THE VIEW WHEN A VALUE IS SELECTED IN THE FILTER? Yes, there are several operations you may perform when a section of a Pie Chart is selected. I had originally intended to show how you can change the displayed color when you select a particular section of the Pie Chart, but I unfortunately ran out of time. In addition, all three of the display Widgets were tied together: when I selected a section of the Pie Chart, the same piece of data was selected on the Map and Time Series (and vice versa for clicking on the other two). You can import the sample Mashup into your Composer to view the configuration settings. IS THERE WAY TO SET PROPERTIES AND GET DATA THROUGH EMAIL USING THINGWORX? Yes, there are several ways to push information to the ThingWorx backend. For instance, you could have an entirely external process which strips data from an e-mail and then makes a REST call to ThingWorx to archive the data or trigger some Service. And, yes, ThingWorx supports sending and receiving e-mail through an Extension which you may download for free from the ThingWorx Marketplace. CAN THE MAXITEMS OF THE QUERYPROPERTYHISTORY BE APPLIED TO A COMBO BOX / LIST, WHICH CAN BE SELECTED FROM THE MASHUP SCREEN ONLINE? If I'm understanding correctly, you're asking whether or not the MaxItems Parameter of the QueryPropertyHistory Mashup Data Service can be set dynamically. The answer is yes. For instance, that TextBox which I made invisible could have been left visible and given a Label of "Max Items to Display". It would still be tied to the QueryPropertyHistory Service in the same way. But when the TextBox has a new value entered, then that would change the Parameter configuration of QueryPropertyHistory to show whatever had been entered. You would still have to figure out how to call the QueryPropertyHistory Service again. Changing a Parameter just sets up how the Service will behave the next time it is called. DO YOU HAVE TIPS FOR MAKING A PRINT-FRIENDLY MASHUP? My only real recommendation would be to use a Static Mashup with a specific resolution (to ensure that everything fit on the page while still looking good), while also setting the Style of various Widgets such that everything was in gray-scale (or something similar) that would be easy on your printer's ink cartridge. HOW WELL DO THE STYLE DEFINITIONS AND CSS WORK WITH TWITTER'S BOOTSTRAP? I'm unfamiliar with Twitter's Bootstrap, but I do know that with Style Definitions and the new CSS functionality, you have a lot of control over exactly how your Mashup looks. So you should be able to configure your Mashup to comply if Twitter has some particular requirements. WHY DO WIDGETS NOT STICK TO THE MASHUP WHEN YOU DRAG THEM INTO A BUSY UI? I HAVE HAD WIDGETS 'FLY' BACK TO THE WIDGET LIST AND HAVE BEEN UNABLE TO GET THEM TO STICK. Unfortunately, that's a known issue. It has something to do with the fact that you're not allowed to drag-and-drop a new Widget on top of an existing Widget. Which is a little strange considering that you explicitly *ARE* allowed to stack Widgets on top of one another after they've been placed in the central Canvas areas. I believe that R&D is investigating. CAN YOU EXPLAIN THE DIFFERENT OPTIONS FOR MASHUP? I believe that this question has something to do with the options in the pop-up when you first create a new Mashup. A Responsive Mashup grows and shrinks to match the viewing-resolution, while Static stays at the resolution you specify. The other options, (Page vs. Template vs. Shape), have to do with a specific setup where you have a Mashup-in-Mashup design. For instance, you could subdivide a Responsive Mashup just as I did in the webinar, and then have one of those sub-sections be an entirely different Mashup. Template and Shape Mashups are used when you have the scenario I describe above with Mashup-in-Mashup, but you want the sub-Mashup to change based off some other metric, such as the selection of a Thing listed in a Grid Widget. You could use some Mashup Data Service like GetImplementingThings. That would return a list of every Thing instantiated from a Thing Template. You could then have a Grid which displays a list of every Thing returned by the GetImplementingThings Service. You could then have a Template Mashup stored within every Thing instantiated from that Template. Whenever a Thing is selected from the Grid, it displays the Template Mashup for that specific Thing. HOW DO YOU HIDE TOOLBAR WHICH ALLOW USER TO SELECT "SHOW/HIDE LOG", "SHOW/HIDE LOG", "RELOAD", ETC. ON YOUR MASHUP SCREEN? Many Widgets have a ""Visible"" Property which may be dynamically set via some other criteria. For instance, you could have a Checkbox Widget, which is great for Boolean values like the Visible Property. You could tie the State Property of the Checkbox Widget to a different Widget's Visible Property. When the Checkbox is checked, the other Widget is visible. When the Checkbox is unchecked, then the other Widget becomes invisible. I WOULD LIKE TO KNOW THE CHALLENGES IN RESPONSIVE MASHUPS VS. STATIC MASHUP DEVELOPMENT. ALSO NEED THE LIST OF WIDGETS NOT SUPPORTED BY THE RESPONSIVE MASHUPS. WHAT IS NEW IN THINGWORX 8.2 FOR RESPONSIVE MASHUPS? The main challenge of a Responsive Mashup is that it's almost necessary to test the Mashup at each resolution that you believe your users may be viewing the page. Responsive does a good job of stretching and shrinking… but this can also lead to undesirable situations where you have scroll bars because the viewing-resolution is too small for everything to fit. The main challenge of a Static Mashup is that it really only works at the specific resolution you set. If you have a 300x200 px Static Mashup, it will essentially be unreadable on a 4k display. As for a list, some Widgets *AREN'T* Responsive. The TextBox Widget will not grow and shrink as the viewing-resolution changes, for instance, but you can still use a TextBox in a Responsive Mashup. The big new item for Mashups in 8.2 was the inclusion of CSS and the Collections Widget. Either or both may be used in any Mashup, regardless of whether it's Responsive or Static.

Alerts via Anomaly Detection This documents objective is to provide information and links about alerts used for anomaly detection. This document covers following topics: What Is Anomaly Detection Implementing Anomaly Detection Creating an Anomaly Alert and Prerequisites Anomaly Stats Certainty Parameter Video Example On How To Create An Alert for Anomaly Detection Tips and troubleshooting What Is Anomaly Detection Anomaly Detection in ThingWorx is implemented via built-in ThingWatcher functionality. ThingWatcher detects anomalies by monitoring a data stream from a device, calculating an expected distribution of data, and validating that the current data point is a member of the expected distribution.   Implementing Anomaly Detection Anomaly Detection is enabled by default in ThingWorx. However, several steps are required to configure the functionality for your specific environment, including the prerequisite activities below.   Creating an Anomaly Alert and Prerequisites   Configuring Anomaly Detection to monitor a stream of data. For information about setting up Anomaly Detection, view Preparing ThingWorx for Anomaly Detection. Anomaly Stats Anomaly Alert Statuses moves through several statuses as it works its way through the corresponding phases. Initialized Calibrating Training Buffering Monitoring Failed Certainty Parameter The Certainty Parameter when implementing anomaly detection requires a number of factors to consider. At its most basic, ThingWatcher functionality compares two sets of data, a validation set (collected during the Calibrating phase) and a test dataset (data streaming from a remote device). ThingWatcher tries to determine the likelihood that the distribution of values in the test dataset is from the same distribution of values contained in the validation dataset. The accuracy of the model plays a large role in this determination, but so does the Certainty parameter used for the statistical analysis of the two data sets.   Video Example On How To Create An Alert for Anomaly Detection Anomaly Detection Part 1. Create connectivity between KEPServer and ThingWorx Platform. Anomaly Detection Part 2. Configure Anomaly Alert to bind simulated data coming through KEPServer for Anomaly Detection. Anomaly Detection Part 3. Viewing data via Anomaly Mashup. Tips and troubleshooting Diagnose and fix the most common issues that may be encountered when working with ThingWatcher. It cannot be stressed strongly enough that you should be familiar with your data including the average time interval between data points, and the collection duration and certainty threshold you specified.

This groovy script creates an xml output of the audit log filtered by the User Access category, so dates of when users logged in or logged out. Parameter: days - number of days to search import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.Context import com.axeda.common.sdk.id.Identifier import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.device.Device import com.axeda.drm.sdk.audit.AuditCategoryList import com.axeda.drm.sdk.audit.AuditCategory import com.axeda.drm.sdk.audit.AuditEntryFinder import com.axeda.drm.sdk.audit.SortType import com.axeda.drm.sdk.audit.AuditEntry import groovy.xml.MarkupBuilder /* * AuditEntryList.groovy * * Creates an xml output of the audit log filtered by the User Access category, so dates of when users logged in or logged out. * * @param days        -   (REQ):Str number of days to search. * * @author Sara Streeter <sstreeter@axeda.com> */ def writer = new StringWriter() def xml = new MarkupBuilder(writer) try {     def ctx = Context.getUserContext()     ModelFinder modelFinder = new ModelFinder(ctx, new Identifier(1))     Model model = modelFinder.find()     DeviceFinder deviceFinder = new DeviceFinder(ctx, new Identifier(1))     Device device = deviceFinder.find()     AuditCategoryList acl = new AuditCategoryList()     acl.add(AuditCategory.USER_ACCESS)     long now = System.currentTimeMillis()     Date today = new Date(now)     def paramdays = parameters.days ? parameters.days: 5     long days = 1000 * 60 * 60 * 24 * Integer.valueOf(paramdays)     AuditEntryFinder aef = new AuditEntryFinder(ctx)     aef.setCategories(acl)     aef.setToDate(today)     aef.setFromDate(new Date(now - (days)))     aef.setSortType(SortType.DATE)     aef.sortDescending()     List<AuditEntry> audits = aef.findAll() // assemble the response     xml.Response() {         audits.each { AuditEntry audit ->                   Audit() {                 id(audit?.id.value)                 user(audit?.user?.username)                 date(audit?.date)                 category(audit?.category?.bundleKey)                 message(audit?.message)             }         }     } } catch (def ex) {     xml.Response() {         Fault {             Code('Groovy Exception')             Message(ex.getMessage())             StringWriter sw = new StringWriter();             PrintWriter pw = new PrintWriter(sw);             ex.printStackTrace(pw);             Detail(sw.toString())         }     } } return ['Content-Type': 'text/xml', 'Content': writer.toString()]

When you install Thingworx with PostgreSQL, you can't import the "PostgreSQL" extension because of the conflict of a library file. So, here is a sample "MetaData.xml" file. You can zip this file and simply import it into your Thingworx so that you can have a thing template for "PostgreSQL" database. <Entities>     <ExtensionPackages>         <ExtensionPackage name="PostgreSQL_ExtensionPackage"                       description="PostgreSQL JDBC Extension"                       vendor="ThingWorx Customer Service"                       packageVersion="1.0"                       minimumThingWorxVersion="4.0.0">         </ExtensionPackage>     </ExtensionPackages>     <ThingTemplates>         <ThingTemplate baseThingTemplate="Database" description="PostgreSQL Server" documentationContent="" effectiveThingPackage="" homeMashup="" lastModifiedDate="2015-11-28T11:40:35.355-05:00" name="PostgreSqlServer" tags="" thingPackage="">             <ThingShape description="" documentationContent="" lastModifiedDate="2015-11-28T11:40:35.355-05:00" name="" tags="">                 <PropertyDefinitions/>                 <ServiceDefinitions/>                 <EventDefinitions/>                 <ServiceImplementations/>                 <ServiceMappings/>                 <Subscriptions/>             </ThingShape>             <ImplementedShapes/>             <ConfigurationTables>                 <ConfigurationTable description="" isMultiRow="false" name="ConnectionInfo">                     <DataShape>                         <FieldDefinitions>                             <FieldDefinition aspect.defaultValue="5.0" baseType="NUMBER" description="Maximum number of connections in the pool" name="maxConnections" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="jdbc" baseType="STRING" description="jDBCConnectionURL" name="jDBCConnectionURL" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="SELECT NOW()" baseType="STRING" description="Connection validation string" name="connectionValidationString" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="org.postgresql.Driver" baseType="STRING" description="jDBCDriverClass" name="jDBCDriverClass" ordinal="0"/>                             <FieldDefinition baseType="STRING" description="Database user name" name="userName" ordinal="0"/>                             <FieldDefinition baseType="PASSWORD" description="Database password" name="password" ordinal="0"/>                         </FieldDefinitions>                     </DataShape>                     <Rows>                         <Row>                             <jDBCConnectionURL><![CDATA[jdbc:postgresql://localhost:5432/demo]]></jDBCConnectionURL>                             <maxConnections>100.0</maxConnections>                             <connectionValidationString><![CDATA[SELECT NOW()]]></connectionValidationString>                             <jDBCDriverClass><![CDATA[org.postgresql.Driver]]></jDBCDriverClass>                             <userName />                             <password />                         </Row>                     </Rows>                 </ConfigurationTable>                 <ConfigurationTable description="" isMultiRow="false" name="ConnectionMonitoring">                     <DataShape>                         <FieldDefinitions>                             <FieldDefinition aspect.defaultValue="1.0" baseType="NUMBER" description="Number of retries" name="numberOfRetries" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="2000.0" baseType="NUMBER" description="Retry delay in milliseconds" name="retryDelay" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="false" baseType="BOOLEAN" description="Enable connection monitoring" name="enableMonitor" ordinal="0"/>                             <FieldDefinition aspect.defaultValue="30000.0" baseType="NUMBER" description="Monitor rate in milliseconds" name="connectionMonitorRate" ordinal="0"/>                         </FieldDefinitions>                     </DataShape>                     <Rows>                         <Row>                             <numberOfRetries>1.0</numberOfRetries>                             <retryDelay>2000.0</retryDelay>                             <enableMonitor>false</enableMonitor>                             <connectionMonitorRate>3000.0</connectionMonitorRate>                         </Row>                     </Rows>                 </ConfigurationTable>             </ConfigurationTables>             <avatar/>             <DesignTimePermissions>                 <Create/>                 <Read/>                 <Update/>                 <Delete/>                 <Metadata/>             </DesignTimePermissions>             <RunTimePermissions/>             <InstanceDesignTimePermissions>                 <Create/>                 <Read/>                 <Update/>                 <Delete/>                 <Metadata/>             </InstanceDesignTimePermissions>             <InstanceRunTimePermissions/>         </ThingTemplate>     </ThingTemplates> </Entities>

Key Functional Highlights ThingWorx Manufacturing Apps enhancements Support for NI InsightCM connected to KEPServerEX as an aggregator Controls Advisor usability improvement to retrieve App Key for a specific KEPServerEX connection Asset Advisor usability improvement for displaying alerts Compatibility ThingWorx 8.1.0 KEPServerEX 6.2, 6.3 KEPServerEX V6.1 and older as well as different OPC Servers (with Kepware OPC aggregator) Documentation ThingWorx Manufacturing Apps Setup and Configuration Guide ThingWorx Manufacturing Apps Customization Guide What’s New in ThingWorx Manufacturing Apps 8.1.0 Download Extensions for ThingWorx Manufacturing Apps and Asset Remoting Note: this release announcement applies to the ThingWorx Manufacturing Apps Extensions 8.1.0. For the ThingWorx Manufacturing Apps Freemium (Express) 8.1.0 release notes, see this page: ThingWorx Manufacturing Apps 8.1 Freemium is Available for Download!

This video is the 2nd part, of a series of two videos, walking you through the configuration of Analysis Event which is applied for Real-Time Scoring. This part 2 video will walk you through the configuration of Analysis Event for Real-Time Scoring, and validate that a predictions job has been executed based on new input data.   Updated Link for access to this video:  Analytics Manger 7.4: Configure Analysis Event & Real Time Scoring Part 2 of 2