IoT & Connectivity Tips

Hello readers,   This week @ Ask Kaya we thought we would give someone else the keyboard for a different point of view on the platform.  I’m Chris, a Product Manager here at PTC working on the ThingWorx platform.  Instead of telling you what is coming in our next release, or interviewing one of our awesome PTC experts, I thought I would take a moment to reflect on the platform’s success and dream about where it could be.  After visiting with customers and partners at PTC Forum Europe this week, it looks like many of you share in our vision.  This is a bit of a fun post and by no means an exact look into ThingWorx 2019, but see what you think.   When I think about the next generation of ThingWorx, here is what I see: I see Mashups that generate themselves with suggested visualizations based on your input for style, user persona, and navigation I see Thing Models that populate, based on your use case, your equipment and your connectivity I see a self-learning platform with understanding of all industrial data sources, presenting options of integration to extend knowledge or informing you of correlations I see applications that automatically master individual pieces of equipment, small processes, and handfuls of KPIs and will command larger fleets, networks, and multi-site operations I see a platform without installation or setup, but is there when you need it I see test code and harnesses that are created based on what you build with our tools and tests that run automatically when things are changed I see developers being notified when things are changed by other developers, or when modules from PTC have new versions I see a central place to manage solutions, with push button access for administrators to deploy to sites I see upgrades happening seamlessly, confidently, with no penalty for failure and with the speed of iterative development I see a self-aware system that monitors and scales itself cost effectively   Readers, what do you see?  Sound off in the comments!   Chris

Warning This post is quite long, has various chapters and you might get bored reading it. If you just want a summary read the "Use case" and "Conclusion" chapter - and maybe the "Required Logic" chapter, because I made a cool graph for it. The rest is all about implementation... Introduction I recently had the opportunity to deliver a ThingWorx training for Saint Gobain. One of the use cases for their ThingWorx application is monitoring machine errors and outages on the production line. If an outage or error status is triggered, the machine operator will see a popup on the monitoring screen where he is forced to select a root cause. This root cause will then be persisted in ThingWorx for more data transformation, analytics and reporting - like cost analysis or optimization opportunities. During the training we were also discussing on how such a forced root cause monitoring can be implemented via Mashups and the usage of modal popups. I've compiled the details into this post as it might also interest other developers. The ThingWorx Entities I'm using in this example can be downloaded from here Note: I'm using the word "Alert" here - but not in the context of a ThingWorx Property Alert... just beware to not be confused due to the wording. Use Case One of the requirements for Saint Gobain's IoT Solution was an interactive alert monitoring directly in the factory on the production machines. Let's say the machine has stopped, the root cause should be recorded. For this an interactive popup will be displayed on the machine's monitoring display and an employee has to choose the root cause from a pre-defined list. This could be planned outages, e.g. for maintenance or unplanned outages, e.g. material jam. The root cause will then be recorded and a history of outage causes can be stored in a ThingWorx value stream. This can then be later analyzed with e.g. ThingWorx Analytics capabilities to understand and optimize the machine's production capabilities and efficiency. As the root cause must be entered, the popup will be forced to be displayed when a certain condition / criteria is met - and it will only disappar when a root cause is chosen. The user should not be able to interact with any other elements of the Mashup and not be able to just close the popup. The popup will close itself and reset the initial condition once the root cause has been identified and chosen. Requirements Required Entities Required Logic Note: Just to make it easier to manage and export Entities, I will add all of the created elements in a new Project called RootCausePopups. All of the elements will have a "rcp_" added in front of their name - just to make it easier for me to find and identify them. Implementation Before we start - set a Project Context Create Entities Create a Popup Mashup Create the Main Mashup Testing the Mashups Conclusion Certain conditions (like the state of a checkbox) can be used to trigger modal popups. A modal popup forces a user interaction and the interaction will not offer any other option until a choice is made. With these parameters it's easy to have mandatory reaction from users when it's important to capture data which rely on the analysis of an engineer or a user - e.g. reasons for machine outages. Using this technique there's not much training required for staff, other than pushing a button with an option of their choice - this saves quite some time in capturing data in any other way (e.g. updating Excel files or manual pen-and-paper techniques). As this data is now part of the ThingWorx instance it can be used for further transformation, analysis or just for monitoring purposes There's of course more possibilities when it comes to states and formatting which would exhaust the context of this post - but feel free to explore... In the example we wouldn't need the textbox, but it's there to demonstrate if the correct values are persisted or not In the example we could of course also set the visibility of the checkbox to false, so that we would only see the popup and the Grid holding historic information We could also create different StateDefinitions to color-format / text-format the input differently from the output in the Grid If you found this interesting (and actually made it to the end of this post) - feel free to play with this concept a bit more... The dependencies might seem a bit difficult, but it should be easy to implement and to adjust to your own ideas and requirements.

Previously Installing & Connecting C SDK to Federated ThingWorx with VNC Tunneling to the Edge device Installing and configuring Web Socket Tunnel Extension on ThingWorx Platform Overview     Using the C SDK Edge client configuration we did earlier, we'll now create above illustrated setup. In this C SDK Client we'll push the data to ThingWorx Publisher with servername : TW802Neo to ThingWorx Subscriber servername : TW81. Notice that the SteamSensor2 on the pulisher server is the one binding to the C SDK client and the FederatedSteamThing on subscriber is only getting data from the SteamSensor2. Let's crack on!   Content   Configure ThingWorx to publish Configure ThingWorx to subscribe Publish entities from Publisher to the Subscriber Create Mashup to view data published to the subscriber Pre-requisite Minimum requirement is to have two ThingWorx servers running. Note that both ThingWorx systems can be publisher & subscriber at the same time.   Configure ThingWorx Publisher   Configuring Federation Subsystem   1. Navigate to ThingWorx Composer > Subsystems > Federation Subsystems and configure the following highlighted sections   Essentially its required to configure the Server Identification, i.e. My Server name (FQDN / IP) , My Server Description (optional) Federation subscribers this server publishes to, i.e. all the server you want to publish to from this server. Refer to the Federation Subsystem doc in the Help Center to check detail description on each configurable parameter.   2. Save the federation subsystem   Configuring a Thing to be published   1. Navigate back to the Composer home page and select the entity which you'd like to publish 2. In this case I'm using SteamSensor2 which is created to connect to the C SDK client 3. To publish edit the entity and click on Publish checkbox, like so 4. Save the entity   Configure ThingWorx Subscriber     Configuring Federation Subsystem   1. Navigate to ThingWorx Composer > Subsystems > Federation Subsystems and configure the following highlighted sections   Essentially its required to configure the Server Identification, i.e. My Server name (FQDN / IP) , My Server Description (optional) Refer to help center doc on Federation Subsystems should you need more detail on the configurable parameter If you only want to use this server as a subscriber of entities from the publishing ThingWorx Server, in that case you don't have to Configure the section Federation subscribers this server publishes to, I've configured here anyway to show that both servers can work as publishers and subscribers   2. Save the federation subsystem   Configuring a Thing to subscribe to a published Thing 1. Subscribing to an entity is fairly straight forward, I'll demonstrate by utilizing the C SDK client which is currently pushing values to my remote thing called SteamSensor2 on server https://tw802neo:443/Thingworx 2. I have already Published the StreamSensor2, see above section Configuring a Thing to be published 3. Create a Thing called FederatedStreamThing with RemoteThingWithTunnels as ThingTemplate, 4. Browser for the Identifier and select the required entity to which binding must be done, like so   5. Navigate to the Properties section for the entity, click Manage Bindings to search for the remote properties like so for adding them to this thing:     6. Save the entity and then we can see the values that were pushed from the client C SDK     7. Finally, we can also quickly see the values pulled via a Mashup created in the subscriber ThingWorx , below a is a simple mashup with grid widget pulling values using QueryPropertyHistory service  

Business logic and actions in a ThingWorx application are driven by events.  Events are interesting or critical property states that a Thing publishes to subscribers.  Events are defined at the thing, thing template, or thing shape level, and can be as simple as a new data value from a device, to complex events from many data points.  An event that is created on a thing shape or thing template level is inherited down into the entities that implement the shape or template that defines the event.   Types of Events Built-in Custom   Built-in Events Every entity in ThingWorx can use several built-in events.  These events are automatically triggered when a prerequisite condition is met.  In some cases, you must provide information that determines the specifics of that prerequisite.   Common built-in events include: Data Change Alert Timer  In ThingWorx, there are standard events and related data packets (defined by Data Shapes).  The most common type of event is a Data Change related to a Thing property.  The Data Change fires when a property value changes. The specifics of this event can be configured from the Data Change Info section of the Property Creation page.  When you define a property, there are many configuration aspects.   Example:   Using DataChangeEvent, you have a few options for setting an event to fire when there is new data for a property: only if the data has changed only if the data evaluates true or false only if the new value changed beyond a defined threshold   The Alert event is also attached to a property and is set up from the Property Definition page.  An Alert event fires when the value of the property reaches a certain threshold or value.  Depending on the base type, you may specify the condition of this event from the Manage Alerts button.   Timer events can be used to run jobs or fire events on a regular basis.  Things themselves can fire specific events such as Thing Start or “special” things, like a Timer type thing that contains special event(s).   How to Set up and Configure Timers   Creating a Custom Event To create a new custom event:   In ThingWorx Composer, open up the Thing, Thing Template, or Thing Shape for edit where the new Event will be added From the sidebar menu under Entity Information, select Events Click Add My Event Provide the event with a name and a data shape that describes the data being sent to a subscriber Click Done and Save   Once created, the event appears on the Subscriptions page for subscribing.  Since the event is custom built, you must specify when it triggers.  This is often accomplished from within a service.   Best Practices There should be a subscriber to the event in the model. The subscriber is sent a data packet and the subscription is initiated. If no one is subscribed to the event (no one is listening), nothing happens. DataChange events hit the event processing subsystem (see Event Processing Subsystem below), which has fewer threads allocated to it than some other subsystems: If queries within subscriptions to data change events take too long, this will cause a back-up in the event processing subsystem Too much activity in this subsystem may result in poor system performance, and even server crashes Be cautious what types of calculations are performed in data change event subscriptions Refer to the following knowledge base article regarding tricky data change event use cases, the implementation of rules for alerts, and conditional operations:  Complicated Event Calculations in ThingWorx Load Test all applications in Sandbox before committing changes to Production to ensure mashups can handle all design choices on large scales   Event Processing Subsystem The Event Processing Subsystem manages event processing for external subscriptions (Things subscribing to other Things) throughout ThingWorx.   Event Queue Processing Settings Base Type Default Notes Min Threads Allocated to Event Processing Pool NUMBER 16   Max Threads Allocated to Event Processing Pool NUMBER 500   Max Queue Entries Before Adding New Working Thread NUMBER 200000 Maximum number of entries to queue before adding a thread to the pool     Additional Information PTC ThingWorx Help Center - Thing Events Event Types Creating and Triggering Custom Events in ThingWorx  

Get Started with ThingWorx for IoT Guide Part 1   Overview   This project will introduce you to the principles of ThingWorx Foundation by creating a working web application. Following the steps in this guide, you will create the building blocks of your first application for the Internet of Things (IoT). You will use ThingWorx Composer to create Thing Templates, which are then used to create Things that model the application domain. A simulator is imported to generate time-series data that is saved to a Value Stream. After modeling the application in ThingWorx Composer, you'll use Mashup Builder to create the web application Graphical User Interface (GUI). No coding is required in modeling the application, or in composing the web GUI that displays dynamically-generated data. NOTE: This guide’s content aligns with ThingWorx 9.3. The estimated time to complete ALL 5 parts of this guide is 30 minutes.      Step 1: Data Model   Model-based design with reusable building blocks makes your applications scalable and flexible. A ThingWorx application is built from Things, each based on a Thing Template that defines the common Properties (characteristics) and Services (behaviors) for a set of entities. Once a Thing Template is created, you can easily instantiate multiple Things without duplicating effort. In this tutorial, we will develop an application for a house including a thermostat, an electrical meter, and a sensor data simulator. We will demonstrate how to capture, store, and visualize data using the ThingWorx Foundation Server.   You will create Thing Shapes that model both a thermostat and an electric meter. You will then create a Thing Template that represents a house based on these shapes and other Properties.   Step 2: Create Thing Shapes Thing Shapes are components that contain Properties and Services. In Java programming terms, they are similar to an interface. In this section, you will build Thing Shapes for an electric meter and a thermostat. Meter Start on the Browse, folder icon tab of ThingWorx Composer. Under the Modeling section of the left-hand navigation panel hover over Thing Shapes, then click the + button.   Type MeterShape in the Name field. NOTE: Thing Shape names are case sensitive   If Project is not already set, choose PTCDefaultProject. Click Save. Add Properties Click Properties and Alerts tab at the top of your shape.   Click + Add. Enter the property name from the first row of the table below into the Name field of the Thing Shape Name Base Type Persistent? Logged? meterID STRING X   currentPower NUMBER   X costPerKWh NUMBER X X currentCost NUMBER     Select the Base Type from the drop-down menu that is listed in the table next to the Property name.   Check Persistent and/or Logged if there is an X in the table row of the Property. NOTE: When Persistent is selected, the property value will be retained when a Thing is restarted. Properties that are not persisted will be reset to the default during a restart. When Logged is selected, every property value change will be automatically logged to a specified Value Stream. Click ✓+ button. TIP: When adding multiple properties at once, click Done and Add after each, once you've entered a Name, selected a Base Type and any other criteria. If adding a single property, click Done. Repeat steps 2 through 6 for each of the properties in the rows of the table. Click the done ✓ Button. You'll see that these Properties have been created for the Meter Thing Shape.   Click Save. Thermostat This time we will use a shortcut to create a Thing Shape. In the top, left of the screen you will find +, click the new entity icon, then select Thing Shape from the list.   TIP: This is a shortcut you can use to create anything you can access from the Home tab in Composer. Type ThermostatShape in the Name field. If Project is not already set, choose PTCDefaultProject. Select the Properties and Alerts tab at the top. Click + Add and create the following properties following the same steps as before: Name Base Type Persistent? Logged? thermostatID STRING X   temperature NUMBER X X setTemperature NUMBER X X message STRING   X Click Save. You'll see that these Properties have been created for the Thermostat Thing Shape.       Step 3: Create Thing Template You can create reusable building blocks called Thing Templates in ThingWorx to maintain scalability and flexibility of your application development. With Thing Templates you define a set of similar objects by specifying the Properties (characteristics) and Services (behaviors) that are common for all the objects. In Java programming terms, a Thing Template is like an abstract class and can be created by extending other Thing Templates. Once a Thing Template is defined and saved in ThingWorx Foundation Server, you can replicate multiple Things to model a complete set without duplicating effort. In this step, you will create a Thing Template that defines properties for a building. This building Template could be used to create multiple Things that each represent a specific home, business, or other building structure. Start on the Browse, folder icon tab on the far left of ThingWorx Composer.   Under the Modeling section of the left-hand navigation panel, hover over Thing Templates and click the + button Type BuildingTemplate in the Name field. NOTE: Thing Template names are case sensitive If Project is not already set, click the + in the Project text box and select the PTCDefaultProject. In the Base Thing Template box, click + to choose GenericThing as the Template.   In the Implemented Shapes field, click the + to select the MeterShape Thing Shape.   Click Save. Add Properties In this step, you will specify the Properties that represent the characteristics of a building. Some Properties like the building location may never change (static), while other properties like power and temperature information may change every few seconds (dynamic). Select the Properties and Alerts tab under Thing Template: BuildingTemplate.   Click the Edit button if the Template is not already open for editing, then click + Add next to My Properties. Enter the property name in the Name field copied from a row of the table below, Select the Base Type of the property from the drop down menu. Check Persistent and/or Logged if there is an X in the table row of the Property. NOTE: When Persistent is selected, the property value will be retained during a system restart. Properties that are not persisted will be reset to the default during a system restart. When Logged is selected, every property value change will be automatically logged to a specified Value Stream. Click the ✓+ button. TIP: When adding multiple properties at once, click Check+ after each, once you've entered a Name, selected a Base Type and any other criteria. If adding a single property, click Check button. Repeat steps 3 through 6 for each of the properties in the rows of the table. Name Base Type Persistent Logged buildingID STRING x   building_lat_long LOCATION x   watts NUMBER x x After entering the final property, click the ✓ button. Click Save. You should see the following properties in your Composer.   In the next part of this introductory exercise, we will create a single Thing based on this Template to represent a house.

Install ThingWorx Kepware Server Guide    Overview   This guide will walk you through the steps to install ThingWorx Kepware Server. NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete this guide is 30 minutes.    Step 1: Learning Path Overview   This guide is the first on the Connect and Monitor Industrial Plant Equipment Learning Path, and it explains how to get up and running with ThingWorx Kepware Server. If you want to learn to install ThingWorx Kepware Server, this guide will be useful to you, and it can be used independently from the full Learning Path. In the next guide on the Learning Path, we will create an Application Key which is used to secure the connection between Kepware Server and ThingWorx Foundation. Later in the Learning Path, we will send information from ThingWorx Kepware Server into ThingWorx Foundation. In other guides in this Learning Path, we will use Foundation's Mashup Builder to construct a website dashboard that displays information from ThingWorx Kepware Server. We hope you enjoy this Learning Path.   Step 2: Install ThingWorx Kepware Server   ThingWorx Kepware Server includes over 150 factory-automation protocols. ThingWorx Kepware Server communicates between industrial assets and ThingWorx Foundation, providing streamlined, real-time access to OT and IT data — whether that data is sourced from on-premise web servers, off-premise cloud applications, or at the edge. This step will download and install ThingWorx Kepware Server. Download the ThingWorx Kepware Server executable installer. Select your Language and click OK 3. On the "Welcome" screen, click Next.        4. the End-User License Agreement and click Next.   5. Set the destination folder for the installation and click Next.   6. Set the Application Data Folder location and click Next. Note that it is recommended NOT to change this path. 7. Select whether you'd like a Shortcut to be created and click Next. 8. On the "Vertical Suite Selection" screen, keep the default of Typical and click Next. 9. On the "Select Features" screen, keep the defaults and click Next. 10. The "External Dependencies" screen simply lists everything that will be installed; click Next. 11. On the "Default Application Settings" screen, leave the default of Allow client applications to request data through Dynamic Tag addressing and click Next. 12. On the “User Manager Credentials” screen, set a unique strong password for the Administrator account and click Next. Note that skipping setting a password can leave your system less secure and is not recommended in a production environment. 13. Click install to begin the installation. 14. Click finish to exit the installer.     Step 3: Open ThingWorx Kepware Server   Now that ThingWorx Kepware Server is installed, you will need to open it. In the bottom-right Windows Taskbar, click Show hidden icons. 2. Double-click on the ThingWorx Kepware Server icon. 3. ThingWorx Kepware Server is now installed. 4. For additional information on ThingWorx Kepware Server, click Server Help on the Menu Bar.   Step 4: Next Steps   Congratulations! You've successfully completed the Install ThingWorx Kepware Server guide. In this guide, you learned how to:   Download, install, and open ThingWorx Kepware Server   The next guide in the Connect and Monitor Industrial Plant Equipment learning path is Connect Kepware Server to ThingWorx Foundation.    The next guide in the Using an Allen-Bradley PLC with ThingWorx learning path is Connect to an Allen-Bradley PLC. . 

  Create An Application Key Guide   Overview   In order for a device to send data to the Platform, it needs to be authenticated. One authentication method is to use an Application Key. Application Keys, or appKeys, are security tokens used for authentication in ThingWorx. They are associated with a given User and have all of the permissions granted to the User with which they are associated. This is one of the most common ways of assigning permission control to a connected device. NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete this guide is 30 minutes.    Step 1: Learning Path Overview   This guide explains the steps to create a ThingWorx Application Key, and is part of a Learning Path. You can use this guide independently from the full Learning Path. If you want to learn to create a ThingWorx Application Key, this guide will be useful to you. When used as part of the Industrial Plant Learning Path, you should already have installed ThingWorx Kepware Server. We will use the Application Key to send information from ThingWorx Kepware Server into ThingWorx Foundation. Other guides demonstrate Foundation's Mashup Builder to construct a website dashboard that displays information from ThingWorx Kepware Server. We hope you enjoy this Learning Path.   Step 2: Create Application Key   Application Keys are assigned to a specific User for secure access to the platform. Using the Application Key for the default User (Administrator) is not recommended. If administrative access is absolutely necessary, create a User and place the User as a member of the SecurityAdministrators and Administrators User groups. Create the User the Application Key will be assigned to. 1. On the Home screen of Composer click + New. 2. In the dropdown list, click Applications Key. 3. Give your application key a name (i.e. MyAppKey). 4. If Project is not already set, click the + in the Project text box and select the PTCDefaultProject. 5. Set the User Name Reference to a User you created. 6. Update the Expiration Date field, otherwise it will default to 1 day. 7. Click Save. A Key ID has been generated and can be used to make secure connections.   IP Whitelisting for Application Keys   One of the features of an Application Key is the ability to set an IP whitelist. This allows the server to specify that only certain IP addresses should be able to use a given Key ID for access. This is a great way to lock down security on the platform for anything that will maintain a static IP address. For example, connected Web-based business systems may have a static IP from which all calls should be made. Similarly, you can use wildcards to specify a network to narrow the range of IP addresses allowed while still offering some flexibility for devices with dynamic IP addresses. Extremely mobile devices should likely not attempt to implement this, however, as they will often change networks and IP addresses and may lose the ability to connect when the IP whitelist feature is used.   Interact with Application Keys Programmatically Service Name Description GetKeyID Returns the ID of this Application Key GetUserName Get the username associated with this Application Key IsExpired Returns if this Application Key is expired ResetExpirationDateToDefault Resets the expiration date of the Application Key to the default time based on configuration in the UserManagement subsystem SetClientName Sets the client name for this Application Key SetExpirationDate Sets the expiration date of this Application Key to a provided date SetIPWhiteList Sets the values for the IP whitelist for this Application Key SetUserName Sets the associated user name for this Application Key   Tip: To learn more about Application Keys, refer to the Help Center   Step 3: Next Steps   Congratulations! You have successfully created an application key. We hope you found this guide useful.     The next guide in the Connect and Monitor Industrial Plant Equipment learning path is Install ThingWorx Kepware Server.    The next guide in the Azure MXChip Development Kit learning path is Connect Azure IoT Devices.   The next guide in the Medical Device Service learning path is Use the Edge MicroServer (EMS) to Connect to ThingWorx.   The next guide in the Using an Allen-Bradley PLC with ThingWorx learning path is Model an Allen-Bradley PLC.

  The latest release of ThingWorx, version 9.7, brings powerful updates across performance, scalability, security, and developer productivity. Designed to meet the evolving demands of IoT and AR solutions, this version equips businesses with the tools to build smarter applications, improve operational efficiency, and stay ahead of the curve.   Performance & Scalability ThingWorx 9.7 introduces key updates that enhance performance and scalability for large-scale operations. The addition of partitioned value streams enables faster queries and purging of massive data sets, streamlining data management. Lazy loading for grid components optimizes application responsiveness, ensuring a seamless user experience, even in data-intensive environments. Additionally, the new Data Ordering feature ensures that ingested data is processed in the exact order it is received, even across asynchronous systems. This capability guarantees accurate execution of application logic and precise end calculations, addressing critical needs in industries requiring serialized data processing.   Reliability Enhancements Reliability is critical in enterprise applications, and ThingWorx 9.7 delivers with automated disaster recovery, providing quick failover mechanisms for deployments on the PTC Cloud. Enhanced diagnostics and tools for high-availability environments address challenges such as large cache collections, helping maintain stability and preventing bottlenecks in mission-critical systems.   Developer Productivity ThingWorx 9.7 offers developers a range of productivity-boosting features. The Mashup Builder enhancements include binding filtering, container zooming, and intuitive design tools that make application development faster and more efficient. Grid customizations, like inline editing and bulk selection, further streamline workflows. Developers also benefit from the latest Eclipse plugin, ensuring compatibility with modern tools and technologies.   Security & Compliance Security continues to be our highest priority in this release. ThingWorx 9.7 fully supports Java 21, delivering enhanced memory management and access to the latest security patches. TLS 1.3 Phase-2 integration brings advanced encryption protocols to secure communication across platforms. Additional security measures, such as granular file permissions, give administrators more control over sensitive data and operations.   Industry-Specific Solutions ThingWorx 9.7 introduces several innovative features tailored for specific industries: Route Versioning in Connected Work Cells: Manufacturers can now revise and track work cell routes with precision, creating complete traceability for serial numbers, routes, and associated work instructions. This feature streamlines workflows, enhances traceability, and boosts operational efficiency. Enhanced Operational Insights: The release includes new key performance indicators (KPIs) and reporting tools to deliver actionable insights at the factory line, site, and enterprise levels. These updates help organizations measure efficiency, reduce downtime, and improve overall productivity. Asset Muting for Chatty Devices: In scenarios where connected devices become overly chatty—sending excessive or unnecessary data—administrators can now mute these assets. This prevents data overload and mitigates risks such as distributed denial-of-service (DDoS) attacks, optimizing platform performance and ensuring uninterrupted operations. Improved Deployment Reliability: Enhancements in software content management ensure reliable and scalable management of large-scale deployments, reducing downtime and improving device performance.   Third-Party Library Upgrades ThingWorx 9.7 includes key updates to its underlying technology stack, ensuring compatibility with modern enterprise systems, improved performance, and robust security. To stay future-ready, we have updated its core technology stack, including Tomcat 9.0.95, Ignite 2.16 and RabbitMQ 3.13.7 to highlight few.   Why Upgrade to ThingWorx 9.7? ThingWorx 9.7 sets a new benchmark for IoT platforms, offering enhanced scalability, state-of-the-art security, and intuitive tools for developers. The updates cater to both enterprise-scale deployments and industry-specific challenges, ensuring that businesses can innovate and operate with confidence in an increasingly complex digital landscape.   Upgrade to ThingWorx 9.7 today and unlock new possibilities for innovation and success! View release notes here for more details on exact updates and be sure to upgrade to 9.7! Vineet Khokhar Principal Product Manager- IOT Security  

In the evolving landscape of software development, ensuring support for the latest, most secure versions of programming languages is essential. At PTC, we continuously evaluate our technology stack, and Java is no exception. As part of our ongoing commitment to providing secure and high-performing products, we’re announcing some important updates to the Java support plans for ThingWorx.   Current Java Support in ThingWorx (Through Version v9.1.X - v9.6.X)   As of ThingWorx v9.6, Java 11 is the only supported version. This version has been a mainstay of our IoT platform, ensuring stability and performance across various use cases. However, Java 11 entered Extended Support in September 2023, meaning its standard support phase has ended. While this version will continue to receive security updates for a while, its lifecycle is winding down.   Introducing Java 21 Support in ThingWorx v9.7 (Planned Release: December 2024)   With ThingWorx 9.7, releasing in December 2024, we will introduce support for Java 21, the next Long-Term Support (LTS) version of Java. This upgrade brings key benefits, including improved performance, enhanced garbage collection, and increased security, ensuring that ThingWorx remains optimized for enterprise-scale IoT deployments. (More details: The Arrival Of JAVA 21) Given the diversity of our customer base, we know that some are still using Java 11, while others are ready to move to Java 21. ThingWorx 9.7 will support both versions, allowing customers the flexibility to upgrade to the latest ThingWorx version while preparing their environments for Java 21.   The Road to Java 21-Only: What to Expect in ThingWorx v10.0 (Planned Release: June 2025)   As we assess the adoption of Java 21 following the ThingWorx 9.7 release, our goal is to phase out support for Java 11 with ThingWorx 10.0, scheduled for release in June 2025. Starting with ThingWorx 10, Java 21 will be the only supported version, marking the end of Java 11 support for the core platform.   This is driven by the need to stay aligned with modern standards and best practices, including support for third-party technologies such as Tomcat v10 and Spring Framework v6, which require latest Java versions. These updates will ensure that ThingWorx continues to benefit from the latest advancements in the Java ecosystem. Next steps for ThingWorx users   As we approach the release of ThingWorx 9.7, we encourage customers to begin planning for the move to Java 21. While ThingWorx 9.7 will support both Java 11 and Java 21, we recommend upgrading to Java 21 to take full advantage of the enhancements it offers. For more detailed information on overall third party support, do check Release Advisor Vineet Khokhar Principal Product Manager, IoT Security   Stay tuned for more updates as we approach the release of ThingWorx v9.7, and as always, in case of issues, feel free to reach out to <support.ptc.com>  This post on ThingWorxTM status & roadmap is a preliminary version and not subject to your license agreement or any other agreement with ThingWorx. This post contains intended strategies, developments, and functionalities of the ThingWorxTM product. The information is furnished for information use only and is not intended to be binding upon ThingWorx to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by ThingWorx at any time without notice; accordingly, you should not rely on this data for production or purchasing decisions. ThingWorx assumes no responsibility for errors or omissions in this document.

Hello everyone,   Following a recent  experience, I felt it was important to share my insights with you. The core of this article is to demonstrate how you can format a Flux request in ThingWorx and post it to InfluxDB, with the aim of reporting the need for performance in calculations to InfluxDB. The following context is renewable energy. This article is not about Kepware neither about connecting to InfluxDB. As a prerequisite, you may like to read this article: Using Influx to store Value Stream properties from... - PTC Community     Introduction   The following InfluxDB usage has been developed for an electricity energy provider.   Technical Context Kepware is used as a source of data. A simulation for Wind assets based on excel file is configured, delivering data in realtime. SQL Database also gather the same data than the simulation in Kepware. It is used to load historical data into InfluxDB, addressing cases of temporary data loss. Once back online, SQL help to records the lost data in InfluxDB and computes the KPIs. InfluxDB is used to store data overtime as well as calculated KPIs. Invoicing third party system is simulated to get electricity price according time of the day.   Orchestration of InfluxDB operations with ThingWorx ThingWorx v9.4.4 Set the numeric property to log Maintain control over execution logic Format Flux request with dynamic inputs to send to Influx DB  InfluxDB Cloud v2 Store logged property Enable quick data read Execute calculation Note: Free InfluxDB version is slower in write and read, and only 30 days data retention max.     ThingWorx model and services   ThingWorx context Due to the fact relevant numeric properties are logged overtime, new KPIs are calculated based on the logged data. In the following example, each Wind asset triggered each minute a calculation to get the monetary gain based on current power produced and current electricity price. The request is formated in ThingWorx, pushed and executed in InfluxDB. Thus, ThingWorx server memory is not used for this calculation.   Services breakdown CalculateMonetaryKPIs Entry point service to calculate monetary KPIs. Use the two following services: Trigger the FormatFlux service then inject it in Post service. Inputs: No input Output: NOTHING FormatFlux_CalculateMonetaryKPI Format the request in Flux format for monetary KPI calculation. Respect the Flux synthax used by InfluxDB. Inputs: bucketName (STRING) thingName (STRING) Output: TEXT PostTextToInflux Generic service to post the request to InfluxDB, whatever the request is Inputs: FluxQuery (TEXT) influxToken (STRING) influxUrl (STRING) influxOrgName (STRING) influxBucket (STRING) thingName (STRING) Output: INFOTABLE   Highlights - CalculateMonetaryKPIs Find in attachments the full script in "CalculateMonetaryKPIs script.docx". Url, token, organization and bucket are configured in the Persitence Provider used by the ValueStream. We dynamically get it from the ValueStream attached to this thing. From here, we can reuse it to set the inputs of two other services using “MyConfig”.   Highlights - FormatFlux_CalculateMonetaryKPI Find in attachments the full script in "FormatFlux_CalculateMonetaryKPI script.docx". The major part of this script is a text, in Flux synthax, where we inject dynamic values. The service get the last values of ElectricityPrice, Power and Capacity to calculate ImmediateMonetaryGain, PotentialMaxMonetaryGain and PotentialMonetaryLoss.   Flux logic might not be easy for beginners, so let's break down the intermediate variables created on the fly in the Flux request. Let’s take the example of the existing data in the bucket (with only two minutes of values): _time _measurement _field _value 2024-07-03T14:00:00Z WindAsset1 ElectricityPrice 0.12 2024-07-03T14:00:00Z WindAsset1 Power 100 2024-07-03T14:00:00Z WindAsset1 Capacity 150 2024-07-03T15:00:00Z WindAsset1 ElectricityPrice 0.15 2024-07-03T15:00:00Z WindAsset1 Power 120 2024-07-03T15:00:00Z WindAsset1 Capacity 160   The request articulates with the following steps: Get source value Get last price, store it in priceData _time ElectricityPrice 2024-07-03T15:00:00Z 0,15 Get last power, store it in powerData _time Power 2024-07-03T15:00:00Z 120 Get last capacity, store it in capacityData _time Capacity 2024-07-03T15:00:00Z 160 Join the three tables *Data on the same time. Last values of price, power and capacity maybe not set at the same time, so final joinedData may be empty. _time ElectricityPrice Power Capacity 2024-07-03T14:00:00Z 0,15 120 160 Perform calculations gainData store the result: ElectricityPrice * Power _time _measurement _field _value 2024-07-03T15:00:00Z WindAsset1 ImmediateMonetaryGain 18 maxGainData store the result: ElectricityPrice * Capacity lossData store the result: ElectricityPrice * (Capacity – Power) Add the result to original bucket   Highlights - PostTextToInflux Find in attachments the full script in "PostTextToInflux script.docx". Pretty straightforward script, the idea is to have a generic script to post a request. The header is quite original with the vnd.flux content type Url needs to be formatted according InfluxDB API     Well done!   Thanks to these steps, calculated values are stored in InfluxDB. Other services can be created to retrieve relevant InfluxDB data and visualize it in a mashup.     Last comment It was the first time I was in touch with Flux script, so I wasn't comfortable, and I am still far to be proficient. After spending more than a week browsing through InfluxDB documentation and running multiple tests, I achieved limited success but nothing substantial for a final outcome. As a last resort, I turned to ChatGPT. Through a few interactions, I quickly obtained convincing results. Within a day, I had a satisfactory outcome, which I fine-tuned for relevant use.   Here is two examples of two consecutive ChatGPT prompts and answers. It might need to be fine-tuned after first answer.   Right after, I asked to convert it to a ThingWorx script format:   In this last picture, the script won’t work. The fluxQuery is not well formatted for TWX. Please, refer to the provided script "FormatFlux_CalculateMonetaryKPI script.docx" to see how to format the Flux query and insert variables inside. Despite mistakes, ChatGPT still mainly provides relevant code structure for beginners in Flux and is an undeniable boost for writing code.  

Check out this new framework to achieving digital manufacturing success. Learn about the top 3 areas teams need to consider!    Identify a unified end goal Align it with the most impactful use cases Formulate a lasting strategy that resonates their long-term vision Discover More! 

Tune in to The Lean Manufacturer podcast where expert guests bring their outside view of the IIoT and discuss various aspects of manufacturing. Over the course of the series, we’ll cover some of the most important ways the IIoT can maximize manufacturing efficiency and bring value to your organization, including the need for reducing planned and unplanned downtime, enabling operational efficiency, ensuring digital continuous improvement, and so much more.      

PTC was recognized an outright leader in the global IoT market.

Note: The following tutorial are based on a Thingworx/CWC 9.5. Steps and names may differ in another version. Context As a human reaction, the tracked time displayed may be misperceived by the Operator. It can lead to a reject of the solution. CWC doesn’t have (yet?) the capability to configure the visibility to hide the timer. The purpose of this tutorial is to create a quick and straight to the point customization to hide the timer in the execution screen. All other features, services and interfaces are left untouched.   As a big picture, here are the 6 modifications you will need to do: Modify the 4 mashups Modify 2 values in 2 tables of the MSSQL database   Status The mashup containing the timer is PTC.FSU.CWC.Execution.Overview_MU. It is easy to duplicate it and hide the timer widget (switch the visible property to false). But now, how to set it in the standard interface? In order to do it, you need to duplicate the mashups linked to the Execution.Overview_MU mashup. PTC.FSU.CWC.Execution.Overview_MU is directly referenced by the following entities: PTC.FSU.CWC.Authoring.Preview_MU PTC.FSU.CWC.Execution.WorkInstructionStart_MU PTC.FSU.CWC.GIobalUI.ApplicationSpecificHeader_HD PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP   Customization Duplicate all those mashups except Authoring.Preview_MU because we will focus only on the authoring side of CWC. Hereafter it will be called the same as the original + _DUPLICATE. Perform the following modifications.   Open PTC.FSU.CWC.GlobalUI.ApplicationSpecificHeader_HD_DUPLICATE, then in Functions: open the expression named NavigateToStationSelection. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP_DUPLICATE open the validator named ShowRaiseHand. Change the name of the 2 mashups to the relevant ones, example: PTC.FSU.CWC.Execution.WorkInstructionStart_MU_DUPLICATE and PTC.FSU.CWC.Execution.Overview_MU_DUPLICATE open the validator named ShowStationSelection. Change the name of the 2 mashups to the relevant ones, example: PTC.FSU.CWC.Execution.WorkInstructionStart_MU_DUPLICATE   Open PTC.FSU.CWC.Execution.Overview_DUPLICATE, then in Functions: open the expression named SetMashupToWorkInstructionStart. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.Execution.WorkInstructionStart_MU_DUPLICATE open the expression named SetMashupToStationSelection. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP_DUPLICATE   Open PTC.FSU.CWC.Execution.WorkInstructionStart_MU_DUPLICATE, then in Functions: open the expression named NavigateToStart. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.Execution.Overview_MU_DUPLICATE open the expression named NavigateToStationSelection. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP_DUPLICATE   Open PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP_DUPLICATE, in functions: open the expression named NavigateToStart. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.Execution.Overview_MU_DUPLICATE open the expression named NavigateToWorkInstructionStart. Change the name of the mashup to the relevant one, example: PTC.FSU.CWC.Execution.WorkInstructionStart_MU_DUPLICATE   Now, let’s change the database value. In MSSQL, navigate to the thingworxapps database, and edit the dbo.menu table. Look the line for AssemblyExecution (by default line 22) and look the value in column targetmashuplink. Switch the original value PTC.FSU.CWC.WorkDefinitionExecution.StationSelectionContainer_EP to the name of the duplication of this mashup. Lastly, edit the dbo.menucontext table. Look the line related to CWC (application UID = 5) and look the value in column targetmashuplink. Switch the original value PTC.FSU.CWC.GlobalUI.ApplicationSpecificHeader_HD to the name of the duplication of this mashup.   Result After this modification, you can start and check an operation. You should see the following result:

As of May 24, 2023, ThingWorx 9.4.0 is available for download!  Here are some of the highlights from the recent ThingWorx release.   What’s new in ThingWorx 9.4? Composer and Mashup Builder  New Combo chart and Pie chart based on modern Web Components architecture along with several other widget enhancements such as Grid toolbar improvements for custom actions, highlighting newly added rows for Grid widget and others Ability to style the focus on each widget when they are selected to get specific styling for different components Absolute positioning option with a beta flag to use by default is available now Several other Mashup improvements, such as Export function support is made available; improved migration dialogue (backported to 9.3 as well) to allow customers to move to new widgets; and legacy widgets language changes   Foundation  Heavily subscribed events could now be distributed across the ThingWorx HA cluster nodes to be scaled horizontally for better performance and processing of ThingWorx subscriptions. Azure Database for PostgreSQL Flex Server GA support with ThingWorx 9.4.1 for customers deploying ThingWorx Foundation on their own Azure Cloud infrastructure Improved ThingWorx APIs for InfluxDB to avoid Data loss at a high scale with additional monitoring metrics in place for guardrails Several security fixes and key third-party stack updates   Remote Access and Control (RAC) Remote Service Edge Extension (RSEE) for C-SDK, currently under Preview and planned to be Generally Available (GA) by Sept 2023, would allow ThingWorx admins to use Remote Access and Control with the C-SDK based connected devices to use Global Access Server (GAS) for a stable, secure, and large number of remote sessions With 9.4, a self-signed certificate or certificate generated by the trusted certificate authority can be used for RAC with the ThingWorx platform Ability to use parameter substitution to update Auto Launch commands dynamically based on data from the ThingWorx platform is now available   Software Content Management  Support for 100k+ assets with redesigned and improved performance on Asset Search Better control over package deployment with redesigned package tracking, filtering, and management Improved overall SCM stability, performance, and scalability and a more user-friendly and intuitive experience   Analytics  Improvements to the scalability of property transforms to enable stream processing of the larger number of properties within a ThingWorx installation Refactoring of Analytics Builder UI to use updated widgets and align with the PTC design system Updates to underlying libraries to enable the creation and scoring of predictive models in the latest version of PMML (v4.4) End of Support for Analytics Manager .NET Agent SDK     View release notes here and be sure to upgrade to 9.4!     Cheers, Ayush Tiwari Director, Product Management, ThingWorx      

  Dev Ops is a crucial process that exists in any software setting, whether you plan on it or not. Chaos in the dev ops process, say because less time is spent here than on the shiny new features that are easy to sell, results in bottlenecks in the dev ops process. Bottlenecks reduce efficiency, and leave you open to vulnerabilities as well. The faster you can get a change properly tested and safely into production, the safer and more stable the system is all along.   Issues will arise, they always arise. Are you ready for them? Watch this video, see some of these additional links, and think about your dev ops process now, before the fires start!   Useful Links:  ThingWorx Monitoring and Alerting Using Prometheus and Grafana, Part 1 ThingWorx Monitoring and Alerting Using Prometheus and Grafana, Part 2 Overview of Monitoring Tools and Diagnostics The System Health Timer

Architecting Reason Code Trees in DPM Tori Firewind, IoT EDC   What are Machine Codes? Factory hardware devices communicate status changes to their human operators and other machines (IoT) via machine codes. The manufacturers often determine the machine codes for different types of factory hardware, so those are often pre-determined. However, how the reason trees map these machine codes to corresponding business logic in ThingWorx is entirely customizable. Knowing the best way to design your reason trees for this purpose can be challenging, so this guide is here to help with your conceptual knowledge. Using the UI to create, edit, and configure reason codes in technical detail can be found in the Help Center.    The Tree Trunk At the highest level of the reason tree, the trunk, there are really 3 categories: Availability (A), Performance or Productivity (P), and Quality (Q). These should look familiar; they are the three dimensions of OEE (Overall Equipment Effectiveness). Fg 1. Calculation of OEE Availability refers to long stops, events that stop planned production long enough that it makes sense to track a reason for being down (typically several minutes, but the threshold between a long stop and a short stop can vary depending on the ideal rate of production of materials).                  Availability = Run Time / Planned Production Time   Productivity/Performance really refers to short stops, things that cause the machine to run at less-than- optimal speeds. This can include stops caused by running out of materials for production, doing minor maintenance like switching out a single, easily-changed part, or even frequent breaks due to ill health of an operator. User error can be a cause as well, say if the machine needs a certain heat to produce parts, and the heat keeps fluctuating (requiring the machine to take the time to calibrate for this before starting on production) because operators are smoking out a back door or adjusting thermostat temperatures. Fg 2. Levels of Runnable Time   Operator influence often is a factor when it comes to the conditions that permit optimal performance from machinery, and every factory may face different challenges. Stops like these are not really outages; the amount of downtime isn’t enough to consider the production block entirely unproductive. Production was continuing and ongoing throughout most of the block despite the issues; the rate was just slower than ideal.                  Performance/Productivity = (Total Count / Run Time) / Ideal Run Rate   Quality refers mostly to the number of items that are considered scrap or rework, and it can be split into two categories: start up scrap (that which is expected because the machine is in the process of warming up or being fine-tuned by the operator) and production scrap (things which come out wrong and must be tossed or reworked because the conditions under which they were produced weren’t ideal; this is called first-pass yield only, meaning it's only a "good" product if it passes inspection the first time).                  Quality = Good Count / Total Count​   The Branches and the Leaves of the Tree The “leaves” are the reason codes which directly map to machine codes , and the “branches” are the method of categorization that connects them to the trunk. Both the leaves and the trees, the children and the parent nodes of the tree, are split into two states: planned versus unplanned downtime. Changeovers, maintenance, and even scrap, can be broken down into this dichotomy.   For scrap, there are startup rejects (planned, because the machines have ramp up periods) and production rejects (unplanned, because the conditions weren’t ideal). For maintenance there is planned and unplanned, small changes that occur on the fly that result in productivity loss, and maybe also reduce availability in the long run. Small, unplanned changes can occasionally shift into the availability loss category if a simple, quick repair winds up being complex and time-consuming. A good reason tree can differentiate easily between short and longer stops in order to respond to each in a deliberate way.   To start off in the process of architecting your reason tree, try writing the three categories on a board in a common room in an average factory (or several as a survey). Ask operators to stop in over the course of a few days and write various machine codes that they see often and find useful under one of these categories, or more than one if the machine code pops up under different circumstances and can mean different things. Have them write a 10 word justification, if the association isn’t obvious. Gather all of the “leaves” in this way, and then begin to associate them with the “trunk”, forming the “branches”.   An example tree can be seen in figure 3 here, with leaves like “Changeover” and “Maintenance” being semi-ambiguous; they could just as easily be seen as unplanned stops. Therefore, there may be multiple reason codes mapping up to the top of the tree in more than one branch, and these can have different categories, which controls how the business logic responds to the different codes. The Help Center has more details about how the events are mapped to types, and each type contains multiple categories, as configured by you when you set-up the DPM model. Fg 3: Different types of changeovers may have different codes, and can map up as either planned or unplanned, but all planned and unplanned stops (long stops) are under the Availability category of the trunk. Similarly, small stops can involve idling, like if there are not enough materials, reduced speed if the conditions are not ideal, or other small stops, usually caused by human error or unforeseeable circumstances. Quality loss then refers to the products which fail quality checks, either because the machine still has the wrong paint in the applicator and needs a few rounds to be ready for the next production item, or because the conditions are again, not ideal, and items wind up scrapped.   Example Reason Tree Fg 5 example tree with more specific tags (there may be dozens or hundreds in a full reason tree, though the fewer are needed to capture the events we care about, the better).   Theory of Constraints Fg 6 theory of constraint wheel: an industry process for gradual OEE improvement in factories that has been adapted into the PTC methodology as well. While architecting your reason tree, always remember the key purpose: gathering only as much data as necessary to analyze the efficiency of a factory and to identify the bottleneck, or the most limiting factor. The important point is to identify not just the bottleneck that seems the most troublesome, but the one that actually results in the greatest impact to OEE across the entire factory.   Without software like DPM, and a properly designed reason code tree, the process of improving a factory can be very challenging, involving a lot of guesswork, and sometimes solving one problem at the cost of another. The issue is that these machines produce a LOT of raw data, and humans are not the best tool available to gather and aggregate this data in a consumable way. A good reason tree ensures a smart application that can quickly prioritize the machine (bottleneck) that most impacts production, and not just the machine that functions in the least optimal way.   So, the theory of constraints is really a process for identifying small, incremental changes, which together can make a big difference, and fast, in factory OEE. The rate at which this cycle can be completed varies, however. The slower the process of identifying constraints and the less information that is gathered, the slower and less precise the first two steps of this process. Alternatively, in a traditional constraint identification process, too much information can be a problem as well, due to human limitation, as discussed above. So, DPM is a great benefit in this regard, because it aggregates the data into a consumable, comparable way every 5 minutes, freeing up your human analytics for problem solving and prioritization, and not data gathering and sorting.   Other Key Tips Also remember that a good tree treats the trunk like a whole unit, with each category occupying a percentage of the overall OEE. Afterall, look back up at the 3 dimensions of OEE in the equation above. For example, the more you see issues with availability, the less you will see issues with scrap, for the machine simply doesn’t have as much time to produce scrap if it is constantly down. The more you see issues with quality loss, the less you should see of productivity loss, because these are simply inversely proportional modes; to say it differently, if a machine is running quickly and seeing few minor maintenance stops, then it is likely to produce more scrap (as well as more good product as well).   Another thing to remember is that even DPM is limited in its capacity to interpret raw data. Even while many magnitudes more efficient than any human gathering and analysis could ever be, there is an upper limit to how much raw data DPM can ingest and analyze before the system gets very expensive. For this reason, you want to ensure your reason trees use only as many reason codes as are required to capture the OEE of a factory site. This will mean using different codes for different types of things, most likely, which is easy to do maintainably across many sites using thing shapes. Keeping things tightly defined and organized is the easiest way to ensure a clean, efficient system for gathering and storing data.   Also remember that data will not need to persist very long once DPM is fully operational and adopted by your factories. DPM ensures that the changes made to the production line to improve efficiency are the highest impact, and the least difficult to implement, meaning that there will be a very rapid return on investment, and a process to ensure future issues are identified and resolved quickly. Data from past issues in the factory won’t be as relevant, and historical data stores can be kept smaller than one might think. It is the power of ingesting data directly into the processing and aggregation process, the automatic reduction of data down into presentable, consumable webpages, that makes DPM and ThingWorx such a great factory solution for optimizing OEE.

ThingWorx Monitoring and Alerting, Part 2 Using Prometheus and Grafana By Tori Firewind, IoT EDC Building Dashboards     To add a panel which monitors some component of the ThingWorx application to a dashboard in Grafana, click to add a new panel. Under “Metrics” in the box at the bottom of the screen, select what ThingWorx metrics you wish to monitor (type “thingworx” in the search box to see them all). For example, select the Platform Subsystem memory in use:     Label filters aren’t necessary, though you may want to sort by instance if you are monitoring multiple ones with the same dashboard. You may also want to take some time to format the Y axis, which by default will show in bytes. Go to the formatting panel on the right side and scroll down to the section called “Standard options”. For the Unit dropdown, start typing “data” and then select “bytes (SI)”. This will automatically determine if the bytes you’ve provided should really print as MB or GB based on how large the numbers are.     Rename the panel, modify it in any other way desired, and then click Apply (last 5 minutes):     Once you add the panel, you can watch the memory usage as it is scraped by selecting the refresh option (10s or 30s, whatever makes sense based on your scrape interval).     The viewing window is stored in the URL, so that you can generate a report for a specific interval (like when a test was occurring), and then store that result or share it in a more compact way: http://localhost:3000/d/nleucPv4k/thingworx-monitoring?orgId=1&from=1668528038732&to=1668536503953  (absolute timestamps):     Dashboards are just collections of panels which report on all of the various metrics of performance and stability that exist for single components of a system. This is because there can be quite a few metrics worth watching for each individual component. Most of the third-party tools come with their own dashboards, but the ThingWorx component is one which for now, requires some thought and creativity.     Consider your use case carefully and look over the various subsystems contained within ThingWorx. Each part of an application is localized to specific subsystems, and some are more business critical than others. What will go at the top of your dashboard? Add rows, add panels per row, and see what the many choices are for watching your system.     Don’t forget that with Telegraf running, VM or machine usage metrics are also available for display on a dashboard. Things like overall CPU and Memory usage are critical to determining the health of a system, as we have demonstrated in our own reasoning in past benchmarks and scale tests. You can create a panel to monitor the mem_used versus mem_total, like so:     Another metric from Telegraf worth adding is the CPU usage, which should be given “percentage” for the units and which needs a label filter of cpu = cpu-total. If we do some resizing and drag-and-dropping, then we now have the first row of a dashboard:     See how the Platform usage climbs steadily and is purged in a cycle? That is the Java Garbage Collection mechanism, and it’s important to remember to leave room for spikes on top of those peaks. Data can also be calculated or processed in some way to make it more useful for determining system health and stability.     The data in the picture below uses the formula submitted = completed + number queued + number failed. It shows the current queue on the left Y-axis and the max queue on the right (since the two numbers usually are drastically different). It looks pretty, but it doesn’t really tell us much about the system in this format, so let’s do some math and find a representation that is a bit more helpful.     Performing a “non-negative derivative” calculation over the submitted and the completed queue counts over time allows for us to look at the status of the queue as a velocity. When the “complete” speed appears behind the “submitted” speed for too long at a time, then that means the queue is filling up and will eventually result in data loss.     If we take this one step further and calculate the average of the submitted minus the completed over time, then we can actually predict approximately when the queue will fill up. This can then be displayed on a dashboard in Grafana, or used as the basis for an alert.   What to Monitor     In addition to monitoring the system which ThingWorx runs upon, ThingWorx itself can easily be monitored down to the subsystems level by Prometheus due to the Metrics endpoint. Many applications have support built into the way they format the data for scraping, including the JVM (which exposes Prometheus-formatted metrics with the JMX Exporter) and the OS (which can use the Node Exporter or Telegraf for the same purpose). For these more generic components, there are popular community dashboards which can be downloaded and used in Grafana for data analysis and review.     For ThingWorx, there’s different kinds of data to track: subsystem data (see the list on the right) and non-subsystem data. There’s queue based versus non-queue data. These different metrics can collectively characterize the overall health of the application, depending on the use case.     For instance, if this is a system with very many connected devices, one metric which may be important to track is the number of total devices defined on the Foundation server vs. the number of devices which are currently connected. If there are relays involved, then many devices suddenly going offline can mean a relay has failed. Another example is if the system sizing depends on an assumption that there will only ever be a fraction of the total number of devices connected at a time. Use cases like these could be monitored easily by keeping track of the total vs. the number of connected devices.     Other common indicators of a healthy ThingWorx application might include the value stream and stream queues. These queues should fluctuate over time as the data is ingested and processed, but they should never be growing in size. If the stream queues are growing, then that means the data is writing to the queues faster than the queues can write to the database. Eventually, when the system runs out of resources to keep track of the queues, data will be lost. Having the stream information displayed in a chart can make it very easy to spot an upward trend in resource usage early on, which can catch a blockage or bottleneck that needs attention before it starts to affect the larger system in catastrophic ways later.     Memory usage information from the various subsystems might be something worth tracking, as well as the event queue. These can indicate that the business logic is functioning with room to handle spikes, and that the server has enough memory to service all three dimensions of an IoT application: the ingestion, the business logic and thing-based alerting, and the user experience and UI. If file transfers are a key part of the use case, then the number of concurrent transfers, the average speed of them, the size of the files, all of this kind of stuff can be tracked and charted in Grafana by making use of the ThingWorx metrics which automatically show up there once you import the Prometheus data source.     A mature dashboard used for a production environment might look a little like this: For further reading about subsystem monitoring, check out the Help Center.   How to Alert     The alerting mechanism built-in to Prometheus is incredibly easy to configure, so it might be tempting to generate tons of alert rules. However, remember that the more noise a system makes, the harder it is for those monitoring that system to know when action is really required. Playbooks which document how to respond to alerts, who to contact, how to act, and all the information necessary to handle an alert, should be created as an ongoing part of the DevOps process.     Alerts should fire with the right severity in the subject line, as well as all of the information about the issue that is currently known, presented in a concise way, so that whoever receives the alert starts thinking about the root cause sooner and recovers the system faster. Those who receive the alert should have the ability to facilitate its resolution, and know who is expected to react to any alerts which come in.     In the ThingWorx monitoring stack, Prometheus handles the alert rules and the generation of alerts, but alert filtering and delivery is managed in an external alerts manager.     Generally, you want your alerts to follow a curve. If the current queue size exceeds 50% of the maximum, perhaps that isn’t a huge deal, if the application catches up quickly. How long are spikes in queue processing expected to last? Perhaps if the queue size is over half-full for 10 seconds, 30 seconds, then that means the queue is falling behind and not catching up. Ok, so this might be a warning level alert. When does this become an error? Well, let’s say the queue exceeds 90% of the max queue size. This might want to alert the moment it hits the mark. Now, farther along the curve, it may not take as long before data gets lost.     As the severity of the situation increases, the threshold for alerting should increase as well. That way when errors do alert, it is a sure thing that they require a response immediately. The alerts are then pushed into the “Alerts Manager” for delivery based on your management rules. The Alerts Manager may decide to withhold warnings altogether, or send them to a much smaller mailing list, whatever filtering helps to ensure the right people receive the right alerts, right when they need them.   In Conclusion, A Healthy Application...     Has stable memory usage that fluctuates predictably and doesn’t grow over time. In a system experiencing mild issues, the memory starts to trend upward:     If left unattended, systems like this may eventually experience outages. Finding the issue this early means there is even time to do some digging, debugging, taking of stack traces, and other such troubleshooting steps before the system must be restarted or recovered. That can really make the difference in identifying and resolving before there are real problems.     One metric which makes for good alerting is the total number of failed stream entries, which can indicate there’s an issue writing to the database even before the queue has started to fill up. Other alerts may include warnings and errors based around percentages of memory used or queues filled, which depend on how long the queues take to fill up and how long the state has been at its increased usage.     Prometheus has all of the tools necessary to make this possible across a variety of infrastructures and use cases. Set it up on a local machine and poke around at what ThingWorx metrics are available to meet your monitoring needs.

ThingWorx Monitoring and Alerting, Part 1 Using Prometheus and Grafana By Tori Firewind, IoT EDC Introduction and Getting Started     As ThingWorx has become a more mature product during the lifetime of the IoT EDC, so too have our dev ops recommendations. As we’ve stated throughout many posts now, testing is a key part of ensuring enterprise readiness, and it occurs at every stage of the process: from unit testing to preserve individual service logic, to integration tests which preserve the functionality of the application as a whole, to user and edge load testing and user experience testing, which ensure enterprise readiness. So testing is a critical component, but the process of dev ops never stops. In order to effectively test the system, a comprehensive monitoring solution is also required.     Once the application is tested and the changes pushed into production, there is no knowing with certainty that everything will run smoothly indefinitely. Random spikes in usage, server bandwidth or availability, any unforeseeable factors like these can come along and cause issues for a system. If these issues aren’t detected and addressed early, then they can very rapidly morph into much larger problems: outages, data loss, inflated data tables which are hard to revert due to their size. It is critical to detect performance issues on a system as early as possible, to have as much information as is necessary to figure out where the problem is heading, and what may have started it. Monitoring is key to a healthy system. CI/CD stands for “Continuous Integration/Continuous Deployment”, a never-ending cycle of improvement. Testing just once before the initial go live isn’t enough. Each system should have automated tests that run continuously, as well as monitors and alerts which reveal problems sooner. Diagnostic tools play a role as well, being the bridge from the end of the dev ops process cycle back to the beginning (monitoring into planning). A good CI/CD dev ops process will ensure that problems are found earlier, fixed more rapidly, and fixed for everyone using the system.       In a fully mature dev ops pipeline, issues are anticipated, discovered and researched before they become production outages or critical issues. These investigations or testing follow-ups produce development tasks (usually bugs, but also features at times) which then start the dev ops cycle all over again. This is why a good, efficient dev ops pipeline is needed, one which allows changes to quickly and safely go from development to production.     This is also why diagnostic tools play a role in the monitoring piece of the dev ops process. They are the bridge between monitoring and planning. Tools like Dynatrace can be configured to provide call stacks and take thread dumps when issues start to occur, before the system is performing so poorly it needs a restart, which happens automatically in a cluster and can clear out any trace of the issue.     Thread dumps are often necessary to diagnosing the root cause of the issue (to permanently fix it), and doing so quickly ensures application stability and availability. That is, after all, the purpose of the dev ops process. Diagnostics is therefore an equally important piece of the dev ops Figure-8-shaped pie, and one which deserves its own spotlight in an article to come.     Every piece of the dev ops process must be viewed as equally important in its own way, lest the dev ops cycle get hung up on bottlenecks of its own. A safe and stable system is not one which never experiences issues, it is one which has a good, efficient plan in place to handle recovery and prevention of repetition. A wholesome dev ops process is a happy dev ops process.   The Monitoring Stack     There are many monitoring options available, but in our experience one of the easiest and most effective monitoring stacks to use with ThingWorx is Prometheus for metrics gathering with Grafana for metrics analysis and review. In a mature monitoring stack, Telegraf is also commonly installed on each VM/host to gather the system metrics (like CPU and Memory usage, things we’ve stated are good metrics of system performance and stability in past articles on scale and size testing) and output them in Prometheus format.     Prometheus is a highly scalable open-source monitoring framework that contains out of the box monitoring and alert capabilities for Kubernetes-based deployments (not covered in this article). Using Prometheus is very simple because the ThingWorx application exposes a metrics endpoint which is formatted directly for use by Prometheus. There is also built-in alerting in Prometheus, but not the ability to form dashboards for reviewing data or screenshotting it for documentation purposes. That’s where Grafana comes into play. Grafana has a preconfigured Prometheus-type data source and many preconfigured dashboard templates for various applications and services. Telegraf is also easily imported into Grafana, as is shown in the section below. The Prometheus targets in the larger diagram are expanded out on the left. For each target, some tool exports the data in a syntax which Prometheus can scrape. For VMs, this can be Telegraf, for Kubernetes, the Node Exporter. JVM has a JMX Exporter, and other tools like CX Server use Graphite. Many apps already have a Prometheus endpoint built-in, like ThingWorx and Zookeeper. Telegraf is not strictly necessary; the node exporter can also be used on VMs, but Telegraf is the more common choice since it is a more mature dev ops tool.     Once Prometheus is scraping the targets, alerting on them can be done with OOTB Prometheus functionality, and dashboards for monitoring can be made easily in Grafana (with built-in support as well). This stack does not include the diagnostics piece, something which triggers thread dumps or the like when issues do occur. There are too many ways to conduct a successful diagnostic piece to cover here.   How to Get Started     Getting started monitoring a ThingWorx application is incredibly easy in the latest versions. Simply open up a browser, and type in the ThingWorx URL, followed by “/Metrics”. At this endpoint, there is a specially formatted response that can automatically be read by the Prometheus monitoring software which contains subsystem and service data. In addition to the application metrics, Prometheus can be configured to collect metrics from a node exporter at the (virtualized) operating system or container (Kubernetes) level as well.     If you haven’t already, install Grafana, install Telegraf as a service, and install Docker Desktop. These are the tools required (in addition to ThingWorx of course) to set-up a simple sandbox system for familiarization with the monitoring stack recommended by PTC. The easiest way to try Prometheus on a local Windows instance is to use Docker. The command for that will be found below, but first open up Docker Desktop to set contextual parameters that the command line will need. Then, modify the configuration file for Telegraf or create one (called telegraf.conf in the same folder as the exe file), and put the following into the file (or uncomment it; the default config file has thousands of lines, so just search for “prometheus”):             Output plugin [[outputs.prometheus_client]] listen = "0.0.0.0:9125"             Alternatively, install the Prometheus Node Exporter tool, which will likely require some additions to the Prometheus config file (not covered here) which we are about to create.     Then, create a configuration file (called prom_config_localhost_scraper.yml in the command to come), add the following (assuming a standard localhost installation of ThingWorx):             # my global config global: scrape_interval: 45s evaluation_interval: 30s scrape_timeout: 30s # scrape_timeout is set to the global default (10s). rule_files: - prom_config_rules.yml scrape_configs: - job_name: thingworx static_configs: - targets: ['host.docker.internal:8080'] basic_auth: username: "Administrator" password: "admin!123456789" metrics_path: /Thingworx/Metrics scheme: http params: x-thingworx-session: - "false" - job_name: prometheus static_configs: - targets: ['localhost:9090'] - job_name: Telegraf # If telegraf is installed, grab stats about the local # machine by default. static_configs: - targets: ['host.docker.internal:9125']                 This example script file uses the host.docker.internal instead of localhost for the server target for ThingWorx because it is running outside of the Docker container which contains Prometheus. This yml file configures Prometheus to monitor both ThingWorx and itself, as well as the server metrics coming from Telegraf (as long as they are configured to push). It’s a sandbox-only configuration, really, as you wouldn’t want to use the Administrator user, or have the password printed in plain text in the config file in a real system. Also note the need for the x-thingworx-session parameter, as runaway sessions which spawn every 30s or so (whatever the scrape interval is) will result in memory issues over time (so we don’t want to use sessions here).     The rules file given here (prom_config_rules.yml) needs to be created separately. This is where all of the alert rules will be defined. This will determine if an alert state is happening, but without configuring the alert manager, there won’t be any notification. That isn’t covered here but is covered extensively in the Grafana docs. Here is an alert example:             groups: - name: alert.rules rules: # Alert for any instance that is unreachable for >5 minutes. - alert: HighMemory expr: mem_used > 14000000 for: 1s labels: severity: page annotations: summary: "High Memory" description: "Localhost Memory Usage is High"             Now, save these files and use Powershell to run the Docker container:             docker run -p 9090:9090 -v C:\<path_to_document>\prom_config_localhost_scraper.yml:/etc/prometheus/prometheus.yml prom/prometheus                 It should download Prometheus and install it in that container (if this is the first time), allowing you to very rapidly deploy it to an endpoint of localhost:9090 by default. If there is an error like the one shown below, this means that you forgot to start Docker Desktop (the application) before opening Powershell. Docker Desktop sets system parameters required for containers to run in a command line (in Linux, it should work if Docker is installed for use by the command line, simple as that).     The localhost endpoints are accessible in a browser. ThingWorx defaults to localhost:8080 endpoint. Prometheus defaults to localhost:9090. Telegraf is on port 9125. Open any of these in a browser tab to see the full monitoring stack. You can see easily if Prometheus is working by clicking “Status” > “Targets” at localhost:9090:     If all of the targets appear as blue and say “last scrape” and a time stamp, then they’re working as expected. If they don’t, ensure you have the right ports, that there aren’t any firewall issues (if things aren’t all on localhost), and that everything is running without errors.     The last step in the process here is to install a dashboard tool like Grafana. Once this is installed and running on localhost:3000 (by default), you can display the data from Prometheus with a few configuration steps the Grafana UI. Highlight over the settings icon in the bottom left of the screen, and then click on “Data sources”. Select the “Add data source” button, and then click on Prometheus. You have to type the URL again  (localhost:9090), but most of the defaults will be ok here, and all you have to do is click “Save and test”.     Now both targets should appear within Grafana, with their metrics showing up throughout the Grafana UI. This data source is what allows for the building of monitoring dashboards.    

This video is Module 11: ThingWorx Analytics Mashup Exercise of the ThingWorx Analytics Training videos. It shows you how to create a ThingWorx project and populate it with entities that collectively comprise a functioning application. 

This video is Module 10: ThingWorx Foundation & Analytics Integration of the ThingWorx Analytics Training videos. It gives a brief review of core ThingWorx Platform functionality, and how the Analytics server works on top of the platform. It also describes the process of creating a simple application, complete with a mashup to display the information from a predictive model.