cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community email notifications are disrupted. While we are working to resolve, please check on your favorite boards regularly to keep up with your conversations and new topics.

IoT Tips

Sort by:
The use of the term “SSO” means different things to different people. Among Navigate Admins, it became shorthand for using PingFederate to provide both authentication with a single sign-on component, as well as authorization (checking permissions for access to files). In Navigate 1.5, this was the only option for configuring a production system, and many people were not ready for it. That was the origin of the “must have SSO” statement. Beginning with Navigate 1.6, PTC added a scenario called “Windchill Authentication”, that is suitable for Production and uses your Enterprise LDAP to authenticate users. It will issue a token so you get some of the benefits of single sign-on, but not all the bells and whistles that come with PingFederate. It’s also easier to configure. People have begun referring to Windchill Authentication as “non-SSO”, to distinguish it from PingFederate, even though Windchill Authentication has some SSO functions.   In the install manual, there are three scenarios: Fixed Authentication, Windchill Authentication, and Single Sign-On with PingFederate. People usually begin with Fixed Authentication (the easiest to configure, but not secure so it’s only good for Proof of Concept demonstrations), then do Windchill Authentication before tackling PingFederate. Windchill Authentication can take a couple of days while Webexing with us to get working, but for PingFederate we plan several Webexes over a period of 8 days for a typical install. During that time you will be coordinating with other administrators (such as the AD admin) and waiting for emails etc. to get remote admin tasks done as part of the install. Be prepared, timewise.
View full tip
Create a new Thing using the Timer Thing Template. The Timer Thing will fire a Timer Event when the Timer's Update Rate has expired. The event is automatically present and does not need to be added manually. Configuration   The Timer Configuration is quite straightforward. It can be accessed via the Thing's Entity Configuration. Configuration allows for Enabling the Timer on Thing-Startup - whenever the Thing is started, e.g. when restarting ThingWorx or via the RestartThing Generic Service, also the Timer is enabled and will fire Events. Changing the Update Rate - in which intervall the Events will be fired (by default every minute [60000 milliseconds]). Changing the User Context - in which the Events will be handled. The user will need visibility and permission on e.g. executing Services or depending Things, which are required to run the Service triggered by the Event.           Services   Timer Things inherit two Services by default from the Thing Template DisableTimer EnableTimer These will activate / de-activate the Timer and allow / disallow firing Events once the Update Rate has expired If a Timer is currently enabled or disabled can be seen in its properties  
View full tip
The following power point contains some reference slides to start up with DSE/ThingWorx integration. Start with understanding DSE architecture and specifically, the differences compared to regular Relational Databases. Free online courses offered by DataStaxAcademy: –https://academy.datastax.com/courses/understanding-cassandra-architecture –https://academy.datastax.com/courses/installing-and-configuring-cassandra   The following section will guide you through some of the specifics: http://datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAbout_c.html
View full tip
Please note that the below configuration is intended for testing purposes only.  Make sure that your final deployment is within your business security policies. The installation guide can be found at: http://support.ptc.com/WCMS/files/173161/en/ThingWorxDockerInstaller.pdf Postgres: Reference the Installation Guide above for a supported version of Postgres Once deployed, configure it to support remote connections: Navigate to: <PostgresInstallPoint>\data Open the following with a text editor: pg_hba.conf Find the line with IPv4 local connections Change 127.0.0.1/32 to 0.0.0.0/0 Restart PostgreSQL server NOTE: This could open up security vulnerabilities to the database, so make sure you take appropriate security measures if the data will be sensitive Docker: Find the appropriate Docker platform for your OS Docker Community Edition For Windows Server 2016, there is a download for the Edge (Windows Server 2016) under the above link -> Docker CE for Windows -> And then scroll down a little bit Docker Toolbox If you try to deploy the Docker Community Edition on a system that doesn't support, it will direct you to this installation instead At some point during or after the installation, it will prompt you to enable Hyper-V If this is a physical server, these settings will be in your Bios For VMWare, while the VM is powered down, go to VM-> Virtual Machine Properties -> Hardware -> Processors -> Enable 'Virtualize Intel VT-x/EPT or AMD-V/RVI' Restart, and make sure Docker is running (whale icon in your system tray for the Windows Server 2016 edge version) With Docker running, open a command prompt and look at your IP settings For windows Server 2016, right click the start menu -> Command Prompt (admin) and run IPCONFIG Write down the IP assigned to DockerNAT, as this is will be your Postgres HOST later Share your main drive with Docker In Windows Server 2016, right click the docker icon in the system tray -> Settings -> Shared Drives -> C: Thingworx Installation: At this point you should have Docker installed and Postgres remotely configured with only the admin user (postgres) The installer will create the image/container inside of Docker, Install Tomcat, and configure your database Below is a capture of the settings used in the above screenshots.  Anything not listed (like specifying the container name, which is twxfoundation by default) was left as the default values:       Installation Directory: C:\Program Files (x86)\twxEnterpriseFoundationPostgresDocker       ThingWorx License Directory: C:\Users\Administrator\Desktop\license.bin       Local ThingWorx Foundation Port: 8080       Java Initial Heap setting for TWX Foundation: 1024       Java Max Heap setting for TWX Foundation: 2048       RDS Instance: 1     PostgreSQL Host: 10.0.75.1       PostgreSQL Port: 5432       PostgreSQL Admin Schema: postgres       PostgreSQL Admin Username: postgres       PostgreSQL Admin Password: <see note>       PostgreSQL ThingWorx Foundation Schema: thingworx       PostgreSQL ThingWorx Foundation Username: thingworx       PostgreSQL ThingWorx Foundation Password: <see note>       PostgreSQL ThingWorx Tablespace Location: /                     ​NOTE:​ It is highly recommended to use a complex password (Letters of all cases, numbers, and symbols) as we have opened up our database to remote connections RDS was set to Yes (Default is no) PostgreSQL Host is the IP taken from the earlier steps In this example, the Tablespace location is defined inside of Docker, not Windows Post Install: Confirm that Thingworx is running properly by opening a broswer and attempting to log in For our example, the URL is http://10.0.75.1:8080/Thingworx Troubleshooting: If the installation fails, refer to the end of the Installation Guide on where to look for logs, and items that need to be cleaned up before attempting to install again If the install was successful, but connecting fails, run the following in the command prompt to look at the Docker Server's startup logs for hints: Docker logs -f twxfoundation *Note that twxfoundation is the default during installation.  If this was changed in your installation, use that instead
View full tip
With the new licensing introduction, it could get confusing at first on how to obtain and apply, especially with more than one app in place. This is an example on how to apply both foundation and manufacturing license when installing Thingworx 8. 1) Install Manufacturing App 8.0 and needed components (ex: Kepware) per  the guide with manufacturing app license - manufacturing app widget can now be accessed. 2) Accessing /Thingworx reports a licensing issue 3) Download Thingworx license from the license portal. 4) Rename the manufacturing app license.bin to <name>.bin and put Thingworx license.bin in the ThingworxPlatform folder. 5) Restart Thingworx service 6) Access /Thingworx and accept license agreement 7) Change license.bin back to the original manufacturing app license.bin (step 4) 😎 Restart Thingworx server 9) Both manufacturing app and foundation functions are available.
View full tip
In this blog post I'm covering the pratical aspects of setting up a Chain of Trust and configure ThingWorx to use it. There are already some examples available on how to do this via command line using a self-signed certificate, e.g. https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS193947 In this example I'll be using the KeyStore Explorer, a graphical tool for managing keystores. http://keystore-explorer.org/ To learn more about the theory behind trust and encryption, see also Trust &amp; Encryption - Theory The Chain of Trust Creating a Root CA Open the KeyStore Explorer and create a new jks KeyStore File > New > JKS Via Tools > Generate Key Pair create a new key-pair. Use the RSA algorithm with a key size of 4096. This size is more than sufficient. Set a validity period of e.g. 302 days. It's important that all subsequent certificates have a lower validity period than the original signer. Leave the other defaults and fill out the Name (addressbook icon). Only the Common Name is required - usually I never use the Email field, as I want to avoid receiving more spam than necessary. Click OK and enter an alias for the Root CA (leave the default). A new password needs to be set for the private key part of the certificate. The new Root CA will now show up in the list of certificates. Creating an Intermediate CA The intermediate CA is signed by the Root CA and used to sign the server specific certificate. This is a common approach as - in case the signing CA for the server specific certificate gets corrupted, not all of the created certificates are affected, but only the ones signed by the corrupted CA. In the list right-click the Root CA and Sign > Sign New Key Pair Use the RSA algorithm with a key size of 4096 again. Ensure the validity period is less than for the Root CA, e.g. 301 days. (Because every second counts) Fill out the Name again - this time as Intermediate Certificate. A Intermediate Certificate is a Certification Authority (CA) in itself. To reflect this, a Basic Constraint must be added via Add Extensions. Mark the checkbox for Subject is a CA Confirm with OK, set the alias (or leave the default) and set the password for the private key part. The new Intermediate CA will now show up in the list of certificates. Creating a Server Specific Certificate In the list right-click the Intermediate CA and Sign > Sign New Key Pair Use the RSA algorithm with a key size of 4096 again. Ensure the validity period is less than for the Root CA, e.g. 300 days. (Because still, every second counts) Fill out the Name again - this time it's important to actually use the public servername, so that browsers can match the server identity with its name in the browser's address bar. If the servername does not match exactly, the certificates and establishing trust will not work! In addition to this, a Subject Alternative Name must be defined - otherwise the latest Chrome versions consider the certificate as invalid. Click on Add Extensions and the + sign, choose Subject Alternative Name. Add a new name with the + sign. Make it the same as the Common Name (CN) above. As before, confirm everything with OK, enter an alias and a password for the private key part. Saving the keystore All certificates should now show with their corresponding properties in the certificate list: Notice the expiry date - the higher up the chain, the sooner the expiry date. If this is not the case - the certificates and establishing trust will not work! Save the keystore via File > Save As and set a password - this time for the keystore itself. As Tomcat has a restriction the password for the keystore must be the password for the private key of the server specific certificate. It's highly recommended and (hopefully) obvious that passwords for the other private keys should be different and that the password for the keystore should be completely different as well. The keystore password will be shared with e.g. Tomcat - the private key password should never be shared with a source you do not trust! Configuring ThingWorx In ThingWorx, configure the <Tomcat>/conf/server.xml as e.g. mentioned in https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS193947 I'm using a connector on port 443 (as it's the default HTTPS port): <Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol" maxThreads="150"           SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" enableLookups="false"           connectionTimeout="20000" keystoreFile="C:\ThingWorx\myKeystore.jks" keystorePass="changeme" redirectPort="8443" /> Restart Tomcat to consider the new configuration. ThingWorx can now be called via https://<myserver.com>/Thingworx It's not yet trusted by the client - so it shows as potentially unsafe, and for me it shows in German as well *sorry* However, it's already using encrypted traffic as every HTTPS connection will use TLS. In this case we're communicating in a secure way, but do not know if the server (and its providers) can be trusted or not. Sharing credit card information or other personal details could be dodgy in such a scenario. Trusting the Root CA To trust our Root CA, we first have to export it. Every browser allows to view the certificate, e.g. in Chrome this is done via the Developer Tools > Security As we're using KeyStore Explorer, we can use it to export the Root CA. Right-click it in the certificate list and Export > Export Certificate Chain. Leave the defaults and export as .cer file. Open the Certificate Manager, by typing certmgr into the Windows search field. Open the Trusted Root Certification Authorities and right-click on the Certificates folder. Under All Tasks > Import. Choose the file you've just exported for the Root CA. The Root CA is now a trusted certificate authority. As the ThingWorx server specific certificate will fall back to the Root CA, it's now also recognized by the browser and the connection is now considered as secure and trustworthy. Congratulations!
View full tip
Do you trust me? if (true) { why } else      { !why } Communication between client and server can be encrypted or not. Communication between client and server can be between trusted parties or not. Both statements are independant of each other. There's no need to trust a server - but still traffic can be encrypted. There's no need to encrypt traffic - but still a server can be trusted. For secure and safe communication in an IoT environment encryption as well as trust should be an integral part of exchanging data. ​ Trust Certificates and Chains of Trust Trust establishes the identity of the parties involved, e.g. is server I'm contacting really the server it pretends to be? This can be ensured with server certificates. On the other hand, client certificates ensure that the client is actual the client it pretends to be. This can be used as an authentication approach and if the identity of the server as well as the client is confirmed, this is also known as mutual authentication. Trust is usually invoked through certificates. Certificates have a subject, usually the servername that the certificate belongs to. This can also be a wildcard subject like *.myserver.com which will be valid for all subdomains of myserver.com. A certificate can be digitally signed to ensure its identity. Usually a Certificate Authority (CA) signs the certificate. This CA can in turn be signed by another CA and so an, creating a whole chain of CAs. This chain is commonly known as the Chain of Trust or the certification path. If a certificate is not signed by a CA it's called a "self-signed certificate". These are mostly used for test scenarios, but should never be used in real production environments. Certificates can also expire by using dates like "Valid From" and "Valid To" to determine if a certificate is valid or not. In case a certificate or CA is expired, all following certificates in the Chain of Trust will no longer be considered as valid. For "commercial" certificates, certificates that are signed by an official CA like Google, Let's Encrypt or VeriSign, a Certificate Signing Request (CSR) is used. For this a key-pair is generated and the CSR is signed with the private key. The CSR contains the public key and is sent to and finally signed by the CA itself, returning the public certificate e.g. signed by VeriSign. This mechanism allows to generate certificates by preparing the main information and just having the CA sign it as a trusted authority. Key-pair All certificates consist of a key-pair - a private key and a public key. The private key is indeed very private to the server and MUST NOT be shared with anyone else. The public key is indeed very public and can be shared with anyone. The key-pair consists of mainly two very large prime numbers and some mathematical magic allows for a mathematical one-way function ​with a trap door. This function allows to en-crypt any message with the public key and to de-crypt this message with the private key. So by giving out the public key to anyone, anyone can en-crypt a message to the server but only the server alone can de-crypt it with its private key. Therefore it's important that the private key is not shared with anyone, otherwise messages can be de-crypted by anyone possessing the private key and secure communication can be compromised. This method is called public-key encryption, or asymmetric encryption (as the key for en- and de-crypting are different). For more information, check out https://en.wikipedia.org/wiki/Public-key_cryptography This key-pair is not related to any encryption while sending data between client and server. Encryption on a transport level is accomplished by using different, random keys. It's just used to generate a unique certificate with a unique fingerprint as a public certificate. The private key part is amongst other things used for signing other certificates in the Chain of Trust. Tomcat needs access to the private key of the server specific certificate, to actually verify that the certificate is authentic. However, certificates can also be used for e.g. emails where the private key is in fact used to en- or decrypt the mail on a content level! ThingWorx ThingWorx is mainly configured for server side trust. In this case all connecting clients can ensure that the identity of the ThingWorx server is a trusted one - and not a fake server or an untrusted source. Certificates and trust are usually triggered when a secure connection is established, via TLS / SSL - e.g. by calling https://<myserver.com>:443/Thingworx The default port for a HTTPS (HTTP Secure / over TLS) connection is 443. For "normal" HTTP configuration, certificates are not required. Keystore To establish trust, any client needs to know the public certificate of the ThingWorx server. This is usually stored in a keystore. In ThingWorx, Tomcat needs to be configured with the location and the password of the keystore containing the certificate. For each web-request Tomcat will then present the server certificate to the client. The client can then verify through its own keystore if the certificate is trusted or not. When operating in browsers, this is done via e.g. the Windows-Keystore. For devices a custom, e.g. a Java-Keystore is required. The keystore must contain the whole Chain of Trust as well as the server specific certificate and its key. If the Chain of Trust is not stored completely in the keystore, or the key is missing / not accessible, the certificate validation process will already fail on server side. Certificates in Windows The default Windows-Keystore is actually pre-filled with lots of Root-Certificates, which start the Chain of Trust - such as Google, VeriSign or Microsoft. Trust needs to start somewhere… If the browser can trace the CAs of certificate back to a trusted Root CA, it's indicated by a green lock and the server will be trusted. If the Root CA cannot be found, it's indicated by a red lock (not a purse) - meaning that the server is not trusted. The server can still be contacted anyway, by manually acknowleding the risks (like identity theft or credit card scams). As an example, here's the certificate for Wikipedia and its Chain of Trust, starting with the GlobalSign Root CA: Trustworthy? In the end, certificates only clarify the identity of a server. If it is trusted or not, is completely up to the user / device. This trustful relationship all comes down to the client keystore / truststore. While the presenting server holds its certificate in the keystore, the verifying client holds the trusted certificates in the truststore. Any certificate in the truststore will ensure the correct server identity. Any unknown certificate will trigger a "Are you sure?" question in the browser. And as devices usually cannot respond to such a question, communication with an untrusted server can be forbidden by default. There are also products, like PTC Navigate which will require a client certificate to not only ensure server identity, but also ensure that also specific clients are able to connect to the server. Encryption No matter if a server is trusted or not, as soon as HTTPS is used as a protocol, communication will be encrypted. The certificate plays a role, but no matter if it's in the client's truststore, TSL forces encryption. Client Hello Usually the client contacts the server and sends a Client Hello. With this it also sends a list of available cipher suites, the intended receiver (server name) and a list of available Signature Hash Algorithms to be used. Cipher suites are used to determine which algorithms are used to establish a connection. The Signature Hash Algorithms define the Hash function, like SHA256 and the key / signature, like RSA. In my environment usually the Elliptic Curve Diffie-Hellman key exchange is used to transfer public keys between server and client. During the Client Hello information about the elliptic curves are also transferred. For more information on these topics, see https://en.wikipedia.org/wiki/SHA-2 https://en.wikipedia.org/wiki/RSA_(cryptosystem) https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange https://www.youtube.com/watch?v=YEBfamv-_do (Diffie-Hellman explained) https://en.wikipedia.org/wiki/Elliptic_curve_cryptography Server Hello After receiving the Client Hello, the server sends back the Server Hello. This consists of multiple handshakes. In this phase, the server sends the preferred cipher suite (intersection of client and server suites) including the Signature Hash Algorithm to be used. In the same step the server is sending off the certificate to the client for identity validation. For establishing trust and validating the certificate, the server needs to know the private key for the server specific certificate. The server also announces in the server key exchange what public key it's using for encryption to the client. This key is not related to the certificate, but a random number from a newly generated key-pair. Keys are exchanged between server and client and vice versa via the Diffie-Hellman key exchange. Client Response The client now sends in the client key exchange what public key it's using for encryption to the server. This is also a random key transmitted via the Diffie-Hellman key exchange. Initiating Communication After announcing the public keys, both server and client will now communicate encrypted by using their custom key-pairs for asymmetric encryption of the payloads. This can also include an additional change of the cipher specification. As the public keys have been exchanged via the Diffie-Hellman key exchange, server and client are now bi-directionally communicating using each other's public key to en-crypt a message and each's own private key to de-crypt the message again. For more information, also see https://tools.ietf.org/html/rfc5246 Trust & Encryption - Hands On Do you trust me now? if (true) { why } else      { !why } To see how this all works in a ThingWorx enviroment, and how easy it actually set up what has been discussed above, see also Trust &amp; Encryption - Hands On
View full tip
In this blog I will be covering the initial setup of ThingWorx Android SDK with a sample app (supplied with the Android SDK) and set it up with a web based revision control system like Bitbucket's (free account plan). I'll also be covering quick information on how to enable the HTTPS connection for the ThingWorx server on Windows platform. This will allow for secured connection to the ThingWorx server from the Android application. Do note this is only a reference guide for setting it up with revision control system, you’re free to choose to setup the Android Studio without Bitbucket or with any other revision control system. It’ll be just fine to have a local Git/SVN/Perforce etc. to manage the code repository, setting Android Studio with Bitbucket is officially not supported. Pre-requisite: 1. Download and unzip the ThingWorx Android SDK from https://support.ptc.com 2. An account with Bitbucket is required 3. Download and install Android Studio Project Setup 1. Unzip the downloaded Android SDK on a local drive 2. Start the Android Studio > Import project > navigate to the sample applications location provided with the ThingWorx Android SDK e.g. Thingworx-Android-SDK-X.x.x\samples 3. Select one of the sample application e.g. androidShell, with ThingWorx Android SDK X.x.x there are 3 sample applications currently available when the Android SDK is downloaded: a. Android File Brwoser b. Android Shell c. Android Steam Thing 4. For this blog I'll be setting up the androidShell Android Application 5. Do note that all the sample projects are built using Gradle, so while importing if required select Gradle as the build system for the sample application 6. Once imported successfully in Android Studio you should be able to see the Android project and its file structure like so 7. We'll need an account with Bitbucket, create one if you don't have already 8. Logon to Bitbucket and create a team and new repository under that team 9. Navigate to the repository created in Bitbucket 10. Create a local GIT code repo if you don't already have one, and copy over all the contents from the Android SDK x.x.x.zip to that location 11. On your local machine open a command prompt and navigate to the drive where the local GIT's code repository resides, i.e. the folder where you unzipped the android SDK and execute the commands as mentioned: a. git remote add origin https://<accountName@bitbucket.org>/<teamName>/<projectName>.git b. git push -u origin master Note: This will add the contents of your local GIT repository to the empty code repository you’ve created under the team on Bitbucket. 12. You can also use SourceTree UI application on windows for creating, GIT and Mercury based code repository and connect it to your Bitbucket account 13. On successful commit following will be logged in the command prompt Enabling HTTPS on Tomcat and connecting to Android Application Securing Tomcat on Windows You can skip this section if you already have Tomcat running ThingWorx configured for HTTPS connection. 1. Execute the command in a command prompt C:\>"%JAVA_HOME%\bin\keytool" -genkey -alias tomcat -keyalg RSA -keystore C:\KeystoreTomcat\tomcat.keystore Note: Executing above step will require you to add additional information to the keystore like, Org name, full name location, etc. 2. Edit the <tomcatInstallation>\conf\server.xml file   <Connector   port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"               maxThreads="200" SSLEnabled="true" scheme="https" secure="true" keystoreFile="C:\KeystoreTomcat\tomcat.keystore" keystorePass="<giveYourPasswordHere>" clientAuth="false" sslProtocol="TLS">       <!-- <SSLHostConfig>             <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"                         type="RSA" />         </SSLHostConfig>-->     </Connector> 3. Restart the tomcat service Note: For detailed information on securing the Tomcat on Windows refer to SSL/TLS Configuration How-TO . Note HTTPS setup is only for the Tomcat where ThingWorx is deployed and does not involve certificate setup on the Android Application side Finally,to test if the HTTPS setup was successful or not navigate to https://<serverName/IP>:8443/Thingworx in a web browser. Port 8443 is the default HTTPS port. Starting and connecting Android Application to ThingWorx Now that we have a working secured Tomcat and Android Studio setup with the sample Android Application, androidShell. Let's build and run the application using an emulated Android device in Android Studio 1. Navigate to https://localhost:8443/Thingworx > Import/Export > Import from file and import the Thing entity which will connect to the SampleThing when you’ll run the Android application, for e.g. I imported the Thingworx-Android-SDK-X.X.X\samples\android-shell\entity\Things_AndroidSampleThing.xml because I will be running the androidShell application 2. Attempting to run the Android application without the entity created in ThingWorx first, Thing will show as unbound in RemoteThing. 3. Create an AppKey in ThingWorx > Security > Application Keys with sufficient rights 4. Go to the Android Studio's toolbar and click on AVD Manager icon 5. This will open the Android Virtual Device Manager and lets you create a virtual Android device 6. You can of course use your own actual device to connect over USB and install and test the application on actual hardware 7. If you already have a Virtual or actual device connected to the system, click on Make Project icon in the toolbar 8. Once the Make finishes run the sample application, in my case androidShell application with the Run icon, like so 9. You may now be prompted with the options to select a device virtual or actual 10. Select as desired and click OK 11. This will now launch the application on the selected device, I have selected to launch on the virtual device which will start and emulated Android Device 12. When initiating/running the application for the first time you will be directed to the Settings screen allowing you to enter the connection URI and the Application Key to connect to the ThingWorx server 13. You have to follow one of the following two URI schemes while attempting to connect to a ThingWorx Server a. For HTTP connection use : ws://<machineIP/Name>:8080/Thingworx/WS b. For HTTPS connection use : wss://<machineIP/Name>:8443/Thingworx/WS Note: Ports may differ as these are the default ports, if you are running ThingWorx on different port enhance the URI accordingly 14. Since my ThingWorx is reachable on HTTPS connection i'll use the HTTPS connectino URI scheme and the application key that I have already created in the ThingWorx, which is bound to Administrator user 15. Once done press the back button on the screen to initiate the connection attempt 16. If all's set as it should be you will be able to see the Connected to Server option checked and the Property count for the Serial Number Property being updated every second For more detail on ThingWorx Android SDK refer to the ThingWorx Edge SDKs and WebScocket based Edge MicroServer (WS EMS) Help Center
View full tip
ThingWorx 7.4 covers the following areas of the product portfolio:  ThingWorx Analytics, ThingWorx Utilities and ThingWorx Foundation which includes Core, Connection Server and Edge capabilities. Key Functional Highlights Highlights of the release include: Source Integration: Improved integration framework making it easy to connect with external systems, including a standard Windchill connector, to retrieve data on demand. Industrial Connectivity: New Industrial Gateway and Discover navigation tool simplifying the mapping of tags to properties, including performance enhancements for data updates. Edge/CSDK: Build process improvements, Subscribed Property Manager (SPM) enhancements, asynchronous service requests and TLS updates to increase developer productivity, improve application performance and strengthen security. AWS IoT Connector: The latest version 1.2 of the Connector allows customers to more fully leverage their investment in AWS IoT. It features improved deployment automation via CloudFormation and automatic extension installation, ThingWorx Edge JavaScript SDK for use with the Connector with support for properties, services and events, and just-in-time certificate registrations. Contextualize Next Generation Composer: Re-imagined Composer using modern browser concepts to improve developer efficiency including enhanced functionality, updated user interface and optimized workflows. Engage These features will be available at the end of March 2017. Advanced Grid: New grid widget with improved design, context menu, multi-column sorting, global search and many more common grid-related features. Tree Grid: New tree-grid widget with same features as advanced grid plus ability to pre-load tree levels, dynamically load child data and auto expand all nodes to build more powerful mashups. Administer & Manage MS SQL Server: New persistence provider for model and run-time data providing customers an alternative to PostgreSQL. Security: ThingWorx worked with industry standard security scanning and auditing tools to identify and correct all non-trivial, and many trivial, vulnerabilities to ensure secure software best practices. Licensing: Link ThingWorx to PTC systems of record to manage user entitlement and provide usage information and auditing capability critical to TWX, PTC and its partners.  Documentation ThingWorx 7.4 Reference Documents ThingWorx Core 7.4 Release Notes ThingWorx Core Help Center ThingWorx Edge SDKs and WebSocket-based Edge MicroServer Help Center ThingWorx Connection Services Help Center ThingWorx Utilities Help Center Additional information ThingWorx eSupport Portal ThingWorx Developer Portal ThingWorx Marketplace Download ThingWorx Platform – Select Release 7.4 ThingWorx Edge C SDK 1.4 – Select Most Recent Datecode, C-SDK-1-4-0 ThingWorx AWS IoT Connector 1.2 – Select Release 7.4 ThingWorx Utilities – Select Release 7.4
View full tip
  PLEASE NOTE DataConnect has now been deprecated and no longer in use and supported.   We are regularly asked in the community how to send data from ThingWorx platform to ThingWorx Analytics in order to perform some analytics computation. There are different ways to achieve this and it will depend on the business needs. If the analytics need is about anomaly detection, the best way forward is to use ThingWatcher without the use of ThingWorx Analytics server. The ThingWatcher Help Center is an excellent place to start, and a quick start up program can be found in this blog. If the requirement is to perform a full blown analytics computation, then sending data to ThingWorx Analytics is required. This can be achieved by Using ThingWorx DataConnect, and this is what this blog will cover Using custom implementation. I will be very pleased to get your feedback on your experience in implementing custom solution as this could give some good ideas to others too. In this blog we will use the example of a smart Tractor in ThingWorx where we collect data points on: Speed Distance covered since last tyre change Tyre pressure Amount of gum left on the tyre Tyre size. From an Analytics perspective the gum left on the tyre would be the goal we want to analyse in order to know when the tyre needs changing.   We are going to cover the following: Background Workflow DataConnect configuration ThingWorx Configuration Data Analysis Definition Configuration Data Analysis Definition Execution Demo files   Background For people not familiar with ThingWorx Analytics, it is important to know that ThingWorx Analytics only accepts a single datafile in a .csv format. Each columns of the .csv file represents a feature that may have an impact on the goal to analyse. For example, in the case of the tyre wear, the distance covered, the speed, the pressure and tyre size will be our features. The goal is also included as a column in this .csv file. So any solution sending data to ThingWorx Analytics will need to prepare such a .csv file. DataConnect will perform this activity, in addition to some transformation too.   Workflow   Decide on the properties of the Thing to be collected, that are relevant to the analysis. Create service(s) that collect those properties. Define a Data Analysis Definition (DAD) object in ThingWorx. The DAD uses a Data Shape to define each feature that is to be collected and sends them to ThingWorx Analytics. Part of the collection process requires the use of the services created in point 2. Upon execution, the DAD will create one skinny csv file per feature and send those skinny .csv files to DataConnect. In the case of our example the DAD will create a speed.csv, distance.csv, pressure.csv, gumleft.csv, tyresize.csv and id.csv. DataConnect processes those skinny csv files to create a final single .csv file that contains all these features. During the processing, DataConnect will perform some transformation and synchronisation of the different skinny .csv files. The resulting dataset csv file is sent to ThingWorx Analytics Server where it can then be used as any other dataset file.     DataConnect configuration   As seen in this workflow a ThingWorx server, DataConnect server and a ThingWorx Analytics server will need to be installed and configured. Thankfully, the installation of DataConnect is rather simple and well described in the ThingWorx DataConnect User’s guide. Below I have provided a sample of a working dataconnect.conf file for reference, as this is one place where syntax can cause a problem:   ThingWorx Configuration The platform Subsystem needs to be configured to point to the DataConnect server . This is done under SYSTEM > Subsystems > PlatformSubsystem:     DAD Configuration The most critical part of the process is to properly configure the DAD as this is what will dictate the format and values filled in the skinny csv files for the specific features. The first step is to create a data shape with as many fields as feature(s)/properties collected.   Note that one field must be defined as the primary key. This field is the one that uniquely identifies the Thing (more on this later). We can then create the DAD using this data shape as shown below   For each feature, a datasource needs to be defined to tell the DAD how to collect the values for the skinny csv files. This is where custom services are usually needed. Indeed, the Out Of The Box (OOTB) services, such as QueryNumberPropertyHistory, help to collect logged values but the id returned by those services is continuously incremented. This does not work for the DAD. The id returned by each services needs to be what uniquely identifies the Thing. It needs to be the same for all records for this Thing amongst the different skinny csv files. It is indeed this field that is then used by DataConnect to merge all the skinny files into one master dataset csv file. A custom service can make use of the OOTB services, however it will need to override the id value. For example the below service uses QueryNumberPropertyHistory to retrieve the logged values and timestamp but then overrides the id with the Thing’s name.     The returned values of the service then needs to be mapped in the DAD to indicate which output corresponds to the actual property’s value, the Thing id and the timestamp (if needed). This is done through the Edit Datasource window (by clicking on Add Datasource link or the Datasource itself if already defined in the Define Feature window).   On the left hand side, we define the origin of the datasource. Here we have selected the service GetHistory from the Thing Template smartTractor. On the right hand side, we define the mapping between the service’s output and the skinny .csv file columns. Circled in grey are the output from the service. Circled in green are what define the columns in the .csv file. A skinny csv file will have 1 to 3 columns, as follow: One column for the ID. Simply tick the radio button corresponding to the service output that represents the ID One column representing the value of the Thing property. This is indicated by selecting the link icon on the left hand side in front of the returned data which represent the value (in our example the output data from the service is named value) One column representing the Timestamp. This is only needed when a property is time dependant (for example, time series dataset). On the example the property is Distance, the distance covered by the tyre does depend on the time, however we would not have a timestamp for the TyreSize property as the wheel size will remain the same. How many columns should we have (and therefore how many output should our service has)? The .csv file representing the ID will have one column, and therefore the service collecting the ID returns only one output (Thing name in our smartTractor example – not shown here but is available in the download) Properties that are not time bound will have a csv file with 2 columns, one for the ID and one for the value of the property. Properties that are time bound will have 3 columns: 1 for the ID, one for the value and one for the timestamp. Therefore the service will have 3 outputs.   Additionally the input for the service may need to be configured, by clicking on the icon.   Once the datasources are configured, you should configure the Time Sampling Interval in the General Information tab. This sampling interval will be used by DataConnect to synchronize all the skinny csv files. See the Help Center for a good explanation on this.   DAD Execution Once the above configuration is done, the DAD can be executed to collect properties’ value already logged on the ThingWorx platform. Select Execution Settings in the DAD and enter the time range for property collection:   A dataset with the same name as the DAD is then created in DataConnect as well as in ThingWorx Analytics Server Dataconnect:     ThingWorx Analytics:   The dataset can then be processed as any other dataset inside ThingWorx Analytics.   Demo files   For convenience I have also attached a ThingWorx entities export that can be imported into a ThingWorx platform for you to take a closer look at the setup discussed in this blog. Attached is also a small simulator to populate the properties of the Tractor_1 Thing. The usage is : java -jar TWXTyreSimulatorClient.jar  hostname_or_IP port AppKey For example: java -jar TWXTyreSimulatorClient.jar 192.168.56.106 8080 d82510b7-5538-449c-af13-0bb60e01a129   Again feel free to share your experience in the comments below as they will be very interesting for all of us. Thank you
View full tip
Validator widgets provide an easy way to evaluate simple expressions and allow users to see different outcomes in a Mashup. Using a validator is fairly intuitive for simple expressions, such as is my field blank? But if we need to evaluate a more complex scenario based on multiple parameters, then we can user our validator with a backing service that will perform more complex analytics. To show how services can work in conjunction with a validator widget, let’s consider a slightly more complicated scenario such as: A web form needs to validate that the zip or postal code entered by the user is consistent with the country the user selected. Let’s go through the steps of validating our form: Create a service with two input parameters. Our service will use regex to validate a postal code based on the user’s country.  Here’s some sample code we could use on an example Thing: //Input parameters: Country and PostalCode (strings) //Country-based regular expressions: var reCAD = /^[ABCEGHJKLMNPRSTVXY]{1}\d{1}[A-Z]{1} *\d{1}[A-Z]{1}\d{1}$/; var reUS = /^\d{5}(-\d{4})?$/; var reUK = /^[A-Za-z]{1,2}[\d]{1,2}([A-Za-z])?\s?[\d][A-Za-z]{2}$/; var search = ""; //Validate based on Country: if (Country==="CAD")                search = reCAD.exec(PostalCode); else if (Country==="US")                search = reUS.exec(PostalCode); else if (Country==="UK")                search = reUK.exec(PostalCode); (search == null) ? result = false: result = true; Set up a simple mashup to collect the parameters and pass them into our service Add a validator widget Configure the validator widget to parse the output of the service. ServiceInvokeComplete on the service should trigger the evaluation and pass the result of the service into a new parameter on the widget called ServiceResult (Boolean). The expression for the evaluator would then be: ServiceResult? true:false Based on the output of the validator, provide a message confirming the postal code is valid or invalid Add a button to activate the service and the validator evaluation Of course, in addition to providing a message we can also use the results of the validator to activate additional services (such as writing the results of the form to the database). For an example you can import into your ThingWorx environment, please see the attached .zip file which contains a sample mashup and a test thing with a validator service.
View full tip
Hi everybody, In this blogpost I want to share with you my local ThingWorx installation, with some optimizations that I did for local development. -use the -XX:+UseConcMarkSweepGC . This uses the older Garbage Collector from the JVM, instead of the newer G1GC recommended by the ThingWorx Installation guide since version 7.2. The advantage of ConcMarkSweepGC is that the startup time is faster and the total memory footprint of the Tomcat is far lower than G1GC. -use -agentlib:jdwp=transport=dt_socket,address=1049,server=y,suspend=n. This allows using your Java IDE of choice to connect directly to the Tomcat server, then debugging your Extension code, or even the ThingWorx code using the Eclipse Class Decompilers for example. Please modify the 1049 to your port of choice for exposing the server debugging port. -use -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=60000 -Dcom.sun.management.jmxremote.ssl=false                  -Dcom.sun.management.jmxremote.authenticate=false           This sets up the server to allow JMX monitoring. I usually use VisualVM from the JDK bin folder, but you can use any JMX monitoring tool.           This uses no Authentication, no SSL and uses port 6000 - modify if you need. I usually startup Tomcat manually from a folder via startup.bat, and the setenv.bat looks like: set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_102 set JRE_HOME=C:\Program Files\Java\jdk1.8.0_102 set THINGWORX_PLATFORM_SETTINGS=D:\Work\servers\apache-tomcat-8.0.33 // this is where the platform-settings.json file is located set CATALINA_OPTS=-d64 -XX:+UseNUMA -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF-8 -agentlib:jdwp=transport=dt_socket,address=1049,server=y,suspend=n -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=60000 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false In this mode I can look at any errors in almost real time from the console and it makes killing the server for Java Extension reload a breeze -> Ctrl+C Please don't hesitate to provide feedback on this document, I certainly welcome it. Be warned: THESE ARE NOT PRODUCTION SETTINGS. Best regards, Vladimir
View full tip
In this video we cover: a short introduction of Thingworx Analytics Builder The import of the Thingworx Analytics Builder extension   This video applies to ThingWorx Analytics 52.1 till 8.1   Updated Link for access to this video:  Installing Thingworx Analytics Builder:  Part 1 of 3
View full tip
Best Practices in Data Preparation for ThingWorx Analytics Data Preparation is an important phase in the process of Data Analysis when using ThingWorx Analytics. Basically, it is getting your Data from being Raw Data that you might have gathered through your Operational system or from your Data warehouse to the kind of Data ready to be analyzed. In this Document we will be using “Talend Data Preparation Free Desktop” as a Tool to illustrate some examples of the Data Preparations process. This tool could be downloaded under the following Link: https://www.talend.com/products/data-preparation (You could also choose to use another tool) We would also use the Beanpro Dataset in our Examples and illustrations. Checking data formats The analysis starts with a raw data file. The user needs to make sure that the data files can be read. Raw data files come in many different shapes and sizes. For example, spreadsheet data is formatted differently than web data or Sensors collected data and so forth. In ThingWorx Analytics the Data Format acceptable are CSV. So the Data retrieved needs to be inputted into that format before it could be uploaded to TWA Data Example (BeanPro dataset used): After that is done the user needs to actually look at what each field contains. For example, a field is listed as a character field could actually contains none character data. Verify data types Verifying the data types for each feature or field in the Dataset used.  All data falls into one of four categories that affect what sort of analytics that could be applied to it: Nominal data is essentially just a name or an identifier. Ordinal data puts records into order from lowest to highest. Interval data represents values where the differences between them are comparable. Ratio data is like interval data except that it also allows for a value of 0. It's important to understand which categories your data falls into before you feed it into ThingWorx Analytics. For example when doing Predictive Analytics TWA would not accept a Nominal Data Field as Goal. The Goal feature data would have to be of a numerical non nominal type so this needs to be confirmed in an early stage.                 Creating a Data Dictionary A data dictionary is a metadata description of the features included in the Dataset when displayed it consists of a table with 3 columns: - The first column represents a label: that is, the name of a feature, or a combination of multiple (up to 3) features which are fields in the used Dataset. It points to “fieldname” in the configuration json file. - The second column is the Datatype value attached to the label. (Integer, String, Datetime…). It points to “dataType” in the configuration json file. - The third column is a description of the Feature related to the label used in the first column. It points to “description” in the configuration json file. In the context of TWA this Metadata is represented by a Data configuration “json” file that would be uploaded before even uploading the Dataset itself. Sample of BeanPro dataset configuration file below: Verify data accuracy Once it is confirmed that the data is formatted the way that is acceptable by TWA, the user still need to make sure it's accurate and that it makes sense. This step requires some knowledge of the subject area that the Dataset is related to. There isn't really a cut-and-dried approach to verifying data accuracy. The basic idea is to formulate some properties that you think the data should exhibit and test the data to see if those properties hold. Are stock prices always positive? Do all the product codes match the list of valid ones? Essentially, you're trying to figure out whether the data really is what you've been told it is. Identifying outliers Outliers are data points that are distant from the rest of the distribution. They are either very large or very small values compared with the rest of the dataset. Outliers are problematic because they can seriously compromise the Training Models that TWA generates. A single outlier can have a huge impact on the value of the mean. Because the mean is supposed to represent the center of the data, in a sense, this one outlier renders the mean useless. When faced with outliers, the most common strategy is to delete them. Example of the effect of an Outlier in the Feature “AVG Technician Tenure” in BeanPro Dataset:   Dataset with No Outlier: Dataset with Outlier: Deal with missing values Missing values are one of the most common (and annoying) data problems you will encounter. In TWA dealing with the Null values is done by one of the below methods: - Dropping records with missing values from your Dataset. The problem with this is that missing values are frequently not just random little data glitches so this would consider as the last option. - Replacing the NULL values with average values of the responses from the other records of the same field to fill in the missing value Transforming the Dataset - Selecting only certain columns to load which would be relevant to records where salary is not present (salary = null). - Translating coded values: (e.g., if the source system codes male as "1" and female as "2", but the warehouse codes male as "M" and female as "F") - Deriving a new calculated value: (e.g., sale_amount = qty * unit_price) - Transposing or pivoting (turning multiple columns into multiple rows or vice versa) - Splitting a column into multiple columns (e.g., converting a comma-separated list, specified as a string in one column, into individual values in different columns) Please note that: Issue with Talend should be reported to the Talend Team Data preparation is outside the scope of PTC Technical Support so please use this article as an advisable Best Practices document
View full tip
This Guide contains all the Linux commands that you may have to use for ThingWorx Analytics Installation or day to day use. Command/Category Description Network/port ip a List the ips of all of the network interfaces ssh How to jump from one machine to another ping Send packets to a remote machine.  useful for testing connectivity netstat –anp Check active port cat < /dev/tcp/localhost/8080 Test connection to a port Replace localhost with desired hostname or ip, replace 8080 with desired port number (/dev/tcp/host/port) exit Exit my current sign in.  this lets one disconnect from remove ssh sessions or if one has changed one's user e.g. switched to root scp Retrieve something via ssh Resource Usage free -m Check memory -m is for output in Mb Mpstat -P ALL CPU usage top Process usage jvmtop Collect cpu usage of jvm and its thread https://github.com/patric-r/jvmtop (requires jdk to be installed) File Interaction cp / mv Copy and move respectively. mv just deletes the source file.  Usage: cp /source/location/file /output/location/file cat Mostly used to just print the contents of a file to the command line.  can also print multiple files at once:  cat /var/log/gridworker/warning.log /var/log/gridworker/error.log vim / vi A command line text editor. Not the most user friendly (none of them are) but really useful. Here's a cheat sheet for the commands https://www.fprintf.net/vimCheatSheet.html rm Remove files chmod Change the access permissions of files chown Change the user or group ownership of files grep A text based filtering.  Useful for making a larger list smaller and more targeted.  Almost always used after a pipe (see pipe below) less Generally used to view the contents of a file with more friendly scrolling locate Find a file by name Directory ls What’s in the directory.  Will do the current directory but you can also pass the directory e.g. ls /var/log/tomcat. Black writing is files Blue writing is directories Red writing is compressed file pwd Tell me which directory I'm currently in cd Change directory.  provide the directory to change to or just use cd to return to the user's home directory clear Clears the screen Terminal clears all provided commands mkdir Creates Directories Running Processes ps Query what services are running. usually use ps -aux to get a full, sorted list.  using grep with this is helpful systemctl The correct way to interact with services that are running Package installation yum install <packageName> Install a package. More useful commands: https://www.centos.org/docs/5/html/5.1/Deployment_Guide/s1-yum-useful-commands.html yum list installed List installed packages yum list <package> List available packages yum --showduplicates list java-1.7.0-openjdk-devel Use --showduplicates to see all versions Can use * for package name: *openjdk* rpm -ql <packagename> Find where package are installed Note: works if package installed with yum Yumdownloader --urls <packageName> Find URL where a package is downloaded from. Note: need to install yum-utils package Repoquery --requires <packageName> Find dependencies of a package Note: need to install yum-utils package repoquery --qf=%{name} -g --list --grouppkgs=all [groups] | xargs repotrack -a x86_64 -p /repos/Packages Download a package with all its dependencies. Need to install yum-utils package From  <http://unix.stackexchange.com/questions/50642/download-all-dependencies-with-yumdownloader-even-if-already-installed> Other Commands curl http://localhost:8080/1.0/about/versioninfo Send REST call via command line Use -X POST (default GET) for a POST (see man page - https://curl.haxx.se/docs/manual.html for example) See also http://www.codingpedia.org/ama/how-to-test-a-rest-api-from-command-line-with-curl/ Find / -type f -exec grep -I mystring {} \; Search string in files Sudo -u user command Execute a command as different user The below helpers are not commands themselves, but can be used in conjunction with the above commands. Helper Description 'pipe' The | character.  lets one chain commands.  e.g. ps -aux | grep java ./ The shorthand way to refer to this directory explicitly ../ The shorthand way to refer to the parent directory 'tab completion' Pressing tab will let linux guess what command/option best fits what's currently written.  very useful for navigating directories and long-named files (NOTE: not necessarily tab based upon one's keyboard layout/language) 'ctrl-r' Look up the mostly likely command that matches what one typing.  so if one earlier used ps -aux | grep java | less and the hit ctrl-r and typed -aux it would likely pull that command or at least the most recent one that matches
View full tip
This is a reference document on how to move existing or fresh install ThingworxStorage location. Note: As of Thingworx 7.0 the platform-settings.json file has changed and therefore the newest version needs to be used  to change the location of the folders. Please refer to this guide pg.45-46 to see the new .json file.http://support.ptc.com/WCMS/files/170230/en/Installing_ThingWorx_7.1.pdf  A sample json is attached at the bottom of this document. Here are the main bullet points that need to be kept in mind when performing a migration process: ThingworxPlatform directory has to be in the same partition as where the Tomcat is installed. I.e. if your tomcat is in /apps, the ThingworxPlatform should also be in /apps, not in root. Note: if ThingworxPlatform will be placed in a different partition than the tomcat, create a THINGWORX_PLATFORM_SETTINGS environmental variable that will point to the correct location. Refer to the Installation Guide for details. Before modifying the location of the platform-settings.json, stop tomcat. There is no need to undeploy the Thingworx app, but do rename the existing folders (as a way of copying/preserving data if any exists) “ThingworxStorage” to “ThingworxStorageCopy” and “ThingworxBackupStorage” to “ThingworxBackupStorageCopy”. If this is a fresh install, proceed to step 3.. Modify the platform-settings.json to have desired locations for the ThingworxStorage folders. Start the tomcat. New ThingworxStorage and ThingworxBackupStorage will be created in the new location. Note: if there is any data in the ThingworxStorage (see ThingworxStorageCopy and ThingworxBackupStorageCopy from step 2), stop the tomcat and place the contents of the original folders into the new ones or just replace the directories. For Postgresql  scenarios: Ensure postgresql database is running. If fresh install/no need to preserve data,  and in case of 401 error, clean the existing database via the script and run the two scripts to install and setup DB and Schema. Start tomcat and verify connection. If data does need to be preserved, then run the schema update script to update from 6.5 to 6.6. then to 7.0 and 7.1 respectively, depending on which version you are starting with. To change the location of where postgresql would write data, find the thingworxPostgresDBSetup.sh script open it to edit: tablespace_location="/ThingworxPostgresqlStorage" Then re-run the DBsetup and Schema setup while tomcat is stopped. For assistance in resolving any difficulties related to the migration process, please contact Thingworx Technical Support.
View full tip
Introduction The Edge MicroServer (EMS) and Lua Script Resource (LSR) are Edge software that can be used to connect remote devices to the ThingWorx platform. Using a Gateway is beneficial because, this will allow you to run one instance of the EMS on a server and then many instances of the LSR on different devices all over the world. All communication to the platform will be handled by this one EMS Gateway server. The EMS Gateway can be set up in two different types of scenarios: Self-Identifying Remote Things and Explicitly defined Remote Things​. The scenario I'm going to discuss below will involve explicitly defined Remote Things, a ThingWorx server, an EMS, and a LSR. We will need at least 1 server to run the ThingWorx platform and EMS, but these can always be on separate servers as well. We will also need some other machine or device that will run the LSR. Visit the support downloads page to find the latest EMS releases. The LSR is contained within the EMS download. You can also navigate to the Edge Support site to read more about the EMS and LSR oif this is the first time you have ever configured one. The "ThingWorx WebSocket-based Edge MicroServer Developer's Guide" is also provided inside of the zip file that contains the EMS for further information. Setting up the EMS Once we have obtained the EMS download from the support site (see the section above for links) we can begin creating our config.json file. The image below is a working config.json file for using the EMS as a Gateway. The settings in here are particular to my personal IP addresses and Application Key, but the concept remains the same, and I will go into further detail on the necessary sections, below the image. ws_servers The host and port parameters are always set to the IP address and port that the ThingWorx platform is being hosted on When the EMS and ThingWorx platform are on the same server, "localhost" can be used instead of an IP address appKey The appKey section is the value of an Application Key in the ThingWorx platform that should be used for the authentication of the EMS to the platform An Application Key will need to be created and assigned a user with proper priveledges prior to authenticating certificates The certificates section should be validating and pointing to proper certificates, but in the example above I am not validating any certificates for the sake of simplicity More can be read about the certificates sections here logger The logging section is out of scope of this article, but further reading on ​logger​ configurations can be found here The section in the example above will work for basic logging needs http_server The http_server section configuration parameters will tell the EMS what host and port to spin up a server on and if there is authentication necessary by any LSRs trying to connect The LSR has settings that will explicitly call out whatever value is set to the host and port in this section, so make sure to set these to an open port that is not in use or blocked behind a firewall Further reading on the http_server section can be found here auto_bind You can see above that there are two objects defined in the auto_bind section. One of these is binding the EMS to an EMSGateway Thing in the platform called "EdgeGateway" and the other is defined in the config.lua file for the LSR The gateway parameter is set to true only in the object, "EdgeGateway", that is being used for the EMS to bind to The host and port defined for the "OtherEdgeThing" should point to the port and IP address that the LSR is running on in the other device By default, the LSR runs on port 8001, but you can always double check the listening port by finding the Process Identification (PID) number of the luaScriptResource.exe and then matching the PID to the corresponding line item in the output of netstat -ao command in a console window The protocol can be set to "http" in an example application, but make sure to use "https" when security is of concern All further reading on the sections of the config.json file can be found in the config.json.complete file included with the EMS download and on the Edge Help Center under the "Creating a Configuration File" section and the "Viewing All Options" section. Setting up the LSR In this example, the LSR is going to run on a separate server and point to the EMS server. Below is a screenshot of two very important additions (rap_host and rap_port) to the default config.lua file: rap_host The rap_host field should be set to the IP address where the EMS is hosted rap_port The rap_port field should be set to the port parameter defined in the config.json http_server section script_resource_host The ​script_resource_host​ field must be set to ensure that the EMS will know what IP address to communicate with the LSR at scripts.OtherEdgeThing This line is necessary to identify what the name of the LSR is that will register with the EMS to bind to the platform "OtherEdgeThing" can be changed to anything, but make sure that the auto_bind section in the config.json aligns with what you've defined in the config.lua file at this line Running the EMS and LSR Now that we have configured the LSR and EMS to point to each other and the platform we can try running both of these applications to make sure we are successful. Make sure the ThingWorx platform is running Create a RemoteThing with the name given in the auto_bind section for the LSR we are connecting Create an EMSGateway with the name given in the auto_bind section for the EMS as a Gateway to bind to Start the EMS This can be done by double clicking the wsems.exe when in Windows, running it as a service, or running it directly from the command line Start the LSR This can be done by double clicking the luaScriptResource.exe when in Windows, running it as a service, or running it directly from the command line Navigate to the ThingWorx platform and make sure that the Things you have created are connected Do this by navigating to the Properties menu option and refreshing the isConnected property You should be able to browse remote properties and services for each bound RemoteThing, and this means you have successfully setup the EMS as a Gateway device to external LSR applications running on remote devices Any further questions about browsing remote properties or other configuration settings in the .config files is most likely addressed in the Edge Help Center under the EMS section​, and if not, feel free to comment directly on this document.
View full tip
I recently had a customer who wanted to run services on ThingWorx from Power BI to retrieve existing operational data, and we were a bit stumped on how to pass the API key over in the headers, so I did a bit of Googling and pieced together the solution. It's not quite intuitive on the Power BI side, so I thought it would be helpful to share. If you have any other experience with integrating ThingWorx with Power BI, feel free to add a comment.    Prepare ThingWorx Create an Application Key that has Run Time execution access to the services you need. Understand the inputs needed for the service you would like. I'll have examples of none, one, an InfoTable, and multiple inputs.   Power BI Following the following steps in Power BI: 1. In Power BI, create a new blank query   2. On the left, right click on Query1 and go to the Advanced Editor:   3. Replace all of the body content with the following, replacing your API key, appropriate end point, and base URL as needed (this is an example with NO input parameters, I'll follow with examples of other parameters):     let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source       4. Click "Done", and now you'll have a warning about how to connect. Click the "Edit Credentials" button. 5. Leave it on Anonymous and click "Connect":   6. You should now see the return data coming from ThingWorx.   Note that I had a little trouble with this authentication initially and it saved the wrong method. To clear that out, go to the ribbon bar item "Data source settings" and select the server and clear it out.   Other Examples Here is an example for sending a single string parameter:   let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""InputParameter"": ""InputValue""}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source     Here's an example of sending a string and an integer: let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""InputString"": ""Hello, world!"", ""InputNumber"" : 42}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source   Here is an example for sending an InfoTable. Note that you must supply the dataShape with fieldDefinitions. If you're using an existing Data Shape, you can get the JSON by using the service GetDataShapeMetadataAsJSON() that is on the data shape.     let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""propertyNames"": { ""rows"": [ { ""name"": ""FirstEntityName"", ""description"": ""The first entity"" }, { ""name"": ""SecondEntityName"", ""description"": ""The second entity"" }], ""dataShape"": { ""fieldDefinitions"": { ""name"": { ""name"": ""name"", ""aspects"": { ""isPrimaryKey"": true }, ""description"": ""Entity name"", ""baseType"": ""STRING"", ""ordinal"": 0 }, ""description"": { ""name"": ""description"", ""aspects"": {}, ""description"": ""Entity description"", ""baseType"": ""STRING"", ""ordinal"": 0 } } } }}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source       If I find any more interesting ways to use Power BI with ThingWorx services, I'll add them on here.  
View full tip
Now that ThingWorx 9.3 is live, let’s take a closer look at some of our new features we released for Composer and Mashup Builder.   Referenced By Find where entities, code references, and project dependencies are used in your existing projects using the new “Referenced by” report feature. This feature is not automatically enabled because it is intended to be used during development, since it can call upon all of the entities in your project and can impact your load times in production. That being said, this is your friendly reminder to turn off this feature during production.     How to enable: Go to the relationship subsystem and tick the check box to enable during development.   How it works: The “Referenced By” feature finds any entity that is referenced in your ThingWorx environment based on the supplied search characteristics. You can run this “Referenced By” report in the Composer or via a ThingWorx service “GetWhereUsed”.  When you use the “Referenced By” feature on an entity, you can find all of the references in the system for that entity in Mashup bindings, service script references, thing property bindings, and more.     Grid Widget The new grid widget component, which was available for preview in ThingWorx 9.2, is now complete, so it’s time to get your grid on! We have improved styling and performance capabilities with this new widget, including greater support for inline editing, autocreation of columns based on infotables, and adding new footer areas. You can also configure the grid using the property editor in Mashup Builder, where previously we had plain text entry.   Style Migration The new Style Migration is a game changer. It allows you to retain the same look and feel of your Mashups as you upgrade from previous versions of ThingWorx to the newest web components and features available in 9.3. Improved from our previous migrator, this allows you to move to the latest platform version and capabilities without having to re-implement or redesign your applications and widgets.     How it works: When you upgrade to the latest version of ThingWorx, you will see a pop-up window appear if you have any legacy widgets or layouts in your Mashup. The window will have the option for you to apply one of three style themes to your Mashup: PTC Convergence Theme (the new ThingWorx Default theme), Legacy Styles Theme (the old ThingWorx theme, from version 8.0 and earlier), or Custom Theme (choose from custom themes you defined using the Theme Editor and Style Theme that will appear when custom theme is selected in the pop-up). Depending on how you already styled or would like to style your Mashups, select an option and click migrate. This migrator maintains previous coloring, spacing, and other design properties better than previous migrators. You, of course, have the option to not upgrade your Mashup, but we recommend that you migrate, especially where we have new widgets available to replace legacy versions. If there are any issues with your migration, you can always click “Undo” in the toolbar.   Things to Consider: This migration will work best with ThingWorx default styling, out of the box styling, and Mashups with widgets that we now have replacements for (these are marked legacy in the builder). Always make sure you review your Mashups to make sure bindings and properties remained. Note that custom CSS will not be migrated, and custom widgets developed outside the standard platform installation will remain the same on the new Mashup.   Other Bonus Features That’s not all we rolled out in ThingWorx 9.3. You will also see Composer enhancements for test execution on ThingTemplates and dynamic use of Master Mashups to allow for swapping out Masters at run time based on predefined conditions based on your users. Plus, we now have truncation support for the breadcrumb component and tabs component, which utilizes an ellipsis pattern for long  text for a more user-friendly application.  With enhancements to our charts, you can now show/hide legends and format axis in new ways. We also support localization for our new web component widgets, .   How has your experience been building solutions with the latest updates to Composer and Mashup Builder? How can we continue to build upon these enhancements? Let us know what you think.   Stay connected, Rachel
View full tip
Background Getting a performance benchmark of your running application is an important thing to do when deploying and scaling up an application in production.  This not only helps focus in on performance issues quickly, but also allows for safely planning for scaling up and resource sizing based on real concrete data.   I recently created a tool and made a post about capturing and analysing ThingWorx utilisation statistics to do such an analysis, as well as identifying potential performance bottlenecks. Although they are rich and precise, utilisation statistics fall short in a number of areas however - specifically being able to count and time specific service executions, as well as identifying and sorting based on the host executing the service.   Tomcat Access Log Analysis As ThingWorx is a Tomcat web application, Tomcat logs details of the requests being made to the application server and ThingWorx REST API.  The default settings include the host (IP address), date/timestamp, and request URI; which can be decoded to reveal relevant details like the calling entities and service executions.   Adding 3 key additional variables (%s %B %D) to the server.xml access log value also gives us the HTTP response code, service execution time, and bytes returned from Tomcat.  This is super useful as we can now determine exact time of service executions, and run statistics on their execution totals and execution time.     Once you have an access log file looking like the one above, you can attempt to load it into the access_log sheet in the analysis Excel workbook that I created.  You do this by click on the access_log table, then selecting "Data > Get Data > Data Source Settings".  You'll then be prompted with the following or similar pop-up allowing you to navigate to your access_log file to select and then load.     It should be noted that you'll have to Refresh the table after selecting the new access_log.txt file so that it is read in and populates the table.  You can do this by right-clicking on the table and saying Refresh, or using the Data > Refresh button.   This workbook relies on a number of formulas to slice and dice the timestamp, and during my attempts at importing I had significant issues with this due to some of the ways that Excel does things automatically without any manual options.  You really need to make sure that the timestamps are imported and converted correctly, or something in the workbook will likely not work as intended.  One thing that I had to do was to add 1 second to round up 00:00:00 for the first entries as this was being imported as a date without the time part, and then the next lines imported as a date/time.   Depending on how many lines your file is, you'll likely also have to "Fill Down" the formulas on the right side of the sheet which may be empty in the table after importing your new data set.  I had the best results by selecting the cells in question on the last row, then going down to the bottom corner, pushing and holding Shift, clicking on the last cell bottom right, and then selecting Home > Fill > Down to pull the formulas down from the top.   Once the data is loaded, you'll be able to start poking around.  The filters and sorting by the named columns is really helpful as you can start out by doing things like removing a particular host, sorting by longest execution times, selecting execution times greater than 4 seconds, or only showing activity aimed at a particular entity or service.     You really need to make sure that the imported data worked fine and looks perfect, as the next steps will totally break if not.  With the data loaded, you can now go to the Summary Data table and right-click on one of the tables and select Refresh.  This is reload the data in into the pivot table and re-run their calculations.   Once the refresh is complete, you should see the table summary like shown here; there are Day, Hour, and Minute expand/collapse buttons.  You should also see the Day, Hour, Month fields showing in the Field Definitions on the right.  This is the part that is painful -- if the dates are in the wrong format and Excel is unable to auto-detect everything in the same way, then you will not get these automatically created fields.     With the data reloaded, and Pivot Tables re-built, you should be able to go over to the Dashboard sheet to start looking at and analysing the graphs.  This one is showing the Top 10 services organised into hourly buckets with cumulated service execution times.     I'm not going to go into all of the workbooks features, but you can also individually select a set of key services that you want to have a look at together across both the execution count and execution time dimensions.     Next you can see the coordinated view of both total service execution time over number or service executions.  This is helpful for looking for patterns where a service may be executing longer but being triggered the same amount of times, compared to both being executed and taking more time.  I've created a YouTube video (see bottom) which goes through using all of the features as well as providing other pointers to using it.     Getting into a finer level of detail, this "bonus" sheet provides a Pivot Table and Pivot Chart which allows for exploring minimum, maximum and average execution time for a specific service.  Comparing this with the utilisation subsystem metrics taken during the same period now provide much deeper insight as we can pinpoint there the peaks were, how long they lasted, and where the slow executions were in relation to other services being executed at that time (example: identifying many queries/data processing occurring simultaneously).     Without further ado, you can download and play with my ThingWorx Tomcat Access Log Analysis Excel Workbook, and check out the recorded demonstration and explanation for more details on loading and analysis use. [YouTube] ThingWorx Tomcat Access Logs - Service Performance Analysis
View full tip
  Hi, everyone!     Today, we’re launching an exciting new series called “PTC Community Spotlights.” Each post in the series explores a community member’s experience with ThingWorx—how they’re using it, what their favorite part about ThingWorx is, and any tips or tricks they may have to share with the PTC Community.   For the first installment, I spoke with @nmilleson of EAC. Check out our conversation below. Our first PTC Community Spotlight Speaker -- Nick Milleson of EAC Product Development Systems. @Kaya: Hi, @nmilleson, welcome! Thank you for taking the time to meet with me and volunteering to be our first ThingWorx Community spotlight!   @nmilleson: Of course, @Kaya. Happy to be here.   @Kaya: To start, can you tell me a little about yourself?   @nmilleson: Absolutely. My name is Nick Milleson.  I work as an IoT Solution Architect at EAC Product Development Systems (a PTC Partner). I’m located in Apple Valley, Minnesota, which is a suburb of the Twin Cities.   @Kaya: Nice! We always love hearing from our partners about the awesome work they do. As a PTC Partner, what industries do you typically work in?   @nmilleson: I consult for many, many different industries, including defense, transportation, medical devices, construction & aerospace.   @Kaya: Wow, so what PTC products are you most familiar with?   @nmilleson:  My schooling is in mechanical engineering, so I’ve also used Creo, Windchill, and MathCAD.  I have been working with the ThingWorx application and helping clients get the most out of ThingWorx for approximately 7 years.   @Kaya:  Seven years—that’s a while! Do you have any “ThingWorx” stories from over the years you can share with your community peers?   @nmilleson:  Sure thing. I think the coolest thing that I’ve done with ThingWorx was create a custom SVG infographic that featured animations, click events, zoom-ins, and heatmaps based on temperature deltas.  It was a custom widget and it worked really well in ThingWorx.  When I first started learning to use ThingWorx, I took apart an old RC car and hooked up an Arduino to the motors and steering.  I was then able to control it using a ThingWorx mashup.  Pretty fun! I’ll be sure to share a visual so people can check it out.   Nick's awesome custom SVG infographic featuring a ton of neat functionality like zoom-ins & heatmaps. @Kaya:  That’s awesome! Sounds like a fun time indeed. I saw that one of your first publications about ThingWorx for EAC was from 2015 and titled “Updating ThingWorx Using an Arduino Uno and a Serial Connection.”  The ThingWorx platform has certainly evolved since then.  What would you say is your favorite thing about ThingWorx today?   @nmilleson:  It sure has evolved. I would say my favorite thing is that it’s flexible enough to allow you the freedom to design all sorts of applications, while also providing you with all these great tools that make it easy to use as well.   @Kaya:  Thanks for that. I can see that you have been a member of the PTC Community for five years.  Thank you for providing such great contributions.  What do you enjoy most about the PTC Community?   @nmilleson:  I enjoy this Community because everyone seems very willing to help each other out, regardless of the complexity of the issue.  I stick mostly with the IoT Developers section, but I’ll meander into the Manufacturing Apps and ThingWorx Ideas once in a while as well.   @Kaya:  Love to hear it. Now, so the PTC Community can learn a little more about you, how do you spend your time when you aren’t playing with ThingWorx or engaging on the PTC Community?   @nmilleson: Great question. I have been a professional piano player for almost 20 years, so I’m often at a piano bar making music when I’m not doing software development with EAC.   @Kaya: Awesome. Well those are all the questions I have for today. Thank you for sharing your experience with ThingWorx! Truly appreciate it.   @nmilleson: Of course. Happy to be a part of it!   Kaya, here. We love hearing from community members like @nmilleson about how ThingWorx creates value for them amongst a variety of use cases. If you’re active on the community and interested in being featured on the PTC Community Spotlight series, send me a direct message and we’ll get the ball rollin’.   For now, we’ll let Nick “play us” out. Until next time, stay connected!   -Kaya
View full tip