IoT Tips

This Expert Session is designed to help beginners get up and running with ThingWorx Analytics. It covers basic concepts like: What are APIs, how to configure the metadata file, and a live Demo that shows you how to interact and use ThingWorx Analytics in real time. This Expert Session would also be useful for experienced users who need a refresher course.   For full-sized viewing, click on the YouTube link in the player controls.   Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

Key Functional Highlights Production Advisor is now available in the Freemium and Developer Kit downloads. Plant Managers are provided with real-time monitoring of production status and critical KPI’s such as utilization, performance, quality and OEE, by unifying data from disparate lines, assets and sensors. With Production Advisor, Plant Managers have the ability to detect and react instantly to production issues- reaching lower downtime, higher production throughput and better quality from the factory resources. Compatibility ThingWorx 8.0.1 KEPServerEX 6.2 KEPServerEX V6,1 and older as well as different OPC Servers (with Kepware OPC aggregator) Documentation ThingWorx Manufacturing Apps Setup and Configuration Guide: https://support.ptc.com/WCMS/files/173133/en/ThingWorxManufacturingAppsSetup_8-0-1.pdf ThingWorx Manufacturing Apps Customization Guide: https://support.ptc.com/WCMS/files/173135/en/ThingWorxManufacturingAppsCust_8-0-1.pdf Get Started Documentation on Portal: https://www.ptc.com/en/thingworx/manufacturing-apps/Dashboard/Get-Started (PTC users should use their normal login credentials and do not need to register on the portal) Download Freemium and Developer Kit (8.0.1) are available for download here: https://www.ptc.com/en/thingworx/manufacturing-apps/Dashboard (PTC users should use their normal login credentials and do not need to register on the portal ThingWorx Platform Extensions (8.1.0, released 1 Nov 2017) are available for download here: https://support.ptc.com/appserver/auth/it/esd/product.jsp?prodFamily=TWA

Check out this new KCS article which links to all known best practice documents available for ThingWorx. This article will get larger in time as more articles are published related to the Dos and Don'ts of building an IoT application! Do you know when to use timers, and where to implement their subscriptions? How about ensuring info tables are used at the proper time, and data tables at others? Pesky performance issues wherein ThingWorx runs slow for apparently no reason? All of these questions and more are addressed here!

Starting with ThingWorx Manufacturing Apps 8.1, a Connection Info box was added to Controls Advisor that will display the Configuration settings needed in KEPServerEX. In Controls Advisor, select the server instance you are connecting with, then select the Key icon Be sure the information displayed here matches the settings you have in place in your KEPServerEX Project Properties in the ThingWorx property group.    

Please refer to the release notes to find information on the new features/changes: PTC Here are some common questions and answers in regards to the Installers feature: Does that mean Thingworx 8 only support docker installation? Or standalone installation is still allowed? Only if using the new installer.  The war file download will still be available for non Docker installs. The WAR files will still be available and usable  the same way as in the past.  Users only need to use Docker if they use the installer. How do customers download/build the docker image? The image is not provided separetly, it is installed and configured by the installer. Does the installer install docker when necessary as well? Or is it expected that the user already has docker installed? No, the user must install it on their operating system before using the installer.  The installer will detect if Docker is properly installed.

In this blog I will be testing with the WindchillSwaggerConnector, but most of the steps also apply to the generic SwaggerConnector.     Overview   The WindchillSwaggerConnector enables the connection to the Windchill REST endpoints through the Swagger (OpenAPI) specification. It is a specialized implementation of the SwaggerConnector. See Integration Connectors for documentation.   It relies on three components : Integration Runtime : microservice that runs outside of ThingWorx and has to be deployed separately, it uses Web Socket to communicate with the ThingWorx platform (similar to EMS). Integration Subsystem : available by default in 7.4 (not extension needed) Integration Connectors (WindchillSwaggerConnector) : available by default in 7.4 (not extension needed)   Currently, in 7.4, the WindchillSwaggerConnector  does not support SSO with Windchill (it is more targeted for a "gateway type" integration). Note that the PTC Navigate PDM apps are using the WindchillConnector and not the WindchillSwaggerConnector.   Integration Runtime microservice setup   The ThingWorx Integration Runtime is a microservice that runs outside of ThingWorx. It can run on the ThingWorx server or a remote machine. It is available for download from the ThingWorx Marketplace (Windows or Linux). The installation media contains 2 files : 1 JAR and 1 JSON configuration file.   For this demo, I'm installing the Integration Runtime on a remote machine and will not be using SSL.   1. Prerequisite for the Integration Runtime : Oracle Jre 8 (and of course a ThingWorx 7.4 platform server accessible) 2. Create an ApplicationKey in the composer for the Integration Runtime to use for communication to the ThingWorx platform. 3. Configure the Integration Runtime communication - ThingWorx host, port, appKey, ... - this is done on the Integration Runtime server via the JSON configuration file.   My integrationRuntime-settings.json (sslEnable=false, storagePath is ignored) : { "traceRoutes": "true", "storagePath": "/ThingworxStorage", "Thingworx": {     "appKey": "1234abcd-xxxx-yyyy-zzzz-5678efgh",     "host": "twx74neo",     "port": "8080",     "basePath": "/Thingworx",     "sslEnable": "false",     "ignoreSSLErrors": "true"   } } Note : It is important to completely remove the "SSL": {} block when not using SSL   4. Launch the Integration Runtime service (update the JAR and JSON filenames if needed) java -DconfigFile=integrationRuntime-settings.json -jar integration-runtime-7.4.0-b12.jar The Integration Runtime service uses Web Socket to communicate with the ThingWorx platform (similar to EMS). It registers itself with the ThingWorx platform.   Monitoring the Integration Runtime microservice        In the ThingWorx composer : Monitoring > Subsystems > Integration Subsystem      SMAINENTE1D1 is the hostname of my Integration Runtime server.   Custom WindchillSwaggerConnector implementation   Use the New Composer UI (some setting, such as API maps, are not available in the ThingWorx legacy composer) 1. Create a DataShape that is used to map the attributes being retrieved from Windchill WNCObjectDS : oid, type, name (all fields of type STRING)   2. Create a Thing named WNC11Connector that uses WindchillSwaggerConnector as Thing Template 3. Setup the Windchill connection under WNC11Connector > Configuration Authentication Type = fixed (SSO currently not supported) Username = <Windchill valid user> Password = <password for the Windchill user> Base URL : <Windchill app URL> (e.g. http://wncserver/Windchill)   4. Create an API maps under WNC11Connector > Services and API Maps > API Maps (New Composer only) My API Map : New API Map Mapping ID : FindBasicObjectsMap EndPoint : findObjects (choose the first one) Select DataShape : WNCObjectDS (created at step 1) and map the following attributes : name <- objName ($.items.attributes) type <- typeId ($.items) oid <- id ($.items)   After pressing [done] verify that the API ID is '/objects GET' (and not /structure/objects - otherwise recreate the mapping and choose the other findObjects endpoint). 5. Create a "Route" service under WNC11Connector > Services and API Maps > Services (New Composer only) Name : FindBasicObjects Type (Next to [Done] button) : Route Route Info | Endpoint : findObjects (same as step 4) Route Info | Mapping ID : FindBasicObjectsMap-xxxx created at step 4 Testing our custom WindchillSwaggerConnector   Test the WNC11Connector::FindBasicObjects service Note that the id (oid) and typeId (type) are returned by default by the /objects REST API - objName has to be explicitly requested.   Monitoring the Integration Connector        In the ThingWorx composer : Monitoring > Integration Connectors

Connection server + InfluxDB + Grafana work together can display connection server metrics in graphic charts Download and set up connection server Download InfluxDB and configure InfluxDB edit <influxdb home>\influxdb.conf file Note: [admin] is InfluxDB admin login page, [http] is Grafana connected settings, [graphite] is connection server connected settings Run influxd.exe file with below commands (windows cmd) <path>/influxd -config <path>/influxdb.conf Run influx.exe file with below commands (in my case the port is 8008) <path>/influx -port 8008 Start connection server Login to InfluxDB by using URL http://localhost:8018. Choose Database graphite and Query SHOW MEASUREMENTS Note: There should be many connection server metrics Double click grafana-server.exe file (<path>/grafana-4.2.0.windows-x64/grafana-4.2.0/bin) Login to Grafana with URL http://localhost:3000 The default login username and password is admin/admin Create a new Data Source Note: Type, Url and Database properties are required. Save & Test. If the connection is set up successfully, there should be a green message pop-up. Create a new dashboard Add a new Row, and set up its metrics by create new querys Sample result:

About This is the third part of a ThingBerry related blog post series. ThingBerry is ThingWorx installed on a RaspBerry Pi, which can be used for portable demonstrations without the need of utilizing e.g. customer networks. Instead the ThingBerry provides its own custom WIFI hotspot and allows Things to connect and send / receive demo data on a small scale. In this particual blog post we'll discuss on how to setup the ThingBerry for encrypted communication and trust via certificates. We will create a custom Root and Intermediate Certificate Authority as well as a server specific certificate. More information on the theory of Trust & Encryption can be found here: Trust &amp; Encryption - Theory As the ThingBerry is a highly unsupported environment for ThingWorx, please see this blog post for all related warnings. Preparing the environment To get started and keep all the working materials safe and secure within the scope of the pi user account, let's start creating a workspace directory and then install the openssl package cd ~ mkdir certs chmod 700 certs cd certs sudo apt-get install openssl Troubleshooting Tips Keytool To look at a specific certificate use the keytool and specify the .crt file to look at keytool -printcert -file rootca.crt This will show information on the certificate's configuration, such as signing authority or validity. Timestamps It's important, that the Chain of Trust follows a timely fashion. The Root CA has the longest expiration date. Within its validity the Intermediate CA must be valid. The server specific certificate must be valid in the timeframe of the Intermediate CA validity. As the validity is calculated based on seconds of the current (signing) action, I'm using valid days of Root CA 999 days Intermediate CA 888 days Server specific certificate 777 days Any other times, that make sense can be used, as long as the time constraints are followed. Otherwise the certificate will not be recognized as valid and will therefore not be trusted. Creating a Chain of Trust For this example we'll create a Root and Intermediate Certificate Authority (CA) as well as the server specific certificate. In a first step we can create all of the required keys. openssl genrsa -des3 -out rootca.key 4096 openssl genrsa -out intermediateca.key 4096 openssl genrsa -out server.key 4096 The -des3 option for the rootca will store a password and encrypt it within the key for more secure access. This tightens up security, so that the key for the rootca is not easily corrupted or exposed. Root Certificate Authority Having the keys, we now create the Root CA itself openssl req -new -x509 -sha256 -days 999 -key rootca.key -reqexts v3_req -extensions v3_ca -out rootca.crt Enter the information, starting with the password for the Root CA key. Use a easy to identify name for the Common Name (CN), e.g. "My Root CA" Intermediate Certificate Authority As the Intermediate CA needs to be signed / approved by the Root CA, we need a Certficiate Signing Request (CSR) openssl req -new -key intermediateca.key -reqexts v3_req -extensions v3_ca -out intermediateca.csr The v3_ca extension will mark this certificate as CA itself. This means that we can use the intermediate CA to sign other certificates as well. The CSR will be signed with the Root CA's key and certificate. For this the CSR must be sent to the whoever own the Root CA. In our case, we're the owner ourselves and have everything in the same directory, so no need to send it off to Let's Encrypt, Google or VeriSign. For signing the Intermediate CA as an actual CA, the Root CA must have a specific configuration to allow the v3_ca extension. This needs to be created via nano v3_ca.ext Into this new file, copy & paste the following: basicConstraints        = CA:TRUE subjectKeyIdentifier    = hash authorityKeyIdentifier  = keyid:always,issuer:always keyUsage                = cRLSign, dataEncipherment, digitalSignature, keyCertSign, keyEncipherment, nonRepudiation Save and exit. Finally, the Root CA is signing the Intermediate CA, using the v3_ca extension via: openssl x509 -req -in intermediateca.csr -CA rootca.crt -CAkey rootca.key -CAcreateserial -CAserial intermediateca.srl -extfile v3_ca.ext -days 888 -sha256 -out intermediateca.crt Certificate Authority Chain Having now two CAs, they need to be chained together. This is required for later to create the keystore used in Tomcat. Creating the Chain is probably the easiest part of this blog: cat intermediateca.crt rootca.crt > cachain.crt Server Specific Certificate The server specific certificate has to be signed and obtained by the Intermediate CA; for this a CSR is required. openssl req -new -key server.key -out server.csr The request is then signed with the Intermediate CA's key and certificate. For commerical certificates this signing process will be done by publicly trusted CAs, like Let's Encrypt, VeriSign or Google. In our case, we're the Intermediate CA ourselves and can sign the CSR. openssl x509 -req -in server.csr -CA intermediateca.crt -CAkey intermediateca.key -CAcreateserial -CAserial server.srl -days 777 -sha256 -out server.crt Creating Keystores For the keystore as used in Tomcat we need the CA Chain as well as the server specific certificate and the corresponding key. To generate a keystore that can be used in Tomcat, we first need to create a PKCS12 type keystore with openssl openssl pkcs12 -export -certfile cachain.crt -in server.crt -inkey server.key -out server.p12 This PKCS12 keystore can now be transformed into a JKS keystore, by importing the PKCS12 information into a new JKS keystore keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 -destkeystore keystore.jks -deststoretype JKS Important: the passwords for the PKCS12 keystore and the JKS keystore must match! Tomcat requires those passwords to be same - in case they are different, the configuration will not work and ThingWorx cannot be accessed through a secure connection. Configuring ThingWorx For ThingWorx, only the keystore is required. It can be copied over e.g. to certificates area in the storage directory. sudo cp keystore.jks /thingworx/storage/certificates sudo chmod 640 /thingworx/storage/certificates/keystore.jks sudo chown tomcat:tomcat /thingworx/storage/certificates/keystore.jks To enable it, Tomcat's server.xml needs to be updated. sudo nano $CATALINA_HOME/conf/server.xml Find the connector for port 80 and add a new connector after the port 80 connector: <Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"            maxThreads="150" connectionTimeout="20000"  redirectPort="8443"            SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLSv1.2" enableLookups="false"            keystoreFile="/thingworx/storage/certificates/keystore.jks" keystorePass="<keystorePass>" /> Don't forget to update the password. Stop Tomcat sudo service tomcat stop Ensure Tomcat is actually down with ps -ef | grep tomcat If it's still running, just sudo kill it and start Tomcat again to activate the new configuration sudo service tomcat start And now? Congratulations! ThingWorx is now running on your ThingBerry using a secure channel. Access it via https://<thingberry>/Thingworx As HTTPS is automatically using port 443, you will automatically connect to the port configured in the server.xml Classic HTTP connections on port 80 can still be used. For more information about setting up ThingWorx with (self-signed) certificates also see https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS193947 - there's also a note on forwarding all HTTP traffic to HTTPS. See also https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS246292 for more information about using specific SSL protocols or cipher suites.

Do you trust me? if (true) { why } else      { !why } Communication between client and server can be encrypted or not. Communication between client and server can be between trusted parties or not. Both statements are independant of each other. There's no need to trust a server - but still traffic can be encrypted. There's no need to encrypt traffic - but still a server can be trusted. For secure and safe communication in an IoT environment encryption as well as trust should be an integral part of exchanging data. ​ Trust Certificates and Chains of Trust Trust establishes the identity of the parties involved, e.g. is server I'm contacting really the server it pretends to be? This can be ensured with server certificates. On the other hand, client certificates ensure that the client is actual the client it pretends to be. This can be used as an authentication approach and if the identity of the server as well as the client is confirmed, this is also known as mutual authentication. Trust is usually invoked through certificates. Certificates have a subject, usually the servername that the certificate belongs to. This can also be a wildcard subject like *.myserver.com which will be valid for all subdomains of myserver.com. A certificate can be digitally signed to ensure its identity. Usually a Certificate Authority (CA) signs the certificate. This CA can in turn be signed by another CA and so an, creating a whole chain of CAs. This chain is commonly known as the Chain of Trust or the certification path. If a certificate is not signed by a CA it's called a "self-signed certificate". These are mostly used for test scenarios, but should never be used in real production environments. Certificates can also expire by using dates like "Valid From" and "Valid To" to determine if a certificate is valid or not. In case a certificate or CA is expired, all following certificates in the Chain of Trust will no longer be considered as valid. For "commercial" certificates, certificates that are signed by an official CA like Google, Let's Encrypt or VeriSign, a Certificate Signing Request (CSR) is used. For this a key-pair is generated and the CSR is signed with the private key. The CSR contains the public key and is sent to and finally signed by the CA itself, returning the public certificate e.g. signed by VeriSign. This mechanism allows to generate certificates by preparing the main information and just having the CA sign it as a trusted authority. Key-pair All certificates consist of a key-pair - a private key and a public key. The private key is indeed very private to the server and MUST NOT be shared with anyone else. The public key is indeed very public and can be shared with anyone. The key-pair consists of mainly two very large prime numbers and some mathematical magic allows for a mathematical one-way function ​with a trap door. This function allows to en-crypt any message with the public key and to de-crypt this message with the private key. So by giving out the public key to anyone, anyone can en-crypt a message to the server but only the server alone can de-crypt it with its private key. Therefore it's important that the private key is not shared with anyone, otherwise messages can be de-crypted by anyone possessing the private key and secure communication can be compromised. This method is called public-key encryption, or asymmetric encryption (as the key for en- and de-crypting are different). For more information, check out https://en.wikipedia.org/wiki/Public-key_cryptography This key-pair is not related to any encryption while sending data between client and server. Encryption on a transport level is accomplished by using different, random keys. It's just used to generate a unique certificate with a unique fingerprint as a public certificate. The private key part is amongst other things used for signing other certificates in the Chain of Trust. Tomcat needs access to the private key of the server specific certificate, to actually verify that the certificate is authentic. However, certificates can also be used for e.g. emails where the private key is in fact used to en- or decrypt the mail on a content level! ThingWorx ThingWorx is mainly configured for server side trust. In this case all connecting clients can ensure that the identity of the ThingWorx server is a trusted one - and not a fake server or an untrusted source. Certificates and trust are usually triggered when a secure connection is established, via TLS / SSL - e.g. by calling https://<myserver.com>:443/Thingworx The default port for a HTTPS (HTTP Secure / over TLS) connection is 443. For "normal" HTTP configuration, certificates are not required. Keystore To establish trust, any client needs to know the public certificate of the ThingWorx server. This is usually stored in a keystore. In ThingWorx, Tomcat needs to be configured with the location and the password of the keystore containing the certificate. For each web-request Tomcat will then present the server certificate to the client. The client can then verify through its own keystore if the certificate is trusted or not. When operating in browsers, this is done via e.g. the Windows-Keystore. For devices a custom, e.g. a Java-Keystore is required. The keystore must contain the whole Chain of Trust as well as the server specific certificate and its key. If the Chain of Trust is not stored completely in the keystore, or the key is missing / not accessible, the certificate validation process will already fail on server side. Certificates in Windows The default Windows-Keystore is actually pre-filled with lots of Root-Certificates, which start the Chain of Trust - such as Google, VeriSign or Microsoft. Trust needs to start somewhere… If the browser can trace the CAs of certificate back to a trusted Root CA, it's indicated by a green lock and the server will be trusted. If the Root CA cannot be found, it's indicated by a red lock (not a purse) - meaning that the server is not trusted. The server can still be contacted anyway, by manually acknowleding the risks (like identity theft or credit card scams). As an example, here's the certificate for Wikipedia and its Chain of Trust, starting with the GlobalSign Root CA: Trustworthy? In the end, certificates only clarify the identity of a server. If it is trusted or not, is completely up to the user / device. This trustful relationship all comes down to the client keystore / truststore. While the presenting server holds its certificate in the keystore, the verifying client holds the trusted certificates in the truststore. Any certificate in the truststore will ensure the correct server identity. Any unknown certificate will trigger a "Are you sure?" question in the browser. And as devices usually cannot respond to such a question, communication with an untrusted server can be forbidden by default. There are also products, like PTC Navigate which will require a client certificate to not only ensure server identity, but also ensure that also specific clients are able to connect to the server. Encryption No matter if a server is trusted or not, as soon as HTTPS is used as a protocol, communication will be encrypted. The certificate plays a role, but no matter if it's in the client's truststore, TSL forces encryption. Client Hello Usually the client contacts the server and sends a Client Hello. With this it also sends a list of available cipher suites, the intended receiver (server name) and a list of available Signature Hash Algorithms to be used. Cipher suites are used to determine which algorithms are used to establish a connection. The Signature Hash Algorithms define the Hash function, like SHA256 and the key / signature, like RSA. In my environment usually the Elliptic Curve Diffie-Hellman key exchange is used to transfer public keys between server and client. During the Client Hello information about the elliptic curves are also transferred. For more information on these topics, see https://en.wikipedia.org/wiki/SHA-2 https://en.wikipedia.org/wiki/RSA_(cryptosystem) https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange https://www.youtube.com/watch?v=YEBfamv-_do (Diffie-Hellman explained) https://en.wikipedia.org/wiki/Elliptic_curve_cryptography Server Hello After receiving the Client Hello, the server sends back the Server Hello. This consists of multiple handshakes. In this phase, the server sends the preferred cipher suite (intersection of client and server suites) including the Signature Hash Algorithm to be used. In the same step the server is sending off the certificate to the client for identity validation. For establishing trust and validating the certificate, the server needs to know the private key for the server specific certificate. The server also announces in the server key exchange what public key it's using for encryption to the client. This key is not related to the certificate, but a random number from a newly generated key-pair. Keys are exchanged between server and client and vice versa via the Diffie-Hellman key exchange. Client Response The client now sends in the client key exchange what public key it's using for encryption to the server. This is also a random key transmitted via the Diffie-Hellman key exchange. Initiating Communication After announcing the public keys, both server and client will now communicate encrypted by using their custom key-pairs for asymmetric encryption of the payloads. This can also include an additional change of the cipher specification. As the public keys have been exchanged via the Diffie-Hellman key exchange, server and client are now bi-directionally communicating using each other's public key to en-crypt a message and each's own private key to de-crypt the message again. For more information, also see https://tools.ietf.org/html/rfc5246 Trust & Encryption - Hands On Do you trust me now? if (true) { why } else      { !why } To see how this all works in a ThingWorx enviroment, and how easy it actually set up what has been discussed above, see also Trust &amp; Encryption - Hands On

Unless created and owned by the Administrator user, by default MySQL Database Thing will not connect to the database as it requires certain permissions on the user. In order for a user other than an administrator to create a working database thing, they need three permissions (in addition to the typical subsystem and resource permissions - refer to https://www.youtube.com/watch?v=HzFqxvgHtpI&index=8&list=PLz1ppcU_kaneagUT9qgQfz3HByf6-9zTF ​ ):: Visibility to the Database Thing Template. Execute service permission on the EncryptPropertyValue service in the Encryption Services resource. Visibility to the DatabaseThing Thing Package. Typically to track down permissions issue, the most convenient and easy way is to use browser developer tools. For example in Chrome, developer tools can be used to view the API calls being sent by Composer, and the errors sent in response.   ThingWorx Composer doesn’t expose Thing Packages, so in order to set visibility to the DatabaseThing Thing Package, one would need to throw a REST API call at it. Hope this information helps in setting up a non-administrator own MSSQL database thing! *In addition refer to The use of System User

Wanting to build a simple environment with ThingWorx, using a Raspbery Pi mini-computer and an Edge MicroServer (EMS), one has to bear in mind some apparently insignificant tips, that can make a huge difference in a well-connected platform.    The Raspbery Pi can have either a few sensors attached on it and/or a Sense HAT integrated with it. A Sense HAT is an add-on board for Raspbery Pi that has an 8x8 LED matrix, a five-button joystick and a sensor, gyroscope. Tips: The ThingWorx guide "Connecting ThingWorx EMS with Rasberry Pi Quickstart" can be expanded to read Sense HAT specific sensor information A Sense HAT integrated with Raspberry Pi can be accessed from anywhere, using the existing APIs to read/write Getting location information while using the Pi board (because The Sense HAT doesn't have this feature): The Raspberry Pi has several USB ports, therefore inserting a USB GPS can be used as a workaround for connecting and reading location information (or the GPIO pins could be used to connect a GPS board) To include services and properties to read Sense HAT sensors data, start from the existing LUA example.lua template and built a Pi specific one. The EMS is a pre-built application that can be installed on Windows or Linux and it provides a bridge between the remote device and the ThingWorx platform, using the REST API AlwaysOn protocol. Tips: To read data from a sensor, it is important to use the corresponding library for that sensor; if LUA scripting is used, then this library has to be easily integrated with LUA (LSR is optional with EMS; EMS can be used standalone). Using the correct library, the EMS will be able to gather the data and push it to ThingWorx. The LSR sends properties to the EMS over the HTTP(s) protocol, which converts it to the AlwaysOn Protocol, and sends it to the Platform LUA is a process manager useful if having to integrate devices from different vendors using different modes to interact To test if the LSR is working: go to Composer -> Properties tab of the remote thing, click "Manage Bindings", select the "Remote" tab, next to "Local" - here can be seen all the remote properties added to the thing by the LSR. For creating and biding more things: create the remote thing in ThingWorx and then add them to the config.lua file in the same way you have the gateway Listing any "Remote Things" that are ready to be connected, and are not associated with any Thing, yet: Monitoring; one method to try to bind them: type the name of the thing in the Identifier section of General Information of the Thing To request a local REST API call to the default EMS port 8000, from a LSR, make sure that the GET works fine (i.e for a property update:  localhost:8000/Thingworx/Things/MyThing/Properties/myString It should return you something like this: {"rows":[{"myString":""}],"fieldDefinitions":{"myString":{"name":"myString","description":"","type":"STRING","aspects":{}}}}

In this blog I will be covering the initial setup of ThingWorx Android SDK with a sample app (supplied with the Android SDK) and set it up with a web based revision control system like Bitbucket's (free account plan). I'll also be covering quick information on how to enable the HTTPS connection for the ThingWorx server on Windows platform. This will allow for secured connection to the ThingWorx server from the Android application. Do note this is only a reference guide for setting it up with revision control system, you’re free to choose to setup the Android Studio without Bitbucket or with any other revision control system. It’ll be just fine to have a local Git/SVN/Perforce etc. to manage the code repository, setting Android Studio with Bitbucket is officially not supported. Pre-requisite: 1. Download and unzip the ThingWorx Android SDK from https://support.ptc.com 2. An account with Bitbucket is required 3. Download and install Android Studio Project Setup 1. Unzip the downloaded Android SDK on a local drive 2. Start the Android Studio > Import project > navigate to the sample applications location provided with the ThingWorx Android SDK e.g. Thingworx-Android-SDK-X.x.x\samples 3. Select one of the sample application e.g. androidShell, with ThingWorx Android SDK X.x.x there are 3 sample applications currently available when the Android SDK is downloaded: a. Android File Brwoser b. Android Shell c. Android Steam Thing 4. For this blog I'll be setting up the androidShell Android Application 5. Do note that all the sample projects are built using Gradle, so while importing if required select Gradle as the build system for the sample application 6. Once imported successfully in Android Studio you should be able to see the Android project and its file structure like so 7. We'll need an account with Bitbucket, create one if you don't have already 8. Logon to Bitbucket and create a team and new repository under that team 9. Navigate to the repository created in Bitbucket 10. Create a local GIT code repo if you don't already have one, and copy over all the contents from the Android SDK x.x.x.zip to that location 11. On your local machine open a command prompt and navigate to the drive where the local GIT's code repository resides, i.e. the folder where you unzipped the android SDK and execute the commands as mentioned: a. git remote add origin https://<accountName@bitbucket.org>/<teamName>/<projectName>.git b. git push -u origin master Note: This will add the contents of your local GIT repository to the empty code repository you’ve created under the team on Bitbucket. 12. You can also use SourceTree UI application on windows for creating, GIT and Mercury based code repository and connect it to your Bitbucket account 13. On successful commit following will be logged in the command prompt Enabling HTTPS on Tomcat and connecting to Android Application Securing Tomcat on Windows You can skip this section if you already have Tomcat running ThingWorx configured for HTTPS connection. 1. Execute the command in a command prompt C:\>"%JAVA_HOME%\bin\keytool" -genkey -alias tomcat -keyalg RSA -keystore C:\KeystoreTomcat\tomcat.keystore Note: Executing above step will require you to add additional information to the keystore like, Org name, full name location, etc. 2. Edit the <tomcatInstallation>\conf\server.xml file   <Connector   port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"               maxThreads="200" SSLEnabled="true" scheme="https" secure="true" keystoreFile="C:\KeystoreTomcat\tomcat.keystore" keystorePass="<giveYourPasswordHere>" clientAuth="false" sslProtocol="TLS">       <!-- <SSLHostConfig>             <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"                         type="RSA" />         </SSLHostConfig>-->     </Connector> 3. Restart the tomcat service Note: For detailed information on securing the Tomcat on Windows refer to SSL/TLS Configuration How-TO . Note HTTPS setup is only for the Tomcat where ThingWorx is deployed and does not involve certificate setup on the Android Application side Finally,to test if the HTTPS setup was successful or not navigate to https://<serverName/IP>:8443/Thingworx in a web browser. Port 8443 is the default HTTPS port. Starting and connecting Android Application to ThingWorx Now that we have a working secured Tomcat and Android Studio setup with the sample Android Application, androidShell. Let's build and run the application using an emulated Android device in Android Studio 1. Navigate to https://localhost:8443/Thingworx > Import/Export > Import from file and import the Thing entity which will connect to the SampleThing when you’ll run the Android application, for e.g. I imported the Thingworx-Android-SDK-X.X.X\samples\android-shell\entity\Things_AndroidSampleThing.xml because I will be running the androidShell application 2. Attempting to run the Android application without the entity created in ThingWorx first, Thing will show as unbound in RemoteThing. 3. Create an AppKey in ThingWorx > Security > Application Keys with sufficient rights 4. Go to the Android Studio's toolbar and click on AVD Manager icon 5. This will open the Android Virtual Device Manager and lets you create a virtual Android device 6. You can of course use your own actual device to connect over USB and install and test the application on actual hardware 7. If you already have a Virtual or actual device connected to the system, click on Make Project icon in the toolbar 8. Once the Make finishes run the sample application, in my case androidShell application with the Run icon, like so 9. You may now be prompted with the options to select a device virtual or actual 10. Select as desired and click OK 11. This will now launch the application on the selected device, I have selected to launch on the virtual device which will start and emulated Android Device 12. When initiating/running the application for the first time you will be directed to the Settings screen allowing you to enter the connection URI and the Application Key to connect to the ThingWorx server 13. You have to follow one of the following two URI schemes while attempting to connect to a ThingWorx Server a. For HTTP connection use : ws://<machineIP/Name>:8080/Thingworx/WS b. For HTTPS connection use : wss://<machineIP/Name>:8443/Thingworx/WS Note: Ports may differ as these are the default ports, if you are running ThingWorx on different port enhance the URI accordingly 14. Since my ThingWorx is reachable on HTTPS connection i'll use the HTTPS connectino URI scheme and the application key that I have already created in the ThingWorx, which is bound to Administrator user 15. Once done press the back button on the screen to initiate the connection attempt 16. If all's set as it should be you will be able to see the Connected to Server option checked and the Property count for the Serial Number Property being updated every second For more detail on ThingWorx Android SDK refer to the ThingWorx Edge SDKs and WebScocket based Edge MicroServer (WS EMS) Help Center

Hi,   I have just been playing around with the Edge MicroServer on a couple of Raspberry Pi's.  I obviously want them connected to ThingWorx, however they stop when the SSH session closes which isn't ideal.  I thought about doing something really quick and dirty using 'nohup', but this could have lead to many running processes, and wouldn't still not have started automatically when the Pi booted.  So, I did it right instead using init.d daemon configurations.   There are two files; one for the EMS and one for the Lua Script Resource.  You need to put these in /etc/init.d, then make sure they are owned by root.   sudo chown root /etc/init.d/thingworx* sudo chgrp root /etc/init.d/thingworx*   You'll need to modify the paths in the first lines of these files to match where you have your microserver folder.  Then you need to update the init.d daemon configs and enable the services.   sudo update-rc.d thingworx-ems defaults sudo update-rc.d thingworx-ems enable sudo update-rc.d thingworx-lsr defaults sudo update-rc.d thingworx-lsr enable   You can then start (stop, etc.) like this:   sudo service thingworx-ems start sudo service thingworx-lsr start   They should both also start automatically after a reboot.   Regards,   Greg

If the ThingWorx 7.4 installation with MSSQL db doesn't start with a license error on the splash screen, despite taking the necessary steps for specifying the license path, the error might be misleading and the problem is actually lying in the database connection. Going to THingworxStorage/logs and opening ApplicationLog.log might reveal the following (or similar ) errors: 2017-04-13 10:26:17.993-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Sending Post Start Notifications... 2017-04-13 10:26:17.999-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.001-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.004-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.005-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.007-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.008-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.008-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Thingworx Server Application...ON A few things to check here. First, how the database was set up. The standard expected port if 1433, however, if you choose a different port - ensure that the port is available. If it isn't - update the port setting in SQL configuration manager. Another possible root cause -- the database scripts did not run properly upon the setup. Currently, there is an active Jira PSPT-3587 for resolving the script issue (NOTE, this thread to be updated once the Jira is fixed). There are two options here, either to run the sql files manually (found in the "install" folder of the downloadable), or edit the bat scripts manually. It's recommended to edit the script files instead because running the sql commands allows more room for a mistake. The following line in the bat scripts needs to be edited to have the ".\" removed: sqlcmd.exe -S %server%\%serverinstance%,%port% -U %adminusername% -v loginname=%loginname% -v database=%database% -v thingworxusername=%thingworxusername% -v schema=%schema%  -i .\thingworx-database-setup.sql Note, that the scripts need to be run from the same directory ("/install")

This video will walk you through the first steps of how to set-up Analytics Manager for Real-Time Scoring. More specifically this video demonstrates how to create an Analysis Provider and start ThingPredictor Agent. NOTE: For version 8.1 the startup command for the Agent has changed view the command in PTC Help center.   Updated Link for access to this video:  ThingWorx Analytics Manager: Create an Analysis Provider & Start the ThingPredictor Agent                                  

About This is the second part of a ThingBerry related blog post series. ThingBerry is ThingWorx installed on a RaspBerry Pi, which can be used for portable demonstrations without the need of utilizing e.g. customer networks. Instead the ThingBerry provides its own custom WIFI hotspot and allows Things to connect and send / receive demo data on a small scale. In this particual blog post we'll discuss on how to setup the ThingBerry as a WIFI hotspot to directly connect with mobile devices or other Raspberry Pis. As the ThingBerry is a highly unsupported environment for ThingWorx, please see this blog post for all related warnings. In case this guide looks familiar, it's probably because it's based on https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/ WIFI Hot Spot As the ThingBerry is currently connected via ethernet we can utilize the Raspberry Pi's WIFI connection to create a private network where all the wireless devices can connect to, e.g. another Raspberry Pi or a ESP8266 First we need to install dnsmasq and hostapd. Those will help setting up the access point and create a private DNS server to dynamically assign IP-addresses to connecting devices. sudo apt-get install dnsmasq hostapd Interfaces We will need to configure the wlan0 interface with a static IP. For this the dhcpcd needs to ignore the wlan0 interface. sudo nano /etc/dhcpcd.conf Paste the following content to the end of the file. This must be ABOVE any interface lines you may have added earlier! denyinterfaces wlan0 Save and exit. Let's now configure the static IP. sudo nano /etc/network/interfaces Comment out ALL lines for the wlan* configurations (e.g. wlan0, wlan1). By default there are three lines which need to be commented out by adding a # at the beginning of the line. After this the wlan0 can be pasted in: allow-hotplug wlan0 iface wlan0 inet static   address 192.168.0.1 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 Save and exit. Now restart the dhcpcd service and reload the wlan0 configuration with sudo service dhcpcd restart sudo ifdown wlan0 sudo ifup wlan0 Hostapd Hostapd is used to configure the actual WIFI hot spot, e.g. the SSID and the WIFI password (wpa_passphrase) that's required to connect to this network. sudo nano /etc/hostapd/hostapd.conf Paste the following content: interface=wlan0 driver=nl80211 ssid=thingberry hw_mode=g channel=6 wmm_enabled=1 ieee80211n=1 country_code=DE macaddr_acl=0 ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40] auth_algs=1 ignore_broadcast_ssid=0 wpa=2 wpa_key_mgmt=WPA-PSK wpa_passphrase=changeme rsn_pairwise=CCMP If you prefer another SSID or a more secure password, please ensure updating above configuration! Save and exit. Check if the configuration is working via sudo /usr/sbin/hostapd /etc/hostapd/hostapd.conf It should return correctly, without any errors and finally show "wlan0: AP-ENABLED". With this you can now connect to the "thingberry" SSID. However there's no IP assigned automatically - so that sucks​ can be improved... Stop hostapd with CTRL+C and let's start it on boot. sudo nano /etc/default/hostapd At the end of the file, paste the following content: DAEMON_CONF="/etc/hostapd/hostapd.conf" Save and exit. DNSMASQ Dnsmasq allows to assign dynamic IP addresses. Let's backup the original configuration file and create a new one. sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig  sudo nano /etc/dnsmasq.conf Paste the following content: interface=wlan0 listen-address=192.168.0.1 bind-interfaces dhcp-range=192.168.0.100,192.168.0.199,255.255.255.0,12h Save and exit. This will make the DNS service listen on 192.168.0.1 and assign IP addresses between 192.168.0.100 and 192.168.0.199 with a 12 hour lease. Next step is to setup the IPV4 forwarding for the wlan0 interface. sudo nano /etc/sysctl.conf Uncomment the following line: You can search in nano with CTRL+W net.ipv4.ip_forward=1 Save and exit. Hostname translation To be able to call the ThingBerry with its actual hostname, the hostname needs to be mapped in the host configuration. sudo nano /etc/hosts Search the line with your hostname and update it to the local IP address (as configured in the listen-address above), e.g. 192.168.0.1 thingberry Save and exit. Please note that this is the hostname and not the SSID of the network! Finalizing the Configuration Run the following command to enable the fowarding and reboot. sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" sudo reboot Optional: Internet Access The ThingBerry is independent of any internet traffic. However if your connected devices or the ThingBerry itself need to contact the internet, the WIFI connection needs to route those packages to and from the (plugged-in) ethernet device. This can done through iptables sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE  sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT  sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT All traffic will be internet related traffic will be forwarded between eth0 and wlan0 and vice versa. To load this configuration every time the ThingBerry is booted, it needs to be saved via sudo sh -c "iptables-save > /etc/iptables.ipv4.nat" Run this file by editing sudo nano /etc/rc.local Above the line exit 0 add the following line: iptables-restore < /etc/iptables.ipv4.nat Save and exit. Verification Connect to the new WIFI hotspot via the SSID and the password configured earlier through any WIFI capable device. When connecting to your new access point check your device's IP settings (in iOS, Android or a laptop / desktop device). It should show a 192.168.0.x IP address and should be pingable from the ThingBerry console. Connect to http://192.168.0.1/Thingworx or http://<servername>/Thingworx to open the Composer.

ThingWorx 7.4 covers the following areas of the product portfolio:  ThingWorx Analytics, ThingWorx Utilities and ThingWorx Foundation which includes Core, Connection Server and Edge capabilities. Key Functional Highlights Highlights of the release include: Source Integration: Improved integration framework making it easy to connect with external systems, including a standard Windchill connector, to retrieve data on demand. Industrial Connectivity: New Industrial Gateway and Discover navigation tool simplifying the mapping of tags to properties, including performance enhancements for data updates. Edge/CSDK: Build process improvements, Subscribed Property Manager (SPM) enhancements, asynchronous service requests and TLS updates to increase developer productivity, improve application performance and strengthen security. AWS IoT Connector: The latest version 1.2 of the Connector allows customers to more fully leverage their investment in AWS IoT. It features improved deployment automation via CloudFormation and automatic extension installation, ThingWorx Edge JavaScript SDK for use with the Connector with support for properties, services and events, and just-in-time certificate registrations. Contextualize Next Generation Composer: Re-imagined Composer using modern browser concepts to improve developer efficiency including enhanced functionality, updated user interface and optimized workflows. Engage These features will be available at the end of March 2017. Advanced Grid: New grid widget with improved design, context menu, multi-column sorting, global search and many more common grid-related features. Tree Grid: New tree-grid widget with same features as advanced grid plus ability to pre-load tree levels, dynamically load child data and auto expand all nodes to build more powerful mashups. Administer & Manage MS SQL Server: New persistence provider for model and run-time data providing customers an alternative to PostgreSQL. Security: ThingWorx worked with industry standard security scanning and auditing tools to identify and correct all non-trivial, and many trivial, vulnerabilities to ensure secure software best practices. Licensing: Link ThingWorx to PTC systems of record to manage user entitlement and provide usage information and auditing capability critical to TWX, PTC and its partners.  Documentation ThingWorx 7.4 Reference Documents ThingWorx Core 7.4 Release Notes ThingWorx Core Help Center ThingWorx Edge SDKs and WebSocket-based Edge MicroServer Help Center ThingWorx Connection Services Help Center ThingWorx Utilities Help Center Additional information ThingWorx eSupport Portal ThingWorx Developer Portal ThingWorx Marketplace Download ThingWorx Platform – Select Release 7.4 ThingWorx Edge C SDK 1.4 – Select Most Recent Datecode, C-SDK-1-4-0 ThingWorx AWS IoT Connector 1.2 – Select Release 7.4 ThingWorx Utilities – Select Release 7.4

Does WSEMS use SSL pinning? Yes, it does. To explain the process a bit more, the ssl pinning is the act of verifying a server certificate by comparing it to the exact certificate that is expected. We “install” a certificate on the EMS (copying the cert to the ems device, and specifying the cert in config.json), EMS will then check all incoming certs against the cert in config.json, looking for an exact match and verifying the certificate chain. Should it be expected that the client, WSEMS.exe, installs the client certificate in order to validate the server certificate using SSL pinning? This does not happen automatically, the cert must be downloaded and manually added to the device. The config must also be manually updated. If so, how we issue the client cert for WSEMS.exe? Does it need to be issued the same way as the server certificate? If talking about pinning, one could use either the server cert directly, or the root cert (recommended). The root cert is the public cert of the signing authority, e.g. if the server uses a verisign issued cert, one can verify the authenticity of the signer by having the verisign public cert. If talking about client certs (2 way auth, client verifies server, server verifies client) then the process is a little different. What happens when the client certificate expires? Do all the devices go offline when the client certificate expires? The device won’t connect with a failure to authorize. Once the server cert expires, the server cert needs to be updated everywhere. This is the advantage to using something like the verisign public cert (root cert) as the installed cert on the client. The root certs usually last longer than the issued certs, but they will have to replaced eventually when they expire. When using Entrust certificates, obtained at https://www.entrust.com/get-support/ssl-certificate-support/root-certificate-downloads/ , how do we know that the certifactes are fed to the WSEMS configuration incorrectly? With the wrong configuration, WSEMS would error out rejecting the Entrust certificate: Non-fips Error code 20: Invalid certificate Fips: Error code 19:  self signed certificate in certificate chain How to properly install Entrust certificate to be accessed by EMS? Public certificate needs to be downloaded and put in a directory that is accessible to the EMS. The path to be added to the cert chain field of the configuration. Even if its trusted it will always need to be installed on the EMS. If a certificate is self-signed, meaning it's non-trusted by default, OpenSSL would throw errors and compain. To solve this, it needs to be installed as a trusted server. If it's signed by a non-trusted CA, that CA's certificate needs to be installed as well. Once the certificates are obtained this if the block of the configuration we are interested in: "certificates":    {                   "validate":          true, //Not validated model is not recommended                       "allow_self_signed":  false, //Self signed model is not recommended, yet theoretically better than non-validated                       "cert_chain":      " " } //recommended, trusted - also the only way to work with a FIPS enabled EMS May use the SSL test tool (for example,https://www.ssllabs.com )to find all chains. For Entrust, both Entrust L1k and G2 need to be downloaded: Because cert_chain is an array type of field, the format to install the certificates paths, would be: "cert_chain": ["path1"], ["path2"] "cert_chain": ["C:\\ThingworxLocation\\MicroServerLocation\\locationofcertificates\\entrust_l1k.pem","C:\\ThingworxLocation\\MicroServerLocation\\locationofcertificates\\entrust_g2_ca.pem"]

In case we would like to create an external application and we aren't sure what's the best solution to use, below are some useful tips. Scenario: Let's say we use a gateway in order to access the external application we want to create. We would like to implement this gateway translating the ThingWorx standard protocol to the SCADA protocol. The system administrator, who manages the grid, has the own secure system, with a standard for communication inside the SCADA system, and we want to be able to get data from our system to the system they have. Let's also consider that the data is connected on the electrical field. Tips: It is recommended to develop a 3rd party that on one side talks to ThingWorx, and on the other side, talks to the SCADA system. This external ThingWorx application that we want to implement would have a series-edge-interface allowing to enter in our customer's Ethernet network, in order for both systems to communicate. JDBC is not recommended - it's mostly for connecting to the data base, which in our case is not the main purpose. Each REST API call to the platform uses a security accreditation (appkey or user/password); depending of the permissions contained in that token, the access can be allowed to certain parts of the platform. Reasons for using REST API: REST API is simple and not dependent on any format that the data comes from ThingWorx. It can be used offline, online, synchronously, asynchronously, and is easy to manage from a formatting point of view. ThingWorx can give a lot of options: like exporting information via xml to a plain xml file, to parse it to whatever protocol we have on the other hand: Either our application would have to handle xml inputs from ThingWorx and process it towards SCADA compatible output. Or our application will have to handle xml input from ThingWorx and process it towards SCADA compatible output. Or we can talk directly via REST API and read on a per-thing basis (using the web services). The interface application just has to know how to read xmls or REST API calls (which are provided with an xml formatted response). SDK has already library written in C, C# and Java. SDK C, C# and Java use the AlwaysOn protocol (web socket)  and are more firewall friendly. It's mostly like for speed and automated processing, so when known exactly what happens and we trust the other side and we know there are little chances for errors.​ If we go with REST API or SDK , the application that is developed will have complete access inside ThingWorx, like change/edit the things. If we want to have access in both ways, not only in reading data, but also in update/delete information, etc, SDK and REST API ​can be used, because we have the whole range of commands, like set property values, call on services, etc. We can limit access, if we want, for security reasons. SDK offers access to the same services as REST API, but in a different way. Otherwise, it's better to go with xml decoupled files. Conclusion: for this particular scenario, better use SDK.

Introduction In-Memory Column stores the data in columnar format contrary to row format. Allowing users to run faster analytics which is also the idea behind this is to push the computation as close to the data store as possible. In this post I'll configure the Oracle database to enable this feature and then populate one or more tables in the In Memory Column store. This could be particularly helpful if you are using Oracle 12c as an external data store for storing data in database table via JDBC connection, current/historic values from DataTable, Streams or ValueStreams for running analytics or DMLs with lots of join and require lot of computation before the data is finally presented on to the Mashup(s). For this post I used the data generated by temperature sensor getting stored in ValueStream, exported to CSV from the ValueStream and imported it in the Oracle table. In-Memory Column Store vs In-Memory database Usage As mentioned above Oracle 12c version 12.1.2 comes with in built In-Memory Column Store feature. As the name suggest it allows data to be populated in RAM enabling high speed transaction and analytics on data without the need to traverse the HDD, and in some cases this is much faster than the buffer cache of the SGA. Without going into too much nitty-gritty it's important to note that In-Memory Column  Store does not equate to In-Memory database. While it could be possible to move the entire schema, if it's small enough, to fit in the defined memory for In-Memory Column Store, the idea however is to speed up the computation requiring analytics on one or more table(s) which are heavily queried by the users. If you are interested in In-Memory Database as persistence provider for ThingWorx please refer to the documentation Using SAP HANA as the Persistence Provider which is one of the option among other available Persistence Providers for ThingWorx. What changes are required to the current Oracle 12c installation or JDBC connection to ThingWorx? In-Memory Column Store feature is an inbuilt feature in Oracle 12.1.2 and only needs to be enabled, as it's not by default. This can be enabled without having to bring any sort of change to the following : 1. The existing SQL services created within ThingWorx 2. General application architecture accessing the tables in the Oracle database over JDBC 3. Existing Oracle 12c installation Getting Started What will it take to enable In-Memory Column Store? This feature can be enabled by following few steps : 1. Enable this feature in the Oracle 12.1.2 installation, by assigning some memory in RAM for InMemory Column 2. Adjust the SGA size for the database to incorporate the memory assigned to the In-Memory Column 3. Bounce the database As mentioned above though this is an inbuilt feature with Oracle 12.1.2, but is not enabled by default and we can confirm this by executing following SQL in SQL*Plus or Oracle SQL Developer connecting to database for which we are enabling this feature. SQL> show parameter INMEMORY; Things to consider before enabling 1. Ensure that the hardware/ VM hosting the Oracle installation have sufficient RAM, 2. Ensure to bump up the SGA by the amount of memory assigned to In-Memory Column store, failing to do so may lead to database failing to start and will require recovery Note: Minimum memory that can be assigned to In-Memory Column Store is 100M Setting it all up For my test setup I will be assigning 5G to the In-Memory Column Store and will add this amount to the current SGA, to do this let's start the SQL*Plus with the rights that will allow me to make changes to the existinng SGA, so i'm using sys@orcl as sysdba  (ORCL is the test DB name i have for my database) Step 1: Start SQL*Plus, e.g. sqlplus sys@orcl as sysdba Step 2: ALTER SYSTEM SET INMEMORY_SIZE = 5G SCOPE=SPFILE; Step 3: ALTER SYSTEM SET SGA_TARGET = 20G SCOPE=SPFILE; Once done, bounce the database. And that's it! We should now be able to confirm that, via SQL*Plus, certain amount of memory, 5G in my case, has been assigned to the In-Memory Column Store feature SQL> show parameter inmemory Populating the In-Memory Column Store In-Memory Column Store will only populate the data from the table only on the first use or if the table is marked critical which will tell Oracle to populate as soon as the database comes online after restart. For more detail on the commands concerning the In-Memory Column Store refer to the OTN webpage I'll now use the SensorHistory table in which i have the ValueStream's exported data in CSV format, currently this table is holding ~32million+ rows, and populate them in columnar architecture of the In Memory Column Store with following command: SQL>ALTER TABLE SENSORHISTORY INMEMORY; // marking the table to be eligible for In-Memory column with default parameters Just to confirm that the data is still not populated since we have only marked the table to be eligible for In-Memory Column Store, if I now query the dynamic view V$IM_SEGMENTS for current usage of the InMemory, it'll confirm this: So now let's populate the In-Memory with a query which would require full table scan, e.g. SQL> select property_name, count(*) from sensorhistory           Group by property_name; Let's recheck the dynamic view V$IM_SEGMENTS As mentioned above, that this is completely transparent to the application layer, so if you already have an existing JDBC connection in ThingWorx to Oracle, all the existing services created for that table will continue to work as expected. If you don't have an existing JDBC connection to Oracle, it can be created with usual steps with no special configuration for In-Memory. Creating JDBC connection I'm including this section for the purpose of completeness, if you already have a working JDBC connection to Oracle 12.1.2 you can skip to Conclusion below.Now for accessing the above database along with the In-Memory Column Store table we'll now setup the JDBC connection to the Oracle, for that download and import the TW_Download_Relational Databases Connectors.zip (ThingWorx Marketplace) > unzip to access the Oracle12Connector_Extension.zip Step 1 : Import the extension in the ThingWorx by navigating to Import/Export > Import > Extensions Step 2: Create a Thing using the OracleDBServer12 Template, part of the extension we just imported Step 3: Here's how a valid configuration would look like to successfully connect to the database, ORCL in this case Step 4: Navigate to the Properties in the Entity Information panel on the left and verify that the isConnected property value is True. Conclusion This is a very short introduction to what could be a setup for improving the data analytics performed on the stored data, manifold. The data in the In-Memory Column Store is not stored in conventional row format, rather in large columnar format. If the need is to have simple SQL queries with not so many joins it could be that the SGA Cache would be sufficient and probably be faster and you may not gain much by configuring the In-Memory Column Store. However, queries requiring heavy computation on large data sets, having In-Memory Column Store configured could bring manifold increase in performance. Therefore if you need more guidelines on where you'd want to use the In-Memory Column Store, feel free to give following listed good reads a try along with real world data use case for reference. I will try to find some time to run my own benchmark and will try to put it out in a separate blog on performance gain. 1. Oracle Database In-Memory with Oracle Database 12c Release 2 : Oracle white paper 2. When to Use Oracle Database In-Memory :  Identifying Use Cases for Application Acceleration 3. Oracle Database 12c In-Memory Option 4. Testing Oracle In-Memory Column Store @ CERN

One of the issues we have encountered recently is the fact that we could not establish a VNC Remote session. The edge was located outside of the internal network where the Tomcat was hosted, and all access to the instance was through an Apache reverse proxy. The EMS was able to connect successfully to the Server, because the Apache had correctly setup the Websocket forwarding through the following directive: ProxyPass "/Thingworx/WS/"  "wss://192.168.0.2/Thingworx/WS" However, we saw that tunnels immediately closed after creation and as a result (or so we thought), we could not connect from the HTML5 VNC viewer. More diagnostics revealed that you need to have ProxyPass directives for the following: -the EMS will make calls to another WS endpoint, called WSTunnelServer. After you setup this, the EMS will be able to create tunnels to the server. -the HTML5 VNC page will make a "websocket" call to yet another WS endpoint, called WSTunnelClient. Only at this step you have the ability to successfully use tunnels through a reverse proxy. Hope it helps!