IoT & Connectivity Tips

Connection server + InfluxDB + Grafana work together can display connection server metrics in graphic charts Download and set up connection server Download InfluxDB and configure InfluxDB edit <influxdb home>\influxdb.conf file Note: [admin] is InfluxDB admin login page, [http] is Grafana connected settings, [graphite] is connection server connected settings Run influxd.exe file with below commands (windows cmd) <path>/influxd -config <path>/influxdb.conf Run influx.exe file with below commands (in my case the port is 8008) <path>/influx -port 8008 Start connection server Login to InfluxDB by using URL http://localhost:8018. Choose Database graphite and Query SHOW MEASUREMENTS Note: There should be many connection server metrics Double click grafana-server.exe file (<path>/grafana-4.2.0.windows-x64/grafana-4.2.0/bin) Login to Grafana with URL http://localhost:3000 The default login username and password is admin/admin Create a new Data Source Note: Type, Url and Database properties are required. Save & Test. If the connection is set up successfully, there should be a green message pop-up. Create a new dashboard Add a new Row, and set up its metrics by create new querys Sample result:

In the process of working with a customer, I was curious as to the throughput of a file sent via the Axeda Connected Content feature to one of the Axeda Agent Gateways.   I took a random 50 megabyte blob of data (/dev/urandom) and sent it to one of my test Gateways via a Package deployment: DEBUG   xgEnterpriseProxy: Enterprise Queue Empty INFO    xgSM:  ... Download percent done = 11% INFO    xgSM:  ... Download percent done = 21% INFO    xgSM:  ... Download percent done = 31% INFO    xgSM:  ... Download percent done = 41% INFO    xgSM:  ... Download percent done = 51% INFO    xgSM:  ... Download percent done = 61% INFO    xgSM:  ... Download percent done = 71% INFO    xgSM:  ... Download percent done = 81% INFO    xgSM:  ... Download percent done = 91% INFO    xgSM:  ... Download percent done = 100% DEBUG   xgSM: >>  INTERNAL DEBUG MESSAGE << :  Download time is 4 seconds DEBUG   xgSM: >>  INTERNAL DEBUG MESSAGE << :  Upgrading.  Backing up files to C:\temp\CFKGW\AxedaBackup DEBUG   xgSM: >>  INTERNAL DEBUG MESSAGE << :  Extracting downloaded files from DefaultProject\CFKGW\Downloads\141581_143281.tar.gz to directory C:\temp\CFKGW\ DEBUG   xgSM: >>  INTERNAL DEBUG MESSAGE << :  Extraction Finished About 12MB per second.  This was a sandbox in the PTC On-Demand Center.  Not bad, but not necessarily representative of a real production system.  This sandbox doesn't have 1000 devices trying to get this file at once.  So some benchmarking in your configuration and environment certainly needs to be done. So that done, I thought I'd up the ante - 700 megabytes this time! DEBUG   xgEnterpriseProxy: Enterprise Queue Empty INFO    xgSM:  ... Download percent done = 10% INFO    xgSM:  ... Download percent done = 20% INFO    xgSM:  ... Download percent done = 30% INFO    xgSM:  ... Download percent done = 40% INFO    xgSM:  ... Download percent done = 50% INFO    xgSM:  ... Download percent done = 60% INFO    xgSM:  ... Download percent done = 70% INFO    xgSM:  ... Download percent done = 80% INFO    xgSM:  ... Download percent done = 90% INFO    xgSM:  ... Download percent done = 100% DEBUG   xgSM: >>  INTERNAL DEBUG MESSAGE << :  Download time is 66 seconds So 10MB per second. Directory of C:\temp\cfkgw 05/17/2017  01:32 PM    <DIR>          . 05/17/2017  01:32 PM    <DIR>          .. 05/17/2017  01:03 PM         1,048,576 1mb.dat 05/17/2017  01:04 PM        52,428,800 50meg-randomdata.dat 05/17/2017  01:32 PM       734,003,200 700mb.dat 05/17/2017  01:03 PM    <DIR>          AxedaBackup                3 File(s)    787,480,576 bytes Not bad at all! 

Twilio extends the ThingWorx functionality to send SMS and voice messages in variety of languages. Starting from scratch I'll cover the steps required to setup the Twilio extension and the basic account registration (trial version) and setup at Twilio.com Twilio allows registering with the trial account, which comes with $15 as initial account balance, something which is quite useful for testing the initial setup and testing of Twilio extension in conjunction to ThingWorx. Prerequisite 1. Sign up, if not already done, with Twilio.com for free trail account 2. Configure the account to setup a verified phone number to receive text and voice messages, free trial account allows setting up 1 verified phone number that can receive text and voice messages Note: See What's the difference between a verified phone number and a Twilio phone number? 3. Choose and configure a Twilio phone number capable of sending text and voice messages, free trial account allows setting up of 1 Twilio number which will be used in the Extension configuration in ThingWorx for sending the messages 4. Download Twilio Extension from ThingWorx Marketplace Note that Twilio phone number should be enabled with the send SMS or send voice message capability, depending on what your use case is. Failing to do so will lead to error while testing the sendSMS or sendVOICE service (two default services provided with the Twilio Extension when imported in ThingWorx) Signing up and setting up Twilio account 1. Sign up on Twilio.com with your details and emailID. When registering for the first time you'll be prompted for your personal phone number as part of the security requirement. Also with free trial account this will be the only phone number where you'll be able to send SMS or Voice message via Twilio Extension in ThingWorx 2. Next would be to setup the Twilio number with voice and text messaging, navigate to https://www.twilio.com/console/phone-numbers/getting-started 3. Do ensure that you setup the voice or text capabilities for the new Twilio number, failing to do so will lead to error in sending message via Twilio Extension in ThingWorx 4. Different Twilio Number have different capabilities, it's quite possible that you don't find the number with required capabilities i.e. Voice, SMS - in that scenario simply search for different number, one that has the capabilities you are looking for 5. With trial account only 1 Twilio number is avilable for setup 6. While setting up the Twilio number you will be required to provide a valid local address of your country The cost of setting up the number and further sending the SMS will only be deducted from the $15 made available when you signed up for the first time. Navigate to your account details on Twilio.com to make note of the following information which will be used when configuring the Twilio Extension in ThingWorx: a. Account SID b. Authentication ID c. Your Verified Phone Number (will be used for receiving the messages) d. Twilio Phone number created with required capabilities, e.g. that number is capable of sending text message Here's how it shows up once you are successfully registered and logged on to the twilio.com/console To check your Twilio Phone Number and the Verified Phone Number navigate to twilio.com/phone-numbers As it can be seen in the above screenshot, the number I have bought from Twilio is capable of sending Voice and SMS. With this all's set  and ready to configure the Twilio Extension in ThingWorx, which is pretty straight forward. Setup Twilio Extension in ThingWorx 1. Let's setup the extension by navigating to the ThingWorx Composer > Import/Export > Extensions > Import > browse to the location where you saved the Twilio Extension from ThingWorx Marketplace and click Import 2. Once imported successfully refresh the composer and navigate to Modeling > Things , to create a new Thing implementing the Twilio Template (imported with the extension) like so, in screenshot below Thing is named DemoTwilioThing 3. Navigate to the the Configuration section of that Thing, to provide the information we noted above from Twilio account Note: this demo thing is setup to send the text SMS to my verified phone number only, therefore the CallerID in the configuration above is the Twilio number I created after signing up on Twilio with text message capability. 4. Save the created the Thing 5. Finally, we can test with the 1 of the 2 default services provided with the Twilio Template i.e. SendSMSMessage service 6. While testing the SendSMSMessage service use your verified phone number in To for receiving the message Troubleshooting some setup related issues I'll try to cover some of the generic errors that came up while doing this whole setup with Twilio Extension Error making outgoing SMS: 404 / TwilioResponse 20404 Reason If the Twilio number is not enabled with SMS sending capabilities you may run into such an error when testing the service. Here's the full error stack Unable to Invoke Service SendSMSMessage on DemoTwilioThing : Error making outgoing SMS: 404 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>20404</Code><Message>The requested resource /2010-04-01/Accounts/<AccountSID>/SMS/Messages was not found</Message><MoreInfo>https://www.twilio.com/docs/errors/20404</MoreInfo><Status>404</Status></RestException></TwilioResponse> Resolution Navigate back to the Phone Number section on Twilio's website and as highlight in screenshot above check if the Twilio number is enabled with SMS capabilities. Error making outgoing SMS:400 / TwilioResponse 21608 Reason You may encounter following error when attempting to send SMS/Voice message, here's the full error detail Wrapped java.lang.Exception: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21608</Code><Message>The number <somePhoneNumber>is unverified. Trial accounts cannot send messages to unverified numbers; verify <somePhoneNumber>at twilio.com/user/account/phone-numbers/verified, or purchase a Twilio number to send messages to unverified numbers.</Message><MoreInfo>https://www.twilio.com/docs/errors/21608</MoreInfo><Status>400</Status></RestException></TwilioResponse> Cause: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21608</Code><Message>The number <somePhoneNUmber>is unverified. Trial accounts cannot send messages to unverified numbers; verify <somePhoneNumber> at twilio.com/user/account/phone-numbers/verified, or purchase a Twilio number to send messages to unverified numbers.</Message><MoreInfo>https://www.twilio.com/docs/errors/21608</MoreInfo><Status>400</Status></RestException></TwilioResponse> Resolution As the error points out clearly the number used in To section while testing the SendSMSMessage service didn't have verified number. With free trial account you can only use the registered verified phone number where SMS/Voice message can be sent. If you want to use different number an account upgrade is required. Error making outgoing SMS:400 / TwilioResponse 21606 Reason Following error is thrown while testing SendSMSMessage service with different Twilio number which is either not the same as the number you bought when setting up the trial account or it doesn have SMS sending capabiltiy. Here's the full error stack Wrapped java.lang.Exception: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21606</Code><Message>The From phone number <TwilioNumber>is not a valid, SMS-capable inbound phone number or short code for your account.</Message><MoreInfo>https://www.twilio.com/docs/errors/21606</MoreInfo><Status>400</Status></RestException></TwilioResponse> Cause: Error making outgoing SMS: 400 <?xml version='1.0' encoding='UTF-8'?><TwilioResponse><RestException><Code>21606</Code><Message>The From phone number <TwilioNumber> is not a valid, SMS-capable inbound phone number or short code for your account.</Message><MoreInfo>https://www.twilio.com/docs/errors/21606</MoreInfo><Status>400</Status></RestException></TwilioResponse> Resolution Check the configuration in the ThingWorx Composer for the Thing created out of Twilio Template whether or not the callerID is configured with correct Twilio account number

This is just a quick note/reminder that starting 7.4, the utilities installation process has slightly changed. Several extensions are no longer bundled with ThingWorx Utilities and must be downloaded from the ThingWorx Marketplace and installed separately. For more information, see the “Prerequisites” topic in the ThingWorx Utilities Installation Guide: http://support.ptc.com/WCMS/files/172616/en/ThingWorxUtilitiesInstall.pdf The following extensions must be installed prior to installing ThingWorx Utilities available at ThingWorx IoT Marketplace ○ Google Maps Widget ○ Mail Extension ○ Web Sockets Tunnel Widget and Library

About This is the third part of a ThingBerry related blog post series. ThingBerry is ThingWorx installed on a RaspBerry Pi, which can be used for portable demonstrations without the need of utilizing e.g. customer networks. Instead the ThingBerry provides its own custom WIFI hotspot and allows Things to connect and send / receive demo data on a small scale. In this particual blog post we'll discuss on how to setup the ThingBerry for encrypted communication and trust via certificates. We will create a custom Root and Intermediate Certificate Authority as well as a server specific certificate. More information on the theory of Trust & Encryption can be found here: Trust &amp; Encryption - Theory As the ThingBerry is a highly unsupported environment for ThingWorx, please see this blog post for all related warnings. Preparing the environment To get started and keep all the working materials safe and secure within the scope of the pi user account, let's start creating a workspace directory and then install the openssl package cd ~ mkdir certs chmod 700 certs cd certs sudo apt-get install openssl Troubleshooting Tips Keytool To look at a specific certificate use the keytool and specify the .crt file to look at keytool -printcert -file rootca.crt This will show information on the certificate's configuration, such as signing authority or validity. Timestamps It's important, that the Chain of Trust follows a timely fashion. The Root CA has the longest expiration date. Within its validity the Intermediate CA must be valid. The server specific certificate must be valid in the timeframe of the Intermediate CA validity. As the validity is calculated based on seconds of the current (signing) action, I'm using valid days of Root CA 999 days Intermediate CA 888 days Server specific certificate 777 days Any other times, that make sense can be used, as long as the time constraints are followed. Otherwise the certificate will not be recognized as valid and will therefore not be trusted. Creating a Chain of Trust For this example we'll create a Root and Intermediate Certificate Authority (CA) as well as the server specific certificate. In a first step we can create all of the required keys. openssl genrsa -des3 -out rootca.key 4096 openssl genrsa -out intermediateca.key 4096 openssl genrsa -out server.key 4096 The -des3 option for the rootca will store a password and encrypt it within the key for more secure access. This tightens up security, so that the key for the rootca is not easily corrupted or exposed. Root Certificate Authority Having the keys, we now create the Root CA itself openssl req -new -x509 -sha256 -days 999 -key rootca.key -reqexts v3_req -extensions v3_ca -out rootca.crt Enter the information, starting with the password for the Root CA key. Use a easy to identify name for the Common Name (CN), e.g. "My Root CA" Intermediate Certificate Authority As the Intermediate CA needs to be signed / approved by the Root CA, we need a Certficiate Signing Request (CSR) openssl req -new -key intermediateca.key -reqexts v3_req -extensions v3_ca -out intermediateca.csr The v3_ca extension will mark this certificate as CA itself. This means that we can use the intermediate CA to sign other certificates as well. The CSR will be signed with the Root CA's key and certificate. For this the CSR must be sent to the whoever own the Root CA. In our case, we're the owner ourselves and have everything in the same directory, so no need to send it off to Let's Encrypt, Google or VeriSign. For signing the Intermediate CA as an actual CA, the Root CA must have a specific configuration to allow the v3_ca extension. This needs to be created via nano v3_ca.ext Into this new file, copy & paste the following: basicConstraints        = CA:TRUE subjectKeyIdentifier    = hash authorityKeyIdentifier  = keyid:always,issuer:always keyUsage                = cRLSign, dataEncipherment, digitalSignature, keyCertSign, keyEncipherment, nonRepudiation Save and exit. Finally, the Root CA is signing the Intermediate CA, using the v3_ca extension via: openssl x509 -req -in intermediateca.csr -CA rootca.crt -CAkey rootca.key -CAcreateserial -CAserial intermediateca.srl -extfile v3_ca.ext -days 888 -sha256 -out intermediateca.crt Certificate Authority Chain Having now two CAs, they need to be chained together. This is required for later to create the keystore used in Tomcat. Creating the Chain is probably the easiest part of this blog: cat intermediateca.crt rootca.crt > cachain.crt Server Specific Certificate The server specific certificate has to be signed and obtained by the Intermediate CA; for this a CSR is required. openssl req -new -key server.key -out server.csr The request is then signed with the Intermediate CA's key and certificate. For commerical certificates this signing process will be done by publicly trusted CAs, like Let's Encrypt, VeriSign or Google. In our case, we're the Intermediate CA ourselves and can sign the CSR. openssl x509 -req -in server.csr -CA intermediateca.crt -CAkey intermediateca.key -CAcreateserial -CAserial server.srl -days 777 -sha256 -out server.crt Creating Keystores For the keystore as used in Tomcat we need the CA Chain as well as the server specific certificate and the corresponding key. To generate a keystore that can be used in Tomcat, we first need to create a PKCS12 type keystore with openssl openssl pkcs12 -export -certfile cachain.crt -in server.crt -inkey server.key -out server.p12 This PKCS12 keystore can now be transformed into a JKS keystore, by importing the PKCS12 information into a new JKS keystore keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 -destkeystore keystore.jks -deststoretype JKS Important: the passwords for the PKCS12 keystore and the JKS keystore must match! Tomcat requires those passwords to be same - in case they are different, the configuration will not work and ThingWorx cannot be accessed through a secure connection. Configuring ThingWorx For ThingWorx, only the keystore is required. It can be copied over e.g. to certificates area in the storage directory. sudo cp keystore.jks /thingworx/storage/certificates sudo chmod 640 /thingworx/storage/certificates/keystore.jks sudo chown tomcat:tomcat /thingworx/storage/certificates/keystore.jks To enable it, Tomcat's server.xml needs to be updated. sudo nano $CATALINA_HOME/conf/server.xml Find the connector for port 80 and add a new connector after the port 80 connector: <Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"            maxThreads="150" connectionTimeout="20000"  redirectPort="8443"            SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLSv1.2" enableLookups="false"            keystoreFile="/thingworx/storage/certificates/keystore.jks" keystorePass="<keystorePass>" /> Don't forget to update the password. Stop Tomcat sudo service tomcat stop Ensure Tomcat is actually down with ps -ef | grep tomcat If it's still running, just sudo kill it and start Tomcat again to activate the new configuration sudo service tomcat start And now? Congratulations! ThingWorx is now running on your ThingBerry using a secure channel. Access it via https://<thingberry>/Thingworx As HTTPS is automatically using port 443, you will automatically connect to the port configured in the server.xml Classic HTTP connections on port 80 can still be used. For more information about setting up ThingWorx with (self-signed) certificates also see https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS193947 - there's also a note on forwarding all HTTP traffic to HTTPS. See also https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS246292 for more information about using specific SSL protocols or cipher suites.

Do you trust me? if (true) { why } else      { !why } Communication between client and server can be encrypted or not. Communication between client and server can be between trusted parties or not. Both statements are independant of each other. There's no need to trust a server - but still traffic can be encrypted. There's no need to encrypt traffic - but still a server can be trusted. For secure and safe communication in an IoT environment encryption as well as trust should be an integral part of exchanging data. ​ Trust Certificates and Chains of Trust Trust establishes the identity of the parties involved, e.g. is server I'm contacting really the server it pretends to be? This can be ensured with server certificates. On the other hand, client certificates ensure that the client is actual the client it pretends to be. This can be used as an authentication approach and if the identity of the server as well as the client is confirmed, this is also known as mutual authentication. Trust is usually invoked through certificates. Certificates have a subject, usually the servername that the certificate belongs to. This can also be a wildcard subject like *.myserver.com which will be valid for all subdomains of myserver.com. A certificate can be digitally signed to ensure its identity. Usually a Certificate Authority (CA) signs the certificate. This CA can in turn be signed by another CA and so an, creating a whole chain of CAs. This chain is commonly known as the Chain of Trust or the certification path. If a certificate is not signed by a CA it's called a "self-signed certificate". These are mostly used for test scenarios, but should never be used in real production environments. Certificates can also expire by using dates like "Valid From" and "Valid To" to determine if a certificate is valid or not. In case a certificate or CA is expired, all following certificates in the Chain of Trust will no longer be considered as valid. For "commercial" certificates, certificates that are signed by an official CA like Google, Let's Encrypt or VeriSign, a Certificate Signing Request (CSR) is used. For this a key-pair is generated and the CSR is signed with the private key. The CSR contains the public key and is sent to and finally signed by the CA itself, returning the public certificate e.g. signed by VeriSign. This mechanism allows to generate certificates by preparing the main information and just having the CA sign it as a trusted authority. Key-pair All certificates consist of a key-pair - a private key and a public key. The private key is indeed very private to the server and MUST NOT be shared with anyone else. The public key is indeed very public and can be shared with anyone. The key-pair consists of mainly two very large prime numbers and some mathematical magic allows for a mathematical one-way function ​with a trap door. This function allows to en-crypt any message with the public key and to de-crypt this message with the private key. So by giving out the public key to anyone, anyone can en-crypt a message to the server but only the server alone can de-crypt it with its private key. Therefore it's important that the private key is not shared with anyone, otherwise messages can be de-crypted by anyone possessing the private key and secure communication can be compromised. This method is called public-key encryption, or asymmetric encryption (as the key for en- and de-crypting are different). For more information, check out https://en.wikipedia.org/wiki/Public-key_cryptography This key-pair is not related to any encryption while sending data between client and server. Encryption on a transport level is accomplished by using different, random keys. It's just used to generate a unique certificate with a unique fingerprint as a public certificate. The private key part is amongst other things used for signing other certificates in the Chain of Trust. Tomcat needs access to the private key of the server specific certificate, to actually verify that the certificate is authentic. However, certificates can also be used for e.g. emails where the private key is in fact used to en- or decrypt the mail on a content level! ThingWorx ThingWorx is mainly configured for server side trust. In this case all connecting clients can ensure that the identity of the ThingWorx server is a trusted one - and not a fake server or an untrusted source. Certificates and trust are usually triggered when a secure connection is established, via TLS / SSL - e.g. by calling https://<myserver.com>:443/Thingworx The default port for a HTTPS (HTTP Secure / over TLS) connection is 443. For "normal" HTTP configuration, certificates are not required. Keystore To establish trust, any client needs to know the public certificate of the ThingWorx server. This is usually stored in a keystore. In ThingWorx, Tomcat needs to be configured with the location and the password of the keystore containing the certificate. For each web-request Tomcat will then present the server certificate to the client. The client can then verify through its own keystore if the certificate is trusted or not. When operating in browsers, this is done via e.g. the Windows-Keystore. For devices a custom, e.g. a Java-Keystore is required. The keystore must contain the whole Chain of Trust as well as the server specific certificate and its key. If the Chain of Trust is not stored completely in the keystore, or the key is missing / not accessible, the certificate validation process will already fail on server side. Certificates in Windows The default Windows-Keystore is actually pre-filled with lots of Root-Certificates, which start the Chain of Trust - such as Google, VeriSign or Microsoft. Trust needs to start somewhere… If the browser can trace the CAs of certificate back to a trusted Root CA, it's indicated by a green lock and the server will be trusted. If the Root CA cannot be found, it's indicated by a red lock (not a purse) - meaning that the server is not trusted. The server can still be contacted anyway, by manually acknowleding the risks (like identity theft or credit card scams). As an example, here's the certificate for Wikipedia and its Chain of Trust, starting with the GlobalSign Root CA: Trustworthy? In the end, certificates only clarify the identity of a server. If it is trusted or not, is completely up to the user / device. This trustful relationship all comes down to the client keystore / truststore. While the presenting server holds its certificate in the keystore, the verifying client holds the trusted certificates in the truststore. Any certificate in the truststore will ensure the correct server identity. Any unknown certificate will trigger a "Are you sure?" question in the browser. And as devices usually cannot respond to such a question, communication with an untrusted server can be forbidden by default. There are also products, like PTC Navigate which will require a client certificate to not only ensure server identity, but also ensure that also specific clients are able to connect to the server. Encryption No matter if a server is trusted or not, as soon as HTTPS is used as a protocol, communication will be encrypted. The certificate plays a role, but no matter if it's in the client's truststore, TSL forces encryption. Client Hello Usually the client contacts the server and sends a Client Hello. With this it also sends a list of available cipher suites, the intended receiver (server name) and a list of available Signature Hash Algorithms to be used. Cipher suites are used to determine which algorithms are used to establish a connection. The Signature Hash Algorithms define the Hash function, like SHA256 and the key / signature, like RSA. In my environment usually the Elliptic Curve Diffie-Hellman key exchange is used to transfer public keys between server and client. During the Client Hello information about the elliptic curves are also transferred. For more information on these topics, see https://en.wikipedia.org/wiki/SHA-2 https://en.wikipedia.org/wiki/RSA_(cryptosystem) https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange https://www.youtube.com/watch?v=YEBfamv-_do (Diffie-Hellman explained) https://en.wikipedia.org/wiki/Elliptic_curve_cryptography Server Hello After receiving the Client Hello, the server sends back the Server Hello. This consists of multiple handshakes. In this phase, the server sends the preferred cipher suite (intersection of client and server suites) including the Signature Hash Algorithm to be used. In the same step the server is sending off the certificate to the client for identity validation. For establishing trust and validating the certificate, the server needs to know the private key for the server specific certificate. The server also announces in the server key exchange what public key it's using for encryption to the client. This key is not related to the certificate, but a random number from a newly generated key-pair. Keys are exchanged between server and client and vice versa via the Diffie-Hellman key exchange. Client Response The client now sends in the client key exchange what public key it's using for encryption to the server. This is also a random key transmitted via the Diffie-Hellman key exchange. Initiating Communication After announcing the public keys, both server and client will now communicate encrypted by using their custom key-pairs for asymmetric encryption of the payloads. This can also include an additional change of the cipher specification. As the public keys have been exchanged via the Diffie-Hellman key exchange, server and client are now bi-directionally communicating using each other's public key to en-crypt a message and each's own private key to de-crypt the message again. For more information, also see https://tools.ietf.org/html/rfc5246 Trust & Encryption - Hands On Do you trust me now? if (true) { why } else      { !why } To see how this all works in a ThingWorx enviroment, and how easy it actually set up what has been discussed above, see also Trust &amp; Encryption - Hands On

Unless created and owned by the Administrator user, by default MySQL Database Thing will not connect to the database as it requires certain permissions on the user. In order for a user other than an administrator to create a working database thing, they need three permissions (in addition to the typical subsystem and resource permissions - refer to https://www.youtube.com/watch?v=HzFqxvgHtpI&index=8&list=PLz1ppcU_kaneagUT9qgQfz3HByf6-9zTF ​ ):: Visibility to the Database Thing Template. Execute service permission on the EncryptPropertyValue service in the Encryption Services resource. Visibility to the DatabaseThing Thing Package. Typically to track down permissions issue, the most convenient and easy way is to use browser developer tools. For example in Chrome, developer tools can be used to view the API calls being sent by Composer, and the errors sent in response.   ThingWorx Composer doesn’t expose Thing Packages, so in order to set visibility to the DatabaseThing Thing Package, one would need to throw a REST API call at it. Hope this information helps in setting up a non-administrator own MSSQL database thing! *In addition refer to The use of System User

Wanting to build a simple environment with ThingWorx, using a Raspbery Pi mini-computer and an Edge MicroServer (EMS), one has to bear in mind some apparently insignificant tips, that can make a huge difference in a well-connected platform.    The Raspbery Pi can have either a few sensors attached on it and/or a Sense HAT integrated with it. A Sense HAT is an add-on board for Raspbery Pi that has an 8x8 LED matrix, a five-button joystick and a sensor, gyroscope. Tips: The ThingWorx guide "Connecting ThingWorx EMS with Rasberry Pi Quickstart" can be expanded to read Sense HAT specific sensor information A Sense HAT integrated with Raspberry Pi can be accessed from anywhere, using the existing APIs to read/write Getting location information while using the Pi board (because The Sense HAT doesn't have this feature): The Raspberry Pi has several USB ports, therefore inserting a USB GPS can be used as a workaround for connecting and reading location information (or the GPIO pins could be used to connect a GPS board) To include services and properties to read Sense HAT sensors data, start from the existing LUA example.lua template and built a Pi specific one. The EMS is a pre-built application that can be installed on Windows or Linux and it provides a bridge between the remote device and the ThingWorx platform, using the REST API AlwaysOn protocol. Tips: To read data from a sensor, it is important to use the corresponding library for that sensor; if LUA scripting is used, then this library has to be easily integrated with LUA (LSR is optional with EMS; EMS can be used standalone). Using the correct library, the EMS will be able to gather the data and push it to ThingWorx. The LSR sends properties to the EMS over the HTTP(s) protocol, which converts it to the AlwaysOn Protocol, and sends it to the Platform LUA is a process manager useful if having to integrate devices from different vendors using different modes to interact To test if the LSR is working: go to Composer -> Properties tab of the remote thing, click "Manage Bindings", select the "Remote" tab, next to "Local" - here can be seen all the remote properties added to the thing by the LSR. For creating and biding more things: create the remote thing in ThingWorx and then add them to the config.lua file in the same way you have the gateway Listing any "Remote Things" that are ready to be connected, and are not associated with any Thing, yet: Monitoring; one method to try to bind them: type the name of the thing in the Identifier section of General Information of the Thing To request a local REST API call to the default EMS port 8000, from a LSR, make sure that the GET works fine (i.e for a property update:  localhost:8000/Thingworx/Things/MyThing/Properties/myString It should return you something like this: {"rows":[{"myString":""}],"fieldDefinitions":{"myString":{"name":"myString","description":"","type":"STRING","aspects":{}}}}

This is part 3 out of 3 videos on Getting Started with ThingWorx Analytics During this video you will learn:   Executing a “Signals” Job Getting the results of the “Signals” Job Executing a “Training Model” Job Getting the results of the “Training Model” Job   Updated Link for access to this video:  Getting Started with ThingWorx Analytics: Part 3 of 3

In this blog I will be covering the initial setup of ThingWorx Android SDK with a sample app (supplied with the Android SDK) and set it up with a web based revision control system like Bitbucket's (free account plan). I'll also be covering quick information on how to enable the HTTPS connection for the ThingWorx server on Windows platform. This will allow for secured connection to the ThingWorx server from the Android application. Do note this is only a reference guide for setting it up with revision control system, you’re free to choose to setup the Android Studio without Bitbucket or with any other revision control system. It’ll be just fine to have a local Git/SVN/Perforce etc. to manage the code repository, setting Android Studio with Bitbucket is officially not supported. Pre-requisite: 1. Download and unzip the ThingWorx Android SDK from https://support.ptc.com 2. An account with Bitbucket is required 3. Download and install Android Studio Project Setup 1. Unzip the downloaded Android SDK on a local drive 2. Start the Android Studio > Import project > navigate to the sample applications location provided with the ThingWorx Android SDK e.g. Thingworx-Android-SDK-X.x.x\samples 3. Select one of the sample application e.g. androidShell, with ThingWorx Android SDK X.x.x there are 3 sample applications currently available when the Android SDK is downloaded: a. Android File Brwoser b. Android Shell c. Android Steam Thing 4. For this blog I'll be setting up the androidShell Android Application 5. Do note that all the sample projects are built using Gradle, so while importing if required select Gradle as the build system for the sample application 6. Once imported successfully in Android Studio you should be able to see the Android project and its file structure like so 7. We'll need an account with Bitbucket, create one if you don't have already 8. Logon to Bitbucket and create a team and new repository under that team 9. Navigate to the repository created in Bitbucket 10. Create a local GIT code repo if you don't already have one, and copy over all the contents from the Android SDK x.x.x.zip to that location 11. On your local machine open a command prompt and navigate to the drive where the local GIT's code repository resides, i.e. the folder where you unzipped the android SDK and execute the commands as mentioned: a. git remote add origin https://<accountName@bitbucket.org>/<teamName>/<projectName>.git b. git push -u origin master Note: This will add the contents of your local GIT repository to the empty code repository you’ve created under the team on Bitbucket. 12. You can also use SourceTree UI application on windows for creating, GIT and Mercury based code repository and connect it to your Bitbucket account 13. On successful commit following will be logged in the command prompt Enabling HTTPS on Tomcat and connecting to Android Application Securing Tomcat on Windows You can skip this section if you already have Tomcat running ThingWorx configured for HTTPS connection. 1. Execute the command in a command prompt C:\>"%JAVA_HOME%\bin\keytool" -genkey -alias tomcat -keyalg RSA -keystore C:\KeystoreTomcat\tomcat.keystore Note: Executing above step will require you to add additional information to the keystore like, Org name, full name location, etc. 2. Edit the <tomcatInstallation>\conf\server.xml file   <Connector   port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"               maxThreads="200" SSLEnabled="true" scheme="https" secure="true" keystoreFile="C:\KeystoreTomcat\tomcat.keystore" keystorePass="<giveYourPasswordHere>" clientAuth="false" sslProtocol="TLS">       <!-- <SSLHostConfig>             <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"                         type="RSA" />         </SSLHostConfig>-->     </Connector> 3. Restart the tomcat service Note: For detailed information on securing the Tomcat on Windows refer to SSL/TLS Configuration How-TO . Note HTTPS setup is only for the Tomcat where ThingWorx is deployed and does not involve certificate setup on the Android Application side Finally,to test if the HTTPS setup was successful or not navigate to https://<serverName/IP>:8443/Thingworx in a web browser. Port 8443 is the default HTTPS port. Starting and connecting Android Application to ThingWorx Now that we have a working secured Tomcat and Android Studio setup with the sample Android Application, androidShell. Let's build and run the application using an emulated Android device in Android Studio 1. Navigate to https://localhost:8443/Thingworx > Import/Export > Import from file and import the Thing entity which will connect to the SampleThing when you’ll run the Android application, for e.g. I imported the Thingworx-Android-SDK-X.X.X\samples\android-shell\entity\Things_AndroidSampleThing.xml because I will be running the androidShell application 2. Attempting to run the Android application without the entity created in ThingWorx first, Thing will show as unbound in RemoteThing. 3. Create an AppKey in ThingWorx > Security > Application Keys with sufficient rights 4. Go to the Android Studio's toolbar and click on AVD Manager icon 5. This will open the Android Virtual Device Manager and lets you create a virtual Android device 6. You can of course use your own actual device to connect over USB and install and test the application on actual hardware 7. If you already have a Virtual or actual device connected to the system, click on Make Project icon in the toolbar 8. Once the Make finishes run the sample application, in my case androidShell application with the Run icon, like so 9. You may now be prompted with the options to select a device virtual or actual 10. Select as desired and click OK 11. This will now launch the application on the selected device, I have selected to launch on the virtual device which will start and emulated Android Device 12. When initiating/running the application for the first time you will be directed to the Settings screen allowing you to enter the connection URI and the Application Key to connect to the ThingWorx server 13. You have to follow one of the following two URI schemes while attempting to connect to a ThingWorx Server a. For HTTP connection use : ws://<machineIP/Name>:8080/Thingworx/WS b. For HTTPS connection use : wss://<machineIP/Name>:8443/Thingworx/WS Note: Ports may differ as these are the default ports, if you are running ThingWorx on different port enhance the URI accordingly 14. Since my ThingWorx is reachable on HTTPS connection i'll use the HTTPS connectino URI scheme and the application key that I have already created in the ThingWorx, which is bound to Administrator user 15. Once done press the back button on the screen to initiate the connection attempt 16. If all's set as it should be you will be able to see the Connected to Server option checked and the Property count for the Serial Number Property being updated every second For more detail on ThingWorx Android SDK refer to the ThingWorx Edge SDKs and WebScocket based Edge MicroServer (WS EMS) Help Center

This blog addresses a few points that are related to scoring with ThingWorx Analytics. It, particularly, brings a clearer understanding of the concepts behind the values of the scores that are generated when performing a scoring job.   Scoring Outputs:   It is important to note that when training an analytics model, the method is to create a generalizable model from a relatively small training dataset.   By its nature, we expect the training process to see a limited subset and not an exhaustive list of all possible values for many constraints, especially for time and practicality.   As such, these generalized models will be expected to handle unseen data in the form of new combinations or values outside of previously observed ranges (more on this below).   One common way to see scores that exceed the observed ranges in training, under the assumption that the goals are continuous, is to use prescriptive scoring.   Prescriptive scoring attempts to find optimal values for a lever, meaning tunable, features in order to maximize or minimize score values. See the prescriptive scoring documentation and functionality for more information.   min/max constraints: these are constraints that are placed upon the inputs for training and expected inputs for scoring.   •          For training: If theses ranges were provided as part of the upload process, then training will raise exceptions regarding invalid data. However, if the ranges are not provided - they will be inferred from the data and, as such, training will not see values outside of observed ranges.   •          For scoring: Validation of the ranges will only be performed on the inputs - not the outputs. It is very important to note that the handling of these "constraints" is dependent upon the data type.   For categorical (e.g. colors) and ordinal data (e.g. shirt sizes), the constraints are strict and data that was not observed in training will raise exceptions during scoring.   However, for continuous values (e.g. temperature ranges) these constraints are more informational in nature. For predictive scoring, our code will accept records with values outside of those ranges.   The rule of thumb is that values slightly outside these ranges are acceptable and that as the values stray farther from the ranges, the accuracy of the model degrades very quickly.   For prescriptive scoring, these constraints are used to determine the acceptable ranges of values to try when determining the optimal values. Values outside of these constraints will NOT be tried.

A Feature - a piece of information that is potentially useful for prediction. Any attribute could be a feature, as long as it is useful to the model. Feature engineering – Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data. It’s a vaguely agreed space of tasks related to designing feature sets for Machine Learning applications. Components: First, understanding the properties of the task you’re trying to solve and how they might interact with the strengths and limitations of the model you are going to use. Second, experimental work were you test your expectations and find out what actually works and what doesn’t. Feature engineering as a technique, has three sub categories of techniques: Feature selection, Dimension reduction and Feature generation. Feature Selection: Sometimes called feature ranking or feature importance, this is the process of ranking the attributes by their value to predictive ability of a model. Algorithms such as decision trees automatically rank the attributes in the data set. The top few nodes in a decision tree are considered the most important features from a predictive stand point. As a part of a process, feature selection using entropy based methods like decision trees can be employed to filter out less valuable attributes before feeding the reduced dataset to another modeling algorithm. Regression type models usually employ methods such as forward selection or backward elimination to select the final set of attributes for a model. For example: Project Development decision-tree:                                                  Dimension Reduction: This is sometimes called feature extraction. The most classic example of dimension reduction is principle component analysis or PCA. PCA allows us to combine existing attributes into a new data frame consisting of a much reduced number of attributes by utilizing the variance in the data. The attributes which "explain" the highest amount of variance in the data form the first few principal components and we can ignore the rest of the attributes if data dimensionality is a problem from a computational standpoint. Feature Generation or Feature Construction: Quite simply, this is the process of manually constructing new attributes from raw data. It involves intelligently combining or splitting existing raw attributes into new one which have a higher predictive power. For example a date stamp may be used to generate 2 new attributes such as AM and PM which may be useful in discriminating whether day or night has a higher propensity to influence the response variable. Feature construction is essentially a data transformation process. Tips for Better Feature Engineering Tip 1: Think about inputs you can create by rolling up existing data fields to a higher/broader level or category. As an example, a person’s title can be categorized into strategic or tactical. Those with titles of “VP” and above can be coded as strategic. Those with titles “Director” and below become tactical. Strategic contacts are those that make high-level budgeting and strategic decisions for a company. Tactical are those in the trenches doing day-to-day work.  Other roll-up examples include: Collating several industries into a higher-level industry: Collate oil and gas companies with utility companies, for instance, and call it the energy industry, or fold high tech and telecommunications industries into a single area called “technology.” Defining “large” companies as those that make $1 billion or more and “small” companies as those that make less than $1 billion.   Tip 2: Think about ways to drill down into more detail in a single field. As an example, a contact within a company may respond to marketing campaigns, and you may have information about his or her number of responses. Drilling down, we can ask how many of these responses occurred in the past two weeks, one to three months, or more than six months in the past. This creates three additional binary (yes=1/no=0) data fields for a model. Other drill-down examples include: Cadence: Number of days between consecutive marketing responses by a contact: 1–7, 8–14, 15–21, 21+ Multiple responses on same day flag (multiple responses = 1, otherwise =0) Tip 3: Split data into separate categories also called bins. For example, annual revenue for companies in your database may range from $50 million (M) to over $1 billion (B). Split the revenue into sequential bins: $50–$200M, $201–$500M, $501M–$1B, and $1B+. Whenever a company falls with the revenue bin it receives a one; otherwise the value is zero. There are now four new data fields created from the annual revenue field. Other examples are: Number of marketing responses by contact: 1–5, 6–10, 10+ Number of employees in company: 1–100, 101–500, 502–1,000, 1,001–5,000, 5,000+ Tip 4: Think about ways to combine existing data fields into new ones. As an example, you may want to create a flag (0/1) that identifies whether someone is a VP or higher and has more than 10 years of experience. Other examples of combining fields include: Title of director or below and in a company with less than 500 employees Public company and located in the Midwestern United States You can even multiply, divide, add, or subtract one data field by another to create a new input. Tip 5: Don’t reinvent the wheel – use variables that others have already fashioned. Tip 6: Think about the problem at hand and be creative. Don’t worry about creating too many variables at first, just let the brainstorming flow.

Hi,   I have just been playing around with the Edge MicroServer on a couple of Raspberry Pi's.  I obviously want them connected to ThingWorx, however they stop when the SSH session closes which isn't ideal.  I thought about doing something really quick and dirty using 'nohup', but this could have lead to many running processes, and wouldn't still not have started automatically when the Pi booted.  So, I did it right instead using init.d daemon configurations.   There are two files; one for the EMS and one for the Lua Script Resource.  You need to put these in /etc/init.d, then make sure they are owned by root.   sudo chown root /etc/init.d/thingworx* sudo chgrp root /etc/init.d/thingworx*   You'll need to modify the paths in the first lines of these files to match where you have your microserver folder.  Then you need to update the init.d daemon configs and enable the services.   sudo update-rc.d thingworx-ems defaults sudo update-rc.d thingworx-ems enable sudo update-rc.d thingworx-lsr defaults sudo update-rc.d thingworx-lsr enable   You can then start (stop, etc.) like this:   sudo service thingworx-ems start sudo service thingworx-lsr start   They should both also start automatically after a reboot.   Regards,   Greg

Merging InfoTables can seem to be an overwhelming task in ThingWorx. What even IS an InfoTable?? The short answer is that an InfoTable is a specifically structured JSON object. Merging InfoTables is therefore as easy as incrementing through each, adding the columns of each to another table, and then populating the new table with data. For InfoTables with the same DataShapes, the built in "Union" service can be used. Find this snippet under the InfoTableResources section. There is also a snippet for incrementing through DataShape fields if the DataShape is known. For InfoTables which don't have the same DataShapes or for which the DataShape is not known, things get a bit trickier. Thankfully, some new KCS documentation provides example code to step you through merging any number of tables together. I hope this documentation is helpful!

As per ThingWorx Documentation: Updating Properties Automatically in a Mashup A mashup using the GetProperties service can be configured to use websockets and receive updates to properties automatically. When creating or editing a mashup, you can configure the GetProperties service so that the properties are automatically updated by selecting the Automatically update values when able checkbox in the service properties panel. So, the feature to update Properties Automatically in a Mashup is limited to GetProperties service only. Following are the steps to invoke our own custom Service automatically when a property change: 1. Find all the Properties in your Thing for which the DataChange should trigger the custom service. 2. In mashup; add value display widget (or some other widgets) for each property in Step1. 3. Bind the properties from GetProperties service to these widget. 4. Set visible property of these widgets to false so that they don't show up at the RunTime. 5. Now bind the ServiceInvokeCompleted event of GetProperties Service to your custom Service. 6. Save and view Mashup. Result: When any of the Property from Step1 is changed; Custom Service will be invoked in our Mashup.

This simple example creates an infotable of hierarchical data from an existing datashape that can be used in a tree or D3 Tree widget.  Attachments are provides that give a PDF document as well as the ThingWorx entities for the example.  If you were just interested in the service code, it is listed below: var params = {   infoTableName : "InfoTable",   dataShapeName : "TreeDataShape" }; // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(TreeDataShape) var result = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); var NewRow = {}; NewRow.Parent = "No Parent"; NewRow.Child = "Enterprise"; NewRow.ChildDescription = "Enterprise"; result.AddRow(NewRow); NewRow.Parent = "Enterprise"; NewRow.Child = "Site1"; NewRow.ChildDescription = "Wilmington Plant"; result.AddRow(NewRow); NewRow.Parent = "Site1"; NewRow.Child = "Site1.Line1"; NewRow.ChildDescription = "Blow Molding"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line1"; NewRow.Child = "Site1.Line1.Asset1"; NewRow.ChildDescription = "Preform Staging"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line1"; NewRow.Child = "Site1.Line1.Asset2"; NewRow.ChildDescription = "Blow Molder"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line1"; NewRow.Child = "Site1.Line1.Asset3"; NewRow.ChildDescription = "Bottle Unscrambler"; result.AddRow(NewRow); NewRow.Parent = "Site1"; NewRow.Child = "Site1.Line2"; NewRow.ChildDescription = "Filling"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line2"; NewRow.Child = "Site1.Line2.Asset1"; NewRow.ChildDescription = "Rinser"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line2"; NewRow.Child = "Site1.Line2.Asset2"; NewRow.ChildDescription = "Filler"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line2"; NewRow.Child = "Site1.Line2.Asset3"; NewRow.ChildDescription = "Capper"; result.AddRow(NewRow); NewRow.Parent = "Site1"; NewRow.Child = "Site1.Line3"; NewRow.ChildDescription = "Packaging"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line3"; NewRow.Child = "Site1.Line3.Asset1"; NewRow.ChildDescription = "Printer Labeler"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line3"; NewRow.Child = "Site1.Line3.Asset2"; NewRow.ChildDescription = "Packer"; result.AddRow(NewRow); NewRow.Parent = "Site1.Line3"; NewRow.Child = "Site1.Line3.Asset3"; NewRow.ChildDescription = "Palletizing"; result.AddRow(NewRow); NewRow.Parent = "Enterprise"; NewRow.Child = "Site2"; NewRow.ChildDescription = "Mobile Plant"; result.AddRow(NewRow); NewRow.Parent = "Site2"; NewRow.Child = "Site2.Line1"; NewRow.ChildDescription = "Blow Molding"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line1"; NewRow.Child = "Site2.Line1.Asset1"; NewRow.ChildDescription = "Preform Staging"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line1"; NewRow.Child = "Site2.Line1.Asset2"; NewRow.ChildDescription = "Blow Molder"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line1"; NewRow.Child = "Site2.Line1.Asset3"; NewRow.ChildDescription = "Bottle Unscrambler"; result.AddRow(NewRow); NewRow.Parent = "Site2"; NewRow.Child = "Site2.Line2"; NewRow.ChildDescription = "Filling"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line2"; NewRow.Child = "Site2.Line2.Asset1"; NewRow.ChildDescription = "Rinser"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line2"; NewRow.Child = "Site2.Line2.Asset2"; NewRow.ChildDescription = "Filler"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line2"; NewRow.Child = "Site2.Line2.Asset3"; NewRow.ChildDescription = "Capper"; result.AddRow(NewRow); NewRow.Parent = "Site2"; NewRow.Child = "Site2.Line3"; NewRow.ChildDescription = "Packaging"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line3"; NewRow.Child = "Site2.Line3.Asset1"; NewRow.ChildDescription = "Printer Labeler"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line3"; NewRow.Child = "Site2.Line3.Asset2"; NewRow.ChildDescription = "Packer"; result.AddRow(NewRow); NewRow.Parent = "Site2.Line3"; NewRow.Child = "Site2.Line3.Asset3"; NewRow.ChildDescription = "Palletizing"; result.AddRow(NewRow);

If the ThingWorx 7.4 installation with MSSQL db doesn't start with a license error on the splash screen, despite taking the necessary steps for specifying the license path, the error might be misleading and the problem is actually lying in the database connection. Going to THingworxStorage/logs and opening ApplicationLog.log might reveal the following (or similar ) errors: 2017-04-13 10:26:17.993-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Sending Post Start Notifications... 2017-04-13 10:26:17.999-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.001-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.004-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.005-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.006-0400 [L: ERROR] [O: E.c.t.p.d.FileTransferDocumentModelProvider] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [context: Could not create a transaction for ThingworxPersistenceProvider][message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.007-0400 [L: ERROR] [O: c.t.p.m.MssqlModelExceptionTranslator] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] [message: Could not create a transaction for ThingworxPersistenceProvider] 2017-04-13 10:26:18.008-0400 [L: ERROR] [O: c.t.s.s.f.FileTransferSubsystem] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Error loading queued transfer jobs from persistence provider 2017-04-13 10:26:18.008-0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [T: localhost-startStop-1] Thingworx Server Application...ON A few things to check here. First, how the database was set up. The standard expected port if 1433, however, if you choose a different port - ensure that the port is available. If it isn't - update the port setting in SQL configuration manager. Another possible root cause -- the database scripts did not run properly upon the setup. Currently, there is an active Jira PSPT-3587 for resolving the script issue (NOTE, this thread to be updated once the Jira is fixed). There are two options here, either to run the sql files manually (found in the "install" folder of the downloadable), or edit the bat scripts manually. It's recommended to edit the script files instead because running the sql commands allows more room for a mistake. The following line in the bat scripts needs to be edited to have the ".\" removed: sqlcmd.exe -S %server%\%serverinstance%,%port% -U %adminusername% -v loginname=%loginname% -v database=%database% -v thingworxusername=%thingworxusername% -v schema=%schema%  -i .\thingworx-database-setup.sql Note, that the scripts need to be run from the same directory ("/install")

This Javascript snippet creates a random value between those limits:   var dbl_Value = Math.floor(Math.random()*(max-min+1)+min);

Usually we want to search out all User list in ThingWorx with Service GetEntityList   But it only shows limited information. In order to see more details like User Extension information etc., and in order to add more search conditions we could encapsulate it in a new created service. Below is an example code: input: emailAddress(String) output: INFOTABLE // Code start here // step 1 Get all user list var params = {            maxItems: undefined /* NUMBER */,           nameMask: undefined /* STRING */,           type: "User" /* STRING */,           tags: undefined /* TAGS */   };   var users = Resources["EntityServices"].GetEntityList(params);   // step 2 get all other properties for user list var params = {            infoTableName: "infotable" /* STRING */,            dataShapeName: "userPropertiesDS" /* DATASHAPENAME */   };   var infotable = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);   for (var v=0;v<users.length;v++){       var row = new Object();       row.EmailAddress= Users[users .name].emailAddress;            row.name = users .name;       row.fullName = Users[users .name].fullName;       infotable.AddRow(row);   // ...... // Add any other user properties you want to display } // step 3 filter the user list with search conditions // You could add as many parameters as you like var query = {   "filters": {     "type": "AND",     "filters": [       {         "fieldName": "EmailAddress",         "type": "EQ",         "value": emailAddress       },       {         "fieldName": "name",         "type": "EQ",         "value": "user1"       }     ]   } }; var params = {          t: infotable /* INFOTABLE */,          query: query /* QUERY */   }; // result: INFOTABLE var result = Resources["InfoTableFunctions"].Query(params);     Besides, to create a query is also a one step operation in Thingworx , you do not need to create it manually:

This example is to achieve to update objects in Windchill thru extensions. It is really hard to find a resource for Windchill extension's services to take an advantage of them. So, I wrote a simple example to update objects in Windchill from Thingworx.   There are three data shapes needed to do this. One is "PTC.PLM.WindchillPartUfids" which has only "value" field (String) in it and another is "PTC.PLM.WindchillPartCheckedOutDS" which has a "ufid" field (String). Last one is "PTC.PLM.WindchillPartPropertyDS" which has a "ufid" field (String) and fields for "attributes". For an instance of the last data shape, there might be three fields as "ufid", "partPrice" and "quantity" to update parts. In this example, this data shape has two fields which are "ufid" and "almProjectId".   In this example, this needs two input parameters. One is ufid (String) and almProjectId (String). If you need to have multiple objects to update at once, you can use InfoTable type as an "ufid" input parameter instead of String type.   Note that this is an example code and need to handle exceptions if needed.     // To var params = {     infoTableName : "InfoTable",     dataShapeName : "PTC.PLM.WindchillPartUfids" };   // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(PTC.PLM.WindchillPartUfids) var ufids = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);   // PTC.PLM.WindchillPartUfids entry object var newValue = new Object(); newValue.value = ufid; // STRING   ufids.AddRow(newValue);   // Check out var params = {     ufids: ufids /* INFOTABLE */,     comment: undefined /* STRING */,     dataShape: "PTC.PLM.WindchillPartCheckedOutDS" /* DATASHAPENAME */ };   // checkedOutObjs: INFOTABLE dataShape: "undefined" var checkedOutObjsFromService = me.CheckOut(params);   var params = {     infoTableName : "InfoTable",     dataShapeName : "PTC.PLM.WindchillPartUfids" };   // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(PTC.PLM.WindchillPartUfids) var checkedOutObjs = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);   try {     var tableLength = checkedOutObjsFromService.rows.length;       for (var x = 0; x < tableLength; x++) {         var row = checkedOutObjsFromService.rows;               // PTC.PLM.WindchillPartUfids entry object         var checkedOutObj = new Object();         checkedOutObj.value = row.ufid.substring(0,row.ufid.lastIndexOf(":")); // STRING               //logger.warn("UFID : " + checkedOutObj.value);         checkedOutObjs.AddRow(checkedOutObj);           /* Update Objects in Windchill */         var params = {             infoTableName : "InfoTable",             dataShapeName : "PTC.PLM.WindchillPartPropertyDS"         };           // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(PTC.ALM.WindchillPartPropertyDS)         var wcInfoTable = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);           // PTC.ALM.WindchillPartPropertyDS entry object         var newEntry = new Object();         newEntry.ufid = checkedOutObj.value; // STRING         newEntry.almProjectId = almProjectId; // STRING           wcInfoTable.AddRow(newEntry);           var params = {             objects: wcInfoTable /* INFOTABLE */,             modification: "REPLACE" /* STRING */,             dataShape: "PTC.PLM.WindchillPartCheckedOutDS" /* DATASHAPENAME */         };           // result: INFOTABLE dataShape: "undefined"         var result = me.Update(params);     }   } catch(err) {     logger.warn("ERROR Catched");     var params = {         ufids: ufids /* INFOTABLE */,         dataShape: "PTC.PLM.WindchillPartCheckedOutDS" /* DATASHAPENAME */     };       // result: INFOTABLE dataShape: "undefined"     var result = me.CancelCheckOut(params);  }   var params = {     ufids: checkedOutObjs /* INFOTABLE */,     comment: undefined /* STRING */,     dataShape: "PTC.PLM.WindchillPartCheckedOutDS" /* DATASHAPENAME */ };   // result: INFOTABLE dataShape: "undefined" var result = me.CheckIn(params);

In the following scenario (for redhat in this case), running the dbsetup script results in the error: ./thingworxPostgresDBSetup.sh psql:./thingworx-database-setup.sql:1: ERROR:  syntax error at or near ":" LINE 1: CREATE TABLESPACE :"tablespace" OWNER :"username" location :... ^ psql:./thingworx-database-setup.sql:3: ERROR:  syntax error at or near ":" LINE 1: GRANT ALL PRIVILEGES ON TABLESPACE :"tablespace" to :"userna... ^ psql:./thingworx-database-setup.sql:5: ERROR:  syntax error at or near ":" LINE 1: GRANT CREATE ON TABLESPACE :"tablespace" to public; ^ psql:./thingworx-database-setup.sql:14: ERROR:  syntax error at or near ":" LINE 1: CREATE DATABASE :"database" WITH ^ psql:./thingworx-database-setup.sql:16: ERROR:  syntax error at or near ":" LINE 1: GRANT ALL PRIVILEGES ON DATABASE :"database" to :"username"; Given that the installed components match the requirements guide (tomcat 8, Postgresql 9.4.5+ for Thingworx 7.x), run the following command: Run this directly from bin directory of postgres deployment – psql -q -h localhost -U twadmin -p 5432 -v database=thingworx -v tablespace=thingworx -v tablespace_location=/app/navigate/ThingworxPostgresqlStorage -v username=twadmin That must get into command line interface. From there  run the following with full qualified path to the sql file on disk (replace FULLPATH with the path to sql file ) \i ./FULLPATH/thingworx-database-setup.sql If you are experiencing the above-mentioned syntax error, then likely the output will be: psql: FATAL:  database "twadmin" does not exist. Then from postgres bin directory, run the following: ./psql postgres \set Then the second command; \q psql -q -h localhost -U twadmin -p 5432 -v database=thingworx -v tablespace=thingworx -v tablespace_location=/app/navigate/ThingworxPostgresqlStorage -v username=twadmin \set   We see the following outputs: ./psql postgres Password: psql.bin (9.4.11) Type "help" for help. postgres=# \set AUTOCOMMIT = 'on' PROMPT1 = '%/%R%# ' PROMPT2 = '%/%R%# ' PROMPT3 = '>> ' VERBOSITY = 'default' VERSION = 'PostgreSQL 9.4.11 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55), 64-bit' DBNAME = 'postgres' USER = 'postgres' PORT = '5432' ENCODING = 'UTF8' postgres=# \q -bash-4.1$ psql -q -h localhost -U twadmin -p 5432 -v database=thingworx -v tablespace=thingworx -v tablespace_location=/ThingworxPostgresqlStorage -v username=twadmin Password for user twadmin: twadmin=# \set AUTOCOMMIT = 'on' QUIET = 'on' PROMPT1 = '%/%R%# ' PROMPT2 = '%/%R%# ' PROMPT3 = '>> ' VERBOSITY = 'default' VERSION = 'PostgreSQL 8.4.20 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-17), 64-bit' database = 'thingworx' tablespace = 'thingworx' tablespace_location = '/ThingworxPostgresqlStorage' username = 'twadmin' DBNAME = 'twadmin' USER = 'twadmin' HOST = 'localhost' PORT = '5432' ENCODING = 'UTF8' Note, even though Postgresql 9.4.5 has been installed by the system administrator, there are still traces of Postgresql 8.4.20 present in the system that cause the syntax error issue (possibly as part of  the default OS packaging). Removing the 8.4.20 rpms will resolve the problem.