IoT Tips

OpenDJ is a directory server which is also the base for WindchillDS. It can be used for centralized user management and ThingWorx can be configured to login with users from this Directory Service.   Before we start Pre-requisiste Docker on Ubuntu JKS keystore with a valid certificate JKS keystore is stored in /docker/certificates - on the machine that runs the Docker environments Certificate is generated with a Subject Alternative Name (SAN) extension for hostname, fully qualified hostname and IP address of the opendj (Docker) server Change the blue phrases to the correct passwords, machine names, etc. when following the instructions If possible, use a more secure password than "Password123456"... the one I use is really bad   Related Links https://hub.docker.com/r/openidentityplatform/opendj/ https://backstage.forgerock.com/docs/opendj/2.6/admin-guide/#chap-change-certs https://backstage.forgerock.com/knowledge/kb/article/a43576900   Configuration Generate the PKCS12 certificate Assume this is our working directory on the Docker machine (with the JKS certificate in it)   cd /docker/certificates   Create .pin file containing the keystore password   echo "Password123456" > keystore.pin   Convert existing JKS keystore into a new PKCS12 keystore   keytool -importkeystore -srcalias muc-twx-docker -destalias server-cert -srckeystore muc-twx-docker.jks -srcstoretype JKS -srcstorepass `cat keystore.pin` -destkeystore keystore -deststoretype PKCS12 -deststorepass `cat keystore.pin` -destkeypass `cat keystore.pin`   Export keystore and Import into truststore   keytool -export -alias server-cert -keystore keystore -storepass `cat keystore.pin` -file server-cert.crt keytool -import -alias server-cert -keystore truststore -storepass `cat keystore.pin` -file server-cert.crt     Docker Image & Container Download and run   sudo docker pull openidentityplatform/opendj sudo docker run -d --name opendj --restart=always -p 389:1389 -p 636:1636 -p 4444:4444 -e BASE_DN=o=opendj -e ROOT_USER_DN=cn=Manager -e ROOT_PASSWORD=Password123456 -e SECRET_VOLUME=/var/secrets/opendj -v /docker/certificates:/var/secrets/opendj:ro openidentityplatform/opendj   After building the container, it MUST be restarted immediately in order for recognizing the new certificates   sudo docker restart opendj   Verify that the certificate is the correct one, execute on the machine that runs the Docker environments: openssl s_client -connect localhost:636 -showcerts   Load the .ldif Use e.g. JXplorer and connect   Select the opendj node LDIF > Import File (my demo breakingbad.ldif is attached to this post) Skip any warnings and messages and continue to import the file   ThingWorx Tomcat If ThingWorx runs in Docker as well, a pre-defined keystore could be copied during image creation. Otherwise connect to the container via commandline: sudo docker exec -it <ThingworxImageName> /bin/sh Tomcat configuration cd /usr/local/openjdk-8/jre/lib/security openssl s_client -connect 10.164.132.9:636 -showcerts Copy the certifcate between BEGIN CERTIFACTE and END CERTIFICATE of above output into opendj.pem, e.g. echo "<cert_goes_here>" > opendj.pem Import the certificate keytool -keystore cacerts -import -alias opendj -file opendj.pem -storepass changeit   ThingWorx Composer As the IP address is used (the hostname is not mapped in Docker container) the certificate must have a SAN containing the IP address     Only works with the TWLDAPExample Directory Service not the ADDS1, because ADDS1 uses hard coded Active Directory queries and structures and therefore does not work with OpenDJ. User ID (cn) must be pre-created in ThingWorx, so the user can login. There is no automatic user creation by the Directory Service. Make sure the Thing is Enabled under General Information   Appendix LDAP Structure for breakingbad.ldif cn=Manager / Password123456 All users with password Password123456    

Contents: Introduction Prerequisites Installing Java Installing PostgreSQL Running the Installer Post Installation Steps Troubleshooting tips   Introduction:   Starting with ThingWorx 8.4, PTC released a new way to install a fresh ThingWorx environment.  This installer takes care of all the permissions, database scripts, credential encryption, and tomcat options that previously needed to be done manually.  More information on the installer can be found in the ThingWorx Help Center   NOTE: This is different than the Docker installer we have available in earlier releases.   As of right now, the installation guide has very basic instructions for the installer.  The purpose of this post is to show you from start to finish what the process looks like.  For this example, I chose to deploy PostgreSQL 10 on the local system to keep things simple.   Prerequisites:   Download the latest Java 8 SE JDK RPM for RHEL Get your database ready: If you're accessing a remote PostgreSQL instance, make sure PSQL is installed and working on your ThingWorx Server Download the appropriate installer from support.ptc.com Ensure the RHEL user that will be executing the installer has SUDO privileges   NOTE: There are pieces of the manual installation guide that I had to reference in order to get JAVA and PostgreSQL properly configured.   Installing Java:   Per Page 83, I downloaded the latest Linux x64 RPM for Java 8 SE JDK (201) and followed steps 2-8 to configure Java. For step 5, I needed to use the -f parameter listed in the guide under NOTE Step 7 make sure you don't accidentally select OpenJDK if it was preinstalled   Installing PostgreSQL:   I'm following along with the Version 10 download instructions found on https://www.postgresql.org/download/linux/redhat/ NOTE: this needs root access, so run all the commands with SUDO Install the client packages Postgresql10 I will Install the optional server packages postgresql10-server since this is a local PostgreSQL instance Complete step 7 to enable automatic start.  We need to set the postgres password so our ThingWorx installer is able to create our thingworx user and the database.  This can be done with the following command: NOTE: Since this is the master user for your database, it is highly recommended to use a password that has a combination of case, numbers, letters, and symbols Sudo passwd postgres Although, this may be redundant, I also run the following command to update the password used in PostgreSQL : sudo -u postgres psql -c "ALTER ROLE postgres WITH password '<password from above>'" Navigate to /var/lib/pgsql/10/data and open pg_hba.conf for editing Review page 91 of the Installation guide to determine which setting best applies to your business needs In the same directory open postgresql.conf Scroll down to "listen_addresses" line and un-comment it.  This would  be the place to make changes if you expect remote connections to access the database.  If it is local, then the default of localhost is fine Restart PostgreSQL to apply these changes: Sudo service postgresql-10 restart   Running the Installer:   Everything should be in place now to run our installer.  Extract the ThingWorxFoundationPostgres-1.2.0-SNAPSHOT.run file to the ~ (home) directory Execute the .run file: NOTE: If it doesn't let you execute the file, it may not have extracted as an executable.  Run the below command to make it executable then try again: Chmod -x ThingWorxFoundationPostgres-1.2.0-SNAPSHOT.run Sudo ./ThingWorxFoundationPostgres-1.2.0-SNAPSHOT.run   At this point you'll be going through text to setup your installation settings.  I'll briefly list out the order you'll see them below: Terms and conditions and whether you agree Where you want ThingWorx deployed (/opt by default) NOTE: this folder will contain ThingworxStorage/ ThingworxPlatform/ tomcat/ etc… Installation Configuration user (twxfoundation by default).  This step creates a user in RHEL that will have ownership of Tomcat, various ThingWorx directory's, etc ThingWorx Administrator Password.  Used to login to ThingWorx Composer. WRITE THIS DOWN SOMEWHERE!  You cannot retrieve this password, and most likely will require you to do a fresh installation if you forget it Tomcat Port http (8080) Tomcat SSL port (8443) Use SSL For simplicity, I chose not to use it for this exercise PostgreSQL information Host Name : mine is local, so localhost Port (5432) Administrator Username (Administrator) : use postgres here, since that's the DB user password we updated above Admin password : use the postgres password ThingWorx Database login username (twadmin).  This user will be created in PostgreSQL and be tied to our ThingWorx database ThingWorx database login password: NOTE There's no place to re-enter your password, so make sure you write this down.   Unexpected issue:   For this particular install, I kept running into a failure saying "Warning: Failed to validate the PostgreSQL connection.  Check the information you entered".  I opened another putty connection and, as root, navigated to /var/lib/pgsql/10/data/log and opened the postgresql log to find the following:   2019-02-28 17:10:30.678 UTC [93377] LOG:  could not connect to Ident server at address "::1", port 113: Connection refused 2019-02-28 17:10:30.678 UTC [93377] FATAL:  Ident authentication failed for user "postgres" 2019-02-28 17:10:30.678 UTC [93377] DETAIL:  Connection matched pg_hba.conf line 84: "host    all             all             ::1/128                 ident"     The solution for me was to go into the pg_hba.conf and change the IPv6 local connections from ident to md5.  Again, make sure you are reading through the PostgreSQL documentation and adjusting these properties in a way that meets both your security and business needs.   Once the change was made, I restarted postgresql, and switched back over to my Putty instance that had the installer going.     A summary pops up for a few items, and then it asks if you're ready to continue NOTE: The progress bar goes to 100% pretty quickly, and doesn't appear to move.  Just let it sit for a few minutes while it finishes up Copy the Thingworx Device ID for future reference To check if ThingWorx is running, run 'sudo service Thingworx-Foundation status' in your command line If it is active (running) try to access it with a remote browser: More information around the command Firewalld can be found here  http://<thingworxurl>:<tomcatport>/Thingworx NOTE: If it just hangs, check your firewall to make sure the port is open for external communication   Post Installation Steps:   Licensing: Navigate to /opt/ThingWorxPostgres-1.2.0-SNAPSHOT/licensingconfigurator and run the twx-licensing-configurator.run as SUDO Choose whether or not you want PTC to store your credentials and download the license for you, or if you want to manually download the license yourself from http://support.ptc.com -> Manage Licenses (bottom right) For this example, I manually downloaded the license Move the license file over to the ThingWorx Server Since you're running the licensingconfigurator as SUDO, don't put this file into your user's home directory.  Instead, put it into /tmp NOTE: Change the downloaded filename to license_capability_response.bin.  Otherwise the file will not be recognized Then it will ask for your ThingWorx Administrator password This appears to be used for verification after the license is in place, and it sees if it can successfully log into your system Once it has completed, and assuming it says "Setup has finished configuration licensing for ThingWorx", open up a web browser and login as Administrator -> Monitor -> Subsystems -> Licensing Subsystem and verify that your licensing information looks correct on the system   Extensions: Extra security has been added as of 8.4 around importing Extensions.  More details can be found in the Help Center In short, adding extensions is disabled by default, and you need to add some lines into your /ThingworxPlatform/platform-settings.json under the "PlatformSettingsConfig" section. For example, here is what I added:    "PlatformSettingsConfig": {                 "BasicSettings": {                         "BackupStorage": "/opt/ThingWorxPostgres-1.2.0-SNAPSHOT/ThingworxBackupStorage",                         "DatabaseLogRetentionPolicy": 7,                         "EnableBackup": true,                         "EnableHA": false,                         "EnableSystemLogging": true,                         "HTTPRequestHeaderMaxLength": 2000,                         "HTTPRequestParameterMaxLength": 2000,                         "InternalAesCryptographicKeyLength": 128,                         "Storage": "/opt/ThingWorxPostgres-1.2.0-SNAPSHOT/ThingworxStorage"                 },                 "ExtensionPackageImportPolicy": {                        "importEnabled": true,                        "allowJarResources": true,                        "allowJavascriptResources": false,                        "allowCSSResources": false,                        "allowJSONResources": false,                        "allowWebAppResources": false,                        "allowEntities": true,                        "allowExtensibleEntities": false       }           }   Make sure you set the appropriate items above to true based on what your extensions require   Troubleshooting:   If things backfire, depending on where you are in the setup process, the following logs should be looked at for clues on the failure:   Installation: /tmp/bitrock_installer.logs I believe the installation directory (default /opt/ThingWorxPostgres-1.2.0-SNAPSHOT) will contain a log file if the installer fails /opt/ThingWorxPostgres-1.2.0-SNAPSHOT/ThingworxStorage/logs/ (need root access) /opt/ThingWorxPostgres-1.2.0-SNAPSHOT/tomcat/apache-tomcat-<version>/logs PostgreSQL (requires root): /var/lib/pgsql/10/data/log LicensingConfigurator : /opt/ThingWorxPostgres-1.2.0-SNAPSHOT/licensingconfigurator

This post adds to my previous post: Deploying H2 Docker versions quickly   In addition to configuring the basic Docker Images and Containers, it's also possible to deploy them with a TLS / SSL certificate and access the instances via HTTPS protocol.   For this a valid certificate is required inside a .jks keystore. I'm using a self-signed certificate, but commercial ones are even better! The certificate must be in the name of the machine which runs Docker and which is accessed by the users via browser. In my case this is "mne-docker". The password for the keystore and the private key must be the same - this is a Tomcat limitation. In my case it's super secret and "Password123456".   I have the following directory structure on my Operating System   /home/ts/docker/ certificates mne-docker.jks twx.8.2.x.h2 Dockerfile settings platform-settings.json <license_file> storage Thingworx.war   The Recipe File   In the Recipe File I make sure that I create a new Connector on port 8443, removing the old one on port 8080. I do this by just replacing via the sed command - also introducing options for content compression. I'm only replacing the first line of the xml node as it holds all the information I need to change.   Changes to the original version I posted are in green   FROM tomcat:latest MAINTAINER mneumann@ptc.com LABEL version = "8.2.0" LABEL database = "H2" RUN mkdir -p /cert RUN mkdir -p /ThingworxPlatform RUN mkdir -p /ThingworxStorage RUN mkdir -p /ThingworxBackupStorage ENV LANG=C.UTF-8 ENV JAVA_OPTS="-server -d64 -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8 -Duser.timezone=GMT -XX:+UseNUMA -XX:+UseG1GC -Djava.library.path=/usr/local/tomcat/webapps/Thingworx/WEB-INF/extensions RUN sed -i 's/<Connector port="8080" protocol="HTTP\/1.1"/<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol" maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" enableLookups="false" keystoreFile="\/cert\/mne-docker.jks" keystorePass="Password123456" ciphers="TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA" compression="on" compressableMimeType="text\/html,text\/xml,text\/plain,text\/css,text\/javascript,application\/javascript,application\/json"/g' /usr/local/tomcat/conf/server.xml COPY Thingworx.war /usr/local/tomcat/webapps VOLUME ["/ThingworxPlatform", "/ThingworxStorage", "/cert"] EXPOSE 8443   Note that I also map the /cert directory to the outside, so all of my Containers can access the same certificate. I will access it read-only.   Deploying     sudo docker build -t twx.8.2.x.h2 . sudo docker run -d --name=twx.8.2.x.h2 -p 88:8443 -v /home/ts/docker/twx.8.2.x.h2/storage:/ThingworxStorage -v /home/ts/docker/twx.8.2.x.h2/settings:/ThingworxPlatform -v /home/ts/docker/certificates:/cert:ro twx.8.2.x.h2   Mapping to the 8443 port ensures to only allow HTTPS connections. The :ro in the directory mapping ensures read-only access.   What next   Go ahead! Only secure stuff is kind of secure 😉 For more information on how to import the certificate into a the Windows Certificate Manager so browsers recognize it, see also the Trusting the Root CA chapter in Trust & Encryption - Hands On

ThingWorx offers Docker based installations utilizing existing PostgreSQL databases. In newer releases ThingWorx Docker installers also offer using other databases.   Personally I'm using a certain method of deployment where I can just easily exchange some files, create new images and have a H2 based environment running for some quick tests.   As H2 is a built-in database, I will not dive into setting up the platform-settings.json for other connectivity. However other databases can be connected to by adjusting the platform-settings.json. This might also require an internal Docker Network structure which I will not elaborate on here.   Note: the following procedure is not fully supported as it's not using the deployment methods provided by the installers!   Create the Directory Structure   My Directory structure looks the following (expanded for the 8.2.x branch):   /home/ts/docker/ twx.8.0.x.h2 twx.8.1.x.h2 twx.8.2.x.h2 Dockerfile settings platform-settings.json <license_file> storage Thingworx.war twx.8.3.x.h2   I have a directory for every version I want to test with.   In each directory there's the Dockerfile - the recipe file I'm using. There's also the version specific Thingworx.war file as well as two directories: settings and storage which I will map to the ThingWorx directories inside the image later.   The Recipe File   FROM tomcat:latest MAINTAINER me@somewhere.com LABEL version = "8.2.0" LABEL database = "H2"  RUN mkdir -p /ThingworxPlatform RUN mkdir -p /ThingworxStorage RUN mkdir -p /ThingworxBackupStorage ENV LANG=C.UTF-8 ENV JAVA_OPTS="-server -d64 -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8 -Duser.timezone=GMT -XX:+UseNUMA -XX:+UseG1GC -Djava.library.path=/usr/local/tomcat/webapps/Thingworx/WEB-INF/extensions COPY Thingworx.war /usr/local/tomcat/webapps VOLUME ["/ThingworxPlatform", "/ThingworxStorage"] EXPOSE 8080   I change the version label to keep track of the versions for each recipe.   Deploying   Build the Docker Image by navigating to the directory where the recipe file is based in   sudo docker build -t twx.8.2.x.h2 .   Create a Docker Container and start it   sudo docker run -d --name=twx.8.2.x.h2 -p 82:8080 -v /home/ts/docker/twx.8.2.x.h2/storage:/ThingworxStorage -v /home/ts/docker/twx.8.2.x.h2/settings:/ThingworxPlatform twx.8.2.x.h2   I change the name of the Image and the Container as well as the external port to distinguish all the different versions. The -v option maps the paths in my Operating System to the paths in the Docker Container, so I can browse the ThingworxStorage and ThingworxPlatform folder without connecting inside the Container. That's quite handy to check the logs, or place the license file.   Starting and Stopping   I can fire up and shut down Containers I need with the following commands:   sudo docker start twx.8.2.x.h2 sudo docker stop twx.8.2.x.h2   What next   That's just my basic setup. Usually I copy & paste a working directory for deploying another version and adjust what needs to be changed. You could use this as a basis for quick and easy deployment where even additional features could be added, i.e. HTTPS configuration or auto-deploying certain ThingWorx Extensions via a REST API call.   To ensure starting with a clean Image, for building new Images I delete the contents of the storage folder and only leave the platform-settings.json in the settings folder (I copy the license later after generating it with my new Device ID).

Let's assume I collect Timeseries Data of two temperature sensors, located next to each other. This is done for redundancy and ensuring the quality of measures. Each of the sensors is logged into its Property in ThingWorx and I can create a Timeseries for the individual sensors. However I would like to create a combined InfoTable that holds information for both sensors, but averages out their values.   Instead of reading values from a stream, I just create some custom data for both InfoTables. After this I use the UNION function to combine the two tables and sort them. Once they are sorted, the INTERPOLATE function allows to group the InfoTable by timestamp.   With this, I have combined the two sensor result into on result set. Taking the average of numbers will give closer results to the real value (as both sensors might not be 100% accurate). In case one sensor does not have data for a given point in time, it will still be considered in the final output.   InfoTable1:   2018-12-18 00:00:00.000 2 2018-12-19 00:00:00.000 3 2018-12-20 00:00:00.000 5 2018-12-21 00:00:00.000 7   InfoTable2:   2018-12-18 00:00:00.000 1 2018-12-19 12:00:00.000 2 2018-12-20 00:00:00.000 3 2018-12-21 00:00:00.000 4   Combined Result:   2018-12-18 00:00:00.000 1.5 2018-12-19 00:00:00.000 3 2018-12-19 12:00:00.000 2 2018-12-20 00:00:00.000 4 2018-12-21 00:00:00.000 5.5     This can be done with the following code:   // Required DataShape "myInfoTableShape": "timestamp" = DATETIME, "value" = NUMBER // The Service Output is an InfoTable based on the same DataShape var params = { infoTableName : "InfoTable", dataShapeName : "myInfoTableShape" }; // Create two InfoTables, representing the data of each sensor var infoTable1 = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); var infoTable2 = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); var newEntry = new Object(); // Create custom data for InfoTable1 newEntry.timestamp = 1545091200000; newEntry.value = 2; infoTable1.AddRow(newEntry); newEntry.timestamp = 1545177600000; newEntry.value = 3; infoTable1.AddRow(newEntry); newEntry.timestamp = 1545264000000; newEntry.value = 5; infoTable1.AddRow(newEntry); newEntry.timestamp = 1545350400000; newEntry.value = 7; infoTable1.AddRow(newEntry); // Create custom data for InfoTable2 newEntry.timestamp = 1545091200000; newEntry.value = 1; infoTable2.AddRow(newEntry); newEntry.timestamp = 1545220800000; newEntry.value = 2; infoTable2.AddRow(newEntry); newEntry.timestamp = 1545264000000; newEntry.value = 3; infoTable2.AddRow(newEntry); newEntry.timestamp = 1545350400000; newEntry.value = 4; infoTable2.AddRow(newEntry); // Combine the two InfoTables via the UNION function var unionTable = Resources["InfoTableFunctions"].Union({ t1: infoTable1, t2: infoTable2 }); // Optional: Sort the table by timestamp var sortedTable = Resources["InfoTableFunctions"].Sort({ sortColumn: "timestamp", t: unionTable, ascending: true }); // Interpolate the (sorted) table by Interval and take average values and build the result var result = Resources["InfoTableFunctions"].Interpolate({ mode: "INTERVAL", timeColumn: "timestamp", t: sortedTable, ignoreMissingData: undefined, stats: "AVG", endDate: 1545609600000, columns: "value", count: undefined, startDate: 1545004800000 });  

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Testing the Mashups Open the rcp_MashupMain in a new browser window For this test I find it easier to have the rcp_AlertThing and the Mashup in two windows side-by-side to each other The Mashup should be completely empty right now Nothing in the historic table (Grid) The Selected Reason is blank The Checkbox is false In the rcp_AlertThing switch the trigger to false The following will now happen The new value will be automatically pushed to Mashup The checkbox will switch to true The validator now throws the TRUE Event, as the condition is met and the trigger is indeed true The TRUE Event will invoke the Navigation Widget's Navigate service and the modal popup will be opened The user now only has the option to select one of the three states offered by the Radio Button selector, everything else will be greyed out After choosing any option, the SelectionChanged Event will be fired and trigger setting the selectedState as well as closing the popup The PopupClosed Event in our MashupMain will then be fired and populate the selectedState parameter into the textbox (just for display) and will also call the SetProperties service on our Thing, updating the selectedReason with the selectedState parameter value Once the property is set and persisted into the ValueStream via the SetProperties' ServiceInvokeCompleted Event, we clear the trigger (back to false) and update the Grid with the new data In the AlertThing, refresh the properties to actually see the trigger false and the selectedReason to whatever the user selected Note: When there is a trigger state and the trigger is set to true the popup will always be shown, even if the user refreshes the UI or the browser window. This is to avoid cheating the system by not entering a root cause for the current issue. As the popup is purely depending on the trigger flag, only clearing the flag can unblock this state. The current logic does not consider to close the popup when the flag is cleared - this could however be implemented using the Validator's FALSE Event and adding additional logic

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Create the Main Mashup Create a new Mashup called "rcp_MashupMain" as Page and Responsive Save and switch to the Design tab Design Add a Layout with two Columns In the right Column add another Layout (vertical) with a Header and one Row Add a Grid to the Row Add a Panel to the Header Add a Panel into the Panel (we will use a Panel-In-Panel technique for a better design experience) Set "Width" to 200 Set "Height" to 50 Set "Horizontal Anchor" to "Center" Set "Vertical Anchor" to "Middle" Delete its current "Style" and add a new custom style - all values to default(this will create a transparent border around the panel) Add a Label to the inner Panel Set "Text" to "Historic data of what went wrong" Set "Alignment" to "Center Aligned" Set "Width" to 200 Set "Top" to 14 Add a Panel to the left Column Add a Navigation Widget to the Panel This will call the Popup Window when its Navigate service is invoked (by a Validator) Set "MashupName" to "rcp_MashupPopup" Set "TargetWindow" to "Modal Popup" Set "ShowCloseButton" to false Set "ModalPopupOpacity" to 0.8 (to make the background darker and give more visual focus to the popup) Set "FixedPopupWidth" to 500 Set "FixedPopupHeight" to 300 Set "PopupScrolling" to "Off" Set "Visible" to false, so it will not be shown to the user during runtime Add a Textbox to the Panel This will show the numeric value corresponding to the State selected in the modal popup This will just be used for displaying with no other functionality - so that we can verify the actual values chosen Set "Read Only" to true Set "Label" to "Selected Reason (numeric value)" Add a Checkbox to the Panel This will be used an input for the Validator to determine if an error state is present or not Set "Prompt" to "Set this box to 'true' to trigger the popup. Set the value via the Thing to simulate a service. Once the value is set, the trigger is set to 'false' as the popup has been dealt with. A new historic entry will be created." Set "Disabled" to true Set "Width" to 250 Add a Validator to the Panel This will determine if the checkbox (based on the trigger / error state) is true or false. If the checkbox switches to true then the validator will call the Navigate service on the Navigation Widget. Otherwise it will do nothing. Click on Configure Validator Add Parameter Name: "Input" Base Type: BOOLEAN Click Done Set "Expression" to "Input" (the Parameter we just created) Set "AutoEvaluate" to true Save the Mashup Data In the Data panel on the right hand side, click on Add entity Choose the "rcp_AlertThing" and select the following services GetProperties (execute when Mashup is loaded) SetProperties QueryPropertyHistory (execute when Mashup is loaded) clearTrigger Click Done and the services will appear in the Data panel Connections After configuring the UI elements and the Data Sources we now have to connect them to implement the logic we decided on earlier GetProperties service Drag and drop the trigger property to the Checkbox and bind it to State Set the Automatically update values when able to true SetProperties service From the Navigation Widget drag and drop the selectedState property and bind it to the SetProperties service selectedReason property From the Navigation Widget drag and drop the PopupClosed event and bind it to the SetProperties service From the SetProperties service drag and drop the ServiceInvokeCompleted event and bind it to the clearTrigger service From the SetProperties service drag and drop the ServiceInvokeCompleted event and bind it to the QueryPropertyHistory service QueryPropertyHistory service Drag and drop the Returned Data's All Data to the Grid and bind it to Data On the Grid click on Configure Grid Columns Switch the position of the timestamp and selectedReason fields with their drag and drop handles For the selectedReason Set the "Column Title" to "Reason for Outage" Switch to the Column Renderer & State Formatting tab Change the format from "0.00" to "0" (as we're only using Integer values anyway) Choose the State-based Formatting Set "Dependent Field" to "selectedReason" Set "State Definition" to "rcp_AlertStateDefinition" Click Done clearTrigger service There's nothing more to configure for this service As the properties will automatically be pushed via the GetProperties service, there's no special action required after the service invoke for the clearTrigger service has been completed Validator Widget Drag and drop the Validator's TRUE event to the Navigation Widget and bind it to the Navigate service Drag and drop the Checkbox State to the Validator and bind it to the Input parameter Navigation Widget Drag and drop the Navigation Widget's selectedState to the Textbox and bind it to the Text property Save the Mashup

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Create a Popup Mashup Create a new Mashup called "rcp_MashupPopup" as Page and Static Save and switch to the Design tab Design Edit the Mashup Properties Set "Width" to 500 Set "Height" to 300 Add a new Label Set "Text" to "Something went wrong - what happend?" Set "Alignment" to "Center Aligned" Set "Width" to 230 Set "Top" to 55 Set "Left" to 130 Add a new Radio Button Set "Button States" to "rcp_AlertStateDefinition" Set "Top" to 145 Set "Left" to 25 Set "Width" to 450 Set "Height" to 100 In the Workspace tab, select the "Mashup" Click on Configure Mashup Parameters Add Parameter Name: "selectedState" BaseType: NUMBER Click Done Save the Mashup Connections Select the Radio Button Drag and drop its Selected Value property to the Mashup and bind it to the selectedState Mashup Parameter Drag and drop its SelectionChanged event to the Mashup and bind it to the CloseIfPopup service Save the Mashup

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Create Entities AlertStateDefinition Create a new StateDefinition called "rcp_AlertStateDefinition" In the State Information tab, select Apply State: Numeric from the list on the right hand side Create a new State: Less than or equal to "1" Display Name: "Something good" Style: a new custom style with text color #f5b83d (orange) Create a new State: Less than or equal to "2" Display Name: "Something bad" Style: a new custom style with text color #f55c3d (red) Create a new State: Less than or equal to "3" Display Name: "Something ugly" Style: a new custom style with text color #ad1f1f (red) with a Font Bold Edit the "Default" State Set the Style: a new custom style with text color #36ad1f (green) We will not use this style, but in case we need a default configuration it will blend into the color schema Save the StateDefinition ValueStream Create a new ValueStream called "rcp_ValueStream" (choose a default ValueStream, not a RemoteValueStream) Save the ValueStream AlertThing Create a new Thing called "rcp_AlertThing" Based on a Generic Thing Base Thing Template Using the rcp_ValueStream Value Stream In the Properties and Alerts tab create the following Properties Name: "trigger" Base Type: BOOLEAN With a Default Value of "false" Check the "Persistent" checkbox Name: "selectedReason" BaseType: NUMBER Check the "Persistent" checkbox Check the "Logged" checkbox Advanced Settings: Data Change Type: ALWAYS In the Services tab create a new Service Name: "clearTrigger" No Inputs and no Outputs Service code me.trigger = false; When this service is executed, it will set the trigger Property to false Click Done to complete the Service creation Save the Thing

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Before we start Create a new Project called "RootCausePopups" and save it. In the New Composer set the Project Context (top left box) to the "RootCausePopups" project. This will automatically add all of our new Entities into our project. Otherwise we would have to add each Entity manually on creation.

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Required Logic The following logic will help us realizing this particular use case: The trigger property on the AlertThing switches from false to true. The MashupMain will receive dynamic Property updates via the AlertThing.GetProperties service. It will validate the value of the trigger Property and if it's true the MashupMain will show the MashupPopup as a modal popup. A modal popup will be exclusively in the foreground, so the user cannot interact with anything else in the Mashup except the modal popup. In the modal popup the user chooses one of the pre-defined AlertStateDefinitions. When a State is selected, the popup will set the State as a Mashup Parameter, pass this to the MashupMain and the popup close itself. When the MashupPopup is closed, the MashupMain will read the Mashup Parameter The MashupMain will set the selectedReason in the AlertThing to the selected value. It will also reset the trigger property to false. This allows the property to be set to true again to trigger another forced popup. On any value change the AlertThing will store the selectedReason State in a ValueStream to capture historic information on which root causes were selected at which time. The ValueStream information will be displayed as a table in a GridWidget in the MashupMain once the new properties have been set.

This post is part of the series Forced Root Cause Monitoring via Mashups and Modal Popups To not feel lost or out of context, it's recommended to read the main post first. Required Entities In this simplified example we'll just use a Thing to set a status triggering the popup. This Thing will have two properties and one service: Properties trigger (Boolean) - to indicate if an error status is present or not, if so - trigger the popup selectedReason (Number) - to indicate the selected reason / root cause chosen in the modal popup Service clearTrigger - to reset the trigger to "false" once a reason has been selected The selectedReason will be logged into a ValueStream. In addition to the Thing and the ValueStream we will need a StateDefinition to pre-define potential root causes to be displayed in the popup. We will use three states to be used in a traffic-light fashion to indicate the severity of the issue in a custom color schema. To display the monitoring Mashup and the popup we will need two Mashups.

Push update what is it? It is a mechanism that GetProperties supports so that the Server can push a value to a client mashup. This will allow you to see values update in your mashup real time without needing the refresh widget. Another great way to use the push updates is to propagate events that tie to specific content fetches. Let's say your mashup has three areas: KPIs, Alerts, Live values. Using some logic server side you can set up a 'tracker' Thing with properties that indicate that one of those areas has updated data. Bring these notifications as property values into the mashup using GetProperties and as the Server pushes updates to the mashup runtime, you can map it to a Validator or Expression widget (set to autoevaluate) which in turn can now run the necessary Service to fetch the updated information for the specific area.

Several times in the past few months I was hit by a quick need to extract some data about Assets for a customer, and find myself continually hand-writing the code to do so.  Rather than repeat myself any more, I figure I can share my work - maybe PTC customers can benefit from the same effort.    import static com.axeda.sdk.v2.dsl.Bridges.* import com.axeda.drm.sdk.Context import com.axeda.common.sdk.id.Identifier import com.axeda.services.v2.* import com.axeda.sdk.v2.exception.* def retStr = "Device and Location Data\n" def modellist = [:] ModelCriteria mc = new ModelCriteria() mc.modelNumber = "*" tcount = 0 def mresults = modelBridge.find(mc) while ( (mresults = modelBridge.find(mc)) != null && tcount < mresults .totalCount) { mresults.models.each { res -> modellist[res.systemId] = res.modelNumber tcount++ } mc.pageNumber = mc.pageNumber + 1 } locationList = [:] LocationCriteria lc = new LocationCriteria() lc.name = "*" tcount = 0 def lresults = locationBridge.find(lc) while ( (lresults = locationBridge.find(lc)) != null && tcount < lresults .totalCount) { lresults.locations.each { res -> locationList[res.systemId] = res.name tcount++ } lc.pageNumber = lc.pageNumber + 1 } AssetCriteria ac = new AssetCriteria() ac.includeDetails = true ac.name = "*" tcount = 0 def results = assetBridge.find(ac) while ( (results = assetBridge.find(ac)) != null && tcount < results .totalCount) { results.assets.each { res -> retStr += "ID: ${res.systemId} MN: ${res.model.systemId},${modellist[res.model.systemId]} SN: ${res.serialNumber} Name: ${res.name} : Location ${res.location.systemId},${locationList[res.location.systemId]}\n"; tcount++ } ac.pageNumber = ac.pageNumber + 1 } return ["Content-Type": "application/text", "Content": retStr] This will output data like so:    ID: 31342 MN: 14682,CKGW SN: Axeda-CK-Windows10VBox Name: Axeda-CK-Windows10VBox : Location 1,Foxboro ID: 26248 MN: 14682,CKGW SN: CK-CKAMINSKI0L1 Name: CK-CKAMINSKI0L1 : Location 1,Foxboro ID: 30082 MN: 14682,CKGW SN: CK-GW1 Name: CK-GW1 : Location 1,Foxboro ID: 26247 MN: 14681,CKGW-ManagedModel1 SN: CK-MM01 Name: CK-MM01 : Location 1,Foxboro This let's me compare the internal systemId of the Asset, the internal systemId of the Model, and the internal systemId of the Location of the device.  This was to help me attempt to isolate an issue with orphaned devices not being returned in a report - exposing some duplicate locations and devices that needed corrections.    You may find yourself needing to do similar things when building logic for Axeda, or eventually integrating or migrating to Thingworx.  Our v2 API bridges help "bridge" the gap.      

Database backups are vital when it comes to ensure data integrity and data safety. PostgreSQL offers simple solutions to generate backups of the existing ThingWorx database instance and recover them when needed.   Please note that this does not replace a proper and well-defined disaster recovery plan. Export and Import are part of this strategy, but are not reflecting the complete strategy. The commands used in this post are for Windows, but can be adjusted to work on Linux-based systems as well.   Backup   To create a Backup, the export / dump functionality of PostgreSQL can be used.     pg_dump -U postgres -C thingworx > thingworxDump.sql     The -C option will include the statement to create the database in the .sql file and map it to the existing tablespace and user (e.g. 'thingworx' and 'twadmin'). The tablespace and user can be seen in the .sql file in the line with "ALTER DATABASE <dbname> OWNER TO <user>;"   In the above example, we're backing up the thingworx database schema and dump it into the thingworxDump.sql file   Tablespace, username & password are also included in the platform_settings.json   Restore   To restore the database, we just assume an empty PostgreSQL installation. We need to create the DB schema user first via the following commandline:     psql -U postgres -c "CREATE USER twadmin WITH PASSWORD 'ts';"     With the user created, we can now re-generate the tablespace and grant permissions to the twadmin user:   psql -U postgres -c "CREATE TABLESPACE thingworx OWNER twadmin LOCATION 'C:\ThingWorx\ThingworxPostgresqlStorage';" psql -U postgres -c "GRANT ALL PRIVILEGES ON TABLESPACE thingworx TO twadmin;" Finally the database itself can be restored by using the following commandline:   psql -U postgres < thingworxDump.sql This will create the database and populate it with tables, functions and sequences and will also restore the data from the .sql file.   It is important to have database, username and tablespace match with the original system - otherwise granting permissions and re-creating data might fail on recovery. User and tablespace can also be reused, so only the database has to be deleted before restoring it.   Part of the strategy   Part of the backup and recovery strategy should be to actually test the backup as well as the recovery part of the procedure. It should be well tested on a test-environment before deployed to any production environments. Take backups on a regular basis and test for disaster recovery once or twice a year, to ensure the procedure is still valid.   Data is the most important source in your application - protect it!  

The purpose of this document is to see how you can setup an MXChip IoT DevKit and also how send the readings of this microprocessor to ThingWorx through an Azure cloud server. You will also learn how to view the values that are being sent.

Help Center link on how to control file transfers from the edge client using the EdgeControlled ThingShape.   The EdgeControlled ThingShape is a default entity included with ThingWorx that allows you to manage the amount of egress being sent from the platform to the Edge.   At the time of writing this post, the available 'When Disconnected' settings for a remotely bound property in ThingWorx are 'Fold' and 'Ignore'. Setting a property to 'Fold' while using this EdgeControlled ThingShape is necessary whether the device is connected all the time or only for brief updates.   To use this ThingShape in a real world scenario you might code your edge client to invoke the DequeueEgress REST API function available through this ThingShape. The parameter you pass in is then the number of messages you would like the client to receive. The result of this function is how many messages the platform then actually sent.   A quick setup: 1. Create a RemoteThing entity in ThingWorx 2. Create an ApplicationKey entity in ThingWorx 3. Setup an edge client to bind to that RemoteThing using the specified ApplicationKey 4. Manage Bindings on the Properties page of the RemoteThing, and pull in a few properties you would like to send property updates to 5. Set the 'When Disconnected' value to 'Fold' for each property you want to queue messages for   5a. Set any other settings on the properties you'd like; ie. persistence, logged 6. Save the Thing 7. Add the EdgeControlled ThingShape to the Thing 8. Save the Thing 9. Update property values, see exceptions thrown, but the value will be queued 10. Invoke DequeueEgress on the RemoteThing, with the number of messages to send to the edge client passed in as the parameter value   10a. Notice 'Fold' means only the last value set for a property will be sent to the edge client. There is currently no retention available for any values previously set to the property and stored as the message to be sent. Those values are lost upon a new value coming in before it's dequeued. 11. Verify the edge client has received the expected egress, and the return result of the DequeueEgress function was the expected # of messages sent.

Disclaimer: This post does of course not express any political views.   Pie Chart Coloring   In ThingWorx Pie Charts use a default color schema based on the DefaultChartStyle Definitions. These schemas are using fixed numbering and coloring systems, e.g. 1 is blue, 2 is green, 3 is red and so on. All Pie Charts will be rendered with these colors in the same order, no matter which data the chart is using. Visualization of data with the default colors might not necessarily help in creating an easy to read chart.   Just take a look at the following example with the default color schema. Let's just take political parties - as they are usually associated with a distinct color - to illustrate how the default color schema will fail depending on the data displayed.         In the first example, just by sheer coincidence the colors are perfectly matching the parties. When introducing a new party to the pool suddenly the blues are rendered green and the yellows rendered light-blue etc. This can be quite confusing, especially on election night 😉   Custom Color Schema   PoliticalParties Thing   To test a custom color schema, we first need to create a new Thing: PoliticalParties as a GenericThing Add a dataset property with the following PoliticalParties DataShape.         Save the Thing and set the InfoTable to:   Key Value The Purples 20 The Blues 20 The Greens 20 The Reds 20 The Yellows 20 Others 20   Number values don't actually matter too much, as the Pie Chart will automatically distribute them according to their percentage.   PoliticalParties Mashup   Create a new Mashup and add a PieChart to the canvas. Bind the PoliticalParties > GetPropertyValues > dataset to the Data input of the Widget. Ensure to set the LabelField to key and the ValueField to value for a correct mapping.     Save the Mashup and preview it.   It should show a non-matching color for each party listed in the InfoTable.   Custom Styles and States   Create new custom Style Definitions for each political party. As the Pie Chart is only using the Background Color other properties can stay on the default. I chose to go with a more muted version of the colors to make the chart easier to look at.         With the newly defined colors we can now generate a new State Definition as follows:       The States allow to evaluate the key-Strings in the Thing's InfoTable and assign a Style Definition depending on the actual value. In this definition we map a color schema based on the InfoTable's key-value to create a 1:1 mapping for the Strings.   This means, no matter where a certain party is positioned in the chart it will be tinted with its associated color.   Refining the Mashup   Back in the Mashup, select the PieChart. In the ColorFormat property choose the newly created State Definition.     Save the Mashup and preview it. With the States and Styles applies, colors are now displayed correctly.       Even when changing positions and numbers in the original InfoTable of the PoliticalParties Thing, the chart now considers the mapping of Strings and still displays the colors correctly.  

This is a follow-up post on my initial document about Edge Microserver (EMS) and Lua Script Resource (LSR) security. While the first part deals with fundamentals on secure configurations, this second part will give some more practical tips and tricks on how to implement these security measurements.   For more information it's also recommended to read through the Setting Up Secure Communications for WS EMS and LSR chapter in the ThingWorx Help Center. See also Trust & Encryption Theory and Hands On for more information and examples - especially around the concept of the Chain of Trust, which will be an important factor for this post as well.   In this post I will only reference the High Security options for both, the EMS and the LSR. Note that all commands and directories are Linux based - Windows equivalents might slightly differ.   Note - some of the configuration options are color coded for easy recognition: LSR resources / EMS resources   Password Encryption   It's recommended to encrypt all passwords and keys, so that they are not stored as cleartext in the config.lua / config.json files.   And of course it's also recommended, to use a more meaningful password than what I use as an example - which also means: do not use any password I mentioned here for your systems, they might too easy to guess now 🙂   The luaScriptResource script can be used for encryption:   ./luaScriptResource -encrypt "pword123" ############ Encrypted String AES:A26fBYKHJq+eMu0Fm2FlDw== ############   The wsems script can be used for encryption:   ./wsems -encrypt "pword123" ############ Encrypted String AES:A26fBYKHJq+eMu0Fm2FlDw== ############   Note that the encryption for both scripts will result in the same encrypted string. This means, either the wsems or luaScriptResource scripts can be used to retrieve the same results.   The string to encrypt can be provided with or without quotation marks. It is however recommended to quote the string, especially when the string contains blanks or spaces. Otherwise unexpected results might occur as blanks will be considered as delimiter symbols.   LSR Configuration   In the config.lua there are two sections to be configured:   scripts.script_resource which deals with the configuration of the LSR itself scripts.rap which deals with the connection to the EMS   HTTP Server Authentication   HTTP Server Authentication will require a username and password for accessing the LSR REST API.     scripts.script_resource_authenticate = true scripts.script_resource_userid = "luauser" scripts.script_resource_password = "pword123"     The password should be encrypted (see above) and the configuration should then be updated to   scripts.script_resource_password = "AES:A26fBYKHJq+eMu0Fm2FlDw=="   HTTP Server TLS Configuration   Configuration   HTTP Server TLS configuration will enable TLS and https for secure and encrypted communication channels from and to the LSR. To enable TLS and https, the following configuration is required:     scripts.script_resource_ssl = true scripts.script_resource_certificate_chain = "/pathToLSR/lsrcertificate.pem" scripts.script_resource_private_key = "/pathToLSR/key.pem" scripts.script_resource_passphrase = "keyForLSR"     It's also encouraged to not use the default certificate, but custom certificates instead. To explicitly set this, the following configuration can be added:     scripts.script_resource_use_default_certificate = false     Certificates, keys and encryption   The passphrase for the private key should be encrypted (see above) and the configuration should then be updated to     scripts.script_resource_passphrase = "AES:A+Uv/xvRWENWUzourErTZQ=="     The private_key should be available as .pem file and starts and ends with the following lines:     -----BEGIN ENCRYPTED PRIVATE KEY----- -----END ENCRYPTED PRIVATE KEY-----     As it's highly recommended to encrypt the private_key, the LSR needs to know the password for how to encrypt and use the key. This is done via the passphrase configuration. Naturally the passphrase should be encrypted in the config.lua to not allow spoofing the actual cleartext passphrase.   The certificate_chain holds the Chain of Trust of the LSR Server Certificate in a .pem file. It holds multiple entries for the the Root, Intermediate and Server Specific certificate starting and ending with the following line for each individual certificate and Certificate Authority (CA):     -----BEGIN CERTIFICATE----- -----END CERTIFICATE-----     After configuring TLS and https, the LSR REST API has to be called via https://lsrserver:8001 (instead of http).   Connection to the EMS   Authentication   To secure the connection to the EMS, the LSR must know the certificates and authentication details for the EMS:     scripts.rap_server_authenticate = true scripts.rap_userid = "emsuser" scripts.rap_password = "AES:A26fBYKHJq+eMu0Fm2FlDw=="     Supply the authentication credentials as defined in the EMS's config.json - as for any other configuration the password can be used in cleartext or encrypted. It's recommended to encrypt it here as well.   HTTPS and TLS   Use the following configuration establish the https connection and using certificates     scripts.rap_ssl = true scripts.rap_cert_file = "/pathToLSR/emscertificate.pem" scripts.rap_deny_selfsigned = true scripts.rap_validate = true     This forces the certificate to be validated and also denies selfsigned certificates. In case selfsigned certificates are used, you might want to adjust above values.   The cert_file is the full Chain of Trust as configured in the EMS' config.json http_server.certificate options. It needs to match exactly, so that the LSR can actually verify and trust the connections from and to the EMS.   EMS Configuration   In the config.lua there are two sections to be configured:   http_server which enables the HTTP Server capabilities for the EMS certificates which holds all certificates that the EMS must verify in order to communicate with other servers (ThingWorx Platform, LSR)   HTTP Server Authentication and TLS Configuration   HTTP Server Authentication will require a username and password for accessing the EMS REST API. HTTP Server TLS configuration will enable TLS and https for secure and encrypted communication channels from and to the EMS.   To enable both the following configuration can be used:   "http_server": { "host": "<emsHostName>", "port": 8000, "ssl": true, "certificate": "/pathToEMS/emscertificate.pem", "private_key": "/pathToEMS/key.pem", "passphrase": "keyForEMS", "authenticate": true, "user": "emsuser", "password": "pword123" }   The passphrase as well as the password should be encrypted (see above) and the configuration should then be updated to   "passphrase": "AES:D6sgxAEwWWdD5ZCcDwq4eg==", "password": "AES:A26fBYKHJq+eMu0Fm2FlDw=="   See LSR configuration for comments on the certificate and the private_key. The same principals apply here. Note that the certificate must hold the full Chain of Trust in a .pem file for the server hosting the EMS.   After configuring TLS and https, the EMS REST API has to be called via https://emsserver:8000 (instead of http).   Certificates Configuration   The certificates configuration hold all certificates that the EMS will need to validate. If ThingWorx is configured for HTTPS and the ws_connection.encryption is set to "ssl" the Chain of Trust for the ThingWorx Platform Server Certificate must be present in the .pem file. If the LSR is configured for HTTPS the Chain of Trust for the LSR Server Certificate must be present in the .pem file.   "certificates": { "validate": true, "allow_self_signed": false, "cert_chain" : "/pathToEMS/listOfCertificates.pem" } The listOfCertificates.pem is basicially a copy of the lsrcertificate.pem with the added ThingWorx certificates and CAs.   Note that all certificates to be validated as well as their full Chain of Trust must be present in this one .pem file. Multiple files cannot be configured.   Binding to the LSR   When binding to the LSR via the auto_bind configuration, the following settings must be configured:   "auto_bind": [{ "name": "<ThingName>", "host": "<lsrHostName>", "port": 8001, "protocol": "https", "user": "luauser", "password": "AES:A26fBYKHJq+eMu0Fm2FlDw==" }]   This will ensure that the EMS connects to the LSR via https and proper authentication.   Tips   Do not use quotation marks (") as part of the strings to be encrypted. This could result in unexpected behavior when running the encryption script. Do not use a semicolon (:) as part of any username. Authentication tokens are passed from browsers as "username:password" and a semicolon in a username could result in unexpected authentication behavior leading to failed authentication requests. In the Server Specific certificates, the CN must match the actual server name and also must match the name of the http_server.host (EMS) or script_resource_host (LSR) In the .pem files first store Server Specific certificates, then all required Intermediate CAs and finally all required Root CAs - any other order could affect the consistency of the files and the certificate might not be fully readable by the scripts and processes. If the EMS is configured with certifcates, the LSR must connect via a secure channel as well and needs to be configured to do so. If the LSR is configured with certifcates, the EMS must connect via a secure channel as well and needs to be configured to do so. For testing REST API calls with resources that require encryptions and authentcation, see also How to run REST API calls with Postman on the Edge Microserver (EMS) and Lua Script Resource (LSR)   Export PEM data from KeyStore Explorer   To generate a .pem file I usually use the KeyStore Explorer for Windows - in which I have created my certificates and manage my keystores. In the keystore, select a certificate and view its details Each certificate and CA in the chain can be viewed: Root, Intermediate and Server Specific Select each certificate and CA and use the "PEM" button on the bottom of the interface to view the actual PEM content Copy to clipboard and paste into .pem file To generate a .pem file for the private key, Right-click the certificate > Export > Export Private Key Choose "PKCS #8" Check "Encrypted" and use the default algorithm; define an "Encryption Password"; check the "PEM" checkbox and export it as .pkcs8 file The .pkcs8 file can then be renamed and used as .pem file The password set during the export process will be the scripts.script_resource_passphrase (LSR) or the http_server.passphrase (EMS) After generating the .pem files I copy them over to my Linux systems where they will need 644 permissions (-rw-r--r--)

This is going to cover one way of configuring an SSL passthrough using HAProxy.  This guide is intended to be a reference document, and administrators looking to configure an SSL passthrough should make sure the end solution meets both their company's business and security needs.   Why use SSL Passthrough instead of SSL Termination? The main reason for ThingWorx would be if a company requires encrypted communication internally, as well as externally.  With SSL Termination, the request between the load balancer and the client is encrypted.  But the load balancer takes on the role to decrypt and passes that back to the server.  With SSL Passthrough, the request goes through the load balancer as is, and the decryption happens on the ThingWorx Application server.   What you will need to continue with this guide:   HAProxy installed A working ThingWorx application server (Guide to getting one setup can be found here) Tomcat configured for ssl NOTE : Always contact your Security team and make sure you have a certificate that meets your business policy For this tutorial, I created a self-signed certificate following along with the below guide.  If you have already obtained a valid certificate, then you can just skip over the step of creating it, and follow along with the Tomcat portion https://www.ptc.com/en/support/article?n=CS193947 Once configured, restart Tomcat and verify it is working by navigating to https://<yourServer>:<port>/Thingworx   With ThingWorx running as SSL and HAProxy installed, we just need to make sure the HAProxy configuration is setup to allow SSL traffic through.  We use 'mode tcp' to accomplish this.   On your HAProxy machine, open /etc/haproxy/haproxy.cfg for editing.  While most of this can be customized to fit your business needs, some variation of the highlighted portions below need to be included in your final configuration:   global         log /dev/log    local0         log /dev/log    local1 notice         chroot /var/lib/haproxy         stats socket /run/haproxy/admin.sock mode 660 level admin         stats timeout 30s         user haproxy         group haproxy         daemon           # Default SSL material locations         ca-base /etc/ssl/certs         crt-base /etc/ssl/private           # Default ciphers to use on SSL-enabled listening sockets.         # For more information, see ciphers(1SSL). This list is from:         #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/         # An alternative list with additional directives can be obtained from         #  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy         ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS         ssl-default-bind-options no-sslv3   defaults         log global          option tcplog          mode tcp          option http-server-close          timeout connect 1s          timeout client  20s          timeout server  20s          timeout client-fin 20s          timeout tunnel 1h          errorfile 400 /etc/haproxy/errors/400.http          errorfile 403 /etc/haproxy/errors/403.http          errorfile 408 /etc/haproxy/errors/408.http          errorfile 500 /etc/haproxy/errors/500.http          errorfile 502 /etc/haproxy/errors/502.http          errorfile 503 /etc/haproxy/errors/503.http          errorfile 504 /etc/haproxy/errors/504.http         frontend https          bind *:443          mode tcp          default_backend bk_app           backend bk_app          mode tcp          server TWXAPP01  <twxapp01IP>:<port>         In this example, the user would connect to https://<loadbalancer>/Thingworx and the load balancer would forward the requests to https://<twxapp01IP>:<port>/Thingworx   That’s it!   A couple of side notes:   The load balancer port the clients connect to does not need to be the same as the ThingWorx port the load balancer will forward to If working in a Highly Available configuration, each ThingWorx Application server needs to have its own certificate configured If HAProxy seems unstable, try updating to the latest release If it is on the latest release according to the Unix repository, check https://www.haproxy.org/ and see if there is a later stable release.  There have been some issues where Ubuntu's latest update in the repository is actually a few years old

Overview This document is targeted towards covering basic PostgreSQL monitoring and health check related system objects like tables, views, etc. This allows simple monitoring of PostgreSQL database via some custom services, which I'll attach at the end of this document, from the ThingWorx Composer itself. I'll also try to cover short detail on some of the services that are included with the Thing: PostgreSQLHealthCheck which implements Database ThingTemplate   Pre-requisite The document assumes that the user already has ThingWorx running with PostgreSQL as a Persistence Provider.   How to install Usage for this is fairly straight forward, import the Entities.twx and it will create required Thing which implements Database ThingTemplate and some DataShapes. Each Service under the Thing: PostgreSQLHealthCheck has its own DataShape. Feel free edit these services / DataShapes if you are looking to use output of these services  as part of your mashup(s).   Make sure to edit the PostgeSQL's JDBC Connection String, Username & password under the configuration section in order to connect to your PostgreSQL instance under the Thing PostgreSQLHealthCheck which will be created when Entities.twx is imported (attached with this blog)   Note : Users can use these services to query non-ThingWorx related database created with PostgreSQL as part of the external JDBC connection.   Reviewing Services from Thing: PostgreSQLHealthCheck   1. DescribeTableStructure - Takes two inputs **Table Name** and the **Schema Name** in which the ThingWorx database tables exists both inputs have default values that can be modified to match your PostgreSQL schema setup and required table name - It provides information on a Table's structure, see below     2. GetAllPSQLConfig - Provides runtime details on all the configurations done in the postgresql.conf which are in-effect - For detail on pg_settings see Postgresql 9.4 Doc     3. GetAllPSQLConfigLimited Similar to GetAllPSQLConfig, however with limited information   4. GetAllPSQLRoles - Lists all the database roles/users - Also lists their access rights permissions together with OID - Helpful in identifying if the role is active/inactive or carries any limitation on the DB connections     5. GetPG_Stat_Activity - Part of the Statistics Collector subsystem for the PostgreSQL DB - Shows current state of the schema e.g. connections, queries, etc. - For more detail on the output refer to the PostgreSQL 9.4 doc   6. GetPSQLDBLocksInformation - Shows the kind of locks in effect on which database and on which relation (table) - Particularly useful in identifying the relations and what lock mode is enabled on them     7. GetPSQLDBStat - Shows database wide statistics - Like Commits, reads, block reads, tupples (rows) fetched, inserted, deadlocks, etc - For more detail refer to PostgreSQL 9.4 doc   8. GetPSQLLogDesitnation - Checks where the PostgreSQL server logs are directed to - I.e. stderr, csvlog or syslog - Default is stderr   9. GetPSQLLogFileName - Fetches the log PostgreSQL log file name and the filename format - E.g. postgresql-%Y-%m-%d_%H%M%S.log    10. GetPSQLLoggingLocation - Fetches the location where the logs are stored for PostgreSQL - e.g. pg_log, which is also the default location - Desired location for the logs can be done in the postgresql.conf file   11. GetPSQLRelationIndexes - Gets information on the Indexes - Information like index size, number of rows, table names on which the index is created   12. GetPSQLReplicationStat - Shows information related to the Replication on PostgreSQL DB - Applicable to the PostgreSQL DBs where replication is enabled   13. GetPSQLTablespaceInfo - Takes tablespace name as input (String DataType), service defaults to 'thingworx' - modify if needed - Fetches information like owner oid, tablespace ACL     14. GetPSQLUserIndexIO - Fetches index that are created only on the User created DB objects - Shows relations (table) vs index relations ids (index on table), together with their names - Also shows additional info like number of disk blocks read from this index & number of buffer hits in this index     15. GetPSQLUserSequencesIOStats - Fetches informtion on Sequence objects used on user defined relations (tables) - Number of disk blocks read from this sequence & buffer hits in this sequence     16. GetPSQLUserTableIOStat - Fetches disk I/O information on the user created tables     17. GetPSQLUserTables - Fetches all the user created tables, together with their name, OID Disk I/O Last auto vacuum , vacuum Also lists the amount of rows each relation (table) has   Finally The attached entity has some additional service not yet covered in this blog, as they are minor services. Therefore for brevity of this blog I've left them out for now, feel free to explore or enhance this. I will continue to look for any additional services and will enhance this document and the entities belong to this.    If you are looking to enhance this feel free to fork from twxPostgreSqlHealthCheck over Github.