Check out the ThingWorx Container Deployment Hub!!

atiwari-4 · ‎Apr 20, 2022

Hello ThingWorx Community members!!
We heard you! We, from ThingWorx IoT Team, would like to inform you that we’re offering enablement and guidance on how to deploy, run, and manage ThingWorx Foundation on Azure Kubernetes Service.

We’d like you to check out the ThingWorx Container Deployment hub where we provide key information to enable you to deploy the ThingWorx Foundation single server or HA cluster on Azure Kubernetes Service (AKS). We also plan to add further details and Knowledge Base (KB) articles over the period to curate all the best practices around containerization and deployment of ThingWorx on Kubernetes in this space. Stay tuned!

Please note that the content offered on the KB articles is for community guidance and is not officially supported by PTC (Tech Support or R&D). For more information on what’s supported, please check out this FAQ KB article: https://www.ptc.com/en/support/article/CS367532

Conceptual diagram showing ThingWorx deployment on Azure Kubernetes Service (AKS)

Should you have any questions or feedback for us or would like to share your own version/insights on the containerized deployment of ThingWorx, please feel free to create a community post with the label “Containers” and we’ll try our best to respond. We’d like to encourage all our ThingWorx developer community members to share what you are doing with your containerization deployment to help us help you in your Day-1 and Day-2 DevOps journey!

Go TWXers!!

Regards,
Ayush Tiwari
ThingWorx Product Management

VladimirRosu · ‎Apr 21, 2022

Awesome content! Thank you @atiwari-4 and team!

Perfect way to start untangling your AKS story!

carloquinonez · ‎Apr 26, 2022

Why not go ALL THE WAY, and actually provide Docker containers?

I'm sure PTC already has a build pipeline for containers so why not just push them to a private container registery? At the minimum, do it for paying customers...

atiwari-4 · ‎Apr 28, 2022

Thanks for your feedback and question @carloquinonez
We do plan to provide Docker container images via the PTC hosted Docker Repo which is on our long-term roadmap. However, in the interim, we have realized that a lot of ThingWorx users have their container registry and like the flexibility to build ThingWorx docker images using our Dockerfiles. Please let us know if you see any challenges building your Docker Container images and hosting them in your internal IT’s repository for your development use cases?

TomDecock · ‎May 25, 2022

Hello Ayush @atiwari-4 ,

I agree that having the opportunity to build the docker images yourself is a good thing. The current build scripts and Dockerfiles provided by PTC work well and building the images yourself is not such a big struggle. After all, it's an opportunity to quicky update them when for example a new security update version is available from either OpenJDK, Tomcat, Thingworx or some other component in the toolchain. So I hope PTC keeps updating the Dockerfiles so they work with upcoming releases of Thingworx.

Regards,

Tom

carloquinonez · ‎Jun 27, 2022

Awesome news. when do you expect the PTC hosted Docker Repo to be ready?

TomDecock · ‎May 25, 2022

Hello Ayush,

The current package PTC has released for Thingworx Deployment in AKS is very helpful, though it still misses some documentation.

SSO configuration for Thingworx in AKS for example. Can some documentation be provided about that also ?

Regards,

Tom

slangley · ‎May 31, 2022

Hi @TomDecock

After speaking with our resident expert, I would like to pass along the recommendation for configuring SSO:

Configure the necessary sso folder content (i.e. ssoSecurityConfig) manually within the PVC (/ThingworxPlatform/ssoSecurityConfig)
Use the helm chart environment variable to set EnableSSO = true (along with the HTTP header/param size variables). Switch back to work through any debugging issues

The core SSO configuration for ThingWorx can be found here:

https://support.ptc.com/help/thingworx/platform/r9/en/index.html#page/ThingWorx/Help/Composer/Securi...

Regards.

--Sharon

TomDecock · ‎Jun 22, 2022

Hi @slangley , that did the job, thank you.

One thing though: when enabling SSO thru the values file for the helm chart, it not only sets the environment variable to enable it in the Platform Settings, it also automatically includes an additional sso-init-container in the pods for the Thingworx platform statefulset. As there are not yet init container images available for sso yet, and as it is not needed if we just mount a predefined ssoSecurityConfig folder from an Azure File Share in the PlatformSettings folder, there should be a way to disable that init container without forking and modifying the Helm chart for thingworx-server. What I did now was just supplying a 'fake' init container image that just touches a file in the ssoSecurityConfig folder, but actually does nothing else. How to deal with that ?

Tom.

TomDecock · ‎Jun 10, 2022

After getting this entire stack running in AKS, I would also like to integrate PingFederate running in clustered mode in our AKS cluster, I deployed PingFederate as described on https://devops.pingidentity.com using the helm chart from https://helm.pingidentity.com .

Everything seems to be deployed correctly, with the pingfederate-admin and pingfederate-engine statefulsets, persistent storage, services & ingress, all pods running...

But I cannot connect to the admin console of pingfederate. The server.log of PingFederate shows the following:

2022-06-10 07:15:39,536 INFO [org.eclipse.jetty.server.Server] Started @153587ms

2022-06-10 07:15:39,541 INFO [com.pingidentity.appserver.jetty.PingFederateInit] PingFederate started in 138s:712ms

2022-06-10 07:15:47,547 WARN [org.eclipse.jetty.server.handler.ContextHandler.pingfederate] unavailable

org.apache.jasper.JasperException: Unable to set last modified date for file [/opt/out/instance/work/jetty-0_0_0_0-9999-pingfederate_war-_pingfederate-any-1628531342419048495/jsp/org/apache/jsp/WEB_002dINF/pages/main_jsp.java]

2022-06-10 07:15:47,549 WARN [org.eclipse.jetty.server.HttpChannel] /pingfederate/app

javax.servlet.ServletException: org.apache.jasper.JasperException: Unable to set last modified date for file [/opt/out/instance/work/jetty-0_0_0_0-9999-pingfederate_war-_pingfederate-any-1628531342419048495/jsp/org/apache/jsp/WEB_002dINF/pages/main_jsp.java]

When I port-forward the service for the admin console and try to connect to https://<admin-server-host>:9999/pingfederate/app in the browser I get:

HTTP ERROR 404 Not Found
URI: /pingfederate/app
STATUS: 404
MESSAGE: Not Found
SERVLET: ReactServlet

Anyone experience with this ?

geva · ‎Nov 03, 2022

Hi @TomDecock

I'd suggest opening a shell to the various pods in question and testing and checking network connectivity. Are the needed and expected ports open and responding on all endpoints? Are you able to connect to and from one to the other? (You can do this with kubectl exec -it <podname> -- /bin/sh). You may need to install the needed tools inside the pod if they're not there (netstat is in net-tools, curl, nc). Otherwise try out k9s it is very helpful and easy to use.

TomDecock · ‎Jun 10, 2022

Hi @atiwari-4 @slangley @VladimirRosu ,

I found an issue when re-deploying a deployment of statefulset for thingworx-platform with an existing persistent volume claim.

In particular, the file /ThingworxPlatform/logback.xml blocks the Thingworx webapp from starting again (this can be seen in the output logs of the pod/container), because it cannot be modified.

After mounting the azure file share as a network drive on my local machine, I saw in the properties that the file had a read-only status. Clearing the read-only attribute made Thingworx start (since the deployment continues to try to start the pods in the kubernetes cluster).

After investigation, I found in the docker-entrypoint.sh script in the imageFiles for the thingworx platform docker image, that this file is deliberately made read-only:

# TW-94100 Prevent runtime changes to logging configuration file
chmod a-w "${THINGWORX_PLATFORM_SETTINGS}/logback.xml

But this status on the existing file in the persistent volume causes a new deployment to fail.

Can someone tell why this file needs to be read-only ?

Tom

cdsouza · ‎Jun 13, 2022

There is no reason to persist /ThingworxPlatform contents (sso config and license file being exceptions) to a persistent volume. With the exceptions noted, the rest are intended to be ephemeral and hence read only.

TomDecock · ‎Jun 13, 2022

Hi @cdsouza, I wanted to mount /ThingworxPlatform as a persistent volume so I could add the SSO config directory in it. (For the license file it is no problem as it can be added as a secret) But even before adding SSO config, I notice that the readonly status of the logback.xml file makes that the statefulset cannot be redeployed with this existing volume claim. The entrypoint script fails when trying to merge xml files into the logback.xml file. Or am I missing something else here ? What would be the best practice then to add SSO config if not doing it with a volume ?

Regards,

Tom

cdsouza · ‎Jun 13, 2022

You can mount just the ssoSecurityConfig as follows:

volumeMounts:
  - mountPath: /ThingworxPlatform/ssoSecurityConfig
    subPath: ThingworxPlatform/ssoSecurityConfig
    name: storage
.
.
.
volumes:
  - name: storage
    persistentVolumeClaim:
      claimName: your_pvc_claim

TomDecock · ‎Jun 13, 2022

Hi @cdsouza, that makes sense - I did not think of mounting only the subpath and not the ThingworxPlatform folder itself. I'll give it a try

Thanks for the tip!

TomDecock · ‎Jun 22, 2022

Hi Ayush @atiwari-4 , be aware that some helm chart versions used in this deployment have limited availability. For example, the chart version used for the zookeepers by Bitnami is no longer available in their repository. As is the corresponding image tag used in the example. Personally I did not yet pull that helm chart to our private repository but now I assume it is best to do it. But for people who only start now with the helmsman config provided by PTC, some helm charts used will not be available from the beginning, and so you are not sure if the provided values files will still work with newer versions of the chart.

Shouldn't there be some listing of what image tags and what helm chart versions are compatible ?

Regards,

Tom

DeShengXu · ‎Jun 22, 2022

Hi, Tom:

Thank you for the question. The fix will be included in a later release. you can fix it now as:

1) go to common.yaml

2) find `helmRepos` section

3) replace the URL for batnami with a new value:

```

#bitnami: 'https://charts.bitnami.com/bitnami'

bitnami: 'https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami'

```

hope this helps.

Desheng

TomDecock · ‎Jun 28, 2022

Hello Guys,

Maybe an idea for the developers of the helm packages: currently the provisioning chart that is included can only create configmaps with text files (.data) but not with binary files (.binaryData). Would be good to change the configmaps.yaml in the provisioning chart to also allow for creation of binaryData

{{- range .Values.configMaps }}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .name }}
  labels: {{- include "provisioning.labels" $ | nindent 4 }}
{{- with .annotations }}
  annotations:
{{ toYaml . | indent 4 }}
{{- end }}
{{- if .data }}
data:
{{- with .data }}
{{- toYaml . | nindent 2 }}
{{- end }}
{{- end }}
{{- if .binaryData }}
binaryData:
{{- with .binaryData }}
{{- toYaml . | nindent 2 }}
{{- end }}
{{- end }}
{{- end }}

Grtz,

Tom

TomDecock · ‎May 12, 2023

Hello @atiwari-4 @geva , are there any plans to provide Dockerfiles and Kubernetes deployment scripts/charts to integrate the Thingworx Azure IoT Connector 4 in this stack ?