Troubleshooting platform issues is generally done by using a layer approach, similar to a simplified OSI Model. From bottom to top, the following layers represent the areas to analyze during each step:
1. Physical (Server, power, wired connections): check the server status and condition, CPU and memory levels.
2. Software (Operating system, tomcat, java versions, compatibility, and configuration): refer to the compatibility matrix to ensure the requirements are met; verify Tomcat java configuration. * Note: Tomcat manager, server status, conveniently provides this information in one place.
3. Network: ensure proper connectivity, port availability, firewall configuration, and additional security, if applicable.
4. Application.
The main focus of this blog post will concentrate on the step 4. As the Thingworx application is driven by Tomcat, first available tools coming "out-of-the-box" is the built-in Tomcat manager app. Clicking on the "Server Status" provides the information on the versions, memory usage, processes, times and thread counts. Keep in mind, the default Tomcat maximum thread number is 200.
Some additional tools that could assist in troubleshooting java applications and gathering performance metrics are: Javamelody, new relic, profiler4j. These have to be obtained, installed, and configured separately.
Javamelody: Free and lightweight monitoring tool which does not do any profiling, safe to use in production environments. It comes with a series of plug-ins including for Grails, Jenkins and Jira.
New relic: Real-time Java application monitoring, features code deployment reports, transaction tracing across different tiers and the ability to create alerts. Subscription fee applies.
Profiler4j: Profiler4J is a free open-source tool for profiling in Java. It is enabled by passing an argument at start-up with a path to the Profiler4J .jar file. It comes with several graphs and charts showing a call graph with method details, a call tree, a memory monitor, a class list and thread monitoring.
From the application perspective, Thingworx composer provides a PlatformSubsystem and LoggingSubsystem:
- PlatformSubsystem contains such services as GetPerformanceMetrics, GetSummaryInformation, GetThingworxVersion, and more to provide fundamental information for any troubleshooting scenario.
- LoggingSubsystem contains the logs, log settings, and other monitoring values.
List of recommended tools for troubleshooting all layers:
- Wireshark: monitors network traffic
- Jstack: monitors memory consumption of specific threads
- Dynatrace: system performance and web application performance
- jconsole: system or application performance
