Hi all,
I have two Thingworx servers in an HA configuration. Recently both servers were taken down for maintenance. After maintenance, one server was started. During the startup, the server took HA leadership and began initializing like normal, but abruptly failed to initialize and shut down the system. The HA leadership was transferred to the passive server, which booted up without an issue. I am trying to understand why the initial startup failed.
The application logs show that the initialization ran up to the step "Initializing Reference Caches", at which point a ConcurrentModificationException was thrown by the Curator-LeaderSelector thread. Based on what I've been able to find, that appears to indicate there were multiple threads which attempted to overwrite the same object. I'm at a loss as to why this might have occured during initialization.
The only other error which was thrown at the time of the exception was an InvalidRequestException due to the Thing [AlwaysOnReporting] not running, but that has always been resolved once initialization is complete, so I doubt it's related to this error.
Has anyone experienced a similar initialization failure and could shed some light?
I've included the stack trace from the exception and the application log of the server startup.
