I am using ThingWorx Platform Release 9.5 and DatecodeSP3
Dear Team we have Thingworx cluster installed in active-active passion on Server. Both server were down. Below error found in Application Log.
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] ChangeWatcher failed to sync
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] Timeout. Cannot synchronize model during 120000 ms
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] Shutting down server..
2024-12-25 18:16:39.696+0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] >>>>>>> PLATFORM SHUTDOWN START <<<<<<<<<
2024-12-25 18:16:39.181+0400 [L: INFO] [O: o.a.i.i.m.d.GridDiscoveryManager] [I: ] [U: ] [S: ] [P: platform1] [T: disco-event-worker-#68%twx-core-server%] Node left topology: TcpDiscoveryNode [id=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, consistentId=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, addrs=ArrayList [10.0.187.40, 127.0.0.1, 192.168.
Here are the errors that I faced
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] ChangeWatcher failed to sync
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] Timeout. Cannot synchronize model during 120000 ms
2024-12-25 18:16:39.598+0400 [L: ERROR] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] Shutting down server..
2024-12-25 18:16:39.696+0400 [L: INFO] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: platform2] [T: pool-8-thread-1] >>>>>>> PLATFORM SHUTDOWN START <<<<<<<<<
2024-12-25 18:16:39.181+0400 [L: INFO] [O: o.a.i.i.m.d.GridDiscoveryManager] [I: ] [U: ] [S: ] [P: platform1] [T: disco-event-worker-#68%twx-core-server%] Node left topology: TcpDiscoveryNode [id=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, consistentId=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, addrs=ArrayList [10.0.187.40, 127.0.0.1, 192.168.8.138], sockAddrs=null, discPort=0, order=22, intOrder=13, lastExchangeTime=1734956211185, loc=false, ver=2.15.0#20230425-sha1:f98f7f35, isClient=true]
2024-12-25 18:16:39.182+0400 [L: INFO] [O: o.a.i.i.m.d.GridDiscoveryManager] [I: ] [U: ] [S: ] [P: platform1] [T: disco-event-worker-#68%twx-core-server%] Topology snapshot [ver=25, locNode=25015be0, servers=2, clients=1, state=ACTIVE, CPUs=32, offheap=31.0GB, heap=51.0GB, aliveNodes=[TcpDiscoveryNode [id=25015be0-fed0-4683-a4a0-79eb07651073, consistentId=25015be0-fed0-4683-a4a0-79eb07651073, isClient=true, ver=2.15.0#20230425-sha1:f98f7f35], TcpDiscoveryNode [id=c606d9dc-88ac-4f47-908b-ea7a0aaf6821, consistentId=0:0:0:0:0:0:0:1%lo,10.0.187.39,127.0.0.1,192.168.8.137:47500, isClient=false, ver=2.15.0#20230425-sha1:f98f7f35], TcpDiscoveryNode [id=4352f61a-c189-4874-9ff2-57ce5e30b59f, consistentId=0:0:0:0:0:0:0:1%lo,10.0.187.40,127.0.0.1,192.168.8.138:47500, isClient=false, ver=2.15.0#20230425-sha1:f98f7f35]]]
2024-12-25 18:16:39.182+0400 [L: INFO] [O: o.a.i.i.m.d.GridDiscoveryManager] [I: ] [U: ] [S: ] [P: platform1] [T: disco-event-worker-#68%twx-core-server%] ^-- Baseline [id=0, size=2, online=2, offline=0]
2024-12-25 18:16:39.191+0400 [L: INFO] [O: o.a.i.i.e.time] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Started exchange init [topVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], crd=false, evt=NODE_LEFT, evtNode=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, customEvt=null, allowMerge=true, exchangeFreeSwitch=false]
2024-12-25 18:16:39.194+0400 [L: INFO] [O: o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Finish exchange future [startVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], err=null, rebalanced=true, wasRebalanced=true]
2024-12-25 18:16:39.198+0400 [L: INFO] [O: o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Completed partition exchange [localNode=25015be0-fed0-4683-a4a0-79eb07651073, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], evt=NODE_LEFT, evtNode=TcpDiscoveryNode [id=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, consistentId=a1a629b9-9f31-450f-b5b4-2a33ded24e1f, addrs=ArrayList [10.0.187.40, 127.0.0.1, 192.168.8.138], sockAddrs=null, discPort=0, order=22, intOrder=13, lastExchangeTime=1734956211185, loc=false, ver=2.15.0#20230425-sha1:f98f7f35, isClient=true], rebalanced=true, done=true, newCrdFut=null], topVer=AffinityTopologyVersion [topVer=25, minorTopVer=0]]
2024-12-25 18:16:39.198+0400 [L: INFO] [O: o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Exchange timings [startVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], stage="Waiting in exchange queue" (0 ms), stage="Exchange parameters initialization" (0 ms), stage="Determine exchange type" (2 ms), stage="Exchange done" (4 ms), stage="Total time" (6 ms)]
2024-12-25 18:16:39.198+0400 [L: INFO] [O: o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Exchange longest local stages [startVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=25, minorTopVer=0]]
2024-12-25 18:16:39.198+0400 [L: INFO] [O: o.a.i.i.e.time] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] Finished exchange init [topVer=AffinityTopologyVersion [topVer=25, minorTopVer=0], crd=false]
2024-12-25 18:16:39.200+0400 [L: INFO] [O: o.a.i.i.p.c.GridCachePartitionExchangeManager] [I: ] [U: ] [S: ] [P: platform1] [T: exchange-worker-#69%twx-core-server%] AffinityTopologyVersion [topVer=25, minorTopVer=0], evt=NOD
