Skip to main content
avillanueva
23-Emerald I
23-Emerald I
June 4, 2025
Solved

SOLR crashing during Bulk Indexing

  • June 4, 2025
  • 2 replies
  • 1084 views

I've completed a rehearsal upgrade to 13.0.2.4 and kicked off a full re-index. In the past, this has been a low risk task that churned away for hours until completion. I've index 850K objects out of 1M but was fighting with Solr to keep running. Rate initially was 2200/min which I guess is good. I have not changed any of the default memory settings. System has 8GB of ram and plenty of HD space. I did not see any areas filling up. I checked the obvious logs but could not find any glaring error messages when it shutdown. I just would notice that I would see failed jobs in the Index Administrator and when I checked, the solr process had ended. I restarted it and resumed but would stop for some reason maybe 1/2 hour later. 

When its running, I am able to search and can see Classification Search so I know its functioning. I did see a curious issue with text preview on documents where it showed only name and number. (see https://www.ptc.com/en/support/article/CS249768?source=search) which indicates to reindex again. No problem since this is a rehearsal but still strange. 

 

Looking to see what I should be looking for in the logs as an indicator to why its stopping. I suspect memory issues but did not see any out of memory messages as I would expect. RHEL 9.2 SELinux. Noting issues from previous where I was not able to run this vis systemctl as root (running as another user). I had to start manually as that user from shell each time. 

Best answer by avillanueva

Closing out thread since it seems to have stabilized and completed Bulk Indexing. No reason why it stopped misbehaving. 

2 replies

20-Turquoise
June 4, 2025

I change the SOLR_HEAP in SolrServer/bin/set_env.sh to 8192m in our environment.

However when I upgraded from 12.1.2.13 to 13.0.2.2 I reused the indexed data.

avillanueva
23-Emerald I
23-Emerald I
June 4, 2025

Well, now it decided to show text preview. Go figure. I made no changes. Running now to complete rest. Let's see if we can make it to the end. 

avillanueva
23-Emerald I
avillanueva23-Emerald IAuthorAnswer
23-Emerald I
June 4, 2025

Closing out thread since it seems to have stabilized and completed Bulk Indexing. No reason why it stopped misbehaving. 

jbailey
18-Opal
June 5, 2025

I opened a case on this while still at NASA. It seemed to be caused only on bulk indexing. The process wouldn't clear the temp files and it would crash the service. Additionally, depending on the size of indexing, the temp files would fill up the drive where your index resides. If you started the service back up, it would properly clear the temp files and resume.

 

PTC Tech said it wasn't reproducible, however it happened in every bulk indexing we did.