How to configure Neo4j logging settings in ThingWorx?

[chrish]

My ThingWorx server is growing Neo4j log files almost exponentially. In normal

installs

I can change configuration and control log retention, etc in neo4j.properties and cneo4j-wrapper.conf but neither appears to be available in ThingWorx installation.  Where is this configured?

[Aanjan]

Christopher, unfortunately the Neo4j log settings are not exposed in the platform. The only logging settings you can control per se, is the ThingWorx logging itself - Subsystems -> LoggingSubsystem has settings for max log size and retention time.

[chrish]

Ok. I went into Subsystems -> LoggingSubsystem and my install is configured max log size of 100000KB (100MB) according to Composer. This is the default.  Why then is my Neo logs growing to over 100GB in the ThingworxStorage/database/data directory??  How can I prevent Neo logs from generating 100's of GB of log files and overwhelming the storage on the server?

[Aanjan]

Christopher, that's the ThingWorx logs - if you go into ThingworxStorage\logs, 100mb would be the maximum size of a log file before it gets pushed into archive. Regarding the neo logs, those are the nioneo_logical logs, right? Do you have anything writing values into streams very frequently?

[chrish]

Yes. Data Tables and Streams. Both are stored in Neo. This is the basis of my original question.

[vranganathan]

I have same issue. I tried the purging of my streams on nightly basis. Still in ThingworxBackupStorage I have lot of files nioneo_logical.log.* Is it safe to delete them?

[Aanjan]

Varathan, although you can delete them, that is not the recommended way. These nioneo logical logs are the backup logs made by neo4j to restore its database if it crashed. What you can do is change the duration the logs are retained. Here's an article on managing retention of neo4j logs.

[vranganathan]

Thanks. I added the platform-settings.json. Set the DatabaseLogRetentionPolicy to 4 days. I noticed that it freed up some space (about 1.5%). However in /ThingworxStorage/database/data I see nioneo_logical.log.* as old as 7 days. I did restart the tomcat. Is there anything I am missing?

[Aanjan]

That should be it. It should've ran another clean up by now, so has there been a change in disk space? You can set it to as low as 1 day.

[vranganathan]

It did not clean the old files. I am manually deleting them. Thanks.

[chrish]

Once I implemented platform-settings.json with PTC support's help; my logs started cleaning themselves up better.