Horizon Log Files

This section describes common log files to look at when troubleshooting.

Debugging

When debugging, it may be helpful to lower the minimum severity at which a component will log from the default level of WARN. To change this setting, edit ${OPENNMS_HOME}/etc/log4j2.xml and locate the following lines:

<KeyValuePair key="..." value="WARN" />

You can alter the log level to one of the following values: DEBUG, INFO, WARN, ERROR. Changes to this file take effect within 60 seconds with no extra action needed.

On busy systems, setting the log level to DEBUG may put additional load on your disk. Make sure to not leave the system at DEBUG level longer than necessary.

manager.log

${OPENNMS_HOME}/logs/manager.log contains information relating to manager server events and the start and stop processes for Horizon services. For example, this log describes a problem connecting to the PostgreSQL database server:

2023-03-20 14:08:11,256 INFO  [Main] o.o.f.s.j.JCEKSSecureCredentialsVault: No existing keystore found at: /opt/opennms/etc/scv.jce. Using empty keystore.
2023-03-20 14:08:11,257 ERROR [Main] o.o.n.v.DatabaseChecker: Unable to connect to data source 'opennms' at URL 'jdbc:postgresql://localhost:5432/opennms' with username 'opennms', check opennms-datasources.xml and your database permissions.
2023-03-20 14:08:11,258 ERROR [Main] o.o.n.v.Controller: error invoking "check" operation: Data source 'opennms' failed.
org.opennms.netmgt.vmmgr.InvalidDataSourceException: Data source 'opennms' failed.

karafStartupMonitor.log

${OPENNMS_HOME}/logs/karafStartupMonitor.log contains information relating to startup of the Horizon Karaf container and its feature functionality. For example, this log describes a problem with the Karaf container’s startup process, and steps you can take to troubleshoot the error:

2023-03-24 14:18:46,751 INFO  [Main] o.o.f.k.h.d.KarafStartupMonitor: KarafStartupMonitor is starting.
2023-03-24 14:18:46,752 INFO  [Main] o.o.f.k.h.d.KarafStartupMonitor: Waiting for loading of org.opennms.features.karaf.health.service.KarafHealthService, will block startup until service is available.
2023-03-24 14:23:46,766 ERROR [Main] o.o.n.d.AbstractSpringContextJmxServiceDaemon: Could not start daemon: java.lang.IllegalStateException: KarafStartupMonitor: It seems Karaf can't be started properly. This is bad, will fail startup.
What can you do about this?
1.) check in logs/karaf.log for problems
2.) clear the 'data' folder - it contains Karaf's cache
3.) run the script bin/fix-karaf-setup.sh
If unexpected results occur when using Karaf, run ${OPENNMS_HOME}/bin/fix-karaf-setup.sh.