Shutdown and Restart Horizon

This section provides an example of the order to follow when shutting down and restarting a sample Horizon system: Horizon core layer, persistence and messaging layer, and database layer.

Diagram displaying Horizon components and the order in which to restart them.

Figure 1. Horizon restart process

Note that restarting the whole stack is rare. Usually, the Horizon core is restarted only to apply configuration changes, or to run upgrades. A core restart usually takes less than 5 minutes, but can take longer depending on the configuration. The Minion and Sentinel components restart more quickly.

When shutting down the Horizon core, there will be no notifications, alarms, outage detection, performance data collection, thresholding, or flows. After restarting, there will be gaps in your performance graphs spanning the shutdown time.

Restarting components in the persistence and messaging layer normally happens only for upgrades or catastrophic failures. Exercise caution with restarting components in this layer, since there is the risk of data loss.

The steps below are illustrative for a sample Linux setup. Your Horizon environment may be different. Documenting procedures for each setup is beyond the scope of this guide. You must be root to run the commands listed below.

Shutdown

Shut down Horizon components in the following order:

Dashboard applications, such as Grafana and Kibana.

systemctl stop grafana-server
systemctl stop kibana

The Horizon core application (core, Sentinel, Minion), in any order.
- Turn off notifications:
  1. Log in to the Horizon web UI.
  2. Click the gear symbol at the top-right of the page.
  3. Under Event Management, toggle Notification Status Off and click Update.
- Access the Horizon system via CLI (putty, SecureCRT, and so on).
- Use the sudo su - command to become a root user.
- Type the following commands:
  systemctl stop opennms systemctl stop minion systemctl stop sentinel
Persistence and messaging components (PostgreSQL, Cassandra, Kafka, and so on), in any order:
```
systemctl stop postgresql-12
systemctl stop cassandra
systemctl stop elasticsearch
```

Neither Kafka nor Zookeeper has systemd definitions. The processes are controlled based on how they were installed. This is beyond the scope of this guide.

Restart

Restart components in the following order:

Persistence and messaging components (PostgreSQL, Cassandra, Kafka, and so on), in any order:
```
systemctl start postgresql-12
systemctl start cassandra
systemctl start elasticsearch
```
The Horizon core application (core, Sentinel, Minion), in any order:
```
systemctl start opennms
systemctl start minion
systemctl start sentinel
```
- Turn on notifications:
  1. Log in to the Horizon web UI.
  2. Click the gear symbol at the top-right of the page.
  3. Under Event Management, toggle Notification Status On and click Update.

Dashboard applications (Grafana, Kibana, and so on):

systemctl start grafana-server
systemctl start kibana

You can restart dashboard applications at any time, but they will not show any useful information until all components in the persistence and Horizon layers have started.