Tuning Apache Kafka

The configuration is shipped with some defaults, but depending on the size and network topology you might need to tune the Apache Kafka environment to meet certain needs. Set Apache Kafka options directly in the org.opennms.core.ipc.rpc.kafka.cfg and org.opennms.core.ipc.sink.kafka.cfg file.

Alternatively, you can set Kafka producer/consumer options can be set by defining additional system properties prefixed with org.opennms.core.ipc.rpc.kafka and org.opennms.core.ipc.sink.kafka.

Find available configuration parameters for Kafka here:

Multiple Horizon instances

Topics will be automatically created and are prefixed by default with OpenNMS. If you want to use an Apache Kafka cluster with multiple Horizon instances, customize the topic prefix by setting org.opennms.core.ipc.rpc.kafka.group.id, org.opennms.core.ipc.sink.kafka.group.id, and org.opennms.instance.id in Horizon and its Minions to a string value that identifies your instance.

Tips for Kafka

For Kafka RPC, the number of partitions should always be greater than the number of minions at a location. When there are multiple locations, partitions >= max number of minions at a location.
By default, Kafka RPC supports buffers greater than >1MB by splitting large buffer into chunks of 900KB(912600). Max buffer size (900KB, by default) can be configured by setting org.opennms.core.ipc.rpc.kafka.max.buffer.size ( in bytes).
Default time to live (time at which request will expire) is 20000 msec (20sec). It can be changed by configuring system property org.opennms.core.ipc.rpc.kafka.ttl in msec.

Disable Single Topic for Kafka RPC

By default OpenNMS uses single topic structure i.e. it creates one request topic for each location and one response topic for all modules, regardless of location.

Note that all Minions at any location must be running the same features in order to make use of single topic.

For any reason, if you need to have different topic for each module or need different features on each minion, you can disable single-topic as below.

Disabling Single topic must be configured on both Minion and OpenNMS.

Disable single topic on Minion
echo 'single-topic=false' >> "$MINION_HOME/etc/org.opennms.core.ipc.rpc.kafka.cfg"

On OpenNMS, disable single topic by setting the org.opennms.core.ipc.rpc.kafka.single-topic system property to false.