Telemetry Daemon

The telemetry daemon (telemetryd) provides an extensible framework that you can use to handle sensor data pushed to Horizon. You can use the framework to support applications that use different protocols to transfer metrics data.

With telemetryd, operators can define listeners that support different protocols to receive the telemetry data, and adapters that transfer the received data into generic formats like flows or performance data.

Network overview diagram showing relationships among telemetryd components

Figure 1. Generic overview of components in telemetryd

The configuration is split in two parts: on one side, listeners and their attached parsers or connectors receive telemetry data that is transported over a specific protocol, and parse the data according to the protocol’s specification. On the other side, adapters pick up the parsed data and enrich it before persisting it.

Queues transport the parsed telemetry data from parsers to adapters by binding a specific parser to a specific adapter.

Listeners and parsers

Listeners receive sensor data from an external source and hand the data off to parsers, which handle the transport protocol format and extract the transported data. Listeners do the transport protocol handling, like opening sockets and accepting incoming connections. The received messages are then passed to the parsers that are configured to the listeners. They parse the concrete protocol implementation and extract all available information from the messages. The extracted data is then handed over to the configured queue for further processing.

For example, a listener may open a UDP socket to receive packets, and an assigned parser parses the incoming IPFIX packets.

Parsers and listeners must work closely together; for that reason, they cannot be combined randomly. The supported parsers are described in the documentation for each listener. For more information on listeners, their supported parsers, and how to configure them, see Telemetryd listeners reference.

Connectors

Connectors initiate connections and maintain sessions with agents. A new connector is created for every agent that it is responsible for connecting to. Here, "agent" refers to a named IP service attached to an IP interface on a specific node.

Connectors do not use parsers; they dispatch messages directly to queues.

Network overview diagram showing relationships among telemetryd connectors

Figure 2. Telemetryd connectors

Adapters

Adapters enrich and process data that is dispatched over a queue. They receive the data from the queue and transform it into a format that can be used by Horizon. They can also enrich incoming data to attach node or interface information, depending on the information that is already available. For example, an adapter may be used to transform telemetry data that is received via sFlow into performance data and persist it to the applicable node.

Since adapters must handle data dependent on a transport protocol, not all of them can be used in combination with every possible parser. The supported parsers are described in the documentation for each adapter.

If you have multiple adapters, the order in which they are run is the same as defined in telemetryd-configuration.xml.

Working with Minions

Listeners and parsers run on Minions, but adapters run on Sentinel. Your main Horizon instance can run listeners, parsers, and adapters.

When using a Minion as a listener, you must use adapters on your Horizon instance or a Sentinel instance to transform the data.

Queues

Queues transfer data between parsers and adapters, and are represented by a channel in the messaging system.

Configuring queues

The following options can help fine-tune queue behavior:

Queue Attribute (Horizon)	Key (Minion/Sentinel)	Description	Default Value
threads	queue.threads	Number of threads used for consuming and dispatching messages.	(2 * number of cores)
queue-size	queue.size	Maximum number of messages to keep in memory while waiting to be dispatched.	10000
use-routing-key	queue.use-routing-key	Whether to use the routing key when forwarding messages to the broker. This enforces message ordering.	true
batch-size	batch.size	Maximum batch size. When the batch reaches this size, it is dispatched.	1000
batch-interval	batch.interval	Maximum batch interval, in milliseconds. When the batch has been created for longer than this interval, it is dispatched, regardless of the current size.	500

Queue Attribute (Horizon)

Key (Minion/Sentinel)

Description

Default Value

threads

queue.threads

Number of threads used for consuming and dispatching messages.

(2 * number of cores)

queue-size

queue.size

Maximum number of messages to keep in memory while waiting to be dispatched.

10000

use-routing-key

queue.use-routing-key

Whether to use the routing key when forwarding messages to the broker. This enforces message ordering.

true

batch-size

batch.size

Maximum batch size. When the batch reaches this size, it is dispatched.

1000

batch-interval

batch.interval

Maximum batch interval, in milliseconds. When the batch has been created for longer than this interval, it is dispatched, regardless of the current size.

500

When using Kafka as a message broker, setting use-routing-key to false balances messages across all partitions. This can be done safely for flows, but is not supported for metrics when using thresholding (order is required).

In Horizon, you can add these options to the <queue> element as attributes:

<queue name="IPFIX" use-routing-key="false">
    ...
</queue>

On a Minion, you can add these options as parser properties, and as adapter properties on Sentinel:

name=IPFIX-Listener
class-name=org.opennms.netmgt.telemetry.listeners.UdpListener
parameters.host=0.0.0.0
parameters.port=4738
parsers.0.name=IPFIX
parsers.0.class-name=org.opennms.netmgt.telemetry.protocols.netflow.parser.IpfixUdpParser
parsers.0.queue.use-routing-key=false

Push sensor data through Minion

Listeners and their parsers may run either on Horizon or a Minion, whereas adapters may run either on Horizon or Sentinel. If a listener and its parsers are running on a Minion, the received messages are dispatched automatically via a queue to associated adapters running in Horizon or Sentinel. Minions can receive data, but require Horizon or Sentinel to process it.

Network overview diagram showing a listener running on a Minion, and the listener’s related components

Figure 3. Running listener on a Minion forwarding packets using the messaging system

Reverse hostname resolution

Some parsers support reverse hostname resolution to annotate IP addresses using the actual hostname. The Minion performs the reverse lookup while it parses the protocol data. The resolved hostname, if found, is attached to the address information. Both are then sent to the adapter for further processing. For more information see DNS Resolver.

The following parsers currently support reverse hostname resolution:

Netflow v5
Netflow v9
IPFIX
sFlow