The Thresholding Service maintains the state of the performance metrics and generates alarms when thresholds are triggered (armed) or cleared (unarmed). The service listens for and compares performance metrics after they are persisted to the time series database. The state of the thresholds are held in memory and pushed to persistent storage only when they change.
Thresholding for streaming telemetry with telemetryd is supported on Sentinel when using Newts. When running on Sentinel, the thresholding state can be stored in either Cassandra or PostgreSQL. Given that Newts requires Cassandra, we recommend using Cassandra to minimize the load on PostgreSQL.
Thresholding on Sentinel uses the same configuration files as Horizon and operates similarly.
When a thresholding changes to or from
cleared, an event is published which is processed by Horizon and the alarm is created or updated.