From ce7d91d1bb7f80a2a07322cad58962924720a5b4 Mon Sep 17 00:00:00 2001 From: Prathamesh Musale Date: Wed, 27 Dec 2023 12:03:59 +0530 Subject: [PATCH] Update watcher monitoring instructions --- .../stacks/monitoring/monitoring-watchers.md | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/stack_orchestrator/data/stacks/monitoring/monitoring-watchers.md b/stack_orchestrator/data/stacks/monitoring/monitoring-watchers.md index 23335efb..3673b530 100644 --- a/stack_orchestrator/data/stacks/monitoring/monitoring-watchers.md +++ b/stack_orchestrator/data/stacks/monitoring/monitoring-watchers.md @@ -35,6 +35,8 @@ laconic-so --stack monitoring deploy create --spec-file monitoring-watchers-spec ## Configure +### Prometheus scrape config + Add the following scrape configs to prometheus config file (`monitoring-watchers-deployment/config/monitoring/prometheus/prometheus.yml`) in the deployment folder: ```yml @@ -100,6 +102,27 @@ Add the following scrape configs to prometheus config file (`monitoring-watchers Add scrape config as done above for any additional watcher to add it to the Watchers dashboard. +### Grafana alerts config + +Place the pre-configured watcher alerts rules in Grafana provisioning directory: + + ```bash + cp monitoring-watchers-deployment/config/monitoring/watcher-alert-rules.yml monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/ + ``` + +Update the alerting contact points config (`monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/contactpoints.yml`) with desired contact points + +Add corresponding routes to the notification policies config (`monitoring-watchers-deployment/monitoring/grafana/provisioning/alerting/policies.yaml`) with appropriate object-matchers: + + ```yml + ... + routes: + - receiver: SlackNotifier + object_matchers: + # Add matchers below + - ['grafana_folder', '=', 'WatcherAlerts'] + ``` + ### Env Set the following env variables in the deployment env config file (`monitoring-watchers-deployment/config.env`):