Add a custom label to blackbox metrics and update instructions

This commit is contained in:
2024-04-15 17:51:17 +05:30
parent 1746f7366c
commit 23d527720f
3 changed files with 17 additions and 10 deletions
@@ -44,9 +44,13 @@ Add the following scrape configs to prometheus config file (`monitoring-watchers
- job_name: 'blackbox'
...
static_configs:
- targets:
- <AZIMUTH_GATEWAY_GQL_ENDPOINT>
- <LACONICD_GQL_ENDPOINT>
- targets: [<AZIMUTH_GATEWAY_GQL_ENDPOINT>]
labels:
# Add destination label for pre-configured alerts
destination: 'azimuth_gateway'
- targets: [<LACONICD_GQL_ENDPOINT>]
labels:
destination: 'laconicd_gql'
...
- job_name: laconicd
...
@@ -120,10 +124,12 @@ Add scrape config as done above for any additional watcher to add it to the Watc
### Grafana alerts config
Place the pre-configured watcher alerts rules in Grafana provisioning directory:
Place the pre-configured watcher and blackbox endpoint alerts rules in Grafana provisioning directory:
```bash
cp monitoring-watchers-deployment/config/monitoring/watcher-alert-rules.yml monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/
cp monitoring-watchers-deployment/config/monitoring/blackbox-alert-rules.yml monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/
```
Update the alerting contact points config (`monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/contactpoints.yml`) with desired contact points
@@ -136,7 +142,7 @@ Add corresponding routes to the notification policies config (`monitoring-watche
- receiver: SlackNotifier
object_matchers:
# Add matchers below
- ['grafana_folder', '=', 'WatcherAlerts']
- ['grafana_folder', '=~', 'WatcherAlerts|BlackboxAlerts']
```
### Env