[WIP] Use templating for watcher dashboard and add Postgres exporter (#695)

* Add Postgres exporter and it's dashboard

* Use templating for watcher dashboard

* Add subgraph related panels to watcher dashboard

* Remove individual watcher dashboards and update instructions

stack_orchestrator/data/stacks/monitoring/README.md

@@ -72,7 +72,7 @@ laconic-so --stack monitoring deploy create --spec-file monitoring-spec.yml --de
       - targets: ['<METRICS_ENDPOINT_HOST>:<METRICS_ENDPOINT_PORT>']
   ```
 
-* Also update the `blackbox` job to add any endpoints to be monitored on the Blackbox dashboard:
+* Blackbox: update the `blackbox` job to add any endpoints to be monitored on the Blackbox dashboard:
 
   ```yml
   ...
@@ -85,6 +85,21 @@ laconic-so --stack monitoring deploy create --spec-file monitoring-spec.yml --de
         - <HTTP_ENDPOINT_2>
   ```
 
+* Postgres:
+  * Update the `postgres` job to add Postgres db targets to be monitored:
+
+    ```yml
+    ...
+    - job_name: 'postgres'
+      ...
+      static_configs:
+        # Add DB targets below
+        - targets: [example-server:5432]
+          labels:
+            instance: 'example-db'
+    ```
+  * Add database credentials to be used in `auth_modules` in the postgres-exporter config file (`monitoring-deployment/config/monitoring/postgres-exporter.yml`)
+
 Note: Use `host.docker.internal` as host to access ports on the host machine
 
 ### Grafana Config

stack_orchestrator/data/stacks/monitoring/monitoring-watchers.md

@@ -35,8 +35,6 @@ laconic-so --stack monitoring deploy create --spec-file monitoring-watchers-spec
 
 ## Configure
 
-### Prometheus Config
-
 Add the following scrape configs to prometheus config file (`monitoring-watchers-deployment/config/monitoring/prometheus/prometheus.yml`) in the deployment folder:
 
   ```yml
@@ -55,27 +53,35 @@ Add the following scrape configs to prometheus config file (`monitoring-watchers
       - targets: ['AZIMUTH_WATCHER_HOST:AZIMUTH_WATCHER_PORT']
         labels:
           instance: 'azimuth'
+          chain: 'ethereum'
       - targets: ['CENSURES_WATCHER_HOST:CENSURES_WATCHER_PORT']
         labels:
           instance: 'censures'
+          chain: 'ethereum'
       - targets: ['CLAIMS_WATCHER_HOST:CLAIMS_WATCHER_PORT']
         labels:
           instance: 'claims'
+          chain: 'ethereum'
       - targets: ['CONDITIONAL_STAR_RELEASE_WATCHER_HOST:CONDITIONAL_STAR_RELEASE_WATCHER_PORT']
         labels:
           instance: 'conditional_star_release'
+          chain: 'ethereum'
       - targets: ['DELEGATED_SENDING_WATCHER_HOST:DELEGATED_SENDING_WATCHER_PORT']
         labels:
-          instance: 'delegated_sending_watcher'
+          instance: 'delegated_sending'
+          chain: 'ethereum'
       - targets: ['ECLIPTIC_WATCHER_HOST:ECLIPTIC_WATCHER_PORT']
         labels:
           instance: 'ecliptic'
+          chain: 'ethereum'
       - targets: ['LINEAR_STAR_WATCHER_HOST:LINEAR_STAR_WATCHER_PORT']
         labels:
           instance: 'linear_star_release'
+          chain: 'ethereum'
       - targets: ['POLLS_WATCHER_HOST:POLLS_WATCHER_PORT']
         labels:
           instance: 'polls'
+          chain: 'ethereum'
 
   - job_name: sushi
     scrape_interval: 20s
@@ -85,18 +91,14 @@ Add the following scrape configs to prometheus config file (`monitoring-watchers
       - targets: ['SUSHISWAP_WATCHER_HOST:SUSHISWAP_WATCHER_PORT']
         labels:
           instance: 'sushiswap'
+          chain: 'filecoin'
       - targets: ['MERKLE_SUSHISWAP_WATCHER_HOST:MERKLE_SUSHISWAP_WATCHER_PORT']
         labels:
           instance: 'merkl_sushiswap'
+          chain: 'filecoin'
   ```
 
-### Grafana Config
-
-In the deployment folder, copy over the pre-configured watcher dashboard JSON files to grafana dashboards config directory:
-
-```bash
-cp -r monitoring-watchers-deployment/config/monitoring/grafana/watcher-dashboards/* monitoring-watchers-deployment/config/monitoring/grafana/dashboards/
-```
+Add scrape config as done above for any additional watcher to add it to the watcher dashboard.
 
 ### Env