Remove latest indexed block value from watcher alerts data (#780 )

Part of https://www.notion.so/Setup-grafana-SO-stack-for-monitoring-watchers-7e23042c296c4de6b8676f1f604aa03c Reviewed-on: cerc-io/stack-orchestrator#780 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Update subgraph watcher versions and instructions to use deployments (#775 )
2024-03-13 07:16:15 +00:00 · 2024-03-12 05:32:55 +00:00 · 2024-03-07 17:38:36 +00:00 · 2024-03-06 18:38:30 +00:00
14 changed files with 211 additions and 305 deletions
@@ -50,22 +50,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -142,22 +126,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="censures", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -234,22 +202,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="claims", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -326,22 +278,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="conditional_star_release", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -418,22 +354,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="delegated_sending", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -510,22 +430,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="ecliptic", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -602,22 +506,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -694,22 +582,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="azimuth", instance="polls", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -788,22 +660,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="sushi", instance="sushiswap", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -880,22 +736,6 @@ groups:
              legendFormat: __auto
              range: false
              refId: latest_external
-          - refId: latest_indexed
-            relativeTimeRange:
-              from: 600
-              to: 0
-            datasourceUid: PBFA97CFB590B2093
-            model:
-              datasource:
-                type: prometheus
-                uid: PBFA97CFB590B2093
-              editorMode: code
-              expr: sync_status_block_number{job="sushi", instance="merkl_sushiswap", kind="latest_indexed"}
-              hide: false
-              instant: true
-              legendFormat: __auto
-              range: false
-              refId: latest_indexed
          - refId: condition
            relativeTimeRange:
              from: 600
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
 CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/nextjs-base:local}

 docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
+rc=$?

-if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
+if [ $rc -ne 0 ]; then
+  echo "BUILD FAILED" 1>&2
+  exit $rc
+fi
+
+if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
  cat <<EOF

 #################################################################
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
 CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/webapp-base:local}

 docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
+rc=$?

-if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
+if [ $rc -ne 0 ]; then
+  echo "BUILD FAILED" 1>&2
+  exit $rc
+fi
+
+if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
  cat <<EOF

 #################################################################
@@ -16,26 +16,55 @@ laconic-so --stack merkl-sushiswap-v3 build-containers

 ## Deploy

-### Configuration
-
-Create and update an env file to be used in the next step:
-
-  ```bash
-  # External Filecoin (ETH RPC) endpoint to point the watcher
-  CERC_ETH_RPC_ENDPOINT=
-  ```
-
-### Deploy the stack
+Create a spec file for the deployment:

 ```bash
-laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
+laconic-so --stack merkl-sushiswap-v3 deploy init --output merkl-sushiswap-v3-spec.yml
+```
+
+### Ports
+
+Edit `network` in the spec file to map container ports to host ports as required:
+
+```
+...
+network:
+  ports:
+    merkl-sushiswap-v3-watcher-db:
+     - '5432'
+    merkl-sushiswap-v3-watcher-job-runner:
+     - 9002:9000
+    merkl-sushiswap-v3-watcher-server:
+     - 127.0.0.1:3007:3008
+     - 9003:9001
+```
+
+### Create a deployment
+
+Create a deployment from the spec file:
+
+```bash
+laconic-so --stack merkl-sushiswap-v3 deploy create --spec-file merkl-sushiswap-v3-spec.yml --deployment-dir merkl-sushiswap-v3-deployment
+```
+
+### Configuration
+
+Inside deployment directory, open the `config.env` file  and set following env variables:
+
+```bash
+# External Filecoin (ETH RPC) endpoint to point the watcher to
+CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
+```
+
+### Start the deployment
+
+```bash
+laconic-so deployment --dir merkl-sushiswap-v3-deployment start
 ```

 * To list down and monitor the running containers:

  ```bash
-  laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 ps
-
  # With status
  docker ps -a

@@ -46,6 +75,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
 * Open the GQL playground at http://localhost:3007/graphql

  ```graphql
+  # Example query
  {
    _meta {
      block {
@@ -54,7 +84,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
      }
      hasIndexingErrors
    }
-    
+
    factories {
      id
      poolCount
@@ -64,18 +94,21 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-

 ## Clean up

-Stop all the services running in background:
+Stop all the merkl-sushiswap-v3 services running in background:

 ```bash
-laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 down
+# Only stop the docker containers
+laconic-so deployment --dir merkl-sushiswap-v3-deployment stop
+
+# Run 'start' to restart the deployment
 ```

-Clear volumes created by this stack:
+To stop all the merkl-sushiswap-v3 services and also delete data:

 ```bash
-# List all relevant volumes
-docker volume ls -q --filter "name=merkl_sushiswap_v3"
+# Stop the docker containers
+laconic-so deployment --dir merkl-sushiswap-v3-deployment stop --delete-volumes

-# Remove all the listed volumes
-docker volume rm $(docker volume ls -q --filter "name=merkl_sushiswap_v3")
+# Remove deployment directory (deployment will have to be recreated for a re-run)
+rm -r merkl-sushiswap-v3-deployment
 ```
@@ -2,7 +2,7 @@ version: "1.0"
 name: merkl-sushiswap-v3
 description: "SushiSwap v3 watcher stack"
 repos:
-  - github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.6
+  - github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.7
 containers:
  - cerc/watcher-merkl-sushiswap-v3
 pods:
@@ -1,7 +1,7 @@
 version: "0.1"
 name: monitoring
 repos:
-  - github.com/cerc-io/watcher-ts@v0.2.79
+  - github.com/cerc-io/watcher-ts@v0.2.81
 containers:
  - cerc/watcher-ts
 pods:
@@ -55,7 +55,7 @@ ports:
 Create deployment:

 ```bash
-laconic-so deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
+laconic-so --stack sushiswap-subgraph deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
 ```

 ## Start the stack
@@ -16,26 +16,55 @@ laconic-so --stack sushiswap-v3 build-containers

 ## Deploy

-### Configuration
-
-Create and update an env file to be used in the next step:
-
-  ```bash
-  # External Filecoin (ETH RPC) endpoint to point the watcher
-  CERC_ETH_RPC_ENDPOINT=
-  ```
-
-### Deploy the stack
+Create a spec file for the deployment:

 ```bash
-laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
+laconic-so --stack sushiswap-v3 deploy init --output sushiswap-v3-spec.yml
+```
+
+### Ports
+
+Edit `network` in the spec file to map container ports to host ports as required:
+
+```
+...
+network:
+  ports:
+    sushiswap-v3-watcher-db:
+     - '5432'
+    sushiswap-v3-watcher-job-runner:
+     - 9000:9000
+    sushiswap-v3-watcher-server:
+     - 127.0.0.1:3008:3008
+     - 9001:9001
+```
+
+### Create a deployment
+
+Create a deployment from the spec file:
+
+```bash
+laconic-so --stack sushiswap-v3 deploy create --spec-file sushiswap-v3-spec.yml --deployment-dir sushiswap-v3-deployment
+```
+
+### Configuration
+
+Inside deployment directory, open the `config.env` file  and set following env variables:
+
+```bash
+# External Filecoin (ETH RPC) endpoint to point the watcher to
+CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
+```
+
+### Start the deployment
+
+```bash
+laconic-so deployment --dir sushiswap-v3-deployment start
 ```

 * To list down and monitor the running containers:

  ```bash
-  laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 ps
-
  # With status
  docker ps -a

@@ -43,20 +72,43 @@ laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_T
  docker logs -f <CONTAINER_ID>
  ```

+* Open the GQL playground at http://localhost:3008/graphql
+
+  ```graphql
+  # Example query
+  {
+    _meta {
+      block {
+        number
+        timestamp
+      }
+      hasIndexingErrors
+    }
+
+    factories {
+      id
+      poolCount
+    }
+  }
+  ```
+
 ## Clean up

-Stop all the services running in background:
+Stop all the sushiswap-v3 services running in background:

 ```bash
-laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 down
+# Only stop the docker containers
+laconic-so deployment --dir sushiswap-v3-deployment stop
+
+# Run 'start' to restart the deployment
 ```

-Clear volumes created by this stack:
+To stop all the sushiswap-v3 services and also delete data:

 ```bash
-# List all relevant volumes
-docker volume ls -q --filter "name=sushiswap_v3"
+# Stop the docker containers
+laconic-so deployment --dir sushiswap-v3-deployment stop --delete-volumes

-# Remove all the listed volumes
-docker volume rm $(docker volume ls -q --filter "name=sushiswap_v3")
+# Remove deployment directory (deployment will have to be recreated for a re-run)
+rm -r sushiswap-v3-deployment
 ```
@@ -2,7 +2,7 @@ version: "1.0"
 name: sushiswap-v3
 description: "SushiSwap v3 watcher stack"
 repos:
-  - github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.6
+  - github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.7
 containers:
  - cerc/watcher-sushiswap-v3
 pods:
@@ -101,7 +101,7 @@ class ClusterInfo:
        )
        return service

-    def get_ingress(self, use_tls=False):
+    def get_ingress(self, use_tls=False, certificate=None, cluster_issuer="letsencrypt-prod"):
        # No ingress for a deployment that has no http-proxy defined, for now
        http_proxy_info_list = self.spec.get_http_proxy()
        ingress = None
@@ -110,32 +110,13 @@ class ClusterInfo:
            http_proxy_info = http_proxy_info_list[0]
            if opts.o.debug:
                print(f"http-proxy: {http_proxy_info}")
-
-            host_name = http_proxy_info["host-name"]
-
-            tls = None
-            tls_issuer = None
-
-            if use_tls:
-                tls_info = http_proxy_info.get("tls", {})
-                tls_hosts = tls_info.get("hosts", [host_name])
-                tls_issuer = tls_info.get("issuer", "letsencrypt-prod")
-                tls_secret_name = f"{self.app_name}-tls"
-                if "secret" in tls_info:
-                    # If an existing secret is specified, unset the issuer so that we don't try to re-request it.
-                    tls_secret_name = tls_info["secret"]
-                    tls_issuer = None
-
-                if opts.o.debug:
-                    print(f"TLS hosts/secret: {tls_hosts}/{tls_secret_name}")
-
-                tls = [client.V1IngressTLS(
-                    hosts=tls_hosts,
-                    secret_name=tls_secret_name
-                )]
-
            # TODO: good enough parsing for webapp deployment for now
+            host_name = http_proxy_info["host-name"]
            rules = []
+            tls = [client.V1IngressTLS(
+                hosts=certificate["spec"]["dnsNames"] if certificate else [host_name],
+                secret_name=certificate["spec"]["secretName"] if certificate else f"{self.app_name}-tls"
+            )] if use_tls else None
            paths = []
            for route in http_proxy_info["routes"]:
                path = route["path"]
@@ -166,15 +147,17 @@ class ClusterInfo:
                tls=tls,
                rules=rules
            )
-            annotations = {
+
+            ingress_annotations = {
                "kubernetes.io/ingress.class": "nginx",
            }
-            if tls_issuer:
-                annotations["cert-manager.io/cluster-issuer"] = tls_issuer
+            if not certificate:
+                ingress_annotations["cert-manager.io/cluster-issuer"] = cluster_issuer
+
            ingress = client.V1Ingress(
                metadata=client.V1ObjectMeta(
                    name=f"{self.app_name}-ingress",
-                    annotations=annotations
+                    annotations=ingress_annotations
                ),
                spec=spec
            )
@@ -169,6 +169,39 @@ class K8sDeployer(Deployer):
                print("Service created:")
                print(f"{service_resp}")

+    def _find_certificate_for_host_name(self, host_name):
+        all_certificates = self.custom_obj_api.list_namespaced_custom_object(
+            group="cert-manager.io",
+            version="v1",
+            namespace=self.k8s_namespace,
+            plural="certificates"
+        )
+
+        host_parts = host_name.split(".", 1)
+        host_as_wild = None
+        if len(host_parts) == 2:
+            host_as_wild = f"*.{host_parts[1]}"
+
+        now = datetime.utcnow().replace(tzinfo=timezone.utc)
+        fmt = "%Y-%m-%dT%H:%M:%S%z"
+
+        # Walk over all the configured certificates.
+        for cert in all_certificates["items"]:
+            dns = cert["spec"]["dnsNames"]
+            # Check for an exact hostname match or a wildcard match.
+            if host_name in dns or host_as_wild in dns:
+                status = cert.get("status", {})
+                # Check the certificate date.
+                if "notAfter" in status and "notBefore" in status:
+                    before = datetime.strptime(status["notBefore"], fmt)
+                    after = datetime.strptime(status["notAfter"], fmt)
+                    if before < now < after:
+                        # Check the status is Ready
+                        for condition in status.get("conditions", []):
+                            if "True" == condition.get("status") and "Ready" == condition.get("type"):
+                                return cert
+        return None
+
    def up(self, detach, services):
        if not opts.o.dry_run:
            if self.is_kind():
@@ -189,8 +222,15 @@ class K8sDeployer(Deployer):
        self._create_volume_data()
        self._create_deployment()

+        http_proxy_info = self.cluster_info.spec.get_http_proxy()
        # Note: at present we don't support tls for kind (and enabling tls causes errors)
-        ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=not self.is_kind())
+        use_tls = http_proxy_info and not self.is_kind()
+        certificate = self._find_certificate_for_host_name(http_proxy_info[0]["host-name"]) if use_tls else None
+        if opts.o.debug:
+            if certificate:
+                print(f"Using existing certificate: {certificate}")
+
+        ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=use_tls, certificate=certificate)
        if ingress:
            if opts.o.debug:
                print(f"Sending this ingress: {ingress}")
@@ -353,7 +393,8 @@ class K8sDeployer(Deployer):
            hostname = ingress.spec.rules[0].host
            ip = ingress.status.load_balancer.ingress[0].ip
            tls = "notBefore: %s; notAfter: %s; names: %s" % (
-                cert["status"]["notBefore"], cert["status"]["notAfter"], ", ".join(ingress.spec.tls[0].hosts))
+                cert["status"]["notBefore"], cert["status"]["notAfter"], ingress.spec.tls[0].hosts
+            )
        except:  # noqa: E722
            pass

@@ -15,7 +15,6 @@

 import click
 import os
-import sys
 from pathlib import Path
 from urllib.parse import urlparse
 from tempfile import NamedTemporaryFile
@@ -24,7 +23,6 @@ from stack_orchestrator.util import error_exit, global_options2
 from stack_orchestrator.deploy.deployment_create import init_operation, create_operation
 from stack_orchestrator.deploy.deploy import create_deploy_context
 from stack_orchestrator.deploy.deploy_types import DeployCommandContext
-from stack_orchestrator.deploy.webapp.util import TlsDetails


 def _fixup_container_tag(deployment_dir: str, image: str):
@@ -38,16 +36,16 @@ def _fixup_container_tag(deployment_dir: str, image: str):
        wfile.write(contents)


-def _fixup_url_spec(spec_file_name: str, url: str, tls_details: TlsDetails = TlsDetails()):
+def _fixup_url_spec(spec_file_name: str, url: str):
    # url is like: https://example.com/path
    parsed_url = urlparse(url)
-    http_proxy_spec = f'''  http-proxy:
+    http_proxy_spec = f'''
+  http-proxy:
    - host-name: {parsed_url.hostname}
      routes:
        - path: '{parsed_url.path if parsed_url.path else "/"}'
          proxy-to: webapp:80
-{tls_details.to_yaml(indent=6)}
-'''
+    '''
    spec_file_path = Path(spec_file_name)
    with open(spec_file_path) as rfile:
        contents = rfile.read()
@@ -56,8 +54,7 @@ def _fixup_url_spec(spec_file_name: str, url: str, tls_details: TlsDetails = Tls
        wfile.write(contents)


-def create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file,
-                      tls_details: TlsDetails = None):
+def create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
    # Do the equivalent of:
    # 1. laconic-so --stack webapp-template deploy --deploy-to k8s init --output webapp-spec.yml
    #   --config (eqivalent of the contents of my-config.env)
@@ -89,7 +86,7 @@ def create_deployment(ctx, deployment_dir, image, url, kube_config, image_regist
        None
    )
    # Add the TLS and DNS spec
-    _fixup_url_spec(spec_file_name, url, tls_details)
+    _fixup_url_spec(spec_file_name, url)
    create_operation(
        deploy_command_context,
        spec_file_name,
@@ -119,16 +116,8 @@ def command(ctx):
@click.option("--image", help="image to deploy", required=True)
@click.option("--url", help="url to serve", required=True)
@click.option("--env-file", help="environment file for webapp")
-@click.option("--tls-host", help="Override TLS hostname (eg, '*.mydomain.com')")
-@click.option("--tls-secret", help="Override TLS secret name")
-@click.option("--tls-issuer", help="TLS issuer to use (default: letsencrypt-prod)")
@click.pass_context
-def create(ctx, deployment_dir, image, url, kube_config, image_registry, env_file, tls_host, tls_secret, tls_issuer):
+def create(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
    '''create a deployment for the specified webapp container'''

-    if (tls_secret and not tls_host) or (tls_host and not tls_secret):
-        print("Cannot specify --tls-host without --tls-secret", file=sys.stderr)
-        sys.exit(2)
-
-    return create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file,
-                             TlsDetails(tls_host, tls_secret, tls_issuer))
+    return create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file)
@@ -26,7 +26,7 @@ import click

 from stack_orchestrator.deploy.images import remote_image_exists, add_tags_to_image
 from stack_orchestrator.deploy.webapp import deploy_webapp
-from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient, TimedLogger, TlsDetails,
+from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient, TimedLogger,
                                                   build_container_image, push_container_image,
                                                   file_hash, deploy_to_k8s, publish_deployment,
                                                   hostname_for_deployment_request, generate_hostname_for_app,
@@ -44,8 +44,7 @@ def process_app_deployment_request(
    kube_config,
    image_registry,
    force_rebuild,
-    tls_details,
-    logger,
+    logger
 ):
    logger.log("BEGIN - process_app_deployment_request")

@@ -107,7 +106,7 @@ def process_app_deployment_request(
                            (app_deployment_crn, deployment_dir))
        print("deploy_webapp", deployment_dir)
        deploy_webapp.create_deployment(ctx, deployment_dir, deployment_container_tag,
-                                        f"https://{fqdn}", kube_config, image_registry, env_filename, tls_details)
+                                        f"https://{fqdn}", kube_config, image_registry, env_filename)
    elif env_filename:
        shutil.copyfile(env_filename, deployment_config_file)

@@ -199,14 +198,11 @@ def dump_known_requests(filename, requests, status="SEEN"):
@click.option("--exclude-tags", help="Exclude requests with matching tags (comma-separated).", default="")
@click.option("--force-rebuild", help="Rebuild even if the image already exists.", is_flag=True)
@click.option("--log-dir", help="Output build/deployment logs to directory.", default=None)
-@click.option("--tls-host", help="Override TLS hostname (eg, '*.mydomain.com')")
-@click.option("--tls-secret", help="Override TLS secret name")
-@click.option("--tls-issuer", help="TLS issuer to use (default: letsencrypt-prod)")
@click.pass_context
 def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_dir,  # noqa: C901
            request_id, discover, state_file, only_update_state,
            dns_suffix, record_namespace_dns, record_namespace_deployments, dry_run,
-            include_tags, exclude_tags, force_rebuild, log_dir, tls_host, tls_secret, tls_issuer):
+            include_tags, exclude_tags, force_rebuild, log_dir):
    if request_id and discover:
        print("Cannot specify both --request-id and --discover", file=sys.stderr)
        sys.exit(2)
@@ -224,10 +220,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
            print("--dns-suffix, --record-namespace-dns, and --record-namespace-deployments are all required", file=sys.stderr)
            sys.exit(2)

-    if (tls_secret and not tls_host) or (tls_host and not tls_secret):
-        print("Cannot specify --tls-host without --tls-secret", file=sys.stderr)
-        sys.exit(2)
-
    # Split CSV and clean up values.
    include_tags = [tag.strip() for tag in include_tags.split(",") if tag]
    exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
@@ -313,7 +305,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
    print("Found %d unsatisfied request(s) to process." % len(requests_to_execute))

    if not dry_run:
-        tls_details = TlsDetails(tls_host, tls_secret, tls_issuer)
        for r in requests_to_execute:
            dump_known_requests(state_file, [r], "DEPLOYING")
            status = "ERROR"
@@ -343,7 +334,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
                    kube_config,
                    image_registry,
                    force_rebuild,
-                    tls_details,
                    logger
                )
                status = "DEPLOYED"
@@ -26,40 +26,6 @@ import uuid
 import yaml


-class TlsDetails:
-    def __init__(self, host_or_hosts=None, secret_name: str = None, issuer_name: str = None):
-        if host_or_hosts:
-            if isinstance(host_or_hosts, list):
-                self.hosts = host_or_hosts
-            else:
-                self.hosts = [host_or_hosts]
-        else:
-            self.hosts = None
-        self.secret_name = secret_name
-        self.issuer_name = issuer_name
-
-    def to_yaml(self, indent=6):
-        if not self.hosts and not self.secret_name and not self.issuer_name:
-            return ""
-
-        ret = " " * indent + "tls:\n"
-        indent += 2
-
-        if self.issuer_name:
-            ret += " " * indent + "issuer: '%s'\n" % self.issuer_name
-
-        if self.secret_name:
-            ret += " " * indent + "secret: '%s'\n" % self.secret_name
-
-        if self.hosts:
-            ret += " " * indent + "hosts:"
-            indent += 2
-            for h in self.hosts:
-                ret += "\n" + " " * indent + "- '%s'" % h
-
-        return ret
-
-
 class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
Author	SHA1	Message	Date
prathamesh	aeddc82ebc	Remove latest indexed block value from watcher alerts data (#780 ) Part of https://www.notion.so/Setup-grafana-SO-stack-for-monitoring-watchers-7e23042c296c4de6b8676f1f604aa03c Reviewed-on: cerc-io/stack-orchestrator#780 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-03-13 07:16:15 +00:00
prathamesh	17e860d6e4	Update subgraph watcher versions and instructions to use deployments (#775 ) Part of https://www.notion.so/Setup-watchers-on-sandman-34b5514a10634c6fbf3ec338967c871c Reviewed-on: cerc-io/stack-orchestrator#775 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-03-12 05:32:55 +00:00
telackey	523b5779be	Auto-detect which certificate to use (including wildcards). (#779 ) Rather than always requesting a certificate, attempt to re-use an existing certificate if it already exists in the k8s cluster. This includes matching to a wildcard certificate. Reviewed-on: cerc-io/stack-orchestrator#779 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-03-07 17:38:36 +00:00
telackey	62f7ce649d	Exit non-0 if docker build fails. (#778 ) Make sure to check the exit code of the docker build and bubble it back up to laconic-so. Reviewed-on: cerc-io/stack-orchestrator#778 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-03-06 18:38:30 +00:00