Run registry CLI tests as part of laconicd fixturenet tests (#791 )

Part of https://www.notion.so/Test-registry-cli-in-SO-fixturenet-laconicd-CI-ef1f497678264362931bd12643ba8a17 Co-authored-by: neeraj <neeraj.rtly@gmail.com> Reviewed-on: cerc-io/stack-orchestrator#791 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
revert 5308ab1e4e (#788 )
2024-04-04 07:16:46 +00:00 · 2024-03-27 20:55:03 +00:00 · 2024-03-26 14:52:57 +00:00 · 2024-03-26 11:50:05 +00:00
12 changed files with 58 additions and 286 deletions
@@ -11,7 +11,7 @@ on:
 jobs:
  test:
-    name: "Run an Laconicd fixturenet test"
+    name: "Run Laconicd fixturenet and Laconic CLI tests"
    runs-on: ubuntu-latest
    steps:
      - name: 'Update'
@@ -46,3 +46,5 @@ jobs:
        run: ./scripts/build_shiv_package.sh
      - name: "Run fixturenet-laconicd tests"
        run: ./tests/fixturenet-laconicd/run-test.sh
      - name: "Run laconic CLI tests"
        run: ./tests/fixturenet-laconicd/run-cli-test.sh
@@ -1,3 +1,4 @@
 Change this file to trigger running the fixturenet-laconicd-test CI job
 Trigger
 Trigger
 Trigger
@@ -1,271 +0,0 @@
 # Deploying WebApps from the Laconic Registry to Kubernetes
 First, ensure you have `laconicd` and the console running via [this tutorial](./laconicd-with-console.md).
 ## Setup Kubernetes
 If merely requesting deployment, access to the Laconic registry is sufficient, but to *process* deployment requests, Kubernetes configuration is also necessary. The configuration format is the same as used by `kubectl`, and should be places in `/etc/config/kube.yml`
 ## Setup the Deployer
 ### Set envs
 ```
 $DEPLOYMENT_DNS_SUFFIX
 $DEPLOYMENT_RECORD_NAMESPACE
 $IMAGE_REGISTRY
 $IMAGE_REGISTRY_CREDS
 $IMAGE_REGISTRY_USER
 ```
 ```
 laconic-so --stack webapp-deployer-backend setup-repositories
 laconic-so --stack webapp-deployer-backend build-containers
 laconic-so --stack webapp-deployer-backend deploy up
 ```
 ## Setup Your Account
 Writing to the Laconic Registry, in order to create records for applications and deployment requests, requires a user key, and a bond with adequate funds available.
 ### Install the registry CLI
 ```
 npm config set @cerc-io:registry https://git.vdb.to/api/packages/cerc-io/npm/
 npm install -g @cerc-io/laconic-registry-cli
 ```
 Alternatively, use the pre-configured container that was setup alongside `laconicd`
 ### Get your private key
 ```
 exec into laconicd
 ```
 ### config.yml
 ```
 services:
  cns:
    restEndpoint: http://console.laconic.com:1317
    gqlEndpoint: http://console.laconic.com:9473/api
    userKey: 3d4789e88508c6230d973ccea283032d3e3948775dbe02f4f0a80dc6c1f7c8d5
   # bondId: 3c61577f3c6197ced599e2fc21ccf76f43373004fd29c29f2e560c77f7c4bb6d
    chainId: laconic_9000-1
    gas: 550000
    fees: 200000aphoton
 ```
 ### Create a bond
 ```
 laconic cns -c config.yml bond create --type aphoton --quantity 1000000000 --gas 200000 --fees 200000aphoton
 ```
 # Creating Deployments
 Deploying an application from the registry requires (and generates)
 several records:
 1. An `ApplicationRecord` which describes the application itself, including its name, version, and repository.
 2. An `ApplicationDeploymentRequest` which references an`ApplicationRecord`, and provides additional information such as the DNS name and configuration.
 3. A `DnsRecord`, which contains the FQDN of the deployment. The ownership of this record will be checked against any future deployment or removal requests.
 4. An `ApplicationDeploymentRecord` which records the successful processing of the deployment request.
 Additionally, since names need to be registered, namespace authorities need to be reserved, for example:
 ```
 $ laconic -c $LACONIC_CONFIG cns authority reserve my-org-name
 $ laconic -c $LACONIC_CONFIG cns authority bond set my-org-name 0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7
 ```
 ## Application Records
 The `ApplicationRecord` should, at a minimum, specify the name of the application (`name`), its version (`app_version`), type (`app_type`), one or more repository URLs (`repository`), and the repository reference (eg, branch, tag, hash, etc.) to use (`repository_ref`).
 ```
 $ cat app.yml
 record:
  type: ApplicationRecord
  version: 0.0.4
  name: "@my-org-name/my-test-webapp"
  repository:
    - "https://github.com/my-org-name/my-test-webapp"
  repository_ref: "v0.1.5"
  app_version: "0.1.5"
  app_type: "webapp"
 $ laconic -c $LACONIC_CONFIG cns record publish -f app.yml
 bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
 ```
 One or more names should be registered for the application, which deployment requests can reference.
 ```
 $ laconic -c $LACONIC_CONFIG cns name set "crn://my-org-name/applications/my-test-webapp" bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
 $ laconic -c $LACONIC_CONFIG cns name set "crn://my-org-name/applications/my-test-webapp@0.1.5" bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
 $ laconic -c $LACONIC_CONFIG cns record get --id bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
 [
  {
    "id": "bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky",
    "names": [
      "crn://my-org-name/applications/my-test-webapp",
      "crn://my-org-name/applications/my-test-webapp@0.1.5",
    ],
    "owners": [
      "2671D38525BDC91A5DF4794EF2059D5771133702"
    ],
    "bondId": "0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7",
    "createTime": "2024-01-12T20:30:33Z",
    "expiryTime": "2025-01-11T20:30:33Z",
    "attributes": {
      "app_type": "webapp",
      "app_version": "0.1.5",
      "name": "@my-org-name/my-test-webapp",
      "repository": [
        "https://github.com/my-org-name/my-test-webapp"
      ],
      "repository_ref": "v0.1.5",
      "type": "ApplicationRecord",
      "version": "0.0.4"
    }
  }
 ]
 ```
 ## Application Deployment Requests
 To create the deployment of specific application, and `ApplicationDeploymentRequest` must be published. This request must reference the application to be deployed (`application`) and may optionally provide configuration (`config`) and a DNS name (`dns`). If no DNS name is supplied, one will be generated. A supplied DNS is usually just the short hostname, not the FQDN, since the suffix is supplied by the deployer.
 ```
 $ cat req.yml
 record:
  type: ApplicationDeploymentRequest
  version: 1.0.0
  name: "my-org-name/my-test-webapp@0.1.5"
  application: "crn://my-org-name/applications/my-test-webapp@0.1.5"
  dns: "my-test-app"
  config:
    env:
      CERC_WEBAPP_DEBUG: my_debug_value_here
 $ laconic -c $LACONIC_CONFIG cns record publish -f req.yml
 bafyreihtpvwjka5ecjmca46y4dip5gb2h25vvmc7t27d7g4zecngjssvky
 ```
 It is not necessary to assign a name to the request.
 ## Building and Deploying
 Building and deploying will happen automatically for records published the production registry, but in other environments it can be triggered manually.
 Laconic `stack-orchestrator` is used build and launch the application. It will clone the repository, build the application, upload the container to a registry, and launch the instance in Kubernetes, with
 automatic DNS and TLS provisioning.
 ```
 $ laconic-so deploy-webapp-from-registry \
    --kube-config $KUBE_CONFIG \
    --image-registry registry.mydeployer.org/app-registry \
    --deployment-parent-dir /opt/deployments \
    --laconic-config $LACONIC_CONFIG \
    --dns-suffix mydeployer.servesthe.world \
    --record-namespace-dns crn://my-deployer-org/dns \
    --record-namespace-deployments crn://my-deployer-org/deployments \
    --request-id bafyreihtpvwjka5ecjmca46y4dip5gb2h25vvmc7t27d7g4zecngjssvky
 ```
 When deployment is complete, an `ApplicationDeploymentRecord` will be created:
 ```
 $ laconic -c $LACONIC_CONFIG cns name resolve crn://my-deployer-org/deployments/my-test-app.mydeployer.servesthe.world
 [
  {
    "id": "bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga",
    "names": [
      "crn://my-deployer-org/deployments/my-test-app.laconic.servesthe.world"
    ],
    "owners": [
      "2671D38525BDC91A5DF4794EF2059D5771133702"
    ],
    "bondId": "0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7",
    "createTime": "2024-01-11T20:45:29Z",
    "expiryTime": "2025-01-10T20:45:29Z",
    "attributes": {
      "application": "bafyreibs6y7jhgjlsoxyqlugvkweanp3bi7ippfmhehk2rslcpoaqic2xi",
      "dns": "bafyreicpac7tar5ua5e42zo7d5zwyp5yv7iord23p3gdox7zdfslrarjle",
      "meta": {
        "config": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
        "so": "82a5f9a3e4924cbb92e28be68759a487"
      },
      "name": "@cerc-io/my-test-webapp",
      "request": "bafyreiex7vmtruiiasra4wvqe7wirl6iuwiso6p3cvb3sjhq55e74zh4ke",
      "type": "ApplicationDeploymentRecord",
      "url": "https://my-test-app.mydeployer.servesthe.world",
      "version": "0.0.1"
    }
  }
 ]
 ```
 ## Checking Status
 The status of the deployment may be checked with `stack-orchestrator`.
 ```
 $ laconic-so deployment --dir /opt/deployments/my-test-app.snowball.servesthe.world status
 Ingress:
        Hostname: my-test-app.mydeployer.servesthe.world
        IP: 204.130.133.199
        TLS: notBefore: 2024-01-12T23:11:09Z, notAfter: 2024-04-11T23:11:08Z
 Pods:
        default/laconic-9560ffc64512e453-deployment-fb58d756f-l77pm: Running (2024-01-13 00:10:42+00:00)
 ```
 # Removing Deployments
 As with deployment, removal involves publishing a request, which is then fulfilled by deployment processor.
 ## Application Deployment Removal Requests
 ```
 $ cat apprm.yml
 record:
  type: ApplicationDeploymentRemovalRequest
  version: 1.0.0
  deployment: bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga
 $ laconic -c $LACONIC_CONFIG cns record publish -f apprm.yml
 bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
 ```
 ## Removal
 As with deployment, for records publish to the production service removal should be processed automatically. In other environments, removal can be processed with `stack-orchestrator`.
 ```
 $ laconic-so undeploy-webapp-from-registry   \
    --deployment-parent-dir /opt/deployments   \
    --laconic-config ~/.laconic/local.yml   \
    --request-id bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
 Request bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge needs to processed.
 Found 1 unsatisfied request(s) to process.
 Matched deployment ownership: 2671D38525BDC91A5DF4794EF2059D5771133702
 record:
  type: ApplicationDeploymentRemovalRecord
  deployment: bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga
  request: bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
  version: 1.0.0
 ```
@@ -3,6 +3,9 @@ services:
    restart: unless-stopped
    image: cerc/laconicd:local
    command: ["sh", "/docker-entrypoint-scripts.d/create-fixturenet.sh"]
    environment:
      TEST_AUCTION_ENABLED: ${TEST_AUCTION_ENABLED}
      TEST_REGISTRY_EXPIRY: ${TEST_REGISTRY_EXPIRY}
    volumes:
      # The cosmos-sdk node's database directory:
      - laconicd-data:/root/.laconicd
@@ -25,6 +28,7 @@ services:
    image: cerc/laconic-registry-cli:local
    volumes:
      - ../config/fixturenet-laconicd/registry-cli-config-template.yml:/registry-cli-config-template.yml
      - ${BASE_DIR:-~/cerc}/laconic-registry-cli:/laconic-registry-cli
 volumes:
  laconicd-data:
@@ -2,7 +2,7 @@
  host = "0.0.0.0"
  port = 3008
  kind = "active"
-  gqlPath = "/graphql"
+  gqlPath = "/"
  # Checkpointing state.
  checkpointing = true
@@ -5,7 +5,7 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
  set -x
 fi
-registry_command="laconic registry"
+registry_command="laconic cns"
 demo_records_dir="scripts/demo-records"
 # Check we have funds
@@ -2,7 +2,7 @@ version: "1.0"
 name: ajna
 description: "Ajna watcher stack"
 repos:
-  - git.vdb.to/cerc-io/ajna-watcher-ts
+  - git.vdb.to/cerc-io/ajna-watcher-ts@v0.1.1
 containers:
  - cerc/watcher-ajna
 pods:
@@ -49,7 +49,7 @@ $ laconic-so --stack fixturenet-laconic-loaded deploy logs
 ```
 ## 6. Test with the Registry CLI
 ```
-$ laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic registry status"
+$ laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic cns status"
 ```
 ## 7. View the laconic console
 Get the URL for the console web app with this command (the port number will be different for each deployment):
@@ -44,5 +44,5 @@ $ laconic-so --stack fixturenet-laconicd deploy logs
 ```
 ## 6. Test with the Registry CLI
 ```
-$ laconic-so --stack fixturenet-laconicd deploy exec cli "laconic registry status"
+$ laconic-so --stack fixturenet-laconicd deploy exec cli "laconic cns status"
 ```
@@ -94,7 +94,7 @@ class LaconicRegistryClient:
        )
    def list_records(self, criteria={}, all=False):
-        args = ["laconic", "-c", self.config_file, "registry", "record", "list"]
+        args = ["laconic", "-c", self.config_file, "cns", "record", "list"]
        if all:
            args.append("--all")
@@ -140,7 +140,7 @@ class LaconicRegistryClient:
        if name in self.cache.name_or_id:
            return self.cache.name_or_id[name]
-        args = ["laconic", "-c", self.config_file, "registry", "name", "resolve", name]
+        args = ["laconic", "-c", self.config_file, "cns", "name", "resolve", name]
        parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
        if parsed:
@@ -165,7 +165,7 @@ class LaconicRegistryClient:
            "laconic",
            "-c",
            self.config_file,
-            "registry",
+            "cns",
            "record",
            "get",
            "--id",
@@ -203,8 +203,7 @@ class LaconicRegistryClient:
            print(open(record_fname, 'r').read(), file=self.log_file)
            new_record_id = json.loads(
-                logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry",
+                logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
                           "record", "publish", "--filename", record_fname)
            )["id"]
            for name in names:
                self.set_name(name, new_record_id)
@@ -213,10 +212,10 @@ class LaconicRegistryClient:
            logged_cmd(self.log_file, "rm", "-rf", tmpdir)
    def set_name(self, name, record_id):
-        logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry", "name", "set", name, record_id)
+        logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
    def delete_name(self, name):
-        logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry", "name", "delete", name)
+        logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "delete", name)
 def file_hash(filename):
@@ -0,0 +1,37 @@
 #!/usr/bin/env bash
 set -e
 if [ -n "$CERC_SCRIPT_DEBUG" ]; then
  set -x
 fi
 echo "$(date +"%Y-%m-%d %T"): Running stack-orchestrator Laconic registry CLI tests"
 env
 cat /etc/hosts
 # Bit of a hack, test the most recent package
 TEST_TARGET_SO=$( ls -t1 ./package/laconic-so* | head -1 )
 echo "$(date +"%Y-%m-%d %T"): Starting stack"
 TEST_AUCTION_ENABLED=true BASE_DIR=~/cerc $TEST_TARGET_SO --stack fixturenet-laconicd deploy --cluster laconicd up
 echo "$(date +"%Y-%m-%d %T"): Stack started"
 # Verify that the fixturenet is up and running
 $TEST_TARGET_SO --stack fixturenet-laconicd deploy --cluster laconicd ps
 # Get the fixturenet account address
 laconicd_account_address=$(docker exec laconicd-laconicd-1 laconicd keys list | awk '/- address:/ {print $3}')
 # Copy over config
 docker exec laconicd-cli-1 cp config.yml laconic-registry-cli/
 # Wait for the laconid endpoint to come up
 echo "Waiting for the RPC endpoint to come up"
 docker exec laconicd-laconicd-1 sh -c "curl --retry 20 --retry-delay 3 --retry-connrefused http://127.0.0.1:9473/api"
 # Run the tests
 echo "Running the tests"
 docker exec -e TEST_ACCOUNT=$laconicd_account_address laconicd-cli-1 sh -c 'cd laconic-registry-cli && yarn && yarn test'
 # Clean up
 $TEST_TARGET_SO --stack fixturenet-laconicd deploy --cluster laconicd down --delete-volumes
 echo "$(date +"%Y-%m-%d %T"): Test finished"
@@ -32,14 +32,14 @@ set +e
 CONTAINER_ID=$(docker run -p 3000:80 -d -e CERC_SCRIPT_DEBUG=$CERC_SCRIPT_DEBUG cerc/test-progressive-web-app:local)
 sleep 3
-wget -t 7 -O test.before -m http://localhost:3000
+wget --tries 20 --retry-connrefused --waitretry=3 -O test.before -m http://localhost:3000
 docker logs $CONTAINER_ID
 docker remove -f $CONTAINER_ID
 CONTAINER_ID=$(docker run -p 3000:80 -e CERC_WEBAPP_DEBUG=$CHECK -e CERC_SCRIPT_DEBUG=$CERC_SCRIPT_DEBUG -d cerc/test-progressive-web-app:local)
 sleep 3
-wget -t 7 -O test.after -m http://localhost:3000
+wget --tries 20 --retry-connrefused --waitretry=3 -O test.after -m http://localhost:3000
 docker logs $CONTAINER_ID
 docker remove -f $CONTAINER_ID
Author	SHA1	Message	Date
prathameshandneeraj	105805cb9b	Run registry CLI tests as part of laconicd fixturenet tests (#791 ) Part of https://www.notion.so/Test-registry-cli-in-SO-fixturenet-laconicd-CI-ef1f497678264362931bd12643ba8a17 Co-authored-by: neeraj <neeraj.rtly@gmail.com> Reviewed-on: cerc-io/stack-orchestrator#791 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-04-04 07:16:46 +00:00
telackey	d2442bcc9b	revert `5308ab1e4e` (#788 ) revert Blind commit to fix laconic CLI calls after rename. (#784) `laconic cns` got renamed to `laconic registry` which breaks all the scripts and commands that use it. Reviewed-on: cerc-io/stack-orchestrator#784 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com> Reviewed-on: cerc-io/stack-orchestrator#788	2024-03-27 20:55:03 +00:00
nabarun	44faf36837	Update ajna-watcher-ts version for using new subgraph (#786 ) Part of https://www.notion.so/Run-ajna-finance-subgraph-watcher-87748d78cd7a471b8d71f50d5fdc2657 Reviewed-on: cerc-io/stack-orchestrator#786 Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Co-committed-by: Nabarun <nabarun@deepstacksoft.com>	2024-03-26 14:52:57 +00:00
nabarun	18b006468d	Update GQL path for ajna subgraph watcher server (#785 ) Part of https://www.notion.so/Run-ajna-finance-subgraph-watcher-87748d78cd7a471b8d71f50d5fdc2657 Reviewed-on: cerc-io/stack-orchestrator#785 Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Co-committed-by: Nabarun <nabarun@deepstacksoft.com>	2024-03-26 11:50:05 +00:00