testnet-laconicd-stack/ops/sp-deployment-from-scratch.md

Service Provider deployments from scratch

container-registry

• Reference: https://github.com/LaconicNetwork/loro-testnet/blob/main/docs/service-provider-setup.md#deploy-docker-image-container-registry

• Target dir: /srv/service-provider/container-registry

• Cleanup an existing deployment if required:

  cd /srv/service-provider/container-registry
  
  # Stop the deployment
  laconic-so deployment --dir container-registry stop --delete-volumes
  
  # Remove the deployment dir
  sudo rm -rf container-registrty
  
  # Remove the existing spec file
  rm container-registry.spec
  

Setup

• Generate the spec file for the container-registry stack

  laconic-so --stack container-registry deploy init --output container-registry.spec
  

• Modify the container-registry.spec as shown below

  stack: container-registry
  deploy-to: k8s
  kube-config: /home/dev/.kube/config-vs-narwhal.yaml
  network:
    ports:
      registry:
      - '5000'
    http-proxy:
      - host-name: container-registry.apps.vaasl.io
        routes:
          - path: '/'
            proxy-to: registry:5000
  volumes:
    registry-data:
  configmaps:
    config: ./configmaps/config
  

• Create the deployment directory for the container-registry stack

  laconic-so --stack container-registry deploy create --deployment-dir container-registry --spec-file container-registry.spec
  

• Modify file container-registry/kubeconfig.yml if required

  apiVersion: v1
  ...
  contexts:
    - context:
        cluster: ***
        user: ***
      name: default
  ...
  

  NOTE: context.name must be default to use with SO

• Base64 encode the container registry credentials
  NOTE: Use actual credentials for container registry (credentials set in container-registry/credentials.txt)

  echo -n "so-reg-user:pXDwO5zLU7M88x3aA" | base64 -w0
  
  # Output: c28tcmVnLXVzZXI6cFhEd081ekxVN004OHgzYUE=
  

• Install apache2-utils for next step

  sudo apt install apache2-utils
  

• Encrypt the container registry credentials to create an htpasswd file

  htpasswd -bB -c container-registry/configmaps/config/htpasswd so-reg-user pXDwO5zLU7M88x3aA
  

  Resulting file should look like this

  cat container-registry/configmaps/config/htpasswd
  # so-reg-user:$2y$05$6EdxIwwDNlJfNhhQxZRr4eNd.aYrdmbBjAdw422w0u2j3TihQXgd2
  

• Using the credentials from the previous steps, create a container-registry/my_password.json file

  {
    "auths": {
      "container-registry.apps.vaasl.io": {
        "username": "so-reg-user",
        "password": "$2y$05$6EdxIwwDNlJfNhhQxZRr4eNd.aYrdmbBjAdw422w0u2j3TihQXgd2",
        "auth": "c28tcmVnLXVzZXI6cFhEd081ekxVN004OHgzYUE="
      }
    }
  }
  

• Configure the file container-registry/config.env as follows

  REGISTRY_AUTH=htpasswd
  REGISTRY_AUTH_HTPASSWD_REALM="VSL Service Provider Image Registry"
  REGISTRY_AUTH_HTPASSWD_PATH="/config/htpasswd"
  REGISTRY_HTTP_SECRET='$2y$05$6EdxIwwDNlJfNhhQxZRr4eNd.aYrdmbBjAdw422w0u2j3TihQXgd2'
  

• Load context for k8s

  kubie ctx vs-narwhal
  

• Add the container registry credentials as a secret available to the cluster

  kubectl create secret generic laconic-registry --from-file=.dockerconfigjson=container-registry/my_password.json --type=kubernetes.io/dockerconfigjson
  

Run

• Deploy the container registry

  laconic-so deployment --dir container-registry start
  

• Check the logs

  laconic-so deployment --dir container-registry logs
  

• Check status and await succesful deployment:

  laconic-so deployment --dir container-registry status
  

• Confirm deployment by logging in:

  docker login container-registry.apps.vaasl.io --username so-reg-user --password pXDwO5zLU7M88x3aA
  

• Set ingress annotations

  • Set the cluster-id found in container-registry/deployment.yml and then run the following commands:

    export CLUSTER_ID=<cluster-id>
    # Example
    # export CLUSTER_ID=laconic-26cc70be8a3db3f4
    
    kubectl annotate ingress $CLUSTER_ID-ingress nginx.ingress.kubernetes.io/proxy-body-size=0
    kubectl annotate ingress $CLUSTER_ID-ingress nginx.ingress.kubernetes.io/proxy-read-timeout=600
    kubectl annotate ingress $CLUSTER_ID-ingress nginx.ingress.kubernetes.io/proxy-send-timeout=600
    

webapp-deployer

Backend

• Reference: https://github.com/LaconicNetwork/loro-testnet/blob/main/docs/service-provider-setup.md#deploy-backend

• Target dir: /srv/service-provider/webapp-deployer

• Cleanup an existing deployment if required:

  cd /srv/service-provider/webapp-deployer
  
  # Stop the deployment
  laconic-so deployment --dir webapp-deployer stop
  
  # Remove the deployment dir
  sudo rm -rf webapp-deployer
  
  # Remove the existing spec file
  rm webapp-deployer.spec
  

Setup

• Initialize a spec file for the deployer backend.

  laconic-so --stack webapp-deployer-backend setup-repositories
  laconic-so --stack webapp-deployer-backend build-containers
  laconic-so --stack webapp-deployer-backend deploy init --output webapp-deployer.spec
  

• Modify the contents of webapp-deployer.spec:

  stack: webapp-deployer-backend
  deploy-to: k8s
  kube-config: /home/dev/.kube/config-vs-narwhal.yaml
  image-registry: container-registry.apps.vaasl.io/laconic-registry
  network:
    ports:
      server:
        - '9555'
    http-proxy:
      - host-name: webapp-deployer-api.apps.vaasl.io
        routes:
          - path: '/'
            proxy-to: server:9555
  volumes:
    srv:
  configmaps:
    config: ./data/config
  annotations:
    container.apparmor.security.beta.kubernetes.io/{name}: unconfined
  labels:
    container.kubeaudit.io/{name}.allow-disabled-apparmor: "podman"
  security:
    privileged: true
  
  resources:
    containers:
      reservations:
        cpus: 3
        memory: 8G
      limits:
        cpus: 7
        memory: 16G
    volumes:
      reservations:
        storage: 200G
  

• Create the deployment directory from the spec file.

  laconic-so --stack webapp-deployer-backend deploy create --deployment-dir webapp-deployer --spec-file webapp-deployer.spec
  

• Modify file webapp-deployer/kubeconfig.yml if required

  apiVersion: v1
  ...
  contexts:
    - context:
        cluster: ***
        user: ***
      name: default
  ...
  

  NOTE: context.name must be default to use with SO

• Copy webapp-deployer/kubeconfig.yml from the k8s cluster creation step to webapp-deployer/data/config/kube.yml

  cp webapp-deployer/kubeconfig.yml webapp-deployer/data/config/kube.yml
  

• Create webapp-deployer/data/config/laconic.yml, it should look like this:

  services:
    registry:
      # Using public endpoint does not work inside machine where laconicd chain is deployed
      rpcEndpoint: 'http://host.docker.internal:36657'
      gqlEndpoint: 'http://host.docker.internal:3473/api'
  
      # Set user key of account with balance and bond owned by the user
      userKey:
      bondId:
  
      chainId: laconic-testnet-2
      gasPrice: 1alnt
  

  NOTE: Modify the user key and bond ID according to your configuration

• Publish a WebappDeployer record for the deployer backend by following the steps below:

  • Setup GPG keys by following these steps to create and export a key

    cd webapp-deployer
    
    # Create a key
    gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.apps.vaasl.io default default never
    
    # Export the public key
    gpg --export webapp-deployer-api.apps.vaasl.io > webapp-deployer-api.apps.vaasl.io.pgp.pub
    
    # Export the private key
    gpg --export-secret-keys webapp-deployer-api.apps.vaasl.io > webapp-deployer-api.apps.vaasl.io.pgp.key
    
    cd -
    

    NOTE: Use "SECRET" for passphrase prompt

  • Copy the GPG pub key file generated above to webapp-deployer/data/config directory. This ensures the Docker container has access to the key during the publish process

    cp webapp-deployer/webapp-deployer-api.apps.vaasl.io.pgp.pub webapp-deployer/data/config
    

  • Publish the webapp deployer record using the publish-deployer-to-registry command

    docker run -i -t \
      --add-host=host.docker.internal:host-gateway \
      -v /srv/service-provider/webapp-deployer/data/config:/config \
      cerc/webapp-deployer-backend:local laconic-so publish-deployer-to-registry \
        --laconic-config /config/laconic.yml \
        --api-url https://webapp-deployer-api.apps.vaasl.io \
        --public-key-file /config/webapp-deployer-api.apps.vaasl.io.pgp.pub \
        --lrn lrn://vaasl-provider/deployers/webapp-deployer-api.apps.vaasl.io \
        --min-required-payment 10000
    

• Modify the contents of webapp-deployer/config.env:

  DEPLOYMENT_DNS_SUFFIX="apps.vaasl.io"
  
  # this should match the name authority reserved above
  DEPLOYMENT_RECORD_NAMESPACE="vaasl-provider"
  
  # url of the deployed docker image registry
  IMAGE_REGISTRY="container-registry.apps.vaasl.io"
  
  # credentials from the htpasswd section above in container-registry setup
  IMAGE_REGISTRY_USER=
  IMAGE_REGISTRY_CREDS=
  
  # configs
  CLEAN_DEPLOYMENTS=false
  CLEAN_LOGS=false
  CLEAN_CONTAINERS=false
  SYSTEM_PRUNE=false
  WEBAPP_IMAGE_PRUNE=true
  CHECK_INTERVAL=10
  FQDN_POLICY="allow"
  
  # lrn of the webapp deployer
  LRN="lrn://vaasl-provider/deployers/webapp-deployer-api.apps.vaasl.io"
  
  # Path to the GPG key file inside the webapp-deployer container
  OPENPGP_PRIVATE_KEY_FILE="webapp-deployer-api.apps.vaasl.io.pgp.key"
  # Passphrase used when creating the GPG key
  OPENPGP_PASSPHRASE="SECRET"
  
  DEPLOYER_STATE="srv-test/deployments/autodeploy.state"
  UNDEPLOYER_STATE="srv-test/deployments/autoundeploy.state"
  UPLOAD_DIRECTORY="srv-test/uploads"
  HANDLE_AUCTION_REQUESTS=true
  AUCTION_BID_AMOUNT=10000
  
  # Minimum payment amount required for single webapp deployment
  MIN_REQUIRED_PAYMENT=10000
  

• Push the image to the container registry

  laconic-so deployment --dir webapp-deployer push-images
  

• Modify webapp-deployer/data/config/laconic.yml:

  services:
    registry:
      rpcEndpoint: 'https://laconicd-sapo.laconic.com/'
      gqlEndpoint: 'https://laconicd-sapo.laconic.com/api'
  
      # Set user key of account with balance and bond owned by the user
      userKey:
      bondId:
  
      chainId: laconic-testnet-2
      gasPrice: 1alnt
  

Run

• Start the deployer.

  laconic-so deployment --dir webapp-deployer start
  

• Load context for k8s

  kubie ctx vs-narwhal
  

• Copy the GPG key file to the webapp-deployer container

  # Get the webapp-deployer pod id
  laconic-so deployment --dir webapp-deployer ps
  
  # Expected output
  # Running containers:
  # id: default/laconic-096fed46af974a47-deployment-644db859c7-snbq6, name: laconic-096fed46af974a47-deployment-644db859c7-snbq6, ports: 10.42.2.11:9555->9555
  
  # Set pod id
  export POD_ID=
  # Example:
  # export POD_ID=laconic-096fed46af974a47-deployment-644db859c7-snbq6
  
  # Copy GPG key files to the pod
  kubectl cp webapp-deployer/webapp-deployer-api.apps.vaasl.io.pgp.key $POD_ID:/app
  kubectl cp webapp-deployer/webapp-deployer-api.apps.vaasl.io.pgp.pub $POD_ID:/app
  

• Publishing records to the registry will trigger deployments in backend now

Frontend

• Target dir: /srv/service-provider/webapp-ui

• Cleanup an existing deployment if required:

  cd /srv/service-provider/webapp-ui
  
  # Stop the deployment
  laconic-so deployment --dir webapp-ui stop
  
  # Remove the deployment dir
  sudo rm -rf webapp-ui
  
  # Remove the existing spec file
  rm webapp-ui.spec
  

Setup

• Clone and build the deployer UI

  git clone https://git.vdb.to/cerc-io/webapp-deployment-status-ui.git ~/cerc/webapp-deployment-status-ui
  
  laconic-so build-webapp --source-repo ~/cerc/webapp-deployment-status-ui
  

• Create a deployment

  export KUBECONFIG_PATH=/home/dev/.kube/config-vs-narwhal.yaml
  # NOTE: Use actual kubeconfig path
  
  laconic-so deploy-webapp create --kube-config $KUBECONFIG_PATH --image-registry container-registry.apps.vaasl.io --deployment-dir webapp-ui --image cerc/webapp-deployment-status-ui:local --url https://webapp-deployer-ui.apps.vaasl.io --env-file ~/cerc/webapp-deployment-status-ui/.env
  

• Modify file webapp-ui/kubeconfig.yml if required

  apiVersion: v1
  ...
  contexts:
    - context:
        cluster: ***
        user: ***
      name: default
  ...
  

  NOTE: context.name must be default to use with SO

• Push the image to the container registry.

  laconic-so deployment --dir webapp-ui push-images
  

• Modify webapp-ui/config.env like this Pull Request but with your host details.

Run

• Start the deployer UI

  laconic-so deployment --dir webapp-ui start
  

• Wait a moment, then go to https://webapp-deployer-ui.apps.vaasl.io for the status and logs of each deployment