Add logs to debug why api is not working

Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75) and cerc-io/stack-orchestrator#948
Requires cerc-io/stack-orchestrator#950

Reviewed-on: cerc-io/webapp-deployment-status-api#19
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>

README.md

@@ -5,10 +5,10 @@ files used by those requests.
 
 ## Build and Run
 
-```
+```bash
-$ yarn
+yarn
-$ yarn build
+yarn build
-$ yarn start
+yarn start
 ```
 
 ## Configuration
@@ -20,22 +20,22 @@ On upload, the configuration is temporarily decrypted for validation, but stored
 
 To create and export a key in the necessary format use:
 
-```
+```bash
 # Create a key
-$ gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never
+gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never
 
 # Export the public key
-$ gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub
+gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub
 
 # Export the private key
-$ gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key
+gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key
 ```
 
 ### Create the Deployer Record
 
 Every webapp deployer should have `WebappDeployer` record in the registry which looks something like:
 
-```
+```yml
 record:
   type: WebappDeployer
   version: 1.0.0
@@ -48,16 +48,34 @@ record:
 
 This record can most easily be created using `laconic-so publish-deployer-to-registry`.
 
-```
+```bash
-$ laconic-so publish-deployer-to-registry \
+laconic-so publish-deployer-to-registry \
   --laconic-config ~/.laconic/registry.yml \
-  --api-url https://webapp-deployer-api.my.domain.com 
+  --api-url https://webapp-deployer-api.my.domain.com
   --public-key-file webapp-deployer-api.my.domain.com.pgp.pub \
   --lrn lrn://laconic/deployers/webapp-deployer-api.my.domain.com  \
   --min-required-payment 100
 ```
 
-This will create the record in the proper format and assign its LRN. 
+This will create the record in the proper format and assign its LRN.
+
+### Publish Deployment Auction
+
+Users can optionally create an auction for app deployment with desired number of providers and max price they are willing to pay for a deployment:
+
+```bash
+laconic-so publish-deployment-auction \
+  --laconic-config ./config.yml \
+  --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
+  --commits-duration 3600 \
+  --reveals-duration 3600 \
+  --commit-fee 10000 \
+  --reveal-fee 10000 \
+  --max-price 5000000 \
+  --num-providers 3
+```
+
+This will create a `provider` auction with given params and publish a deployment auction record.
 
 ### Request Deployment
 
@@ -67,11 +85,11 @@ Users can now request deployment using the LRN of the deployer.  This will allow
 1. Obtain the public key for encrypting config.
 1. See the minimum required payment.
 
-The request can be made using `laconic-so request-webapp-deployment`.  This will handle encrypting and uploading the
+The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the
 config automatically, as well as making a payment (if necessary).
 
-```
+```bash
-$ laconic-so request-webapp-deployment \
+laconic-so request-webapp-deployment \
   --laconic-config ~/.laconic/registry.yml \
   --deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
   --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
@@ -79,9 +97,33 @@ $ laconic-so request-webapp-deployment \
   --make-payment auto
 ```
 
+Alternatively, users can also use a deployment auction they created instead of making the payment to any specific deployer directly:
+
+```bash
+laconic-so request-webapp-deployment \
+  --laconic-config ~/.laconic/registry.yml \
+  --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
+  --env-file hello.env \
+  --auction-id 4c9701c22651e143202e991056b6e7649853acc5bc0e97e3a98e09c9f3355909
+```
+
+This creates deployment requests targeted towards all the deployers who have won the auction. Similar to requests with payments, the config is automatically encrypted and uploaded to all the deployers.
+
+### Request Undeployment
+
+Users can also request removal of an existing deployment using the deployment record id:
+
+```bash
+laconic-so request-webapp-undeployment \
+  --laconic-config ~/.laconic/registry.yml \
+  --deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
+  --deployment bafyreigeopr72dmp6rhvnomgdz3cljbqzhh75epcrigit7ue6i6vjullme \
+  --make-payment auto
+```
+
 ### Example Config
 
-```
+```bash
 UPLOAD_DIRECTORY="/srv/uploads/config"
 UPLOAD_MAX_SIZE="1MB"
 DEPLOYER_STATE="/srv/deployments/autodeploy.state"
@@ -90,4 +132,7 @@ BUILD_LOGS="/srv/logs"
 OPENPGP_PASSPHRASE="SECRET"
 OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key"
 LACONIC_CONFIG="/etc/config/registry.yml"
+LRN=lrn://laconic/deployers/webapp-deployer-api.my.domain.com
+HANDLE_AUCTION_REQUESTS=true
+AUCTION_BID_AMOUNT=50000
 ```

run.sh

@@ -35,6 +35,16 @@ if [ ! -f "/etc/config/kube.yml" ]; then
   exit 2
 fi
 
+AUCTION_OPTS=""
+if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
+  if [ -z "$AUCTION_BID_AMOUNT" ]; then
+    echo "AUCTION_BID_AMOUNT is required when handling auction requsts."
+    exit 2
+  fi
+
+  AUCTION_OPTS="--auction-requests"
+fi
+
 STORAGE_ROOT="${STORAGE_ROOT:-/srv}"
 DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}"
 LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}"
@@ -113,67 +123,82 @@ while true; do
     yarn start &
   fi
 
-  echo "########### UNDEPLOY ############"
+#   echo "########### UNDEPLOY ############"
-  laconic-so undeploy-webapp-from-registry \
+#   laconic-so undeploy-webapp-from-registry \
-    --laconic-config /etc/config/laconic.yml \
+#     --laconic-config /etc/config/laconic.yml \
-    --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
+#     --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
-    --delete-names \
+#     --delete-names \
-    --delete-volumes \
+#     --delete-volumes \
-    --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
+#     --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
-    --include-tags "$INCLUDE_TAGS" \
+#     --include-tags "$INCLUDE_TAGS" \
-    --exclude-tags "$EXCLUDE_TAGS" \
+#     --exclude-tags "$EXCLUDE_TAGS" \
-    --lrn "$LRN" \
+#     --lrn "$LRN" \
-    --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
+#     --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
-    $EXTRA_UNDEPLOY_OPTS \
+#     $EXTRA_UNDEPLOY_OPTS \
-    $UPDATE_OPTS \
+#     $UPDATE_OPTS \
-    --discover
+#     --discover
-  rc=$?
+#   rc=$?
-  if [ $rc -eq 0 ]; then
+#   if [ $rc -eq 0 ]; then
-    echo "############ UNDEPLOY SUCCESS #############"
+#     echo "############ UNDEPLOY SUCCESS #############"
-  else
+#   else
-    echo "############ UNDEPLOY FAILURE STATUS $rc #############"
+#     echo "############ UNDEPLOY FAILURE STATUS $rc #############"
-  fi
+#   fi
 
-  echo "############ DEPLOY #############"
+#   echo "############ DEPLOY #############"
-  laconic-so deploy-webapp-from-registry \
+#   laconic-so deploy-webapp-from-registry \
-    --kube-config /etc/config/kube.yml \
+#     --kube-config /etc/config/kube.yml \
-    --laconic-config /etc/config/laconic.yml \
+#     --laconic-config /etc/config/laconic.yml \
-    --image-registry ${IMAGE_REGISTRY} \
+#     --image-registry ${IMAGE_REGISTRY} \
-    --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
+#     --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
-    --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
+#     --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
-    --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
+#     --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
-    --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
+#     --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
-    --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
+#     --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
-    --include-tags "$INCLUDE_TAGS" \
+#     --include-tags "$INCLUDE_TAGS" \
-    --exclude-tags "$EXCLUDE_TAGS" \
+#     --exclude-tags "$EXCLUDE_TAGS" \
-    --fqdn-policy "${FQDN_POLICY:-prohibit}" \
+#     --fqdn-policy "${FQDN_POLICY:-prohibit}" \
-    --lrn "$LRN" \
+#     --lrn "$LRN" \
-    --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
+#     --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
-    --config-upload-dir "$UPLOAD_DIRECTORY" \
+#     --config-upload-dir "$UPLOAD_DIRECTORY" \
-    --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \
+#     --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \
-    --private-key-passphrase "$OPENPGP_PASSPHRASE" \
+#     --private-key-passphrase "$OPENPGP_PASSPHRASE" \
-    $LOG_OPTS \
+#     $AUCTION_OPTS \
-    $EXTRA_DEPLOY_OPTS \
+#     $LOG_OPTS \
-    $UPDATE_OPTS \
+#     $EXTRA_DEPLOY_OPTS \
-    --discover
+#     $UPDATE_OPTS \
-  rc=$?
+#     --discover
-  if [ $rc -eq 0 ]; then
+#   rc=$?
-    echo "############ DEPLOY SUCCESS #############"
+#   if [ $rc -eq 0 ]; then
-  else
+#     echo "############ DEPLOY SUCCESS #############"
-    echo "############ DEPLOY FAILURE STATUS $rc #############"
+#   else
-  fi
+#     echo "############ DEPLOY FAILURE STATUS $rc #############"
+#   fi
 
-  # Cleanup any build leftovers
+#   if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
-  if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
+#     echo "############ DEPLOYMENT AUCTION #############"
-    docker system prune --all --force
+#     laconic-so handle-deployment-auction \
-  fi
+#       --laconic-config /etc/config/laconic.yml \
-  if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
+#       --state-file "${DEPLOYMENTS_DIR}/autoauction.state" \
-    APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
+#       --bid-amount ${AUCTION_BID_AMOUNT}
-    DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
+#     rc=$?
-    if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
+#     if [ $rc -eq 0 ]; then
-      echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
+#       echo "############ DEPLOYMENT AUCTION SUCCESS #############"
-      docker image rm -f $APP_IMAGES $DANGLING_IMAGES
+#     else
-    fi
+#       echo "############ DEPLOYMENT AUCTION FAILURE STATUS $rc #############"
-  fi
+#     fi
+#   fi
+
+#   # Cleanup any build leftovers
+#   if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
+#     docker system prune --all --force
+#   fi
+#   if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
+#     APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
+#     DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
+#     if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
+#       echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
+#       docker image rm -f $APP_IMAGES $DANGLING_IMAGES
+#     fi
+#   fi
   sleep ${CHECK_INTERVAL:-15}
 done

src/deployments.ts

@@ -108,8 +108,12 @@ export class RegHelper {
   }
 
   async deploymentRequestStatus(requestId?: string) {
+    console.log('Starting deploymentRequestStatus with requestId:', requestId);
+
     const requests: any[] = [];
     const deployments: any[] = [];
+
+    // Querying for removal requests
     const removalRequests = await this.queryRecords({
       type: 'ApplicationDeploymentRemovalRequest',
     });
@@ -118,53 +122,75 @@ export class RegHelper {
       const request = await this.getRecordById(requestId);
       if (request) {
         requests.push(request);
+        console.log('Found request:', request);
+      } else {
+        console.log('Request not found for requestId:', requestId);
       }
-      deployments.push(...await this.queryRecords({
+
-        type: 'ApplicationDeploymentRecord', request: requestId
+      const foundDeployments = await this.queryRecords({
-      }));
-    } else {
-      requests.push(...await this.queryRecords({
-        type: 'ApplicationDeploymentRequest',
-      }));
-      deployments.push(...await this.queryRecords({
         type: 'ApplicationDeploymentRecord',
-      }));
+        request: requestId
+      });
+      deployments.push(...foundDeployments);
+    } else {
+      console.log('Fetching all ApplicationDeploymentRequests');
+      const allRequests = await this.queryRecords({
+        type: 'ApplicationDeploymentRequest',
+      });
+      requests.push(...allRequests);
+      console.log('All requests:', allRequests);
+
+      console.log('Fetching all ApplicationDeploymentRecords');
+      const allDeployments = await this.queryRecords({
+        type: 'ApplicationDeploymentRecord',
+      });
+      deployments.push(...allDeployments);
+      console.log('All deployments:', allDeployments);
     }
 
+    console.log('Sorting requests by createTime');
     requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,);
     requests.reverse();
+    console.log('Sorted requests:', requests);
 
     const deploymentsByRequest = new Map<string, any>();
     for (const d of deployments) {
       deploymentsByRequest.set(d.attributes.request, d);
     }
+    console.log('Deployments by request:', deploymentsByRequest);
+
     const removalsByRequest = new Map<string, any>();
     for (const rr of removalRequests) {
       if (rr.attributes.request) {
         removalsByRequest.set(rr.attributes.request, rr);
       }
     }
+    console.log('Removals by request:', removalsByRequest);
 
     const latestByHostname = new Map<string, any>();
 
     const ret = [];
     for (const r of requests) {
+      console.log('Processing request:', r.id);
       const status = new RequestStatus(r.id, r.createTime);
       ret.push(status);
 
       const app = await this.getRecord(r.attributes.application);
       if (!app) {
+        console.log('Error: Application not found for request:', r.id);
         status.lastState = 'ERROR';
         continue;
       }
 
       status.app = r.attributes.application;
       const hostname = r.attributes.dns ?? generateHostnameForApp(app);
+      console.log('Hostname for app:', hostname);
 
       if (deploymentsByRequest.has(r.id)) {
         const deployment = deploymentsByRequest.get(r.id);
         status.url = deployment.attributes.url;
         status.lastUpdate = deployment.createTime;
+        console.log('Deployment found for request:', r.id, 'with deployment:', deployment);
 
         if (!latestByHostname.has(hostname)) {
           latestByHostname.set(hostname, status);
@@ -179,11 +205,13 @@ export class RegHelper {
       }
 
       if (removalsByRequest.has(r.id)) {
+        console.log('Removal request found for request:', r.id);
         status.lastState = 'CANCELLED';
         continue;
       }
 
       if (latestByHostname.has(hostname)) {
+        console.log('Cancellation found for hostname:', hostname);
         status.lastState = 'CANCELLED';
         continue;
       }
@@ -191,6 +219,7 @@ export class RegHelper {
       latestByHostname.set(hostname, status);
     }
 
+    console.log('Final status array:', ret);
     return ret;
   }