deep-stack/webapp-deployment-status-api
3
0
Fork 0
forked from cerc-io/webapp-deployment-status-api
Code Pull Requests Activity

Compare commits

base: deep-stack:ag-debug-api
Branches Tags
deep-stack:main
deep-stack:pm-handle-multiple-dns
deep-stack:pm-without-mutex
deep-stack:ag-debug-api
deep-stack:zramsay-patch-1
cerc-io:zach/atom-payments
cerc-io:main
cerc-io:zramsay-patch-1
cerc-io:v0.1.0
..
compare: deep-stack:main
Branches Tags
deep-stack:pm-handle-multiple-dns
deep-stack:pm-without-mutex
deep-stack:ag-debug-api
deep-stack:main
deep-stack:zramsay-patch-1
cerc-io:zach/atom-payments
cerc-io:main
cerc-io:zramsay-patch-1
cerc-io:v0.1.0

No commits in common. "ag-debug-api" and "main" have entirely different histories.
ag-debug-a ... main

3 changed files with 84 additions and 183 deletions
Show Stats Expand all files Collapse all files

73
README.md
View File

@ -5,10 +5,10 @@ files used by those requests.
## Build and Run ## Build and Run
```bash ```
yarn $ yarn
yarn build $ yarn build
yarn start $ yarn start
``` ```
## Configuration ## Configuration
@ -20,22 +20,22 @@ On upload, the configuration is temporarily decrypted for validation, but stored
To create and export a key in the necessary format use: To create and export a key in the necessary format use:
```bash ```
# Create a key # Create a key
gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never $ gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never
# Export the public key # Export the public key
gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub $ gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub
# Export the private key # Export the private key
gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key $ gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key
``` ```
### Create the Deployer Record ### Create the Deployer Record
Every webapp deployer should have `WebappDeployer` record in the registry which looks something like: Every webapp deployer should have `WebappDeployer` record in the registry which looks something like:
```yml ```
record: record:
type: WebappDeployer type: WebappDeployer
version: 1.0.0 version: 1.0.0
@ -48,8 +48,8 @@ record:
This record can most easily be created using `laconic-so publish-deployer-to-registry`. This record can most easily be created using `laconic-so publish-deployer-to-registry`.
```bash ```
laconic-so publish-deployer-to-registry \ $ laconic-so publish-deployer-to-registry \
--laconic-config ~/.laconic/registry.yml \ --laconic-config ~/.laconic/registry.yml \
--api-url https://webapp-deployer-api.my.domain.com --api-url https://webapp-deployer-api.my.domain.com
--public-key-file webapp-deployer-api.my.domain.com.pgp.pub \ --public-key-file webapp-deployer-api.my.domain.com.pgp.pub \
@ -59,24 +59,6 @@ laconic-so publish-deployer-to-registry \
This will create the record in the proper format and assign its LRN. This will create the record in the proper format and assign its LRN.
### Publish Deployment Auction
Users can optionally create an auction for app deployment with desired number of providers and max price they are willing to pay for a deployment:
```bash
laconic-so publish-deployment-auction \
--laconic-config ./config.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--commits-duration 3600 \
--reveals-duration 3600 \
--commit-fee 10000 \
--reveal-fee 10000 \
--max-price 5000000 \
--num-providers 3
```
This will create a `provider` auction with given params and publish a deployment auction record.
### Request Deployment ### Request Deployment
Users can now request deployment using the LRN of the deployer. This will allow them to: Users can now request deployment using the LRN of the deployer. This will allow them to:
@ -88,8 +70,8 @@ Users can now request deployment using the LRN of the deployer. This will allow
The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the
config automatically, as well as making a payment (if necessary). config automatically, as well as making a payment (if necessary).
```bash ```
laconic-so request-webapp-deployment \ $ laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \ --laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \ --deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \ --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
@ -97,33 +79,9 @@ laconic-so request-webapp-deployment \
--make-payment auto --make-payment auto
``` ```
Alternatively, users can also use a deployment auction they created instead of making the payment to any specific deployer directly:
```bash
laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--env-file hello.env \
--auction-id 4c9701c22651e143202e991056b6e7649853acc5bc0e97e3a98e09c9f3355909
```
This creates deployment requests targeted towards all the deployers who have won the auction. Similar to requests with payments, the config is automatically encrypted and uploaded to all the deployers.
### Request Undeployment
Users can also request removal of an existing deployment using the deployment record id:
```bash
laconic-so request-webapp-undeployment \
--laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--deployment bafyreigeopr72dmp6rhvnomgdz3cljbqzhh75epcrigit7ue6i6vjullme \
--make-payment auto
```
### Example Config ### Example Config
```bash ```
UPLOAD_DIRECTORY="/srv/uploads/config" UPLOAD_DIRECTORY="/srv/uploads/config"
UPLOAD_MAX_SIZE="1MB" UPLOAD_MAX_SIZE="1MB"
DEPLOYER_STATE="/srv/deployments/autodeploy.state" DEPLOYER_STATE="/srv/deployments/autodeploy.state"
@ -132,7 +90,4 @@ BUILD_LOGS="/srv/logs"
OPENPGP_PASSPHRASE="SECRET" OPENPGP_PASSPHRASE="SECRET"
OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key" OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key"
LACONIC_CONFIG="/etc/config/registry.yml" LACONIC_CONFIG="/etc/config/registry.yml"
LRN=lrn://laconic/deployers/webapp-deployer-api.my.domain.com
HANDLE_AUCTION_REQUESTS=true
AUCTION_BID_AMOUNT=50000
``` ```

145
run.sh
View File

@ -35,16 +35,6 @@ if [ ! -f "/etc/config/kube.yml" ]; then
exit 2 exit 2
fi fi
AUCTION_OPTS=""
if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
if [ -z "$AUCTION_BID_AMOUNT" ]; then
echo "AUCTION_BID_AMOUNT is required when handling auction requsts."
exit 2
fi
AUCTION_OPTS="--auction-requests"
fi
STORAGE_ROOT="${STORAGE_ROOT:-/srv}" STORAGE_ROOT="${STORAGE_ROOT:-/srv}"
DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}" DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}"
LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}" LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}"
@ -123,82 +113,67 @@ while true; do
yarn start & yarn start &
fi fi
# echo "########### UNDEPLOY ############" echo "########### UNDEPLOY ############"
# laconic-so undeploy-webapp-from-registry \ laconic-so undeploy-webapp-from-registry \
# --laconic-config /etc/config/laconic.yml \ --laconic-config /etc/config/laconic.yml \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \ --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --delete-names \ --delete-names \
# --delete-volumes \ --delete-volumes \
# --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \ --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
# --include-tags "$INCLUDE_TAGS" \ --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \ --exclude-tags "$EXCLUDE_TAGS" \
# --lrn "$LRN" \ --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \ --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# $EXTRA_UNDEPLOY_OPTS \ $EXTRA_UNDEPLOY_OPTS \
# $UPDATE_OPTS \ $UPDATE_OPTS \
# --discover --discover
# rc=$? rc=$?
# if [ $rc -eq 0 ]; then if [ $rc -eq 0 ]; then
# echo "############ UNDEPLOY SUCCESS #############" echo "############ UNDEPLOY SUCCESS #############"
# else else
# echo "############ UNDEPLOY FAILURE STATUS $rc #############" echo "############ UNDEPLOY FAILURE STATUS $rc #############"
# fi fi
# echo "############ DEPLOY #############" echo "############ DEPLOY #############"
# laconic-so deploy-webapp-from-registry \ laconic-so deploy-webapp-from-registry \
# --kube-config /etc/config/kube.yml \ --kube-config /etc/config/kube.yml \
# --laconic-config /etc/config/laconic.yml \ --laconic-config /etc/config/laconic.yml \
# --image-registry ${IMAGE_REGISTRY} \ --image-registry ${IMAGE_REGISTRY} \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \ --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \ --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
# --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \ --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
# --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \ --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
# --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \ --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
# --include-tags "$INCLUDE_TAGS" \ --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \ --exclude-tags "$EXCLUDE_TAGS" \
# --fqdn-policy "${FQDN_POLICY:-prohibit}" \ --fqdn-policy "${FQDN_POLICY:-prohibit}" \
# --lrn "$LRN" \ --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \ --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# --config-upload-dir "$UPLOAD_DIRECTORY" \ --config-upload-dir "$UPLOAD_DIRECTORY" \
# --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \ --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \
# --private-key-passphrase "$OPENPGP_PASSPHRASE" \ --private-key-passphrase "$OPENPGP_PASSPHRASE" \
# $AUCTION_OPTS \ $LOG_OPTS \
# $LOG_OPTS \ $EXTRA_DEPLOY_OPTS \
# $EXTRA_DEPLOY_OPTS \ $UPDATE_OPTS \
# $UPDATE_OPTS \ --discover
# --discover rc=$?
# rc=$? if [ $rc -eq 0 ]; then
# if [ $rc -eq 0 ]; then echo "############ DEPLOY SUCCESS #############"
# echo "############ DEPLOY SUCCESS #############" else
# else echo "############ DEPLOY FAILURE STATUS $rc #############"
# echo "############ DEPLOY FAILURE STATUS $rc #############" fi
# fi
# if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then # Cleanup any build leftovers
# echo "############ DEPLOYMENT AUCTION #############" if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
# laconic-so handle-deployment-auction \ docker system prune --all --force
# --laconic-config /etc/config/laconic.yml \ fi
# --state-file "${DEPLOYMENTS_DIR}/autoauction.state" \ if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
# --bid-amount ${AUCTION_BID_AMOUNT} APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
# rc=$? DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
# if [ $rc -eq 0 ]; then if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
# echo "############ DEPLOYMENT AUCTION SUCCESS #############" echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
# else docker image rm -f $APP_IMAGES $DANGLING_IMAGES
# echo "############ DEPLOYMENT AUCTION FAILURE STATUS $rc #############" fi
# fi fi
# fi
# # Cleanup any build leftovers
# if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
# docker system prune --all --force
# fi
# if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
# APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
# DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
# if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
# echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
# docker image rm -f $APP_IMAGES $DANGLING_IMAGES
# fi
# fi
sleep ${CHECK_INTERVAL:-15} sleep ${CHECK_INTERVAL:-15}
done done

43
src/deployments.ts
View File

@ -108,12 +108,8 @@ export class RegHelper {
} }
async deploymentRequestStatus(requestId?: string) { async deploymentRequestStatus(requestId?: string) {
console.log('Starting deploymentRequestStatus with requestId:', requestId);
const requests: any[] = []; const requests: any[] = [];
const deployments: any[] = []; const deployments: any[] = [];
// Querying for removal requests
const removalRequests = await this.queryRecords({ const removalRequests = await this.queryRecords({
type: 'ApplicationDeploymentRemovalRequest', type: 'ApplicationDeploymentRemovalRequest',
}); });
@ -122,75 +118,53 @@ export class RegHelper {
const request = await this.getRecordById(requestId); const request = await this.getRecordById(requestId);
if (request) { if (request) {
requests.push(request); requests.push(request);
console.log('Found request:', request);
} else {
console.log('Request not found for requestId:', requestId);
} }
deployments.push(...await this.queryRecords({
const foundDeployments = await this.queryRecords({ type: 'ApplicationDeploymentRecord', request: requestId
type: 'ApplicationDeploymentRecord', }));
request: requestId
});
deployments.push(...foundDeployments);
} else { } else {
console.log('Fetching all ApplicationDeploymentRequests'); requests.push(...await this.queryRecords({
const allRequests = await this.queryRecords({
type: 'ApplicationDeploymentRequest', type: 'ApplicationDeploymentRequest',
}); }));
requests.push(...allRequests); deployments.push(...await this.queryRecords({
console.log('All requests:', allRequests);
console.log('Fetching all ApplicationDeploymentRecords');
const allDeployments = await this.queryRecords({
type: 'ApplicationDeploymentRecord', type: 'ApplicationDeploymentRecord',
}); }));
deployments.push(...allDeployments);
console.log('All deployments:', allDeployments);
} }
console.log('Sorting requests by createTime');
requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,); requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,);
requests.reverse(); requests.reverse();
console.log('Sorted requests:', requests);
const deploymentsByRequest = new Map<string, any>(); const deploymentsByRequest = new Map<string, any>();
for (const d of deployments) { for (const d of deployments) {
deploymentsByRequest.set(d.attributes.request, d); deploymentsByRequest.set(d.attributes.request, d);
} }
console.log('Deployments by request:', deploymentsByRequest);
const removalsByRequest = new Map<string, any>(); const removalsByRequest = new Map<string, any>();
for (const rr of removalRequests) { for (const rr of removalRequests) {
if (rr.attributes.request) { if (rr.attributes.request) {
removalsByRequest.set(rr.attributes.request, rr); removalsByRequest.set(rr.attributes.request, rr);
} }
} }
console.log('Removals by request:', removalsByRequest);
const latestByHostname = new Map<string, any>(); const latestByHostname = new Map<string, any>();
const ret = []; const ret = [];
for (const r of requests) { for (const r of requests) {
console.log('Processing request:', r.id);
const status = new RequestStatus(r.id, r.createTime); const status = new RequestStatus(r.id, r.createTime);
ret.push(status); ret.push(status);
const app = await this.getRecord(r.attributes.application); const app = await this.getRecord(r.attributes.application);
if (!app) { if (!app) {
console.log('Error: Application not found for request:', r.id);
status.lastState = 'ERROR'; status.lastState = 'ERROR';
continue; continue;
} }
status.app = r.attributes.application; status.app = r.attributes.application;
const hostname = r.attributes.dns ?? generateHostnameForApp(app); const hostname = r.attributes.dns ?? generateHostnameForApp(app);
console.log('Hostname for app:', hostname);
if (deploymentsByRequest.has(r.id)) { if (deploymentsByRequest.has(r.id)) {
const deployment = deploymentsByRequest.get(r.id); const deployment = deploymentsByRequest.get(r.id);
status.url = deployment.attributes.url; status.url = deployment.attributes.url;
status.lastUpdate = deployment.createTime; status.lastUpdate = deployment.createTime;
console.log('Deployment found for request:', r.id, 'with deployment:', deployment);
if (!latestByHostname.has(hostname)) { if (!latestByHostname.has(hostname)) {
latestByHostname.set(hostname, status); latestByHostname.set(hostname, status);
@ -205,13 +179,11 @@ export class RegHelper {
} }
if (removalsByRequest.has(r.id)) { if (removalsByRequest.has(r.id)) {
console.log('Removal request found for request:', r.id);
status.lastState = 'CANCELLED'; status.lastState = 'CANCELLED';
continue; continue;
} }
if (latestByHostname.has(hostname)) { if (latestByHostname.has(hostname)) {
console.log('Cancellation found for hostname:', hostname);
status.lastState = 'CANCELLED'; status.lastState = 'CANCELLED';
continue; continue;
} }
@ -219,7 +191,6 @@ export class RegHelper {
latestByHostname.set(hostname, status); latestByHostname.set(hostname, status);
} }
console.log('Final status array:', ret);
return ret; return ret;
} }