deep-stack/webapp-deployment-status-api
3
0
Fork 0
forked from cerc-io/webapp-deployment-status-api
Code Pull Requests Activity

Compare commits

base: deep-stack:ag-debug-api
Branches Tags
deep-stack:main
deep-stack:pm-handle-multiple-dns
deep-stack:pm-without-mutex
deep-stack:ag-debug-api
deep-stack:zramsay-patch-1
cerc-io:zach/atom-payments
cerc-io:main
cerc-io:zramsay-patch-1
cerc-io:v0.1.0
..
compare: deep-stack:main
Branches Tags
deep-stack:pm-handle-multiple-dns
deep-stack:pm-without-mutex
deep-stack:ag-debug-api
deep-stack:main
deep-stack:zramsay-patch-1
cerc-io:zach/atom-payments
cerc-io:main
cerc-io:zramsay-patch-1
cerc-io:v0.1.0

No commits in common. "ag-debug-api" and "main" have entirely different histories.
ag-debug-a ... main

3 changed files with 84 additions and 183 deletions
Show Stats Expand all files Collapse all files

79
README.md
View File

@ -5,10 +5,10 @@ files used by those requests.
## Build and Run ## Build and Run
```bash ```
yarn $ yarn
yarn build $ yarn build
yarn start $ yarn start
``` ```
## Configuration ## Configuration
@ -20,22 +20,22 @@ On upload, the configuration is temporarily decrypted for validation, but stored
To create and export a key in the necessary format use: To create and export a key in the necessary format use:
```bash ```
# Create a key # Create a key
gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never $ gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never
# Export the public key # Export the public key
gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub $ gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub
# Export the private key # Export the private key
gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key $ gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key
``` ```
### Create the Deployer Record ### Create the Deployer Record
Every webapp deployer should have `WebappDeployer` record in the registry which looks something like: Every webapp deployer should have `WebappDeployer` record in the registry which looks something like:
```yml ```
record: record:
type: WebappDeployer type: WebappDeployer
version: 1.0.0 version: 1.0.0
@ -48,34 +48,16 @@ record:
This record can most easily be created using `laconic-so publish-deployer-to-registry`. This record can most easily be created using `laconic-so publish-deployer-to-registry`.
```bash ```
laconic-so publish-deployer-to-registry \ $ laconic-so publish-deployer-to-registry \
--laconic-config ~/.laconic/registry.yml \ --laconic-config ~/.laconic/registry.yml \
--api-url https://webapp-deployer-api.my.domain.com --api-url https://webapp-deployer-api.my.domain.com
--public-key-file webapp-deployer-api.my.domain.com.pgp.pub \ --public-key-file webapp-deployer-api.my.domain.com.pgp.pub \
--lrn lrn://laconic/deployers/webapp-deployer-api.my.domain.com \ --lrn lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--min-required-payment 100 --min-required-payment 100
``` ```
This will create the record in the proper format and assign its LRN. This will create the record in the proper format and assign its LRN.
### Publish Deployment Auction
Users can optionally create an auction for app deployment with desired number of providers and max price they are willing to pay for a deployment:
```bash
laconic-so publish-deployment-auction \
--laconic-config ./config.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--commits-duration 3600 \
--reveals-duration 3600 \
--commit-fee 10000 \
--reveal-fee 10000 \
--max-price 5000000 \
--num-providers 3
```
This will create a `provider` auction with given params and publish a deployment auction record.
### Request Deployment ### Request Deployment
@ -85,11 +67,11 @@ Users can now request deployment using the LRN of the deployer. This will allow
1. Obtain the public key for encrypting config. 1. Obtain the public key for encrypting config.
1. See the minimum required payment. 1. See the minimum required payment.
The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the
config automatically, as well as making a payment (if necessary). config automatically, as well as making a payment (if necessary).
```bash ```
laconic-so request-webapp-deployment \ $ laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \ --laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \ --deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \ --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
@ -97,33 +79,9 @@ laconic-so request-webapp-deployment \
--make-payment auto --make-payment auto
``` ```
Alternatively, users can also use a deployment auction they created instead of making the payment to any specific deployer directly:
```bash
laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--env-file hello.env \
--auction-id 4c9701c22651e143202e991056b6e7649853acc5bc0e97e3a98e09c9f3355909
```
This creates deployment requests targeted towards all the deployers who have won the auction. Similar to requests with payments, the config is automatically encrypted and uploaded to all the deployers.
### Request Undeployment
Users can also request removal of an existing deployment using the deployment record id:
```bash
laconic-so request-webapp-undeployment \
--laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--deployment bafyreigeopr72dmp6rhvnomgdz3cljbqzhh75epcrigit7ue6i6vjullme \
--make-payment auto
```
### Example Config ### Example Config
```bash ```
UPLOAD_DIRECTORY="/srv/uploads/config" UPLOAD_DIRECTORY="/srv/uploads/config"
UPLOAD_MAX_SIZE="1MB" UPLOAD_MAX_SIZE="1MB"
DEPLOYER_STATE="/srv/deployments/autodeploy.state" DEPLOYER_STATE="/srv/deployments/autodeploy.state"
@ -132,7 +90,4 @@ BUILD_LOGS="/srv/logs"
OPENPGP_PASSPHRASE="SECRET" OPENPGP_PASSPHRASE="SECRET"
OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key" OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key"
LACONIC_CONFIG="/etc/config/registry.yml" LACONIC_CONFIG="/etc/config/registry.yml"
LRN=lrn://laconic/deployers/webapp-deployer-api.my.domain.com
HANDLE_AUCTION_REQUESTS=true
AUCTION_BID_AMOUNT=50000
``` ```

145
run.sh
View File

@ -35,16 +35,6 @@ if [ ! -f "/etc/config/kube.yml" ]; then
exit 2 exit 2
fi fi
AUCTION_OPTS=""
if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
if [ -z "$AUCTION_BID_AMOUNT" ]; then
echo "AUCTION_BID_AMOUNT is required when handling auction requsts."
exit 2
fi
AUCTION_OPTS="--auction-requests"
fi
STORAGE_ROOT="${STORAGE_ROOT:-/srv}" STORAGE_ROOT="${STORAGE_ROOT:-/srv}"
DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}" DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}"
LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}" LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}"
@ -123,82 +113,67 @@ while true; do
yarn start & yarn start &
fi fi
# echo "########### UNDEPLOY ############" echo "########### UNDEPLOY ############"
# laconic-so undeploy-webapp-from-registry \ laconic-so undeploy-webapp-from-registry \
# --laconic-config /etc/config/laconic.yml \ --laconic-config /etc/config/laconic.yml \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \ --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --delete-names \ --delete-names \
# --delete-volumes \ --delete-volumes \
# --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \ --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
# --include-tags "$INCLUDE_TAGS" \ --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \ --exclude-tags "$EXCLUDE_TAGS" \
# --lrn "$LRN" \ --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \ --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# $EXTRA_UNDEPLOY_OPTS \ $EXTRA_UNDEPLOY_OPTS \
# $UPDATE_OPTS \ $UPDATE_OPTS \
# --discover --discover
# rc=$? rc=$?
# if [ $rc -eq 0 ]; then if [ $rc -eq 0 ]; then
# echo "############ UNDEPLOY SUCCESS #############" echo "############ UNDEPLOY SUCCESS #############"
# else else
# echo "############ UNDEPLOY FAILURE STATUS $rc #############" echo "############ UNDEPLOY FAILURE STATUS $rc #############"
# fi fi
# echo "############ DEPLOY #############" echo "############ DEPLOY #############"
# laconic-so deploy-webapp-from-registry \ laconic-so deploy-webapp-from-registry \
# --kube-config /etc/config/kube.yml \ --kube-config /etc/config/kube.yml \
# --laconic-config /etc/config/laconic.yml \ --laconic-config /etc/config/laconic.yml \
# --image-registry ${IMAGE_REGISTRY} \ --image-registry ${IMAGE_REGISTRY} \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \ --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \ --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
# --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \ --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
# --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \ --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
# --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \ --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
# --include-tags "$INCLUDE_TAGS" \ --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \ --exclude-tags "$EXCLUDE_TAGS" \
# --fqdn-policy "${FQDN_POLICY:-prohibit}" \ --fqdn-policy "${FQDN_POLICY:-prohibit}" \
# --lrn "$LRN" \ --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \ --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# --config-upload-dir "$UPLOAD_DIRECTORY" \ --config-upload-dir "$UPLOAD_DIRECTORY" \
# --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \ --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \
# --private-key-passphrase "$OPENPGP_PASSPHRASE" \ --private-key-passphrase "$OPENPGP_PASSPHRASE" \
# $AUCTION_OPTS \ $LOG_OPTS \
# $LOG_OPTS \ $EXTRA_DEPLOY_OPTS \
# $EXTRA_DEPLOY_OPTS \ $UPDATE_OPTS \
# $UPDATE_OPTS \ --discover
# --discover rc=$?
# rc=$? if [ $rc -eq 0 ]; then
# if [ $rc -eq 0 ]; then echo "############ DEPLOY SUCCESS #############"
# echo "############ DEPLOY SUCCESS #############" else
# else echo "############ DEPLOY FAILURE STATUS $rc #############"
# echo "############ DEPLOY FAILURE STATUS $rc #############" fi
# fi
# if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then # Cleanup any build leftovers
# echo "############ DEPLOYMENT AUCTION #############" if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
# laconic-so handle-deployment-auction \ docker system prune --all --force
# --laconic-config /etc/config/laconic.yml \ fi
# --state-file "${DEPLOYMENTS_DIR}/autoauction.state" \ if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
# --bid-amount ${AUCTION_BID_AMOUNT} APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
# rc=$? DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
# if [ $rc -eq 0 ]; then if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
# echo "############ DEPLOYMENT AUCTION SUCCESS #############" echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
# else docker image rm -f $APP_IMAGES $DANGLING_IMAGES
# echo "############ DEPLOYMENT AUCTION FAILURE STATUS $rc #############" fi
# fi fi
# fi
# # Cleanup any build leftovers
# if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
# docker system prune --all --force
# fi
# if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
# APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
# DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
# if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
# echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
# docker image rm -f $APP_IMAGES $DANGLING_IMAGES
# fi
# fi
sleep ${CHECK_INTERVAL:-15} sleep ${CHECK_INTERVAL:-15}
done done

43
src/deployments.ts
View File

@ -108,12 +108,8 @@ export class RegHelper {
} }
async deploymentRequestStatus(requestId?: string) { async deploymentRequestStatus(requestId?: string) {
console.log('Starting deploymentRequestStatus with requestId:', requestId);
const requests: any[] = []; const requests: any[] = [];
const deployments: any[] = []; const deployments: any[] = [];
// Querying for removal requests
const removalRequests = await this.queryRecords({ const removalRequests = await this.queryRecords({
type: 'ApplicationDeploymentRemovalRequest', type: 'ApplicationDeploymentRemovalRequest',
}); });
@ -122,75 +118,53 @@ export class RegHelper {
const request = await this.getRecordById(requestId); const request = await this.getRecordById(requestId);
if (request) { if (request) {
requests.push(request); requests.push(request);
console.log('Found request:', request);
} else {
console.log('Request not found for requestId:', requestId);
} }
deployments.push(...await this.queryRecords({
const foundDeployments = await this.queryRecords({ type: 'ApplicationDeploymentRecord', request: requestId
type: 'ApplicationDeploymentRecord', }));
request: requestId
});
deployments.push(...foundDeployments);
} else { } else {
console.log('Fetching all ApplicationDeploymentRequests'); requests.push(...await this.queryRecords({
const allRequests = await this.queryRecords({
type: 'ApplicationDeploymentRequest', type: 'ApplicationDeploymentRequest',
}); }));
requests.push(...allRequests); deployments.push(...await this.queryRecords({
console.log('All requests:', allRequests);
console.log('Fetching all ApplicationDeploymentRecords');
const allDeployments = await this.queryRecords({
type: 'ApplicationDeploymentRecord', type: 'ApplicationDeploymentRecord',
}); }));
deployments.push(...allDeployments);
console.log('All deployments:', allDeployments);
} }
console.log('Sorting requests by createTime');
requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,); requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,);
requests.reverse(); requests.reverse();
console.log('Sorted requests:', requests);
const deploymentsByRequest = new Map<string, any>(); const deploymentsByRequest = new Map<string, any>();
for (const d of deployments) { for (const d of deployments) {
deploymentsByRequest.set(d.attributes.request, d); deploymentsByRequest.set(d.attributes.request, d);
} }
console.log('Deployments by request:', deploymentsByRequest);
const removalsByRequest = new Map<string, any>(); const removalsByRequest = new Map<string, any>();
for (const rr of removalRequests) { for (const rr of removalRequests) {
if (rr.attributes.request) { if (rr.attributes.request) {
removalsByRequest.set(rr.attributes.request, rr); removalsByRequest.set(rr.attributes.request, rr);
} }
} }
console.log('Removals by request:', removalsByRequest);
const latestByHostname = new Map<string, any>(); const latestByHostname = new Map<string, any>();
const ret = []; const ret = [];
for (const r of requests) { for (const r of requests) {
console.log('Processing request:', r.id);
const status = new RequestStatus(r.id, r.createTime); const status = new RequestStatus(r.id, r.createTime);
ret.push(status); ret.push(status);
const app = await this.getRecord(r.attributes.application); const app = await this.getRecord(r.attributes.application);
if (!app) { if (!app) {
console.log('Error: Application not found for request:', r.id);
status.lastState = 'ERROR'; status.lastState = 'ERROR';
continue; continue;
} }
status.app = r.attributes.application; status.app = r.attributes.application;
const hostname = r.attributes.dns ?? generateHostnameForApp(app); const hostname = r.attributes.dns ?? generateHostnameForApp(app);
console.log('Hostname for app:', hostname);
if (deploymentsByRequest.has(r.id)) { if (deploymentsByRequest.has(r.id)) {
const deployment = deploymentsByRequest.get(r.id); const deployment = deploymentsByRequest.get(r.id);
status.url = deployment.attributes.url; status.url = deployment.attributes.url;
status.lastUpdate = deployment.createTime; status.lastUpdate = deployment.createTime;
console.log('Deployment found for request:', r.id, 'with deployment:', deployment);
if (!latestByHostname.has(hostname)) { if (!latestByHostname.has(hostname)) {
latestByHostname.set(hostname, status); latestByHostname.set(hostname, status);
@ -205,13 +179,11 @@ export class RegHelper {
} }
if (removalsByRequest.has(r.id)) { if (removalsByRequest.has(r.id)) {
console.log('Removal request found for request:', r.id);
status.lastState = 'CANCELLED'; status.lastState = 'CANCELLED';
continue; continue;
} }
if (latestByHostname.has(hostname)) { if (latestByHostname.has(hostname)) {
console.log('Cancellation found for hostname:', hostname);
status.lastState = 'CANCELLED'; status.lastState = 'CANCELLED';
continue; continue;
} }
@ -219,7 +191,6 @@ export class RegHelper {
latestByHostname.set(hostname, status); latestByHostname.set(hostname, status);
} }
console.log('Final status array:', ret);
return ret; return ret;
} }