Author SHA1 Message Date
zramsay b32dd3a8cd add token to env. 2024-08-15 00:01:09 +00:00
10 changed files with 161 additions and 460 deletions
+2 -1
View File
@@ -9,4 +9,5 @@ CLEAN_CONTAINERS=false
SYSTEM_PRUNE=false
WEBAPP_IMAGE_PRUNE=true
CHECK_INTERVAL=5
FQDN_POLICY="allow"
FQDN_POLICY="allow"
DEPLOYER_GITHUB_TOKEN="optional, set for deploying private repos"
+90 -115
View File
@@ -1,138 +1,113 @@
# webapp-deployment-status-api
# node-typescript-boilerplate
This API provides status information about webapp deployment requests and a mechanism for upload encrypted configuration
files used by those requests.
[![Sponsor][sponsor-badge]][sponsor]
[![TypeScript version][ts-badge]][typescript-5-3]
[![Node.js version][nodejs-badge]][nodejs]
[![APLv2][license-badge]][license]
[![Build Status - GitHub Actions][gha-badge]][gha-ci]
## Build and Run
👩🏻‍💻 Developer Ready: A comprehensive template. Works out of the box for most [Node.js][nodejs] projects.
```bash
yarn
yarn build
yarn start
🏃🏽 Instant Value: All basic tools included and configured:
- [TypeScript][typescript] [5.3][typescript-5-3]
- [ESM][esm]
- [ESLint][eslint] with some initial rules recommendation
- [Jest][jest] for fast unit testing and code coverage
- Type definitions for Node.js and Jest
- [Prettier][prettier] to enforce consistent code style
- NPM [scripts](#available-scripts) for common operations
- [EditorConfig][editorconfig] for consistent coding style
- Reproducible environments thanks to [Volta][volta]
- Example configuration for [GitHub Actions][gh-actions]
- Simple example of TypeScript code and unit test
🤲 Free as in speech: available under the APLv2 license.
## Getting Started
This project is intended to be used with the latest Active LTS release of [Node.js][nodejs].
### Use as a repository template
To start, just click the **[Use template][repo-template-action]** link (or the green button). Start adding your code in the `src` and unit tests in the `__tests__` directories.
### Clone repository
To clone the repository, use the following commands:
```sh
git clone https://github.com/jsynowiec/node-typescript-boilerplate
cd node-typescript-boilerplate
npm install
```
## Configuration
### Download latest release
### Keys
Download and unzip the current **main** branch or one of the tags:
Configuration files are encrypted prior to being uploaded using an RSA `publicKey` specified in the `WebappDeployer` record.
On upload, the configuration is temporarily decrypted for validation, but stored in its encrypted format.
To create and export a key in the necessary format use:
```bash
# Create a key
gpg --batch --passphrase "SECRET" --quick-generate-key webapp-deployer-api.my.domain.com default default never
# Export the public key
gpg --export webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.pub
# Export the private key
gpg --export-secret-keys webapp-deployer-api.my.domain.com > webapp-deployer-api.my.domain.com.pgp.key
```sh
wget https://github.com/jsynowiec/node-typescript-boilerplate/archive/main.zip -O node-typescript-boilerplate.zip
unzip node-typescript-boilerplate.zip && rm node-typescript-boilerplate.zip
```
### Create the Deployer Record
## Available Scripts
Every webapp deployer should have `WebappDeployer` record in the registry which looks something like:
- `clean` - remove coverage data, Jest cache and transpiled files,
- `prebuild` - lint source files and tests before building,
- `build` - transpile TypeScript to ES6,
- `build:watch` - interactive watch mode to automatically transpile source files,
- `lint` - lint source files and tests,
- `prettier` - reformat files,
- `test` - run tests,
- `test:watch` - interactive watch mode to automatically re-run tests
```yml
record:
type: WebappDeployer
version: 1.0.0
name: webapp-deployer-api.my.domain.com
apiUrl: https://webapp-deployer-api.my.domain.com
minimumPayment: 100alnt
paymentAddress: laconic1clpc8smrhx5k25zmk3vwna8kddxrsem7a1jlry
publicKey: mQGNBGbJUk0BDAC3j3CiaVtoEf1jrgtsjJnTA5u1a3BExP72mv0eE8y84TgY5rVcf ...
```
## Additional Information
This record can most easily be created using `laconic-so publish-deployer-to-registry`.
### Why include Volta
```bash
laconic-so publish-deployer-to-registry \
--laconic-config ~/.laconic/registry.yml \
--api-url https://webapp-deployer-api.my.domain.com
--public-key-file webapp-deployer-api.my.domain.com.pgp.pub \
--lrn lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--min-required-payment 100
```
[Volta][volta]s toolchain always keeps track of where you are, it makes sure the tools you use always respect the settings of the project youre working on. This means you dont have to worry about changing the state of your installed software when switching between projects. For example, it's [used by engineers at LinkedIn][volta-tomdale] to standardize tools and have reproducible development environments.
This will create the record in the proper format and assign its LRN.
I recommend to [install][volta-getting-started] Volta and use it to manage your project's toolchain.
### Publish Deployment Auction
### ES Modules
Users can optionally create an auction for app deployment with desired number of providers and max price they are willing to pay for a deployment:
This template uses native [ESM][esm]. Make sure to read [this][nodejs-esm], and [this][ts47-esm] first.
```bash
laconic-so publish-deployment-auction \
--laconic-config ./config.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--commits-duration 3600 \
--reveals-duration 3600 \
--commit-fee 10000 \
--reveal-fee 10000 \
--max-price 5000000 \
--num-providers 3
```
If your project requires CommonJS, you will have to [convert to ESM][sindresorhus-esm].
This will create a `provider` auction with given params and publish a deployment auction record.
Please do not open issues for questions regarding CommonJS or ESM on this repo.
### Request Deployment
## Backers & Sponsors
Users can now request deployment using the LRN of the deployer. This will allow them to:
Support this project by becoming a [sponsor][sponsor].
1. Discover the API URL for config uploads.
1. Obtain the public key for encrypting config.
1. See the minimum required payment.
## License
The request can be made using `laconic-so request-webapp-deployment`. This will handle encrypting and uploading the
config automatically, as well as making a payment (if necessary).
Licensed under the APLv2. See the [LICENSE](https://github.com/jsynowiec/node-typescript-boilerplate/blob/main/LICENSE) file for details.
```bash
laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--env-file hello.env \
--make-payment auto
```
Alternatively, users can also use a deployment auction they created instead of making the payment to any specific deployer directly:
```bash
laconic-so request-webapp-deployment \
--laconic-config ~/.laconic/registry.yml \
--app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
--env-file hello.env \
--auction-id 4c9701c22651e143202e991056b6e7649853acc5bc0e97e3a98e09c9f3355909
```
This creates deployment requests targeted towards all the deployers who have won the auction. Similar to requests with payments, the config is automatically encrypted and uploaded to all the deployers.
### Request Undeployment
Users can also request removal of an existing deployment using the deployment record id:
```bash
laconic-so request-webapp-undeployment \
--laconic-config ~/.laconic/registry.yml \
--deployer lrn://laconic/deployers/webapp-deployer-api.my.domain.com \
--deployment bafyreigeopr72dmp6rhvnomgdz3cljbqzhh75epcrigit7ue6i6vjullme \
--make-payment auto
```
### Example Config
```bash
UPLOAD_DIRECTORY="/srv/uploads/config"
UPLOAD_MAX_SIZE="1MB"
DEPLOYER_STATE="/srv/deployments/autodeploy.state"
UNDEPLOYER_STATE="/srv/deployments/autoundeploy.state"
BUILD_LOGS="/srv/logs"
OPENPGP_PASSPHRASE="SECRET"
OPENPGP_PRIVATE_KEY_FILE="/etc/config/webapp-deployer-api.my.domain.com.pgp.key"
LACONIC_CONFIG="/etc/config/registry.yml"
LRN=lrn://laconic/deployers/webapp-deployer-api.my.domain.com
HANDLE_AUCTION_REQUESTS=true
AUCTION_BID_AMOUNT=50000
```
[ts-badge]: https://img.shields.io/badge/TypeScript-5.3-blue.svg
[nodejs-badge]: https://img.shields.io/badge/Node.js->=%2020.9-blue.svg
[nodejs]: https://nodejs.org/dist/latest-v20.x/docs/api/
[gha-badge]: https://github.com/jsynowiec/node-typescript-boilerplate/actions/workflows/nodejs.yml/badge.svg
[gha-ci]: https://github.com/jsynowiec/node-typescript-boilerplate/actions/workflows/nodejs.yml
[typescript]: https://www.typescriptlang.org/
[typescript-5-3]: https://devblogs.microsoft.com/typescript/announcing-typescript-5-3/
[license-badge]: https://img.shields.io/badge/license-APLv2-blue.svg
[license]: https://github.com/jsynowiec/node-typescript-boilerplate/blob/main/LICENSE
[sponsor-badge]: https://img.shields.io/badge/♥-Sponsor-fc0fb5.svg
[sponsor]: https://github.com/sponsors/jsynowiec
[jest]: https://facebook.github.io/jest/
[eslint]: https://github.com/eslint/eslint
[wiki-js-tests]: https://github.com/jsynowiec/node-typescript-boilerplate/wiki/Unit-tests-in-plain-JavaScript
[prettier]: https://prettier.io
[volta]: https://volta.sh
[volta-getting-started]: https://docs.volta.sh/guide/getting-started
[volta-tomdale]: https://twitter.com/tomdale/status/1162017336699838467
[gh-actions]: https://github.com/features/actions
[repo-template-action]: https://github.com/jsynowiec/node-typescript-boilerplate/generate
[esm]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules
[sindresorhus-esm]: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
[nodejs-esm]: https://nodejs.org/docs/latest-v16.x/api/esm.html
[ts47-esm]: https://devblogs.microsoft.com/typescript/announcing-typescript-4-7/#esm-nodejs
[editorconfig]: https://editorconfig.org
+2 -7
View File
@@ -37,19 +37,14 @@
"license": "Apache-2.0",
"dependencies": {
"@cerc-io/laconic-sdk": "^0.1.15",
"@openpgp/web-stream-tools": "^0.1.3",
"body-parser": "^1.20.2",
"express": "^4.18.2",
"express-async-handler": "^1.2.0",
"express-serve-static-core": "^0.1.1",
"js-yaml": "^4.1.0",
"json-stable-stringify": "^1.1.1",
"openpgp": "^5.11.2",
"tslib": "~2.6",
"yaml": "^2.5.0"
"tslib": "~2.6"
},
"volta": {
"node": "20.10.0"
},
"packageManager": "yarn@1.22.22+sha1.ac34549e6aa8e7ead463a7407e1c7390f61a6610"
}
}
+41 -89
View File
@@ -35,16 +35,6 @@ if [ ! -f "/etc/config/kube.yml" ]; then
exit 2
fi
AUCTION_OPTS=""
if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
if [ -z "$AUCTION_BID_AMOUNT" ]; then
echo "AUCTION_BID_AMOUNT is required when handling auction requsts."
exit 2
fi
AUCTION_OPTS="--auction-requests"
fi
STORAGE_ROOT="${STORAGE_ROOT:-/srv}"
DEPLOYMENTS_DIR="${DEPLOYMENTS_DIR:-$STORAGE_ROOT/deployments}"
LOG_DIR="${LOG_DIR:-$STORAGE_ROOT/logs}"
@@ -81,10 +71,6 @@ if [[ "$CLEAN_LOGS" == "true" ]] && [[ -n "$LOG_DIR" ]]; then
rm -rf ${LOG_DIR}/*
fi
if [[ ! -d "${UPLOAD_DIRECTORY}" ]]; then
mkdir -p "${UPLOAD_DIRECTORY}"
fi
STORAGE_DRIVER="${STORAGE_DRIVER}"
if [[ -z "${STORAGE_DRIVER}" ]]; then
if [[ "true" == "`is_privileged`" ]]; then
@@ -123,82 +109,48 @@ while true; do
yarn start &
fi
# echo "########### UNDEPLOY ############"
# laconic-so undeploy-webapp-from-registry \
# --laconic-config /etc/config/laconic.yml \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --delete-names \
# --delete-volumes \
# --state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
# --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \
# --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# $EXTRA_UNDEPLOY_OPTS \
# $UPDATE_OPTS \
# --discover
# rc=$?
# if [ $rc -eq 0 ]; then
# echo "############ UNDEPLOY SUCCESS #############"
# else
# echo "############ UNDEPLOY FAILURE STATUS $rc #############"
# fi
echo "########### UNDEPLOY ############"
laconic-so undeploy-webapp-from-registry \
--laconic-config /etc/config/laconic.yml \
--deployment-parent-dir "${DEPLOYMENTS_DIR}" \
--delete-names \
--delete-volumes \
--state-file "${DEPLOYMENTS_DIR}/autoremove.state" \
--include-tags "$INCLUDE_TAGS" \
--exclude-tags "$EXCLUDE_TAGS" \
$EXTRA_UNDEPLOY_OPTS \
$UPDATE_OPTS \
--discover
# echo "############ DEPLOY #############"
# laconic-so deploy-webapp-from-registry \
# --kube-config /etc/config/kube.yml \
# --laconic-config /etc/config/laconic.yml \
# --image-registry ${IMAGE_REGISTRY} \
# --deployment-parent-dir "${DEPLOYMENTS_DIR}" \
# --dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
# --record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
# --record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
# --state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
# --include-tags "$INCLUDE_TAGS" \
# --exclude-tags "$EXCLUDE_TAGS" \
# --fqdn-policy "${FQDN_POLICY:-prohibit}" \
# --lrn "$LRN" \
# --min-required-payment ${MIN_REQUIRED_PAYMENT:-0} \
# --config-upload-dir "$UPLOAD_DIRECTORY" \
# --private-key-file "$OPENPGP_PRIVATE_KEY_FILE" \
# --private-key-passphrase "$OPENPGP_PASSPHRASE" \
# $AUCTION_OPTS \
# $LOG_OPTS \
# $EXTRA_DEPLOY_OPTS \
# $UPDATE_OPTS \
# --discover
# rc=$?
# if [ $rc -eq 0 ]; then
# echo "############ DEPLOY SUCCESS #############"
# else
# echo "############ DEPLOY FAILURE STATUS $rc #############"
# fi
echo "############ DEPLOY #############"
laconic-so deploy-webapp-from-registry \
--kube-config /etc/config/kube.yml \
--laconic-config /etc/config/laconic.yml \
--image-registry ${IMAGE_REGISTRY} \
--deployment-parent-dir "${DEPLOYMENTS_DIR}" \
--dns-suffix ${DEPLOYMENT_DNS_SUFFIX} \
--record-namespace-dns lrn://${DEPLOYMENT_RECORD_NAMESPACE}/dns \
--record-namespace-deployments lrn://${DEPLOYMENT_RECORD_NAMESPACE}/deployments \
--state-file "${DEPLOYMENTS_DIR}/autodeploy.state" \
--include-tags "$INCLUDE_TAGS" \
--exclude-tags "$EXCLUDE_TAGS" \
--fqdn-policy "${FQDN_POLICY:-prohibit}" \
$LOG_OPTS \
$EXTRA_DEPLOY_OPTS \
$UPDATE_OPTS \
--discover
# if [ "$HANDLE_AUCTION_REQUESTS" = "true" ]; then
# echo "############ DEPLOYMENT AUCTION #############"
# laconic-so handle-deployment-auction \
# --laconic-config /etc/config/laconic.yml \
# --state-file "${DEPLOYMENTS_DIR}/autoauction.state" \
# --bid-amount ${AUCTION_BID_AMOUNT}
# rc=$?
# if [ $rc -eq 0 ]; then
# echo "############ DEPLOYMENT AUCTION SUCCESS #############"
# else
# echo "############ DEPLOYMENT AUCTION FAILURE STATUS $rc #############"
# fi
# fi
# # Cleanup any build leftovers
# if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
# docker system prune --all --force
# fi
# if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
# APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
# DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
# if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
# echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
# docker image rm -f $APP_IMAGES $DANGLING_IMAGES
# fi
# fi
# Cleanup any build leftovers
if [[ "${SYSTEM_PRUNE:-false}" == "true" ]]; then
docker system prune --all --force
fi
if [[ "${WEBAPP_IMAGE_PRUNE:-true}" == "true" ]]; then
APP_IMAGES="$(docker image ls --quiet --filter 'reference=laconic-webapp')"
DANGLING_IMAGES="$(docker image ls --quiet --filter 'dangling=true')"
if [[ -n "$APP_IMAGES" ]] || [[ -n "$DANGLING_IMAGES" ]]; then
echo "Pruning images: $APP_IMAGES $DANGLING_IMAGES"
docker image rm -f $APP_IMAGES $DANGLING_IMAGES
fi
fi
sleep ${CHECK_INTERVAL:-15}
done
-4
View File
@@ -13,15 +13,11 @@ export const Config = {
LISTEN_PORT: parseInt(process.env.LISTEN_PORT || '9555'),
LISTEN_ADDR: process.env.LISTEN_ADDR || '0.0.0.0',
LACONIC_CONFIG: process.env.LACONIC_CONFIG || '/etc/config/laconic.yml',
UPLOAD_DIRECTORY: process.env.UPLOAD_DIRECTORY || '/srv/uploads',
DEPLOYER_STATE:
process.env.DEPLOYER_STATE || '/srv/deployments/autodeploy.state',
UNDEPLOYER_STATE:
process.env.UNDEPLOYER_STATE || '/srv/deployments/autoundeploy.state',
BUILD_LOGS: process.env.BUILD_LOGS || '/srv/logs',
UPLOAD_MAX_SIZE: process.env.BUILD_LOGS || '1MB',
OPENPGP_PASSPHRASE: process.env.OPENPGP_PASSPHRASE,
OPENPGP_PRIVATE_KEY_FILE: process.env.OPENPGP_PRIVATE_KEY_FILE,
};
export const getRegistry = (): Registry => {
+19 -48
View File
@@ -108,12 +108,8 @@ export class RegHelper {
}
async deploymentRequestStatus(requestId?: string) {
console.log('Starting deploymentRequestStatus with requestId:', requestId);
const requests: any[] = [];
const deployments: any[] = [];
// Querying for removal requests
const removalRequests = await this.queryRecords({
type: 'ApplicationDeploymentRemovalRequest',
});
@@ -122,78 +118,49 @@ export class RegHelper {
const request = await this.getRecordById(requestId);
if (request) {
requests.push(request);
console.log('Found request:', request);
} else {
console.log('Request not found for requestId:', requestId);
}
const foundDeployments = await this.queryRecords({
type: 'ApplicationDeploymentRecord',
request: requestId
});
deployments.push(...foundDeployments);
deployments.push(...await this.queryRecords({
type: 'ApplicationDeploymentRecord', request: requestId
}));
} else {
console.log('Fetching all ApplicationDeploymentRequests');
const allRequests = await this.queryRecords({
requests.push(...await this.queryRecords({
type: 'ApplicationDeploymentRequest',
});
requests.push(...allRequests);
console.log('All requests:', allRequests);
console.log('Fetching all ApplicationDeploymentRecords');
const allDeployments = await this.queryRecords({
}));
deployments.push(...await this.queryRecords({
type: 'ApplicationDeploymentRecord',
});
deployments.push(...allDeployments);
console.log('All deployments:', allDeployments);
}));
}
console.log('Sorting requests by createTime');
requests.sort((a, b) => a.createTime === b.createTime ? 0 : a.createTime > b.createTime ? 1 : -1,);
requests.reverse();
console.log('Sorted requests:', requests);
const deploymentsByRequest = new Map<string, any>();
for (const d of deployments) {
deploymentsByRequest.set(d.attributes.request, d);
}
console.log('Deployments by request:', deploymentsByRequest);
const removalsByRequest = new Map<string, any>();
for (const rr of removalRequests) {
if (rr.attributes.request) {
removalsByRequest.set(rr.attributes.request, rr);
}
}
console.log('Removals by request:', removalsByRequest);
const latestByHostname = new Map<string, any>();
const ret = [];
for (const r of requests) {
console.log('Processing request:', r.id);
const status = new RequestStatus(r.id, r.createTime);
ret.push(status);
const app = await this.getRecord(r.attributes.application);
if (!app) {
console.log('Error: Application not found for request:', r.id);
status.lastState = 'ERROR';
continue;
}
status.app = r.attributes.application;
const hostname = r.attributes.dns ?? generateHostnameForApp(app);
console.log('Hostname for app:', hostname);
if (deploymentsByRequest.has(r.id)) {
const deployment = deploymentsByRequest.get(r.id);
status.url = deployment.attributes.url;
status.lastUpdate = deployment.createTime;
console.log('Deployment found for request:', r.id, 'with deployment:', deployment);
if (!latestByHostname.has(hostname)) {
latestByHostname.set(hostname, status);
const shortHost = new URL(status.url).host.split('.').shift();
if (!latestByHostname.has(shortHost)) {
latestByHostname.set(shortHost, status);
}
status.deployment = deployment.names ? deployment.names[0] : null;
if (status.deployment) {
@@ -205,21 +172,25 @@ export class RegHelper {
}
if (removalsByRequest.has(r.id)) {
console.log('Removal request found for request:', r.id);
status.lastState = 'CANCELLED';
continue;
}
if (latestByHostname.has(hostname)) {
console.log('Cancellation found for hostname:', hostname);
const app = await this.getRecord(r.attributes.application);
if (!app) {
status.lastState = 'ERROR';
continue;
}
const shortHost = r.attributes.dns ?? generateHostnameForApp(app);
if (latestByHostname.has(shortHost)) {
status.lastState = 'CANCELLED';
continue;
}
latestByHostname.set(hostname, status);
latestByHostname.set(shortHost, status);
}
console.log('Final status array:', ret);
return ret;
}
-17
View File
@@ -1,18 +1,13 @@
import express from 'express';
import bodyParser from 'body-parser';
import {existsSync, readdirSync, readFileSync} from 'fs';
import {Config} from './config.js';
import {RegHelper} from './deployments.js';
import { Uploader } from './upload.js';
const app = express();
app.use(express.json());
const configUploader = new Uploader(Config.UPLOAD_DIRECTORY);
const configUploadParser = bodyParser.raw({limit: Config.UPLOAD_MAX_SIZE, type: "*/*"})
app.use(function (_req, res, next) {
res.header('Access-Control-Allow-Origin', '*');
res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept',);
@@ -100,18 +95,6 @@ app.get('/:id/log', async (req, res) => {
}
});
app.post('/upload/config', configUploadParser, async (req, res) => {
try {
const id = await configUploader.upload(req.body);
res.json({
id
});
} catch (e) {
console.error(e);
res.sendStatus(500);
}
});
// deprecated
app.get('/log/:id', async (req, res) => {
try {
-91
View File
@@ -1,91 +0,0 @@
import crypto from 'crypto';
import fs from 'fs';
import assert from 'node:assert';
import openpgp from 'openpgp';
import YAML from 'yaml';
import { atob } from 'node:buffer';
import { Config } from './config.js';
let privateKey: openpgp.PrivateKey | null = null;
const loadPrivateKey = async () => {
if (null == privateKey) {
privateKey = await openpgp.decryptKey({
privateKey: await openpgp.readPrivateKey({
binaryKey: fs.readFileSync(Config.OPENPGP_PRIVATE_KEY_FILE)
}),
passphrase: Config.OPENPGP_PASSPHRASE,
});
}
return privateKey;
}
const randomId = (): string =>
crypto
.randomUUID({ disableEntropyCache: true })
.replaceAll('-', '')
.toUpperCase();
const validateConfig = (obj): undefined => {
assert(obj.authorized, "'authorized' is required");
assert(Array.isArray(obj.authorized), "'authorized' must be an array");
assert(obj.authorized.length >= 1, "'authorized' cannot be empty");
assert(obj.config, "'config' is required");
};
export const b64ToBytes = (base64): Uint8Array => {
const binaryString = atob(base64);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return bytes;
};
const decrypt = async (binaryMessage: Uint8Array): Promise<any> => {
const message = await openpgp.readMessage({
binaryMessage,
});
const { data } = await openpgp.decrypt({
message,
decryptionKeys: await loadPrivateKey(),
});
const config = data.toString();
return config.charAt(0) === '{' ? JSON.parse(config) : YAML.parse(config);
};
export class Uploader {
directory: string;
constructor(dir: string) {
this.directory = dir;
}
async upload(body: string | Uint8Array): Promise<string> {
let raw: any;
try {
raw = b64ToBytes(body);
} catch {
raw = body;
}
// We decrypt only to make sure the content is valid.
// Once we know it is good, we want to store the encrypted copy.
const obj = await decrypt(raw);
validateConfig(obj);
let id: string;
let destination: string;
do {
id = randomId();
destination = `${this.directory}/${id}`;
} while (fs.existsSync(destination));
console.log(`Wrote config to: ${destination}`);
fs.writeFileSync(destination, raw);
return id;
}
}
+1 -2
View File
@@ -17,8 +17,7 @@
"noUnusedParameters": true,
"noImplicitAny": false,
"noImplicitThis": false,
"strictNullChecks": false,
"skipLibCheck": true
"strictNullChecks": false
},
"include": ["src/**/*", "__tests__/**/*"]
}
+6 -86
View File
@@ -1146,11 +1146,6 @@
resolved "https://registry.yarnpkg.com/@octetstream/promisify/-/promisify-2.0.2.tgz#29ac3bd7aefba646db670227f895d812c1a19615"
integrity sha512-7XHoRB61hxsz8lBQrjC1tq/3OEIgpvGWg6DKAdwi7WRzruwkmsdwmOoUXbU4Dtd4RSOMDwed0SkP3y8UlMt1Bg==
"@openpgp/web-stream-tools@^0.1.3":
version "0.1.3"
resolved "https://registry.yarnpkg.com/@openpgp/web-stream-tools/-/web-stream-tools-0.1.3.tgz#a9750f12a634b5a15e711b6c1de559511fb53732"
integrity sha512-mT/ds43cH6c+AO5RFpxs+LkACr7KjC3/dZWHrP6KPrWJu4uJ/XJ+p7telaoYiqUfdjiiIvdNSOfhezW9fkmboQ==
"@pkgjs/parseargs@^0.11.0":
version "0.11.0"
resolved "https://registry.yarnpkg.com/@pkgjs/parseargs/-/parseargs-0.11.0.tgz#a77ea742fab25775145434eb1d2328cf5013ac33"
@@ -1662,16 +1657,6 @@ array-union@^2.1.0:
resolved "https://registry.yarnpkg.com/array-union/-/array-union-2.1.0.tgz#b798420adbeb1de828d84acd8a2e23d3efe85e8d"
integrity sha512-HGyxoOTYUyCM6stUe6EJgnd4EoewAI7zMdfqO+kGjnlZmBDz/cR5pf8r/cR4Wq60sL/p0IkcjUEEPwS3GFrIyw==
asn1.js@^5.0.0:
version "5.4.1"
resolved "https://registry.yarnpkg.com/asn1.js/-/asn1.js-5.4.1.tgz#11a980b84ebb91781ce35b0fdc2ee294e3783f07"
integrity sha512-+I//4cYPccV8LdmBLiX8CYvf9Sp3vQsrqu2QNXRcrbiWvcx/UdlFiqUJJzxRQxgsZmvhXhn4cSKeSmoFjVdupA==
dependencies:
bn.js "^4.0.0"
inherits "^2.0.1"
minimalistic-assert "^1.0.0"
safer-buffer "^2.1.0"
axios@^0.26.1:
version "0.26.1"
resolved "https://registry.yarnpkg.com/axios/-/axios-0.26.1.tgz#1ede41c51fcf51bbbd6fd43669caaa4f0495aaa9"
@@ -1802,7 +1787,7 @@ blakejs@^1.1.0:
resolved "https://registry.yarnpkg.com/blakejs/-/blakejs-1.2.1.tgz#5057e4206eadb4a97f7c0b6e197a505042fc3814"
integrity sha512-QXUSXI3QVc/gJME0dBpXrag1kbzOqCjCX8/b54ntNyW6sjtoqxqRk3LTmXzaJoh71zMsDCjM+47jS7XiwN/+fQ==
bn.js@^4.0.0, bn.js@^4.11.0, bn.js@^4.11.8, bn.js@^4.11.9:
bn.js@^4.11.0, bn.js@^4.11.8, bn.js@^4.11.9:
version "4.12.0"
resolved "https://registry.yarnpkg.com/bn.js/-/bn.js-4.12.0.tgz#775b3f278efbb9718eec7361f483fb36fbbfea88"
integrity sha512-c98Bf3tPniI+scsdk237ku1Dc3ujXQTSgyiPUDEOe7tRkhrqridvh8klBv0HCEso1OLOYcHuCv/cS6DNxKH+ZA==
@@ -1830,24 +1815,6 @@ body-parser@1.20.1:
type-is "~1.6.18"
unpipe "1.0.0"
body-parser@^1.20.2:
version "1.20.2"
resolved "https://registry.yarnpkg.com/body-parser/-/body-parser-1.20.2.tgz#6feb0e21c4724d06de7ff38da36dad4f57a747fd"
integrity sha512-ml9pReCu3M61kGlqoTm2umSXTlRTuGTx0bfYj+uIUKKYycG5NtSbeetV3faSU6R7ajOPw0g/J1PvK4qNy7s5bA==
dependencies:
bytes "3.1.2"
content-type "~1.0.5"
debug "2.6.9"
depd "2.0.0"
destroy "1.2.0"
http-errors "2.0.0"
iconv-lite "0.4.24"
on-finished "2.4.1"
qs "6.11.0"
raw-body "2.5.2"
type-is "~1.6.18"
unpipe "1.0.0"
brace-expansion@^1.1.7:
version "1.1.11"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.11.tgz#3c7fcbf529d87226f3d2f52b966ff5271eb441dd"
@@ -2084,7 +2051,7 @@ content-disposition@0.5.4:
dependencies:
safe-buffer "5.2.1"
content-type@~1.0.4, content-type@~1.0.5:
content-type@~1.0.4:
version "1.0.5"
resolved "https://registry.yarnpkg.com/content-type/-/content-type-1.0.5.tgz#8b773162656d1d1086784c8f23a54ce6d73d7918"
integrity sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==
@@ -3949,13 +3916,6 @@ onetime@^5.1.2:
dependencies:
mimic-fn "^2.1.0"
openpgp@^5.11.2:
version "5.11.2"
resolved "https://registry.yarnpkg.com/openpgp/-/openpgp-5.11.2.tgz#2c035a26b13feb3b0bb5180718ec91c8e65cc686"
integrity sha512-f8dJFVLwdkvPvW3VPFs6q9Vs2+HNhdvwls7a/MIFcQUB+XiQzRe7alfa3RtwfGJU7oUDDMAWPZ0nYsHa23Az+A==
dependencies:
asn1.js "^5.0.0"
optionator@^0.9.3:
version "0.9.3"
resolved "https://registry.yarnpkg.com/optionator/-/optionator-0.9.3.tgz#007397d44ed1872fdc6ed31360190f81814e2c64"
@@ -4199,16 +4159,6 @@ raw-body@2.5.1:
iconv-lite "0.4.24"
unpipe "1.0.0"
raw-body@2.5.2:
version "2.5.2"
resolved "https://registry.yarnpkg.com/raw-body/-/raw-body-2.5.2.tgz#99febd83b90e08975087e8f1f9419a149366b68a"
integrity sha512-8zGqypfENjCIqGhgXToC8aB2r7YrBX+AQAfIPs/Mlk+BtPTztOvTS01NRW/3Eh60J+a48lt8qsCzirQ6loCVfA==
dependencies:
bytes "3.1.2"
http-errors "2.0.0"
iconv-lite "0.4.24"
unpipe "1.0.0"
react-is@^18.0.0:
version "18.2.0"
resolved "https://registry.yarnpkg.com/react-is/-/react-is-18.2.0.tgz#199431eeaaa2e09f86427efbb4f1473edb47609b"
@@ -4322,7 +4272,7 @@ safe-buffer@5.2.1, safe-buffer@^5.0.1, safe-buffer@^5.1.0, safe-buffer@^5.1.1, s
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.2.1.tgz#1eaf9fa9bdb1fdd4ec75f58f9cdb4e6b7827eec6"
integrity sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==
"safer-buffer@>= 2.1.2 < 3", safer-buffer@^2.1.0:
"safer-buffer@>= 2.1.2 < 3":
version "2.1.2"
resolved "https://registry.yarnpkg.com/safer-buffer/-/safer-buffer-2.1.2.tgz#44fa161b0187b9549dd84bb91802f9bd8385cd6a"
integrity sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==
@@ -4514,16 +4464,7 @@ string-length@^4.0.1:
char-regex "^1.0.2"
strip-ansi "^6.0.0"
"string-width-cjs@npm:string-width@^4.2.0":
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
dependencies:
emoji-regex "^8.0.0"
is-fullwidth-code-point "^3.0.0"
strip-ansi "^6.0.1"
string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
"string-width-cjs@npm:string-width@^4.2.0", string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
@@ -4548,14 +4489,7 @@ string_decoder@^1.1.1:
dependencies:
safe-buffer "~5.2.0"
"strip-ansi-cjs@npm:strip-ansi@^6.0.1":
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
dependencies:
ansi-regex "^5.0.1"
strip-ansi@^6.0.0, strip-ansi@^6.0.1:
"strip-ansi-cjs@npm:strip-ansi@^6.0.1", strip-ansi@^6.0.0, strip-ansi@^6.0.1:
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
@@ -4820,16 +4754,7 @@ wif@^2.0.6:
dependencies:
bs58check "<3.0.0"
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0":
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
dependencies:
ansi-styles "^4.0.0"
string-width "^4.1.0"
strip-ansi "^6.0.0"
wrap-ansi@^7.0.0:
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0", wrap-ansi@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
@@ -4880,11 +4805,6 @@ yallist@^4.0.0:
resolved "https://registry.yarnpkg.com/yallist/-/yallist-4.0.0.tgz#9bb92790d9c0effec63be73519e11a35019a3a72"
integrity sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==
yaml@^2.5.0:
version "2.5.0"
resolved "https://registry.yarnpkg.com/yaml/-/yaml-2.5.0.tgz#c6165a721cf8000e91c36490a41d7be25176cf5d"
integrity sha512-2wWLbGbYDiSqqIKoPjar3MPgB94ErzCtrNE1FdqGuaO0pi2JGjmE8aW8TDZwzU7vuxcGRdL/4gPQwQ7hD5AMSw==
yargs-parser@^21.0.1, yargs-parser@^21.1.1:
version "21.1.1"
resolved "https://registry.yarnpkg.com/yargs-parser/-/yargs-parser-21.1.1.tgz#9096bceebf990d21bb31fa9516e0ede294a77d35"