Archived
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1fa67db025 | ||
|
|
3af3915f15 |
@@ -48,5 +48,5 @@ jobs:
|
||||
run: |
|
||||
source /opt/bash-utils/cgroup-helper.sh
|
||||
join_cgroup
|
||||
CERC_SCRIPT_DEBUG=1 ./tests/database/run-test.sh
|
||||
./tests/database/run-test.sh
|
||||
|
||||
|
||||
+97
-104
@@ -15,139 +15,132 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
|
||||
16 GB Memory / 8 Intel vCPUs / 160 GB Disk.
|
||||
|
||||
1. Login to the droplet as root (either by SSH key or password set in the DO console)
|
||||
```
|
||||
ssh root@IP
|
||||
```
|
||||
|
||||
1. Get the install script, give it executable permissions, and run it:
|
||||
```
|
||||
ssh root@IP
|
||||
```
|
||||
|
||||
```
|
||||
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
|
||||
```
|
||||
```
|
||||
chmod +x install.sh
|
||||
```
|
||||
```
|
||||
bash install.sh
|
||||
```
|
||||
2. Get the install script, give it executable permissions, and run it:
|
||||
|
||||
1. Confirm docker was installed and activate the changes in `~/.profile`:
|
||||
```
|
||||
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
|
||||
```
|
||||
```
|
||||
chmod +x install.sh
|
||||
```
|
||||
```
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
```
|
||||
docker run hello-world
|
||||
```
|
||||
```
|
||||
source ~/.profile
|
||||
```
|
||||
3. Confirm docker was installed and activate the changes in `~/.profile`:
|
||||
|
||||
1. Verify installation:
|
||||
```
|
||||
docker run hello-world
|
||||
```
|
||||
```
|
||||
source ~/.profile
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so version
|
||||
```
|
||||
4. Verify installation:
|
||||
|
||||
```
|
||||
laconic-so version
|
||||
```
|
||||
|
||||
## Setup the laconic fixturenet stack
|
||||
|
||||
1. Get the repositories
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
|
||||
```
|
||||
|
||||
1. Build the containers:
|
||||
2. Set this environment variable to the Laconic self-hosted Gitea instance:
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded build-containers
|
||||
```
|
||||
```
|
||||
export CERC_NPM_REGISTRY_URL=https://git.vdb.to/api/packages/cerc-io/npm/
|
||||
```
|
||||
|
||||
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
|
||||
3. Build the containers:
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded build-containers
|
||||
```
|
||||
|
||||
1. Set this environment variable to your droplet's IP address or fully qualified DNS host name if it has one:
|
||||
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
|
||||
|
||||
```
|
||||
export BACKEND_ENDPOINT=http://<your-IP-or-hostname>:9473
|
||||
```
|
||||
e.g.
|
||||
```
|
||||
export BACKEND_ENDPOINT=http://my-test-server.example.com:9473
|
||||
```
|
||||
4. Set this environment variable to your droplet's IP address:
|
||||
|
||||
1. Create a deployment directory for the stack:
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy init --output laconic-loaded.spec --map-ports-to-host any-same --config LACONIC_HOSTED_ENDPOINT=$BACKEND_ENDPOINT
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy create --deployment-dir laconic-loaded-deployment --spec-file laconic-loaded.spec
|
||||
```
|
||||
2. Start the stack:
|
||||
```
|
||||
export LACONIC_HOSTED_ENDPOINT=http://<your-IP>
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment start
|
||||
```
|
||||
5. Deploy the stack:
|
||||
|
||||
3. Check the logs:
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy up
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment logs
|
||||
```
|
||||
6. Check the logs:
|
||||
|
||||
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy logs
|
||||
```
|
||||
|
||||
```
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
|
||||
```
|
||||
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
|
||||
|
||||
4. Confirm operation of the registry CLI:
|
||||
```
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment exec cli "laconic cns status"
|
||||
```
|
||||
7. Confirm operation of the registry CLI:
|
||||
|
||||
```
|
||||
{
|
||||
"version": "0.3.0",
|
||||
"node": {
|
||||
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
|
||||
"network": "laconic_9000-1",
|
||||
"moniker": "localtestnet"
|
||||
},
|
||||
"sync": {
|
||||
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
|
||||
"latest_block_height": "5",
|
||||
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
|
||||
"catching_up": false
|
||||
},
|
||||
"validator": {
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000"
|
||||
},
|
||||
"validators": [
|
||||
{
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000",
|
||||
"proposer_priority": "0"
|
||||
}
|
||||
],
|
||||
"num_peers": "0",
|
||||
"peers": [],
|
||||
"disk_usage": "292.0K"
|
||||
}
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic cns status"
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"version": "0.3.0",
|
||||
"node": {
|
||||
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
|
||||
"network": "laconic_9000-1",
|
||||
"moniker": "localtestnet"
|
||||
},
|
||||
"sync": {
|
||||
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
|
||||
"latest_block_height": "5",
|
||||
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
|
||||
"catching_up": false
|
||||
},
|
||||
"validator": {
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000"
|
||||
},
|
||||
"validators": [
|
||||
{
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000",
|
||||
"proposer_priority": "0"
|
||||
}
|
||||
],
|
||||
"num_peers": "0",
|
||||
"peers": [],
|
||||
"disk_usage": "292.0K"
|
||||
}
|
||||
```
|
||||
|
||||
## Configure Digital Ocean firewall
|
||||
|
||||
(Note this step may not be necessary depending on the droplet image used)
|
||||
|
||||
Let's open some ports.
|
||||
|
||||
1. In the Digital Ocean web console, navigate to your droplet's main page. Select the "Networking" tab and scroll down to "Firewall".
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
services:
|
||||
snowballtools-base-backend:
|
||||
image: cerc/snowballtools-base-backend:local
|
||||
restart: always
|
||||
volumes:
|
||||
- data:/data
|
||||
- config:/config:ro
|
||||
ports:
|
||||
- 8000
|
||||
|
||||
volumes:
|
||||
data:
|
||||
config:
|
||||
@@ -6,7 +6,6 @@ services:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_TEST_PARAM_1: ${CERC_TEST_PARAM_1:-FAILED}
|
||||
CERC_TEST_PARAM_2: "CERC_TEST_PARAM_2_VALUE"
|
||||
CERC_TEST_PARAM_3: ${CERC_TEST_PARAM_3:-FAILED}
|
||||
volumes:
|
||||
- test-data-bind:/data
|
||||
- test-data-auto:/data2
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
FROM cerc/snowballtools-base-backend-base:local
|
||||
|
||||
WORKDIR /app/packages/backend
|
||||
COPY run.sh .
|
||||
|
||||
ENTRYPOINT ["./run.sh"]
|
||||
-26
@@ -1,26 +0,0 @@
|
||||
FROM ubuntu:22.04 as builder
|
||||
|
||||
RUN apt update && \
|
||||
apt install -y --no-install-recommends --no-install-suggests \
|
||||
ca-certificates curl gnupg
|
||||
|
||||
# Node
|
||||
ARG NODE_MAJOR=20
|
||||
RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \
|
||||
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
|
||||
apt update && apt install -y nodejs
|
||||
|
||||
# npm setup
|
||||
RUN npm config set @cerc-io:registry https://git.vdb.to/api/packages/cerc-io/npm/ && npm install -g yarn
|
||||
|
||||
COPY . /app/
|
||||
WORKDIR /app/
|
||||
|
||||
RUN find . -name 'node_modules' | xargs -n1 rm -rf
|
||||
RUN yarn && yarn build --ignore frontend
|
||||
|
||||
FROM cerc/webapp-base:local
|
||||
|
||||
COPY --from=builder /app /app
|
||||
|
||||
WORKDIR /app/packages/backend
|
||||
@@ -1,10 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build cerc/webapp-deployer-backend
|
||||
|
||||
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
|
||||
|
||||
# See: https://stackoverflow.com/a/246128/1701505
|
||||
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
||||
|
||||
docker build -t cerc/snowballtools-base-backend-base:local ${build_command_args} -f ${SCRIPT_DIR}/Dockerfile-base ${CERC_REPO_BASE_DIR}/snowballtools-base
|
||||
docker build -t cerc/snowballtools-base-backend:local ${build_command_args} ${SCRIPT_DIR}
|
||||
@@ -1,19 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
|
||||
LACONIC_HOSTED_CONFIG_FILE=${LACONIC_HOSTED_CONFIG_FILE}
|
||||
if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
if [ -f "/config/laconic-hosted-config.yml" ]; then
|
||||
LACONIC_HOSTED_CONFIG_FILE="/config/laconic-hosted-config.yml"
|
||||
elif [ -f "/config/config.yml" ]; then
|
||||
LACONIC_HOSTED_CONFIG_FILE="/config/config.yml"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE "`pwd`/dist"
|
||||
fi
|
||||
|
||||
/scripts/apply-runtime-env.sh "`pwd`/dist"
|
||||
|
||||
yarn start
|
||||
@@ -39,15 +39,6 @@ fi
|
||||
if [ -n "$CERC_TEST_PARAM_2" ]; then
|
||||
echo "Test-param-2: ${CERC_TEST_PARAM_2}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_3" ]; then
|
||||
echo "Test-param-3: ${CERC_TEST_PARAM_3}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_4" ]; then
|
||||
echo "Test-param-4: ${CERC_TEST_PARAM_4}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_5" ]; then
|
||||
echo "Test-param-5: ${CERC_TEST_PARAM_5}"
|
||||
fi
|
||||
|
||||
if [ -d "/config" ]; then
|
||||
echo "/config: EXISTS"
|
||||
|
||||
@@ -43,6 +43,7 @@ COPY scripts /scripts
|
||||
# RUN su node -c "npm install -g <your-package-list-here>"
|
||||
|
||||
RUN mkdir -p /config
|
||||
COPY ./config.yml /config
|
||||
|
||||
# Install simple web server for now (use nginx perhaps later)
|
||||
RUN yarn global add http-server
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
# Put config here.
|
||||
@@ -11,13 +11,7 @@ WORK_DIR="${1:-/app}"
|
||||
OUTPUT_DIR="${2:-build}"
|
||||
DEST_DIR="${3:-/data}"
|
||||
|
||||
if [ -f "${WORK_DIR}/build-webapp.sh" ]; then
|
||||
echo "Building webapp with ${WORK_DIR}/build-webapp.sh ..."
|
||||
cd "${WORK_DIR}" || exit 1
|
||||
|
||||
rm -rf "${DEST_DIR}"
|
||||
./build-webapp.sh "${DEST_DIR}" || exit 1
|
||||
elif [ -f "${WORK_DIR}/package.json" ]; then
|
||||
if [ -f "${WORK_DIR}/package.json" ]; then
|
||||
echo "Building node-based webapp ..."
|
||||
cd "${WORK_DIR}" || exit 1
|
||||
|
||||
|
||||
+1
-13
@@ -10,18 +10,6 @@ if [ "true" == "$CERC_ENABLE_CORS" ]; then
|
||||
CERC_HTTP_EXTRA_ARGS="$CERC_HTTP_EXTRA_ARGS --cors"
|
||||
fi
|
||||
|
||||
LACONIC_HOSTED_CONFIG_FILE=${LACONIC_HOSTED_CONFIG_FILE}
|
||||
if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
if [ -f "/config/laconic-hosted-config.yml" ]; then
|
||||
LACONIC_HOSTED_CONFIG_FILE="/config/laconic-hosted-config.yml"
|
||||
elif [ -f "/config/config.yml" ]; then
|
||||
LACONIC_HOSTED_CONFIG_FILE="/config/config.yml"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE ${CERC_WEBAPP_FILES_DIR}
|
||||
fi
|
||||
|
||||
/scripts/apply-webapp-config.sh /config/config.yml ${CERC_WEBAPP_FILES_DIR}
|
||||
/scripts/apply-runtime-env.sh ${CERC_WEBAPP_FILES_DIR}
|
||||
http-server $CERC_HTTP_EXTRA_ARGS -p ${CERC_LISTEN_PORT:-80} ${CERC_WEBAPP_FILES_DIR}
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
version: "1.0"
|
||||
name: snowballtools-base-backend
|
||||
description: "snowballtools-base-backend"
|
||||
repos:
|
||||
- github.com/snowball-tools/snowballtools-base
|
||||
containers:
|
||||
- cerc/webapp-base
|
||||
- cerc/snowballtools-base-backend
|
||||
pods:
|
||||
- snowballtools-base-backend
|
||||
@@ -33,7 +33,6 @@ from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient,
|
||||
|
||||
|
||||
def process_app_deployment_request(
|
||||
run_id,
|
||||
ctx,
|
||||
laconic: LaconicRegistryClient,
|
||||
app_deployment_request,
|
||||
@@ -43,8 +42,18 @@ def process_app_deployment_request(
|
||||
deployment_parent_dir,
|
||||
kube_config,
|
||||
image_registry,
|
||||
log_file=None
|
||||
log_parent_dir
|
||||
):
|
||||
run_id = f"{app_deployment_request.id}-{str(time.time()).split('.')[0]}-{str(uuid.uuid4()).split('-')[0]}"
|
||||
log_file = None
|
||||
if log_parent_dir:
|
||||
log_dir = os.path.join(log_parent_dir, app_deployment_request.id)
|
||||
if not os.path.exists(log_dir):
|
||||
os.mkdir(log_dir)
|
||||
log_file_path = os.path.join(log_dir, f"{run_id}.log")
|
||||
print(f"Directing build logs to: {log_file_path}")
|
||||
log_file = open(log_file_path, "wt")
|
||||
|
||||
# 1. look up application
|
||||
app = laconic.get_record(app_deployment_request.attributes.application, require=True)
|
||||
|
||||
@@ -282,23 +291,10 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
for r in requests_to_execute:
|
||||
dump_known_requests(state_file, [r], "DEPLOYING")
|
||||
status = "ERROR"
|
||||
run_log_file = None
|
||||
run_reg_client = laconic
|
||||
try:
|
||||
run_id = f"{r.id}-{str(time.time()).split('.')[0]}-{str(uuid.uuid4()).split('-')[0]}"
|
||||
if log_dir:
|
||||
run_log_dir = os.path.join(log_dir, r.id)
|
||||
if not os.path.exists(run_log_dir):
|
||||
os.mkdir(run_log_dir)
|
||||
run_log_file_path = os.path.join(run_log_dir, f"{run_id}.log")
|
||||
print(f"Directing deployment logs to: {run_log_file_path}")
|
||||
run_log_file = open(run_log_file_path, "wt")
|
||||
run_reg_client = LaconicRegistryClient(laconic_config, log_file=run_log_file)
|
||||
|
||||
process_app_deployment_request(
|
||||
run_id,
|
||||
ctx,
|
||||
run_reg_client,
|
||||
laconic,
|
||||
r,
|
||||
record_namespace_deployments,
|
||||
record_namespace_dns,
|
||||
@@ -306,12 +302,8 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
os.path.abspath(deployment_parent_dir),
|
||||
kube_config,
|
||||
image_registry,
|
||||
run_log_file
|
||||
log_dir
|
||||
)
|
||||
status = "DEPLOYED"
|
||||
except Exception as e:
|
||||
print("ERROR: " + str(e), file=run_log_file)
|
||||
finally:
|
||||
dump_known_requests(state_file, [r], status)
|
||||
if run_log_file:
|
||||
run_log_file.close()
|
||||
|
||||
@@ -39,19 +39,13 @@ class AttrDict(dict):
|
||||
return v
|
||||
|
||||
|
||||
def logged_cmd(log_file, *vargs):
|
||||
result = None
|
||||
def cmd(*vargs):
|
||||
try:
|
||||
if log_file:
|
||||
print(" ".join(vargs), file=log_file)
|
||||
result = subprocess.run(vargs, capture_output=True)
|
||||
result.check_returncode()
|
||||
return result.stdout.decode()
|
||||
except Exception as err:
|
||||
if result:
|
||||
print(result.stderr.decode(), file=log_file)
|
||||
else:
|
||||
print(str(err), file=log_file)
|
||||
print(result.stderr.decode())
|
||||
raise err
|
||||
|
||||
|
||||
@@ -64,9 +58,8 @@ def match_owner(recordA, *records):
|
||||
|
||||
|
||||
class LaconicRegistryClient:
|
||||
def __init__(self, config_file, log_file=None):
|
||||
def __init__(self, config_file):
|
||||
self.config_file = config_file
|
||||
self.log_file = log_file
|
||||
self.cache = AttrDict(
|
||||
{
|
||||
"name_or_id": {},
|
||||
@@ -84,7 +77,7 @@ class LaconicRegistryClient:
|
||||
args.append("--%s" % k)
|
||||
args.append(str(v))
|
||||
|
||||
results = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
results = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
|
||||
# Most recent records first
|
||||
results.sort(key=lambda r: r.createTime)
|
||||
@@ -122,7 +115,7 @@ class LaconicRegistryClient:
|
||||
|
||||
args = ["laconic", "-c", self.config_file, "cns", "name", "resolve", name]
|
||||
|
||||
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
if parsed:
|
||||
self._add_to_cache(parsed)
|
||||
return parsed[0]
|
||||
@@ -152,7 +145,7 @@ class LaconicRegistryClient:
|
||||
name_or_id,
|
||||
]
|
||||
|
||||
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
if len(parsed):
|
||||
self._add_to_cache(parsed)
|
||||
return parsed[0]
|
||||
@@ -180,22 +173,22 @@ class LaconicRegistryClient:
|
||||
record_file = open(record_fname, 'w')
|
||||
yaml.dump(record, record_file)
|
||||
record_file.close()
|
||||
print(open(record_fname, 'r').read(), file=self.log_file)
|
||||
print(open(record_fname, 'r').read())
|
||||
|
||||
new_record_id = json.loads(
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
|
||||
cmd("laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
|
||||
)["id"]
|
||||
for name in names:
|
||||
self.set_name(name, new_record_id)
|
||||
return new_record_id
|
||||
finally:
|
||||
logged_cmd(self.log_file, "rm", "-rf", tmpdir)
|
||||
cmd("rm", "-rf", tmpdir)
|
||||
|
||||
def set_name(self, name, record_id):
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
|
||||
cmd("laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
|
||||
|
||||
def delete_name(self, name):
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "delete", name)
|
||||
cmd("laconic", "-c", self.config_file, "cns", "name", "delete", name)
|
||||
|
||||
|
||||
def file_hash(filename):
|
||||
@@ -234,16 +227,8 @@ def build_container_image(app_record, tag, extra_build_args=[], log_file=None):
|
||||
git_env = dict(os.environ.copy())
|
||||
# Never prompt
|
||||
git_env["GIT_TERMINAL_PROMPT"] = "0"
|
||||
try:
|
||||
subprocess.check_call(["git", "clone", repo, clone_dir], env=git_env, stdout=log_file, stderr=log_file)
|
||||
except Exception as e:
|
||||
print(f"git clone failed. Is the repository {repo} private?", file=log_file)
|
||||
raise e
|
||||
try:
|
||||
subprocess.check_call(["git", "checkout", ref], cwd=clone_dir, env=git_env, stdout=log_file, stderr=log_file)
|
||||
except Exception as e:
|
||||
print(f"git checkout failed. Does ref {ref} exist?", file=log_file)
|
||||
raise e
|
||||
subprocess.check_call(["git", "clone", repo, clone_dir], env=git_env, stdout=log_file, stderr=log_file)
|
||||
subprocess.check_call(["git", "checkout", ref], cwd=clone_dir, env=git_env, stdout=log_file, stderr=log_file)
|
||||
else:
|
||||
result = subprocess.run(["git", "clone", "--depth", "1", repo, clone_dir], stdout=log_file, stderr=log_file)
|
||||
result.check_returncode()
|
||||
@@ -264,7 +249,7 @@ def build_container_image(app_record, tag, extra_build_args=[], log_file=None):
|
||||
result = subprocess.run(build_command, stdout=log_file, stderr=log_file)
|
||||
result.check_returncode()
|
||||
finally:
|
||||
logged_cmd(log_file, "rm", "-rf", tmpdir)
|
||||
cmd("rm", "-rf", tmpdir)
|
||||
|
||||
|
||||
def push_container_image(deployment_dir, log_file=None):
|
||||
|
||||
@@ -83,7 +83,7 @@ $TEST_TARGET_SO --stack test deploy down --delete-volumes
|
||||
# Basic test of creating a deployment
|
||||
test_deployment_dir=$CERC_REPO_BASE_DIR/test-deployment-dir
|
||||
test_deployment_spec=$CERC_REPO_BASE_DIR/test-deployment-spec.yml
|
||||
$TEST_TARGET_SO --stack test deploy init --output $test_deployment_spec --config CERC_TEST_PARAM_1=PASSED,CERC_TEST_PARAM_3=FAST
|
||||
$TEST_TARGET_SO --stack test deploy init --output $test_deployment_spec --config CERC_TEST_PARAM_1=PASSED
|
||||
# Check the file now exists
|
||||
if [ ! -f "$test_deployment_spec" ]; then
|
||||
echo "deploy init test: spec file not present"
|
||||
@@ -141,13 +141,6 @@ else
|
||||
echo "deployment compose config test: FAILED"
|
||||
exit 1
|
||||
fi
|
||||
# Check the config variable CERC_TEST_PARAM_3 was passed correctly
|
||||
if [[ "$log_output_3" == *"Test-param-3: FAST"* ]]; then
|
||||
echo "deployment config test: passed"
|
||||
else
|
||||
echo "deployment config test: FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check that the ConfigMap is mounted and contains the expected content.
|
||||
log_output_4=$( $TEST_TARGET_SO deployment --dir $test_deployment_dir logs )
|
||||
|
||||
Reference in New Issue
Block a user