forked from cerc-io/stack-orchestrator
Compare commits
46
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
42a4035a64 | ||
|
|
7cce68959f | ||
|
|
0cd90ad3dc | ||
|
|
90d92c3e15 | ||
|
|
202c68d996 | ||
|
|
236dea1b44 | ||
|
|
9aa02fc26a | ||
|
|
80e039d70e | ||
|
|
d7f06ad51e | ||
|
|
343d31c324 | ||
|
|
fd00143b76 | ||
|
|
36e89b5a78 | ||
|
|
80b299c1e5 | ||
|
|
f20dcb8457 | ||
|
|
7929a137b6 | ||
|
|
247025de28 | ||
|
|
76821f00a0 | ||
|
|
78219c0e04 | ||
|
|
9e87088328 | ||
|
|
523b5779be | ||
|
|
62f7ce649d | ||
|
|
cc541ac20f | ||
|
|
10e2311a8b | ||
|
|
f32bbf9e48 | ||
|
|
0302153162 | ||
|
|
01e4437b62 | ||
|
|
64cec163b3 | ||
|
|
170ad71397 | ||
|
|
da1ff609fe | ||
|
|
21eb9f036f | ||
|
|
a0413659f7 | ||
|
|
a16fc657bf | ||
|
|
704c42c404 | ||
|
|
202f187172 | ||
|
|
aaed356d32 | ||
|
|
2af6ffce77 | ||
|
|
7bb86cf35e | ||
|
|
9e0892cb6b | ||
|
|
cf9cf6346f | ||
|
|
642c0ead0d | ||
|
|
6bd77c893a | ||
|
|
4a4d48ddb9 | ||
|
|
08438b1cd5 | ||
|
|
9f1dd284a5 | ||
|
|
5985242b8c | ||
|
|
d4152b7ce3 |
@@ -0,0 +1,45 @@
|
||||
name: Fixturenet-Eth-Plugeth-Arm-Test
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: '*'
|
||||
paths:
|
||||
- '!**'
|
||||
- '.gitea/workflows/triggers/fixturenet-eth-plugeth-arm-test'
|
||||
schedule: # Note: coordinate with other tests to not overload runners at the same time of day
|
||||
- cron: '2 14 * * *'
|
||||
|
||||
# Needed until we can incorporate docker startup into the executor container
|
||||
env:
|
||||
DOCKER_HOST: unix:///var/run/dind.sock
|
||||
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: "Run an Ethereum plugeth fixturenet test"
|
||||
runs-on: ubuntu-latest-arm
|
||||
steps:
|
||||
- name: "Clone project repository"
|
||||
uses: actions/checkout@v3
|
||||
# At present the stock setup-python action fails on Linux/aarch64
|
||||
# Conditional steps below workaroud this by using deadsnakes for that case only
|
||||
- name: "Install Python for ARM on Linux"
|
||||
if: ${{ runner.arch == 'arm64' && runner.os == 'Linux' }}
|
||||
uses: deadsnakes/action@v3.0.1
|
||||
with:
|
||||
python-version: '3.8'
|
||||
- name: "Install Python cases other than ARM on Linux"
|
||||
if: ${{ ! (runner.arch == 'arm64' && runner.os == 'Linux') }}
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.8'
|
||||
- name: "Print Python version"
|
||||
run: python3 --version
|
||||
- name: "Install shiv"
|
||||
run: pip install shiv
|
||||
- name: "Generate build version file"
|
||||
run: ./scripts/create_build_tag_file.sh
|
||||
- name: "Build local shiv package"
|
||||
run: ./scripts/build_shiv_package.sh
|
||||
- name: "Run fixturenet-eth tests"
|
||||
run: ./tests/fixturenet-eth-plugeth/run-test.sh
|
||||
@@ -0,0 +1,2 @@
|
||||
Change this file to trigger running the fixturenet-eth-plugeth-arm-test CI job
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
# Fetching pre-built container images
|
||||
When Stack Orchestrator deploys a stack containing a suite of one or more containers it expects images for those containers to be on the local machine with a tag of the form `<image-name>:local` Images for these containers can be built from source (and optionally base container images from public registries) with the `build-containers` subcommand.
|
||||
|
||||
However, the task of building a large number of containers from source may consume considerable time and machine resources. This is where the `fetch-containers` subcommand steps in. It is designed to work exactly like `build-containers` but instead the images, pre-built, are fetched from an image registry then re-tagged for deployment. It can be used in place of `build-containers` for any stack provided the necessary containers, built for the local machine architecture (e.g. arm64 or x86-64) have already been published in an image registry.
|
||||
## Usage
|
||||
To use `fetch-containers`, provide an image registry path, a username and token/password with read access to the registry, and optionally specify `--force-local-overwrite`. If this argument is not specified, if there is already a locally built or previously fetched image for a stack container on the machine, it will not be overwritten and a warning issued.
|
||||
```
|
||||
$ laconic-so --stack mobymask-v3-demo fetch-containers --image-registry git.vdb.to/cerc-io --registry-username <registry-user> --registry-token <registry-token> --force-local-overwrite
|
||||
```
|
||||
+104
-97
@@ -15,132 +15,139 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
|
||||
16 GB Memory / 8 Intel vCPUs / 160 GB Disk.
|
||||
|
||||
1. Login to the droplet as root (either by SSH key or password set in the DO console)
|
||||
```
|
||||
ssh root@IP
|
||||
```
|
||||
|
||||
```
|
||||
ssh root@IP
|
||||
```
|
||||
1. Get the install script, give it executable permissions, and run it:
|
||||
|
||||
2. Get the install script, give it executable permissions, and run it:
|
||||
```
|
||||
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
|
||||
```
|
||||
```
|
||||
chmod +x install.sh
|
||||
```
|
||||
```
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
```
|
||||
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
|
||||
```
|
||||
```
|
||||
chmod +x install.sh
|
||||
```
|
||||
```
|
||||
bash install.sh
|
||||
```
|
||||
1. Confirm docker was installed and activate the changes in `~/.profile`:
|
||||
|
||||
3. Confirm docker was installed and activate the changes in `~/.profile`:
|
||||
```
|
||||
docker run hello-world
|
||||
```
|
||||
```
|
||||
source ~/.profile
|
||||
```
|
||||
|
||||
```
|
||||
docker run hello-world
|
||||
```
|
||||
```
|
||||
source ~/.profile
|
||||
```
|
||||
1. Verify installation:
|
||||
|
||||
4. Verify installation:
|
||||
|
||||
```
|
||||
laconic-so version
|
||||
```
|
||||
```
|
||||
laconic-so version
|
||||
```
|
||||
|
||||
## Setup the laconic fixturenet stack
|
||||
|
||||
1. Get the repositories
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
|
||||
```
|
||||
|
||||
2. Set this environment variable to the Laconic self-hosted Gitea instance:
|
||||
1. Build the containers:
|
||||
|
||||
```
|
||||
export CERC_NPM_REGISTRY_URL=https://git.vdb.to/api/packages/cerc-io/npm/
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded build-containers
|
||||
```
|
||||
|
||||
3. Build the containers:
|
||||
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded build-containers
|
||||
```
|
||||
|
||||
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
|
||||
1. Set this environment variable to your droplet's IP address or fully qualified DNS host name if it has one:
|
||||
|
||||
4. Set this environment variable to your droplet's IP address:
|
||||
```
|
||||
export BACKEND_ENDPOINT=http://<your-IP-or-hostname>:9473
|
||||
```
|
||||
e.g.
|
||||
```
|
||||
export BACKEND_ENDPOINT=http://my-test-server.example.com:9473
|
||||
```
|
||||
|
||||
```
|
||||
export LACONIC_HOSTED_ENDPOINT=http://<your-IP>
|
||||
```
|
||||
1. Create a deployment directory for the stack:
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy init --output laconic-loaded.spec --map-ports-to-host any-same --config LACONIC_HOSTED_ENDPOINT=$BACKEND_ENDPOINT
|
||||
```
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy create --deployment-dir laconic-loaded-deployment --spec-file laconic-loaded.spec
|
||||
```
|
||||
2. Start the stack:
|
||||
|
||||
5. Deploy the stack:
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment start
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy up
|
||||
```
|
||||
3. Check the logs:
|
||||
|
||||
6. Check the logs:
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment logs
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy logs
|
||||
```
|
||||
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
|
||||
|
||||
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
|
||||
```
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
|
||||
```
|
||||
|
||||
```
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
|
||||
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
|
||||
```
|
||||
4. Confirm operation of the registry CLI:
|
||||
|
||||
7. Confirm operation of the registry CLI:
|
||||
```
|
||||
laconic-so deployment --dir laconic-loaded-deployment exec cli "laconic cns status"
|
||||
```
|
||||
|
||||
```
|
||||
laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic cns status"
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"version": "0.3.0",
|
||||
"node": {
|
||||
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
|
||||
"network": "laconic_9000-1",
|
||||
"moniker": "localtestnet"
|
||||
},
|
||||
"sync": {
|
||||
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
|
||||
"latest_block_height": "5",
|
||||
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
|
||||
"catching_up": false
|
||||
},
|
||||
"validator": {
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000"
|
||||
},
|
||||
"validators": [
|
||||
{
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000",
|
||||
"proposer_priority": "0"
|
||||
}
|
||||
],
|
||||
"num_peers": "0",
|
||||
"peers": [],
|
||||
"disk_usage": "292.0K"
|
||||
}
|
||||
```
|
||||
```
|
||||
{
|
||||
"version": "0.3.0",
|
||||
"node": {
|
||||
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
|
||||
"network": "laconic_9000-1",
|
||||
"moniker": "localtestnet"
|
||||
},
|
||||
"sync": {
|
||||
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
|
||||
"latest_block_height": "5",
|
||||
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
|
||||
"catching_up": false
|
||||
},
|
||||
"validator": {
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000"
|
||||
},
|
||||
"validators": [
|
||||
{
|
||||
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
|
||||
"voting_power": "1000000000000000",
|
||||
"proposer_priority": "0"
|
||||
}
|
||||
],
|
||||
"num_peers": "0",
|
||||
"peers": [],
|
||||
"disk_usage": "292.0K"
|
||||
}
|
||||
```
|
||||
|
||||
## Configure Digital Ocean firewall
|
||||
|
||||
(Note this step may not be necessary depending on the droplet image used)
|
||||
|
||||
Let's open some ports.
|
||||
|
||||
1. In the Digital Ocean web console, navigate to your droplet's main page. Select the "Networking" tab and scroll down to "Firewall".
|
||||
|
||||
@@ -25,10 +25,13 @@ import sys
|
||||
from decouple import config
|
||||
import subprocess
|
||||
import click
|
||||
import importlib.resources
|
||||
from pathlib import Path
|
||||
from stack_orchestrator.util import include_exclude_check, get_parsed_stack_config, stack_is_external, warn_exit
|
||||
from stack_orchestrator.opts import opts
|
||||
from stack_orchestrator.util import include_exclude_check, stack_is_external, error_exit
|
||||
from stack_orchestrator.base import get_npm_registry_url
|
||||
from stack_orchestrator.build.build_types import BuildContext
|
||||
from stack_orchestrator.build.publish import publish_image
|
||||
from stack_orchestrator.build.build_util import get_containers_in_scope
|
||||
|
||||
# TODO: find a place for this
|
||||
# epilog="Config provided either in .env or settings.ini or env vars: CERC_REPO_BASE_DIR (defaults to ~/cerc)"
|
||||
@@ -59,69 +62,58 @@ def make_container_build_env(dev_root_path: str,
|
||||
return container_build_env
|
||||
|
||||
|
||||
def process_container(stack: str,
|
||||
container,
|
||||
container_build_dir: str,
|
||||
container_build_env: dict,
|
||||
dev_root_path: str,
|
||||
quiet: bool,
|
||||
verbose: bool,
|
||||
dry_run: bool,
|
||||
continue_on_error: bool,
|
||||
):
|
||||
if not quiet:
|
||||
print(f"Building: {container}")
|
||||
def process_container(build_context: BuildContext) -> bool:
|
||||
if not opts.o.quiet:
|
||||
print(f"Building: {build_context.container}")
|
||||
|
||||
default_container_tag = f"{container}:local"
|
||||
container_build_env.update({"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag})
|
||||
default_container_tag = f"{build_context.container}:local"
|
||||
build_context.container_build_env.update({"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag})
|
||||
|
||||
# Check if this is in an external stack
|
||||
if stack_is_external(stack):
|
||||
container_parent_dir = Path(stack).joinpath("container-build")
|
||||
temp_build_dir = container_parent_dir.joinpath(container.replace("/", "-"))
|
||||
if stack_is_external(build_context.stack):
|
||||
container_parent_dir = Path(build_context.stack).joinpath("container-build")
|
||||
temp_build_dir = container_parent_dir.joinpath(build_context.container.replace("/", "-"))
|
||||
temp_build_script_filename = temp_build_dir.joinpath("build.sh")
|
||||
# Now check if the container exists in the external stack.
|
||||
if not temp_build_script_filename.exists():
|
||||
# If not, revert to building an internal container
|
||||
container_parent_dir = container_build_dir
|
||||
container_parent_dir = build_context.container_build_dir
|
||||
else:
|
||||
container_parent_dir = container_build_dir
|
||||
container_parent_dir = build_context.container_build_dir
|
||||
|
||||
build_dir = container_parent_dir.joinpath(container.replace("/", "-"))
|
||||
build_dir = container_parent_dir.joinpath(build_context.container.replace("/", "-"))
|
||||
build_script_filename = build_dir.joinpath("build.sh")
|
||||
|
||||
if verbose:
|
||||
if opts.o.verbose:
|
||||
print(f"Build script filename: {build_script_filename}")
|
||||
if os.path.exists(build_script_filename):
|
||||
build_command = build_script_filename.as_posix()
|
||||
else:
|
||||
if verbose:
|
||||
if opts.o.verbose:
|
||||
print(f"No script file found: {build_script_filename}, using default build script")
|
||||
repo_dir = container.split('/')[1]
|
||||
repo_dir = build_context.container.split('/')[1]
|
||||
# TODO: make this less of a hack -- should be specified in some metadata somewhere
|
||||
# Check if we have a repo for this container. If not, set the context dir to the container-build subdir
|
||||
repo_full_path = os.path.join(dev_root_path, repo_dir)
|
||||
repo_full_path = os.path.join(build_context.dev_root_path, repo_dir)
|
||||
repo_dir_or_build_dir = repo_full_path if os.path.exists(repo_full_path) else build_dir
|
||||
build_command = os.path.join(container_build_dir,
|
||||
build_command = os.path.join(build_context.container_build_dir,
|
||||
"default-build.sh") + f" {default_container_tag} {repo_dir_or_build_dir}"
|
||||
if not dry_run:
|
||||
if not opts.o.dry_run:
|
||||
# No PATH at all causes failures with podman.
|
||||
if "PATH" not in container_build_env:
|
||||
container_build_env["PATH"] = os.environ["PATH"]
|
||||
if verbose:
|
||||
print(f"Executing: {build_command} with environment: {container_build_env}")
|
||||
build_result = subprocess.run(build_command, shell=True, env=container_build_env)
|
||||
if verbose:
|
||||
if "PATH" not in build_context.container_build_env:
|
||||
build_context.container_build_env["PATH"] = os.environ["PATH"]
|
||||
if opts.o.verbose:
|
||||
print(f"Executing: {build_command} with environment: {build_context.container_build_env}")
|
||||
build_result = subprocess.run(build_command, shell=True, env=build_context.container_build_env)
|
||||
if opts.o.verbose:
|
||||
print(f"Return code is: {build_result.returncode}")
|
||||
if build_result.returncode != 0:
|
||||
print(f"Error running build for {container}")
|
||||
if not continue_on_error:
|
||||
print("FATAL Error: container build failed and --continue-on-error not set, exiting")
|
||||
sys.exit(1)
|
||||
else:
|
||||
print("****** Container Build Error, continuing because --continue-on-error is set")
|
||||
return False
|
||||
else:
|
||||
return True
|
||||
else:
|
||||
print("Skipped")
|
||||
return True
|
||||
|
||||
|
||||
@click.command()
|
||||
@@ -129,17 +121,14 @@ def process_container(stack: str,
|
||||
@click.option('--exclude', help="don\'t build these containers")
|
||||
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
|
||||
@click.option("--extra-build-args", help="Supply extra arguments to build")
|
||||
@click.option("--publish-images", is_flag=True, default=False, help="Publish the built images in the specified image registry")
|
||||
@click.option("--image-registry", help="Specify the image registry for --publish-images")
|
||||
@click.pass_context
|
||||
def command(ctx, include, exclude, force_rebuild, extra_build_args):
|
||||
def command(ctx, include, exclude, force_rebuild, extra_build_args, publish_images, image_registry):
|
||||
'''build the set of containers required for a complete stack'''
|
||||
|
||||
quiet = ctx.obj.quiet
|
||||
verbose = ctx.obj.verbose
|
||||
dry_run = ctx.obj.dry_run
|
||||
debug = ctx.obj.debug
|
||||
local_stack = ctx.obj.local_stack
|
||||
stack = ctx.obj.stack
|
||||
continue_on_error = ctx.obj.continue_on_error
|
||||
|
||||
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
|
||||
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
|
||||
@@ -150,41 +139,45 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
|
||||
else:
|
||||
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
|
||||
|
||||
if not quiet:
|
||||
if not opts.o.quiet:
|
||||
print(f'Dev Root is: {dev_root_path}')
|
||||
|
||||
if not os.path.isdir(dev_root_path):
|
||||
print('Dev root directory doesn\'t exist, creating')
|
||||
|
||||
# See: https://stackoverflow.com/a/20885799/1701505
|
||||
from stack_orchestrator import data
|
||||
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
|
||||
all_containers = container_list_file.read().splitlines()
|
||||
if publish_images:
|
||||
if not image_registry:
|
||||
error_exit("--image-registry must be supplied with --publish-images")
|
||||
|
||||
containers_in_scope = []
|
||||
if stack:
|
||||
stack_config = get_parsed_stack_config(stack)
|
||||
if "containers" not in stack_config or stack_config["containers"] is None:
|
||||
warn_exit(f"stack {stack} does not define any containers")
|
||||
containers_in_scope = stack_config['containers']
|
||||
else:
|
||||
containers_in_scope = all_containers
|
||||
|
||||
if verbose:
|
||||
print(f'Containers: {containers_in_scope}')
|
||||
if stack:
|
||||
print(f"Stack: {stack}")
|
||||
containers_in_scope = get_containers_in_scope(stack)
|
||||
|
||||
container_build_env = make_container_build_env(dev_root_path,
|
||||
container_build_dir,
|
||||
debug,
|
||||
opts.o.debug,
|
||||
force_rebuild,
|
||||
extra_build_args)
|
||||
|
||||
for container in containers_in_scope:
|
||||
if include_exclude_check(container, include, exclude):
|
||||
process_container(stack, container, container_build_dir, container_build_env,
|
||||
dev_root_path, quiet, verbose, dry_run, continue_on_error)
|
||||
|
||||
build_context = BuildContext(
|
||||
stack,
|
||||
container,
|
||||
container_build_dir,
|
||||
container_build_env,
|
||||
dev_root_path
|
||||
)
|
||||
result = process_container(build_context)
|
||||
if result:
|
||||
if publish_images:
|
||||
publish_image(f"{container}:local", image_registry)
|
||||
else:
|
||||
print(f"Error running build for {build_context.container}")
|
||||
if not opts.o.continue_on_error:
|
||||
error_exit("container build failed and --continue-on-error not set, exiting")
|
||||
sys.exit(1)
|
||||
else:
|
||||
print("****** Container Build Error, continuing because --continue-on-error is set")
|
||||
else:
|
||||
if verbose:
|
||||
if opts.o.verbose:
|
||||
print(f"Excluding: {container}")
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Copyright © 2024 Vulcanize
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Affero General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Mapping
|
||||
|
||||
|
||||
@dataclass
|
||||
class BuildContext:
|
||||
stack: str
|
||||
container: str
|
||||
container_build_dir: Path
|
||||
container_build_env: Mapping[str,str]
|
||||
dev_root_path: str
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# Copyright © 2024 Vulcanize
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Affero General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
import importlib.resources
|
||||
|
||||
from stack_orchestrator.opts import opts
|
||||
from stack_orchestrator.util import get_parsed_stack_config, warn_exit
|
||||
|
||||
|
||||
def get_containers_in_scope(stack: str):
|
||||
|
||||
# See: https://stackoverflow.com/a/20885799/1701505
|
||||
from stack_orchestrator import data
|
||||
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
|
||||
all_containers = container_list_file.read().splitlines()
|
||||
|
||||
containers_in_scope = []
|
||||
if stack:
|
||||
stack_config = get_parsed_stack_config(stack)
|
||||
if "containers" not in stack_config or stack_config["containers"] is None:
|
||||
warn_exit(f"stack {stack} does not define any containers")
|
||||
containers_in_scope = stack_config['containers']
|
||||
else:
|
||||
containers_in_scope = all_containers
|
||||
|
||||
if opts.o.verbose:
|
||||
print(f'Containers: {containers_in_scope}')
|
||||
if stack:
|
||||
print(f"Stack: {stack}")
|
||||
|
||||
return containers_in_scope
|
||||
@@ -21,11 +21,14 @@
|
||||
# TODO: display the available list of containers; allow re-build of either all or specific containers
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
from decouple import config
|
||||
import click
|
||||
from pathlib import Path
|
||||
from stack_orchestrator.build import build_containers
|
||||
from stack_orchestrator.deploy.webapp.util import determine_base_container
|
||||
from stack_orchestrator.deploy.webapp.util import determine_base_container, TimedLogger
|
||||
from stack_orchestrator.build.build_types import BuildContext
|
||||
|
||||
|
||||
@click.command()
|
||||
@@ -37,26 +40,25 @@ from stack_orchestrator.deploy.webapp.util import determine_base_container
|
||||
@click.pass_context
|
||||
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag):
|
||||
'''build the specified webapp container'''
|
||||
logger = TimedLogger()
|
||||
|
||||
quiet = ctx.obj.quiet
|
||||
verbose = ctx.obj.verbose
|
||||
dry_run = ctx.obj.dry_run
|
||||
debug = ctx.obj.debug
|
||||
verbose = ctx.obj.verbose
|
||||
local_stack = ctx.obj.local_stack
|
||||
stack = ctx.obj.stack
|
||||
continue_on_error = ctx.obj.continue_on_error
|
||||
|
||||
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
|
||||
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
|
||||
|
||||
if local_stack:
|
||||
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
|
||||
print(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
|
||||
logger.log(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
|
||||
else:
|
||||
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
|
||||
|
||||
if not quiet:
|
||||
print(f'Dev Root is: {dev_root_path}')
|
||||
if verbose:
|
||||
logger.log(f'Dev Root is: {dev_root_path}')
|
||||
|
||||
if not base_container:
|
||||
base_container = determine_base_container(source_repo)
|
||||
@@ -65,8 +67,23 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
|
||||
container_build_env = build_containers.make_container_build_env(dev_root_path, container_build_dir, debug,
|
||||
force_rebuild, extra_build_args)
|
||||
|
||||
build_containers.process_container(None, base_container, container_build_dir, container_build_env, dev_root_path, quiet,
|
||||
verbose, dry_run, continue_on_error)
|
||||
if verbose:
|
||||
logger.log(f"Building base container: {base_container}")
|
||||
|
||||
build_context_1 = BuildContext(
|
||||
stack,
|
||||
base_container,
|
||||
container_build_dir,
|
||||
container_build_env,
|
||||
dev_root_path,
|
||||
)
|
||||
ok = build_containers.process_container(build_context_1)
|
||||
if not ok:
|
||||
logger.log("ERROR: Build failed.")
|
||||
sys.exit(1)
|
||||
|
||||
if verbose:
|
||||
logger.log(f"Base container {base_container} build finished.")
|
||||
|
||||
# Now build the target webapp. We use the same build script, but with a different Dockerfile and work dir.
|
||||
container_build_env["CERC_WEBAPP_BUILD_RUNNING"] = "true"
|
||||
@@ -76,9 +93,25 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
|
||||
"Dockerfile.webapp")
|
||||
if not tag:
|
||||
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
|
||||
container_build_env["CERC_CONTAINER_BUILD_TAG"] = f"cerc/{webapp_name}:local"
|
||||
else:
|
||||
container_build_env["CERC_CONTAINER_BUILD_TAG"] = tag
|
||||
tag = f"cerc/{webapp_name}:local"
|
||||
|
||||
build_containers.process_container(None, base_container, container_build_dir, container_build_env, dev_root_path, quiet,
|
||||
verbose, dry_run, continue_on_error)
|
||||
container_build_env["CERC_CONTAINER_BUILD_TAG"] = tag
|
||||
|
||||
if verbose:
|
||||
logger.log(f"Building app container: {tag}")
|
||||
|
||||
build_context_2 = BuildContext(
|
||||
stack,
|
||||
base_container,
|
||||
container_build_dir,
|
||||
container_build_env,
|
||||
dev_root_path,
|
||||
)
|
||||
ok = build_containers.process_container(build_context_2)
|
||||
if not ok:
|
||||
logger.log("ERROR: Build failed.")
|
||||
sys.exit(1)
|
||||
|
||||
if verbose:
|
||||
logger.log(f"App container {base_container} build finished.")
|
||||
logger.log("build-webapp complete", show_step_time=False, show_total_time=True)
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
# Copyright © 2024 Vulcanize
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Affero General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
import click
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import platform
|
||||
from python_on_whales import DockerClient
|
||||
from python_on_whales.components.manifest.cli_wrapper import ManifestCLI, ManifestList
|
||||
from python_on_whales.utils import run
|
||||
import requests
|
||||
from typing import List
|
||||
|
||||
from stack_orchestrator.opts import opts
|
||||
from stack_orchestrator.util import include_exclude_check, error_exit
|
||||
from stack_orchestrator.build.build_util import get_containers_in_scope
|
||||
|
||||
# Experimental fetch-container command
|
||||
|
||||
|
||||
@dataclass
|
||||
class RegistryInfo:
|
||||
registry: str
|
||||
registry_username: str
|
||||
registry_token: str
|
||||
|
||||
|
||||
# Extending this code to support the --verbose option, cnosider contributing upstream
|
||||
# https://github.com/gabrieldemarmiesse/python-on-whales/blob/master/python_on_whales/components/manifest/cli_wrapper.py#L129
|
||||
class ExtendedManifestCLI(ManifestCLI):
|
||||
def inspect_verbose(self, x: str) -> ManifestList:
|
||||
"""Returns a Docker manifest list object."""
|
||||
json_str = run(self.docker_cmd + ["manifest", "inspect", "--verbose", x])
|
||||
return json.loads(json_str)
|
||||
|
||||
|
||||
def _local_tag_for(container: str):
|
||||
return f"{container}:local"
|
||||
|
||||
|
||||
# See: https://docker-docs.uclv.cu/registry/spec/api/
|
||||
# Emulate this:
|
||||
# $ curl -u "my-username:my-token" -X GET "https://<container-registry-hostname>/v2/cerc-io/cerc/test-container/tags/list"
|
||||
# {"name":"cerc-io/cerc/test-container","tags":["202402232130","202402232208"]}
|
||||
def _get_tags_for_container(container: str, registry_info: RegistryInfo) -> List[str]:
|
||||
# registry looks like: git.vdb.to/cerc-io
|
||||
registry_parts = registry_info.registry.split("/")
|
||||
url = f"https://{registry_parts[0]}/v2/{registry_parts[1]}/{container}/tags/list"
|
||||
if opts.o.debug:
|
||||
print(f"Fetching tags from: {url}")
|
||||
response = requests.get(url, auth=(registry_info.registry_username, registry_info.registry_token))
|
||||
if response.status_code == 200:
|
||||
tag_info = response.json()
|
||||
if opts.o.debug:
|
||||
print(f"container tags list: {tag_info}")
|
||||
tags_array = tag_info["tags"]
|
||||
return tags_array
|
||||
else:
|
||||
error_exit(f"failed to fetch tags from image registry, status code: {response.status_code}")
|
||||
|
||||
|
||||
def _find_latest(candidate_tags: List[str]):
|
||||
# Lex sort should give us the latest first
|
||||
sorted_candidates = sorted(candidate_tags)
|
||||
if opts.o.debug:
|
||||
print(f"sorted candidates: {sorted_candidates}")
|
||||
return sorted_candidates[-1]
|
||||
|
||||
|
||||
def _filter_for_platform(container: str,
|
||||
registry_info: RegistryInfo,
|
||||
tag_list: List[str]) -> List[str] :
|
||||
filtered_tags = []
|
||||
this_machine = platform.machine()
|
||||
# Translate between Python and docker platform names
|
||||
if this_machine == "x86_64":
|
||||
this_machine = "amd64"
|
||||
if this_machine == "aarch64":
|
||||
this_machine = "arm64"
|
||||
if opts.o.debug:
|
||||
print(f"Python says the architecture is: {this_machine}")
|
||||
docker = DockerClient()
|
||||
for tag in tag_list:
|
||||
remote_tag = f"{registry_info.registry}/{container}:{tag}"
|
||||
manifest_cmd = ExtendedManifestCLI(docker.client_config)
|
||||
manifest = manifest_cmd.inspect_verbose(remote_tag)
|
||||
if opts.o.debug:
|
||||
print(f"manifest: {manifest}")
|
||||
image_architecture = manifest["Descriptor"]["platform"]["architecture"]
|
||||
if opts.o.debug:
|
||||
print(f"image_architecture: {image_architecture}")
|
||||
if this_machine == image_architecture:
|
||||
filtered_tags.append(tag)
|
||||
if opts.o.debug:
|
||||
print(f"Tags filtered for platform: {filtered_tags}")
|
||||
return filtered_tags
|
||||
|
||||
|
||||
def _get_latest_image(container: str, registry_info: RegistryInfo):
|
||||
all_tags = _get_tags_for_container(container, registry_info)
|
||||
tags_for_platform = _filter_for_platform(container, registry_info, all_tags)
|
||||
if len(tags_for_platform) > 0:
|
||||
latest_tag = _find_latest(tags_for_platform)
|
||||
return f"{container}:{latest_tag}"
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def _fetch_image(tag: str, registry_info: RegistryInfo):
|
||||
docker = DockerClient()
|
||||
remote_tag = f"{registry_info.registry}/{tag}"
|
||||
if opts.o.debug:
|
||||
print(f"Attempting to pull this image: {remote_tag}")
|
||||
docker.image.pull(remote_tag)
|
||||
|
||||
|
||||
def _exists_locally(container: str):
|
||||
docker = DockerClient()
|
||||
return docker.image.exists(_local_tag_for(container))
|
||||
|
||||
|
||||
def _add_local_tag(remote_tag: str, registry: str, local_tag: str):
|
||||
docker = DockerClient()
|
||||
docker.image.tag(f"{registry}/{remote_tag}", local_tag)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option('--include', help="only fetch these containers")
|
||||
@click.option('--exclude', help="don\'t fetch these containers")
|
||||
@click.option("--force-local-overwrite", is_flag=True, default=False, help="Overwrite a locally built image, if present")
|
||||
@click.option("--image-registry", required=True, help="Specify the image registry to fetch from")
|
||||
@click.option("--registry-username", required=True, help="Specify the image registry username")
|
||||
@click.option("--registry-token", required=True, help="Specify the image registry access token")
|
||||
@click.pass_context
|
||||
def command(ctx, include, exclude, force_local_overwrite, image_registry, registry_username, registry_token):
|
||||
'''EXPERIMENTAL: fetch the images for a stack from remote registry'''
|
||||
|
||||
registry_info = RegistryInfo(image_registry, registry_username, registry_token)
|
||||
docker = DockerClient()
|
||||
if not opts.o.quiet:
|
||||
print("Logging into container registry:")
|
||||
docker.login(registry_info.registry, registry_info.registry_username, registry_info.registry_token)
|
||||
# Generate list of target containers
|
||||
stack = ctx.obj.stack
|
||||
containers_in_scope = get_containers_in_scope(stack)
|
||||
all_containers_found = True
|
||||
for container in containers_in_scope:
|
||||
local_tag = _local_tag_for(container)
|
||||
if include_exclude_check(container, include, exclude):
|
||||
if opts.o.debug:
|
||||
print(f"Processing: {container}")
|
||||
# For each container, attempt to find the latest of a set of
|
||||
# images with the correct name and platform in the specified registry
|
||||
image_to_fetch = _get_latest_image(container, registry_info)
|
||||
if not image_to_fetch:
|
||||
print(f"Warning: no image found to fetch for container: {container}")
|
||||
all_containers_found = False
|
||||
continue
|
||||
if opts.o.debug:
|
||||
print(f"Fetching: {image_to_fetch}")
|
||||
_fetch_image(image_to_fetch, registry_info)
|
||||
# Now check if the target container already exists exists locally already
|
||||
if (_exists_locally(container)):
|
||||
if not opts.o.quiet:
|
||||
print(f"Container image {container} already exists locally")
|
||||
# if so, fail unless the user specified force-local-overwrite
|
||||
if (force_local_overwrite):
|
||||
# In that case remove the existing :local tag
|
||||
if not opts.o.quiet:
|
||||
print(f"Warning: overwriting local tag from this image: {container} because "
|
||||
"--force-local-overwrite was specified")
|
||||
else:
|
||||
if not opts.o.quiet:
|
||||
print(f"Skipping local tagging for this image: {container} because that would "
|
||||
"overwrite an existing :local tagged image, use --force-local-overwrite to do so.")
|
||||
continue
|
||||
# Tag the fetched image with the :local tag
|
||||
_add_local_tag(image_to_fetch, image_registry, local_tag)
|
||||
else:
|
||||
if opts.o.verbose:
|
||||
print(f"Excluding: {container}")
|
||||
if not all_containers_found:
|
||||
print("Warning: couldn't find usable images for one or more containers, this stack will not deploy")
|
||||
@@ -0,0 +1,48 @@
|
||||
# Copyright © 2024 Vulcanize
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Affero General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
from datetime import datetime
|
||||
from python_on_whales import DockerClient
|
||||
|
||||
from stack_orchestrator.opts import opts
|
||||
from stack_orchestrator.util import error_exit
|
||||
|
||||
|
||||
def _publish_tag_for_image(local_image_tag: str, remote_repo: str, version: str):
|
||||
# Turns image tags of the form: foo/bar:local into remote.repo/org/bar:deploy
|
||||
(image_name, image_version) = local_image_tag.split(":")
|
||||
if image_version == "local":
|
||||
return f"{remote_repo}/{image_name}:{version}"
|
||||
else:
|
||||
error_exit("Asked to publish a non-locally built image")
|
||||
|
||||
|
||||
def publish_image(local_tag, registry):
|
||||
if opts.o.verbose:
|
||||
print(f"Publishing this image: {local_tag} to this registry: {registry}")
|
||||
docker = DockerClient()
|
||||
# Figure out the target image tag
|
||||
# Eventually this version will be generated from the source repo state
|
||||
# Using a timestemp is an intermediate step
|
||||
version = datetime.now().strftime("%Y%m%d%H%M")
|
||||
remote_tag = _publish_tag_for_image(local_tag, registry, version)
|
||||
# Tag the image thus
|
||||
if opts.o.debug:
|
||||
print(f"Tagging {local_tag} to {remote_tag}")
|
||||
docker.image.tag(local_tag, remote_tag)
|
||||
# Push it to the desired registry
|
||||
if opts.o.verbose:
|
||||
print(f"Pushing image {remote_tag}")
|
||||
docker.image.push(remote_tag)
|
||||
@@ -5,9 +5,11 @@ services:
|
||||
environment:
|
||||
REGISTRY_LOG_LEVEL: ${REGISTRY_LOG_LEVEL}
|
||||
volumes:
|
||||
- config:/config:ro
|
||||
- registry-data:/var/lib/registry
|
||||
ports:
|
||||
- "5000"
|
||||
|
||||
volumes:
|
||||
config:
|
||||
registry-data:
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
version: '3.7'
|
||||
|
||||
services:
|
||||
go-nitro-alice:
|
||||
image: cerc/go-nitro:local
|
||||
hostname: go-nitro-alice
|
||||
restart: on-failure
|
||||
depends_on:
|
||||
- go-nitro-bootnode
|
||||
environment:
|
||||
CERC_NITRO_CHAIN_PK: ${CERC_NITRO_CHAIN_PK_ALICE:-888814df89c4358d7ddb3fa4b0213e7331239a80e1f013eaa7b2deca2a41a218}
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL:-ws://fixturenet-eth-geth-1:8546}
|
||||
CERC_NA_ADDRESS: ${CERC_NA_ADDRESS}
|
||||
CERC_VPA_ADDRESS: ${CERC_VPA_ADDRESS}
|
||||
CERC_CA_ADDRESS: ${CERC_CA_ADDRESS}
|
||||
env_file:
|
||||
- ../config/go-nitro-auth/alice.env
|
||||
entrypoint: ["bash", "-c", "/app/run-nitro-node.sh"]
|
||||
volumes:
|
||||
- nitro_deployment:/app/deployment
|
||||
- go_nitro_data_alice:/app/data
|
||||
- ../config/go-nitro/run-nitro-node.sh:/app/run-nitro-node.sh
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-vz", "localhost", "4006"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 100
|
||||
start_period: 100s
|
||||
ports:
|
||||
- "3006"
|
||||
- "4006"
|
||||
- "5006"
|
||||
- "4106"
|
||||
- "4206"
|
||||
- "4216"
|
||||
|
||||
go-nitro-bob:
|
||||
image: cerc/go-nitro:local
|
||||
hostname: go-nitro-bob
|
||||
restart: on-failure
|
||||
depends_on:
|
||||
- go-nitro-bootnode
|
||||
environment:
|
||||
CERC_NITRO_CHAIN_PK: ${CERC_NITRO_CHAIN_PK_BOB:-570b909da9669b2f35a0b1ac70b8358516d55ae1b5b3710e95e9a94395090597}
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL:-ws://fixturenet-eth-geth-1:8546}
|
||||
CERC_NA_ADDRESS: ${CERC_NA_ADDRESS}
|
||||
CERC_VPA_ADDRESS: ${CERC_VPA_ADDRESS}
|
||||
CERC_CA_ADDRESS: ${CERC_CA_ADDRESS}
|
||||
env_file:
|
||||
- ../config/go-nitro-auth/bob.env
|
||||
entrypoint: ["bash", "-c", "/app/run-nitro-node.sh"]
|
||||
volumes:
|
||||
- nitro_deployment:/app/deployment
|
||||
- go_nitro_data_bob:/app/data
|
||||
- ../config/go-nitro/run-nitro-node.sh:/app/run-nitro-node.sh
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-vz", "localhost", "4007"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 100
|
||||
start_period: 100s
|
||||
ports:
|
||||
- "3007"
|
||||
- "4007"
|
||||
- "5007"
|
||||
- "4107"
|
||||
- "4207"
|
||||
- "4217"
|
||||
|
||||
go-nitro-bootnode:
|
||||
image: cerc/go-nitro:local
|
||||
hostname: go-nitro-bootnode
|
||||
restart: on-failure
|
||||
environment:
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL:-ws://fixturenet-eth-geth-1:8546}
|
||||
CERC_NA_ADDRESS: ${CERC_NA_ADDRESS}
|
||||
CERC_VPA_ADDRESS: ${CERC_VPA_ADDRESS}
|
||||
CERC_CA_ADDRESS: ${CERC_CA_ADDRESS}
|
||||
CERC_NITRO_RELAY_ON: true
|
||||
env_file:
|
||||
- ../config/go-nitro-auth/bootnode.env
|
||||
entrypoint: [ "bash", "-c", "/app/run-nitro-node.sh" ]
|
||||
volumes:
|
||||
- nitro_deployment:/app/deployment
|
||||
- go_nitro_data_bootnode:/app/data
|
||||
- ../config/go-nitro/run-nitro-node.sh:/app/run-nitro-node.sh
|
||||
healthcheck:
|
||||
test: [ "CMD", "nc", "-vz", "localhost", "4008" ]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 100
|
||||
start_period: 100s
|
||||
ports:
|
||||
- "9090"
|
||||
|
||||
nitro-caddy:
|
||||
image: caddy:2-alpine
|
||||
restart: always
|
||||
environment:
|
||||
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT:-http://fixturenet-eth-geth-1:8545}
|
||||
volumes:
|
||||
- ../config/go-nitro-auth/caddy/Caddyfile:/etc/caddy/Caddyfile
|
||||
ports:
|
||||
- "5678"
|
||||
depends_on:
|
||||
- go-nitro-alice
|
||||
- go-nitro-bob
|
||||
|
||||
volumes:
|
||||
go_nitro_data_alice:
|
||||
go_nitro_data_bob:
|
||||
go_nitro_data_bootnode:
|
||||
nitro_deployment:
|
||||
@@ -1,32 +0,0 @@
|
||||
version: '3.2'
|
||||
|
||||
services:
|
||||
# Builds and serves the MobyMask v3 react-app
|
||||
mobymask-v3-app:
|
||||
restart: unless-stopped
|
||||
image: cerc/mobymask-ui:local
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_APP_WATCHER_URL: ${CERC_APP_WATCHER_URL}
|
||||
CERC_SNAP_URL: ${CERC_SNAP_URL}
|
||||
working_dir: /app
|
||||
command: ["bash", "/scripts/mobymask-app-start.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-mobymask-v3-demo/mobymask-app-start.sh:/scripts/mobymask-app-start.sh
|
||||
- ../config/watcher-mobymask-v3-demo/local/keys:/server/keys
|
||||
- mobymask_deployment:/server
|
||||
- nitro_deployment:/nitro
|
||||
ports:
|
||||
- "127.0.0.1:3004:80"
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-vz", "localhost", "80"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 10s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
volumes:
|
||||
mobymask_deployment:
|
||||
nitro_deployment:
|
||||
@@ -6,6 +6,7 @@ services:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_TEST_PARAM_1: ${CERC_TEST_PARAM_1:-FAILED}
|
||||
CERC_TEST_PARAM_2: "CERC_TEST_PARAM_2_VALUE"
|
||||
CERC_TEST_PARAM_3: ${CERC_TEST_PARAM_3:-FAILED}
|
||||
volumes:
|
||||
- test-data-bind:/data
|
||||
- test-data-auto:/data2
|
||||
|
||||
@@ -1,162 +0,0 @@
|
||||
version: '3.2'
|
||||
|
||||
services:
|
||||
# Starts the PostgreSQL database for watcher
|
||||
mobymask-watcher-db:
|
||||
restart: unless-stopped
|
||||
image: postgres:14-alpine
|
||||
environment:
|
||||
- POSTGRES_USER=vdbm
|
||||
- POSTGRES_MULTIPLE_DATABASES=mobymask-watcher-1,mobymask-watcher-1-job-queue,mobymask-watcher-2,mobymask-watcher-2-job-queue,mobymask-watcher-3,mobymask-watcher-3-job-queue
|
||||
- POSTGRES_EXTENSION=mobymask-watcher-1-job-queue:pgcrypto,mobymask-watcher-2-job-queue:pgcrypto,mobymask-watcher-3-job-queue:pgcrypto
|
||||
- POSTGRES_PASSWORD=password
|
||||
volumes:
|
||||
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
|
||||
- mobymask_watcher_db_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "5432"
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-v", "localhost", "5432"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 10s
|
||||
|
||||
# Deploys the MobyMask contract and generates an invite link
|
||||
# Deployment is skipped if CERC_DEPLOYED_CONTRACT env is set
|
||||
mobymask:
|
||||
image: cerc/mobymask:local
|
||||
working_dir: /app/packages/server
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
# ENV: "PROD" # Check if required
|
||||
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT}
|
||||
CERC_PRIVATE_KEY_DEPLOYER: ${CERC_PRIVATE_KEY_DEPLOYER}
|
||||
CERC_MOBYMASK_APP_BASE_URI: ${CERC_MOBYMASK_APP_BASE_URI}
|
||||
CERC_DEPLOYED_CONTRACT: ${CERC_DEPLOYED_CONTRACT}
|
||||
command: ["bash", "-c", "./deploy-and-generate-invite.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-mobymask-v3-demo/deploy-and-generate-invite.sh:/app/packages/server/deploy-and-generate-invite.sh
|
||||
- mobymask_deployment:/app/packages/server
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
# Starts a MobyMask v3 watcher server #1
|
||||
mobymask-watcher-1-server:
|
||||
image: cerc/watcher-mobymask-v3:local
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
mobymask-watcher-db:
|
||||
condition: service_healthy
|
||||
mobymask:
|
||||
condition: service_completed_successfully
|
||||
nitro-contracts:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_WATCHER_INDEX: 1
|
||||
CERC_ETH_RPC_QUERY_ENDPOINT: ${CERC_ETH_RPC_QUERY_ENDPOINT}
|
||||
CERC_ETH_RPC_MUTATION_ENDPOINT: ${CERC_ETH_RPC_MUTATION_ENDPOINT}
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL}
|
||||
command: ["bash", "./start-server.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-template.toml:/app/environments/watcher-config-template.toml
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-rates.toml:/app/environments/rates.toml
|
||||
- ../config/watcher-mobymask-v3-demo/local/keys:/app/keys
|
||||
- ../config/watcher-mobymask-v3-demo/start-server.sh:/app/start-server.sh
|
||||
- watcher_nitro_data:/app/out/nitro-db
|
||||
- nitro_deployment:/nitro
|
||||
- mobymask_deployment:/server
|
||||
# Expose GQL, metrics and relay node ports
|
||||
ports:
|
||||
- "0.0.0.0:3001:3001"
|
||||
- "9001"
|
||||
- "0.0.0.0:9090:9090"
|
||||
healthcheck:
|
||||
test: ["CMD", "busybox", "nc", "localhost", "9090"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 5s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
# Starts a MobyMask v3 watcher server #2
|
||||
mobymask-watcher-2-server:
|
||||
image: cerc/watcher-mobymask-v3:local
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
mobymask-watcher-1-server:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_WATCHER_INDEX: 2
|
||||
CERC_ETH_RPC_QUERY_ENDPOINT: ${CERC_ETH_RPC_QUERY_ENDPOINT}
|
||||
CERC_ETH_RPC_MUTATION_ENDPOINT: ${CERC_ETH_RPC_MUTATION_ENDPOINT}
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL}
|
||||
command: ["bash", "./start-server.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-template.toml:/app/environments/watcher-config-template.toml
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-rates.toml:/app/environments/rates.toml
|
||||
- ../config/watcher-mobymask-v3-demo/local/keys:/app/keys
|
||||
- ../config/watcher-mobymask-v3-demo/start-server.sh:/app/start-server.sh
|
||||
- watcher_nitro_data:/app/out/nitro-db
|
||||
- nitro_deployment:/nitro
|
||||
- mobymask_deployment:/server
|
||||
# Expose GQL, metrics and relay node ports
|
||||
ports:
|
||||
- "3001"
|
||||
- "9001"
|
||||
- "0.0.0.0:9091:9090"
|
||||
healthcheck:
|
||||
test: ["CMD", "busybox", "nc", "localhost", "9090"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 5s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
# Starts a MobyMask v3 watcher server #3
|
||||
mobymask-watcher-3-server:
|
||||
image: cerc/watcher-mobymask-v3:local
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
mobymask-watcher-1-server:
|
||||
condition: service_healthy
|
||||
mobymask-watcher-2-server:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_WATCHER_INDEX: 3
|
||||
CERC_ETH_RPC_QUERY_ENDPOINT: ${CERC_ETH_RPC_QUERY_ENDPOINT}
|
||||
CERC_ETH_RPC_MUTATION_ENDPOINT: ${CERC_ETH_RPC_MUTATION_ENDPOINT}
|
||||
CERC_NITRO_CHAIN_URL: ${CERC_NITRO_CHAIN_URL}
|
||||
command: ["bash", "./start-server.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-template.toml:/app/environments/watcher-config-template.toml
|
||||
- ../config/watcher-mobymask-v3-demo/watcher-config-rates.toml:/app/environments/rates.toml
|
||||
- ../config/watcher-mobymask-v3-demo/local/keys:/app/keys
|
||||
- ../config/watcher-mobymask-v3-demo/start-server.sh:/app/start-server.sh
|
||||
- watcher_nitro_data:/app/out/nitro-db
|
||||
- nitro_deployment:/nitro
|
||||
- mobymask_deployment:/server
|
||||
# Expose GQL, metrics and relay node ports
|
||||
ports:
|
||||
- "3001"
|
||||
- "9001"
|
||||
- "0.0.0.0:9092:9090"
|
||||
healthcheck:
|
||||
test: ["CMD", "busybox", "nc", "localhost", "9090"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 5s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
volumes:
|
||||
mobymask_watcher_db_data:
|
||||
mobymask_deployment:
|
||||
nitro_deployment:
|
||||
watcher_nitro_data:
|
||||
+2
-2
@@ -65,7 +65,7 @@ if [ -n "$CERC_L1_ADDRESS" ] && [ -n "$CERC_L1_PRIV_KEY" ]; then
|
||||
# Sequencer
|
||||
SEQ=$(echo "$wallet3" | awk '/Address:/{print $2}')
|
||||
SEQ_KEY=$(echo "$wallet3" | awk '/Private key:/{print $3}')
|
||||
|
||||
|
||||
echo "Funding accounts."
|
||||
wait_for_block 1 300
|
||||
cast send --from $ADMIN --rpc-url $CERC_L1_RPC --value 5ether $PROPOSER --private-key $ADMIN_KEY
|
||||
@@ -169,4 +169,4 @@ cp -a /app/packages/contracts-bedrock/deployments/$DEPLOYMENT_CONTEXT /l1-deploy
|
||||
cp /app/packages/contracts-bedrock/deploy-config/$DEPLOYMENT_CONTEXT.json /l2-config
|
||||
openssl rand -hex 32 > /l2-config/l2-jwt.txt
|
||||
|
||||
echo "Deployment successful. Exiting."
|
||||
echo "Deployment successful. Exiting"
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
# SET ME! Your on-chain private key.
|
||||
#CERC_NITRO_CHAIN_PK=<MY_PK_GOES_HERE>
|
||||
|
||||
# Default is CERC_NITRO_CHAIN_PK.
|
||||
#CERC_NITRO_PK=<MY_PK_GOES_HERE>
|
||||
|
||||
# SET ME! The WebSocket to connect to.
|
||||
#CERC_NITRO_CHAIN_URL=wss://linea-sepolia.infura.io/ws/v3/<MY_API_KEY_GOES_HERE>
|
||||
|
||||
CERC_NITRO_AUTH_ON=false
|
||||
CERC_NITRO_AUTH_RPC_PORT=4006
|
||||
CERC_NITRO_AUTH_UI_PORT=4206
|
||||
CERC_NITRO_BOOT_PEERS="/dns4/go-nitro-bootnode/tcp/3008/p2p/16Uiu2HAmFYz5YPf3GtkfQsAwYJfDsKSm34U9qAiTwxW7RJUVGMRK"
|
||||
CERC_NITRO_DURABLE_STORE_FOLDER=/app/data/nitro-store
|
||||
CERC_NITRO_MSG_PORT=3006
|
||||
CERC_NITRO_RPC_PORT=4006
|
||||
CERC_NITRO_UI_PORT=4106
|
||||
CERC_NITRO_USE_DURABLE_STORE=true
|
||||
CERC_NITRO_WS_MSG_PORT=5006
|
||||
@@ -0,0 +1,19 @@
|
||||
# SET ME! Your on-chain private key.
|
||||
#CERC_NITRO_CHAIN_PK=<MY_PK_GOES_HERE>
|
||||
|
||||
# Default is CERC_NITRO_CHAIN_PK.
|
||||
#CERC_NITRO_PK=<MY_PK_GOES_HERE>
|
||||
|
||||
# SET ME! The WebSocket to connect to.
|
||||
#CERC_NITRO_CHAIN_URL=wss://linea-sepolia.infura.io/ws/v3/<MY_API_KEY_GOES_HERE>
|
||||
|
||||
CERC_NITRO_AUTH_ON=true
|
||||
CERC_NITRO_AUTH_RPC_PORT=4007
|
||||
CERC_NITRO_AUTH_UI_PORT=4207
|
||||
CERC_NITRO_BOOT_PEERS="/dns4/go-nitro-bootnode/tcp/3008/p2p/16Uiu2HAmFYz5YPf3GtkfQsAwYJfDsKSm34U9qAiTwxW7RJUVGMRK"
|
||||
CERC_NITRO_DURABLE_STORE_FOLDER=/app/data/nitro-store
|
||||
CERC_NITRO_MSG_PORT=3007
|
||||
CERC_NITRO_RPC_PORT=4007
|
||||
CERC_NITRO_UI_PORT=4107
|
||||
CERC_NITRO_USE_DURABLE_STORE=true
|
||||
CERC_NITRO_WS_MSG_PORT=5007
|
||||
@@ -0,0 +1,17 @@
|
||||
CERC_NITRO_CHAIN_PK=596db2ac27479cfdf60f708bf64ae44a1c5090e9446cca011d72bc9a59b47d3d
|
||||
CERC_NITRO_PK=f41086394674cf00a66448c6688295d7330af39f6f38fed89ea023e39382d6a0
|
||||
|
||||
# The WebSocket to connect to.
|
||||
#CERC_NITRO_CHAIN_URL=wss://linea-sepolia.infura.io/ws/v3/<MY_API_KEY_GOES_HERE>
|
||||
|
||||
CERC_NITRO_AUTH_ON=false
|
||||
CERC_NITRO_AUTH_RPC_PORT=4008
|
||||
CERC_NITRO_AUTH_UI_PORT=4208
|
||||
CERC_NITRO_DURABLE_STORE_FOLDER=/app/data/nitro-store
|
||||
CERC_NITRO_MSG_PORT=3008
|
||||
CERC_NITRO_RPC_PORT=4008
|
||||
CERC_NITRO_UI_PORT=4108
|
||||
CERC_NITRO_USE_DURABLE_STORE=true
|
||||
CERC_NITRO_WS_MSG_PORT=5008
|
||||
|
||||
CERC_SCRIPT_DEBUG=true
|
||||
@@ -0,0 +1,42 @@
|
||||
:5678 {
|
||||
handle /pay* {
|
||||
reverse_proxy http://go-nitro-bob:8547
|
||||
}
|
||||
|
||||
@eth-key-in-header {
|
||||
path_regexp /eth/?$
|
||||
header X-Api-Key *
|
||||
}
|
||||
|
||||
handle @eth-key-in-header {
|
||||
forward_auth go-nitro-bob:8547 {
|
||||
uri /auth/{header.x-api-key}
|
||||
}
|
||||
rewrite * /
|
||||
reverse_proxy {$CERC_ETH_RPC_ENDPOINT} {
|
||||
header_up Host {upstream_hostport}
|
||||
}
|
||||
}
|
||||
|
||||
@eth-key-in-path {
|
||||
path_regexp apikey eth/(.+)$
|
||||
}
|
||||
|
||||
handle @eth-key-in-path {
|
||||
forward_auth go-nitro-bob:8547 {
|
||||
uri /auth/{re.apikey.1}
|
||||
}
|
||||
rewrite * /
|
||||
reverse_proxy {$CERC_ETH_RPC_ENDPOINT} {
|
||||
header_up Host {upstream_hostport}
|
||||
}
|
||||
}
|
||||
|
||||
handle /eth* {
|
||||
respond "401 Unauthorized" 401
|
||||
}
|
||||
|
||||
handle {
|
||||
reverse_proxy http://go-nitro-alice:4206
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,11 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
if [ -z "$CERC_NITRO_CHAIN_PK" ] || [ -z "$CERC_NITRO_CHAIN_URL" ]; then
|
||||
echo "You most set both CERC_NITRO_CHAIN_PK and CERC_NITRO_CHAIN_URL." 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
nitro_addresses_file="/app/deployment/nitro-addresses.json"
|
||||
|
||||
# Check if CERC_NA_ADDRESS environment variable is set
|
||||
@@ -29,22 +34,86 @@ fi
|
||||
|
||||
echo "Running Nitro node"
|
||||
|
||||
# Assuming CERC_NITRO_CHAIN_URL is of format <ws|http>://host:port
|
||||
ws_host=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d ':' -f 1)
|
||||
ws_port=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d ':' -f 2)
|
||||
|
||||
# Wait till chain endpoint is available
|
||||
retry_interval=5
|
||||
while true; do
|
||||
nc -z -w 1 "$ws_host" "$ws_port"
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "Chain endpoint is available"
|
||||
break
|
||||
if [[ "${CERC_GO_NITRO_WAIT_FOR_CHAIN:-true}" == "true" ]]; then
|
||||
# Assuming CERC_NITRO_CHAIN_URL is of format <ws|http>://host[:port][/foo]
|
||||
ws_host=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d'/' -f 1 | cut -d ':' -f 1)
|
||||
ws_port=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d'/' -f 1 | cut -d ':' -f 2)
|
||||
if [[ "$ws_port" == "$ws_host" ]]; then
|
||||
ws_port=""
|
||||
fi
|
||||
|
||||
echo "Chain endpoint not yet available, retrying in $retry_interval seconds..."
|
||||
sleep $retry_interval
|
||||
done
|
||||
# Wait till chain endpoint is available
|
||||
retry_interval=5
|
||||
while true; do
|
||||
nc -z -w 1 "$ws_host" "${ws_port:-443}"
|
||||
|
||||
./nitro -chainurl ${CERC_NITRO_CHAIN_URL} -msgport ${CERC_NITRO_MSG_PORT} -rpcport ${CERC_NITRO_RPC_PORT} -wsmsgport ${CERC_NITRO_WS_MSG_PORT} -publicip "0.0.0.0" -pk ${CERC_NITRO_PK} -chainpk ${CERC_NITRO_CHAIN_PK} -naaddress ${NA_ADDRESS} -vpaaddress ${VPA_ADDRESS} -caaddress ${CA_ADDRESS} -usedurablestore ${CERC_NITRO_USE_DURABLE_STORE} -durablestorefolder ${CERC_NITRO_DURABLE_STORE_FOLDER} -tlscertfilepath "" -tlsKeyFilepath ""
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "Chain endpoint is available"
|
||||
break
|
||||
fi
|
||||
|
||||
echo "Chain endpoint not yet available, retrying in $retry_interval seconds..."
|
||||
sleep $retry_interval
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ -n "$CERC_NITRO_UI_PORT" ]] && [[ -d "/app-node/packages/nitro-gui/dist" ]]; then
|
||||
for f in `ls /app-node/packages/nitro-gui/dist/assets/*.js`; do
|
||||
sed -i "s#\"CERC_RUNTIME_ENV_RPC_HOST\"#\"localhost:${CERC_NITRO_RPC_PORT}\"#g" "$f"
|
||||
sed -i "s#\"CERC_RUNTIME_ENV_TARGET_URL\"#\"http://localhost:5678\"#g" "$f"
|
||||
done
|
||||
http-server -p $CERC_NITRO_UI_PORT /app-node/packages/nitro-gui/dist &
|
||||
fi
|
||||
|
||||
if [[ -n "$CERC_NITRO_AUTH_UI_PORT" ]] && [[ -d "/app-node/packages/nitro-auth-gui/dist" ]]; then
|
||||
for f in `ls /app-node/packages/nitro-auth-gui/dist/assets/*.js`; do
|
||||
sed -i "s#\"CERC_RUNTIME_ENV_RPC_URL\"#\"http://localhost:${CERC_NITRO_RPC_PORT}\"#g" "$f"
|
||||
sed -i "s#\"CERC_RUNTIME_ENV_TARGET_URL\"#\"http://localhost:5678\"#g" "$f"
|
||||
done
|
||||
http-server -p $CERC_NITRO_AUTH_UI_PORT /app-node/packages/nitro-auth-gui/dist &
|
||||
fi
|
||||
|
||||
if [[ "$CERC_NITRO_AUTH_ON" == "true" ]] && [[ -d "/app-node/packages/nitro-auth/dist" ]]; then
|
||||
bash -c "sleep 6 && cd /app-node/packages/nitro-auth && yarn start" &
|
||||
fi
|
||||
|
||||
if [[ "$CERC_NITRO_RELAY_ON" == "true" ]]; then
|
||||
if [[ ! -f "/app/deployment/relay-node.json" ]]; then
|
||||
node /usr/local/lib/node_modules/@cerc-io/peer/dist/cli/create-peer.js \
|
||||
-f /app/deployment/relay-node.json
|
||||
fi
|
||||
DEBUG='laconic:*' node /usr/local/lib/node_modules/@cerc-io/peer/dist/cli/relay.js \
|
||||
--host 0.0.0.0 \
|
||||
-f /app/deployment/relay-node.json &
|
||||
fi
|
||||
|
||||
if [[ -z "$CERC_CHAIN_START_BLOCK" ]]; then
|
||||
if [[ ! -f "/app/deployment/chainstartblock.json" ]]; then
|
||||
curl --location "$(echo $CERC_NITRO_CHAIN_URL | sed 's/^ws/http/' | sed 's#/ws/#/#')" \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 124,
|
||||
"method": "eth_blockNumber",
|
||||
"params": []
|
||||
}' > /app/deployment/chainstartblock.json
|
||||
fi
|
||||
CERC_CHAIN_START_BLOCK=$(printf "%d" `cat /app/deployment/chainstartblock.json | jq -r '.result'`)
|
||||
fi
|
||||
|
||||
cd /app
|
||||
./nitro \
|
||||
-chainurl ${CERC_NITRO_CHAIN_URL} \
|
||||
-msgport ${CERC_NITRO_MSG_PORT} \
|
||||
-rpcport ${CERC_NITRO_RPC_PORT} \
|
||||
-wsmsgport ${CERC_NITRO_WS_MSG_PORT} \
|
||||
-publicip "0.0.0.0" \
|
||||
-pk ${CERC_NITRO_PK:-$CERC_NITRO_CHAIN_PK} \
|
||||
-chainpk ${CERC_NITRO_CHAIN_PK} \
|
||||
-naaddress ${NA_ADDRESS} \
|
||||
-vpaaddress ${VPA_ADDRESS} \
|
||||
-caaddress ${CA_ADDRESS} \
|
||||
-usedurablestore=${CERC_NITRO_USE_DURABLE_STORE} \
|
||||
-durablestorefolder ${CERC_NITRO_DURABLE_STORE_FOLDER} \
|
||||
-bootpeers "${CERC_NITRO_BOOT_PEERS}" \
|
||||
-chainstartblock $CERC_CHAIN_START_BLOCK
|
||||
@@ -0,0 +1,99 @@
|
||||
#!/bin/bash
|
||||
|
||||
CERC_NITRO_RPC_FUND_AMOUNT=${CERC_NITRO_RPC_FUND_AMOUNT:-0}
|
||||
CERC_NITRO_RPC_HOST_ALICE=${CERC_NITRO_RPC_HOST_ALICE:-go-nitro-alice}
|
||||
CERC_NITRO_RPC_PORT_ALICE=${CERC_NITRO_RPC_PORT_ALICE:-4006}
|
||||
CERC_NITRO_USE_TLS=${CERC_NITRO_USE_TLS:-false}
|
||||
CERC_NITRO_ADDRESS_BOB=${CERC_NITRO_ADDRESS_BOB:-0xe07e314501cc73b24cf45a6577486017300e153c}
|
||||
|
||||
|
||||
# Wait till chain endpoint is available
|
||||
retry_interval=5
|
||||
while true; do
|
||||
nc -z -w 1 "$CERC_NITRO_RPC_HOST_ALICE" "$CERC_NITRO_RPC_PORT_ALICE"
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "Nitro endpoint is available"
|
||||
break
|
||||
fi
|
||||
|
||||
echo "Nitro endpoint not yet available, retrying in $retry_interval seconds..."
|
||||
sleep $retry_interval
|
||||
done
|
||||
|
||||
if [[ "$CERC_NITRO_RPC_FUND_AMOUNT" -gt 0 ]]; then
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
get-all-ledger-channels | \
|
||||
jq "[.[] | select(.Status == \"Open\") | select(.Balance.Them == \"$CERC_NITRO_ADDRESS_BOB\")] | first" > \
|
||||
/app/deployment/nitro-ledger-channel-alice-to-bob.json
|
||||
|
||||
ledger_channel=$(jq -r '.ID' /app/deployment/nitro-ledger-channel-alice-to-bob.json 2>/dev/null | sed 's/^null$//')
|
||||
|
||||
if [[ -z "${ledger_channel}" ]]; then
|
||||
echo "Creating new ledger channel ..."
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
-n \
|
||||
direct-fund --amount $CERC_NITRO_RPC_FUND_AMOUNT $CERC_NITRO_ADDRESS_BOB
|
||||
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
get-all-ledger-channels | \
|
||||
jq "[.[] | select(.Status == \"Open\") | select(.Balance.Them == \"$CERC_NITRO_ADDRESS_BOB\")] | first" > \
|
||||
/app/deployment/nitro-ledger-channel-alice-to-bob.json
|
||||
|
||||
ledger_channel=$(jq -r '.ID' /app/deployment/nitro-ledger-channel-alice-to-bob.json)
|
||||
fi
|
||||
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
get-payment-channels-by-ledger $ledger_channel > \
|
||||
/app/deployment/nitro-payment-channels-alice-to-bob.json
|
||||
|
||||
first_open_channel=$(jq '[.[] | select(.Status == "Open")] | first' /app/deployment/nitro-payment-channels-alice-to-bob.json | sed 's/^null$//')
|
||||
|
||||
if [[ -z "$first_open_channel" ]]; then
|
||||
echo "Creating new payment channel ..."
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
-n \
|
||||
virtual-fund --amount $((CERC_NITRO_RPC_FUND_AMOUNT/2)) $CERC_NITRO_ADDRESS_BOB
|
||||
|
||||
nitro-rpc-client -h $CERC_NITRO_RPC_HOST_ALICE \
|
||||
-p $CERC_NITRO_RPC_PORT_ALICE \
|
||||
-s=$CERC_NITRO_USE_TLS \
|
||||
get-payment-channels-by-ledger $ledger_channel > \
|
||||
/app/deployment/nitro-payment-channels-alice-to-bob.json
|
||||
|
||||
first_open_channel=$(jq '[.[] | select(.Status == "Open")] | first' /app/deployment/nitro-payment-channels-alice-to-bob.json | sed 's/^null$//')
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "################################################################"
|
||||
echo ""
|
||||
|
||||
echo "LEDGER:"
|
||||
cat /app/deployment/nitro-ledger-channel-alice-to-bob.json | jq
|
||||
echo ""
|
||||
echo ""
|
||||
|
||||
echo "PAYMENT:"
|
||||
cat /app/deployment/nitro-payment-channels-alice-to-bob.json | jq
|
||||
echo ""
|
||||
echo ""
|
||||
fi
|
||||
|
||||
if [ -n "$1" ]; then
|
||||
exec "$@"
|
||||
exit $?
|
||||
fi
|
||||
|
||||
while [ 1 -eq 1 ]; do
|
||||
sleep 100
|
||||
done
|
||||
@@ -1,80 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
# From env:
|
||||
# CERC_ETH_RPC_ENDPOINT
|
||||
# CERC_MOBYMASK_APP_BASE_URI
|
||||
# CERC_DEPLOYED_CONTRACT
|
||||
# CERC_PRIVATE_KEY_DEPLOYER
|
||||
|
||||
# Check if CERC_DEPLOYED_CONTRACT environment variable set to skip contract deployment
|
||||
if [ -n "$CERC_DEPLOYED_CONTRACT" ]; then
|
||||
echo "CERC_DEPLOYED_CONTRACT is set to '$CERC_DEPLOYED_CONTRACT'"
|
||||
echo "Skipping contract deployment"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Using ETH RPC endpoint ${CERC_ETH_RPC_ENDPOINT}"
|
||||
|
||||
# Wait till ETH RPC endpoint is available with block number > 1
|
||||
retry_interval=5
|
||||
while true; do
|
||||
block_number_hex=$(curl -s -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' ${CERC_ETH_RPC_ENDPOINT} | jq -r '.result')
|
||||
|
||||
# Check if the request call was successful
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "RPC endpoint not yet available, retrying in $retry_interval seconds..."
|
||||
sleep $retry_interval
|
||||
continue
|
||||
fi
|
||||
|
||||
# Convert hex to decimal
|
||||
block_number_dec=$(printf %u ${block_number_hex})
|
||||
|
||||
# Check if block number is > 1 to avoid failures in the deployment
|
||||
if [ "$block_number_dec" -ge 1 ]; then
|
||||
echo "RPC endpoint is up"
|
||||
break
|
||||
else
|
||||
echo "RPC endpoint not yet available, retrying in $retry_interval seconds..."
|
||||
sleep $retry_interval
|
||||
continue
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Using CERC_PRIVATE_KEY_DEPLOYER from env"
|
||||
|
||||
# Create the required JSON and write it to a file
|
||||
secrets_file="secrets.json"
|
||||
secrets_json=$(jq -n \
|
||||
--arg privateKey "$CERC_PRIVATE_KEY_DEPLOYER" \
|
||||
--arg rpcUrl "$CERC_ETH_RPC_ENDPOINT" \
|
||||
--arg baseURI "$CERC_MOBYMASK_APP_BASE_URI" \
|
||||
'.privateKey = $privateKey | .rpcUrl = $rpcUrl | .baseURI = $baseURI')
|
||||
echo "$secrets_json" > "${secrets_file}"
|
||||
|
||||
export RPC_URL="${CERC_ETH_RPC_ENDPOINT}"
|
||||
|
||||
# Check and exit if a deployment already exists (on restarts)
|
||||
if [ -f ./config.json ]; then
|
||||
echo "config.json already exists, checking the contract deployment"
|
||||
|
||||
# Read JSON file
|
||||
deployment_details=$(cat config.json)
|
||||
deployed_contract=$(echo "$deployment_details" | jq -r '.address')
|
||||
|
||||
cd ../hardhat
|
||||
if yarn verifyDeployment --network optimism --contract "${deployed_contract}"; then
|
||||
echo "Deployment verfication successful"
|
||||
cd ../server
|
||||
else
|
||||
echo "Deployment verfication failed, please clear MobyMask deployment volume before starting"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
npm run deployAndGenerateInvite
|
||||
@@ -1,16 +0,0 @@
|
||||
# Watchers
|
||||
|
||||
CERC_ETH_RPC_ENDPOINT="http://op-geth:8545"
|
||||
|
||||
# L1 pre-funded account #4, funded on L2 as well
|
||||
CERC_PRIVATE_KEY_DEPLOYER="0xbe4aa664815ea3bc3d63118649a733f6c96b243744310806ecb6d96359ab62cf"
|
||||
CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3004/#"
|
||||
|
||||
CERC_ETH_RPC_QUERY_ENDPOINT="http://op-geth:8545"
|
||||
CERC_ETH_RPC_MUTATION_ENDPOINT="http://op-geth:8545"
|
||||
CERC_NITRO_CHAIN_URL="http://op-geth:8546"
|
||||
|
||||
# App
|
||||
|
||||
CERC_APP_WATCHER_URL="http://localhost:3001"
|
||||
CERC_SNAP_URL=http://localhost:8080
|
||||
@@ -1,72 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
echo "*************************************"
|
||||
|
||||
CERC_L1_RPC="${CERC_L1_RPC:-${DEFAULT_CERC_L1_RPC}}"
|
||||
CERC_L1_CHAIN_ID="${CERC_L1_CHAIN_ID:-${DEFAULT_CERC_L1_CHAIN_ID}}"
|
||||
CERC_L1_ACCOUNTS_CSV_URL="${CERC_L1_ACCOUNTS_CSV_URL:-${DEFAULT_CERC_L1_ACCOUNTS_CSV_URL}}"
|
||||
|
||||
# Fixture accounts
|
||||
WATCHER_1_ACCOUNT=0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266
|
||||
WATCHER_2_ACCOUNT=0x70997970C51812dc3A010C7d01b50e0d17dc79C8
|
||||
WATCHER_3_ACCOUNT=0x15d34AAf54267DB7D7c367839AAf71A00a2C6A65
|
||||
NITRO_APP_ACCOUNT=0xbDA5747bFD65F08deb54cb465eB87D40e51B197E
|
||||
|
||||
WATCHER_1_ACCOUNT_KEY=0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
|
||||
WATCHER_2_ACCOUNT_KEY=0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d
|
||||
WATCHER_3_ACCOUNT_KEY=0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a
|
||||
NITRO_APP_ACCOUNT_KEY=0x689af8efa8c651a91ad287602527f3af2fe9f6501a7ac4b061667b5a93e037fd
|
||||
|
||||
# Check if watcher 1 account already funded
|
||||
WATCHER_1_ACCOUNT_BALANCE=$(cast balance ${WATCHER_1_ACCOUNT} --rpc-url $CERC_L1_RPC)
|
||||
echo "WATCHER_1_ACCOUNT_BALANCE ${WATCHER_1_ACCOUNT_BALANCE}"
|
||||
if [ "$WATCHER_1_ACCOUNT_BALANCE" != "0" ]; then
|
||||
echo "Watcher account already funded, exiting."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Funding accounts on L2"
|
||||
|
||||
# Fetch the L1 funded accounts
|
||||
curl -o accounts.csv $CERC_L1_ACCOUNTS_CSV_URL
|
||||
|
||||
# Use the accounts other than the ones used for Optimism deployment
|
||||
ACCOUNT_1=$(awk -F ',' 'NR == 5 {print $2}' accounts.csv)
|
||||
ACCOUNT_1_KEY=$(awk -F ',' 'NR == 5 {print $3}' accounts.csv)
|
||||
|
||||
ACCOUNT_2=$(awk -F ',' 'NR == 6 {print $2}' accounts.csv)
|
||||
ACCOUNT_2_KEY=$(awk -F ',' 'NR == 6 {print $3}' accounts.csv)
|
||||
|
||||
# Get the bridge contract address
|
||||
DEPLOYMENT_CONTEXT="$CERC_L1_CHAIN_ID"
|
||||
BRIDGE=$(cat /l1-deployment/$DEPLOYMENT_CONTEXT/L1StandardBridgeProxy.json | jq -r .address)
|
||||
|
||||
# Send balance to bridge contract on L1
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_1 --value 10000ether $BRIDGE --private-key $ACCOUNT_1_KEY
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_2 --value 10000ether $BRIDGE --private-key $ACCOUNT_2_KEY
|
||||
|
||||
echo "Following accounts have been funded; use them for transactions on L2:"
|
||||
echo "${ACCOUNT_1}"
|
||||
echo "${ACCOUNT_2}"
|
||||
|
||||
echo "*************************************"
|
||||
echo "Funding the watcher and app Nitro accounts on L2"
|
||||
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_1 --value 200ether $WATCHER_1_ACCOUNT --private-key $ACCOUNT_1_KEY
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_1 --value 200ether $WATCHER_2_ACCOUNT --private-key $ACCOUNT_1_KEY
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_1 --value 200ether $WATCHER_3_ACCOUNT --private-key $ACCOUNT_1_KEY
|
||||
cast send --rpc-url $CERC_L1_RPC --from $ACCOUNT_1 --value 200ether $NITRO_APP_ACCOUNT --private-key $ACCOUNT_1_KEY
|
||||
|
||||
cast send --rpc-url $CERC_L1_RPC --from $WATCHER_1_ACCOUNT --value 100ether $BRIDGE --private-key $WATCHER_1_ACCOUNT_KEY
|
||||
sleep 5
|
||||
cast send --rpc-url $CERC_L1_RPC --from $WATCHER_2_ACCOUNT --value 100ether $BRIDGE --private-key $WATCHER_2_ACCOUNT_KEY
|
||||
sleep 5
|
||||
cast send --rpc-url $CERC_L1_RPC --from $WATCHER_3_ACCOUNT --value 100ether $BRIDGE --private-key $WATCHER_3_ACCOUNT_KEY
|
||||
sleep 5
|
||||
cast send --rpc-url $CERC_L1_RPC --from $NITRO_APP_ACCOUNT --value 100ether $BRIDGE --private-key $NITRO_APP_ACCOUNT_KEY
|
||||
|
||||
echo "Done, exiting."
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"peerId": {
|
||||
"id": "12D3KooWAMjBkFCT9DtCnSDcxftxJzSuTBvzVojabv64cnEvX4AZ",
|
||||
"privKey": "CAESQAKCrnY0QKTky1I18fqn+VPydXGUv1NYiV+nVKqBFkw/CAjE9sKKIDGnYAo8mivnI6dngFenERY+0Q8AJrPTaXY=",
|
||||
"pubKey": "CAESIAgIxPbCiiAxp2AKPJor5yOnZ4BXpxEWPtEPACaz02l2"
|
||||
},
|
||||
"relayPeerId": {
|
||||
"id": "12D3KooWNQJDGtH3YAZk2qG1ZbcoQvw6HaD814pfd9HHQthgX63K",
|
||||
"privKey": "CAESQGc996T2OuyV3aY+hnna+GIexKBEcE6jDNX/RaofHr58uv2EYxXm4kcL2tO2tKgnPgtkegHFdjqy6h/xOA+YMZo=",
|
||||
"pubKey": "CAESILr9hGMV5uJHC9rTtrSoJz4LZHoBxXY6suof8TgPmDGa"
|
||||
},
|
||||
"peer": {
|
||||
"privateKey": "ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80",
|
||||
"nitroAddress": "0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE",
|
||||
"nitroPrivateKey": "2d999770f7b5d49b694080f987b82bbc9fc9ac2b4dcc10b0f8aba7d700f69c6d"
|
||||
},
|
||||
"consensus": {
|
||||
"publicKey": "02cd17b05ca998955be5ca7bf4fd4531243d438f1aae7ce8a0ed5159f53cee5b40",
|
||||
"privateKey": "67d80505614bdf61fca11cbad31d93acb2c7df1c653dc25975d77d05f05f154f"
|
||||
}
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"peerId": {
|
||||
"id": "12D3KooWBNEbY3QS4y23ngupDw9PDc4bvNvRJGVRejjV9EZLjux5",
|
||||
"privKey": "CAESQGSTw0ymvn8+wX9Dbvyr4/Gib1q2voe0CC0VyeClMQP6FwW14x0fpRbBIx0XhLdxWHkRndphVg3gVAHyC+7ZI8o=",
|
||||
"pubKey": "CAESIBcFteMdH6UWwSMdF4S3cVh5EZ3aYVYN4FQB8gvu2SPK"
|
||||
},
|
||||
"relayPeerId": {
|
||||
"id": "12D3KooWPpBt2coC5BKEsewogHSkdgdyY9guUHJBgJFZoh66R4UL",
|
||||
"privKey": "CAESQBdoGZzpiHsE8GlIhGGP+tl6A13sIPxOhgSwsWZR4cwmz/f7627X5GQRLD41ikH6oZ7ZeSbiiPsBtelL1+t3M80=",
|
||||
"pubKey": "CAESIM/3++tu1+RkESw+NYpB+qGe2Xkm4oj7AbXpS9frdzPN"
|
||||
},
|
||||
"peer": {
|
||||
"privateKey": "59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d",
|
||||
"nitroAddress": "0xBBB676f9cFF8D242e9eaC39D063848807d3D1D94",
|
||||
"nitroPrivateKey": "0279651921cd800ac560c21ceea27aab0107b67daf436cdd25ce84cad30159b4"
|
||||
},
|
||||
"consensus": {
|
||||
"publicKey": "029c8035b3e9401b8f178f7c37285b5cb22501e017340e2058b3b842f2a1f0ae45",
|
||||
"privateKey": "0261008e8e3ec808168e99333599da38ca59a056a2ae4510a6ad3d8b5cb0918c"
|
||||
}
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"peerId": {
|
||||
"id": "12D3KooWSRH6ftgkAZsKZK7UX1Zr6Hx6YAsEepHqzopFszqfTxxi",
|
||||
"privKey": "CAESQHBjlHxfVhZ2gXsBItrIEEgSGKcjMkFiGs3PPz9E3ace9qyWEkvR4oit5ve9SAROVoh20hoa42IC91NIafMaqws=",
|
||||
"pubKey": "CAESIPaslhJL0eKIreb3vUgETlaIdtIaGuNiAvdTSGnzGqsL"
|
||||
},
|
||||
"relayPeerId": {
|
||||
"id": "12D3KooWLudDC3NnS8k9xLiqTZcS9xbFNrGSttu5GPynQgNgenPW",
|
||||
"privKey": "CAESQHuEXPB1hTQATlZaNRQnAfSvnfYUeLF9PjSNKp9L33K2pMl37tTBYghnfQ0yluse0zHOIXRJM+9+ARn9+jIr0gU=",
|
||||
"pubKey": "CAESIKTJd+7UwWIIZ30NMpbrHtMxziF0STPvfgEZ/foyK9IF"
|
||||
},
|
||||
"peer": {
|
||||
"privateKey": "47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a",
|
||||
"nitroAddress": "0x67D5b55604d1aF90074FcB69b8C51838FFF84f8d",
|
||||
"nitroPrivateKey": "58368d20ff12f17669c06158c21d885897aa56f9be430edc789614bf9851d53f"
|
||||
},
|
||||
"consensus": {
|
||||
"publicKey": "039160c244a7ad8be16a64bdb69e6dbacdcfe20b37076792a0d06032a8097468ca",
|
||||
"privateKey": "8894685fe81001d75662b079905472699373967451d1255ee5fc669d0a09a9ca"
|
||||
}
|
||||
}
|
||||
@@ -1,68 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
watcher_keys_dir="/server/keys"
|
||||
|
||||
# Configure relay nodes to connect to
|
||||
# Note: Assuming host ports for watchers (1-3) relay nodes
|
||||
RELAY_NODES="[\"/ip4/127.0.0.1/tcp/9090/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-1.json)\", \"/ip4/127.0.0.1/tcp/9091/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-2.json)\", \"/ip4/127.0.0.1/tcp/9092/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-3.json)\"]"
|
||||
|
||||
# Configure watcher (1) Nitro account to make payments to
|
||||
PAYMENT_NITRO_ADDRESS=$(jq -r '.peer.nitroAddress' ${watcher_keys_dir}/watcher-1.json)
|
||||
|
||||
echo "Using RELAY_NODES $RELAY_NODES"
|
||||
echo "Using PAYMENT_NITRO_ADDRESS $PAYMENT_NITRO_ADDRESS"
|
||||
|
||||
# Use config from mounted volume
|
||||
echo "Taking config for deployed contracts from mounted volume"
|
||||
while [ ! -f /server/config.json ]; do
|
||||
echo "Config not found, retrying in 5 seconds..."
|
||||
sleep 5
|
||||
done
|
||||
|
||||
# Get deployed contract address and chain id
|
||||
DEPLOYED_CONTRACT=$(jq -r '.address' /server/config.json | tr -d '"')
|
||||
CHAIN_ID=$(jq -r '.chainId' /server/config.json)
|
||||
|
||||
echo "Using DEPLOYED_CONTRACT $DEPLOYED_CONTRACT"
|
||||
echo "Using CHAIN_ID $CHAIN_ID"
|
||||
|
||||
nitro_addresses_file="/nitro/nitro-addresses.json"
|
||||
nitro_addresses_destination_file="/app/src/utils/nitro-addresses.json"
|
||||
if [ -f ${nitro_addresses_file} ]; then
|
||||
echo "Using Nitro addresses from ${nitro_addresses_file}:"
|
||||
cat "$nitro_addresses_file"
|
||||
cat "$nitro_addresses_file" > "$nitro_addresses_destination_file"
|
||||
else
|
||||
echo "Nitro addresses not available"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Export config values in a json file
|
||||
app_config_file="/app/src/utils/config.json"
|
||||
app_config_json=$(jq -n \
|
||||
--arg name "MobyMask" \
|
||||
--argjson enableDebugInfo true \
|
||||
--arg address "$DEPLOYED_CONTRACT" \
|
||||
--argjson chainId "$CHAIN_ID" \
|
||||
--argjson relayNodes "$RELAY_NODES" \
|
||||
--argjson denyMultiaddrs "[]" \
|
||||
--arg pubsub "" \
|
||||
--argjson directPeers "[]" \
|
||||
'.name = $name | .address = $address | .chainId = $chainId | .relayNodes = $relayNodes | .peer.enableDebugInfo = $enableDebugInfo | .peer.denyMultiaddrs = $denyMultiaddrs | .peer.pubsub = $pubsub | .peer.directPeers = $directPeers')
|
||||
echo "$app_config_json" > "${app_config_file}"
|
||||
|
||||
echo "Using CERC_APP_WATCHER_URL $CERC_APP_WATCHER_URL"
|
||||
echo "Using CERC_SNAP_URL $CERC_SNAP_URL"
|
||||
|
||||
REACT_APP_DEBUG_PEER=true \
|
||||
REACT_APP_WATCHER_URI="$CERC_APP_WATCHER_URL/graphql" \
|
||||
REACT_APP_PAY_TO_NITRO_ADDRESS="$PAYMENT_NITRO_ADDRESS" \
|
||||
REACT_APP_SNAP_ORIGIN="local:$CERC_SNAP_URL" \
|
||||
yarn build
|
||||
|
||||
http-server -p 80 /app/build
|
||||
@@ -1,119 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
watcher_keys_dir="./keys"
|
||||
|
||||
echo "Watcher index: ${CERC_WATCHER_INDEX}"
|
||||
echo "Using RPC query endpoint ${CERC_ETH_RPC_QUERY_ENDPOINT}"
|
||||
echo "Using RPC mutation endpoint ${CERC_ETH_RPC_MUTATION_ENDPOINT}"
|
||||
echo "Using Nitro chain URL ${CERC_NITRO_CHAIN_URL}"
|
||||
|
||||
WATCHER_DB="mobymask-watcher-${CERC_WATCHER_INDEX}"
|
||||
|
||||
# Use the docker container's host IP for relay multiaddr in peer config
|
||||
CERC_RELAY_MULTIADDR="/dns4/mobymask-watcher-${CERC_WATCHER_INDEX}-server/tcp/9090/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-${CERC_WATCHER_INDEX}.json)"
|
||||
|
||||
# Assign deployed contract address from server config (created by mobymask container after deploying contract)
|
||||
CONTRACT_ADDRESS=$(jq -r '.address' /server/config.json | tr -d '"')
|
||||
|
||||
nitro_addresses_file="/nitro/nitro-addresses.json"
|
||||
nitro_addresses_destination_file="./src/nitro-addresses.json"
|
||||
if [ -f ${nitro_addresses_file} ]; then
|
||||
echo "Using Nitro addresses from ${nitro_addresses_file}:"
|
||||
cat "$nitro_addresses_file"
|
||||
cat "$nitro_addresses_file" > "$nitro_addresses_destination_file"
|
||||
else
|
||||
echo "Nitro addresses not available"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build after setting the Nitro addresses
|
||||
yarn build
|
||||
|
||||
PEER_ID_FILE='./peer-id.json'
|
||||
RELAY_PEER_ID_FILE='./relay-id.json'
|
||||
|
||||
# Create watcher party array
|
||||
watcher_parties=()
|
||||
|
||||
# Iterate over each fixture JSON file
|
||||
for watcher_keys_file in "$watcher_keys_dir"/*.json; do
|
||||
# Extract the filename without the path and extension
|
||||
filename=$(basename "$watcher_keys_file" .json)
|
||||
|
||||
# Read the consensus keys
|
||||
peer_id=$(jq -r '.peerId.id' "$watcher_keys_file")
|
||||
|
||||
# Read the consensus keys
|
||||
consensus_public_key=$(jq -r '.consensus.publicKey' "$watcher_keys_file")
|
||||
consensus_private_key=$(jq -r '.consensus.privateKey' "$watcher_keys_file")
|
||||
|
||||
# Append watcher party
|
||||
watcher_party=$(jq -n \
|
||||
--arg peerId "$peer_id" \
|
||||
--arg publicKey "$consensus_public_key" \
|
||||
'.peerId = $peerId | .publicKey = $publicKey')
|
||||
watcher_parties+=("$watcher_party")
|
||||
|
||||
if [ "$filename" = "watcher-${CERC_WATCHER_INDEX}" ]; then
|
||||
# Export watcher peer and relay ids
|
||||
peer_id_data=$(jq '.peerId' "$watcher_keys_file")
|
||||
relay_peer_id_data=$(jq '.relayPeerId' "$watcher_keys_file")
|
||||
echo "$peer_id_data" > "${PEER_ID_FILE}"
|
||||
echo "$relay_peer_id_data" > "${RELAY_PEER_ID_FILE}"
|
||||
|
||||
# Set peer and nitro account keys
|
||||
CERC_PRIVATE_KEY_PEER=$(jq -r '.peer.privateKey' "$watcher_keys_file")
|
||||
CERC_WATCHER_NITRO_PK=$(jq -r '.peer.nitroPrivateKey' "$watcher_keys_file")
|
||||
|
||||
# Set consensus keys for this peer
|
||||
CONSENSUS_PUBLIC_KEY=${consensus_public_key}
|
||||
CONSENSUS_PRIVATE_KEY=${consensus_private_key}
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Consensus module enabled"
|
||||
CONSENSUS_ENABLED=true
|
||||
WATCHER_PARTY_PEERS_FILE='./watcher-party-peers.json'
|
||||
|
||||
# Export watcher party file
|
||||
watcher_parties_json=$(printf '%s\n' "${watcher_parties[@]}" | jq -s .)
|
||||
echo "$watcher_parties_json" > "${WATCHER_PARTY_PEERS_FILE}"
|
||||
echo "Watcher party peers exported to ${WATCHER_PARTY_PEERS_FILE}"
|
||||
|
||||
# Configure relay peers for watcher 2 and 3
|
||||
CERC_RELAY_PEERS=[]
|
||||
if [ "${CERC_WATCHER_INDEX}" = "2" ]; then
|
||||
# 2 -> 1
|
||||
CERC_RELAY_PEERS="['/dns4/mobymask-watcher-1-server/tcp/9090/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-1.json)']"
|
||||
elif [ "${CERC_WATCHER_INDEX}" = "3" ]; then
|
||||
# 3 -> 1, 2
|
||||
CERC_RELAY_PEERS="['/dns4/mobymask-watcher-1-server/tcp/9090/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-1.json)', '/dns4/mobymask-watcher-2-server/tcp/9090/ws/p2p/$(jq -r '.relayPeerId.id' ${watcher_keys_dir}/watcher-2.json)']"
|
||||
fi
|
||||
|
||||
# Read in the config template TOML file and modify it
|
||||
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
|
||||
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
|
||||
sed -E "s|REPLACE_WITH_CERC_RELAY_PEERS|${CERC_RELAY_PEERS}|g; \
|
||||
s|REPLACE_WITH_CERC_RELAY_MULTIADDR|${CERC_RELAY_MULTIADDR}|g; \
|
||||
s/REPLACE_WITH_CERC_PRIVATE_KEY_PEER/${CERC_PRIVATE_KEY_PEER}/g; \
|
||||
s/REPLACE_WITH_CERC_WATCHER_NITRO_PK/${CERC_WATCHER_NITRO_PK}/g; \
|
||||
s/REPLACE_WITH_CONTRACT_ADDRESS/${CONTRACT_ADDRESS}/g; \
|
||||
s|REPLACE_WITH_CERC_NITRO_CHAIN_URL|${CERC_NITRO_CHAIN_URL}|g; \
|
||||
s/REPLACE_WITH_CONSENSUS_ENABLED/${CONSENSUS_ENABLED}/g; \
|
||||
s/REPLACE_WITH_CONSENSUS_PUBLIC_KEY/${CONSENSUS_PUBLIC_KEY}/g; \
|
||||
s/REPLACE_WITH_CONSENSUS_PRIVATE_KEY/${CONSENSUS_PRIVATE_KEY}/g; \
|
||||
s|REPLACE_WITH_WATCHER_PARTY_PEERS_FILE|${WATCHER_PARTY_PEERS_FILE}|g; \
|
||||
s|REPLACE_WITH_CERC_ETH_RPC_QUERY_ENDPOINT|${CERC_ETH_RPC_QUERY_ENDPOINT}|g; \
|
||||
s|REPLACE_WITH_CERC_ETH_RPC_MUTATION_ENDPOINT|${CERC_ETH_RPC_MUTATION_ENDPOINT}|g; \
|
||||
s|REPLACE_WITH_WATCHER_DB|${WATCHER_DB}| ")
|
||||
|
||||
# Write the modified content to a new file
|
||||
echo "$WATCHER_CONFIG" > environments/local.toml
|
||||
|
||||
echo 'Running watcher server'
|
||||
yarn server
|
||||
@@ -1,14 +0,0 @@
|
||||
freeQueriesLimit = 10
|
||||
|
||||
freeQueriesList = []
|
||||
|
||||
[queries]
|
||||
multiNonce = '50'
|
||||
_owner = '50'
|
||||
isRevoked = '50'
|
||||
isPhisher = '50'
|
||||
isMember = '50'
|
||||
|
||||
[mutations]
|
||||
invoke = '100'
|
||||
revoke = '100'
|
||||
@@ -1,111 +0,0 @@
|
||||
[server]
|
||||
host = "0.0.0.0"
|
||||
port = 3001
|
||||
kind = "lazy"
|
||||
|
||||
# Checkpointing state.
|
||||
checkpointing = false
|
||||
|
||||
# Checkpoint interval in number of blocks.
|
||||
checkpointInterval = 2000
|
||||
|
||||
# Enable state creation
|
||||
enableState = false
|
||||
|
||||
# Boolean to filter logs by contract.
|
||||
filterLogs = true
|
||||
|
||||
# Max block range for which to return events in eventsInRange GQL query.
|
||||
# Use -1 for skipping check on block range.
|
||||
maxEventsBlockRange = -1
|
||||
|
||||
# Flag to specify whether RPC endpoint supports block hash as block tag parameter
|
||||
rpcSupportsBlockHashParam = true
|
||||
|
||||
[server.p2p]
|
||||
enableRelay = true
|
||||
enablePeer = true
|
||||
|
||||
[server.p2p.relay]
|
||||
host = "0.0.0.0"
|
||||
port = 9090
|
||||
relayPeers = REPLACE_WITH_CERC_RELAY_PEERS
|
||||
denyMultiaddrs = []
|
||||
peerIdFile = './relay-id.json'
|
||||
announce = ''
|
||||
pubsub = ''
|
||||
enableDebugInfo = true
|
||||
|
||||
[server.p2p.peer]
|
||||
relayMultiaddr = 'REPLACE_WITH_CERC_RELAY_MULTIADDR'
|
||||
pubSubTopic = 'mobymask'
|
||||
denyMultiaddrs = []
|
||||
peerIdFile = './peer-id.json'
|
||||
pubsub = ''
|
||||
enableDebugInfo = true
|
||||
enableL2Txs = true
|
||||
pingInterval = 4000
|
||||
pingTimeout = 1500
|
||||
maxRelayConnections = 10
|
||||
|
||||
[server.p2p.peer.l2TxsConfig]
|
||||
privateKey = 'REPLACE_WITH_CERC_PRIVATE_KEY_PEER'
|
||||
contractAddress = 'REPLACE_WITH_CONTRACT_ADDRESS'
|
||||
|
||||
[server.p2p.nitro]
|
||||
chainUrl = 'REPLACE_WITH_CERC_NITRO_CHAIN_URL'
|
||||
store = './out/nitro-db'
|
||||
privateKey = 'REPLACE_WITH_CERC_WATCHER_NITRO_PK'
|
||||
chainPrivateKey = 'REPLACE_WITH_CERC_PRIVATE_KEY_PEER'
|
||||
|
||||
[server.p2p.nitro.payments]
|
||||
ratesFile = './environments/rates.toml'
|
||||
requestTimeoutInSecs = 10
|
||||
|
||||
[server.p2p.nitro.payments.cache]
|
||||
maxAccounts = 1000
|
||||
accountTTLInSecs = 1800
|
||||
maxVouchersPerAccount = 1000
|
||||
voucherTTLInSecs = 300
|
||||
maxPaymentChannels = 10000
|
||||
paymentChannelTTLInSecs = 1800
|
||||
|
||||
[server.p2p.consensus]
|
||||
enabled = REPLACE_WITH_CONSENSUS_ENABLED
|
||||
publicKey = 'REPLACE_WITH_CONSENSUS_PUBLIC_KEY'
|
||||
privateKey = 'REPLACE_WITH_CONSENSUS_PRIVATE_KEY'
|
||||
watcherPartyPeersFile = 'REPLACE_WITH_WATCHER_PARTY_PEERS_FILE'
|
||||
|
||||
[metrics]
|
||||
host = "0.0.0.0"
|
||||
port = 9000
|
||||
[metrics.gql]
|
||||
port = 9001
|
||||
|
||||
[database]
|
||||
type = "postgres"
|
||||
host = "mobymask-watcher-db"
|
||||
port = 5432
|
||||
database = "REPLACE_WITH_WATCHER_DB"
|
||||
username = "vdbm"
|
||||
password = "password"
|
||||
synchronize = true
|
||||
logging = false
|
||||
|
||||
[upstream]
|
||||
[upstream.ethServer]
|
||||
gqlApiEndpoint = "http://ipld-eth-server:8083/graphql"
|
||||
rpcProviderEndpoint = 'REPLACE_WITH_CERC_ETH_RPC_QUERY_ENDPOINT'
|
||||
rpcProviderMutationEndpoint = 'REPLACE_WITH_CERC_ETH_RPC_MUTATION_ENDPOINT'
|
||||
|
||||
[upstream.cache]
|
||||
name = "requests"
|
||||
enabled = false
|
||||
deleteOnStart = false
|
||||
|
||||
[jobQueue]
|
||||
dbConnectionString = "postgres://vdbm:password@mobymask-watcher-db/REPLACE_WITH_WATCHER_DB-job-queue"
|
||||
maxCompletionLagInSecs = 300
|
||||
jobDelayInMilliSecs = 100
|
||||
eventsInBatch = 50
|
||||
blockDelayInMilliSecs = 60000
|
||||
@@ -8,9 +8,30 @@ COPY . .
|
||||
RUN go build -v -o nitro .
|
||||
|
||||
# Reduce image size
|
||||
FROM debian:bullseye-slim
|
||||
FROM node:18-bullseye-slim as builder-node
|
||||
RUN apt-get update
|
||||
RUN apt-get install -y ca-certificates jq netcat
|
||||
RUN apt-get install -y make
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN find . -name 'node_modules' | xargs -n1 rm -rf
|
||||
RUN find . -name 'dist' | xargs -n1 rm -rf
|
||||
RUN yarn
|
||||
WORKDIR /app/packages/nitro-gui
|
||||
RUN VITE_RPC_HOST=CERC_RUNTIME_ENV_RPC_HOST yarn build
|
||||
RUN if [ -d /app/packages/nitro-auth-gui ]; then \
|
||||
cd /app/packages/nitro-auth-gui && \
|
||||
VITE_RPC_URL=CERC_RUNTIME_ENV_RPC_URL VITE_TARGET_URL=CERC_RUNTIME_ENV_TARGET_URL yarn build; \
|
||||
fi
|
||||
WORKDIR /app/packages/nitro-auth
|
||||
RUN if [ -d /app/packages/nitro-auth ]; then cd /app/packages/nitro-auth && yarn build; fi
|
||||
|
||||
FROM node:18-bullseye-slim
|
||||
RUN apt-get update
|
||||
RUN apt-get install -y ca-certificates jq netcat make curl wget
|
||||
RUN rm -rf /var/lib/apt/lists/*
|
||||
RUN npm install -g http-server
|
||||
RUN npm config set @cerc-io:registry https://git.vdb.to/api/packages/cerc-io/npm/
|
||||
RUN npm install -g @cerc-io/peer
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/nitro .
|
||||
COPY --from=builder-node /app /app-node
|
||||
@@ -26,6 +26,8 @@ RUN \
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g eslint" \
|
||||
# Install semver
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g semver" \
|
||||
# Install pnpm
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g pnpm" \
|
||||
&& npm cache clean --force > /dev/null 2>&1
|
||||
|
||||
# [Optional] Uncomment this section to install additional OS packages.
|
||||
@@ -35,6 +37,9 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
|
||||
# [Optional] Uncomment if you want to install more global node modules
|
||||
# RUN su node -c "npm install -g <your-package-list-here>"
|
||||
|
||||
# We do this to get a yq binary from the published container, for the correct architecture we're building here
|
||||
COPY --from=docker.io/mikefarah/yq:latest /usr/bin/yq /usr/local/bin/yq
|
||||
|
||||
# Expose port for http
|
||||
EXPOSE 80
|
||||
|
||||
|
||||
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
|
||||
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/nextjs-base:local}
|
||||
|
||||
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
|
||||
rc=$?
|
||||
|
||||
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "BUILD FAILED" 1>&2
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
|
||||
cat <<EOF
|
||||
|
||||
#################################################################
|
||||
|
||||
+5
-3
@@ -10,10 +10,12 @@ TRG_DIR="${3:-.next-r}"
|
||||
|
||||
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
|
||||
if [ -z "$CERC_BUILD_TOOL" ]; then
|
||||
if [ -f "yarn.lock" ]; then
|
||||
CERC_BUILD_TOOL=npm
|
||||
else
|
||||
if [ -f "pnpm-lock.yaml" ]; then
|
||||
CERC_BUILD_TOOL=pnpm
|
||||
elif [ -f "yarn.lock" ]; then
|
||||
CERC_BUILD_TOOL=yarn
|
||||
else
|
||||
CERC_BUILD_TOOL=npm
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -9,7 +9,9 @@ CERC_MIN_NEXTVER=13.4.2
|
||||
CERC_NEXT_VERSION="${CERC_NEXT_VERSION:-keep}"
|
||||
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
|
||||
if [ -z "$CERC_BUILD_TOOL" ]; then
|
||||
if [ -f "yarn.lock" ]; then
|
||||
if [ -f "pnpm-lock.yaml" ]; then
|
||||
CERC_BUILD_TOOL=pnpm
|
||||
elif [ -f "yarn.lock" ]; then
|
||||
CERC_BUILD_TOOL=yarn
|
||||
else
|
||||
CERC_BUILD_TOOL=npm
|
||||
@@ -113,7 +115,7 @@ if [ "$CERC_NEXT_VERSION" != "keep" ] && [ "$CUR_NEXT_VERSION" != "$CERC_NEXT_VE
|
||||
mv package.json.$$ package.json
|
||||
fi
|
||||
|
||||
$CERC_BUILD_TOOL install || exit 1
|
||||
time $CERC_BUILD_TOOL install || exit 1
|
||||
|
||||
CUR_NEXT_VERSION=`jq -r '.version' node_modules/next/package.json`
|
||||
|
||||
@@ -136,9 +138,9 @@ to use for the build with:
|
||||
EOF
|
||||
cat package.json | jq ".dependencies.next = \"^$CERC_MIN_NEXTVER\"" > package.json.$$
|
||||
mv package.json.$$ package.json
|
||||
$CERC_BUILD_TOOL install || exit 1
|
||||
time $CERC_BUILD_TOOL install || exit 1
|
||||
fi
|
||||
|
||||
$CERC_BUILD_TOOL run cerc_compile || exit 1
|
||||
time $CERC_BUILD_TOOL run cerc_compile || exit 1
|
||||
|
||||
exit 0
|
||||
|
||||
+3
-1
@@ -16,7 +16,9 @@ trap ctrl_c INT
|
||||
|
||||
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
|
||||
if [ -z "$CERC_BUILD_TOOL" ]; then
|
||||
if [ -f "yarn.lock" ] && [ ! -f "package-lock.json" ]; then
|
||||
if [ -f "pnpm-lock.yaml" ]; then
|
||||
CERC_BUILD_TOOL=pnpm
|
||||
elif [ -f "yarn.lock" ]; then
|
||||
CERC_BUILD_TOOL=yarn
|
||||
else
|
||||
CERC_BUILD_TOOL=npm
|
||||
|
||||
@@ -4,9 +4,10 @@ RUN apk --update --no-cache add python3 alpine-sdk bash curl jq
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY . .
|
||||
COPY packages/nitro-rpc-client .
|
||||
|
||||
RUN echo "Installing dependencies" && \
|
||||
yarn
|
||||
|
||||
RUN cd packages/nitro-rpc-client
|
||||
RUN yarn build
|
||||
RUN ln -s /app/bin/nitro-rpc-client.js /bin/nitro-rpc-client
|
||||
|
||||
@@ -3,17 +3,12 @@ FROM cerc/foundry:local
|
||||
# Install node (local foundry is a debian based image)
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y curl wget \
|
||||
&& apt-get install -y git busybox jq python3 make gcc g++
|
||||
|
||||
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash \
|
||||
&& export NVM_DIR="$HOME/.nvm" \
|
||||
&& [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" \
|
||||
&& nvm install 16.16.0 \
|
||||
&& curl --silent --location https://deb.nodesource.com/setup_16.x | bash - \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y nodejs git busybox jq \
|
||||
&& node -v
|
||||
|
||||
SHELL ["/bin/bash", "-c"]
|
||||
RUN source $HOME/.bashrc \
|
||||
&& corepack enable \
|
||||
RUN corepack enable \
|
||||
&& yarn --version
|
||||
|
||||
WORKDIR /app
|
||||
@@ -21,10 +16,7 @@ WORKDIR /app
|
||||
# Copy optimism repo contents
|
||||
COPY . .
|
||||
|
||||
SHELL ["/bin/bash", "-c"]
|
||||
RUN source $HOME/.bashrc \
|
||||
&& echo "Building optimism" \
|
||||
&& pnpm i -g pnpm@8.15.3 \
|
||||
&& pnpm install && pnpm build
|
||||
RUN echo "Building optimism" && \
|
||||
pnpm install && pnpm build
|
||||
|
||||
WORKDIR /app/packages/contracts-bedrock
|
||||
|
||||
@@ -39,6 +39,15 @@ fi
|
||||
if [ -n "$CERC_TEST_PARAM_2" ]; then
|
||||
echo "Test-param-2: ${CERC_TEST_PARAM_2}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_3" ]; then
|
||||
echo "Test-param-3: ${CERC_TEST_PARAM_3}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_4" ]; then
|
||||
echo "Test-param-4: ${CERC_TEST_PARAM_4}"
|
||||
fi
|
||||
if [ -n "$CERC_TEST_PARAM_5" ]; then
|
||||
echo "Test-param-5: ${CERC_TEST_PARAM_5}"
|
||||
fi
|
||||
|
||||
if [ -d "/config" ]; then
|
||||
echo "/config: EXISTS"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Originally from: https://github.com/devcontainers/images/blob/main/src/javascript-node/.devcontainer/Dockerfile
|
||||
# [Choice] Node.js version (use -bullseye variants on local arm64/Apple Silicon): 18, 16, 14, 18-bullseye, 16-bullseye, 14-bullseye, 18-buster, 16-buster, 14-buster
|
||||
ARG VARIANT=20-bullseye
|
||||
ARG VARIANT=20-bullseye-slim
|
||||
FROM node:${VARIANT}
|
||||
|
||||
ARG USERNAME=node
|
||||
@@ -24,6 +24,10 @@ RUN \
|
||||
&& su ${USERNAME} -c "npm config -g set prefix ${NPM_GLOBAL}" \
|
||||
# Install eslint
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g eslint" \
|
||||
# Install semver
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g semver" \
|
||||
# Install pnpm
|
||||
&& su ${USERNAME} -c "umask 0002 && npm install -g pnpm" \
|
||||
&& npm cache clean --force > /dev/null 2>&1
|
||||
|
||||
# [Optional] Uncomment this section to install additional OS packages.
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
FROM cerc/webapp-base:local as builder
|
||||
|
||||
ARG CERC_BUILD_TOOL
|
||||
ARG CERC_BUILD_OUTPUT_DIR
|
||||
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN rm -rf node_modules build .next*
|
||||
RUN /scripts/build-app.sh /app build /data
|
||||
RUN rm -rf node_modules build dist .next*
|
||||
RUN /scripts/build-app.sh /app /data
|
||||
|
||||
FROM cerc/webapp-base:local
|
||||
COPY --from=builder /data /data
|
||||
|
||||
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
|
||||
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/webapp-base:local}
|
||||
|
||||
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
|
||||
rc=$?
|
||||
|
||||
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "BUILD FAILED" 1>&2
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
|
||||
cat <<EOF
|
||||
|
||||
#################################################################
|
||||
|
||||
@@ -7,9 +7,10 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
fi
|
||||
|
||||
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
|
||||
CERC_BUILD_OUTPUT_DIR="${CERC_BUILD_OUTPUT_DIR}"
|
||||
|
||||
WORK_DIR="${1:-/app}"
|
||||
OUTPUT_DIR="${2:-build}"
|
||||
DEST_DIR="${3:-/data}"
|
||||
DEST_DIR="${2:-/data}"
|
||||
|
||||
if [ -f "${WORK_DIR}/build-webapp.sh" ]; then
|
||||
echo "Building webapp with ${WORK_DIR}/build-webapp.sh ..."
|
||||
@@ -22,18 +23,30 @@ elif [ -f "${WORK_DIR}/package.json" ]; then
|
||||
cd "${WORK_DIR}" || exit 1
|
||||
|
||||
if [ -z "$CERC_BUILD_TOOL" ]; then
|
||||
if [ -f "yarn.lock" ]; then
|
||||
if [ -f "pnpm-lock.yaml" ]; then
|
||||
CERC_BUILD_TOOL=pnpm
|
||||
elif [ -f "yarn.lock" ]; then
|
||||
CERC_BUILD_TOOL=yarn
|
||||
else
|
||||
CERC_BUILD_TOOL=npm
|
||||
fi
|
||||
fi
|
||||
|
||||
$CERC_BUILD_TOOL install || exit 1
|
||||
$CERC_BUILD_TOOL build || exit 1
|
||||
time $CERC_BUILD_TOOL install || exit 1
|
||||
time $CERC_BUILD_TOOL build || exit 1
|
||||
|
||||
rm -rf "${DEST_DIR}"
|
||||
mv "${WORK_DIR}/${OUTPUT_DIR}" "${DEST_DIR}"
|
||||
if [ -z "${CERC_BUILD_OUTPUT_DIR}" ]; then
|
||||
if [ -d "${WORK_DIR}/dist" ]; then
|
||||
CERC_BUILD_OUTPUT_DIR="${WORK_DIR}/dist"
|
||||
elif [ -d "${WORK_DIR}/build" ]; then
|
||||
CERC_BUILD_OUTPUT_DIR="${WORK_DIR}/build"
|
||||
else
|
||||
echo "ERROR: Unable to locate build output. Set with --extra-build-args \"--build-arg CERC_BUILD_OUTPUT_DIR=path\"" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
mv "${CERC_BUILD_OUTPUT_DIR}" "${DEST_DIR}"
|
||||
else
|
||||
echo "Copying static app ..."
|
||||
mv "${WORK_DIR}" "${DEST_DIR}"
|
||||
|
||||
+17
-2
@@ -3,13 +3,28 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
CERC_LISTEN_PORT=${CERC_LISTEN_PORT:-80}
|
||||
CERC_WEBAPP_FILES_DIR="${CERC_WEBAPP_FILES_DIR:-/data}"
|
||||
CERC_ENABLE_CORS="${CERC_ENABLE_CORS:-false}"
|
||||
CERC_SINGLE_PAGE_APP="${CERC_SINGLE_PAGE_APP}"
|
||||
|
||||
if [ -z "${CERC_SINGLE_PAGE_APP}" ]; then
|
||||
if [ 1 -eq $(find "${CERC_WEBAPP_FILES_DIR}" -name '*.html' | wc -l) ] && [ -d "${CERC_WEBAPP_FILES_DIR}/static" ]; then
|
||||
CERC_SINGLE_PAGE_APP=true
|
||||
else
|
||||
CERC_SINGLE_PAGE_APP=false
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "true" == "$CERC_ENABLE_CORS" ]; then
|
||||
CERC_HTTP_EXTRA_ARGS="$CERC_HTTP_EXTRA_ARGS --cors"
|
||||
fi
|
||||
|
||||
if [ "true" == "$CERC_SINGLE_PAGE_APP" ]; then
|
||||
# Create a catchall redirect back to /
|
||||
CERC_HTTP_EXTRA_ARGS="$CERC_HTTP_EXTRA_ARGS --proxy http://localhost:${CERC_LISTEN_PORT}?"
|
||||
fi
|
||||
|
||||
LACONIC_HOSTED_CONFIG_FILE=${LACONIC_HOSTED_CONFIG_FILE}
|
||||
if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
if [ -f "/config/laconic-hosted-config.yml" ]; then
|
||||
@@ -20,8 +35,8 @@ if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
fi
|
||||
|
||||
if [ -f "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
|
||||
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE ${CERC_WEBAPP_FILES_DIR}
|
||||
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE "${CERC_WEBAPP_FILES_DIR}"
|
||||
fi
|
||||
|
||||
/scripts/apply-runtime-env.sh ${CERC_WEBAPP_FILES_DIR}
|
||||
http-server $CERC_HTTP_EXTRA_ARGS -p ${CERC_LISTEN_PORT:-80} ${CERC_WEBAPP_FILES_DIR}
|
||||
http-server $CERC_HTTP_EXTRA_ARGS -p ${CERC_LISTEN_PORT} "${CERC_WEBAPP_FILES_DIR}"
|
||||
@@ -0,0 +1,104 @@
|
||||
# go-nitro-auth
|
||||
|
||||
Deploy a stack for demoing Nitro-based auth, using either a local fixturenet (fully self-contained) or remote testnet.
|
||||
|
||||
## Local Fixturenet (Self-Contained)
|
||||
|
||||
### Clone required repositories
|
||||
|
||||
```
|
||||
$ laconic-so --stack fixturenet-eth setup-repositories
|
||||
$ laconic-so --stack go-nitro-auth setup-repositories
|
||||
```
|
||||
|
||||
### Build containers
|
||||
|
||||
```
|
||||
$ laconic-so --stack fixturenet-eth build-containers
|
||||
$ laconic-so --stack go-nitro-auth build-containers
|
||||
```
|
||||
|
||||
### Create a deployment
|
||||
|
||||
```
|
||||
$ laconic-so --stack fixturenet-eth deploy init --output nitro-net.yml
|
||||
$ laconic-so --stack fixturenet-eth deploy create --spec-file nitro-net.yml --deployment-dir /srv/nitro-net
|
||||
|
||||
$ laconic-so --stack go-nitro-auth deploy init --map-ports-to-host any-same --output nitro-auth.yml
|
||||
$ laconic-so --stack go-nitro-auth deploy create --spec-file nitro-auth.yml --deployment-dir /srv/nitro-auth
|
||||
|
||||
# Place them both in the same namespace (TODO: support setting the deployment name via --cluster).
|
||||
$ cp /srv/nitro-net/deployment.yml /srv/nitro-auth/deployment.yml
|
||||
```
|
||||
|
||||
### Start the containers
|
||||
|
||||
```
|
||||
$ laconic-so deployment --dir /srv/nitro-net up
|
||||
$ laconic-so deployment --dir /srv/nitro-auth up
|
||||
```
|
||||
|
||||
### Open the webapp
|
||||
|
||||
Visit http://localhost:5678
|
||||
|
||||
## Remote Testnet
|
||||
|
||||
This example will use the Linea Sepolia testnet.
|
||||
|
||||
### Clone required repositories
|
||||
|
||||
```
|
||||
$ laconic-so --stack go-nitro-auth setup-repositories
|
||||
```
|
||||
|
||||
### Build containers
|
||||
|
||||
```
|
||||
$ laconic-so --stack go-nitro-auth build-containers
|
||||
```
|
||||
|
||||
### Create a deployment
|
||||
|
||||
```
|
||||
$ laconic-so --stack go-nitro-auth deploy init --map-ports-to-host any-same --output nitro-auth.yml
|
||||
$ laconic-so --stack go-nitro-auth deploy create --spec-file nitro-auth.yml --deployment-dir /srv/nitro-auth
|
||||
```
|
||||
|
||||
### Set your keys, contract addresses, etc.
|
||||
|
||||
You must set the private keys for two accounts with funds on the target network, as well as the contract addresses
|
||||
(if they already exist) or else an account to create them. You must also set the URL to use for WebSocket connections,
|
||||
eg, `wss://linea-sepolia.infura.io/ws/v3/<MY_API_KEY>`
|
||||
|
||||
#### Config
|
||||
|
||||
```
|
||||
$ vim /srv/nitro-auth/config.env
|
||||
# Addresses of existing contracts.
|
||||
CERC_CA_ADDRESS="0x1Ae815c3e7556e16ceaB6B6d46306C1870EB6d24"
|
||||
CERC_NA_ADDRESS="0xc453C5E3f304bb545A3Df7bBa02fe6274A056636"
|
||||
CERC_VPA_ADDRESS="0xA11af80D75b1150631FA78178c94fa451c7172a8"
|
||||
|
||||
# Else the private key of an account and RPC URL to use create them.
|
||||
CERC_PRIVATE_KEY_DEPLOYER=<PRIV_KEY_HERE>
|
||||
CERC_ETH_RPC_ENDPOINT=https://rpc.sepolia.linea.build
|
||||
|
||||
# The WebSocket chain URL.
|
||||
CERC_NITRO_CHAIN_URL=wss://linea-sepolia.infura.io/ws/v3/<MY_API_KEY_HERE>
|
||||
|
||||
# Private key for "Alice" account (payer)
|
||||
CERC_NITRO_CHAIN_PK_ALICE=<ALICE_PRIVATE_KEY_HERE>
|
||||
|
||||
# Private key for "Bob" account (payee)
|
||||
CERC_NITRO_CHAIN_PK_BOB=<BOB_PRIVATE_KEY_HERE>
|
||||
```
|
||||
|
||||
### Start the stack
|
||||
```
|
||||
$ laconic-so deployment --dir /srv/nitro-auth up
|
||||
```
|
||||
|
||||
### Open the webapp
|
||||
|
||||
Visit http://localhost:5678
|
||||
@@ -0,0 +1,16 @@
|
||||
version: "1.0"
|
||||
name: go-nitro-auth
|
||||
description: "Stack to demonstrate payments between various services"
|
||||
repos:
|
||||
# for nitro-contracts
|
||||
- github.com/cerc-io/ts-nitro@v0.1.15
|
||||
# for go-nitro and nitro-rpc-client
|
||||
- github.com/cerc-io/go-nitro@telackey/update
|
||||
containers:
|
||||
# nitro
|
||||
- cerc/nitro-contracts
|
||||
- cerc/go-nitro
|
||||
- cerc/nitro-rpc-client
|
||||
pods:
|
||||
- nitro-contracts
|
||||
- go-nitro-auth
|
||||
@@ -1,95 +0,0 @@
|
||||
# MobyMask v3 Demo
|
||||
|
||||
Instructions to setup and deploy an end-to-end MobyMask v3 stack (fixturenet-optimism + watchers + web-app) locally using [laconic-stack-orchestrator](/README.md#install)
|
||||
|
||||
## Setup
|
||||
|
||||
* Clone required repositories:
|
||||
|
||||
```bash
|
||||
laconic-so --stack mobymask-v3-demo setup-repositories --pull
|
||||
```
|
||||
|
||||
* Build the container images:
|
||||
|
||||
```bash
|
||||
laconic-so --stack mobymask-v3-demo build-containers
|
||||
```
|
||||
|
||||
* Install MetaMask Flask extension in a chromium browser from chrome web store and follow the setup instructions
|
||||
|
||||
## Deploy
|
||||
|
||||
* Create a spec file for the deployment:
|
||||
|
||||
```bash
|
||||
laconic-so --stack mobymask-v3-demo deploy init --output mobymask-v3-demo-spec.yml
|
||||
```
|
||||
|
||||
* Create a deployment from the generated spec file:
|
||||
|
||||
```bash
|
||||
laconic-so --stack mobymask-v3-demo deploy create --spec-file mobymask-v3-demo-spec.yml --deployment-dir mobymask-v3-demo-deployment
|
||||
```
|
||||
|
||||
* Copy over the demo config to place it at an appropriate path:
|
||||
|
||||
```bash
|
||||
cp mobymask-v3-demo-deployment/config/watcher-mobymask-v3-demo/local/config.env mobymask-v3-demo-deployment/
|
||||
```
|
||||
|
||||
### Start the stack
|
||||
|
||||
* Start the deployment:
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir mobymask-v3-demo-deployment start
|
||||
|
||||
# Useful ports exposed on host
|
||||
# 3001: MobyMask v3 watcher 1 GQL endpoint
|
||||
# 9090: MobyMask v3 watcher 1 relay node endpoint
|
||||
# 9091: MobyMask v3 watcher 2 relay node endpoint
|
||||
# 9092: MobyMask v3 watcher 3 relay node endpoint
|
||||
# 8080: MobyMask snap
|
||||
# 3004: MobyMask v3 app
|
||||
```
|
||||
|
||||
Note: This may take several minutes as it configures and runs the L1, L2 chains and the watchers; you can follow the progress from containers' logs
|
||||
|
||||
* To list and monitor the running containers:
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir mobymask-v3-demo-deployment ps
|
||||
|
||||
# With status
|
||||
docker ps
|
||||
|
||||
# Check logs for a container
|
||||
docker logs -f <CONTAINER_ID>
|
||||
```
|
||||
|
||||
## Demo
|
||||
|
||||
Follow [demo](./demo.md) to try out the MobyMask app
|
||||
|
||||
## Clean up
|
||||
|
||||
To stop all services running in the background, while preserving data:
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir mobymask-v3-demo-deployment stop
|
||||
```
|
||||
|
||||
To stop all services and also delete data:
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir mobymask-v3-demo-deployment stop --delete-volumes
|
||||
|
||||
# Also remove the deployment directory
|
||||
rm -rf mobymask-v3-demo-deployment
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
* Resource requirements (memory + time) for building the `cerc/foundry` image are on the higher side
|
||||
* `cerc/optimism-contracts` image is currently based on `cerc/foundry` (Optimism requires foundry installation)
|
||||
@@ -1,195 +0,0 @@
|
||||
# Demo
|
||||
|
||||
## Setup
|
||||
|
||||
* Follow logs and check that all 3 watchers are running in consensus:
|
||||
|
||||
```bash
|
||||
# Follow the logs in three different terminals and keep them running
|
||||
docker logs -f $(docker ps --filter "name=mobymask-watcher-1" -q)
|
||||
docker logs -f $(docker ps --filter "name=mobymask-watcher-2" -q)
|
||||
docker logs -f $(docker ps --filter "name=mobymask-watcher-3" -q)
|
||||
|
||||
# Expected output when all three are running in consensus:
|
||||
# ...
|
||||
# 2024-02-21T10:42:23.932Z laconic:consensus State changed to 3 (FOLLOWER) with term 286
|
||||
# 2024-02-21T10:42:23.932Z laconic:consensus State changed to 2 (CANDIDATE) with term 287
|
||||
# 2024-02-21T10:42:24.406Z laconic:consensus State changed to 1 (LEADER) with term 287
|
||||
# ...
|
||||
|
||||
# At any moment, only one of watchers is the 'LEADER'
|
||||
```
|
||||
|
||||
* In MetaMask, go to settings add a custom network with the following settings:
|
||||
|
||||
```bash
|
||||
# Network name
|
||||
Local Optimism
|
||||
|
||||
# New RPC URL
|
||||
http://127.0.0.1:8545
|
||||
|
||||
# Chain ID
|
||||
42069
|
||||
|
||||
# Currency symbol
|
||||
ETH
|
||||
```
|
||||
|
||||
Switch to the newly added network
|
||||
|
||||
* Import a pre-funded account (using it's private key) for Nitro client in the MobyMask app:
|
||||
|
||||
```bash
|
||||
# PK: 689af8efa8c651a91ad287602527f3af2fe9f6501a7ac4b061667b5a93e037fd
|
||||
# Address: 0xbDA5747bFD65F08deb54cb465eB87D40e51B197E
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
### Open MobyMask app
|
||||
|
||||
* Copy the generated invite link from MobyMask deployment container logs:
|
||||
|
||||
```bash
|
||||
docker logs -f $(docker ps -a --filter "name=mobymask-1" -q)
|
||||
|
||||
# A SIGNED DELEGATION/INVITE LINK:
|
||||
# ...
|
||||
# http://127.0.0.1:3004/#/members?invitation=<INVITATION>
|
||||
```
|
||||
|
||||
* Open the invite link in browser where MetaMask was setup
|
||||
|
||||
* In the app’s debug panel (bottom-right), check from `PEERS` and `GRAPH` tabs that the peer gets connected to relay nodes and watcher peers
|
||||
|
||||
* Perform phisher status checks from the app
|
||||
* First 10 queries are served for free; repeat until the free quota is exhausted
|
||||
* Same can be seen in the watcher-1's logs (the app makes all the GQL queries to watcher-1):
|
||||
|
||||
```bash
|
||||
# ...
|
||||
# 2024-02-21T11:01:20.084Z laconic:payments Query rate not configured for "latestBlock", serving free query
|
||||
# 2024-02-21T11:01:20.084Z vulcanize:resolver latestBlock
|
||||
# 2024-02-21T11:01:20.108Z laconic:payments Serving a free query to 0x3c9B491ACA5cf17B6C11E39bbFddCA603F387d41
|
||||
# 2024-02-21T11:01:20.109Z vulcanize:resolver isPhisher 0xf8995f83bbab2bc13fb9a43c6fea4a605616e6f59503f36cbe2ad3abffd0efd3 0xAFA36c47E130d89bcE4470a9030d99f3CEcaD146 TWT:dummyPhisher
|
||||
# 2024-02-21T11:01:20.113Z vulcanize:indexer isPhisher: db miss, fetching from upstream server
|
||||
# ...
|
||||
|
||||
# After free quota has been exhausted:
|
||||
# 2024-02-21T11:06:49.311Z laconic:payments Query rate not configured for "latestBlock", serving free query
|
||||
# 2024-02-21T11:06:49.312Z vulcanize:resolver latestBlock
|
||||
# 2024-02-21T11:06:49.337Z laconic:payments Rejecting query from 0x3c9B491ACA5cf17B6C11E39bbFddCA603F387d41: Free quota exhausted
|
||||
```
|
||||
|
||||
### Setup app's Nitro node
|
||||
|
||||
* Open the `NITRO` tab in debug panel
|
||||
|
||||
* Click on `Connect Wallet` button to connect to MetaMask (use the imported account with funds)
|
||||
|
||||
* Click on `Connect Snap` to install / connect snap; the watcher Nitro clients should show up in the `NITRO` tab
|
||||
|
||||
* Click on `DIRECT FUND` button against watcher-1's Nitro account (`0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE`) to create a ledger channel with the pre-set amount
|
||||
* Confirm the tx in MetaMask popup; wait some time for the tx to be confirmed and ledger channel to be created
|
||||
* The progress can be followed from watcher-1's logs
|
||||
* The created ledger channel should now be visible in the `NITRO` tab; click on `REFRESH` button otherwise
|
||||
|
||||
* Change amount to `10000` and click on `VIRTUAL FUND` button to create a virtual payment channel
|
||||
* This results in a payment channel between the app and watcher-1
|
||||
* The payment channel's details should now be visible along with a `PAY` and `VIRTUAL DEFUND` buttons
|
||||
|
||||
* Close the debug panel
|
||||
|
||||
### Paid queries and mutations
|
||||
|
||||
* Perform phisher status checks now that a payment channel has been created
|
||||
* Amount set in the debug panel's `NITRO` tab is sent along with each request to the watcher
|
||||
* Check the watcher-1 logs for the received payments:
|
||||
|
||||
```bash
|
||||
# ...
|
||||
# 2024-02-21T11:35:25.538Z ts-nitro:engine {"msg":"Received message","_msg":{"to":"0xAAA662","from":"0x5D12ac","payloadSummaries":[],"proposalSummaries":[],"payments":[{"amount":50,"channelId":"0x654a85725442828f89b497e3973640613c03b5f5ec47302bfa4402d42c07de30"}],"rejectedObjectives":[]}}
|
||||
# 2024-02-21T11:35:25.552Z laconic:payments Query rate not configured for "latestBlock", serving free query
|
||||
# 2024-02-21T11:35:25.553Z vulcanize:resolver latestBlock
|
||||
# 2024-02-21T11:35:25.557Z laconic:payments Received a payment voucher of 50 from 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# 2024-02-21T11:35:25.570Z laconic:payments Serving a paid query for 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# 2024-02-21T11:35:25.570Z vulcanize:resolver isPhisher 0x6a1f0dce967aefd4adf7762c523cde358960236f05734f616ebe69c0abfcb0cc 0xAFA36c47E130d89bcE4470a9030d99f3CEcaD146 TWT:dummyPhisher
|
||||
# 2024-02-21T11:35:25.580Z vulcanize:indexer isPhisher: db miss, fetching from upstream server
|
||||
# ...
|
||||
```
|
||||
|
||||
* Rate for mutations is set to `100` in the watcher; go back to the `NITRO` tab in the debug panel and change amount value besides `PAY` button to >=100
|
||||
|
||||
* Perform a phisher report
|
||||
* Enter a new record(s) and click on `Submit batch to p2p network` button
|
||||
* Among all three watchers running in consensus, whoever is the `LEADER` at time of reporting sends a tx to the chain
|
||||
* Check all the watchers' logs:
|
||||
|
||||
If the payment receiving watcher (1) is leader at that moment:
|
||||
|
||||
```bash
|
||||
# On watcher-1 (payment received + tx sent)
|
||||
# ...
|
||||
# 2024-02-21T11:42:55.088Z vulcanize:libp2p-utils [11:42:55] Received a message on mobymask P2P network from peer: 12D3KooWGXxcwevUY7KCfw8fcGhxqxPaiFMGSU4tgJDjE54QGKzf
|
||||
# 2024-02-21T11:42:55.102Z ts-nitro:engine {"msg":"Received message","_msg":{"to":"0xAAA662","from":"0x5D12ac","payloadSummaries":[],"proposalSummaries":[],"payments":[{"amount":150,"channelId":"0x654a85725442828f89b497e3973640613c03b5f5ec47302bfa4402d42c07de30"}],"rejectedObjectives":[]}}
|
||||
# 2024-02-21T11:42:55.115Z laconic:payments Received a payment voucher of 100 from 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# 2024-02-21T11:42:55.115Z vulcanize:libp2p-utils Payment received for a mutation request from 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# 2024-02-21T11:42:59.115Z vulcanize:libp2p-utils Transaction receipt for invoke message {
|
||||
# to: '0xAFA36c47E130d89bcE4470a9030d99f3CEcaD146',
|
||||
# blockNumber: 4638,
|
||||
# blockHash: '0x23a42bc2ae43771c62b0d59cc48b5858e6e2e488953527ba6a9f5119ae72b42a',
|
||||
# transactionHash: '0xb61cfdfd0ffe937a191a230e1355b9bc5cdd32507f7b4f24a8e6356adf089b64',
|
||||
# effectiveGasPrice: '1500000050',
|
||||
# gasUsed: '136450'
|
||||
# }
|
||||
# ...
|
||||
|
||||
# On other watchers (payment not received + tx not sent)
|
||||
...
|
||||
# 2024-02-21T11:42:50.329Z laconic:consensus State changed to 3 (FOLLOWER) with term 478
|
||||
# 2024-02-21T11:42:55.089Z vulcanize:libp2p-utils [11:42:55] Received a message on mobymask P2P network from peer: 12D3KooWGXxcwevUY7KCfw8fcGhxqxPaiFMGSU4tgJDjE54QGKzf
|
||||
# 2024-02-21T11:42:55.096Z vulcanize:libp2p-utils Not a leader, skipped sending L2 tx
|
||||
# 2024-02-21T11:43:05.095Z vulcanize:libp2p-utils Payment not received
|
||||
# ...
|
||||
```
|
||||
|
||||
If the payment receiving watcher (1) is NOT leader at that moment:
|
||||
|
||||
```bash
|
||||
# On watcher 1 (payment received + tx not sent)
|
||||
# ...
|
||||
# 2024-02-21T11:46:52.049Z vulcanize:libp2p-utils [11:46:52] Received a message on mobymask P2P network from peer: 12D3KooWGXxcwevUY7KCfw8fcGhxqxPaiFMGSU4tgJDjE54QGKzf
|
||||
# 2024-02-21T11:46:52.051Z vulcanize:libp2p-utils Not a leader, skipped sending L2 tx
|
||||
# 2024-02-21T11:46:52.074Z ts-nitro:engine {"msg":"Received message","_msg":{"to":"0xAAA662","from":"0x5D12ac","payloadSummaries":[],"proposalSummaries":[],"payments":[{"amount":450,"channelId":"0x654a85725442828f89b497e3973640613c03b5f5ec47302bfa4402d42c07de30"}],"rejectedObjectives":[]}}
|
||||
# 2024-02-21T11:46:52.097Z laconic:payments Received a payment voucher of 100 from 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# 2024-02-21T11:46:52.097Z vulcanize:libp2p-utils Payment received for a mutation request from 0x5D12acfbBB1caD65fD61983003a50E0CB6900Fd3
|
||||
# ...
|
||||
|
||||
# On the leader watcher (payment not received + tx sent)
|
||||
# ...
|
||||
# 2024-02-21T11:46:40.111Z vulcanize:libp2p-utils Payment not received
|
||||
# 2024-02-21T11:46:52.048Z vulcanize:libp2p-utils [11:46:52] Received a message on mobymask P2P network from peer: 12D3KooWGXxcwevUY7KCfw8fcGhxqxPaiFMGSU4tgJDjE54QGKzf
|
||||
# 2024-02-21T11:46:56.048Z vulcanize:libp2p-utils Transaction receipt for invoke message {
|
||||
# to: '0xAFA36c47E130d89bcE4470a9030d99f3CEcaD146',
|
||||
# blockNumber: 4757,
|
||||
# blockHash: '0x859aa54cb02e8a3f910a01b85c2a7bf9bff7540e83018e7e846f87ca1770e55e',
|
||||
# transactionHash: '0x92f087e4b6ac5604c9d4ecd823d526ca5f8f1bf5c2a92861d6b0f69bd899ba83',
|
||||
# effectiveGasPrice: '1500000050',
|
||||
# gasUsed: '136438'
|
||||
# }
|
||||
```
|
||||
|
||||
* Check phisher status for the reported phishers to confirm state update
|
||||
|
||||
## Clean up
|
||||
|
||||
* From the `NITRO` tab in debug panel, perform `VIRTUAL DEFUND` and `DIRECT DEFUND` (in order) for any payment channels created
|
||||
|
||||
* In the browser's console, delete all indexedDBs:
|
||||
|
||||
```bash
|
||||
await clearNodeStorage()
|
||||
```
|
||||
|
||||
* Remove the snap from MetaMask flask extension
|
||||
@@ -1,68 +0,0 @@
|
||||
# Copyright © 2023 Vulcanize
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU Affero General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
from stack_orchestrator.deploy.deployment_context import DeploymentContext
|
||||
from ruamel.yaml import YAML
|
||||
|
||||
|
||||
def create(context: DeploymentContext, extra_args):
|
||||
fixturenet_eth_compose_file = context.deployment_dir.joinpath('compose', 'docker-compose-fixturenet-eth.yml')
|
||||
|
||||
with open(fixturenet_eth_compose_file, 'r') as yaml_file:
|
||||
yaml = YAML()
|
||||
yaml_data = yaml.load(yaml_file)
|
||||
|
||||
new_script = '../config/fixturenet-optimism/run-geth.sh:/opt/testnet/run.sh'
|
||||
if new_script not in yaml_data['services']['fixturenet-eth-geth-1']['volumes']:
|
||||
yaml_data['services']['fixturenet-eth-geth-1']['volumes'].append(new_script)
|
||||
|
||||
with open(fixturenet_eth_compose_file, 'w') as yaml_file:
|
||||
yaml = YAML()
|
||||
yaml.dump(yaml_data, yaml_file)
|
||||
|
||||
# load fixturenet-optimism compose file
|
||||
fixturenet_optimism_compose_file = context.deployment_dir.joinpath(
|
||||
"compose",
|
||||
"docker-compose-fixturenet-optimism.yml"
|
||||
)
|
||||
|
||||
with open(fixturenet_optimism_compose_file, 'r') as yaml_file:
|
||||
yaml = YAML()
|
||||
yaml_data = yaml.load(yaml_file)
|
||||
|
||||
# mount the funding script to volumes
|
||||
fund_accounts_script = (
|
||||
'../config/watcher-mobymask-v3-demo/local/fund-accounts-on-l2.sh'
|
||||
':/app/packages/contracts-bedrock/fund-accounts-on-l2.sh'
|
||||
)
|
||||
yaml_data['services']['fixturenet-optimism-contracts']['volumes'].append(fund_accounts_script)
|
||||
|
||||
# update command to run the script
|
||||
yaml_data['services']['fixturenet-optimism-contracts']['command'] = (
|
||||
'"./wait-for-it.sh -h ${CERC_L1_HOST:-$${DEFAULT_CERC_L1_HOST}}'
|
||||
' -p ${CERC_L1_PORT:-$${DEFAULT_CERC_L1_PORT}} -s -t 60'
|
||||
' -- ./deploy-contracts.sh && ./fund-accounts-on-l2.sh"'
|
||||
)
|
||||
|
||||
# update port mapping for op-geth: 0.0.0.0:8545:8545
|
||||
existing_ports = yaml_data['services']['op-geth']['ports']
|
||||
new_ports = ["0.0.0.0:8545:8545" if "8545" in s else s for s in existing_ports]
|
||||
yaml_data['services']['op-geth']['ports'] = new_ports
|
||||
|
||||
with open(fixturenet_optimism_compose_file, 'w') as yaml_file:
|
||||
yaml = YAML()
|
||||
yaml.dump(yaml_data, yaml_file)
|
||||
|
||||
return None
|
||||
@@ -1,47 +0,0 @@
|
||||
version: "1.0"
|
||||
description: "MobyMask v3 demo stack"
|
||||
name: mobymask-v3-demo
|
||||
repos:
|
||||
# fixturenet repos
|
||||
- git.vdb.to/cerc-io/go-ethereum@v1.11.6-statediff-v5
|
||||
- git.vdb.to/cerc-io/lighthouse
|
||||
- github.com/dboreham/foundry
|
||||
- github.com/ethereum-optimism/optimism@op-node/v1.3.0
|
||||
- github.com/ethereum-optimism/op-geth@v1.101304.0
|
||||
# nitro repo
|
||||
- github.com/cerc-io/ts-nitro@v0.1.15
|
||||
# mobymask watcher repos
|
||||
- github.com/cerc-io/mobymask-v2-watcher-ts@v0.2.4
|
||||
- github.com/cerc-io/MobyMask@v0.1.3
|
||||
# mobymask app repos
|
||||
- github.com/cerc-io/mobymask-snap
|
||||
- github.com/cerc-io/mobymask-ui@v0.2.2
|
||||
containers:
|
||||
# fixturenet images
|
||||
- cerc/go-ethereum
|
||||
- cerc/lighthouse
|
||||
- cerc/lighthouse-cli
|
||||
- cerc/fixturenet-eth-genesis
|
||||
- cerc/fixturenet-eth-geth
|
||||
- cerc/fixturenet-eth-lighthouse
|
||||
- cerc/foundry
|
||||
- cerc/optimism-contracts
|
||||
- cerc/optimism-op-node
|
||||
- cerc/optimism-l2geth
|
||||
- cerc/optimism-op-batcher
|
||||
- cerc/optimism-op-proposer
|
||||
# nitro contract deployment image
|
||||
- cerc/nitro-contracts
|
||||
# mobymask watcher images
|
||||
- cerc/watcher-mobymask-v3
|
||||
- cerc/mobymask
|
||||
# mobymask app images
|
||||
- cerc/mobymask-snap
|
||||
- cerc/mobymask-ui
|
||||
pods:
|
||||
- fixturenet-eth
|
||||
- fixturenet-optimism
|
||||
- nitro-contracts
|
||||
- watcher-mobymask-v3-demo
|
||||
- mobymask-snap
|
||||
- mobymask-app-v3-demo
|
||||
@@ -75,7 +75,7 @@ Create and update an env file to be used in the next step ([defaults](../../conf
|
||||
# Uses a generated peer id if not set (disables consensus)
|
||||
CERC_PEER_ID=
|
||||
|
||||
# Enable payments to upstream ETH server
|
||||
# Disable payments to upstream ETH server
|
||||
CERC_ENABLE_UPSTREAM_PAYMENTS=false
|
||||
```
|
||||
|
||||
|
||||
@@ -112,7 +112,7 @@ Place the pre-configured watcher alerts rules in Grafana provisioning directory:
|
||||
|
||||
Update the alerting contact points config (`monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/contactpoints.yml`) with desired contact points
|
||||
|
||||
Add corresponding routes to the notification policies config (`monitoring-watchers-deployment/config/monitoring/grafana/provisioning/alerting/policies.yml`) with appropriate object-matchers:
|
||||
Add corresponding routes to the notification policies config (`monitoring-watchers-deployment/monitoring/grafana/provisioning/alerting/policies.yaml`) with appropriate object-matchers:
|
||||
|
||||
```yml
|
||||
...
|
||||
|
||||
@@ -33,23 +33,29 @@ laconic-so --stack uniswap-urbit-app deploy init --output uniswap-urbit-app-spec
|
||||
|
||||
### Ports
|
||||
|
||||
Edit `network` in spec file to map container ports to same ports in host:
|
||||
Edit `uniswap-urbit-app-spec.yml` such that it looks like:
|
||||
|
||||
```yml
|
||||
...
|
||||
stack: uniswap-urbit-app
|
||||
deploy-to: compose
|
||||
network:
|
||||
ports:
|
||||
urbit-fake-ship:
|
||||
- '8080:80'
|
||||
proxy-server:
|
||||
- '4000:4000'
|
||||
urbit-fake-ship:
|
||||
- '8080:80'
|
||||
ipfs:
|
||||
- '8081:8080'
|
||||
- '5001:5001'
|
||||
...
|
||||
- '4001'
|
||||
- '8081:8080'
|
||||
- 0.0.0.0:5001:5001
|
||||
volumes:
|
||||
urbit_app_builds: ./data/urbit_app_builds
|
||||
urbit_data: ./data/urbit_data
|
||||
ipfs-import: ./data/ipfs-import
|
||||
ipfs-data: ./data/ipfs-data
|
||||
```
|
||||
|
||||
Note: Skip the `ipfs` ports if need to use an externally running IPFS node
|
||||
Note: Skip the `ipfs` ports if using an externally running IPFS node, set via `config.env`, below.
|
||||
|
||||
### Data volumes
|
||||
|
||||
|
||||
@@ -327,12 +327,14 @@ def init_operation(deploy_command_context, stack, deployer_type, config,
|
||||
default_spec_file_content = call_stack_deploy_init(deploy_command_context)
|
||||
spec_file_content = {"stack": stack, constants.deploy_to_key: deployer_type}
|
||||
if deployer_type == "k8s":
|
||||
if kube_config is None:
|
||||
if kube_config:
|
||||
spec_file_content.update({constants.kube_config_key: kube_config})
|
||||
else:
|
||||
error_exit("--kube-config must be supplied with --deploy-to k8s")
|
||||
if image_registry is None:
|
||||
error_exit("--image-registry must be supplied with --deploy-to k8s")
|
||||
spec_file_content.update({constants.kube_config_key: kube_config})
|
||||
spec_file_content.update({constants.image_registry_key: image_registry})
|
||||
if image_registry:
|
||||
spec_file_content.update({constants.image_registry_key: image_registry})
|
||||
else:
|
||||
print("WARNING: --image-registry not specified, only default container registries (eg, Docker Hub) will be available")
|
||||
else:
|
||||
# Check for --kube-config supplied for non-relevant deployer types
|
||||
if kube_config is not None:
|
||||
|
||||
@@ -29,6 +29,29 @@ def _image_needs_pushed(image: str):
|
||||
return image.endswith(":local")
|
||||
|
||||
|
||||
def remote_image_exists(remote_repo_url: str, local_tag: str):
|
||||
docker = DockerClient()
|
||||
try:
|
||||
remote_tag = remote_tag_for_image(local_tag, remote_repo_url)
|
||||
result = docker.manifest.inspect(remote_tag)
|
||||
return True if result else False
|
||||
except Exception: # noqa: E722
|
||||
return False
|
||||
|
||||
|
||||
def add_tags_to_image(remote_repo_url: str, local_tag: str, *additional_tags):
|
||||
if not additional_tags:
|
||||
return
|
||||
|
||||
if not remote_image_exists(remote_repo_url, local_tag):
|
||||
raise Exception(f"{local_tag} does not exist in {remote_repo_url}")
|
||||
|
||||
docker = DockerClient()
|
||||
remote_tag = remote_tag_for_image(local_tag, remote_repo_url)
|
||||
new_remote_tags = [remote_tag_for_image(tag, remote_repo_url) for tag in additional_tags]
|
||||
docker.buildx.imagetools.create(sources=[remote_tag], tags=new_remote_tags)
|
||||
|
||||
|
||||
def remote_tag_for_image(image: str, remote_repo_url: str):
|
||||
# Turns image tags of the form: foo/bar:local into remote.repo/org/bar:deploy
|
||||
major_parts = image.split("/", 2)
|
||||
|
||||
@@ -101,7 +101,7 @@ class ClusterInfo:
|
||||
)
|
||||
return service
|
||||
|
||||
def get_ingress(self, use_tls=False):
|
||||
def get_ingress(self, use_tls=False, certificate=None, cluster_issuer="letsencrypt-prod"):
|
||||
# No ingress for a deployment that has no http-proxy defined, for now
|
||||
http_proxy_info_list = self.spec.get_http_proxy()
|
||||
ingress = None
|
||||
@@ -114,8 +114,8 @@ class ClusterInfo:
|
||||
host_name = http_proxy_info["host-name"]
|
||||
rules = []
|
||||
tls = [client.V1IngressTLS(
|
||||
hosts=[host_name],
|
||||
secret_name=f"{self.app_name}-tls"
|
||||
hosts=certificate["spec"]["dnsNames"] if certificate else [host_name],
|
||||
secret_name=certificate["spec"]["secretName"] if certificate else f"{self.app_name}-tls"
|
||||
)] if use_tls else None
|
||||
paths = []
|
||||
for route in http_proxy_info["routes"]:
|
||||
@@ -147,13 +147,17 @@ class ClusterInfo:
|
||||
tls=tls,
|
||||
rules=rules
|
||||
)
|
||||
|
||||
ingress_annotations = {
|
||||
"kubernetes.io/ingress.class": "nginx",
|
||||
}
|
||||
if not certificate:
|
||||
ingress_annotations["cert-manager.io/cluster-issuer"] = cluster_issuer
|
||||
|
||||
ingress = client.V1Ingress(
|
||||
metadata=client.V1ObjectMeta(
|
||||
name=f"{self.app_name}-ingress",
|
||||
annotations={
|
||||
"kubernetes.io/ingress.class": "nginx",
|
||||
"cert-manager.io/cluster-issuer": "letsencrypt-prod"
|
||||
}
|
||||
annotations=ingress_annotations
|
||||
),
|
||||
spec=spec
|
||||
)
|
||||
|
||||
@@ -169,6 +169,39 @@ class K8sDeployer(Deployer):
|
||||
print("Service created:")
|
||||
print(f"{service_resp}")
|
||||
|
||||
def _find_certificate_for_host_name(self, host_name):
|
||||
all_certificates = self.custom_obj_api.list_namespaced_custom_object(
|
||||
group="cert-manager.io",
|
||||
version="v1",
|
||||
namespace=self.k8s_namespace,
|
||||
plural="certificates"
|
||||
)
|
||||
|
||||
host_parts = host_name.split(".", 1)
|
||||
host_as_wild = None
|
||||
if len(host_parts) == 2:
|
||||
host_as_wild = f"*.{host_parts[1]}"
|
||||
|
||||
now = datetime.utcnow().replace(tzinfo=timezone.utc)
|
||||
fmt = "%Y-%m-%dT%H:%M:%S%z"
|
||||
|
||||
# Walk over all the configured certificates.
|
||||
for cert in all_certificates["items"]:
|
||||
dns = cert["spec"]["dnsNames"]
|
||||
# Check for an exact hostname match or a wildcard match.
|
||||
if host_name in dns or host_as_wild in dns:
|
||||
status = cert.get("status", {})
|
||||
# Check the certificate date.
|
||||
if "notAfter" in status and "notBefore" in status:
|
||||
before = datetime.strptime(status["notBefore"], fmt)
|
||||
after = datetime.strptime(status["notAfter"], fmt)
|
||||
if before < now < after:
|
||||
# Check the status is Ready
|
||||
for condition in status.get("conditions", []):
|
||||
if "True" == condition.get("status") and "Ready" == condition.get("type"):
|
||||
return cert
|
||||
return None
|
||||
|
||||
def up(self, detach, services):
|
||||
if not opts.o.dry_run:
|
||||
if self.is_kind():
|
||||
@@ -189,8 +222,15 @@ class K8sDeployer(Deployer):
|
||||
self._create_volume_data()
|
||||
self._create_deployment()
|
||||
|
||||
http_proxy_info = self.cluster_info.spec.get_http_proxy()
|
||||
# Note: at present we don't support tls for kind (and enabling tls causes errors)
|
||||
ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=not self.is_kind())
|
||||
use_tls = http_proxy_info and not self.is_kind()
|
||||
certificate = self._find_certificate_for_host_name(http_proxy_info[0]["host-name"]) if use_tls else None
|
||||
if opts.o.debug:
|
||||
if certificate:
|
||||
print(f"Using existing certificate: {certificate}")
|
||||
|
||||
ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=use_tls, certificate=certificate)
|
||||
if ingress:
|
||||
if opts.o.debug:
|
||||
print(f"Sending this ingress: {ingress}")
|
||||
@@ -350,9 +390,11 @@ class K8sDeployer(Deployer):
|
||||
name=ingress.spec.tls[0].secret_name
|
||||
)
|
||||
|
||||
hostname = ingress.spec.tls[0].hosts[0]
|
||||
hostname = ingress.spec.rules[0].host
|
||||
ip = ingress.status.load_balancer.ingress[0].ip
|
||||
tls = "notBefore: %s, notAfter: %s" % (cert["status"]["notBefore"], cert["status"]["notAfter"])
|
||||
tls = "notBefore: %s; notAfter: %s; names: %s" % (
|
||||
cert["status"]["notBefore"], cert["status"]["notAfter"], ingress.spec.tls[0].hosts
|
||||
)
|
||||
except: # noqa: E722
|
||||
pass
|
||||
|
||||
|
||||
@@ -27,7 +27,9 @@ class ResourceLimits:
|
||||
memory: int = None
|
||||
storage: int = None
|
||||
|
||||
def __init__(self, obj={}):
|
||||
def __init__(self, obj=None):
|
||||
if obj is None:
|
||||
obj = {}
|
||||
if "cpus" in obj:
|
||||
self.cpus = float(obj["cpus"])
|
||||
if "memory" in obj:
|
||||
@@ -50,7 +52,9 @@ class Resources:
|
||||
limits: ResourceLimits = None
|
||||
reservations: ResourceLimits = None
|
||||
|
||||
def __init__(self, obj={}):
|
||||
def __init__(self, obj=None):
|
||||
if obj is None:
|
||||
obj = {}
|
||||
if "reservations" in obj:
|
||||
self.reservations = ResourceLimits(obj["reservations"])
|
||||
if "limits" in obj:
|
||||
@@ -72,7 +76,9 @@ class Spec:
|
||||
obj: typing.Any
|
||||
file_path: Path
|
||||
|
||||
def __init__(self, file_path: Path = None, obj={}) -> None:
|
||||
def __init__(self, file_path: Path = None, obj=None) -> None:
|
||||
if obj is None:
|
||||
obj = {}
|
||||
self.file_path = file_path
|
||||
self.obj = obj
|
||||
|
||||
@@ -91,49 +97,41 @@ class Spec:
|
||||
self.file_path = file_path
|
||||
|
||||
def get_image_registry(self):
|
||||
return (self.obj[constants.image_registry_key]
|
||||
if self.obj and constants.image_registry_key in self.obj
|
||||
else None)
|
||||
return self.obj.get(constants.image_registry_key)
|
||||
|
||||
def get_volumes(self):
|
||||
return (self.obj["volumes"]
|
||||
if self.obj and "volumes" in self.obj
|
||||
else {})
|
||||
return self.obj.get(constants.volumes_key, {})
|
||||
|
||||
def get_configmaps(self):
|
||||
return (self.obj["configmaps"]
|
||||
if self.obj and "configmaps" in self.obj
|
||||
else {})
|
||||
return self.obj.get(constants.configmaps_key, {})
|
||||
|
||||
def get_container_resources(self):
|
||||
return Resources(self.obj.get("resources", {}).get("containers", {}))
|
||||
return Resources(self.obj.get(constants.resources_key, {}).get("containers", {}))
|
||||
|
||||
def get_volume_resources(self):
|
||||
return Resources(self.obj.get("resources", {}).get("volumes", {}))
|
||||
return Resources(self.obj.get(constants.resources_key, {}).get(constants.volumes_key, {}))
|
||||
|
||||
def get_http_proxy(self):
|
||||
return (self.obj[constants.network_key][constants.http_proxy_key]
|
||||
if self.obj and constants.network_key in self.obj
|
||||
and constants.http_proxy_key in self.obj[constants.network_key]
|
||||
else None)
|
||||
return self.obj.get(constants.network_key, {}).get(constants.http_proxy_key, [])
|
||||
|
||||
def get_annotations(self):
|
||||
return self.obj.get("annotations", {})
|
||||
return self.obj.get(constants.annotations_key, {})
|
||||
|
||||
def get_labels(self):
|
||||
return self.obj.get("labels", {})
|
||||
return self.obj.get(constants.labels_key, {})
|
||||
|
||||
def get_privileged(self):
|
||||
return "true" == str(self.obj.get("security", {}).get("privileged", "false")).lower()
|
||||
return "true" == str(self.obj.get(constants.security_key, {}).get("privileged", "false")).lower()
|
||||
|
||||
def get_capabilities(self):
|
||||
return self.obj.get("security", {}).get("capabilities", [])
|
||||
return self.obj.get(constants.security_key, {}).get("capabilities", [])
|
||||
|
||||
def get_deployment_type(self):
|
||||
return self.obj[constants.deploy_to_key]
|
||||
return self.obj.get(constants.deploy_to_key)
|
||||
|
||||
def is_kubernetes_deployment(self):
|
||||
return self.get_deployment_type() in [constants.k8s_kind_deploy_type, constants.k8s_deploy_type]
|
||||
return self.get_deployment_type() in [constants.k8s_kind_deploy_type,
|
||||
constants.k8s_deploy_type]
|
||||
|
||||
def is_kind_deployment(self):
|
||||
return self.get_deployment_type() in [constants.k8s_kind_deploy_type]
|
||||
|
||||
@@ -24,8 +24,9 @@ import uuid
|
||||
|
||||
import click
|
||||
|
||||
from stack_orchestrator.deploy.images import remote_image_exists, add_tags_to_image
|
||||
from stack_orchestrator.deploy.webapp import deploy_webapp
|
||||
from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient,
|
||||
from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient, TimedLogger,
|
||||
build_container_image, push_container_image,
|
||||
file_hash, deploy_to_k8s, publish_deployment,
|
||||
hostname_for_deployment_request, generate_hostname_for_app,
|
||||
@@ -38,36 +39,33 @@ def process_app_deployment_request(
|
||||
app_deployment_request,
|
||||
deployment_record_namespace,
|
||||
dns_record_namespace,
|
||||
dns_suffix,
|
||||
default_dns_suffix,
|
||||
deployment_parent_dir,
|
||||
kube_config,
|
||||
image_registry,
|
||||
log_parent_dir
|
||||
force_rebuild,
|
||||
fqdn_policy,
|
||||
logger
|
||||
):
|
||||
run_id = f"{app_deployment_request.id}-{str(time.time()).split('.')[0]}-{str(uuid.uuid4()).split('-')[0]}"
|
||||
log_file = None
|
||||
if log_parent_dir:
|
||||
log_dir = os.path.join(log_parent_dir, app_deployment_request.id)
|
||||
if not os.path.exists(log_dir):
|
||||
os.mkdir(log_dir)
|
||||
log_file_path = os.path.join(log_dir, f"{run_id}.log")
|
||||
print(f"Directing build logs to: {log_file_path}")
|
||||
log_file = open(log_file_path, "wt")
|
||||
logger.log("BEGIN - process_app_deployment_request")
|
||||
|
||||
# 1. look up application
|
||||
app = laconic.get_record(app_deployment_request.attributes.application, require=True)
|
||||
logger.log(f"Retrieved app record {app_deployment_request.attributes.application}")
|
||||
|
||||
# 2. determine dns
|
||||
requested_name = hostname_for_deployment_request(app_deployment_request, laconic)
|
||||
logger.log(f"Determined requested name: {requested_name}")
|
||||
|
||||
# HACK
|
||||
if "." in requested_name:
|
||||
raise Exception("Only unqualified hostnames allowed at this time.")
|
||||
|
||||
fqdn = f"{requested_name}.{dns_suffix}"
|
||||
if "allow" == fqdn_policy or "preexisting" == fqdn_policy:
|
||||
fqdn = requested_name
|
||||
else:
|
||||
raise Exception(f"{requested_name} is invalid: only unqualified hostnames are allowed.")
|
||||
else:
|
||||
fqdn = f"{requested_name}.{default_dns_suffix}"
|
||||
|
||||
# 3. check ownership of existing dnsrecord vs this request
|
||||
# TODO: Support foreign DNS
|
||||
dns_crn = f"{dns_record_namespace}/{fqdn}"
|
||||
dns_record = laconic.get_record(dns_crn)
|
||||
if dns_record:
|
||||
@@ -76,10 +74,12 @@ def process_app_deployment_request(
|
||||
matched_owner = match_owner(app_deployment_request, laconic.get_record(dns_record.attributes.request, require=True))
|
||||
|
||||
if matched_owner:
|
||||
print("Matched DnsRecord ownership:", matched_owner)
|
||||
logger.log(f"Matched DnsRecord ownership: {matched_owner}")
|
||||
else:
|
||||
raise Exception("Unable to confirm ownership of DnsRecord %s for request %s" %
|
||||
(dns_record.id, app_deployment_request.id))
|
||||
(dns_crn, app_deployment_request.id))
|
||||
elif "preexisting" == fqdn_policy:
|
||||
raise Exception(f"No pre-existing DnsRecord {dns_crn} could be found for request {app_deployment_request.id}.")
|
||||
|
||||
# 4. get build and runtime config from request
|
||||
env_filename = None
|
||||
@@ -100,7 +100,9 @@ def process_app_deployment_request(
|
||||
deployment_record = laconic.get_record(app_deployment_crn)
|
||||
deployment_dir = os.path.join(deployment_parent_dir, fqdn)
|
||||
deployment_config_file = os.path.join(deployment_dir, "config.env")
|
||||
# TODO: Is there any reason not to simplify the hash input to the app_deployment_crn?
|
||||
deployment_container_tag = "laconic-webapp/%s:local" % hashlib.md5(deployment_dir.encode()).hexdigest()
|
||||
app_image_shared_tag = f"laconic-webapp/{app.id}:local"
|
||||
# b. check for deployment directory (create if necessary)
|
||||
if not os.path.exists(deployment_dir):
|
||||
if deployment_record:
|
||||
@@ -115,11 +117,26 @@ def process_app_deployment_request(
|
||||
needs_k8s_deploy = False
|
||||
# 6. build container (if needed)
|
||||
if not deployment_record or deployment_record.attributes.application != app.id:
|
||||
# TODO: pull from request
|
||||
extra_build_args = []
|
||||
build_container_image(app, deployment_container_tag, extra_build_args, log_file)
|
||||
push_container_image(deployment_dir, log_file)
|
||||
needs_k8s_deploy = True
|
||||
# check if the image already exists
|
||||
shared_tag_exists = remote_image_exists(image_registry, app_image_shared_tag)
|
||||
if shared_tag_exists and not force_rebuild:
|
||||
# simply add our unique tag to the existing image and we are done
|
||||
logger.log(f"Using existing app image {app_image_shared_tag} for {deployment_container_tag}")
|
||||
add_tags_to_image(image_registry, app_image_shared_tag, deployment_container_tag)
|
||||
logger.log("Tag complete")
|
||||
else:
|
||||
extra_build_args = [] # TODO: pull from request
|
||||
logger.log(f"Building container image {deployment_container_tag}")
|
||||
build_container_image(app, deployment_container_tag, extra_build_args, logger)
|
||||
logger.log("Build complete")
|
||||
logger.log(f"Pushing container image {deployment_container_tag}")
|
||||
push_container_image(deployment_dir, logger)
|
||||
logger.log("Push complete")
|
||||
# The build/push commands above will use the unique deployment tag, so now we need to add the shared tag.
|
||||
logger.log(f"Updating app image tag {app_image_shared_tag} from build of {deployment_container_tag}")
|
||||
add_tags_to_image(image_registry, deployment_container_tag, app_image_shared_tag)
|
||||
logger.log("Tag complete")
|
||||
|
||||
# 7. update config (if needed)
|
||||
if not deployment_record or file_hash(deployment_config_file) != deployment_record.attributes.meta.config:
|
||||
@@ -127,13 +144,13 @@ def process_app_deployment_request(
|
||||
|
||||
# 8. update k8s deployment
|
||||
if needs_k8s_deploy:
|
||||
print("Deploying to k8s")
|
||||
deploy_to_k8s(
|
||||
deployment_record,
|
||||
deployment_dir,
|
||||
log_file
|
||||
logger
|
||||
)
|
||||
|
||||
logger.log("Publishing deployment to registry.")
|
||||
publish_deployment(
|
||||
laconic,
|
||||
app,
|
||||
@@ -142,8 +159,11 @@ def process_app_deployment_request(
|
||||
dns_record,
|
||||
dns_crn,
|
||||
deployment_dir,
|
||||
app_deployment_request
|
||||
app_deployment_request,
|
||||
logger
|
||||
)
|
||||
logger.log("Publication complete.")
|
||||
logger.log("END - process_app_deployment_request")
|
||||
|
||||
|
||||
def load_known_requests(filename):
|
||||
@@ -175,17 +195,19 @@ def dump_known_requests(filename, requests, status="SEEN"):
|
||||
@click.option("--state-file", help="File to store state about previously seen requests.")
|
||||
@click.option("--only-update-state", help="Only update the state file, don't process any requests anything.", is_flag=True)
|
||||
@click.option("--dns-suffix", help="DNS domain to use eg, laconic.servesthe.world")
|
||||
@click.option("--fqdn-policy", help="How to handle requests with an FQDN: prohibit, allow, preexisting", default="prohibit")
|
||||
@click.option("--record-namespace-dns", help="eg, crn://laconic/dns")
|
||||
@click.option("--record-namespace-deployments", help="eg, crn://laconic/deployments")
|
||||
@click.option("--dry-run", help="Don't do anything, just report what would be done.", is_flag=True)
|
||||
@click.option("--include-tags", help="Only include requests with matching tags (comma-separated).", default="")
|
||||
@click.option("--exclude-tags", help="Exclude requests with matching tags (comma-separated).", default="")
|
||||
@click.option("--force-rebuild", help="Rebuild even if the image already exists.", is_flag=True)
|
||||
@click.option("--log-dir", help="Output build/deployment logs to directory.", default=None)
|
||||
@click.pass_context
|
||||
def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_dir, # noqa: C901
|
||||
request_id, discover, state_file, only_update_state,
|
||||
dns_suffix, record_namespace_dns, record_namespace_deployments, dry_run,
|
||||
include_tags, exclude_tags, log_dir):
|
||||
dns_suffix, fqdn_policy, record_namespace_dns, record_namespace_deployments, dry_run,
|
||||
include_tags, exclude_tags, force_rebuild, log_dir):
|
||||
if request_id and discover:
|
||||
print("Cannot specify both --request-id and --discover", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
@@ -203,6 +225,10 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
print("--dns-suffix, --record-namespace-dns, and --record-namespace-deployments are all required", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
if fqdn_policy not in ["prohibit", "allow", "preexisting"]:
|
||||
print("--fqdn-policy must be one of 'prohibit', 'allow', or 'preexisting'", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# Split CSV and clean up values.
|
||||
include_tags = [tag.strip() for tag in include_tags.split(",") if tag]
|
||||
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
|
||||
@@ -291,10 +317,24 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
for r in requests_to_execute:
|
||||
dump_known_requests(state_file, [r], "DEPLOYING")
|
||||
status = "ERROR"
|
||||
run_log_file = None
|
||||
run_reg_client = laconic
|
||||
try:
|
||||
run_id = f"{r.id}-{str(time.time()).split('.')[0]}-{str(uuid.uuid4()).split('-')[0]}"
|
||||
if log_dir:
|
||||
run_log_dir = os.path.join(log_dir, r.id)
|
||||
if not os.path.exists(run_log_dir):
|
||||
os.mkdir(run_log_dir)
|
||||
run_log_file_path = os.path.join(run_log_dir, f"{run_id}.log")
|
||||
print(f"Directing deployment logs to: {run_log_file_path}")
|
||||
run_log_file = open(run_log_file_path, "wt")
|
||||
run_reg_client = LaconicRegistryClient(laconic_config, log_file=run_log_file)
|
||||
|
||||
logger = TimedLogger(run_id, run_log_file)
|
||||
logger.log("Processing ...")
|
||||
process_app_deployment_request(
|
||||
ctx,
|
||||
laconic,
|
||||
run_reg_client,
|
||||
r,
|
||||
record_namespace_deployments,
|
||||
record_namespace_dns,
|
||||
@@ -302,8 +342,16 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
os.path.abspath(deployment_parent_dir),
|
||||
kube_config,
|
||||
image_registry,
|
||||
log_dir
|
||||
force_rebuild,
|
||||
fqdn_policy,
|
||||
logger
|
||||
)
|
||||
status = "DEPLOYED"
|
||||
except Exception as e:
|
||||
logger.log("ERROR: " + str(e))
|
||||
finally:
|
||||
if logger:
|
||||
logger.log(f"DONE with status {status}", show_step_time=False, show_total_time=True)
|
||||
dump_known_requests(state_file, [r], status)
|
||||
if run_log_file:
|
||||
run_log_file.close()
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
# You should have received a copy of the GNU Affero General Public License
|
||||
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
|
||||
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -39,13 +40,38 @@ class AttrDict(dict):
|
||||
return v
|
||||
|
||||
|
||||
def cmd(*vargs):
|
||||
class TimedLogger:
|
||||
def __init__(self, id="", file=None):
|
||||
self.start = datetime.datetime.now()
|
||||
self.last = self.start
|
||||
self.id = id
|
||||
self.file = file
|
||||
|
||||
def log(self, msg, show_step_time=True, show_total_time=False):
|
||||
prefix = f"{datetime.datetime.utcnow()} - {self.id}"
|
||||
if show_step_time:
|
||||
prefix += f" - {datetime.datetime.now() - self.last} (step)"
|
||||
if show_total_time:
|
||||
prefix += f" - {datetime.datetime.now() - self.start} (total)"
|
||||
print(f"{prefix}: {msg}", file=self.file)
|
||||
if self.file:
|
||||
self.file.flush()
|
||||
self.last = datetime.datetime.now()
|
||||
|
||||
|
||||
def logged_cmd(log_file, *vargs):
|
||||
result = None
|
||||
try:
|
||||
if log_file:
|
||||
print(" ".join(vargs), file=log_file)
|
||||
result = subprocess.run(vargs, capture_output=True)
|
||||
result.check_returncode()
|
||||
return result.stdout.decode()
|
||||
except Exception as err:
|
||||
print(result.stderr.decode())
|
||||
if result:
|
||||
print(result.stderr.decode(), file=log_file)
|
||||
else:
|
||||
print(str(err), file=log_file)
|
||||
raise err
|
||||
|
||||
|
||||
@@ -58,8 +84,9 @@ def match_owner(recordA, *records):
|
||||
|
||||
|
||||
class LaconicRegistryClient:
|
||||
def __init__(self, config_file):
|
||||
def __init__(self, config_file, log_file=None):
|
||||
self.config_file = config_file
|
||||
self.log_file = log_file
|
||||
self.cache = AttrDict(
|
||||
{
|
||||
"name_or_id": {},
|
||||
@@ -77,7 +104,7 @@ class LaconicRegistryClient:
|
||||
args.append("--%s" % k)
|
||||
args.append(str(v))
|
||||
|
||||
results = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
results = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
|
||||
# Most recent records first
|
||||
results.sort(key=lambda r: r.createTime)
|
||||
@@ -115,7 +142,7 @@ class LaconicRegistryClient:
|
||||
|
||||
args = ["laconic", "-c", self.config_file, "cns", "name", "resolve", name]
|
||||
|
||||
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
if parsed:
|
||||
self._add_to_cache(parsed)
|
||||
return parsed[0]
|
||||
@@ -145,7 +172,7 @@ class LaconicRegistryClient:
|
||||
name_or_id,
|
||||
]
|
||||
|
||||
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
|
||||
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
if len(parsed):
|
||||
self._add_to_cache(parsed)
|
||||
return parsed[0]
|
||||
@@ -173,22 +200,22 @@ class LaconicRegistryClient:
|
||||
record_file = open(record_fname, 'w')
|
||||
yaml.dump(record, record_file)
|
||||
record_file.close()
|
||||
print(open(record_fname, 'r').read())
|
||||
print(open(record_fname, 'r').read(), file=self.log_file)
|
||||
|
||||
new_record_id = json.loads(
|
||||
cmd("laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
|
||||
)["id"]
|
||||
for name in names:
|
||||
self.set_name(name, new_record_id)
|
||||
return new_record_id
|
||||
finally:
|
||||
cmd("rm", "-rf", tmpdir)
|
||||
logged_cmd(self.log_file, "rm", "-rf", tmpdir)
|
||||
|
||||
def set_name(self, name, record_id):
|
||||
cmd("laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
|
||||
|
||||
def delete_name(self, name):
|
||||
cmd("laconic", "-c", self.config_file, "cns", "name", "delete", name)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "delete", name)
|
||||
|
||||
|
||||
def file_hash(filename):
|
||||
@@ -212,7 +239,7 @@ def determine_base_container(clone_dir, app_type="webapp"):
|
||||
return base_container
|
||||
|
||||
|
||||
def build_container_image(app_record, tag, extra_build_args=[], log_file=None):
|
||||
def build_container_image(app_record, tag, extra_build_args=[], logger=None):
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
|
||||
try:
|
||||
@@ -221,23 +248,33 @@ def build_container_image(app_record, tag, extra_build_args=[], log_file=None):
|
||||
repo = random.choice(app_record.attributes.repository)
|
||||
clone_dir = os.path.join(tmpdir, record_id)
|
||||
|
||||
print(f"Cloning repository {repo} to {clone_dir} ...")
|
||||
logger.log(f"Cloning repository {repo} to {clone_dir} ...")
|
||||
if ref:
|
||||
# TODO: Determing branch or hash, and use depth 1 if we can.
|
||||
git_env = dict(os.environ.copy())
|
||||
# Never prompt
|
||||
git_env["GIT_TERMINAL_PROMPT"] = "0"
|
||||
subprocess.check_call(["git", "clone", repo, clone_dir], env=git_env, stdout=log_file, stderr=log_file)
|
||||
subprocess.check_call(["git", "checkout", ref], cwd=clone_dir, env=git_env, stdout=log_file, stderr=log_file)
|
||||
try:
|
||||
subprocess.check_call(["git", "clone", repo, clone_dir], env=git_env, stdout=logger.file, stderr=logger.file)
|
||||
except Exception as e:
|
||||
logger.log(f"git clone failed. Is the repository {repo} private?")
|
||||
raise e
|
||||
try:
|
||||
subprocess.check_call(["git", "checkout", ref], cwd=clone_dir, env=git_env, stdout=logger.file, stderr=logger.file)
|
||||
except Exception as e:
|
||||
logger.log(f"git checkout failed. Does ref {ref} exist?")
|
||||
raise e
|
||||
else:
|
||||
result = subprocess.run(["git", "clone", "--depth", "1", repo, clone_dir], stdout=log_file, stderr=log_file)
|
||||
result = subprocess.run(["git", "clone", "--depth", "1", repo, clone_dir], stdout=logger.file, stderr=logger.file)
|
||||
result.check_returncode()
|
||||
|
||||
base_container = determine_base_container(clone_dir, app_record.attributes.app_type)
|
||||
|
||||
print("Building webapp ...")
|
||||
logger.log("Building webapp ...")
|
||||
build_command = [
|
||||
sys.argv[0], "build-webapp",
|
||||
sys.argv[0],
|
||||
"--verbose",
|
||||
"build-webapp",
|
||||
"--source-repo", clone_dir,
|
||||
"--tag", tag,
|
||||
"--base-container", base_container
|
||||
@@ -246,28 +283,31 @@ def build_container_image(app_record, tag, extra_build_args=[], log_file=None):
|
||||
build_command.append("--extra-build-args")
|
||||
build_command.append(" ".join(extra_build_args))
|
||||
|
||||
result = subprocess.run(build_command, stdout=log_file, stderr=log_file)
|
||||
result = subprocess.run(build_command, stdout=logger.file, stderr=logger.file)
|
||||
result.check_returncode()
|
||||
finally:
|
||||
cmd("rm", "-rf", tmpdir)
|
||||
logged_cmd(logger.file, "rm", "-rf", tmpdir)
|
||||
|
||||
|
||||
def push_container_image(deployment_dir, log_file=None):
|
||||
print("Pushing image ...")
|
||||
def push_container_image(deployment_dir, logger):
|
||||
logger.log("Pushing images ...")
|
||||
result = subprocess.run([sys.argv[0], "deployment", "--dir", deployment_dir, "push-images"],
|
||||
stdout=log_file, stderr=log_file)
|
||||
stdout=logger.file, stderr=logger.file)
|
||||
result.check_returncode()
|
||||
logger.log("Finished pushing images.")
|
||||
|
||||
|
||||
def deploy_to_k8s(deploy_record, deployment_dir, log_file=None):
|
||||
def deploy_to_k8s(deploy_record, deployment_dir, logger):
|
||||
if not deploy_record:
|
||||
command = "up"
|
||||
else:
|
||||
command = "update"
|
||||
|
||||
logger.log("Deploying to k8s ...")
|
||||
result = subprocess.run([sys.argv[0], "deployment", "--dir", deployment_dir, command],
|
||||
stdout=log_file, stderr=log_file)
|
||||
stdout=logger.file, stderr=logger.file)
|
||||
result.check_returncode()
|
||||
logger.log("Finished deploying to k8s.")
|
||||
|
||||
|
||||
def publish_deployment(laconic: LaconicRegistryClient,
|
||||
@@ -277,7 +317,8 @@ def publish_deployment(laconic: LaconicRegistryClient,
|
||||
dns_record,
|
||||
dns_crn,
|
||||
deployment_dir,
|
||||
app_deployment_request=None):
|
||||
app_deployment_request=None,
|
||||
logger=None):
|
||||
if not deploy_record:
|
||||
deploy_ver = "0.0.1"
|
||||
else:
|
||||
@@ -307,6 +348,8 @@ def publish_deployment(laconic: LaconicRegistryClient,
|
||||
if app_deployment_request:
|
||||
new_dns_record["record"]["request"] = app_deployment_request.id
|
||||
|
||||
if logger:
|
||||
logger.log("Publishing DnsRecord.")
|
||||
dns_id = laconic.publish(new_dns_record, [dns_crn])
|
||||
|
||||
new_deployment_record = {
|
||||
@@ -326,6 +369,8 @@ def publish_deployment(laconic: LaconicRegistryClient,
|
||||
if app_deployment_request:
|
||||
new_deployment_record["record"]["request"] = app_deployment_request.id
|
||||
|
||||
if logger:
|
||||
logger.log("Publishing ApplicationDeploymentRecord.")
|
||||
deployment_id = laconic.publish(new_deployment_record, [deployment_crn])
|
||||
return {"dns": dns_id, "deployment": deployment_id}
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ import click
|
||||
|
||||
from stack_orchestrator.command_types import CommandOptions
|
||||
from stack_orchestrator.repos import setup_repositories
|
||||
from stack_orchestrator.build import build_containers
|
||||
from stack_orchestrator.build import build_containers, fetch_containers
|
||||
from stack_orchestrator.build import build_npms
|
||||
from stack_orchestrator.build import build_webapp
|
||||
from stack_orchestrator.deploy.webapp import (run_webapp,
|
||||
@@ -52,6 +52,7 @@ def cli(ctx, stack, quiet, verbose, dry_run, local_stack, debug, continue_on_err
|
||||
|
||||
cli.add_command(setup_repositories.command, "setup-repositories")
|
||||
cli.add_command(build_containers.command, "build-containers")
|
||||
cli.add_command(fetch_containers.command, "fetch-containers")
|
||||
cli.add_command(build_npms.command, "build-npms")
|
||||
cli.add_command(build_webapp.command, "build-webapp")
|
||||
cli.add_command(run_webapp.command, "run-webapp")
|
||||
|
||||
@@ -83,7 +83,7 @@ $TEST_TARGET_SO --stack test deploy down --delete-volumes
|
||||
# Basic test of creating a deployment
|
||||
test_deployment_dir=$CERC_REPO_BASE_DIR/test-deployment-dir
|
||||
test_deployment_spec=$CERC_REPO_BASE_DIR/test-deployment-spec.yml
|
||||
$TEST_TARGET_SO --stack test deploy init --output $test_deployment_spec --config CERC_TEST_PARAM_1=PASSED
|
||||
$TEST_TARGET_SO --stack test deploy init --output $test_deployment_spec --config CERC_TEST_PARAM_1=PASSED,CERC_TEST_PARAM_3=FAST
|
||||
# Check the file now exists
|
||||
if [ ! -f "$test_deployment_spec" ]; then
|
||||
echo "deploy init test: spec file not present"
|
||||
@@ -141,6 +141,13 @@ else
|
||||
echo "deployment compose config test: FAILED"
|
||||
exit 1
|
||||
fi
|
||||
# Check the config variable CERC_TEST_PARAM_3 was passed correctly
|
||||
if [[ "$log_output_3" == *"Test-param-3: FAST"* ]]; then
|
||||
echo "deployment config test: passed"
|
||||
else
|
||||
echo "deployment config test: FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check that the ConfigMap is mounted and contains the expected content.
|
||||
log_output_4=$( $TEST_TARGET_SO deployment --dir $test_deployment_dir logs )
|
||||
|
||||
Reference in New Issue
Block a user