Compare commits

..
Author SHA1 Message Date
Your Name 8684f64eb6 ports etc 2023-07-27 00:53:02 +00:00
zramsay 525d7ee49b uniswap updates 2023-07-26 19:44:59 -04:00
dboreham 29fc611885 Dboreham/mainnet eth (#464) 2023-07-25 10:16:19 -06:00
dboreham 1f9131ff5a Add deployment scripting (#444) 2023-07-23 20:54:05 -06:00
dboreham 950857fa84 Add --delete-volumes option to the down command (#461) 2023-07-20 06:25:31 -06:00
dboreham 54f50aa09e Move files into config subdirectory (#460) 2023-07-20 05:37:51 -06:00
telackey a5945c9e59 Back to Github-based actions. (#453)
* Back to Github-based actions.

* Use fork.

* Use new release plugin tag.
2023-07-19 20:40:04 -06:00
dboreham c8b4f89335 Copy all referenced config files into deployment dir (#452) 2023-07-18 08:59:07 -06:00
telackey 809889f9f0 Update plugeth stack for plugeth-statediff plugin. (#443)
* Use latest plugeth-statediff.

* Change plugeth path

* EOL

* CERC_PLUGINS_DIR
2023-06-29 15:11:28 -06:00
dboreham 6595659a7a Specify the git tag/branch for opera in stack.yml (#442) 2023-06-29 15:10:08 -06:00
Ian 9872ce33bb Merge pull request #435 from cerc-io/reth
Add reth stack
2023-06-28 14:22:59 -04:00
Ian 2ff490d121 Merge branch 'main' into reth 2023-06-28 14:11:43 -04:00
dboreham c76195c491 Auto-create bind mount directories inside the deployment dir (#440)
* Auto-create bind mount directories inside the deployment dir

* Fix deploy test
2023-06-27 21:18:04 -06:00
dboreham 831a8cd5f7 Deployments feature (#433)
Basic deployments framework
2023-06-27 16:58:41 -06:00
prathamesh0 f0f40cad4a Map ports in mobymask-v2 and apps stacks to localhost (#439) 2023-06-27 12:23:56 +05:30
Ian f473567bd7 Update README.md 2023-06-26 08:41:51 -04:00
prathamesh0 3e7037e06f Update instructions in the mobymask-v2 stack (#438) 2023-06-26 15:37:08 +05:30
dboreham c465153cc7 Fixes for repos checked out from a tag rather than a branch (#437)
* Fixes for repos checked out from a tag rather than a branch

* Fix happy path
2023-06-24 11:19:20 -06:00
iskay b120682d8e node sync, initial 2023-06-23 21:48:18 +00:00
Zach c9bb0d0ccb Lasso stack (#434)
* lasso stack

* add readme

* copy pasta
2023-06-23 14:27:05 -04:00
dboreham c94fa3cccf Use cerc npm registry as default (#432) 2023-06-22 06:36:05 -06:00
Zach 55af33e0e8 Merge pull request #427 from cerc-io/fantom
Fantom
2023-06-20 17:24:38 -04:00
zramsay c0f07c2d4f typo 2023-06-20 11:21:26 -04:00
Zach 7ca7bcc952 Cloud init scripts for user/dev mode (#430)
* cloud init install

* add dev mode script + description

* instructions
2023-06-20 10:09:30 -04:00
zramsay ee8c8a0c13 add fantom docs 2023-06-20 10:02:14 -04:00
iskay 30ce3deb31 update build script for rc.5 2023-06-16 18:39:59 +00:00
Ian a6cf131cd3 add network mode "host" 2023-06-13 12:19:23 -04:00
zramsay 25b13d7822 doc fixes 2023-06-12 21:00:30 -04:00
iskay 01500e78b6 initial node 2023-06-12 20:54:39 -04:00
iskay 4ecc98dd46 simple node 2023-06-12 20:54:39 -04:00
nabarun 32f8d65bb8 Update mobymask-v2 stack with lighthouse-cli and branch checkout feature (#425)
* Update optimism stack yml for lighthouse-cli

* Use branch checkout feature in mobymask stack
2023-06-07 18:48:59 +05:30
David Boreham d19b9a65b9 Fix typo 2023-06-05 21:59:42 -06:00
dborehamandDavid Boreham 98e1d120cc Add missing lighthouse-cli container to pocket stack (#424)
Co-authored-by: David Boreham <david@bozemanpas.com>
2023-06-05 21:08:05 -06:00
telackey 26ff7a969c Fix plugeth build. (#423) 2023-06-05 21:10:17 -05:00
telackey a8e198ad55 Allow configuring the number of statediff workers. (#422)
* Allow configuring the number of statediff workers.

* Leave logging alone
2023-06-05 18:16:42 -05:00
f1a626ddf5 build local lighthouse cli (#420)
* Build lcli locally

* Pull lighthouse repo

* Enable portable lcli build

* Update ldcli options

* Add lcli container to fixturenet-eth stack

* Include --eth1-block-hash

---------

Co-authored-by: David Boreham <david@bozemanpas.com>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
2023-06-05 16:54:22 -05:00
roysc ff616db4ad Updates for running IPLD-ETH CI tests (#414)
* random nits

* geth - visibility of migration status

* forward CERC_RUN_STATEDIFF to geth container

* fix ipld-eth-server vars

* fix fixturenet-eth-loaded stack

* fixturenet geth genesis - include mergeNetsplitBlock

* forward CERC_STATEDIFF_DB_GOOSE_MIN_VER to env file

* add TAG_SUFFIX arg to lighthouse build

  intended to avoid sporadic failures when running lcli on github CI runners, likely related to non-portable builds
2023-05-31 03:10:58 -05:00
dboreham 9880b48b78 Add foundry to fixturenet-plugeth-tx (#418) 2023-05-30 23:51:01 -06:00
telackey 23a336020c Make a separate lighthouse container for the plugeth fixturenet. (#412)
* Make a separate lighthouse container for the plugeth fixturenet.
2023-05-26 16:57:15 -05:00
Zach 605db8a4d2 Update pokt README (#413)
* Update pokt README

* split cmds from responses
2023-05-26 10:37:59 -04:00
telackey 6ec55ba460 Add a plugeth-based version of the fixturenet (#411)
* plugeth version of the fixturenet

* Use pre-built plugeth.
2023-05-25 11:21:08 -05:00
dboreham 938f51ef8c Specify chunker stack branches (#410)
* Specify v5 branches

* Fix logic for branch switch
2023-05-24 20:00:42 -06:00
dboreham 6d620ba9c2 git branch in stack and on command line (#409)
* Support @branch notation in stack.yml

* Refactor and support branches directive
2023-05-24 19:49:26 -06:00
erikdiesanddboreham 0c4c128465 cleanup Options boilerplate (#402)
Co-authored-by: David Boreham <david@bozemanpass.com>
2023-05-24 18:02:25 -06:00
111 changed files with 2208 additions and 354 deletions
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
- name: "Clone project repository"
uses: actions/checkout@v3
- name: "Install Python"
uses: cerc-io/setup-python@v4
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
+2 -2
View File
@@ -19,7 +19,7 @@ jobs:
build_tag=$(./scripts/create_build_tag_file.sh)
echo "build-tag=v${build_tag}" >> $GITHUB_OUTPUT
- name: "Install Python"
uses: cerc-io/setup-python@v4
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
@@ -37,7 +37,7 @@ jobs:
run: |
cp ${{ steps.build.outputs.package-file }} ./laconic-so
- name: "Create release"
uses: cerc-io/action-gh-release@gitea-v1
uses: https://gitea.com/cerc-io/action-gh-release@gitea-v2
with:
tag_name: ${{ steps.build-info.outputs.build-tag }}
# On the publish test branch, mark our release as a draft
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
- name: "Clone project repository"
uses: actions/checkout@v3
- name: "Install Python"
uses: cerc-io/setup-python@v4
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
- name: "Clone project repository"
uses: actions/checkout@v3
- name: "Install Python"
uses: cerc-io/setup-python@v4
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
+9 -1
View File
@@ -15,7 +15,8 @@
import os
from abc import ABC, abstractmethod
from .deploy_system import get_stack_status
from app.deploy import get_stack_status
from decouple import config
def get_stack(config, stack):
@@ -69,3 +70,10 @@ class package_registry_stack(base_stack):
def get_url(self):
return self.url
def get_npm_registry_url():
# If an auth token is not defined, we assume the default should be the cerc registry
# If an auth token is defined, we assume the local gitea should be used.
default_npm_registry_url = "http://gitea.local:3000/api/packages/cerc-io/npm/" if config("CERC_NPM_AUTH_TOKEN", default=None) else "https://git.vdb.to/api/packages/cerc-io/npm/"
return config("CERC_NPM_REGISTRY_URL", default=default_npm_registry_url)
+4 -3
View File
@@ -27,7 +27,8 @@ import subprocess
import click
import importlib.resources
from pathlib import Path
from .util import include_exclude_check, get_parsed_stack_config
from app.util import include_exclude_check, get_parsed_stack_config
from app.base import get_npm_registry_url
# TODO: find a place for this
# epilog="Config provided either in .env or settings.ini or env vars: CERC_REPO_BASE_DIR (defaults to ~/cerc)"
@@ -66,7 +67,7 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
print('Dev root directory doesn\'t exist, creating')
# See: https://stackoverflow.com/a/20885799/1701505
from . import data
from app import data
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
all_containers = container_list_file.read().splitlines()
@@ -84,7 +85,7 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
# TODO: make this configurable
container_build_env = {
"CERC_NPM_REGISTRY_URL": config("CERC_NPM_REGISTRY_URL", default="http://gitea.local:3000/api/packages/cerc-io/npm/"),
"CERC_NPM_REGISTRY_URL": get_npm_registry_url(),
"CERC_NPM_AUTH_TOKEN": config("CERC_NPM_AUTH_TOKEN", default=""),
"CERC_REPO_BASE_DIR": dev_root_path,
"CERC_CONTAINER_BASE_DIR": container_build_dir,
+3 -3
View File
@@ -25,8 +25,8 @@ from decouple import config
import click
import importlib.resources
from python_on_whales import docker, DockerException
from .base import get_stack
from .util import include_exclude_check, get_parsed_stack_config
from app.base import get_stack
from app.util import include_exclude_check, get_parsed_stack_config
builder_js_image_name = "cerc/builder-js:local"
@@ -81,7 +81,7 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
os.makedirs(build_root_path)
# See: https://stackoverflow.com/a/20885799/1701505
from . import data
from app import data
with importlib.resources.open_text(data, "npm-package-list.txt") as package_list_file:
all_packages = package_list_file.read().splitlines()
@@ -22,7 +22,7 @@ services:
- SYS_PTRACE
environment:
CERC_REMOTE_DEBUG: "true"
CERC_RUN_STATEDIFF: "detect"
CERC_RUN_STATEDIFF: ${CERC_RUN_STATEDIFF:-detect}
CERC_STATEDIFF_DB_NODE_ID: 1
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
env_file:
@@ -25,3 +25,6 @@ services:
image: cerc/laconic-registry-cli:local
volumes:
- ../config/fixturenet-laconicd/registry-cli-config-template.yml:/registry-cli-config-template.yml
volumes:
laconicd-data:
@@ -23,10 +23,10 @@ services:
command: |
"./wait-for-it.sh -h ${CERC_L1_HOST:-$${DEFAULT_CERC_L1_HOST}} -p ${CERC_L1_PORT:-$${DEFAULT_CERC_L1_PORT}} -s -t 60 -- ./run.sh"
volumes:
- ../config/wait-for-it.sh:/app/packages/contracts-bedrock/wait-for-it.sh
- ../container-build/cerc-optimism-contracts/hardhat-tasks/verify-contract-deployment.ts:/app/packages/contracts-bedrock/tasks/verify-contract-deployment.ts
- ../container-build/cerc-optimism-contracts/hardhat-tasks/rekey-json.ts:/app/packages/contracts-bedrock/tasks/rekey-json.ts
- ../container-build/cerc-optimism-contracts/hardhat-tasks/send-balance.ts:/app/packages/contracts-bedrock/tasks/send-balance.ts
- ../config/network/wait-for-it.sh:/app/packages/contracts-bedrock/wait-for-it.sh
- ../config/optimism-contracts/hardhat-tasks/verify-contract-deployment.ts:/app/packages/contracts-bedrock/tasks/verify-contract-deployment.ts
- ../config/optimism-contracts/hardhat-tasks/rekey-json.ts:/app/packages/contracts-bedrock/tasks/rekey-json.ts
- ../config/optimism-contracts/hardhat-tasks/send-balance.ts:/app/packages/contracts-bedrock/tasks/send-balance.ts
- ../config/fixturenet-optimism/optimism-contracts/update-config.js:/app/packages/contracts-bedrock/update-config.js
- ../config/fixturenet-optimism/optimism-contracts/run.sh:/app/packages/contracts-bedrock/run.sh
- l2_accounts:/l2-accounts
@@ -120,7 +120,7 @@ services:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_L1_RPC: ${CERC_L1_RPC}
volumes:
- ../config/wait-for-it.sh:/wait-for-it.sh
- ../config/network/wait-for-it.sh:/wait-for-it.sh
- ../config/fixturenet-optimism/run-op-batcher.sh:/run-op-batcher.sh
- l2_accounts:/l2-accounts:ro
entrypoint: ["sh", "-c"]
@@ -145,7 +145,7 @@ services:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_L1_RPC: ${CERC_L1_RPC}
volumes:
- ../config/wait-for-it.sh:/wait-for-it.sh
- ../config/network/wait-for-it.sh:/wait-for-it.sh
- ../config/fixturenet-optimism/run-op-proposer.sh:/run-op-proposer.sh
- l1_deployment:/contracts-bedrock:ro
- l2_accounts:/l2-accounts:ro
@@ -0,0 +1,127 @@
services:
fixturenet-eth-bootnode-geth:
restart: always
hostname: fixturenet-eth-bootnode-geth
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
environment:
RUN_BOOTNODE: "true"
image: cerc/fixturenet-plugeth-plugeth:local
volumes:
- fixturenet_plugeth_bootnode_geth_data:/root/ethdata
- ../config/fixturenet-plugeth/plugins:/root/ethdata/plugins
ports:
- "9898"
- "30303"
fixturenet-eth-geth-1:
restart: always
hostname: fixturenet-eth-geth-1
cap_add:
- SYS_PTRACE
environment:
CERC_REMOTE_DEBUG: "true"
CERC_RUN_STATEDIFF: "detect"
CERC_STATEDIFF_DB_NODE_ID: 1
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
image: cerc/fixturenet-plugeth-plugeth:local
volumes:
- fixturenet_plugeth_geth_1_data:/root/ethdata
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8545/"]
interval: 30s
timeout: 10s
retries: 10
start_period: 3s
depends_on:
- fixturenet-eth-bootnode-geth
ports:
- "8545"
- "40000"
- "6060"
fixturenet-eth-geth-2:
restart: always
hostname: fixturenet-eth-geth-2
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8545/"]
interval: 30s
timeout: 10s
retries: 10
start_period: 3s
environment:
CERC_KEEP_RUNNING_AFTER_GETH_EXIT: "true"
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
image: cerc/fixturenet-plugeth-plugeth:local
depends_on:
- fixturenet-eth-bootnode-geth
volumes:
- fixturenet_plugeth_geth_2_data:/root/ethdata
fixturenet-eth-bootnode-lighthouse:
restart: always
hostname: fixturenet-eth-bootnode-lighthouse
environment:
RUN_BOOTNODE: "true"
image: cerc/fixturenet-plugeth-lighthouse:local
fixturenet-eth-lighthouse-1:
restart: always
hostname: fixturenet-eth-lighthouse-1
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8001/eth/v2/beacon/blocks/head"]
interval: 30s
timeout: 10s
retries: 10
start_period: 30s
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
environment:
NODE_NUMBER: "1"
ETH1_ENDPOINT: "http://fixturenet-eth-geth-1:8545"
EXECUTION_ENDPOINT: "http://fixturenet-eth-geth-1:8551"
image: cerc/fixturenet-plugeth-lighthouse:local
volumes:
- fixturenet_plugeth_lighthouse_1_data:/opt/testnet/build/cl
depends_on:
fixturenet-eth-bootnode-lighthouse:
condition: service_started
fixturenet-eth-geth-1:
condition: service_healthy
ports:
- "8001"
fixturenet-eth-lighthouse-2:
restart: always
hostname: fixturenet-eth-lighthouse-2
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8001/eth/v2/beacon/blocks/head"]
interval: 30s
timeout: 10s
retries: 10
start_period: 30s
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
environment:
NODE_NUMBER: "2"
ETH1_ENDPOINT: "http://fixturenet-eth-geth-2:8545"
EXECUTION_ENDPOINT: "http://fixturenet-eth-geth-2:8551"
LIGHTHOUSE_GENESIS_STATE_URL: "http://fixturenet-eth-lighthouse-1:8001/eth/v2/debug/beacon/states/0"
image: cerc/fixturenet-plugeth-lighthouse:local
volumes:
- fixturenet_plugeth_lighthouse_2_data:/opt/testnet/build/cl
depends_on:
fixturenet-eth-bootnode-lighthouse:
condition: service_started
fixturenet-eth-geth-2:
condition: service_healthy
volumes:
fixturenet_plugeth_bootnode_geth_data:
fixturenet_plugeth_geth_1_data:
fixturenet_plugeth_geth_2_data:
fixturenet_plugeth_lighthouse_1_data:
fixturenet_plugeth_lighthouse_2_data:
@@ -7,11 +7,9 @@ services:
condition: service_healthy
image: cerc/ipld-eth-server:local
environment:
IPLD_SERVER_GRAPHQL: "true"
IPLD_POSTGRAPHILEPATH: http://graphql:5000
ETH_SERVER_HTTPPATH: 0.0.0.0:8081
ETH_SERVER_GRAPHQL: "true"
ETH_SERVER_GRAPHQLPATH: 0.0.0.0:8082
SERVER_HTTP_PATH: 0.0.0.0:8081
SERVER_GRAPHQL: "true"
SERVER_GRAPHQLPATH: 0.0.0.0:8082
VDB_COMMAND: "serve"
ETH_CHAIN_CONFIG: "/tmp/chain.json"
DATABASE_NAME: cerc_testing
@@ -0,0 +1,8 @@
version: "3.2"
services:
lasso:
image: cerc/lasso:local
restart: always
ports:
- "0.0.0.0:3000:3000"
@@ -0,0 +1,56 @@
services:
mainnet-eth-geth-1:
restart: always
hostname: mainnet-eth-geth-1
cap_add:
- SYS_PTRACE
environment:
CERC_REMOTE_DEBUG: "true"
CERC_RUN_STATEDIFF: ${CERC_RUN_STATEDIFF:-detect}
CERC_STATEDIFF_DB_NODE_ID: 1
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
image: cerc/go-ethereum:local
entrypoint: /bin/sh
command: -c "/opt/run-geth.sh"
volumes:
- mainnet_eth_geth_1_data:/root/ethdata
- mainnet_eth_config_data:/etc/mainnet-eth
- ../config/mainnet-eth/scripts/run-geth.sh:/opt/run-geth.sh
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "8545"]
interval: 30s
timeout: 10s
retries: 10
start_period: 3s
ports:
- "8545"
- "40000"
- "6060"
mainnet-eth-lighthouse-1:
restart: always
hostname: mainnet-eth-lighthouse-1
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8001/eth/v2/beacon/blocks/head"]
interval: 30s
timeout: 10s
retries: 10
start_period: 30s
environment:
EXECUTION_ENDPOINT: "http://mainnet-eth-geth-1:8551"
image: cerc/lighthouse:local
entrypoint: /bin/sh
command: -c "/opt/run-lighthouse.sh"
volumes:
- mainnet_eth_lighthouse_1_data:/var/lighthouse-data-dir
- mainnet_eth_config_data:/etc/mainnet-eth
- ../config/mainnet-eth/scripts/run-lighthouse.sh:/opt/run-lighthouse.sh
ports:
- "8001"
volumes:
mainnet_eth_config_data:
mainnet_eth_geth_1_data:
mainnet_eth_lighthouse_1_data:
@@ -0,0 +1,17 @@
version: "3.8"
services:
go-opera:
restart: unless-stopped
image: cerc/go-opera:local
entrypoint: ["sh", "/docker-entrypoint-scripts.d/start-node.sh"]
volumes:
- ../config/mainnet-go-opera/start-node.sh:/docker-entrypoint-scripts.d/start-node.sh
# TODO: ports taken from dockerfile, determine which are needed
ports:
- "5050:5050" # p2p port, needed
- "5050:5050/udp"
- "18545:18545" # http rpc port
- "18546:18546" # websockets rpc port
#- "18547" # unknown
#- "19090" # unknown
@@ -0,0 +1,30 @@
services:
laconicd:
restart: no
image: cerc/laconicd:local
command: ["/bin/sh", "-c", "while :; do sleep 600; done"]
volumes:
# The cosmos-sdk node's database directory:
- laconicd-data:/root/.laconicd/data
# TODO: look at folding these scripts into the container
- ../config/mainnet-laconicd/create-fixturenet.sh:/docker-entrypoint-scripts.d/create-fixturenet.sh
- ../config/mainnet-laconicd/export-mykey.sh:/docker-entrypoint-scripts.d/export-mykey.sh
- ../config/mainnet-laconicd/export-myaddress.sh:/docker-entrypoint-scripts.d/export-myaddress.sh
# TODO: determine which of the ports below is really needed
ports:
- "6060"
- "26657"
- "26656"
- "9473:9473"
- "8545"
- "8546"
- "9090"
- "9091"
- "1317"
cli:
image: cerc/laconic-registry-cli:local
volumes:
- ../config/mainnet-laconicd/registry-cli-config-template.yml:/registry-cli-config-template.yml
volumes:
laconicd-data:
@@ -18,12 +18,12 @@ services:
working_dir: /scripts
command: ["sh", "mobymask-app-start.sh"]
volumes:
- ../config/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/network/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/watcher-mobymask-v2/mobymask-app-start.sh:/scripts/mobymask-app-start.sh
- peers_ids:/peers
- mobymask_deployment:/server
ports:
- "0.0.0.0:3002:80"
- "127.0.0.1:3002:80"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "80"]
interval: 20s
@@ -50,12 +50,12 @@ services:
working_dir: /scripts
command: ["sh", "mobymask-app-start.sh"]
volumes:
- ../config/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/network/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/watcher-mobymask-v2/mobymask-app-start.sh:/scripts/mobymask-app-start.sh
- peers_ids:/peers
- mobymask_deployment:/server
ports:
- "0.0.0.0:3004:80"
- "127.0.0.1:3004:80"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "80"]
interval: 20s
@@ -1,8 +1,9 @@
version: '3.2'
services:
# Builds and serves the peer-test react-app
peer-test-app:
# Builds and serves the peer-test react-app
restart: unless-stopped
image: cerc/react-peer:local
working_dir: /scripts
env_file:
@@ -13,11 +14,11 @@ services:
CERC_DENY_MULTIADDRS: ${CERC_DENY_MULTIADDRS}
command: ["sh", "test-app-start.sh"]
volumes:
- ../config/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/network/wait-for-it.sh:/scripts/wait-for-it.sh
- ../config/watcher-mobymask-v2/test-app-start.sh:/scripts/test-app-start.sh
- peers_ids:/peers
ports:
- "0.0.0.0:3003:80"
- "127.0.0.1:3003:80"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "80"]
interval: 20s
+35
View File
@@ -0,0 +1,35 @@
version: "3.8"
services:
reth:
restart: unless-stopped
hostname: reth
image: cerc/reth:local
entrypoint: ["sh", "/docker-entrypoint-scripts.d/start-reth.sh"]
volumes:
- ../config/reth/start-reth.sh:/docker-entrypoint-scripts.d/start-reth.sh
- reth_data:/root/.local/share/reth
- shared_data:/root/.shared_data
ports:
- "8545:8545" # http rpc
- "8546:8546" # ws rpc
- "30303:30303" # network listening port
- "30303:30303/udp"
- "8551" # consensus auth
lighthouse:
restart: unless-stopped
hostname: lighthouse
image: cerc/lighthouse:local
entrypoint: ["sh", "/docker-entrypoint-scripts.d/start-lighthouse.sh"]
volumes:
- ../config/reth/start-lighthouse.sh:/docker-entrypoint-scripts.d/start-lighthouse.sh
- lighthouse_data:/root/.lighthouse/mainnet
- shared_data:/root/.shared_data
ports:
- "8001"
volumes:
reth_data:
lighthouse_data:
shared_data:
+1 -1
View File
@@ -5,7 +5,7 @@ services:
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
volumes:
- test-data:/var
- test-data:/data
ports:
- "80"
@@ -14,7 +14,7 @@ services:
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
- mobymask_watcher_db_data:/var/lib/postgresql/data
ports:
- "0.0.0.0:15432:5432"
- "127.0.0.1:15432:5432"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "5432"]
interval: 20s
@@ -44,7 +44,7 @@ services:
CERC_L2_NODE_PORT: ${CERC_L2_NODE_PORT}
command: ["sh", "deploy-and-generate-invite.sh"]
volumes:
- ../config/wait-for-it.sh:/app/packages/server/wait-for-it.sh
- ../config/network/wait-for-it.sh:/app/packages/server/wait-for-it.sh
- ../config/watcher-mobymask-v2/secrets-template.json:/app/packages/server/secrets-template.json
- ../config/watcher-mobymask-v2/deploy-and-generate-invite.sh:/app/packages/server/deploy-and-generate-invite.sh
- mobymask_deployment:/app/packages/server
@@ -95,9 +95,9 @@ services:
- mobymask_deployment:/server
# Expose GQL, metrics and relay node ports
ports:
- "0.0.0.0:3001:3001"
- "0.0.0.0:9001:9001"
- "0.0.0.0:9090:9090"
- "127.0.0.1:3001:3001"
- "127.0.0.1:9001:9001"
- "127.0.0.1:9090:9090"
healthcheck:
test: ["CMD", "busybox", "nc", "localhost", "9090"]
interval: 20s
@@ -75,6 +75,8 @@ services:
uni-watcher-server:
restart: unless-stopped
depends_on:
ipld-eth-server:
condition: service_healthy
uniswap-watcher-db:
condition: service_healthy
uni-watcher-job-runner:
@@ -17,7 +17,8 @@ CERC_STATEDIFF_DB_PORT=5432
CERC_STATEDIFF_DB_NAME="cerc_testing"
CERC_STATEDIFF_DB_USER="vdbm"
CERC_STATEDIFF_DB_PASSWORD="password"
CERC_STATEDIFF_DB_GOOSE_MIN_VER=23
CERC_STATEDIFF_DB_GOOSE_MIN_VER=${CERC_STATEDIFF_DB_GOOSE_MIN_VER:-18}
CERC_STATEDIFF_DB_LOG_STATEMENTS="false"
CERC_STATEDIFF_WORKERS=2
CERC_GETH_VMODULE="statediff/*=5,rpc/*=5"
@@ -0,0 +1 @@
See: https://docs.plugeth.org/
+52
View File
@@ -0,0 +1,52 @@
#!/bin/sh
if [[ -n "$CERC_SCRIPT_DEBUG" ]]; then
set -x
fi
CERC_ETH_DATADIR=/root/ethdata
START_CMD="geth"
if [ "true" == "$CERC_REMOTE_DEBUG" ] && [ -x "/usr/local/bin/dlv" ]; then
START_CMD="/usr/local/bin/dlv --listen=:40000 --headless=true --api-version=2 --accept-multiclient exec /usr/local/bin/geth --continue --"
fi
# See https://linuxconfig.org/how-to-propagate-a-signal-to-child-processes-from-a-bash-script
cleanup() {
echo "Signal received, cleaning up..."
# Kill the child process first (CERC_REMOTE_DEBUG=true uses dlv which starts geth as a child process)
pkill -P ${geth_pid}
sleep 2
kill $(jobs -p)
wait
echo "Done"
}
$START_CMD \
--datadir="${CERC_ETH_DATADIR}" \
--authrpc.addr="0.0.0.0" \
--authrpc.port 8551 \
--authrpc.vhosts="*" \
--authrpc.jwtsecret="/etc/mainnet-eth/jwtsecret" \
--ws \
--ws.addr="0.0.0.0" \
--ws.origins="*" \
--ws.api="${CERC_GETH_WS_APIS:-eth,web3,net,admin,personal,debug,statediff}" \
--http.corsdomain="*" \
--gcmode full \
--txlookuplimit=0 \
--cache.preimages \
--syncmode=snap \
&
geth_pid=$!
wait $geth_pid
if [ "true" == "$CERC_KEEP_RUNNING_AFTER_GETH_EXIT" ]; then
while [ 1 -eq 1 ]; do
sleep 60
done
fi
+22
View File
@@ -0,0 +1,22 @@
#!/bin/bash
if [[ -n "$CERC_SCRIPT_DEBUG" ]]; then
set -x
fi
DEBUG_LEVEL=${CERC_LIGHTHOUSE_DEBUG_LEVEL:-info}
data_dir=/var/lighthouse-data-dir
network_port=9001
http_port=8001
authrpc_port=8551
exec lighthouse \
bn \
--debug-level $DEBUG_LEVEL \
--datadir $data_dir \
--network mainnet \
--execution-endpoint $EXECUTION_ENDPOINT \
--execution-jwt /etc/mainnet-eth/jwtsecret \
--disable-deposit-contract-sync \
--checkpoint-sync-url https://beaconstate.ethstaker.cc
@@ -0,0 +1 @@
@@ -0,0 +1,7 @@
#!/bin/bash
# download genesis file
wget https://download.fantom.network/mainnet-109331-no-history.g
./opera --genesis=mainnet-109331-no-history.g --db.preset ldb-1 --syncmode snap --http --http.addr="0.0.0.0" --http.corsdomain="*" --http.api=eth,web3,net,txpool,ftm --ws --ws.addr="0.0.0.0" --ws.origins="*" --ws.api=eth,web3,net,txpool,ftm --cache 8192
#tail -f /dev/null
@@ -0,0 +1,118 @@
#!/bin/bash
# TODO: this file is now an unmodified copy of cerc-io/laconicd/init.sh
# so we should have a mechanism to bundle it inside the container rather than link from here
# at deploy time.
KEY="mykey"
CHAINID="laconic_9000-1"
MONIKER="localtestnet"
KEYRING="test"
KEYALGO="eth_secp256k1"
LOGLEVEL="info"
# trace evm
TRACE="--trace"
# TRACE=""
# validate dependencies are installed
command -v jq > /dev/null 2>&1 || { echo >&2 "jq not installed. More info: https://stedolan.github.io/jq/download/"; exit 1; }
# remove existing daemon and client
rm -rf ~/.laconic*
make install
laconicd config keyring-backend $KEYRING
laconicd config chain-id $CHAINID
# if $KEY exists it should be deleted
laconicd keys add $KEY --keyring-backend $KEYRING --algo $KEYALGO
# Set moniker and chain-id for Ethermint (Moniker can be anything, chain-id must be an integer)
laconicd init $MONIKER --chain-id $CHAINID
# Change parameter token denominations to aphoton
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["staking"]["params"]["bond_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["crisis"]["constant_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["gov"]["deposit_params"]["min_deposit"][0]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["mint"]["params"]["mint_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Custom modules
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commit_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveal_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_minimum_bid"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
if [[ "$TEST_REGISTRY_EXPIRY" == "true" ]]; then
echo "Setting timers for expiry tests."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
if [[ "$TEST_AUCTION_ENABLED" == "true" ]]; then
echo "Enabling auction and setting timers."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
# increase block time (?)
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["time_iota_ms"]="1000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Set gas limit in genesis
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["max_gas"]="10000000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# disable produce empty block
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
fi
if [[ $1 == "pending" ]]; then
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
fi
fi
# Allocate genesis accounts (cosmos formatted addresses)
laconicd add-genesis-account $KEY 100000000000000000000000000aphoton --keyring-backend $KEYRING
# Sign genesis transaction
laconicd gentx $KEY 1000000000000000000000aphoton --keyring-backend $KEYRING --chain-id $CHAINID
# Collect genesis tx
laconicd collect-gentxs
# Run this to ensure everything worked and that the genesis file is setup correctly
laconicd validate-genesis
if [[ $1 == "pending" ]]; then
echo "pending mode is on, please wait for the first block committed."
fi
# Start the node (remove the --pruning=nothing flag if historical queries are not needed)
laconicd start --pruning=nothing --evm.tracer=json $TRACE --log_level $LOGLEVEL --minimum-gas-prices=0.0001aphoton --json-rpc.api eth,txpool,personal,net,debug,web3,miner --api.enable --gql-server --gql-playground
@@ -0,0 +1,2 @@
#!/bin/sh
laconicd keys show mykey | grep address | cut -d ' ' -f 3
@@ -0,0 +1,2 @@
#!/bin/sh
echo y | laconicd keys export mykey --unarmored-hex --unsafe
@@ -0,0 +1,9 @@
services:
cns:
restEndpoint: 'http://laconicd:1317'
gqlEndpoint: 'http://laconicd:9473/api'
userKey: REPLACE_WITH_MYKEY
bondId:
chainId: laconic_9000-1
gas: 250000
fees: 200000aphoton
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Wait for reth container to create jwt auth token
while [ ! -f /root/.shared_data/jwt.hex ]; do
echo "Jwt auth token not found, sleeping for 5s..."
sleep 5
done
echo "Jwt token found. Starting Lighthouse..."
export RUST_LOG=info
lighthouse bn \
--network mainnet \
--execution-endpoint http://reth:8551 \
--execution-jwt /root/.shared_data/jwt.hex \
--checkpoint-sync-url https://mainnet.checkpoint.sigp.io \
--disable-deposit-contract-sync
+24
View File
@@ -0,0 +1,24 @@
#!/bin/bash
# generate jwt token for reth/lighthouse authentication
echo "Installing OpenSSL..."
apt update
apt install openssl
echo "Generating jwt token for lighthouse auth..."
openssl rand -hex 32 | tr -d "\n" | tee /root/.shared_data/jwt.hex
# start reth
echo "Starting Reth..."
export RUST_LOG=info
reth node \
--authrpc.jwtsecret /root/.shared_data/jwt.hex \
--authrpc.addr 0.0.0.0 \
--authrpc.port 8551 \
--http \
--http.addr 0.0.0.0 \
--http.corsdomain * \
--http.api eth,web3,net,rpc \
--ws \
--ws.addr 0.0.0.0 \
--ws.origins * \
--ws.api eth,web3,net,rpc
@@ -10,7 +10,7 @@ DEFAULT_CERC_RELAY_PEERS=[]
DEFAULT_CERC_RELAY_ANNOUNCE_DOMAIN=
# Base URI for mobymask-app (used for generating invite)
DEFAULT_CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3002/#"
DEFAULT_CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3004/#"
# Set to false for disabling watcher peer to send txs to L2
DEFAULT_CERC_ENABLE_PEER_L2_TXS=true
@@ -23,8 +23,8 @@
[upstream]
[upstream.ethServer]
gqlApiEndpoint = "http://ipld-eth-server.example.com:8083/graphql"
rpcProviderEndpoint = "http://ipld-eth-server.example.com:8082"
gqlApiEndpoint = "http://ipld-eth-server:8082/graphql"
rpcProviderEndpoint = "http://ipld-eth-server:8081"
[upstream.cache]
name = "requests"
@@ -63,8 +63,8 @@
[upstream]
[upstream.ethServer]
gqlApiEndpoint = "http://ipld-eth-server.example.com:8083/graphql"
rpcProviderEndpoint = "http://ipld-eth-server.example.com:8082"
gqlApiEndpoint = "http://ipld-eth-server:8082/graphql"
rpcProviderEndpoint = "http://ipld-eth-server:8081"
[upstream.cache]
name = "requests"
@@ -22,8 +22,8 @@
[upstream]
[upstream.ethServer]
gqlApiEndpoint = "http://ipld-eth-server.example.com:8083/graphql"
rpcProviderEndpoint = "http://ipld-eth-server.example.com:8082"
gqlApiEndpoint = "http://ipld-eth-server:8082/graphql"
rpcProviderEndpoint = "http://ipld-eth-server:8081"
[upstream.cache]
name = "requests"
@@ -22,6 +22,18 @@ COPY run-el.sh /opt/testnet/run.sh
RUN cd /opt/testnet && make genesis-el
COPY --from=geth /usr/local/bin/geth /usr/local/bin/
# Snag the genesis block info.
RUN geth --datadir ~/ethdata init /opt/testnet/build/el/geth.json && rm -f ~/ethdata/geth/nodekey
RUN cp -rp ~/ethdata ~/tmpeth && \
geth --datadir ~/tmpeth init /opt/testnet/build/el/geth.json && \
geth --datadir ~/tmpeth --http & \
sleep 5 && \
curl -q --location 'localhost:8545' \
--header 'Content-Type: application/json' \
--data '{ "jsonrpc": "2.0", "id": 14, "method": "eth_getBlockByNumber", "params": ["0x0", false] }' \
-o /opt/testnet/build/el/genesis_block.json && \
killall -9 geth && \
rm -rf ~/tmpeth
ENTRYPOINT ["/opt/testnet/run.sh"]
@@ -34,5 +34,7 @@ python3 /apps/el-gen/genesis_geth.py $tmp_dir/genesis-config.yaml | \
jq ".config.istanbulBlock=$istanbul_block" | \
jq ".config.berlinBlock=$berlin_block" | \
jq ".config.londonBlock=$london_block" | \
jq ".config.mergeForkBlock=$merge_fork_block" > ../build/el/geth.json
jq ".config.mergeForkBlock=$merge_fork_block" | \
jq ".config.mergeNetsplitBlock=$merge_fork_block" \
> ../build/el/geth.json
python3 ../accounts/mnemonic_to_csv.py $tmp_dir/genesis-config.yaml > ../build/el/accounts.csv
@@ -7,11 +7,12 @@ fi
ETHERBASE=`cat /opt/testnet/build/el/accounts.csv | head -1 | cut -d',' -f2`
NETWORK_ID=`cat /opt/testnet/el/el-config.yaml | grep 'chain_id' | awk '{ print $2 }'`
NETRESTRICT=`ip addr | grep inet | grep -v '127.0' | awk '{print $2}'`
CERC_ETH_DATADIR="${CERC_ETH_DATADIR:-$HOME/ethdata}"
CERC_PLUGINS_DIR="${CERC_PLUGINS_DIR:-/usr/local/lib/plugeth}"
HOME_DIR=`pwd`
cd /opt/testnet/build/el
python3 -m http.server 9898 &
cd $HOME_DIR
cd $HOME
START_CMD="geth"
if [ "true" == "$CERC_REMOTE_DEBUG" ] && [ -x "/usr/local/bin/dlv" ]; then
@@ -34,7 +35,7 @@ trap 'cleanup' SIGINT SIGTERM
if [ "true" == "$RUN_BOOTNODE" ]; then
$START_CMD \
--datadir=~/ethdata \
--datadir="${CERC_ETH_DATADIR}" \
--nodekeyhex="${BOOTNODE_KEY}" \
--nodiscover \
--ipcdisable \
@@ -64,8 +65,8 @@ else
STATEDIFF_OPTS=""
if [ "$CERC_RUN_STATEDIFF" == "true" ]; then
ready=0
echo "Waiting for statediff DB..."
while [ $ready -eq 0 ]; do
echo "Waiting for statediff DB..."
sleep 1
export PGPASSWORD="$CERC_STATEDIFF_DB_PASSWORD"
result=$(psql -h "$CERC_STATEDIFF_DB_HOST" \
@@ -73,12 +74,16 @@ else
-U "$CERC_STATEDIFF_DB_USER" \
-d "$CERC_STATEDIFF_DB_NAME" \
-t -c 'select max(version_id) from goose_db_version;' 2>/dev/null | awk '{ print $1 }')
if [ -n "$result" ] && [ $result -ge $CERC_STATEDIFF_DB_GOOSE_MIN_VER ]; then
if [ -n "$result" ]; then
echo "DB ready..."
ready=1
if [ $result -ge $CERC_STATEDIFF_DB_GOOSE_MIN_VER ]; then
ready=1
else
echo "DB not at required version (want $CERC_STATEDIFF_DB_GOOSE_MIN_VER, have $result)"
fi
fi
done
STATEDIFF_OPTS="--statediff=true \
STATEDIFF_OPTS="--statediff \
--statediff.db.host=$CERC_STATEDIFF_DB_HOST \
--statediff.db.name=$CERC_STATEDIFF_DB_NAME \
--statediff.db.nodeid=$CERC_STATEDIFF_DB_NODE_ID \
@@ -88,11 +93,17 @@ else
--statediff.db.logstatements=${CERC_STATEDIFF_DB_LOG_STATEMENTS:-false} \
--statediff.db.copyfrom=${CERC_STATEDIFF_DB_COPY_FROM:-true} \
--statediff.waitforsync=true \
--statediff.workers=${CERC_STATEDIFF_WORKERS:-1} \
--statediff.writing=true"
if [ -d "${CERC_PLUGINS_DIR}" ]; then
# With plugeth, we separate the statediff options by prefixing with ' -- '
STATEDIFF_OPTS="--pluginsdir "${CERC_PLUGINS_DIR}" -- ${STATEDIFF_OPTS}"
fi
fi
$START_CMD \
--datadir=~/ethdata \
--datadir="${CERC_ETH_DATADIR}" \
--bootnodes="${ENODE}" \
--allow-insecure-unlock \
--http \
@@ -119,8 +130,9 @@ else
--metrics \
--metrics.addr="0.0.0.0" \
--verbosity=${CERC_GETH_VERBOSITY:-3} \
--vmodule="${CERC_GETH_VMODULE:-statediff/*=5}" \
--miner.etherbase="${ETHERBASE}" ${STATEDIFF_OPTS} \
--log.vmodule="${CERC_GETH_VMODULE:-statediff/*=5}" \
--miner.etherbase="${ETHERBASE}" \
${STATEDIFF_OPTS} \
&
geth_pid=$!
@@ -1,4 +1,4 @@
FROM sigp/lcli:v4.1.0 AS lcli
FROM cerc/lighthouse-cli:local AS lcli
FROM skylenet/ethereum-genesis-generator@sha256:210353ce7c898686bc5092f16c61220a76d357f51eff9c451e9ad1b9ad03d4d3 AS ethgen
FROM cerc/fixturenet-eth-geth:local AS fnetgeth
@@ -27,12 +27,14 @@ lcli \
--deposit-contract-address $ETH1_DEPOSIT_CONTRACT_ADDRESS \
--testnet-dir $TESTNET_DIR \
--min-genesis-active-validator-count $GENESIS_VALIDATOR_COUNT \
--validator-count $VALIDATOR_COUNT \
--min-genesis-time $GENESIS_TIME \
--genesis-delay $GENESIS_DELAY \
--genesis-fork-version $GENESIS_FORK_VERSION \
--altair-fork-epoch $ALTAIR_FORK_EPOCH \
--merge-fork-epoch $MERGE_FORK_EPOCH \
--bellatrix-fork-epoch $MERGE_FORK_EPOCH \
--eth1-id $ETH1_CHAIN_ID \
--eth1-block-hash $ETH1_BLOCK_HASH \
--eth1-follow-distance 1 \
--seconds-per-slot $SECONDS_PER_SLOT \
--seconds-per-eth1-block $SECONDS_PER_ETH1_BLOCK \
@@ -15,9 +15,6 @@ GENESIS_VALIDATOR_COUNT=${GENESIS_VALIDATOR_COUNT:-80}
# Number of beacon_node instances that you intend to run
BN_COUNT=${BN_COUNT:-2}
# Number of validator clients
VC_COUNT=${VC_COUNT:-$BN_COUNT}
# Number of seconds to delay to start genesis block.
# If started by a script this can be 0, if starting by hand
# use something like 180.
@@ -45,7 +42,9 @@ VC_ARGS=${VC_ARGS:-""}
EXECUTION_ENDPOINT=${EXECUTION_ENDPOINT:-http://localhost:8551}
ETH1_GENESIS_JSON=${ETH1_GENESIS_JSON:-"../build/el/geth.json"}
ETH1_GENESIS_BLOCK_JSON=${ETH1_GENESIS_BLOCK_JSON:-"../build/el/genesis_block.json"}
ETH1_CONFIG_YAML=${ETH1_CONFIG_YAML:-"../el/el-config.yaml"}
ETH1_BLOCK_HASH=${ETH1_BLOCK_HASH:-`cat $ETH1_GENESIS_BLOCK_JSON | jq -r '.result.hash' | cut -d'x' -f2`}
ETH1_CHAIN_ID=${ETH1_CHAIN_ID:-`cat $ETH1_GENESIS_JSON | jq -r '.config.chainId'`}
ETH1_TTD=${ETH1_TTD:-`cat $ETH1_GENESIS_JSON | jq -r '.config.terminalTotalDifficulty'`}
@@ -4,7 +4,14 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
fi
MIN_BLOCK_NUM=${1:-${MIN_BLOCK_NUM:-3}}
STATUSES=("geth to generate DAG" "beacon phase0" "beacon altair" "beacon bellatrix pre-merge" "beacon bellatrix merge" "block number $MIN_BLOCK_NUM")
STATUSES=(
"geth to generate DAG"
"beacon phase0"
"beacon altair"
"beacon bellatrix pre-merge"
"beacon bellatrix merge"
"block number $MIN_BLOCK_NUM"
)
STATUS=0
LIGHTHOUSE_BASE_URL=${LIGHTHOUSE_BASE_URL}
@@ -36,7 +43,6 @@ MARKER="."
function inc_status() {
echo " done"
MARKEr="."
STATUS=$((STATUS + 1))
if [ $STATUS -lt ${#STATUSES[@]} ]; then
echo -n "Waiting for ${STATUSES[$STATUS]}..."
@@ -55,7 +61,7 @@ while [ $STATUS -lt ${#STATUSES[@]} ]; do
inc_status
fi
;;
1)
1)
result=`wget --no-check-certificate --quiet -O - "$LIGHTHOUSE_BASE_URL/eth/v2/beacon/blocks/head" | jq -r '.data.message.slot'`
if [ ! -z "$result" ] && [ $result -gt 0 ]; then
inc_status
@@ -0,0 +1,34 @@
FROM cerc/lighthouse-cli:local AS lcli
FROM skylenet/ethereum-genesis-generator@sha256:210353ce7c898686bc5092f16c61220a76d357f51eff9c451e9ad1b9ad03d4d3 AS ethgen
FROM cerc/fixturenet-plugeth-plugeth:local AS fnetgeth
FROM cerc/lighthouse:local
# cerc/lighthouse is based on Ubuntu
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install-recommends \
libssl-dev ca-certificates \
curl socat iproute2 telnet wget jq \
build-essential python3 python3-dev python3-pip gettext-base \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY genesis /opt/testnet
COPY run-cl.sh /opt/testnet/run.sh
COPY --from=lcli /usr/local/bin/lcli /usr/local/bin/lcli
COPY --from=ethgen /usr/local/bin/eth2-testnet-genesis /usr/local/bin/eth2-testnet-genesis
COPY --from=ethgen /usr/local/bin/eth2-val-tools /usr/local/bin/eth2-val-tools
COPY --from=ethgen /apps /apps
COPY --from=fnetgeth /opt/testnet/el /opt/testnet/el
COPY --from=fnetgeth /opt/testnet/build/el /opt/testnet/build/el
RUN cd /opt/testnet && make genesis-cl
# Work around some bugs in lcli where the default path is always used.
RUN mkdir -p /root/.lighthouse && cd /root/.lighthouse && ln -s /opt/testnet/build/cl/testnet
RUN mkdir -p /scripts
COPY scripts/status-internal.sh /scripts
COPY scripts/status.sh /scripts
ENTRYPOINT ["/opt/testnet/run.sh"]
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
# Build cerc/fixturenet-plugeth-lighthouse
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
if [ ! -d "${SCRIPT_DIR}/genesis" ]; then
cp -frp ${SCRIPT_DIR}/../cerc-fixturenet-eth-lighthouse/genesis ${SCRIPT_DIR}/genesis
fi
if [ ! -e "${SCRIPT_DIR}/run-cl.sh" ]; then
cp -fp ${SCRIPT_DIR}/../cerc-fixturenet-eth-lighthouse/run-cl.sh ${SCRIPT_DIR}/
fi
if [ ! -d "${SCRIPT_DIR}/scripts" ]; then
cp -frp ${SCRIPT_DIR}/../cerc-fixturenet-eth-lighthouse/scripts ${SCRIPT_DIR}/
fi
docker build -t cerc/fixturenet-plugeth-lighthouse:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} $SCRIPT_DIR
@@ -0,0 +1,42 @@
FROM skylenet/ethereum-genesis-generator@sha256:210353ce7c898686bc5092f16c61220a76d357f51eff9c451e9ad1b9ad03d4d3 AS ethgen
FROM golang:1.19-alpine as delve
RUN go install github.com/go-delve/delve/cmd/dlv@latest
FROM cerc/plugeth:local as geth
FROM cerc/plugeth-statediff:local as statediff
FROM alpine:3.17
RUN apk add --no-cache python3 python3-dev py3-pip curl wget jq build-base gettext libintl openssl bash bind-tools postgresql-client
COPY --from=delve /go/bin/dlv /usr/local/bin/
COPY --from=ethgen /usr/local/bin/eth2-testnet-genesis /usr/local/bin/
COPY --from=ethgen /usr/local/bin/eth2-val-tools /usr/local/bin/
COPY --from=ethgen /apps /apps
RUN cd /apps/el-gen && pip3 install -r requirements.txt
COPY genesis /opt/testnet
COPY run-el.sh /opt/testnet/run.sh
RUN cd /opt/testnet && make genesis-el
COPY --from=geth /usr/local/bin/geth /usr/local/bin/
RUN mkdir -p /usr/local/lib/plugeth/
COPY --from=statediff /usr/local/lib/statediff.so /usr/local/lib/plugeth/
# Snag the genesis block info.
RUN geth --datadir ~/ethdata init /opt/testnet/build/el/geth.json && rm -f ~/ethdata/geth/nodekey
RUN cp -rp ~/ethdata ~/tmpeth && \
geth --datadir ~/tmpeth init /opt/testnet/build/el/geth.json && \
geth --datadir ~/tmpeth --http & \
sleep 5 && \
curl -q --location 'localhost:8545' \
--header 'Content-Type: application/json' \
--data '{ "jsonrpc": "2.0", "id": 14, "method": "eth_getBlockByNumber", "params": ["0x0", false] }' \
-o /opt/testnet/build/el/genesis_block.json && \
killall -9 geth && \
rm -rf ~/tmpeth
ENTRYPOINT ["/opt/testnet/run.sh"]
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Build cerc/fixturenet-plugeth-plugeth
set -x
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
if [ ! -d "${SCRIPT_DIR}/genesis" ]; then
cp -frp ${SCRIPT_DIR}/../cerc-fixturenet-eth-geth/genesis ${SCRIPT_DIR}/genesis
fi
if [ ! -e "${SCRIPT_DIR}/run-el.sh" ]; then
cp -fp ${SCRIPT_DIR}/../cerc-fixturenet-eth-geth/run-el.sh ${SCRIPT_DIR}/
fi
docker build -t cerc/fixturenet-plugeth-plugeth:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} $SCRIPT_DIR
@@ -16,7 +16,7 @@ db-waitforsync=bool Should the statediff service start once geth has synced to
rpc-port=port change RPC port (default: 8545)
rpc-addr=address change RPC address (default: 127.0.0.1)
chain-id=number change chain ID (default: 99)
extra-args=name extra args to pass to geth on startup
extra-args=name extra args to pass to geth on startup
period=seconds use a block time instead of instamine
accounts=number create multiple accounts (default: 1)
address=address eth address to add to genesis
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
# Build cerc/go-opera
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# Repo's dockerfile gives build error because it's hardcoded for go 1.17; go 1.19 is required
sed -i 's/FROM golang:1\.[0-9]*-alpine as builder/FROM golang:1.19-alpine as builder/' ${CERC_REPO_BASE_DIR}/go-opera/docker/Dockerfile.opera
docker build -f ${CERC_REPO_BASE_DIR}/go-opera/docker/Dockerfile.opera -t cerc/go-opera:local ${build_command_args} ${CERC_REPO_BASE_DIR}/go-opera
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build the lasso image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
docker build -t cerc/lasso:local -f ${CERC_REPO_BASE_DIR}/lasso/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/lasso
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
# Build cerc/lighthouse-cli
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
project_dir=${CERC_REPO_BASE_DIR}/lighthouse
docker build -t cerc/lighthouse-cli:local --build-arg PORTABLE=true -f ${project_dir}/lcli/Dockerfile ${build_command_args} ${project_dir}
@@ -1,4 +1,5 @@
FROM sigp/lighthouse:v4.1.0-modern
ARG TAG_SUFFIX="-modern"
FROM sigp/lighthouse:v4.1.0${TAG_SUFFIX}
RUN apt-get update; apt-get install bash netcat curl less jq -y;
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# Build cerc/go-opera
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
docker build -t cerc/reth:local ${build_command_args} ${CERC_REPO_BASE_DIR}/reth
@@ -4,7 +4,7 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Test if the container's filesystem is old (run previously) or new
EXISTSFILENAME=/var/exists
EXISTSFILENAME=/data/exists
echo "Test container starting"
if [[ -f "$EXISTSFILENAME" ]];
then
+3
View File
@@ -41,3 +41,6 @@ cerc/watcher-azimuth
cerc/ipld-eth-state-snapshot
cerc/watcher-gelato
cerc/lotus
cerc/go-opera
cerc/lasso
cerc/reth
+3 -1
View File
@@ -5,7 +5,6 @@ go-ethereum-foundry
ipld-eth-beacon-db
ipld-eth-beacon-indexer
ipld-eth-server
lighthouse
laconicd
fixturenet-laconicd
fixturenet-eth
@@ -28,3 +27,6 @@ fixturenet-pocket
watcher-azimuth
watcher-gelato
fixturenet-lotus
mainnet-go-opera
lasso
reth
+3
View File
@@ -34,3 +34,6 @@ github.com/cerc-io/ipld-eth-state-snapshot
github.com/cerc-io/gelato-watcher-ts
github.com/filecoin-project/lotus
git.vdb.to/cerc-io/test-project
github.com/Fantom-foundation/go-opera
github.com/cerc-io/lasso
github.com/paradigmxyz/reth
+4 -4
View File
@@ -2,10 +2,10 @@ version: "1.0"
name: chain-chunker
decription: "Stack to build containers for chain-chunker"
repos:
- github.com/cerc-io/ipld-eth-state-snapshot
- github.com/cerc-io/eth-statediff-service
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/ipld-eth-server
- github.com/cerc-io/ipld-eth-state-snapshot@v5
- github.com/cerc-io/eth-statediff-service@v5
- github.com/cerc-io/ipld-eth-db@v5
- github.com/cerc-io/ipld-eth-server@v5
containers:
- cerc/ipld-eth-state-snapshot
- cerc/eth-statediff-service
@@ -6,9 +6,11 @@ repos:
- github.com/cerc-io/tx-spammer
- github.com/cerc-io/ipld-eth-server
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/ipld-eth-server
+3 -1
View File
@@ -4,10 +4,12 @@ decription: "Ethereum Fixturenet w/ tx-spammer"
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/tx-spammer
- dboreham/foundry
- github.com/dboreham/foundry
- github.com/cerc-io/lighthouse
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/tx-spammer
+1 -1
View File
@@ -1,6 +1,6 @@
# fixturenet-eth
Instructions for deploying a local a geth + lighthouse blockchain "fixturenet" for development and testing purposes using laconic-stack-orchestrator (the installation of which is covered [here](https://github.com/cerc-io/stack-orchestrator#user-mode)):
Instructions for deploying a local a geth + lighthouse blockchain "fixturenet" for development and testing purposes using laconic-stack-orchestrator (the installation of which is covered [here](https://github.com/cerc-io/stack-orchestrator)):
## Clone required repositories
+2
View File
@@ -3,10 +3,12 @@ name: fixturenet-eth
decription: "Ethereum Fixturenet"
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
- github.com/dboreham/foundry
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/foundry
+1 -11
View File
@@ -14,14 +14,6 @@ laconic-so --stack fixturenet-optimism setup-repositories
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Checkout to the required versions and branches in repos:
```bash
# Optimism
cd ~/cerc/optimism
git checkout v1.0.4
```
Build the container images:
```bash
@@ -50,12 +42,11 @@ Deploy the stack:
laconic-so --stack fixturenet-optimism deploy up
```
If you get the error `service "fixturenet-optimism-contracts" didn't complete successfully: exit 1` with ~25 lines of Traceback, wait 15-20 mins then re-run the command.
The `fixturenet-optimism-contracts` service takes a while to complete running as it:
1. waits for the 'Merge' to happen on L1
2. waits for a finalized block to exist on L1 (so that it can be taken as a starting block for roll ups)
3. deploys the L1 contracts
It may restart a few times after running into errors.
To list and monitor the running containers:
@@ -123,6 +114,5 @@ docker volume rm $(docker volume ls -q --filter "name=.*l1_deployment|.*l2_accou
## Known Issues
* `fixturenet-eth` currently starts fresh on a restart
* Resource requirements (memory + time) for building the `cerc/foundry` image are on the higher side
* `cerc/optimism-contracts` image is currently based on `cerc/foundry` (Optimism requires foundry installation)
@@ -14,14 +14,6 @@ laconic-so --stack fixturenet-optimism setup-repositories --exclude github.com/c
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Checkout to the required versions and branches in repos:
```bash
# Optimism
cd ~/cerc/optimism
git checkout v1.0.4
```
Build the container images:
```bash
@@ -3,12 +3,14 @@ name: fixturenet-optimism
decription: "Optimism Fixturenet"
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
- github.com/dboreham/foundry
- github.com/ethereum-optimism/optimism
- github.com/ethereum-optimism/op-geth
- github.com/ethereum-optimism/optimism@v1.0.4
- github.com/ethereum-optimism/op-geth@v1.101105.2
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/foundry
@@ -0,0 +1,19 @@
# fixturenet-plugeth-tx
A variation of `fixturenet-eth` that uses `plugeth` instead of `go-ethereum`.
See `stacks/fixturenet-eth/README.md` for more information.
## Containers
* cerc/lighthouse
* cerc/fixturenet-eth-plugeth
* cerc/fixturenet-eth-lighthouse
* cerc/tx-spammer
## Deploy the stack
```
$ laconic-so --stack fixturenet-plugeth-tx setup-repositories
$ laconic-so --stack fixturenet-plugeth-tx build-containers
$ laconic-so --stack fixturenet-plugeth-tx deploy up
```
@@ -0,0 +1,28 @@
version: "1.2"
name: fixturenet-plugeth-tx
decription: "plugeth Ethereum Fixturenet w/ tx-spammer"
repos:
- github.com/cerc-io/tx-spammer
- github.com/dboreham/foundry
- github.com/cerc-io/lighthouse
- github.com/cerc-io/ipld-eth-db@v5
- github.com/cerc-io/ipld-eth-server@v5
- git.vdb.to/cerc-io/plugeth@statediff-wip
- git.vdb.to/cerc-io/plugeth-statediff@dev-local-build
containers:
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/plugeth-statediff
- cerc/plugeth
- cerc/fixturenet-plugeth-plugeth
- cerc/fixturenet-plugeth-lighthouse
- cerc/tx-spammer
- cerc/foundry
- cerc/ipld-eth-db
- cerc/ipld-eth-server
pods:
- ipld-eth-db
- ipld-eth-server
- fixturenet-plugeth
- foundry
- tx-spammer
+18 -17
View File
@@ -1,41 +1,41 @@
# Pocket Fixturenet
Instructions for deploying a local single-node Pocket chain alongside a geth + lighthouse blockchain "fixturenet" for development and testing purposes using laconic-stack-orchestrator.
Instructions for deploying a local single-node Pocket chain alongside a geth + lighthouse blockchain "fixturenet" for development and testing purposes using Stack Orchestrator.
## 1. Build Laconic Stack Orchestrator
Build this fork of Laconic Stack Orchestrator which includes the fixturenet-pocket stack:
```
$ scripts/build_shiv_package.sh
$ cd package
$ mv laconic-so-{version} /usr/local/bin/laconic-so # Or move laconic-so to ~/bin or your favorite on-path directory
```
## 2. Clone required repositories
## 1. Clone required repositories
```
$ laconic-so --stack fixturenet-pocket setup-repositories
```
## 3. Build the stack's containers
## 2. Build the stack's containers
```
$ laconic-so --stack fixturenet-pocket build-containers
```
## 4. Deploy the stack
## 3. Deploy the stack
```
$ laconic-so --stack fixturenet-pocket deploy up
```
It may take up to 10 minutes for the Eth Fixturenet to fully come online and start producing blocks.
## 5. Check status
## 4. Check status
**Eth Fixturenet:**
```
$ laconic-so --stack fixturenet-pocket deploy exec fixturenet-eth-bootnode-lighthouse /scripts/status-internal.sh
```
Response:
```
Waiting for geth to generate DAG.... done
Waiting for beacon phase0.... done
Waiting for beacon altair.... done
Waiting for beacon bellatrix pre-merge.... done
Waiting for beacon bellatrix merge.... done
```
**Pocket node:**
```
$ laconic-so --stack fixturenet-pocket deploy exec pocket "pocket query height"
```
Response:
```
2023/04/20 08:07:46 Initializing Pocket Datadir
2023/04/20 08:07:46 datadir = /home/app/.pocket
http://localhost:8081/v1/query/height
@@ -43,17 +43,18 @@ http://localhost:8081/v1/query/height
"height": 4
}
```
or
or see the full logs:
```
$ laconic-so --stack fixturenet-pocket deploy logs pocket
```
## 6. Send a relay request to Pocket node
## 5. Send a relay request to Pocket node
The Pocket node serves relay requests at `http://localhost:8081/v1/client/sim`
**Example request:**
Example request:
```
$ curl -X POST --data '{"relay_network_id":"0021","payload":{"data":"{\"jsonrpc\": \"2.0\",\"id\": 1,\"method\": \"eth_blockNumber\",\"params\": []}","method":"POST","path":"","headers":{}}}' http://localhost:8081/v1/client/sim
```
**Response:**
Response:
```
"{\"jsonrpc\":\"2.0\",\"id\":1,\"result\":\"0x6fe\"}\n"
```
@@ -3,11 +3,13 @@ name: fixturenet-pocket
description: "A single node pocket chain that can serve relays from the geth-1 node in eth-fixturenet"
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
- github.com/pokt-network/pocket-core
- github.com/pokt-network/pocket-core-deployments # contains the dockerfile
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/pocket
+7
View File
@@ -0,0 +1,7 @@
# lasso
```
laconic-so --stack lasso setup-repositories
laconic-so --stack lasso build-containers
laconic-so --stack lasso deploy up
```
+8
View File
@@ -0,0 +1,8 @@
version: "0.1"
name: lasso
repos:
- github.com/cerc-io/lasso
containers:
- cerc/lasso
pods:
- lasso
+104
View File
@@ -0,0 +1,104 @@
# mainnet-eth
Deploys a "head-tracking" mainnet Ethereum stack comprising a [go-ethereum](https://github.com/cerc-io/go-ethereum) execution layer node and a [lighthouse](https://github.com/sigp/lighthouse) consensus layer node.
## Clone required repositories
```
$ laconic-so --stack mainnet-eth setup-repositories
```
## Build containers
```
$ laconic-so --stack mainnet-eth build-containers
```
## Create a deployment
```
$ laconic-so --stack mainnet-eth deploy init --output mainnet-eth-spec.yml
$ laconic-so deploy create --spec-file mainnet-eth-spec.yml --deployment-dir mainnet-eth-deployment
```
## Start the stack
```
$ laconic-so deployment --dir mainnet-eth-deployment start
```
Display stack status:
```
$ laconic-so deployment --dir mainnet-eth-deployment ps
Running containers:
id: f39608eca04d72d6b0f1f3acefc5ebb52908da06e221d20c7138f7e3dff5e423, name: laconic-ef641b4d13eb61ed561b19be67063241-foundry-1, ports:
id: 4052b1eddd886ae0d6b41f9ff22e68a70f267b2bfde10f4b7b79b5bd1eeddcac, name: laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-geth-1-1, ports: 30303/tcp, 30303/udp, 0.0.0.0:49184->40000/tcp, 0.0.0.0:49185->6060/tcp, 0.0.0.0:49186->8545/tcp, 8546/tcp
id: ac331232e597944b621b3b8942ace5dafb14524302cab338ff946c7f6e5a1d52, name: laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1, ports: 0.0.0.0:49187->8001/tcp
```
See stack logs:
```
$ laconic-so deployment --dir mainnet-eth-deployment logs
time="2023-07-25T09:46:29-06:00" level=warning msg="The \"CERC_SCRIPT_DEBUG\" variable is not set. Defaulting to a blank string."
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.362 INFO Logging to file path: "/var/lighthouse-data-dir/beacon/logs/beacon.log"
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.365 INFO Lighthouse started version: Lighthouse/v4.1.0-693886b
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.365 INFO Configured for network name: mainnet
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.366 INFO Data directory initialised datadir: /var/lighthouse-data-dir
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.366 INFO Deposit contract address: 0x00000000219ab540356cbb839cbe05303d7705fa, deploy_block: 11184524
laconic-ef641b4d13eb61ed561b19be67063241-mainnet-eth-lighthouse-1-1 | Jul 25 15:45:13.424 INFO Starting checkpoint sync remote_url: https://beaconstate.ethstaker.cc/, service: beacon
```
## Monitoring stack sync progress
Both go-ethereum and lighthouse will engage in an initial chain sync phase that will last up to several hours depending on hardware performance and network capacity.
Syncing can be monitored by looking for these log messages:
```
Jul 24 12:34:17.001 INFO Downloading historical blocks est_time: 5 days 11 hrs, speed: 14.67 slots/sec, distance: 6932481 slots (137 weeks 3 days), service: slot_notifier
INFO [07-24|12:14:52.493] Syncing beacon headers downloaded=145,920 left=17,617,968 eta=1h23m32.815s
INFO [07-24|12:33:15.238] Syncing: chain download in progress synced=1.86% chain=148.94MiB headers=368,640@95.03MiB bodies=330,081@40.56MiB receipts=330,081@13.35MiB eta=37m54.505s
INFO [07-24|12:35:13.028] Syncing: state download in progress synced=1.32% state=4.64GiB accounts=2,850,314@677.57MiB slots=18,663,070@3.87GiB codes=26662@111.14MiB eta=3h18m0.699s
```
Once synced up these log messages will be observed:
```
INFO Synced slot: 6952515, block: 0x5bcb…f6d9, epoch: 217266, finalized_epoch: 217264, finalized_root: 0x6342…2c5c, exec_hash: 0x8d8c…2443 (verified), peers: 31, service: slot_notifier
INFO [07-25|03:04:48.941] Imported new potential chain segment number=17,767,316 hash=84f6e7..bc2cb0 blocks=1 txs=137 mgas=16.123 elapsed=57.087ms mgasps=282.434 dirty=461.46MiB
INFO [07-25|03:04:49.042] Chain head was updated number=17,767,316 hash=84f6e7..bc2cb0 root=ca58b2..8258c1 elapsed=2.480111ms
```
## Clean up
Stop the stack:
```
$ laconic-so deployment --dir mainnet-eth-deployment stop
```
This leaves data volumes in place, allowing the stack to be subsequently re-started.
To permanently *delete* the stack's data volumes run:
```
$ laconic-so deployment --dir mainnet-eth-deployment stop --delete-data-volumes
```
After deleting the volumes, any subsequent re-start will begin chain sync from cold.
## Data volumes
Container data volumes are bind-mounted to specified paths in the host filesystem.
The default setup (generated by `laconic-so deploy init`) places the volumes in the `./data` subdirectory of the deployment directory:
```
$ cat mainnet-eth-spec.yml
stack: mainnet-eth
volumes:
mainnet_eth_config_data: ./data/mainnet_eth_config_data
mainnet_eth_geth_1_data: ./data/mainnet_eth_geth_1_data
mainnet_eth_lighthouse_1_data: ./data/mainnet_eth_lighthouse_1_data
```
A synced-up stack will consume around 900GB of data volume space:
```
$ sudo du -h mainnet-eth-deployment/data/
150M mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data/beacon/freezer_db
25G mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data/beacon/chain_db
16K mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data/beacon/network
368M mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data/beacon/logs
26G mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data/beacon
26G mainnet-eth-deployment/data/mainnet_eth_lighthouse_1_data
8.0K mainnet-eth-deployment/data/mainnet_eth_config_data
4.0K mainnet-eth-deployment/data/mainnet_eth_geth_1_data/keystore
527G mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/chaindata/ancient/chain
527G mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/chaindata/ancient
859G mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/chaindata
4.8M mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/nodes
242M mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/ethash
669M mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth/triecache
860G mainnet-eth-deployment/data/mainnet_eth_geth_1_data/geth
860G mainnet-eth-deployment/data/mainnet_eth_geth_1_data
885G mainnet-eth-deployment/data/
```
@@ -0,0 +1,31 @@
# Copyright © 2023 Cerc
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from secrets import token_hex
def init(ctx):
return None
def setup(ctx):
return None
def create(ctx):
# Generate the JWT secret and save to its config file
secret = token_hex(32)
jwt_file_path = ctx.deployment_dir.joinpath("data", "mainnet_eth_config_data", "jwtsecret")
with open(jwt_file_path, 'w+') as jwt_file:
jwt_file.write(secret)
+15
View File
@@ -0,0 +1,15 @@
version: "1.1"
name: mainnet-eth
decription: "Ethereum Mainnet"
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
- github.com/dboreham/foundry
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/foundry
pods:
- mainnet-eth
- foundry
@@ -0,0 +1,88 @@
# Opera (Fantom)
Deploy a Fantom API node.
## Clone required repositories
```
$ laconic-so --stack mainnet-go-opera setup-repositories
```
## Build the fixturenet-eth containers
```
$ laconic-so --stack mainnet-go-opera build-containers
```
## Deploy the stack
```
$ laconic-so --stack mainnet-go-opera deploy up
```
## Check logs
```
$ laconic-so --stack mainnet-go-opera deploy logs
```
You'll see something like:
```
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | Connecting to download.fantom.network (65.108.45.88:443)
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | saving to 'mainnet-109331-no-history.g'
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | mainnet-109331-no-hi 100% |********************************| 16326 0:00:00 ETA
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | 'mainnet-109331-no-history.g' saved
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.034] Maximum peer count total=50
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.034] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.034] Genesis file is a known preset name="Mainnet-109331 without history"
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.052] Applying genesis state
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.052] - Reading epochs unit 0
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.054] - Reading blocks unit 0
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.530] Applied genesis state name=main id=250 genesis=0x4a53c5445584b3bfc20dbfb2ec18ae20037c716f3ba2d9e1da768a9deca17cb4
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.531] Regenerated local transaction journal transactions=0 accounts=0
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.532] Starting peer-to-peer node instance=go-opera/v1.1.2-rc.5-50cd051d-1677276206/linux-amd64/go1.19.10
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.536] New local node record seq=1 id=5e40f984908317cd ip=127.0.0.1 udp=5050 tcp=5050
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.537] Started P2P networking self=enode://3ffb15988ca5a79b63dbe48be89d9d8b48dc4845d318fe08231a0ab49d3b23476e2561044311dc257405f882f7c52ff7b128c8bd1b6d85cf7205a6fed6555443@127.0.0.1:5050
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.537] IPC endpoint opened url=/root/.opera/opera.ipc
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] HTTP server started endpoint=[::]:18545 prefix= cors=* vhosts=localhost
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] WebSocket enabled url=ws://[::]:18546
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] Rebuilding state snapshot
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] EVM snapshot module=gossip-store at=000000..000000 generating=true
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] Resuming state snapshot generation accounts=0 slots=0 storage=0.00B elapsed="189.74µs"
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:33.538] Generated state snapshot accounts=0 slots=0 storage=0.00B elapsed="265.061µs"
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:34.788] New LLR summary last_epoch=0 last_block=37676611 new_evs=0 new_ers=0 new_bvs=64 new_brs=0 age=none
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:35.040] New local node record seq=2 id=5e40f984908317cd ip=186.233.184.56 udp=5050 tcp=5050
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:42.788] New LLR summary last_epoch=114604 last_block=37753891 new_evs=24581 new_ers=5272 new_bvs=233257 new_brs=780 age=1y1mo5d
laconic-f028f14527b95e2eb97f0c0229d00939-go-opera-1 | INFO [06-20|13:32:50.827] New LLR summary last_epoch=115574 last_block=38118749 new_evs=4907 new_ers=971 new_bvs=1098760 new_brs=3768 age=1y1mo2d
```
Consecutive lines of "New LLR summary" shows that your node is sync'ing.
## Use the opera admin console
```
$ docker exec -it $(docker ps -q --filter "name=go-opera") /bin/sh
```
then:
```
$ ./opera attach
```
and check the node info:
```
> admin.nodeInfo
```
Run `exit` twice to return to your terminal.
## Clean up
Stop all services running in the background:
```bash
$ laconic-so --stack mainnet-go-opera deploy down
```
@@ -0,0 +1,9 @@
version: "1.1"
name: mainnet-opera
decription: "Fantom mainnet node"
repos:
- github.com/Fantom-foundation/go-opera@release/1.1.2-rc.5
containers:
- cerc/go-opera
pods:
- mainnet-go-opera
@@ -0,0 +1,2 @@
# Laconic Mainnet Deployment (experimental)
@@ -0,0 +1,62 @@
# Copyright © 2022, 2023 Cerc
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from dataclasses import dataclass
from app.util import get_yaml
from app.stack_state import State
default_spec_file_content = """config:
node_moniker: my-node-name
chain_id: my-chain-id
"""
init_help_text = """Add helpful text here on setting config variables.
"""
@dataclass
class VolumeMapping:
host_path: str
container_path: str
# In order to make this, we need the ability to run the stack
# In theory we can make this same way as we would run deploy up
def run_container_command(ctx, ontainer, command, mounts):
deploy_context = ctx.obj
pass
def setup(ctx):
node_moniker = "dbdb-node"
chain_id = "laconic_81337-1"
mounts = [
VolumeMapping("./path", "~/.laconicd")
]
output, status = run_container_command(ctx, "laconicd", f"laconicd init {node_moniker} --chain-id {chain_id}", mounts)
def init(command_context):
print(init_help_text)
yaml = get_yaml()
return yaml.load(default_spec_file_content)
def get_state(command_context):
print("Here we get state")
return State.CONFIGURED
def change_state(command_context):
pass
+28
View File
@@ -0,0 +1,28 @@
version: "1.0"
name: mainnet-laconic
description: "Mainnet laconic node"
repos:
- cerc-io/laconicd
- lirewine/debug
- lirewine/crypto
- lirewine/gem
- lirewine/sdk
- cerc-io/laconic-sdk
- cerc-io/laconic-registry-cli
- cerc-io/laconic-console
npms:
- laconic-sdk
- laconic-registry-cli
- debug
- crypto
- sdk
- gem
- laconic-console
containers:
- cerc/laconicd
- cerc/laconic-registry-cli
- cerc/laconic-console-host
pods:
- mainnet-laconicd
- fixturenet-laconic-console
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Dump environment variables for debugging
echo "Environment variables:"
env
# Test laconic stack
echo "Running laconic stack test"
# Bit of a hack, test the most recent package
TEST_TARGET_SO=$( ls -t1 ./package/laconic-so* | head -1 )
# Set a non-default repo dir
export CERC_REPO_BASE_DIR=~/stack-orchestrator-test/repo-base-dir
echo "Testing this package: $TEST_TARGET_SO"
echo "Test version command"
reported_version_string=$( $TEST_TARGET_SO version )
echo "Version reported is: ${reported_version_string}"
echo "Cloning repositories into: $CERC_REPO_BASE_DIR"
rm -rf $CERC_REPO_BASE_DIR
mkdir -p $CERC_REPO_BASE_DIR
# Test bringing the test container up and down
# with and without volume removal
$TEST_TARGET_SO --stack test setup-repositories
$TEST_TARGET_SO --stack test build-containers
$TEST_TARGET_SO --stack test deploy up
$TEST_TARGET_SO --stack test deploy down
# The next time we bring the container up the volume will be old (from the previous run above)
$TEST_TARGET_SO --stack test deploy up
log_output_1=$( $TEST_TARGET_SO --stack test deploy logs )
if [[ "$log_output_1" == *"Filesystem is old"* ]]; then
echo "Retain volumes test: passed"
else
echo "Retain volumes test: FAILED"
exit 1
fi
$TEST_TARGET_SO --stack test deploy down --delete-volumes
# Now when we bring the container up the volume will be new again
$TEST_TARGET_SO --stack test deploy up
log_output_2=$( $TEST_TARGET_SO --stack test deploy logs )
if [[ "$log_output_2" == *"Filesystem is fresh"* ]]; then
echo "Delete volumes test: passed"
else
echo "Delete volumes test: FAILED"
exit 1
fi
$TEST_TARGET_SO --stack test deploy down --delete-volumes
echo "Test passed"
+15 -30
View File
@@ -18,26 +18,6 @@ laconic-so --stack mobymask-v2 setup-repositories
NOTE: If repositories already exist and are checked out to different versions, `setup-repositories` command will throw an error.
For getting around this, the repositories mentioned below can be removed and then run the command.
Checkout to the required versions and branches in repos
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
git checkout v0.1.2
# Optimism
cd ~/cerc/optimism
git checkout v1.0.4
```
Build the container images:
```bash
@@ -51,17 +31,21 @@ Deploy the stack:
* Deploy the containers:
```bash
laconic-so --stack mobymask-v2 deploy-system up
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 up
```
* List and check the health status of all the containers using `docker ps` and wait for them to be `healthy`
NOTE: The `fixturenet-optimism-contracts` service takes a while to run to completion and it may restart a few times after running into errors.
NOTE: The `mobymask-app` container might not start; if the app is not running at http://localhost:3002, restart the container using it's id:
* To list down and monitor the running containers:
```bash
docker ps -a | grep "mobymask-app"
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 ps
docker restart <CONTAINER_ID>
# With status
docker ps -a
# Check logs for a container
docker logs -f <CONTAINER_ID>
```
## Tests
@@ -88,11 +72,12 @@ docker ps | grep -E 'mobymask-app|peer-test-app'
### mobymask-app
The mobymask-app should be running at http://localhost:3002
* The mobymask-app should be running at http://localhost:3002
* The lxdao-mobymask-app should be running at http://localhost:3004
### peer-test-app
The peer-test-app should be running at http://localhost:3003
* The peer-test-app should be running at http://localhost:3003
## Details
@@ -111,15 +96,15 @@ Follow the [demo](./demo.md) to try out the MobyMask app with L2 chain
Stop all the services running in background run:
```bash
laconic-so --stack mobymask-v2 deploy-system down 30
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 down 30
```
Clear volumes created by this stack:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=.*mobymask_watcher_db_data|.*peers_ids|.*mobymask_deployment|.*l1_deployment|.*l2_accounts|.*l2_config|.*l2_geth_data"
docker volume ls -q --filter "name=mobymask_v2"
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=.*mobymask_watcher_db_data|.*peers_ids|.*mobymask_deployment|.*l1_deployment|.*l2_accounts|.*l2_config|.*l2_geth_data")
docker volume rm $(docker volume ls -q --filter "name=mobymask_v2")
```
+10 -11
View File
@@ -9,23 +9,23 @@
The invite link is seen at the end of the logs. Example log:
```bash
laconic-bfb01caf98b1b8f7c8db4d33f11b905a-mobymask-1 | http://127.0.0.1:3002/#/members?invitation=%7B%22v%22%3A1%2C%22signedDelegations%22%3A%5B%7B%22signature%22%3A%220x7559bd412f02677d60820e38243acf61547f79339395a34f7d4e1630e645aeb30535fc219f79b6fbd3af0ce3bd05132ad46d2b274a9fbc4c36bc71edd09850891b%22%2C%22delegation%22%3A%7B%22delegate%22%3A%220xc0838c92B2b71756E0eAD5B3C1e1F186baeEAAac%22%2C%22authority%22%3A%220x0000000000000000000000000000000000000000000000000000000000000000%22%2C%22caveats%22%3A%5B%7B%22enforcer%22%3A%220x558024C7d593B840E1BfD83E9B287a5CDad4db15%22%2C%22terms%22%3A%220x0000000000000000000000000000000000000000000000000000000000000000%22%7D%5D%7D%7D%5D%2C%22key%22%3A%220x98da9805821f1802196443e578fd32af567bababa0a249c07c82df01ecaa7d8d%22%7D
http://127.0.0.1:3004/#/members?invitation=%7B%22v%22%3A1%2C%22signedDelegations%22%3A%5B%7B%22signature%22%3A%220x7559bd412f02677d60820e38243acf61547f79339395a34f7d4e1630e645aeb30535fc219f79b6fbd3af0ce3bd05132ad46d2b274a9fbc4c36bc71edd09850891b%22%2C%22delegation%22%3A%7B%22delegate%22%3A%220xc0838c92B2b71756E0eAD5B3C1e1F186baeEAAac%22%2C%22authority%22%3A%220x0000000000000000000000000000000000000000000000000000000000000000%22%2C%22caveats%22%3A%5B%7B%22enforcer%22%3A%220x558024C7d593B840E1BfD83E9B287a5CDad4db15%22%2C%22terms%22%3A%220x0000000000000000000000000000000000000000000000000000000000000000%22%7D%5D%7D%7D%5D%2C%22key%22%3A%220x98da9805821f1802196443e578fd32af567bababa0a249c07c82df01ecaa7d8d%22%7D
```
* Open the invite link in a browser to use the mobymask-app.
NOTE: Before opening the invite link, clear the browser cache (local storage) for http://127.0.0.1:3002 to remove old invitations
NOTE: Before opening the invite link, clear the browser cache (local storage) for http://127.0.0.1:3004 to remove old invitations
* In the debug panel, check if it is connected to the p2p network (it should be connected to at least one other peer for pubsub to work).
* Create an invite link in the app by clicking on `Create new invite link` button.
* Create an invite link in the app by clicking on `Create new invite link` button in the `My invitees` section.
* Switch to the `MESSAGES` tab in debug panel for viewing incoming messages later.
* Open the invite link in a new browser with different profile (to simulate remote browser)
* Check that it is connected to any other peer in the network.
* Check that it is connected to a peer in the network.
* In `Report a phishing attempt` section, report multiple phishers using the `Submit` button. Click on the `Submit batch to p2p network` button. This broadcasts signed invocations to the connected peers.
* In the `Pending reports` section, enter multiple phisher records and click on the `Submit batch to p2p network` button. This broadcasts signed invocations to the connected peers.
* In the `MESSAGES` tab of other browsers, a message can be seen with the signed invocations.
@@ -66,7 +66,7 @@
* Get the deployed contract address:
```bash
docker exec -it $(docker ps -aq --filter name="mobymask-app") cat /config/config.yml
docker exec -it $(docker ps -aq --filter name="lxdao-mobymask-app") cat /config/config.yml
```
The value of `address` field is the deployed contract address
@@ -91,15 +91,14 @@
* Watcher internally is using L2 chain `eth_getStorageAt` method.
* Check the phisher name in mobymask app in `Check Phisher Status` section.
* Watcher GQL API is used for checking phisher.
* Watcher GQL API is used for checking phisher.
* Manage the invitations by clicking on the `Outstanding Invitations in p2p network`.
* Revoke the created invitation by clicking on `Revoke (p2p network)`
* Manage invitations in the `Outstanding invitations (p2p network)` tab in `My Invitations` section.
* Revoke the created invitation by clicking on the `Revoke` button.
* Revocation messages can be seen in the debug panel `MESSAGES` tab of other browsers.
* Check the watcher peer logs. It should receive a message and log the transaction receipt for a `revoke` message.
* Also, check the watcher peer logs. It should receive a message and log the transaction receipt for a `revoke` message.
* Try reporting a phisher from the revoked invitee's browser.
+7 -23
View File
@@ -14,22 +14,6 @@ laconic-so --stack mobymask-v2 setup-repositories --include github.com/cerc-io/M
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Checkout to the required versions and branches in repos:
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
git checkout v0.1.2
```
Build the container images:
```bash
@@ -65,7 +49,7 @@ Create and update an env file to be used in the next step ([defaults](../../conf
# Base URI for mobymask-app
# (used for generating a root invite link after deploying the contract)
CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3002/#"
CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3004/#"
# (Optional) Domain to be used in the relay node's announce address
CERC_RELAY_ANNOUNCE_DOMAIN=
@@ -88,16 +72,16 @@ Create and update an env file to be used in the next step ([defaults](../../conf
### Deploy the stack
```bash
laconic-so --stack mobymask-v2 deploy --include watcher-mobymask-v2 --env-file <PATH_TO_ENV_FILE> up
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include watcher-mobymask-v2 --env-file <PATH_TO_ENV_FILE> up
```
To list down and monitor the running containers:
```bash
laconic-so --stack mobymask-v2 deploy --include watcher-mobymask-v2 ps
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include watcher-mobymask-v2 ps
# With status
docker ps
docker ps -a
# Check logs for a container
docker logs -f <CONTAINER_ID>
@@ -124,15 +108,15 @@ For deploying the web-app(s) separately after deploying the watcher, follow [web
Stop all services running in the background:
```bash
laconic-so --stack mobymask-v2 deploy --include watcher-mobymask-v2 down
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include watcher-mobymask-v2 down
```
Clear volumes created by this stack:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=.*mobymask_watcher_db_data|.*peers_ids|.*mobymask_deployment"
docker volume ls -q --filter "name=mobymask_v2"
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=.*mobymask_watcher_db_data|.*peers_ids|.*mobymask_deployment")
docker volume rm $(docker volume ls -q --filter "name=mobymask_v2")
```
+8 -5
View File
@@ -2,21 +2,24 @@ version: "1.0"
name: mobymask-v2
repos:
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/lighthouse
- github.com/dboreham/foundry
- github.com/ethereum-optimism/optimism
- github.com/ethereum-optimism/op-geth
- github.com/cerc-io/watcher-ts
- github.com/cerc-io/mobymask-v2-watcher-ts
- github.com/cerc-io/MobyMask
- github.com/ethereum-optimism/optimism@v1.0.4
- github.com/ethereum-optimism/op-geth@v1.101105.2
- github.com/cerc-io/watcher-ts@v0.2.43
- github.com/cerc-io/mobymask-v2-watcher-ts@v0.1.1
- github.com/cerc-io/MobyMask@v0.1.2
containers:
- cerc/go-ethereum
- cerc/lighthouse
- cerc/lighthouse-cli
- cerc/fixturenet-eth-geth
- cerc/fixturenet-eth-lighthouse
- cerc/foundry
- cerc/optimism-contracts
- cerc/optimism-l2geth
- cerc/optimism-op-batcher
- cerc/optimism-op-proposer
- cerc/optimism-op-node
- cerc/watcher-ts
- cerc/watcher-mobymask-v2
@@ -30,22 +30,6 @@ Clone required repositories:
# 100%|##############################################################################################################################################| 1.41k/1.41k [00:18<00:00, 76.4B/s]
```
Checkout to the required versions and branches in repos:
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
git checkout v0.1.2
```
Build the container images:
```bash
@@ -126,8 +110,8 @@ To list down and monitor the running containers:
# Expected output:
# Running containers:
# id: 25cc3a1cbda27fcd9c2ad4c772bd753ccef1e178f901a70e6ff4191d4a8684e9, name: mobymask_v2-mobymask-watcher-db-1, ports: 0.0.0.0:15432->5432/tcp
# id: c9806f78680d68292ffe942222af2003aa3ed5d5c69d7121b573f5028444391d, name: mobymask_v2-mobymask-watcher-server-1, ports: 0.0.0.0:3001->3001/tcp, 0.0.0.0:9001->9001/tcp, 0.0.0.0:9090->9090/tcp
# id: 25cc3a1cbda27fcd9c2ad4c772bd753ccef1e178f901a70e6ff4191d4a8684e9, name: mobymask_v2-mobymask-watcher-db-1, ports: 127.0.0.1:15432->5432/tcp
# id: c9806f78680d68292ffe942222af2003aa3ed5d5c69d7121b573f5028444391d, name: mobymask_v2-mobymask-watcher-server-1, ports: 127.0.0.1:3001->3001/tcp, 127.0.0.1:9001->9001/tcp, 127.0.0.1:9090->9090/tcp
# id: 6b30a1d313a88fb86f8a3b37a1b1a3bc053f238664e4b2d196c3ec74e04faf13, name: mobymask_v2-peer-tests-1, ports:
@@ -138,8 +122,8 @@ To list down and monitor the running containers:
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# 6b30a1d313a8 cerc/watcher-ts:local "docker-entrypoint.s…" 5 minutes ago Up 4 minutes mobymask_v2-peer-tests-1
# c9806f78680d cerc/watcher-mobymask-v2:local "sh start-server.sh" 5 minutes ago Up 5 minutes (healthy) 0.0.0.0:3001->3001/tcp, 0.0.0.0:9001->9001/tcp, 0.0.0.0:9090->9090/tcp mobymask_v2-mobymask-watcher-server-1
# 25cc3a1cbda2 postgres:14-alpine "docker-entrypoint.s…" 5 minutes ago Up 5 minutes (healthy) 0.0.0.0:15432->5432/tcp mobymask_v2-mobymask-watcher-db-1
# c9806f78680d cerc/watcher-mobymask-v2:local "sh start-server.sh" 5 minutes ago Up 5 minutes (healthy) 127.0.0.1:3001->3001/tcp, 127.0.0.1:9001->9001/tcp, 127.0.0.1:9090->9090/tcp mobymask_v2-mobymask-watcher-server-1
# 25cc3a1cbda2 postgres:14-alpine "docker-entrypoint.s…" 5 minutes ago Up 5 minutes (healthy) 127.0.0.1:15432->5432/tcp mobymask_v2-mobymask-watcher-db-1
# Check logs for a container
@@ -78,7 +78,7 @@ To monitor the running container:
# Expected output:
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# f1369dbae1c9 cerc/mobymask-ui:local "docker-entrypoint.s…" 2 minutes ago Up 2 minutes (healthy) 0.0.0.0:3004->80/tcp mm_v2-lxdao-mobymask-app-1
# f1369dbae1c9 cerc/mobymask-ui:local "docker-entrypoint.s…" 2 minutes ago Up 2 minutes (healthy) 127.0.0.1:3004->80/tcp mm_v2-lxdao-mobymask-app-1
# Check logs for a container
docker logs -f mm_v2-lxdao-mobymask-app-1
+9 -8
View File
@@ -47,14 +47,14 @@ Create and update an env file to be used in the next step ([defaults](../../conf
For running mobymask-app
```bash
laconic-so --stack mobymask-v2 deploy --include mobymask-app --env-file <PATH_TO_ENV_FILE> up
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include mobymask-app --env-file <PATH_TO_ENV_FILE> up
# Runs mobymask-app on host port 3002 and lxdao-mobymask-app on host port 3004
```
For running peer-test-app
```bash
laconic-so --stack mobymask-v2 deploy --include peer-test-app --env-file <PATH_TO_ENV_FILE> up
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include peer-test-app --env-file <PATH_TO_ENV_FILE> up
# Runs on host port 3003
```
@@ -62,9 +62,10 @@ laconic-so --stack mobymask-v2 deploy --include peer-test-app --env-file <PATH_T
To list down and monitor the running containers:
```bash
laconic-so --stack mobymask-v2 deploy --include [mobymask-app | peer-test-app] ps
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include [mobymask-app | peer-test-app] ps
docker ps
# With status
docker ps -a
# Check logs for a container
docker logs -f <CONTAINER_ID>
@@ -80,20 +81,20 @@ Stop all services running in the background:
For mobymask-app
```bash
laconic-so --stack mobymask-v2 deploy --include mobymask-app down
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include mobymask-app down
```
For peer-test-app
```bash
laconic-so --stack mobymask-v2 deploy --include peer-test-app down
laconic-so --stack mobymask-v2 deploy --cluster mobymask_v2 --include peer-test-app down
```
Clear volumes created by this stack:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=.*mobymask_deployment|.*peers_ids"
docker volume ls -q --filter "name=mobymask_v2"
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=.*mobymask_deployment|.*peers_ids")
docker volume rm $(docker volume ls -q --filter "name=mobymask_v2")
```
+65
View File
@@ -0,0 +1,65 @@
# Reth
Deploy a Reth API node alongside Lighthouse.
## Clone required repositories
```
$ laconic-so --stack reth setup-repositories
```
## Build the Reth stack containers
```
$ laconic-so --stack reth build-containers
```
## Deploy the stack
```
$ laconic-so --stack reth deploy up
```
## Check logs
```
$ laconic-so --stack reth deploy logs
```
Verify that your node is syncing. You should see entries similar to this from the Lighthouse container:
```
laconic-200e8f8ff7891515d777cd0f719078e3-lighthouse-1 | Jun 23 20:59:01.226 INFO New block received root: 0x9cd4a2dd9333cf802c2963c2f029deb0f94e511d2481fa0724ae8752e4c49b15, slot: 6727493
```
and entries similar to this from the Reth container:
```
laconic-200e8f8ff7891515d777cd0f719078e3-reth-1 | 2023-06-23T20:59:11.557389Z INFO reth::node::events: Stage committed progress pipeline_stages=1/13 stage=Headers block=0 checkpoint=4.9% eta=1h 3m 57s
```
## Test the API
Reth's http api is accessible on port `8545` and the websocket api is accessible on port `8546`.
```
$ curl --request POST \
--url http://localhost:8545/ \
--header 'Content-Type: application/json' \
--data '{
"jsonrpc": "2.0",
"method": "eth_blockNumber",
"params": [],
"id": 0
}'
# Response
{"jsonrpc":"2.0","result":"0x0","id":0}
```
## Clean up
Stop all services running in the background:
```bash
$ laconic-so --stack reth deploy down
```
To also delete the docker data volumes:
```bash
$ laconic-so --stack reth deploy down --delete-volumes
+10
View File
@@ -0,0 +1,10 @@
version: "1.1"
name: reth
decription: "Reth node"
repos:
- github.com/paradigmxyz/reth
containers:
- cerc/reth
- cerc/lighthouse
pods:
- reth
+1 -1
View File
@@ -3,7 +3,7 @@ name: test
description: "A test stack"
repos:
- github.com/cerc-io/laconicd
- git.vdb.to/cerc-io/test-project
- git.vdb.to/cerc-io/test-project@test-branch
containers:
- cerc/test-container
pods:
+2 -2
View File
@@ -22,13 +22,13 @@ Instructions to deploy Uniswap v3 watcher stack (watcher + uniswap-v3-info front
* Clone / pull required repositories:
```bash
$ laconic-so setup-repositories --include vulcanize/uniswap-watcher-ts,vulcanize/uniswap-v3-info --git-ssh --pull
$ laconic-so --stack uniswap-v3 setup-repositories
```
* Build watcher and info app container images:
```bash
$ laconic-so build-containers --include cerc/watcher-uniswap-v3,cerc/uniswap-v3-info
$ laconic-so --stack uniswap-v3 build-containers
```
This should create the required docker images in the local image registry.
+6
View File
@@ -3,8 +3,14 @@ name: uniswap-v3
repos:
- github.com/vulcanize/uniswap-watcher-ts
- github.com/vulcanize/uniswap-v3-info
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/ipld-eth-server
containers:
- cerc/watcher-uniswap-v3
- cerc/uniswap-v3-info
- cerc/ipld-eth-db
- cerc/ipld-eth-server
pods:
- watcher-uniswap-v3
- ipld-eth-db
- ipld-eth-server
+94 -48
View File
@@ -21,12 +21,16 @@ import os
import sys
from dataclasses import dataclass
from decouple import config
from importlib import resources
import subprocess
from python_on_whales import DockerClient, DockerException
import click
import importlib.resources
from pathlib import Path
from .util import include_exclude_check, get_parsed_stack_config
from app.util import include_exclude_check, get_parsed_stack_config, global_options2
from app.deployment_create import create as deployment_create
from app.deployment_create import init as deployment_init
from app.deployment_create import setup as deployment_setup
class DeployCommandContext(object):
def __init__(self, cluster_context, docker):
@@ -43,44 +47,40 @@ class DeployCommandContext(object):
def command(ctx, include, exclude, env_file, cluster):
'''deploy a stack'''
cluster_context = _make_cluster_context(ctx.obj, include, exclude, cluster, env_file)
# See: https://gabrieldemarmiesse.github.io/python-on-whales/sub-commands/compose/
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster,
compose_env_file=cluster_context.env_file)
ctx.obj = DeployCommandContext(cluster_context, docker)
if ctx.parent.obj.debug:
print(f"ctx.parent.obj: {ctx.parent.obj}")
ctx.obj = create_deploy_context(global_options2(ctx), global_options2(ctx).stack, include, exclude, cluster, env_file)
# Subcommand is executed now, by the magic of click
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: up <service1> <service2>
@click.pass_context
def up(ctx, extra_args):
def create_deploy_context(global_context, stack, include, exclude, cluster, env_file):
cluster_context = _make_cluster_context(global_context, stack, include, exclude, cluster, env_file)
# See: https://gabrieldemarmiesse.github.io/python-on-whales/sub-commands/compose/
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster,
compose_env_file=cluster_context.env_file)
return DeployCommandContext(cluster_context, docker)
def up_operation(ctx, services_list, stay_attached=False):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
deploy_context = ctx.obj
if not global_context.dry_run:
cluster_context = ctx.obj.cluster_context
cluster_context = deploy_context.cluster_context
container_exec_env = _make_runtime_env(global_context)
for attr, value in container_exec_env.items():
os.environ[attr] = value
if global_context.verbose:
print(f"Running compose up with container_exec_env: {container_exec_env}, extra_args: {extra_args_list}")
print(f"Running compose up with container_exec_env: {container_exec_env}, extra_args: {services_list}")
for pre_start_command in cluster_context.pre_start_commands:
_run_command(global_context, cluster_context.cluster, pre_start_command)
ctx.obj.docker.compose.up(detach=True, services=extra_args_list)
deploy_context.docker.compose.up(detach=not stay_attached, services=services_list)
for post_start_command in cluster_context.post_start_commands:
_run_command(global_context, cluster_context.cluster, post_start_command)
_orchestrate_cluster_config(global_context, cluster_context.config, ctx.obj.docker, container_exec_env)
_orchestrate_cluster_config(global_context, cluster_context.config, deploy_context.docker, container_exec_env)
@command.command()
@click.option("--delete-volumes/--preserve-volumes", default=False, help="delete data volumes")
@click.argument('extra_args', nargs=-1) # help: command: down<service1> <service2>
@click.pass_context
def down(ctx, delete_volumes, extra_args):
def down_operation(ctx, delete_volumes, extra_args_list):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
if global_context.verbose:
print("Running compose down")
@@ -91,9 +91,7 @@ def down(ctx, delete_volumes, extra_args):
ctx.obj.docker.compose.down(timeout=timeout_arg, volumes=delete_volumes)
@command.command()
@click.pass_context
def ps(ctx):
def ps_operation(ctx):
global_context = ctx.parent.parent.obj
if not global_context.dry_run:
if global_context.verbose:
@@ -118,10 +116,7 @@ def ps(ctx):
print("No containers running")
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: port <service1> <service2>
@click.pass_context
def port(ctx, extra_args):
def port_operation(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
@@ -136,10 +131,7 @@ def port(ctx, extra_args):
print(f"{mapped_port_data[0]}:{mapped_port_data[1]}")
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: exec <service> <command>
@click.pass_context
def exec(ctx, extra_args):
def exec_operation(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
@@ -157,10 +149,7 @@ def exec(ctx, extra_args):
print(f"container command returned error exit status")
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: logs <service1> <service2>
@click.pass_context
def logs(ctx, extra_args):
def logs_operation(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
@@ -170,12 +159,56 @@ def logs(ctx, extra_args):
print(logs_output)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: up <service1> <service2>
@click.pass_context
def up(ctx, extra_args):
extra_args_list = list(extra_args) or None
up_operation(ctx, extra_args_list)
@command.command()
@click.option("--delete-volumes/--preserve-volumes", default=False, help="delete data volumes")
@click.argument('extra_args', nargs=-1) # help: command: down<service1> <service2>
@click.pass_context
def down(ctx, delete_volumes, extra_args):
extra_args_list = list(extra_args) or None
down_operation(ctx, delete_volumes, extra_args_list)
@command.command()
@click.pass_context
def ps(ctx):
ps_operation(ctx)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: port <service1> <service2>
@click.pass_context
def port(ctx, extra_args):
port_operation(ctx, extra_args)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: exec <service> <command>
@click.pass_context
def exec(ctx, extra_args):
exec_operation(ctx, extra_args)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: logs <service1> <service2>
@click.pass_context
def logs(ctx, extra_args):
logs_operation(ctx, extra_args)
def get_stack_status(ctx, stack):
ctx_copy = copy.copy(ctx)
ctx_copy.stack = stack
cluster_context = _make_cluster_context(ctx_copy, None, None, None, None)
cluster_context = _make_cluster_context(ctx_copy, stack, None, None, None, None)
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster)
# TODO: refactor to avoid duplicating this code above
if ctx.verbose:
@@ -200,7 +233,8 @@ def _make_runtime_env(ctx):
return container_exec_env
def _make_cluster_context(ctx, include, exclude, cluster, env_file):
# stack has to be either PathLike pointing to a stack yml file, or a string with the name of a known stack
def _make_cluster_context(ctx, stack, include, exclude, cluster, env_file):
if ctx.local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
@@ -208,14 +242,20 @@ def _make_cluster_context(ctx, include, exclude, cluster, env_file):
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
compose_dir = Path(__file__).absolute().parent.joinpath("data", "compose")
# TODO: huge hack, fix this
# If the caller passed a path for the stack file, then we know that we can get the compose files
# from the same directory
if isinstance(stack, os.PathLike):
compose_dir = stack.parent.joinpath("compose")
else:
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
compose_dir = Path(__file__).absolute().parent.joinpath("data", "compose")
if cluster is None:
# Create default unique, stable cluster name from confile file path and stack name if provided
# TODO: change this to the config file path
path = os.path.realpath(sys.argv[0])
unique_cluster_descriptor = f"{path},{ctx.stack},{include},{exclude}"
unique_cluster_descriptor = f"{path},{stack},{include},{exclude}"
if ctx.debug:
print(f"pre-hash descriptor: {unique_cluster_descriptor}")
hash = hashlib.md5(unique_cluster_descriptor.encode()).hexdigest()
@@ -224,13 +264,13 @@ def _make_cluster_context(ctx, include, exclude, cluster, env_file):
print(f"Using cluster name: {cluster}")
# See: https://stackoverflow.com/a/20885799/1701505
from . import data
with importlib.resources.open_text(data, "pod-list.txt") as pod_list_file:
from app import data
with resources.open_text(data, "pod-list.txt") as pod_list_file:
all_pods = pod_list_file.read().splitlines()
pods_in_scope = []
if ctx.stack:
stack_config = get_parsed_stack_config(ctx.stack)
if stack:
stack_config = get_parsed_stack_config(stack)
# TODO: syntax check the input here
pods_in_scope = stack_config['pods']
cluster_config = stack_config['config'] if 'config' in stack_config else None
@@ -342,6 +382,7 @@ def _orchestrate_cluster_config(ctx, cluster_config, docker, container_exec_env)
f" = {pd.source_container}.{pd.source_variable}")
# TODO: add a timeout
waiting_for_data = True
destination_output = "*** no output received yet ***"
while waiting_for_data:
# TODO: fix the script paths so they're consistent between containers
source_value = None
@@ -376,3 +417,8 @@ def _orchestrate_cluster_config(ctx, cluster_config, docker, container_exec_env)
waiting_for_data = False
if ctx.debug:
print(f"destination output: {destination_output}")
command.add_command(deployment_init)
command.add_command(deployment_create)
command.add_command(deployment_setup)
+132
View File
@@ -0,0 +1,132 @@
# Copyright © 2022, 2023 Cerc
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import click
from dataclasses import dataclass
from pathlib import Path
import sys
from app.deploy import up_operation, down_operation, ps_operation, port_operation, exec_operation, logs_operation, create_deploy_context
from app.util import global_options
@dataclass
class DeploymentContext:
dir: Path
@click.group()
@click.option("--dir", required=True, help="path to deployment directory")
@click.pass_context
def command(ctx, dir):
# Check that --stack wasn't supplied
if ctx.parent.obj.stack:
print("Error: --stack can't be supplied with the deployment command")
sys.exit(1)
# Check dir is valid
dir_path = Path(dir)
if not dir_path.exists():
print(f"Error: deployment directory {dir} does not exist")
sys.exit(1)
if not dir_path.is_dir():
print(f"Error: supplied deployment directory path {dir} exists but is a file not a directory")
sys.exit(1)
# Store the deployment context for subcommands
ctx.obj = DeploymentContext(dir_path)
def make_deploy_context(ctx):
# Get the stack config file name
stack_file_path = ctx.obj.dir.joinpath("stack.yml")
# TODO: add cluster name and env file here
return create_deploy_context(ctx.parent.parent.obj, stack_file_path, None, None, None, None)
@command.command()
@click.option("--stay-attached/--detatch-terminal", default=False, help="detatch or not to see container stdout")
@click.argument('extra_args', nargs=-1) # help: command: up <service1> <service2>
@click.pass_context
def up(ctx, stay_attached, extra_args):
ctx.obj = make_deploy_context(ctx)
services_list = list(extra_args) or None
up_operation(ctx, services_list, stay_attached)
# start is the preferred alias for up
@command.command()
@click.option("--stay-attached/--detatch-terminal", default=False, help="detatch or not to see container stdout")
@click.argument('extra_args', nargs=-1) # help: command: up <service1> <service2>
@click.pass_context
def start(ctx, stay_attached, extra_args):
ctx.obj = make_deploy_context(ctx)
services_list = list(extra_args) or None
up_operation(ctx, services_list, stay_attached)
@command.command()
@click.option("--delete-volumes/--preserve-volumes", default=False, help="delete data volumes")
@click.argument('extra_args', nargs=-1) # help: command: down <service1> <service2>
@click.pass_context
def down(ctx, delete_volumes, extra_args):
# Get the stack config file name
# TODO: add cluster name and env file here
ctx.obj = make_deploy_context(ctx)
down_operation(ctx, delete_volumes, extra_args)
# stop is the preferred alias for down
@command.command()
@click.option("--delete-volumes/--preserve-volumes", default=False, help="delete data volumes")
@click.argument('extra_args', nargs=-1) # help: command: down <service1> <service2>
@click.pass_context
def stop(ctx, delete_volumes, extra_args):
# TODO: add cluster name and env file here
ctx.obj = make_deploy_context(ctx)
down_operation(ctx, delete_volumes, extra_args)
@command.command()
@click.pass_context
def ps(ctx):
ctx.obj = make_deploy_context(ctx)
ps_operation(ctx)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: port <service1> <service2>
@click.pass_context
def port(ctx, extra_args):
port_operation(ctx, extra_args)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: exec <service> <command>
@click.pass_context
def exec(ctx, extra_args):
ctx.obj = make_deploy_context(ctx)
exec_operation(ctx, extra_args)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: logs <service1> <service2>
@click.pass_context
def logs(ctx, extra_args):
ctx.obj = make_deploy_context(ctx)
logs_operation(ctx, extra_args)
@command.command()
@click.pass_context
def status(ctx):
print(f"Context: {ctx.parent.obj}")

Some files were not shown because too many files have changed in this diff Show More