Compare commits

...
Author SHA1 Message Date
dboreham 97c1ae1c43 Use upstream act_runner project (#408) 2023-05-24 18:01:49 -06:00
dboreham ec6b5439f4 Support for git hosts other than github (#407)
* Update repository list file

* Add host part to repo name

* Allow git hosts other than github
2023-05-24 17:19:21 -06:00
dboreham 1d8f252a51 Detect bad reponse from yarn info (#406) 2023-05-22 13:42:55 -06:00
dboreham 161665ef72 Fix deploy commands (#404)
* Fix bugs

* Add test for deploy port command
2023-05-22 12:43:59 -06:00
dboreham 9c5f6469ff Allow docker buildkit to be enabled via env var (#403) 2023-05-22 11:38:34 -06:00
dboreham 85225c72d7 Fix another typo 2023-05-21 15:43:15 -06:00
dboreham 223d1171e8 Change test display name 2023-05-21 07:42:09 -06:00
dboreham 1e38e16550 Fix typo 2023-05-21 07:40:22 -06:00
dboreham dddae8cc7a Dboreham/deploy volume control (#401)
* Implement volume control

* Deploy test

* Add test for volumes

* Enable CI for deploy test
2023-05-21 07:39:00 -06:00
telackey aa702737ef Fix 397 by pegging alpine version. 2023-05-19 11:26:09 -05:00
prathamesh0 c9155eafd2 Add restart policies to fixturenet-eth and fixturenet-opimism stacks (#396)
* Add restart policies for fixturenet-optimism stack containers


Former-commit-id: e749699188c733614423ccc7ef43525b9805e23d

* Add restart policies for fixturenet-eth stack containers


Former-commit-id: 716e132300d88dbe6121ed3968a9c78b561196ef

* Remove existing bootnode ENR directory on start
2023-05-19 13:46:39 +05:30
dboreham 1ffc6b1687 Refactor deploy into click subcommands (#399)
Former-commit-id: cb58fdb58ce1686f4638946745830f391d820f4b
2023-05-18 17:01:46 -06:00
dboreham 87c25dfb5e Fix up test stack (#398)
Former-commit-id: 088105c7829254fc8ff1f31b71d28fd916def7eb
2023-05-18 13:54:27 -06:00
Ianandiskay 0691c22db4 Lotus (#392)
* fist commit

* manual peer connect

* add build to gitignore

* add shared volume

* connect to bootnode

* fix volume init bug

* todo generate genesis

* remove build dir

---------

Co-authored-by: iskay <ian@knowable.vc>
Former-commit-id: 5ecfcae5cc
2023-05-17 17:11:56 -04:00
prathamesh0 5c7d445500 Add a stack for Gelato watcher (#394)
* Add a stack for Gelato watcher

* Add option to create and use a state snapshot

* Add commands to create and import a state checkpoint

* Rename ipld-eth-server endpoint env variables

* Fix default env variable

Former-commit-id: 8b4b5deba8
2023-05-16 09:09:08 +05:30
dboreham a93fa93d26 Small doc fix
Former-commit-id: d26dd4b531
2023-05-09 17:08:25 -06:00
dboreham 1852d7d4c1 Chain chunker stack (#389)
* Fix bug in default container build flow

* Add convenience stack for chain-chunker

Former-commit-id: 3e78c321b0
2023-05-09 14:00:58 -06:00
Zach fce41994a3 match tokens (#388)
Former-commit-id: e5faeb9d3b
2023-05-09 15:23:56 -04:00
Zach a5d3d6bae7 Update laconicd-fixturenet.md (#386)
Former-commit-id: b6a0af4e95
2023-05-09 14:57:49 -04:00
nabarunandprathamesh 8add4671c0 Add environment variables for multiaddrs blacklist (#381)
* Add env variable for web apps config denyMultiaddrs

* Add watcher config option for blacklisted multiaddrs

* Update package versions

* Use provided domain for relay multiaddr in peer config

* Change delimeter while replacing deny multiaddrs list

---------

Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com>
Former-commit-id: b678a3ecb4
2023-05-05 13:32:19 +05:30
Marten O'Grady b1b1464205 Update CONTRIBUTING.md (#383)
Wrong output for Step 3 in Build A ZipApp.  Fixed it to what I just experienced while smoke testing.

Former-commit-id: bce604e4bb
2023-05-04 09:47:59 -04:00
Zach fbe901a0fb Merge pull request #382 from Escape613/patch-2
Update CONTRIBUTING.md

Former-commit-id: ff0a67f45f
2023-05-04 09:20:46 -04:00
Marten O'Grady a9558aa874 Update CONTRIBUTING.md
Missing ")" in Step #1 of INSTALL

Former-commit-id: f1bc8aa4e1
2023-05-04 09:16:14 -04:00
95 changed files with 1402 additions and 231 deletions
+39
View File
@@ -0,0 +1,39 @@
name: Deploy Test
on:
pull_request:
branches: '*'
push:
branches:
- main
- ci-test
# Needed until we can incorporate docker startup into the executor container
env:
DOCKER_HOST: unix:///var/run/dind.sock
jobs:
test:
name: "Run deploy test suite"
runs-on: ubuntu-latest
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
- name: "Install Python"
uses: cerc-io/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: Start dockerd # Also needed until we can incorporate into the executor
run: |
dockerd -H $DOCKER_HOST --userland-proxy=false &
sleep 5
- name: "Run deploy tests"
run: ./tests/deploy/run-deploy-test.sh
+1 -1
View File
@@ -1,4 +1,4 @@
name: Integration Test
name: Smoke Test
on:
pull_request:
+29
View File
@@ -0,0 +1,29 @@
name: Deploy Test
on:
pull_request:
branches: '*'
push:
branches: '*'
jobs:
test:
name: "Run deploy test suite"
runs-on: ubuntu-latest
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
- name: "Install Python"
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: "Run deploy tests"
run: ./tests/deploy/run-deploy-test.sh
+1 -1
View File
@@ -1,4 +1,4 @@
name: Test
name: Smoke Test
on:
pull_request:
+1
View File
@@ -7,3 +7,4 @@ __pycache__
*~
package
app/data/build_tag.txt
build
+2 -2
View File
@@ -90,7 +90,7 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
"CERC_CONTAINER_BASE_DIR": container_build_dir,
"CERC_HOST_UID": f"{os.getuid()}",
"CERC_HOST_GID": f"{os.getgid()}",
"DOCKER_BUILDKIT": "0"
"DOCKER_BUILDKIT": config("DOCKER_BUILDKIT", default="0")
}
container_build_env.update({"CERC_SCRIPT_DEBUG": "true"} if debug else {})
container_build_env.update({"CERC_FORCE_REBUILD": "true"} if force_rebuild else {})
@@ -115,7 +115,7 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
# TODO: make this less of a hack -- should be specified in some metadata somewhere
# Check if we have a repo for this container. If not, set the context dir to the container-build subdir
repo_full_path = os.path.join(dev_root_path, repo_dir)
repo_dir_or_build_dir = repo_dir if os.path.exists(repo_full_path) else build_dir
repo_dir_or_build_dir = repo_full_path if os.path.exists(repo_full_path) else build_dir
build_command = os.path.join(container_build_dir, "default-build.sh") + f" {container}:local {repo_dir_or_build_dir}"
if not dry_run:
if verbose:
@@ -2,6 +2,7 @@ version: '3.7'
services:
fixturenet-eth-bootnode-geth:
restart: always
hostname: fixturenet-eth-bootnode-geth
env_file:
- ../config/fixturenet-eth/fixturenet-eth.env
@@ -15,6 +16,7 @@ services:
- "30303"
fixturenet-eth-geth-1:
restart: always
hostname: fixturenet-eth-geth-1
cap_add:
- SYS_PTRACE
@@ -42,6 +44,7 @@ services:
- "6060"
fixturenet-eth-geth-2:
restart: always
hostname: fixturenet-eth-geth-2
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "8545"]
@@ -60,12 +63,14 @@ services:
- fixturenet_eth_geth_2_data:/root/ethdata
fixturenet-eth-bootnode-lighthouse:
restart: always
hostname: fixturenet-eth-bootnode-lighthouse
environment:
RUN_BOOTNODE: "true"
image: cerc/fixturenet-eth-lighthouse:local
fixturenet-eth-lighthouse-1:
restart: always
hostname: fixturenet-eth-lighthouse-1
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8001/eth/v2/beacon/blocks/head"]
@@ -91,6 +96,7 @@ services:
- "8001"
fixturenet-eth-lighthouse-2:
restart: always
hostname: fixturenet-eth-lighthouse-2
healthcheck:
test: ["CMD", "wget", "--tries=1", "--connect-timeout=1", "--quiet", "-O", "-", "http://localhost:8001/eth/v2/beacon/blocks/head"]
@@ -1,10 +1,11 @@
version: "3.2"
services:
laconicd:
restart: unless-stopped
image: cerc/laconicd:local
command: ["sh", "/docker-entrypoint-scripts.d/create-fixturenet.sh"]
volumes:
# The cosmos-sdk node's database directory:
- laconicd-data:/root/.laconicd/data
# TODO: look at folding these scripts into the container
- ../config/fixturenet-laconicd/create-fixturenet.sh:/docker-entrypoint-scripts.d/create-fixturenet.sh
- ../config/fixturenet-laconicd/export-mykey.sh:/docker-entrypoint-scripts.d/export-mykey.sh
@@ -0,0 +1,68 @@
version: "3.8"
services:
lotus-miner:
hostname: lotus-miner
env_file:
- ../config/fixturenet-lotus/lotus-env.env
image: cerc/lotus:local
volumes:
- ../config/fixturenet-lotus/setup-miner.sh:/docker-entrypoint-scripts.d/setup-miner.sh
- ../config/fixturenet-lotus/genesis/devgen.car:/devgen.car
- $HOME/stack-orchestrator/app/data/config/fixturenet-lotus/genesis/.genesis-sectors:/root/.genesis-sectors
- lotus-shared:/root/.lotus-shared
healthcheck:
# test: ["CMD-SHELL", "grep 'started ChainNotify channel' /var/log/lotus.log"]
# test: ["CMD-SHELL", "[ -f /root/.lotus-shared/miner.addr ]"]
test: ["CMD-SHELL", "[ -d /root/.lotus-miner-local-net ]"]
interval: 10s
timeout: 10s
retries: 10
start_period: 60s
entrypoint: ["sh", "/docker-entrypoint-scripts.d/setup-miner.sh"]
ports:
- "1234"
- "2345"
- "3456"
- "1777"
lotus-node-1:
hostname: lotus-node-1
env_file:
- ../config/fixturenet-lotus/lotus-env.env
image: cerc/lotus:local
volumes:
- ../config/fixturenet-lotus/setup-node.sh:/docker-entrypoint-scripts.d/setup-node.sh
- ../config/fixturenet-lotus/genesis/devgen.car:/devgen.car
- lotus-shared:/root/.lotus-shared
depends_on:
lotus-miner:
condition: service_healthy
entrypoint: ["sh", "/docker-entrypoint-scripts.d/setup-node.sh"]
ports:
- "1234"
- "2345"
- "3456"
- "1777"
lotus-node-2:
hostname: lotus-node-2
env_file:
- ../config/fixturenet-lotus/lotus-env.env
image: cerc/lotus:local
volumes:
- ../config/fixturenet-lotus/setup-node.sh:/docker-entrypoint-scripts.d/setup-node.sh
- ../config/fixturenet-lotus/genesis/devgen.car:/devgen.car
- lotus-shared:/root/.lotus-shared
depends_on:
lotus-miner:
condition: service_healthy
entrypoint: ["sh", "/docker-entrypoint-scripts.d/setup-node.sh"]
ports:
- "1234"
- "2345"
- "3456"
- "1777"
volumes:
lotus-shared:
@@ -5,6 +5,7 @@ services:
# Creates / updates the configuration for L1 contracts deployment
# Deploys the L1 smart contracts (outputs to volume l1_deployment)
fixturenet-optimism-contracts:
restart: on-failure
hostname: fixturenet-optimism-contracts
image: cerc/optimism-contracts:local
env_file:
@@ -35,6 +36,7 @@ services:
# Generates the config files required for L2 (outputs to volume l2_config)
op-node-l2-config-gen:
restart: on-failure
image: cerc/optimism-op-node:local
depends_on:
fixturenet-optimism-contracts:
@@ -54,6 +56,7 @@ services:
# Initializes and runs the L2 execution client (outputs to volume l2_geth_data)
op-geth:
restart: always
image: cerc/optimism-l2geth:local
depends_on:
op-node-l2-config-gen:
@@ -76,6 +79,7 @@ services:
# Runs the L2 consensus client (Sequencer node)
op-node:
restart: always
image: cerc/optimism-op-node:local
depends_on:
op-geth:
@@ -103,6 +107,7 @@ services:
# Runs the batcher (takes transactions from the Sequencer and publishes them to L1)
op-batcher:
restart: always
image: cerc/optimism-op-batcher:local
depends_on:
op-node:
@@ -129,6 +134,7 @@ services:
# Runs the proposer (periodically submits new state roots to L1)
op-proposer:
restart: always
image: cerc/optimism-op-proposer:local
depends_on:
op-node:
@@ -1,6 +1,7 @@
# Add-on pod to include foundry tooling within a fixturenet
services:
foundry:
restart: always
image: cerc/foundry:local
command: ["while :; do sleep 600; done"]
volumes:
@@ -13,6 +13,7 @@ services:
CERC_DEPLOYED_CONTRACT: ${CERC_DEPLOYED_CONTRACT}
CERC_APP_WATCHER_URL: ${CERC_APP_WATCHER_URL}
CERC_RELAY_NODES: ${CERC_RELAY_NODES}
CERC_DENY_MULTIADDRS: ${CERC_DENY_MULTIADDRS}
CERC_BUILD_DIR: "@cerc-io/mobymask-ui/build"
working_dir: /scripts
command: ["sh", "mobymask-app-start.sh"]
@@ -44,6 +45,7 @@ services:
CERC_DEPLOYED_CONTRACT: ${CERC_DEPLOYED_CONTRACT}
CERC_APP_WATCHER_URL: ${CERC_APP_WATCHER_URL}
CERC_RELAY_NODES: ${CERC_RELAY_NODES}
CERC_DENY_MULTIADDRS: ${CERC_DENY_MULTIADDRS}
CERC_BUILD_DIR: "@cerc-io/mobymask-ui-lxdao/build"
working_dir: /scripts
command: ["sh", "mobymask-app-start.sh"]
@@ -10,6 +10,7 @@ services:
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_RELAY_NODES: ${CERC_RELAY_NODES}
CERC_DENY_MULTIADDRS: ${CERC_DENY_MULTIADDRS}
command: ["sh", "test-app-start.sh"]
volumes:
- ../config/wait-for-it.sh:/scripts/wait-for-it.sh
+7 -1
View File
@@ -1,7 +1,13 @@
version: "3.2"
services:
test:
image: cerc/test-container:local
restart: always
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
volumes:
- test-data:/var
ports:
- "80"
volumes:
test-data:
@@ -0,0 +1,91 @@
version: '3.2'
services:
# Starts the PostgreSQL database for watcher
gelato-watcher-db:
restart: unless-stopped
image: postgres:14-alpine
environment:
- POSTGRES_USER=vdbm
- POSTGRES_MULTIPLE_DATABASES=gelato-watcher,gelato-watcher-job-queue
- POSTGRES_EXTENSION=gelato-watcher-job-queue:pgcrypto
- POSTGRES_PASSWORD=password
volumes:
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
- gelato_watcher_db_data:/var/lib/postgresql/data
ports:
- "0.0.0.0:15432:5432"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "5432"]
interval: 10s
timeout: 5s
retries: 15
start_period: 10s
# Starts the gelato-watcher job runner
gelato-watcher-job-runner:
image: cerc/watcher-gelato:local
restart: unless-stopped
depends_on:
gelato-watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-gelato/watcher-params.env
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
command: ["./start-job-runner.sh"]
volumes:
- ../config/watcher-gelato/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-gelato/start-job-runner.sh:/app/start-job-runner.sh
ports:
- "0.0.0.0:9000:9000"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "9000"]
interval: 10s
timeout: 5s
retries: 15
start_period: 10s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the gelato-watcher server
gelato-watcher-server:
image: cerc/watcher-gelato:local
restart: unless-stopped
depends_on:
gelato-watcher-db:
condition: service_healthy
gelato-watcher-job-runner:
condition: service_healthy
env_file:
- ../config/watcher-gelato/watcher-params.env
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CERC_USE_STATE_SNAPSHOT: ${CERC_USE_STATE_SNAPSHOT}
CERC_SNAPSHOT_GQL_ENDPOINT: ${CERC_SNAPSHOT_GQL_ENDPOINT}
CERC_SNAPSHOT_BLOCKHASH: ${CERC_SNAPSHOT_BLOCKHASH}
command: ["./start-server.sh"]
volumes:
- ../config/watcher-gelato/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-gelato/start-server.sh:/app/start-server.sh
- ../config/watcher-gelato/create-and-import-checkpoint.sh:/app/create-and-import-checkpoint.sh
- gelato_watcher_state_gql:/app/state_checkpoint
ports:
- "0.0.0.0:3008:3008"
- "0.0.0.0:9001:9001"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "3008"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
gelato_watcher_db_data:
gelato_watcher_state_gql:
@@ -83,6 +83,7 @@ services:
CERC_L1_ACCOUNTS_CSV_URL: ${CERC_L1_ACCOUNTS_CSV_URL}
CERC_PRIVATE_KEY_PEER: ${CERC_PRIVATE_KEY_PEER}
CERC_RELAY_PEERS: ${CERC_RELAY_PEERS}
CERC_DENY_MULTIADDRS: ${CERC_DENY_MULTIADDRS}
CERC_RELAY_ANNOUNCE_DOMAIN: ${CERC_RELAY_ANNOUNCE_DOMAIN}
CERC_ENABLE_PEER_L2_TXS: ${CERC_ENABLE_PEER_L2_TXS}
CERC_DEPLOYED_CONTRACT: ${CERC_DEPLOYED_CONTRACT}
@@ -0,0 +1 @@
}+V{iνΆΠΉ²¨ΣΗ\k»qς␍ —?δΪAΒfκ’~μ©™LΉtb·yqτ·²ηξΔΟ?ξaΣJ
@@ -0,0 +1 @@
Βfκ’~μ©™LΉtb·yqτ·²ηξΔΟ?ξaΣJ
@@ -0,0 +1,71 @@
{
"t01000": {
"ID": "t01000",
"Owner": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Worker": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"PeerId": "12D3KooWG5q6pWJVdPBhDBv9AjWVbUh4xxTAZ7xvgZSjczWuD2Z9",
"MarketBalance": "0",
"PowerBalance": "0",
"SectorSize": 2048,
"Sectors": [
{
"CommR": {
"/": "bagboea4b5abcboxypcewlkmrat2myu4vthk3ii2pcomak7nhqmdbb6sxlolp2wdf"
},
"CommD": {
"/": "baga6ea4seaqn3jfixthmdgksv4vhfeuyvr6upw6tvaqbmzmsyxnzosm4pwgnmlq"
},
"SectorID": 0,
"Deal": {
"PieceCID": {
"/": "baga6ea4seaqn3jfixthmdgksv4vhfeuyvr6upw6tvaqbmzmsyxnzosm4pwgnmlq"
},
"PieceSize": 2048,
"VerifiedDeal": false,
"Client": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Provider": "t01000",
"Label": "0",
"StartEpoch": 0,
"EndEpoch": 9001,
"StoragePricePerEpoch": "0",
"ProviderCollateral": "0",
"ClientCollateral": "0"
},
"DealClientKey": {
"Type": "bls",
"PrivateKey": "tFvSRiSg2G3Ssgg0PSYy23XyjaIMXpsmdyG2B7UFLT4="
},
"ProofType": 5
},
{
"CommR": {
"/": "bagboea4b5abcb6krzypqcczhcnbeyjcqkeo6omfergm336o3kitugh3jgjog2yqq"
},
"CommD": {
"/": "baga6ea4seaqhondpb2373hjasjplxvbjzi5n5mm4fbbhjxp5ptnbq4cibapkeii"
},
"SectorID": 1,
"Deal": {
"PieceCID": {
"/": "baga6ea4seaqhondpb2373hjasjplxvbjzi5n5mm4fbbhjxp5ptnbq4cibapkeii"
},
"PieceSize": 2048,
"VerifiedDeal": false,
"Client": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Provider": "t01000",
"Label": "1",
"StartEpoch": 0,
"EndEpoch": 9001,
"StoragePricePerEpoch": "0",
"ProviderCollateral": "0",
"ClientCollateral": "0"
},
"DealClientKey": {
"Type": "bls",
"PrivateKey": "tFvSRiSg2G3Ssgg0PSYy23XyjaIMXpsmdyG2B7UFLT4="
},
"ProofType": 5
}
]
}
}
@@ -0,0 +1 @@
7b2254797065223a22626c73222c22507269766174654b6579223a227446765352695367324733537367673050535979323358796a61494d5870736d64794732423755464c54343d227d
@@ -0,0 +1,11 @@
{
"ID": "f355523e-69d0-4984-bd0e-9588487c6231",
"Weight": 0,
"CanSeal": false,
"CanStore": false,
"MaxStorage": 0,
"Groups": null,
"AllowTo": null,
"AllowTypes": null,
"DenyTypes": null
}
Binary file not shown.
@@ -0,0 +1,108 @@
{
"NetworkVersion": 18,
"Accounts": [
{
"Type": "account",
"Balance": "50000000000000000000000000",
"Meta": {
"Owner": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q"
}
}
],
"Miners": [
{
"ID": "t01000",
"Owner": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Worker": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"PeerId": "12D3KooWG5q6pWJVdPBhDBv9AjWVbUh4xxTAZ7xvgZSjczWuD2Z9",
"MarketBalance": "0",
"PowerBalance": "0",
"SectorSize": 2048,
"Sectors": [
{
"CommR": {
"/": "bagboea4b5abcboxypcewlkmrat2myu4vthk3ii2pcomak7nhqmdbb6sxlolp2wdf"
},
"CommD": {
"/": "baga6ea4seaqn3jfixthmdgksv4vhfeuyvr6upw6tvaqbmzmsyxnzosm4pwgnmlq"
},
"SectorID": 0,
"Deal": {
"PieceCID": {
"/": "baga6ea4seaqn3jfixthmdgksv4vhfeuyvr6upw6tvaqbmzmsyxnzosm4pwgnmlq"
},
"PieceSize": 2048,
"VerifiedDeal": false,
"Client": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Provider": "t01000",
"Label": "0",
"StartEpoch": 0,
"EndEpoch": 9001,
"StoragePricePerEpoch": "0",
"ProviderCollateral": "0",
"ClientCollateral": "0"
},
"DealClientKey": {
"Type": "bls",
"PrivateKey": "tFvSRiSg2G3Ssgg0PSYy23XyjaIMXpsmdyG2B7UFLT4="
},
"ProofType": 5
},
{
"CommR": {
"/": "bagboea4b5abcb6krzypqcczhcnbeyjcqkeo6omfergm336o3kitugh3jgjog2yqq"
},
"CommD": {
"/": "baga6ea4seaqhondpb2373hjasjplxvbjzi5n5mm4fbbhjxp5ptnbq4cibapkeii"
},
"SectorID": 1,
"Deal": {
"PieceCID": {
"/": "baga6ea4seaqhondpb2373hjasjplxvbjzi5n5mm4fbbhjxp5ptnbq4cibapkeii"
},
"PieceSize": 2048,
"VerifiedDeal": false,
"Client": "t3spusn5ia57qezc3fwpe3n2lhb4y4xt67xoflqbqy2muliparw2uktevletuv7gl4qakjpafgcl7jk2s2er3q",
"Provider": "t01000",
"Label": "1",
"StartEpoch": 0,
"EndEpoch": 9001,
"StoragePricePerEpoch": "0",
"ProviderCollateral": "0",
"ClientCollateral": "0"
},
"DealClientKey": {
"Type": "bls",
"PrivateKey": "tFvSRiSg2G3Ssgg0PSYy23XyjaIMXpsmdyG2B7UFLT4="
},
"ProofType": 5
}
]
}
],
"NetworkName": "localnet-6d52dae5-ff29-4bac-a45d-f84e6c07564c",
"VerifregRootKey": {
"Type": "multisig",
"Balance": "0",
"Meta": {
"Signers": [
"t1ceb34gnsc6qk5dt6n7xg6ycwzasjhbxm3iylkiy"
],
"Threshold": 1,
"VestingDuration": 0,
"VestingStart": 0
}
},
"RemainderAccount": {
"Type": "multisig",
"Balance": "0",
"Meta": {
"Signers": [
"t1ceb34gnsc6qk5dt6n7xg6ycwzasjhbxm3iylkiy"
],
"Threshold": 1,
"VestingDuration": 0,
"VestingStart": 0
}
}
}
@@ -0,0 +1,5 @@
LOTUS_PATH=~/.lotus-local-net
LOTUS_MINER_PATH=~/.lotus-miner-local-net
LOTUS_SKIP_GENESIS_CHECK=_yes_
CGO_CFLAGS_ALLOW="-D__BLST_PORTABLE__"
CGO_CFLAGS="-D__BLST_PORTABLE__"
@@ -0,0 +1,39 @@
#!/bin/bash
lotus --version
# # remove old bootnode peer info if present
# [ -f /root/.lotus-shared/miner.addr ] && rm /root/.lotus-shared/miner.addr
##TODO: generate genesis files inside container instead of bundling in config dir
##something like commands below should work, other scripts/compose will have to be updated to corresponding directories
# lotus fetch-params 2048
# lotus-seed pre-seal --sector-size 2KiB --num-sectors 2
# lotus-seed genesis new localnet.json
# lotus-seed genesis add-miner localnet.json ~/.genesis-sectors/pre-seal-t01000.json
# start daemon
nohup lotus daemon --genesis=/devgen.car --profile=bootstrapper --bootstrap=false > /var/log/lotus.log 2>&1 &
# Loop until the daemon is started
echo "Waiting for daemon to start..."
while ! grep -q "started ChainNotify channel" /var/log/lotus.log ; do
sleep 5
done
echo "Daemon started."
# publish bootnode peer info to shared volume
lotus net listen | awk 'NR==1{print}' > /root/.lotus-shared/miner.addr
# if miner not already initialized
if [ ! -d /root/.lotus-miner-local-net ]; then
# initialize miner
lotus wallet import --as-default ~/.genesis-sectors/pre-seal-t01000.key
lotus-miner init --genesis-miner --actor=t01000 --sector-size=2KiB --pre-sealed-sectors=~/.genesis-sectors --pre-sealed-metadata=~/.genesis-sectors/pre-seal-t01000.json --nosync
fi
# start miner
nohup lotus-miner run --nosync &
tail -f /dev/null
@@ -0,0 +1,24 @@
#!/bin/bash
lotus --version
##TODO: paths can use values from lotus-env.env file
# if not already initialized
if [ ! -f /root/.lotus-local-net/config.toml ]; then
# init node config
mkdir $HOME/.lotus-local-net
lotus config default > $HOME/.lotus-local-net/config.toml
# add bootstrap peer info if available
if [ -f /root/.lotus-shared/miner.addr ]; then
MINER_ADDR=\"$(cat /root/.lotus-shared/miner.addr)\"
# add bootstrap peer id to config file
sed -i "/^\[Libp2p\]/a \ \ BootstrapPeers = [$MINER_ADDR]" $HOME/.lotus-local-net/config.toml
else
echo "Bootstrap peer info not found, unable to configure. Manual peering will be required."
fi
fi
# start node
lotus daemon --genesis=/devgen.car
@@ -0,0 +1,28 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_SNAPSHOT_GQL_ENDPOINT="${CERC_SNAPSHOT_GQL_ENDPOINT:-${DEFAULT_CERC_SNAPSHOT_GQL_ENDPOINT}}"
CERC_SNAPSHOT_BLOCKHASH="${CERC_SNAPSHOT_BLOCKHASH:-${DEFAULT_CERC_SNAPSHOT_BLOCKHASH}}"
CHECKPOINT_FILE_PATH="./state_checkpoint/state-gql-${CERC_SNAPSHOT_BLOCKHASH}"
if [ -f "${CHECKPOINT_FILE_PATH}" ]; then
# Skip checkpoint creation if the file already exists
echo "File at ${CHECKPOINT_FILE_PATH} already exists, skipping checkpoint creation..."
else
# Create a checkpoint using GQL endpoint
echo "Creating a state checkpoint using GQL endpoint..."
yarn create-state-gql \
--snapshot-block-hash "${CERC_SNAPSHOT_BLOCKHASH}" \
--gql-endpoint "${CERC_SNAPSHOT_GQL_ENDPOINT}" \
--output "${CHECKPOINT_FILE_PATH}"
fi
echo "Initializing watcher using a state snapshot..."
# Import the state checkpoint
# (skips if snapshot block is already indexed)
yarn import-state --import-file "${CHECKPOINT_FILE_PATH}"
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_IPLD_ETH_RPC="${CERC_IPLD_ETH_RPC:-${DEFAULT_CERC_IPLD_ETH_RPC}}"
CERC_IPLD_ETH_GQL="${CERC_IPLD_ETH_GQL:-${DEFAULT_CERC_IPLD_ETH_GQL}}"
echo "Using ETH server RPC endpoint ${CERC_IPLD_ETH_RPC}"
echo "Using ETH server GQL endpoint ${CERC_IPLD_ETH_GQL}"
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_IPLD_ETH_GQL|${CERC_IPLD_ETH_GQL}|g; \
s|REPLACE_WITH_CERC_IPLD_ETH_RPC|${CERC_IPLD_ETH_RPC}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/local.toml
echo "Running job-runner"
DEBUG=vulcanize:* exec node --enable-source-maps dist/job-runner.js
+32
View File
@@ -0,0 +1,32 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_IPLD_ETH_RPC="${CERC_IPLD_ETH_RPC:-${DEFAULT_CERC_IPLD_ETH_RPC}}"
CERC_IPLD_ETH_GQL="${CERC_IPLD_ETH_GQL:-${DEFAULT_CERC_IPLD_ETH_GQL}}"
CERC_USE_STATE_SNAPSHOT="${CERC_USE_STATE_SNAPSHOT:-${DEFAULT_CERC_USE_STATE_SNAPSHOT}}"
echo "Using ETH server RPC endpoint ${CERC_IPLD_ETH_RPC}"
echo "Using ETH server GQL endpoint ${CERC_IPLD_ETH_GQL}"
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_IPLD_ETH_GQL|${CERC_IPLD_ETH_GQL}|g; \
s|REPLACE_WITH_CERC_IPLD_ETH_RPC|${CERC_IPLD_ETH_RPC}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/local.toml
if [ "$CERC_USE_STATE_SNAPSHOT" = true ] ; then
./create-and-import-checkpoint.sh
else
echo "Initializing watcher using fill..."
yarn fill --start-block $DEFAULT_CERC_GELATO_START_BLOCK --end-block $DEFAULT_CERC_GELATO_START_BLOCK
fi
echo "Running active server"
DEBUG=vulcanize:* exec node --enable-source-maps dist/server.js
@@ -0,0 +1,75 @@
[server]
host = "0.0.0.0"
port = 3008
kind = "active"
# Checkpointing state.
checkpointing = true
# Checkpoint interval in number of blocks.
checkpointInterval = 2000
# Enable state creation
# CAUTION: Disable only if state creation is not desired or can be filled subsequently
enableState = true
subgraphPath = "./subgraph"
# Interval to restart wasm instance periodically
wasmRestartBlocksInterval = 20
# Interval in number of blocks at which to clear entities cache.
clearEntitiesCacheInterval = 1000
# Boolean to filter logs by contract.
filterLogs = true
# Max block range for which to return events in eventsInRange GQL query.
# Use -1 for skipping check on block range.
maxEventsBlockRange = 1000
# GQL cache settings
[server.gqlCache]
enabled = true
# Max in-memory cache size (in bytes) (default 8 MB)
# maxCacheSize
# GQL cache-control max-age settings (in seconds)
maxAge = 15
timeTravelMaxAge = 86400 # 1 day
[metrics]
host = "0.0.0.0"
port = 9000
[metrics.gql]
port = 9001
[database]
type = "postgres"
host = "gelato-watcher-db"
port = 5432
database = "gelato-watcher"
username = "vdbm"
password = "password"
synchronize = true
logging = false
[upstream]
[upstream.ethServer]
gqlApiEndpoint = "REPLACE_WITH_CERC_IPLD_ETH_GQL"
rpcProviderEndpoint = "REPLACE_WITH_CERC_IPLD_ETH_RPC"
[upstream.cache]
name = "requests"
enabled = false
deleteOnStart = false
[jobQueue]
dbConnectionString = "postgres://vdbm:password@gelato-watcher-db/gelato-watcher-job-queue"
maxCompletionLagInSecs = 300
jobDelayInMilliSecs = 100
eventsInBatch = 50
blockDelayInMilliSecs = 2000
prefetchBlocksInMem = true
prefetchBlockCount = 10
@@ -0,0 +1,13 @@
# ipld-eth-server endpoints
DEFAULT_CERC_IPLD_ETH_RPC="http://ipld-eth-server:8082"
DEFAULT_CERC_IPLD_ETH_GQL="http://ipld-eth-server:8083/graphql"
# Gelato start block
DEFAULT_CERC_GELATO_START_BLOCK=11361987
# Whether to use a state snapshot to initialize the watcher
DEFAULT_CERC_USE_STATE_SNAPSHOT=false
# State snapshot params
DEFAULT_CERC_SNAPSHOT_GQL_ENDPOINT=
DEFAULT_CERC_SNAPSHOT_BLOCKHASH=
@@ -7,6 +7,7 @@ fi
CERC_CHAIN_ID="${CERC_CHAIN_ID:-${DEFAULT_CERC_CHAIN_ID}}"
CERC_DEPLOYED_CONTRACT="${CERC_DEPLOYED_CONTRACT:-${DEFAULT_CERC_DEPLOYED_CONTRACT}}"
CERC_RELAY_NODES="${CERC_RELAY_NODES:-${DEFAULT_CERC_RELAY_NODES}}"
CERC_DENY_MULTIADDRS="${CERC_DENY_MULTIADDRS:-${DEFAULT_CERC_DENY_MULTIADDRS}}"
CERC_APP_WATCHER_URL="${CERC_APP_WATCHER_URL:-${DEFAULT_CERC_APP_WATCHER_URL}}"
# If not set (or []), check the mounted volume for relay peer id
@@ -37,5 +38,6 @@ yq -n ".address = env(CERC_DEPLOYED_CONTRACT)" > /config/config.yml
yq ".watcherUrl = env(CERC_APP_WATCHER_URL)" -i /config/config.yml
yq ".chainId = env(CERC_CHAIN_ID)" -i /config/config.yml
yq ".relayNodes = strenv(CERC_RELAY_NODES)" -i /config/config.yml
yq ".denyMultiaddrs = strenv(CERC_DENY_MULTIADDRS)" -i /config/config.yml
/scripts/start-serving-app.sh
@@ -24,3 +24,6 @@ DEFAULT_CERC_CHAIN_ID=42069
# Set of relay nodes to be used by web-apps
DEFAULT_CERC_RELAY_NODES=[]
# Set of multiaddrs to be avoided while dialling
DEFAULT_CERC_DENY_MULTIADDRS=[]
@@ -8,13 +8,20 @@ CERC_L2_GETH_RPC="${CERC_L2_GETH_RPC:-${DEFAULT_CERC_L2_GETH_RPC}}"
CERC_L1_ACCOUNTS_CSV_URL="${CERC_L1_ACCOUNTS_CSV_URL:-${DEFAULT_CERC_L1_ACCOUNTS_CSV_URL}}"
CERC_RELAY_PEERS="${CERC_RELAY_PEERS:-${DEFAULT_CERC_RELAY_PEERS}}"
CERC_DENY_MULTIADDRS="${CERC_DENY_MULTIADDRS:-${DEFAULT_CERC_DENY_MULTIADDRS}}"
CERC_RELAY_ANNOUNCE_DOMAIN="${CERC_RELAY_ANNOUNCE_DOMAIN:-${DEFAULT_CERC_RELAY_ANNOUNCE_DOMAIN}}"
CERC_ENABLE_PEER_L2_TXS="${CERC_ENABLE_PEER_L2_TXS:-${DEFAULT_CERC_ENABLE_PEER_L2_TXS}}"
CERC_DEPLOYED_CONTRACT="${CERC_DEPLOYED_CONTRACT:-${DEFAULT_CERC_DEPLOYED_CONTRACT}}"
echo "Using L2 RPC endpoint ${CERC_L2_GETH_RPC}"
CERC_RELAY_MULTIADDR="/dns4/mobymask-watcher-server/tcp/9090/ws/p2p/$(jq -r '.id' /app/peers/relay-id.json)"
# Use public domain for relay multiaddr in peer config if specified
# Otherwise, use the docker container's host IP
if [ -n "$CERC_RELAY_ANNOUNCE_DOMAIN" ]; then
CERC_RELAY_MULTIADDR="/dns4/${CERC_RELAY_ANNOUNCE_DOMAIN}/tcp/443/wss/p2p/$(jq -r '.id' /app/peers/relay-id.json)"
else
CERC_RELAY_MULTIADDR="/dns4/mobymask-watcher-server/tcp/9090/ws/p2p/$(jq -r '.id' /app/peers/relay-id.json)"
fi
# Use contract address from environment variable or set from config.json in mounted volume
if [ -n "$CERC_DEPLOYED_CONTRACT" ]; then
@@ -42,6 +49,7 @@ fi
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_RELAY_PEERS|${CERC_RELAY_PEERS}|g; \
s|REPLACE_WITH_CERC_DENY_MULTIADDRS|${CERC_DENY_MULTIADDRS}|g; \
s/REPLACE_WITH_CERC_RELAY_ANNOUNCE_DOMAIN/${CERC_RELAY_ANNOUNCE_DOMAIN}/g; \
s|REPLACE_WITH_CERC_RELAY_MULTIADDR|${CERC_RELAY_MULTIADDR}|g; \
s/REPLACE_WITH_CERC_ENABLE_PEER_L2_TXS/${CERC_ENABLE_PEER_L2_TXS}/g; \
@@ -1,6 +1,7 @@
{
"relayNodes": [],
"peer": {
"denyMultiaddrs": [],
"enableDebugInfo": true
}
}
@@ -5,6 +5,7 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
fi
CERC_RELAY_NODES="${CERC_RELAY_NODES:-${DEFAULT_CERC_RELAY_NODES}}"
CERC_DENY_MULTIADDRS="${CERC_DENY_MULTIADDRS:-${DEFAULT_CERC_DENY_MULTIADDRS}}"
# If not set (or []), check the mounted volume for relay peer id
if [ -z "$CERC_RELAY_NODES" ] || [ "$CERC_RELAY_NODES" = "[]" ]; then
@@ -16,5 +17,6 @@ echo "Using CERC_RELAY_NODES $CERC_RELAY_NODES"
# Use yq to create config.yml with environment variables
yq -n ".relayNodes = strenv(CERC_RELAY_NODES)" > /config/config.yml
yq ".denyMultiaddrs = strenv(CERC_DENY_MULTIADDRS)" -i /config/config.yml
/scripts/start-serving-app.sh
@@ -27,6 +27,7 @@
host = "0.0.0.0"
port = 9090
relayPeers = REPLACE_WITH_CERC_RELAY_PEERS
denyMultiaddrs = REPLACE_WITH_CERC_DENY_MULTIADDRS
peerIdFile = './peers/relay-id.json'
announce = 'REPLACE_WITH_CERC_RELAY_ANNOUNCE_DOMAIN'
enableDebugInfo = true
@@ -34,6 +35,7 @@
[server.p2p.peer]
relayMultiaddr = 'REPLACE_WITH_CERC_RELAY_MULTIADDR'
pubSubTopic = 'mobymask'
denyMultiaddrs = REPLACE_WITH_CERC_DENY_MULTIADDRS
peerIdFile = './peers/peer-id.json'
enableDebugInfo = true
enableL2Txs = REPLACE_WITH_CERC_ENABLE_PEER_L2_TXS
@@ -27,8 +27,8 @@ yarn_info_output=$(yarn info --json $versioned_target_package 2>/dev/null)
# If it doesn't exist there will be no .data.dist.tarball element,
# and jq will output the string "null"
package_tarball=$(echo $yarn_info_output | jq -r .data.dist.tarball)
if [[ $package_tarball == "null" ]]; then
echo "FATAL: Target package version ($versioned_target_package) not found" >&2
if [[ "$yarn_info_output" == "" || $package_tarball == "null" ]]; then
echo "FATAL: Target package version ($versioned_target_package) not found (or bad npm auth token)" >&2
exit 1
fi
# Code below parses out the values we need
@@ -6,7 +6,7 @@ RUN go install github.com/go-delve/delve/cmd/dlv@latest
FROM cerc/go-ethereum:local as geth
FROM alpine:latest
FROM alpine:3.17
RUN apk add --no-cache python3 python3-dev py3-pip curl wget jq build-base gettext libintl openssl bash bind-tools postgresql-client
COPY --from=delve /go/bin/dlv /usr/local/bin/
@@ -13,22 +13,26 @@ DEBUG_LEVEL=${1:-info}
echo "Starting bootnode"
if [ ! -f "$DATADIR/bootnode/enr.dat" ]; then
echo "Generating bootnode enr"
lcli \
generate-bootnode-enr \
--ip $ENR_IP \
--udp-port $BOOTNODE_PORT \
--tcp-port $BOOTNODE_PORT \
--genesis-fork-version $GENESIS_FORK_VERSION \
--output-dir $DATADIR/bootnode
bootnode_enr=`cat $DATADIR/bootnode/enr.dat`
echo "- $bootnode_enr" > $TESTNET_DIR/boot_enr.yaml
echo "Generated bootnode enr and written to $TESTNET_DIR/boot_enr.yaml"
# Clean up existing ENR dir to avoid node connectivity issues on a restart
if [ -d "$DATADIR/bootnode" ]; then
echo "Removing existing bootnode enr directory"
rm -r "$DATADIR/bootnode"
fi
echo "Generating bootnode enr"
lcli \
generate-bootnode-enr \
--ip $ENR_IP \
--udp-port $BOOTNODE_PORT \
--tcp-port $BOOTNODE_PORT \
--genesis-fork-version $GENESIS_FORK_VERSION \
--output-dir $DATADIR/bootnode
bootnode_enr=`cat $DATADIR/bootnode/enr.dat`
echo "- $bootnode_enr" > $TESTNET_DIR/boot_enr.yaml
echo "Generated bootnode enr and written to $TESTNET_DIR/boot_enr.yaml"
exec lighthouse boot_node \
--testnet-dir $TESTNET_DIR \
--port $BOOTNODE_PORT \
@@ -0,0 +1,138 @@
#####################################
FROM golang:1.19.7-buster AS lotus-builder
MAINTAINER Lotus Development Team
RUN apt-get update && apt-get install -y ca-certificates build-essential clang ocl-icd-opencl-dev ocl-icd-libopencl1 jq libhwloc-dev
ENV XDG_CACHE_HOME="/tmp"
### taken from https://github.com/rust-lang/docker-rust/blob/master/1.63.0/buster/Dockerfile
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH \
RUST_VERSION=1.63.0
RUN set -eux; \
dpkgArch="$(dpkg --print-architecture)"; \
case "${dpkgArch##*-}" in \
amd64) rustArch='x86_64-unknown-linux-gnu'; rustupSha256='5cc9ffd1026e82e7fb2eec2121ad71f4b0f044e88bca39207b3f6b769aaa799c' ;; \
arm64) rustArch='aarch64-unknown-linux-gnu'; rustupSha256='e189948e396d47254103a49c987e7fb0e5dd8e34b200aa4481ecc4b8e41fb929' ;; \
*) echo >&2 "unsupported architecture: ${dpkgArch}"; exit 1 ;; \
esac; \
url="https://static.rust-lang.org/rustup/archive/1.25.1/${rustArch}/rustup-init"; \
wget "$url"; \
echo "${rustupSha256} *rustup-init" | sha256sum -c -; \
chmod +x rustup-init; \
./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch}; \
rm rustup-init; \
chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \
rustup --version; \
cargo --version; \
rustc --version;
COPY ./ /opt/filecoin
WORKDIR /opt/filecoin
#RUN scripts/docker-git-state-check.sh
### make configurable filecoin-ffi build
ARG FFI_BUILD_FROM_SOURCE=0
ENV FFI_BUILD_FROM_SOURCE=${FFI_BUILD_FROM_SOURCE}
RUN make clean deps
ARG RUSTFLAGS=""
ARG GOFLAGS=""
#RUN make buildall
RUN make 2k
#####################################
FROM ubuntu:20.04 AS lotus-base
MAINTAINER Lotus Development Team
# Base resources
COPY --from=lotus-builder /etc/ssl/certs /etc/ssl/certs
COPY --from=lotus-builder /lib/*/libdl.so.2 /lib/
COPY --from=lotus-builder /lib/*/librt.so.1 /lib/
COPY --from=lotus-builder /lib/*/libgcc_s.so.1 /lib/
COPY --from=lotus-builder /lib/*/libutil.so.1 /lib/
COPY --from=lotus-builder /usr/lib/*/libltdl.so.7 /lib/
COPY --from=lotus-builder /usr/lib/*/libnuma.so.1 /lib/
COPY --from=lotus-builder /usr/lib/*/libhwloc.so.5 /lib/
COPY --from=lotus-builder /usr/lib/*/libOpenCL.so.1 /lib/
RUN useradd -r -u 532 -U fc \
&& mkdir -p /etc/OpenCL/vendors \
&& echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
#####################################
FROM lotus-base AS lotus
MAINTAINER Lotus Development Team
COPY --from=lotus-builder /opt/filecoin/lotus /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-shed /usr/local/bin/
#COPY scripts/docker-lotus-entrypoint.sh /
#COPY myscripts/setup-node.sh /docker-entrypoint-scripts.d/setup-node.sh
ARG DOCKER_LOTUS_IMPORT_SNAPSHOT https://snapshots.mainnet.filops.net/minimal/latest
ENV DOCKER_LOTUS_IMPORT_SNAPSHOT ${DOCKER_LOTUS_IMPORT_SNAPSHOT}
ENV FILECOIN_PARAMETER_CACHE /var/tmp/filecoin-proof-parameters
ENV LOTUS_PATH /var/lib/lotus
ENV DOCKER_LOTUS_IMPORT_WALLET ""
RUN mkdir /var/lib/lotus /var/tmp/filecoin-proof-parameters
RUN chown fc: /var/lib/lotus /var/tmp/filecoin-proof-parameters
VOLUME /var/lib/lotus
VOLUME /var/tmp/filecoin-proof-parameters
USER fc
EXPOSE 1234
ENTRYPOINT ["/docker-lotus-entrypoint.sh"]
CMD ["-help"]
#####################################
FROM lotus-base AS lotus-all-in-one
ENV FILECOIN_PARAMETER_CACHE /var/tmp/filecoin-proof-parameters
ENV LOTUS_MINER_PATH /var/lib/lotus-miner
ENV LOTUS_PATH /var/lib/lotus
ENV LOTUS_WORKER_PATH /var/lib/lotus-worker
ENV WALLET_PATH /var/lib/lotus-wallet
COPY --from=lotus-builder /opt/filecoin/lotus /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-seed /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-shed /usr/local/bin/
#COPY --from=lotus-builder /opt/filecoin/lotus-wallet /usr/local/bin/
#COPY --from=lotus-builder /opt/filecoin/lotus-gateway /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-miner /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-worker /usr/local/bin/
#COPY --from=lotus-builder /opt/filecoin/lotus-stats /usr/local/bin/
#COPY --from=lotus-builder /opt/filecoin/lotus-fountain /usr/local/bin/
RUN mkdir /var/tmp/filecoin-proof-parameters
RUN mkdir /var/lib/lotus
RUN mkdir /var/lib/lotus-miner
RUN mkdir /var/lib/lotus-worker
RUN mkdir /var/lib/lotus-wallet
RUN chown fc: /var/tmp/filecoin-proof-parameters
RUN chown fc: /var/lib/lotus
RUN chown fc: /var/lib/lotus-miner
RUN chown fc: /var/lib/lotus-worker
RUN chown fc: /var/lib/lotus-wallet
#VOLUME /var/tmp/filecoin-proof-parameters
#VOLUME /var/lib/lotus
#VOLUME /var/lib/lotus-miner
#VOLUME /var/lib/lotus-worker
#VOLUME /var/lib/lotus-wallet
EXPOSE 1234
EXPOSE 2345
EXPOSE 3456
EXPOSE 1777
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Build cerc/lotus
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
# Per lotus docs, 'releases' branch always contains latest stable release
git -C ${CERC_REPO_BASE_DIR}/lotus checkout releases
# Replace repo's Dockerfile with modified one
cp ${SCRIPT_DIR}/Dockerfile ${CERC_REPO_BASE_DIR}/lotus/Dockerfile
docker build -t cerc/lotus:local ${build_command_args} ${CERC_REPO_BASE_DIR}/lotus
@@ -50,9 +50,9 @@ RUN yarn global add http-server
# Globally install both versions of the payload web app package
# Install old version of MobyMask web app
RUN yarn global add @cerc-io/mobymask-ui@0.1.3
RUN yarn global add @cerc-io/mobymask-ui@0.1.4
# Install the LXDAO version of MobyMask web app
RUN yarn global add @cerc-io/mobymask-ui-lxdao@npm:@cerc-io/mobymask-ui@0.1.3-lxdao-0.1.1
RUN yarn global add @cerc-io/mobymask-ui-lxdao@npm:@cerc-io/mobymask-ui@0.1.4-lxdao-0.1.1
# Expose port for http
EXPOSE 80
@@ -33,7 +33,7 @@ do
echo "Substituting: ${template_string_to_replace} = ${template_value_to_substitute}"
# TODO: Pass keys to be replaced without double quotes
if [[ "$template_string_to_replace" =~ ^${config_prefix}_(relayNodes|chainId)$ ]]; then
if [[ "$template_string_to_replace" =~ ^${config_prefix}_(relayNodes|chainId|denyMultiaddrs)$ ]]; then
find ${webapp_files_dir} -type f -exec sed -i 's#"'"${template_string_to_replace}"'"#'"${template_value_to_substitute}"'#g' {} +
else
# Note: we do not escape our strings, on the expectation they do not container the '#' char.
@@ -21,7 +21,7 @@ RUN mkdir -p /config
RUN yarn global add http-server
# Globally install the payload web app package
RUN yarn global add @cerc-io/test-app@0.2.33
RUN yarn global add @cerc-io/test-app@0.2.34
# Expose port for http
EXPOSE 80
@@ -33,7 +33,7 @@ do
echo "Substituting: ${template_string_to_replace} = ${template_value_to_substitute}"
# TODO: Pass keys to be replaced without double quotes
if [[ "$template_string_to_replace" == "${config_prefix}_relayNodes" ]]; then
if [[ "$template_string_to_replace" =~ ^${config_prefix}_(relayNodes|denyMultiaddrs)$ ]]; then
find ${webapp_files_dir} -type f -exec sed -i 's#"'"${template_string_to_replace}"'"#'"${template_value_to_substitute}"'#g' {} +
else
# Note: we do not escape our strings, on the expectation they do not container the '#' char.
@@ -1,11 +1,14 @@
#!/bin/sh
#!/usr/bin/env bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Test if the container's filesystem is old (run previously) or new
EXISTSFILENAME=/var/exists
echo "Test container starting"
if [[ -f "$EXISTSFILENAME" ]];
then
TIMESTAMP = `cat $EXISTSFILENAME`
TIMESTAMP=`cat $EXISTSFILENAME`
echo "Filesystem is old, created: $TIMESTAMP"
else
echo "Filesystem is fresh"
@@ -0,0 +1,12 @@
FROM node:18.15.0-alpine3.16
RUN apk --update --no-cache add git python3 alpine-sdk bash
WORKDIR /app
COPY . .
RUN echo "Building gelato-watcher-ts" && \
yarn && yarn build
WORKDIR /app
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Build cerc/watcher-gelato
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/watcher-gelato:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/gelato-watcher-ts
@@ -15,6 +15,6 @@ WORKDIR /app
COPY . .
RUN echo "Building mobymask-v2-watcher-ts" && \
yarn && yarn build
yarn && yarn build
WORKDIR /app
+3
View File
@@ -38,3 +38,6 @@ cerc/optimism-op-node
cerc/optimism-op-proposer
cerc/pocket
cerc/watcher-azimuth
cerc/ipld-eth-state-snapshot
cerc/watcher-gelato
cerc/lotus
+2
View File
@@ -26,3 +26,5 @@ foundry
fixturenet-optimism
fixturenet-pocket
watcher-azimuth
watcher-gelato
fixturenet-lotus
+36 -32
View File
@@ -1,32 +1,36 @@
cerc-io/ipld-eth-db
cerc-io/go-ethereum
cerc-io/ipld-eth-server
cerc-io/eth-statediff-service
cerc-io/eth-statediff-fill-service
cerc-io/ipld-eth-db-validator
cerc-io/ipld-eth-beacon-indexer
cerc-io/ipld-eth-beacon-db
cerc-io/laconicd
cerc-io/laconic-sdk
cerc-io/laconic-registry-cli
cerc-io/laconic-console
cerc-io/mobymask-watcher
cerc-io/watcher-ts
cerc-io/mobymask-v2-watcher-ts
cerc-io/MobyMask
vulcanize/uniswap-watcher-ts
vulcanize/uniswap-v3-info
vulcanize/assemblyscript
cerc-io/eth-probe
cerc-io/tx-spammer
dboreham/foundry
lirewine/gem
lirewine/debug
lirewine/crypto
lirewine/sdk
telackey/act_runner
ethereum-optimism/op-geth
ethereum-optimism/optimism
pokt-network/pocket-core
pokt-network/pocket-core-deployments
cerc-io/azimuth-watcher-ts
github.com/cerc-io/ipld-eth-db
github.com/cerc-io/go-ethereum
github.com/cerc-io/ipld-eth-server
github.com/cerc-io/eth-statediff-service
github.com/cerc-io/eth-statediff-fill-service
github.com/cerc-io/ipld-eth-db-validator
github.com/cerc-io/ipld-eth-beacon-indexer
github.com/cerc-io/ipld-eth-beacon-db
github.com/cerc-io/laconicd
github.com/cerc-io/laconic-sdk
github.com/cerc-io/laconic-registry-cli
github.com/cerc-io/laconic-console
github.com/cerc-io/mobymask-watcher
github.com/cerc-io/watcher-ts
github.com/cerc-io/mobymask-v2-watcher-ts
github.com/cerc-io/MobyMask
github.com/vulcanize/uniswap-watcher-ts
github.com/vulcanize/uniswap-v3-info
github.com/vulcanize/assemblyscript
github.com/cerc-io/eth-probe
github.com/cerc-io/tx-spammer
github.com/dboreham/foundry
github.com/lirewine/gem
github.com/lirewine/debug
github.com/lirewine/crypto
github.com/lirewine/sdk
github.com/telackey/act_runner
github.com/ethereum-optimism/op-geth
github.com/ethereum-optimism/optimism
github.com/pokt-network/pocket-core
github.com/pokt-network/pocket-core-deployments
github.com/cerc-io/azimuth-watcher-ts
github.com/cerc-io/ipld-eth-state-snapshot
github.com/cerc-io/gelato-watcher-ts
github.com/filecoin-project/lotus
git.vdb.to/cerc-io/test-project
+2 -2
View File
@@ -4,7 +4,7 @@ Instructions to setup and deploy Azimuth Watcher stack
## Setup
Prerequisite: ipld-eth-server RPC and GQL endpoints
Prerequisite: `ipld-eth-server` RPC and GQL endpoints
Clone required repositories:
@@ -41,7 +41,7 @@ This should create the required docker images in the local image registry.
CERC_IPLD_ETH_GQL=
```
* NOTE: If ipld-eth-server is running on the host machine, use `host.docker.internal` as the hostname to access host ports
* NOTE: If `ipld-eth-server` is running on the host machine, use `host.docker.internal` as the hostname to access host ports
### Deploy the stack
+1 -1
View File
@@ -1,7 +1,7 @@
version: "1.0"
name: azimuth
repos:
- cerc-io/azimuth-watcher-ts
- github.com/cerc-io/azimuth-watcher-ts
containers:
- cerc/watcher-azimuth
pods:
+1 -1
View File
@@ -30,7 +30,7 @@ $ laconic-so --stack package-registry deploy up
⠿ Container laconic-aecc4a21d3a502b14522db97d427e850-server-1 Started 1.9s
New user 'gitea_admin' has been successfully created!
This is your gitea access token: 84fe66a73698bf11edbdccd0a338236b7d1d5c45. Keep it safe and secure, it can not be fetched again from gitea.
To use with laconic-so set this environment variable: export CERC_NPM_AUTH_TOKEN=3e493e77b3e83fe9e882f7e3a79dd4d5441c308b
To use with laconic-so set this environment variable: export CERC_NPM_AUTH_TOKEN=84fe66a73698bf11edbdccd0a338236b7d1d5c45
Created the organization cerc-io
Gitea was configured to use host name: gitea.local, ensure that this resolves to localhost, e.g. with sudo vi /etc/hosts
Success, gitea is properly initialized
+13
View File
@@ -0,0 +1,13 @@
version: "1.0"
name: chain-chunker
decription: "Stack to build containers for chain-chunker"
repos:
- github.com/cerc-io/ipld-eth-state-snapshot
- github.com/cerc-io/eth-statediff-service
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/ipld-eth-server
containers:
- cerc/ipld-eth-state-snapshot
- cerc/eth-statediff-service
- cerc/ipld-eth-db
- cerc/ipld-eth-server
+5 -5
View File
@@ -1,11 +1,11 @@
version: "1.0"
name: erc20-watcher
repos:
- cerc-io/go-ethereum
- cerc-io/ipld-eth-db
- cerc-io/ipld-eth-server
- cerc-io/watcher-ts
- dboreham/foundry
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/ipld-eth-server
- github.com/cerc-io/watcher-ts
- github.com/dboreham/foundry
containers:
- cerc/foundry
- cerc/go-ethereum
+4 -4
View File
@@ -1,10 +1,10 @@
version: "1.0"
name: erc721-watcher
repos:
- cerc-io/go-ethereum
- cerc-io/ipld-eth-db
- cerc-io/ipld-eth-server
- cerc-io/watcher-ts
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/ipld-eth-server
- github.com/cerc-io/watcher-ts
containers:
- cerc/go-ethereum
- cerc/go-ethereum-foundry
@@ -2,11 +2,11 @@ version: "1.0"
name: fixturenet-eth-loaded
decription: "Loaded Ethereum Fixturenet"
repos:
- cerc-io/go-ethereum
- cerc-io/tx-spammer
- cerc-io/ipld-eth-server
- cerc-io/ipld-eth-db
- cerc/go-ethereum
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/tx-spammer
- github.com/cerc-io/ipld-eth-server
- github.com/cerc-io/ipld-eth-db
- github.com/cerc-io/go-ethereum
containers:
- cerc/lighthouse
- cerc/fixturenet-eth-geth
+2 -2
View File
@@ -2,8 +2,8 @@ version: "1.2"
name: fixturenet-eth-tx
decription: "Ethereum Fixturenet w/ tx-spammer"
repos:
- cerc-io/go-ethereum
- cerc-io/tx-spammer
- github.com/cerc-io/go-ethereum
- github.com/cerc-io/tx-spammer
- dboreham/foundry
containers:
- cerc/go-ethereum
+1 -1
View File
@@ -66,7 +66,7 @@ It is not necessary to use them all at once, but a complete example follows:
```
# Setup
$ laconic-so setup-repositories --include cerc-io/go-ethereum,cerc-io/ipld-eth-db,cerc-io/ipld-eth-server,cerc-io/ipld-eth-beacon-db,cerc-io/ipld-eth-beacon-indexer,cerc-io/eth-probe,cerc-io/tx-spammer
$ laconic-so setup-repositories --include github.com/cerc-io/go-ethereum,github.com/cerc-io/ipld-eth-db,github.com/cerc-io/ipld-eth-server,github.com/cerc-io/ipld-eth-beacon-db,github.com/cerc-io/ipld-eth-beacon-indexer,github.com/cerc-io/eth-probe,github.com/cerc-io/tx-spammer
# Build
$ laconic-so build-containers --include cerc/go-ethereum,cerc/lighthouse,cerc/fixturenet-eth-geth,cerc/fixturenet-eth-lighthouse,cerc/ipld-eth-db,cerc/ipld-eth-server,cerc/ipld-eth-beacon-db,cerc/ipld-eth-beacon-indexer,cerc/eth-probe,cerc/keycloak,cerc/tx-spammer
+2 -2
View File
@@ -2,8 +2,8 @@ version: "1.1"
name: fixturenet-eth
decription: "Ethereum Fixturenet"
repos:
- cerc-io/go-ethereum
- dboreham/foundry
- github.com/cerc-io/go-ethereum
- github.com/dboreham/foundry
containers:
- cerc/go-ethereum
- cerc/lighthouse
@@ -2,14 +2,14 @@ version: "1.1"
name: fixturenet-laconic-loaded
description: "A full featured laconic fixturenet"
repos:
- cerc-io/laconicd
- lirewine/debug
- lirewine/crypto
- lirewine/gem
- lirewine/sdk
- cerc-io/laconic-sdk
- cerc-io/laconic-registry-cli
- cerc-io/laconic-console
- github.com/cerc-io/laconicd
- github.com/lirewine/debug
- github.com/lirewine/crypto
- github.com/lirewine/gem
- github.com/lirewine/sdk
- github.com/cerc-io/laconic-sdk
- github.com/cerc-io/laconic-registry-cli
- github.com/cerc-io/laconic-console
npms:
- laconic-sdk
- laconic-registry-cli
@@ -2,9 +2,9 @@ version: "1.0"
name: fixturenet-laconicd
description: "A laconicd fixturenet"
repos:
- cerc-io/laconicd
- cerc-io/laconic-sdk
- cerc-io/laconic-registry-cli
- github.com/cerc-io/laconicd
- github.com/cerc-io/laconic-sdk
- github.com/cerc-io/laconic-registry-cli
npms:
- laconic-sdk
- laconic-registry-cli
@@ -0,0 +1,28 @@
# Lotus Fixturenet
Instructions for deploying a local Lotus (Filecoin) chain for development and testing purposes using laconic-stack-orchestrator.
## 1. Clone required repositories
```
$ laconic-so --stack fixturenet-lotus setup-repositories
```
## 2. Build the stack's packages and containers
```
$ laconic-so --stack fixturenet-lotus build-containers
```
## 3. Deploy the stack
```
$ laconic-so --stack fixturenet-lotus deploy up
```
Correct operation should be verified by checking the container logs with:
```
$ laconic-so --stack fixturenet-lotus deploy logs lotus-miner
$ laconic-so --stack fixturenet-lotus deploy logs lotus-node-1
$ laconic-so --stack fixturenet-lotus deploy logs lotus-node-2
```
or by checking the chain status on each node:
```
$ laconic-so --stack fixturenet-lotus deploy exec lotus-miner "lotus status"
$ laconic-so --stack fixturenet-lotus deploy exec lotus-node-1 "lotus status"
$ laconic-so --stack fixturenet-lotus deploy exec lotus-node-2 "lotus status"
```
@@ -0,0 +1,9 @@
version: "1.0"
name: fixturenet-lotus
description: "A lotus fixturenet"
repos:
- github.com/filecoin-project/lotus
containers:
- cerc/lotus
pods:
- fixturenet-lotus
@@ -9,7 +9,7 @@ Prerequisite: An L1 Ethereum RPC endpoint
Clone required repositories:
```bash
laconic-so --stack fixturenet-optimism setup-repositories --exclude cerc-io/go-ethereum
laconic-so --stack fixturenet-optimism setup-repositories --exclude github.com/cerc-io/go-ethereum
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
@@ -2,10 +2,10 @@ version: "1.0"
name: fixturenet-optimism
decription: "Optimism Fixturenet"
repos:
- cerc-io/go-ethereum
- dboreham/foundry
- ethereum-optimism/optimism
- ethereum-optimism/op-geth
- github.com/cerc-io/go-ethereum
- github.com/dboreham/foundry
- github.com/ethereum-optimism/optimism
- github.com/ethereum-optimism/op-geth
containers:
- cerc/go-ethereum
- cerc/lighthouse
+3 -3
View File
@@ -2,9 +2,9 @@ version: "1.0"
name: fixturenet-pocket
description: "A single node pocket chain that can serve relays from the geth-1 node in eth-fixturenet"
repos:
- cerc-io/go-ethereum
- pokt-network/pocket-core
- pokt-network/pocket-core-deployments # contains the dockerfile
- github.com/cerc-io/go-ethereum
- github.com/pokt-network/pocket-core
- github.com/pokt-network/pocket-core-deployments # contains the dockerfile
containers:
- cerc/go-ethereum
- cerc/lighthouse
+98
View File
@@ -0,0 +1,98 @@
# Gelato watcher
Instructions to setup and deploy Gelato watcher using [laconic-stack-orchestrator](/README.md#install)
## Setup
Prerequisite: `ipld-eth-server` RPC and GQL endpoints
Clone required repositories:
```bash
laconic-so --stack gelato setup-repositories
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Checkout to required version in the repo:
```bash
# gelato-watcher-ts
cd ~/cerc/gelato-watcher-ts
git checkout v0.1.0
```
Build the container images:
```bash
laconic-so --stack gelato build-containers
```
This should create the required docker images in the local image registry.
## Deploy
### Configuration
Create and update an env file to be used in the next step ([defaults](../../config/watcher-gelato/watcher-params.env)):
```bash
# External ipld-eth-server endpoints
CERC_IPLD_ETH_RPC=
CERC_IPLD_ETH_GQL=
# Whether to use a state snapshot to initialize the watcher
CERC_USE_STATE_SNAPSHOT=false
# State snapshot params
# Required if CERC_USE_STATE_SNAPSHOT is set to true
CERC_SNAPSHOT_GQL_ENDPOINT=
CERC_SNAPSHOT_BLOCKHASH=
```
* NOTE: If `ipld-eth-server` is running on the host machine, use `host.docker.internal` as the hostname to access the host port(s)
### Deploy the stack
```bash
laconic-so --stack gelato deploy --env-file <PATH_TO_ENV_FILE> up
```
To list down and monitor the running containers:
```bash
laconic-so --stack gelato deploy ps
# With status
docker ps
# Check logs for a container
docker logs -f <CONTAINER_ID>
```
The stack runs an active watcher with following endpoints exposed on the host ports:
* `3008`: watcher endpoint
* `9000`: watcher metrics
* `9001`: watcher GQL metrics
## Web Apps
TODO
## Clean up
Stop all services running in the background:
```bash
laconic-so --stack gelato deploy down
```
Clear volumes created by this stack:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=.*gelato_watcher_db_data|.*gelato_watcher_state_gql"
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=.*gelato_watcher_db_data|.*gelato_watcher_state_gql")
```
+8
View File
@@ -0,0 +1,8 @@
version: "1.0"
name: gelato
repos:
- github.com/cerc-io/gelato-watcher-ts
containers:
- cerc/watcher-gelato
pods:
- watcher-gelato
+2 -2
View File
@@ -12,10 +12,10 @@ If running in the cloud, visit `IP:5001/webui` and you'll likely see this error:
1. Get the container name with `docker ps`:
2. Go into the container (replace with your container name):
2. Go into the container:
```
docker exec -it laconic-dbbf5498fd7d322930b9484121a6a5f4-ipfs-1 sh
laconic-so --stack kubo deploy exec ipfs sh
```
3. Enable CORS as described in point 2 of the error message. Copy/paste/run each line in sequence, then run `exit` to exit the container.
+2 -2
View File
@@ -23,11 +23,11 @@ Checkout to the required versions and branches in repos
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.39
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.0
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
+8 -5
View File
@@ -9,7 +9,7 @@ Prerequisite: L2 Optimism Geth and Node RPC endpoints
Clone required repositories:
```bash
laconic-so --stack mobymask-v2 setup-repositories --include cerc-io/MobyMask,cerc-io/watcher-ts,cerc-io/mobymask-v2-watcher-ts
laconic-so --stack mobymask-v2 setup-repositories --include github.com/cerc-io/MobyMask,github.com/cerc-io/watcher-ts,github.com/cerc-io/mobymask-v2-watcher-ts
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
@@ -19,11 +19,11 @@ Checkout to the required versions and branches in repos:
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.39
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.0
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
@@ -67,11 +67,14 @@ Create and update an env file to be used in the next step ([defaults](../../conf
# (used for generating a root invite link after deploying the contract)
CERC_MOBYMASK_APP_BASE_URI="http://127.0.0.1:3002/#"
# (Optional) Domain to be used in the relay node's announce address
CERC_RELAY_ANNOUNCE_DOMAIN=
# (Optional) Set of relay peers to connect to from the relay node
CERC_RELAY_PEERS=[]
# (Optional) Domain to be used in the relay node's announce address
CERC_RELAY_ANNOUNCE_DOMAIN=
# (Optional) Set of multiaddrs to be avoided while dialling
CERC_DENY_MULTIADDRS=[]
# Set to false for disabling watcher peer to send txs to L2
CERC_ENABLE_PEER_L2_TXS=true
+7 -7
View File
@@ -1,13 +1,13 @@
version: "1.0"
name: mobymask-v2
repos:
- cerc-io/go-ethereum
- dboreham/foundry
- ethereum-optimism/optimism
- ethereum-optimism/op-geth
- cerc-io/watcher-ts
- cerc-io/mobymask-v2-watcher-ts
- cerc-io/MobyMask
- github.com/cerc-io/go-ethereum
- github.com/dboreham/foundry
- github.com/ethereum-optimism/optimism
- github.com/ethereum-optimism/op-geth
- github.com/cerc-io/watcher-ts
- github.com/cerc-io/mobymask-v2-watcher-ts
- github.com/cerc-io/MobyMask
containers:
- cerc/go-ethereum
- cerc/lighthouse
@@ -14,7 +14,7 @@ This demo has been tested on a `Ubuntu 22.04 LTS` machine with `8GB` of RAM
Clone required repositories:
```bash
laconic-so --stack mobymask-v2 setup-repositories --include cerc-io/MobyMask,cerc-io/watcher-ts,cerc-io/mobymask-v2-watcher-ts
laconic-so --stack mobymask-v2 setup-repositories --include github.com/cerc-io/MobyMask,github.com/cerc-io/watcher-ts,github.com/cerc-io/mobymask-v2-watcher-ts
# This will clone the required repositories at ~/cerc
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned in the next step and re-run the command
@@ -35,11 +35,11 @@ Checkout to the required versions and branches in repos:
```bash
# watcher-ts
cd ~/cerc/watcher-ts
git checkout v0.2.39
git checkout v0.2.41
# mobymask-v2-watcher-ts
cd ~/cerc/mobymask-v2-watcher-ts
git checkout v0.1.0
git checkout v0.1.1
# MobyMask
cd ~/cerc/MobyMask
+3
View File
@@ -26,6 +26,9 @@ Create and update an env file to be used in the next step ([defaults](../../conf
# Eg. CERC_RELAY_NODES=["/dns4/example.com/tcp/443/wss/p2p/12D3KooWGHmDDCc93XUWL16FMcTPCGu2zFaMkf67k8HZ4gdQbRDr"]
CERC_RELAY_NODES=[]
# Set of multiaddrs to be avoided while dialling
CERC_DENY_MULTIADDRS=[]
# Also add if running MobyMask app:
# Watcher endpoint used by the app for GQL queries
+1 -1
View File
@@ -11,7 +11,7 @@ This deployment expects that ipld-eth-server's endpoints are available on the lo
## Clone required repositories
```
$ laconic-so setup-repositories --include cerc-io/watcher-ts
$ laconic-so setup-repositories --include github.com/cerc-io/watcher-ts
```
## Build the watcher container
+1 -1
View File
@@ -1,7 +1,7 @@
version: "1.0"
name: mobymask-watcher
repos:
- cerc-io/watcher-ts/v0.2.19
- github.com/cerc-io/watcher-ts/v0.2.19
containers:
- cerc/watcher-mobymask
pods:
+2 -2
View File
@@ -2,8 +2,8 @@ version: "1.1"
name: package-registry
decription: "Local Package Registry"
repos:
- cerc-io/hosting
- telackey/act_runner
- github.com/cerc-io/hosting
- gitea.com/gitea/act_runner
containers:
- cerc/act-runner
- cerc/act-runner-task-executor
+2 -1
View File
@@ -2,7 +2,8 @@ version: "1.0"
name: test
description: "A test stack"
repos:
- cerc-io/laconicd
- github.com/cerc-io/laconicd
- git.vdb.to/cerc-io/test-project
containers:
- cerc/test-container
pods:
+2 -2
View File
@@ -1,8 +1,8 @@
version: "1.0"
name: uniswap-v3
repos:
- vulcanize/uniswap-watcher-ts
- vulcanize/uniswap-v3-info
- github.com/vulcanize/uniswap-watcher-ts
- github.com/vulcanize/uniswap-v3-info
containers:
- cerc/watcher-uniswap-v3
- cerc/uniswap-v3-info
+128 -89
View File
@@ -28,108 +28,146 @@ import importlib.resources
from pathlib import Path
from .util import include_exclude_check, get_parsed_stack_config
class DeployCommandContext(object):
def __init__(self, cluster_context, docker):
self.cluster_context = cluster_context
self.docker = docker
@click.command()
@click.group()
@click.option("--include", help="only start these components")
@click.option("--exclude", help="don\'t start these components")
@click.option("--env-file", help="env file to be used")
@click.option("--cluster", help="specify a non-default cluster name")
@click.argument('command', required=True) # help: command: up|down|ps
@click.argument('extra_args', nargs=-1) # help: command: up|down|ps <service1> <service2>
@click.pass_context
def command(ctx, include, exclude, env_file, cluster, command, extra_args):
def command(ctx, include, exclude, env_file, cluster):
'''deploy a stack'''
# TODO: implement option exclusion and command value constraint lost with the move from argparse to click
debug = ctx.obj.debug
quiet = ctx.obj.quiet
verbose = ctx.obj.verbose
local_stack = ctx.obj.local_stack
dry_run = ctx.obj.dry_run
stack = ctx.obj.stack
cluster_context = _make_cluster_context(ctx.obj, include, exclude, cluster)
cluster_context = _make_cluster_context(ctx.obj, include, exclude, cluster, env_file)
# See: https://gabrieldemarmiesse.github.io/python-on-whales/sub-commands/compose/
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster, compose_env_file=env_file)
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster,
compose_env_file=cluster_context.env_file)
ctx.obj = DeployCommandContext(cluster_context, docker)
# Subcommand is executed now, by the magic of click
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: up <service1> <service2>
@click.pass_context
def up(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
cluster_context = ctx.obj.cluster_context
container_exec_env = _make_runtime_env(global_context)
for attr, value in container_exec_env.items():
os.environ[attr] = value
if global_context.verbose:
print(f"Running compose up with container_exec_env: {container_exec_env}, extra_args: {extra_args_list}")
for pre_start_command in cluster_context.pre_start_commands:
_run_command(global_context, cluster_context.cluster, pre_start_command)
ctx.obj.docker.compose.up(detach=True, services=extra_args_list)
for post_start_command in cluster_context.post_start_commands:
_run_command(global_context, cluster_context.cluster, post_start_command)
_orchestrate_cluster_config(global_context, cluster_context.config, ctx.obj.docker, container_exec_env)
if not dry_run:
if command == "up":
container_exec_env = _make_runtime_env(ctx.obj)
for attr, value in container_exec_env.items():
os.environ[attr] = value
if verbose:
print(f"Running compose up with container_exec_env: {container_exec_env}, extra_args: {extra_args_list}")
for pre_start_command in cluster_context.pre_start_commands:
_run_command(ctx.obj, cluster_context.cluster, pre_start_command)
docker.compose.up(detach=True, services=extra_args_list)
for post_start_command in cluster_context.post_start_commands:
_run_command(ctx.obj, cluster_context.cluster, post_start_command)
_orchestrate_cluster_config(ctx.obj, cluster_context.config, docker, container_exec_env)
@command.command()
@click.option("--delete-volumes/--preserve-volumes", default=False, help="delete data volumes")
@click.argument('extra_args', nargs=-1) # help: command: down<service1> <service2>
@click.pass_context
def down(ctx, delete_volumes, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
if global_context.verbose:
print("Running compose down")
timeout_arg = None
if extra_args_list:
timeout_arg = extra_args_list[0]
# Specify shutdown timeout (default 10s) to give services enough time to shutdown gracefully
ctx.obj.docker.compose.down(timeout=timeout_arg, volumes=delete_volumes)
elif command == "down":
if verbose:
print("Running compose down")
timeout_arg = None
if extra_args_list:
timeout_arg=extra_args_list[0]
@command.command()
@click.pass_context
def ps(ctx):
global_context = ctx.parent.parent.obj
if not global_context.dry_run:
if global_context.verbose:
print("Running compose ps")
container_list = ctx.obj.docker.compose.ps()
if len(container_list) > 0:
print("Running containers:")
for container in container_list:
print(f"id: {container.id}, name: {container.name}, ports: ", end="")
ports = container.network_settings.ports
comma = ""
for port_mapping in ports.keys():
mapping = ports[port_mapping]
print(comma, end="")
if mapping is None:
print(f"{port_mapping}", end="")
else:
print(f"{mapping[0]['HostIp']}:{mapping[0]['HostPort']}->{port_mapping}", end="")
comma = ", "
print()
else:
print("No containers running")
# Specify shutdown timeout (default 10s) to give services enough time to shutdown gracefully
docker.compose.down(timeout=timeout_arg)
elif command == "exec":
if extra_args_list is None or len(extra_args_list) < 2:
print("Usage: exec <service> <cmd>")
sys.exit(1)
service_name = extra_args_list[0]
command_to_exec = ["sh", "-c"] + extra_args_list[1:]
container_exec_env = _make_runtime_env(ctx.obj)
if verbose:
print(f"Running compose exec {service_name} {command_to_exec}")
try:
docker.compose.execute(service_name, command_to_exec, envs=container_exec_env)
except DockerException as error:
print(f"container command returned error exit status")
elif command == "port":
if extra_args_list is None or len(extra_args_list) < 2:
print("Usage: port <service> <exposed-port>")
sys.exit(1)
service_name = extra_args_list[0]
exposed_port = extra_args_list[1]
if verbose:
print(f"Running compose port {service_name} {exposed_port}")
mapped_port_data = docker.compose.port(service_name, exposed_port)
print(f"{mapped_port_data[0]}:{mapped_port_data[1]}")
elif command == "ps":
if verbose:
print("Running compose ps")
container_list = docker.compose.ps()
if len(container_list) > 0:
print("Running containers:")
for container in container_list:
print(f"id: {container.id}, name: {container.name}, ports: ", end="")
ports = container.network_settings.ports
comma = ""
for port_mapping in ports.keys():
mapping = ports[port_mapping]
print(comma, end="")
if mapping is None:
print(f"{port_mapping}", end="")
else:
print(f"{mapping[0]['HostIp']}:{mapping[0]['HostPort']}->{port_mapping}", end="")
comma = ", "
print()
else:
print("No containers running")
elif command == "logs":
if verbose:
print("Running compose logs")
logs_output = docker.compose.logs(services=extra_args_list if extra_args_list is not None else [])
print(logs_output)
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: port <service1> <service2>
@click.pass_context
def port(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
if extra_args_list is None or len(extra_args_list) < 2:
print("Usage: port <service> <exposed-port>")
sys.exit(1)
service_name = extra_args_list[0]
exposed_port = extra_args_list[1]
if global_context.verbose:
print(f"Running compose port {service_name} {exposed_port}")
mapped_port_data = ctx.obj.docker.compose.port(service_name, exposed_port)
print(f"{mapped_port_data[0]}:{mapped_port_data[1]}")
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: exec <service> <command>
@click.pass_context
def exec(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
if extra_args_list is None or len(extra_args_list) < 2:
print("Usage: exec <service> <cmd>")
sys.exit(1)
service_name = extra_args_list[0]
command_to_exec = ["sh", "-c"] + extra_args_list[1:]
container_exec_env = _make_runtime_env(global_context)
if global_context.verbose:
print(f"Running compose exec {service_name} {command_to_exec}")
try:
ctx.obj.docker.compose.execute(service_name, command_to_exec, envs=container_exec_env)
except DockerException as error:
print(f"container command returned error exit status")
@command.command()
@click.argument('extra_args', nargs=-1) # help: command: logs <service1> <service2>
@click.pass_context
def logs(ctx, extra_args):
global_context = ctx.parent.parent.obj
extra_args_list = list(extra_args) or None
if not global_context.dry_run:
if global_context.verbose:
print("Running compose logs")
logs_output = ctx.obj.docker.compose.logs(services=extra_args_list if extra_args_list is not None else [])
print(logs_output)
def get_stack_status(ctx, stack):
@@ -137,7 +175,7 @@ def get_stack_status(ctx, stack):
ctx_copy = copy.copy(ctx)
ctx_copy.stack = stack
cluster_context = _make_cluster_context(ctx_copy, None, None, None)
cluster_context = _make_cluster_context(ctx_copy, None, None, None, None)
docker = DockerClient(compose_files=cluster_context.compose_files, compose_project_name=cluster_context.cluster)
# TODO: refactor to avoid duplicating this code above
if ctx.verbose:
@@ -162,7 +200,7 @@ def _make_runtime_env(ctx):
return container_exec_env
def _make_cluster_context(ctx, include, exclude, cluster):
def _make_cluster_context(ctx, include, exclude, cluster, env_file):
if ctx.local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
@@ -235,16 +273,17 @@ def _make_cluster_context(ctx, include, exclude, cluster):
if ctx.verbose:
print(f"files: {compose_files}")
return cluster_context(cluster, compose_files, pre_start_commands, post_start_commands, cluster_config)
return cluster_context(cluster, compose_files, pre_start_commands, post_start_commands, cluster_config, env_file)
class cluster_context:
def __init__(self, cluster, compose_files, pre_start_commands, post_start_commands, config) -> None:
def __init__(self, cluster, compose_files, pre_start_commands, post_start_commands, config, env_file) -> None:
self.cluster = cluster
self.compose_files = compose_files
self.pre_start_commands = pre_start_commands
self.post_start_commands = post_start_commands
self.config = config
self.env_file = env_file
def _convert_to_new_format(old_pod_array):
+16 -5
View File
@@ -125,11 +125,22 @@ def command(ctx, include, exclude, git_ssh, check_only, pull, branches_file):
if verbose:
print(f"Excluding: {repo}")
def process_repo(repo):
git_ssh_prefix = "git@github.com:"
git_http_prefix = "https://github.com/"
full_github_repo_path = f"{git_ssh_prefix if git_ssh else git_http_prefix}{repo}"
repoName = repo.split("/")[-1]
def host_and_path_for_repo(fully_qualified_repo):
repo_split = fully_qualified_repo.split("/")
# Legacy unqualified repo means github
if len(repo_split) == 2:
return "github.com", "/".join(repo_split)
else:
if len(repo_split) == 3:
# First part is the host
return repo_split[0], "/".join(repo_split[1:])
def process_repo(fully_qualified_repo):
repo_host, repo_path = host_and_path_for_repo(fully_qualified_repo)
git_ssh_prefix = f"git@{repo_host}:"
git_http_prefix = f"https://{repo_host}/"
full_github_repo_path = f"{git_ssh_prefix if git_ssh else git_http_prefix}{repo_path}"
repoName = repo_path.split("/")[-1]
full_filesystem_repo_path = os.path.join(dev_root_path, repoName)
is_present = os.path.isdir(full_filesystem_repo_path)
if not quiet:
+10 -3
View File
@@ -26,7 +26,7 @@ In addition to the pre-requisites listed in the [README](/README.md), the follow
1. Clone this repository:
```
$ git clone (https://github.com/cerc-io/stack-orchestrator.git
$ git clone https://github.com/cerc-io/stack-orchestrator.git
```
2. Enter the project directory:
@@ -87,20 +87,27 @@ Use shiv to build a single file Python executable zip archive of laconic-so:
```
$ cp stack-orchetrator/laconic-so ~/bin
$ laconic-so
Usage: python -m laconic-so [OPTIONS] COMMAND [ARGS]...
Usage: laconic-so [OPTIONS] COMMAND [ARGS]...
Laconic Stack Orchestrator
Options:
--stack TEXT specify a stack to build/deploy
--quiet
--verbose
--dry-run
-h, --help Show this message and exit.
--local-stack
--debug
--continue-on-error
-h, --help Show this message and exit.
Commands:
build-containers build the set of containers required for a complete...
build-npms build the set of npm packages required for a...
deploy deploy a stack
deploy-system deploy a stack
setup-repositories git clone the set of repositories required to build...
version print tool version
```
For cutting releases, use the [shiv build script](/scripts/build_shiv_package.sh).
+7 -1
View File
@@ -162,7 +162,13 @@ Published demo-record-1.yml with id: bafyreierh3xnfivexlscdwubvczmddsnf46uytyfvr
The sample record we deployed looks like:
```
TODO
record:
type: WebsiteRegistrationRecord
url: 'https://cerc.io'
repo_registration_record_cid: QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D
build_artifact_cid: QmP8jTG1m9GSDJLCbeWhVSVgEzCPPwXRdCRuJtQ5Tz9Kc9
tls_cert_cid: QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
version: 1.0.23
```
2. Return to the laconic-console
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Dump environment variables for debugging
echo "Environment variables:"
env
# Test basic stack-orchestrator deploy
echo "Running stack-orchestrator deploy test"
# Bit of a hack, test the most recent package
TEST_TARGET_SO=$( ls -t1 ./package/laconic-so* | head -1 )
# Set a non-default repo dir
export CERC_REPO_BASE_DIR=~/stack-orchestrator-test/repo-base-dir
echo "Testing this package: $TEST_TARGET_SO"
echo "Test version command"
reported_version_string=$( $TEST_TARGET_SO version )
echo "Version reported is: ${reported_version_string}"
echo "Cloning repositories into: $CERC_REPO_BASE_DIR"
rm -rf $CERC_REPO_BASE_DIR
mkdir -p $CERC_REPO_BASE_DIR
# Test bringing the test container up and down
# with and without volume removal
$TEST_TARGET_SO --stack test setup-repositories
$TEST_TARGET_SO --stack test build-containers
$TEST_TARGET_SO --stack test deploy up
# Test deploy port command
deploy_port_output=$( $TEST_TARGET_SO --stack test deploy port test 80 )
if [[ "$deploy_port_output" =~ ^0.0.0.0:[1-9][0-9]* ]]; then
echo "Deploy port test: passed"
else
echo "Deploy port test: FAILED"
exit 1
fi
$TEST_TARGET_SO --stack test deploy down
# The next time we bring the container up the volume will be old (from the previous run above)
$TEST_TARGET_SO --stack test deploy up
log_output_1=$( $TEST_TARGET_SO --stack test deploy logs )
if [[ "$log_output_1" == *"Filesystem is old"* ]]; then
echo "Retain volumes test: passed"
else
echo "Retain volumes test: FAILED"
exit 1
fi
$TEST_TARGET_SO --stack test deploy down --delete-volumes
# Now when we bring the container up the volume will be new again
$TEST_TARGET_SO --stack test deploy up
log_output_2=$( $TEST_TARGET_SO --stack test deploy logs )
if [[ "$log_output_2" == *"Filesystem is fresh"* ]]; then
echo "Delete volumes test: passed"
else
echo "Delete volumes test: FAILED"
exit 1
fi
$TEST_TARGET_SO --stack test deploy down --delete-volumes
echo "Test passed"