Compare commits

...
18 Commits
Author SHA1 Message Date
dboreham 8a4f189286 Merge pull request #234 from cerc-io/dboreham/lirewine-builds
Add support for lirewine npm package build and consumption

Former-commit-id: ebe1dfa4bf
2023-03-24 02:54:21 -06:00
dboreham aa01feb98a Fix up script and npm list
Former-commit-id: e6b91acdea
2023-03-24 02:52:37 -06:00
dboreham c4e3aa54f1 Remove trailing whitespace
Former-commit-id: 2e7ef1e52e
2023-03-24 02:31:19 -06:00
dboreham 364a71d694 Support the @lirewine npm scope
Former-commit-id: 9e322203e9
2023-03-24 02:30:04 -06:00
dboreham d96a6def14 Add lirewine repos and npms
Former-commit-id: 870a987760
2023-03-24 02:08:56 -06:00
dboreham 1a75a270f5 Merge pull request #232 from cerc-io/dboreham/foundry-debian-container
Foundry debian-based container

Former-commit-id: 76029d24de
2023-03-23 17:46:42 -06:00
David Boreham 23fab04ec8 Fix filename
Former-commit-id: 06288fd9e1
2023-03-23 16:09:27 -06:00
David Boreham 4420f45924 Build foundry with Debian base container
Former-commit-id: 21e9f96771
2023-03-23 16:03:35 -06:00
dboreham dac22b19b7 Update version
Former-commit-id: d5fd880bba
2023-03-22 21:35:48 -06:00
dboreham 08ed69e4cd Merge pull request #227 from cerc-io/dboreham/fix-fixturenet-laconicd
Update fixturenet create script

Former-commit-id: 5fca9bdac7
2023-03-22 15:01:59 -06:00
David Boreham 84bad13934 Update fixturenet create script
Former-commit-id: 01c6999027
2023-03-22 14:52:39 -06:00
dboreham c537d787c2 Update version
Former-commit-id: 91ba0ae011
2023-03-21 09:55:40 -06:00
Zach 538bccd19a Merge pull request #225 from cerc-io/dboreham/update-readme
Update README to reflect user experience and to use --stack

Former-commit-id: 8fb93758ee
2023-03-21 09:25:53 -04:00
David Boreham d5f5ecceda Update README to reflect user experience and to use --stack directive in examples
Former-commit-id: 2c151cb8ce
2023-03-20 19:36:05 -06:00
dboreham 39741b2e8f Add Gitea note
Former-commit-id: 74be51b892
2023-03-20 10:14:58 -06:00
dboreham 2ce04afd1e Merge pull request #224 from cerc-io/dboreham/update-registry-doc
Update registry deploy instructions

Former-commit-id: 33bbbccef7
2023-03-20 10:09:49 -06:00
dboreham 48a761875a Update registry deploy instructions
Former-commit-id: ea6f28a3de
2023-03-20 10:08:59 -06:00
dboreham 6ee7c3bbba Update version
Former-commit-id: 4c92391faf
2023-03-20 08:21:41 -06:00
10 changed files with 85 additions and 68 deletions
+16 -29
View File
@@ -8,11 +8,10 @@ Stack Orchestrator allows building and deployment of a Laconic Stack on a single
Ensure that the following are already installed:
- [Python3](https://wiki.python.org/moin/BeginnersGuide/Download): `python3 --version` >= `3.10.8`
- [Python3](https://wiki.python.org/moin/BeginnersGuide/Download): `python3 --version` >= `3.8.10` (the Python3 shipped in Ubuntu 20+ is good to go)
- [Docker](https://docs.docker.com/get-docker/): `docker --version` >= `20.10.21`
- [Docker Compose](https://docs.docker.com/compose/install/): `docker-compose --version` >= `2.13.0`
Note: if installing docker-compose via package manager (as opposed to Docker Desktop), you must [install the plugin](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually), e.g., on Linux:
Note: if installing docker-compose via package manager on Linux (as opposed to Docker Desktop), you must [install the plugin](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually), e.g. :
```bash
mkdir -p ~/.docker/cli-plugins
@@ -20,39 +19,27 @@ curl -SL https://github.com/docker/compose/releases/download/v2.11.2/docker-comp
chmod +x ~/.docker/cli-plugins/docker-compose
```
Next, download the latest release from [this page](https://github.com/cerc-io/stack-orchestrator/tags), into a suitable directory (e.g. `~/bin`):
Next decide on a directory where you would like to put the stack-orchestrator program. Typically this would be
a "user" binary directory such as `~/bin` or perhaps `/usr/local/laconic` or possibly just the current working directory.
Now, having selected that directory, download the latest release from [this page](https://github.com/cerc-io/stack-orchestrator/tags) into it (we're using `~/bin` below for concreteness but edit to suit if you selected a different directory). Also be sure that the destination directory exists and is writable:
```bash
curl -L -o ~/bin/laconic-so https://github.com/cerc-io/stack-orchestrator/releases/latest/download/laconic-so
```
Give it permissions:
Give it execute permissions:
```bash
chmod +x ~/bin/laconic-so
```
Ensure `laconic-so` is on the [`PATH`](https://unix.stackexchange.com/a/26059)
Verify operation:
Verify operation (your version will probably be different, just check here that you see some version outut and not an error):
```
laconic-so --help
Usage: python -m laconic-so [OPTIONS] COMMAND [ARGS]...
Laconic Stack Orchestrator
Options:
--quiet
--verbose
--dry-run
--local-stack
-h, --help Show this message and exit.
Commands:
build-containers build the set of containers required for a complete...
build-npms build the set of npm packages required for a...
deploy-system deploy a stack
setup-repositories git clone the set of repositories required to build...
laconic-so version
Version: v1.0.27-7831078
```
## Usage
@@ -64,25 +51,25 @@ Three sub-commands: `setup-repositories`, `build-containers` and `deploy-system`
Clone the set of git repositories necessary to build a system:
```bash
laconic-so --verbose setup-repositories --include cerc-io/go-ethereum,cerc-io/ipld-eth-db,cerc-io/ipld-eth-server,cerc-io/watcher-ts
laconic-so --stack erc20 setup-repositories
```
This will default to `~/cerc` or - if set - the environment variable `CERC_REPO_BASE_DIR`
This will default to cloning git reposiories into: `~/cerc` or - if set - the environment variable `CERC_REPO_BASE_DIR`
### Build Containers
Build the set of docker container images required to run a system. It takes around 10 minutes to build all the containers from scratch.
```bash
laconic-so --verbose build-containers --include cerc/go-ethereum,cerc/go-ethereum-foundry,cerc/ipld-eth-db,cerc/ipld-eth-server,cerc/watcher-erc20
laconic-so --stack erc20 build-containers
```
### Deploy System
Uses `docker-compose` to deploy a system (with most recently built container images).
Uses `docker compose` to deploy a system (with most recently built container images).
```bash
laconic-so --verbose deploy-system --include ipld-eth-db,go-ethereum-foundry,ipld-eth-server,watcher-erc20 up
laconic-so --stack erc20 deploy-system up
```
Check out he GraphQL playground here: [http://localhost:3002/graphql](http://localhost:3002/graphql)
@@ -92,7 +79,7 @@ See the [erc20 watcher demo](/app/data/stacks/erc20) to continue further.
### Cleanup
```bash
laconic-so --verbose deploy-system --include ipld-eth-db,go-ethereum-foundry,ipld-eth-server,watcher-erc20 down
laconic-so --stack erc20 deploy-system down
```
## Contributing
@@ -1,8 +1,8 @@
#!/bin/sh
# Originally from: https://github.com/cerc-io/laconicd/blob/main/init.sh
# TODO: fold this back into the laconicd repo
#!/bin/bash
# TODO: this file is now an unmodified copy of cerc-io/laconicd/init.sh
# so we should have a mechanism to bundle it inside the container rather than link from here
# at deploy time.
KEY="mykey"
CHAINID="laconic_9000-1"
@@ -10,7 +10,7 @@ MONIKER="localtestnet"
KEYRING="test"
KEYALGO="eth_secp256k1"
LOGLEVEL="info"
# to trace evm
# trace evm
TRACE="--trace"
# TRACE=""
@@ -28,7 +28,7 @@ laconicd config chain-id $CHAINID
# if $KEY exists it should be deleted
laconicd keys add $KEY --keyring-backend $KEYRING --algo $KEYALGO
# Set moniker and chain-id for laconic (Moniker can be anything, chain-id must be an integer)
# Set moniker and chain-id for Ethermint (Moniker can be anything, chain-id must be an integer)
laconicd init $MONIKER --chain-id $CHAINID
# Change parameter token denominations to aphoton
@@ -37,28 +37,28 @@ cat $HOME/.laconicd/config/genesis.json | jq '.app_state["crisis"]["constant_fee
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["gov"]["deposit_params"]["min_deposit"][0]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["mint"]["params"]["mint_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Custom modules
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["record_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_commit_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_reveal_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_minimum_bid"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commit_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveal_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_minimum_bid"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
if [[ "$TEST_NAMESERVICE_EXPIRY" == "true" ]]; then
if [[ "$TEST_REGISTRY_EXPIRY" == "true" ]]; then
echo "Setting timers for expiry tests."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
if [[ "$TEST_AUCTION_ENABLED" == "true" ]]; then
echo "Enabling auction and setting timers."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["nameservice"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
# increase block time (?)
@@ -1,7 +1,7 @@
#!/bin/bash
# Usage: build-npm-package-local-dependencies.sh <registry-url> <publish-with-this-version>
# Runs build-npm-package.sh after first fixing up yarn.lock to use a local
# npm registry for all packages in a spcific scope (currently @cerc-io)
# npm registry for all packages in a spcific scope (currently @cerc-io and @lirewine)
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
@@ -17,18 +17,21 @@ fi
set -e
local_npm_registry_url=$1
package_publish_version=$2
# TODO: make this a paramater and allow a list of scopes
npm_scope_for_local="@cerc-io"
# We need to configure the local registry
npm config set ${npm_scope_for_local}:registry ${local_npm_registry_url}
npm config set -- ${local_npm_registry_url}:_authToken ${CERC_NPM_AUTH_TOKEN}
# Find the set of dependencies from the specified scope
mapfile -t dependencies_from_scope < <(cat package.json | jq -r '.dependencies | with_entries(if (.key|test("^'${npm_scope_for_local}'/.*$")) then ( {key: .key, value: .value } ) else empty end ) | keys[]')
echo "Fixing up dependencies"
for package in "${dependencies_from_scope[@]}"
# If we need to handle an additional scope, add it to the list below:
npm_scopes_to_handle=("@cerc-io" "@lirewine")
for npm_scope_for_local in ${npm_scopes_to_handle[@]}
do
echo "Fixing up package ${package}"
yarn-local-registry-fixup.sh $package ${local_npm_registry_url}
# We need to configure the local registry
npm config set ${npm_scope_for_local}:registry ${local_npm_registry_url}
npm config set -- ${local_npm_registry_url}:_authToken ${CERC_NPM_AUTH_TOKEN}
# Find the set of dependencies from the specified scope
mapfile -t dependencies_from_scope < <(cat package.json | jq -r '.dependencies | with_entries(if (.key|test("^'${npm_scope_for_local}'/.*$")) then ( {key: .key, value: .value } ) else empty end ) | keys[]')
echo "Fixing up dependencies in scope ${npm_scope_for_local}"
for package in "${dependencies_from_scope[@]}"
do
echo "Fixing up package ${package}"
yarn-local-registry-fixup.sh $package ${local_npm_registry_url}
done
done
echo "Running build"
build-npm-package.sh ${local_npm_registry_url} ${package_publish_version}
@@ -22,8 +22,8 @@ set -e
# Get the name of this package from package.json since we weren't passed that
package_name=$( cat package.json | jq -r .name )
local_npm_registry_url=$1
npm config set @lirewine:registry ${local_npm_registry_url}
npm config set @cerc-io:registry ${local_npm_registry_url}
npm config set @lirewine:registry ${local_npm_registry_url}
npm config set -- ${local_npm_registry_url}:_authToken ${CERC_NPM_AUTH_TOKEN}
# First check if the version of this package we're trying to build already exists in the registry
package_exists=$( yarn info --json ${package_name}@${package_publish_version} 2>/dev/null | jq -r .data.dist.tarball )
@@ -1,3 +1,3 @@
#!/usr/bin/env bash
# Build a local version of the foundry-rs/foundry image
docker build -t cerc/foundry:local ${CERC_REPO_BASE_DIR}/foundry
docker build -t cerc/foundry:local -f ${CERC_REPO_BASE_DIR}/foundry/Dockerfile-debian ${CERC_REPO_BASE_DIR}/foundry
@@ -40,8 +40,7 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# RUN su node -c "npm install -g <your-package-list-here>"
# Configure the local npm registry
RUN npm config set @lirewine:registry ${CERC_NPM_URL} \
&& npm config set @cerc-io:registry ${CERC_NPM_URL} \
RUN npm config set @cerc-io:registry ${CERC_NPM_URL} \
&& npm config set @lirewine:registry ${CERC_NPM_URL} \
&& npm config set -- ${CERC_NPM_URL}:_authToken ${CERC_NPM_AUTH_TOKEN}
+3 -2
View File
@@ -1,4 +1,5 @@
gem
laconic-sdk
debug
laconic-registry-cli
gem
debug
crypto
+3
View File
@@ -19,3 +19,6 @@ vulcanize/assemblyscript
cerc-io/eth-probe
cerc-io/tx-spammer
dboreham/foundry
lirewine/gem
lirewine/debug
lirewine/crypto
+24
View File
@@ -24,8 +24,32 @@ $ laconic-so --stack build-support build-containers --exclude cerc/builder-gerbi
```
$ laconic-so --stack package-registry setup-repositories
$ laconic-so --stack package-registry deploy up
[+] Running 3/3
⠿ Network laconic-aecc4a21d3a502b14522db97d427e850_gitea Created 0.0s
⠿ Container laconic-aecc4a21d3a502b14522db97d427e850-db-1 Started 1.2s
⠿ Container laconic-aecc4a21d3a502b14522db97d427e850-server-1 Started 1.9s
New user 'gitea_admin' has been successfully created!
This is your gitea access token: 84fe66a73698bf11edbdccd0a338236b7d1d5c45. Keep it safe and secure, it can not be fetched again from gitea.
To use with laconic-so set this environment variable: export CERC_NPM_AUTH_TOKEN=3e493e77b3e83fe9e882f7e3a79dd4d5441c308b
Created the organization cerc-io
Gitea was configured to use host name: gitea.local, ensure that this resolves to localhost, e.g. with sudo vi /etc/hosts
Success, gitea is properly initialized
$
```
### 3. Configure the hostname gitea.local
How to do this is OS-dependent but usually involves editing a `hosts` file. For example on Linux add this line to the file `/etc/hosts` (needs sudo):
```
127.0.0.1 gitea.local
```
Test with:
```
$ ping gitea.local
PING gitea.local (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.147 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.033 ms
```
Although not necessary in order to build and publish packages, you can now access the Gitea web interface at: [http://gitea.local:3000](http://gitea.local:3000) using these credentials: gitea_admin/admin1234 (Note: please properly secure Gitea if public internet access is allowed).
Now npm packages can be built:
### Build npm Packages
Ensure that `CERC_NPM_AUTH_TOKEN` is set with the token printed above when the package-registry stack was deployed (the actual token value will be different than shown in this example):
+1 -1
View File
@@ -1,2 +1,2 @@
# This file should be re-generated running: scripts/update-version-file.sh script
v1.0.26-ee77835
v1.0.29-5fca9bd