1
0
stack-orchestrator/README.md

154 lines
5.2 KiB
Markdown
Raw Normal View History

2022-08-24 20:31:44 +00:00
# Stack Orchestrator
2022-08-22 02:26:55 +00:00
2022-08-24 20:48:55 +00:00
Stack Orchestrator allows building and deployment of a Laconic stack on a single machine with minimial prerequisites.
2022-08-24 20:31:44 +00:00
## Setup
### Developer Mode
Developer mode runs the orchestrator from a cloned git repository.
#### Prerequisites
2022-08-24 20:48:55 +00:00
Stack Orchestrator is a Python3 CLI tool that runs on any OS with Python3 and Docker. Tested on: Ubuntu 20/22.
Ensure that the following are already installed:
1. Python3 (the version 3.8 available in Ubuntu 20/22 works)
```
$ python3 --version
Python 3.8.10
```
1. Docker (Install a current version from dockerco, don't use the version from any Linux distro)
```
$ docker --version
Docker version 20.10.17, build 100c701
```
2022-08-24 20:31:44 +00:00
#### Install
1. Clone this repository:
```
$ git clone (https://github.com/cerc-io/stack-orchestrator.git
```
1. Enter the project directory:
```
$ cd stack-orchestrator
```
1. Create and activate a venv:
```
$ python3 -m venv venv
$ source ./venv/bin/activate
(venv) $
```
1. Install the cli in edit mode:
```
$ pip install --editable .
```
1. Verify installation:
```
(venv) $ laconic-so
Usage: laconic-so [OPTIONS] COMMAND [ARGS]...
Laconic Stack Orchestrator
Options:
--quiet
--verbose
--dry-run
-h, --help Show this message and exit.
Commands:
build-containers build the set of containers required for a complete...
deploy-system deploy a stack
setup-repositories git clone the set of repositories required to build...
```
2022-09-27 21:40:00 +00:00
#### Build a zipapp (single file distributable script)
Use shiv to build a single file Python executable zip archive of laconic-so:
1. Install [shiv](https://github.com/linkedin/shiv):
```
$ (venv) pip install shiv
```
1. Run shiv to create a zipapp file:
```
$ (venv) shiv -c laconic-so -o laconic-so .
```
This creates a file `./laconic-so` that is executable outside of any venv, and on other machines and OSes and architectures, and requiring only the system Python3:
1. Verify it works:
```
$ cp stack-orchetrator/laconic-so ~/bin
$ laconic-so
Usage: python -m laconic-so [OPTIONS] COMMAND [ARGS]...
Laconic Stack Orchestrator
Options:
--quiet
--verbose
--dry-run
-h, --help Show this message and exit.
Commands:
build-containers build the set of containers required for a complete...
deploy-system deploy a stack
setup-repositories git clone the set of repositories required to build...
```
2022-08-24 20:31:44 +00:00
### CI Mode
_write-me_
## Usage
There are three sub-commands: `setup-repositories`, `build-containers` and `deploy-system` that are generally run in order:
### Setup Repositories
Clones the set of git repositories necessary to build a system.
2022-08-24 20:48:55 +00:00
Note: the use of `ssh-agent` is recommended in order to avoid entering your ssh key passphrase for each repository.
2022-08-24 20:31:44 +00:00
```
$ laconic-so --verbose setup-repositories
```
### Build Containers
2022-08-24 20:48:55 +00:00
Builds the set of docker container images required to run a system. It takes around 10 minutes to build all the containers from cold.
2022-08-22 02:26:55 +00:00
```
2022-08-24 20:31:44 +00:00
$ laconic-so --verbose build-containers
```
### Deploy System
Uses `docker compose` to deploy a system.
2022-08-24 20:48:55 +00:00
Use `---include <list of components>` to deploy a subset of all containers:
2022-08-24 20:31:44 +00:00
```
$ laconic-so --verbose deploy-system --include db-sharding,contract,ipld-eth-server,go-ethereum-foundry up
```
2022-08-24 20:48:55 +00:00
```
$ laconic-so --verbose deploy-system --include db-sharding,contract,ipld-eth-server,go-ethereum-foundry down
```
2022-09-27 21:40:00 +00:00
## Platform Support
Native aarm64 is _not_ currently supported. x64 emulation on ARM64 macos should work (not yet tested).
2022-08-24 20:31:44 +00:00
## Implementation
2022-08-24 20:48:55 +00:00
The orchestrator's operation is driven by files shown below. `repository-list.txt` container the list of git repositories; `container-image-list.txt` contains
the list of container image names, while `clister-list.txt` specifies the set of compose components (corresponding to individual docker-compose-xxx.yml files which may in turn specify more than one container).
Files required to build each container image are stored under `./container-build/<container-name>`
Files required at deploy-time are stored under `./config/<component-name>`
2022-08-24 20:31:44 +00:00
```
├── cluster-list.txt
├── compose
│   ├── docker-compose-contract.yml
│   ├── docker-compose-db-sharding.yml
│   ├── docker-compose-db.yml
│   ├── docker-compose-eth-statediff-fill-service.yml
│   ├── docker-compose-go-ethereum-foundry.yml
│   ├── docker-compose-ipld-eth-beacon-db.yml
│   ├── docker-compose-ipld-eth-beacon-indexer.yml
│   ├── docker-compose-ipld-eth-server.yml
│   ├── docker-compose-lighthouse.yml
│   └── docker-compose-prometheus-grafana.yml
├── config
│   └── ipld-eth-server
├── container-build
│   ├── cerc-eth-statediff-fill-service
│   ├── cerc-go-ethereum
│   ├── cerc-go-ethereum-foundry
│   ├── cerc-ipld-eth-beacon-db
│   ├── cerc-ipld-eth-beacon-indexer
│   ├── cerc-ipld-eth-db
│   ├── cerc-ipld-eth-server
│   ├── cerc-lighthouse
│   └── cerc-test-contract
├── container-image-list.txt
├── repository-list.txt
```
_write-more-of-me_