Compare commits

..
Author SHA1 Message Date
telackey 88f66a3626 Add deployment update and deploy-webapp-from-registry commands. (#676) 2023-12-13 21:02:34 -06:00
prathamesh0 1ef0b316c6 Expose metrics endpoints for sushiswap and merkl sushiswap watchers (#685) 2023-12-13 14:58:26 +05:30
prathamesh0 232d5618cb Update instructions in osmosis stack (#684) 2023-12-12 13:54:00 +05:30
fa6b570f4a Add stack for running osmosis frontend app (#673)
* osmosis FE stack

* chmod

* dont use 3000

* fix for neww stack format

* updates

* update osmosis readme

* Update stack.yml

* Update osmosis frontend stack to serve app

* Host osmosis app static build using python server

* Fix mapped ports in deployment for containers

* Update instructions

* Use nginx server to host files and handle page reloads

* Fix typo

---------

Co-authored-by: zramsay <zach@bluecollarcoding.ca>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2023-12-11 14:10:54 +05:30
prathamesh0 f9eb5a4ba8 Refactor to make Urbit setup generic (#682)
* Refactor to make Urbit app deployment script generic

* Rename urbit pod and update instructions

* Add a flag to allow skipping app installation on Urbit

* Make remote Urbit app deployment scripts generic

* Move remote deployment scripts to urbit fixturenet

* Update and use existing kubo pod for Urbit glob hosting
2023-12-08 09:35:00 +05:30
telackey 077ea80c70 Add deployment status command and fix k8s output for deployment ps (#679) 2023-12-06 09:27:47 -07:00
dboreham 15faed00de Generate a unique deployment id for each deployment (#680)
* Move cluster name generation into a function

* Generate a unique deployment id for each deployment
2023-12-05 22:56:58 -07:00
prathamesh0 6bef0c5b2f Separate out GQL proxy server from uniswap-urbit stack (#681)
* Separate out uniswap gql proxy in a stack

* Use proxy server from watcher-ts

* Add a flag to enable/disable the proxy server

* Update env configuratoin for uniswap urbit app stack

* Update stack file for uniswap urbit app stack

* Fix env variables in instructions
2023-12-06 10:41:10 +05:30
prathamesh0 f27da19808 Use IPFS for hosting glob files for Urbit (#677)
* Use IPFS for hosting glob files for Urbit

* Add env configuration for IPFS endpoints to instructions

* Make ship pier dir configurable in remote deployment script

* Update remote deployment script to accept glob hash arg
2023-12-05 15:00:03 +05:30
telackey 2dd54892a1 Allow specifying the webapp tag explicitly (#675) 2023-12-04 21:39:16 -06:00
telackey ab0e70ed83 Change path portion of unique cluster name to point to compose file, not argv[0]. (#678) 2023-12-04 13:39:14 -06:00
nabarunandprathamesh c319e90ddd Add a stack for running uniswap frontend on urbit (#670)
* Create uniswap-frontend stack

* Add stack for building uniswap frontend app

* Add a container for Urbit fake ship

* Update with deployment command

* Add a service for uniswap app deployment to urbit

* Use a script to start urbit ship to handle restarts

* Rename stack name to uniswap-urbit-app

* Rename build.sh to build-app.sh and check if build already exists

* Rename stack directory name

* Update uniswap build restart on failure

* Perform uniswap app deployment in the urbit container

* Add steps to create glob for the app

* Tail /dev/null after deployment

* Add steps to install the app to desk

* Host glob files for uniswap

* Update repo branch

* Update readme with command to get urbit password

* Update readme

* Update readme to open urbit web UI

* Expose the port on glob hosting container

* Avoid exposing urbit http port

* Add scripts for installing uniswap on remote urbit instance

* Configure GQL proxy for uniswap app

* Use laconic branch for app repo

* Rename urbit pod for uniswap app deployment

---------

Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2023-12-04 18:39:19 +05:30
telackey 2173e7ce6a If the next version is unsupported, print a big warning and try higher version. (#674) 2023-11-30 12:33:06 -06:00
dboreham c19559967d Add doc for deploy-webapp (#672) 2023-11-29 20:55:14 -07:00
dboreham d7093277b4 Use constants (#671) 2023-11-29 20:50:53 -07:00
telackey 03a3645b3c Add --port option to run-webapp. (#667)
* Add --port option to run-webapp

* Fixed merge

* lint
2023-11-29 11:32:28 -06:00
dboreham 113c0bfbf1 Propagate env file for webapp deployment (#669) 2023-11-28 21:14:02 -07:00
dboreham 1a069a6816 Use a temp file for the spec file name (#668) 2023-11-28 19:56:12 -07:00
dboreham a68cd5d65c Webapp deploy (#662) 2023-11-27 22:02:16 -07:00
nabarun 1b94db27c1 Upgrade azimuth watcher release version to 0.1.2 (#666)
* Upgrade azimuth watcher release version

* Fix version for azimuth watcher repo
2023-11-24 14:05:37 +05:30
prathamesh0 9499941891 Increase max connections for Azimuth watcher dbs (#665) 2023-11-23 19:11:02 +05:30
3fefc67e77 Run azimuth contract watcher in active mode (#661)
* Update stack to run azimuth job runner

* Run azimuth watcher in active mode

* Update stack to run job-runners for all watchers

* Update ports in job-runner health checks

* Map metrics ports to host

* Configure historical block processing batch size for Azimuth watcher

* Use deployment command for azimuth stack

---------

Co-authored-by: neeraj <neeraj.rtly@gmail.com>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2023-11-23 15:00:18 +05:30
telackey 1a37255c18 Tweak laconicd config to allow setting endpoint port and to make the fixturenet restartable. (#660)
* Endpoint includes port

* Make it restartable

* Don't try to remove the mounted directory

* Make copy of init.sh
2023-11-22 11:31:30 -06:00
dboreham 87bedde5cb Support for k8s ingress and tls (#659) 2023-11-21 16:04:36 -07:00
76 changed files with 2779 additions and 255 deletions
+11 -1
View File
@@ -34,7 +34,7 @@ To test locally run:
## Running
With `run-webapp` a new container will be launched with runtime configuration provided by `--env-file` (if specified) and published on an available port. Multiple instances can be launched with different configuration.
With `run-webapp` a new container will be launched on the local machine, with runtime configuration provided by `--env-file` (if specified) and published on an available port. Multiple instances can be launched with different configuration.
**Example**:
```
@@ -52,3 +52,13 @@ Image: cerc/test-progressive-web-app:local
ID: 9ab96494f563aafb6c057d88df58f9eca81b90f8721a4e068493a289a976051c
URL: http://localhost:32769
```
## Deploying
Use the subcommand `deploy-webapp create` to make a deployment directory that can be subsequently deployed to a Kubernetes cluster.
Example commands are shown below, assuming that the webapp container image `cerc/test-progressive-web-app:local` has already been built:
```
$ laconic-so deploy-webapp create --kube-config ~/kubectl/k8s-kubeconfig.yaml --image-registry registry.digitalocean.com/laconic-registry --deployment-dir webapp-k8s-deployment --image cerc/test-progressive-web-app:local --url https://test-pwa-app.hosting.laconic.com/ --env-file test-webapp.env
$ laconic-so deployment --dir webapp-k8s-deployment push-images
$ laconic-so deployment --dir webapp-k8s-deployment start
```
+7 -3
View File
@@ -32,8 +32,9 @@ from stack_orchestrator.build import build_containers
@click.option('--source-repo', help="directory containing the webapp to build", required=True)
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
@click.option("--extra-build-args", help="Supply extra arguments to build")
@click.option("--tag", help="Container tag (default: cerc/<app_name>:local)")
@click.pass_context
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args):
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag):
'''build the specified webapp container'''
quiet = ctx.obj.quiet
@@ -70,8 +71,11 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args):
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = os.path.join(container_build_dir,
base_container.replace("/", "-"),
"Dockerfile.webapp")
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
container_build_env["CERC_CONTAINER_BUILD_TAG"] = f"cerc/{webapp_name}:local"
if not tag:
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
container_build_env["CERC_CONTAINER_BUILD_TAG"] = f"cerc/{webapp_name}:local"
else:
container_build_env["CERC_CONTAINER_BUILD_TAG"] = tag
build_containers.process_container(None, base_container, container_build_dir, container_build_env, dev_root_path, quiet,
verbose, dry_run, continue_on_error)
+8
View File
@@ -13,12 +13,20 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
cluster_name_prefix = "laconic-"
stack_file_name = "stack.yml"
spec_file_name = "spec.yml"
config_file_name = "config.env"
deployment_file_name = "deployment.yml"
compose_dir_name = "compose"
compose_deploy_type = "compose"
k8s_kind_deploy_type = "k8s-kind"
k8s_deploy_type = "k8s"
cluster_id_key = "cluster-id"
kube_config_key = "kube-config"
deploy_to_key = "deploy-to"
network_key = "network"
http_proxy_key = "http-proxy"
image_resigtry_key = "image-registry"
kind_config_filename = "kind-config.yml"
kube_config_filename = "kubeconfig.yml"
@@ -4,6 +4,6 @@ services:
image: cerc/laconic-console-host:local
environment:
- CERC_WEBAPP_FILES_DIR=${CERC_WEBAPP_FILES_DIR:-/usr/local/share/.config/yarn/global/node_modules/@cerc-io/console-app/dist/production}
- LACONIC_HOSTED_ENDPOINT=${LACONIC_HOSTED_ENDPOINT:-http://localhost}
- LACONIC_HOSTED_ENDPOINT=${LACONIC_HOSTED_ENDPOINT:-http://localhost:9473}
ports:
- "80"
@@ -5,7 +5,7 @@ services:
command: ["sh", "/docker-entrypoint-scripts.d/create-fixturenet.sh"]
volumes:
# The cosmos-sdk node's database directory:
- laconicd-data:/root/.laconicd/data
- laconicd-data:/root/.laconicd
# TODO: look at folding these scripts into the container
- ../config/fixturenet-laconicd/create-fixturenet.sh:/docker-entrypoint-scripts.d/create-fixturenet.sh
- ../config/fixturenet-laconicd/export-mykey.sh:/docker-entrypoint-scripts.d/export-mykey.sh
@@ -0,0 +1,36 @@
version: '3.7'
services:
# Runs an Urbit fake ship and attempts an app installation using given data
# Uploads the app glob to given IPFS endpoint
# From urbit_app_builds volume:
# - takes app build from ${CERC_URBIT_APP}/build (waits for it to appear)
# - takes additional mark files from ${CERC_URBIT_APP}/mar
# - takes the docket file from ${CERC_URBIT_APP}/desk.docket-0
urbit-fake-ship:
restart: unless-stopped
image: tloncorp/vere
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_URBIT_APP: ${CERC_URBIT_APP}
CERC_ENABLE_APP_INSTALL: ${CERC_ENABLE_APP_INSTALL:-true}
CERC_IPFS_GLOB_HOST_ENDPOINT: ${CERC_IPFS_GLOB_HOST_ENDPOINT:-http://ipfs:5001}
CERC_IPFS_SERVER_ENDPOINT: ${CERC_IPFS_SERVER_ENDPOINT:-http://ipfs:8080}
entrypoint: ["bash", "-c", "./run-urbit-ship.sh && ./deploy-app.sh && tail -f /dev/null"]
volumes:
- urbit_data:/urbit
- urbit_app_builds:/app-builds
- ../config/urbit/run-urbit-ship.sh:/urbit/run-urbit-ship.sh
- ../config/urbit/deploy-app.sh:/urbit/deploy-app.sh
ports:
- "80"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "80"]
interval: 20s
timeout: 5s
retries: 15
start_period: 10s
volumes:
urbit_data:
urbit_app_builds:
@@ -1,13 +1,24 @@
version: "3.2"
# See: https://docs.ipfs.tech/install/run-ipfs-inside-docker/#set-up
services:
ipfs:
image: ipfs/kubo:master-2023-02-20-714a968
image: ipfs/kubo:v0.24.0
restart: always
volumes:
- ./ipfs/import:/import
- ./ipfs/data:/data/ipfs
- ipfs-import:/import
- ipfs-data:/data/ipfs
ports:
- "0.0.0.0:8080:8080"
- "0.0.0.0:4001:4001"
- "4001"
- "8080"
- "0.0.0.0:5001:5001"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "5001"]
interval: 20s
timeout: 5s
retries: 15
start_period: 10s
volumes:
ipfs-import:
ipfs-data:
@@ -0,0 +1,34 @@
version: "3.2"
services:
osmosis-front-end:
image: cerc/osmosis-front-end:local
restart: on-failure
environment:
- NEXT_PUBLIC_WEB_API_BASE_URL=${CERC_WEB_API_BASE_URL}
working_dir: /app/packages/web
command: ["./build-app.sh"]
volumes:
- ../config/osmosis/build-app.sh:/app/packages/web/build-app.sh
- app_builds:/app-builds
nginx:
image: nginx:1.23-alpine
restart: always
depends_on:
osmosis-front-end:
condition: service_completed_successfully
volumes:
- ../config/osmosis/nginx:/etc/nginx/conf.d
- app_builds:/usr/share/nginx
ports:
- "80"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "80"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
volumes:
app_builds:
@@ -0,0 +1,22 @@
version: "3.2"
services:
proxy-server:
image: cerc/watcher-ts:local
restart: on-failure
working_dir: /app/packages/cli
environment:
ENABLE_PROXY: ${CERC_ENABLE_PROXY:-true}
PROXY_UPSTREAM: ${CERC_PROXY_UPSTREAM}
PROXY_ORIGIN_HEADER: ${CERC_PROXY_ORIGIN_HEADER}
command: ["sh", "-c", "./run.sh"]
volumes:
- ../config/proxy-server/run.sh:/app/packages/cli/run.sh
ports:
- "4000"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "4000"]
interval: 20s
timeout: 5s
retries: 15
start_period: 10s
@@ -0,0 +1,18 @@
version: "3.2"
services:
uniswap-interface:
image: cerc/uniswap-interface:local
restart: on-failure
environment:
- REACT_APP_INFURA_KEY=${CERC_INFURA_KEY}
- REACT_APP_AWS_API_ENDPOINT=${CERC_UNISWAP_GQL}
command: ["./build-app.sh"]
volumes:
- ../config/uniswap-interface/build-app.sh:/app/build-app.sh
- urbit_app_builds:/app-builds
- ../config/uniswap-interface/urbit-files/mar:/app/mar
- ../config/uniswap-interface/urbit-files/desk.docket-0:/app/desk.docket-0
volumes:
urbit_app_builds:
@@ -10,6 +10,7 @@ services:
- POSTGRES_MULTIPLE_DATABASES=azimuth-watcher,azimuth-watcher-job-queue,censures-watcher,censures-watcher-job-queue,claims-watcher,claims-watcher-job-queue,conditional-star-release-watcher,conditional-star-release-watcher-job-queue,delegated-sending-watcher,delegated-sending-watcher-job-queue,ecliptic-watcher,ecliptic-watcher-job-queue,linear-star-release-watcher,linear-star-release-watcher-job-queue,polls-watcher,polls-watcher-job-queue
- POSTGRES_EXTENSION=azimuth-watcher-job-queue:pgcrypto,censures-watcher-job-queue:pgcrypto,claims-watcher-job-queue:pgcrypto,conditional-star-release-watcher-job-queue:pgcrypto,delegated-sending-watcher-job-queue:pgcrypto,ecliptic-watcher-job-queue:pgcrypto,linear-star-release-watcher-job-queue:pgcrypto,polls-watcher-job-queue:pgcrypto,
- POSTGRES_PASSWORD=password
command: ["postgres", "-c", "max_connections=200"]
volumes:
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
- watcher_db_data:/var/lib/postgresql/data
@@ -22,6 +23,38 @@ services:
retries: 15
start_period: 10s
# Starts the azimuth-watcher job runner
azimuth-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CERC_HISTORICAL_BLOCK_RANGE: 500
CONTRACT_ADDRESS: 0x223c067F8CF28ae173EE5CafEa60cA44C335fecB
CONTRACT_NAME: Azimuth
STARTING_BLOCK: 6784880
working_dir: /app/packages/azimuth-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/azimuth-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/azimuth-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/azimuth-watcher/start-job-runner.sh
ports:
- "9000"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9000"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the azimuth-watcher server
azimuth-watcher-server:
image: cerc/watcher-azimuth:local
@@ -29,8 +62,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
azimuth-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -52,6 +85,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the censures-watcher job runner
censures-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0x325f68d32BdEe6Ed86E7235ff2480e2A433D6189
CONTRACT_NAME: Censures
STARTING_BLOCK: 6784954
working_dir: /app/packages/censures-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/censures-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/censures-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/censures-watcher/start-job-runner.sh
ports:
- "9002"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9002"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the censures-watcher server
censures-watcher-server:
image: cerc/watcher-azimuth:local
@@ -59,8 +123,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
censures-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -82,6 +146,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the claims-watcher job runner
claims-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0xe7e7f69b34D7d9Bd8d61Fb22C33b22708947971A
CONTRACT_NAME: Claims
STARTING_BLOCK: 6784941
working_dir: /app/packages/claims-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/claims-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/claims-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/claims-watcher/start-job-runner.sh
ports:
- "9004"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9004"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the claims-watcher server
claims-watcher-server:
image: cerc/watcher-azimuth:local
@@ -89,8 +184,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
claims-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -112,6 +207,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the conditional-star-release-watcher job runner
conditional-star-release-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0x8C241098C3D3498Fe1261421633FD57986D74AeA
CONTRACT_NAME: ConditionalStarRelease
STARTING_BLOCK: 6828004
working_dir: /app/packages/conditional-star-release-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/conditional-star-release-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/conditional-star-release-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/conditional-star-release-watcher/start-job-runner.sh
ports:
- "9006"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9006"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the conditional-star-release-watcher server
conditional-star-release-watcher-server:
image: cerc/watcher-azimuth:local
@@ -119,8 +245,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
conditional-star-release-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -142,6 +268,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the delegated-sending-watcher job runner
delegated-sending-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0xf6b461fE1aD4bd2ce25B23Fe0aff2ac19B3dFA76
CONTRACT_NAME: DelegatedSending
STARTING_BLOCK: 6784956
working_dir: /app/packages/delegated-sending-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/delegated-sending-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/delegated-sending-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/delegated-sending-watcher/start-job-runner.sh
ports:
- "9008"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9008"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the delegated-sending-watcher server
delegated-sending-watcher-server:
image: cerc/watcher-azimuth:local
@@ -149,8 +306,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
delegated-sending-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -172,6 +329,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the ecliptic-watcher job runner
ecliptic-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0x33EeCbf908478C10614626A9D304bfe18B78DD73
CONTRACT_NAME: Ecliptic
STARTING_BLOCK: 13692129
working_dir: /app/packages/ecliptic-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/ecliptic-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/ecliptic-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/ecliptic-watcher/start-job-runner.sh
ports:
- "9010"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9010"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the ecliptic-watcher server
ecliptic-watcher-server:
image: cerc/watcher-azimuth:local
@@ -179,8 +367,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
ecliptic-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -202,6 +390,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the linear-star-release-watcher job runner
linear-star-release-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0x86cd9cd0992F04231751E3761De45cEceA5d1801
CONTRACT_NAME: LinearStarRelease
STARTING_BLOCK: 6784943
working_dir: /app/packages/linear-star-release-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/linear-star-release-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/linear-star-release-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/linear-star-release-watcher/start-job-runner.sh
ports:
- "9012"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9012"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the linear-star-release-watcher server
linear-star-release-watcher-server:
image: cerc/watcher-azimuth:local
@@ -209,8 +428,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
linear-star-release-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -232,6 +451,37 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the polls-watcher job runner
polls-watcher-job-runner:
image: cerc/watcher-azimuth:local
restart: unless-stopped
depends_on:
watcher-db:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
CERC_IPLD_ETH_GQL: ${CERC_IPLD_ETH_GQL}
CONTRACT_ADDRESS: 0x7fEcaB617c868Bb5996d99D95200D2Fa708218e4
CONTRACT_NAME: Polls
STARTING_BLOCK: 6784912
working_dir: /app/packages/polls-watcher
command: "./start-job-runner.sh"
volumes:
- ../config/watcher-azimuth/watcher-config-template.toml:/app/packages/polls-watcher/environments/watcher-config-template.toml
- ../config/watcher-azimuth/merge-toml.js:/app/packages/polls-watcher/merge-toml.js
- ../config/watcher-azimuth/start-job-runner.sh:/app/packages/polls-watcher/start-job-runner.sh
ports:
- "9014"
healthcheck:
test: ["CMD", "nc", "-vz", "localhost", "9014"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
# Starts the polls-watcher server
polls-watcher-server:
image: cerc/watcher-azimuth:local
@@ -239,8 +489,8 @@ services:
depends_on:
watcher-db:
condition: service_healthy
env_file:
- ../config/watcher-azimuth/watcher-params.env
polls-watcher-job-runner:
condition: service_healthy
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_IPLD_ETH_RPC: ${CERC_IPLD_ETH_RPC}
@@ -35,7 +35,7 @@ services:
- ../config/watcher-merkl-sushiswap-v3/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-merkl-sushiswap-v3/start-job-runner.sh:/app/start-job-runner.sh
ports:
- "9000"
- "9002:9000"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "9000"]
interval: 20s
@@ -62,7 +62,7 @@ services:
- ../config/watcher-merkl-sushiswap-v3/start-server.sh:/app/start-server.sh
ports:
- "127.0.0.1:3007:3008"
- "9001"
- "9003:9001"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "3008"]
interval: 20s
@@ -35,7 +35,7 @@ services:
- ../config/watcher-sushiswap-v3/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-sushiswap-v3/start-job-runner.sh:/app/start-job-runner.sh
ports:
- "9000"
- "9000:9000"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "9000"]
interval: 20s
@@ -62,7 +62,7 @@ services:
- ../config/watcher-sushiswap-v3/start-server.sh:/app/start-server.sh
ports:
- "127.0.0.1:3008:3008"
- "9001"
- "9001:9001"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "3008"]
interval: 20s
@@ -0,0 +1,8 @@
services:
webapp:
image: cerc/webapp-container:local
restart: always
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
ports:
- "3000"
@@ -14,104 +14,111 @@ LOGLEVEL="info"
TRACE="--trace"
# TRACE=""
# validate dependencies are installed
command -v jq > /dev/null 2>&1 || { echo >&2 "jq not installed. More info: https://stedolan.github.io/jq/download/"; exit 1; }
if [ "$1" == "clean" ] || [ ! -d "$HOME/.laconicd/data/blockstore.db" ]; then
# validate dependencies are installed
command -v jq > /dev/null 2>&1 || { echo >&2 "jq not installed. More info: https://stedolan.github.io/jq/download/"; exit 1; }
# remove existing daemon and client
rm -rf ~/.laconic*
# remove existing daemon and client
rm -rf $HOME/.laconicd/*
rm -rf $HOME/.laconic/*
make install
laconicd config keyring-backend $KEYRING
laconicd config chain-id $CHAINID
# if $KEY exists it should be deleted
laconicd keys add $KEY --keyring-backend $KEYRING --algo $KEYALGO
# Set moniker and chain-id for Ethermint (Moniker can be anything, chain-id must be an integer)
laconicd init $MONIKER --chain-id $CHAINID
# Change parameter token denominations to aphoton
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["staking"]["params"]["bond_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["crisis"]["constant_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["gov"]["deposit_params"]["min_deposit"][0]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["mint"]["params"]["mint_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Custom modules
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commit_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveal_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_minimum_bid"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
if [[ "$TEST_REGISTRY_EXPIRY" == "true" ]]; then
echo "Setting timers for expiry tests."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
if [[ "$TEST_AUCTION_ENABLED" == "true" ]]; then
echo "Enabling auction and setting timers."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
# increase block time (?)
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["time_iota_ms"]="1000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Set gas limit in genesis
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["max_gas"]="10000000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# disable produce empty block
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
fi
if [[ $1 == "pending" ]]; then
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
if [ -n "`which make`" ]; then
make install
fi
fi
# Allocate genesis accounts (cosmos formatted addresses)
laconicd add-genesis-account $KEY 100000000000000000000000000aphoton --keyring-backend $KEYRING
laconicd config keyring-backend $KEYRING
laconicd config chain-id $CHAINID
# Sign genesis transaction
laconicd gentx $KEY 1000000000000000000000aphoton --keyring-backend $KEYRING --chain-id $CHAINID
# if $KEY exists it should be deleted
laconicd keys add $KEY --keyring-backend $KEYRING --algo $KEYALGO
# Collect genesis tx
laconicd collect-gentxs
# Set moniker and chain-id for Ethermint (Moniker can be anything, chain-id must be an integer)
laconicd init $MONIKER --chain-id $CHAINID
# Run this to ensure everything worked and that the genesis file is setup correctly
laconicd validate-genesis
# Change parameter token denominations to aphoton
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["staking"]["params"]["bond_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["crisis"]["constant_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["gov"]["deposit_params"]["min_deposit"][0]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["mint"]["params"]["mint_denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Custom modules
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commit_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveal_fee"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_minimum_bid"]["denom"]="aphoton"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
if [[ $1 == "pending" ]]; then
echo "pending mode is on, please wait for the first block committed."
if [[ "$TEST_REGISTRY_EXPIRY" == "true" ]]; then
echo "Setting timers for expiry tests."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
if [[ "$TEST_AUCTION_ENABLED" == "true" ]]; then
echo "Enabling auction and setting timers."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
fi
# increase block time (?)
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["time_iota_ms"]="1000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Set gas limit in genesis
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["max_gas"]="10000000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# disable produce empty block
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
fi
if [[ $1 == "pending" ]]; then
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
fi
fi
# Allocate genesis accounts (cosmos formatted addresses)
laconicd add-genesis-account $KEY 100000000000000000000000000aphoton --keyring-backend $KEYRING
# Sign genesis transaction
laconicd gentx $KEY 1000000000000000000000aphoton --keyring-backend $KEYRING --chain-id $CHAINID
# Collect genesis tx
laconicd collect-gentxs
# Run this to ensure everything worked and that the genesis file is setup correctly
laconicd validate-genesis
if [[ $1 == "pending" ]]; then
echo "pending mode is on, please wait for the first block committed."
fi
else
echo "Using existing database at $HOME/.laconicd. To replace, run '`basename $0` clean'"
fi
# Start the node (remove the --pruning=nothing flag if historical queries are not needed)
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Check and exit if a deployment already exists (on restarts)
if [ -d /app-builds/osmosis/build ]; then
echo "Build already exists, remove volume to rebuild"
exit 0
fi
yarn build:static
# Move build to app-builds
mkdir -p /app-builds/osmosis
cp -r ./out /app-builds/osmosis/build
@@ -0,0 +1,16 @@
server {
listen 80;
listen [::]:80;
server_name _;
error_page 500 502 503 504 /50x.html;
location / {
root /usr/share/nginx/osmosis/build;
index index.html index.htm index.nginx-debian.html;
# First attempt to serve request as file, then as html,
# then as directory, then fall back to displaying a 404.
try_files $uri $uri.html $uri/ /index.html =404;
}
}
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
if [ "$ENABLE_PROXY" = "true" ]; then
echo "Proxy server enabled"
yarn proxy
else
echo "Proxy server disabled, exiting"
exit 0
fi
@@ -0,0 +1,21 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Check and exit if a deployment already exists (on restarts)
if [ -d /app-builds/uniswap/build ]; then
echo "Build already exists, remove volume to rebuild"
exit 0
fi
yarn build
# Copy over build and other files to app-builds for urbit deployment
mkdir -p /app-builds/uniswap
cp -r ./build /app-builds/uniswap/
cp -r mar /app-builds/uniswap/
cp desk.docket-0 /app-builds/uniswap/
@@ -0,0 +1,10 @@
:~ title+'Uniswap'
info+'Self-hosted uniswap frontend.'
color+0xcd.75df
image+'https://logowik.com/content/uploads/images/uniswap-uni7403.jpg'
base+'uniswap'
glob-http+['REPLACE_WITH_GLOB_URL' REPLACE_WITH_GLOB_HASH]
version+[0 0 1]
website+'https://uniswap.org/'
license+'MIT'
==
@@ -0,0 +1,18 @@
::
:::: /hoon/map/mar
:: Mark for js source maps
/? 310
::
=, eyre
|_ mud=@
++ grow
|%
++ mime [/application/octet-stream (as-octs:mimes:html (@t mud))]
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] (@t q.q))
++ noun cord :: clam from %noun
--
++ grad %mime
--
@@ -0,0 +1,12 @@
|_ dat=octs
++ grow
|%
++ mime [/font/ttf dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -0,0 +1,12 @@
|_ dat=octs
++ grow
|%
++ mime [/font/woff dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -0,0 +1,34 @@
#!/bin/bash
# $1: Remote user host
# $2: App name (eg. uniswap)
# $3: Assets dir path (local) for app (eg. /home/user/myapp/urbit-files)
# $4: Remote Urbit ship's pier dir path (eg. /home/user/zod)
# $5: Glob file URL (eg. https://xyz.com/glob-0vabcd.glob)
# $6: Glob file hash (eg. 0vabcd)
if [ "$#" -ne 6 ]; then
echo "Incorrect number of arguments"
echo "Usage: $0 <username@remote_host> <app_name> </path/to/app/assets/folder> </path/to/remote/pier/folder> <glob_url> <glob_hash>"
exit 1
fi
remote_user_host="$1"
app_name=$2
app_assets_folder=$3
remote_pier_folder="$4"
glob_url="$5"
glob_hash="$6"
installation_script="./install-urbit-app.sh"
# Copy over the assets to remote machine in a tmp dir
remote_app_assets_folder=/tmp/urbit-app-assets/$app_name
ssh "$remote_user_host" "mkdir -p $remote_app_assets_folder"
scp -r $app_assets_folder/* $remote_user_host:$remote_app_assets_folder
# Run the installation script
ssh "$remote_user_host" "bash -s $app_name $remote_app_assets_folder '${glob_url}' $glob_hash $remote_pier_folder" < "$installation_script"
# Remove the tmp assets dir
ssh "$remote_user_host" "rm -rf $remote_app_assets_folder"
+110
View File
@@ -0,0 +1,110 @@
#!/bin/bash
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
if [ -z "$CERC_URBIT_APP" ]; then
echo "CERC_URBIT_APP not set, exiting"
exit 0
fi
echo "Creating Urbit application for ${CERC_URBIT_APP}"
app_desk_dir=/urbit/zod/${CERC_URBIT_APP}
if [ -d ${app_desk_dir} ]; then
echo "Desk dir already exists for ${CERC_URBIT_APP}, skipping deployment..."
exit 0
fi
app_build=/app-builds/${CERC_URBIT_APP}/build
app_mark_files=/app-builds/${CERC_URBIT_APP}/mar
app_docket_file=/app-builds/${CERC_URBIT_APP}/desk.docket-0
echo "Reading app build from ${app_build}"
echo "Reading additional mark files from ${app_mark_files}"
echo "Reading docket file ${app_docket_file}"
# Loop until the app's build appears
while [ ! -d ${app_build} ]; do
echo "${CERC_URBIT_APP} app build not found, retrying in 5s..."
sleep 5
done
echo "Build found..."
echo "Using IPFS endpoint ${CERC_IPFS_GLOB_HOST_ENDPOINT} for hosting the ${CERC_URBIT_APP} glob"
echo "Using IPFS server endpoint ${CERC_IPFS_SERVER_ENDPOINT} for reading ${CERC_URBIT_APP} glob"
ipfs_host_endpoint=${CERC_IPFS_GLOB_HOST_ENDPOINT}
ipfs_server_endpoint=${CERC_IPFS_SERVER_ENDPOINT}
# Fire curl requests to perform operations on the ship
dojo () {
curl -s --data '{"source":{"dojo":"'"$1"'"},"sink":{"stdout":null}}' http://localhost:12321
}
hood () {
curl -s --data '{"source":{"dojo":"+hood/'"$1"'"},"sink":{"app":"hood"}}' http://localhost:12321
}
# Create / mount the app's desk
hood "merge %${CERC_URBIT_APP} our %landscape"
hood "mount %${CERC_URBIT_APP}"
# Copy over build to desk data dir
cp -r ${app_build} ${app_desk_dir}
# Copy over the additional mark files
cp ${app_mark_files}/* ${app_desk_dir}/mar/
rm "${app_desk_dir}/desk.bill"
rm "${app_desk_dir}/desk.ship"
# Commit changes and create a glob
hood "commit %${CERC_URBIT_APP}"
dojo "-landscape!make-glob %${CERC_URBIT_APP} /build"
glob_file=$(ls -1 -c zod/.urb/put | head -1)
echo "Created glob file: ${glob_file}"
# Upload the glob file to IPFS
echo "Uploading glob file to ${ipfs_host_endpoint}"
upload_response=$(curl -X POST -F file=@./zod/.urb/put/${glob_file} ${ipfs_host_endpoint}/api/v0/add)
glob_cid=$(echo "$upload_response" | grep -o '"Hash":"[^"]*' | sed 's/"Hash":"//')
glob_url="${ipfs_server_endpoint}/ipfs/${glob_cid}?filename=${glob_file}"
glob_hash=$(echo "$glob_file" | sed "s/glob-\([a-z0-9\.]*\).glob/\1/")
echo "Glob file uploaded to IFPS:"
echo "{ cid: ${glob_cid}, filename: ${glob_file} }"
echo "{ url: ${glob_url}, hash: ${glob_hash} }"
# Exit if the installation not required
if [ "$CERC_ENABLE_APP_INSTALL" = "false" ]; then
echo "CERC_ENABLE_APP_INSTALL set to false, skipping app installation"
exit 0
fi
# Curl and wait for the glob to be hosted
echo "Checking if glob file hosted at ${glob_url}"
while true; do
response=$(curl -sL -w "%{http_code}" -o /dev/null "$glob_url")
if [ $response -eq 200 ]; then
echo "File found at $glob_url"
break # Exit the loop if the file is found
else
echo "File not found, retrying in a 5s..."
sleep 5
fi
done
# Replace the docket file for app
# Substitue the glob URL and hash
cp ${app_docket_file} ${app_desk_dir}/
sed -i "s|REPLACE_WITH_GLOB_URL|${glob_url}|g; s|REPLACE_WITH_GLOB_HASH|${glob_hash}|g" ${app_desk_dir}/desk.docket-0
# Commit changes and install the app
hood "commit %${CERC_URBIT_APP}"
hood "install our %${CERC_URBIT_APP}"
echo "${CERC_URBIT_APP} app installed"
+60
View File
@@ -0,0 +1,60 @@
#!/bin/bash
# $1: App name (eg. uniswap)
# $2: Assets dir path (local) for app (eg. /home/user/myapp/urbit-files)
# $3: Glob file URL (eg. https://xyz.com/glob-0vabcd.glob)
# $4: Glob file hash (eg. 0vabcd)
# $5: Urbit ship's pier dir (default: ./zod)
if [ "$#" -lt 4 ]; then
echo "Insufficient arguments"
echo "Usage: $0 <app_name> </path/to/app/assets/folder> <glob_url> <glob_hash> [/path/to/remote/pier/folder]"
exit 1
fi
app_name=$1
app_mark_files=$2/mar
app_docket_file=$2/desk.docket-0
echo "Creating Urbit application for ${app_name}"
echo "Reading additional mark files from ${app_mark_files}"
echo "Reading docket file ${app_docket_file}"
glob_url=$3
glob_hash=$4
echo "Using glob file from ${glob_url} with hash ${glob_hash}"
# Default pier dir: ./zod
# Default desk dir: ./zod/<app_name>
pier_dir="${5:-./zod}"
app_desk_dir="${pier_dir}/${app_name}"
echo "Using ${app_desk_dir} as the ${app_name} desk dir path"
# Fire curl requests to perform operations on the ship
dojo () {
curl -s --data '{"source":{"dojo":"'"$1"'"},"sink":{"stdout":null}}' http://localhost:12321
}
hood () {
curl -s --data '{"source":{"dojo":"+hood/'"$1"'"},"sink":{"app":"hood"}}' http://localhost:12321
}
# Create / mount the app's desk
hood "merge %${app_name} our %landscape"
hood "mount %${app_name}"
# Copy over the additional mark files
cp ${app_mark_files}/* ${app_desk_dir}/mar/
rm "${app_desk_dir}/desk.bill"
rm "${app_desk_dir}/desk.ship"
# Replace the docket file for app
# Substitue the glob URL and hash
cp ${app_docket_file} ${app_desk_dir}/
sed -i "s|REPLACE_WITH_GLOB_URL|${glob_url}|g; s|REPLACE_WITH_GLOB_HASH|${glob_hash}|g" ${app_desk_dir}/desk.docket-0
# Commit changes and install the app
hood "commit %${app_name}"
hood "install our %${app_name}"
echo "${app_name} app installed"
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
pier_dir="/urbit/zod"
# TODO: Bootstrap fake ship on the first run
# Run urbit ship in daemon mode
# Check if the directory exists
if [ -d "$pier_dir" ]; then
echo "Pier directory already exists, rebooting..."
/urbit/zod/.run -d
else
echo "Creating a new fake ship..."
urbit -d -F zod
fi
@@ -0,0 +1,28 @@
#!/bin/sh
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
echo "Using IPLD ETH RPC endpoint ${CERC_IPLD_ETH_RPC}"
echo "Using IPLD GQL endpoint ${CERC_IPLD_ETH_GQL}"
echo "Using historicalLogsBlockRange ${CERC_HISTORICAL_BLOCK_RANGE:-2000}"
# Replace env variables in template TOML file
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_IPLD_ETH_RPC|${CERC_IPLD_ETH_RPC}|g; \
s|REPLACE_WITH_CERC_IPLD_ETH_GQL|${CERC_IPLD_ETH_GQL}|g; \
s|REPLACE_WITH_CERC_HISTORICAL_BLOCK_RANGE|${CERC_HISTORICAL_BLOCK_RANGE:-2000}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/watcher-config.toml
# Merge SO watcher config with existing config file
node merge-toml.js
yarn watch:contract --address $CONTRACT_ADDRESS --kind $CONTRACT_NAME --checkpoint true --starting-block $STARTING_BLOCK
echo 'yarn job-runner'
yarn job-runner
@@ -4,18 +4,17 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_IPLD_ETH_RPC="${CERC_IPLD_ETH_RPC:-${DEFAULT_CERC_IPLD_ETH_RPC}}"
CERC_IPLD_ETH_GQL="${CERC_IPLD_ETH_GQL:-${DEFAULT_CERC_IPLD_ETH_GQL}}"
echo "Using IPLD ETH RPC endpoint ${CERC_IPLD_ETH_RPC}"
echo "Using IPLD GQL endpoint ${CERC_IPLD_ETH_GQL}"
echo "Using historicalLogsBlockRange ${CERC_HISTORICAL_BLOCK_RANGE:-2000}"
# Replace env variables in template TOML file
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_IPLD_ETH_RPC|${CERC_IPLD_ETH_RPC}|g; \
s|REPLACE_WITH_CERC_IPLD_ETH_GQL|${CERC_IPLD_ETH_GQL}| ")
s|REPLACE_WITH_CERC_IPLD_ETH_GQL|${CERC_IPLD_ETH_GQL}|g; \
s|REPLACE_WITH_CERC_HISTORICAL_BLOCK_RANGE|${CERC_HISTORICAL_BLOCK_RANGE:-2000}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/watcher-config.toml
@@ -2,6 +2,9 @@
host = "0.0.0.0"
maxSimultaneousRequests = -1
[metrics]
host = "0.0.0.0"
[database]
host = "watcher-db"
port = 5432
@@ -12,3 +15,7 @@
[upstream.ethServer]
gqlApiEndpoint = "REPLACE_WITH_CERC_IPLD_ETH_GQL"
rpcProviderEndpoint = "REPLACE_WITH_CERC_IPLD_ETH_RPC"
[jobQueue]
historicalLogsBlockRange = REPLACE_WITH_CERC_HISTORICAL_BLOCK_RANGE
blockDelayInMilliSecs = 12000
@@ -1,5 +0,0 @@
# Defaults
# ipld-eth-server endpoints
DEFAULT_CERC_IPLD_ETH_RPC=
DEFAULT_CERC_IPLD_ETH_GQL=
@@ -41,7 +41,7 @@
timeTravelMaxAge = 86400 # 1 day
[metrics]
host = "127.0.0.1"
host = "0.0.0.0"
port = 9000
[metrics.gql]
port = 9001
@@ -41,7 +41,7 @@
timeTravelMaxAge = 86400 # 1 day
[metrics]
host = "127.0.0.1"
host = "0.0.0.0"
port = 9000
[metrics.gql]
port = 9001
@@ -2,5 +2,5 @@
services:
wns:
server: 'LACONIC_HOSTED_ENDPOINT:9473/api'
webui: 'LACONIC_HOSTED_ENDPOINT:9473/console'
server: 'LACONIC_HOSTED_ENDPOINT/api'
webui: 'LACONIC_HOSTED_ENDPOINT/console'
@@ -0,0 +1,14 @@
record:
type: ApplicationDeploymentRecord
version: 1.2.3
name: name
description: description
application: application
url: url
dns: dns
request: request
meta:
foo: bar
tags:
- a
- b
@@ -0,0 +1,12 @@
record:
type: GeneralRecord
version: 1.2.3
name: name
description: description
category: category
value: value
meta:
foo: bar
tags:
- a
- b
@@ -0,0 +1,17 @@
record:
type: ApplicationArtifact
version: 1.2.3
name: name
description: description
application: appidgoeshere
content_type: content_type
os: os
cpu: cpu
uri:
- uri://a
- uri://b
meta:
foo: bar
tags:
- a
- b
@@ -0,0 +1,11 @@
record:
type: DnsRecord
version: 0.0.1
name: "foo"
resource_type: "A"
value: "bar"
meta:
foo: bar
tags:
- a
- b
@@ -0,0 +1,18 @@
record:
type: ApplicationRecord
version: 0.0.1
name: my-demo-app
description: "Description of my app"
homepage: http://my.demo.app
license: license
author: author
repository:
- "https://my.demo.repo"
repository_ref: "v0.1.0"
app_version: "0.1.0"
app_type: "webapp"
meta:
foo: bar
tags:
- a
- b
@@ -0,0 +1,17 @@
record:
type: ApplicationDeploymentRequest
version: 1.2.3
application: application
dns: dns
config:
env:
ENV_VAR_A: A
ENV_VAR_B: B
crn:
- crn://foo.bar
- crn://bar.baz
meta:
foo: bar
tags:
- a
- b
@@ -24,6 +24,8 @@ RUN \
&& su ${USERNAME} -c "npm config -g set prefix ${NPM_GLOBAL}" \
# Install eslint
&& su ${USERNAME} -c "umask 0002 && npm install -g eslint" \
# Install semver
&& su ${USERNAME} -c "umask 0002 && npm install -g semver" \
&& npm cache clean --force > /dev/null 2>&1
# [Optional] Uncomment this section to install additional OS packages.
@@ -4,10 +4,12 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_MIN_NEXTVER=13.4.2
CERC_NEXT_VERSION="${CERC_NEXT_VERSION:-keep}"
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "yarn.lock" ] && [ ! -f "package-lock.json" ]; then
if [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=yarn
else
CERC_BUILD_TOOL=npm
@@ -101,13 +103,35 @@ cat package.dist | jq '.scripts.cerc_compile = "next experimental-compile"' | jq
CUR_NEXT_VERSION="`jq -r '.dependencies.next' package.json`"
if [ "$CERC_NEXT_VERSION" != "keep" ] && [ "$CUR_NEXT_VERSION" != "$CERC_NEXT_VERSION" ]; then
echo "Changing 'next' version specifier from '$CUR_NEXT_VERSION' to '$CERC_NEXT_VERSION' (set with --build-arg CERC_NEXT_VERSION)"
echo "Changing 'next' version specifier from '$CUR_NEXT_VERSION' to '$CERC_NEXT_VERSION' (set with '--extra-build-args \"--build-arg CERC_NEXT_VERSION=$CERC_NEXT_VERSION\"')"
cat package.json | jq ".dependencies.next = \"$CERC_NEXT_VERSION\"" | sponge package.json
else
echo "'next' version specifier '$CUR_NEXT_VERSION' (override with --build-arg CERC_NEXT_VERSION)"
fi
$CERC_BUILD_TOOL install || exit 1
CUR_NEXT_VERSION=`jq -r '.version' node_modules/next/package.json`
semver -p -r ">=$CERC_MIN_NEXTVER" $CUR_NEXT_VERSION
if [ $? -ne 0 ]; then
cat <<EOF
###############################################################################
WARNING: 'next' $CUR_NEXT_VERSION < minimum version $CERC_MIN_NEXTVER.
Attempting to build with '^$CERC_MIN_NEXTVER'. If this fails, you should upgrade
the dependency in your webapp, or specify an explicit 'next' version
to use for the build with:
--extra-build-args "--build-arg CERC_NEXT_VERSION=<version>"
###############################################################################
EOF
cat package.json | jq ".dependencies.next = \"^$CERC_MIN_NEXTVER\"" | sponge package.json
$CERC_BUILD_TOOL install || exit 1
fi
$CERC_BUILD_TOOL run cerc_compile || exit 1
exit 0
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build the osmosis front end image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
docker build -t cerc/osmosis-front-end:local -f ${CERC_REPO_BASE_DIR}/osmosis-frontend/docker/Dockerfile.static ${build_command_args} ${CERC_REPO_BASE_DIR}/osmosis-frontend
@@ -0,0 +1,10 @@
FROM node:18.17.1-alpine3.18
RUN apk --update --no-cache add git make alpine-sdk bash
WORKDIR /app
COPY . .
RUN echo "Building uniswap-interface" && \
yarn
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
# Build the uniswap-interface image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/uniswap-interface:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/uniswap-interface
@@ -0,0 +1,7 @@
FROM python:3.13.0a2-alpine3.18
RUN apk --update --no-cache add alpine-sdk jq bash curl wget
WORKDIR /app
ENTRYPOINT [ "bash" ]
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Build the urbit-globs-host image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/urbit-globs-host:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} ${SCRIPT_DIR}
@@ -59,3 +59,4 @@ cerc/ponder
cerc/nitro-rpc-client
cerc/watcher-merkl-sushiswap-v3
cerc/watcher-sushiswap-v3
cerc/uniswap-interface
@@ -49,3 +49,4 @@ github.com/cerc-io/mobymask-snap
github.com/cerc-io/ponder
github.com/cerc-io/merkl-sushiswap-v3-watcher-ts
github.com/cerc-io/sushiswap-v3-watcher-ts
github.com/cerc-io/uniswap-interface
@@ -9,19 +9,11 @@ Prerequisite: `ipld-eth-server` RPC and GQL endpoints
Clone required repositories:
```bash
laconic-so --stack azimuth setup-repositories
laconic-so --stack azimuth setup-repositories --pull
```
NOTE: If the repository already exists and checked out to a different version, `setup-repositories` command will throw an error.
For getting around this, the `azimuth-watcher-ts` repository can be removed and then run the command.
Checkout to the required versions and branches in repos
```bash
# azimuth-watcher-ts
cd ~/cerc/azimuth-watcher-ts
git checkout v0.1.0
```
For getting around this, the `azimuth-watcher-ts` repository can be removed and then run the command again.
Build the container images:
@@ -31,42 +23,85 @@ laconic-so --stack azimuth build-containers
This should create the required docker images in the local image registry.
### Configuration
## Create a deployment
* Create and update an env file to be used in the next step:
First, create a spec file for the deployment, which will map the stack's ports and volumes to the host:
```bash
laconic-so --stack azimuth deploy init --output azimuth-spec.yml
```
### Ports
Edit `network` in spec file to map container ports to same ports in host
```yaml
...
network:
ports:
watcher-db:
- 0.0.0.0:15432:5432
azimuth-watcher-server:
- 0.0.0.0:3001:3001
censures-watcher-server:
- 0.0.0.0:3002:3002
claims-watcher-server:
- 0.0.0.0:3003:3003
conditional-star-release-watcher-server:
- 0.0.0.0:3004:3004
delegated-sending-watcher-server:
- 0.0.0.0:3005:3005
ecliptic-watcher-server:
- 0.0.0.0:3006:3006
linear-star-release-watcher-server:
- 0.0.0.0:3007:3007
polls-watcher-server:
- 0.0.0.0:3008:3008
gateway-server:
- 0.0.0.0:4000:4000
...
```
### Data volumes
Container data volumes are bind-mounted to specified paths in the host filesystem.
The default setup (generated by `laconic-so deploy init`) places the volumes in the `./data` subdirectory of the deployment directory. The default mappings can be customized by editing the "spec" file generated by `laconic-so deploy init`.
---
Once you've made any needed changes to the spec file, create a deployment from it:
```bash
laconic-so --stack azimuth deploy create --spec-file azimuth-spec.yml --deployment-dir azimuth-deployment
```
## Set env variables
Inside the deployment directory, open the file `config.env` and add variable to update RPC endpoint :
```bash
# External ipld-eth-server endpoints
# External RPC endpoints
CERC_IPLD_ETH_RPC=
CERC_IPLD_ETH_GQL=
```
* NOTE: If `ipld-eth-server` is running on the host machine, use `host.docker.internal` as the hostname to access host ports
* NOTE: If RPC endpoint is on the host machine, use `host.docker.internal` as the hostname to access the host port, or use the `ip a` command to find the IP address of the `docker0` interface (this will usually be something like `172.17.0.1` or `172.18.0.1`)
### Deploy the stack
## Start the stack
* Deploy the containers:
```bash
laconic-so --stack azimuth deploy-system --env-file <PATH_TO_ENV_FILE> up
```
Start the deployment:
```bash
laconic-so deployment --dir azimuth-deployment start
```
* List and check the health status of all the containers using `docker ps` and wait for them to be `healthy`
## Clean up
Stop all the services running in background:
To stop all azimuth services running in the background, while preserving chain data:
```bash
laconic-so --stack azimuth deploy-system down
laconic-so deployment --dir azimuth-deployment stop
```
Clear volumes created by this stack:
To stop all azimuth services and also delete data:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=.*watcher_db_data"
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=.*watcher_db_data")
laconic-so deployment --dir azimuth-deployment stop --delete-volumes
```
@@ -1,7 +1,7 @@
version: "1.0"
name: azimuth
repos:
- github.com/cerc-io/azimuth-watcher-ts@v0.1.1
- github.com/cerc-io/azimuth-watcher-ts@v0.1.2
containers:
- cerc/watcher-azimuth
pods:
@@ -0,0 +1,151 @@
# self-hosted osmosis
Instructions to build and deploy:
- self-hosted gitea
- an ipfs node
- the osmosis front end
- a laconicd chain
## Setup
Clone required repositories:
```bash
laconic-so --stack osmosis setup-repositories
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories and re-run the command
```
Build the container images:
```bash
# TODO: support image for the gitea package registry
# laconic-so --stack build-support build-containers
laconic-so --stack osmosis build-containers
```
## Create a deployment
First, create a spec file for the deployment, which will map the stack's ports and volumes to the host:
```bash
laconic-so --stack osmosis deploy init --output osmosis-spec.yml
```
### Ports
Edit `network` in spec file to map container ports to same ports in host
```
...
network:
ports:
proxy-server:
- '4000:4000'
nginx:
- '3000:80'
```
### Data volumes
Container data volumes are bind-mounted to specified paths in the host filesystem.
The default setup (generated by `laconic-so deploy init`) places the volumes in the `./data` subdirectory of the deployment directory. The default mappings can be customized by editing the "spec" file generated by `laconic-so deploy init`.
---
Once you've made any needed changes to the spec file, create a deployment from it:
```bash
laconic-so --stack osmosis deploy create --spec-file osmosis-spec.yml --deployment-dir osmosis-deployment
```
## Set env variables
Inside the deployment directory, open the file `config.env` and set the following env variables:
(Note: Following config can be used as is if the stack is being run locally)
```bash
# Osmosis API base URL
# Set this to proxy server endpoint for osmosis app
# (Eg. http://localhost:4000 - in case stack is being run locally with proxy enabled)
# (Eg. https://abc.xyz.com - in case https://abc.xyz.com is pointed to the proxy endpoint)
CERC_WEB_API_BASE_URL=http://localhost:4000
# Optional
# Whether to run the proxy server
# (Disable only if proxy not required to be run) (Default: true)
CERC_ENABLE_PROXY=
# Proxy server configuration
# Used only if proxy is enabled
# Upstream API URL
CERC_PROXY_UPSTREAM=https://app.osmosis.zone
```
## Start the stack
Start the deployment:
```bash
laconic-so deployment --dir osmosis-deployment start
```
* The `osmosis-front-end` container will take some time to run to completion as it builds the front-end app with given configuration; same can be tracked in the logs:
```bash
laconic-so deployment --dir osmosis-deployment logs -f osmosis-front-end
```
* List and check the health status of all the containers using `docker ps` and wait for them to be `healthy`
* The web app can be accessed at http://localhost:3000
## Laconic registry
Setup a test chain:
```bash
export CERC_NPM_REGISTRY_URL=https://git.vdb.to/api/packages/cerc-io/npm/
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
laconic-so --stack fixturenet-laconic-loaded build-containers
export LACONIC_HOSTED_ENDPOINT=http://<your-IP>
laconic-so --stack fixturenet-laconic-loaded deploy up
```
then `docker exec` into the `laconicd` container and either export the private key or create a new one and send funds to it. Use that private key for `LACONIC_HOTWALLET_KEY`.
## Clean up
To stop all osmosis services running in the background, while preserving data:
```bash
# Only stop the docker containers
laconic-so deployment --dir osmosis-deployment stop
# Run 'start' to restart the deployment
```
To stop all osmosis services and also delete data:
```bash
# Stop the docker containers
laconic-so deployment --dir osmosis-deployment stop --delete-volumes
# Remove deployment directory (deployment will have to be recreated for a re-run)
rm -r osmosis-deployment
```
To stop stack running Laconic registry
```bash
laconic-so --stack fixturenet-laconic-loaded deploy down
# Delete volumes
laconic-so --stack fixturenet-laconic-loaded deploy down --delete-volumes
```
@@ -0,0 +1,30 @@
version: "0.1"
name: osmosis
repos:
# For gitea
# - git.vdb.to/cerc-io/hosting@names-for-so
# - gitea.com/gitea/act_runner
# For osmosis frontend
- github.com/cerc-io/watcher-ts@v0.2.78
- github.com/cerc-io/osmosis-frontend@laconic # TODO: Use release
containers:
# - cerc/act-runner
# - cerc/act-runner-task-executor
- cerc/watcher-ts
- cerc/osmosis-front-end
pods:
- kubo
- proxy-server
- osmosis-front-end
# TODO: Fix pod config for deployment command
# TODO: mirroring all of osmosis repos: https://git.vdb.to/cerc-io/hosting/pulls/42
# - name: gitea
# repository: cerc-io/hosting
# path: gitea
# pre_start_command: "run-this-first.sh"
# post_start_command: "initialize-gitea.sh"
# - name: act-runner
# repository: cerc-io/hosting
# path: act-runner
# pre_start_command: "pre_start.sh"
# post_start_command: "post_start.sh"
@@ -0,0 +1,79 @@
# Proxy Server
Instructions to setup and deploy a HTTP proxy server
## Setup
Clone required repository:
```bash
laconic-so --stack proxy-server setup-repositories --pull
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Build the container image:
```bash
laconic-so --stack proxy-server build-containers
```
## Create a deployment
* First, create a spec file for the deployment, which will allow mapping the stack's ports and volumes to the host:
```bash
laconic-so --stack proxy-server deploy init --output proxy-server-spec.yml
```
* Edit `network` in spec file to map container ports to same ports in host:
```yml
...
network:
ports:
proxy-server:
- '4000:4000'
...
```
* Once you've made any needed changes to the spec file, create a deployment from it:
```bash
laconic-so --stack proxy-server deploy create --spec-file proxy-server-spec.yml --deployment-dir proxy-server-deployment
```
* Inside the deployment directory, open the file `config.env` and set the following env variables:
```bash
# Whether to run the proxy server (Optional) (Default: true)
CERC_ENABLE_PROXY=
# Upstream endpoint
# (Eg. https://api.example.org)
CERC_PROXY_UPSTREAM=
# Origin header to be used (Optional)
# (Eg. https://app.example.org)
CERC_PROXY_ORIGIN_HEADER=
```
## Start the stack
Start the deployment:
```bash
laconic-so deployment --dir proxy-server-deployment start
```
* List and check the health status of the container using `docker ps`
* The proxy server will now be listening at http://localhost:4000
## Clean up
To stop the service running in background:
```bash
laconic-so deployment --dir proxy-server-deployment stop
```
@@ -0,0 +1,8 @@
version: "0.1"
name: proxy-server
repos:
- github.com/cerc-io/watcher-ts@v0.2.78
containers:
- cerc/watcher-ts
pods:
- proxy-server
@@ -0,0 +1,167 @@
# Self-hosted Uniswap Frontend
Instructions to setup and deploy Uniswap app on Urbit
Build and deploy:
- Urbit
- Uniswap app
## Setup
Clone required repositories:
```bash
laconic-so --stack uniswap-urbit-app setup-repositories --pull
# If this throws an error as a result of being already checked out to a branch/tag in a repo, remove the repositories mentioned below and re-run the command
```
Build the container images:
```bash
laconic-so --stack uniswap-urbit-app build-containers
```
## Create a deployment
First, create a spec file for the deployment, which will map the stack's ports and volumes to the host:
```bash
laconic-so --stack uniswap-urbit-app deploy init --output uniswap-urbit-app-spec.yml
```
### Ports
Edit `network` in spec file to map container ports to same ports in host:
```
...
network:
ports:
urbit-fake-ship:
- '8080:80'
proxy-server:
- '4000:4000'
ipfs:
- '8081:8080'
- '5001:5001'
...
```
Note: Skip the `ipfs` ports if need to use an externally running IPFS node
### Data volumes
Container data volumes are bind-mounted to specified paths in the host filesystem.
The default setup (generated by `laconic-so deploy init`) places the volumes in the `./data` subdirectory of the deployment directory. The default mappings can be customized by editing the "spec" file generated by `laconic-so deploy init`.
---
Once you've made any needed changes to the spec file, create a deployment from it:
```bash
laconic-so --stack uniswap-urbit-app deploy create --spec-file uniswap-urbit-app-spec.yml --deployment-dir uniswap-urbit-app-deployment
```
## Set env variables
Inside the deployment directory, open the file `config.env` and set the following env variables:
```bash
# App to be installed (Do not change)
CERC_URBIT_APP=uniswap
# External RPC endpoints
# https://docs.infura.io/getting-started#2-create-an-api-key
CERC_INFURA_KEY=
# Uniswap API GQL Endpoint
# Set this to GQL proxy server endpoint for uniswap app
# (Eg. http://localhost:4000/v1/graphql - in case stack is being run locally with proxy enabled)
# (Eg. https://abc.xyz.com/v1/graphql - in case https://abc.xyz.com is pointed to the proxy endpoint)
CERC_UNISWAP_GQL=http://localhost:4000/v1/graphql
# Optional
# Whether to enable app installation on Urbit
# (just builds and uploads the glob file if disabled) (Default: true)
CERC_ENABLE_APP_INSTALL=
# Whether to run the proxy GQL server
# (disable only if proxy not required to be run) (Default: true)
CERC_ENABLE_PROXY=
# Proxy server configuration
# Used only if proxy is enabled
# Upstream API URL
# (Eg. https://api.example.org)
CERC_PROXY_UPSTREAM=https://api.uniswap.org
# Origin header to be used in the proxy
# (Eg. https://app.example.org)
CERC_PROXY_ORIGIN_HEADER=https://app.uniswap.org
# IPFS configuration
# IFPS endpoint to host the glob file on
# (Default: http://ipfs:5001 pointing to in-stack IPFS node)
CERC_IPFS_GLOB_HOST_ENDPOINT=
# IFPS endpoint to fetch the glob file from
# (Default: http://ipfs:8080 pointing to in-stack IPFS node)
CERC_IPFS_SERVER_ENDPOINT=
```
## Start the stack
Start the deployment:
```bash
laconic-so deployment --dir uniswap-urbit-app-deployment start
```
* List and check the health status of all the containers using `docker ps` and wait for them to be `healthy`
* Run the following to get login password for Urbit web interface:
```bash
laconic-so deployment --dir uniswap-urbit-app-deployment exec urbit-fake-ship "curl -s --data '{\"source\":{\"dojo\":\"+code\"},\"sink\":{\"stdout\":null}}' http://localhost:12321"
# Expected output: "<PASSWORD>\n"%
```
* Open the Urbit web UI at http://localhost:8080 and use the `PASSWORD` from previous step to login
* The uniswap app is not available when starting stack for the first time. Check `urbit-fake-ship` logs to see that app has installed
```
laconic-so deployment --dir uniswap-urbit-app-deployment logs -f
# Expected output:
# laconic-3ccf7ee79bdae874-urbit-fake-ship-1 | docket: fetching %http glob for %uniswap desk
# laconic-3ccf7ee79bdae874-urbit-fake-ship-1 | ">="">="Uniswap app installed
```
* The uniswap app will be now visible at http://localhost:8080
## Clean up
To stop all uniswap-urbit-app services running in the background, while preserving data:
```bash
# Only stop the docker containers
laconic-so deployment --dir uniswap-urbit-app-deployment stop
# Run 'start' to restart the deployment
```
To stop all uniswap-urbit-app services and also delete data:
```bash
# Stop the docker containers
laconic-so deployment --dir uniswap-urbit-app-deployment stop --delete-volumes
# Remove deployment directory (deployment will have to be recreated for a re-run)
rm -r uniswap-urbit-app-deployment
```
@@ -0,0 +1,14 @@
version: "0.1"
name: uniswap-urbit-app
repos:
- github.com/cerc-io/uniswap-interface@laconic # TODO: Use release
- github.com/cerc-io/watcher-ts@v0.2.78
containers:
- cerc/uniswap-interface
- cerc/watcher-ts
- cerc/urbit-globs-host
pods:
- uniswap-interface
- proxy-server
- fixturenet-urbit
- kubo
@@ -0,0 +1 @@
# Template stack for webapp deployments
@@ -0,0 +1,7 @@
version: "1.0"
name: test
description: "Webapp deployment stack"
containers:
- cerc/webapp-template-container
pods:
- webapp-template
@@ -40,6 +40,19 @@ class DockerDeployer(Deployer):
except DockerException as e:
raise DeployerException(e)
def update(self):
try:
return self.docker.compose.restart()
except DockerException as e:
raise DeployerException(e)
def status(self):
try:
for p in self.docker.compose.ps():
print(f"{p.name}\t{p.state.status}")
except DockerException as e:
raise DeployerException(e)
def ps(self):
try:
return self.docker.compose.ps()
@@ -64,10 +77,10 @@ class DockerDeployer(Deployer):
except DockerException as e:
raise DeployerException(e)
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, detach=False):
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, ports=[], detach=False):
try:
return self.docker.run(image=image, command=command, user=user, volumes=volumes,
entrypoint=entrypoint, envs=env, detach=detach, publish_all=True)
entrypoint=entrypoint, envs=env, detach=detach, publish=ports, publish_all=len(ports) == 0)
except DockerException as e:
raise DeployerException(e)
+40 -10
View File
@@ -24,6 +24,8 @@ from importlib import resources
import subprocess
import click
from pathlib import Path
from stack_orchestrator import constants
from stack_orchestrator.opts import opts
from stack_orchestrator.util import include_exclude_check, get_parsed_stack_config, global_options2, get_dev_root_path
from stack_orchestrator.deploy.deployer import Deployer, DeployerException
from stack_orchestrator.deploy.deployer_factory import getDeployer
@@ -70,6 +72,9 @@ def create_deploy_context(
cluster,
env_file,
deploy_to) -> DeployCommandContext:
# Extract the cluster name from the deployment, if we have one
if deployment_context and cluster is None:
cluster = deployment_context.get_cluster_id()
cluster_context = _make_cluster_context(global_context, stack, include, exclude, cluster, env_file)
deployer = getDeployer(deploy_to, deployment_context, compose_files=cluster_context.compose_files,
compose_project_name=cluster_context.cluster,
@@ -107,6 +112,22 @@ def down_operation(ctx, delete_volumes, extra_args_list):
ctx.obj.deployer.down(timeout=timeout_arg, volumes=delete_volumes)
def status_operation(ctx):
global_context = ctx.parent.parent.obj
if not global_context.dry_run:
if global_context.verbose:
print("Running compose status")
ctx.obj.deployer.status()
def update_operation(ctx):
global_context = ctx.parent.parent.obj
if not global_context.dry_run:
if global_context.verbose:
print("Running compose update")
ctx.obj.deployer.update()
def ps_operation(ctx):
global_context = ctx.parent.parent.obj
if not global_context.dry_run:
@@ -253,6 +274,22 @@ def _make_runtime_env(ctx):
return container_exec_env
def _make_default_cluster_name(deployment, compose_dir, stack, include, exclude):
# Create default unique, stable cluster name from confile file path and stack name if provided
if deployment:
path = os.path.realpath(os.path.abspath(compose_dir))
else:
path = "internal"
unique_cluster_descriptor = f"{path},{stack},{include},{exclude}"
if opts.o.debug:
print(f"pre-hash descriptor: {unique_cluster_descriptor}")
hash = hashlib.md5(unique_cluster_descriptor.encode()).hexdigest()[:16]
cluster = f"{constants.cluster_name_prefix}{hash}"
if opts.o.debug:
print(f"Using cluster name: {cluster}")
return cluster
# stack has to be either PathLike pointing to a stack yml file, or a string with the name of a known stack
def _make_cluster_context(ctx, stack, include, exclude, cluster, env_file):
@@ -270,16 +307,9 @@ def _make_cluster_context(ctx, stack, include, exclude, cluster, env_file):
compose_dir = Path(__file__).absolute().parent.parent.joinpath("data", "compose")
if cluster is None:
# Create default unique, stable cluster name from confile file path and stack name if provided
# TODO: change this to the config file path
path = os.path.realpath(sys.argv[0])
unique_cluster_descriptor = f"{path},{stack},{include},{exclude}"
if ctx.debug:
print(f"pre-hash descriptor: {unique_cluster_descriptor}")
hash = hashlib.md5(unique_cluster_descriptor.encode()).hexdigest()
cluster = f"laconic-{hash}"
if ctx.verbose:
print(f"Using cluster name: {cluster}")
cluster = _make_default_cluster_name(deployment, compose_dir, stack, include, exclude)
else:
_make_default_cluster_name(deployment, compose_dir, stack, include, exclude)
# See: https://stackoverflow.com/a/20885799/1701505
from stack_orchestrator import data
+9 -1
View File
@@ -27,10 +27,18 @@ class Deployer(ABC):
def down(self, timeout, volumes):
pass
@abstractmethod
def update(self):
pass
@abstractmethod
def ps(self):
pass
@abstractmethod
def status(self):
pass
@abstractmethod
def port(self, service, private_port):
pass
@@ -44,7 +52,7 @@ class Deployer(ABC):
pass
@abstractmethod
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, detach=False):
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, ports=[], detach=False):
pass
+12 -4
View File
@@ -18,8 +18,8 @@ from pathlib import Path
import sys
from stack_orchestrator import constants
from stack_orchestrator.deploy.images import push_images_operation
from stack_orchestrator.deploy.deploy import up_operation, down_operation, ps_operation, port_operation
from stack_orchestrator.deploy.deploy import exec_operation, logs_operation, create_deploy_context
from stack_orchestrator.deploy.deploy import up_operation, down_operation, ps_operation, port_operation, status_operation
from stack_orchestrator.deploy.deploy import exec_operation, logs_operation, create_deploy_context, update_operation
from stack_orchestrator.deploy.deploy_types import DeployCommandContext
from stack_orchestrator.deploy.deployment_context import DeploymentContext
@@ -52,7 +52,7 @@ def make_deploy_context(ctx) -> DeployCommandContext:
context: DeploymentContext = ctx.obj
stack_file_path = context.get_stack_file()
env_file = context.get_env_file()
cluster_name = context.get_cluster_name()
cluster_name = context.get_cluster_id()
if constants.deploy_to_key in context.spec.obj:
deployment_type = context.spec.obj[constants.deploy_to_key]
else:
@@ -147,4 +147,12 @@ def logs(ctx, tail, follow, extra_args):
@command.command()
@click.pass_context
def status(ctx):
print(f"Context: {ctx.parent.obj}")
ctx.obj = make_deploy_context(ctx)
status_operation(ctx)
@command.command()
@click.pass_context
def update(ctx):
ctx.obj = make_deploy_context(ctx)
update_operation(ctx)
@@ -14,29 +14,39 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import hashlib
import os
from pathlib import Path
from stack_orchestrator import constants
from stack_orchestrator.util import get_yaml
from stack_orchestrator.deploy.stack import Stack
from stack_orchestrator.deploy.spec import Spec
class DeploymentContext:
deployment_dir: Path
id: str
spec: Spec
stack: Stack
def get_stack_file(self):
return self.deployment_dir.joinpath("stack.yml")
return self.deployment_dir.joinpath(constants.stack_file_name)
def get_spec_file(self):
return self.deployment_dir.joinpath("spec.yml")
return self.deployment_dir.joinpath(constants.spec_file_name)
def get_env_file(self):
return self.deployment_dir.joinpath("config.env")
return self.deployment_dir.joinpath(constants.config_file_name)
# TODO: implement me
def get_cluster_name(self):
return None
def get_deployment_file(self):
return self.deployment_dir.joinpath(constants.deployment_file_name)
def get_compose_dir(self):
return self.deployment_dir.joinpath(constants.compose_dir_name)
def get_cluster_id(self):
return self.id
def init(self, dir):
self.deployment_dir = dir
@@ -44,3 +54,16 @@ class DeploymentContext:
self.spec.init_from_file(self.get_spec_file())
self.stack = Stack(self.spec.obj["stack"])
self.stack.init_from_file(self.get_stack_file())
deployment_file_path = self.get_deployment_file()
if deployment_file_path.exists():
with deployment_file_path:
obj = get_yaml().load(open(deployment_file_path, "r"))
self.id = obj[constants.cluster_id_key]
# Handle the case of a legacy deployment with no file
# Code below is intended to match the output from _make_default_cluster_name()
# TODO: remove when we no longer need to support legacy deployments
else:
path = os.path.realpath(os.path.abspath(self.get_compose_dir()))
unique_cluster_descriptor = f"{path},{self.get_stack_file()},None,None"
hash = hashlib.md5(unique_cluster_descriptor.encode()).hexdigest()[:16]
self.id = f"{constants.cluster_name_prefix}{hash}"
+65 -19
View File
@@ -20,11 +20,13 @@ from pathlib import Path
from typing import List
import random
from shutil import copy, copyfile, copytree
from secrets import token_hex
import sys
from stack_orchestrator import constants
from stack_orchestrator.opts import opts
from stack_orchestrator.util import (get_stack_file_path, get_parsed_deployment_spec, get_parsed_stack_config,
global_options, get_yaml, get_pod_list, get_pod_file_path, pod_has_scripts,
get_pod_script_paths, get_plugin_code_paths, error_exit)
get_pod_script_paths, get_plugin_code_paths, error_exit, env_var_map_from_file)
from stack_orchestrator.deploy.deploy_types import LaconicStackSetupCommand
from stack_orchestrator.deploy.deployer_factory import getDeployerConfigGenerator
from stack_orchestrator.deploy.deployment_context import DeploymentContext
@@ -103,8 +105,8 @@ def _fixup_pod_file(pod, spec, compose_dir):
}
pod["volumes"][volume] = new_volume_spec
# Fix up ports
if "ports" in spec:
spec_ports = spec["ports"]
if "network" in spec and "ports" in spec["network"]:
spec_ports = spec["network"]["ports"]
for container_name, container_ports in spec_ports.items():
if container_name in pod["services"]:
pod["services"][container_name]["ports"] = container_ports
@@ -243,12 +245,13 @@ def _parse_config_variables(variable_values: str):
variable_name = variable_value_pair[0]
variable_value = variable_value_pair[1]
result_values[variable_name] = variable_value
result = {"config": result_values}
result = result_values
return result
@click.command()
@click.option("--config", help="Provide config variables for the deployment")
@click.option("--config-file", help="Provide config variables in a file for the deployment")
@click.option("--kube-config", help="Provide a config file for a k8s deployment")
@click.option("--image-registry", help="Provide a container image registry url for this k8s cluster")
@click.option("--output", required=True, help="Write yaml spec file here")
@@ -256,14 +259,32 @@ def _parse_config_variables(variable_values: str):
help="Map ports to the host as one of: any-variable-random (default), "
"localhost-same, any-same, localhost-fixed-random, any-fixed-random")
@click.pass_context
def init(ctx, config, kube_config, image_registry, output, map_ports_to_host):
yaml = get_yaml()
def init(ctx, config, config_file, kube_config, image_registry, output, map_ports_to_host):
stack = global_options(ctx).stack
debug = global_options(ctx).debug
deployer_type = ctx.obj.deployer.type
default_spec_file_content = call_stack_deploy_init(ctx.obj)
deploy_command_context = ctx.obj
return init_operation(
deploy_command_context,
stack, deployer_type,
config, config_file,
kube_config,
image_registry,
output,
map_ports_to_host)
# The init command's implementation is in a separate function so that we can
# call it from other commands, bypassing the click decoration stuff
def init_operation(deploy_command_context, stack, deployer_type, config,
config_file, kube_config, image_registry, output, map_ports_to_host):
default_spec_file_content = call_stack_deploy_init(deploy_command_context)
spec_file_content = {"stack": stack, constants.deploy_to_key: deployer_type}
if deployer_type == "k8s":
if kube_config is None:
error_exit("--kube-config must be supplied with --deploy-to k8s")
if image_registry is None:
error_exit("--image-registry must be supplied with --deploy-to k8s")
spec_file_content.update({constants.kube_config_key: kube_config})
spec_file_content.update({constants.image_resigtry_key: image_registry})
else:
@@ -275,17 +296,25 @@ def init(ctx, config, kube_config, image_registry, output, map_ports_to_host):
if default_spec_file_content:
spec_file_content.update(default_spec_file_content)
config_variables = _parse_config_variables(config)
# Implement merge, since update() overwrites
if config_variables:
# Implement merge, since update() overwrites
orig_config = spec_file_content.get("config", {})
new_config = config_variables["config"]
new_config = config_variables
merged_config = {**new_config, **orig_config}
spec_file_content.update({"config": merged_config})
if debug:
print(f"Creating spec file for stack: {stack} with content: {spec_file_content}")
if config_file:
config_file_path = Path(config_file)
if not config_file_path.exists():
error_exit(f"config file: {config_file} does not exist")
config_file_variables = env_var_map_from_file(config_file_path)
if config_file_variables:
orig_config = spec_file_content.get("config", {})
new_config = config_file_variables
merged_config = {**new_config, **orig_config}
spec_file_content.update({"config": merged_config})
ports = _get_mapped_ports(stack, map_ports_to_host)
spec_file_content["ports"] = ports
spec_file_content.update({"network": {"ports": ports}})
named_volumes = _get_named_volumes(stack)
if named_volumes:
@@ -294,8 +323,11 @@ def init(ctx, config, kube_config, image_registry, output, map_ports_to_host):
volume_descriptors[named_volume] = f"./data/{named_volume}"
spec_file_content["volumes"] = volume_descriptors
if opts.o.debug:
print(f"Creating spec file for stack: {stack} with content: {spec_file_content}")
with open(output, "w") as output_file:
yaml.dump(spec_file_content, output_file)
get_yaml().dump(spec_file_content, output_file)
def _write_config_file(spec_file: Path, config_env_file: Path):
@@ -321,6 +353,13 @@ def _copy_files_to_directory(file_paths: List[Path], directory: Path):
copy(path, os.path.join(directory, os.path.basename(path)))
def _create_deployment_file(deployment_dir: Path):
deployment_file_path = deployment_dir.joinpath(constants.deployment_file_name)
cluster = f"{constants.cluster_name_prefix}{token_hex(8)}"
with open(deployment_file_path, "w") as output_file:
output_file.write(f"{constants.cluster_id_key}: {cluster}\n")
@click.command()
@click.option("--spec-file", required=True, help="Spec file to use to create this deployment")
@click.option("--deployment-dir", help="Create deployment files in this directory")
@@ -329,12 +368,19 @@ def _copy_files_to_directory(file_paths: List[Path], directory: Path):
@click.option("--initial-peers", help="Initial set of persistent peers")
@click.pass_context
def create(ctx, spec_file, deployment_dir, network_dir, initial_peers):
deployment_command_context = ctx.obj
return create_operation(deployment_command_context, spec_file, deployment_dir, network_dir, initial_peers)
# The init command's implementation is in a separate function so that we can
# call it from other commands, bypassing the click decoration stuff
def create_operation(deployment_command_context, spec_file, deployment_dir, network_dir, initial_peers):
parsed_spec = get_parsed_deployment_spec(spec_file)
stack_name = parsed_spec["stack"]
deployment_type = parsed_spec[constants.deploy_to_key]
stack_file = get_stack_file_path(stack_name)
parsed_stack = get_parsed_stack_config(stack_name)
if global_options(ctx).debug:
if opts.o.debug:
print(f"parsed spec: {parsed_spec}")
if deployment_dir is None:
deployment_dir_path = _make_default_deployment_dir()
@@ -344,10 +390,11 @@ def create(ctx, spec_file, deployment_dir, network_dir, initial_peers):
error_exit(f"{deployment_dir_path} already exists")
os.mkdir(deployment_dir_path)
# Copy spec file and the stack file into the deployment dir
copyfile(spec_file, deployment_dir_path.joinpath("spec.yml"))
copyfile(spec_file, deployment_dir_path.joinpath(constants.spec_file_name))
copyfile(stack_file, deployment_dir_path.joinpath(os.path.basename(stack_file)))
_create_deployment_file(deployment_dir_path)
# Copy any config varibles from the spec file into an env file suitable for compose
_write_config_file(spec_file, deployment_dir_path.joinpath("config.env"))
_write_config_file(spec_file, deployment_dir_path.joinpath(constants.config_file_name))
# Copy any k8s config file into the deployment dir
if deployment_type == "k8s":
_write_kube_config_file(Path(parsed_spec[constants.kube_config_key]),
@@ -366,7 +413,7 @@ def create(ctx, spec_file, deployment_dir, network_dir, initial_peers):
extra_config_dirs = _find_extra_config_dirs(parsed_pod_file, pod)
destination_pod_dir = destination_pods_dir.joinpath(pod)
os.mkdir(destination_pod_dir)
if global_options(ctx).debug:
if opts.o.debug:
print(f"extra config dirs: {extra_config_dirs}")
_fixup_pod_file(parsed_pod_file, parsed_spec, destination_compose_dir)
with open(destination_compose_dir.joinpath("docker-compose-%s.yml" % pod), "w") as output_file:
@@ -390,7 +437,6 @@ def create(ctx, spec_file, deployment_dir, network_dir, initial_peers):
# Delegate to the stack's Python code
# The deploy create command doesn't require a --stack argument so we need to insert the
# stack member here.
deployment_command_context = ctx.obj
deployment_command_context.stack = stack_name
deployment_context = DeploymentContext()
deployment_context.init(deployment_dir_path)
+98 -9
View File
@@ -17,34 +17,117 @@ from kubernetes import client
from typing import Any, List, Set
from stack_orchestrator.opts import opts
from stack_orchestrator.util import env_var_map_from_file
from stack_orchestrator.deploy.k8s.helpers import named_volumes_from_pod_files, volume_mounts_for_service, volumes_for_pod_files
from stack_orchestrator.deploy.k8s.helpers import get_node_pv_mount_path
from stack_orchestrator.deploy.k8s.helpers import env_var_map_from_file, envs_from_environment_variables_map
from stack_orchestrator.deploy.k8s.helpers import envs_from_environment_variables_map
from stack_orchestrator.deploy.deploy_util import parsed_pod_files_map_from_file_names, images_for_deployment
from stack_orchestrator.deploy.deploy_types import DeployEnvVars
from stack_orchestrator.deploy.spec import Spec
from stack_orchestrator.deploy.images import remote_tag_for_image
class ClusterInfo:
parsed_pod_yaml_map: Any
image_set: Set[str] = set()
app_name: str = "test-app"
deployment_name: str = "test-deployment"
app_name: str
environment_variables: DeployEnvVars
remote_image_repo: str
spec: Spec
def __init__(self) -> None:
pass
def int(self, pod_files: List[str], compose_env_file, remote_image_repo):
def int(self, pod_files: List[str], compose_env_file, deployment_name, spec: Spec):
self.parsed_pod_yaml_map = parsed_pod_files_map_from_file_names(pod_files)
# Find the set of images in the pods
self.image_set = images_for_deployment(pod_files)
self.environment_variables = DeployEnvVars(env_var_map_from_file(compose_env_file))
self.remote_image_repo = remote_image_repo
self.app_name = deployment_name
self.spec = spec
if (opts.o.debug):
print(f"Env vars: {self.environment_variables.map}")
def get_ingress(self):
# No ingress for a deployment that has no http-proxy defined, for now
http_proxy_info_list = self.spec.get_http_proxy()
ingress = None
if http_proxy_info_list:
# TODO: handle multiple definitions
http_proxy_info = http_proxy_info_list[0]
if opts.o.debug:
print(f"http-proxy: {http_proxy_info}")
# TODO: good enough parsing for webapp deployment for now
host_name = http_proxy_info["host-name"]
rules = []
tls = [client.V1IngressTLS(
hosts=[host_name],
secret_name=f"{self.app_name}-tls"
)]
paths = []
for route in http_proxy_info["routes"]:
path = route["path"]
proxy_to = route["proxy-to"]
if opts.o.debug:
print(f"proxy config: {path} -> {proxy_to}")
# proxy_to has the form <service>:<port>
proxy_to_port = int(proxy_to.split(":")[1])
paths.append(client.V1HTTPIngressPath(
path_type="Prefix",
path=path,
backend=client.V1IngressBackend(
service=client.V1IngressServiceBackend(
# TODO: this looks wrong
name=f"{self.app_name}-service",
# TODO: pull port number from the service
port=client.V1ServiceBackendPort(number=proxy_to_port)
)
)
))
rules.append(client.V1IngressRule(
host=host_name,
http=client.V1HTTPIngressRuleValue(
paths=paths
)
))
spec = client.V1IngressSpec(
tls=tls,
rules=rules
)
ingress = client.V1Ingress(
metadata=client.V1ObjectMeta(
name=f"{self.app_name}-ingress",
annotations={
"kubernetes.io/ingress.class": "nginx",
"cert-manager.io/cluster-issuer": "letsencrypt-prod"
}
),
spec=spec
)
return ingress
# TODO: suppoprt multiple services
def get_service(self):
for pod_name in self.parsed_pod_yaml_map:
pod = self.parsed_pod_yaml_map[pod_name]
services = pod["services"]
for service_name in services:
service_info = services[service_name]
port = int(service_info["ports"][0])
if opts.o.debug:
print(f"service port: {port}")
service = client.V1Service(
metadata=client.V1ObjectMeta(name=f"{self.app_name}-service"),
spec=client.V1ServiceSpec(
type="ClusterIP",
ports=[client.V1ServicePort(
port=port,
target_port=port
)],
selector={"app": self.app_name}
)
)
return service
def get_pvcs(self):
result = []
volumes = named_volumes_from_pod_files(self.parsed_pod_yaml_map)
@@ -95,14 +178,20 @@ class ClusterInfo:
container_name = service_name
service_info = services[service_name]
image = service_info["image"]
port = int(service_info["ports"][0])
if opts.o.debug:
print(f"image: {image}")
print(f"service port: {port}")
# Re-write the image tag for remote deployment
image_to_use = remote_tag_for_image(image, self.remote_image_repo) if self.remote_image_repo is not None else image
image_to_use = remote_tag_for_image(
image, self.spec.get_image_registry()) if self.spec.get_image_registry() is not None else image
volume_mounts = volume_mounts_for_service(self.parsed_pod_yaml_map, service_name)
container = client.V1Container(
name=container_name,
image=image_to_use,
image_pull_policy="Always",
env=envs_from_environment_variables_map(self.environment_variables.map),
ports=[client.V1ContainerPort(container_port=80)],
ports=[client.V1ContainerPort(container_port=port)],
volume_mounts=volume_mounts,
resources=client.V1ResourceRequirements(
requests={"cpu": "100m", "memory": "200Mi"},
@@ -123,7 +212,7 @@ class ClusterInfo:
deployment = client.V1Deployment(
api_version="apps/v1",
kind="Deployment",
metadata=client.V1ObjectMeta(name=self.deployment_name),
metadata=client.V1ObjectMeta(name=f"{self.app_name}-deployment"),
spec=spec,
)
return deployment
+192 -23
View File
@@ -13,6 +13,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from datetime import datetime, timezone
from pathlib import Path
from kubernetes import client, config
@@ -23,6 +25,21 @@ from stack_orchestrator.deploy.k8s.helpers import pods_in_deployment, log_stream
from stack_orchestrator.deploy.k8s.cluster_info import ClusterInfo
from stack_orchestrator.opts import opts
from stack_orchestrator.deploy.deployment_context import DeploymentContext
from stack_orchestrator.util import error_exit
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def _check_delete_exception(e: client.exceptions.ApiException):
if e.status == 404:
if opts.o.debug:
print("Failed to delete object, continuing")
else:
error_exit(f"k8s api error: {e}")
class K8sDeployer(Deployer):
@@ -30,9 +47,10 @@ class K8sDeployer(Deployer):
type: str
core_api: client.CoreV1Api
apps_api: client.AppsV1Api
networking_api: client.NetworkingV1Api
k8s_namespace: str = "default"
kind_cluster_name: str
cluster_info : ClusterInfo
cluster_info: ClusterInfo
deployment_dir: Path
deployment_context: DeploymentContext
@@ -45,7 +63,7 @@ class K8sDeployer(Deployer):
self.deployment_context = deployment_context
self.kind_cluster_name = compose_project_name
self.cluster_info = ClusterInfo()
self.cluster_info.int(compose_files, compose_env_file, deployment_context.spec.obj[constants.image_resigtry_key])
self.cluster_info.int(compose_files, compose_env_file, compose_project_name, deployment_context.spec)
if (opts.o.debug):
print(f"Deployment dir: {deployment_context.deployment_dir}")
print(f"Compose files: {compose_files}")
@@ -60,9 +78,12 @@ class K8sDeployer(Deployer):
# Get the config file and pass to load_kube_config()
config.load_kube_config(config_file=self.deployment_dir.joinpath(constants.kube_config_filename).as_posix())
self.core_api = client.CoreV1Api()
self.networking_api = client.NetworkingV1Api()
self.apps_api = client.AppsV1Api()
self.custom_obj_api = client.CustomObjectsApi()
def up(self, detach, services):
if self.is_kind():
# Create the kind cluster
create_cluster(self.kind_cluster_name, self.deployment_dir.joinpath(constants.kind_config_filename))
@@ -102,6 +123,28 @@ class K8sDeployer(Deployer):
print(f"{deployment_resp.metadata.namespace} {deployment_resp.metadata.name} \
{deployment_resp.metadata.generation} {deployment_resp.spec.template.spec.containers[0].image}")
service: client.V1Service = self.cluster_info.get_service()
service_resp = self.core_api.create_namespaced_service(
namespace=self.k8s_namespace,
body=service
)
if opts.o.debug:
print("Service created:")
print(f"{service_resp}")
# TODO: disable ingress for kind
ingress: client.V1Ingress = self.cluster_info.get_ingress()
if opts.o.debug:
print(f"Sending this ingress: {ingress}")
ingress_resp = self.networking_api.create_namespaced_ingress(
namespace=self.k8s_namespace,
body=ingress
)
if opts.o.debug:
print("Ingress created:")
print(f"{ingress_resp}")
def down(self, timeout, volumes):
self.connect_api()
# Delete the k8s objects
@@ -110,41 +153,140 @@ class K8sDeployer(Deployer):
for pv in pvs:
if opts.o.debug:
print(f"Deleting this pv: {pv}")
pv_resp = self.core_api.delete_persistent_volume(name=pv.metadata.name)
if opts.o.debug:
print("PV deleted:")
print(f"{pv_resp}")
try:
pv_resp = self.core_api.delete_persistent_volume(name=pv.metadata.name)
if opts.o.debug:
print("PV deleted:")
print(f"{pv_resp}")
except client.exceptions.ApiException as e:
_check_delete_exception(e)
# Figure out the PVCs for this deployment
pvcs = self.cluster_info.get_pvcs()
for pvc in pvcs:
if opts.o.debug:
print(f"Deleting this pvc: {pvc}")
pvc_resp = self.core_api.delete_namespaced_persistent_volume_claim(name=pvc.metadata.name, namespace=self.k8s_namespace)
if opts.o.debug:
print("PVCs deleted:")
print(f"{pvc_resp}")
# Process compose files into a Deployment
try:
pvc_resp = self.core_api.delete_namespaced_persistent_volume_claim(
name=pvc.metadata.name, namespace=self.k8s_namespace
)
if opts.o.debug:
print("PVCs deleted:")
print(f"{pvc_resp}")
except client.exceptions.ApiException as e:
_check_delete_exception(e)
deployment = self.cluster_info.get_deployment()
# Create the k8s objects
if opts.o.debug:
print(f"Deleting this deployment: {deployment}")
self.apps_api.delete_namespaced_deployment(
name=deployment.metadata.name, namespace=self.k8s_namespace
)
try:
self.apps_api.delete_namespaced_deployment(
name=deployment.metadata.name, namespace=self.k8s_namespace
)
except client.exceptions.ApiException as e:
_check_delete_exception(e)
service: client.V1Service = self.cluster_info.get_service()
if opts.o.debug:
print(f"Deleting service: {service}")
try:
self.core_api.delete_namespaced_service(
namespace=self.k8s_namespace,
name=service.metadata.name
)
except client.exceptions.ApiException as e:
_check_delete_exception(e)
# TODO: disable ingress for kind
ingress: client.V1Ingress = self.cluster_info.get_ingress()
if opts.o.debug:
print(f"Deleting this ingress: {ingress}")
try:
self.networking_api.delete_namespaced_ingress(
name=ingress.metadata.name, namespace=self.k8s_namespace
)
except client.exceptions.ApiException as e:
_check_delete_exception(e)
if self.is_kind():
# Destroy the kind cluster
destroy_cluster(self.kind_cluster_name)
def ps(self):
def status(self):
self.connect_api()
# Call whatever API we need to get the running container list
ret = self.core_api.list_pod_for_all_namespaces(watch=False)
if ret.items:
for i in ret.items:
print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))
ret = self.core_api.list_node(pretty=True, watch=False)
return []
all_pods = self.core_api.list_pod_for_all_namespaces(watch=False)
pods = []
if all_pods.items:
for p in all_pods.items:
if self.cluster_info.app_name in p.metadata.name:
pods.append(p)
if not pods:
return
hostname = "?"
ip = "?"
tls = "?"
try:
ingress = self.networking_api.read_namespaced_ingress(namespace=self.k8s_namespace,
name=self.cluster_info.get_ingress().metadata.name)
cert = self.custom_obj_api.get_namespaced_custom_object(
group="cert-manager.io",
version="v1",
namespace=self.k8s_namespace,
plural="certificates",
name=ingress.spec.tls[0].secret_name
)
hostname = ingress.spec.tls[0].hosts[0]
ip = ingress.status.load_balancer.ingress[0].ip
tls = "notBefore: %s, notAfter: %s" % (cert["status"]["notBefore"], cert["status"]["notAfter"])
except: # noqa: E722
pass
print("Ingress:")
print("\tHostname:", hostname)
print("\tIP:", ip)
print("\tTLS:", tls)
print("")
print("Pods:")
for p in pods:
if p.metadata.deletion_timestamp:
print(f"\t{p.metadata.namespace}/{p.metadata.name}: Terminating ({p.metadata.deletion_timestamp})")
else:
print(f"\t{p.metadata.namespace}/{p.metadata.name}: Running ({p.metadata.creation_timestamp})")
def ps(self):
self.connect_api()
pods = self.core_api.list_pod_for_all_namespaces(watch=False)
ret = []
for p in pods.items:
if self.cluster_info.app_name in p.metadata.name:
pod_ip = p.status.pod_ip
ports = AttrDict()
for c in p.spec.containers:
if c.ports:
for prt in c.ports:
ports[str(prt.container_port)] = [AttrDict({
"HostIp": pod_ip,
"HostPort": prt.container_port
})]
ret.append(AttrDict({
"id": f"{p.metadata.namespace}/{p.metadata.name}",
"name": p.metadata.name,
"namespace": p.metadata.namespace,
"network_settings": AttrDict({
"ports": ports
})
}))
return ret
def port(self, service, private_port):
# Since we handle the port mapping, need to figure out where this comes from
@@ -164,7 +306,34 @@ class K8sDeployer(Deployer):
log_data = self.core_api.read_namespaced_pod_log(k8s_pod_name, namespace="default", container="test")
return log_stream_from_string(log_data)
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, detach=False):
def update(self):
self.connect_api()
ref_deployment = self.cluster_info.get_deployment()
deployment = self.apps_api.read_namespaced_deployment(
name=ref_deployment.metadata.name,
namespace=self.k8s_namespace
)
new_env = ref_deployment.spec.template.spec.containers[0].env
for container in deployment.spec.template.spec.containers:
old_env = container.env
if old_env != new_env:
container.env = new_env
deployment.spec.template.metadata.annotations = {
"kubectl.kubernetes.io/restartedAt": datetime.utcnow()
.replace(tzinfo=timezone.utc)
.isoformat()
}
self.apps_api.patch_namespaced_deployment(
name=ref_deployment.metadata.name,
namespace=self.k8s_namespace,
body=deployment
)
def run(self, image: str, command=None, user=None, volumes=None, entrypoint=None, env={}, ports=[], detach=False):
# We need to figure out how to do this -- check why we're being called first
pass
-5
View File
@@ -14,7 +14,6 @@
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from kubernetes import client
from dotenv import dotenv_values
import os
from pathlib import Path
import subprocess
@@ -224,7 +223,3 @@ def generate_kind_config(deployment_dir: Path):
f"{port_mappings_yml}\n"
f"{mounts_yml}\n"
)
def env_var_map_from_file(file: Path) -> Mapping[str, str]:
return dotenv_values(file)
+12
View File
@@ -16,6 +16,7 @@
from pathlib import Path
import typing
from stack_orchestrator.util import get_yaml
from stack_orchestrator import constants
class Spec:
@@ -28,3 +29,14 @@ class Spec:
def init_from_file(self, file_path: Path):
with file_path:
self.obj = get_yaml().load(open(file_path, "r"))
def get_image_registry(self):
return (self.obj[constants.image_resigtry_key]
if self.obj and constants.image_resigtry_key in self.obj
else None)
def get_http_proxy(self):
return (self.obj[constants.network_key][constants.http_proxy_key]
if self.obj and constants.network_key in self.obj
and constants.http_proxy_key in self.obj[constants.network_key]
else None)
@@ -0,0 +1,123 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import click
import os
from pathlib import Path
from urllib.parse import urlparse
from tempfile import NamedTemporaryFile
from stack_orchestrator.util import error_exit, global_options2
from stack_orchestrator.deploy.deployment_create import init_operation, create_operation
from stack_orchestrator.deploy.deploy import create_deploy_context
from stack_orchestrator.deploy.deploy_types import DeployCommandContext
def _fixup_container_tag(deployment_dir: str, image: str):
deployment_dir_path = Path(deployment_dir)
compose_file = deployment_dir_path.joinpath("compose", "docker-compose-webapp-template.yml")
# replace "cerc/webapp-container:local" in the file with our image tag
with open(compose_file) as rfile:
contents = rfile.read()
contents = contents.replace("cerc/webapp-container:local", image)
with open(compose_file, "w") as wfile:
wfile.write(contents)
def _fixup_url_spec(spec_file_name: str, url: str):
# url is like: https://example.com/path
parsed_url = urlparse(url)
http_proxy_spec = f'''
http-proxy:
- host-name: {parsed_url.hostname}
routes:
- path: '{parsed_url.path if parsed_url.path else "/"}'
proxy-to: webapp:3000
'''
spec_file_path = Path(spec_file_name)
with open(spec_file_path) as rfile:
contents = rfile.read()
contents = contents + http_proxy_spec
with open(spec_file_path, "w") as wfile:
wfile.write(contents)
def create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
# Do the equivalent of:
# 1. laconic-so --stack webapp-template deploy --deploy-to k8s init --output webapp-spec.yml
# --config (eqivalent of the contents of my-config.env)
# 2. laconic-so --stack webapp-template deploy --deploy-to k8s create --deployment-dir test-deployment
# --spec-file webapp-spec.yml
# 3. Replace the container image tag with the specified image
deployment_dir_path = Path(deployment_dir)
# Check the deployment dir does not exist
if deployment_dir_path.exists():
error_exit(f"Deployment dir {deployment_dir} already exists")
# Generate a temporary file name for the spec file
tf = NamedTemporaryFile(prefix="webapp-", suffix=".yml", delete=False)
spec_file_name = tf.name
# Specify the webapp template stack
stack = "webapp-template"
# TODO: support env file
deploy_command_context: DeployCommandContext = create_deploy_context(
global_options2(ctx), None, stack, None, None, None, env_file, "k8s"
)
init_operation(
deploy_command_context,
stack,
"k8s",
None,
env_file,
kube_config,
image_registry,
spec_file_name,
None
)
# Add the TLS and DNS spec
_fixup_url_spec(spec_file_name, url)
create_operation(
deploy_command_context,
spec_file_name,
deployment_dir,
None,
None
)
# Fix up the container tag inside the deployment compose file
_fixup_container_tag(deployment_dir, image)
os.remove(spec_file_name)
@click.group()
@click.pass_context
def command(ctx):
'''manage a webapp deployment'''
# Check that --stack wasn't supplied
if ctx.parent.obj.stack:
error_exit("--stack can't be supplied with the deploy-webapp command")
@command.command()
@click.option("--kube-config", help="Provide a config file for a k8s deployment")
@click.option("--image-registry", help="Provide a container image registry url for this k8s cluster")
@click.option("--deployment-dir", help="Create deployment files in this directory", required=True)
@click.option("--image", help="image to deploy", required=True)
@click.option("--url", help="url to serve", required=True)
@click.option("--env-file", help="environment file for webapp")
@click.pass_context
def create(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
'''create a deployment for the specified webapp container'''
return create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file)
@@ -0,0 +1,267 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import json
import os
import shlex
import shutil
import sys
import tempfile
import click
from stack_orchestrator.deploy.webapp import deploy_webapp
from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient,
build_container_image, push_container_image,
file_hash, deploy_to_k8s, publish_deployment,
hostname_for_deployment_request, generate_hostname_for_app)
def process_app_deployment_request(
ctx,
laconic: LaconicRegistryClient,
app_deployment_request,
deployment_record_namespace,
dns_record_namespace,
dns_suffix,
deployment_parent_dir,
kube_config,
image_registry
):
# 1. look up application
app = laconic.get_record(app_deployment_request.attributes.application, require=True)
# 2. determine dns
requested_name = hostname_for_deployment_request(app_deployment_request, laconic)
# HACK
if "." in requested_name:
raise Exception("Only unqualified hostnames allowed at this time.")
fqdn = f"{requested_name}.{dns_suffix}"
container_tag = "%s:local" % app.attributes.name.replace("@", "")
# 3. check ownership of existing dnsrecord vs this request
# TODO: Support foreign DNS
dns_crn = f"{dns_record_namespace}/{fqdn}"
dns_record = laconic.get_record(dns_crn)
if dns_record:
dns_record_owners = dns_record.owners
dns_request_owners = []
if dns_record.request:
prev_request = laconic.get_record(dns_record.request, require=True)
dns_request_owners = prev_request.owners
owner_match = None
for owner in app_deployment_request.owners:
if owner in dns_request_owners or owner in dns_record_owners:
owner_match = owner
if owner_match:
print("Matched DnsRecord ownership to", owner)
else:
raise Exception("Unable to confirm ownership of DnsRecord %s for request %s" %
(dns_record.id, app_deployment_request.id))
# 4. get build and runtime config from request
env_filename = None
if app_deployment_request.attributes.config and "env" in app_deployment_request.attributes.config:
env_filename = tempfile.mktemp()
with open(env_filename, 'w') as file:
for k, v in app_deployment_request.attributes.config["env"].items():
file.write("%s=%s\n" % (k, shlex.quote(str(v))))
# 5. determine new or existing deployment
# a. check for deployment crn
app_deployment_crn = f"{deployment_record_namespace}/{fqdn}"
if app_deployment_request.attributes.deployment:
app_deployment_crn = app_deployment_request.attributes.deployment
if not app_deployment_crn.startswith(deployment_record_namespace):
raise Exception("Deployment CRN %s is not in a supported namespace" % app_deployment_request.attributes.deployment)
deployment_record = laconic.get_record(app_deployment_crn)
deployment_dir = os.path.join(deployment_parent_dir, fqdn)
deployment_config_file = os.path.join(deployment_dir, "config.env")
# b. check for deployment directory (create if necessary)
if not os.path.exists(deployment_dir):
if deployment_record:
raise ("Deployment record %s exists, but not deployment dir %s. Please remove name." %
(app_deployment_crn, deployment_dir))
print("deploy_webapp", deployment_dir)
deploy_webapp.create_deployment(ctx, deployment_dir, container_tag,
f"https://{fqdn}", kube_config, image_registry, env_filename)
elif env_filename:
shutil.copyfile(env_filename, deployment_config_file)
needs_k8s_deploy = False
# 6. build container (if needed)
if not deployment_record or deployment_record.attributes.application != app.id:
build_container_image(app, container_tag)
push_container_image(deployment_dir)
needs_k8s_deploy = True
# 7. update config (if needed)
if not deployment_record or file_hash(deployment_config_file) != deployment_record.attributes.meta.config:
needs_k8s_deploy = True
# 8. update k8s deployment
if needs_k8s_deploy:
print("Deploying to k8s")
deploy_to_k8s(
deployment_record,
deployment_dir,
)
publish_deployment(
laconic,
app,
deployment_record,
app_deployment_crn,
dns_record,
dns_crn,
deployment_dir,
app_deployment_request
)
def load_known_requests(filename):
if filename and os.path.exists(filename):
return json.load(open(filename, "r"))
return {}
def dump_known_requests(filename, requests):
if not filename:
return
known_requests = load_known_requests(filename)
for r in requests:
known_requests[r.id] = r.createTime
json.dump(known_requests, open(filename, "w"))
@click.command()
@click.option("--kube-config", help="Provide a config file for a k8s deployment")
@click.option("--kube-config", help="Provide a config file for a k8s deployment")
@click.option("--laconic-config", help="Provide a config file for laconicd", required=True)
@click.option("--image-registry", help="Provide a container image registry url for this k8s cluster")
@click.option("--deployment-parent-dir", help="Create deployment directories beneath this directory", required=True)
@click.option("--request-id", help="The ApplicationDeploymentRequest to process")
@click.option("--discover", help="Discover and process all pending ApplicationDeploymentRequests", is_flag=True, default=False)
@click.option("--state-file", help="File to store state about previously seen requests.")
@click.option("--only-update-state", help="Only update the state file, don't process any requests anything.", is_flag=True)
@click.option("--dns-suffix", help="DNS domain to use eg, laconic.servesthe.world")
@click.option("--record-namespace-dns", help="eg, crn://laconic/dns")
@click.option("--record-namespace-deployments", help="eg, crn://laconic/deployments")
@click.pass_context
def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_dir,
request_id, discover, state_file, only_update_state,
dns_suffix, record_namespace_dns, record_namespace_deployments):
if request_id and discover:
print("Cannot specify both --request-id and --discover", file=sys.stderr)
sys.exit(2)
if not request_id and not discover:
print("Must specify either --request-id or --discover", file=sys.stderr)
sys.exit(2)
if only_update_state and not state_file:
print("--only-update-state requires --state-file", file=sys.stderr)
sys.exit(2)
if not only_update_state:
if not record_namespace_dns or not record_namespace_deployments or not dns_suffix:
print("--dns-suffix, --record-namespace-dns, and --record-namespace-deployments are all required", file=sys.stderr)
sys.exit(2)
laconic = LaconicRegistryClient(laconic_config)
# Find deployment requests.
# single request
if request_id:
requests = [laconic.get_record(request_id, require=True)]
# all requests
elif discover:
requests = laconic.app_deployment_requests()
if only_update_state:
dump_known_requests(state_file, requests)
return
previous_requests = load_known_requests(state_file)
# Collapse related requests.
requests.sort(key=lambda r: r.createTime)
requests.reverse()
requests_by_name = {}
for r in requests:
app = laconic.get_record(r.attributes.application)
if not app:
print("Skipping request %s, cannot locate app." % r.id)
continue
requested_name = r.attributes.dns
if not requested_name:
requested_name = generate_hostname_for_app(app)
print("Generating name %s for request %s." % (requested_name, r.id))
if requested_name not in requests_by_name:
print(
"Found request %s to run application %s on %s."
% (r.id, r.attributes.application, requested_name)
)
requests_by_name[requested_name] = r
else:
print(
"Ignoring request %s, it is superseded by %s."
% (r.id, requests_by_name[requested_name].id)
)
# Find deployments.
deployments = laconic.app_deployments()
deployments_by_request = {}
for d in deployments:
if d.attributes.request:
deployments_by_request[d.attributes.request] = d
requests_to_execute = []
for r in requests_by_name.values():
if r.id not in deployments_by_request:
if r.id not in previous_requests:
print(f"Request {r.id} needs to processed.")
requests_to_execute.append(r)
else:
print(
f"Skipping unsatisfied request {r.id} because we have seen it before."
)
else:
print(f"Found satisfied request {r.id} at {deployments_by_request[r.id].names[0]}")
print("Found %d unsatisfied request(s) to process." % len(requests_to_execute))
for r in requests_to_execute:
try:
process_app_deployment_request(
ctx,
laconic,
r,
record_namespace_deployments,
record_namespace_dns,
dns_suffix,
deployment_parent_dir,
kube_config,
image_registry
)
finally:
dump_known_requests(state_file, [r])
@@ -22,18 +22,21 @@
import hashlib
import click
from dotenv import dotenv_values
from stack_orchestrator import constants
from stack_orchestrator.deploy.deployer_factory import getDeployer
WEBAPP_PORT = 3000
@click.command()
@click.option("--image", help="image to deploy", required=True)
@click.option("--deploy-to", default="compose", help="deployment type ([Docker] 'compose' or 'k8s')")
@click.option("--env-file", help="environment file for webapp")
@click.option("--port", help="port to use (default random)")
@click.pass_context
def command(ctx, image, deploy_to, env_file):
'''build the specified webapp container'''
def command(ctx, image, env_file, port):
'''run the specified webapp container'''
env = {}
if env_file:
@@ -43,16 +46,19 @@ def command(ctx, image, deploy_to, env_file):
hash = hashlib.md5(unique_cluster_descriptor.encode()).hexdigest()
cluster = f"laconic-webapp-{hash}"
deployer = getDeployer(deploy_to,
deployer = getDeployer(type=constants.compose_deploy_type,
deployment_context=None,
compose_files=None,
compose_project_name=cluster,
compose_env_file=None)
container = deployer.run(image, command=[], user=None, volumes=[], entrypoint=None, env=env, detach=True)
ports = []
if port:
ports = [(port, WEBAPP_PORT)]
container = deployer.run(image, command=[], user=None, volumes=[], entrypoint=None, env=env, ports=ports, detach=True)
# Make configurable?
webappPort = "3000/tcp"
webappPort = f"{WEBAPP_PORT}/tcp"
# TODO: This assumes a Docker container object...
if webappPort in container.network_settings.ports:
mapping = container.network_settings.ports[webappPort][0]
+303
View File
@@ -0,0 +1,303 @@
# Copyright © 2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import hashlib
import json
import os
import random
import subprocess
import sys
import tempfile
import uuid
import yaml
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
def __getattribute__(self, attr):
__dict__ = super(AttrDict, self).__getattribute__("__dict__")
if attr in __dict__:
v = super(AttrDict, self).__getattribute__(attr)
if isinstance(v, dict):
return AttrDict(v)
return v
def cmd(*vargs):
try:
result = subprocess.run(vargs, capture_output=True)
result.check_returncode()
return result.stdout.decode()
except Exception as err:
print(result.stderr.decode())
raise err
class LaconicRegistryClient:
def __init__(self, config_file):
self.config_file = config_file
self.cache = AttrDict(
{
"name_or_id": {},
}
)
def list_records(self, criteria={}, all=False):
args = ["laconic", "-c", self.config_file, "cns", "record", "list"]
if all:
args.append("--all")
if criteria:
for k, v in criteria.items():
args.append("--%s" % k)
args.append(str(v))
results = [AttrDict(r) for r in json.loads(cmd(*args))]
# Most recent records first
results.sort(key=lambda r: r.createTime)
results.reverse()
return results
def is_crn(self, name_or_id: str):
if name_or_id:
return str(name_or_id).startswith("crn://")
return False
def is_id(self, name_or_id: str):
return not self.is_crn(name_or_id)
def _add_to_cache(self, records):
if not records:
return
for p in records:
self.cache["name_or_id"][p.id] = p
if p.names:
for crn in p.names:
self.cache["name_or_id"][crn] = p
if p.attributes.type not in self.cache:
self.cache[p.attributes.type] = []
self.cache[p.attributes.type].append(p)
def resolve(self, name):
if not name:
return None
if name in self.cache.name_or_id:
return self.cache.name_or_id[name]
args = ["laconic", "-c", self.config_file, "cns", "name", "resolve", name]
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
if parsed:
self._add_to_cache(parsed)
return parsed[0]
return None
def get_record(self, name_or_id, require=False):
if not name_or_id:
if require:
raise Exception("Cannot locate record:", name_or_id)
return None
if name_or_id in self.cache.name_or_id:
return self.cache.name_or_id[name_or_id]
if self.is_crn(name_or_id):
return self.resolve(name_or_id)
args = [
"laconic",
"-c",
self.config_file,
"cns",
"record",
"get",
"--id",
name_or_id,
]
parsed = [AttrDict(r) for r in json.loads(cmd(*args))]
if len(parsed):
self._add_to_cache(parsed)
return parsed[0]
if require:
raise Exception("Cannot locate record:", name_or_id)
return None
def app_deployment_requests(self):
return self.list_records({"type": "ApplicationDeploymentRequest"}, True)
def app_deployments(self):
return self.list_records({"type": "ApplicationDeploymentRecord"})
def publish(self, record, names=[]):
tmpdir = tempfile.mkdtemp()
try:
record_fname = os.path.join(tmpdir, "record.yml")
record_file = open(record_fname, 'w')
yaml.dump(record, record_file)
record_file.close()
print(open(record_fname, 'r').read())
new_record_id = json.loads(
cmd("laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
)["id"]
for name in names:
cmd("laconic", "-c", self.config_file, "cns", "name", "set", name, new_record_id)
return new_record_id
finally:
cmd("rm", "-rf", tmpdir)
def file_hash(filename):
return hashlib.sha1(open(filename).read().encode()).hexdigest()
def build_container_image(app_record, tag, extra_build_args=[]):
tmpdir = tempfile.mkdtemp()
try:
record_id = app_record["id"]
ref = app_record.attributes.repository_ref
repo = random.choice(app_record.attributes.repository)
clone_dir = os.path.join(tmpdir, record_id)
print(f"Cloning repository {repo} to {clone_dir} ...")
if ref:
result = subprocess.run(["git", "clone", "--depth", "1", "--branch", ref, repo, clone_dir])
result.check_returncode()
else:
result = subprocess.run(["git", "clone", "--depth", "1", repo, clone_dir])
result.check_returncode()
print("Building webapp ...")
build_command = [sys.argv[0], "build-webapp", "--source-repo", clone_dir, "--tag", tag]
if extra_build_args:
build_command.append("--extra-build-args")
build_command.append(" ".join(extra_build_args))
result = subprocess.run(build_command)
result.check_returncode()
finally:
cmd("rm", "-rf", tmpdir)
def push_container_image(deployment_dir):
print("Pushing image ...")
result = subprocess.run([sys.argv[0], "deployment", "--dir", deployment_dir, "push-images"])
result.check_returncode()
def deploy_to_k8s(deploy_record, deployment_dir):
if not deploy_record:
command = "up"
else:
command = "update"
result = subprocess.run([sys.argv[0], "deployment", "--dir", deployment_dir, command])
result.check_returncode()
def publish_deployment(laconic: LaconicRegistryClient,
app_record,
deploy_record,
deployment_crn,
dns_record,
dns_crn,
deployment_dir,
app_deployment_request=None):
if not deploy_record:
deploy_ver = "0.0.1"
else:
deploy_ver = "0.0.%d" % (int(deploy_record.attributes.version.split(".")[-1]) + 1)
if not dns_record:
dns_ver = "0.0.1"
else:
dns_ver = "0.0.%d" % (int(dns_record.attributes.version.split(".")[-1]) + 1)
spec = yaml.full_load(open(os.path.join(deployment_dir, "spec.yml")))
fqdn = spec["network"]["http-proxy"][0]["host-name"]
uniq = uuid.uuid4()
new_dns_record = {
"record": {
"type": "DnsRecord",
"version": dns_ver,
"name": fqdn,
"resource_type": "A",
"meta": {
"so": uniq.hex
},
}
}
if app_deployment_request:
new_dns_record["record"]["request"] = app_deployment_request.id
dns_id = laconic.publish(new_dns_record, [dns_crn])
new_deployment_record = {
"record": {
"type": "ApplicationDeploymentRecord",
"version": deploy_ver,
"url": f"https://{fqdn}",
"name": app_record.attributes.name,
"application": app_record.id,
"dns": dns_id,
"meta": {
"config": file_hash(os.path.join(deployment_dir, "config.env")),
"so": uniq.hex
},
}
}
if app_deployment_request:
new_deployment_record["record"]["request"] = app_deployment_request.id
deployment_id = laconic.publish(new_deployment_record, [deployment_crn])
return {"dns": dns_id, "deployment": deployment_id}
def hostname_for_deployment_request(app_deployment_request, laconic):
dns_name = app_deployment_request.attributes.dns
if not dns_name:
app = laconic.get_record(app_deployment_request.attributes.application, require=True)
dns_name = generate_hostname_for_app(app)
elif dns_name.startswith("crn://"):
record = laconic.get_record(dns_name, require=True)
dns_name = record.attributes.name
return dns_name
def generate_hostname_for_app(app):
last_part = app.attributes.name.split("/")[-1]
m = hashlib.sha256()
m.update(app.attributes.name.encode())
m.update(b"|")
if isinstance(app.attributes.repository, list):
m.update(app.attributes.repository[0].encode())
else:
m.update(app.attributes.repository.encode())
return "%s-%s" % (last_part, m.hexdigest()[0:10])
+3 -1
View File
@@ -20,7 +20,7 @@ from stack_orchestrator.repos import setup_repositories
from stack_orchestrator.build import build_containers
from stack_orchestrator.build import build_npms
from stack_orchestrator.build import build_webapp
from stack_orchestrator.deploy import run_webapp
from stack_orchestrator.deploy.webapp import run_webapp, deploy_webapp, deploy_webapp_from_registry
from stack_orchestrator.deploy import deploy
from stack_orchestrator import version
from stack_orchestrator.deploy import deployment
@@ -52,6 +52,8 @@ cli.add_command(build_containers.command, "build-containers")
cli.add_command(build_npms.command, "build-npms")
cli.add_command(build_webapp.command, "build-webapp")
cli.add_command(run_webapp.command, "run-webapp")
cli.add_command(deploy_webapp.command, "deploy-webapp")
cli.add_command(deploy_webapp_from_registry.command, "deploy-webapp-from-registry")
cli.add_command(deploy.command, "deploy") # deploy is an alias for deploy-system
cli.add_command(deploy.command, "deploy-system")
cli.add_command(deployment.command, "deployment")
+6
View File
@@ -18,6 +18,8 @@ import os.path
import sys
import ruamel.yaml
from pathlib import Path
from dotenv import dotenv_values
from typing import Mapping
def include_exclude_check(s, include, exclude):
@@ -178,3 +180,7 @@ def global_options2(ctx):
def error_exit(s):
print(f"ERROR: {s}")
sys.exit(1)
def env_var_map_from_file(file: Path) -> Mapping[str, str]:
return dotenv_values(file)