Compare commits

...
39 Commits
Author SHA1 Message Date
Alex e7aa2e944f Add urbit-uniswap proxy 2024-06-04 13:26:23 -07:00
Alex 7f0452cd73 Add vega stack 2024-03-29 21:50:32 -07:00
telackey d2442bcc9b revert 5308ab1e4e (#788)
revert Blind commit to fix laconic CLI calls after rename. (#784)

`laconic cns` got renamed to `laconic registry` which breaks all the scripts and commands that use it.

Reviewed-on: cerc-io/stack-orchestrator#784
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>

Reviewed-on: cerc-io/stack-orchestrator#788
2024-03-27 20:55:03 +00:00
nabarun 44faf36837 Update ajna-watcher-ts version for using new subgraph (#786)
Part of https://www.notion.so/Run-ajna-finance-subgraph-watcher-87748d78cd7a471b8d71f50d5fdc2657

Reviewed-on: cerc-io/stack-orchestrator#786
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-03-26 14:52:57 +00:00
nabarun 18b006468d Update GQL path for ajna subgraph watcher server (#785)
Part of https://www.notion.so/Run-ajna-finance-subgraph-watcher-87748d78cd7a471b8d71f50d5fdc2657

Reviewed-on: cerc-io/stack-orchestrator#785
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-03-26 11:50:05 +00:00
telackey 5308ab1e4e Blind commit to fix laconic CLI calls after rename. (#784)
`laconic cns` got renamed to `laconic registry` which breaks all the scripts and commands that use it.

Reviewed-on: cerc-io/stack-orchestrator#784
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-25 19:09:26 +00:00
prathamesh cd50832038 Add a Ajna watcher stack (#781)
Part of https://www.notion.so/Generate-ajna-finance-subgraph-watcher-with-codegen-5b80ac149b3f449fb138f5d92cc5485e

Reviewed-on: cerc-io/stack-orchestrator#781
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-03-21 07:17:01 +00:00
prathamesh aeddc82ebc Remove latest indexed block value from watcher alerts data (#780)
Part of https://www.notion.so/Setup-grafana-SO-stack-for-monitoring-watchers-7e23042c296c4de6b8676f1f604aa03c

Reviewed-on: cerc-io/stack-orchestrator#780
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-03-13 07:16:15 +00:00
prathamesh 17e860d6e4 Update subgraph watcher versions and instructions to use deployments (#775)
Part of https://www.notion.so/Setup-watchers-on-sandman-34b5514a10634c6fbf3ec338967c871c

Reviewed-on: cerc-io/stack-orchestrator#775
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-03-12 05:32:55 +00:00
telackey 523b5779be Auto-detect which certificate to use (including wildcards). (#779)
Rather than always requesting a certificate, attempt to re-use an existing certificate if it already exists in the k8s cluster.  This includes matching to a wildcard certificate.

Reviewed-on: cerc-io/stack-orchestrator#779
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-07 17:38:36 +00:00
telackey 62f7ce649d Exit non-0 if docker build fails. (#778)
Make sure to check the exit code of the docker build and bubble it back up to laconic-so.

Reviewed-on: cerc-io/stack-orchestrator#778
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-06 18:38:30 +00:00
telackey cc541ac20f Use -slim variant for Dockerfile (#773)
This saves about 1GB of space in the image.

Reviewed-on: cerc-io/stack-orchestrator#773
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-28 04:37:57 +00:00
telackey 10e2311a8b Add timed logging for the webapp build (#771)
Add lots of log and timer output to webapp builds.

Reviewed-on: cerc-io/stack-orchestrator#771
2024-02-28 00:38:11 +00:00
dboreham f32bbf9e48 Merge pull request 'Doc for fetch-containers command' (#772) from dboreham/fetch-containers-doc into main
Reviewed-on: cerc-io/stack-orchestrator#772
2024-02-27 18:45:18 +00:00
dboreham 0302153162 Doc for fetch-containers command 2024-02-27 11:44:28 -07:00
dboreham 01e4437b62 Merge pull request 'Sort order was backwards' (#770) from dboreham/fix-container-age-sort into main
Reviewed-on: cerc-io/stack-orchestrator#770
2024-02-27 16:01:17 +00:00
dboreham 64cec163b3 Sort order was backwards 2024-02-27 09:00:36 -07:00
dboreham 170ad71397 fetch-containers-fixes (#769)
Reviewed-on: cerc-io/stack-orchestrator#769
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-02-27 15:53:05 +00:00
dboreham da1ff609fe fetch-images command (#768)
Implementation of a command to fetch pre-built images from a remote registry, complementing the --push-images option already present on build-containers.

The two subcommands used together allow a stack to be deployed without needing to built its images, provided they have been already built and pushed to the specified container image registry.

This implementation simply picks the newest image with the right name and platform (matches against the platform Python is running on, so watch out for scenarios where Python is an x86 binary on M1 macs).

Reviewed-on: cerc-io/stack-orchestrator#768
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-02-27 15:15:08 +00:00
telackey 21eb9f036f Add support for pnpm as a webapp build tool. (#767)
This adds support for auto-detecting pnpm as a build tool for webapps.

Reviewed-on: cerc-io/stack-orchestrator#767
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-26 23:31:52 +00:00
telackey a0413659f7 Check for existing tag in remote repo before building. (#764)
webapps are meant to be build-once/deploy-many, but we were rebuilding them for every request.  This changes that, so that we rebuild only for every unique ApplicationRecord.

When we push the image, we now tag it according to its ApplicationRecord.

We don't want to use that tag directly in the compose file for the deployment, however, as the deployment needs to be able to adjust to new builds w/o re-writing the file all the time.  Instead, we use a per-deployment unique tag (same as before), we just update what image it references as needed.

Reviewed-on: cerc-io/stack-orchestrator#764
2024-02-24 03:22:49 +00:00
zramsayandzramsay a16fc657bf clarify uniswap urbit readme (#766)
Co-authored-by: zramsay <zach@bluecollarcoding.ca>
Reviewed-on: cerc-io/stack-orchestrator#766
2024-02-24 00:15:53 +00:00
telackey 704c42c404 Use a catchall for single page apps. (#763)
This creates a new environment variable, CERC_SINGLE_PAGE_APP, which controls whether a catchall redirection back to / is applied.

If the value is not explicitly set, we try to detect if the page looks like a single-page app.

Reviewed-on: cerc-io/stack-orchestrator#763
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-23 20:32:24 +00:00
dboreham 202f187172 Fix copy/paste error 2024-02-23 13:15:37 -07:00
dboreham aaed356d32 Simple container image publication (#762)
Reviewed-on: cerc-io/stack-orchestrator#762
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-02-23 19:57:47 +00:00
telackey 2af6ffce77 Tweaks for running the container registry in k8s (#760)
Minor tweaks for running the container-registry in k8s.  The big change is not requiring --image-registry.

Reviewed-on: cerc-io/stack-orchestrator#760
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
2024-02-22 21:11:06 +00:00
dboreham 7bb86cf35e Merge pull request 'Add ARM fixturenet-plugeth test job' (#759) from dboreham/arm-fixturenet-test into main
Reviewed-on: cerc-io/stack-orchestrator#759
2024-02-22 20:45:26 +00:00
dboreham 9e0892cb6b No need to start docker now 2024-02-22 13:36:00 -07:00
dboreham cf9cf6346f Add an arm-specific version of the plugeth fixturenet test 2024-02-22 13:34:37 -07:00
dboreham 642c0ead0d Add test for two config parameters (#758)
Reviewed-on: cerc-io/stack-orchestrator#758
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-02-22 19:35:55 +00:00
telackey 6bd77c893a Even more logging fixes (#757)
Hopefully the last one for a bit.

This only output the cmdline if log_file is present (ie, not to stdout).  It also fixes a bug where the log_file was not passed in one line.

Reviewed-on: cerc-io/stack-orchestrator#757
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-22 01:24:44 +00:00
telackey 4a4d48ddb9 Fix error when logging exception. (#756)
Reviewed-on: cerc-io/stack-orchestrator#756
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-22 00:11:06 +00:00
telackey 08438b1cd5 More logging for webapp deployments (#755)
Reviewed-on: cerc-io/stack-orchestrator#755
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 23:48:52 +00:00
telackey 9f1dd284a5 Better error logging for registry deployments. (#754)
We were missing errors related to registration, this should fix that.

Reviewed-on: cerc-io/stack-orchestrator#754
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 20:12:52 +00:00
dboreham 5985242b8c Merge pull request 'Update doc for fixturenet-laconic-loaded stack' (#753) from dboreham/update-laconic-stack-doc into main
Reviewed-on: cerc-io/stack-orchestrator#753
2024-02-21 17:33:04 +00:00
dboreham d4152b7ce3 Update doc for fixturenet-laconic-loaded stack 2024-02-21 10:29:15 -07:00
telackey db4986dcc6 snowballtool-base backend stack (#751)
This adds a stack for the backend from snowball/snowballtools-base.

Reviewed-on: cerc-io/stack-orchestrator#751
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 04:45:46 +00:00
telackey 65f05ea80c Run a manual build script, if present. (#750)
If the tree has a 'build-webapp.sh' script, use that.

Reviewed-on: cerc-io/stack-orchestrator#750
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 00:20:50 +00:00
zramsayandzramsay 01f9fe67ed add Mars v2 interface (#744)
Tested on DO with real funds on mainnet

Co-authored-by: zramsay <zach@bluecollarcoding.ca>
Reviewed-on: cerc-io/stack-orchestrator#744
2024-02-19 19:11:59 +00:00
111 changed files with 3846 additions and 577 deletions
@@ -0,0 +1,45 @@
name: Fixturenet-Eth-Plugeth-Arm-Test
on:
push:
branches: '*'
paths:
- '!**'
- '.gitea/workflows/triggers/fixturenet-eth-plugeth-arm-test'
schedule: # Note: coordinate with other tests to not overload runners at the same time of day
- cron: '2 14 * * *'
# Needed until we can incorporate docker startup into the executor container
env:
DOCKER_HOST: unix:///var/run/dind.sock
jobs:
test:
name: "Run an Ethereum plugeth fixturenet test"
runs-on: ubuntu-latest-arm
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
# At present the stock setup-python action fails on Linux/aarch64
# Conditional steps below workaroud this by using deadsnakes for that case only
- name: "Install Python for ARM on Linux"
if: ${{ runner.arch == 'arm64' && runner.os == 'Linux' }}
uses: deadsnakes/action@v3.0.1
with:
python-version: '3.8'
- name: "Install Python cases other than ARM on Linux"
if: ${{ ! (runner.arch == 'arm64' && runner.os == 'Linux') }}
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: "Run fixturenet-eth tests"
run: ./tests/fixturenet-eth-plugeth/run-test.sh
@@ -0,0 +1,2 @@
Change this file to trigger running the fixturenet-eth-plugeth-arm-test CI job
+9
View File
@@ -0,0 +1,9 @@
# Fetching pre-built container images
When Stack Orchestrator deploys a stack containing a suite of one or more containers it expects images for those containers to be on the local machine with a tag of the form `<image-name>:local` Images for these containers can be built from source (and optionally base container images from public registries) with the `build-containers` subcommand.
However, the task of building a large number of containers from source may consume considerable time and machine resources. This is where the `fetch-containers` subcommand steps in. It is designed to work exactly like `build-containers` but instead the images, pre-built, are fetched from an image registry then re-tagged for deployment. It can be used in place of `build-containers` for any stack provided the necessary containers, built for the local machine architecture (e.g. arm64 or x86-64) have already been published in an image registry.
## Usage
To use `fetch-containers`, provide an image registry path, a username and token/password with read access to the registry, and optionally specify `--force-local-overwrite`. If this argument is not specified, if there is already a locally built or previously fetched image for a stack container on the machine, it will not be overwritten and a warning issued.
```
$ laconic-so --stack mobymask-v3-demo fetch-containers --image-registry git.vdb.to/cerc-io --registry-username <registry-user> --registry-token <registry-token> --force-local-overwrite
```
+104 -97
View File
@@ -15,132 +15,139 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
16 GB Memory / 8 Intel vCPUs / 160 GB Disk.
1. Login to the droplet as root (either by SSH key or password set in the DO console)
```
ssh root@IP
```
```
ssh root@IP
```
1. Get the install script, give it executable permissions, and run it:
2. Get the install script, give it executable permissions, and run it:
```
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
```
```
chmod +x install.sh
```
```
bash install.sh
```
```
curl -o install.sh https://raw.githubusercontent.com/cerc-io/stack-orchestrator/main/scripts/quick-install-linux.sh
```
```
chmod +x install.sh
```
```
bash install.sh
```
1. Confirm docker was installed and activate the changes in `~/.profile`:
3. Confirm docker was installed and activate the changes in `~/.profile`:
```
docker run hello-world
```
```
source ~/.profile
```
```
docker run hello-world
```
```
source ~/.profile
```
1. Verify installation:
4. Verify installation:
```
laconic-so version
```
```
laconic-so version
```
## Setup the laconic fixturenet stack
1. Get the repositories
```
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
```
```
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
```
2. Set this environment variable to the Laconic self-hosted Gitea instance:
1. Build the containers:
```
export CERC_NPM_REGISTRY_URL=https://git.vdb.to/api/packages/cerc-io/npm/
```
```
laconic-so --stack fixturenet-laconic-loaded build-containers
```
3. Build the containers:
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
```
laconic-so --stack fixturenet-laconic-loaded build-containers
```
It's possible to run into an `ESOCKETTIMEDOUT` error, e.g., `error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.11.3.tgz: ESOCKETTIMEDOUT"`. This may happen even if you have a great internet connection. In that case, re-run the `build-containers` command.
1. Set this environment variable to your droplet's IP address or fully qualified DNS host name if it has one:
4. Set this environment variable to your droplet's IP address:
```
export BACKEND_ENDPOINT=http://<your-IP-or-hostname>:9473
```
e.g.
```
export BACKEND_ENDPOINT=http://my-test-server.example.com:9473
```
```
export LACONIC_HOSTED_ENDPOINT=http://<your-IP>
```
1. Create a deployment directory for the stack:
```
laconic-so --stack fixturenet-laconic-loaded deploy init --output laconic-loaded.spec --map-ports-to-host any-same --config LACONIC_HOSTED_ENDPOINT=$BACKEND_ENDPOINT
```
```
laconic-so --stack fixturenet-laconic-loaded deploy create --deployment-dir laconic-loaded-deployment --spec-file laconic-loaded.spec
```
2. Start the stack:
5. Deploy the stack:
```
laconic-so deployment --dir laconic-loaded-deployment start
```
```
laconic-so --stack fixturenet-laconic-loaded deploy up
```
3. Check the logs:
6. Check the logs:
```
laconic-so deployment --dir laconic-loaded-deployment logs
```
```
laconic-so --stack fixturenet-laconic-loaded deploy logs
```
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
```
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
```
```
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643aphoton from=mint module=x/bank
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
```
4. Confirm operation of the registry CLI:
7. Confirm operation of the registry CLI:
```
laconic-so deployment --dir laconic-loaded-deployment exec cli "laconic cns status"
```
```
laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic cns status"
```
```
{
"version": "0.3.0",
"node": {
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
"network": "laconic_9000-1",
"moniker": "localtestnet"
},
"sync": {
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
"latest_block_height": "5",
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
"catching_up": false
},
"validator": {
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000"
},
"validators": [
{
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000",
"proposer_priority": "0"
}
],
"num_peers": "0",
"peers": [],
"disk_usage": "292.0K"
}
```
```
{
"version": "0.3.0",
"node": {
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
"network": "laconic_9000-1",
"moniker": "localtestnet"
},
"sync": {
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
"latest_block_height": "5",
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
"catching_up": false
},
"validator": {
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000"
},
"validators": [
{
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000",
"proposer_priority": "0"
}
],
"num_peers": "0",
"peers": [],
"disk_usage": "292.0K"
}
```
## Configure Digital Ocean firewall
(Note this step may not be necessary depending on the droplet image used)
Let's open some ports.
1. In the Digital Ocean web console, navigate to your droplet's main page. Select the "Networking" tab and scroll down to "Firewall".
+61 -68
View File
@@ -25,10 +25,13 @@ import sys
from decouple import config
import subprocess
import click
import importlib.resources
from pathlib import Path
from stack_orchestrator.util import include_exclude_check, get_parsed_stack_config, stack_is_external, warn_exit
from stack_orchestrator.opts import opts
from stack_orchestrator.util import include_exclude_check, stack_is_external, error_exit
from stack_orchestrator.base import get_npm_registry_url
from stack_orchestrator.build.build_types import BuildContext
from stack_orchestrator.build.publish import publish_image
from stack_orchestrator.build.build_util import get_containers_in_scope
# TODO: find a place for this
# epilog="Config provided either in .env or settings.ini or env vars: CERC_REPO_BASE_DIR (defaults to ~/cerc)"
@@ -59,69 +62,58 @@ def make_container_build_env(dev_root_path: str,
return container_build_env
def process_container(stack: str,
container,
container_build_dir: str,
container_build_env: dict,
dev_root_path: str,
quiet: bool,
verbose: bool,
dry_run: bool,
continue_on_error: bool,
):
if not quiet:
print(f"Building: {container}")
def process_container(build_context: BuildContext) -> bool:
if not opts.o.quiet:
print(f"Building: {build_context.container}")
default_container_tag = f"{container}:local"
container_build_env.update({"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag})
default_container_tag = f"{build_context.container}:local"
build_context.container_build_env.update({"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag})
# Check if this is in an external stack
if stack_is_external(stack):
container_parent_dir = Path(stack).joinpath("container-build")
temp_build_dir = container_parent_dir.joinpath(container.replace("/", "-"))
if stack_is_external(build_context.stack):
container_parent_dir = Path(build_context.stack).joinpath("container-build")
temp_build_dir = container_parent_dir.joinpath(build_context.container.replace("/", "-"))
temp_build_script_filename = temp_build_dir.joinpath("build.sh")
# Now check if the container exists in the external stack.
if not temp_build_script_filename.exists():
# If not, revert to building an internal container
container_parent_dir = container_build_dir
container_parent_dir = build_context.container_build_dir
else:
container_parent_dir = container_build_dir
container_parent_dir = build_context.container_build_dir
build_dir = container_parent_dir.joinpath(container.replace("/", "-"))
build_dir = container_parent_dir.joinpath(build_context.container.replace("/", "-"))
build_script_filename = build_dir.joinpath("build.sh")
if verbose:
if opts.o.verbose:
print(f"Build script filename: {build_script_filename}")
if os.path.exists(build_script_filename):
build_command = build_script_filename.as_posix()
else:
if verbose:
if opts.o.verbose:
print(f"No script file found: {build_script_filename}, using default build script")
repo_dir = container.split('/')[1]
repo_dir = build_context.container.split('/')[1]
# TODO: make this less of a hack -- should be specified in some metadata somewhere
# Check if we have a repo for this container. If not, set the context dir to the container-build subdir
repo_full_path = os.path.join(dev_root_path, repo_dir)
repo_full_path = os.path.join(build_context.dev_root_path, repo_dir)
repo_dir_or_build_dir = repo_full_path if os.path.exists(repo_full_path) else build_dir
build_command = os.path.join(container_build_dir,
build_command = os.path.join(build_context.container_build_dir,
"default-build.sh") + f" {default_container_tag} {repo_dir_or_build_dir}"
if not dry_run:
if not opts.o.dry_run:
# No PATH at all causes failures with podman.
if "PATH" not in container_build_env:
container_build_env["PATH"] = os.environ["PATH"]
if verbose:
print(f"Executing: {build_command} with environment: {container_build_env}")
build_result = subprocess.run(build_command, shell=True, env=container_build_env)
if verbose:
if "PATH" not in build_context.container_build_env:
build_context.container_build_env["PATH"] = os.environ["PATH"]
if opts.o.verbose:
print(f"Executing: {build_command} with environment: {build_context.container_build_env}")
build_result = subprocess.run(build_command, shell=True, env=build_context.container_build_env)
if opts.o.verbose:
print(f"Return code is: {build_result.returncode}")
if build_result.returncode != 0:
print(f"Error running build for {container}")
if not continue_on_error:
print("FATAL Error: container build failed and --continue-on-error not set, exiting")
sys.exit(1)
else:
print("****** Container Build Error, continuing because --continue-on-error is set")
return False
else:
return True
else:
print("Skipped")
return True
@click.command()
@@ -129,17 +121,14 @@ def process_container(stack: str,
@click.option('--exclude', help="don\'t build these containers")
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
@click.option("--extra-build-args", help="Supply extra arguments to build")
@click.option("--publish-images", is_flag=True, default=False, help="Publish the built images in the specified image registry")
@click.option("--image-registry", help="Specify the image registry for --publish-images")
@click.pass_context
def command(ctx, include, exclude, force_rebuild, extra_build_args):
def command(ctx, include, exclude, force_rebuild, extra_build_args, publish_images, image_registry):
'''build the set of containers required for a complete stack'''
quiet = ctx.obj.quiet
verbose = ctx.obj.verbose
dry_run = ctx.obj.dry_run
debug = ctx.obj.debug
local_stack = ctx.obj.local_stack
stack = ctx.obj.stack
continue_on_error = ctx.obj.continue_on_error
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
@@ -150,41 +139,45 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
if not quiet:
if not opts.o.quiet:
print(f'Dev Root is: {dev_root_path}')
if not os.path.isdir(dev_root_path):
print('Dev root directory doesn\'t exist, creating')
# See: https://stackoverflow.com/a/20885799/1701505
from stack_orchestrator import data
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
all_containers = container_list_file.read().splitlines()
if publish_images:
if not image_registry:
error_exit("--image-registry must be supplied with --publish-images")
containers_in_scope = []
if stack:
stack_config = get_parsed_stack_config(stack)
if "containers" not in stack_config or stack_config["containers"] is None:
warn_exit(f"stack {stack} does not define any containers")
containers_in_scope = stack_config['containers']
else:
containers_in_scope = all_containers
if verbose:
print(f'Containers: {containers_in_scope}')
if stack:
print(f"Stack: {stack}")
containers_in_scope = get_containers_in_scope(stack)
container_build_env = make_container_build_env(dev_root_path,
container_build_dir,
debug,
opts.o.debug,
force_rebuild,
extra_build_args)
for container in containers_in_scope:
if include_exclude_check(container, include, exclude):
process_container(stack, container, container_build_dir, container_build_env,
dev_root_path, quiet, verbose, dry_run, continue_on_error)
build_context = BuildContext(
stack,
container,
container_build_dir,
container_build_env,
dev_root_path
)
result = process_container(build_context)
if result:
if publish_images:
publish_image(f"{container}:local", image_registry)
else:
print(f"Error running build for {build_context.container}")
if not opts.o.continue_on_error:
error_exit("container build failed and --continue-on-error not set, exiting")
sys.exit(1)
else:
print("****** Container Build Error, continuing because --continue-on-error is set")
else:
if verbose:
if opts.o.verbose:
print(f"Excluding: {container}")
+29
View File
@@ -0,0 +1,29 @@
# Copyright © 2024 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from dataclasses import dataclass
from pathlib import Path
from typing import Mapping
@dataclass
class BuildContext:
stack: str
container: str
container_build_dir: Path
container_build_env: Mapping[str,str]
dev_root_path: str
+43
View File
@@ -0,0 +1,43 @@
# Copyright © 2024 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import importlib.resources
from stack_orchestrator.opts import opts
from stack_orchestrator.util import get_parsed_stack_config, warn_exit
def get_containers_in_scope(stack: str):
# See: https://stackoverflow.com/a/20885799/1701505
from stack_orchestrator import data
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
all_containers = container_list_file.read().splitlines()
containers_in_scope = []
if stack:
stack_config = get_parsed_stack_config(stack)
if "containers" not in stack_config or stack_config["containers"] is None:
warn_exit(f"stack {stack} does not define any containers")
containers_in_scope = stack_config['containers']
else:
containers_in_scope = all_containers
if opts.o.verbose:
print(f'Containers: {containers_in_scope}')
if stack:
print(f"Stack: {stack}")
return containers_in_scope
+47 -14
View File
@@ -21,11 +21,14 @@
# TODO: display the available list of containers; allow re-build of either all or specific containers
import os
import sys
from decouple import config
import click
from pathlib import Path
from stack_orchestrator.build import build_containers
from stack_orchestrator.deploy.webapp.util import determine_base_container
from stack_orchestrator.deploy.webapp.util import determine_base_container, TimedLogger
from stack_orchestrator.build.build_types import BuildContext
@click.command()
@@ -37,26 +40,25 @@ from stack_orchestrator.deploy.webapp.util import determine_base_container
@click.pass_context
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag):
'''build the specified webapp container'''
logger = TimedLogger()
quiet = ctx.obj.quiet
verbose = ctx.obj.verbose
dry_run = ctx.obj.dry_run
debug = ctx.obj.debug
verbose = ctx.obj.verbose
local_stack = ctx.obj.local_stack
stack = ctx.obj.stack
continue_on_error = ctx.obj.continue_on_error
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
if local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
print(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
logger.log(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
if not quiet:
print(f'Dev Root is: {dev_root_path}')
if verbose:
logger.log(f'Dev Root is: {dev_root_path}')
if not base_container:
base_container = determine_base_container(source_repo)
@@ -65,8 +67,23 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
container_build_env = build_containers.make_container_build_env(dev_root_path, container_build_dir, debug,
force_rebuild, extra_build_args)
build_containers.process_container(None, base_container, container_build_dir, container_build_env, dev_root_path, quiet,
verbose, dry_run, continue_on_error)
if verbose:
logger.log(f"Building base container: {base_container}")
build_context_1 = BuildContext(
stack,
base_container,
container_build_dir,
container_build_env,
dev_root_path,
)
ok = build_containers.process_container(build_context_1)
if not ok:
logger.log("ERROR: Build failed.")
sys.exit(1)
if verbose:
logger.log(f"Base container {base_container} build finished.")
# Now build the target webapp. We use the same build script, but with a different Dockerfile and work dir.
container_build_env["CERC_WEBAPP_BUILD_RUNNING"] = "true"
@@ -76,9 +93,25 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
"Dockerfile.webapp")
if not tag:
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
container_build_env["CERC_CONTAINER_BUILD_TAG"] = f"cerc/{webapp_name}:local"
else:
container_build_env["CERC_CONTAINER_BUILD_TAG"] = tag
tag = f"cerc/{webapp_name}:local"
build_containers.process_container(None, base_container, container_build_dir, container_build_env, dev_root_path, quiet,
verbose, dry_run, continue_on_error)
container_build_env["CERC_CONTAINER_BUILD_TAG"] = tag
if verbose:
logger.log(f"Building app container: {tag}")
build_context_2 = BuildContext(
stack,
base_container,
container_build_dir,
container_build_env,
dev_root_path,
)
ok = build_containers.process_container(build_context_2)
if not ok:
logger.log("ERROR: Build failed.")
sys.exit(1)
if verbose:
logger.log(f"App container {base_container} build finished.")
logger.log("build-webapp complete", show_step_time=False, show_total_time=True)
@@ -0,0 +1,195 @@
# Copyright © 2024 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import click
from dataclasses import dataclass
import json
import platform
from python_on_whales import DockerClient
from python_on_whales.components.manifest.cli_wrapper import ManifestCLI, ManifestList
from python_on_whales.utils import run
import requests
from typing import List
from stack_orchestrator.opts import opts
from stack_orchestrator.util import include_exclude_check, error_exit
from stack_orchestrator.build.build_util import get_containers_in_scope
# Experimental fetch-container command
@dataclass
class RegistryInfo:
registry: str
registry_username: str
registry_token: str
# Extending this code to support the --verbose option, cnosider contributing upstream
# https://github.com/gabrieldemarmiesse/python-on-whales/blob/master/python_on_whales/components/manifest/cli_wrapper.py#L129
class ExtendedManifestCLI(ManifestCLI):
def inspect_verbose(self, x: str) -> ManifestList:
"""Returns a Docker manifest list object."""
json_str = run(self.docker_cmd + ["manifest", "inspect", "--verbose", x])
return json.loads(json_str)
def _local_tag_for(container: str):
return f"{container}:local"
# See: https://docker-docs.uclv.cu/registry/spec/api/
# Emulate this:
# $ curl -u "my-username:my-token" -X GET "https://<container-registry-hostname>/v2/cerc-io/cerc/test-container/tags/list"
# {"name":"cerc-io/cerc/test-container","tags":["202402232130","202402232208"]}
def _get_tags_for_container(container: str, registry_info: RegistryInfo) -> List[str]:
# registry looks like: git.vdb.to/cerc-io
registry_parts = registry_info.registry.split("/")
url = f"https://{registry_parts[0]}/v2/{registry_parts[1]}/{container}/tags/list"
if opts.o.debug:
print(f"Fetching tags from: {url}")
response = requests.get(url, auth=(registry_info.registry_username, registry_info.registry_token))
if response.status_code == 200:
tag_info = response.json()
if opts.o.debug:
print(f"container tags list: {tag_info}")
tags_array = tag_info["tags"]
return tags_array
else:
error_exit(f"failed to fetch tags from image registry, status code: {response.status_code}")
def _find_latest(candidate_tags: List[str]):
# Lex sort should give us the latest first
sorted_candidates = sorted(candidate_tags)
if opts.o.debug:
print(f"sorted candidates: {sorted_candidates}")
return sorted_candidates[-1]
def _filter_for_platform(container: str,
registry_info: RegistryInfo,
tag_list: List[str]) -> List[str] :
filtered_tags = []
this_machine = platform.machine()
# Translate between Python and docker platform names
if this_machine == "x86_64":
this_machine = "amd64"
if this_machine == "aarch64":
this_machine = "arm64"
if opts.o.debug:
print(f"Python says the architecture is: {this_machine}")
docker = DockerClient()
for tag in tag_list:
remote_tag = f"{registry_info.registry}/{container}:{tag}"
manifest_cmd = ExtendedManifestCLI(docker.client_config)
manifest = manifest_cmd.inspect_verbose(remote_tag)
if opts.o.debug:
print(f"manifest: {manifest}")
image_architecture = manifest["Descriptor"]["platform"]["architecture"]
if opts.o.debug:
print(f"image_architecture: {image_architecture}")
if this_machine == image_architecture:
filtered_tags.append(tag)
if opts.o.debug:
print(f"Tags filtered for platform: {filtered_tags}")
return filtered_tags
def _get_latest_image(container: str, registry_info: RegistryInfo):
all_tags = _get_tags_for_container(container, registry_info)
tags_for_platform = _filter_for_platform(container, registry_info, all_tags)
if len(tags_for_platform) > 0:
latest_tag = _find_latest(tags_for_platform)
return f"{container}:{latest_tag}"
else:
return None
def _fetch_image(tag: str, registry_info: RegistryInfo):
docker = DockerClient()
remote_tag = f"{registry_info.registry}/{tag}"
if opts.o.debug:
print(f"Attempting to pull this image: {remote_tag}")
docker.image.pull(remote_tag)
def _exists_locally(container: str):
docker = DockerClient()
return docker.image.exists(_local_tag_for(container))
def _add_local_tag(remote_tag: str, registry: str, local_tag: str):
docker = DockerClient()
docker.image.tag(f"{registry}/{remote_tag}", local_tag)
@click.command()
@click.option('--include', help="only fetch these containers")
@click.option('--exclude', help="don\'t fetch these containers")
@click.option("--force-local-overwrite", is_flag=True, default=False, help="Overwrite a locally built image, if present")
@click.option("--image-registry", required=True, help="Specify the image registry to fetch from")
@click.option("--registry-username", required=True, help="Specify the image registry username")
@click.option("--registry-token", required=True, help="Specify the image registry access token")
@click.pass_context
def command(ctx, include, exclude, force_local_overwrite, image_registry, registry_username, registry_token):
'''EXPERIMENTAL: fetch the images for a stack from remote registry'''
registry_info = RegistryInfo(image_registry, registry_username, registry_token)
docker = DockerClient()
if not opts.o.quiet:
print("Logging into container registry:")
docker.login(registry_info.registry, registry_info.registry_username, registry_info.registry_token)
# Generate list of target containers
stack = ctx.obj.stack
containers_in_scope = get_containers_in_scope(stack)
all_containers_found = True
for container in containers_in_scope:
local_tag = _local_tag_for(container)
if include_exclude_check(container, include, exclude):
if opts.o.debug:
print(f"Processing: {container}")
# For each container, attempt to find the latest of a set of
# images with the correct name and platform in the specified registry
image_to_fetch = _get_latest_image(container, registry_info)
if not image_to_fetch:
print(f"Warning: no image found to fetch for container: {container}")
all_containers_found = False
continue
if opts.o.debug:
print(f"Fetching: {image_to_fetch}")
_fetch_image(image_to_fetch, registry_info)
# Now check if the target container already exists exists locally already
if (_exists_locally(container)):
if not opts.o.quiet:
print(f"Container image {container} already exists locally")
# if so, fail unless the user specified force-local-overwrite
if (force_local_overwrite):
# In that case remove the existing :local tag
if not opts.o.quiet:
print(f"Warning: overwriting local tag from this image: {container} because "
"--force-local-overwrite was specified")
else:
if not opts.o.quiet:
print(f"Skipping local tagging for this image: {container} because that would "
"overwrite an existing :local tagged image, use --force-local-overwrite to do so.")
continue
# Tag the fetched image with the :local tag
_add_local_tag(image_to_fetch, image_registry, local_tag)
else:
if opts.o.verbose:
print(f"Excluding: {container}")
if not all_containers_found:
print("Warning: couldn't find usable images for one or more containers, this stack will not deploy")
+48
View File
@@ -0,0 +1,48 @@
# Copyright © 2024 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from datetime import datetime
from python_on_whales import DockerClient
from stack_orchestrator.opts import opts
from stack_orchestrator.util import error_exit
def _publish_tag_for_image(local_image_tag: str, remote_repo: str, version: str):
# Turns image tags of the form: foo/bar:local into remote.repo/org/bar:deploy
(image_name, image_version) = local_image_tag.split(":")
if image_version == "local":
return f"{remote_repo}/{image_name}:{version}"
else:
error_exit("Asked to publish a non-locally built image")
def publish_image(local_tag, registry):
if opts.o.verbose:
print(f"Publishing this image: {local_tag} to this registry: {registry}")
docker = DockerClient()
# Figure out the target image tag
# Eventually this version will be generated from the source repo state
# Using a timestemp is an intermediate step
version = datetime.now().strftime("%Y%m%d%H%M")
remote_tag = _publish_tag_for_image(local_tag, registry, version)
# Tag the image thus
if opts.o.debug:
print(f"Tagging {local_tag} to {remote_tag}")
docker.image.tag(local_tag, remote_tag)
# Push it to the desired registry
if opts.o.verbose:
print(f"Pushing image {remote_tag}")
docker.image.push(remote_tag)
@@ -5,9 +5,11 @@ services:
environment:
REGISTRY_LOG_LEVEL: ${REGISTRY_LOG_LEVEL}
volumes:
- config:/config:ro
- registry-data:/var/lib/registry
ports:
- "5000"
volumes:
config:
registry-data:
@@ -26,9 +26,9 @@ services:
- "80"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "80"]
interval: 20s
interval: 10s
timeout: 5s
retries: 15
retries: 30
start_period: 10s
volumes:
@@ -0,0 +1,12 @@
version: "3.2"
services:
mars:
image: cerc/mars-v2:local
restart: always
ports:
- "3000:3000"
environment:
- URL_OSMOSIS_REST=https://lcd-osmosis.blockapsis.com
- URL_OSMOSIS_RPC=https://rpc-osmosis.blockapsis.com
- WALLET_CONNECT_ID=0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x
@@ -0,0 +1,13 @@
services:
snowballtools-base-backend:
image: cerc/snowballtools-base-backend:local
restart: always
volumes:
- data:/data
- config:/config:ro
ports:
- 8000
volumes:
data:
config:
@@ -6,6 +6,7 @@ services:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_TEST_PARAM_1: ${CERC_TEST_PARAM_1:-FAILED}
CERC_TEST_PARAM_2: "CERC_TEST_PARAM_2_VALUE"
CERC_TEST_PARAM_3: ${CERC_TEST_PARAM_3:-FAILED}
volumes:
- test-data-bind:/data
- test-data-auto:/data2
@@ -7,12 +7,12 @@ services:
environment:
- REACT_APP_INFURA_KEY=${CERC_INFURA_KEY}
- REACT_APP_AWS_API_ENDPOINT=${CERC_UNISWAP_GQL}
- REACT_APP_UNISWAP_API_URL=${CERC_UNISWAP_API}
command: ["./build-app.sh"]
volumes:
- ../config/uniswap-interface/build-app.sh:/app/build-app.sh
- urbit_app_builds:/app-builds
- ../config/uniswap-interface/urbit-files/mar:/app/mar
- ../config/uniswap-interface/urbit-files/desk.docket-0:/app/desk.docket-0
- ../config/uniswap-interface/urbit-files:/app/urbit-files
volumes:
urbit_app_builds:
@@ -0,0 +1,19 @@
version: "3.2"
# This is a short-lived container. It runs the Vega build and then copies the output to the
# `urbit_app_builds` volume.
services:
vega-interface:
image: cerc/vega-interface:local
restart: on-failure
# environment:
# - REACT_APP_INFURA_KEY=${CERC_INFURA_KEY}
# - REACT_APP_AWS_API_ENDPOINT=${CERC_UNISWAP_GQL}
entrypoint: ["./build-app.sh"]
volumes:
- ../config/vega-interface/urbit-files:/app/urbit-files
- ../config/vega-interface/build-app.sh:/app/build-app.sh
- urbit_app_builds:/app-builds
volumes:
urbit_app_builds:
@@ -0,0 +1,76 @@
version: '3.2'
services:
ajna-watcher-db:
restart: unless-stopped
image: postgres:14-alpine
environment:
- POSTGRES_USER=vdbm
- POSTGRES_MULTIPLE_DATABASES=ajna-watcher,ajna-watcher-job-queue
- POSTGRES_EXTENSION=ajna-watcher-job-queue:pgcrypto
- POSTGRES_PASSWORD=password
volumes:
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
- ajna_watcher_db_data:/var/lib/postgresql/data
ports:
- "5432"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "5432"]
interval: 20s
timeout: 5s
retries: 15
start_period: 10s
ajna-watcher-job-runner:
restart: unless-stopped
depends_on:
ajna-watcher-db:
condition: service_healthy
image: cerc/watcher-ajna:local
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT}
command: ["bash", "./start-job-runner.sh"]
volumes:
- ../config/watcher-ajna/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-ajna/start-job-runner.sh:/app/start-job-runner.sh
ports:
- "9000"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "9000"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
ajna-watcher-server:
restart: unless-stopped
depends_on:
ajna-watcher-db:
condition: service_healthy
ajna-watcher-job-runner:
condition: service_healthy
image: cerc/watcher-ajna:local
environment:
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT}
command: ["bash", "./start-server.sh"]
volumes:
- ../config/watcher-ajna/watcher-config-template.toml:/app/environments/watcher-config-template.toml
- ../config/watcher-ajna/start-server.sh:/app/start-server.sh
ports:
- "3008"
- "9001"
healthcheck:
test: ["CMD", "nc", "-v", "localhost", "3008"]
interval: 20s
timeout: 5s
retries: 15
start_period: 5s
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
ajna_watcher_db_data:
@@ -50,22 +50,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -142,22 +126,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="censures", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -234,22 +202,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="claims", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -326,22 +278,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="conditional_star_release", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -418,22 +354,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="delegated_sending", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -510,22 +430,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="ecliptic", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -602,22 +506,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -694,22 +582,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="azimuth", instance="polls", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -788,22 +660,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="sushi", instance="sushiswap", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -880,22 +736,6 @@ groups:
legendFormat: __auto
range: false
refId: latest_external
- refId: latest_indexed
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: sync_status_block_number{job="sushi", instance="merkl_sushiswap", kind="latest_indexed"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_indexed
- refId: condition
relativeTimeRange:
from: 600
@@ -15,7 +15,5 @@ yarn build
# Copy over build and other files to app-builds for urbit deployment
mkdir -p /app-builds/uniswap
cp -r ./build /app-builds/uniswap/
cp -r mar /app-builds/uniswap/
cp desk.docket-0 /app-builds/uniswap/
cp -r urbit-files /app-builds/uniswap/
cp -r ./build /app-builds/uniswap/urbit-files
@@ -0,0 +1,225 @@
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
:: Proxy server
:: ------------
::
:: This is a simple agent which accepts requests at the path given in the `on-init` arm-- in this
:: case, "/apps/uniswap-proxy/"-- and proxies them to the upstream, in this case,
:: "https://api.uniswap.org/".
::
:: For example, assuming your ship is running on `localhost:8080`, a request to the url
::
:: `http://localhost:8080/apps/uniswap-proxy/v1/graphql`
::
:: will be proxied to
::
:: `https://api.uniswap.org/v1/graphql`
::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
/+ dbug, server, default-agent
::
|%
+$ card card:agent:gall
+$ state
$% %0
==
--
::
:: Path config
|%
++ base-url-path /apps/uniswap-proxy :: HTTP path on the ship that this agent will bind to
++ upstream-api-url "https://api.uniswap.org/" :: Upstream HTTP api that requests should be proxied to
--
::
%- agent:dbug
::
=| state
=* state -
::
^- agent:gall
=<
|_ =bowl:gall
+* this .
default ~(. (default-agent this) bowl)
::
++ on-init
^- (quip card _this)
:_ this
:~
[%pass /eyre/connect %arvo %e %connect [~ base-url-path] %proxy]
==
::
++ on-poke
|= [=mark =vase]
^- (quip card _this)
:_ this :: We have no state
?+ mark (on-poke:default mark vase)
%handle-http-request
=/ [eyre-id=@ta inbound-req=inbound-request:eyre] !<([@ta =inbound-request:eyre] vase)
~& "Eyre id: {<eyre-id>}"
=/ old-req request:inbound-req
=/ new-req=request:http
%= old-req
:: Remove path prefix from the url, replace with Uniswap domain and path prefix
url (rewrite-path url:old-req)
:: Change the "Origin" header to match the upstream (this is hacky as fuck and will break eventually)
header-list (rewrite-headers header-list:old-req)
==
~& "Proxying {<url.old-req>} to {<url.new-req>}"
~& "Outgoing headers: {<header-list:new-req>}"
:~
[%pass /response/[eyre-id] %arvo %i %request new-req *outbound-config:iris]
==
==
::
++ on-arvo
|= [=wire =sign-arvo]
^- (quip card _this)
:_ this :: We don't have any state
?+ sign-arvo (on-arvo:default wire sign-arvo)
::
:: Arvo will respond when we initially connect to Eyre in `on-init`. We will accept (and ignore)
:: that and reject any other communications.
[%eyre %bound *]
~& "Got eyre bound: {<sign-arvo>}"
~
[%iris %http-response %finished *]
?+ wire (on-arvo:default wire sign-arvo)
[%response @ ~]
=/ original-eyre-id=@ta (snag 1 `(list @ta)`wire)
=/ resp=client-response:iris +:+:sign-arvo
?> ?= %finished -.resp
=/ resp-header=response-header:http response-header:resp
=/ the-octs=octs data:(need full-file:resp)
::
=/ thedata=@t q.the-octs
~& resp-header
~& "Proxied HTTP {<status-code:resp-header>}: {<thedata>}"
%+ give-simple-payload:app:server
original-eyre-id
:- %_(resp-header headers [['Content-type'^'application/json'] ~])
`the-octs
==
==
::
:: Each time Eyre pokes a request to us, it will subscribe for the response. We will just accept
:: those connections (wire = /http-response/[eyre-id]) and reject any others.
:: See: https://docs.urbit.org/system/kernel/eyre/reference/tasks#connect
++ on-watch
|= =path
^- (quip card _this)
?+ path (on-watch:default path)
[%http-response *]
`this
==
::
++ on-save on-save:default
++ on-load on-load:default
++ on-leave on-leave:default
++ on-peek on-peek:default
++ on-agent on-agent:default
++ on-fail on-fail:default
--
::
:: Helpers core
|%
:: Rewrites a path like "/apps/uniswap-proxy/v1/graphql" to "https://api.uniswap.org/v1/graphql".
:: `url-path` is a path, not a full URL!
++ rewrite-path
|= [url-path=@t]
%+ rash
url-path
:: Prepend the `upstream-api-url` base url and convert back to a cord
%+ cook
|= [a=tape]
(crip (weld upstream-api-url a))
:: Strip the base path
;~ pfix
(ifix [(just '/') (just '/')] (jest (crip (join '/' `(list @t)`base-url-path))))
(star prn)
==
::
:: Change the 'Origin' http header to "https://api.uniswap.org".
:: I don't know why this makes the request work, but it does.
:: Also drop the "x-forwarded-for" header for Red Horizon; otherwise this breaks it.
++ rewrite-headers
|= [headers=header-list:http]
^- header-list:http
=/ ret *header-list:http
|-
?~ headers
ret
%= $
ret ?+ (crip (cass (trip key:(head headers))))
::
:: Default: use the header
(snoc ret (head headers))
::
:: Drop "x-forwarded-for" header
%'x-forwarded-for'
ret
::
:: Drop the "cookie" header, which contains the Urbit auth token (!)
%'cookie'
ret
::
:: Rewrite "origin" header
%'origin'
(snoc ret ['origin' 'https://api.uniswap.org'])
==
headers +.headers
==
::
:: Manually construct a card that can be passed to iris, simulating a Uniswap graphql request.
:: Can be useful for testing purposes.
++ iris-request-card
::
:: The resulting card produces an HTTP request equivalent to the following cURL command:
::
:: curl 'https://api.uniswap.org/v1/graphql' \
:: -H 'authority: interface.gateway.uniswap.org' \
:: -H 'accept: */*' \
:: -H 'accept-language: en-US,en;q=0.7' \
:: -H 'content-type: application/json' \
:: -H 'origin: https://app.uniswap.org' \
:: -H 'referer: https://app.uniswap.org/' \
:: -H 'sec-ch-ua: "Not A(Brand";v="99", "Brave";v="121", "Chromium";v="121"' \
:: -H 'sec-ch-ua-mobile: ?0' \
:: -H 'sec-ch-ua-platform: "Linux"' \
:: -H 'sec-fetch-dest: empty' \
:: -H 'sec-fetch-mode: cors' \
:: -H 'sec-fetch-site: same-site' \
:: -H 'sec-gpc: 1' \
:: -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36' \
:: --data-raw $'{"operationName":"TokenSpotPrice","variables":{"address":null,"chain":"ETHEREUM"},"query":"query TokenSpotPrice($chain: Chain\u0021, $address: String = null) {\\n token(chain: $chain, address: $address) {\\n id\\n address\\n chain\\n name\\n symbol\\n project {\\n id\\n markets(currencies: [USD]) {\\n id\\n price {\\n id\\n value\\n __typename\\n }\\n __typename\\n }\\n __typename\\n }\\n __typename\\n }\\n}"}' \
:: --compressed
::
=/ the-request=request:http
:* %'POST' 'https://api.uniswap.org/v1/graphql'
:~ ['Accept-Language' 'en-US,en']
['Cache-Control' 'no-cache']
['Connection' 'keep-alive']
['Origin' 'http://localhost:8080']
['Pragma' 'no-cache']
['Referer' 'http://localhost:8080/']
['Sec-Fetch-Dest' 'empty']
['Sec-Fetch-Mode' 'cors']
['Sec-Fetch-Site' 'same-site']
['Sec-GPC' '1']
['User-Agent' 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36']
['accept' '*/*']
['content-type' 'application/json']
['sec-ch-ua' '"Not A(Brand";v="99", "Brave";v="121", "Chromium";v="121"']
['sec-ch-ua-mobile' '?0']
['sec-ch-ua-platform' '"Linux"']
==
:- ~ %- as-octs:mimes:html
'''
{"operationName":"TokenSpotPrice","variables":{"address":null,"chain":"ETHEREUM"},"query":"query TokenSpotPrice($chain: Chain\u0021, $address: String = null) {\n token(chain: $chain, address: $address) {\n id\n address\n chain\n name\n symbol\n project {\n id\n markets(currencies: [USD]) {\n id\n price {\n id\n value\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}"}
'''
==
:~
[%pass /response/1 %arvo %i %request the-request *outbound-config:iris]
==
--
@@ -0,0 +1,2 @@
:~ %proxy
==
@@ -0,0 +1,155 @@
:: dbug: agent wrapper for generic debugging tools
::
:: usage: %-(agent:dbug your-agent)
::
|%
+$ poke
$% [%bowl ~]
[%state grab=cord]
[%incoming =about]
[%outgoing =about]
==
::
+$ about
$@ ~
$% [%ship =ship]
[%path =path]
[%wire =wire]
[%term =term]
==
::
++ agent
|= =agent:gall
^- agent:gall
!.
|_ =bowl:gall
+* this .
ag ~(. agent bowl)
::
++ on-poke
|= [=mark =vase]
^- (quip card:agent:gall agent:gall)
?. ?=(%dbug mark)
=^ cards agent (on-poke:ag mark vase)
[cards this]
=/ dbug
!<(poke vase)
=; =tang
((%*(. slog pri 1) tang) [~ this])
?- -.dbug
%bowl [(sell !>(bowl))]~
::
%state
=? grab.dbug =('' grab.dbug) '-'
=; product=^vase
[(sell product)]~
=/ state=^vase
:: if the underlying app has implemented a /dbug/state scry endpoint,
:: use that vase in place of +on-save's.
::
=/ result=(each ^vase tang)
(mule |.(q:(need (need (on-peek:ag /x/dbug/state)))))
?:(?=(%& -.result) p.result on-save:ag)
%+ slap
(slop state !>([bowl=bowl ..zuse]))
(ream grab.dbug)
::
%incoming
=; =tang
?^ tang tang
[%leaf "no matching subscriptions"]~
%+ murn
%+ sort ~(tap by sup.bowl)
|= [[* a=[=ship =path]] [* b=[=ship =path]]]
(aor [path ship]:a [path ship]:b)
|= [=duct [=ship =path]]
^- (unit tank)
=; relevant=?
?. relevant ~
`>[path=path from=ship duct=duct]<
?: ?=(~ about.dbug) &
?- -.about.dbug
%ship =(ship ship.about.dbug)
%path ?=(^ (find path.about.dbug path))
%wire %+ lien duct
|=(=wire ?=(^ (find wire.about.dbug wire)))
%term !!
==
::
%outgoing
=; =tang
?^ tang tang
[%leaf "no matching subscriptions"]~
%+ murn
%+ sort ~(tap by wex.bowl)
|= [[[a=wire *] *] [[b=wire *] *]]
(aor a b)
|= [[=wire =ship =term] [acked=? =path]]
^- (unit tank)
=; relevant=?
?. relevant ~
`>[wire=wire agnt=[ship term] path=path ackd=acked]<
?: ?=(~ about.dbug) &
?- -.about.dbug
%ship =(ship ship.about.dbug)
%path ?=(^ (find path.about.dbug path))
%wire ?=(^ (find wire.about.dbug wire))
%term =(term term.about.dbug)
==
==
::
++ on-peek
|= =path
^- (unit (unit cage))
?. ?=([@ %dbug *] path)
(on-peek:ag path)
?+ path [~ ~]
[%u %dbug ~] ``noun+!>(&)
[%x %dbug %state ~] ``noun+!>(on-save:ag)
[%x %dbug %subscriptions ~] ``noun+!>([wex sup]:bowl)
==
::
++ on-init
^- (quip card:agent:gall agent:gall)
=^ cards agent on-init:ag
[cards this]
::
++ on-save on-save:ag
::
++ on-load
|= old-state=vase
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-load:ag old-state)
[cards this]
::
++ on-watch
|= =path
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-watch:ag path)
[cards this]
::
++ on-leave
|= =path
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-leave:ag path)
[cards this]
::
++ on-agent
|= [=wire =sign:agent:gall]
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-agent:ag wire sign)
[cards this]
::
++ on-arvo
|= [=wire =sign-arvo]
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-arvo:ag wire sign-arvo)
[cards this]
::
++ on-fail
|= [=term =tang]
^- (quip card:agent:gall agent:gall)
=^ cards agent (on-fail:ag term tang)
[cards this]
--
--
@@ -0,0 +1,65 @@
|= [agent=agent:gall]
|_ =bowl:gall
++ on-init
`agent
::
++ on-save
!>(~)
::
++ on-load
|= old-state=vase
`agent
::
++ on-poke
|= =cage
~| "unexpected poke to {<dap.bowl>} with mark {<p.cage>}"
!!
::
++ on-watch
|= =path
~| "unexpected subscription to {<dap.bowl>} on path {<path>}"
!!
::
++ on-leave
|= path
`agent
::
++ on-peek
|= =path
~| "unexpected scry into {<dap.bowl>} on path {<path>}"
!!
::
++ on-agent
|= [=wire =sign:agent:gall]
^- (quip card:agent:gall _agent)
?- -.sign
%poke-ack
?~ p.sign
`agent
%- (slog leaf+"poke failed from {<dap.bowl>} on wire {<wire>}" u.p.sign)
`agent
::
%watch-ack
?~ p.sign
`agent
=/ =tank leaf+"subscribe failed from {<dap.bowl>} on wire {<wire>}"
%- (slog tank u.p.sign)
`agent
::
%kick `agent
%fact
~| "unexpected subscription update to {<dap.bowl>} on wire {<wire>}"
~| "with mark {<p.cage.sign>}"
!!
==
::
++ on-arvo
|= [=wire =sign-arvo]
~| "unexpected system response {<sign-arvo>} to {<dap.bowl>} on wire {<wire>}"
!!
::
++ on-fail
|= [=term =tang]
%- (slog leaf+"error in {<dap.bowl>}" >term< tang)
`agent
--
@@ -0,0 +1,159 @@
=, eyre
|%
+$ request-line
$: [ext=(unit @ta) site=(list @t)]
args=(list [key=@t value=@t])
==
:: +parse-request-line: take a cord and parse out a url
::
++ parse-request-line
|= url=@t
^- request-line
(fall (rush url ;~(plug apat:de-purl:html yque:de-purl:html)) [[~ ~] ~])
::
++ manx-to-octs
|= man=manx
^- octs
(as-octt:mimes:html (en-xml:html man))
::
++ json-to-octs
|= jon=json
^- octs
(as-octs:mimes:html (en:json:html jon))
::
++ app
|%
::
:: +require-authorization:
:: redirect to the login page when unauthenticated
:: otherwise call handler on inbound request
::
++ require-authorization
|= $: =inbound-request:eyre
handler=$-(inbound-request:eyre simple-payload:http)
==
^- simple-payload:http
::
?: authenticated.inbound-request
~! this
~! +:*handler
(handler inbound-request)
::
=- [[307 ['location' -]~] ~]
%^ cat 3
'/~/login?redirect='
url.request.inbound-request
::
:: +require-authorization-simple:
:: redirect to the login page when unauthenticated
:: otherwise pass through simple-paylod
::
++ require-authorization-simple
|= [=inbound-request:eyre =simple-payload:http]
^- simple-payload:http
::
?: authenticated.inbound-request
~! this
simple-payload
::
=- [[307 ['location' -]~] ~]
%^ cat 3
'/~/login?redirect='
url.request.inbound-request
::
++ give-simple-payload
|= [eyre-id=@ta =simple-payload:http]
^- (list card:agent:gall)
=/ header-cage
[%http-response-header !>(response-header.simple-payload)]
=/ data-cage
[%http-response-data !>(data.simple-payload)]
:~ [%give %fact ~[/http-response/[eyre-id]] header-cage]
[%give %fact ~[/http-response/[eyre-id]] data-cage]
[%give %kick ~[/http-response/[eyre-id]] ~]
==
--
++ gen
|%
::
++ max-1-da ['cache-control' 'max-age=86400']
++ max-1-wk ['cache-control' 'max-age=604800']
::
++ html-response
=| cache=?
|= =octs
^- simple-payload:http
:_ `octs
[200 [['content-type' 'text/html'] ?:(cache [max-1-wk ~] ~)]]
::
++ css-response
=| cache=?
|= =octs
^- simple-payload:http
:_ `octs
[200 [['content-type' 'text/css'] ?:(cache [max-1-wk ~] ~)]]
::
++ js-response
=| cache=?
|= =octs
^- simple-payload:http
:_ `octs
[200 [['content-type' 'text/javascript'] ?:(cache [max-1-wk ~] ~)]]
::
++ png-response
=| cache=?
|= =octs
^- simple-payload:http
:_ `octs
[200 [['content-type' 'image/png'] ?:(cache [max-1-wk ~] ~)]]
::
++ svg-response
=| cache=?
|= =octs
^- simple-payload:http
:_ `octs
[200 [['content-type' 'image/svg+xml'] ?:(cache [max-1-wk ~] ~)]]
::
++ ico-response
|= =octs
^- simple-payload:http
[[200 [['content-type' 'image/x-icon'] max-1-wk ~]] `octs]
::
++ woff2-response
=| cache=?
|= =octs
^- simple-payload:http
[[200 [['content-type' 'font/woff2'] max-1-wk ~]] `octs]
::
++ json-response
=| cache=_|
|= =json
^- simple-payload:http
:_ `(json-to-octs json)
[200 [['content-type' 'application/json'] ?:(cache [max-1-da ~] ~)]]
::
++ manx-response
=| cache=_|
|= man=manx
^- simple-payload:http
:_ `(manx-to-octs man)
[200 [['content-type' 'text/html'] ?:(cache [max-1-da ~] ~)]]
::
++ not-found
^- simple-payload:http
[[404 ~] ~]
::
++ login-redirect
|= =request:http
^- simple-payload:http
=- [[307 ['location' -]~] ~]
%^ cat 3
'/~/login?redirect='
url.request
::
++ redirect
|= redirect=cord
^- simple-payload:http
[[307 ['location' redirect]~] ~]
--
--
@@ -0,0 +1,34 @@
|_ bil=(list dude:gall)
++ grow
|%
++ mime `^mime`[/text/x-bill (as-octs:mimes:html hoon)]
++ noun bil
++ hoon
^- @t
|^ (crip (of-wall:format (wrap-lines (spit-duz bil))))
::
++ wrap-lines
|= taz=wall
^- wall
?~ taz ["~"]~
:- (weld ":~ " i.taz)
%- snoc :_ "=="
(turn t.taz |=(t=tape (weld " " t)))
::
++ spit-duz
|= duz=(list dude:gall)
^- wall
(turn duz |=(=dude:gall ['%' (trip dude)]))
--
++ txt (to-wain:format hoon)
--
++ grab
|%
++ noun (list dude:gall)
++ mime
|= [=mite len=@ud tex=@]
~_ tex
!<((list dude:gall) (slap !>(~) (ream tex)))
--
++ grad %noun
--
@@ -0,0 +1,36 @@
:::: /hoon/hoon/mar
::
/? 310
::
=, eyre
|_ own=@t
::
++ grow :: convert to
|%
++ mime `^mime`[/text/x-hoon (as-octs:mimes:html own)] :: convert to %mime
++ hymn
;html
;head
;title:"Source"
;script@"//cdnjs.cloudflare.com/ajax/libs/codemirror/4.3.0/codemirror.js";
;script@"/lib/syntax/hoon.js";
;link(rel "stylesheet", href "//cdnjs.cloudflare.com/ajax/libs/".
"codemirror/4.3.0/codemirror.min.css");
;link/"/lib/syntax/codemirror.css"(rel "stylesheet");
==
;body
;textarea#src:"{(trip own)}"
;script:'CodeMirror.fromTextArea(src, {lineNumbers:true, readOnly:true})'
==
==
++ txt
(to-wain:format own)
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] q.q)
++ noun @t :: clam from %noun
++ txt of-wain:format
--
++ grad %txt
--
@@ -0,0 +1,28 @@
|_ kal=waft:clay
++ grow
|%
++ mime `^mime`[/text/x-kelvin (as-octs:mimes:html hoon)]
++ noun kal
++ hoon
%+ rap 3
%+ turn
%+ sort
~(tap in (waft-to-wefts:clay kal))
|= [a=weft b=weft]
?: =(lal.a lal.b)
(gte num.a num.b)
(gte lal.a lal.b)
|= =weft
(rap 3 '[%' (scot %tas lal.weft) ' ' (scot %ud num.weft) ']\0a' ~)
::
++ txt (to-wain:format hoon)
--
++ grab
|%
++ noun waft:clay
++ mime
|= [=mite len=@ud tex=@]
(cord-to-waft:clay tex)
--
++ grad %noun
--
@@ -1,18 +0,0 @@
::
:::: /hoon/map/mar
:: Mark for js source maps
/? 310
::
=, eyre
|_ mud=@
++ grow
|%
++ mime [/application/octet-stream (as-octs:mimes:html (@t mud))]
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] (@t q.q))
++ noun cord :: clam from %noun
--
++ grad %mime
--
@@ -0,0 +1,32 @@
::
:::: /hoon/mime/mar
::
/? 310
::
|_ own=mime
++ grow
^?
|%
++ jam `@`q.q.own
--
::
++ grab :: convert from
^?
|%
++ noun mime :: clam from %noun
++ tape
|=(a=_"" [/application/x-urb-unknown (as-octt:mimes:html a)])
--
++ grad
^?
|%
++ form %mime
++ diff |=(mime +<)
++ pact |=(mime +<)
++ join |=([mime mime] `(unit mime)`~)
++ mash
|= [[ship desk mime] [ship desk mime]]
^- mime
~|(%mime-mash !!)
--
--
@@ -0,0 +1,22 @@
::
:::: /hoon/noun/mar
::
/? 310
!:
:::: A minimal noun mark
|_ non=*
++ grab |%
++ noun *
--
++ grow |%
++ mime [/application/x-urb-jam (as-octs:mimes:html (jam non))]
--
++ grad
|%
++ form %noun
++ diff |=(* +<)
++ pact |=(* +<)
++ join |=([* *] *(unit *))
++ mash |=([[ship desk *] [ship desk *]] `*`~|(%noun-mash !!))
--
--
@@ -1,12 +0,0 @@
|_ dat=octs
++ grow
|%
++ mime [/font/ttf dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -0,0 +1,274 @@
::
:::: /hoon/txt/mar
::
/? 310
::
=, clay
=, differ
=, format
=, mimes:html
|_ txt=wain
::
++ grab :: convert from
|%
++ mime |=((pair mite octs) (to-wain q.q))
++ noun wain :: clam from %noun
--
++ grow
=> v=.
|%
++ mime => v [/text/plain (as-octs (of-wain txt))]
--
++ grad
|%
++ form %txt-diff
++ diff
|= tyt=wain
^- (urge cord)
(lusk txt tyt (loss txt tyt))
::
++ pact
|= dif=(urge cord)
~| [%pacting dif]
^- wain
(lurk txt dif)
::
++ join
|= [ali=(urge cord) bob=(urge cord)]
^- (unit (urge cord))
|^
=. ali (clean ali)
=. bob (clean bob)
|- ^- (unit (urge cord))
?~ ali `bob
?~ bob `ali
?- -.i.ali
%&
?- -.i.bob
%&
?: =(p.i.ali p.i.bob)
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
?: (gth p.i.ali p.i.bob)
%+ bind $(p.i.ali (sub p.i.ali p.i.bob), bob t.bob)
|=(cud=(urge cord) [i.bob cud])
%+ bind $(ali t.ali, p.i.bob (sub p.i.bob p.i.ali))
|=(cud=(urge cord) [i.ali cud])
::
%|
?: =(p.i.ali (lent p.i.bob))
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.bob cud])
?: (gth p.i.ali (lent p.i.bob))
%+ bind $(p.i.ali (sub p.i.ali (lent p.i.bob)), bob t.bob)
|=(cud=(urge cord) [i.bob cud])
~
==
::
%|
?- -.i.bob
%|
?. =(i.ali i.bob)
~
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
::
%&
?: =(p.i.bob (lent p.i.ali))
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
?: (gth p.i.bob (lent p.i.ali))
%+ bind $(ali t.ali, p.i.bob (sub p.i.bob (lent p.i.ali)))
|=(cud=(urge cord) [i.ali cud])
~
==
==
++ clean :: clean
|= wig=(urge cord)
^- (urge cord)
?~ wig ~
?~ t.wig wig
?: ?=(%& -.i.wig)
?: ?=(%& -.i.t.wig)
$(wig [[%& (add p.i.wig p.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
?: ?=(%| -.i.t.wig)
$(wig [[%| (welp p.i.wig p.i.t.wig) (welp q.i.wig q.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
--
::
++ mash
|= $: [als=ship ald=desk ali=(urge cord)]
[bos=ship bod=desk bob=(urge cord)]
==
^- (urge cord)
|^
=. ali (clean ali)
=. bob (clean bob)
|- ^- (urge cord)
?~ ali bob
?~ bob ali
?- -.i.ali
%&
?- -.i.bob
%&
?: =(p.i.ali p.i.bob)
[i.ali $(ali t.ali, bob t.bob)]
?: (gth p.i.ali p.i.bob)
[i.bob $(p.i.ali (sub p.i.ali p.i.bob), bob t.bob)]
[i.ali $(ali t.ali, p.i.bob (sub p.i.bob p.i.ali))]
::
%|
?: =(p.i.ali (lent p.i.bob))
[i.bob $(ali t.ali, bob t.bob)]
?: (gth p.i.ali (lent p.i.bob))
[i.bob $(p.i.ali (sub p.i.ali (lent p.i.bob)), bob t.bob)]
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
:: ~ :: here, alice is good for a while, but not for the whole
== :: length of bob's changes
::
%|
?- -.i.bob
%|
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
::
%&
?: =(p.i.bob (lent p.i.ali))
[i.ali $(ali t.ali, bob t.bob)]
?: (gth p.i.bob (lent p.i.ali))
[i.ali $(ali t.ali, p.i.bob (sub p.i.bob (lent p.i.ali)))]
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
==
==
::
++ annotate :: annotate conflict
|= $: ali=(list @t)
bob=(list @t)
bas=(list @t)
==
^- (list @t)
%- zing
^- (list (list @t))
%- flop
^- (list (list @t))
:- :_ ~
%^ cat 3 '<<<<<<<<<<<<'
%^ cat 3 ' '
%^ cat 3 `@t`(scot %p bos)
%^ cat 3 '/'
bod
:- bob
:- ~['------------']
:- bas
:- ~['++++++++++++']
:- ali
:- :_ ~
%^ cat 3 '>>>>>>>>>>>>'
%^ cat 3 ' '
%^ cat 3 `@t`(scot %p als)
%^ cat 3 '/'
ald
~
::
++ clean :: clean
|= wig=(urge cord)
^- (urge cord)
?~ wig ~
?~ t.wig wig
?: ?=(%& -.i.wig)
?: ?=(%& -.i.t.wig)
$(wig [[%& (add p.i.wig p.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
?: ?=(%| -.i.t.wig)
$(wig [[%| (welp p.i.wig p.i.t.wig) (welp q.i.wig q.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
::
++ resolve
|= [ali=(urge cord) bob=(urge cord)]
^- [fic=[%| p=(list cord) q=(list cord)] ali=(urge cord) bob=(urge cord)]
=- [[%| bac (annotate alc boc bac)] ali bob]
|- ^- $: $: bac=(list cord)
alc=(list cord)
boc=(list cord)
==
ali=(urge cord)
bob=(urge cord)
==
?~ ali [[~ ~ ~] ali bob]
?~ bob [[~ ~ ~] ali bob]
?- -.i.ali
%&
?- -.i.bob
%& [[~ ~ ~] ali bob] :: no conflict
%|
=+ lob=(lent p.i.bob)
?: =(lob p.i.ali)
[[p.i.bob p.i.bob q.i.bob] t.ali t.bob]
?: (lth lob p.i.ali)
[[p.i.bob p.i.bob q.i.bob] [[%& (sub p.i.ali lob)] t.ali] t.bob]
=+ wat=(scag (sub lob p.i.ali) p.i.bob)
=+ ^= res
%= $
ali t.ali
bob [[%| (scag (sub lob p.i.ali) p.i.bob) ~] t.bob]
==
:* :* (welp bac.res wat)
(welp alc.res wat)
(welp boc.res q.i.bob)
==
ali.res
bob.res
==
==
::
%|
?- -.i.bob
%&
=+ loa=(lent p.i.ali)
?: =(loa p.i.bob)
[[p.i.ali q.i.ali p.i.ali] t.ali t.bob]
?: (lth loa p.i.bob)
[[p.i.ali q.i.ali p.i.ali] t.ali [[%& (sub p.i.bob loa)] t.bob]]
=+ wat=(slag (sub loa p.i.bob) p.i.ali)
=+ ^= res
%= $
ali [[%| (scag (sub loa p.i.bob) p.i.ali) ~] t.ali]
bob t.bob
==
:* :* (welp bac.res wat)
(welp alc.res q.i.ali)
(welp boc.res wat)
==
ali.res
bob.res
==
::
%|
=+ loa=(lent p.i.ali)
=+ lob=(lent p.i.bob)
?: =(loa lob)
[[p.i.ali q.i.ali q.i.bob] t.ali t.bob]
=+ ^= res
?: (gth loa lob)
$(ali [[%| (scag (sub loa lob) p.i.ali) ~] t.ali], bob t.bob)
~& [%scagging loa=loa pibob=p.i.bob slag=(scag loa p.i.bob)]
$(ali t.ali, bob [[%| (scag (sub lob loa) p.i.bob) ~] t.bob])
:* :* (welp bac.res ?:((gth loa lob) p.i.bob p.i.ali))
(welp alc.res q.i.ali)
(welp boc.res q.i.bob)
==
ali.res
bob.res
==
==
==
--
--
--
@@ -1,12 +0,0 @@
|_ dat=octs
++ grow
|%
++ mime [/font/woff dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -17,16 +17,10 @@ if [ -d ${app_desk_dir} ]; then
exit 0
fi
app_build=/app-builds/${CERC_URBIT_APP}/build
app_mark_files=/app-builds/${CERC_URBIT_APP}/mar
app_docket_file=/app-builds/${CERC_URBIT_APP}/desk.docket-0
echo "Reading app build from ${app_build}"
echo "Reading additional mark files from ${app_mark_files}"
echo "Reading docket file ${app_docket_file}"
app_files=/app-builds/${CERC_URBIT_APP}/urbit-files
# Loop until the app's build appears
while [ ! -d ${app_build} ]; do
while [ ! -d "${app_files}/build" ]; do
echo "${CERC_URBIT_APP} app build not found, retrying in 5s..."
sleep 5
done
@@ -50,15 +44,16 @@ hood () {
hood "merge %${CERC_URBIT_APP} our %landscape"
hood "mount %${CERC_URBIT_APP}"
# Copy over build to desk data dir
cp -r ${app_build} ${app_desk_dir}
# Copy over the additional mark files
cp ${app_mark_files}/* ${app_desk_dir}/mar/
rm "${app_desk_dir}/desk.bill"
rm "${app_desk_dir}/desk.bill" # Don't install the landscape apps
rm "${app_desk_dir}/desk.ship"
echo "Copying files from ${app_files}"
# Copy over build to desk data dir
cp -r ${app_files}/* ${app_desk_dir}
rm ${app_desk_dir}/desk.docket-0 # Remove until we have a valid glob path; it's added back again below
# Commit changes and create a glob
hood "commit %${CERC_URBIT_APP}"
dojo "-landscape!make-glob %${CERC_URBIT_APP} /build"
@@ -99,8 +94,8 @@ while true; do
done
# Replace the docket file for app
# Substitue the glob URL and hash
cp ${app_docket_file} ${app_desk_dir}/
# Substitute the glob URL and hash
cp ${app_files}/desk.docket-0 ${app_desk_dir}/
sed -i "s|REPLACE_WITH_GLOB_URL|${glob_url}|g; s|REPLACE_WITH_GLOB_HASH|${glob_hash}|g" ${app_desk_dir}/desk.docket-0
# Commit changes and install the app
+32
View File
@@ -0,0 +1,32 @@
#!/bin/bash
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
# Check and exit if a deployment already exists (on restarts)
if [ -d /app-builds/vega/build ]; then
echo "Build already exists, remove volume to rebuild"
exit 0
fi
./docker/prepare-dist.sh
# Post-process the output: rename files, e.g., `AlphaLyrae.woff2` => `alphalyrae.woff2`
while read filepath; do
filename=$(basename "$filepath")
dir=$(dirname "$filepath")
if echo "$filename" | grep -q "[A-Z]"; then
newfilename=$(tr '[:upper:]' '[:lower:]' <<< "$filename")
echo "Translating '$filepath' => $dir/$newfilename'"
mv "$filepath" "$dir/$newfilename"
find dist-result -type f | xargs sed -i "s/$filename/$newfilename/g"
fi
done < <(find dist-result/ -type f)
# Copy over dist-result and other files to `app-builds` volume for urbit deployment
mkdir -p /app-builds/vega
cp -r ./dist-result /app-builds/vega/build
cp -r urbit-files/* /app-builds/vega/
@@ -0,0 +1,10 @@
:~ title+'Vegas'
info+'vegas'
color+0xcd.75df
image+'https://vegaprotocol.eth.limo/favicon.ico'
base+'vegas'
glob-http+['REPLACE_WITH_GLOB_URL' REPLACE_WITH_GLOB_HASH]
version+[0 0 1]
website+'https://vega.xyz/'
license+'MIT'
==
@@ -0,0 +1,223 @@
/- *docket
|%
::
++ mime
|%
+$ draft
$: title=(unit @t)
info=(unit @t)
color=(unit @ux)
glob-http=(unit [=url hash=@uvH])
glob-ames=(unit [=ship hash=@uvH])
base=(unit term)
site=(unit path)
image=(unit url)
version=(unit version)
website=(unit url)
license=(unit cord)
==
::
++ finalize
|= =draft
^- (unit docket)
?~ title.draft ~
?~ info.draft ~
?~ color.draft ~
?~ version.draft ~
?~ website.draft ~
?~ license.draft ~
=/ href=(unit href)
?^ site.draft `[%site u.site.draft]
?~ base.draft ~
?^ glob-http.draft
`[%glob u.base hash.u.glob-http %http url.u.glob-http]:draft
?~ glob-ames.draft
~
`[%glob u.base hash.u.glob-ames %ames ship.u.glob-ames]:draft
?~ href ~
=, draft
:- ~
:* %1
u.title
u.info
u.color
u.href
image
u.version
u.website
u.license
==
::
++ from-clauses
=| =draft
|= cls=(list clause)
^- (unit docket)
=* loop $
?~ cls (finalize draft)
=* clause i.cls
=. draft
?- -.clause
%title draft(title `title.clause)
%info draft(info `info.clause)
%color draft(color `color.clause)
%glob-http draft(glob-http `[url hash]:clause)
%glob-ames draft(glob-ames `[ship hash]:clause)
%base draft(base `base.clause)
%site draft(site `path.clause)
%image draft(image `url.clause)
%version draft(version `version.clause)
%website draft(website `website.clause)
%license draft(license `license.clause)
==
loop(cls t.cls)
::
++ to-clauses
|= d=docket
^- (list clause)
%- zing
:~ :~ title+title.d
info+info.d
color+color.d
version+version.d
website+website.d
license+license.d
==
?~ image.d ~ ~[image+u.image.d]
?: ?=(%site -.href.d) ~[site+path.href.d]
=/ ref=glob-reference glob-reference.href.d
:~ base+base.href.d
?- -.location.ref
%http [%glob-http url.location.ref hash.ref]
%ames [%glob-ames ship.location.ref hash.ref]
== == ==
::
++ spit-clause
|= =clause
^- tape
%+ weld " {(trip -.clause)}+"
?+ -.clause "'{(trip +.clause)}'"
%color (scow %ux color.clause)
%site (spud path.clause)
::
%glob-http
"['{(trip url.clause)}' {(scow %uv hash.clause)}]"
::
%glob-ames
"[{(scow %p ship.clause)} {(scow %uv hash.clause)}]"
::
%version
=, version.clause
"[{(scow %ud major)} {(scow %ud minor)} {(scow %ud patch)}]"
==
::
++ spit-docket
|= dock=docket
^- tape
;: welp
":~\0a"
`tape`(zing (join "\0a" (turn (to-clauses dock) spit-clause)))
"\0a=="
==
--
::
++ enjs
=, enjs:format
|%
::
++ charge-update
|= u=^charge-update
^- json
%+ frond -.u
^- json
?- -.u
%del-charge s+desk.u
::
%initial
%- pairs
%+ turn ~(tap by initial.u)
|=([=desk c=^charge] [desk (charge c)])
::
%add-charge
%- pairs
:~ desk+s+desk.u
charge+(charge charge.u)
==
==
::
++ num
|= a=@u
^- ^tape
=/ p=json (numb a)
?> ?=(%n -.p)
(trip p.p)
::
++ version
|= v=^version
^- json
:- %s
%- crip
"{(num major.v)}.{(num minor.v)}.{(num patch.v)}"
::
++ merge
|= [a=json b=json]
^- json
?> &(?=(%o -.a) ?=(%o -.b))
[%o (~(uni by p.a) p.b)]
::
++ href
|= h=^href
%+ frond -.h
?- -.h
%site s+(spat path.h)
%glob
%- pairs
:~ base+s+base.h
glob-reference+(glob-reference glob-reference.h)
==
==
::
++ glob-reference
|= ref=^glob-reference
%- pairs
:~ hash+s+(scot %uv hash.ref)
location+(glob-location location.ref)
==
::
++ glob-location
|= loc=^glob-location
^- json
%+ frond -.loc
?- -.loc
%http s+url.loc
%ames s+(scot %p ship.loc)
==
::
++ charge
|= c=^charge
%+ merge (docket docket.c)
%- pairs
:~ chad+(chad chad.c)
==
::
++ docket
|= d=^docket
^- json
%- pairs
:~ title+s+title.d
info+s+info.d
color+s+(scot %ux color.d)
href+(href href.d)
image+?~(image.d ~ s+u.image.d)
version+(version version.d)
license+s+license.d
website+s+website.d
==
::
++ chad
|= c=^chad
%+ frond -.c
?+ -.c ~
%hung s+err.c
==
--
--
@@ -0,0 +1,34 @@
|_ bil=(list dude:gall)
++ grow
|%
++ mime `^mime`[/text/x-bill (as-octs:mimes:html hoon)]
++ noun bil
++ hoon
^- @t
|^ (crip (of-wall:format (wrap-lines (spit-duz bil))))
::
++ wrap-lines
|= taz=wall
^- wall
?~ taz ["~"]~
:- (weld ":~ " i.taz)
%- snoc :_ "=="
(turn t.taz |=(t=tape (weld " " t)))
::
++ spit-duz
|= duz=(list dude:gall)
^- wall
(turn duz |=(=dude:gall ['%' (trip dude)]))
--
++ txt (to-wain:format hoon)
--
++ grab
|%
++ noun (list dude:gall)
++ mime
|= [=mite len=@ud tex=@]
~_ tex
!<((list dude:gall) (slap !>(~) (ream tex)))
--
++ grad %noun
--
@@ -0,0 +1,25 @@
::
:::: /hoon/css/mar
::
/? 310
=, eyre
=, mimes:html
|_ mud=@t
++ grow :: convert to
|%
++ mime [/text/css (as-octs mud)] :: convert to %mime
++ hymn :: convert to %hymn
|^ html
++ style ;style
;- (trip mud)
==
++ html ;html:(head:"{style}" body)
--
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] (@t q.q))
++ noun @t :: clam from %noun
--
++ grad %mime
--
@@ -0,0 +1,25 @@
/+ dock=docket
|_ =docket:dock
++ grow
|%
++ mime
^- ^mime
[/text/x-docket (as-octt:mimes:html (spit-docket:mime:dock docket))]
++ noun docket
++ json (docket:enjs:dock docket)
--
++ grab
|%
::
++ mime
|= [=mite len=@ud tex=@]
^- docket:dock
%- need
%- from-clauses:mime:dock
!<((list clause:dock) (slap !>(~) (ream tex)))
::
++ noun docket:dock
--
++ grad %noun
--
@@ -0,0 +1,36 @@
:::: /hoon/hoon/mar
::
/? 310
::
=, eyre
|_ own=@t
::
++ grow :: convert to
|%
++ mime `^mime`[/text/x-hoon (as-octs:mimes:html own)] :: convert to %mime
++ hymn
;html
;head
;title:"Source"
;script@"//cdnjs.cloudflare.com/ajax/libs/codemirror/4.3.0/codemirror.js";
;script@"/lib/syntax/hoon.js";
;link(rel "stylesheet", href "//cdnjs.cloudflare.com/ajax/libs/".
"codemirror/4.3.0/codemirror.min.css");
;link/"/lib/syntax/codemirror.css"(rel "stylesheet");
==
;body
;textarea#src:"{(trip own)}"
;script:'CodeMirror.fromTextArea(src, {lineNumbers:true, readOnly:true})'
==
==
++ txt
(to-wain:format own)
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] q.q)
++ noun @t :: clam from %noun
++ txt of-wain:format
--
++ grad %txt
--
@@ -0,0 +1,22 @@
::
:::: /hoon/html/mar
::
/? 310
::
:::: compute
::
=, html
|_ htm=@t
++ grow :: convert to
^?
|% ::
++ mime [/text/html (met 3 htm) htm] :: to %mime
++ hymn (need (de-xml htm)) :: to %hymn
-- ::
++ grab ^?
|% :: convert from
++ noun @t :: clam from %noun
++ mime |=([p=mite q=octs] q.q) :: retrieve form %mime
--
++ grad %mime
--
@@ -0,0 +1,12 @@
|_ dat=@
++ grow
|%
++ mime [/image/x-icon (as-octs:mimes:html dat)]
--
++ grab
|%
++ mime |=([p=mite q=octs] q.q)
++ noun @
--
++ grad %mime
--
@@ -0,0 +1,12 @@
|_ dat=@
++ grow
|%
++ mime [/image/jpeg (as-octs:mimes:html dat)]
--
++ grab
|%
++ mime |=([p=mite q=octs] q.q)
++ noun @
--
++ grad %mime
--
@@ -0,0 +1,25 @@
::
:::: /hoon/js/mar
::
/? 310
::
=, eyre
|_ mud=@
++ grow
|%
++ mime [/application/javascript (as-octs:mimes:html (@t mud))]
++ hymn :: convert to %hymn
|^ html
++ script ;script
;- (trip (@t mud))
==
++ html ;html:(head:"{script}" body)
--
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] (@t q.q))
++ noun cord :: clam from %noun
--
++ grad %mime
--
@@ -0,0 +1,26 @@
::
:::: /hoon/json/mar
::
/? 310
::
:::: compute
::
=, eyre
=, format
=, html
|_ jon=^json
::
++ grow :: convert to
|%
++ mime [/application/json (as-octs:mimes -:txt)] :: convert to %mime
++ txt [(en:json jon)]~
--
++ grab
|% :: convert from
++ mime |=([p=mite q=octs] (fall (de:json (@t q.q)) *^json))
++ noun ^json :: clam from %noun
++ numb numb:enjs
++ time time:enjs
--
++ grad %mime
--
@@ -0,0 +1,42 @@
::
/- *json-rpc
::
|_ res=response
::
++ grad %noun
++ grow
|%
++ noun res
--
++ grab :: convert from
|%
++ noun response :: from noun
++ httr :: from httr
|= hit=httr:eyre
^- response
~| hit
?: ?=(%2 (div p.hit 100))
%- json
?~ r.hit
a+~
(need (de:json:html q:u.r.hit))
fail+hit
++ json :: from json
=, dejs-soft:format
|= a=json
^- response
=; dere
=+ res=((ar dere) a)
?~ res (need (dere a))
[%batch u.res]
|= a=json
^- (unit response)
=/ res=(unit [@t json])
::TODO breaks when no id present
((ot id+so result+some ~) a)
?^ res `[%result u.res]
~| a
:+ ~ %error %- need
((ot id+so error+(ot code+no message+so ~) ~) a)
--
--
@@ -0,0 +1,28 @@
|_ kal=waft:clay
++ grow
|%
++ mime `^mime`[/text/x-kelvin (as-octs:mimes:html hoon)]
++ noun kal
++ hoon
%+ rap 3
%+ turn
%+ sort
~(tap in (waft-to-wefts:clay kal))
|= [a=weft b=weft]
?: =(lal.a lal.b)
(gte num.a num.b)
(gte lal.a lal.b)
|= =weft
(rap 3 '[%' (scot %tas lal.weft) ' ' (scot %ud num.weft) ']\0a' ~)
::
++ txt (to-wain:format hoon)
--
++ grab
|%
++ noun waft:clay
++ mime
|= [=mite len=@ud tex=@]
(cord-to-waft:clay tex)
--
++ grad %noun
--
@@ -0,0 +1,32 @@
::
:::: /hoon/mime/mar
::
/? 310
::
|_ own=mime
++ grow
^?
|%
++ jam `@`q.q.own
--
::
++ grab :: convert from
^?
|%
++ noun mime :: clam from %noun
++ tape
|=(a=_"" [/application/x-urb-unknown (as-octt:mimes:html a)])
--
++ grad
^?
|%
++ form %mime
++ diff |=(mime +<)
++ pact |=(mime +<)
++ join |=([mime mime] `(unit mime)`~)
++ mash
|= [[ship desk mime] [ship desk mime]]
^- mime
~|(%mime-mash !!)
--
--
@@ -0,0 +1,22 @@
::
:::: /hoon/noun/mar
::
/? 310
!:
:::: A minimal noun mark
|_ non=*
++ grab |%
++ noun *
--
++ grow |%
++ mime [/application/x-urb-jam (as-octs:mimes:html (jam non))]
--
++ grad
|%
++ form %noun
++ diff |=(* +<)
++ pact |=(* +<)
++ join |=([* *] *(unit *))
++ mash |=([[ship desk *] [ship desk *]] `*`~|(%noun-mash !!))
--
--
@@ -0,0 +1,12 @@
|_ dat=@
++ grow
|%
++ mime [/image/png (as-octs:mimes:html dat)]
--
++ grab
|%
++ mime |=([p=mite q=octs] q.q)
++ noun @
--
++ grad %mime
--
@@ -0,0 +1,20 @@
|_ s=ship
++ grad %noun
++ grow
|%
++ noun s
++ json s+(scot %p s)
++ mime
^- ^mime
[/text/x-ship (as-octt:mimes:html (scow %p s))]
--
++ grab
|%
++ noun ship
++ json (su:dejs:format ;~(pfix sig fed:ag))
++ mime
|= [=mite len=@ tex=@]
(slav %p (snag 0 (to-wain:format tex)))
--
--
@@ -0,0 +1,16 @@
::
:::: /hoon/txt-diff/mar
::
/? 310
|_ txt-diff=(urge:clay cord)
::
++ grad %noun
++ grow
|%
++ noun txt-diff
--
++ grab :: convert from
|%
++ noun (urge:clay cord) :: make from %noun
--
--
@@ -0,0 +1,274 @@
::
:::: /hoon/txt/mar
::
/? 310
::
=, clay
=, differ
=, format
=, mimes:html
|_ txt=wain
::
++ grab :: convert from
|%
++ mime |=((pair mite octs) (to-wain q.q))
++ noun wain :: clam from %noun
--
++ grow
=> v=.
|%
++ mime => v [/text/plain (as-octs (of-wain txt))]
--
++ grad
|%
++ form %txt-diff
++ diff
|= tyt=wain
^- (urge cord)
(lusk txt tyt (loss txt tyt))
::
++ pact
|= dif=(urge cord)
~| [%pacting dif]
^- wain
(lurk txt dif)
::
++ join
|= [ali=(urge cord) bob=(urge cord)]
^- (unit (urge cord))
|^
=. ali (clean ali)
=. bob (clean bob)
|- ^- (unit (urge cord))
?~ ali `bob
?~ bob `ali
?- -.i.ali
%&
?- -.i.bob
%&
?: =(p.i.ali p.i.bob)
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
?: (gth p.i.ali p.i.bob)
%+ bind $(p.i.ali (sub p.i.ali p.i.bob), bob t.bob)
|=(cud=(urge cord) [i.bob cud])
%+ bind $(ali t.ali, p.i.bob (sub p.i.bob p.i.ali))
|=(cud=(urge cord) [i.ali cud])
::
%|
?: =(p.i.ali (lent p.i.bob))
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.bob cud])
?: (gth p.i.ali (lent p.i.bob))
%+ bind $(p.i.ali (sub p.i.ali (lent p.i.bob)), bob t.bob)
|=(cud=(urge cord) [i.bob cud])
~
==
::
%|
?- -.i.bob
%|
?. =(i.ali i.bob)
~
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
::
%&
?: =(p.i.bob (lent p.i.ali))
%+ bind $(ali t.ali, bob t.bob)
|=(cud=(urge cord) [i.ali cud])
?: (gth p.i.bob (lent p.i.ali))
%+ bind $(ali t.ali, p.i.bob (sub p.i.bob (lent p.i.ali)))
|=(cud=(urge cord) [i.ali cud])
~
==
==
++ clean :: clean
|= wig=(urge cord)
^- (urge cord)
?~ wig ~
?~ t.wig wig
?: ?=(%& -.i.wig)
?: ?=(%& -.i.t.wig)
$(wig [[%& (add p.i.wig p.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
?: ?=(%| -.i.t.wig)
$(wig [[%| (welp p.i.wig p.i.t.wig) (welp q.i.wig q.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
--
::
++ mash
|= $: [als=ship ald=desk ali=(urge cord)]
[bos=ship bod=desk bob=(urge cord)]
==
^- (urge cord)
|^
=. ali (clean ali)
=. bob (clean bob)
|- ^- (urge cord)
?~ ali bob
?~ bob ali
?- -.i.ali
%&
?- -.i.bob
%&
?: =(p.i.ali p.i.bob)
[i.ali $(ali t.ali, bob t.bob)]
?: (gth p.i.ali p.i.bob)
[i.bob $(p.i.ali (sub p.i.ali p.i.bob), bob t.bob)]
[i.ali $(ali t.ali, p.i.bob (sub p.i.bob p.i.ali))]
::
%|
?: =(p.i.ali (lent p.i.bob))
[i.bob $(ali t.ali, bob t.bob)]
?: (gth p.i.ali (lent p.i.bob))
[i.bob $(p.i.ali (sub p.i.ali (lent p.i.bob)), bob t.bob)]
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
:: ~ :: here, alice is good for a while, but not for the whole
== :: length of bob's changes
::
%|
?- -.i.bob
%|
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
::
%&
?: =(p.i.bob (lent p.i.ali))
[i.ali $(ali t.ali, bob t.bob)]
?: (gth p.i.bob (lent p.i.ali))
[i.ali $(ali t.ali, p.i.bob (sub p.i.bob (lent p.i.ali)))]
=/ [fic=(unce cord) ali=(urge cord) bob=(urge cord)]
(resolve ali bob)
[fic $(ali ali, bob bob)]
==
==
::
++ annotate :: annotate conflict
|= $: ali=(list @t)
bob=(list @t)
bas=(list @t)
==
^- (list @t)
%- zing
^- (list (list @t))
%- flop
^- (list (list @t))
:- :_ ~
%^ cat 3 '<<<<<<<<<<<<'
%^ cat 3 ' '
%^ cat 3 `@t`(scot %p bos)
%^ cat 3 '/'
bod
:- bob
:- ~['------------']
:- bas
:- ~['++++++++++++']
:- ali
:- :_ ~
%^ cat 3 '>>>>>>>>>>>>'
%^ cat 3 ' '
%^ cat 3 `@t`(scot %p als)
%^ cat 3 '/'
ald
~
::
++ clean :: clean
|= wig=(urge cord)
^- (urge cord)
?~ wig ~
?~ t.wig wig
?: ?=(%& -.i.wig)
?: ?=(%& -.i.t.wig)
$(wig [[%& (add p.i.wig p.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
?: ?=(%| -.i.t.wig)
$(wig [[%| (welp p.i.wig p.i.t.wig) (welp q.i.wig q.i.t.wig)] t.t.wig])
[i.wig $(wig t.wig)]
::
++ resolve
|= [ali=(urge cord) bob=(urge cord)]
^- [fic=[%| p=(list cord) q=(list cord)] ali=(urge cord) bob=(urge cord)]
=- [[%| bac (annotate alc boc bac)] ali bob]
|- ^- $: $: bac=(list cord)
alc=(list cord)
boc=(list cord)
==
ali=(urge cord)
bob=(urge cord)
==
?~ ali [[~ ~ ~] ali bob]
?~ bob [[~ ~ ~] ali bob]
?- -.i.ali
%&
?- -.i.bob
%& [[~ ~ ~] ali bob] :: no conflict
%|
=+ lob=(lent p.i.bob)
?: =(lob p.i.ali)
[[p.i.bob p.i.bob q.i.bob] t.ali t.bob]
?: (lth lob p.i.ali)
[[p.i.bob p.i.bob q.i.bob] [[%& (sub p.i.ali lob)] t.ali] t.bob]
=+ wat=(scag (sub lob p.i.ali) p.i.bob)
=+ ^= res
%= $
ali t.ali
bob [[%| (scag (sub lob p.i.ali) p.i.bob) ~] t.bob]
==
:* :* (welp bac.res wat)
(welp alc.res wat)
(welp boc.res q.i.bob)
==
ali.res
bob.res
==
==
::
%|
?- -.i.bob
%&
=+ loa=(lent p.i.ali)
?: =(loa p.i.bob)
[[p.i.ali q.i.ali p.i.ali] t.ali t.bob]
?: (lth loa p.i.bob)
[[p.i.ali q.i.ali p.i.ali] t.ali [[%& (sub p.i.bob loa)] t.bob]]
=+ wat=(slag (sub loa p.i.bob) p.i.ali)
=+ ^= res
%= $
ali [[%| (scag (sub loa p.i.bob) p.i.ali) ~] t.ali]
bob t.bob
==
:* :* (welp bac.res wat)
(welp alc.res q.i.ali)
(welp boc.res wat)
==
ali.res
bob.res
==
::
%|
=+ loa=(lent p.i.ali)
=+ lob=(lent p.i.bob)
?: =(loa lob)
[[p.i.ali q.i.ali q.i.bob] t.ali t.bob]
=+ ^= res
?: (gth loa lob)
$(ali [[%| (scag (sub loa lob) p.i.ali) ~] t.ali], bob t.bob)
~& [%scagging loa=loa pibob=p.i.bob slag=(scag loa p.i.bob)]
$(ali t.ali, bob [[%| (scag (sub lob loa) p.i.bob) ~] t.bob])
:* :* (welp bac.res ?:((gth loa lob) p.i.bob p.i.ali))
(welp alc.res q.i.ali)
(welp boc.res q.i.bob)
==
ali.res
bob.res
==
==
==
--
--
--
@@ -0,0 +1,12 @@
|_ dat=octs
++ grow
|%
++ mime [/font/woff dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -0,0 +1,12 @@
|_ dat=octs
++ grow
|%
++ mime [/font/woff2 dat]
--
++ grab
|%
++ mime |=([=mite =octs] octs)
++ noun octs
--
++ grad %mime
--
@@ -0,0 +1,5 @@
::
:::: /hoon/xhtml/mar
::
/= xhtml /mar/html
xhtml
@@ -0,0 +1,82 @@
|%
::
+$ version
[major=@ud minor=@ud patch=@ud]
::
+$ glob (map path mime)
::
+$ url cord
:: $glob-location: How to retrieve a glob
::
+$ glob-reference
[hash=@uvH location=glob-location]
::
+$ glob-location
$% [%http =url]
[%ames =ship]
==
:: $href: Where a tile links to
::
+$ href
$% [%glob base=term =glob-reference]
[%site =path]
==
:: $chad: State of a docket
::
+$ chad
$~ [%install ~]
$% :: Done
[%glob =glob]
[%site ~]
:: Waiting
[%install ~]
[%suspend glob=(unit glob)]
:: Error
[%hung err=cord]
==
::
:: $charge: A realized $docket
::
+$ charge
$: =docket
=chad
==
::
:: $clause: A key and value, as part of a docket
::
:: Only used to parse $docket
::
+$ clause
$% [%title title=@t]
[%info info=@t]
[%color color=@ux]
[%glob-http url=cord hash=@uvH]
[%glob-ames =ship hash=@uvH]
[%image =url]
[%site =path]
[%base base=term]
[%version =version]
[%website website=url]
[%license license=cord]
==
::
:: $docket: A description of JS bundles for a desk
::
+$ docket
$: %1
title=@t
info=@t
color=@ux
=href
image=(unit url)
=version
website=url
license=cord
==
::
+$ charge-update
$% [%initial initial=(map desk charge)]
[%add-charge =desk =charge]
[%del-charge =desk]
==
--
@@ -0,0 +1 @@
[%zuse 412]
@@ -0,0 +1,20 @@
#!/bin/sh
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
set -u
echo "Using ETH RPC endpoint ${CERC_ETH_RPC_ENDPOINT}"
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_ETH_RPC_ENDPOINT|${CERC_ETH_RPC_ENDPOINT}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/local.toml
echo "Running job-runner..."
DEBUG=vulcanize:* exec node --enable-source-maps dist/job-runner.js
@@ -0,0 +1,20 @@
#!/bin/sh
set -e
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
set -u
echo "Using ETH RPC endpoint ${CERC_ETH_RPC_ENDPOINT}"
# Read in the config template TOML file and modify it
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
sed -E "s|REPLACE_WITH_CERC_ETH_RPC_ENDPOINT|${CERC_ETH_RPC_ENDPOINT}| ")
# Write the modified content to a new file
echo "$WATCHER_CONFIG" > environments/local.toml
echo "Running server..."
DEBUG=vulcanize:* exec node --enable-source-maps dist/server.js
@@ -0,0 +1,98 @@
[server]
host = "0.0.0.0"
port = 3008
kind = "active"
gqlPath = "/"
# Checkpointing state.
checkpointing = true
# Checkpoint interval in number of blocks.
checkpointInterval = 2000
# Enable state creation
# CAUTION: Disable only if state creation is not desired or can be filled subsequently
enableState = false
subgraphPath = "./subgraph-build"
# Interval to restart wasm instance periodically
wasmRestartBlocksInterval = 20
# Interval in number of blocks at which to clear entities cache.
clearEntitiesCacheInterval = 1000
# Max block range for which to return events in eventsInRange GQL query.
# Use -1 for skipping check on block range.
maxEventsBlockRange = 1000
# Flag to specify whether RPC endpoint supports block hash as block tag parameter
rpcSupportsBlockHashParam = false
# GQL cache settings
[server.gqlCache]
enabled = true
# Max in-memory cache size (in bytes) (default 8 MB)
# maxCacheSize
# GQL cache-control max-age settings (in seconds)
maxAge = 15
timeTravelMaxAge = 86400 # 1 day
[metrics]
host = "0.0.0.0"
port = 9000
[metrics.gql]
port = 9001
[database]
type = "postgres"
host = "ajna-watcher-db"
port = 5432
database = "ajna-watcher"
username = "vdbm"
password = "password"
synchronize = true
logging = false
[upstream]
[upstream.ethServer]
rpcProviderEndpoint = "REPLACE_WITH_CERC_ETH_RPC_ENDPOINT"
# Boolean flag to specify if rpc-eth-client should be used for RPC endpoint instead of ipld-eth-client (ipld-eth-server GQL client)
rpcClient = true
# Boolean flag to specify if rpcProviderEndpoint is an FEVM RPC endpoint
isFEVM = true
# Boolean flag to filter event logs by contracts
filterLogsByAddresses = true
# Boolean flag to filter event logs by topics
filterLogsByTopics = true
[upstream.cache]
name = "requests"
enabled = false
deleteOnStart = false
[jobQueue]
dbConnectionString = "postgres://vdbm:password@ajna-watcher-db/ajna-watcher-job-queue"
maxCompletionLagInSecs = 300
jobDelayInMilliSecs = 100
eventsInBatch = 50
subgraphEventsOrder = true
# Filecoin block time: https://docs.filecoin.io/basics/the-blockchain/blocks-and-tipsets#blocktime
blockDelayInMilliSecs = 30000
# Boolean to switch between modes of processing events when starting the server.
# Setting to true will fetch filtered events and required blocks in a range of blocks and then process them.
# Setting to false will fetch blocks consecutively with its events and then process them (Behaviour is followed in realtime processing near head).
useBlockRanges = true
# Block range in which logs are fetched during historical blocks processing
historicalLogsBlockRange = 2000
# Max block range of historical processing after which it waits for completion of events processing
# If set to -1 historical processing does not wait for events processing and completes till latest canonical block
historicalMaxFetchAhead = 10000
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build the mars-v2 image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
docker build -t cerc/mars-v2:local -f ${CERC_REPO_BASE_DIR}/mars-v2-frontend/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/mars-v2-frontend
@@ -26,6 +26,8 @@ RUN \
&& su ${USERNAME} -c "umask 0002 && npm install -g eslint" \
# Install semver
&& su ${USERNAME} -c "umask 0002 && npm install -g semver" \
# Install pnpm
&& su ${USERNAME} -c "umask 0002 && npm install -g pnpm" \
&& npm cache clean --force > /dev/null 2>&1
# [Optional] Uncomment this section to install additional OS packages.
@@ -35,6 +37,9 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# [Optional] Uncomment if you want to install more global node modules
# RUN su node -c "npm install -g <your-package-list-here>"
# We do this to get a yq binary from the published container, for the correct architecture we're building here
COPY --from=docker.io/mikefarah/yq:latest /usr/bin/yq /usr/local/bin/yq
# Expose port for http
EXPOSE 80
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/nextjs-base:local}
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
rc=$?
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
if [ $rc -ne 0 ]; then
echo "BUILD FAILED" 1>&2
exit $rc
fi
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
cat <<EOF
#################################################################
@@ -10,10 +10,12 @@ TRG_DIR="${3:-.next-r}"
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=npm
else
if [ -f "pnpm-lock.yaml" ]; then
CERC_BUILD_TOOL=pnpm
elif [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=yarn
else
CERC_BUILD_TOOL=npm
fi
fi
@@ -9,7 +9,9 @@ CERC_MIN_NEXTVER=13.4.2
CERC_NEXT_VERSION="${CERC_NEXT_VERSION:-keep}"
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "yarn.lock" ]; then
if [ -f "pnpm-lock.yaml" ]; then
CERC_BUILD_TOOL=pnpm
elif [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=yarn
else
CERC_BUILD_TOOL=npm
@@ -113,7 +115,7 @@ if [ "$CERC_NEXT_VERSION" != "keep" ] && [ "$CUR_NEXT_VERSION" != "$CERC_NEXT_VE
mv package.json.$$ package.json
fi
$CERC_BUILD_TOOL install || exit 1
time $CERC_BUILD_TOOL install || exit 1
CUR_NEXT_VERSION=`jq -r '.version' node_modules/next/package.json`
@@ -136,9 +138,9 @@ to use for the build with:
EOF
cat package.json | jq ".dependencies.next = \"^$CERC_MIN_NEXTVER\"" > package.json.$$
mv package.json.$$ package.json
$CERC_BUILD_TOOL install || exit 1
time $CERC_BUILD_TOOL install || exit 1
fi
$CERC_BUILD_TOOL run cerc_compile || exit 1
time $CERC_BUILD_TOOL run cerc_compile || exit 1
exit 0
@@ -16,7 +16,9 @@ trap ctrl_c INT
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "yarn.lock" ] && [ ! -f "package-lock.json" ]; then
if [ -f "pnpm-lock.yaml" ]; then
CERC_BUILD_TOOL=pnpm
elif [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=yarn
else
CERC_BUILD_TOOL=npm
@@ -0,0 +1,6 @@
FROM cerc/snowballtools-base-backend-base:local
WORKDIR /app/packages/backend
COPY run.sh .
ENTRYPOINT ["./run.sh"]
@@ -0,0 +1,26 @@
FROM ubuntu:22.04 as builder
RUN apt update && \
apt install -y --no-install-recommends --no-install-suggests \
ca-certificates curl gnupg
# Node
ARG NODE_MAJOR=20
RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
apt update && apt install -y nodejs
# npm setup
RUN npm config set @cerc-io:registry https://git.vdb.to/api/packages/cerc-io/npm/ && npm install -g yarn
COPY . /app/
WORKDIR /app/
RUN find . -name 'node_modules' | xargs -n1 rm -rf
RUN yarn && yarn build --ignore frontend
FROM cerc/webapp-base:local
COPY --from=builder /app /app
WORKDIR /app/packages/backend
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# Build cerc/webapp-deployer-backend
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/snowballtools-base-backend-base:local ${build_command_args} -f ${SCRIPT_DIR}/Dockerfile-base ${CERC_REPO_BASE_DIR}/snowballtools-base
docker build -t cerc/snowballtools-base-backend:local ${build_command_args} ${SCRIPT_DIR}
@@ -0,0 +1,19 @@
#!/bin/bash
LACONIC_HOSTED_CONFIG_FILE=${LACONIC_HOSTED_CONFIG_FILE}
if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
if [ -f "/config/laconic-hosted-config.yml" ]; then
LACONIC_HOSTED_CONFIG_FILE="/config/laconic-hosted-config.yml"
elif [ -f "/config/config.yml" ]; then
LACONIC_HOSTED_CONFIG_FILE="/config/config.yml"
fi
fi
if [ -f "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE "`pwd`/dist"
fi
/scripts/apply-runtime-env.sh "`pwd`/dist"
yarn start
@@ -39,6 +39,15 @@ fi
if [ -n "$CERC_TEST_PARAM_2" ]; then
echo "Test-param-2: ${CERC_TEST_PARAM_2}"
fi
if [ -n "$CERC_TEST_PARAM_3" ]; then
echo "Test-param-3: ${CERC_TEST_PARAM_3}"
fi
if [ -n "$CERC_TEST_PARAM_4" ]; then
echo "Test-param-4: ${CERC_TEST_PARAM_4}"
fi
if [ -n "$CERC_TEST_PARAM_5" ]; then
echo "Test-param-5: ${CERC_TEST_PARAM_5}"
fi
if [ -d "/config" ]; then
echo "/config: EXISTS"
@@ -0,0 +1,12 @@
from node:20.9.0
# RUN apk --update --no-cache add git make alpine-sdk bash
WORKDIR /app
# Copy the Vega monorepo fork into `/app`
COPY . .
RUN yarn
# RUN ./docker/prepare-dist.sh
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Build the vega-interface image
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build \
-t cerc/vega-interface:local \
-f ${SCRIPT_DIR}/Dockerfile \
${build_command_args} \
${CERC_REPO_BASE_DIR}/vega-frontend-monorepo
@@ -0,0 +1,10 @@
FROM node:18.17.1-alpine3.18
RUN apk --update --no-cache add git python3 alpine-sdk bash curl jq
WORKDIR /app
COPY . .
RUN echo "Installing dependencies and building ajna-watcher-ts" && \
yarn && yarn build
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Build cerc/watcher-ajna
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
# See: https://stackoverflow.com/a/246128/1701505
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/watcher-ajna:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/ajna-watcher-ts
@@ -1,6 +1,6 @@
# Originally from: https://github.com/devcontainers/images/blob/main/src/javascript-node/.devcontainer/Dockerfile
# [Choice] Node.js version (use -bullseye variants on local arm64/Apple Silicon): 18, 16, 14, 18-bullseye, 16-bullseye, 14-bullseye, 18-buster, 16-buster, 14-buster
ARG VARIANT=20-bullseye
ARG VARIANT=20-bullseye-slim
FROM node:${VARIANT}
ARG USERNAME=node
@@ -24,6 +24,10 @@ RUN \
&& su ${USERNAME} -c "npm config -g set prefix ${NPM_GLOBAL}" \
# Install eslint
&& su ${USERNAME} -c "umask 0002 && npm install -g eslint" \
# Install semver
&& su ${USERNAME} -c "umask 0002 && npm install -g semver" \
# Install pnpm
&& su ${USERNAME} -c "umask 0002 && npm install -g pnpm" \
&& npm cache clean --force > /dev/null 2>&1
# [Optional] Uncomment this section to install additional OS packages.
@@ -43,7 +47,6 @@ COPY scripts /scripts
# RUN su node -c "npm install -g <your-package-list-here>"
RUN mkdir -p /config
COPY ./config.yml /config
# Install simple web server for now (use nginx perhaps later)
RUN yarn global add http-server
@@ -1,11 +1,12 @@
FROM cerc/webapp-base:local as builder
ARG CERC_BUILD_TOOL
ARG CERC_BUILD_OUTPUT_DIR
WORKDIR /app
COPY . .
RUN rm -rf node_modules build .next*
RUN /scripts/build-app.sh /app build /data
RUN rm -rf node_modules build dist .next*
RUN /scripts/build-app.sh /app /data
FROM cerc/webapp-base:local
COPY --from=builder /data /data
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/webapp-base:local}
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
rc=$?
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
if [ $rc -ne 0 ]; then
echo "BUILD FAILED" 1>&2
exit $rc
fi
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
cat <<EOF
#################################################################
@@ -1 +0,0 @@
# Put config here.
@@ -7,27 +7,46 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
fi
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
WORK_DIR="${1:-/app}"
OUTPUT_DIR="${2:-build}"
DEST_DIR="${3:-/data}"
CERC_BUILD_OUTPUT_DIR="${CERC_BUILD_OUTPUT_DIR}"
if [ -f "${WORK_DIR}/package.json" ]; then
WORK_DIR="${1:-/app}"
DEST_DIR="${2:-/data}"
if [ -f "${WORK_DIR}/build-webapp.sh" ]; then
echo "Building webapp with ${WORK_DIR}/build-webapp.sh ..."
cd "${WORK_DIR}" || exit 1
rm -rf "${DEST_DIR}"
./build-webapp.sh "${DEST_DIR}" || exit 1
elif [ -f "${WORK_DIR}/package.json" ]; then
echo "Building node-based webapp ..."
cd "${WORK_DIR}" || exit 1
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "yarn.lock" ]; then
if [ -f "pnpm-lock.yaml" ]; then
CERC_BUILD_TOOL=pnpm
elif [ -f "yarn.lock" ]; then
CERC_BUILD_TOOL=yarn
else
CERC_BUILD_TOOL=npm
fi
fi
$CERC_BUILD_TOOL install || exit 1
$CERC_BUILD_TOOL build || exit 1
time $CERC_BUILD_TOOL install || exit 1
time $CERC_BUILD_TOOL build || exit 1
rm -rf "${DEST_DIR}"
mv "${WORK_DIR}/${OUTPUT_DIR}" "${DEST_DIR}"
if [ -z "${CERC_BUILD_OUTPUT_DIR}" ]; then
if [ -d "${WORK_DIR}/dist" ]; then
CERC_BUILD_OUTPUT_DIR="${WORK_DIR}/dist"
elif [ -d "${WORK_DIR}/build" ]; then
CERC_BUILD_OUTPUT_DIR="${WORK_DIR}/build"
else
echo "ERROR: Unable to locate build output. Set with --extra-build-args \"--build-arg CERC_BUILD_OUTPUT_DIR=path\"" 1>&2
exit 1
fi
fi
mv "${CERC_BUILD_OUTPUT_DIR}" "${DEST_DIR}"
else
echo "Copying static app ..."
mv "${WORK_DIR}" "${DEST_DIR}"
@@ -3,13 +3,40 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
CERC_LISTEN_PORT=${CERC_LISTEN_PORT:-80}
CERC_WEBAPP_FILES_DIR="${CERC_WEBAPP_FILES_DIR:-/data}"
CERC_ENABLE_CORS="${CERC_ENABLE_CORS:-false}"
CERC_SINGLE_PAGE_APP="${CERC_SINGLE_PAGE_APP}"
if [ -z "${CERC_SINGLE_PAGE_APP}" ]; then
if [ 1 -eq $(find "${CERC_WEBAPP_FILES_DIR}" -name '*.html' | wc -l) ] && [ -d "${CERC_WEBAPP_FILES_DIR}/static" ]; then
CERC_SINGLE_PAGE_APP=true
else
CERC_SINGLE_PAGE_APP=false
fi
fi
if [ "true" == "$CERC_ENABLE_CORS" ]; then
CERC_HTTP_EXTRA_ARGS="$CERC_HTTP_EXTRA_ARGS --cors"
fi
/scripts/apply-webapp-config.sh /config/config.yml ${CERC_WEBAPP_FILES_DIR}
if [ "true" == "$CERC_SINGLE_PAGE_APP" ]; then
# Create a catchall redirect back to /
CERC_HTTP_EXTRA_ARGS="$CERC_HTTP_EXTRA_ARGS --proxy http://localhost:${CERC_LISTEN_PORT}?"
fi
LACONIC_HOSTED_CONFIG_FILE=${LACONIC_HOSTED_CONFIG_FILE}
if [ -z "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
if [ -f "/config/laconic-hosted-config.yml" ]; then
LACONIC_HOSTED_CONFIG_FILE="/config/laconic-hosted-config.yml"
elif [ -f "/config/config.yml" ]; then
LACONIC_HOSTED_CONFIG_FILE="/config/config.yml"
fi
fi
if [ -f "${LACONIC_HOSTED_CONFIG_FILE}" ]; then
/scripts/apply-webapp-config.sh $LACONIC_HOSTED_CONFIG_FILE "${CERC_WEBAPP_FILES_DIR}"
fi
/scripts/apply-runtime-env.sh ${CERC_WEBAPP_FILES_DIR}
http-server $CERC_HTTP_EXTRA_ARGS -p ${CERC_LISTEN_PORT:-80} ${CERC_WEBAPP_FILES_DIR}
http-server $CERC_HTTP_EXTRA_ARGS -p ${CERC_LISTEN_PORT} "${CERC_WEBAPP_FILES_DIR}"
@@ -60,3 +60,4 @@ cerc/nitro-rpc-client
cerc/watcher-merkl-sushiswap-v3
cerc/watcher-sushiswap-v3
cerc/uniswap-interface
cerc/vega-interface
+1
View File
@@ -45,3 +45,4 @@ ponder
ipld-eth-server-payments
merkl-sushiswap-v3
sushiswap-v3
vega-interface
@@ -50,3 +50,4 @@ github.com/cerc-io/ponder
github.com/cerc-io/merkl-sushiswap-v3-watcher-ts
github.com/cerc-io/sushiswap-v3-watcher-ts
github.com/cerc-io/uniswap-interface
git.vdb.to/LaconicNetwork/vega-frontend-monorepo
@@ -0,0 +1,118 @@
# Ajna Watcher
## Setup
Clone required repositories:
```bash
laconic-so --stack ajna setup-repositories --git-ssh --pull
```
Build the container images:
```bash
laconic-so --stack ajna build-containers
```
## Deploy
Create a spec file for the deployment:
```bash
laconic-so --stack ajna deploy init --output ajna-spec.yml
```
### Ports
Edit `network` in the spec file to map container ports to host ports as required:
```yml
...
network:
ports:
ajna-watcher-db:
- 15432:5432
ajna-watcher-job-runner:
- 9000:9000
ajna-watcher-server:
- 3008:3008
- 9001:9001
```
### Create a deployment
Create a deployment from the spec file:
```bash
laconic-so --stack ajna deploy create --spec-file ajna-spec.yml --deployment-dir ajna-deployment
```
### Configuration
Inside deployment directory, open the `config.env` file and set following env variables:
```bash
# External Filecoin (ETH RPC) endpoint to point the watcher to
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
```
### Start the deployment
```bash
laconic-so deployment --dir ajna-deployment start
```
* To list down and monitor the running containers:
```bash
# With status
docker ps -a
# Check logs for a container
docker logs -f <CONTAINER_ID>
```
* Open the GQL playground at <http://localhost:3008/graphql>
```graphql
# Example query
query {
_meta {
block {
hash
number
timestamp
}
deployment
hasIndexingErrors
}
accounts {
id
txCount
tokensDelegated
rewardsClaimed
}
}
```
## Clean up
Stop all the ajna services running in background:
```bash
# Only stop the docker containers
laconic-so deployment --dir ajna-deployment stop
# Run 'start' to restart the deployment
```
To stop all the ajna services and also delete data:
```bash
# Stop the docker containers
laconic-so deployment --dir ajna-deployment stop --delete-volumes
# Remove deployment directory (deployment will have to be recreated for a re-run)
rm -r ajna-deployment
```
@@ -0,0 +1,9 @@
version: "1.0"
name: ajna
description: "Ajna watcher stack"
repos:
- git.vdb.to/cerc-io/ajna-watcher-ts@v0.1.1
containers:
- cerc/watcher-ajna
pods:
- watcher-ajna
@@ -0,0 +1,16 @@
# mars
On a fresh Digital Ocean droplet with Ubuntu:
```
git clone https://github.com/cerc-io/stack-orchestrator
cd stack-orchestrator
./scripts/quick-install-linux.sh
```
Read and follow the instructions output from the above output to complete installation, then:
```
laconic-so --stack mars-v2 setup-repositories
laconic-so --stack mars-v2 build-containers
laconic-so --stack mars-v2 deploy up
```
@@ -0,0 +1,8 @@
version: "0.1"
name: mars-v2
repos:
- github.com/mars-protocol/mars-v2-frontend
containers:
- cerc/mars-v2
pods:
- mars-v2
@@ -16,26 +16,55 @@ laconic-so --stack merkl-sushiswap-v3 build-containers
## Deploy
### Configuration
Create and update an env file to be used in the next step:
```bash
# External Filecoin (ETH RPC) endpoint to point the watcher
CERC_ETH_RPC_ENDPOINT=
```
### Deploy the stack
Create a spec file for the deployment:
```bash
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
laconic-so --stack merkl-sushiswap-v3 deploy init --output merkl-sushiswap-v3-spec.yml
```
### Ports
Edit `network` in the spec file to map container ports to host ports as required:
```
...
network:
ports:
merkl-sushiswap-v3-watcher-db:
- '5432'
merkl-sushiswap-v3-watcher-job-runner:
- 9002:9000
merkl-sushiswap-v3-watcher-server:
- 127.0.0.1:3007:3008
- 9003:9001
```
### Create a deployment
Create a deployment from the spec file:
```bash
laconic-so --stack merkl-sushiswap-v3 deploy create --spec-file merkl-sushiswap-v3-spec.yml --deployment-dir merkl-sushiswap-v3-deployment
```
### Configuration
Inside deployment directory, open the `config.env` file and set following env variables:
```bash
# External Filecoin (ETH RPC) endpoint to point the watcher to
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
```
### Start the deployment
```bash
laconic-so deployment --dir merkl-sushiswap-v3-deployment start
```
* To list down and monitor the running containers:
```bash
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 ps
# With status
docker ps -a
@@ -46,6 +75,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
* Open the GQL playground at http://localhost:3007/graphql
```graphql
# Example query
{
_meta {
block {
@@ -54,7 +84,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
}
hasIndexingErrors
}
factories {
id
poolCount
@@ -64,18 +94,21 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
## Clean up
Stop all the services running in background:
Stop all the merkl-sushiswap-v3 services running in background:
```bash
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 down
# Only stop the docker containers
laconic-so deployment --dir merkl-sushiswap-v3-deployment stop
# Run 'start' to restart the deployment
```
Clear volumes created by this stack:
To stop all the merkl-sushiswap-v3 services and also delete data:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=merkl_sushiswap_v3"
# Stop the docker containers
laconic-so deployment --dir merkl-sushiswap-v3-deployment stop --delete-volumes
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=merkl_sushiswap_v3")
# Remove deployment directory (deployment will have to be recreated for a re-run)
rm -r merkl-sushiswap-v3-deployment
```
@@ -2,7 +2,7 @@ version: "1.0"
name: merkl-sushiswap-v3
description: "SushiSwap v3 watcher stack"
repos:
- github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.6
- github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.7
containers:
- cerc/watcher-merkl-sushiswap-v3
pods:
@@ -1,7 +1,7 @@
version: "0.1"
name: monitoring
repos:
- github.com/cerc-io/watcher-ts@v0.2.79
- github.com/cerc-io/watcher-ts@v0.2.81
containers:
- cerc/watcher-ts
pods:
@@ -0,0 +1,10 @@
version: "1.0"
name: snowballtools-base-backend
description: "snowballtools-base-backend"
repos:
- github.com/snowball-tools/snowballtools-base
containers:
- cerc/webapp-base
- cerc/snowballtools-base-backend
pods:
- snowballtools-base-backend
@@ -55,7 +55,7 @@ ports:
Create deployment:
```bash
laconic-so deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
laconic-so --stack sushiswap-subgraph deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
```
## Start the stack
@@ -16,26 +16,55 @@ laconic-so --stack sushiswap-v3 build-containers
## Deploy
### Configuration
Create and update an env file to be used in the next step:
```bash
# External Filecoin (ETH RPC) endpoint to point the watcher
CERC_ETH_RPC_ENDPOINT=
```
### Deploy the stack
Create a spec file for the deployment:
```bash
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
laconic-so --stack sushiswap-v3 deploy init --output sushiswap-v3-spec.yml
```
### Ports
Edit `network` in the spec file to map container ports to host ports as required:
```
...
network:
ports:
sushiswap-v3-watcher-db:
- '5432'
sushiswap-v3-watcher-job-runner:
- 9000:9000
sushiswap-v3-watcher-server:
- 127.0.0.1:3008:3008
- 9001:9001
```
### Create a deployment
Create a deployment from the spec file:
```bash
laconic-so --stack sushiswap-v3 deploy create --spec-file sushiswap-v3-spec.yml --deployment-dir sushiswap-v3-deployment
```
### Configuration
Inside deployment directory, open the `config.env` file and set following env variables:
```bash
# External Filecoin (ETH RPC) endpoint to point the watcher to
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
```
### Start the deployment
```bash
laconic-so deployment --dir sushiswap-v3-deployment start
```
* To list down and monitor the running containers:
```bash
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 ps
# With status
docker ps -a
@@ -43,20 +72,43 @@ laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_T
docker logs -f <CONTAINER_ID>
```
* Open the GQL playground at http://localhost:3008/graphql
```graphql
# Example query
{
_meta {
block {
number
timestamp
}
hasIndexingErrors
}
factories {
id
poolCount
}
}
```
## Clean up
Stop all the services running in background:
Stop all the sushiswap-v3 services running in background:
```bash
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 down
# Only stop the docker containers
laconic-so deployment --dir sushiswap-v3-deployment stop
# Run 'start' to restart the deployment
```
Clear volumes created by this stack:
To stop all the sushiswap-v3 services and also delete data:
```bash
# List all relevant volumes
docker volume ls -q --filter "name=sushiswap_v3"
# Stop the docker containers
laconic-so deployment --dir sushiswap-v3-deployment stop --delete-volumes
# Remove all the listed volumes
docker volume rm $(docker volume ls -q --filter "name=sushiswap_v3")
# Remove deployment directory (deployment will have to be recreated for a re-run)
rm -r sushiswap-v3-deployment
```
@@ -2,7 +2,7 @@ version: "1.0"
name: sushiswap-v3
description: "SushiSwap v3 watcher stack"
repos:
- github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.6
- github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.7
containers:
- cerc/watcher-sushiswap-v3
pods:
@@ -33,23 +33,27 @@ laconic-so --stack uniswap-urbit-app deploy init --output uniswap-urbit-app-spec
### Ports
Edit `network` in spec file to map container ports to same ports in host:
Edit `uniswap-urbit-app-spec.yml` such that it looks like:
```yml
...
stack: uniswap-urbit-app
deploy-to: compose
network:
ports:
urbit-fake-ship:
- '8080:80'
proxy-server:
- '4000:4000'
ipfs:
- '8081:8080'
- '5001:5001'
...
- '4001'
- '8081:8080'
- 0.0.0.0:5001:5001
volumes:
urbit_app_builds: ./data/urbit_app_builds
urbit_data: ./data/urbit_data
ipfs-import: ./data/ipfs-import
ipfs-data: ./data/ipfs-data
```
Note: Skip the `ipfs` ports if need to use an externally running IPFS node
Note: Skip the `ipfs` ports if using an externally running IPFS node, set via `config.env`, below.
### Data volumes
@@ -80,7 +84,8 @@ Inside the deployment directory, open the file `config.env` and set the followin
# Set this to GQL proxy server endpoint for uniswap app
# (Eg. http://localhost:4000/v1/graphql - in case stack is being run locally with proxy enabled)
# (Eg. https://abc.xyz.com/v1/graphql - in case https://abc.xyz.com is pointed to the proxy endpoint)
CERC_UNISWAP_GQL=http://localhost:4000/v1/graphql
CERC_UNISWAP_GQL=/apps/uniswap-proxy/v1/graphql
CERC_UNISWAP_API=/apps/uniswap-proxy/v2
# Optional
@@ -2,12 +2,9 @@ version: "0.1"
name: uniswap-urbit-app
repos:
- github.com/cerc-io/uniswap-interface@laconic # TODO: Use release
- github.com/cerc-io/watcher-ts@v0.2.78
containers:
- cerc/uniswap-interface
- cerc/watcher-ts
pods:
- uniswap-interface
- proxy-server
- fixturenet-urbit
- kubo

Some files were not shown because too many files have changed in this diff Show More