Compare commits

..
Author SHA1 Message Date
A. F. DudleyandClaude Opus 4.5 811bbd9db4 Add TODO.md with planned features and refactoring
Lint Checks / Run linter (push) Successful in 4m1s
- Update stack command for continuous deployment workflow
- Separate deployer from CLI
- Separate stacks from orchestrator repo

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 10:43:12 -05:00
A. F. DudleyandClaude Opus 4.5 8d9682eb47 Use caddy ingress class instead of nginx in cluster_info.py
Lint Checks / Run linter (push) Successful in 14s
The ingress annotation was still set to nginx class even though we're now
using Caddy as the ingress controller. Caddy won't pick up ingresses
annotated with the nginx class.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 03:41:35 -05:00
A. F. DudleyandClaude Opus 4.5 638435873c Add port 443 mapping for kind clusters with Caddy ingress
Lint Checks / Run linter (push) Successful in 14s
Caddy provides automatic HTTPS with Let's Encrypt, but needs port 443
mapped from the kind container to the host. Previously only port 80 was
mapped.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 03:35:03 -05:00
A. F. DudleyandClaude Opus 4.5 97a85359ff Fix helpers.py to use Caddy ingress instead of nginx
Lint Checks / Run linter (push) Successful in 14s
The helm-charts-with-caddy branch had the Caddy manifest file but was still
using nginx in the code. This change:

- Switch install_ingress_for_kind() to use ingress-caddy-kind-deploy.yaml
- Update wait_for_ingress_in_kind() to watch caddy-system namespace
- Use correct label selector for Caddy ingress controller pods

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 03:22:07 -05:00
A. F. DudleyandClaude Opus 4.5 ffa00767d4 Add extra_args support to deploy create command
Lint Checks / Run linter (push) Successful in 13s
- Add @click.argument for generic args passthrough to stack commands
- Keep explicit --network-dir and --initial-peers options
- Add DeploymentContext.get_compose_file() helper
- Add DeploymentContext.modify_yaml() helper for stack commands
- Update init() to use absolute paths

This allows stack-specific create commands to receive arbitrary
arguments via: laconic-so deploy create ... -- --custom-arg value

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 03:06:45 -05:00
A. F. DudleyandClaude Opus 4.5 86462c940f Fix high-memlock spec to include complete OCI runtime config
Lint Checks / Run linter (push) Successful in 14s
The base_runtime_spec for containerd requires a complete OCI spec,
not just the rlimits section. The minimal spec was causing runc to
fail with "open /proc/self/fd: no such file or directory" because
essential mounts and namespaces were missing.

This commit uses kind's default cri-base.json as the base and adds
the rlimits configuration on top. The spec includes all necessary
mounts, namespaces, capabilities, and kind-specific hooks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 02:12:11 -05:00
A. F. DudleyandClaude Opus 4.5 87db167d7f Add RuntimeClass support for unlimited RLIMIT_MEMLOCK
Lint Checks / Run linter (push) Successful in 14s
The previous approach of mounting cri-base.json into kind nodes failed
because we didn't tell containerd to use it via containerdConfigPatches.

RuntimeClass allows different stacks to have different rlimit profiles,
which is essential since kind only supports one cluster per host and
multiple stacks share the same cluster.

Changes:
- Add containerdConfigPatches to kind-config.yml to define runtime handlers
- Create RuntimeClass resources after cluster creation
- Add runtimeClassName to pod specs based on stack's security settings
- Rename cri-base.json to high-memlock-spec.json for clarity
- Add get_runtime_class() method to Spec that auto-derives from
  unlimited-memlock setting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 01:58:38 -05:00
A. F. DudleyandClaude Opus 4.5 dd856af2d3 Fix pyright type errors across codebase
- Add pyrightconfig.json for pyright 1.1.408 TOML parsing workaround
- Add NoReturn annotations to fatal() functions for proper type narrowing
- Add None checks and assertions after require=True get_record() calls
- Fix AttrDict class with __getattr__ for dynamic attribute access
- Add type annotations and casts for Kubernetes client objects
- Store compose config as DockerDeployer instance attributes
- Filter None values from dotenv and environment mappings
- Use hasattr/getattr patterns for optional container attributes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 01:10:36 -05:00
A. F. DudleyandClaude Opus 4.5 cd3d908d0d Apply pre-commit linting fixes
- Format code with black (line length 88)
- Fix E501 line length errors by breaking long strings and comments
- Fix F841 unused variable (removed unused 'quiet' variable)
- Configure pyright to disable common type issues in existing codebase
  (reportGeneralTypeIssues, reportOptionalMemberAccess, etc.)
- All pre-commit hooks now pass

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 20:58:31 -05:00
A. F. DudleyandClaude Opus 4.5 03f9acf869 Add unlimited-memlock support for Kind clusters
Lint Checks / Run linter (push) Successful in 14s
Add spec.yml option `security.unlimited-memlock` that configures
RLIMIT_MEMLOCK to unlimited for Kind cluster pods. This is needed
for workloads like Solana validators that require large amounts of
locked memory for memory-mapped files during snapshot decompression.

When enabled, generates a cri-base.json file with rlimits and mounts
it into the Kind node to override the default containerd runtime spec.

Also includes flake8 line-length fixes for affected files.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 20:20:19 -05:00
A. F. DudleyandClaude Opus 4.5 ba1aad9fa6 Add black, pyright, yamllint to pre-commit hooks
- Add black formatter (rev 23.12.1)
- Add pyright type checker (rev v1.1.345)
- Add yamllint with relaxed mode (rev v1.35.1)
- Update flake8 args: max-line-length=88, extend-ignore=E203,W503,E402
- Remove ansible-lint from dev dependencies (no ansible files in repo)
- Sync pyproject.toml flake8 config with pre-commit

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 20:04:15 -05:00
A. F. Dudley dc36a6564a Fix misleading error message in load_images_into_kind
Lint Checks / Run linter (push) Successful in 14s
2026-01-21 19:32:53 -05:00
A. F. Dudley c5c3fc1618 Retrigger test-container-registry CI
Lint Checks / Run linter (push) Successful in 14s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 1m36s
2026-01-21 19:28:29 -05:00
A. F. DudleyandClaude Opus 4.5 2e384b7179 Trigger test-container-registry CI
Lint Checks / Run linter (push) Successful in 14s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 2m33s
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 19:12:05 -05:00
A. F. DudleyandClaude Opus 4.5 b708836aa9 Add flake8 to pre-commit hooks
Lint Checks / Run linter (push) Successful in 14s
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 19:05:12 -05:00
A. F. DudleyandClaude Opus 4.5 d8da9b6515 Add missing get_kind_cluster function to helpers.py
Lint Checks / Run linter (push) Successful in 13s
Fixes ImportError in k8s_command.py that was causing CI failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 19:04:46 -05:00
A. F. DudleyandClaude Opus 4.5 5a1399f2b2 Apply pre-commit linting fixes
Lint Checks / Run linter (push) Successful in 14s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (push) Failing after 31s
Database Test / Run database hosting test on kind/k8s (push) Failing after 31s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 36s
Fix trailing whitespace and end-of-file issues across codebase.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 23:16:44 -05:00
A. F. DudleyandClaude Opus 4.5 89db6e1e92 Add Caddy ingress and k8s cluster management features
- Add Caddy ingress controller manifest for kind deployments
- Add k8s cluster list command for kind cluster management
- Add k8s_command import and registration in deploy.py
- Fix network section merge to preserve http-proxy settings
- Increase default container resources (4 CPUs, 8GB memory)
- Add UDP protocol support for K8s port definitions
- Add command/entrypoint support for K8s deployments
- Implement docker-compose variable expansion for K8s
- Set ConfigMap defaultMode to 0755 for executable scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 23:14:22 -05:00
A. F. DudleyandClaude Opus 4.5 9bd59f29d9 Add CLAUDE.md, pre-commit config, and pyproject.toml
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 22:40:59 -05:00
prathamesh 8afae1904b Add support for running jobs from a stack (#975)
Lint Checks / Run linter (push) Successful in 30s
Part of https://plan.wireit.in/deepstack/browse/VUL-265/

Reviewed-on: #975
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2025-12-04 06:13:28 +00:00
prathamesh 7acabb0743 Add support for generating Helm charts when creating a deployment (#974)
Lint Checks / Run linter (push) Successful in 29s
Part of https://plan.wireit.in/deepstack/browse/VUL-265/

- Added a flag `--helm-chart` to `deploy create` command
- Uses Kompose CLI wrapper to generate a helm chart from compose files in a stack
- To be handled in a follow on PR(s):
  - Templatize generated charts and generate a `values.yml` file with defaults

Reviewed-on: #974
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2025-11-27 06:43:07 +00:00
roysc 34f3b719e4 Path is not a context manager in python 3.13 (#971)
Publish / Build and publish (push) Successful in 15m24s
Deploy Test / Run deploy test suite (push) Successful in 17m26s
Webapp Test / Run webapp test suite (push) Successful in 19m19s
Smoke Test / Run basic test suite (push) Successful in 21m22s
Lint Checks / Run linter (push) Failing after 11s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m22s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 4m33s
Database Test / Run database hosting test on kind/k8s (push) Successful in 4m47s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 2m3s
External Stack Test / Run external stack test suite (push) Successful in 2m33s
Reviewed-on: #971
Reviewed-by: rachmaninovquar <rachmaninovquar@noreply.git.vdb.to>
2025-10-16 17:55:44 +00:00
nabarun 0e814bd4da Support custom build and run for NextJS app (#969)
Lint Checks / Run linter (push) Successful in 17s
Publish / Build and publish (push) Successful in 31s
Smoke Test / Run basic test suite (push) Successful in 1m54s
Webapp Test / Run webapp test suite (push) Successful in 2m2s
Deploy Test / Run deploy test suite (push) Successful in 2m48s
Database Test / Run database hosting test on kind/k8s (push) Successful in 7m53s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 4m43s
External Stack Test / Run external stack test suite (push) Successful in 5m11s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 16m15s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m21s
Part of https://www.notion.so/Laconic-Mainnet-Plan-1eca6b22d47280569cd0d1e6d711d949
For deploying frontend https://git.vdb.to/NasSharaf/laconic-deployer-frontend

Reviewed-on: #969
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2025-08-20 12:26:11 +00:00
prathamesh 873a6d472c Update webapp deployment flow for supporting custom domains (#963)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m7s
Smoke Test / Run basic test suite (push) Successful in 4m1s
Webapp Test / Run webapp test suite (push) Successful in 4m44s
Deploy Test / Run deploy test suite (push) Successful in 4m44s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m33s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 4m32s
Database Test / Run database hosting test on kind/k8s (push) Successful in 5m22s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 2m18s
External Stack Test / Run external stack test suite (push) Successful in 2m32s
Part of https://www.notion.so/Support-custom-domains-in-deploy-laconic-com-18aa6b22d4728067a44ae27090c02ce5 and cerc-io/snowballtools-base#47

- Set `value` (IP address for `A` resource) in the DNS records
- Update `deploy-webapp-from-registry` command with an option to pass k8s cluster IP address (only required with fqdn policy `allow`)

Reviewed-on: #963
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2025-02-04 13:26:31 +00:00
prathamesh 39df4683ac Allow payment reuse for same app LRN (#961)
Lint Checks / Run linter (push) Successful in 33s
Publish / Build and publish (push) Successful in 1m12s
Deploy Test / Run deploy test suite (push) Successful in 4m54s
Smoke Test / Run basic test suite (push) Successful in 3m52s
Webapp Test / Run webapp test suite (push) Successful in 4m38s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 8m21s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m53s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m58s
External Stack Test / Run external stack test suite (push) Successful in 4m51s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m22s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75)

Reviewed-on: #961
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-29 11:30:03 +00:00
prathamesh 23ca4c4341 Allow payment reuse for application redeployment (#960)
Lint Checks / Run linter (push) Successful in 39s
Publish / Build and publish (push) Successful in 1m10s
Smoke Test / Run basic test suite (push) Successful in 3m54s
Webapp Test / Run webapp test suite (push) Successful in 4m40s
Deploy Test / Run deploy test suite (push) Successful in 4m51s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75)

Reviewed-on: #960
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-29 06:51:48 +00:00
prathamesh f64ef5d128 Use file existence for registry mutex (#959)
Lint Checks / Run linter (push) Successful in 1m1s
Publish / Build and publish (push) Successful in 1m27s
Webapp Test / Run webapp test suite (push) Successful in 4m59s
Smoke Test / Run basic test suite (push) Successful in 4m10s
Deploy Test / Run deploy test suite (push) Successful in 5m33s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75)

Reviewed-on: #959
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-29 04:05:35 +00:00
prathamesh 5f8e809b2d Add mutex lock file path to registry CLI wrapper class (#958)
Lint Checks / Run linter (push) Successful in 33s
Publish / Build and publish (push) Successful in 1m24s
Deploy Test / Run deploy test suite (push) Successful in 4m53s
Webapp Test / Run webapp test suite (push) Successful in 4m39s
Smoke Test / Run basic test suite (push) Successful in 3m58s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m38s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 6m59s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m59s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m49s
External Stack Test / Run external stack test suite (push) Successful in 4m38s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75)
Follows #957

Reviewed-on: #958
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-28 06:03:13 +00:00
prathamesh 4a7df2de33 Use a mutex for registry CLI txs in webapp deployment commands (#957)
Lint Checks / Run linter (push) Successful in 37s
Publish / Build and publish (push) Successful in 1m19s
Webapp Test / Run webapp test suite (push) Successful in 4m45s
Smoke Test / Run basic test suite (push) Successful in 4m16s
Deploy Test / Run deploy test suite (push) Successful in 4m58s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m17s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m33s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m41s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m36s
External Stack Test / Run external stack test suite (push) Successful in 4m43s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75) and #948

- Add a registry mutex decorator over tx methods in `LaconicRegistryClient` wrapper
- Required to allow multiple process to run webapp deployment tooling without running into account sequence errors when sending laconicd txs

Reviewed-on: #957
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-25 08:40:54 +00:00
prathamesh 0c47da42fe Integrate SP auctions in webapp deployment flow (#950)
Lint Checks / Run linter (push) Successful in 39s
Publish / Build and publish (push) Successful in 1m15s
Smoke Test / Run basic test suite (push) Successful in 4m16s
Webapp Test / Run webapp test suite (push) Successful in 4m47s
Deploy Test / Run deploy test suite (push) Successful in 5m2s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m41s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m51s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m30s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m54s
External Stack Test / Run external stack test suite (push) Successful in 4m52s
Part of [Service provider auctions for web deployments](https://www.notion.so/Service-provider-auctions-for-web-deployments-104a6b22d47280dbad51d28aa3a91d75) and #948

- Add a command `publish-deployment-auction` to create and publish an app deployment auction
- Add a command `handle-deployment-auction` to handle auctions on deployer side
- Update `request-webapp-deployment` command to allow using an auction id in deployment requests
- Update `deploy-webapp-from-registry` command to handle deployment requests with auction
- Add a command `request-webapp-undeployment` to request an application undeployment

Reviewed-on: #950
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-10-21 07:02:06 +00:00
nabarun e290c62aca Pin shiv version to resolve failing CI (#956)
Lint Checks / Run linter (push) Successful in 41s
Publish / Build and publish (push) Successful in 1m15s
Webapp Test / Run webapp test suite (push) Successful in 5m25s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (push) Successful in 9m4s
Smoke Test / Run basic test suite (push) Successful in 6m11s
Deploy Test / Run deploy test suite (push) Successful in 7m5s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m32s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m35s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m42s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m20s
External Stack Test / Run external stack test suite (push) Successful in 4m30s
Part of #955
- Using `shiv` version 1.0.6

Reviewed-on: #956
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-10-17 06:37:32 +00:00
telackey f1fdc48aaa Work around this bug: https://github.com/python/cpython/pull/14064 (#941)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m30s
Smoke Test / Run basic test suite (push) Successful in 4m18s
Webapp Test / Run webapp test suite (push) Successful in 5m2s
Deploy Test / Run deploy test suite (push) Successful in 5m20s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m7s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Failing after 30s
Database Test / Run database hosting test on kind/k8s (push) Failing after 32s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 34s
External Stack Test / Run external stack test suite (push) Failing after 31s
Otherwise we sometimes see errors like:

```
cerc-webapp-deployer:   File "/root/.shiv/laconic-so_0f937aa98c2748ef9af8585d6f441dbc01546ace0d6660cbb159d1e5040aeddf/site-packages/stack_orchestrator/deploy/webapp/deploy_webapp_from_registry.py", line 671, in command
cerc-webapp-deployer:     shutil.rmtree(tempdir)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 725, in rmtree
cerc-webapp-deployer:     _rmtree_safe_fd(fd, path, onerror)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 681, in _rmtree_safe_fd
cerc-webapp-deployer:     onerror(os.unlink, fullname, sys.exc_info())
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 679, in _rmtree_safe_fd
cerc-webapp-deployer:     os.unlink(entry.name, dir_fd=topfd)
cerc-webapp-deployer: FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
```

Reviewed-on: #941
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 23:17:13 +00:00
telackey a54072de6c Add --config-ref flag. (#939)
Lint Checks / Run linter (push) Successful in 43s
Publish / Build and publish (push) Successful in 1m15s
Smoke Test / Run basic test suite (push) Successful in 3m55s
Webapp Test / Run webapp test suite (push) Successful in 4m38s
Deploy Test / Run deploy test suite (push) Successful in 4m53s
Database Test / Run database hosting test on kind/k8s (push) Successful in 8m46s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m24s
External Stack Test / Run external stack test suite (push) Successful in 4m32s
Add a flag to re-use config.

Reviewed-on: #939
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 17:32:52 +00:00
telackey fa21ff2627 Support uploaded config, add 'publish-webapp-deployer' and 'request-webapp-deployment' commands (#938)
Lint Checks / Run linter (push) Successful in 36s
Publish / Build and publish (push) Successful in 1m6s
Smoke Test / Run basic test suite (push) Successful in 3m53s
Webapp Test / Run webapp test suite (push) Successful in 4m33s
Deploy Test / Run deploy test suite (push) Successful in 4m39s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m10s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m25s
This adds two new commands: `publish-webapp-deployer` and `request-webapp-deployment`.

`publish-webapp-deployer` creates a `WebappDeployer` record, which provides information to requestors like the API URL, minimum required payment, payment address, and public key to use for encrypting config.

```
$ laconic-so publish-deployer-to-registry \
  --laconic-config ~/.laconic/laconic.yml \
  --api-url https://webapp-deployer-api.dev.vaasl.io \
  --public-key-file webapp-deployer-api.dev.vaasl.io.pgp.pub  \
  --lrn lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io  \
  --min-required-payment 100000
```

`request-webapp-deployment` simplifies publishing a `WebappDeploymentRequest` and can also handle automatic payment, and encryption and upload of configuration.

```
$ laconic-so request-webapp-deployment \
  --laconic-config ~/.laconic/laconic.yml \
  --deployer lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io \
  --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
  --env-file ~/yaml/hello.env \
  --make-payment auto
```

Related changes are included for the deploy/undeploy commands for decrypting and using config, using the payment address from the WebappDeployer record, etc.

Reviewed-on: #938
2024-08-27 19:55:06 +00:00
prathamesh 33d395e213 Add package registry stack instructions (#937)
Lint Checks / Run linter (push) Successful in 36s
Publish / Build and publish (push) Successful in 1m9s
Smoke Test / Run basic test suite (push) Successful in 4m0s
Webapp Test / Run webapp test suite (push) Successful in 4m33s
Deploy Test / Run deploy test suite (push) Successful in 4m55s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m59s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m42s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m52s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m37s
External Stack Test / Run external stack test suite (push) Successful in 4m46s
- The instructions to `Deploy Gitea Package Registry` from build-support [readme](https://git.vdb.to/deep-stack/stack-orchestrator/src/branch/pm-update-registry-steps/stack_orchestrator/data/stacks/build-support#2-deploy-gitea-package-registry) don't seem to be in a working state
- Updated `package-registry` stack instructions to use deployment pattern

Reviewed-on: #937
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-08-23 09:42:44 +00:00
telackey 75ff60752a Require payment for app deployment requests. (#928)
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m33s
Lint Checks / Run linter (push) Successful in 35s
Publish / Build and publish (push) Successful in 1m18s
Smoke Test / Run basic test suite (push) Successful in 3m58s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m44s
Webapp Test / Run webapp test suite (push) Successful in 4m45s
External Stack Test / Run external stack test suite (push) Successful in 4m39s
Deploy Test / Run deploy test suite (push) Successful in 5m10s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m5s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m19s
Adds three new options for deployment/undeployment:

```
    "--min-required-payment",
    help="Requests must have a minimum payment to be processed",

    "--payment-address",
    help="The address to which payments should be made.  Default is the current laconic account.",

    "--all-requests",
    help="Handle requests addressed to anyone (by default only requests to my payment address are examined).",
```

In this mode, requests should be designated for a particular address with the attribute `to` and include a `payment` attribute which is the tx hash for the payment.

The deployer will confirm the payment (to the right account, right amount, not used before, etc.) and then proceed with the deployment or undeployment.

Reviewed-on: #928
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-21 14:39:20 +00:00
dboreham 44b9709717 Use Laconic version of ping-pub (#930)
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m33s
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m23s
Smoke Test / Run basic test suite (push) Successful in 4m32s
Deploy Test / Run deploy test suite (push) Successful in 5m20s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m34s
Webapp Test / Run webapp test suite (push) Successful in 5m9s
External Stack Test / Run external stack test suite (push) Successful in 4m34s
Database Test / Run database hosting test on kind/k8s (push) Successful in 8m58s
Reviewed-on: #930
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-20 17:44:00 +00:00
dboreham e56da7dcc1 Add support for k8s pod to node affinity and taint toleration (#917)
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m12s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m24s
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m15s
Smoke Test / Run basic test suite (push) Successful in 4m40s
Webapp Test / Run webapp test suite (push) Successful in 5m5s
Deploy Test / Run deploy test suite (push) Successful in 5m42s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (push) Successful in 6m16s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m22s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m30s
External Stack Test / Run external stack test suite (push) Successful in 4m31s
Reviewed-on: #917
Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to>
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-15 20:32:58 +00:00
telackey 60d34217f8 More logging for webapp deployment (#923)
Lint Checks / Run linter (push) Successful in 37s
Publish / Build and publish (push) Successful in 1m11s
Smoke Test / Run basic test suite (push) Successful in 3m57s
Webapp Test / Run webapp test suite (push) Successful in 4m31s
Deploy Test / Run deploy test suite (push) Successful in 4m50s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m14s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m37s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m28s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 4m22s
External Stack Test / Run external stack test suite (push) Successful in 5m5s
```
cerc-webapp-deployer: ############ DEPLOY #############
cerc-webapp-deployer: 2024-08-15 02:13:08.321991 -  - 0:00:00.000031 (step): Discovering deployment requests...
cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRequest
cerc-webapp-deployer: 2024-08-15 02:13:08.815428 -  - 0:00:00.493420 (step): Loading known requests from /srv/deployments/autodeploy.state...
cerc-webapp-deployer: 2024-08-15 02:13:08.815626 -  - 0:00:00.000158 (step): BEGIN: Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi
cerc-webapp-deployer: 2024-08-15 02:13:08.815645 -  - 0:00:00.000008 (step): Skipping request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815653 -  - 0:00:00.000005 (step): DONE Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815664 -  - 0:00:00.000005 (step): BEGIN: Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue
cerc-webapp-deployer: 2024-08-15 02:13:08.815674 -  - 0:00:00.000006 (step): Skipping request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815684 -  - 0:00:00.000004 (step): DONE Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815692 -  - 0:00:00.000005 (step): BEGIN: Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne
cerc-webapp-deployer: 2024-08-15 02:13:08.815705 -  - 0:00:00.000007 (step): Skipping request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815714 -  - 0:00:00.000005 (step): DONE Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815724 -  - 0:00:00.000004 (step): BEGIN: Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua
cerc-webapp-deployer: 2024-08-15 02:13:08.815733 -  - 0:00:00.000005 (step): Skipping request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815743 -  - 0:00:00.000005 (step): DONE Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815751 -  - 0:00:00.000004 (step): BEGIN: Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy
cerc-webapp-deployer: 2024-08-15 02:13:08.815761 -  - 0:00:00.000006 (step): Skipping request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815770 -  - 0:00:00.000005 (step): DONE Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815779 -  - 0:00:00.000005 (step): BEGIN: Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy
cerc-webapp-deployer: 2024-08-15 02:13:08.815791 -  - 0:00:00.000007 (step): Skipping request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy, we've already seen it.
cerc-webapp-deployer: 2024-08-15 02:13:08.815800 -  - 0:00:00.000004 (step): DONE Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy with result SKIP.
cerc-webapp-deployer: 2024-08-15 02:13:08.815808 -  - 0:00:00.000004 (step): Discovering existing app deployments...
cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRecord
cerc-webapp-deployer: 2024-08-15 02:13:09.330655 -  - 0:00:00.514858 (step): Discovering deployment removal and cancellation requests...
cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRemovalRequest
cerc-webapp-deployer: 2024-08-15 02:13:09.825145 -  - 0:00:00.494460 (step): Found 0 unsatisfied request(s) to process.
cerc-webapp-deployer: ############ DEPLOY SUCCESS #############
```

Reviewed-on: #923
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-15 02:57:47 +00:00
telackey 952389abb0 Add option to recreate deployments rather than update them. (#920)
Lint Checks / Run linter (push) Successful in 48s
Publish / Build and publish (push) Successful in 1m21s
Smoke Test / Run basic test suite (push) Successful in 4m46s
Webapp Test / Run webapp test suite (push) Successful in 5m16s
Deploy Test / Run deploy test suite (push) Successful in 5m41s
cherry-pick from #912

Reviewed-on: #920
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
2024-08-14 20:14:40 +00:00
telackey 5c275aa622 Defensively handle errors examining app requests. (#922)
Lint Checks / Run linter (push) Successful in 34s
Publish / Build and publish (push) Successful in 1m7s
Smoke Test / Run basic test suite (push) Successful in 4m0s
Webapp Test / Run webapp test suite (push) Successful in 4m42s
Deploy Test / Run deploy test suite (push) Successful in 4m58s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m2s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m44s
External Stack Test / Run external stack test suite (push) Successful in 4m26s
Related to cerc-io/webapp-deployment-status-api#10

There are two issues in that.  One is that the output probably changed recently, whether in the client or server, where no matching record is found by ID (Note this is specific to `laconic record get --id <v>` and does not seem to apply to the similar command to retrieve a record by name, `laconic name resolve <n>`).

Rather than returning `[]` it is now returning `[ null ]`.  This cause us to think there *was* an application record found, and we attempt to treat the `null` entry like an Application object.  That's fixed by filtering out null responses, which is a good precaution for the deployer, though I think it makes sense to ask whether this new behavior by the client/server is correct.  Seems suspicious.

The other issue is that all the defensive checks we had in place to deal with broken/bad AppDeploymentRequests were around the _build_.  This error was coming much earlier, merely when parsing and examining the request to see if it needed to be handled at all.

I have now added similar defensive error handling around that portion of the code.

Reviewed-on: #922
Reviewed-by: zramsay <zramsay@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-14 18:04:31 +00:00
telackey 8576137557 Convert port to string. (#919)
Lint Checks / Run linter (push) Successful in 40s
Publish / Build and publish (push) Successful in 1m19s
Smoke Test / Run basic test suite (push) Successful in 4m15s
Webapp Test / Run webapp test suite (push) Successful in 4m41s
Deploy Test / Run deploy test suite (push) Successful in 4m59s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m12s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m48s
The str type check doesn't work if the port is a ruamel.yaml.scalarstring.SingleQuotedScalarString or ruamel.yaml.scalarstring.DoubleQuotedScalarString

Reviewed-on: #919
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-14 00:25:35 +00:00
dboreham 65c1cdf6b1 Fix crash if port has int type in yaml (#918)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m18s
Deploy Test / Run deploy test suite (push) Successful in 4m36s
Webapp Test / Run webapp test suite (push) Successful in 4m24s
Smoke Test / Run basic test suite (push) Successful in 3m52s
Reviewed-on: #918
Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to>
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-13 20:47:09 +00:00
dboreham 265699bc38 Allow to disable kind cluster management for testing (#915)
Lint Checks / Run linter (push) Successful in 37s
Publish / Build and publish (push) Successful in 1m9s
Smoke Test / Run basic test suite (push) Successful in 4m23s
Webapp Test / Run webapp test suite (push) Successful in 4m38s
Deploy Test / Run deploy test suite (push) Successful in 5m0s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m49s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m37s
External Stack Test / Run external stack test suite (push) Successful in 4m54s
Reviewed-on: #915
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-13 17:48:14 +00:00
dboreham 4a7670a5d6 Open the json-rpc port (#916)
Lint Checks / Run linter (push) Has been cancelled
Deploy Test / Run deploy test suite (push) Has been cancelled
Publish / Build and publish (push) Has been cancelled
Webapp Test / Run webapp test suite (push) Has been cancelled
Smoke Test / Run basic test suite (push) Has been cancelled
Reviewed-on: #916
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-13 17:47:56 +00:00
telackey 6087e1cd31 Copy config under a volume for Docker (similar to a ConfigMap for K8S). (#914)
Lint Checks / Run linter (push) Successful in 42s
Publish / Build and publish (push) Successful in 1m15s
Deploy Test / Run deploy test suite (push) Successful in 4m43s
Webapp Test / Run webapp test suite (push) Successful in 4m40s
Smoke Test / Run basic test suite (push) Successful in 3m49s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m9s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m33s
External Stack Test / Run external stack test suite (push) Successful in 4m34s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m43s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m19s
This emulates the K8S ConfigMap behavior on Docker by using a regular volume.

If a directory exists under `config/` which matches a named volume, the contents will be copied to the volume on `create` (provided the destination volume is empty).  That is, rather than a ConfigMap, it is essentially a "config volume".

For example, with a compose file like:

```
version: '3.7'
services:
  caddy:
    image: cerc/caddy-ethcache:local
    restart: always
    volumes:
      - caddyconfig:/etc/caddy:ro
volumes:
  caddyconfig:
```

And a directory:

```
❯ ls stack-orchestrator/config/caddyconfig/
Caddyfile
```

After `laconic-so deploy create --spec-file caddy.yml --deployment-dir /srv/caddy` there will be:

```
❯ ls /srv/caddy/data/caddyconfig
Caddyfile
```

Mounted at `/etc/caddy`

Reviewed-on: #914
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-10 02:32:21 +00:00
telackey 1def279d26 Support multiple NodePorts, static NodePort mapping, and add 'replicas' spec option (#913)
Lint Checks / Run linter (push) Successful in 33s
Publish / Build and publish (push) Successful in 1m7s
Smoke Test / Run basic test suite (push) Successful in 3m51s
Webapp Test / Run webapp test suite (push) Successful in 4m30s
Deploy Test / Run deploy test suite (push) Successful in 4m42s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 12m52s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m27s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m35s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m54s
External Stack Test / Run external stack test suite (push) Successful in 5m19s
NodePort example:

```
network:
  ports:
    caddy:
     - 1234
     - 32020:2020
```

Replicas example:

```
replicas: 2
```

This also adds an optimization for k8s where if a directory matching the name of a configmap exists in beneath config/ in the stack, its contents will be copied into the corresponding configmap.

For example:

```
# Config files in the stack
❯ ls stack-orchestrator/config/caddyconfig
Caddyfile  Caddyfile.one-req-per-upstream-example

# ConfigMap in the spec
❯ cat foo.yml | grep config
...
configmaps:
  caddyconfig: ./configmaps/caddyconfig

# Create the deployment
❯ laconic-so --stack ~/cerc/caddy-ethcache/stack-orchestrator/stacks/caddy-ethcache deploy create --spec-file foo.yml

# The files from beneath config/<config_map_name> have been copied to the ConfigMap directory from the spec.
❯ ls deployment-001/configmaps/caddyconfig
Caddyfile  Caddyfile.one-req-per-upstream-example
```

Reviewed-on: #913
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-09 02:32:06 +00:00
dboreham 64691bd206 Merge pull request 'Allow gentx-files to be omitted' (#911) from dboreham/allow-zero-gentx into main
Lint Checks / Run linter (push) Successful in 43s
Publish / Build and publish (push) Successful in 1m21s
Deploy Test / Run deploy test suite (push) Successful in 5m20s
Webapp Test / Run webapp test suite (push) Successful in 5m0s
Smoke Test / Run basic test suite (push) Successful in 4m18s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m5s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m22s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m15s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m40s
External Stack Test / Run external stack test suite (push) Successful in 4m29s
Reviewed-on: #911
2024-08-07 20:13:40 +00:00
dboreham aef5986135 Allow gentx-files to be omitted
Lint Checks / Run linter (pull_request) Successful in 34s
Webapp Test / Run webapp test suite (pull_request) Successful in 5m3s
Smoke Test / Run basic test suite (pull_request) Successful in 4m36s
Deploy Test / Run deploy test suite (pull_request) Successful in 5m13s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 8m9s
2024-08-07 14:11:06 -06:00
dboreham 6f8f0340d3 Merge pull request 'Add stage 1 support' (#910) from dboreham/stage1-support into main
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m19s
Webapp Test / Run webapp test suite (push) Successful in 5m1s
Deploy Test / Run deploy test suite (push) Successful in 5m42s
Smoke Test / Run basic test suite (push) Successful in 4m32s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m23s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 4m9s
External Stack Test / Run external stack test suite (push) Successful in 4m52s
Reviewed-on: #910
2024-08-07 17:44:28 +00:00
dboreham 7590d6e237 Add stage 1 support
Lint Checks / Run linter (pull_request) Successful in 37s
Webapp Test / Run webapp test suite (pull_request) Successful in 6m10s
Smoke Test / Run basic test suite (pull_request) Successful in 6m9s
Deploy Test / Run deploy test suite (pull_request) Successful in 7m7s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 9m57s
2024-08-07 11:28:10 -06:00
dboreham 573f99dbbe Listen on 0.0.0.0 (#909)
Lint Checks / Run linter (push) Successful in 45s
Publish / Build and publish (push) Successful in 1m18s
Smoke Test / Run basic test suite (push) Successful in 4m8s
Webapp Test / Run webapp test suite (push) Successful in 4m46s
Deploy Test / Run deploy test suite (push) Successful in 4m59s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m41s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m42s
External Stack Test / Run external stack test suite (push) Successful in 4m30s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m30s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m58s
Reviewed-on: #909
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-02 14:06:06 +00:00
dboreham 8052c1c25e Merge pull request 'Laconicd needs to be told its currency' (#908) from dboreham/mainnet-laconic-specify-currency into main
Lint Checks / Run linter (push) Successful in 34s
Publish / Build and publish (push) Successful in 1m13s
Deploy Test / Run deploy test suite (push) Successful in 4m59s
Webapp Test / Run webapp test suite (push) Successful in 4m53s
Smoke Test / Run basic test suite (push) Successful in 4m8s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 12m49s
Reviewed-on: #908
2024-08-02 03:10:34 +00:00
dboreham a674d13493 Laconicd needs to be told its currency
Lint Checks / Run linter (pull_request) Successful in 34s
Smoke Test / Run basic test suite (pull_request) Successful in 4m23s
Webapp Test / Run webapp test suite (pull_request) Successful in 4m57s
Deploy Test / Run deploy test suite (pull_request) Successful in 5m17s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 8m7s
2024-08-01 21:09:30 -06:00
roysc 0d4f4509c8 Remove Eth fixturenet workflows (#906)
Lint Checks / Run linter (push) Successful in 41s
Publish / Build and publish (push) Successful in 1m14s
Smoke Test / Run basic test suite (push) Successful in 4m21s
Webapp Test / Run webapp test suite (push) Successful in 4m37s
Deploy Test / Run deploy test suite (push) Successful in 4m49s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m8s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m39s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m43s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m34s
External Stack Test / Run external stack test suite (push) Successful in 4m31s
Deletes the now-failing CI workflows for the old `fixturenet-eth` and `fixturenet-plugeth` stacks.

Part of #905.

Reviewed-on: #906
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
2024-08-01 02:28:05 +00:00
dboreham 5af27b1b3a Merge pull request 'Fix for sh as shell not bash' (#907) from dboreham/fix-script-for-ubuntu into main
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m31s
Smoke Test / Run basic test suite (push) Successful in 4m13s
Deploy Test / Run deploy test suite (push) Successful in 5m8s
Webapp Test / Run webapp test suite (push) Successful in 4m55s
Reviewed-on: #907
2024-07-31 20:39:37 +00:00
dboreham 6c91b87348 Fix for sh as shell not bash
Lint Checks / Run linter (pull_request) Successful in 32s
Deploy Test / Run deploy test suite (pull_request) Successful in 4m39s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 7m55s
Webapp Test / Run webapp test suite (pull_request) Successful in 4m59s
Smoke Test / Run basic test suite (pull_request) Successful in 4m30s
2024-07-31 14:30:53 -06:00
dboreham 7d18334953 Mainnet-laconic stack fixes for laconicd2 (#904)
Lint Checks / Run linter (push) Successful in 39s
Publish / Build and publish (push) Successful in 1m13s
Smoke Test / Run basic test suite (push) Successful in 4m7s
Webapp Test / Run webapp test suite (push) Successful in 4m35s
Deploy Test / Run deploy test suite (push) Successful in 4m58s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 9m58s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 10m55s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m44s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m15s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m52s
External Stack Test / Run external stack test suite (push) Successful in 4m48s
Reviewed-on: #904
2024-07-31 13:51:28 +00:00
prathamesh 79c1c5ed99 Update fixturenet-laconicd stack to use alnt denom (#902)
Lint Checks / Run linter (push) Successful in 45s
Publish / Build and publish (push) Successful in 1m10s
Deploy Test / Run deploy test suite (push) Successful in 5m26s
Smoke Test / Run basic test suite (push) Successful in 4m5s
Webapp Test / Run webapp test suite (push) Successful in 4m49s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m32s
Part of [laconicd testnet validator enrollment](https://www.notion.so/laconicd-testnet-validator-enrollment-6fc1d3cafcc64fef8c5ed3affa27c675)

Reviewed-on: #902
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-07-31 13:27:54 +00:00
zramsayandzramsay dfedd9e9ff rename laconic-sdk to registry-sdk (#897)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m21s
Deploy Test / Run deploy test suite (push) Successful in 4m53s
Smoke Test / Run basic test suite (push) Successful in 3m47s
Webapp Test / Run webapp test suite (push) Successful in 4m41s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Failing after 12m28s
Co-authored-by: zramsay <zach@bluecollarcoding.ca>
Reviewed-on: #897
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Co-authored-by: zramsay <zramsay@noreply.git.vdb.to>
Co-committed-by: zramsay <zramsay@noreply.git.vdb.to>
2024-07-31 08:01:02 +00:00
telackey f64683f03b Use a more flexible mechanism to inject config into next.config.js for runtime env. (#901)
Lint Checks / Run linter (push) Successful in 32s
Publish / Build and publish (push) Successful in 1m11s
Smoke Test / Run basic test suite (push) Successful in 4m1s
Webapp Test / Run webapp test suite (push) Successful in 4m39s
Deploy Test / Run deploy test suite (push) Successful in 4m44s
Instead of attempting to rewriting the nextConfig file directly, inject a helper function to add the config we need.

Reviewed-on: #901
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-07-31 03:22:23 +00:00
telackey 913c3a8557 Back to v2 now that we have a working webapp deployer build again. (#896)
Lint Checks / Run linter (push) Successful in 46s
Publish / Build and publish (push) Successful in 1m27s
Smoke Test / Run basic test suite (push) Successful in 4m2s
Deploy Test / Run deploy test suite (push) Successful in 5m10s
Webapp Test / Run webapp test suite (push) Successful in 4m40s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 12m47s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 10m31s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 11m40s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 8m40s
Database Test / Run database hosting test on kind/k8s (push) Successful in 11m4s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 4m7s
External Stack Test / Run external stack test suite (push) Successful in 5m25s
Reviewed-on: #896
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-07-27 18:59:42 +00:00
telackey 2f5b0cdd13 Revert recent laconicd deployment changes to restore production webapp deployer function. (#895)
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m26s
Smoke Test / Run basic test suite (push) Successful in 4m31s
Webapp Test / Run webapp test suite (push) Successful in 4m55s
Deploy Test / Run deploy test suite (push) Successful in 5m22s
Database Test / Run database hosting test on kind/k8s (push) Successful in 8m42s
Reviewed-on: #895
2024-07-27 17:04:03 +00:00
telackey 432bd4113d 881: Support next.config.mjs (#890)
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m24s
Deploy Test / Run deploy test suite (push) Successful in 5m49s
Webapp Test / Run webapp test suite (push) Successful in 5m7s
Smoke Test / Run basic test suite (push) Successful in 4m44s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m3s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m41s
External Stack Test / Run external stack test suite (push) Successful in 4m42s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m37s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 7m28s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 11m11s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 8m4s
Reviewed-on: #890
2024-07-25 16:47:17 +00:00
prathamesh b26698b756 Update fixturenet-laconicd stack for renaming changes (#891)
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m31s
Deploy Test / Run deploy test suite (push) Successful in 5m54s
Webapp Test / Run webapp test suite (push) Successful in 5m10s
Smoke Test / Run basic test suite (push) Successful in 4m34s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m16s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m47s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 3h7m0s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 3h6m59s
Part of [Rename laconic2d to laconicd](https://www.notion.so/Rename-laconic2d-to-laconicd-9028d0c020d24d1288e92ebcb773d7a7)
Handles #882, #889

Reviewed-on: #891
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-07-25 08:50:15 +00:00
telackey 01deac78c4 880: Support new compile/generate syntax for next >=14.2.0 (#886)
Lint Checks / Run linter (push) Successful in 34s
Publish / Build and publish (push) Successful in 1m17s
Deploy Test / Run deploy test suite (push) Successful in 4m34s
Smoke Test / Run basic test suite (push) Successful in 3m50s
Webapp Test / Run webapp test suite (push) Successful in 4m39s
Fix for #880 to support the next compile/generate syntax.

Reviewed-on: #886
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-07-24 20:20:16 +00:00
177 changed files with 7116 additions and 1816 deletions
@@ -1,57 +0,0 @@
name: Fixturenet-Eth-Plugeth-Test
on:
push:
branches: '*'
paths:
- '!**'
- '.gitea/workflows/triggers/fixturenet-eth-plugeth-test'
schedule: # Note: coordinate with other tests to not overload runners at the same time of day
- cron: '2 14 * * *'
jobs:
test:
name: "Run an Ethereum plugeth fixturenet test"
runs-on: ubuntu-latest
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
# At present the stock setup-python action fails on Linux/aarch64
# Conditional steps below workaroud this by using deadsnakes for that case only
- name: "Install Python for ARM on Linux"
if: ${{ runner.arch == 'arm64' && runner.os == 'Linux' }}
uses: deadsnakes/action@v3.0.1
with:
python-version: '3.8'
- name: "Install Python cases other than ARM on Linux"
if: ${{ ! (runner.arch == 'arm64' && runner.os == 'Linux') }}
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: "Run fixturenet-eth tests"
run: ./tests/fixturenet-eth-plugeth/run-test.sh
- name: Notify Vulcanize Slack on CI failure
if: ${{ always() && github.ref_name == 'main' }}
uses: ravsamhq/notify-slack-action@v2
with:
status: ${{ job.status }}
notify_when: 'failure'
env:
SLACK_WEBHOOK_URL: ${{ secrets.VULCANIZE_SLACK_CI_ALERTS }}
- name: Notify DeepStack Slack on CI failure
if: ${{ always() && github.ref_name == 'main' }}
uses: ravsamhq/notify-slack-action@v2
with:
status: ${{ job.status }}
notify_when: 'failure'
env:
SLACK_WEBHOOK_URL: ${{ secrets.DEEPSTACK_SLACK_CI_ALERTS }}
-55
View File
@@ -1,55 +0,0 @@
name: Fixturenet-Eth-Test
on:
push:
branches: '*'
paths:
- '!**'
- '.gitea/workflows/triggers/fixturenet-eth-test'
jobs:
test:
name: "Run an Ethereum fixturenet test"
runs-on: ubuntu-latest
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
# At present the stock setup-python action fails on Linux/aarch64
# Conditional steps below workaroud this by using deadsnakes for that case only
- name: "Install Python for ARM on Linux"
if: ${{ runner.arch == 'arm64' && runner.os == 'Linux' }}
uses: deadsnakes/action@v3.0.1
with:
python-version: '3.8'
- name: "Install Python cases other than ARM on Linux"
if: ${{ ! (runner.arch == 'arm64' && runner.os == 'Linux') }}
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: "Run fixturenet-eth tests"
run: ./tests/fixturenet-eth/run-test.sh
- name: Notify Vulcanize Slack on CI failure
if: ${{ always() && github.ref_name == 'main' }}
uses: ravsamhq/notify-slack-action@v2
with:
status: ${{ job.status }}
notify_when: 'failure'
env:
SLACK_WEBHOOK_URL: ${{ secrets.VULCANIZE_SLACK_CI_ALERTS }}
- name: Notify DeepStack Slack on CI failure
if: ${{ always() && github.ref_name == 'main' }}
uses: ravsamhq/notify-slack-action@v2
with:
status: ${{ job.status }}
notify_when: 'failure'
env:
SLACK_WEBHOOK_URL: ${{ secrets.DEEPSTACK_SLACK_CI_ALERTS }}
@@ -39,7 +39,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -35,7 +35,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Build local shiv package"
id: build
run: |
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -35,7 +35,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
@@ -1,19 +1,22 @@
name: Fixturenet-Eth-Plugeth-Arm-Test
name: K8s Deployment Control Test
on:
pull_request:
branches: '*'
push:
branches: '*'
paths:
- '!**'
- '.gitea/workflows/triggers/fixturenet-eth-plugeth-arm-test'
- '.gitea/workflows/triggers/test-k8s-deployment-control'
- '.gitea/workflows/test-k8s-deployment-control.yml'
- 'tests/k8s-deployment-control/run-test.sh'
schedule: # Note: coordinate with other tests to not overload runners at the same time of day
- cron: '2 14 * * *'
- cron: '3 30 * * *'
jobs:
test:
name: "Run an Ethereum plugeth fixturenet test"
runs-on: ubuntu-latest-arm
name: "Run deployment control suite on kind/k8s"
runs-on: ubuntu-22.04
steps:
- name: "Clone project repository"
uses: actions/checkout@v3
@@ -32,13 +35,22 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
run: ./scripts/build_shiv_package.sh
- name: "Run fixturenet-eth tests"
run: ./tests/fixturenet-eth-plugeth/run-test.sh
- name: "Check cgroups version"
run: mount | grep cgroup
- name: "Install kind"
run: ./tests/scripts/install-kind.sh
- name: "Install Kubectl"
run: ./tests/scripts/install-kubectl.sh
- name: "Run k8s deployment control test"
run: |
source /opt/bash-utils/cgroup-helper.sh
join_cgroup
./tests/k8s-deployment-control/run-test.sh
- name: Notify Vulcanize Slack on CI failure
if: ${{ always() && github.ref_name == 'main' }}
uses: ravsamhq/notify-slack-action@v2
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: "Print Python version"
run: python3 --version
- name: "Install shiv"
run: pip install shiv
run: pip install shiv==1.0.6
- name: "Generate build version file"
run: ./scripts/create_build_tag_file.sh
- name: "Build local shiv package"
@@ -1,2 +0,0 @@
Change this file to trigger running the fixturenet-eth-plugeth-arm-test CI job
@@ -1,3 +0,0 @@
Change this file to trigger running the fixturenet-eth-plugeth-test CI job
trigger
trigger
@@ -1,2 +0,0 @@
Change this file to trigger running the fixturenet-eth-test CI job
@@ -7,3 +7,4 @@ Trigger
Trigger
Trigger
Trigger
Trigger
@@ -1 +1,3 @@
Change this file to trigger running the test-container-registry CI job
Change this file to trigger running the test-container-registry CI job
Triggered: 2026-01-21
Triggered: 2026-01-21 19:28:29
+1 -1
View File
@@ -1,2 +1,2 @@
Change this file to trigger running the test-database CI job
Trigger test run
Trigger test run
@@ -1,2 +1 @@
Change this file to trigger running the fixturenet-eth-test CI job
+34
View File
@@ -0,0 +1,34 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
args: ['--allow-multiple-documents']
- id: check-json
- id: check-merge-conflict
- id: check-added-large-files
- repo: https://github.com/psf/black
rev: 23.12.1
hooks:
- id: black
language_version: python3
- repo: https://github.com/PyCQA/flake8
rev: 7.1.1
hooks:
- id: flake8
args: ['--max-line-length=88', '--extend-ignore=E203,W503,E402']
- repo: https://github.com/RobertCraigie/pyright-python
rev: v1.1.345
hooks:
- id: pyright
- repo: https://github.com/adrienverge/yamllint
rev: v1.35.1
hooks:
- id: yamllint
args: [-d, relaxed]
+50
View File
@@ -0,0 +1,50 @@
# CLAUDE.md
This file provides guidance to Claude Code when working with the stack-orchestrator project.
## Some rules to follow
NEVER speculate about the cause of something
NEVER assume your hypotheses are true without evidence
ALWAYS clearly state when something is a hypothesis
ALWAYS use evidence from the systems your interacting with to support your claims and hypotheses
## Key Principles
### Development Guidelines
- **Single responsibility** - Each component has one clear purpose
- **Fail fast** - Let errors propagate, don't hide failures
- **DRY/KISS** - Minimize duplication and complexity
## Development Philosophy: Conversational Literate Programming
### Approach
This project follows principles inspired by literate programming, where development happens through explanatory conversation rather than code-first implementation.
### Core Principles
- **Documentation-First**: All changes begin with discussion of intent and reasoning
- **Narrative-Driven**: Complex systems are explained through conversational exploration
- **Justification Required**: Every coding task must have a corresponding TODO.md item explaining the "why"
- **Iterative Understanding**: Architecture and implementation evolve through dialogue
### Working Method
1. **Explore and Understand**: Read existing code to understand current state
2. **Discuss Architecture**: Workshop complex design decisions through conversation
3. **Document Intent**: Update TODO.md with clear justification before coding
4. **Explain Changes**: Each modification includes reasoning and context
5. **Maintain Narrative**: Conversations serve as living documentation of design evolution
### Implementation Guidelines
- Treat conversations as primary documentation
- Explain architectural decisions before implementing
- Use TODO.md as the "literate document" that justifies all work
- Maintain clear narrative threads across sessions
- Workshop complex ideas before coding
This approach treats the human-AI collaboration as a form of **conversational literate programming** where understanding emerges through dialogue before code implementation.
## Insights and Observations
### Design Principles
- **When something times out that doesn't mean it needs a longer timeout it means something that was expected never happened, not that we need to wait longer for it.**
- **NEVER change a timeout because you believe something truncated, you don't understand timeouts, don't edit them unless told to explicitly by user.**
+1 -1
View File
@@ -658,4 +658,4 @@
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<http://www.gnu.org/licenses/>.
<http://www.gnu.org/licenses/>.
+1 -3
View File
@@ -26,7 +26,7 @@ curl -SL https://github.com/docker/compose/releases/download/v2.11.2/docker-comp
chmod +x ~/.docker/cli-plugins/docker-compose
```
Next decide on a directory where you would like to put the stack-orchestrator program. Typically this would be
Next decide on a directory where you would like to put the stack-orchestrator program. Typically this would be
a "user" binary directory such as `~/bin` or perhaps `/usr/local/laconic` or possibly just the current working directory.
Now, having selected that directory, download the latest release from [this page](https://git.vdb.to/cerc-io/stack-orchestrator/tags) into it (we're using `~/bin` below for concreteness but edit to suit if you selected a different directory). Also be sure that the destination directory exists and is writable:
@@ -78,5 +78,3 @@ See the [CONTRIBUTING.md](/docs/CONTRIBUTING.md) for developer mode install.
## Platform Support
Native aarm64 is _not_ currently supported. x64 emulation on ARM64 macos should work (not yet tested).
+16
View File
@@ -0,0 +1,16 @@
# TODO
## Features Needed
### Update Stack Command
We need an "update stack" command in stack orchestrator and cleaner documentation regarding how to do continuous deployment with and without payments.
**Context**: Currently, `deploy init` generates a spec file and `deploy create` creates a deployment directory. The `deployment update` command (added by Thomas Lackey) only syncs env vars and restarts - it doesn't regenerate configurations. There's a gap in the workflow for updating stack configurations after initial deployment.
## Architecture Refactoring
### Separate Deployer from Stack Orchestrator CLI
The deployer logic should be decoupled from the CLI tool to allow independent development and reuse.
### Separate Stacks from Stack Orchestrator Repo
Stacks should live in their own repositories, not bundled with the orchestrator tool. This allows stacks to evolve independently and be maintained by different teams.
+1 -1
View File
@@ -51,7 +51,7 @@ $ laconic-so build-npms --include <package-name>
```
e.g.
```
$ laconic-so build-npms --include laconic-sdk
$ laconic-so build-npms --include registry-sdk
```
Build the packages for a stack:
```
+2 -2
View File
@@ -1,9 +1,9 @@
# Fetching pre-built container images
When Stack Orchestrator deploys a stack containing a suite of one or more containers it expects images for those containers to be on the local machine with a tag of the form `<image-name>:local` Images for these containers can be built from source (and optionally base container images from public registries) with the `build-containers` subcommand.
When Stack Orchestrator deploys a stack containing a suite of one or more containers it expects images for those containers to be on the local machine with a tag of the form `<image-name>:local` Images for these containers can be built from source (and optionally base container images from public registries) with the `build-containers` subcommand.
However, the task of building a large number of containers from source may consume considerable time and machine resources. This is where the `fetch-containers` subcommand steps in. It is designed to work exactly like `build-containers` but instead the images, pre-built, are fetched from an image registry then re-tagged for deployment. It can be used in place of `build-containers` for any stack provided the necessary containers, built for the local machine architecture (e.g. arm64 or x86-64) have already been published in an image registry.
## Usage
To use `fetch-containers`, provide an image registry path, a username and token/password with read access to the registry, and optionally specify `--force-local-overwrite`. If this argument is not specified, if there is already a locally built or previously fetched image for a stack container on the machine, it will not be overwritten and a warning issued.
```
$ laconic-so --stack mobymask-v3-demo fetch-containers --image-registry git.vdb.to/cerc-io --registry-username <registry-user> --registry-token <registry-token> --force-local-overwrite
```
```
+2 -2
View File
@@ -7,7 +7,7 @@ Deploy a local Gitea server, publish NPM packages to it, then use those packages
```bash
laconic-so --stack build-support build-containers
laconic-so --stack package-registry setup-repositories
laconic-so --stack package-registry build-containers
laconic-so --stack package-registry build-containers
laconic-so --stack package-registry deploy up
```
@@ -56,7 +56,7 @@ laconic-so --stack fixturenet-laconicd build-npms
Navigate to the Gitea console and switch to the `cerc-io` user then find the `Packages` tab to confirm that these two npm packages have been published:
- `@cerc-io/laconic-registry-cli`
- `@cerc-io/laconic-sdk`
- `@cerc-io/registry-sdk`
### Build and deploy fixturenet containers
+113
View File
@@ -0,0 +1,113 @@
# Helm Chart Generation
Generate Kubernetes Helm charts from stack compose files using Kompose.
## Prerequisites
Install Kompose:
```bash
# Linux
curl -L https://github.com/kubernetes/kompose/releases/download/v1.34.0/kompose-linux-amd64 -o kompose
chmod +x kompose
sudo mv kompose /usr/local/bin/
# macOS
brew install kompose
# Verify
kompose version
```
## Usage
### 1. Create spec file
```bash
laconic-so --stack <stack-name> deploy --deploy-to k8s init \
--kube-config ~/.kube/config \
--output spec.yml
```
### 2. Generate Helm chart
```bash
laconic-so --stack <stack-name> deploy create \
--spec-file spec.yml \
--deployment-dir my-deployment \
--helm-chart
```
### 3. Deploy to Kubernetes
```bash
helm install my-release my-deployment/chart
kubectl get pods -n zenith
```
## Output Structure
```bash
my-deployment/
├── spec.yml # Reference
├── stack.yml # Reference
└── chart/ # Helm chart
├── Chart.yaml
├── README.md
└── templates/
└── *.yaml
```
## Example
```bash
# Generate chart for stage1-zenithd
laconic-so --stack stage1-zenithd deploy --deploy-to k8s init \
--kube-config ~/.kube/config \
--output stage1-spec.yml
laconic-so --stack stage1-zenithd deploy create \
--spec-file stage1-spec.yml \
--deployment-dir stage1-deployment \
--helm-chart
# Deploy
helm install stage1-zenithd stage1-deployment/chart
```
## Production Deployment (TODO)
### Local Development
```bash
# Access services using port-forward
kubectl port-forward service/zenithd 26657:26657
kubectl port-forward service/nginx-api-proxy 1317:80
kubectl port-forward service/cosmos-explorer 4173:4173
```
### Production Access Options
- Option 1: Ingress + cert-manager (Recommended)
- Install ingress-nginx + cert-manager
- Point DNS to cluster LoadBalancer IP
- Auto-provisions Let's Encrypt TLS certs
- Access: `https://api.zenith.example.com`
- Option 2: Cloud LoadBalancer
- Use cloud provider's LoadBalancer service type
- Point DNS to assigned external IP
- Manual TLS cert management
- Option 3: Bare Metal (MetalLB + Ingress)
- MetalLB provides LoadBalancer IPs from local network
- Same Ingress setup as cloud
- Option 4: NodePort + External Proxy
- Expose services on 30000-32767 range
- External nginx/Caddy proxies 80/443 → NodePort
- Manual cert management
### Changes Needed
- Add Ingress template to charts
- Add TLS configuration to values.yaml
- Document cert-manager setup
- Add production deployment guide
+26
View File
@@ -0,0 +1,26 @@
# K8S Deployment Enhancements
## Controlling pod placement
The placement of pods created as part of a stack deployment can be controlled to either avoid certain nodes, or require certain nodes.
### Pod/Node Affinity
Node affinity rules applied to pods target node labels. The effect is that a pod can only be placed on a node having the specified label value. Note that other pods that do not have any node affinity rules can also be placed on those same nodes. Thus node affinity for a pod controls where that pod can be placed, but does not control where other pods are placed.
Node affinity for stack pods is specified in the deployment's `spec.yml` file as follows:
```
node-affinities:
- label: nodetype
value: typeb
```
This example denotes that the stack's pods should only be placed on nodes that have the label `nodetype` with value `typeb`.
### Node Taint Toleration
K8s nodes can be given one or more "taints". These are special fields (distinct from labels) with a name (key) and optional value.
When placing pods, the k8s scheduler will only assign a pod to a tainted node if the pod posesses a corresponding "toleration".
This is metadata associated with the pod that specifies that the pod "tolerates" a given taint.
Therefore taint toleration provides a mechanism by which only certain pods can be placed on specific nodes, and provides a complementary mechanism to node affinity.
Taint toleration for stack pods is specified in the deployment's `spec.yml` file as follows:
```
node-tolerations:
- key: nodetype
value: typeb
```
This example denotes that the stack's pods will tolerate a taint: `nodetype=typeb`
+29 -29
View File
@@ -1,9 +1,8 @@
# Running a laconicd fixturenet with console
The following tutorial explains the steps to run a laconicd fixturenet with CLI and web console that displays records in the registry. It is designed as an introduction to Stack Orchestrator and to showcase one component of the Laconic Stack. Prior to Stack Orchestrator, the following 4 repositories had to be cloned and setup manually:
The following tutorial explains the steps to run a laconicd fixturenet with CLI and web console that displays records in the registry. It is designed as an introduction to Stack Orchestrator and to showcase one component of the Laconic Stack. Prior to Stack Orchestrator, the following repositories had to be cloned and setup manually:
- https://git.vdb.to/cerc-io/laconicd
- https://git.vdb.to/cerc-io/laconic-sdk
- https://git.vdb.to/cerc-io/laconic-registry-cli
- https://git.vdb.to/cerc-io/laconic-console
@@ -11,7 +10,7 @@ Now, with Stack Orchestrator, it is a few quick commands. Additionally, the `doc
## Setup laconic-so
To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the user experience, this tutorial is focused on using a fresh Digital Ocean (DO) droplet with similar specs:
To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the user experience, this tutorial is focused on using a fresh Digital Ocean (DO) droplet with similar specs:
16 GB Memory / 8 Intel vCPUs / 160 GB Disk.
1. Login to the droplet as root (either by SSH key or password set in the DO console)
@@ -51,7 +50,7 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
1. Get the repositories
```
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd,git.vdb.to/cerc-io/laconic-sdk,git.vdb.to/cerc-io/laconic-registry-cli,git.vdb.to/cerc-io/laconic-console
laconic-so --stack fixturenet-laconic-loaded setup-repositories --include git.vdb.to/cerc-io/laconicd
```
1. Build the containers:
@@ -76,6 +75,8 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
1. Create a deployment directory for the stack:
```
laconic-so --stack fixturenet-laconic-loaded deploy init --output laconic-loaded.spec --map-ports-to-host any-same --config LACONIC_HOSTED_ENDPOINT=$BACKEND_ENDPOINT
# Update port mapping in the laconic-loaded.spec file to resolve port conflicts on host if any
```
```
laconic-so --stack fixturenet-laconic-loaded deploy create --deployment-dir laconic-loaded-deployment --spec-file laconic-loaded.spec
@@ -95,16 +96,15 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
You'll see output from `laconicd` and the block height should be >1 to confirm it is running:
```
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:29PM INF indexed block exents height=12 module=txindex server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF Timed out dur=4976.960115 height=13 module=consensus round=0 server=node step=1
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received proposal module=consensus proposal={"Type":32,"block_id":{"hash":"D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4","parts":{"hash":"22411A20B7F14CDA33244420FBDDAF24450C0628C7A06034FF22DAC3699DDCC8","total":1}},"height":13,"pol_round":-1,"round":0,"signature":"DEuqnaQmvyYbUwckttJmgKdpRu6eVm9i+9rQ1pIrV2PidkMNdWRZBLdmNghkIrUzGbW8Xd7UVJxtLRmwRASgBg==","timestamp":"2023-04-18T21:30:01.49450663Z"} server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF received complete proposal block hash=D26C088A711F912ADB97888C269F628DA33153795621967BE44DCB43C3D03CA4 height=13 module=consensus server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF finalizing commit of block hash={} height=13 module=consensus num_txs=0 root=1A8CA1AF139CCC80EC007C6321D8A63A46A793386EE2EDF9A5CA0AB2C90728B7 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF minted coins from module account amount=2059730459416582643photon from=mint module=x/bank
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF executed block height=13 module=state num_invalid_txs=0 num_valid_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF commit synced commit=436F6D6D697449447B5B363520313037203630203232372039352038352032303820313334203231392032303520313433203130372031343920313431203139203139322038362031323720362031383520323533203137362031333820313735203135392031383620323334203135382031323120313431203230342037335D3A447D
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF committed state app_hash=416B3CE35F55D086DBCD8F6B958D13C0567F06B9FDB08AAF9FBAEA9E798DCC49 height=13 module=state num_txs=0 server=node
laconic-5cd0a80c1442c3044c8b295d26426bae-laconicd-1 | 9:30PM INF indexed block exents height=13 module=txindex server=node
laconicd-1 | 6:12AM INF indexed block events height=16 module=txindex
laconicd-1 | 6:12AM INF Timed out dur=2993.893332 height=17 module=consensus round=0 step=RoundStepNewHeight
laconicd-1 | 6:12AM INF received proposal module=consensus proposal="Proposal{17/0 (E15D03C180CE607AE8340A1325A0C134DFB4E1ADD992E173C701EBD362523267:1:DF138772FEF0, -1) 6A6F3B0A42B3 @ 2024-07-25T06:12:31.952967053Z}" proposer=86970D950BC9C16F3991A52D9C6DC55BA478A7C6
laconicd-1 | 6:12AM INF received complete proposal block hash=E15D03C180CE607AE8340A1325A0C134DFB4E1ADD992E173C701EBD362523267 height=17 module=consensus
laconicd-1 | 6:12AM INF finalizing commit of block hash=E15D03C180CE607AE8340A1325A0C134DFB4E1ADD992E173C701EBD362523267 height=17 module=consensus num_txs=0 root=AF4941107DC718ED1425E77A3DC7F1154FB780B7A7DE20288DC43442203527E3
laconicd-1 | 6:12AM INF finalized block block_app_hash=26A665360BB1EE64E54F97F2A5AB7F621B33A86D9896574000C05DE63F43F788 height=17 module=state num_txs_res=0 num_val_updates=0
laconicd-1 | 6:12AM INF executed block app_hash=26A665360BB1EE64E54F97F2A5AB7F621B33A86D9896574000C05DE63F43F788 height=17 module=state
laconicd-1 | 6:12AM INF committed state block_app_hash=AF4941107DC718ED1425E77A3DC7F1154FB780B7A7DE20288DC43442203527E3 height=17 module=state
laconicd-1 | 6:12AM INF indexed block events height=17 module=txindex
```
4. Confirm operation of the registry CLI:
@@ -117,30 +117,30 @@ To avoid hiccups on Mac M1/M2 and any local machine nuances that may affect the
{
"version": "0.3.0",
"node": {
"id": "4216af2ac9f68bda33a38803fc1b5c9559312c1d",
"id": "6e072894aa1f5d9535a1127a0d7a7f8e65100a2c",
"network": "laconic_9000-1",
"moniker": "localtestnet"
},
"sync": {
"latest_block_hash": "1BDF4CB9AE2390DA65BCF997C83133C18014FCDDCAE03708488F0B56FCEEA429",
"latest_block_height": "5",
"latest_block_time": "2023-08-09 16:00:30.386903172 +0000 UTC",
"catching_up": false
"latestBlockHash": "260102C283D0411CFBA0270F7DC182650FFCA737A2F6F652A985F6065696F590",
"latestBlockHeight": "49",
"latestBlockTime": "2024-07-25 06:14:05.626744215 +0000 UTC",
"catchingUp": false
},
"validator": {
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000"
"address": "86970D950BC9C16F3991A52D9C6DC55BA478A7C6",
"votingPower": "1000000000000000"
},
"validators": [
{
"address": "651FBC700B747C76E90ACFC18CC9508C3D0905B9",
"voting_power": "1000000000000000",
"proposer_priority": "0"
"address": "86970D950BC9C16F3991A52D9C6DC55BA478A7C6",
"votingPower": "1000000000000000",
"proposerPriority": "0"
}
],
"num_peers": "0",
"numPeers": "0",
"peers": [],
"disk_usage": "292.0K"
"diskUsage": "688K"
}
```
@@ -186,13 +186,13 @@ wns
1. The following command will create a bond and publish a record:
```
laconic-so --stack fixturenet-laconic-loaded deploy exec cli ./scripts/create-demo-records.sh
laconic-so deployment --dir laconic-loaded-deployment exec cli ./scripts/create-demo-records.sh
```
You'll get an output like:
```
Balance is: 99998999999999998999600000
Balance is: 9.9999e+25
Created bond with id: dd88e8d6f9567b32b28e70552aea4419c5dd3307ebae85a284d1fe38904e301a
Published demo-record-1.yml with id: bafyreierh3xnfivexlscdwubvczmddsnf46uytyfvrbdhkjzztvsz6ruly
```
@@ -223,5 +223,5 @@ record:
- e.g,:
```
laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic registry record list"
laconic-so deployment --dir laconic-loaded-deployment exec cli "laconic registry record list"
```
-1
View File
@@ -26,4 +26,3 @@ $ ./scripts/tag_new_release.sh 1 0 17
$ ./scripts/build_shiv_package.sh
$ ./scripts/publish_shiv_package_github.sh 1 0 17
```
+2 -2
View File
@@ -4,9 +4,9 @@ Note: this page is out of date (but still useful) - it will no longer be useful
## Implementation
The orchestrator's operation is driven by files shown below.
The orchestrator's operation is driven by files shown below.
- `repository-list.txt` contains the list of git repositories;
- `repository-list.txt` contains the list of git repositories;
- `container-image-list.txt` contains the list of container image names
- `pod-list.txt` specifies the set of compose components (corresponding to individual docker-compose-xxx.yml files which may in turn specify more than one container).
- `container-build/` contains the files required to build each container image
+1 -1
View File
@@ -7,7 +7,7 @@ compilation and static page generation are separated in the `build-webapp` and `
This offers much more flexibilty than standard Next.js build methods, since any environment variables accessed
via `process.env`, whether for pages or for API, will have values drawn from their runtime deployment environment,
not their build environment.
not their build environment.
## Building
+110
View File
@@ -0,0 +1,110 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "laconic-stack-orchestrator"
version = "1.1.0"
description = "Orchestrates deployment of the Laconic stack"
readme = "README.md"
license = {text = "GNU Affero General Public License"}
authors = [
{name = "Cerc", email = "info@cerc.io"}
]
requires-python = ">=3.8"
classifiers = [
"Programming Language :: Python :: 3.8",
"Operating System :: OS Independent",
]
dependencies = [
"python-decouple>=3.8",
"python-dotenv==1.0.0",
"GitPython>=3.1.32",
"tqdm>=4.65.0",
"python-on-whales>=0.64.0",
"click>=8.1.6",
"PyYAML>=6.0.1",
"ruamel.yaml>=0.17.32",
"pydantic==1.10.9",
"tomli==2.0.1",
"validators==0.22.0",
"kubernetes>=28.1.0",
"humanfriendly>=10.0",
"python-gnupg>=0.5.2",
"requests>=2.3.2",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"black>=22.0.0",
"flake8>=5.0.0",
"pyright>=1.1.0",
"yamllint>=1.28.0",
"pre-commit>=3.0.0",
]
[project.scripts]
laconic-so = "stack_orchestrator.main:cli"
[project.urls]
Homepage = "https://git.vdb.to/cerc-io/stack-orchestrator"
[tool.setuptools.packages.find]
where = ["."]
[tool.setuptools.package-data]
"*" = ["data/**"]
[tool.black]
line-length = 88
target-version = ['py38']
[tool.flake8]
max-line-length = 88
extend-ignore = ["E203", "W503", "E402"]
[tool.pyright]
pythonVersion = "3.9"
typeCheckingMode = "basic"
reportMissingImports = "none"
reportMissingModuleSource = "none"
reportUnusedImport = "error"
include = ["stack_orchestrator/**/*.py", "tests/**/*.py"]
exclude = ["**/build/**", "**/__pycache__/**"]
[tool.mypy]
python_version = "3.8"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
markers = [
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
"e2e: marks tests as end-to-end (requires real infrastructure)",
]
addopts = [
"--cov",
"--cov-report=term-missing",
"--cov-report=html",
"--strict-markers",
]
asyncio_default_fixture_loop_scope = "function"
[tool.coverage.run]
source = ["stack_orchestrator"]
disable_warnings = ["couldnt-parse"]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"def __repr__",
"raise AssertionError",
"raise NotImplementedError",
]
+9
View File
@@ -0,0 +1,9 @@
{
"pythonVersion": "3.9",
"typeCheckingMode": "basic",
"reportMissingImports": "none",
"reportMissingModuleSource": "none",
"reportUnusedImport": "error",
"include": ["stack_orchestrator/**/*.py", "tests/**/*.py"],
"exclude": ["**/build/**", "**/__pycache__/**"]
}
+2
View File
@@ -11,3 +11,5 @@ tomli==2.0.1
validators==0.22.0
kubernetes>=28.1.0
humanfriendly>=10.0
python-gnupg>=0.5.2
requests>=2.3.2
+1 -1
View File
@@ -4,7 +4,7 @@
# https://github.com/cerc-io/github-release-api
# User must define: CERC_GH_RELEASE_SCRIPTS_DIR
# pointing to the location of that cloned repository
# e.g.
# e.g.
# cd ~/projects
# git clone https://github.com/cerc-io/github-release-api
# cd ./stack-orchestrator
+1 -1
View File
@@ -94,7 +94,7 @@ sudo apt -y install jq
# laconic-so depends on git
sudo apt -y install git
# curl used below
sudo apt -y install curl
sudo apt -y install curl
# docker repo add depends on gnupg and updated ca-certificates
sudo apt -y install ca-certificates gnupg
+1 -1
View File
@@ -3,7 +3,7 @@
# Uses this script package to tag a new release:
# User must define: CERC_GH_RELEASE_SCRIPTS_DIR
# pointing to the location of that cloned repository
# e.g.
# e.g.
# cd ~/projects
# git clone https://github.com/cerc-io/github-release-api
# cd ./stack-orchestrator
+14 -12
View File
@@ -1,5 +1,7 @@
# See https://medium.com/nerd-for-tech/how-to-build-and-distribute-a-cli-tool-with-python-537ae41d9d78
# See
# https://medium.com/nerd-for-tech/how-to-build-and-distribute-a-cli-tool-with-python-537ae41d9d78
from setuptools import setup, find_packages
with open("README.md", "r", encoding="utf-8") as fh:
long_description = fh.read()
with open("requirements.txt", "r", encoding="utf-8") as fh:
@@ -7,26 +9,26 @@ with open("requirements.txt", "r", encoding="utf-8") as fh:
with open("stack_orchestrator/data/version.txt", "r", encoding="utf-8") as fh:
version = fh.readlines()[-1].strip(" \n")
setup(
name='laconic-stack-orchestrator',
name="laconic-stack-orchestrator",
version=version,
author='Cerc',
author_email='info@cerc.io',
license='GNU Affero General Public License',
description='Orchestrates deployment of the Laconic stack',
author="Cerc",
author_email="info@cerc.io",
license="GNU Affero General Public License",
description="Orchestrates deployment of the Laconic stack",
long_description=long_description,
long_description_content_type="text/markdown",
url='https://git.vdb.to/cerc-io/stack-orchestrator',
py_modules=['stack_orchestrator'],
url="https://git.vdb.to/cerc-io/stack-orchestrator",
py_modules=["stack_orchestrator"],
packages=find_packages(),
install_requires=[requirements],
python_requires='>=3.7',
python_requires=">=3.7",
include_package_data=True,
package_data={'': ['data/**']},
package_data={"": ["data/**"]},
classifiers=[
"Programming Language :: Python :: 3.8",
"Operating System :: OS Independent",
],
entry_points={
'console_scripts': ['laconic-so=stack_orchestrator.main:cli'],
}
"console_scripts": ["laconic-so=stack_orchestrator.main:cli"],
},
)
+30 -11
View File
@@ -23,11 +23,10 @@ def get_stack(config, stack):
if stack == "package-registry":
return package_registry_stack(config, stack)
else:
return base_stack(config, stack)
return default_stack(config, stack)
class base_stack(ABC):
def __init__(self, config, stack):
self.config = config
self.stack = stack
@@ -41,15 +40,27 @@ class base_stack(ABC):
pass
class package_registry_stack(base_stack):
class default_stack(base_stack):
"""Default stack implementation for stacks without specific handling."""
def ensure_available(self):
return True
def get_url(self):
return None
class package_registry_stack(base_stack):
def ensure_available(self):
self.url = "<no registry url set>"
# Check if we were given an external registry URL
url_from_environment = os.environ.get("CERC_NPM_REGISTRY_URL")
if url_from_environment:
if self.config.verbose:
print(f"Using package registry url from CERC_NPM_REGISTRY_URL: {url_from_environment}")
print(
f"Using package registry url from CERC_NPM_REGISTRY_URL: "
f"{url_from_environment}"
)
self.url = url_from_environment
else:
# Otherwise we expect to use the local package-registry stack
@@ -62,10 +73,16 @@ class package_registry_stack(base_stack):
# TODO: get url from deploy-stack
self.url = "http://gitea.local:3000/api/packages/cerc-io/npm/"
else:
# If not, print a message about how to start it and return fail to the caller
print("ERROR: The package-registry stack is not running, and no external registry "
"specified with CERC_NPM_REGISTRY_URL")
print("ERROR: Start the local package registry with: laconic-so --stack package-registry deploy-system up")
# If not, print a message about how to start it and return fail to the
# caller
print(
"ERROR: The package-registry stack is not running, "
"and no external registry specified with CERC_NPM_REGISTRY_URL"
)
print(
"ERROR: Start the local package registry with: "
"laconic-so --stack package-registry deploy-system up"
)
return False
return True
@@ -76,7 +93,9 @@ class package_registry_stack(base_stack):
def get_npm_registry_url():
# If an auth token is not defined, we assume the default should be the cerc registry
# If an auth token is defined, we assume the local gitea should be used.
default_npm_registry_url = "http://gitea.local:3000/api/packages/cerc-io/npm/" if config(
"CERC_NPM_AUTH_TOKEN", default=None
) else "https://git.vdb.to/api/packages/cerc-io/npm/"
default_npm_registry_url = (
"http://gitea.local:3000/api/packages/cerc-io/npm/"
if config("CERC_NPM_AUTH_TOKEN", default=None)
else "https://git.vdb.to/api/packages/cerc-io/npm/"
)
return config("CERC_NPM_REGISTRY_URL", default=default_npm_registry_url)
+106 -44
View File
@@ -18,7 +18,8 @@
# env vars:
# CERC_REPO_BASE_DIR defaults to ~/cerc
# TODO: display the available list of containers; allow re-build of either all or specific containers
# TODO: display the available list of containers;
# allow re-build of either all or specific containers
import os
import sys
@@ -34,14 +35,17 @@ from stack_orchestrator.build.publish import publish_image
from stack_orchestrator.build.build_util import get_containers_in_scope
# TODO: find a place for this
# epilog="Config provided either in .env or settings.ini or env vars: CERC_REPO_BASE_DIR (defaults to ~/cerc)"
# epilog="Config provided either in .env or settings.ini or env vars:
# CERC_REPO_BASE_DIR (defaults to ~/cerc)"
def make_container_build_env(dev_root_path: str,
container_build_dir: str,
debug: bool,
force_rebuild: bool,
extra_build_args: str):
def make_container_build_env(
dev_root_path: str,
container_build_dir: str,
debug: bool,
force_rebuild: bool,
extra_build_args: str,
):
container_build_env = {
"CERC_NPM_REGISTRY_URL": get_npm_registry_url(),
"CERC_GO_AUTH_TOKEN": config("CERC_GO_AUTH_TOKEN", default=""),
@@ -50,11 +54,15 @@ def make_container_build_env(dev_root_path: str,
"CERC_CONTAINER_BASE_DIR": container_build_dir,
"CERC_HOST_UID": f"{os.getuid()}",
"CERC_HOST_GID": f"{os.getgid()}",
"DOCKER_BUILDKIT": config("DOCKER_BUILDKIT", default="0")
"DOCKER_BUILDKIT": config("DOCKER_BUILDKIT", default="0"),
}
container_build_env.update({"CERC_SCRIPT_DEBUG": "true"} if debug else {})
container_build_env.update({"CERC_FORCE_REBUILD": "true"} if force_rebuild else {})
container_build_env.update({"CERC_CONTAINER_EXTRA_BUILD_ARGS": extra_build_args} if extra_build_args else {})
container_build_env.update(
{"CERC_CONTAINER_EXTRA_BUILD_ARGS": extra_build_args}
if extra_build_args
else {}
)
docker_host_env = os.getenv("DOCKER_HOST")
if docker_host_env:
container_build_env.update({"DOCKER_HOST": docker_host_env})
@@ -67,12 +75,18 @@ def process_container(build_context: BuildContext) -> bool:
print(f"Building: {build_context.container}")
default_container_tag = f"{build_context.container}:local"
build_context.container_build_env.update({"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag})
build_context.container_build_env.update(
{"CERC_DEFAULT_CONTAINER_IMAGE_TAG": default_container_tag}
)
# Check if this is in an external stack
if stack_is_external(build_context.stack):
container_parent_dir = Path(build_context.stack).parent.parent.joinpath("container-build")
temp_build_dir = container_parent_dir.joinpath(build_context.container.replace("/", "-"))
container_parent_dir = Path(build_context.stack).parent.parent.joinpath(
"container-build"
)
temp_build_dir = container_parent_dir.joinpath(
build_context.container.replace("/", "-")
)
temp_build_script_filename = temp_build_dir.joinpath("build.sh")
# Now check if the container exists in the external stack.
if not temp_build_script_filename.exists():
@@ -90,21 +104,34 @@ def process_container(build_context: BuildContext) -> bool:
build_command = build_script_filename.as_posix()
else:
if opts.o.verbose:
print(f"No script file found: {build_script_filename}, using default build script")
repo_dir = build_context.container.split('/')[1]
# TODO: make this less of a hack -- should be specified in some metadata somewhere
# Check if we have a repo for this container. If not, set the context dir to the container-build subdir
print(
f"No script file found: {build_script_filename}, "
"using default build script"
)
repo_dir = build_context.container.split("/")[1]
# TODO: make this less of a hack -- should be specified in
# some metadata somewhere. Check if we have a repo for this
# container. If not, set the context dir to container-build subdir
repo_full_path = os.path.join(build_context.dev_root_path, repo_dir)
repo_dir_or_build_dir = repo_full_path if os.path.exists(repo_full_path) else build_dir
build_command = os.path.join(build_context.container_build_dir,
"default-build.sh") + f" {default_container_tag} {repo_dir_or_build_dir}"
repo_dir_or_build_dir = (
repo_full_path if os.path.exists(repo_full_path) else build_dir
)
build_command = (
os.path.join(build_context.container_build_dir, "default-build.sh")
+ f" {default_container_tag} {repo_dir_or_build_dir}"
)
if not opts.o.dry_run:
# No PATH at all causes failures with podman.
if "PATH" not in build_context.container_build_env:
build_context.container_build_env["PATH"] = os.environ["PATH"]
if opts.o.verbose:
print(f"Executing: {build_command} with environment: {build_context.container_build_env}")
build_result = subprocess.run(build_command, shell=True, env=build_context.container_build_env)
print(
f"Executing: {build_command} with environment: "
f"{build_context.container_build_env}"
)
build_result = subprocess.run(
build_command, shell=True, env=build_context.container_build_env
)
if opts.o.verbose:
print(f"Return code is: {build_result.returncode}")
if build_result.returncode != 0:
@@ -117,33 +144,61 @@ def process_container(build_context: BuildContext) -> bool:
@click.command()
@click.option('--include', help="only build these containers")
@click.option('--exclude', help="don\'t build these containers")
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
@click.option("--include", help="only build these containers")
@click.option("--exclude", help="don't build these containers")
@click.option(
"--force-rebuild",
is_flag=True,
default=False,
help="Override dependency checking -- always rebuild",
)
@click.option("--extra-build-args", help="Supply extra arguments to build")
@click.option("--publish-images", is_flag=True, default=False, help="Publish the built images in the specified image registry")
@click.option("--image-registry", help="Specify the image registry for --publish-images")
@click.option(
"--publish-images",
is_flag=True,
default=False,
help="Publish the built images in the specified image registry",
)
@click.option(
"--image-registry", help="Specify the image registry for --publish-images"
)
@click.pass_context
def command(ctx, include, exclude, force_rebuild, extra_build_args, publish_images, image_registry):
'''build the set of containers required for a complete stack'''
def command(
ctx,
include,
exclude,
force_rebuild,
extra_build_args,
publish_images,
image_registry,
):
"""build the set of containers required for a complete stack"""
local_stack = ctx.obj.local_stack
stack = ctx.obj.stack
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
# See: https://stackoverflow.com/questions/25389095/
# python-get-path-of-root-project-structure
container_build_dir = (
Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
)
if local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
print(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
dev_root_path = os.getcwd()[0 : os.getcwd().rindex("stack-orchestrator")]
print(
f"Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: "
f"{dev_root_path}"
)
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
dev_root_path = os.path.expanduser(
config("CERC_REPO_BASE_DIR", default="~/cerc")
)
if not opts.o.quiet:
print(f'Dev Root is: {dev_root_path}')
print(f"Dev Root is: {dev_root_path}")
if not os.path.isdir(dev_root_path):
print('Dev root directory doesn\'t exist, creating')
print("Dev root directory doesn't exist, creating")
if publish_images:
if not image_registry:
@@ -151,21 +206,22 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args, publish_imag
containers_in_scope = get_containers_in_scope(stack)
container_build_env = make_container_build_env(dev_root_path,
container_build_dir,
opts.o.debug,
force_rebuild,
extra_build_args)
container_build_env = make_container_build_env(
dev_root_path,
container_build_dir,
opts.o.debug,
force_rebuild,
extra_build_args,
)
for container in containers_in_scope:
if include_exclude_check(container, include, exclude):
build_context = BuildContext(
stack,
container,
container_build_dir,
container_build_env,
dev_root_path
dev_root_path,
)
result = process_container(build_context)
if result:
@@ -174,10 +230,16 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args, publish_imag
else:
print(f"Error running build for {build_context.container}")
if not opts.o.continue_on_error:
error_exit("container build failed and --continue-on-error not set, exiting")
error_exit(
"container build failed and --continue-on-error "
"not set, exiting"
)
sys.exit(1)
else:
print("****** Container Build Error, continuing because --continue-on-error is set")
print(
"****** Container Build Error, continuing because "
"--continue-on-error is set"
)
else:
if opts.o.verbose:
print(f"Excluding: {container}")
+81 -40
View File
@@ -32,14 +32,18 @@ builder_js_image_name = "cerc/builder-js:local"
@click.command()
@click.option('--include', help="only build these packages")
@click.option('--exclude', help="don\'t build these packages")
@click.option("--force-rebuild", is_flag=True, default=False,
help="Override existing target package version check -- force rebuild")
@click.option("--include", help="only build these packages")
@click.option("--exclude", help="don't build these packages")
@click.option(
"--force-rebuild",
is_flag=True,
default=False,
help="Override existing target package version check -- force rebuild",
)
@click.option("--extra-build-args", help="Supply extra arguments to build")
@click.pass_context
def command(ctx, include, exclude, force_rebuild, extra_build_args):
'''build the set of npm packages required for a complete stack'''
"""build the set of npm packages required for a complete stack"""
quiet = ctx.obj.quiet
verbose = ctx.obj.verbose
@@ -65,45 +69,54 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
sys.exit(1)
if local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
print(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
dev_root_path = os.getcwd()[0 : os.getcwd().rindex("stack-orchestrator")]
print(
f"Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: "
f"{dev_root_path}"
)
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
dev_root_path = os.path.expanduser(
config("CERC_REPO_BASE_DIR", default="~/cerc")
)
build_root_path = os.path.join(dev_root_path, "build-trees")
if verbose:
print(f'Dev Root is: {dev_root_path}')
print(f"Dev Root is: {dev_root_path}")
if not os.path.isdir(dev_root_path):
print('Dev root directory doesn\'t exist, creating')
print("Dev root directory doesn't exist, creating")
os.makedirs(dev_root_path)
if not os.path.isdir(dev_root_path):
print('Build root directory doesn\'t exist, creating')
print("Build root directory doesn't exist, creating")
os.makedirs(build_root_path)
# See: https://stackoverflow.com/a/20885799/1701505
from stack_orchestrator import data
with importlib.resources.open_text(data, "npm-package-list.txt") as package_list_file:
with importlib.resources.open_text(
data, "npm-package-list.txt"
) as package_list_file:
all_packages = package_list_file.read().splitlines()
packages_in_scope = []
if stack:
stack_config = get_parsed_stack_config(stack)
# TODO: syntax check the input here
packages_in_scope = stack_config['npms']
packages_in_scope = stack_config["npms"]
else:
packages_in_scope = all_packages
if verbose:
print(f'Packages: {packages_in_scope}')
print(f"Packages: {packages_in_scope}")
def build_package(package):
if not quiet:
print(f"Building npm package: {package}")
repo_dir = package
repo_full_path = os.path.join(dev_root_path, repo_dir)
# Copy the repo and build that to avoid propagating JS tooling file changes back into the cloned repo
# Copy the repo and build that to avoid propagating
# JS tooling file changes back into the cloned repo
repo_copy_path = os.path.join(build_root_path, repo_dir)
# First delete any old build tree
if os.path.isdir(repo_copy_path):
@@ -116,41 +129,63 @@ def command(ctx, include, exclude, force_rebuild, extra_build_args):
print(f"Copying build tree from: {repo_full_path} to: {repo_copy_path}")
if not dry_run:
copytree(repo_full_path, repo_copy_path)
build_command = ["sh", "-c", f"cd /workspace && build-npm-package-local-dependencies.sh {npm_registry_url}"]
build_command = [
"sh",
"-c",
"cd /workspace && "
f"build-npm-package-local-dependencies.sh {npm_registry_url}",
]
if not dry_run:
if verbose:
print(f"Executing: {build_command}")
# Originally we used the PEP 584 merge operator:
# envs = {"CERC_NPM_AUTH_TOKEN": npm_registry_url_token} | ({"CERC_SCRIPT_DEBUG": "true"} if debug else {})
# but that isn't available in Python 3.8 (default in Ubuntu 20) so for now we use dict.update:
envs = {"CERC_NPM_AUTH_TOKEN": npm_registry_url_token,
"LACONIC_HOSTED_CONFIG_FILE": "config-hosted.yml" # Convention used by our web app packages
}
# envs = {"CERC_NPM_AUTH_TOKEN": npm_registry_url_token} |
# ({"CERC_SCRIPT_DEBUG": "true"} if debug else {})
# but that isn't available in Python 3.8 (default in Ubuntu 20)
# so for now we use dict.update:
envs = {
"CERC_NPM_AUTH_TOKEN": npm_registry_url_token,
# Convention used by our web app packages
"LACONIC_HOSTED_CONFIG_FILE": "config-hosted.yml",
}
envs.update({"CERC_SCRIPT_DEBUG": "true"} if debug else {})
envs.update({"CERC_FORCE_REBUILD": "true"} if force_rebuild else {})
envs.update({"CERC_CONTAINER_EXTRA_BUILD_ARGS": extra_build_args} if extra_build_args else {})
envs.update(
{"CERC_CONTAINER_EXTRA_BUILD_ARGS": extra_build_args}
if extra_build_args
else {}
)
try:
docker.run(builder_js_image_name,
remove=True,
interactive=True,
tty=True,
user=f"{os.getuid()}:{os.getgid()}",
envs=envs,
# TODO: detect this host name in npm_registry_url rather than hard-wiring it
add_hosts=[("gitea.local", "host-gateway")],
volumes=[(repo_copy_path, "/workspace")],
command=build_command
)
# Note that although the docs say that build_result should contain
# the command output as a string, in reality it is always the empty string.
# Since we detect errors via catching exceptions below, we can safely ignore it here.
docker.run(
builder_js_image_name,
remove=True,
interactive=True,
tty=True,
user=f"{os.getuid()}:{os.getgid()}",
envs=envs,
# TODO: detect this host name in npm_registry_url
# rather than hard-wiring it
add_hosts=[("gitea.local", "host-gateway")],
volumes=[(repo_copy_path, "/workspace")],
command=build_command,
)
# Note that although the docs say that build_result should
# contain the command output as a string, in reality it is
# always the empty string. Since we detect errors via catching
# exceptions below, we can safely ignore it here.
except DockerException as e:
print(f"Error executing build for {package} in container:\n {e}")
if not continue_on_error:
print("FATAL Error: build failed and --continue-on-error not set, exiting")
print(
"FATAL Error: build failed and --continue-on-error "
"not set, exiting"
)
sys.exit(1)
else:
print("****** Build Error, continuing because --continue-on-error is set")
print(
"****** Build Error, continuing because "
"--continue-on-error is set"
)
else:
print("Skipped")
@@ -168,6 +203,12 @@ def _ensure_prerequisites():
# Tell the user how to build it if not
images = docker.image.list(builder_js_image_name)
if len(images) == 0:
print(f"FATAL: builder image: {builder_js_image_name} is required but was not found")
print("Please run this command to create it: laconic-so --stack build-support build-containers")
print(
f"FATAL: builder image: {builder_js_image_name} is required "
"but was not found"
)
print(
"Please run this command to create it: "
"laconic-so --stack build-support build-containers"
)
sys.exit(1)
+1 -2
View File
@@ -24,6 +24,5 @@ class BuildContext:
stack: str
container: str
container_build_dir: Path
container_build_env: Mapping[str,str]
container_build_env: Mapping[str, str]
dev_root_path: str
+6 -4
View File
@@ -20,21 +20,23 @@ from stack_orchestrator.util import get_parsed_stack_config, warn_exit
def get_containers_in_scope(stack: str):
containers_in_scope = []
if stack:
stack_config = get_parsed_stack_config(stack)
if "containers" not in stack_config or stack_config["containers"] is None:
warn_exit(f"stack {stack} does not define any containers")
containers_in_scope = stack_config['containers']
containers_in_scope = stack_config["containers"]
else:
# See: https://stackoverflow.com/a/20885799/1701505
from stack_orchestrator import data
with importlib.resources.open_text(data, "container-image-list.txt") as container_list_file:
with importlib.resources.open_text(
data, "container-image-list.txt"
) as container_list_file:
containers_in_scope = container_list_file.read().splitlines()
if opts.o.verbose:
print(f'Containers: {containers_in_scope}')
print(f"Containers: {containers_in_scope}")
if stack:
print(f"Stack: {stack}")
+35 -18
View File
@@ -18,7 +18,8 @@
# env vars:
# CERC_REPO_BASE_DIR defaults to ~/cerc
# TODO: display the available list of containers; allow re-build of either all or specific containers
# TODO: display the available list of containers;
# allow re-build of either all or specific containers
import os
import sys
@@ -32,40 +33,55 @@ from stack_orchestrator.build.build_types import BuildContext
@click.command()
@click.option('--base-container')
@click.option('--source-repo', help="directory containing the webapp to build", required=True)
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
@click.option("--base-container")
@click.option(
"--source-repo", help="directory containing the webapp to build", required=True
)
@click.option(
"--force-rebuild",
is_flag=True,
default=False,
help="Override dependency checking -- always rebuild",
)
@click.option("--extra-build-args", help="Supply extra arguments to build")
@click.option("--tag", help="Container tag (default: cerc/<app_name>:local)")
@click.pass_context
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag):
'''build the specified webapp container'''
"""build the specified webapp container"""
logger = TimedLogger()
quiet = ctx.obj.quiet
debug = ctx.obj.debug
verbose = ctx.obj.verbose
local_stack = ctx.obj.local_stack
stack = ctx.obj.stack
# See: https://stackoverflow.com/questions/25389095/python-get-path-of-root-project-structure
container_build_dir = Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
# See: https://stackoverflow.com/questions/25389095/
# python-get-path-of-root-project-structure
container_build_dir = (
Path(__file__).absolute().parent.parent.joinpath("data", "container-build")
)
if local_stack:
dev_root_path = os.getcwd()[0:os.getcwd().rindex("stack-orchestrator")]
logger.log(f'Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: {dev_root_path}')
dev_root_path = os.getcwd()[0 : os.getcwd().rindex("stack-orchestrator")]
logger.log(
f"Local stack dev_root_path (CERC_REPO_BASE_DIR) overridden to: "
f"{dev_root_path}"
)
else:
dev_root_path = os.path.expanduser(config("CERC_REPO_BASE_DIR", default="~/cerc"))
dev_root_path = os.path.expanduser(
config("CERC_REPO_BASE_DIR", default="~/cerc")
)
if verbose:
logger.log(f'Dev Root is: {dev_root_path}')
logger.log(f"Dev Root is: {dev_root_path}")
if not base_container:
base_container = determine_base_container(source_repo)
# First build the base container.
container_build_env = build_containers.make_container_build_env(dev_root_path, container_build_dir, debug,
force_rebuild, extra_build_args)
container_build_env = build_containers.make_container_build_env(
dev_root_path, container_build_dir, debug, force_rebuild, extra_build_args
)
if verbose:
logger.log(f"Building base container: {base_container}")
@@ -85,12 +101,13 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
if verbose:
logger.log(f"Base container {base_container} build finished.")
# Now build the target webapp. We use the same build script, but with a different Dockerfile and work dir.
# Now build the target webapp. We use the same build script,
# but with a different Dockerfile and work dir.
container_build_env["CERC_WEBAPP_BUILD_RUNNING"] = "true"
container_build_env["CERC_CONTAINER_BUILD_WORK_DIR"] = os.path.abspath(source_repo)
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = os.path.join(container_build_dir,
base_container.replace("/", "-"),
"Dockerfile.webapp")
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = os.path.join(
container_build_dir, base_container.replace("/", "-"), "Dockerfile.webapp"
)
if not tag:
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
tag = f"cerc/{webapp_name}:local"
+60 -23
View File
@@ -52,7 +52,8 @@ def _local_tag_for(container: str):
# See: https://docker-docs.uclv.cu/registry/spec/api/
# Emulate this:
# $ curl -u "my-username:my-token" -X GET "https://<container-registry-hostname>/v2/cerc-io/cerc/test-container/tags/list"
# $ curl -u "my-username:my-token" -X GET \
# "https://<container-registry-hostname>/v2/cerc-io/cerc/test-container/tags/list"
# {"name":"cerc-io/cerc/test-container","tags":["202402232130","202402232208"]}
def _get_tags_for_container(container: str, registry_info: RegistryInfo) -> List[str]:
# registry looks like: git.vdb.to/cerc-io
@@ -60,7 +61,9 @@ def _get_tags_for_container(container: str, registry_info: RegistryInfo) -> List
url = f"https://{registry_parts[0]}/v2/{registry_parts[1]}/{container}/tags/list"
if opts.o.debug:
print(f"Fetching tags from: {url}")
response = requests.get(url, auth=(registry_info.registry_username, registry_info.registry_token))
response = requests.get(
url, auth=(registry_info.registry_username, registry_info.registry_token)
)
if response.status_code == 200:
tag_info = response.json()
if opts.o.debug:
@@ -68,7 +71,10 @@ def _get_tags_for_container(container: str, registry_info: RegistryInfo) -> List
tags_array = tag_info["tags"]
return tags_array
else:
error_exit(f"failed to fetch tags from image registry, status code: {response.status_code}")
error_exit(
f"failed to fetch tags from image registry, "
f"status code: {response.status_code}"
)
def _find_latest(candidate_tags: List[str]):
@@ -79,9 +85,9 @@ def _find_latest(candidate_tags: List[str]):
return sorted_candidates[-1]
def _filter_for_platform(container: str,
registry_info: RegistryInfo,
tag_list: List[str]) -> List[str] :
def _filter_for_platform(
container: str, registry_info: RegistryInfo, tag_list: List[str]
) -> List[str]:
filtered_tags = []
this_machine = platform.machine()
# Translate between Python and docker platform names
@@ -98,7 +104,7 @@ def _filter_for_platform(container: str,
manifest = manifest_cmd.inspect_verbose(remote_tag)
if opts.o.debug:
print(f"manifest: {manifest}")
image_architecture = manifest["Descriptor"]["platform"]["architecture"]
image_architecture = manifest["Descriptor"]["platform"]["architecture"]
if opts.o.debug:
print(f"image_architecture: {image_architecture}")
if this_machine == image_architecture:
@@ -137,21 +143,44 @@ def _add_local_tag(remote_tag: str, registry: str, local_tag: str):
@click.command()
@click.option('--include', help="only fetch these containers")
@click.option('--exclude', help="don\'t fetch these containers")
@click.option("--force-local-overwrite", is_flag=True, default=False, help="Overwrite a locally built image, if present")
@click.option("--image-registry", required=True, help="Specify the image registry to fetch from")
@click.option("--registry-username", required=True, help="Specify the image registry username")
@click.option("--registry-token", required=True, help="Specify the image registry access token")
@click.option("--include", help="only fetch these containers")
@click.option("--exclude", help="don't fetch these containers")
@click.option(
"--force-local-overwrite",
is_flag=True,
default=False,
help="Overwrite a locally built image, if present",
)
@click.option(
"--image-registry", required=True, help="Specify the image registry to fetch from"
)
@click.option(
"--registry-username", required=True, help="Specify the image registry username"
)
@click.option(
"--registry-token", required=True, help="Specify the image registry access token"
)
@click.pass_context
def command(ctx, include, exclude, force_local_overwrite, image_registry, registry_username, registry_token):
'''EXPERIMENTAL: fetch the images for a stack from remote registry'''
def command(
ctx,
include,
exclude,
force_local_overwrite,
image_registry,
registry_username,
registry_token,
):
"""EXPERIMENTAL: fetch the images for a stack from remote registry"""
registry_info = RegistryInfo(image_registry, registry_username, registry_token)
docker = DockerClient()
if not opts.o.quiet:
print("Logging into container registry:")
docker.login(registry_info.registry, registry_info.registry_username, registry_info.registry_token)
docker.login(
registry_info.registry,
registry_info.registry_username,
registry_info.registry_token,
)
# Generate list of target containers
stack = ctx.obj.stack
containers_in_scope = get_containers_in_scope(stack)
@@ -172,19 +201,24 @@ def command(ctx, include, exclude, force_local_overwrite, image_registry, regist
print(f"Fetching: {image_to_fetch}")
_fetch_image(image_to_fetch, registry_info)
# Now check if the target container already exists exists locally already
if (_exists_locally(container)):
if _exists_locally(container):
if not opts.o.quiet:
print(f"Container image {container} already exists locally")
# if so, fail unless the user specified force-local-overwrite
if (force_local_overwrite):
if force_local_overwrite:
# In that case remove the existing :local tag
if not opts.o.quiet:
print(f"Warning: overwriting local tag from this image: {container} because "
"--force-local-overwrite was specified")
print(
f"Warning: overwriting local tag from this image: "
f"{container} because --force-local-overwrite was specified"
)
else:
if not opts.o.quiet:
print(f"Skipping local tagging for this image: {container} because that would "
"overwrite an existing :local tagged image, use --force-local-overwrite to do so.")
print(
f"Skipping local tagging for this image: {container} "
"because that would overwrite an existing :local tagged "
"image, use --force-local-overwrite to do so."
)
continue
# Tag the fetched image with the :local tag
_add_local_tag(image_to_fetch, image_registry, local_tag)
@@ -192,4 +226,7 @@ def command(ctx, include, exclude, force_local_overwrite, image_registry, regist
if opts.o.verbose:
print(f"Excluding: {container}")
if not all_containers_found:
print("Warning: couldn't find usable images for one or more containers, this stack will not deploy")
print(
"Warning: couldn't find usable images for one or more containers, "
"this stack will not deploy"
)
+8
View File
@@ -34,5 +34,13 @@ volumes_key = "volumes"
security_key = "security"
annotations_key = "annotations"
labels_key = "labels"
replicas_key = "replicas"
node_affinities_key = "node-affinities"
node_tolerations_key = "node-tolerations"
kind_config_filename = "kind-config.yml"
kube_config_filename = "kubeconfig.yml"
cri_base_filename = "cri-base.json"
unlimited_memlock_key = "unlimited-memlock"
runtime_class_key = "runtime-class"
high_memlock_runtime = "high-memlock"
high_memlock_spec_filename = "high-memlock-spec.json"
@@ -20,7 +20,7 @@ services:
depends_on:
generate-jwt:
condition: service_completed_successfully
env_file:
env_file:
- ../config/fixturenet-blast/${NETWORK:-fixturenet}.config
blast-geth:
image: blastio/blast-geth:${NETWORK:-testnet-sepolia}
@@ -51,7 +51,7 @@ services:
--nodiscover
--maxpeers=0
--rollup.disabletxpoolgossip=true
env_file:
env_file:
- ../config/fixturenet-blast/${NETWORK:-fixturenet}.config
depends_on:
geth-init:
@@ -73,7 +73,7 @@ services:
--rollup.config="/blast/rollup.json"
depends_on:
- blast-geth
env_file:
env_file:
- ../config/fixturenet-blast/${NETWORK:-fixturenet}.config
volumes:
@@ -2,10 +2,11 @@ services:
laconicd:
restart: unless-stopped
image: cerc/laconicd:local
command: ["sh", "/docker-entrypoint-scripts.d/create-fixturenet.sh"]
command: ["bash", "/docker-entrypoint-scripts.d/create-fixturenet.sh"]
environment:
TEST_AUCTION_ENABLED: ${TEST_AUCTION_ENABLED}
TEST_REGISTRY_EXPIRY: ${TEST_REGISTRY_EXPIRY}
TEST_AUCTION_ENABLED: ${TEST_AUCTION_ENABLED:-false}
TEST_REGISTRY_EXPIRY: ${TEST_REGISTRY_EXPIRY:-false}
ONBOARDING_ENABLED: ${ONBOARDING_ENABLED:-false}
volumes:
# The cosmos-sdk node's database directory:
- laconicd-data:/root/.laconicd
@@ -19,11 +20,9 @@ services:
- "26657"
- "26656"
- "9473"
- "8545"
- "8546"
- "9090"
- "9091"
- "1317"
cli:
image: cerc/laconic-registry-cli:local
volumes:
@@ -14,4 +14,3 @@ services:
- "9090"
- "9091"
- "1317"
@@ -19,7 +19,7 @@ services:
depends_on:
generate-jwt:
condition: service_completed_successfully
env_file:
env_file:
- ../config/mainnet-blast/${NETWORK:-mainnet}.config
blast-geth:
image: blastio/blast-geth:${NETWORK:-mainnet}
@@ -53,7 +53,7 @@ services:
--nodiscover
--maxpeers=0
--rollup.disabletxpoolgossip=true
env_file:
env_file:
- ../config/mainnet-blast/${NETWORK:-mainnet}.config
depends_on:
geth-init:
@@ -76,7 +76,7 @@ services:
--rollup.config="/blast/rollup.json"
depends_on:
- blast-geth
env_file:
env_file:
- ../config/mainnet-blast/${NETWORK:-mainnet}.config
volumes:
@@ -17,4 +17,3 @@ services:
- URL_NEUTRON_TEST_REST=https://rest-palvus.pion-1.ntrn.tech
- URL_NEUTRON_TEST_RPC=https://rpc-palvus.pion-1.ntrn.tech
- WALLET_CONNECT_ID=0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x0x
@@ -32,4 +32,4 @@ services:
volumes:
reth_data:
lighthouse_data:
shared_data:
shared_data:
@@ -12,7 +12,7 @@ services:
POSTGRES_INITDB_ARGS: "-E UTF8 --locale=C"
ports:
- "5432"
test-client:
image: cerc/test-database-client:local
@@ -1,2 +1,2 @@
GETH_ROLLUP_SEQUENCERHTTP=https://sequencer.s2.testblast.io
OP_NODE_P2P_BOOTNODES=enr:-J-4QM3GLUFfKMSJQuP1UvuKQe8DyovE7Eaiit0l6By4zjTodkR4V8NWXJxNmlg8t8rP-Q-wp3jVmeAOml8cjMj__ROGAYznzb_HgmlkgnY0gmlwhA-cZ_eHb3BzdGFja4X947FQAIlzZWNwMjU2azGhAiuDqvB-AsVSRmnnWr6OHfjgY8YfNclFy9p02flKzXnOg3RjcIJ2YYN1ZHCCdmE,enr:-J-4QDCVpByqQ8nFqCS9aHicqwUfXgzFDslvpEyYz19lvkHLIdtcIGp2d4q5dxHdjRNTO6HXCsnIKxUeuZSPcEbyVQCGAYznzz0RgmlkgnY0gmlwhANiQfuHb3BzdGFja4X947FQAIlzZWNwMjU2azGhAy3AtF2Jh_aPdOohg506Hjmtx-fQ1AKmu71C7PfkWAw9g3RjcIJ2YYN1ZHCCdmE
OP_NODE_P2P_BOOTNODES=enr:-J-4QM3GLUFfKMSJQuP1UvuKQe8DyovE7Eaiit0l6By4zjTodkR4V8NWXJxNmlg8t8rP-Q-wp3jVmeAOml8cjMj__ROGAYznzb_HgmlkgnY0gmlwhA-cZ_eHb3BzdGFja4X947FQAIlzZWNwMjU2azGhAiuDqvB-AsVSRmnnWr6OHfjgY8YfNclFy9p02flKzXnOg3RjcIJ2YYN1ZHCCdmE,enr:-J-4QDCVpByqQ8nFqCS9aHicqwUfXgzFDslvpEyYz19lvkHLIdtcIGp2d4q5dxHdjRNTO6HXCsnIKxUeuZSPcEbyVQCGAYznzz0RgmlkgnY0gmlwhANiQfuHb3BzdGFja4X947FQAIlzZWNwMjU2azGhAy3AtF2Jh_aPdOohg506Hjmtx-fQ1AKmu71C7PfkWAw9g3RjcIJ2YYN1ZHCCdmE
@@ -1411,4 +1411,4 @@
"uid": "nT9VeZoVk",
"version": 2,
"weekStart": ""
}
}
@@ -8,19 +8,20 @@ KEY="mykey"
CHAINID="laconic_9000-1"
MONIKER="localtestnet"
KEYRING="test"
KEYALGO="eth_secp256k1"
LOGLEVEL="info"
# trace evm
TRACE="--trace"
# TRACE=""
KEYALGO="secp256k1"
LOGLEVEL="${LOGLEVEL:-info}"
DENOM="alnt"
if [ "$1" == "clean" ] || [ ! -d "$HOME/.laconicd/data/blockstore.db" ]; then
# validate dependencies are installed
command -v jq > /dev/null 2>&1 || { echo >&2 "jq not installed. More info: https://stedolan.github.io/jq/download/"; exit 1; }
command -v jq > /dev/null 2>&1 || {
echo >&2 "jq not installed. More info: https://stedolan.github.io/jq/download/"
exit 1
}
# remove existing daemon and client
rm -rf $HOME/.laconicd/*
rm -rf $HOME/.laconic/*
if [ -n "`which make`" ]; then
make install
@@ -33,43 +34,42 @@ if [ "$1" == "clean" ] || [ ! -d "$HOME/.laconicd/data/blockstore.db" ]; then
laconicd keys add $KEY --keyring-backend $KEYRING --algo $KEYALGO
# Set moniker and chain-id for Ethermint (Moniker can be anything, chain-id must be an integer)
laconicd init $MONIKER --chain-id $CHAINID
laconicd init $MONIKER --chain-id $CHAINID --default-denom $DENOM
# Change parameter token denominations to photon
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["staking"]["params"]["bond_denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["crisis"]["constant_fee"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["gov"]["deposit_params"]["min_deposit"][0]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["mint"]["params"]["mint_denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
# Custom modules
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commit_fee"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveal_fee"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_minimum_bid"]["denom"]="photon"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
update_genesis() {
jq "$1" $HOME/.laconicd/config/genesis.json > $HOME/.laconicd/config/tmp_genesis.json &&
mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
}
if [[ "$TEST_REGISTRY_EXPIRY" == "true" ]]; then
echo "Setting timers for expiry tests."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["record_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
update_genesis '.app_state["registry"]["params"]["record_rent_duration"]="60s"'
update_genesis '.app_state["registry"]["params"]["authority_grace_period"]="60s"'
update_genesis '.app_state["registry"]["params"]["authority_rent_duration"]="60s"'
fi
if [[ "$TEST_AUCTION_ENABLED" == "true" ]]; then
echo "Enabling auction and setting timers."
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_enabled"]=true' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_rent_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_grace_period"]="300s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
cat $HOME/.laconicd/config/genesis.json | jq '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
update_genesis '.app_state["registry"]["params"]["authority_auction_enabled"]=true'
update_genesis '.app_state["registry"]["params"]["authority_rent_duration"]="60s"'
update_genesis '.app_state["registry"]["params"]["authority_grace_period"]="300s"'
update_genesis '.app_state["registry"]["params"]["authority_auction_commits_duration"]="60s"'
update_genesis '.app_state["registry"]["params"]["authority_auction_reveals_duration"]="60s"'
fi
if [[ "$ONBOARDING_ENABLED" == "true" ]]; then
echo "Enabling validator onboarding."
update_genesis '.app_state["onboarding"]["params"]["onboarding_enabled"]=true'
fi
# increase block time (?)
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["time_iota_ms"]="1000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
update_genesis '.consensus["params"]["block"]["time_iota_ms"]="1000"'
# Set gas limit in genesis
cat $HOME/.laconicd/config/genesis.json | jq '.consensus_params["block"]["max_gas"]="10000000"' > $HOME/.laconicd/config/tmp_genesis.json && mv $HOME/.laconicd/config/tmp_genesis.json $HOME/.laconicd/config/genesis.json
update_genesis '.consensus["params"]["block"]["max_gas"]="10000000"'
# disable produce empty block
if [[ "$OSTYPE" == "darwin"* ]]; then
@@ -78,30 +78,6 @@ if [ "$1" == "clean" ] || [ ! -d "$HOME/.laconicd/data/blockstore.db" ]; then
sed -i 's/create_empty_blocks = true/create_empty_blocks = false/g' $HOME/.laconicd/config/config.toml
fi
if [[ $1 == "pending" ]]; then
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i '' 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
else
sed -i 's/create_empty_blocks_interval = "0s"/create_empty_blocks_interval = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose = "3s"/timeout_propose = "30s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_propose_delta = "500ms"/timeout_propose_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote = "1s"/timeout_prevote = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_prevote_delta = "500ms"/timeout_prevote_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit = "1s"/timeout_precommit = "10s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_precommit_delta = "500ms"/timeout_precommit_delta = "5s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_commit = "5s"/timeout_commit = "150s"/g' $HOME/.laconicd/config/config.toml
sed -i 's/timeout_broadcast_tx_commit = "10s"/timeout_broadcast_tx_commit = "150s"/g' $HOME/.laconicd/config/config.toml
fi
fi
# Enable telemetry (prometheus metrics: http://localhost:1317/metrics?format=prometheus)
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's/enabled = false/enabled = true/g' $HOME/.laconicd/config/app.toml
@@ -114,23 +90,27 @@ if [ "$1" == "clean" ] || [ ! -d "$HOME/.laconicd/data/blockstore.db" ]; then
fi
# Allocate genesis accounts (cosmos formatted addresses)
laconicd add-genesis-account $KEY 100000000000000000000000000photon --keyring-backend $KEYRING
# 10^30 alnt | 10^12 lnt
laconicd genesis add-genesis-account $KEY 1000000000000000000000000000000$DENOM --keyring-backend $KEYRING
# Sign genesis transaction
laconicd gentx $KEY 1000000000000000000000photon --keyring-backend $KEYRING --chain-id $CHAINID
# 10^24 alnt | 10^6 lnt
laconicd genesis gentx $KEY 1000000000000000000000000$DENOM --keyring-backend $KEYRING --chain-id $CHAINID
# Collect genesis tx
laconicd collect-gentxs
laconicd genesis collect-gentxs
# Run this to ensure everything worked and that the genesis file is setup correctly
laconicd validate-genesis
if [[ $1 == "pending" ]]; then
echo "pending mode is on, please wait for the first block committed."
fi
laconicd genesis validate
else
echo "Using existing database at $HOME/.laconicd. To replace, run '`basename $0` clean'"
fi
# Start the node (remove the --pruning=nothing flag if historical queries are not needed)
laconicd start --pruning=nothing --evm.tracer=json $TRACE --log_level $LOGLEVEL --minimum-gas-prices=0.0001photon --json-rpc.api eth,txpool,personal,net,debug,web3,miner --api.enable --gql-server --gql-playground
laconicd start \
--pruning=nothing \
--log_level $LOGLEVEL \
--minimum-gas-prices=1$DENOM \
--api.enable \
--rpc.laddr="tcp://0.0.0.0:26657" \
--gql-server --gql-playground
@@ -6,4 +6,4 @@ services:
bondId:
chainId: laconic_9000-1
gas: 350000
fees: 200000photon
fees: 2000000alnt
@@ -65,7 +65,7 @@ if [ -n "$CERC_L1_ADDRESS" ] && [ -n "$CERC_L1_PRIV_KEY" ]; then
# Sequencer
SEQ=$(echo "$wallet3" | awk '/Address:/{print $2}')
SEQ_KEY=$(echo "$wallet3" | awk '/Private key:/{print $3}')
echo "Funding accounts."
wait_for_block 1 300
cast send --from $ADMIN --rpc-url $CERC_L1_RPC --value 5ether $PROPOSER --private-key $ADMIN_KEY
@@ -56,7 +56,7 @@
"value": "!validator-pubkey"
}
}
}
}
],
"supply": []
},
@@ -269,4 +269,4 @@
"claims": null
}
}
}
}
@@ -2084,4 +2084,4 @@
"clientPolicies": {
"policies": []
}
}
}
@@ -2388,4 +2388,4 @@
"clientPolicies": {
"policies": []
}
}
}
@@ -29,4 +29,3 @@
"l1_system_config_address": "0x5531dcff39ec1ec727c4c5d2fc49835368f805a9",
"protocol_versions_address": "0x0000000000000000000000000000000000000000"
}
@@ -2388,4 +2388,4 @@
"clientPolicies": {
"policies": []
}
}
}
@@ -12,7 +12,10 @@ from fabric import Connection
def dump_src_db_to_file(db_host, db_port, db_user, db_password, db_name, file_name):
command = f"pg_dump -h {db_host} -p {db_port} -U {db_user} -d {db_name} -c --inserts -f {file_name}"
command = (
f"pg_dump -h {db_host} -p {db_port} -U {db_user} "
f"-d {db_name} -c --inserts -f {file_name}"
)
my_env = os.environ.copy()
my_env["PGPASSWORD"] = db_password
print(f"Exporting from {db_host}:{db_port}/{db_name} to {file_name}... ", end="")
@@ -1 +0,0 @@
@@ -6,4 +6,4 @@ services:
bondId:
chainId: laconic_9000-1
gas: 250000
fees: 200000photon
fees: 2000000alnt
@@ -1,18 +1,15 @@
#!/bin/sh
if [[ -n "$CERC_SCRIPT_DEBUG" ]]; then
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
set -x
fi
#TODO: pass these in from the caller
TRACE="--trace"
LOGLEVEL="info"
laconicd start \
--pruning=nothing \
--evm.tracer=json $TRACE \
--log_level $LOGLEVEL \
--minimum-gas-prices=0.0001photon \
--json-rpc.api eth,txpool,personal,net,debug,web3,miner \
--minimum-gas-prices=1alnt \
--api.enable \
--gql-server \
--gql-playground
@@ -1901,4 +1901,4 @@
"uid": "b54352dd-35f6-4151-97dc-265bab0c67e9",
"version": 18,
"weekStart": ""
}
}
@@ -849,7 +849,7 @@ groups:
annotations:
summary: Watcher {{ index $labels "instance" }} of group {{ index $labels "job" }} is falling behind external head by {{ index $values "diff" }}
isPaused: false
# Secured Finance
- uid: secured_finance_diff_external
title: secured_finance_watcher_head_tracking
@@ -14,7 +14,7 @@ echo ACCOUNT_PRIVATE_KEY=${CERC_PRIVATE_KEY_DEPLOYER} >> .env
if [ -f ${erc20_address_file} ]; then
echo "${erc20_address_file} already exists, skipping ERC20 contract deployment"
cat ${erc20_address_file}
# Keep the container running
tail -f
fi
@@ -940,4 +940,3 @@ ALTER TABLE ONLY public.state
--
-- PostgreSQL database dump complete
--
@@ -18,4 +18,3 @@ root@7c4124bb09e3:/src#
```
Now gerbil commands can be run.
@@ -68,5 +68,5 @@ ENV PATH="${PATH}:/scripts"
COPY entrypoint.sh .
ENTRYPOINT ["./entrypoint.sh"]
# Placeholder CMD : generally this will be overridden at run time like :
# docker run -it -v /home/builder/cerc/laconic-sdk:/workspace cerc/builder-js sh -c 'cd /workspace && yarn && yarn build'
# docker run -it -v /home/builder/cerc/registry-sdk:/workspace cerc/builder-js sh -c 'cd /workspace && yarn && yarn build'
CMD node --version
@@ -23,7 +23,7 @@ local_npm_registry_url=$2
versioned_target_package=$(yarn list --pattern ${target_package} --depth=0 --json --non-interactive --no-progress | jq -r '.data.trees[].name')
# Use yarn info to get URL checksums etc from the new registry
yarn_info_output=$(yarn info --json $versioned_target_package 2>/dev/null)
# First check if the target version actually exists.
# First check if the target version actually exists.
# If it doesn't exist there will be no .data.dist.tarball element,
# and jq will output the string "null"
package_tarball=$(echo $yarn_info_output | jq -r .data.dist.tarball)
@@ -11,6 +11,8 @@ if len(sys.argv) > 1:
with open(testnet_config_path) as stream:
data = yaml.safe_load(stream)
for key, value in data['el_premine'].items():
acct = w3.eth.account.from_mnemonic(data['mnemonic'], account_path=key, passphrase='')
for key, value in data["el_premine"].items():
acct = w3.eth.account.from_mnemonic(
data["mnemonic"], account_path=key, passphrase=""
)
print("%s,%s,%s" % (key, acct.address, acct.key.hex()))
@@ -4,4 +4,4 @@ out = 'out'
libs = ['lib']
remappings = ['ds-test/=lib/ds-test/src/']
# See more config options https://github.com/gakonst/foundry/tree/master/config
# See more config options https://github.com/gakonst/foundry/tree/master/config
@@ -20,4 +20,4 @@ contract Stateful {
function inc() public {
x = x + 1;
}
}
}
@@ -50,7 +50,7 @@ RUN npm config set @cerc-io:registry ${CERC_NPM_REGISTRY_URL} \
# that install different Node-based CLI commands
# Globally install the cli package
RUN yarn global add @cerc-io/laconic-registry-cli@0.2.0
RUN yarn global add @cerc-io/laconic-registry-cli
# Add scripts
RUN mkdir /scripts
@@ -10,11 +10,11 @@ demo_records_dir="scripts/demo-records"
# Check we have funds
funds_response=$(${registry_command} account get --address $(cat my-address.txt))
funds_balance=$(echo ${funds_response} | jq -r .[0].balance[0].quantity)
funds_balance=$(echo ${funds_response} | jq -r ".[0].balance[0].quantity")
echo "Balance is: ${funds_balance}"
# Create a bond
bond_create_result=$(${registry_command} bond create --type photon --quantity 1000000000)
bond_create_result=$(${registry_command} bond create --type alnt --quantity 1000000000)
bond_id=$(echo ${bond_create_result} | jq -r .bondId)
echo "Created bond with id: ${bond_id}"
@@ -11,4 +11,4 @@ record:
foo: bar
tags:
- a
- b
- b
@@ -9,4 +9,4 @@ record:
foo: bar
tags:
- a
- b
- b
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
# Build cerc/laconicd
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
docker build -t cerc/laconicd:local ${build_command_args} ${CERC_REPO_BASE_DIR}/laconicd
docker build -t cerc/laconicd:local ${build_command_args} ${CERC_REPO_BASE_DIR}/laconicd
@@ -5,8 +5,11 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
fi
CERC_MIN_NEXTVER=13.4.2
CERC_DEFAULT_WEBPACK_VER="5.93.0"
CERC_NEXT_VERSION="${CERC_NEXT_VERSION:-keep}"
CERC_WEBPACK_VERSION="${CERC_WEBPACK_VERSION:-keep}"
CERC_BUILD_TOOL="${CERC_BUILD_TOOL}"
if [ -z "$CERC_BUILD_TOOL" ]; then
if [ -f "pnpm-lock.yaml" ]; then
@@ -23,15 +26,29 @@ fi
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
WORK_DIR="${1:-/app}"
if [ -f "${WORK_DIR}/build-webapp.sh" ]; then
echo "Building webapp with ${WORK_DIR}/build-webapp.sh ..."
cd "${WORK_DIR}" || exit 1
./build-webapp.sh || exit 1
exit 0
fi
if [ -f "next.config.mjs" ]; then
NEXT_CONFIG_JS="next.config.mjs"
IMPORT_OR_REQUIRE="import"
else
NEXT_CONFIG_JS="next.config.js"
IMPORT_OR_REQUIRE="require"
fi
# If this file doesn't exist at all, we'll get errors below.
if [ ! -f "next.config.js" ]; then
touch next.config.js
if [ ! -f "${NEXT_CONFIG_JS}" ]; then
touch ${NEXT_CONFIG_JS}
fi
if [ ! -f "next.config.dist" ]; then
cp next.config.js next.config.dist
cp $NEXT_CONFIG_JS next.config.dist
fi
which js-beautify >/dev/null
@@ -39,17 +56,34 @@ if [ $? -ne 0 ]; then
npm i -g js-beautify
fi
js-beautify next.config.dist > next.config.js
echo "" >> next.config.js
# js-beautify formats NEXTJS_CONFIG_FILE (ie next.config.js / next.config.mjs) so we can reliably transformable later
js-beautify next.config.dist > ${NEXT_CONFIG_JS}
echo "" >> ${NEXT_CONFIG_JS}
WEBPACK_REQ_LINE=$(grep -n "require([\'\"]webpack[\'\"])" next.config.js | cut -d':' -f1)
if [ -z "$WEBPACK_REQ_LINE" ]; then
cat > next.config.js.0 <<EOF
if [ "${IMPORT_OR_REQUIRE}" == "require" ]; then
WEBPACK_REQ_LINE=$(grep -n "require([\'\"]webpack[\'\"])" ${NEXT_CONFIG_JS} | cut -d':' -f1)
if [ -z "$WEBPACK_REQ_LINE" ]; then
cat > ${NEXT_CONFIG_JS}.0 <<EOF
const webpack = require('webpack');
EOF
fi
else
WEBPACK_IMPORT_LINE=$(grep -n "^import .*[\'\"]webpack[\'\"];?$" ${NEXT_CONFIG_JS} | cut -d':' -f1)
if [ -z "$WEBPACK_IMPORT_LINE" ]; then
cat > ${NEXT_CONFIG_JS}.0 <<EOF
import webpack from 'webpack';
EOF
fi
CREATE_REQUIRE_LINE=$(grep -n "require = createRequire" ${NEXT_CONFIG_JS} | cut -d':' -f1)
if [ -z "$CREATE_REQUIRE_LINE" ]; then
cat >> ${NEXT_CONFIG_JS}.0 <<EOF
import { createRequire } from "module";
const require = createRequire(import.meta.url);
EOF
fi
fi
cat > next.config.js.1 <<EOF
cat > ${NEXT_CONFIG_JS}.1 <<EOF
let envMap;
try {
// .env-list.json provides us a list of identifiers which should be replaced at runtime.
@@ -57,7 +91,8 @@ try {
a[v] = \`"CERC_RUNTIME_ENV_\${v.split(/\./).pop()}"\`;
return a;
}, {});
} catch {
} catch (e) {
console.error(e);
// If .env-list.json cannot be loaded, we are probably running in dev mode, so use process.env instead.
envMap = Object.keys(process.env).reduce((a, v) => {
if (v.startsWith('CERC_')) {
@@ -66,40 +101,80 @@ try {
return a;
}, {});
}
console.log(envMap);
EOF
CONFIG_LINES=$(wc -l next.config.js | awk '{ print $1 }')
ENV_LINE=$(grep -n 'env:' next.config.js | cut -d':' -f1)
WEBPACK_CONF_LINE=$(egrep -n 'webpack:\s+\([^,]+,' next.config.js | cut -d':' -f1)
NEXT_SECTION_ADJUSTMENT=0
grep 'withPWA' ${NEXT_CONFIG_JS} >/dev/null && HAS_WITHPWA=true || HAS_WITHPWA=false
if [ -n "$WEBPACK_CONF_LINE" ]; then
WEBPACK_CONF_VAR=$(egrep -n 'webpack:\s+\([^,]+,' next.config.js | cut -d',' -f1 | cut -d'(' -f2)
head -$(( ${WEBPACK_CONF_LINE} )) next.config.js > next.config.js.2
cat > next.config.js.3 <<EOF
$WEBPACK_CONF_VAR.plugins.push(new webpack.DefinePlugin(envMap));
EOF
NEXT_SECTION_LINE=$((WEBPACK_CONF_LINE))
elif [ -n "$ENV_LINE" ]; then
head -$(( ${ENV_LINE} - 1 )) next.config.js > next.config.js.2
cat > next.config.js.3 <<EOF
webpack: (config) => {
if [ "$HAS_WITHPWA" == "true" ]; then
if [ "$IMPORT_OR_REQUIRE" == "import" ]; then
cat > ${NEXT_CONFIG_JS}.2 <<EOF
const __xPWA__ = (p) => {
const realPWA = withPWA(p);
return (nextConfig) => {
const modConfig = {...nextConfig};
modConfig.webpack = (config) => {
config.plugins.push(new webpack.DefinePlugin(envMap));
return config;
},
return nextConfig.webpack ? nextConfig.webpack(config) : config;
};
return realPWA(modConfig);
};
};
EOF
NEXT_SECTION_ADJUSTMENT=1
NEXT_SECTION_LINE=$ENV_LINE
else
cat > ${NEXT_CONFIG_JS}.3 <<EOF
const __xPWA__ = (nextConfig) => {
const modConfig = {...nextConfig};
modConfig.webpack = (config) => {
config.plugins.push(new webpack.DefinePlugin(envMap));
return nextConfig.webpack ? nextConfig.webpack(config) : config;
};
return withPWA(modConfig);
};
EOF
fi
cat ${NEXT_CONFIG_JS} | js-beautify | sed 's/withPWA(/__xPWA__(/g' > ${NEXT_CONFIG_JS}.4
else
echo "WARNING: Cannot find location to insert environment variable map in next.config.js" 1>&2
rm -f next.config.js.*
NEXT_SECTION_LINE=0
cat > ${NEXT_CONFIG_JS}.3 <<EOF
const __xCfg__ = (nextConfig) => {
const modConfig = {...nextConfig};
modConfig.webpack = (config) => {
config.plugins.push(new webpack.DefinePlugin(envMap));
return nextConfig.webpack ? nextConfig.webpack(config) : config;
};
return modConfig;
};
EOF
if [ "$IMPORT_OR_REQUIRE" == "import" ]; then
cat ${NEXT_CONFIG_JS} | js-beautify | sed 's/export\s\+default\s\+/const __orig_cfg__ = /g' > ${NEXT_CONFIG_JS}.4
echo "export default __xCfg__(__orig_cfg__);" > ${NEXT_CONFIG_JS}.5
else
cat ${NEXT_CONFIG_JS} | js-beautify | sed 's/module.exports\s\+=\s\+/const __orig_cfg__ = /g' > ${NEXT_CONFIG_JS}.4
echo "module.exports = __xCfg__(__orig_cfg__);" > ${NEXT_CONFIG_JS}.5
fi
fi
tail -$(( ${CONFIG_LINES} - ${NEXT_SECTION_LINE} + ${NEXT_SECTION_ADJUSTMENT} )) next.config.js > next.config.js.5
cat next.config.js.* | sed 's/^ *//g' | js-beautify | grep -v 'process\.\env\.' | js-beautify > next.config.js
rm next.config.js.*
rm -f ${NEXT_CONFIG_JS}
for ((i=0; i <= 10; i++)); do
if [ -s "${NEXT_CONFIG_JS}.${i}" ]; then
if [ $i -le 2 ] ; then
cat ${NEXT_CONFIG_JS}.${i} >> ${NEXT_CONFIG_JS}
else
cat ${NEXT_CONFIG_JS}.${i} | sed 's/^ *//g' | js-beautify | grep -v 'process\.\env\.' | js-beautify >> ${NEXT_CONFIG_JS}
fi
fi
done
rm ${NEXT_CONFIG_JS}.*
cat ${NEXT_CONFIG_JS} | js-beautify > ${NEXT_CONFIG_JS}.pretty
mv ${NEXT_CONFIG_JS}.pretty ${NEXT_CONFIG_JS}
"${SCRIPT_DIR}/find-env.sh" "$(pwd)" > .env-list.json
@@ -107,8 +182,6 @@ if [ ! -f "package.dist" ]; then
cp package.json package.dist
fi
cat package.dist | jq '.scripts.cerc_compile = "next experimental-compile"' | jq '.scripts.cerc_generate = "next experimental-generate"' > package.json
CUR_NEXT_VERSION="`jq -r '.dependencies.next' package.json`"
if [ "$CERC_NEXT_VERSION" != "keep" ] && [ "$CUR_NEXT_VERSION" != "$CERC_NEXT_VERSION" ]; then
@@ -117,10 +190,38 @@ if [ "$CERC_NEXT_VERSION" != "keep" ] && [ "$CUR_NEXT_VERSION" != "$CERC_NEXT_VE
mv package.json.$$ package.json
fi
CUR_WEBPACK_VERSION="`jq -r '.dependencies.webpack' package.json`"
if [ -z "$CUR_WEBPACK_VERSION" ]; then
CUR_WEBPACK_VERSION="`jq -r '.devDependencies.webpack' package.json`"
fi
if [ "${CERC_WEBPACK_VERSION}" != "keep" ] || [ "${CUR_WEBPACK_VERSION}" == "null" ]; then
if [ -z "$CERC_WEBPACK_VERSION" ] || [ "$CERC_WEBPACK_VERSION" == "keep" ]; then
CERC_WEBPACK_VERSION="${CERC_DEFAULT_WEBPACK_VER}"
fi
echo "Webpack is required for env variable substitution. Adding to webpack@$CERC_WEBPACK_VERSION to dependencies..." 1>&2
cat package.json | jq ".dependencies.webpack = \"$CERC_WEBPACK_VERSION\"" > package.json.$$
mv package.json.$$ package.json
fi
time $CERC_BUILD_TOOL install || exit 1
CUR_NEXT_VERSION=`jq -r '.version' node_modules/next/package.json`
# See https://github.com/vercel/next.js/discussions/46544
semver -p -r ">=14.2.0" "$CUR_NEXT_VERSION"
if [ $? -eq 0 ]; then
# For >= 14.2.0
CERC_NEXT_COMPILE_COMMAND="next build --experimental-build-mode compile"
CERC_NEXT_GENERATE_COMMAND="next build --experimental-build-mode generate"
else
# For 13.4.2 to 14.1.x
CERC_NEXT_COMPILE_COMMAND="next experimental-compile"
CERC_NEXT_GENERATE_COMMAND="next experimental-generate"
fi
cat package.json | jq ".scripts.cerc_compile = \"$CERC_NEXT_COMPILE_COMMAND\"" | jq ".scripts.cerc_generate = \"$CERC_NEXT_GENERATE_COMMAND\"" > package.json.$$
mv package.json.$$ package.json
semver -p -r ">=$CERC_MIN_NEXTVER" $CUR_NEXT_VERSION
if [ $? -ne 0 ]; then
cat <<EOF
@@ -20,9 +20,11 @@ for d in $(find . -maxdepth 1 -type d | grep -v '\./\.' | grep '/' | cut -d'/' -
done
done
NEXT_CONF="next.config.js next.config.dist"
NEXT_CONF="next.config.mjs next.config.js next.config.dist"
for f in $NEXT_CONF; do
cat "$f" | tr -s '[:blank:]' '\n' | tr -s '[{},()]' '\n' | egrep -o 'process.env.[A-Za-z0-9_]+' >> $TMPF
if [ -f "$f" ]; then
cat "$f" | tr -s '[:blank:]' '\n' | tr -s '[{},()]' '\n' | egrep -o 'process.env.[A-Za-z0-9_]+' >> $TMPF
fi
done
cat $TMPF | sort -u | jq --raw-input . | jq --slurp .
@@ -30,36 +30,44 @@ fi
CERC_WEBAPP_FILES_DIR="${CERC_WEBAPP_FILES_DIR:-/app}"
cd "$CERC_WEBAPP_FILES_DIR"
"$SCRIPT_DIR/apply-runtime-env.sh" "`pwd`" .next .next-r
mv .next .next.old
mv .next-r/.next .
if [ -f "./run-webapp.sh" ]; then
echo "Running webapp with run-webapp.sh ..."
cd "${WORK_DIR}" || exit 1
./run-webapp.sh &
tpid=$!
wait $tpid
else
"$SCRIPT_DIR/apply-runtime-env.sh" "`pwd`" .next .next-r
mv .next .next.old
mv .next-r/.next .
if [ "$CERC_NEXTJS_SKIP_GENERATE" != "true" ]; then
jq -e '.scripts.cerc_generate' package.json >/dev/null
if [ $? -eq 0 ]; then
npm run cerc_generate > gen.out 2>&1 &
tail -f gen.out &
tpid=$!
if [ "$CERC_NEXTJS_SKIP_GENERATE" != "true" ]; then
jq -e '.scripts.cerc_generate' package.json >/dev/null
if [ $? -eq 0 ]; then
npm run cerc_generate > gen.out 2>&1 &
tail -f gen.out &
tpid=$!
count=0
generate_done="false"
while [ $count -lt $CERC_MAX_GENERATE_TIME ] && [ "$generate_done" == "false" ]; do
sleep 1
count=$((count + 1))
grep 'rendered as static' gen.out > /dev/null
if [ $? -eq 0 ]; then
generate_done="true"
count=0
generate_done="false"
while [ $count -lt $CERC_MAX_GENERATE_TIME ] && [ "$generate_done" == "false" ]; do
sleep 1
count=$((count + 1))
grep 'rendered as static' gen.out > /dev/null
if [ $? -eq 0 ]; then
generate_done="true"
fi
done
if [ $generate_done != "true" ]; then
echo "ERROR: 'npm run cerc_generate' not successful within CERC_MAX_GENERATE_TIME" 1>&2
exit 1
fi
done
if [ $generate_done != "true" ]; then
echo "ERROR: 'npm run cerc_generate' not successful within CERC_MAX_GENERATE_TIME" 1>&2
exit 1
kill $tpid $(ps -ef | grep node | grep next | grep generate | awk '{print $2}') 2>/dev/null
tpid=""
fi
kill $tpid $(ps -ef | grep node | grep next | grep generate | awk '{print $2}') 2>/dev/null
tpid=""
fi
fi
$CERC_BUILD_TOOL start . -- -p ${CERC_LISTEN_PORT:-80}
$CERC_BUILD_TOOL start . -- -p ${CERC_LISTEN_PORT:-80}
fi
@@ -5,4 +5,3 @@ WORKDIR /app
COPY . .
RUN yarn
@@ -4,5 +4,9 @@ source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
# Two-stage build is to allow us to pick up both the upstream repo's files, and local files here for config
docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/explorer
docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/cosmos-explorer
if [[ $? -ne 0 ]]; then
echo "FATAL: Base container build failed, exiting"
exit 1
fi
docker build -t cerc/ping-pub:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile $SCRIPT_DIR
@@ -7,16 +7,16 @@
"rpc": [
{"provider": "LX-tendermint-rpc", "address": "LACONIC_LACONICD_RPC_URL"}
],
"sdk_version": "0.45.1",
"sdk_version": "0.50.3",
"coin_type": "118",
"min_tx_fee": "800",
"addr_prefix": "ethm",
"addr_prefix": "laconic",
"logo": "/logos/cosmos.svg",
"assets": [{
"base": "photon",
"base": "alnt",
"symbol": "LNT",
"exponent": "6",
"coingecko_id": "cosmos",
"exponent": "18",
"coingecko_id": "cosmos",
"logo": "/logos/cosmos.svg"
}]
}
@@ -22,7 +22,7 @@ fi
# infers the directory from which to load chain configuration files
# by the presence or absense of the substring "testnet" in the host name
# (browser side -- the host name of the host in the address bar of the browser)
# Accordingly we configure our network in both directories in order to
# Accordingly we configure our network in both directories in order to
# subvert this lunacy.
explorer_mainnet_config_dir=/app/chains/mainnet
explorer_testnet_config_dir=/app/chains/testnet
@@ -2,4 +2,4 @@
# Build cerc/test-container
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
docker build -t cerc/test-database-client:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} $SCRIPT_DIR
docker build -t cerc/test-database-client:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} $SCRIPT_DIR
@@ -8,7 +8,7 @@ CERC_WEBAPP_FILES_DIR="${CERC_WEBAPP_FILES_DIR:-/data}"
CERC_ENABLE_CORS="${CERC_ENABLE_CORS:-false}"
CERC_SINGLE_PAGE_APP="${CERC_SINGLE_PAGE_APP}"
if [ -z "${CERC_SINGLE_PAGE_APP}" ]; then
if [ -z "${CERC_SINGLE_PAGE_APP}" ]; then
# If there is only one HTML file, assume an SPA.
if [ 1 -eq $(find "${CERC_WEBAPP_FILES_DIR}" -name '*.html' | wc -l) ]; then
CERC_SINGLE_PAGE_APP=true
@@ -0,0 +1,260 @@
# Caddy Ingress Controller for kind
# Based on: https://github.com/caddyserver/ingress
# Provides automatic HTTPS with Let's Encrypt
apiVersion: v1
kind: Namespace
metadata:
name: caddy-system
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: caddy-ingress-controller
namespace: caddy-system
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: caddy-ingress-controller
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- namespaces
- services
verbs:
- list
- watch
- get
- apiGroups:
- ""
resources:
- secrets
verbs:
- list
- watch
- get
- create
- update
- delete
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- create
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: caddy-ingress-controller
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: caddy-ingress-controller
subjects:
- kind: ServiceAccount
name: caddy-ingress-controller
namespace: caddy-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: caddy-ingress-controller-configmap
namespace: caddy-system
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
data:
# Caddy global options
acmeCA: "https://acme-v02.api.letsencrypt.org/directory"
email: ""
---
apiVersion: v1
kind: Service
metadata:
name: caddy-ingress-controller
namespace: caddy-system
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
app.kubernetes.io/component: controller
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
- name: https
port: 443
targetPort: https
protocol: TCP
selector:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
app.kubernetes.io/component: controller
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: caddy-ingress-controller
namespace: caddy-system
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
app.kubernetes.io/component: controller
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
app.kubernetes.io/component: controller
template:
metadata:
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
app.kubernetes.io/component: controller
spec:
serviceAccountName: caddy-ingress-controller
terminationGracePeriodSeconds: 60
nodeSelector:
ingress-ready: "true"
kubernetes.io/os: linux
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Equal
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Equal
containers:
- name: caddy-ingress-controller
image: caddy/ingress:latest
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
hostPort: 80
protocol: TCP
- name: https
containerPort: 443
hostPort: 443
protocol: TCP
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- -config-map=caddy-system/caddy-ingress-controller-configmap
- -class-name=caddy
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 1000m
memory: 512Mi
readinessProbe:
httpGet:
path: /healthz
port: 9765
initialDelaySeconds: 3
periodSeconds: 10
livenessProbe:
httpGet:
path: /healthz
port: 9765
initialDelaySeconds: 3
periodSeconds: 10
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 0
runAsGroup: 0
volumeMounts:
- name: caddy-data
mountPath: /data
- name: caddy-config
mountPath: /config
volumes:
- name: caddy-data
emptyDir: {}
- name: caddy-config
emptyDir: {}
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: caddy
labels:
app.kubernetes.io/name: caddy-ingress-controller
app.kubernetes.io/instance: caddy-ingress
annotations:
ingressclass.kubernetes.io/is-default-class: "true"
spec:
controller: caddy.io/ingress-controller
+1 -1
View File
@@ -1,4 +1,4 @@
laconic-sdk
registry-sdk
laconic-registry-cli
laconic-console
debug
+1 -1
View File
@@ -7,7 +7,7 @@ github.com/cerc-io/ipld-eth-db-validator
github.com/cerc-io/ipld-eth-beacon-indexer
github.com/cerc-io/ipld-eth-beacon-db
github.com/cerc-io/laconicd
github.com/cerc-io/laconic-sdk
github.com/cerc-io/registry-sdk
github.com/cerc-io/laconic-registry-cli
github.com/cerc-io/laconic-console
github.com/cerc-io/mobymask-watcher-ts
@@ -6,7 +6,7 @@ JS/TS/NPM builds need an npm registry to store intermediate package artifacts.
This can be supplied by the user (e.g. using a hosted registry or even npmjs.com), or a local registry using gitea can be deployed by stack orchestrator.
To use a user-supplied registry set these environment variables:
`CERC_NPM_REGISTRY_URL` and
`CERC_NPM_REGISTRY_URL` and
`CERC_NPM_AUTH_TOKEN`
Leave `CERC_NPM_REGISTRY_URL` un-set to use the local gitea registry.
@@ -22,7 +22,7 @@ $ laconic-so --stack build-support build-containers
```
$ laconic-so --stack package-registry setup-repositories
$ laconic-so --stack package-registry build-containers
$ laconic-so --stack package-registry build-containers
$ laconic-so --stack package-registry deploy up
[+] Running 3/3
⠿ Network laconic-aecc4a21d3a502b14522db97d427e850_gitea Created 0.0s
@@ -58,5 +58,5 @@ Now npm packages can be built:
Ensure that `CERC_NPM_AUTH_TOKEN` is set with the token printed above when the package-registry stack was deployed (the actual token value will be different than shown in this example):
```
$ export CERC_NPM_AUTH_TOKEN=84fe66a73698bf11edbdccd0a338236b7d1d5c45
$ laconic-so build-npms --include laconic-sdk,laconic-registry-cli
$ laconic-so build-npms --include registry-sdk,laconic-registry-cli
```
@@ -14,4 +14,3 @@ containers:
pods:
- fixturenet-blast
- foundry
@@ -3,4 +3,3 @@
A "loaded" version of fixturenet-eth, with all the bells and whistles enabled.
TODO: write me

Some files were not shown because too many files have changed in this diff Show More