deep-stack/stack-orchestrator
3
0
Fork 0
forked from cerc-io/stack-orchestrator
Code Pull Requests Activity
971 Commits 44 Branches 393 Tags
fee32ec7032ee3cbf785ddfce9f72a8d6c8d3b64
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
telackey cc541ac20f Use -slim variant for Dockerfile (#773) 
This saves about 1GB of space in the image.

Reviewed-on: cerc-io/stack-orchestrator#773
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-28 04:37:57 +00:00
telackey 21eb9f036f Add support for pnpm as a webapp build tool. (#767) 
This adds support for auto-detecting pnpm as a build tool for webapps.

Reviewed-on: cerc-io/stack-orchestrator#767
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-26 23:31:52 +00:00
telackey 704c42c404 Use a catchall for single page apps. (#763) 
This creates a new environment variable, CERC_SINGLE_PAGE_APP, which controls whether a catchall redirection back to / is applied.

If the value is not explicitly set, we try to detect if the page looks like a single-page app.

Reviewed-on: cerc-io/stack-orchestrator#763
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-23 20:32:24 +00:00
telackey 6bd77c893a Even more logging fixes (#757) 
Hopefully the last one for a bit.

This only output the cmdline if log_file is present (ie, not to stdout).  It also fixes a bug where the log_file was not passed in one line.

Reviewed-on: cerc-io/stack-orchestrator#757
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-22 01:24:44 +00:00
telackey 4a4d48ddb9 Fix error when logging exception. (#756) 
Reviewed-on: cerc-io/stack-orchestrator#756
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-22 00:11:06 +00:00
telackey 08438b1cd5 More logging for webapp deployments (#755) 
Reviewed-on: cerc-io/stack-orchestrator#755
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-21 23:48:52 +00:00
telackey 9f1dd284a5 Better error logging for registry deployments. (#754) 
We were missing errors related to registration, this should fix that.

Reviewed-on: cerc-io/stack-orchestrator#754
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-21 20:12:52 +00:00
telackey db4986dcc6 snowballtool-base backend stack (#751) 
This adds a stack for the backend from snowball/snowballtools-base.

Reviewed-on: cerc-io/stack-orchestrator#751
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-21 04:45:46 +00:00
telackey 65f05ea80c Run a manual build script, if present. (#750) 
If the tree has a 'build-webapp.sh' script, use that.

Reviewed-on: cerc-io/stack-orchestrator#750
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-21 00:20:50 +00:00
telackey 3c3e582939 Minor envsubst improvements. (#746) 
Minor fixes to envsubst for webapps.  Somewhat specially treated is `LACONIC_HOSTED_CONFIG_homepage` which can be used to replace the homepage in package.json.  With react, this gets an extra `/` though, which we need to remove.

Reviewed-on: cerc-io/stack-orchestrator#746
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-16 04:11:09 +00:00
telackey 65d67dba10 Fix k8s and enable it by default on PRs (#742) 
Reviewed-on: cerc-io/stack-orchestrator#742
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-14 23:50:09 +00:00
telackey b22c72e715 For k8s, use provisioner-managed volumes when an absolute host path is not specified. (#741) 
In kind, when we bind-mount a host directory it is first mounted into the kind container at /mnt, then into the pod at the desired location.

We accidentally picked this up for full-blown k8s, and were creating volumes at /mnt.  This changes the behavior for both kind and regular k8s so that bind mounts are only allowed if a fully-qualified path is specified.  If no path is specified at all, a default storageClass is assumed to be present, and the volume managed by a provisioner.

Eg, for kind, the default provisioner is: https://github.com/rancher/local-path-provisioner

```
stack: test
deploy-to: k8s-kind
config:
  test-variable-1: test-value-1
network:
  ports:
    test:
     - '80'
volumes:
  # this will be bind-mounted to a host-path
  test-data-bind: /srv/data
  # this will be managed by the k8s node
  test-data-auto:
configmaps:
  test-config: ./configmap/test-config
```

Reviewed-on: cerc-io/stack-orchestrator#741
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-14 21:45:01 +00:00
telackey c9444591f5 Fix default webapp port number. (#740) 
Reviewed-on: cerc-io/stack-orchestrator#740
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-09 01:20:41 +00:00
telackey 903f3b10e2 Add support for annotations and labels in spec. (#739) 
```
stack: webapp-deployer-backend
deploy-to: k8s
annotations:
  foo.bar.annot/{name}: baz
labels:
  a.b.c/{name}.blah: "value"
```

Reviewed-on: cerc-io/stack-orchestrator#739
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-09 00:11:07 +00:00
telackey 72ed2eb91a Fix bad test in tag check. (#738) 
Reviewed-on: cerc-io/stack-orchestrator#738
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-08 20:38:41 +00:00
telackey 3309782439 Refactor 2024-02-08 00:47:46 -06:00
telackey 4b3b3478e7 Switch to Docker-style limits 2024-02-08 00:43:41 -06:00
telackey 2a9955055c debug 2024-02-07 16:56:35 -06:00
telackey 8964e1c0fe Add resource limit options to spec. 2024-02-07 16:48:02 -06:00
telackey d2ebb81d77 Tags for undeploy (#734) 
```
  --include-tags TEXT             Only include requests with matching tags
                                  (comma-separated).
  --exclude-tags TEXT             Exclude requests with matching tags (comma-
                                  separated).
```

Reviewed-on: cerc-io/stack-orchestrator#734
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-07 21:45:16 +00:00
telackey 4a981d8d2e Fix repo URL (#733) 
Needs a '/' (http) not ':' (ssh).

Reviewed-on: cerc-io/stack-orchestrator#733
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-07 18:52:07 +00:00
telackey 937b983ec9 Update links from github.com to git.vdb.to (#732) 
Update links and references to github.com to git.vdb.to.

Also enable the flake8 lint action in gitea.

Reviewed-on: cerc-io/stack-orchestrator#732
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
 2024-02-06 22:42:44 +00:00
telackeyanddboreham 36bb068983 Add ConfigMap test. (#726) 
* Add ConfigMap test.

* eof

* Minor tweak

* Trigger test

---------

Co-authored-by: David Boreham <david@bozemanpass.com>
 2024-02-05 14:15:11 -06:00
telackey 2fcd416e29 Basic webapp deployer stack. (#722) 2024-02-02 19:05:15 -07:00
telackey 6629017d6a Support other webapp types (react, static). (#721) 
* Support other webapp types (react, static).
 2024-02-02 18:04:06 -06:00
telackey b398050787 Don't include volumes in spec if we don't have any. (#720) 2024-01-31 15:11:32 -06:00
telackey 12ec1bec43 Add ConfigMap support for k8s. (#714) 
* Minor fixes for deploying with k8s and podman.

* ConfigMap support
 2024-01-30 23:09:48 -06:00
telackey 62af03077f Add deployed/error status output to the state file. (#719) 
* More status info
* Up default resource limits.
* Need ps
 2024-01-30 22:13:45 -06:00
telackey 428b05158e Fix DnsRecord ownership check. (#718) 
* Fix DnsRecord ownership check.

* Var names
 2024-01-30 13:31:59 -06:00
telackey 554f05de87 Fix pip3 error (#709) 2024-01-22 21:06:15 -06:00
telackey 0587813dd0 Create next.config.js if it is missing. (#701) 
* Create next.config.js if it is missing.

* Add comment.
 2024-01-15 12:12:59 -06:00
telackey 4b3ea7c30f Update independent act-runner stack to use custom act as well. (#702) 
* Update independent act-runner stack to use custom act as well.

* Remove branches which are not needed or already merged.
 2024-01-15 12:10:48 -06:00
telackey b83030f63b Use custom act with gitea. (#700) 
* Use custom act with gitea.

* Make sure wget is available

* Fix repo url
 2024-01-09 22:53:43 -06:00
telackey 837e443800 Support application removal requests. (#697) 
* Support application removal request.

* Git should never prompt when deploying a webapp
 2023-12-21 18:05:40 -06:00
telackey 997496b8a5 Update script for new nextjs build output. (#691) 2023-12-14 19:47:30 -06:00
telackey 61f2884505 Reduce base image size (first round of improvements) (#690) 2023-12-14 17:46:03 -06:00
telackey 27a14737f8 Make the container tag based on the deployment path. (#688) 2023-12-14 09:49:21 -06:00
telackey 1f4eb57069 Add --dry-run option (#686) 2023-12-13 22:56:40 -06:00
telackey 88f66a3626 Add deployment update and deploy-webapp-from-registry commands. (#676) 2023-12-13 21:02:34 -06:00
telackey 077ea80c70 Add deployment status command and fix k8s output for deployment ps (#679) 2023-12-06 09:27:47 -07:00
telackey 2dd54892a1 Allow specifying the webapp tag explicitly (#675) 2023-12-04 21:39:16 -06:00
telackey ab0e70ed83 Change path portion of unique cluster name to point to compose file, not argv[0]. (#678) 2023-12-04 13:39:14 -06:00
telackey 2173e7ce6a If the next version is unsupported, print a big warning and try higher version. (#674) 2023-11-30 12:33:06 -06:00
telackey 03a3645b3c Add --port option to run-webapp. (#667) 
* Add --port option to run-webapp

* Fixed merge

* lint
 2023-11-29 11:32:28 -06:00
telackey 1a37255c18 Tweak laconicd config to allow setting endpoint port and to make the fixturenet restartable. (#660) 
* Endpoint includes port

* Make it restartable

* Don't try to remove the mounted directory

* Make copy of init.sh
 2023-11-22 11:31:30 -06:00
telackey 5c80887215 Fix missing tty parameter. (#653) 2023-11-16 12:58:03 -06:00
telackey 1e9d24a8ce Update webapp.md 2023-11-15 12:52:34 -06:00
telackey 9900565714 Update webapp.md 2023-11-15 12:48:58 -06:00
telackey a13f841f34 Update webapp.md 2023-11-15 12:37:30 -06:00
telackey d37f80553d Add webapp doc (#652) 2023-11-15 12:28:07 -06:00
telackey 2059d67dca Add run-webapp command. (#651) 2023-11-15 10:54:27 -07:00
telackey 4ae4d3b61d Print docker container logs in webapp test. (#649) 2023-11-14 17:30:01 -06:00
telackey 9687d84468 646: Add error message for webapp startup hang (#647) 
This fixes three issues:

1. #644 (build output)
2. #646 (error on startup)
3. automatic env quote handling (related to 2)


For the build output we now have:

```
#################################################################

Built host container for /home/telackey/tmp/iglootools-home with tag:

    cerc/iglootools-home:local

To test locally run:

    docker run -p 3000:3000 cerc/iglootools-home:local
```

For the startup error, it was hung waiting for the "success" message from the next generate output (itself a workaround for a nextjs bug fixed by this PR we submitted: https://github.com/vercel/next.js/pull/58276).

I added a timeout which will cause it to wait up to a maximum _n_ seconds before issuing:

```
ERROR: 'npm run cerc_generate' exceeded CERC_MAX_GENERATE_TIME.
```

On the quoting itself, I plan on adding a new run-webapp command, but I realized I had a decent spot to do effect the quote replacement on-the-fly after all when I am already escaping the values for insertion/replacement into JS.

The "dequoting" can be disabled with `CERC_RETAIN_ENV_QUOTES=true`.
 2023-11-14 16:07:26 -06:00
telackey f1f618c57a Don't change the next.js version by default. (#640) 2023-11-13 11:56:04 -06:00
telackey 414b887036 Allow setting build tool (npm/yarn) and next.js version. (#639) 
* Allow setting build tool (npm/yarn) and next.js version.
 2023-11-10 17:44:25 -06:00
telackey 042b413598 Support the case where webpack config is already present next.config.js (#631) 
* Support the case where webpack config is already present next.config.js

* Update scripts for experimental-compile/experimental-generate
 2023-11-08 23:44:48 -06:00
telackey 36e13f7199 Remove test output from tree. (#628) 2023-11-07 23:10:32 -06:00
telackey d9bcc088a8 Enable webapp test in GitHub CI. (#627) 2023-11-07 18:27:08 -06:00
telackey 660326f713 Add new build-webapp command and related scripts and containers. (#626) 
* Add new build-webapp command and related scripts and containers.
 2023-11-07 18:15:04 -06:00
telackey fd5779f967 Fix KeyError accessing config. (#620) 2023-10-31 12:29:19 -05:00
telackey 8cac598679 Split act-runner into its own pod and offer as a distinct stack. (#612) 
* Split act-runner into its own pod and offer as a distinct stack.
 2023-10-27 13:57:13 -05:00
telackey 0f5b1a097b Add plugeth to chain-chunker stack (needed for new verify option). (#610) 2023-10-25 14:47:53 -05:00
telackeyanddboreham 20d633f81c Plugeth-based full mainnet stack. (#592) 
* Plugeth-based full mainnet stack.

---------

Co-authored-by: David Boreham <david@bozemanpass.com>
 2023-10-25 14:42:52 -05:00
telackey 3291c16466 Update stacks for migrated laconic repos. (#594) 2023-10-20 11:59:06 -05:00
telackey 8246e3551f 53: Make sure curl and jq are present in the container. (#585) 2023-10-17 13:36:43 -05:00
telackey 80b0c07736 Open ports for 2nd geth instance and add missing lcli param. (#555) 2023-10-03 19:52:23 -05:00
telackey 6fa3ca2b6d Update from github.com to git.vdb.to where applicable. (#553) 
* Update from github.com to git.vdb.to for many repos.

* Use ipld-eth-server@v1.11.6-statediff-v5 for most stacks

* Specify go-ethereum branch/tag
 2023-10-03 13:55:33 -05:00
telackey ed9c253f61 Simplified, improved default dashboard for mainnet-eth-metrics. (#536) 
* Simplified, improved default dashboard for mainnet-eth-metrics.

* Add missing lighthouse prometheus config.
 2023-09-19 20:29:08 -05:00
telackey bdf4f069b4 Include all of app/data regardless of depth. (#535) 2023-09-19 20:26:48 -05:00
telackey d3c26948fc Add eth-api-proxy to mainnet-eth stack. (#514) 
* Add eth-api-proxy to mainnet-eth stack.
 2023-08-29 16:49:09 -05:00
telackey 7f9b556f9d Update mainnet-eth stack to include auth and metrics. (#492) 2023-08-29 11:32:37 -05:00
telackey f55a14bd6c Create new base container for pre-compiled webapps. (#507) 
* Create new base container for pre-compiled webapps.

* Set the webapp dir appropriately.
 2023-08-22 13:32:45 -05:00
telackey 18e45af463 Add port config to deployment spec. (#498) 
* Add port config to deployment spec.
 2023-08-11 15:25:54 -05:00
telackey a5945c9e59 Back to Github-based actions. (#453) 
* Back to Github-based actions.

* Use fork.

* Use new release plugin tag.
 2023-07-19 20:40:04 -06:00
telackey 809889f9f0 Update plugeth stack for plugeth-statediff plugin. (#443) 
* Use latest plugeth-statediff.

* Change plugeth path

* EOL

* CERC_PLUGINS_DIR
 2023-06-29 15:11:28 -06:00
telackey 26ff7a969c Fix plugeth build. (#423) 2023-06-05 21:10:17 -05:00
telackey a8e198ad55 Allow configuring the number of statediff workers. (#422) 
* Allow configuring the number of statediff workers.

* Leave logging alone
 2023-06-05 18:16:42 -05:00
telackey 23a336020c Make a separate lighthouse container for the plugeth fixturenet. (#412) 
* Make a separate lighthouse container for the plugeth fixturenet.
 2023-05-26 16:57:15 -05:00
telackey 6ec55ba460 Add a plugeth-based version of the fixturenet (#411) 
* plugeth version of the fixturenet

* Use pre-built plugeth.
 2023-05-25 11:21:08 -05:00
telackey aa702737ef Fix 397 by pegging alpine version. 2023-05-19 11:26:09 -05:00
telackey 03f6d027f9 Minor script cleanup. (#372) 
Former-commit-id: 323ca3b238
 2023-04-26 23:26:50 -05:00
telackey d8522211f4 Add script for exporting ethdb from fixturenet. (#370) 
* Add script for exporting ethdb from fixturenet.

* Update README

* Script

Former-commit-id: 7a607c2994
 2023-04-26 00:13:35 -05:00
telackey 80c16d2ced Update for latest act_runner. (#316) 
Former-commit-id: c7c3cbde8e
 2023-04-11 15:05:08 -05:00
telackey 287468b7c0 Update run script to support COPY and WebSockets. (#275) 
Former-commit-id: 4da69ebf4c
 2023-04-03 14:59:31 -05:00
telackey 788b214116 Add Gitea action support via act_runner. 
Former-commit-id: 74077d7704
 2023-03-24 22:24:40 -05:00
telackey bc5eff71b5 Use latest keycloak plugin. (#173) 
Former-commit-id: 8645ab0619
 2023-02-01 16:02:49 -06:00
telackey bdde1b2c9e Include all eth API operation metrics. (#162) 
Former-commit-id: ce0c7ee9ba
 2023-01-26 13:53:02 -06:00
telackey b5c7ad114c Integrate grafana/prometheus with fixturenet-eth. (#156) 
* Integrate grafana/prometheus with fixturenet-eth.

* Integrate grafana/prometheus with fixturenet-eth.

* Revert port change

* EOL

* EOL

* Simplify ipld-eth-server config.

Former-commit-id: c279bd49a4
 2023-01-25 11:50:08 -06:00
telackey c77cc4e111 Update keycloak to include X-User-Id header. (#147) 
Former-commit-id: 65cdba7705
 2023-01-23 10:39:15 -06:00
telackey 08d5c22f32 Enable metrics endpoint in fixturenet. 
Former-commit-id: 0ea29bc480
 2023-01-17 12:04:28 -06:00
telackey b13300ca81 Add CERC_STATEDIFF_DB_LOG_STATEMENTS=true|false option. (#120) 2023-01-13 16:33:59 -06:00
telackey 6396de1ecf Improvements for building/deploying geth. (#119) 
* Improvements for building/deploying geth.

* Improvements for building/deploying geth.
 2023-01-12 11:37:23 -06:00
telackey 8e184f509d Add remote debugging support to fixturenet-geth. (#113) 
* Add remote debugging support to fixturenet-geth.
 2023-01-11 12:54:22 -06:00
telackey 8669a822df fix: import Path 2023-01-09 21:43:04 -06:00
telackey e7296c2ce3 Add tx-spammer (#74) 
* Add tx-spammer

* Update README
 2022-12-21 09:59:40 -06:00
telackey 8f63328d60 fixturenet-eth README (#73) 
* fixturenet-eth README

* Fix bullets

* Minor edit

* Update README.md
 2022-12-20 14:08:10 -06:00
telackey efbc86bdea Moved repos. (#72) 2022-12-20 14:06:48 -06:00
telackey 8a9c1fb781 Use new repo (#68) 2022-12-16 15:57:26 -06:00
telackey 5d482658a7 Download pre-built keycloak api-key-module-0.1.jar rather than building it. (#66) 2022-12-16 14:15:56 -06:00
telackey df0bf549a2 Keycloak + nginx reverse proxy (#63) 
* Example Keycloak config.

* Proxy for geth, ipld-eth-server, and lighthouse.

* Add metrics.
 2022-12-14 12:59:40 -06:00