Commit Graph
100 Commits
Author SHA1 Message Date
telackey fef7649683 Refactor SPA check. (#831)
Reviewed-on: cerc-io/stack-orchestrator#831
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-05-22 19:00:42 +00:00
telackey 579b402f2f Make the SPA detection even simpler. (#830)
Reviewed-on: cerc-io/stack-orchestrator#830
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-05-22 18:31:57 +00:00
telackey 25d0bc8a98 Case insensitive comparison (#829)
Reviewed-on: cerc-io/stack-orchestrator#829
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-05-22 18:18:27 +00:00
telackey 855288368c Add messages around the SPA auto-detect. (#828)
Tweak the auto-detection logic slightly for single-page apps, and also print the results.

Reviewed-on: cerc-io/stack-orchestrator#828
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-05-22 17:42:28 +00:00
telackey 9043a67c7c Skip checks on requests we've already seen (#805)
Reviewed-on: cerc-io/stack-orchestrator#805
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-04-15 20:27:35 +00:00
telackey 4126f2fc43 Add --fqdn-policy option to deploy-webapp-from-registry. (#802)
This add a new option `--fqdn-policy` to the `deploy-webapp-from-registry`.

The default policy, `prohibit` means that `ApplicationDeploymentRequests` which specify a FQDN will be rejected.  The `allow` policy will cause them to be processed.  The `preexisting` policy will only process them if an existing `DnsRecord` exists in the registry with the correct ownership.

The latter would be useful in conjunction with a pre-checking scheme in the UI (eg, that the DNS entry is properly configured, the domain is under the control of the requestor, etc.)  Only after all the checks were successful would the `DnsRecord` be created, allowing for `ApplicationDeploymentRequests` to use it.

Reviewed-on: cerc-io/stack-orchestrator#802
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-04-15 12:20:35 +00:00
telackey 5308ab1e4e Blind commit to fix laconic CLI calls after rename. (#784)
`laconic cns` got renamed to `laconic registry` which breaks all the scripts and commands that use it.

Reviewed-on: cerc-io/stack-orchestrator#784
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-25 19:09:26 +00:00
telackey 523b5779be Auto-detect which certificate to use (including wildcards). (#779)
Rather than always requesting a certificate, attempt to re-use an existing certificate if it already exists in the k8s cluster.  This includes matching to a wildcard certificate.

Reviewed-on: cerc-io/stack-orchestrator#779
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-07 17:38:36 +00:00
telackey 62f7ce649d Exit non-0 if docker build fails. (#778)
Make sure to check the exit code of the docker build and bubble it back up to laconic-so.

Reviewed-on: cerc-io/stack-orchestrator#778
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-03-06 18:38:30 +00:00
telackey cc541ac20f Use -slim variant for Dockerfile (#773)
This saves about 1GB of space in the image.

Reviewed-on: cerc-io/stack-orchestrator#773
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-28 04:37:57 +00:00
telackey 21eb9f036f Add support for pnpm as a webapp build tool. (#767)
This adds support for auto-detecting pnpm as a build tool for webapps.

Reviewed-on: cerc-io/stack-orchestrator#767
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-26 23:31:52 +00:00
telackey 704c42c404 Use a catchall for single page apps. (#763)
This creates a new environment variable, CERC_SINGLE_PAGE_APP, which controls whether a catchall redirection back to / is applied.

If the value is not explicitly set, we try to detect if the page looks like a single-page app.

Reviewed-on: cerc-io/stack-orchestrator#763
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-23 20:32:24 +00:00
telackey 6bd77c893a Even more logging fixes (#757)
Hopefully the last one for a bit.

This only output the cmdline if log_file is present (ie, not to stdout).  It also fixes a bug where the log_file was not passed in one line.

Reviewed-on: cerc-io/stack-orchestrator#757
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-22 01:24:44 +00:00
telackey 4a4d48ddb9 Fix error when logging exception. (#756)
Reviewed-on: cerc-io/stack-orchestrator#756
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-22 00:11:06 +00:00
telackey 08438b1cd5 More logging for webapp deployments (#755)
Reviewed-on: cerc-io/stack-orchestrator#755
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 23:48:52 +00:00
telackey 9f1dd284a5 Better error logging for registry deployments. (#754)
We were missing errors related to registration, this should fix that.

Reviewed-on: cerc-io/stack-orchestrator#754
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 20:12:52 +00:00
telackey db4986dcc6 snowballtool-base backend stack (#751)
This adds a stack for the backend from snowball/snowballtools-base.

Reviewed-on: cerc-io/stack-orchestrator#751
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 04:45:46 +00:00
telackey 65f05ea80c Run a manual build script, if present. (#750)
If the tree has a 'build-webapp.sh' script, use that.

Reviewed-on: cerc-io/stack-orchestrator#750
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-21 00:20:50 +00:00
telackey 3c3e582939 Minor envsubst improvements. (#746)
Minor fixes to envsubst for webapps.  Somewhat specially treated is `LACONIC_HOSTED_CONFIG_homepage` which can be used to replace the homepage in package.json.  With react, this gets an extra `/` though, which we need to remove.

Reviewed-on: cerc-io/stack-orchestrator#746
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-16 04:11:09 +00:00
telackey 65d67dba10 Fix k8s and enable it by default on PRs (#742)
Reviewed-on: cerc-io/stack-orchestrator#742
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-14 23:50:09 +00:00
telackey b22c72e715 For k8s, use provisioner-managed volumes when an absolute host path is not specified. (#741)
In kind, when we bind-mount a host directory it is first mounted into the kind container at /mnt, then into the pod at the desired location.

We accidentally picked this up for full-blown k8s, and were creating volumes at /mnt.  This changes the behavior for both kind and regular k8s so that bind mounts are only allowed if a fully-qualified path is specified.  If no path is specified at all, a default storageClass is assumed to be present, and the volume managed by a provisioner.

Eg, for kind, the default provisioner is: https://github.com/rancher/local-path-provisioner

```
stack: test
deploy-to: k8s-kind
config:
  test-variable-1: test-value-1
network:
  ports:
    test:
     - '80'
volumes:
  # this will be bind-mounted to a host-path
  test-data-bind: /srv/data
  # this will be managed by the k8s node
  test-data-auto:
configmaps:
  test-config: ./configmap/test-config
```

Reviewed-on: cerc-io/stack-orchestrator#741
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-14 21:45:01 +00:00
telackey c9444591f5 Fix default webapp port number. (#740)
Reviewed-on: cerc-io/stack-orchestrator#740
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-09 01:20:41 +00:00
telackey 903f3b10e2 Add support for annotations and labels in spec. (#739)
```
stack: webapp-deployer-backend
deploy-to: k8s
annotations:
  foo.bar.annot/{name}: baz
labels:
  a.b.c/{name}.blah: "value"
```

Reviewed-on: cerc-io/stack-orchestrator#739
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-09 00:11:07 +00:00
telackey 72ed2eb91a Fix bad test in tag check. (#738)
Reviewed-on: cerc-io/stack-orchestrator#738
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-08 20:38:41 +00:00
telackey 3309782439 Refactor 2024-02-08 00:47:46 -06:00
telackey 4b3b3478e7 Switch to Docker-style limits 2024-02-08 00:43:41 -06:00
telackey 2a9955055c debug 2024-02-07 16:56:35 -06:00
telackey 8964e1c0fe Add resource limit options to spec. 2024-02-07 16:48:02 -06:00
telackey d2ebb81d77 Tags for undeploy (#734)
```
  --include-tags TEXT             Only include requests with matching tags
                                  (comma-separated).
  --exclude-tags TEXT             Exclude requests with matching tags (comma-
                                  separated).
```

Reviewed-on: cerc-io/stack-orchestrator#734
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-07 21:45:16 +00:00
telackey 4a981d8d2e Fix repo URL (#733)
Needs a '/' (http) not ':' (ssh).

Reviewed-on: cerc-io/stack-orchestrator#733
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-07 18:52:07 +00:00
telackey 937b983ec9 Update links from github.com to git.vdb.to (#732)
Update links and references to github.com to git.vdb.to.

Also enable the flake8 lint action in gitea.

Reviewed-on: cerc-io/stack-orchestrator#732
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-06 22:42:44 +00:00
telackeyanddboreham 36bb068983 Add ConfigMap test. (#726)
* Add ConfigMap test.

* eof

* Minor tweak

* Trigger test

---------

Co-authored-by: David Boreham <david@bozemanpass.com>
2024-02-05 14:15:11 -06:00
telackey 2fcd416e29 Basic webapp deployer stack. (#722) 2024-02-02 19:05:15 -07:00
telackey 6629017d6a Support other webapp types (react, static). (#721)
* Support other webapp types (react, static).
2024-02-02 18:04:06 -06:00
telackey b398050787 Don't include volumes in spec if we don't have any. (#720) 2024-01-31 15:11:32 -06:00
telackey 12ec1bec43 Add ConfigMap support for k8s. (#714)
* Minor fixes for deploying with k8s and podman.

* ConfigMap support
2024-01-30 23:09:48 -06:00
telackey 62af03077f Add deployed/error status output to the state file. (#719)
* More status info
* Up default resource limits.
* Need ps
2024-01-30 22:13:45 -06:00
telackey 428b05158e Fix DnsRecord ownership check. (#718)
* Fix DnsRecord ownership check.

* Var names
2024-01-30 13:31:59 -06:00
telackey 554f05de87 Fix pip3 error (#709) 2024-01-22 21:06:15 -06:00
telackey 0587813dd0 Create next.config.js if it is missing. (#701)
* Create next.config.js if it is missing.

* Add comment.
2024-01-15 12:12:59 -06:00
telackey 4b3ea7c30f Update independent act-runner stack to use custom act as well. (#702)
* Update independent act-runner stack to use custom act as well.

* Remove branches which are not needed or already merged.
2024-01-15 12:10:48 -06:00
telackey b83030f63b Use custom act with gitea. (#700)
* Use custom act with gitea.

* Make sure wget is available

* Fix repo url
2024-01-09 22:53:43 -06:00
telackey 837e443800 Support application removal requests. (#697)
* Support application removal request.

* Git should never prompt when deploying a webapp
2023-12-21 18:05:40 -06:00
telackey 997496b8a5 Update script for new nextjs build output. (#691) 2023-12-14 19:47:30 -06:00
telackey 61f2884505 Reduce base image size (first round of improvements) (#690) 2023-12-14 17:46:03 -06:00
telackey 27a14737f8 Make the container tag based on the deployment path. (#688) 2023-12-14 09:49:21 -06:00
telackey 1f4eb57069 Add --dry-run option (#686) 2023-12-13 22:56:40 -06:00
telackey 88f66a3626 Add deployment update and deploy-webapp-from-registry commands. (#676) 2023-12-13 21:02:34 -06:00
telackey 077ea80c70 Add deployment status command and fix k8s output for deployment ps (#679) 2023-12-06 09:27:47 -07:00
telackey 2dd54892a1 Allow specifying the webapp tag explicitly (#675) 2023-12-04 21:39:16 -06:00
telackey ab0e70ed83 Change path portion of unique cluster name to point to compose file, not argv[0]. (#678) 2023-12-04 13:39:14 -06:00
telackey 2173e7ce6a If the next version is unsupported, print a big warning and try higher version. (#674) 2023-11-30 12:33:06 -06:00
telackey 03a3645b3c Add --port option to run-webapp. (#667)
* Add --port option to run-webapp

* Fixed merge

* lint
2023-11-29 11:32:28 -06:00
telackey 1a37255c18 Tweak laconicd config to allow setting endpoint port and to make the fixturenet restartable. (#660)
* Endpoint includes port

* Make it restartable

* Don't try to remove the mounted directory

* Make copy of init.sh
2023-11-22 11:31:30 -06:00
telackey 5c80887215 Fix missing tty parameter. (#653) 2023-11-16 12:58:03 -06:00
telackey 1e9d24a8ce Update webapp.md 2023-11-15 12:52:34 -06:00
telackey 9900565714 Update webapp.md 2023-11-15 12:48:58 -06:00
telackey a13f841f34 Update webapp.md 2023-11-15 12:37:30 -06:00
telackey d37f80553d Add webapp doc (#652) 2023-11-15 12:28:07 -06:00
telackey 2059d67dca Add run-webapp command. (#651) 2023-11-15 10:54:27 -07:00
telackey 4ae4d3b61d Print docker container logs in webapp test. (#649) 2023-11-14 17:30:01 -06:00
telackey 9687d84468 646: Add error message for webapp startup hang (#647)
This fixes three issues:

1. #644 (build output)
2. #646 (error on startup)
3. automatic env quote handling (related to 2)


For the build output we now have:

```
#################################################################

Built host container for /home/telackey/tmp/iglootools-home with tag:

    cerc/iglootools-home:local

To test locally run:

    docker run -p 3000:3000 cerc/iglootools-home:local
```

For the startup error, it was hung waiting for the "success" message from the next generate output (itself a workaround for a nextjs bug fixed by this PR we submitted: https://github.com/vercel/next.js/pull/58276).

I added a timeout which will cause it to wait up to a maximum _n_ seconds before issuing:

```
ERROR: 'npm run cerc_generate' exceeded CERC_MAX_GENERATE_TIME.
```

On the quoting itself, I plan on adding a new run-webapp command, but I realized I had a decent spot to do effect the quote replacement on-the-fly after all when I am already escaping the values for insertion/replacement into JS.

The "dequoting" can be disabled with `CERC_RETAIN_ENV_QUOTES=true`.
2023-11-14 16:07:26 -06:00
telackey f1f618c57a Don't change the next.js version by default. (#640) 2023-11-13 11:56:04 -06:00
telackey 414b887036 Allow setting build tool (npm/yarn) and next.js version. (#639)
* Allow setting build tool (npm/yarn) and next.js version.
2023-11-10 17:44:25 -06:00
telackey 042b413598 Support the case where webpack config is already present next.config.js (#631)
* Support the case where webpack config is already present next.config.js

* Update scripts for experimental-compile/experimental-generate
2023-11-08 23:44:48 -06:00
telackey 36e13f7199 Remove test output from tree. (#628) 2023-11-07 23:10:32 -06:00
telackey d9bcc088a8 Enable webapp test in GitHub CI. (#627) 2023-11-07 18:27:08 -06:00
telackey 660326f713 Add new build-webapp command and related scripts and containers. (#626)
* Add new build-webapp command and related scripts and containers.
2023-11-07 18:15:04 -06:00
telackey fd5779f967 Fix KeyError accessing config. (#620) 2023-10-31 12:29:19 -05:00
telackey 8cac598679 Split act-runner into its own pod and offer as a distinct stack. (#612)
* Split act-runner into its own pod and offer as a distinct stack.
2023-10-27 13:57:13 -05:00
telackey 0f5b1a097b Add plugeth to chain-chunker stack (needed for new verify option). (#610) 2023-10-25 14:47:53 -05:00
telackeyanddboreham 20d633f81c Plugeth-based full mainnet stack. (#592)
* Plugeth-based full mainnet stack.

---------

Co-authored-by: David Boreham <david@bozemanpass.com>
2023-10-25 14:42:52 -05:00
telackey 3291c16466 Update stacks for migrated laconic repos. (#594) 2023-10-20 11:59:06 -05:00
telackey 8246e3551f 53: Make sure curl and jq are present in the container. (#585) 2023-10-17 13:36:43 -05:00
telackey 80b0c07736 Open ports for 2nd geth instance and add missing lcli param. (#555) 2023-10-03 19:52:23 -05:00
telackey 6fa3ca2b6d Update from github.com to git.vdb.to where applicable. (#553)
* Update from github.com to git.vdb.to for many repos.

* Use ipld-eth-server@v1.11.6-statediff-v5 for most stacks

* Specify go-ethereum branch/tag
2023-10-03 13:55:33 -05:00
telackey ed9c253f61 Simplified, improved default dashboard for mainnet-eth-metrics. (#536)
* Simplified, improved default dashboard for mainnet-eth-metrics.

* Add missing lighthouse prometheus config.
2023-09-19 20:29:08 -05:00
telackey bdf4f069b4 Include all of app/data regardless of depth. (#535) 2023-09-19 20:26:48 -05:00
telackey d3c26948fc Add eth-api-proxy to mainnet-eth stack. (#514)
* Add eth-api-proxy to mainnet-eth stack.
2023-08-29 16:49:09 -05:00
telackey 7f9b556f9d Update mainnet-eth stack to include auth and metrics. (#492) 2023-08-29 11:32:37 -05:00
telackey f55a14bd6c Create new base container for pre-compiled webapps. (#507)
* Create new base container for pre-compiled webapps.

* Set the webapp dir appropriately.
2023-08-22 13:32:45 -05:00
telackey 18e45af463 Add port config to deployment spec. (#498)
* Add port config to deployment spec.
2023-08-11 15:25:54 -05:00
telackey a5945c9e59 Back to Github-based actions. (#453)
* Back to Github-based actions.

* Use fork.

* Use new release plugin tag.
2023-07-19 20:40:04 -06:00
telackey 809889f9f0 Update plugeth stack for plugeth-statediff plugin. (#443)
* Use latest plugeth-statediff.

* Change plugeth path

* EOL

* CERC_PLUGINS_DIR
2023-06-29 15:11:28 -06:00
telackey 26ff7a969c Fix plugeth build. (#423) 2023-06-05 21:10:17 -05:00
telackey a8e198ad55 Allow configuring the number of statediff workers. (#422)
* Allow configuring the number of statediff workers.

* Leave logging alone
2023-06-05 18:16:42 -05:00
telackey 23a336020c Make a separate lighthouse container for the plugeth fixturenet. (#412)
* Make a separate lighthouse container for the plugeth fixturenet.
2023-05-26 16:57:15 -05:00
telackey 6ec55ba460 Add a plugeth-based version of the fixturenet (#411)
* plugeth version of the fixturenet

* Use pre-built plugeth.
2023-05-25 11:21:08 -05:00
telackey aa702737ef Fix 397 by pegging alpine version. 2023-05-19 11:26:09 -05:00
telackey 03f6d027f9 Minor script cleanup. (#372)
Former-commit-id: 323ca3b238
2023-04-26 23:26:50 -05:00
telackey d8522211f4 Add script for exporting ethdb from fixturenet. (#370)
* Add script for exporting ethdb from fixturenet.

* Update README

* Script

Former-commit-id: 7a607c2994
2023-04-26 00:13:35 -05:00
telackey 80c16d2ced Update for latest act_runner. (#316)
Former-commit-id: c7c3cbde8e
2023-04-11 15:05:08 -05:00
telackey 287468b7c0 Update run script to support COPY and WebSockets. (#275)
Former-commit-id: 4da69ebf4c
2023-04-03 14:59:31 -05:00
telackey 788b214116 Add Gitea action support via act_runner.
Former-commit-id: 74077d7704
2023-03-24 22:24:40 -05:00
telackey bc5eff71b5 Use latest keycloak plugin. (#173)
Former-commit-id: 8645ab0619
2023-02-01 16:02:49 -06:00
telackey bdde1b2c9e Include all eth API operation metrics. (#162)
Former-commit-id: ce0c7ee9ba
2023-01-26 13:53:02 -06:00
telackey b5c7ad114c Integrate grafana/prometheus with fixturenet-eth. (#156)
* Integrate grafana/prometheus with fixturenet-eth.

* Integrate grafana/prometheus with fixturenet-eth.

* Revert port change

* EOL

* EOL

* Simplify ipld-eth-server config.

Former-commit-id: c279bd49a4
2023-01-25 11:50:08 -06:00
telackey c77cc4e111 Update keycloak to include X-User-Id header. (#147)
Former-commit-id: 65cdba7705
2023-01-23 10:39:15 -06:00
telackey 08d5c22f32 Enable metrics endpoint in fixturenet.
Former-commit-id: 0ea29bc480
2023-01-17 12:04:28 -06:00
telackey b13300ca81 Add CERC_STATEDIFF_DB_LOG_STATEMENTS=true|false option. (#120) 2023-01-13 16:33:59 -06:00