225 Commits

Author	SHA1	Message	Date
Thomas E Lackey	60d34217f8	More logging for webapp deployment (#923) ... ``` cerc-webapp-deployer: ############ DEPLOY ############# cerc-webapp-deployer: 2024-08-15 02:13:08.321991 - - 0:00:00.000031 (step): Discovering deployment requests... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRequest cerc-webapp-deployer: 2024-08-15 02:13:08.815428 - - 0:00:00.493420 (step): Loading known requests from /srv/deployments/autodeploy.state... cerc-webapp-deployer: 2024-08-15 02:13:08.815626 - - 0:00:00.000158 (step): BEGIN: Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi cerc-webapp-deployer: 2024-08-15 02:13:08.815645 - - 0:00:00.000008 (step): Skipping request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815653 - - 0:00:00.000005 (step): DONE Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815664 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue cerc-webapp-deployer: 2024-08-15 02:13:08.815674 - - 0:00:00.000006 (step): Skipping request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815684 - - 0:00:00.000004 (step): DONE Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815692 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne cerc-webapp-deployer: 2024-08-15 02:13:08.815705 - - 0:00:00.000007 (step): Skipping request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815714 - - 0:00:00.000005 (step): DONE Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815724 - - 0:00:00.000004 (step): BEGIN: Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua cerc-webapp-deployer: 2024-08-15 02:13:08.815733 - - 0:00:00.000005 (step): Skipping request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815743 - - 0:00:00.000005 (step): DONE Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815751 - - 0:00:00.000004 (step): BEGIN: Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy cerc-webapp-deployer: 2024-08-15 02:13:08.815761 - - 0:00:00.000006 (step): Skipping request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815770 - - 0:00:00.000005 (step): DONE Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815779 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy cerc-webapp-deployer: 2024-08-15 02:13:08.815791 - - 0:00:00.000007 (step): Skipping request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815800 - - 0:00:00.000004 (step): DONE Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815808 - - 0:00:00.000004 (step): Discovering existing app deployments... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRecord cerc-webapp-deployer: 2024-08-15 02:13:09.330655 - - 0:00:00.514858 (step): Discovering deployment removal and cancellation requests... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRemovalRequest cerc-webapp-deployer: 2024-08-15 02:13:09.825145 - - 0:00:00.494460 (step): Found 0 unsatisfied request(s) to process. cerc-webapp-deployer: ############ DEPLOY SUCCESS ############# ``` Reviewed-on: #923 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-15 02:57:47 +00:00
Thomas E Lackey	952389abb0	Add option to recreate deployments rather than update them. (#920) ... cherry-pick from #912 Reviewed-on: #920 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>	2024-08-14 20:14:40 +00:00
Thomas E Lackey	5c275aa622	Defensively handle errors examining app requests. (#922) ... Related to cerc-io/webapp-deployment-status-api#10 There are two issues in that. One is that the output probably changed recently, whether in the client or server, where no matching record is found by ID (Note this is specific to `laconic record get --id <v>` and does not seem to apply to the similar command to retrieve a record by name, `laconic name resolve <n>`). Rather than returning `[]` it is now returning `[ null ]`. This cause us to think there *was* an application record found, and we attempt to treat the `null` entry like an Application object. That's fixed by filtering out null responses, which is a good precaution for the deployer, though I think it makes sense to ask whether this new behavior by the client/server is correct. Seems suspicious. The other issue is that all the defensive checks we had in place to deal with broken/bad AppDeploymentRequests were around the _build_. This error was coming much earlier, merely when parsing and examining the request to see if it needed to be handled at all. I have now added similar defensive error handling around that portion of the code. Reviewed-on: #922 Reviewed-by: zramsay <zramsay@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-14 18:04:31 +00:00
Thomas E Lackey	8576137557	Convert port to string. (#919) ... The str type check doesn't work if the port is a ruamel.yaml.scalarstring.SingleQuotedScalarString or ruamel.yaml.scalarstring.DoubleQuotedScalarString Reviewed-on: #919 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-14 00:25:35 +00:00
David Boreham	65c1cdf6b1	Fix crash if port has int type in yaml (#918) ... Reviewed-on: #918 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 20:47:09 +00:00
David Boreham	265699bc38	Allow to disable kind cluster management for testing (#915) ... Reviewed-on: #915 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 17:48:14 +00:00
David Boreham	4a7670a5d6	Open the json-rpc port (#916) ... Reviewed-on: #916 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 17:47:56 +00:00
Thomas E Lackey	6087e1cd31	Copy config under a volume for Docker (similar to a ConfigMap for K8S). (#914) ... This emulates the K8S ConfigMap behavior on Docker by using a regular volume. If a directory exists under `config/` which matches a named volume, the contents will be copied to the volume on `create` (provided the destination volume is empty). That is, rather than a ConfigMap, it is essentially a "config volume". For example, with a compose file like: ``` version: '3.7' services: caddy: image: cerc/caddy-ethcache:local restart: always volumes: - caddyconfig:/etc/caddy:ro volumes: caddyconfig: ``` And a directory: ``` ❯ ls stack-orchestrator/config/caddyconfig/ Caddyfile ``` After `laconic-so deploy create --spec-file caddy.yml --deployment-dir /srv/caddy` there will be: ``` ❯ ls /srv/caddy/data/caddyconfig Caddyfile ``` Mounted at `/etc/caddy` Reviewed-on: #914 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-10 02:32:21 +00:00
Thomas E Lackey	1def279d26	Support multiple NodePorts, static NodePort mapping, and add 'replicas' spec option (#913) ... NodePort example: ``` network: ports: caddy: - 1234 - 32020:2020 ``` Replicas example: ``` replicas: 2 ``` This also adds an optimization for k8s where if a directory matching the name of a configmap exists in beneath config/ in the stack, its contents will be copied into the corresponding configmap. For example: ``` # Config files in the stack ❯ ls stack-orchestrator/config/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example # ConfigMap in the spec ❯ cat foo.yml | grep config ... configmaps: caddyconfig: ./configmaps/caddyconfig # Create the deployment ❯ laconic-so --stack ~/cerc/caddy-ethcache/stack-orchestrator/stacks/caddy-ethcache deploy create --spec-file foo.yml # The files from beneath config/<config_map_name> have been copied to the ConfigMap directory from the spec. ❯ ls deployment-001/configmaps/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example ``` Reviewed-on: #913 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-09 02:32:06 +00:00
David Boreham	aef5986135	Allow gentx-files to be omitted	2024-08-07 14:11:06 -06:00
David Boreham	7590d6e237	Add stage 1 support	2024-08-07 11:28:10 -06:00
David Boreham	573f99dbbe	Listen on 0.0.0.0 (#909) ... Reviewed-on: #909 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-02 14:06:06 +00:00
David Boreham	a674d13493	Laconicd needs to be told its currency	2024-08-01 21:09:30 -06:00
David Boreham	6c91b87348	Fix for sh as shell not bash	2024-07-31 14:30:53 -06:00
David Boreham	7d18334953	Mainnet-laconic stack fixes for laconicd2 (#904) ... Reviewed-on: #904	2024-07-31 13:51:28 +00:00
Prathamesh Musale	79c1c5ed99	Update fixturenet-laconicd stack to use alnt denom (#902) ... Part of [laconicd testnet validator enrollment](https://www.notion.so/laconicd-testnet-validator-enrollment-6fc1d3cafcc64fef8c5ed3affa27c675) Reviewed-on: #902 Reviewed-by: ashwin <ashwin@noreply.git.vdb.to> Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-07-31 13:27:54 +00:00
zramsay	dfedd9e9ff	rename laconic-sdk to registry-sdk (#897) ... Co-authored-by: zramsay <zach@bluecollarcoding.ca> Reviewed-on: #897 Reviewed-by: ashwin <ashwin@noreply.git.vdb.to> Co-authored-by: zramsay <zramsay@noreply.git.vdb.to> Co-committed-by: zramsay <zramsay@noreply.git.vdb.to>	2024-07-31 08:01:02 +00:00
Thomas E Lackey	f64683f03b	Use a more flexible mechanism to inject config into next.config.js for runtime env. (#901) ... Instead of attempting to rewriting the nextConfig file directly, inject a helper function to add the config we need. Reviewed-on: #901 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-07-31 03:22:23 +00:00
Thomas E Lackey	913c3a8557	Back to v2 now that we have a working webapp deployer build again. (#896) ... Reviewed-on: #896 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-07-27 18:59:42 +00:00
Thomas E Lackey	2f5b0cdd13	Revert recent laconicd deployment changes to restore production webapp deployer function. (#895) ... Reviewed-on: #895	2024-07-27 17:04:03 +00:00
Thomas E Lackey	432bd4113d	881: Support next.config.mjs (#890) ... Reviewed-on: #890	2024-07-25 16:47:17 +00:00
Prathamesh Musale	b26698b756	Update fixturenet-laconicd stack for renaming changes (#891) ... Part of [Rename laconic2d to laconicd](https://www.notion.so/Rename-laconic2d-to-laconicd-9028d0c020d24d1288e92ebcb773d7a7) Handles #882, #889 Reviewed-on: #891 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-07-25 08:50:15 +00:00
Thomas E Lackey	01deac78c4	880: Support new compile/generate syntax for next >=14.2.0 (#886) ... Fix for #880 to support the next compile/generate syntax. Reviewed-on: #886 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-07-24 20:20:16 +00:00
zramsay	80cff73344	crn --> lrn	2024-07-23 20:20:01 -04:00
zramsay	21b1270d27	fix lint	2024-07-23 20:16:16 -04:00
zramsay	008389dcd8	cns --> registry and other fixes	2024-07-23 20:10:06 -04:00
David Boreham	c81fb9581a	Fix stack path check (#877) ... Reviewed-on: #877 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-19 17:16:40 +00:00
David Boreham	83397bbae4	Enable cors in laconicd http services (#875) ... Reviewed-on: #875 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-15 05:23:18 +00:00
David Boreham	7fb9ccdfd8	Work around explorer host name sensitivity	2024-07-14 20:51:11 -06:00
David Boreham	2bad59dfcd	Add missing file to explorer container (#873) ... Reviewed-on: #873 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-14 17:40:35 +00:00
David Boreham	13d04e9256	Integrate ping-pub explorer (#872) ... Reviewed-on: #872	2024-07-13 14:24:23 +00:00
zramsay	1f017c9066	increase CERC_MAX_GENERATE_TIME for webapps (#857) ... sort of addresses #856 Co-authored-by: zramsay <zach@bluecollarcoding.ca> Co-authored-by: David Boreham <dboreham@noreply.git.vdb.to> Reviewed-on: #857 Co-authored-by: zramsay <zramsay@noreply.git.vdb.to> Co-committed-by: zramsay <zramsay@noreply.git.vdb.to>	2024-07-12 19:01:35 +00:00
VPhung24	3b9422095c	Add support for bun as a webapp package manager (#800) ... This is working off pull request "[Add support for pnpm as a webapp build tool. #767](https://git.vdb.to/cerc-io/stack-orchestrator/pulls/767/files)" that adds `pnpm` package manager support for `nextjs` & `webapps`. `bun` default build output directory (defined as `CERC_BUILD_OUTPUT_DIR`) is `dist` which should already be handled with `pnpm` support in the previously mentioned [pull request](https://git.vdb.to/cerc-io/stack-orchestrator/pulls/767/files) Installing `bun` using `npm` following our previous `pnpm` installation ```zsh npm install -g bun ``` We'll be using `bun` as a package manager that works with `Node.js` projects as defined in bun's [docs](https://bun.sh/docs/cli/install) > The bun CLI contains a Node.js-compatible package manager designed to be a dramatically faster replacement for npm, yarn, and pnpm. It's a standalone tool that will work in pre-existing Node.js projects; if your project has a package.json, bun install can help you speed up your workflow. To test `next.js` apps using `node.js` and compatibility with all four packager managers -- `npm`, `yarn`, `pnpm`, and `bun` -- use the branches of snowball's [nextjs-package-manager-example-app](https://git.vdb.to/snowball/nextjs-package-manager-example-app) repo: `nextjs-package-manager/npm`, `nextjs-package-manager/yarn`, `nextjs-package-manager/pnpm`, `nextjs-package-manager/bun`. Co-authored-by: Vivian Phung <dev+github@vivianphung.com> Co-authored-by: David Boreham <dboreham@noreply.git.vdb.to> Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/800 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: VPhung24 <vphung24@noreply.git.vdb.to> Co-committed-by: VPhung24 <vphung24@noreply.git.vdb.to>	2024-07-09 18:00:14 +00:00
Roy Crihfield	36d4969b2d	Fixes for external stack deployment (#851) ... Fixes - stack path resolution for `build` - external stack path resolution for deployments - "extra" config detection - `deployment ports` command - `version` command in dist or source install (without build_tag.txt) - `setup-repos`, so it won't die when an existing repo is not at a branch or exact tag Used in cerc-io/fixturenet-eth-stacks#14 Reviewed-on: #851 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>	2024-07-09 15:37:35 +00:00
David Boreham	62f7825ec2	Remove quotes from git config	2024-07-05 09:55:14 -06:00
David Boreham	6d24d4a7e6	Set github auth token if present (#868) ... Reviewed-on: #868 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-05 12:27:22 +00:00
David Boreham	f06e5f9a2a	Don't try to tag remote images (#866) ... Reviewed-on: #866 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-04 23:51:06 +00:00
David Boreham	c3a1402042	Derive the webapp host container id from stable data (#865) ... Reviewed-on: #865 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-04 22:38:07 +00:00
David Boreham	ca5fffaed5	Add logging to webapp deployer (#863) ... Helps with diagnosing failures and odd behavior seen in the deployer in production. Reviewed-on: #863 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-07-04 19:46:42 +00:00
David Boreham	df776c1b4c	Install git for apps that want to get their commit sha (#854) ... Reviewed-on: #854 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <dboreham@noreply.git.vdb.to> Co-committed-by: David Boreham <dboreham@noreply.git.vdb.to>	2024-06-25 05:03:49 +00:00
David Boreham	0eaa5b8f09	Fix argument errors in command code	2024-06-24 14:15:46 -06:00
David Boreham	fd15252c3f	Fixes for current SO objects	2024-06-24 13:41:15 -06:00
David Boreham	2364924a59	Fix mainnet laconic deploy setup	2024-06-21 19:24:33 -06:00
Nabarun	a223797b4a	Update graph-node dashboard to show individual subgraph increase in query count (#846) ... Part of [Metrics and logging for GQL queries in watcher](https://www.notion.so/Metrics-and-logging-for-GQL-queries-in-watcher-928c692292b140a2a4f52cda9795df5e) Reviewed-on: #846 Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Co-committed-by: Nabarun <nabarun@deepstacksoft.com>	2024-06-20 09:27:23 +00:00
Nabarun	b8004e9870	Add Grafana panels for graph-node subgraph GQL queries (#845) ... Part of [Deploy v2 and updated v3 sushiswap subgraphs](https://www.notion.so/Deploy-v2-and-updated-v3-sushiswap-subgraphs-e331945fdeea487c890706fc22c6cc94) Reviewed-on: #845 Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Co-committed-by: Nabarun <nabarun@deepstacksoft.com>	2024-06-19 10:40:54 +00:00
Nabarun	842d999792	Add alert rules for secured secured-finance subgraph watcher (#842) ... Part of [Generate secured-finance subgraph watcher with codegen](https://www.notion.so/Generate-secured-finance-subgraph-watcher-with-codegen-2923413e0af54ea787c5435d6966f3bb) - Update watcher dashboard labels Reviewed-on: #842 Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Co-committed-by: Nabarun <nabarun@deepstacksoft.com>	2024-06-18 12:28:02 +00:00
David Boreham	bf1eccd486	Fix image tag name	2024-06-13 08:31:45 -06:00
David Boreham	3fb025b5c9	Make remote image tags unique to the deployment (#838) ... Reviewed-on: #838 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-06-13 03:26:58 +00:00
Prathamesh Musale	4acb06325b	Update watcher dashboard and config templates (#835) ... Part of [Metrics and logging for GQL queries in watcher](https://www.notion.so/Metrics-and-logging-for-GQL-queries-in-watcher-928c692292b140a2a4f52cda9795df5e) - Update watcher config templates after config refactoring - Mount watcher GQL query log files on volumes - Update watcher dashboard to - add a panel to show latest processed block number - use latest processed block from sync status for diff values Co-authored-by: Nabarun <nabarun@deepstacksoft.com> Reviewed-on: #835 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-06-12 11:52:51 +00:00
David Boreham	9a1d3bb0f1	Remove files migrated to external repo	2024-06-06 20:44:51 -06:00