1123 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Prathamesh Musale	c24ff99e62	Resolve app name when verifying auction	2024-10-09 16:39:49 +05:30
Prathamesh Musale	4e4d498913	Verify app in deployment request with auction when deploying	2024-10-09 12:00:26 +05:30
Prathamesh Musale	67599898dc	Remove auction already used check	2024-10-09 11:24:17 +05:30
Prathamesh Musale	78874a4d7f	Cross check app in deployment request and auction record	2024-10-09 11:23:10 +05:30
Prathamesh Musale	ca903b55be	Remove debug log	2024-10-08 14:14:40 +05:30
Prathamesh Musale	9810b46f7f	Fix gpg key usage when uploading config to multiple deployers	2024-10-08 12:53:05 +05:30
Prathamesh Musale	2bf1ed5653	Add default auction params for deployment auction	2024-10-07 14:26:47 +05:30
Prathamesh Musale	e89fcb6687	Add a command to request an application undeployment	2024-10-07 12:42:47 +05:30
Prathamesh Musale	5a237b8f8d	Prefer deployers with names set	2024-10-07 09:47:17 +05:30
Prathamesh Musale	3349e8e0b4	Set config ref after uploading it	2024-10-04 15:56:55 +05:30
Prathamesh Musale	6168612ddb	Handle lint errors	2024-10-03 18:38:22 +05:30
Prathamesh Musale	48de1b0539	Fix target deployer resolution	2024-10-03 17:50:18 +05:30
Prathamesh Musale	d1c68448c8	Check if auction is already used for a deployment	2024-10-03 12:45:49 +05:30
Prathamesh Musale	e9de97f371	Update handling for deployment requests with auction	2024-10-03 10:00:31 +05:30
Prathamesh Musale	0694c31807	Handle requests with auction when deploying webapps from registry	2024-10-01 19:25:17 +05:30
Prathamesh Musale	74edf16d14	Convert amounts to int for comparison	2024-10-01 18:44:51 +05:30
Prathamesh Musale	19e1ce211b	Fix imports	2024-10-01 17:47:09 +05:30
Prathamesh Musale	08109355bc	Handle deployment auction stages	2024-10-01 17:37:21 +05:30
Prathamesh Musale	d79834a182	Add a command to handle deployment auctions	2024-10-01 15:11:52 +05:30
Prathamesh Musale	02fac0feb7	Match auction winner address to deployer payment address	2024-09-30 12:22:45 +05:30
Prathamesh Musale	b84527d9c6	Set auction id in deployment request	2024-09-30 12:22:04 +05:30
Prathamesh Musale	61e1116e33	Update request-webapp-deployment command to handle deployment auction	2024-09-27 16:06:50 +05:30
Prathamesh Musale	8051a3fc6b	Add a command to create and publish a app deployment auction	2024-09-26 19:08:06 +05:30
Prathamesh Musale	2319699c68	Add a util method to create a provider auction	2024-09-26 12:19:14 +05:30
Thomas E Lackey	f1fdc48aaa	Work around this bug: https://github.com/python/cpython/pull/14064 (#941) ... Otherwise we sometimes see errors like: ``` cerc-webapp-deployer: File "/root/.shiv/laconic-so_0f937aa98c2748ef9af8585d6f441dbc01546ace0d6660cbb159d1e5040aeddf/site-packages/stack_orchestrator/deploy/webapp/deploy_webapp_from_registry.py", line 671, in command cerc-webapp-deployer: shutil.rmtree(tempdir) cerc-webapp-deployer: File "/usr/lib/python3.10/shutil.py", line 725, in rmtree cerc-webapp-deployer: _rmtree_safe_fd(fd, path, onerror) cerc-webapp-deployer: File "/usr/lib/python3.10/shutil.py", line 681, in _rmtree_safe_fd cerc-webapp-deployer: onerror(os.unlink, fullname, sys.exc_info()) cerc-webapp-deployer: File "/usr/lib/python3.10/shutil.py", line 679, in _rmtree_safe_fd cerc-webapp-deployer: os.unlink(entry.name, dir_fd=topfd) cerc-webapp-deployer: FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra' ``` Reviewed-on: #941 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-28 23:17:13 +00:00
Thomas E Lackey	a54072de6c	Add --config-ref flag. (#939) ... Add a flag to re-use config. Reviewed-on: #939 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-28 17:32:52 +00:00
Thomas E Lackey	fa21ff2627	Support uploaded config, add 'publish-webapp-deployer' and 'request-webapp-deployment' commands (#938) ... This adds two new commands: `publish-webapp-deployer` and `request-webapp-deployment`. `publish-webapp-deployer` creates a `WebappDeployer` record, which provides information to requestors like the API URL, minimum required payment, payment address, and public key to use for encrypting config. ``` $ laconic-so publish-deployer-to-registry \ --laconic-config ~/.laconic/laconic.yml \ --api-url https://webapp-deployer-api.dev.vaasl.io \ --public-key-file webapp-deployer-api.dev.vaasl.io.pgp.pub \ --lrn lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io \ --min-required-payment 100000 ``` `request-webapp-deployment` simplifies publishing a `WebappDeploymentRequest` and can also handle automatic payment, and encryption and upload of configuration. ``` $ laconic-so request-webapp-deployment \ --laconic-config ~/.laconic/laconic.yml \ --deployer lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io \ --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \ --env-file ~/yaml/hello.env \ --make-payment auto ``` Related changes are included for the deploy/undeploy commands for decrypting and using config, using the payment address from the WebappDeployer record, etc. Reviewed-on: #938	2024-08-27 19:55:06 +00:00
Prathamesh Musale	33d395e213	Add package registry stack instructions (#937) ... - The instructions to `Deploy Gitea Package Registry` from build-support [readme](https://git.vdb.to/deep-stack/stack-orchestrator/src/branch/pm-update-registry-steps/stack_orchestrator/data/stacks/build-support#2-deploy-gitea-package-registry) don't seem to be in a working state - Updated `package-registry` stack instructions to use deployment pattern Reviewed-on: #937 Reviewed-by: ashwin <ashwin@noreply.git.vdb.to> Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>	2024-08-23 09:42:44 +00:00
Thomas E Lackey	75ff60752a	Require payment for app deployment requests. (#928) ... Adds three new options for deployment/undeployment: ``` "--min-required-payment", help="Requests must have a minimum payment to be processed", "--payment-address", help="The address to which payments should be made. Default is the current laconic account.", "--all-requests", help="Handle requests addressed to anyone (by default only requests to my payment address are examined).", ``` In this mode, requests should be designated for a particular address with the attribute `to` and include a `payment` attribute which is the tx hash for the payment. The deployer will confirm the payment (to the right account, right amount, not used before, etc.) and then proceed with the deployment or undeployment. Reviewed-on: #928 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-21 14:39:20 +00:00
David Boreham	44b9709717	Use Laconic version of ping-pub (#930) ... Reviewed-on: #930 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-20 17:44:00 +00:00
David Boreham	e56da7dcc1	Add support for k8s pod to node affinity and taint toleration (#917) ... Reviewed-on: #917 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-15 20:32:58 +00:00
Thomas E Lackey	60d34217f8	More logging for webapp deployment (#923) ... ``` cerc-webapp-deployer: ############ DEPLOY ############# cerc-webapp-deployer: 2024-08-15 02:13:08.321991 - - 0:00:00.000031 (step): Discovering deployment requests... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRequest cerc-webapp-deployer: 2024-08-15 02:13:08.815428 - - 0:00:00.493420 (step): Loading known requests from /srv/deployments/autodeploy.state... cerc-webapp-deployer: 2024-08-15 02:13:08.815626 - - 0:00:00.000158 (step): BEGIN: Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi cerc-webapp-deployer: 2024-08-15 02:13:08.815645 - - 0:00:00.000008 (step): Skipping request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815653 - - 0:00:00.000005 (step): DONE Examining request bafyreigiltcdscwt7rqldnilo4ohrhgoulrlfceixde5ycewsym64sefgi with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815664 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue cerc-webapp-deployer: 2024-08-15 02:13:08.815674 - - 0:00:00.000006 (step): Skipping request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815684 - - 0:00:00.000004 (step): DONE Examining request bafyreicoxippgdwab6cz72py4rgv63rvvbsea73y62hashlhqpcsxyfkue with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815692 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne cerc-webapp-deployer: 2024-08-15 02:13:08.815705 - - 0:00:00.000007 (step): Skipping request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815714 - - 0:00:00.000005 (step): DONE Examining request bafyreih3gt44pvahnbg7ag26mlk3iie4s5m5znhygajja5dcovheti72ne with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815724 - - 0:00:00.000004 (step): BEGIN: Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua cerc-webapp-deployer: 2024-08-15 02:13:08.815733 - - 0:00:00.000005 (step): Skipping request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815743 - - 0:00:00.000005 (step): DONE Examining request bafyreigjnbio47rug6x5tufzc6cwfcqpl3ck3xldzotrlz5bt663dh2pua with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815751 - - 0:00:00.000004 (step): BEGIN: Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy cerc-webapp-deployer: 2024-08-15 02:13:08.815761 - - 0:00:00.000006 (step): Skipping request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815770 - - 0:00:00.000005 (step): DONE Examining request bafyreihsfno4s6lkxcp5a7g7pjj7kklrp3xaqo57mr2pz76nk3h4jukayy with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815779 - - 0:00:00.000005 (step): BEGIN: Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy cerc-webapp-deployer: 2024-08-15 02:13:08.815791 - - 0:00:00.000007 (step): Skipping request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy, we've already seen it. cerc-webapp-deployer: 2024-08-15 02:13:08.815800 - - 0:00:00.000004 (step): DONE Examining request bafyreicyfyj4ncmtuy5pain2rvc67v645cg2bbsiakizvhdiwvkx7asvdy with result SKIP. cerc-webapp-deployer: 2024-08-15 02:13:08.815808 - - 0:00:00.000004 (step): Discovering existing app deployments... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRecord cerc-webapp-deployer: 2024-08-15 02:13:09.330655 - - 0:00:00.514858 (step): Discovering deployment removal and cancellation requests... cerc-webapp-deployer: laconic -c /etc/config/laconic.yml registry record list --all --type ApplicationDeploymentRemovalRequest cerc-webapp-deployer: 2024-08-15 02:13:09.825145 - - 0:00:00.494460 (step): Found 0 unsatisfied request(s) to process. cerc-webapp-deployer: ############ DEPLOY SUCCESS ############# ``` Reviewed-on: #923 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-15 02:57:47 +00:00
Thomas E Lackey	952389abb0	Add option to recreate deployments rather than update them. (#920) ... cherry-pick from #912 Reviewed-on: #920 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>	2024-08-14 20:14:40 +00:00
Thomas E Lackey	5c275aa622	Defensively handle errors examining app requests. (#922) ... Related to cerc-io/webapp-deployment-status-api#10 There are two issues in that. One is that the output probably changed recently, whether in the client or server, where no matching record is found by ID (Note this is specific to `laconic record get --id <v>` and does not seem to apply to the similar command to retrieve a record by name, `laconic name resolve <n>`). Rather than returning `[]` it is now returning `[ null ]`. This cause us to think there *was* an application record found, and we attempt to treat the `null` entry like an Application object. That's fixed by filtering out null responses, which is a good precaution for the deployer, though I think it makes sense to ask whether this new behavior by the client/server is correct. Seems suspicious. The other issue is that all the defensive checks we had in place to deal with broken/bad AppDeploymentRequests were around the _build_. This error was coming much earlier, merely when parsing and examining the request to see if it needed to be handled at all. I have now added similar defensive error handling around that portion of the code. Reviewed-on: #922 Reviewed-by: zramsay <zramsay@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-14 18:04:31 +00:00
Thomas E Lackey	8576137557	Convert port to string. (#919) ... The str type check doesn't work if the port is a ruamel.yaml.scalarstring.SingleQuotedScalarString or ruamel.yaml.scalarstring.DoubleQuotedScalarString Reviewed-on: #919 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-14 00:25:35 +00:00
David Boreham	65c1cdf6b1	Fix crash if port has int type in yaml (#918) ... Reviewed-on: #918 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 20:47:09 +00:00
David Boreham	265699bc38	Allow to disable kind cluster management for testing (#915) ... Reviewed-on: #915 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 17:48:14 +00:00
David Boreham	4a7670a5d6	Open the json-rpc port (#916) ... Reviewed-on: #916 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-13 17:47:56 +00:00
Thomas E Lackey	6087e1cd31	Copy config under a volume for Docker (similar to a ConfigMap for K8S). (#914) ... This emulates the K8S ConfigMap behavior on Docker by using a regular volume. If a directory exists under `config/` which matches a named volume, the contents will be copied to the volume on `create` (provided the destination volume is empty). That is, rather than a ConfigMap, it is essentially a "config volume". For example, with a compose file like: ``` version: '3.7' services: caddy: image: cerc/caddy-ethcache:local restart: always volumes: - caddyconfig:/etc/caddy:ro volumes: caddyconfig: ``` And a directory: ``` ❯ ls stack-orchestrator/config/caddyconfig/ Caddyfile ``` After `laconic-so deploy create --spec-file caddy.yml --deployment-dir /srv/caddy` there will be: ``` ❯ ls /srv/caddy/data/caddyconfig Caddyfile ``` Mounted at `/etc/caddy` Reviewed-on: #914 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-10 02:32:21 +00:00
Thomas E Lackey	1def279d26	Support multiple NodePorts, static NodePort mapping, and add 'replicas' spec option (#913) ... NodePort example: ``` network: ports: caddy: - 1234 - 32020:2020 ``` Replicas example: ``` replicas: 2 ``` This also adds an optimization for k8s where if a directory matching the name of a configmap exists in beneath config/ in the stack, its contents will be copied into the corresponding configmap. For example: ``` # Config files in the stack ❯ ls stack-orchestrator/config/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example # ConfigMap in the spec ❯ cat foo.yml | grep config ... configmaps: caddyconfig: ./configmaps/caddyconfig # Create the deployment ❯ laconic-so --stack ~/cerc/caddy-ethcache/stack-orchestrator/stacks/caddy-ethcache deploy create --spec-file foo.yml # The files from beneath config/<config_map_name> have been copied to the ConfigMap directory from the spec. ❯ ls deployment-001/configmaps/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example ``` Reviewed-on: #913 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>	2024-08-09 02:32:06 +00:00
David Boreham	64691bd206	Merge pull request 'Allow gentx-files to be omitted' (#911) from dboreham/allow-zero-gentx into main ... Reviewed-on: #911	2024-08-07 20:13:40 +00:00
David Boreham	aef5986135	Allow gentx-files to be omitted	2024-08-07 14:11:06 -06:00
David Boreham	6f8f0340d3	Merge pull request 'Add stage 1 support' (#910) from dboreham/stage1-support into main ... Reviewed-on: #910	2024-08-07 17:44:28 +00:00
David Boreham	7590d6e237	Add stage 1 support	2024-08-07 11:28:10 -06:00
David Boreham	573f99dbbe	Listen on 0.0.0.0 (#909) ... Reviewed-on: #909 Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>	2024-08-02 14:06:06 +00:00
David Boreham	8052c1c25e	Merge pull request 'Laconicd needs to be told its currency' (#908) from dboreham/mainnet-laconic-specify-currency into main ... Reviewed-on: #908	2024-08-02 03:10:34 +00:00
David Boreham	a674d13493	Laconicd needs to be told its currency	2024-08-01 21:09:30 -06:00
Roy Crihfield	0d4f4509c8	Remove Eth fixturenet workflows (#906) ... Deletes the now-failing CI workflows for the old `fixturenet-eth` and `fixturenet-plugeth` stacks. Part of #905. Reviewed-on: #906 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>	2024-08-01 02:28:05 +00:00
David Boreham	5af27b1b3a	Merge pull request 'Fix for sh as shell not bash' (#907) from dboreham/fix-script-for-ubuntu into main ... Reviewed-on: #907	2024-07-31 20:39:37 +00:00
David Boreham	6c91b87348	Fix for sh as shell not bash	2024-07-31 14:30:53 -06:00