Compare commits

..
Author SHA1 Message Date
telackey 4e5b5ff031 Honor DOCKER_DEFAULT_PLATFORM
Lint Checks / Run linter (pull_request) Successful in 39s
Deploy Test / Run deploy test suite (pull_request) Successful in 6m1s
Webapp Test / Run webapp test suite (pull_request) Successful in 5m33s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (pull_request) Successful in 7m39s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 9m0s
Smoke Test / Run basic test suite (pull_request) Successful in 4m16s
2024-08-29 14:57:35 -05:00
telackey f1fdc48aaa Work around this bug: https://github.com/python/cpython/pull/14064 (#941)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m30s
Smoke Test / Run basic test suite (push) Successful in 4m18s
Webapp Test / Run webapp test suite (push) Successful in 5m2s
Deploy Test / Run deploy test suite (push) Successful in 5m20s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m7s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Failing after 30s
Database Test / Run database hosting test on kind/k8s (push) Failing after 32s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 34s
External Stack Test / Run external stack test suite (push) Failing after 31s
Otherwise we sometimes see errors like:

```
cerc-webapp-deployer:   File "/root/.shiv/laconic-so_0f937aa98c2748ef9af8585d6f441dbc01546ace0d6660cbb159d1e5040aeddf/site-packages/stack_orchestrator/deploy/webapp/deploy_webapp_from_registry.py", line 671, in command
cerc-webapp-deployer:     shutil.rmtree(tempdir)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 725, in rmtree
cerc-webapp-deployer:     _rmtree_safe_fd(fd, path, onerror)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 681, in _rmtree_safe_fd
cerc-webapp-deployer:     onerror(os.unlink, fullname, sys.exc_info())
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 679, in _rmtree_safe_fd
cerc-webapp-deployer:     os.unlink(entry.name, dir_fd=topfd)
cerc-webapp-deployer: FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
```

Reviewed-on: #941
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 23:17:13 +00:00
telackey a54072de6c Add --config-ref flag. (#939)
Lint Checks / Run linter (push) Successful in 43s
Publish / Build and publish (push) Successful in 1m15s
Smoke Test / Run basic test suite (push) Successful in 3m55s
Webapp Test / Run webapp test suite (push) Successful in 4m38s
Deploy Test / Run deploy test suite (push) Successful in 4m53s
Database Test / Run database hosting test on kind/k8s (push) Successful in 8m46s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m24s
External Stack Test / Run external stack test suite (push) Successful in 4m32s
Add a flag to re-use config.

Reviewed-on: #939
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 17:32:52 +00:00
telackey fa21ff2627 Support uploaded config, add 'publish-webapp-deployer' and 'request-webapp-deployment' commands (#938)
Lint Checks / Run linter (push) Successful in 36s
Publish / Build and publish (push) Successful in 1m6s
Smoke Test / Run basic test suite (push) Successful in 3m53s
Webapp Test / Run webapp test suite (push) Successful in 4m33s
Deploy Test / Run deploy test suite (push) Successful in 4m39s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m10s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m25s
This adds two new commands: `publish-webapp-deployer` and `request-webapp-deployment`.

`publish-webapp-deployer` creates a `WebappDeployer` record, which provides information to requestors like the API URL, minimum required payment, payment address, and public key to use for encrypting config.

```
$ laconic-so publish-deployer-to-registry \
  --laconic-config ~/.laconic/laconic.yml \
  --api-url https://webapp-deployer-api.dev.vaasl.io \
  --public-key-file webapp-deployer-api.dev.vaasl.io.pgp.pub  \
  --lrn lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io  \
  --min-required-payment 100000
```

`request-webapp-deployment` simplifies publishing a `WebappDeploymentRequest` and can also handle automatic payment, and encryption and upload of configuration.

```
$ laconic-so request-webapp-deployment \
  --laconic-config ~/.laconic/laconic.yml \
  --deployer lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io \
  --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
  --env-file ~/yaml/hello.env \
  --make-payment auto
```

Related changes are included for the deploy/undeploy commands for decrypting and using config, using the payment address from the WebappDeployer record, etc.

Reviewed-on: #938
2024-08-27 19:55:06 +00:00
prathamesh 33d395e213 Add package registry stack instructions (#937)
Lint Checks / Run linter (push) Successful in 36s
Publish / Build and publish (push) Successful in 1m9s
Smoke Test / Run basic test suite (push) Successful in 4m0s
Webapp Test / Run webapp test suite (push) Successful in 4m33s
Deploy Test / Run deploy test suite (push) Successful in 4m55s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m59s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m42s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m52s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m37s
External Stack Test / Run external stack test suite (push) Successful in 4m46s
- The instructions to `Deploy Gitea Package Registry` from build-support [readme](https://git.vdb.to/deep-stack/stack-orchestrator/src/branch/pm-update-registry-steps/stack_orchestrator/data/stacks/build-support#2-deploy-gitea-package-registry) don't seem to be in a working state
- Updated `package-registry` stack instructions to use deployment pattern

Reviewed-on: #937
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-08-23 09:42:44 +00:00
telackey 75ff60752a Require payment for app deployment requests. (#928)
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m33s
Lint Checks / Run linter (push) Successful in 35s
Publish / Build and publish (push) Successful in 1m18s
Smoke Test / Run basic test suite (push) Successful in 3m58s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m44s
Webapp Test / Run webapp test suite (push) Successful in 4m45s
External Stack Test / Run external stack test suite (push) Successful in 4m39s
Deploy Test / Run deploy test suite (push) Successful in 5m10s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m5s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m19s
Adds three new options for deployment/undeployment:

```
    "--min-required-payment",
    help="Requests must have a minimum payment to be processed",

    "--payment-address",
    help="The address to which payments should be made.  Default is the current laconic account.",

    "--all-requests",
    help="Handle requests addressed to anyone (by default only requests to my payment address are examined).",
```

In this mode, requests should be designated for a particular address with the attribute `to` and include a `payment` attribute which is the tx hash for the payment.

The deployer will confirm the payment (to the right account, right amount, not used before, etc.) and then proceed with the deployment or undeployment.

Reviewed-on: #928
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-21 14:39:20 +00:00
dboreham 44b9709717 Use Laconic version of ping-pub (#930)
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m33s
Lint Checks / Run linter (push) Successful in 49s
Publish / Build and publish (push) Successful in 1m23s
Smoke Test / Run basic test suite (push) Successful in 4m32s
Deploy Test / Run deploy test suite (push) Successful in 5m20s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m34s
Webapp Test / Run webapp test suite (push) Successful in 5m9s
External Stack Test / Run external stack test suite (push) Successful in 4m34s
Database Test / Run database hosting test on kind/k8s (push) Successful in 8m58s
Reviewed-on: #930
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-20 17:44:00 +00:00
12 changed files with 455 additions and 58 deletions
+2
View File
@@ -11,3 +11,5 @@ tomli==2.0.1
validators==0.22.0
kubernetes>=28.1.0
humanfriendly>=10.0
python-gnupg>=0.5.2
requests>=2.3.2
+13 -7
View File
@@ -14,6 +14,7 @@
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import click
import os
from dataclasses import dataclass
import json
import platform
@@ -84,13 +85,18 @@ def _filter_for_platform(container: str,
tag_list: List[str]) -> List[str] :
filtered_tags = []
this_machine = platform.machine()
# Translate between Python and docker platform names
if this_machine == "x86_64":
this_machine = "amd64"
if this_machine == "aarch64":
this_machine = "arm64"
if opts.o.debug:
print(f"Python says the architecture is: {this_machine}")
if "DOCKER_DEFAULT_PLATFORM" in os.environ:
this_machine = os.environ["DOCKER_DEFAULT_PLATFORM"].split("/")[-1]
if opts.o.debug:
print(f"DOCKER_DEFAULT_PLATFORM says the architecture is: {this_machine}")
else:
# Translate between Python and docker platform names
if this_machine == "x86_64":
this_machine = "amd64"
if this_machine == "aarch64":
this_machine = "arm64"
if opts.o.debug:
print(f"Python says the architecture is: {this_machine}")
docker = DockerClient()
for tag in tag_list:
remote_tag = f"{registry_info.registry}/{container}:{tag}"
@@ -4,5 +4,9 @@ source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
# Two-stage build is to allow us to pick up both the upstream repo's files, and local files here for config
docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/explorer
docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/cosmos-explorer
if [[ $? -ne 0 ]]; then
echo "FATAL: Base container build failed, exiting"
exit 1
fi
docker build -t cerc/ping-pub:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile $SCRIPT_DIR
@@ -10,7 +10,7 @@ repos:
- git.vdb.to/cerc-io/registry-sdk
- git.vdb.to/cerc-io/laconic-registry-cli
- git.vdb.to/cerc-io/laconic-console
- github.com/ping-pub/explorer
- git.vdb.to/cerc-io/cosmos-explorer
npms:
- registry-sdk
- laconic-registry-cli
@@ -2,4 +2,28 @@
The Package Registry Stack supports a build environment that requires a package registry (initially for NPM packages only).
Setup instructions can be found [here](../build-support/README.md).
## Setup
* Setup required repos and build containers:
```bash
laconic-so --stack package-registry setup-repositories
laconic-so --stack package-registry build-containers
```
* Create a deployment:
```bash
laconic-so --stack package-registry deploy init --output package-registry-spec.yml
# Update port mapping in the laconic-loaded.spec file to resolve port conflicts on host if any
laconic-so --stack package-registry deploy create --deployment-dir package-registry-deployment --spec-file package-registry-spec.yml
```
* Start the deployment:
```bash
laconic-so deployment --dir package-registry-deployment start
```
* The local gitea registry can now be accessed at <http://localhost:3000> (the username and password can be taken from the deployment logs)
@@ -14,6 +14,7 @@
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import os
import base64
from kubernetes import client
from typing import Any, List, Set
@@ -260,12 +261,12 @@ class ClusterInfo:
for f in os.listdir(cfg_map_path):
full_path = os.path.join(cfg_map_path, f)
if os.path.isfile(full_path):
data[f] = open(full_path, 'rt').read()
data[f] = base64.b64encode(open(full_path, 'rb').read()).decode('ASCII')
spec = client.V1ConfigMap(
metadata=client.V1ObjectMeta(name=f"{self.app_name}-{cfg_map_name}",
labels={"configmap-label": cfg_map_name}),
data=data
binary_data=data
)
result.append(spec)
return result
@@ -21,12 +21,15 @@ import sys
import tempfile
import time
import uuid
import yaml
import click
import gnupg
from stack_orchestrator.deploy.images import remote_image_exists
from stack_orchestrator.deploy.webapp import deploy_webapp
from stack_orchestrator.deploy.webapp.util import (
AttrDict,
LaconicRegistryClient,
TimedLogger,
build_container_image,
@@ -55,7 +58,10 @@ def process_app_deployment_request(
force_rebuild,
fqdn_policy,
recreate_on_deploy,
payment_address,
webapp_deployer_record,
gpg,
private_key_passphrase,
config_upload_dir,
logger,
):
logger.log("BEGIN - process_app_deployment_request")
@@ -107,14 +113,31 @@ def process_app_deployment_request(
)
# 4. get build and runtime config from request
env = {}
if app_deployment_request.attributes.config:
if "ref" in app_deployment_request.attributes.config:
with open(
f"{config_upload_dir}/{app_deployment_request.attributes.config.ref}",
"rb",
) as file:
record_owner = laconic.get_owner(app_deployment_request)
decrypted = gpg.decrypt_file(file, passphrase=private_key_passphrase)
parsed = AttrDict(yaml.safe_load(decrypted.data))
if record_owner not in parsed.authorized:
raise Exception(
f"{record_owner} not authorized to access config {app_deployment_request.attributes.config.ref}"
)
if "env" in parsed.config:
env.update(parsed.config.env)
if "env" in app_deployment_request.attributes.config:
env.update(app_deployment_request.attributes.config.env)
env_filename = None
if (
app_deployment_request.attributes.config
and "env" in app_deployment_request.attributes.config
):
if env:
env_filename = tempfile.mktemp()
with open(env_filename, "w") as file:
for k, v in app_deployment_request.attributes.config["env"].items():
for k, v in env.items():
file.write("%s=%s\n" % (k, shlex.quote(str(v))))
# 5. determine new or existing deployment
@@ -227,7 +250,7 @@ def process_app_deployment_request(
dns_lrn,
deployment_dir,
app_deployment_request,
payment_address,
webapp_deployer_record,
logger,
)
logger.log("Publication complete.")
@@ -285,8 +308,12 @@ def dump_known_requests(filename, requests, status="SEEN"):
help="How to handle requests with an FQDN: prohibit, allow, preexisting",
default="prohibit",
)
@click.option("--record-namespace-dns", help="eg, lrn://laconic/dns")
@click.option("--record-namespace-deployments", help="eg, lrn://laconic/deployments")
@click.option("--record-namespace-dns", help="eg, lrn://laconic/dns", required=True)
@click.option(
"--record-namespace-deployments",
help="eg, lrn://laconic/deployments",
required=True,
)
@click.option(
"--dry-run", help="Don't do anything, just report what would be done.", is_flag=True
)
@@ -313,21 +340,29 @@ def dump_known_requests(filename, requests, status="SEEN"):
)
@click.option(
"--min-required-payment",
help="Requests must have a minimum payment to be processed",
help="Requests must have a minimum payment to be processed (in alnt)",
default=0,
)
@click.option(
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option("--lrn", help="The LRN of this deployer.", required=True)
@click.option(
"--all-requests",
help="Handle requests addressed to anyone (by default only requests to"
"my payment address are examined).",
is_flag=True,
)
@click.option(
"--config-upload-dir",
help="The directory containing uploaded config.",
required=True,
)
@click.option(
"--private-key-file", help="The private key for decrypting config.", required=True
)
@click.option(
"--private-key-passphrase",
help="The passphrase for the private key.",
required=True,
)
@click.pass_context
def command( # noqa: C901
ctx,
@@ -350,7 +385,10 @@ def command( # noqa: C901
recreate_on_deploy,
log_dir,
min_required_payment,
payment_address,
lrn,
config_upload_dir,
private_key_file,
private_key_passphrase,
all_requests,
):
if request_id and discover:
@@ -384,6 +422,18 @@ def command( # noqa: C901
)
sys.exit(2)
tempdir = tempfile.mkdtemp()
gpg = gnupg.GPG(gnupghome=tempdir)
# Import the deployer's public key
result = gpg.import_keys(open(private_key_file, "rb").read())
if 1 != result.imported:
print(
f"Failed to load private key file: {private_key_file}.",
file=sys.stderr,
)
sys.exit(2)
main_logger = TimedLogger(file=sys.stderr)
try:
@@ -392,11 +442,17 @@ def command( # noqa: C901
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr)
if not payment_address:
payment_address = laconic.whoami().address
webapp_deployer_record = laconic.get_record(lrn, require=True)
payment_address = webapp_deployer_record.attributes.paymentAddress
main_logger.log(f"Payment address: {payment_address}")
if min_required_payment and not payment_address:
print(
f"Minimum payment required, but no payment address listed for deployer: {lrn}.",
file=sys.stderr,
)
sys.exit(2)
# Find deployment requests.
# single request
if request_id:
@@ -408,7 +464,7 @@ def command( # noqa: C901
if all_requests:
requests = laconic.app_deployment_requests()
else:
requests = laconic.app_deployment_requests({"to": payment_address})
requests = laconic.app_deployment_requests({"deployer": lrn})
if only_update_state:
if not dry_run:
@@ -487,7 +543,7 @@ def command( # noqa: C901
if all_requests:
deployments = laconic.app_deployments()
else:
deployments = laconic.app_deployments({"by": payment_address})
deployments = laconic.app_deployments({"deployer": lrn})
deployments_by_request = {}
for d in deployments:
if d.attributes.request:
@@ -530,7 +586,11 @@ def command( # noqa: C901
for r in requests_to_check_for_payment:
main_logger.log(f"{r.id}: Confirming payment...")
if confirm_payment(
laconic, r, payment_address, min_required_payment, main_logger
laconic,
r,
payment_address,
min_required_payment,
main_logger,
):
main_logger.log(f"{r.id}: Payment confirmed.")
requests_to_execute.append(r)
@@ -583,7 +643,10 @@ def command( # noqa: C901
force_rebuild,
fqdn_policy,
recreate_on_deploy,
payment_address,
webapp_deployer_record,
gpg,
private_key_passphrase,
config_upload_dir,
build_logger,
)
status = "DEPLOYED"
@@ -604,3 +667,5 @@ def command( # noqa: C901
except Exception as e:
main_logger.log("UNCAUGHT ERROR:" + str(e))
raise e
finally:
shutil.rmtree(tempdir, ignore_errors=True)
@@ -0,0 +1,91 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import base64
import click
import sys
import yaml
from urllib.parse import urlparse
from stack_orchestrator.deploy.webapp.util import LaconicRegistryClient
@click.command()
@click.option(
"--laconic-config", help="Provide a config file for laconicd", required=True
)
@click.option("--api-url", help="The API URL of the deployer.", required=True)
@click.option(
"--public-key-file",
help="The public key to use. This should be a binary file.",
required=True,
)
@click.option(
"--lrn", help="eg, lrn://laconic/deployers/my.deployer.name", required=True
)
@click.option(
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option(
"--min-required-payment",
help="List the minimum required payment (in alnt) to process a deployment request.",
default=0,
)
@click.option(
"--dry-run",
help="Don't publish anything, just report what would be done.",
is_flag=True,
)
@click.pass_context
def command( # noqa: C901
ctx,
laconic_config,
api_url,
public_key_file,
lrn,
payment_address,
min_required_payment,
dry_run,
):
laconic = LaconicRegistryClient(laconic_config)
if not payment_address:
payment_address = laconic.whoami().address
pub_key = base64.b64encode(open(public_key_file, "rb").read()).decode("ASCII")
hostname = urlparse(api_url).hostname
webapp_deployer_record = {
"record": {
"type": "WebappDeployer",
"version": "1.0.0",
"apiUrl": api_url,
"name": hostname,
"publicKey": pub_key,
"paymentAddress": payment_address,
}
}
if min_required_payment:
webapp_deployer_record["record"][
"minimumPayment"
] = f"{min_required_payment}alnt"
if dry_run:
yaml.dump(webapp_deployer_record, sys.stdout)
return
laconic.publish(webapp_deployer_record, [lrn])
@@ -0,0 +1,175 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
import base64
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import shutil
import sys
import tempfile
from datetime import datetime
import gnupg
import click
import requests
import yaml
from stack_orchestrator.deploy.webapp.util import (
LaconicRegistryClient,
)
from dotenv import dotenv_values
def fatal(msg: str):
print(msg, file=sys.stderr)
sys.exit(1)
@click.command()
@click.option(
"--laconic-config", help="Provide a config file for laconicd", required=True
)
@click.option(
"--app",
help="The LRN of the application to deploy.",
required=True,
)
@click.option(
"--deployer",
help="The LRN of the deployer to process this request.",
required=True,
)
@click.option("--env-file", help="environment file for webapp")
@click.option("--config-ref", help="The ref of an existing config upload to use.")
@click.option(
"--make-payment",
help="The payment to make (in alnt). The value should be a number or 'auto' to use the deployer's minimum required payment.",
)
@click.option(
"--use-payment", help="The TX id of an existing, unused payment", default=None
)
@click.option("--dns", help="the DNS name to request (default is autogenerated)")
@click.option(
"--dry-run",
help="Don't publish anything, just report what would be done.",
is_flag=True,
)
@click.pass_context
def command(
ctx,
laconic_config,
app,
deployer,
env_file,
config_ref,
make_payment,
use_payment,
dns,
dry_run,
): # noqa: C901
tempdir = tempfile.mkdtemp()
try:
laconic = LaconicRegistryClient(laconic_config)
app_record = laconic.get_record(app)
if not app_record:
fatal(f"Unable to locate app: {app}")
deployer_record = laconic.get_record(deployer)
if not deployer_record:
fatal(f"Unable to locate deployer: {deployer}")
if env_file and config_ref:
fatal("Cannot use --env-file and --config-ref at the same time.")
# If env_file
if env_file:
gpg = gnupg.GPG(gnupghome=tempdir)
# Import the deployer's public key
result = gpg.import_keys(
base64.b64decode(deployer_record.attributes.publicKey)
)
if 1 != result.imported:
fatal("Failed to import deployer's public key.")
recip = gpg.list_keys()[0]["uids"][0]
# Wrap the config
config = {
# Include account (and payment?) details
"authorized": [laconic.whoami().address],
"config": {"env": dict(dotenv_values(env_file))},
}
serialized = yaml.dump(config)
# Encrypt
result = gpg.encrypt(serialized, recip, always_trust=True, armor=False)
if not result.ok:
fatal("Failed to encrypt config.")
# Upload it to the deployer's API
response = requests.post(
f"{deployer_record.attributes.apiUrl}/upload/config",
data=result.data,
headers={"Content-Type": "application/octet-stream"},
)
if not response.ok:
response.raise_for_status()
config_ref = response.json()["id"]
deployment_request = {
"record": {
"type": "ApplicationDeploymentRequest",
"application": app,
"version": "1.0.0",
"name": f"{app_record.attributes.name}@{app_record.attributes.version}",
"deployer": deployer,
"meta": {"when": str(datetime.utcnow())},
}
}
if config_ref:
deployment_request["record"]["config"] = {"ref": config_ref}
if dns:
deployment_request["record"]["dns"] = dns.lower()
if make_payment:
amount = 0
if dry_run:
deployment_request["record"]["payment"] = "DRY_RUN"
elif "auto" == make_payment:
if "minimumPayment" in deployer_record.attributes:
amount = int(
deployer_record.attributes.minimumPayment.replace("alnt", "")
)
else:
amount = make_payment
if amount:
receipt = laconic.send_tokens(
deployer_record.attributes.paymentAddress, amount
)
deployment_request["record"]["payment"] = receipt.tx.hash
print("Payment TX:", receipt.tx.hash)
elif use_payment:
deployment_request["record"]["payment"] = use_payment
if dry_run:
print(yaml.dump(deployment_request))
return
# Send the request
laconic.publish(deployment_request)
finally:
shutil.rmtree(tempdir, ignore_errors=True)
@@ -38,7 +38,7 @@ def process_app_removal_request(
deployment_parent_dir,
delete_volumes,
delete_names,
payment_address,
webapp_deployer_record,
):
deployment_record = laconic.get_record(
app_removal_request.attributes.deployment, require=True
@@ -84,7 +84,7 @@ def process_app_removal_request(
"version": "1.0.0",
"request": app_removal_request.id,
"deployment": deployment_record.id,
"by": payment_address,
"deployer": webapp_deployer_record.names[0],
}
}
@@ -168,15 +168,10 @@ def dump_known_requests(filename, requests):
)
@click.option(
"--min-required-payment",
help="Requests must have a minimum payment to be processed",
help="Requests must have a minimum payment to be processed (in alnt)",
default=0,
)
@click.option(
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option("--lrn", help="The LRN of this deployer.", required=True)
@click.option(
"--all-requests",
help="Handle requests addressed to anyone (by default only requests to"
@@ -198,7 +193,7 @@ def command( # noqa: C901
include_tags,
exclude_tags,
min_required_payment,
payment_address,
lrn,
all_requests,
):
if request_id and discover:
@@ -218,8 +213,16 @@ def command( # noqa: C901
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr)
if not payment_address:
payment_address = laconic.whoami().address
deployer_record = laconic.get_record(lrn, require=True)
payment_address = deployer_record.attributes.paymentAddress
main_logger.log(f"Payment address: {payment_address}")
if min_required_payment and not payment_address:
print(
f"Minimum payment required, but no payment address listed for deployer: {lrn}.",
file=sys.stderr,
)
sys.exit(2)
# Find deployment removal requests.
# single request
@@ -233,7 +236,7 @@ def command( # noqa: C901
if all_requests:
requests = laconic.app_deployment_removal_requests()
else:
requests = laconic.app_deployment_removal_requests({"to": payment_address})
requests = laconic.app_deployment_removal_requests({"deployer": lrn})
if only_update_state:
if not dry_run:
@@ -312,7 +315,11 @@ def command( # noqa: C901
for r in requests_to_check_for_payment:
main_logger.log(f"{r.id}: Confirming payment...")
if confirm_payment(
laconic, r, payment_address, min_required_payment, main_logger
laconic,
r,
payment_address,
min_required_payment,
main_logger,
):
main_logger.log(f"{r.id}: Payment confirmed.")
requests_to_execute.append(r)
@@ -336,7 +343,7 @@ def command( # noqa: C901
os.path.abspath(deployment_parent_dir),
delete_volumes,
delete_names,
payment_address,
deployer_record,
)
except Exception as e:
main_logger.log(f"ERROR processing removal request {r.id}: {e}")
+24 -6
View File
@@ -1,4 +1,4 @@
# Copyright © 2023 Vulcanize
# = str(min_required_payment) Copyright © 2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
@@ -142,14 +142,14 @@ def confirm_payment(laconic, record, payment_address, min_amount, logger):
# Check if the payment was already used on a
used = laconic.app_deployments(
{"by": payment_address, "payment": tx.hash}, all=True
{"deployer": payment_address, "payment": tx.hash}, all=True
)
if len(used):
logger.log(f"{record.id}: payment {tx.hash} already used on deployment {used}")
return False
used = laconic.app_deployment_removals(
{"by": payment_address, "payment": tx.hash}, all=True
{"deployer": payment_address, "payment": tx.hash}, all=True
)
if len(used):
logger.log(
@@ -453,6 +453,24 @@ class LaconicRegistryClient:
name,
)
def send_tokens(self, address, amount, type="alnt"):
args = [
"laconic",
"-c",
self.config_file,
"registry",
"tokens",
"send",
"--address",
address,
"--quantity",
str(amount),
"--type",
type,
]
return AttrDict(json.loads(logged_cmd(self.log_file, *args)))
def file_hash(filename):
return hashlib.sha1(open(filename).read().encode()).hexdigest()
@@ -609,7 +627,7 @@ def publish_deployment(
dns_lrn,
deployment_dir,
app_deployment_request=None,
payment_address=None,
webapp_deployer_record=None,
logger=None,
):
if not deploy_record:
@@ -666,8 +684,8 @@ def publish_deployment(
"payment"
] = app_deployment_request.attributes.payment
if payment_address:
new_deployment_record["record"]["by"] = payment_address
if webapp_deployer_record:
new_deployment_record["record"]["deployer"] = webapp_deployer_record.names[0]
if logger:
logger.log("Publishing ApplicationDeploymentRecord.")
+5 -1
View File
@@ -24,7 +24,9 @@ from stack_orchestrator.build import build_webapp
from stack_orchestrator.deploy.webapp import (run_webapp,
deploy_webapp,
deploy_webapp_from_registry,
undeploy_webapp_from_registry)
undeploy_webapp_from_registry,
publish_webapp_deployer,
request_webapp_deployment)
from stack_orchestrator.deploy import deploy
from stack_orchestrator import version
from stack_orchestrator.deploy import deployment
@@ -61,6 +63,8 @@ cli.add_command(run_webapp.command, "run-webapp")
cli.add_command(deploy_webapp.command, "deploy-webapp")
cli.add_command(deploy_webapp_from_registry.command, "deploy-webapp-from-registry")
cli.add_command(undeploy_webapp_from_registry.command, "undeploy-webapp-from-registry")
cli.add_command(publish_webapp_deployer.command, "publish-deployer-to-registry")
cli.add_command(request_webapp_deployment.command, "request-webapp-deployment")
cli.add_command(deploy.command, "deploy") # deploy is an alias for deploy-system
cli.add_command(deploy.command, "deploy-system")
cli.add_command(deployment.command, "deployment")