Compare commits

..

8 Commits

Author SHA1 Message Date
dceda936b7 env 2024-10-07 18:55:42 +00:00
bca332b598 run dockerfile if exists 2024-10-07 18:27:10 +00:00
f1fdc48aaa Work around this bug: https://github.com/python/cpython/pull/14064 (#941)
Otherwise we sometimes see errors like:

```
cerc-webapp-deployer:   File "/root/.shiv/laconic-so_0f937aa98c2748ef9af8585d6f441dbc01546ace0d6660cbb159d1e5040aeddf/site-packages/stack_orchestrator/deploy/webapp/deploy_webapp_from_registry.py", line 671, in command
cerc-webapp-deployer:     shutil.rmtree(tempdir)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 725, in rmtree
cerc-webapp-deployer:     _rmtree_safe_fd(fd, path, onerror)
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 681, in _rmtree_safe_fd
cerc-webapp-deployer:     onerror(os.unlink, fullname, sys.exc_info())
cerc-webapp-deployer:   File "/usr/lib/python3.10/shutil.py", line 679, in _rmtree_safe_fd
cerc-webapp-deployer:     os.unlink(entry.name, dir_fd=topfd)
cerc-webapp-deployer: FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
```

Reviewed-on: cerc-io/stack-orchestrator#941
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 23:17:13 +00:00
a54072de6c Add --config-ref flag. (#939)
Add a flag to re-use config.

Reviewed-on: cerc-io/stack-orchestrator#939
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-28 17:32:52 +00:00
fa21ff2627 Support uploaded config, add 'publish-webapp-deployer' and 'request-webapp-deployment' commands (#938)
This adds two new commands: `publish-webapp-deployer` and `request-webapp-deployment`.

`publish-webapp-deployer` creates a `WebappDeployer` record, which provides information to requestors like the API URL, minimum required payment, payment address, and public key to use for encrypting config.

```
$ laconic-so publish-deployer-to-registry \
  --laconic-config ~/.laconic/laconic.yml \
  --api-url https://webapp-deployer-api.dev.vaasl.io \
  --public-key-file webapp-deployer-api.dev.vaasl.io.pgp.pub  \
  --lrn lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io  \
  --min-required-payment 100000
```

`request-webapp-deployment` simplifies publishing a `WebappDeploymentRequest` and can also handle automatic payment, and encryption and upload of configuration.

```
$ laconic-so request-webapp-deployment \
  --laconic-config ~/.laconic/laconic.yml \
  --deployer lrn://laconic/deployers/webapp-deployer-api.dev.vaasl.io \
  --app lrn://cerc-io/applications/webapp-hello-world@0.1.3 \
  --env-file ~/yaml/hello.env \
  --make-payment auto
```

Related changes are included for the deploy/undeploy commands for decrypting and using config, using the payment address from the WebappDeployer record, etc.

Reviewed-on: cerc-io/stack-orchestrator#938
2024-08-27 19:55:06 +00:00
33d395e213 Add package registry stack instructions (#937)
- The instructions to `Deploy Gitea Package Registry` from build-support [readme](https://git.vdb.to/deep-stack/stack-orchestrator/src/branch/pm-update-registry-steps/stack_orchestrator/data/stacks/build-support#2-deploy-gitea-package-registry) don't seem to be in a working state
- Updated `package-registry` stack instructions to use deployment pattern

Reviewed-on: cerc-io/stack-orchestrator#937
Reviewed-by: ashwin <ashwin@noreply.git.vdb.to>
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-08-23 09:42:44 +00:00
75ff60752a Require payment for app deployment requests. (#928)
Adds three new options for deployment/undeployment:

```
    "--min-required-payment",
    help="Requests must have a minimum payment to be processed",

    "--payment-address",
    help="The address to which payments should be made.  Default is the current laconic account.",

    "--all-requests",
    help="Handle requests addressed to anyone (by default only requests to my payment address are examined).",
```

In this mode, requests should be designated for a particular address with the attribute `to` and include a `payment` attribute which is the tx hash for the payment.

The deployer will confirm the payment (to the right account, right amount, not used before, etc.) and then proceed with the deployment or undeployment.

Reviewed-on: cerc-io/stack-orchestrator#928
Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-21 14:39:20 +00:00
44b9709717 Use Laconic version of ping-pub (#930)
Reviewed-on: cerc-io/stack-orchestrator#930
Co-authored-by: David Boreham <david@bozemanpass.com>
Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-20 17:44:00 +00:00
12 changed files with 472 additions and 55 deletions

View File

@ -11,3 +11,5 @@ tomli==2.0.1
validators==0.22.0 validators==0.22.0
kubernetes>=28.1.0 kubernetes>=28.1.0
humanfriendly>=10.0 humanfriendly>=10.0
python-gnupg>=0.5.2
requests>=2.3.2

View File

@ -30,6 +30,12 @@ from stack_orchestrator.build import build_containers
from stack_orchestrator.deploy.webapp.util import determine_base_container, TimedLogger from stack_orchestrator.deploy.webapp.util import determine_base_container, TimedLogger
from stack_orchestrator.build.build_types import BuildContext from stack_orchestrator.build.build_types import BuildContext
def create_env_file(env_vars, repo_root):
env_file_path = os.path.join(repo_root, '.env')
with open(env_file_path, 'w') as env_file:
for key, value in env_vars.items():
env_file.write(f"{key}={value}\n")
return env_file_path
@click.command() @click.command()
@click.option('--base-container') @click.option('--base-container')
@ -37,8 +43,9 @@ from stack_orchestrator.build.build_types import BuildContext
@click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild") @click.option("--force-rebuild", is_flag=True, default=False, help="Override dependency checking -- always rebuild")
@click.option("--extra-build-args", help="Supply extra arguments to build") @click.option("--extra-build-args", help="Supply extra arguments to build")
@click.option("--tag", help="Container tag (default: cerc/<app_name>:local)") @click.option("--tag", help="Container tag (default: cerc/<app_name>:local)")
@click.option("--env", help="Environment variables for webapp (format: KEY1=VALUE1,KEY2=VALUE2)", default="")
@click.pass_context @click.pass_context
def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag): def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, tag, env):
'''build the specified webapp container''' '''build the specified webapp container'''
logger = TimedLogger() logger = TimedLogger()
@ -88,9 +95,28 @@ def command(ctx, base_container, source_repo, force_rebuild, extra_build_args, t
# Now build the target webapp. We use the same build script, but with a different Dockerfile and work dir. # Now build the target webapp. We use the same build script, but with a different Dockerfile and work dir.
container_build_env["CERC_WEBAPP_BUILD_RUNNING"] = "true" container_build_env["CERC_WEBAPP_BUILD_RUNNING"] = "true"
container_build_env["CERC_CONTAINER_BUILD_WORK_DIR"] = os.path.abspath(source_repo) container_build_env["CERC_CONTAINER_BUILD_WORK_DIR"] = os.path.abspath(source_repo)
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = os.path.join(container_build_dir,
# Check if Dockerfile exists in the repository
repo_dockerfile = os.path.join(container_build_env["CERC_CONTAINER_BUILD_WORK_DIR"], "Dockerfile")
default_dockerfile = os.path.join(container_build_dir,
base_container.replace("/", "-"), base_container.replace("/", "-"),
"Dockerfile.webapp") "Dockerfile.webapp")
if os.path.isfile(repo_dockerfile):
env_vars = {}
if env:
for pair in env.split(','):
key, value = pair.split('=')
env_vars[key.strip()] = value.strip()
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = repo_dockerfile
# Create .env file with environment variables
env_file_path = create_env_file(env_vars, container_build_env["CERC_CONTAINER_BUILD_WORK_DIR"])
container_build_env["CERC_CONTAINER_BUILD_ENV_FILE"] = env_file_path
else:
container_build_env["CERC_CONTAINER_BUILD_DOCKERFILE"] = default_dockerfile
if not tag: if not tag:
webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1] webapp_name = os.path.abspath(source_repo).split(os.path.sep)[-1]
tag = f"cerc/{webapp_name}:local" tag = f"cerc/{webapp_name}:local"

View File

@ -4,5 +4,9 @@ source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
# Two-stage build is to allow us to pick up both the upstream repo's files, and local files here for config # Two-stage build is to allow us to pick up both the upstream repo's files, and local files here for config
docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/explorer docker build -t cerc/ping-pub-base:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile.base $CERC_REPO_BASE_DIR/cosmos-explorer
if [[ $? -ne 0 ]]; then
echo "FATAL: Base container build failed, exiting"
exit 1
fi
docker build -t cerc/ping-pub:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile $SCRIPT_DIR docker build -t cerc/ping-pub:local ${build_command_args} -f $SCRIPT_DIR/Dockerfile $SCRIPT_DIR

View File

@ -10,7 +10,7 @@ repos:
- git.vdb.to/cerc-io/registry-sdk - git.vdb.to/cerc-io/registry-sdk
- git.vdb.to/cerc-io/laconic-registry-cli - git.vdb.to/cerc-io/laconic-registry-cli
- git.vdb.to/cerc-io/laconic-console - git.vdb.to/cerc-io/laconic-console
- github.com/ping-pub/explorer - git.vdb.to/cerc-io/cosmos-explorer
npms: npms:
- registry-sdk - registry-sdk
- laconic-registry-cli - laconic-registry-cli

View File

@ -2,4 +2,28 @@
The Package Registry Stack supports a build environment that requires a package registry (initially for NPM packages only). The Package Registry Stack supports a build environment that requires a package registry (initially for NPM packages only).
Setup instructions can be found [here](../build-support/README.md). ## Setup
* Setup required repos and build containers:
```bash
laconic-so --stack package-registry setup-repositories
laconic-so --stack package-registry build-containers
```
* Create a deployment:
```bash
laconic-so --stack package-registry deploy init --output package-registry-spec.yml
# Update port mapping in the laconic-loaded.spec file to resolve port conflicts on host if any
laconic-so --stack package-registry deploy create --deployment-dir package-registry-deployment --spec-file package-registry-spec.yml
```
* Start the deployment:
```bash
laconic-so deployment --dir package-registry-deployment start
```
* The local gitea registry can now be accessed at <http://localhost:3000> (the username and password can be taken from the deployment logs)

View File

@ -14,6 +14,7 @@
# along with this program. If not, see <http:#www.gnu.org/licenses/>. # along with this program. If not, see <http:#www.gnu.org/licenses/>.
import os import os
import base64
from kubernetes import client from kubernetes import client
from typing import Any, List, Set from typing import Any, List, Set
@ -260,12 +261,12 @@ class ClusterInfo:
for f in os.listdir(cfg_map_path): for f in os.listdir(cfg_map_path):
full_path = os.path.join(cfg_map_path, f) full_path = os.path.join(cfg_map_path, f)
if os.path.isfile(full_path): if os.path.isfile(full_path):
data[f] = open(full_path, 'rt').read() data[f] = base64.b64encode(open(full_path, 'rb').read()).decode('ASCII')
spec = client.V1ConfigMap( spec = client.V1ConfigMap(
metadata=client.V1ObjectMeta(name=f"{self.app_name}-{cfg_map_name}", metadata=client.V1ObjectMeta(name=f"{self.app_name}-{cfg_map_name}",
labels={"configmap-label": cfg_map_name}), labels={"configmap-label": cfg_map_name}),
data=data binary_data=data
) )
result.append(spec) result.append(spec)
return result return result

View File

@ -21,12 +21,15 @@ import sys
import tempfile import tempfile
import time import time
import uuid import uuid
import yaml
import click import click
import gnupg
from stack_orchestrator.deploy.images import remote_image_exists from stack_orchestrator.deploy.images import remote_image_exists
from stack_orchestrator.deploy.webapp import deploy_webapp from stack_orchestrator.deploy.webapp import deploy_webapp
from stack_orchestrator.deploy.webapp.util import ( from stack_orchestrator.deploy.webapp.util import (
AttrDict,
LaconicRegistryClient, LaconicRegistryClient,
TimedLogger, TimedLogger,
build_container_image, build_container_image,
@ -55,7 +58,10 @@ def process_app_deployment_request(
force_rebuild, force_rebuild,
fqdn_policy, fqdn_policy,
recreate_on_deploy, recreate_on_deploy,
payment_address, webapp_deployer_record,
gpg,
private_key_passphrase,
config_upload_dir,
logger, logger,
): ):
logger.log("BEGIN - process_app_deployment_request") logger.log("BEGIN - process_app_deployment_request")
@ -107,14 +113,31 @@ def process_app_deployment_request(
) )
# 4. get build and runtime config from request # 4. get build and runtime config from request
env = {}
if app_deployment_request.attributes.config:
if "ref" in app_deployment_request.attributes.config:
with open(
f"{config_upload_dir}/{app_deployment_request.attributes.config.ref}",
"rb",
) as file:
record_owner = laconic.get_owner(app_deployment_request)
decrypted = gpg.decrypt_file(file, passphrase=private_key_passphrase)
parsed = AttrDict(yaml.safe_load(decrypted.data))
if record_owner not in parsed.authorized:
raise Exception(
f"{record_owner} not authorized to access config {app_deployment_request.attributes.config.ref}"
)
if "env" in parsed.config:
env.update(parsed.config.env)
if "env" in app_deployment_request.attributes.config:
env.update(app_deployment_request.attributes.config.env)
env_filename = None env_filename = None
if ( if env:
app_deployment_request.attributes.config
and "env" in app_deployment_request.attributes.config
):
env_filename = tempfile.mktemp() env_filename = tempfile.mktemp()
with open(env_filename, "w") as file: with open(env_filename, "w") as file:
for k, v in app_deployment_request.attributes.config["env"].items(): for k, v in env.items():
file.write("%s=%s\n" % (k, shlex.quote(str(v)))) file.write("%s=%s\n" % (k, shlex.quote(str(v))))
# 5. determine new or existing deployment # 5. determine new or existing deployment
@ -227,7 +250,7 @@ def process_app_deployment_request(
dns_lrn, dns_lrn,
deployment_dir, deployment_dir,
app_deployment_request, app_deployment_request,
payment_address, webapp_deployer_record,
logger, logger,
) )
logger.log("Publication complete.") logger.log("Publication complete.")
@ -285,8 +308,12 @@ def dump_known_requests(filename, requests, status="SEEN"):
help="How to handle requests with an FQDN: prohibit, allow, preexisting", help="How to handle requests with an FQDN: prohibit, allow, preexisting",
default="prohibit", default="prohibit",
) )
@click.option("--record-namespace-dns", help="eg, lrn://laconic/dns") @click.option("--record-namespace-dns", help="eg, lrn://laconic/dns", required=True)
@click.option("--record-namespace-deployments", help="eg, lrn://laconic/deployments") @click.option(
"--record-namespace-deployments",
help="eg, lrn://laconic/deployments",
required=True,
)
@click.option( @click.option(
"--dry-run", help="Don't do anything, just report what would be done.", is_flag=True "--dry-run", help="Don't do anything, just report what would be done.", is_flag=True
) )
@ -313,21 +340,29 @@ def dump_known_requests(filename, requests, status="SEEN"):
) )
@click.option( @click.option(
"--min-required-payment", "--min-required-payment",
help="Requests must have a minimum payment to be processed", help="Requests must have a minimum payment to be processed (in alnt)",
default=0, default=0,
) )
@click.option( @click.option("--lrn", help="The LRN of this deployer.", required=True)
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option( @click.option(
"--all-requests", "--all-requests",
help="Handle requests addressed to anyone (by default only requests to" help="Handle requests addressed to anyone (by default only requests to"
"my payment address are examined).", "my payment address are examined).",
is_flag=True, is_flag=True,
) )
@click.option(
"--config-upload-dir",
help="The directory containing uploaded config.",
required=True,
)
@click.option(
"--private-key-file", help="The private key for decrypting config.", required=True
)
@click.option(
"--private-key-passphrase",
help="The passphrase for the private key.",
required=True,
)
@click.pass_context @click.pass_context
def command( # noqa: C901 def command( # noqa: C901
ctx, ctx,
@ -350,7 +385,10 @@ def command( # noqa: C901
recreate_on_deploy, recreate_on_deploy,
log_dir, log_dir,
min_required_payment, min_required_payment,
payment_address, lrn,
config_upload_dir,
private_key_file,
private_key_passphrase,
all_requests, all_requests,
): ):
if request_id and discover: if request_id and discover:
@ -384,6 +422,18 @@ def command( # noqa: C901
) )
sys.exit(2) sys.exit(2)
tempdir = tempfile.mkdtemp()
gpg = gnupg.GPG(gnupghome=tempdir)
# Import the deployer's public key
result = gpg.import_keys(open(private_key_file, "rb").read())
if 1 != result.imported:
print(
f"Failed to load private key file: {private_key_file}.",
file=sys.stderr,
)
sys.exit(2)
main_logger = TimedLogger(file=sys.stderr) main_logger = TimedLogger(file=sys.stderr)
try: try:
@ -392,11 +442,17 @@ def command( # noqa: C901
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag] exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr) laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr)
if not payment_address: webapp_deployer_record = laconic.get_record(lrn, require=True)
payment_address = laconic.whoami().address payment_address = webapp_deployer_record.attributes.paymentAddress
main_logger.log(f"Payment address: {payment_address}") main_logger.log(f"Payment address: {payment_address}")
if min_required_payment and not payment_address:
print(
f"Minimum payment required, but no payment address listed for deployer: {lrn}.",
file=sys.stderr,
)
sys.exit(2)
# Find deployment requests. # Find deployment requests.
# single request # single request
if request_id: if request_id:
@ -408,7 +464,7 @@ def command( # noqa: C901
if all_requests: if all_requests:
requests = laconic.app_deployment_requests() requests = laconic.app_deployment_requests()
else: else:
requests = laconic.app_deployment_requests({"to": payment_address}) requests = laconic.app_deployment_requests({"deployer": lrn})
if only_update_state: if only_update_state:
if not dry_run: if not dry_run:
@ -487,7 +543,7 @@ def command( # noqa: C901
if all_requests: if all_requests:
deployments = laconic.app_deployments() deployments = laconic.app_deployments()
else: else:
deployments = laconic.app_deployments({"by": payment_address}) deployments = laconic.app_deployments({"deployer": lrn})
deployments_by_request = {} deployments_by_request = {}
for d in deployments: for d in deployments:
if d.attributes.request: if d.attributes.request:
@ -530,7 +586,11 @@ def command( # noqa: C901
for r in requests_to_check_for_payment: for r in requests_to_check_for_payment:
main_logger.log(f"{r.id}: Confirming payment...") main_logger.log(f"{r.id}: Confirming payment...")
if confirm_payment( if confirm_payment(
laconic, r, payment_address, min_required_payment, main_logger laconic,
r,
payment_address,
min_required_payment,
main_logger,
): ):
main_logger.log(f"{r.id}: Payment confirmed.") main_logger.log(f"{r.id}: Payment confirmed.")
requests_to_execute.append(r) requests_to_execute.append(r)
@ -583,7 +643,10 @@ def command( # noqa: C901
force_rebuild, force_rebuild,
fqdn_policy, fqdn_policy,
recreate_on_deploy, recreate_on_deploy,
payment_address, webapp_deployer_record,
gpg,
private_key_passphrase,
config_upload_dir,
build_logger, build_logger,
) )
status = "DEPLOYED" status = "DEPLOYED"
@ -604,3 +667,5 @@ def command( # noqa: C901
except Exception as e: except Exception as e:
main_logger.log("UNCAUGHT ERROR:" + str(e)) main_logger.log("UNCAUGHT ERROR:" + str(e))
raise e raise e
finally:
shutil.rmtree(tempdir, ignore_errors=True)

View File

@ -0,0 +1,91 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import base64
import click
import sys
import yaml
from urllib.parse import urlparse
from stack_orchestrator.deploy.webapp.util import LaconicRegistryClient
@click.command()
@click.option(
"--laconic-config", help="Provide a config file for laconicd", required=True
)
@click.option("--api-url", help="The API URL of the deployer.", required=True)
@click.option(
"--public-key-file",
help="The public key to use. This should be a binary file.",
required=True,
)
@click.option(
"--lrn", help="eg, lrn://laconic/deployers/my.deployer.name", required=True
)
@click.option(
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option(
"--min-required-payment",
help="List the minimum required payment (in alnt) to process a deployment request.",
default=0,
)
@click.option(
"--dry-run",
help="Don't publish anything, just report what would be done.",
is_flag=True,
)
@click.pass_context
def command( # noqa: C901
ctx,
laconic_config,
api_url,
public_key_file,
lrn,
payment_address,
min_required_payment,
dry_run,
):
laconic = LaconicRegistryClient(laconic_config)
if not payment_address:
payment_address = laconic.whoami().address
pub_key = base64.b64encode(open(public_key_file, "rb").read()).decode("ASCII")
hostname = urlparse(api_url).hostname
webapp_deployer_record = {
"record": {
"type": "WebappDeployer",
"version": "1.0.0",
"apiUrl": api_url,
"name": hostname,
"publicKey": pub_key,
"paymentAddress": payment_address,
}
}
if min_required_payment:
webapp_deployer_record["record"][
"minimumPayment"
] = f"{min_required_payment}alnt"
if dry_run:
yaml.dump(webapp_deployer_record, sys.stdout)
return
laconic.publish(webapp_deployer_record, [lrn])

View File

@ -0,0 +1,175 @@
# Copyright ©2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
import base64
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
import shutil
import sys
import tempfile
from datetime import datetime
import gnupg
import click
import requests
import yaml
from stack_orchestrator.deploy.webapp.util import (
LaconicRegistryClient,
)
from dotenv import dotenv_values
def fatal(msg: str):
print(msg, file=sys.stderr)
sys.exit(1)
@click.command()
@click.option(
"--laconic-config", help="Provide a config file for laconicd", required=True
)
@click.option(
"--app",
help="The LRN of the application to deploy.",
required=True,
)
@click.option(
"--deployer",
help="The LRN of the deployer to process this request.",
required=True,
)
@click.option("--env-file", help="environment file for webapp")
@click.option("--config-ref", help="The ref of an existing config upload to use.")
@click.option(
"--make-payment",
help="The payment to make (in alnt). The value should be a number or 'auto' to use the deployer's minimum required payment.",
)
@click.option(
"--use-payment", help="The TX id of an existing, unused payment", default=None
)
@click.option("--dns", help="the DNS name to request (default is autogenerated)")
@click.option(
"--dry-run",
help="Don't publish anything, just report what would be done.",
is_flag=True,
)
@click.pass_context
def command(
ctx,
laconic_config,
app,
deployer,
env_file,
config_ref,
make_payment,
use_payment,
dns,
dry_run,
): # noqa: C901
tempdir = tempfile.mkdtemp()
try:
laconic = LaconicRegistryClient(laconic_config)
app_record = laconic.get_record(app)
if not app_record:
fatal(f"Unable to locate app: {app}")
deployer_record = laconic.get_record(deployer)
if not deployer_record:
fatal(f"Unable to locate deployer: {deployer}")
if env_file and config_ref:
fatal("Cannot use --env-file and --config-ref at the same time.")
# If env_file
if env_file:
gpg = gnupg.GPG(gnupghome=tempdir)
# Import the deployer's public key
result = gpg.import_keys(
base64.b64decode(deployer_record.attributes.publicKey)
)
if 1 != result.imported:
fatal("Failed to import deployer's public key.")
recip = gpg.list_keys()[0]["uids"][0]
# Wrap the config
config = {
# Include account (and payment?) details
"authorized": [laconic.whoami().address],
"config": {"env": dict(dotenv_values(env_file))},
}
serialized = yaml.dump(config)
# Encrypt
result = gpg.encrypt(serialized, recip, always_trust=True, armor=False)
if not result.ok:
fatal("Failed to encrypt config.")
# Upload it to the deployer's API
response = requests.post(
f"{deployer_record.attributes.apiUrl}/upload/config",
data=result.data,
headers={"Content-Type": "application/octet-stream"},
)
if not response.ok:
response.raise_for_status()
config_ref = response.json()["id"]
deployment_request = {
"record": {
"type": "ApplicationDeploymentRequest",
"application": app,
"version": "1.0.0",
"name": f"{app_record.attributes.name}@{app_record.attributes.version}",
"deployer": deployer,
"meta": {"when": str(datetime.utcnow())},
}
}
if config_ref:
deployment_request["record"]["config"] = {"ref": config_ref}
if dns:
deployment_request["record"]["dns"] = dns.lower()
if make_payment:
amount = 0
if dry_run:
deployment_request["record"]["payment"] = "DRY_RUN"
elif "auto" == make_payment:
if "minimumPayment" in deployer_record.attributes:
amount = int(
deployer_record.attributes.minimumPayment.replace("alnt", "")
)
else:
amount = make_payment
if amount:
receipt = laconic.send_tokens(
deployer_record.attributes.paymentAddress, amount
)
deployment_request["record"]["payment"] = receipt.tx.hash
print("Payment TX:", receipt.tx.hash)
elif use_payment:
deployment_request["record"]["payment"] = use_payment
if dry_run:
print(yaml.dump(deployment_request))
return
# Send the request
laconic.publish(deployment_request)
finally:
shutil.rmtree(tempdir, ignore_errors=True)

View File

@ -38,7 +38,7 @@ def process_app_removal_request(
deployment_parent_dir, deployment_parent_dir,
delete_volumes, delete_volumes,
delete_names, delete_names,
payment_address, webapp_deployer_record,
): ):
deployment_record = laconic.get_record( deployment_record = laconic.get_record(
app_removal_request.attributes.deployment, require=True app_removal_request.attributes.deployment, require=True
@ -84,7 +84,7 @@ def process_app_removal_request(
"version": "1.0.0", "version": "1.0.0",
"request": app_removal_request.id, "request": app_removal_request.id,
"deployment": deployment_record.id, "deployment": deployment_record.id,
"by": payment_address, "deployer": webapp_deployer_record.names[0],
} }
} }
@ -168,15 +168,10 @@ def dump_known_requests(filename, requests):
) )
@click.option( @click.option(
"--min-required-payment", "--min-required-payment",
help="Requests must have a minimum payment to be processed", help="Requests must have a minimum payment to be processed (in alnt)",
default=0, default=0,
) )
@click.option( @click.option("--lrn", help="The LRN of this deployer.", required=True)
"--payment-address",
help="The address to which payments should be made. "
"Default is the current laconic account.",
default=None,
)
@click.option( @click.option(
"--all-requests", "--all-requests",
help="Handle requests addressed to anyone (by default only requests to" help="Handle requests addressed to anyone (by default only requests to"
@ -198,7 +193,7 @@ def command( # noqa: C901
include_tags, include_tags,
exclude_tags, exclude_tags,
min_required_payment, min_required_payment,
payment_address, lrn,
all_requests, all_requests,
): ):
if request_id and discover: if request_id and discover:
@ -218,8 +213,16 @@ def command( # noqa: C901
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag] exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr) laconic = LaconicRegistryClient(laconic_config, log_file=sys.stderr)
if not payment_address: deployer_record = laconic.get_record(lrn, require=True)
payment_address = laconic.whoami().address payment_address = deployer_record.attributes.paymentAddress
main_logger.log(f"Payment address: {payment_address}")
if min_required_payment and not payment_address:
print(
f"Minimum payment required, but no payment address listed for deployer: {lrn}.",
file=sys.stderr,
)
sys.exit(2)
# Find deployment removal requests. # Find deployment removal requests.
# single request # single request
@ -233,7 +236,7 @@ def command( # noqa: C901
if all_requests: if all_requests:
requests = laconic.app_deployment_removal_requests() requests = laconic.app_deployment_removal_requests()
else: else:
requests = laconic.app_deployment_removal_requests({"to": payment_address}) requests = laconic.app_deployment_removal_requests({"deployer": lrn})
if only_update_state: if only_update_state:
if not dry_run: if not dry_run:
@ -312,7 +315,11 @@ def command( # noqa: C901
for r in requests_to_check_for_payment: for r in requests_to_check_for_payment:
main_logger.log(f"{r.id}: Confirming payment...") main_logger.log(f"{r.id}: Confirming payment...")
if confirm_payment( if confirm_payment(
laconic, r, payment_address, min_required_payment, main_logger laconic,
r,
payment_address,
min_required_payment,
main_logger,
): ):
main_logger.log(f"{r.id}: Payment confirmed.") main_logger.log(f"{r.id}: Payment confirmed.")
requests_to_execute.append(r) requests_to_execute.append(r)
@ -336,7 +343,7 @@ def command( # noqa: C901
os.path.abspath(deployment_parent_dir), os.path.abspath(deployment_parent_dir),
delete_volumes, delete_volumes,
delete_names, delete_names,
payment_address, deployer_record,
) )
except Exception as e: except Exception as e:
main_logger.log(f"ERROR processing removal request {r.id}: {e}") main_logger.log(f"ERROR processing removal request {r.id}: {e}")

View File

@ -1,4 +1,4 @@
# Copyright © 2023 Vulcanize # = str(min_required_payment) Copyright © 2023 Vulcanize
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by # it under the terms of the GNU Affero General Public License as published by
@ -142,14 +142,14 @@ def confirm_payment(laconic, record, payment_address, min_amount, logger):
# Check if the payment was already used on a # Check if the payment was already used on a
used = laconic.app_deployments( used = laconic.app_deployments(
{"by": payment_address, "payment": tx.hash}, all=True {"deployer": payment_address, "payment": tx.hash}, all=True
) )
if len(used): if len(used):
logger.log(f"{record.id}: payment {tx.hash} already used on deployment {used}") logger.log(f"{record.id}: payment {tx.hash} already used on deployment {used}")
return False return False
used = laconic.app_deployment_removals( used = laconic.app_deployment_removals(
{"by": payment_address, "payment": tx.hash}, all=True {"deployer": payment_address, "payment": tx.hash}, all=True
) )
if len(used): if len(used):
logger.log( logger.log(
@ -453,6 +453,24 @@ class LaconicRegistryClient:
name, name,
) )
def send_tokens(self, address, amount, type="alnt"):
args = [
"laconic",
"-c",
self.config_file,
"registry",
"tokens",
"send",
"--address",
address,
"--quantity",
str(amount),
"--type",
type,
]
return AttrDict(json.loads(logged_cmd(self.log_file, *args)))
def file_hash(filename): def file_hash(filename):
return hashlib.sha1(open(filename).read().encode()).hexdigest() return hashlib.sha1(open(filename).read().encode()).hexdigest()
@ -609,7 +627,7 @@ def publish_deployment(
dns_lrn, dns_lrn,
deployment_dir, deployment_dir,
app_deployment_request=None, app_deployment_request=None,
payment_address=None, webapp_deployer_record=None,
logger=None, logger=None,
): ):
if not deploy_record: if not deploy_record:
@ -666,8 +684,8 @@ def publish_deployment(
"payment" "payment"
] = app_deployment_request.attributes.payment ] = app_deployment_request.attributes.payment
if payment_address: if webapp_deployer_record:
new_deployment_record["record"]["by"] = payment_address new_deployment_record["record"]["deployer"] = webapp_deployer_record.names[0]
if logger: if logger:
logger.log("Publishing ApplicationDeploymentRecord.") logger.log("Publishing ApplicationDeploymentRecord.")

View File

@ -24,7 +24,9 @@ from stack_orchestrator.build import build_webapp
from stack_orchestrator.deploy.webapp import (run_webapp, from stack_orchestrator.deploy.webapp import (run_webapp,
deploy_webapp, deploy_webapp,
deploy_webapp_from_registry, deploy_webapp_from_registry,
undeploy_webapp_from_registry) undeploy_webapp_from_registry,
publish_webapp_deployer,
request_webapp_deployment)
from stack_orchestrator.deploy import deploy from stack_orchestrator.deploy import deploy
from stack_orchestrator import version from stack_orchestrator import version
from stack_orchestrator.deploy import deployment from stack_orchestrator.deploy import deployment
@ -61,6 +63,8 @@ cli.add_command(run_webapp.command, "run-webapp")
cli.add_command(deploy_webapp.command, "deploy-webapp") cli.add_command(deploy_webapp.command, "deploy-webapp")
cli.add_command(deploy_webapp_from_registry.command, "deploy-webapp-from-registry") cli.add_command(deploy_webapp_from_registry.command, "deploy-webapp-from-registry")
cli.add_command(undeploy_webapp_from_registry.command, "undeploy-webapp-from-registry") cli.add_command(undeploy_webapp_from_registry.command, "undeploy-webapp-from-registry")
cli.add_command(publish_webapp_deployer.command, "publish-deployer-to-registry")
cli.add_command(request_webapp_deployment.command, "request-webapp-deployment")
cli.add_command(deploy.command, "deploy") # deploy is an alias for deploy-system cli.add_command(deploy.command, "deploy") # deploy is an alias for deploy-system
cli.add_command(deploy.command, "deploy-system") cli.add_command(deploy.command, "deploy-system")
cli.add_command(deployment.command, "deployment") cli.add_command(deployment.command, "deployment")