Compare commits

..
Author SHA1 Message Date
A. F. DudleyandClaude Opus 4.6 6a2f2a5dde fix(k8s): drop stale ACME accounts during etcd cleanup
Lint Checks / Run linter (push) Successful in 13s
Lint Checks / Run linter (pull_request) Successful in 16s
Smoke Test / Run basic test suite (pull_request) Successful in 1m23s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (pull_request) Failing after 1m53s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Failing after 2m7s
Webapp Test / Run webapp test suite (pull_request) Successful in 2m8s
Deploy Test / Run deploy test suite (pull_request) Successful in 2m17s
_clean_etcd_keeping_certs() preserved ALL caddy-system secrets across
cluster recreations, including ACME account secrets registered with
wrong/empty email. Caddy reuses these stale accounts instead of
registering fresh ones, causing recurring "unable to parse email
address" errors.

Filter the etcd restore loop to only keep certificate secrets (keys
matching *certificates*). ACME accounts, OCSP staples, and locks are
transient and get recreated automatically by Caddy on startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 01:43:48 -05:00
7 changed files with 24 additions and 87 deletions
+10 -66
View File
@@ -68,7 +68,7 @@ $ laconic-so build-npms --include <package-name> --force-rebuild
## deploy
The `deploy` command group manages persistent deployments. The general workflow is `deploy init` to generate a spec file, then `deploy create` to create a deployment directory from the spec, then runtime commands like `deployment start` and `deployment stop`.
The `deploy` command group manages persistent deployments. The general workflow is `deploy init` to generate a spec file, then `deploy create` to create a deployment directory from the spec, then runtime commands like `deploy up` and `deploy down`.
### deploy init
@@ -101,91 +101,35 @@ Options:
- `--spec-file` (required): spec file to use
- `--deployment-dir`: target directory for deployment files
- `--update`: update an existing deployment directory, preserving data volumes and env file. Changed files are backed up with a `.bak` suffix. The deployment's `config.env` and `deployment.yml` are also preserved.
- `--helm-chart`: generate Helm chart instead of deploying (k8s only)
- `--network-dir`: network configuration supplied in this directory
- `--initial-peers`: initial set of persistent peers
## deployment
### deploy up
Runtime commands for managing a created deployment. Use `--dir` to specify the deployment directory.
### deployment start
Start a deployment (`up` is a legacy alias):
Start a deployment:
```
$ laconic-so deployment --dir <deployment-dir> start
$ laconic-so deployment --dir <deployment-dir> up
```
Options:
- `--stay-attached` / `--detatch-terminal`: attach to container stdout (default: detach)
- `--skip-cluster-management` / `--perform-cluster-management`: skip kind cluster creation/teardown (default: perform management). Only affects k8s-kind deployments. Use this when multiple stacks share a single cluster.
### deploy down
### deployment stop
Stop a deployment (`down` is a legacy alias):
Stop a deployment:
```
$ laconic-so deployment --dir <deployment-dir> stop
$ laconic-so deployment --dir <deployment-dir> down
```
Use `--delete-volumes` to also remove data volumes.
Options:
- `--delete-volumes` / `--preserve-volumes`: delete data volumes on stop (default: preserve)
- `--skip-cluster-management` / `--perform-cluster-management`: skip kind cluster teardown (default: perform management). Use this to stop a single deployment without destroying a shared cluster.
### deployment restart
Restart a deployment with GitOps-aware workflow. Pulls latest stack code, syncs the deployment directory from the git-tracked spec, and restarts services:
```
$ laconic-so deployment --dir <deployment-dir> restart
```
See [deployment_patterns.md](deployment_patterns.md) for the recommended GitOps workflow.
### deployment ps
### deploy ps
Show running services:
```
$ laconic-so deployment --dir <deployment-dir> ps
```
### deployment logs
### deploy logs
View service logs:
```
$ laconic-so deployment --dir <deployment-dir> logs
```
Use `-f` to follow and `-n <count>` to tail.
### deployment exec
Execute a command in a running service container:
```
$ laconic-so deployment --dir <deployment-dir> exec <service-name> "<command>"
```
### deployment status
Show deployment status:
```
$ laconic-so deployment --dir <deployment-dir> status
```
### deployment port
Show mapped ports for a service:
```
$ laconic-so deployment --dir <deployment-dir> port <service-name> <port>
```
### deployment push-images
Push deployment images to a registry:
```
$ laconic-so deployment --dir <deployment-dir> push-images
```
### deployment run-job
Run a one-time job in the deployment:
```
$ laconic-so deployment --dir <deployment-dir> run-job <job-name>
```
+1 -1
View File
@@ -30,7 +30,7 @@ git commit -m "Add my-stack deployment configuration"
git push
# On deployment server: deploy from git-tracked spec
laconic-so --stack my-stack deploy create \
laconic-so deploy create \
--spec-file /path/to/operator-repo/spec.yml \
--deployment-dir my-deployment
-1
View File
@@ -29,7 +29,6 @@ network_key = "network"
http_proxy_key = "http-proxy"
image_registry_key = "image-registry"
configmaps_key = "configmaps"
secrets_key = "secrets"
resources_key = "resources"
volumes_key = "volumes"
security_key = "security"
@@ -477,9 +477,6 @@ def init_operation(
spec_file_content["volumes"] = {**volume_descriptors, **orig_volumes}
if configmap_descriptors:
spec_file_content["configmaps"] = configmap_descriptors
if "k8s" in deployer_type:
if "secrets" not in spec_file_content:
spec_file_content["secrets"] = {}
if opts.o.debug:
print(
@@ -483,16 +483,6 @@ class ClusterInfo:
)
)
]
# Mount user-declared secrets from spec.yml
for user_secret_name in self.spec.get_secrets():
env_from.append(
client.V1EnvFromSource(
secret_ref=client.V1SecretEnvSource(
name=user_secret_name,
optional=True,
)
)
)
container = client.V1Container(
name=container_name,
image=image_to_use,
+13 -3
View File
@@ -118,12 +118,17 @@ def _get_etcd_host_path_from_kind_config(config_file: str) -> Optional[str]:
def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
"""Clean persisted etcd, keeping only TLS certificates.
"""Clean persisted etcd, keeping only TLS certificate secrets.
When etcd is persisted and a cluster is recreated, kind tries to install
resources fresh but they already exist. Instead of trying to delete
specific stale resources (blacklist), we keep only the valuable data
(caddy TLS certs) and delete everything else (whitelist approach).
(caddy TLS certificate secrets) and delete everything else.
ACME account secrets are intentionally NOT preserved — they may contain
stale registrations (e.g. with wrong/empty email) that cause "unable to
parse email address" errors. Caddy re-registers accounts automatically
on startup using the email from the ConfigMap.
The etcd image is distroless (no shell), so we extract the statically-linked
etcdctl binary and run it from alpine which has shell support.
@@ -218,6 +223,11 @@ def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
jq -r ".kvs[] | @base64" /backup/kept.json 2>/dev/null | \
while read encoded; do
key=$(echo $encoded | base64 -d | jq -r ".key" | base64 -d)
# Keep only certificate secrets, drop ACME accounts/locks/OCSP
case "$key" in
*certificates*) ;;
*) continue ;;
esac
val=$(echo $encoded | base64 -d | jq -r ".value" | base64 -d)
echo "$val" | /backup/etcdctl put "$key"
done
@@ -259,7 +269,7 @@ def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
return False
if opts.o.debug:
print("Cleaned etcd, kept only TLS certificates")
print("Cleaned etcd, kept only TLS certificate secrets (dropped ACME accounts)")
return True
-3
View File
@@ -115,9 +115,6 @@ class Spec:
def get_configmaps(self):
return self.obj.get(constants.configmaps_key, {})
def get_secrets(self):
return self.obj.get(constants.secrets_key, {})
def get_container_resources(self):
return Resources(
self.obj.get(constants.resources_key, {}).get("containers", {})