Compare commits

...
Author SHA1 Message Date
A. F. DudleyandClaude Opus 4.6 6a2f2a5dde fix(k8s): drop stale ACME accounts during etcd cleanup
Lint Checks / Run linter (push) Successful in 13s
Lint Checks / Run linter (pull_request) Successful in 16s
Smoke Test / Run basic test suite (pull_request) Successful in 1m23s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (pull_request) Failing after 1m53s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Failing after 2m7s
Webapp Test / Run webapp test suite (pull_request) Successful in 2m8s
Deploy Test / Run deploy test suite (pull_request) Successful in 2m17s
_clean_etcd_keeping_certs() preserved ALL caddy-system secrets across
cluster recreations, including ACME account secrets registered with
wrong/empty email. Caddy reuses these stale accounts instead of
registering fresh ones, causing recurring "unable to parse email
address" errors.

Filter the etcd restore loop to only keep certificate secrets (keys
matching *certificates*). ACME accounts, OCSP staples, and locks are
transient and get recreated automatically by Caddy on startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 01:43:48 -05:00
AFDudley 4a1b5d86fd Merge pull request 'fix(k8s): translate service names to localhost for sidecar containers' (#989) from fix-sidecar-localhost into main
Lint Checks / Run linter (push) Successful in 16s
Publish / Build and publish (push) Successful in 29s
Deploy Test / Run deploy test suite (push) Successful in 2m10s
Webapp Test / Run webapp test suite (push) Successful in 3m51s
Smoke Test / Run basic test suite (push) Successful in 3m51s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 19m36s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Failing after 2m52s
Database Test / Run database hosting test on kind/k8s (push) Failing after 8m16s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Failing after 3m50s
External Stack Test / Run external stack test suite (push) Failing after 3m19s
Reviewed-on: #989
2026-02-03 23:13:27 +00:00
A. F. DudleyandClaude Opus 4.5 019225ca18 fix(k8s): translate service names to localhost for sidecar containers
Lint Checks / Run linter (push) Failing after 3s
Lint Checks / Run linter (pull_request) Failing after 4s
Deploy Test / Run deploy test suite (pull_request) Failing after 4s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Failing after 5s
K8s Deployment Control Test / Run deployment control suite on kind/k8s (pull_request) Failing after 4s
Webapp Test / Run webapp test suite (pull_request) Failing after 5s
Smoke Test / Run basic test suite (pull_request) Failing after 5s
In docker-compose, services can reference each other by name (e.g., 'db:5432').
In Kubernetes, when multiple containers are in the same pod (sidecars), they
share the same network namespace and must use 'localhost' instead.

This fix adds translate_sidecar_service_names() which replaces docker-compose
service name references with 'localhost' in environment variable values for
containers that share the same pod.

Fixes issue where multi-container pods fail because one container tries to
connect to a sibling using the compose service name instead of localhost.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 18:10:32 -05:00
AFDudley 0296da6f64 Merge pull request 'feat(k8s): namespace-per-deployment for resource isolation and cleanup' (#988) from feat-namespace-per-deployment into main
Lint Checks / Run linter (push) Failing after 5s
Deploy Test / Run deploy test suite (push) Failing after 5s
Publish / Build and publish (push) Failing after 6s
Webapp Test / Run webapp test suite (push) Failing after 5s
Smoke Test / Run basic test suite (push) Failing after 5s
Reviewed-on: #988
2026-02-03 23:09:16 +00:00
2 changed files with 55 additions and 3 deletions
@@ -31,6 +31,7 @@ from stack_orchestrator.deploy.k8s.helpers import (
envs_from_environment_variables_map,
envs_from_compose_file,
merge_envs,
translate_sidecar_service_names,
)
from stack_orchestrator.deploy.deploy_util import (
parsed_pod_files_map_from_file_names,
@@ -439,6 +440,12 @@ class ClusterInfo:
if "environment" in service_info
else self.environment_variables.map
)
# Translate docker-compose service names to localhost for sidecars
# All services in the same pod share the network namespace
sibling_services = [s for s in services.keys() if s != service_name]
merged_envs = translate_sidecar_service_names(
merged_envs, sibling_services
)
envs = envs_from_environment_variables_map(merged_envs)
if opts.o.debug:
print(f"Merged envs: {envs}")
+48 -3
View File
@@ -118,12 +118,17 @@ def _get_etcd_host_path_from_kind_config(config_file: str) -> Optional[str]:
def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
"""Clean persisted etcd, keeping only TLS certificates.
"""Clean persisted etcd, keeping only TLS certificate secrets.
When etcd is persisted and a cluster is recreated, kind tries to install
resources fresh but they already exist. Instead of trying to delete
specific stale resources (blacklist), we keep only the valuable data
(caddy TLS certs) and delete everything else (whitelist approach).
(caddy TLS certificate secrets) and delete everything else.
ACME account secrets are intentionally NOT preserved — they may contain
stale registrations (e.g. with wrong/empty email) that cause "unable to
parse email address" errors. Caddy re-registers accounts automatically
on startup using the email from the ConfigMap.
The etcd image is distroless (no shell), so we extract the statically-linked
etcdctl binary and run it from alpine which has shell support.
@@ -218,6 +223,11 @@ def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
jq -r ".kvs[] | @base64" /backup/kept.json 2>/dev/null | \
while read encoded; do
key=$(echo $encoded | base64 -d | jq -r ".key" | base64 -d)
# Keep only certificate secrets, drop ACME accounts/locks/OCSP
case "$key" in
*certificates*) ;;
*) continue ;;
esac
val=$(echo $encoded | base64 -d | jq -r ".value" | base64 -d)
echo "$val" | /backup/etcdctl put "$key"
done
@@ -259,7 +269,7 @@ def _clean_etcd_keeping_certs(etcd_path: str) -> bool:
return False
if opts.o.debug:
print("Cleaned etcd, kept only TLS certificates")
print("Cleaned etcd, kept only TLS certificate secrets (dropped ACME accounts)")
return True
@@ -942,6 +952,41 @@ def envs_from_compose_file(
return result
def translate_sidecar_service_names(
envs: Mapping[str, str], sibling_service_names: List[str]
) -> Mapping[str, str]:
"""Translate docker-compose service names to localhost for sidecar containers.
In docker-compose, services can reference each other by name (e.g., 'db:5432').
In Kubernetes, when multiple containers are in the same pod (sidecars), they
share the same network namespace and must use 'localhost' instead.
This function replaces service name references with 'localhost' in env values.
"""
import re
if not sibling_service_names:
return envs
result = {}
for env_var, env_val in envs.items():
if env_val is None:
result[env_var] = env_val
continue
new_val = str(env_val)
for service_name in sibling_service_names:
# Match service name followed by optional port (e.g., 'db:5432', 'db')
# Handle URLs like: postgres://user:pass@db:5432/dbname
# and simple refs like: db:5432 or just db
pattern = rf"\b{re.escape(service_name)}(:\d+)?\b"
new_val = re.sub(pattern, lambda m: f'localhost{m.group(1) or ""}', new_val)
result[env_var] = new_val
return result
def envs_from_environment_variables_map(
map: Mapping[str, str]
) -> List[client.V1EnvVar]: