Archived
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
98dcf5d967 | ||
|
|
9b91213bf8 |
@@ -7,25 +7,6 @@ We need an "update stack" command in stack orchestrator and cleaner documentatio
|
|||||||
|
|
||||||
**Context**: Currently, `deploy init` generates a spec file and `deploy create` creates a deployment directory. The `deployment update` command (added by Thomas Lackey) only syncs env vars and restarts - it doesn't regenerate configurations. There's a gap in the workflow for updating stack configurations after initial deployment.
|
**Context**: Currently, `deploy init` generates a spec file and `deploy create` creates a deployment directory. The `deployment update` command (added by Thomas Lackey) only syncs env vars and restarts - it doesn't regenerate configurations. There's a gap in the workflow for updating stack configurations after initial deployment.
|
||||||
|
|
||||||
## Bugs
|
|
||||||
|
|
||||||
### `deploy create` doesn't auto-generate volume mappings for new pods
|
|
||||||
|
|
||||||
When a new pod is added to `stack.yml` (e.g. `monitoring`), `deploy create`
|
|
||||||
does not generate default host path mappings in spec.yml for the new pod's
|
|
||||||
volumes. The deployment then fails at scheduling because the PVCs don't exist.
|
|
||||||
|
|
||||||
**Expected**: `deploy create` enumerates all volumes from all compose files
|
|
||||||
in the stack and generates default host paths for any that aren't already
|
|
||||||
mapped in the spec.yml `volumes:` section.
|
|
||||||
|
|
||||||
**Actual**: Only volumes already in spec.yml get PVs. New volumes are silently
|
|
||||||
missing, causing `FailedScheduling: persistentvolumeclaim not found`.
|
|
||||||
|
|
||||||
**Workaround**: Manually add volume entries to spec.yml and create host dirs.
|
|
||||||
|
|
||||||
**Files**: `deployment_create.py` (`_write_config_file`, volume handling)
|
|
||||||
|
|
||||||
## Architecture Refactoring
|
## Architecture Refactoring
|
||||||
|
|
||||||
### Separate Deployer from Stack Orchestrator CLI
|
### Separate Deployer from Stack Orchestrator CLI
|
||||||
|
|||||||
+68
-12
@@ -68,7 +68,7 @@ $ laconic-so build-npms --include <package-name> --force-rebuild
|
|||||||
|
|
||||||
## deploy
|
## deploy
|
||||||
|
|
||||||
The `deploy` command group manages persistent deployments. The general workflow is `deploy init` to generate a spec file, then `deploy create` to create a deployment directory from the spec, then runtime commands like `deploy up` and `deploy down`.
|
The `deploy` command group manages persistent deployments. The general workflow is `deploy init` to generate a spec file, then `deploy create` to create a deployment directory from the spec, then runtime commands like `deployment start` and `deployment stop`.
|
||||||
|
|
||||||
### deploy init
|
### deploy init
|
||||||
|
|
||||||
@@ -101,35 +101,91 @@ Options:
|
|||||||
- `--spec-file` (required): spec file to use
|
- `--spec-file` (required): spec file to use
|
||||||
- `--deployment-dir`: target directory for deployment files
|
- `--deployment-dir`: target directory for deployment files
|
||||||
- `--update`: update an existing deployment directory, preserving data volumes and env file. Changed files are backed up with a `.bak` suffix. The deployment's `config.env` and `deployment.yml` are also preserved.
|
- `--update`: update an existing deployment directory, preserving data volumes and env file. Changed files are backed up with a `.bak` suffix. The deployment's `config.env` and `deployment.yml` are also preserved.
|
||||||
|
- `--helm-chart`: generate Helm chart instead of deploying (k8s only)
|
||||||
- `--network-dir`: network configuration supplied in this directory
|
- `--network-dir`: network configuration supplied in this directory
|
||||||
- `--initial-peers`: initial set of persistent peers
|
- `--initial-peers`: initial set of persistent peers
|
||||||
|
|
||||||
### deploy up
|
## deployment
|
||||||
|
|
||||||
Start a deployment:
|
Runtime commands for managing a created deployment. Use `--dir` to specify the deployment directory.
|
||||||
|
|
||||||
|
### deployment start
|
||||||
|
|
||||||
|
Start a deployment (`up` is a legacy alias):
|
||||||
```
|
```
|
||||||
$ laconic-so deployment --dir <deployment-dir> up
|
$ laconic-so deployment --dir <deployment-dir> start
|
||||||
```
|
```
|
||||||
|
|
||||||
### deploy down
|
Options:
|
||||||
|
- `--stay-attached` / `--detatch-terminal`: attach to container stdout (default: detach)
|
||||||
|
- `--skip-cluster-management` / `--perform-cluster-management`: skip kind cluster creation/teardown (default: perform management). Only affects k8s-kind deployments. Use this when multiple stacks share a single cluster.
|
||||||
|
|
||||||
Stop a deployment:
|
### deployment stop
|
||||||
```
|
|
||||||
$ laconic-so deployment --dir <deployment-dir> down
|
|
||||||
```
|
|
||||||
Use `--delete-volumes` to also remove data volumes.
|
|
||||||
|
|
||||||
### deploy ps
|
Stop a deployment (`down` is a legacy alias):
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> stop
|
||||||
|
```
|
||||||
|
|
||||||
|
Options:
|
||||||
|
- `--delete-volumes` / `--preserve-volumes`: delete data volumes on stop (default: preserve)
|
||||||
|
- `--skip-cluster-management` / `--perform-cluster-management`: skip kind cluster teardown (default: perform management). Use this to stop a single deployment without destroying a shared cluster.
|
||||||
|
|
||||||
|
### deployment restart
|
||||||
|
|
||||||
|
Restart a deployment with GitOps-aware workflow. Pulls latest stack code, syncs the deployment directory from the git-tracked spec, and restarts services:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> restart
|
||||||
|
```
|
||||||
|
|
||||||
|
See [deployment_patterns.md](deployment_patterns.md) for the recommended GitOps workflow.
|
||||||
|
|
||||||
|
### deployment ps
|
||||||
|
|
||||||
Show running services:
|
Show running services:
|
||||||
```
|
```
|
||||||
$ laconic-so deployment --dir <deployment-dir> ps
|
$ laconic-so deployment --dir <deployment-dir> ps
|
||||||
```
|
```
|
||||||
|
|
||||||
### deploy logs
|
### deployment logs
|
||||||
|
|
||||||
View service logs:
|
View service logs:
|
||||||
```
|
```
|
||||||
$ laconic-so deployment --dir <deployment-dir> logs
|
$ laconic-so deployment --dir <deployment-dir> logs
|
||||||
```
|
```
|
||||||
Use `-f` to follow and `-n <count>` to tail.
|
Use `-f` to follow and `-n <count>` to tail.
|
||||||
|
|
||||||
|
### deployment exec
|
||||||
|
|
||||||
|
Execute a command in a running service container:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> exec <service-name> "<command>"
|
||||||
|
```
|
||||||
|
|
||||||
|
### deployment status
|
||||||
|
|
||||||
|
Show deployment status:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> status
|
||||||
|
```
|
||||||
|
|
||||||
|
### deployment port
|
||||||
|
|
||||||
|
Show mapped ports for a service:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> port <service-name> <port>
|
||||||
|
```
|
||||||
|
|
||||||
|
### deployment push-images
|
||||||
|
|
||||||
|
Push deployment images to a registry:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> push-images
|
||||||
|
```
|
||||||
|
|
||||||
|
### deployment run-job
|
||||||
|
|
||||||
|
Run a one-time job in the deployment:
|
||||||
|
```
|
||||||
|
$ laconic-so deployment --dir <deployment-dir> run-job <job-name>
|
||||||
|
```
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ git commit -m "Add my-stack deployment configuration"
|
|||||||
git push
|
git push
|
||||||
|
|
||||||
# On deployment server: deploy from git-tracked spec
|
# On deployment server: deploy from git-tracked spec
|
||||||
laconic-so deploy create \
|
laconic-so --stack my-stack deploy create \
|
||||||
--spec-file /path/to/operator-repo/spec.yml \
|
--spec-file /path/to/operator-repo/spec.yml \
|
||||||
--deployment-dir my-deployment
|
--deployment-dir my-deployment
|
||||||
|
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ network_key = "network"
|
|||||||
http_proxy_key = "http-proxy"
|
http_proxy_key = "http-proxy"
|
||||||
image_registry_key = "image-registry"
|
image_registry_key = "image-registry"
|
||||||
configmaps_key = "configmaps"
|
configmaps_key = "configmaps"
|
||||||
|
secrets_key = "secrets"
|
||||||
resources_key = "resources"
|
resources_key = "resources"
|
||||||
volumes_key = "volumes"
|
volumes_key = "volumes"
|
||||||
security_key = "security"
|
security_key = "security"
|
||||||
|
|||||||
@@ -477,6 +477,9 @@ def init_operation(
|
|||||||
spec_file_content["volumes"] = {**volume_descriptors, **orig_volumes}
|
spec_file_content["volumes"] = {**volume_descriptors, **orig_volumes}
|
||||||
if configmap_descriptors:
|
if configmap_descriptors:
|
||||||
spec_file_content["configmaps"] = configmap_descriptors
|
spec_file_content["configmaps"] = configmap_descriptors
|
||||||
|
if "k8s" in deployer_type:
|
||||||
|
if "secrets" not in spec_file_content:
|
||||||
|
spec_file_content["secrets"] = {}
|
||||||
|
|
||||||
if opts.o.debug:
|
if opts.o.debug:
|
||||||
print(
|
print(
|
||||||
|
|||||||
@@ -394,43 +394,13 @@ class ClusterInfo:
|
|||||||
result.append(pv)
|
result.append(pv)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def _any_service_has_host_network(self):
|
|
||||||
for pod_name in self.parsed_pod_yaml_map:
|
|
||||||
pod = self.parsed_pod_yaml_map[pod_name]
|
|
||||||
for svc in pod.get("services", {}).values():
|
|
||||||
if svc.get("network_mode") == "host":
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
def _resolve_container_resources(
|
|
||||||
self, container_name: str, service_info: dict, global_resources: Resources
|
|
||||||
) -> Resources:
|
|
||||||
"""Resolve resources for a container using layered priority.
|
|
||||||
|
|
||||||
Priority: spec per-container > compose deploy.resources
|
|
||||||
> spec global > DEFAULT
|
|
||||||
"""
|
|
||||||
# 1. Check spec.yml for per-container override
|
|
||||||
per_container = self.spec.get_container_resources_for(container_name)
|
|
||||||
if per_container:
|
|
||||||
return per_container
|
|
||||||
|
|
||||||
# 2. Check compose service_info for deploy.resources
|
|
||||||
deploy_block = service_info.get("deploy", {})
|
|
||||||
compose_resources = deploy_block.get("resources", {}) if deploy_block else {}
|
|
||||||
if compose_resources:
|
|
||||||
return Resources(compose_resources)
|
|
||||||
|
|
||||||
# 3. Fall back to spec.yml global (already resolved with DEFAULT fallback)
|
|
||||||
return global_resources
|
|
||||||
|
|
||||||
# TODO: put things like image pull policy into an object-scope struct
|
# TODO: put things like image pull policy into an object-scope struct
|
||||||
def get_deployment(self, image_pull_policy: Optional[str] = None):
|
def get_deployment(self, image_pull_policy: Optional[str] = None):
|
||||||
containers = []
|
containers = []
|
||||||
services = {}
|
services = {}
|
||||||
global_resources = self.spec.get_container_resources()
|
resources = self.spec.get_container_resources()
|
||||||
if not global_resources:
|
if not resources:
|
||||||
global_resources = DEFAULT_CONTAINER_RESOURCES
|
resources = DEFAULT_CONTAINER_RESOURCES
|
||||||
for pod_name in self.parsed_pod_yaml_map:
|
for pod_name in self.parsed_pod_yaml_map:
|
||||||
pod = self.parsed_pod_yaml_map[pod_name]
|
pod = self.parsed_pod_yaml_map[pod_name]
|
||||||
services = pod["services"]
|
services = pod["services"]
|
||||||
@@ -513,9 +483,16 @@ class ClusterInfo:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
container_resources = self._resolve_container_resources(
|
# Mount user-declared secrets from spec.yml
|
||||||
container_name, service_info, global_resources
|
for user_secret_name in self.spec.get_secrets():
|
||||||
)
|
env_from.append(
|
||||||
|
client.V1EnvFromSource(
|
||||||
|
secret_ref=client.V1SecretEnvSource(
|
||||||
|
name=user_secret_name,
|
||||||
|
optional=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
container = client.V1Container(
|
container = client.V1Container(
|
||||||
name=container_name,
|
name=container_name,
|
||||||
image=image_to_use,
|
image=image_to_use,
|
||||||
@@ -534,7 +511,7 @@ class ClusterInfo:
|
|||||||
if self.spec.get_capabilities()
|
if self.spec.get_capabilities()
|
||||||
else None,
|
else None,
|
||||||
),
|
),
|
||||||
resources=to_k8s_resource_requirements(container_resources),
|
resources=to_k8s_resource_requirements(resources),
|
||||||
)
|
)
|
||||||
containers.append(container)
|
containers.append(container)
|
||||||
volumes = volumes_for_pod_files(
|
volumes = volumes_for_pod_files(
|
||||||
@@ -601,7 +578,6 @@ class ClusterInfo:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
use_host_network = self._any_service_has_host_network()
|
|
||||||
template = client.V1PodTemplateSpec(
|
template = client.V1PodTemplateSpec(
|
||||||
metadata=client.V1ObjectMeta(annotations=annotations, labels=labels),
|
metadata=client.V1ObjectMeta(annotations=annotations, labels=labels),
|
||||||
spec=client.V1PodSpec(
|
spec=client.V1PodSpec(
|
||||||
@@ -611,8 +587,6 @@ class ClusterInfo:
|
|||||||
affinity=affinity,
|
affinity=affinity,
|
||||||
tolerations=tolerations,
|
tolerations=tolerations,
|
||||||
runtime_class_name=self.spec.get_runtime_class(),
|
runtime_class_name=self.spec.get_runtime_class(),
|
||||||
host_network=use_host_network or None,
|
|
||||||
dns_policy=("ClusterFirstWithHostNet" if use_host_network else None),
|
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
spec = client.V1DeploymentSpec(
|
spec = client.V1DeploymentSpec(
|
||||||
|
|||||||
@@ -115,32 +115,14 @@ class Spec:
|
|||||||
def get_configmaps(self):
|
def get_configmaps(self):
|
||||||
return self.obj.get(constants.configmaps_key, {})
|
return self.obj.get(constants.configmaps_key, {})
|
||||||
|
|
||||||
|
def get_secrets(self):
|
||||||
|
return self.obj.get(constants.secrets_key, {})
|
||||||
|
|
||||||
def get_container_resources(self):
|
def get_container_resources(self):
|
||||||
return Resources(
|
return Resources(
|
||||||
self.obj.get(constants.resources_key, {}).get("containers", {})
|
self.obj.get(constants.resources_key, {}).get("containers", {})
|
||||||
)
|
)
|
||||||
|
|
||||||
def get_container_resources_for(
|
|
||||||
self, container_name: str
|
|
||||||
) -> typing.Optional[Resources]:
|
|
||||||
"""Look up per-container resource overrides from spec.yml.
|
|
||||||
|
|
||||||
Checks resources.containers.<container_name> in the spec. Returns None
|
|
||||||
if no per-container override exists (caller falls back to other sources).
|
|
||||||
"""
|
|
||||||
containers_block = self.obj.get(constants.resources_key, {}).get(
|
|
||||||
"containers", {}
|
|
||||||
)
|
|
||||||
if container_name in containers_block:
|
|
||||||
entry = containers_block[container_name]
|
|
||||||
# Only treat it as a per-container override if it's a dict with
|
|
||||||
# reservations/limits nested inside (not a top-level global key)
|
|
||||||
if isinstance(entry, dict) and (
|
|
||||||
"reservations" in entry or "limits" in entry
|
|
||||||
):
|
|
||||||
return Resources(entry)
|
|
||||||
return None
|
|
||||||
|
|
||||||
def get_volume_resources(self):
|
def get_volume_resources(self):
|
||||||
return Resources(
|
return Resources(
|
||||||
self.obj.get(constants.resources_key, {}).get(constants.volumes_key, {})
|
self.obj.get(constants.resources_key, {}).get(constants.volumes_key, {})
|
||||||
|
|||||||
@@ -1,53 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
# Run a test suite locally in an isolated venv.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./tests/scripts/run-test-local.sh <test-script>
|
|
||||||
#
|
|
||||||
# Examples:
|
|
||||||
# ./tests/scripts/run-test-local.sh tests/webapp-test/run-webapp-test.sh
|
|
||||||
# ./tests/scripts/run-test-local.sh tests/smoke-test/run-smoke-test.sh
|
|
||||||
# ./tests/scripts/run-test-local.sh tests/k8s-deploy/run-deploy-test.sh
|
|
||||||
#
|
|
||||||
# The script creates a temporary venv, installs shiv, builds the laconic-so
|
|
||||||
# package, runs the requested test, then cleans up.
|
|
||||||
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
if [ $# -lt 1 ]; then
|
|
||||||
echo "Usage: $0 <test-script> [args...]"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
TEST_SCRIPT="$1"
|
|
||||||
shift
|
|
||||||
|
|
||||||
if [ ! -f "$TEST_SCRIPT" ]; then
|
|
||||||
echo "Error: $TEST_SCRIPT not found"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
VENV_DIR=$(mktemp -d /tmp/so-test-XXXXXX)
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
echo "Cleaning up venv: $VENV_DIR"
|
|
||||||
rm -rf "$VENV_DIR"
|
|
||||||
}
|
|
||||||
trap cleanup EXIT
|
|
||||||
|
|
||||||
cd "$REPO_DIR"
|
|
||||||
|
|
||||||
echo "==> Creating venv in $VENV_DIR"
|
|
||||||
python3 -m venv "$VENV_DIR"
|
|
||||||
source "$VENV_DIR/bin/activate"
|
|
||||||
|
|
||||||
echo "==> Installing shiv"
|
|
||||||
pip install -q shiv
|
|
||||||
|
|
||||||
echo "==> Building laconic-so package"
|
|
||||||
./scripts/create_build_tag_file.sh
|
|
||||||
./scripts/build_shiv_package.sh
|
|
||||||
|
|
||||||
echo "==> Running: $TEST_SCRIPT $*"
|
|
||||||
exec "./$TEST_SCRIPT" "$@"
|
|
||||||
Reference in New Issue
Block a user