diff --git a/AI-FRIENDLY-PLAN.md b/AI-FRIENDLY-PLAN.md new file mode 100644 index 00000000..58d239f3 --- /dev/null +++ b/AI-FRIENDLY-PLAN.md @@ -0,0 +1,151 @@ +# Plan: Make Stack-Orchestrator AI-Friendly + +## Goal + +Make the stack-orchestrator repository easier for AI tools (Claude Code, Cursor, Copilot) to understand and use for generating stacks, including adding a `create-stack` command. + +--- + +## Part 1: Documentation & Context Files + +### 1.1 Add CLAUDE.md + +Create a root-level context file for AI assistants. + +**File:** `CLAUDE.md` + +Contents: +- Project overview (what stack-orchestrator does) +- Stack creation workflow (step-by-step) +- File naming conventions +- Required vs optional fields in stack.yml +- Common patterns and anti-patterns +- Links to example stacks (simple, medium, complex) + +### 1.2 Add JSON Schema for stack.yml + +Create formal validation schema. + +**File:** `schemas/stack-schema.json` + +Benefits: +- AI tools can validate generated stacks +- IDEs provide autocomplete +- CI can catch errors early + +### 1.3 Add Template Stack with Comments + +Create an annotated template for reference. + +**File:** `stack_orchestrator/data/stacks/_template/stack.yml` + +```yaml +# Stack definition template - copy this directory to create a new stack +version: "1.2" # Required: 1.0, 1.1, or 1.2 +name: my-stack # Required: lowercase, hyphens only +description: "Human-readable description" # Optional +repos: # Git repositories to clone + - github.com/org/repo +containers: # Container images to build (must have matching container-build/) + - cerc/my-container +pods: # Deployment units (must have matching docker-compose-{pod}.yml) + - my-pod +``` + +### 1.4 Document Validation Rules + +Create explicit documentation of constraints currently scattered in code. + +**File:** `docs/stack-format.md` + +Contents: +- Container names must start with `cerc/` +- Pod names must match compose file: `docker-compose-{pod}.yml` +- Repository format: `host/org/repo[@ref]` +- Stack directory name should match `name` field +- Version field options and differences + +--- + +## Part 2: Add `create-stack` Command + +### 2.1 Command Overview + +```bash +laconic-so create-stack --repo github.com/org/my-app [--name my-app] [--type webapp] +``` + +**Behavior:** +1. Parse repo URL to extract app name (if --name not provided) +2. Create `stacks/{name}/stack.yml` +3. Create `container-build/cerc-{name}/Dockerfile` and `build.sh` +4. Create `compose/docker-compose-{name}.yml` +5. Update list files (repository-list.txt, container-image-list.txt, pod-list.txt) + +### 2.2 Files to Create + +| File | Purpose | +|------|---------| +| `stack_orchestrator/create/__init__.py` | Package init | +| `stack_orchestrator/create/create_stack.py` | Command implementation | + +### 2.3 Files to Modify + +| File | Change | +|------|--------| +| `stack_orchestrator/main.py` | Add import and `cli.add_command()` | + +### 2.4 Command Options + +| Option | Required | Description | +|--------|----------|-------------| +| `--repo` | Yes | Git repository URL (e.g., github.com/org/repo) | +| `--name` | No | Stack name (defaults to repo name) | +| `--type` | No | Template type: webapp, service, empty (default: webapp) | +| `--force` | No | Overwrite existing files | + +### 2.5 Template Types + +| Type | Base Image | Port | Use Case | +|------|------------|------|----------| +| webapp | node:20-bullseye-slim | 3000 | React/Vue/Next.js apps | +| service | python:3.11-slim | 8080 | Python backend services | +| empty | none | none | Custom from scratch | + +--- + +## Part 3: Implementation Summary + +### New Files (6) + +1. `CLAUDE.md` - AI assistant context +2. `schemas/stack-schema.json` - Validation schema +3. `stack_orchestrator/data/stacks/_template/stack.yml` - Annotated template +4. `docs/stack-format.md` - Stack format documentation +5. `stack_orchestrator/create/__init__.py` - Package init +6. `stack_orchestrator/create/create_stack.py` - Command implementation + +### Modified Files (1) + +1. `stack_orchestrator/main.py` - Register create-stack command + +--- + +## Verification + +```bash +# 1. Command appears in help +laconic-so --help | grep create-stack + +# 2. Dry run works +laconic-so --dry-run create-stack --repo github.com/org/test-app + +# 3. Creates all expected files +laconic-so create-stack --repo github.com/org/test-app +ls stack_orchestrator/data/stacks/test-app/ +ls stack_orchestrator/data/container-build/cerc-test-app/ +ls stack_orchestrator/data/compose/docker-compose-test-app.yml + +# 4. Build works with generated stack +laconic-so --stack test-app build-containers +``` diff --git a/STACK-CREATION-GUIDE.md b/STACK-CREATION-GUIDE.md new file mode 100644 index 00000000..57d4c7f5 --- /dev/null +++ b/STACK-CREATION-GUIDE.md @@ -0,0 +1,413 @@ +# Implementing `laconic-so create-stack` Command + +A plan for adding a new CLI command to scaffold stack files automatically. + +--- + +## Overview + +Add a `create-stack` command that generates all required files for a new stack: + +```bash +laconic-so create-stack --name my-stack --type webapp +``` + +**Output:** +``` +stack_orchestrator/data/ +├── stacks/my-stack/stack.yml +├── container-build/cerc-my-stack/ +│ ├── Dockerfile +│ └── build.sh +└── compose/docker-compose-my-stack.yml + +Updated: repository-list.txt, container-image-list.txt, pod-list.txt +``` + +--- + +## CLI Architecture Summary + +### Command Registration Pattern + +Commands are Click functions registered in `main.py`: + +```python +# main.py (line ~70) +from stack_orchestrator.create import create_stack +cli.add_command(create_stack.command, "create-stack") +``` + +### Global Options Access + +```python +from stack_orchestrator.opts import opts + +if not opts.o.quiet: + print("message") +if opts.o.dry_run: + print("(would create files)") +``` + +### Key Utilities + +| Function | Location | Purpose | +|----------|----------|---------| +| `get_yaml()` | `util.py` | YAML parser (ruamel.yaml) | +| `get_stack_path(stack)` | `util.py` | Resolve stack directory path | +| `error_exit(msg)` | `util.py` | Print error and exit(1) | + +--- + +## Files to Create + +### 1. Command Module + +**`stack_orchestrator/create/__init__.py`** +```python +# Empty file to make this a package +``` + +**`stack_orchestrator/create/create_stack.py`** +```python +import click +import os +from pathlib import Path +from shutil import copy +from stack_orchestrator.opts import opts +from stack_orchestrator.util import error_exit, get_yaml + +# Template types +STACK_TEMPLATES = { + "webapp": { + "description": "Web application with Node.js", + "base_image": "node:20-bullseye-slim", + "port": 3000, + }, + "service": { + "description": "Backend service", + "base_image": "python:3.11-slim", + "port": 8080, + }, + "empty": { + "description": "Minimal stack with no defaults", + "base_image": None, + "port": None, + }, +} + + +def get_data_dir() -> Path: + """Get path to stack_orchestrator/data directory""" + return Path(__file__).absolute().parent.parent.joinpath("data") + + +def validate_stack_name(name: str) -> None: + """Validate stack name follows conventions""" + import re + if not re.match(r'^[a-z0-9][a-z0-9-]*[a-z0-9]$', name) and len(name) > 2: + error_exit(f"Invalid stack name '{name}'. Use lowercase alphanumeric with hyphens.") + if name.startswith("cerc-"): + error_exit("Stack name should not start with 'cerc-' (container names will add this prefix)") + + +def create_stack_yml(stack_dir: Path, name: str, template: dict, repo_url: str) -> None: + """Create stack.yml file""" + config = { + "version": "1.2", + "name": name, + "description": template.get("description", f"Stack: {name}"), + "repos": [repo_url] if repo_url else [], + "containers": [f"cerc/{name}"], + "pods": [name], + } + + stack_dir.mkdir(parents=True, exist_ok=True) + with open(stack_dir / "stack.yml", "w") as f: + get_yaml().dump(config, f) + + +def create_dockerfile(container_dir: Path, name: str, template: dict) -> None: + """Create Dockerfile""" + base_image = template.get("base_image", "node:20-bullseye-slim") + port = template.get("port", 3000) + + dockerfile_content = f'''# Build stage +FROM {base_image} AS builder + +WORKDIR /app +COPY package*.json ./ +RUN npm ci +COPY . . +RUN npm run build + +# Production stage +FROM {base_image} + +WORKDIR /app +COPY package*.json ./ +RUN npm ci --only=production +COPY --from=builder /app/dist ./dist + +EXPOSE {port} +CMD ["npm", "run", "start"] +''' + + container_dir.mkdir(parents=True, exist_ok=True) + with open(container_dir / "Dockerfile", "w") as f: + f.write(dockerfile_content) + + +def create_build_script(container_dir: Path, name: str) -> None: + """Create build.sh script""" + build_script = f'''#!/usr/bin/env bash +# Build cerc/{name} + +source ${{CERC_CONTAINER_BASE_DIR}}/build-base.sh + +SCRIPT_DIR=$( cd -- "$( dirname -- "${{BASH_SOURCE[0]}}" )" &> /dev/null && pwd ) + +docker build -t cerc/{name}:local \\ + -f ${{SCRIPT_DIR}}/Dockerfile \\ + ${{build_command_args}} \\ + ${{CERC_REPO_BASE_DIR}}/{name} +''' + + build_path = container_dir / "build.sh" + with open(build_path, "w") as f: + f.write(build_script) + + # Make executable + os.chmod(build_path, 0o755) + + +def create_compose_file(compose_dir: Path, name: str, template: dict) -> None: + """Create docker-compose file""" + port = template.get("port", 3000) + + compose_content = { + "version": "3.8", + "services": { + name: { + "image": f"cerc/{name}:local", + "restart": "unless-stopped", + "ports": [f"${{HOST_PORT:-{port}}}:{port}"], + "environment": { + "NODE_ENV": "${NODE_ENV:-production}", + }, + } + } + } + + with open(compose_dir / f"docker-compose-{name}.yml", "w") as f: + get_yaml().dump(compose_content, f) + + +def update_list_file(data_dir: Path, filename: str, entry: str) -> None: + """Add entry to a list file if not already present""" + list_path = data_dir / filename + + # Read existing entries + existing = set() + if list_path.exists(): + with open(list_path, "r") as f: + existing = set(line.strip() for line in f if line.strip()) + + # Add new entry + if entry not in existing: + with open(list_path, "a") as f: + f.write(f"{entry}\n") + + +@click.command() +@click.option("--name", required=True, help="Name of the new stack (lowercase, hyphens)") +@click.option("--type", "stack_type", default="webapp", + type=click.Choice(list(STACK_TEMPLATES.keys())), + help="Stack template type") +@click.option("--repo", help="Git repository URL (e.g., github.com/org/repo)") +@click.option("--force", is_flag=True, help="Overwrite existing files") +@click.pass_context +def command(ctx, name: str, stack_type: str, repo: str, force: bool): + """Create a new stack with all required files. + + Examples: + + laconic-so create-stack --name my-app --type webapp + + laconic-so create-stack --name my-service --type service --repo github.com/org/repo + """ + # Validate + validate_stack_name(name) + + template = STACK_TEMPLATES[stack_type] + data_dir = get_data_dir() + + # Define paths + stack_dir = data_dir / "stacks" / name + container_dir = data_dir / "container-build" / f"cerc-{name}" + compose_dir = data_dir / "compose" + + # Check for existing files + if not force: + if stack_dir.exists(): + error_exit(f"Stack already exists: {stack_dir}\nUse --force to overwrite") + if container_dir.exists(): + error_exit(f"Container build dir exists: {container_dir}\nUse --force to overwrite") + + # Dry run check + if opts.o.dry_run: + print(f"Would create stack '{name}' with template '{stack_type}':") + print(f" - {stack_dir}/stack.yml") + print(f" - {container_dir}/Dockerfile") + print(f" - {container_dir}/build.sh") + print(f" - {compose_dir}/docker-compose-{name}.yml") + print(f" - Update repository-list.txt") + print(f" - Update container-image-list.txt") + print(f" - Update pod-list.txt") + return + + # Create files + if not opts.o.quiet: + print(f"Creating stack '{name}' with template '{stack_type}'...") + + create_stack_yml(stack_dir, name, template, repo) + if opts.o.verbose: + print(f" Created {stack_dir}/stack.yml") + + create_dockerfile(container_dir, name, template) + if opts.o.verbose: + print(f" Created {container_dir}/Dockerfile") + + create_build_script(container_dir, name) + if opts.o.verbose: + print(f" Created {container_dir}/build.sh") + + create_compose_file(compose_dir, name, template) + if opts.o.verbose: + print(f" Created {compose_dir}/docker-compose-{name}.yml") + + # Update list files + if repo: + update_list_file(data_dir, "repository-list.txt", repo) + if opts.o.verbose: + print(f" Added {repo} to repository-list.txt") + + update_list_file(data_dir, "container-image-list.txt", f"cerc/{name}") + if opts.o.verbose: + print(f" Added cerc/{name} to container-image-list.txt") + + update_list_file(data_dir, "pod-list.txt", name) + if opts.o.verbose: + print(f" Added {name} to pod-list.txt") + + # Summary + if not opts.o.quiet: + print(f"\nStack '{name}' created successfully!") + print(f"\nNext steps:") + print(f" 1. Edit {stack_dir}/stack.yml") + print(f" 2. Customize {container_dir}/Dockerfile") + print(f" 3. Run: laconic-so --stack {name} build-containers") + print(f" 4. Run: laconic-so --stack {name} deploy-system up") +``` + +### 2. Register Command in main.py + +**Edit `stack_orchestrator/main.py`** + +Add import: +```python +from stack_orchestrator.create import create_stack +``` + +Add command registration (after line ~78): +```python +cli.add_command(create_stack.command, "create-stack") +``` + +--- + +## Implementation Steps + +### Step 1: Create module structure +```bash +mkdir -p stack_orchestrator/create +touch stack_orchestrator/create/__init__.py +``` + +### Step 2: Create the command file +Create `stack_orchestrator/create/create_stack.py` with the code above. + +### Step 3: Register in main.py +Add the import and `cli.add_command()` line. + +### Step 4: Test the command +```bash +# Show help +laconic-so create-stack --help + +# Dry run +laconic-so --dry-run create-stack --name test-app --type webapp + +# Create a stack +laconic-so create-stack --name test-app --type webapp --repo github.com/org/test-app + +# Verify +ls -la stack_orchestrator/data/stacks/test-app/ +cat stack_orchestrator/data/stacks/test-app/stack.yml +``` + +--- + +## Template Types + +| Type | Base Image | Port | Use Case | +|------|------------|------|----------| +| `webapp` | node:20-bullseye-slim | 3000 | React/Vue/Next.js apps | +| `service` | python:3.11-slim | 8080 | Python backend services | +| `empty` | none | none | Custom from scratch | + +--- + +## Future Enhancements + +1. **Interactive mode** - Prompt for values if not provided +2. **More templates** - Go, Rust, database stacks +3. **Template from existing** - `--from-stack existing-stack` +4. **External stack support** - Create in custom directory +5. **Validation command** - `laconic-so validate-stack --name my-stack` + +--- + +## Files Modified + +| File | Change | +|------|--------| +| `stack_orchestrator/create/__init__.py` | New (empty) | +| `stack_orchestrator/create/create_stack.py` | New (command implementation) | +| `stack_orchestrator/main.py` | Add import and `cli.add_command()` | + +--- + +## Verification + +```bash +# 1. Command appears in help +laconic-so --help | grep create-stack + +# 2. Dry run works +laconic-so --dry-run create-stack --name verify-test --type webapp + +# 3. Full creation works +laconic-so create-stack --name verify-test --type webapp +ls stack_orchestrator/data/stacks/verify-test/ +ls stack_orchestrator/data/container-build/cerc-verify-test/ +ls stack_orchestrator/data/compose/docker-compose-verify-test.yml + +# 4. Build works +laconic-so --stack verify-test build-containers + +# 5. Cleanup +rm -rf stack_orchestrator/data/stacks/verify-test +rm -rf stack_orchestrator/data/container-build/cerc-verify-test +rm stack_orchestrator/data/compose/docker-compose-verify-test.yml +``` diff --git a/docs/docker-compose-deployment.md b/docs/docker-compose-deployment.md new file mode 100644 index 00000000..55688600 --- /dev/null +++ b/docs/docker-compose-deployment.md @@ -0,0 +1,550 @@ +# Docker Compose Deployment Guide + +## Introduction + +### What is a Deployer? + +In stack-orchestrator, a **deployer** provides a uniform interface for orchestrating containerized applications. This guide focuses on Docker Compose deployments, which is the default and recommended deployment mode. + +While stack-orchestrator also supports Kubernetes (`k8s`) and Kind (`k8s-kind`) deployments, those are out of scope for this guide. See the [Kubernetes Enhancements](./k8s-deployment-enhancements.md) documentation for advanced deployment options. + +## Prerequisites + +To deploy stacks using Docker Compose, you need: + +- Docker Engine (20.10+) +- Docker Compose plugin (v2.0+) +- Python 3.8+ +- stack-orchestrator installed (`laconic-so`) + +**That's it!** No additional infrastructure is required. If you have Docker installed, you're ready to deploy. + +## Deployment Workflow + +The typical deployment workflow consists of four main steps: + +1. **Setup repositories and build containers** (first time only) +2. **Initialize deployment specification** +3. **Create deployment directory** +4. **Start and manage services** + +## Quick Start Example + +Here's a complete example using the built-in `test` stack: + +```bash +# Step 1: Setup (first time only) +laconic-so --stack test setup-repositories +laconic-so --stack test build-containers + +# Step 2: Initialize deployment spec +laconic-so --stack test deploy init --output test-spec.yml + +# Step 3: Create deployment directory +laconic-so --stack test deploy create \ + --spec-file test-spec.yml \ + --deployment-dir test-deployment + +# Step 4: Start services +laconic-so deployment --dir test-deployment start + +# View running services +laconic-so deployment --dir test-deployment ps + +# View logs +laconic-so deployment --dir test-deployment logs + +# Stop services (preserves data) +laconic-so deployment --dir test-deployment stop +``` + +## Deployment Workflows + +Stack-orchestrator supports two deployment workflows: + +### 1. Deployment Directory Workflow (Recommended) + +This workflow creates a persistent deployment directory that contains all configuration and data. + +**When to use:** +- Production deployments +- When you need to preserve configuration +- When you want to manage multiple deployments +- When you need persistent volume data + +**Example:** + +```bash +# Initialize deployment spec +laconic-so --stack fixturenet-eth deploy init --output eth-spec.yml + +# Optionally edit eth-spec.yml to customize configuration + +# Create deployment directory +laconic-so --stack fixturenet-eth deploy create \ + --spec-file eth-spec.yml \ + --deployment-dir my-eth-deployment + +# Start the deployment +laconic-so deployment --dir my-eth-deployment start + +# Manage the deployment +laconic-so deployment --dir my-eth-deployment ps +laconic-so deployment --dir my-eth-deployment logs +laconic-so deployment --dir my-eth-deployment stop +``` + +### 2. Quick Deploy Workflow + +This workflow deploys directly without creating a persistent deployment directory. + +**When to use:** +- Quick testing +- Temporary deployments +- Simple stacks that don't require customization + +**Example:** + +```bash +# Start the stack directly +laconic-so --stack test deploy up + +# Check service status +laconic-so --stack test deploy port test 80 + +# View logs +laconic-so --stack test deploy logs + +# Stop (preserves volumes) +laconic-so --stack test deploy down + +# Stop and remove volumes +laconic-so --stack test deploy down --delete-volumes +``` + +## Real-World Example: Ethereum Fixturenet + +Deploy a local Ethereum testnet with Geth and Lighthouse: + +```bash +# Setup (first time only) +laconic-so --stack fixturenet-eth setup-repositories +laconic-so --stack fixturenet-eth build-containers + +# Initialize with default configuration +laconic-so --stack fixturenet-eth deploy init --output eth-spec.yml + +# Create deployment +laconic-so --stack fixturenet-eth deploy create \ + --spec-file eth-spec.yml \ + --deployment-dir fixturenet-eth-deployment + +# Start the network +laconic-so deployment --dir fixturenet-eth-deployment start + +# Check status +laconic-so deployment --dir fixturenet-eth-deployment ps + +# Access logs from specific service +laconic-so deployment --dir fixturenet-eth-deployment logs fixturenet-eth-geth-1 + +# Stop the network (preserves blockchain data) +laconic-so deployment --dir fixturenet-eth-deployment stop + +# Start again - blockchain data is preserved +laconic-so deployment --dir fixturenet-eth-deployment start + +# Clean up everything including data +laconic-so deployment --dir fixturenet-eth-deployment stop --delete-volumes +``` + +## Configuration + +### Passing Configuration Parameters + +Configuration can be passed in three ways: + +**1. At init time via `--config` flag:** + +```bash +laconic-so --stack test deploy init --output spec.yml \ + --config PARAM1=value1,PARAM2=value2 +``` + +**2. Edit the spec file after init:** + +```bash +# Initialize +laconic-so --stack test deploy init --output spec.yml + +# Edit spec.yml +vim spec.yml +``` + +Example spec.yml: +```yaml +stack: test +config: + PARAM1: value1 + PARAM2: value2 +``` + +**3. Docker Compose defaults:** + +Environment variables defined in the stack's `docker-compose-*.yml` files are used as defaults. Configuration from the spec file overrides these defaults. + +### Port Mapping + +By default, services are accessible on randomly assigned host ports. To find the mapped port: + +```bash +# Find the host port for container port 80 on service 'webapp' +laconic-so deployment --dir my-deployment port webapp 80 + +# Output example: 0.0.0.0:32768 +``` + +To configure fixed ports, edit the spec file before creating the deployment: + +```yaml +network: + ports: + webapp: + - '8080:80' # Maps host port 8080 to container port 80 + api: + - '3000:3000' +``` + +Then create the deployment: + +```bash +laconic-so --stack my-stack deploy create \ + --spec-file spec.yml \ + --deployment-dir my-deployment +``` + +### Volume Persistence + +Volumes are preserved between stop/start cycles by default: + +```bash +# Stop but keep data +laconic-so deployment --dir my-deployment stop + +# Start again - data is still there +laconic-so deployment --dir my-deployment start +``` + +To completely remove all data: + +```bash +# Stop and delete all volumes +laconic-so deployment --dir my-deployment stop --delete-volumes +``` + +Volume data is stored in `/data/`. + +## Common Operations + +### Viewing Logs + +```bash +# All services, continuous follow +laconic-so deployment --dir my-deployment logs --follow + +# Last 100 lines from all services +laconic-so deployment --dir my-deployment logs --tail 100 + +# Specific service only +laconic-so deployment --dir my-deployment logs webapp + +# Combine options +laconic-so deployment --dir my-deployment logs --tail 50 --follow webapp +``` + +### Executing Commands in Containers + +```bash +# Execute a command in a running service +laconic-so deployment --dir my-deployment exec webapp ls -la + +# Interactive shell +laconic-so deployment --dir my-deployment exec webapp /bin/bash + +# Run command with specific environment variables +laconic-so deployment --dir my-deployment exec webapp env VAR=value command +``` + +### Checking Service Status + +```bash +# List all running services +laconic-so deployment --dir my-deployment ps + +# Check using Docker directly +docker ps +``` + +### Updating a Running Deployment + +If you need to change configuration after deployment: + +```bash +# 1. Edit the spec file +vim my-deployment/spec.yml + +# 2. Regenerate configuration +laconic-so deployment --dir my-deployment update + +# 3. Restart services to apply changes +laconic-so deployment --dir my-deployment stop +laconic-so deployment --dir my-deployment start +``` + +## Multi-Service Deployments + +Many stacks deploy multiple services that work together: + +```bash +# Deploy a stack with multiple services +laconic-so --stack laconicd-with-console deploy init --output spec.yml +laconic-so --stack laconicd-with-console deploy create \ + --spec-file spec.yml \ + --deployment-dir laconicd-deployment + +laconic-so deployment --dir laconicd-deployment start + +# View all services +laconic-so deployment --dir laconicd-deployment ps + +# View logs from specific services +laconic-so deployment --dir laconicd-deployment logs laconicd +laconic-so deployment --dir laconicd-deployment logs console +``` + +## ConfigMaps + +ConfigMaps allow you to mount configuration files into containers: + +```bash +# 1. Create the config directory in your deployment +mkdir -p my-deployment/data/my-config +echo "database_url=postgres://localhost" > my-deployment/data/my-config/app.conf + +# 2. Reference in spec file +vim my-deployment/spec.yml +``` + +Add to spec.yml: +```yaml +configmaps: + my-config: ./data/my-config +``` + +```bash +# 3. Restart to apply +laconic-so deployment --dir my-deployment stop +laconic-so deployment --dir my-deployment start +``` + +The files will be mounted in the container at `/config/` (or as specified by the stack). + +## Deployment Directory Structure + +A typical deployment directory contains: + +``` +my-deployment/ +├── compose/ +│ └── docker-compose-*.yml # Generated compose files +├── config.env # Environment variables +├── deployment.yml # Deployment metadata +├── spec.yml # Deployment specification +└── data/ # Volume mounts and configs + ├── service-data/ # Persistent service data + └── config-maps/ # ConfigMap files +``` + +## Troubleshooting + +### Common Issues + +**Problem: "Cannot connect to Docker daemon"** + +```bash +# Ensure Docker is running +docker ps + +# Start Docker if needed (macOS) +open -a Docker + +# Start Docker (Linux) +sudo systemctl start docker +``` + +**Problem: "Port already in use"** + +```bash +# Either stop the conflicting service or use different ports +# Edit spec.yml before creating deployment: + +network: + ports: + webapp: + - '8081:80' # Use 8081 instead of 8080 +``` + +**Problem: "Image not found"** + +```bash +# Build containers first +laconic-so --stack your-stack build-containers +``` + +**Problem: Volumes not persisting** + +```bash +# Check if you used --delete-volumes when stopping +# Volume data is in: /data/ + +# Don't use --delete-volumes if you want to keep data: +laconic-so deployment --dir my-deployment stop + +# Only use --delete-volumes when you want to reset completely: +laconic-so deployment --dir my-deployment stop --delete-volumes +``` + +**Problem: Services not starting** + +```bash +# Check logs for errors +laconic-so deployment --dir my-deployment logs + +# Check Docker container status +docker ps -a + +# Try stopping and starting again +laconic-so deployment --dir my-deployment stop +laconic-so deployment --dir my-deployment start +``` + +### Inspecting Deployment State + +```bash +# Check deployment directory structure +ls -la my-deployment/ + +# Check running containers +docker ps + +# Check container details +docker inspect + +# Check networks +docker network ls + +# Check volumes +docker volume ls +``` + +## CLI Commands Reference + +### Stack Operations + +```bash +# Clone required repositories +laconic-so --stack setup-repositories + +# Build container images +laconic-so --stack build-containers +``` + +### Deployment Initialization + +```bash +# Initialize deployment spec with defaults +laconic-so --stack deploy init --output + +# Initialize with configuration +laconic-so --stack deploy init --output \ + --config PARAM1=value1,PARAM2=value2 +``` + +### Deployment Creation + +```bash +# Create deployment directory from spec +laconic-so --stack deploy create \ + --spec-file \ + --deployment-dir +``` + +### Deployment Management + +```bash +# Start all services +laconic-so deployment --dir start + +# Stop services (preserves volumes) +laconic-so deployment --dir stop + +# Stop and remove volumes +laconic-so deployment --dir stop --delete-volumes + +# List running services +laconic-so deployment --dir ps + +# View logs +laconic-so deployment --dir logs [--tail N] [--follow] [service] + +# Show mapped port +laconic-so deployment --dir port + +# Execute command in service +laconic-so deployment --dir exec + +# Update configuration +laconic-so deployment --dir update +``` + +### Quick Deploy Commands + +```bash +# Start stack directly +laconic-so --stack deploy up + +# Stop stack +laconic-so --stack deploy down [--delete-volumes] + +# View logs +laconic-so --stack deploy logs + +# Show port mapping +laconic-so --stack deploy port +``` + +## Related Documentation + +- [CLI Reference](./cli.md) - Complete CLI command documentation +- [Adding a New Stack](./adding-a-new-stack.md) - Creating custom stacks +- [Specification](./spec.md) - Internal structure and design +- [Kubernetes Enhancements](./k8s-deployment-enhancements.md) - Advanced K8s deployment options +- [Web App Deployment](./webapp.md) - Deploying web applications + +## Examples + +For more examples, see the test scripts: +- `scripts/quick-deploy-test.sh` - Quick deployment example +- `tests/deploy/run-deploy-test.sh` - Comprehensive test showing all features + +## Summary + +- Docker Compose is the default and recommended deployment mode +- Two workflows: deployment directory (recommended) or quick deploy +- The standard workflow is: setup → build → init → create → start +- Configuration is flexible with multiple override layers +- Volume persistence is automatic unless explicitly deleted +- All deployment state is contained in the deployment directory +- For Kubernetes deployments, see separate K8s documentation + +You're now ready to deploy stacks using stack-orchestrator with Docker Compose! diff --git a/laconic-network-deployment.md b/laconic-network-deployment.md new file mode 100644 index 00000000..d30661b2 --- /dev/null +++ b/laconic-network-deployment.md @@ -0,0 +1,128 @@ +# Deploying to the Laconic Network + +## Overview + +The Laconic network uses a **registry-based deployment model** where everything is published as blockchain records. + +## Key Documentation in stack-orchestrator + +- `docs/laconicd-with-console.md` - Setting up a laconicd network +- `docs/webapp.md` - Webapp building/running +- `stack_orchestrator/deploy/webapp/` - Implementation (14 modules) + +## Core Concepts + +### LRN (Laconic Resource Name) +Format: `lrn://laconic/[namespace]/[name]` + +Examples: +- `lrn://laconic/deployers/my-deployer-name` +- `lrn://laconic/dns/example.com` +- `lrn://laconic/deployments/example.com` + +### Registry Record Types + +| Record Type | Purpose | +|-------------|---------| +| `ApplicationRecord` | Published app metadata | +| `WebappDeployer` | Deployment service offering | +| `ApplicationDeploymentRequest` | User's request to deploy | +| `ApplicationDeploymentAuction` | Optional bidding for deployers | +| `ApplicationDeploymentRecord` | Completed deployment result | + +## Deployment Workflows + +### 1. Direct Deployment + +``` +User publishes ApplicationDeploymentRequest + → targets specific WebappDeployer (by LRN) + → includes payment TX hash + → Deployer picks up request, builds, deploys, publishes result +``` + +### 2. Auction-Based Deployment + +``` +User publishes ApplicationDeploymentAuction + → Deployers bid (commit/reveal phases) + → Winner selected + → User publishes request targeting winner +``` + +## Key CLI Commands + +### Publish a Deployer Service +```bash +laconic-so publish-webapp-deployer --laconic-config config.yml \ + --api-url https://deployer-api.example.com \ + --name my-deployer \ + --payment-address laconic1... \ + --minimum-payment 1000alnt +``` + +### Request Deployment (User Side) +```bash +laconic-so request-webapp-deployment --laconic-config config.yml \ + --app lrn://laconic/apps/my-app \ + --deployer lrn://laconic/deployers/xyz \ + --make-payment auto +``` + +### Run Deployer Service (Deployer Side) +```bash +laconic-so deploy-webapp-from-registry --laconic-config config.yml --discover +``` + +## Laconic Config File + +All tools require a laconic config file (`laconic.toml`): + +```toml +[cosmos] +address_prefix = "laconic" +chain_id = "laconic_9000-1" +endpoint = "http://localhost:26657" +key = "" +password = "" +``` + +## Setting Up a Local Laconicd Network + +```bash +# Clone and build +laconic-so --stack fixturenet-laconic-loaded setup-repositories +laconic-so --stack fixturenet-laconic-loaded build-containers +laconic-so --stack fixturenet-laconic-loaded deploy create +laconic-so deployment --dir laconic-loaded-deployment start + +# Check status +laconic-so deployment --dir laconic-loaded-deployment exec cli "laconic registry status" +``` + +## Key Implementation Files + +| File | Purpose | +|------|---------| +| `publish_webapp_deployer.py` | Register deployment service on network | +| `publish_deployment_auction.py` | Create auction for deployers to bid on | +| `handle_deployment_auction.py` | Monitor and bid on auctions (deployer-side) | +| `request_webapp_deployment.py` | Create deployment request (user-side) | +| `deploy_webapp_from_registry.py` | Process requests and deploy (deployer-side) | +| `request_webapp_undeployment.py` | Request app removal | +| `undeploy_webapp_from_registry.py` | Process removal requests | +| `util.py` | LaconicRegistryClient - all registry interactions | + +## Payment System + +- **Token Denom**: `alnt` (Laconic network tokens) +- **Payment Options**: + - `--make-payment`: Create new payment with amount (or "auto" for deployer's minimum) + - `--use-payment`: Reference existing payment TX + +## What's NOT Well-Documented + +1. No end-to-end tutorial for full deployment workflow +2. Stack publishing (vs webapp) process unclear +3. LRN naming conventions not formally specified +4. Payment economics and token mechanics