lockdrop-simulation/README.md

# lockdrop-simulation

This repository includes the simulation of Zenith Network's token lockdrop distribution and tests to validate the simulation run. It simulates a realistic lockdrop scenario using mock participants and validates token allocation calculations against a live zenithd node.

## Table of Contents

- [Overview](#overview)
  - [Interactive Experimentation](#interactive-experimentation)
  - [Full Simulation and Validation](#full-simulation-and-validation)
- [Approach](#approach)
  - [Simulation Features](#simulation-features)
- [Prerequisites](#prerequisites)
- [Configuration](#configuration)
  - [Service Configuration](#service-configuration)
  - [View Configuration](#view-configuration)
- [Setup](#setup)
- [Run Simulation](#run-simulation)
  - [Step 1: Simulated Token Genesis Event](#step-1-simulated-token-genesis-event)
  - [Step 2: Genesis Transaction (Gentx) Signing](#step-2-genesis-transaction-gentx-signing)
  - [Step 3: Start Validator Node](#step-3-start-validator-node)
  - [Step 4: Run Lockdrop Distribution Notebook](#step-4-run-lockdrop-distribution-notebook)
  - [Step 5: Run Simulation Tests](#step-5-run-simulation-tests)
- [Cleanup](#cleanup)

## Overview

This repository provides two main ways to work with lockdrop calculations:

### Interactive Experimentation

For independent experimentation with lockdrop allocation scenarios, see [EXPERIMENT.md](./EXPERIMENT.md). This allows you to:
- Experiment with different participation distributions using an interactive Jupyter notebook
- Test various scenarios (balanced, five-year focused, short-term focused, etc.)
- Export timestamped results for analysis

### Full Simulation and Validation

To reproduce results from pre-generated simulation runs, see [test-runs](./test-runs). There are several test runs with existing participant data and test outputs for validation.

OR

Continue reading below for the complete simulation workflow that validates token distribution against a live zenithd node.

---

## Approach

The lockdrop simulation validates the Zenith Network's token distribution mechanism by creating a realistic test environment without requiring real Ethereum data or live participants.

- **Generate mock participants** with proper Ethereum/Zenith addresses and distribute Urbit points (galaxies/stars) based on configurable participation rates
- **Generate watcher events** to simulate Ethereum Lockdrop contract interactions
- **Simulate the TGE event** to process participants and generate a genesis file with treasury allocations and unlock schedules as per [distribution-simulate-lockdrop.json](./distribution-simulate-lockdrop.json)
- **Deploy a test zenithd validator** using the simulated genesis file to validate the actual token distribution and accrual implementation
- **Compare expected calculations** (via Jupyter notebook) against live node's API responses to ensure mathematical correctness
- **Provide end-to-end verification** that the entire flow from participant generation to live blockchain queries works correctly in a controlled, reproducible environment

### Simulation Features

- **Urbit Point Allocation**: Uses real Urbit naming conventions and point hierarchy
- **Valid Ethereum Addresses**: Generates Ethereum key pairs for all participants
- **Valid Zenith Addresses**: Generates Zenith addresses for all participants
- **Realistic Attestations**: Create attestations following the expected format
- **Token Calculations**: Treasury calculations for lockdrop participants based on total supply and participant count

## Prerequisites

- Software Requirements

  - Ubuntu 22 or 24

    - **Note**: Other platforms are not supported yet for deployment

  - [Python3](https://wiki.python.org/moin/BeginnersGuide/Download) `3.12.x` >= `python3 --version` >= `3.8.10` (the Python3 shipped in Ubuntu 20+ is good to go)

    - Make sure that `ensurepip` is available for creating virtual environments:

      ```bash
      # Ubuntu/Debian
      sudo apt install python3-venv python3-dev build-essential

      python3 -m ensurepip --version
      ```

  - Repository Access: SSH access to <https://git.vdb.to/LaconicNetwork/zenith-stack>

  - If you are logging in to the host using SSH, make sure to use agent forwarding:

    ```bash
    # Example
    ssh -A <control-node-ip>
    ```

- Set zenith-stack version to use:

  ```bash
  ZENITH_STACK_VERSION=v0.2.9
  ```

  Check [releases](https://git.vdb.to/LaconicNetwork/zenith-stack/releases) page for version history.

- **lockdrop-simulation repository**

  Clone the lockdrop simulation repository containing the simulation test suite:

  ```bash
  git clone git@git.vdb.to:LaconicNetwork/lockdrop-simulation.git

  # Set repo directory path for further usage
  LOCKDROP_SIMULATION_DIR=$(pwd)/lockdrop-simulation
  ```

- **zenith-stack repository**

  Clone the zenith-stack repository containing required Ansible playbooks:

  ```bash
  git clone git@git.vdb.to:LaconicNetwork/zenith-stack.git

  # Set repo directory path for further usage
  ZENITH_STACK_DIR=$(pwd)/zenith-stack

  # Checkout to the required version
  cd zenith-stack
  git checkout $ZENITH_STACK_VERSION
  ```

- **zenith-ansible binary**

  **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice.

  You may need to run the following commands with necessary permissions, as root or through sudo.

  ```bash
  # Download the binary from generic package registry
  OUTPUT_DIR=~/bin
  mkdir -p $OUTPUT_DIR

  curl -L -o $OUTPUT_DIR/zenith-ansible https://git.vdb.to/api/packages/LaconicNetwork/generic/zenith-stack/$ZENITH_STACK_VERSION/zenith-ansible
  ```

  Make it executable:

  ```bash
  chmod +x $OUTPUT_DIR/zenith-ansible
  ```

  Add `OUTPUT_DIR` to your PATH:

  ```bash
  export PATH="$OUTPUT_DIR:$PATH"
  ```

  Verify installation:

  ```bash
  which zenith-ansible

  zenith-ansible --help
  ```

  **Always run zenith-ansible from the ansible directory on the control node.**

- **configure-zenith-vars binary**

  `configure-zenith-vars` is an interactive CLI tool to simplify the configuration process.

  ```bash
  # Navigate to the ansible directory
  cd $ZENITH_STACK_DIR/ansible

  # Run a playbook to install configure-zenith-vars
  # Use the same OUTPUT_DIR used for zenith-ansible
  zenith-ansible install-zenith-config-cli.yml -e "cli_install_dir=${OUTPUT_DIR}" -K
  ```

  Verify installation:

  ```bash
  which configure-zenith-vars

  # Working directory: zenith-stack/ansible
  configure-zenith-vars --help
  ```

  **Always run configure-zenith-vars from the ansible directory on the control node**: The CLI creates `inventories/` directory relative to your current working directory by default.

## Configuration

Before running a lockdrop simulation for Stage 1 of the Zenith Stack, we need to configure the required parameters.

```bash
# Navigate to ansible directory where `inventories` directory is present
cd $ZENITH_STACK_DIR/ansible

configure-zenith-vars --stage stage1-lockdrop-simulation
```

Set **`Generate fresh participants data` to `true`**.

This interactive tool will guide you through configuring all the necessary variables for your lockdrop simulation deployment.

It allows flexible simulation parameters:
- **Participant count**: Configure the total number of mock participants
- **Galaxy allocation**: Determines how many participants will be validators
- **Star distribution**: Controls the total star pool available for allocation

**To update or edit the generated configuration**, simply re-run the `configure-zenith-vars` command before proceeding with further system setup.

### Service Configuration

**Genesis Generator Configuration**

- **Data directory for lockdrop simulation deployments**: Absolute path to the parent directory where deployments for lockdrop simulation will be created.

- **Ethereum RPC endpoint**: RPC endpoint for fetching current ETH block height.

  ```bash
  Example: https://eth.rpc.laconic.com/<API_KEY>
  ```

- **Number of participants**: Total number of mock participants to generate for simulation (default: 400).

- **Galaxy count**: Number of galaxies to allocate among participants - determines validator count in real scenario (default: 200).

- **Star count**: Number of stars to allocate among participants - each participant needs at least 1 (default: 2000).

**Bootstrap Validator Configuration**

- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (use the same parent directory as set for Genesis Generator).

- **Validator display name (moniker)**: Human-readable validator name for the simulation (default: ZodNode).

**Gentx Signer Configuration**

- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (use the same parent directory as set for Genesis Generator).

### View Configuration

To view existing variables configuration, run with `list` flag:

```bash
# Working directory: zenith-stack/ansible
configure-zenith-vars --stage stage1-lockdrop-simulation --list

# Select `development` environment when prompted
```

## Setup

First, create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables):

```bash
# Make sure you are in ansible directory
cd $ZENITH_STACK_DIR/ansible

DATA_DIRECTORY=$(grep 'data_directory:' inventories/development/group_vars/stage1.yml | awk '{print $2; exit}')
mkdir -p $DATA_DIRECTORY
```

Now, run required system setup (installs docker and laconic-so):

```bash
# sudo access required for installing docker
zenith-ansible -i ./inventories/development/hosts.yml ./stage1-site.yml -e "mode=system-setup" -K --skip-tags onboarding
```

Setup the deployment directories and pull required docker images to generate base genesis file along with other artifacts:

```bash
zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=setup"
```

Setup the deployment directories and pull required docker images to sign the gentx and setup stage 1 validator node:

```bash
zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=setup" --skip-tags onboarding
```

These steps will create following in the [configured](#view-configuration) deployments directory:
- `mock-lockdrop-watcher-deployment`
- `mainnet-zenithd-deployment`

## Run Simulation

Now that all the deployment directories are setup, we are ready to run the simulation.

### Step 1: Simulated Token Genesis Event

Remove any existing data from previous runs:

```bash
rm -rf $ZENITH_STACK_DIR/generated
```

Following command generates the simulated participants with respective point lockup events. It also creates a base genesis file with treasury initialized with the participants data:

```bash
zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=simulate-lockdrop"
```

This will generate following files in `zenith-stack/generated`:

```bash
<path/to/zenith-stack>/generated/
├── generated-participants.json      # Mock participant data with attestations
├── generated-accounts.json          # Ethereum and Zenith account pairs
├── point-allocation-stats.json      # Statistics about galaxy/star allocation
└── watcher-events.json              # Simulated lockdrop contract events
```

And a base genesis file at `zenith-stack/base-genesis-file/genesis.json`.

[distribution-simulate-lockdrop.json](./distribution-simulate-lockdrop.json) is used for category-wise allocation of `$Z` with respective vesting/unlock schedules (unlock frequency reduced to 60 seconds or 30 blocks for lockdrop participants for demo purposes).

### Step 2: Genesis Transaction (Gentx) Signing

Since we have generated dummy accounts, we can access there private keys present in `generated-accounts.json`.

Get the private key of first account present in this file:

```bash
# Working directory: zenith-stack/ansible
jq -r '.[0].zenithPrivateKey' ../generated/generated-accounts.json
```

Note this private key down as it will be required in next step.

Now run the playbook to sign the gentx and generate final genesis file:

```bash
zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=sign"
```

Use the private key noted above when prompted.

This will:
- Automatically extract the pubkey of your validator node
- Create a genesis transaction (gentx) using the validator public key and private key
- Combine the base genesis file with the bootstrap validator gentx
- Generate the final genesis file
- Copy final genesis to `<path/to/zenith-stack>/genesis-file/genesis.json`

### Step 3: Start Validator Node

Now, we can use this genesis file to run the stage 1 validator node:

```bash
zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=start" --skip-tags onboarding
```

After starting the node, verify it's running correctly:

```bash
# Check validator logs
laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd
```

Now we have a zenithd node running with the simulated participants data.

### Step 4: Run Lockdrop Distribution Notebook

Now we can execute the reference Jupyter notebook to perform lockdrop allocation calculations on the generated data and produce analysis outputs. The notebook output is used further in the simulation test suite.

1. **Create Virtual Environment and Install Dependencies**

    Create and activate a Python virtual environment:

    ```bash
    # Navigate to the directory where you had cloned the lockdrop-simulation repo
    cd $LOCKDROP_SIMULATION_DIR

    python3 -m venv venv
    source venv/bin/activate
    ```

    Install required Python packages:

    ```bash
    pip install -r requirements.txt
    ```

    Export path to the `generated` directory in `zenith-stack` repo from [Step 1](#step-1-simulated-token-genesis-event):

    ```bash
    export GENERATED_DIR=$ZENITH_STACK_DIR/generated
    ```

2. **Execute the Notebook**

    Run the notebook to generate allocation calculations:

    ```bash
    jupyter nbconvert --to notebook --execute --inplace --log-level WARN lockdrop-calculations-simulated.ipynb
    ```

    This will:
    - Process the generated lockdrop participant data
    - Calculate allocation amounts for different lock periods
    - Generate artifacts (`lockdrop_allocations_notebook.json`) for comparison with the data from zenithd node

3. **View Notebook Results (Optional)**

    To view the analysis on generated data, open the notebook in your browser at <http://localhost:8888/notebooks/lockdrop-calculations-simulated.ipynb>:

    ```bash
    jupyter notebook lockdrop-calculations-simulated.ipynb
    ```

    The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics.

### Step 5: Run Simulation Tests

Now we can run the comprehensive test suite to validate that the zenithd node's TGE allocations match notebook results and run-time accruals happen as expected.

1. **Set Environment Variables**

    Configure API endpoints for the running zenithd node:

    ```bash
    export REST_API_ENDPOINT="http://localhost:1317"
    export RPC_API_ENDPOINT="http://localhost:26657"
    ```

2. **Run All Tests**

    Activate `venv`:

    ```bash
    # Navigate to the lockdrop-simulation repo directory
    source venv/bin/activate
    ```

    Execute the complete test suite:

    ```bash
    python3 tests/run_all_tests.py
    ```

    This will run tests in the following order:
    - **Allocation Tests**: Compare star, galaxy, and total allocations between notebook and zenithd
    - **Unlock Schedule Tests**: Validate unlock block calculations (considering each point's locking time) and initial unlock amounts
    - **Accrual State Tests**: Verify accrual state calculations at current block height

3. **Run Individual Test Modules** (Optional)

    You can also run specific test categories:

    ```bash
    # Test only allocations
    python3 tests/test_allocations.py

    # Test only unlock schedules
    python3 tests/test_unlock_schedule.py

    # Test only accrual states
    python3 tests/test_accrual_state.py
    ```

4. **Test Output**

    The tests provide detailed tabular output showing:
    - Comparison between notebook calculations and zenithd responses
    - Any differences or mismatches
    - Validation of the lockdrop implementation

## Cleanup

### Validator Deployment

Navigate to the Ansible directory:

```bash
cd $ZENITH_STACK_DIR/ansible
```

Stop validator deployment:

```bash
zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=stop" --skip-tags onboarding
```

Clean up validator deployment:

```bash
zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=cleanup" --skip-tags onboarding -K
```

### Python Virtual Environment

Navigate to the `lockdrop-simulation` repo directory:

```bash
cd $LOCKDROP_SIMULATION_DIR
```

Clean up Python virtual environment:

```bash
# Deactivate virtual environment (if currently active)
deactivate

# Remove virtual environment directory
rm -rf venv
```