From cea8e660cd7465db33565906de7b2c96d7cc18d1 Mon Sep 17 00:00:00 2001 From: Prathamesh Musale Date: Thu, 14 Aug 2025 12:55:52 +0530 Subject: [PATCH] Add instructions to reproduce results from a test run --- README.md | 14 +- test-runs/README.md | 415 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 420 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index c1dae26..aa17e33 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ This repository includes the simulation of Zenith Network's token lockdrop distr - [Simulation Features](#simulation-features) - [Prerequisites](#prerequisites) - [Configuration](#configuration) - - [Generated Files](#generated-files) + - [Service Configuration](#service-configuration) - [View Configuration](#view-configuration) - [Setup](#setup) - [Run Simulation](#run-simulation) @@ -35,6 +35,10 @@ For independent experimentation with lockdrop allocation scenarios, see [EXPERIM ### Full Simulation and Validation +To reproduce results from pre-generated simulation runs, see [test-runs](./test-runs). There are several test runs with existing participant data and test outputs for validation. + +OR + Continue reading below for the complete simulation workflow that validates token distribution against a live zenithd node. --- @@ -99,6 +103,7 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ```bash # Download the binary from generic package registry OUTPUT_DIR=~/bin + curl -L -o $OUTPUT_DIR/zenith-ansible https://git.vdb.to/api/packages/LaconicNetwork/generic/zenith-stack/$ZENITH_STACK_VERSION/zenith-ansible ``` @@ -192,13 +197,6 @@ It allows flexible simulation parameters: - **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (can use the same parent directory as set for Genesis Generator). -### Generated Files - -The CLI tool generates the following configuration files: - -- `ansible/inventories/development/group_vars/stage1.yml` - Stage 1 specific configuration -- `ansible/inventories/development/group_vars/tge.yml` - Genesis Generator configuration with simulation parameters - ### View Configuration To view existing variables configuration, run with `list` flag: diff --git a/test-runs/README.md b/test-runs/README.md index 73b59cb..248a6ad 100644 --- a/test-runs/README.md +++ b/test-runs/README.md @@ -1,3 +1,416 @@ # Simulation Test Runs - +This directory contains pre-generated data and outputs from simulation test runs. These can be used to reproduce test results using existing participant data instead of generating it afresh. + +## Table of Contents + +- [Overview](#overview) +- [Prerequisites](#prerequisites) +- [Configuration](#configuration) + - [Service Configuration](#service-configuration) + - [View Configuration](#view-configuration) +- [Setup](#setup) +- [Run Simulation](#run-simulation) + - [Step 0: Place Existing Generated Data](#step-0-place-existing-generated-data) + - [Step 1: Simulated Token Genesis Event](#step-1-simulated-token-genesis-event) + - [Step 2: Genesis Transaction (Gentx) Signing](#step-2-genesis-transaction-gentx-signing) + - [Step 3: Start Validator Node](#step-3-start-validator-node) + - [Step 4: Run Lockdrop Distribution Notebook](#step-4-run-lockdrop-distribution-notebook) + - [Step 5: Run Simulation Tests](#step-5-run-simulation-tests) +- [Compare Results](#compare-results) +- [Cleanup](#cleanup) + +## Overview + +To reproduce the results from any one of the test runs, follow these steps to run through the simulation with existing generated data. + +## Prerequisites + +- Set zenith-stack version to use: + + ```bash + ZENITH_STACK_VERSION=v0.2.9 + ``` + + Check [releases](https://git.vdb.to/LaconicNetwork/zenith-stack/releases) page for version history. + +- **lockdrop-simulation repository** + + Clone the lockdrop simulation repository containing the simulation test suite and test runs: + + ```bash + git clone git@git.vdb.to:LaconicNetwork/lockdrop-simulation.git + ``` + +- **zenith-stack repository** + + Clone the zenith-stack repository containing required Ansible playbooks: + + ```bash + git clone git@git.vdb.to:LaconicNetwork/zenith-stack.git + + # Checkout to the required version + cd zenith-stack + git checkout $ZENITH_STACK_VERSION + ``` + + **Note**: Replace `` in the further commands in deployment with the actual path where you cloned the zenith-stack repository. + +- **zenith-ansible binary** + + **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice. Make sure that it exists and is in your [`PATH`](https://unix.stackexchange.com/a/26059). + + You may need to run the following commands with necessary permissions, as root or through sudo. + + ```bash + # Download the binary from generic package registry + OUTPUT_DIR=~/bin + + curl -L -o $OUTPUT_DIR/zenith-ansible https://git.vdb.to/api/packages/LaconicNetwork/generic/zenith-stack/$ZENITH_STACK_VERSION/zenith-ansible + ``` + + Make it executable: + + ```bash + chmod +x $OUTPUT_DIR/zenith-ansible + ``` + + Verify installation: + + ```bash + which zenith-ansible + + zenith-ansible --help + ``` + + **Always run zenith-ansible from the ansible directory on the control node.** + +- **configure-zenith-vars binary** + + `configure-zenith-vars` is an interactive CLI tool to simplify the configuration process. + + ```bash + # Navigate to the ansible directory + cd /ansible + + # Run a playbook to install configure-zenith-vars + # Use the same OUTPUT_DIR used for zenith-ansible + zenith-ansible install-zenith-config-cli.yml -e "cli_install_dir=${OUTPUT_DIR}" -K + ``` + + Verify installation: + + ```bash + which configure-zenith-vars + + # Working directory: /ansible + configure-zenith-vars --help + ``` + + **Always run configure-zenith-vars from the ansible directory on the control node**: The CLI creates `inventories/` directory relative to your current working directory by default. + +## Configuration + +Before running a lockdrop simulation for Stage 1 of the Zenith Stack, we need to configure the required parameters. + +```bash +# Navigate to ansible directory where `inventories` directory is present +cd /ansible + +configure-zenith-vars --stage stage1-lockdrop-simulation +``` + +Set **`Generate fresh participants data` to `false`** since we will be using existing generated data from a test run. + +This interactive tool will guide you through configuring all the necessary variables for your lockdrop simulation deployment. + +**To update or edit the generated configuration**, simply re-run the `configure-zenith-vars` command before proceeding with further system setup. + +### Service Configuration + +**Genesis Generator Configuration** + +- **Data directory for lockdrop simulation deployments**: Absolute path to the parent directory where deployments for lockdrop simulation will be created. + +- **Ethereum RPC endpoint**: RPC endpoint for fetching current ETH block height. + + ```bash + Example: https://eth.rpc.laconic.com/ + ``` + +- **Number of participants**: Total number of mock participants to generate for simulation (default: 400). + +- **Galaxy count**: Number of galaxies to allocate among participants - determines validator count in real scenario (default: 200). + +- **Star count**: Number of stars to allocate among participants - each participant needs at least 1 (default: 2000). + +**Bootstrap Validator Configuration** + +- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (can use the same parent directory as set for Genesis Generator). + +- **Validator display name (moniker)**: Human-readable validator name for the simulation (default: ZodNode). + +**Gentx Signer Configuration** + +- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (can use the same parent directory as set for Genesis Generator). + +### View Configuration + +To view existing variables configuration, run with `list` flag: + +```bash +# Working directory: /ansible +configure-zenith-vars --stage stage1-lockdrop-simulation --list + +# Select `development` environment when prompted +``` + +## Setup + +Create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables): + +```bash +# For example: +mkdir -p /home/$USER/stage1-lockdrop-simulation +``` + +Setup the deployment directories and pull required docker images to generate base genesis file along with other artifacts: + +```bash +# Make sure you are in ansible directory: +cd /ansible + +zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=setup" +``` + +Setup the deployment directories and pull required docker images to sign the gentx and setup stage 1 validator node: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=setup" --skip-tags onboarding +``` + +These steps will create following in the [configured](#view-configuration) deployments directory: +- `mock-lockdrop-watcher-deployment` +- `mainnet-zenithd-deployment` + +## Run Simulation + +Now that all the deployment directories are setup, we are ready to run the simulation. + +### Step 0: Place Existing Generated Data + +Copy the generated folder from your chosen test run to zenith-stack: + +```bash +# Example for run1 +cp -r /test-runs/run1/generated /generated + +# Verify +ls /generated + +# generated-accounts.json generated-participants.json point-allocation-stats.json watcher-events.json +``` + +### Step 1: Simulated Token Genesis Event + +Since we've placed existing generated data in zenith-stack, following command skips generating any new data. It creates a base genesis file with treasury initialized with the participants data: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=simulate-lockdrop" +``` + +This will generate a base genesis file at `/base-genesis-file/genesis.json` + +Note the path to `/generated` directory as it will be required in [Step 4](#step-4-run-lockdrop-distribution-notebook). + +[distribution-simulate-lockdrop.json](./distribution-simulate-lockdrop.json) is used for category-wise allocation of `$Z` with respective vesting/unlock schedules (unlock frequency reduced to 60 seconds or 30 blocks for lockdrop participants for demo purposes). + +### Step 2: Genesis Transaction (Gentx) Signing + +Since we have dummy accounts from the test run, we can access there private keys present in `generated-accounts.json`. + +Get the private key of first account present in this file: + +```bash +# Working directory: /ansible +jq -r '.[0].zenithPrivateKey' ../generated/generated-accounts.json +``` + +Note this private key down as it will be required in next step. + +Now run the playbook to sign the gentx and generate final genesis file: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=sign" +``` + +Use the private key noted above when prompted. + +This will: +- Automatically extract the pubkey of your validator node +- Create a genesis transaction (gentx) using the validator public key and private key +- Combine the base genesis file with the bootstrap validator gentx +- Generate the final genesis file +- Copy final genesis to `/genesis-file/genesis.json` + +### Step 3: Start Validator Node + +Now, we can use this genesis file to run the stage 1 validator node: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=start" --skip-tags onboarding +``` + +After starting the node, verify it's running correctly: + +```bash +# Set deployments data directory (should match configuration) +DATA_DIRECTORY=/absolute/path/to/deployments/directory + +# Check validator logs +laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd -f +``` + +Now we have a zenithd node running with the simulated participants data. + +### Step 4: Run Lockdrop Distribution Notebook + +Now we can execute the reference Jupyter notebook to perform lockdrop allocation calculations on the test participants data and produce analysis outputs. The notebook output is used further in the simulation test suite. + +1. **Create Virtual Environment and Install Dependencies** + + Create and activate a Python virtual environment: + + ```bash + # Navigate to the directory where you had cloned the lockdrop-simulation repo + cd lockdrop-simulation + + python3 -m venv venv + source venv/bin/activate + ``` + + Install required Python packages: + + ```bash + pip install -r requirements.txt + ``` + + Export path to the `generated` directory in `zenith-stack` repo from [Step 1](#step-1-simulated-token-genesis-event): + + ```bash + export GENERATED_DIR="/generated" + ``` + +2. **Execute the Notebook** + + Run the notebook to generate allocation calculations: + + ```bash + jupyter nbconvert --to notebook --execute --inplace --log-level WARN lockdrop-calculations-simulated.ipynb + ``` + + This will: + - Process the lockdrop participants data from chosen test run + - Calculate allocation amounts for different lock periods + - Generate artifacts (`lockdrop_allocations_notebook.json`) for comparison with the data from zenithd node + +3. **View Notebook Results (Optional)** + + To view the analysis on participants data, open the notebook in your browser at : + + ```bash + jupyter notebook lockdrop-calculations-simulated.ipynb + ``` + + The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics. + +### Step 5: Run Simulation Tests + +Now we can run the comprehensive test suite to validate that the zenithd node's TGE allocations match notebook results and run-time accruals happen as expected. + +1. **Set Environment Variables** + + Configure API endpoints for the running zenithd node: + + ```bash + export REST_API_ENDPOINT="http://localhost:1317" + export RPC_API_ENDPOINT="http://localhost:26657" + ``` + +2. **Run All Tests** + + Navigate to the lockdrop-simulation directory (if not already there): + + ```bash + cd + ``` + + Activate `venv`: + + ```bash + source venv/bin/activate + ``` + + Execute the complete test suite: + + ```bash + python3 tests/run_all_tests.py + ``` + + This will run tests in the following order: + - **Allocation Tests**: Compare star, galaxy, and total allocations between notebook and zenithd + - **Unlock Schedule Tests**: Validate unlock block calculations (considering each point's locking time) and initial unlock amounts + - **Accrual State Tests**: Verify accrual state calculations at current block height + +## Compare Results + +After running the tests, compare your output from the tests above to output from the chosen test run (eg. `test-runs/run1/output.log`): + +- `test_allocations`: The allocations comparisons (`Star Allocations Comparison`, `Galaxy Allocations Comparison` and `Total Allocations Comparison`) should match exactly as the TGE process is time-invariant +- `test_unlock_schedule`: + - Since the chain from your run starts at a later time than that from the original test run, the effective calculated unlock schedule for points will be different + - In `Unlock Blocks Comparison`, the `Expected Blocks` (your run) should be `<` `Expected Blocks` (test run) + - In `Initial Unlock Amounts Comparison`, the `Expected ($sZ)` (your run) should be `>` `Expected ($sZ)` (test run) +- `test_accrual_state`: Since this checks accrual state at time of running the test, `Expected ($sZ)` will be different across runs + +If all the tests pass and above conditions hold, it confirms that the simulation produces consistent results. + +## Cleanup + +### Validator Deployment + +Navigate to the Ansible directory: + +```bash +cd /ansible +``` + +Stop validator deployment: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=stop" --skip-tags onboarding +``` + +Clean up validator deployment: + +```bash +zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=cleanup" --skip-tags onboarding -K +``` + +### Python Virtual Environment + +Go to the `lockdrop-simulation` directory: + +```bash +cd +``` + +Clean up Python virtual environment: + +```bash +# Deactivate virtual environment (if currently active) +deactivate + +# Remove virtual environment directory +rm -rf venv +```