diff --git a/EXPERIMENT.md b/EXPERIMENT.md index bb5ff35..33cdd93 100644 --- a/EXPERIMENT.md +++ b/EXPERIMENT.md @@ -4,8 +4,20 @@ This guide explains how to use the interactive notebook to experiment with diffe ## Prerequisites +- Ubuntu 22 or 24 + + - **Note**: Other platforms are not supported yet for deployment + - [Python3](https://wiki.python.org/moin/BeginnersGuide/Download) `3.12.x` >= `python3 --version` >= `3.8.10` (the Python3 shipped in Ubuntu 20+ is good to go) -- [pip](https://pip.pypa.io/en/stable/installation/) (Python package manager) + + - Make sure that `ensurepip` is available for creating virtual environments: + + ```bash + # Ubuntu/Debian + sudo apt install python3-venv python3-dev build-essential + + python3 -m ensurepip --version + ``` ## Quick Start diff --git a/README.md b/README.md index f032b63..312d727 100644 --- a/README.md +++ b/README.md @@ -64,10 +64,36 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ## Prerequisites +- Software Requirements + + - Ubuntu 22 or 24 + + - **Note**: Other platforms are not supported yet for deployment + + - [Python3](https://wiki.python.org/moin/BeginnersGuide/Download) `3.12.x` >= `python3 --version` >= `3.8.10` (the Python3 shipped in Ubuntu 20+ is good to go) + + - Make sure that `ensurepip` is available for creating virtual environments: + + ```bash + # Ubuntu/Debian + sudo apt install python3-venv python3-dev build-essential + + python3 -m ensurepip --version + ``` + +- Repository Access: SSH access to + +- If you are logging in to the host using SSH, make sure to use agent forwarding: + + ```bash + # Example + ssh -A @ + ``` + - Set zenith-stack version to use: ```bash - ZENITH_STACK_VERSION=v0.2.9 + ZENITH_STACK_VERSION=v0.2.10 ``` Check [releases](https://git.vdb.to/LaconicNetwork/zenith-stack/releases) page for version history. @@ -78,6 +104,9 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ```bash git clone git@git.vdb.to:LaconicNetwork/lockdrop-simulation.git + + # Set repo directory path for further usage + LOCKDROP_SIMULATION_DIR=$(pwd)/lockdrop-simulation ``` - **zenith-stack repository** @@ -87,22 +116,24 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ```bash git clone git@git.vdb.to:LaconicNetwork/zenith-stack.git + # Set repo directory path for further usage + ZENITH_STACK_DIR=$(pwd)/zenith-stack + # Checkout to the required version cd zenith-stack git checkout $ZENITH_STACK_VERSION ``` - **Note**: Replace `` in the further commands in deployment with the actual path where you cloned the zenith-stack repository. - - **zenith-ansible binary** - **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice. Make sure that it exists and is in your [`PATH`](https://unix.stackexchange.com/a/26059). + **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice. You may need to run the following commands with necessary permissions, as root or through sudo. ```bash # Download the binary from generic package registry OUTPUT_DIR=~/bin + mkdir -p $OUTPUT_DIR curl -L -o $OUTPUT_DIR/zenith-ansible https://git.vdb.to/api/packages/LaconicNetwork/generic/zenith-stack/$ZENITH_STACK_VERSION/zenith-ansible ``` @@ -113,6 +144,12 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan chmod +x $OUTPUT_DIR/zenith-ansible ``` + Add `OUTPUT_DIR` to your PATH: + + ```bash + export PATH="$OUTPUT_DIR:$PATH" + ``` + Verify installation: ```bash @@ -129,7 +166,7 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ```bash # Navigate to the ansible directory - cd /ansible + cd $ZENITH_STACK_DIR/ansible # Run a playbook to install configure-zenith-vars # Use the same OUTPUT_DIR used for zenith-ansible @@ -141,7 +178,7 @@ The lockdrop simulation validates the Zenith Network's token distribution mechan ```bash which configure-zenith-vars - # Working directory: /ansible + # Working directory: zenith-stack/ansible configure-zenith-vars --help ``` @@ -153,7 +190,7 @@ Before running a lockdrop simulation for Stage 1 of the Zenith Stack, we need to ```bash # Navigate to ansible directory where `inventories` directory is present -cd /ansible +cd $ZENITH_STACK_DIR/ansible configure-zenith-vars --stage stage1-lockdrop-simulation ``` @@ -189,20 +226,20 @@ It allows flexible simulation parameters: **Bootstrap Validator Configuration** -- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (can use the same parent directory as set for Genesis Generator). +- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (use the same parent directory as set for Genesis Generator). - **Validator display name (moniker)**: Human-readable validator name for the simulation (default: ZodNode). **Gentx Signer Configuration** -- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (can use the same parent directory as set for Genesis Generator). +- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (use the same parent directory as set for Genesis Generator). ### View Configuration To view existing variables configuration, run with `list` flag: ```bash -# Working directory: /ansible +# Working directory: zenith-stack/ansible configure-zenith-vars --stage stage1-lockdrop-simulation --list # Select `development` environment when prompted @@ -210,19 +247,26 @@ configure-zenith-vars --stage stage1-lockdrop-simulation --list ## Setup -Create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables): +First, create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables): ```bash -# For example: -mkdir -p /home/$USER/stage1-lockdrop-simulation +# Make sure you are in ansible directory +cd $ZENITH_STACK_DIR/ansible + +DATA_DIRECTORY=$(grep 'data_directory:' inventories/development/group_vars/stage1.yml | awk '{print $2; exit}') +mkdir -p $DATA_DIRECTORY +``` + +Now, run required system setup (installs docker and laconic-so): + +```bash +# sudo access required for installing docker +zenith-ansible -i ./inventories/development/hosts.yml ./stage1-site.yml -e "mode=system-setup" -K --skip-tags onboarding ``` Setup the deployment directories and pull required docker images to generate base genesis file along with other artifacts: ```bash -# Make sure you are in ansible directory: -cd /ansible - zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=setup" ``` @@ -245,7 +289,7 @@ Now that all the deployment directories are setup, we are ready to run the simul Remove any existing data from previous runs: ```bash -rm -rf /generated +rm -rf $ZENITH_STACK_DIR/generated ``` Following command generates the simulated participants with respective point lockup events. It also creates a base genesis file with treasury initialized with the participants data: @@ -254,7 +298,7 @@ Following command generates the simulated participants with respective point loc zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=simulate-lockdrop" ``` -This will generate following files in `/generated`: +This will generate following files in `zenith-stack/generated`: ```bash /generated/ @@ -264,9 +308,7 @@ This will generate following files in `/generated`: └── watcher-events.json # Simulated lockdrop contract events ``` -And a base genesis file at `/base-genesis-file/genesis.json`. - -Note the path to `/generated` directory as it will be required in [Step 4](#step-4-run-lockdrop-distribution-notebook). +And a base genesis file at `zenith-stack/base-genesis-file/genesis.json`. [distribution-simulate-lockdrop.json](./distribution-simulate-lockdrop.json) is used for category-wise allocation of `$Z` with respective vesting/unlock schedules (unlock frequency reduced to 60 seconds or 30 blocks for lockdrop participants for demo purposes). @@ -277,7 +319,7 @@ Since we have generated dummy accounts, we can access there private keys present Get the private key of first account present in this file: ```bash -# Working directory: /ansible +# Working directory: zenith-stack/ansible jq -r '.[0].zenithPrivateKey' ../generated/generated-accounts.json ``` @@ -309,11 +351,8 @@ zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=s After starting the node, verify it's running correctly: ```bash -# Set deployments data directory (should match configuration) -DATA_DIRECTORY=/absolute/path/to/deployments/directory - # Check validator logs -laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd -f +laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd ``` Now we have a zenithd node running with the simulated participants data. @@ -328,7 +367,7 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation ```bash # Navigate to the directory where you had cloned the lockdrop-simulation repo - cd lockdrop-simulation + cd $LOCKDROP_SIMULATION_DIR python3 -m venv venv source venv/bin/activate @@ -343,7 +382,7 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation Export path to the `generated` directory in `zenith-stack` repo from [Step 1](#step-1-simulated-token-genesis-event): ```bash - export GENERATED_DIR="/generated" + export GENERATED_DIR=$ZENITH_STACK_DIR/generated ``` 2. **Execute the Notebook** @@ -359,16 +398,6 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation - Calculate allocation amounts for different lock periods - Generate artifacts (`lockdrop_allocations_notebook.json`) for comparison with the data from zenithd node -3. **View Notebook Results (Optional)** - - To view the analysis on generated data, open the notebook in your browser at : - - ```bash - jupyter notebook lockdrop-calculations-simulated.ipynb - ``` - - The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics. - ### Step 5: Run Simulation Tests Now we can run the comprehensive test suite to validate that the zenithd node's TGE allocations match notebook results and run-time accruals happen as expected. @@ -384,15 +413,10 @@ Now we can run the comprehensive test suite to validate that the zenithd node's 2. **Run All Tests** - Navigate to the lockdrop-simulation directory (if not already there): - - ```bash - cd - ``` - Activate `venv`: ```bash + # Navigate to the lockdrop-simulation repo directory source venv/bin/activate ``` @@ -429,6 +453,26 @@ Now we can run the comprehensive test suite to validate that the zenithd node's - Any differences or mismatches - Validation of the lockdrop implementation +5. **View Lockdrop Distribution Notebook Results (Optional)** + + To view the analysis on generated data, open it using jupyter: + + ```bash + jupyter notebook lockdrop-calculations-simulated.ipynb + ``` + + This should automatically open the notebook in you browser. + + If jupyter is running on a remote host, you can tunnel the port `8888` to load the notebook in your local browser: + + ```bash + ssh @ -L localhost:8888:localhost:8888 -Nv + ``` + + Open the URL from server logs and load `lockdrop-calculations-simulated.ipynb`. + + The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics. + ## Cleanup ### Validator Deployment @@ -436,7 +480,7 @@ Now we can run the comprehensive test suite to validate that the zenithd node's Navigate to the Ansible directory: ```bash -cd /ansible +cd $ZENITH_STACK_DIR/ansible ``` Stop validator deployment: @@ -453,10 +497,10 @@ zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=c ### Python Virtual Environment -Go to the `lockdrop-simulation` directory: +Navigate to the `lockdrop-simulation` repo directory: ```bash -cd +cd $LOCKDROP_SIMULATION_DIR ``` Clean up Python virtual environment: diff --git a/test-runs/README.md b/test-runs/README.md index ada1837..4e170ef 100644 --- a/test-runs/README.md +++ b/test-runs/README.md @@ -31,10 +31,36 @@ To reproduce the results from any one of the test runs, follow these steps to ru ## Prerequisites +- Software Requirements + + - Ubuntu 22 or 24 + + - **Note**: Other platforms are not supported yet for deployment + + - [Python3](https://wiki.python.org/moin/BeginnersGuide/Download) `3.12.x` >= `python3 --version` >= `3.8.10` (the Python3 shipped in Ubuntu 20+ is good to go) + + - Make sure that `ensurepip` is available for creating virtual environments: + + ```bash + # Ubuntu/Debian + sudo apt install python3-venv python3-dev build-essential + + python3 -m ensurepip --version + ``` + +- Repository Access: SSH access to + +- If you are logging in to the host using SSH, make sure to use agent forwarding: + + ```bash + # Example + ssh -A @ + ``` + - Set zenith-stack version to use: ```bash - ZENITH_STACK_VERSION=v0.2.9 + ZENITH_STACK_VERSION=v0.2.10 ``` Check [releases](https://git.vdb.to/LaconicNetwork/zenith-stack/releases) page for version history. @@ -45,6 +71,9 @@ To reproduce the results from any one of the test runs, follow these steps to ru ```bash git clone git@git.vdb.to:LaconicNetwork/lockdrop-simulation.git + + # Set repo directory path for further usage + LOCKDROP_SIMULATION_DIR=$(pwd)/lockdrop-simulation ``` - **zenith-stack repository** @@ -54,6 +83,9 @@ To reproduce the results from any one of the test runs, follow these steps to ru ```bash git clone git@git.vdb.to:LaconicNetwork/zenith-stack.git + # Set repo directory path for further usage + ZENITH_STACK_DIR=$(pwd)/zenith-stack + # Checkout to the required version cd zenith-stack git checkout $ZENITH_STACK_VERSION @@ -63,13 +95,14 @@ To reproduce the results from any one of the test runs, follow these steps to ru - **zenith-ansible binary** - **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice. Make sure that it exists and is in your [`PATH`](https://unix.stackexchange.com/a/26059). + **Note**: Set `OUTPUT_DIR` in the following command to output directory of your choice. You may need to run the following commands with necessary permissions, as root or through sudo. ```bash # Download the binary from generic package registry OUTPUT_DIR=~/bin + mkdir -p $OUTPUT_DIR curl -L -o $OUTPUT_DIR/zenith-ansible https://git.vdb.to/api/packages/LaconicNetwork/generic/zenith-stack/$ZENITH_STACK_VERSION/zenith-ansible ``` @@ -80,6 +113,12 @@ To reproduce the results from any one of the test runs, follow these steps to ru chmod +x $OUTPUT_DIR/zenith-ansible ``` + Add `OUTPUT_DIR` to your PATH: + + ```bash + export PATH="$OUTPUT_DIR:$PATH" + ``` + Verify installation: ```bash @@ -96,7 +135,7 @@ To reproduce the results from any one of the test runs, follow these steps to ru ```bash # Navigate to the ansible directory - cd /ansible + cd $ZENITH_STACK_DIR/ansible # Run a playbook to install configure-zenith-vars # Use the same OUTPUT_DIR used for zenith-ansible @@ -108,7 +147,7 @@ To reproduce the results from any one of the test runs, follow these steps to ru ```bash which configure-zenith-vars - # Working directory: /ansible + # Working directory: zenith-stack/ansible configure-zenith-vars --help ``` @@ -120,7 +159,7 @@ Before running a lockdrop simulation for Stage 1 of the Zenith Stack, we need to ```bash # Navigate to ansible directory where `inventories` directory is present -cd /ansible +cd $ZENITH_STACK_DIR/ansible configure-zenith-vars --stage stage1-lockdrop-simulation ``` @@ -151,20 +190,20 @@ This interactive tool will guide you through configuring all the necessary varia **Bootstrap Validator Configuration** -- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (can use the same parent directory as set for Genesis Generator). +- **Bootstrap validator data directory**: Absolute path to the parent directory where the bootstrap validator deployment will be created (use the same parent directory as set for Genesis Generator). - **Validator display name (moniker)**: Human-readable validator name for the simulation (default: ZodNode). **Gentx Signer Configuration** -- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (can use the same parent directory as set for Genesis Generator). +- **Gentx signer data directory**: Absolute path to the parent directory where gentx will be signed and the genesis file created (use the same parent directory as set for Genesis Generator). ### View Configuration To view existing variables configuration, run with `list` flag: ```bash -# Working directory: /ansible +# Working directory: zenith-stack/ansible configure-zenith-vars --stage stage1-lockdrop-simulation --list # Select `development` environment when prompted @@ -172,19 +211,26 @@ configure-zenith-vars --stage stage1-lockdrop-simulation --list ## Setup -Create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables): +First, create the data directory required for simulation (must be same as the path configured for `lockdrop simulation deployments directory` in previous step to configure variables): ```bash -# For example: -mkdir -p /home/$USER/stage1-lockdrop-simulation +# Make sure you are in ansible directory +cd $ZENITH_STACK_DIR/ansible + +DATA_DIRECTORY=$(grep 'data_directory:' inventories/development/group_vars/stage1.yml | awk '{print $2; exit}') +mkdir -p $DATA_DIRECTORY +``` + +Now, run required system setup (installs docker and laconic-so): + +```bash +# sudo access required for installing docker +zenith-ansible -i ./inventories/development/hosts.yml ./stage1-site.yml -e "mode=system-setup" -K --skip-tags onboarding ``` Setup the deployment directories and pull required docker images to generate base genesis file along with other artifacts: ```bash -# Make sure you are in ansible directory: -cd /ansible - zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=setup" ``` @@ -207,11 +253,14 @@ Now that all the deployment directories are setup, we are ready to run the simul Copy the generated folder from your chosen test run to zenith-stack: ```bash +# Navigate to the lockdrop-simulation repo directory +cd $LOCKDROP_SIMULATION_DIR + # Example for run1 -cp -r /test-runs/run1/generated /generated +cp -r ./test-runs/run1/generated $ZENITH_STACK_DIR/generated # Verify -ls /generated +ls $ZENITH_STACK_DIR/generated # generated-accounts.json generated-participants.json point-allocation-stats.json watcher-events.json ``` @@ -221,13 +270,14 @@ ls /generated Since we've placed existing generated data in zenith-stack, following command skips generating any new data. It creates a base genesis file with treasury initialized with the participants data: ```bash +# Navigate back to ansible directory in zenith-stack +cd $ZENITH_STACK_DIR/ansible + zenith-ansible -i ./inventories/development/hosts.yml tge-site.yml -e "mode=simulate-lockdrop" ``` This will generate a base genesis file at `/base-genesis-file/genesis.json`. -Note the path to `/generated` directory as it will be required in [Step 4](#step-4-run-lockdrop-distribution-notebook). - [distribution-simulate-lockdrop.json](./distribution-simulate-lockdrop.json) is used for category-wise allocation of `$Z` with respective vesting/unlock schedules (unlock frequency reduced to 60 seconds or 30 blocks for lockdrop participants for demo purposes). ### Step 2: Genesis Transaction (Gentx) Signing @@ -237,7 +287,7 @@ Since we have dummy accounts from the test run, we can access there private keys Get the private key of first account present in this file: ```bash -# Working directory: /ansible +# Working directory: zenith-stack/ansible jq -r '.[0].zenithPrivateKey' ../generated/generated-accounts.json ``` @@ -269,11 +319,8 @@ zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=s After starting the node, verify it's running correctly: ```bash -# Set deployments data directory (should match configuration) -DATA_DIRECTORY=/absolute/path/to/deployments/directory - # Check validator logs -laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd -f +laconic-so deployment --dir $DATA_DIRECTORY/mainnet-zenithd-deployment logs zenithd ``` Now we have a zenithd node running with the simulated participants data. @@ -288,7 +335,7 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation ```bash # Navigate to the directory where you had cloned the lockdrop-simulation repo - cd lockdrop-simulation + cd $LOCKDROP_SIMULATION_DIR python3 -m venv venv source venv/bin/activate @@ -303,7 +350,7 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation Export path to the `generated` directory in `zenith-stack` repo from [Step 1](#step-1-simulated-token-genesis-event): ```bash - export GENERATED_DIR="/generated" + export GENERATED_DIR=$ZENITH_STACK_DIR/generated ``` 2. **Execute the Notebook** @@ -319,16 +366,6 @@ Now we can execute the reference Jupyter notebook to perform lockdrop allocation - Calculate allocation amounts for different lock periods - Generate artifacts (`lockdrop_allocations_notebook.json`) for comparison with the data from zenithd node -3. **View Notebook Results (Optional)** - - To view the analysis on participants data, open the notebook in your browser at : - - ```bash - jupyter notebook lockdrop-calculations-simulated.ipynb - ``` - - The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics. - ### Step 5: Run Simulation Tests Now we can run the comprehensive test suite to validate that the zenithd node's TGE allocations match notebook results and run-time accruals happen as expected. @@ -344,15 +381,10 @@ Now we can run the comprehensive test suite to validate that the zenithd node's 2. **Run All Tests** - Navigate to the lockdrop-simulation directory (if not already there): - - ```bash - cd - ``` - Activate `venv`: ```bash + # Navigate to the lockdrop-simulation repo directory source venv/bin/activate ``` @@ -367,6 +399,26 @@ Now we can run the comprehensive test suite to validate that the zenithd node's - **Unlock Schedule Tests**: Validate unlock block calculations (considering each point's locking time) and initial unlock amounts - **Accrual State Tests**: Verify accrual state calculations at current block height +3. **View Lockdrop Distribution Notebook Results (Optional)** + + To view the analysis on generated data, open it using jupyter: + + ```bash + jupyter notebook lockdrop-calculations-simulated.ipynb + ``` + + This should automatically open the notebook in you browser. + + If jupyter is running on a remote host, you can tunnel the port `8888` to load the notebook in your local browser: + + ```bash + ssh @ -L localhost:8888:localhost:8888 -Nv + ``` + + Open the URL from server logs and load `lockdrop-calculations-simulated.ipynb`. + + The notebook contains useful visualizations including allocation distributions, lock period analysis, and participant statistics. + ## Compare Results After running the tests, compare your output from the tests above to output from the chosen test run (eg. `test-runs/run1/output.log`): @@ -387,7 +439,7 @@ If all the tests pass and above conditions hold, it confirms that the simulation Navigate to the Ansible directory: ```bash -cd /ansible +cd $ZENITH_STACK_DIR/ansible ``` Stop validator deployment: @@ -407,7 +459,7 @@ zenith-ansible -i ./inventories/development/hosts.yml stage1-site.yml -e "mode=c Go to the `lockdrop-simulation` directory: ```bash -cd +cd $LOCKDROP_SIMULATION_DIR ``` Clean up Python virtual environment: