Add a script to generate stage1 participants and allocations with given validators (#25)

Part of [laconicd testnet validator enrollment](https://www.notion.so/laconicd-testnet-validator-enrollment-6fc1d3cafcc64fef8c5ed3affa27c675)
Requires cerc-io/fixturenet-laconicd-stack#14

Reviewed-on: cerc-io/testnet-laconicd-stack#25
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>

ops/deployments-from-scratch.md

@@ -99,7 +99,7 @@
   cat <<EOF > stage0-deployment/config.env
   # Set to true to enable adding participants functionality of the onboarding module
   ONBOARDING_ENABLED=true
-  
+
   # A custom human readable name for this node
   MONIKER=LaconicStage0
   EOF
@@ -625,7 +625,7 @@
   ```bash
   network:
     ports:
-      laconic-console:
+      console:
         - '127.0.0.1:4001:80'
   ```
 

ops/stage0-to-stage1.md

@@ -2,10 +2,6 @@
 
 Once all the participants have completed their onboarding, stage0 laconicd chain can be halted and stage1 chain can be initialized and started
 
-## Prerequisite
-
-* A CSV having the subscribers (`subscribers.csv`) placed at `/srv/laconicd/csvs`
-
 ## Login
 
 * Log in as `dev` user on the deployments VM
@@ -16,58 +12,6 @@ Once all the participants have completed their onboarding, stage0 laconicd chain
   cd /srv
   ```
 
-## Map subscribers to participants
-
-* Fetch updated scripts in testnet-laconicd-stack:
-
-  ```bash
-  laconic-so fetch-stack git.vdb.to/cerc-io/testnet-laconicd-stack --pull
-  ```
-
-* List the participants on stage0:
-
-  ```bash
-  cd /src/laconicd
-  
-  laconic-so deployment --dir stage0-deployment exec laconicd "laconicd query onboarding list"
-  ```
-
-* Create a CSV with subscribers who have onboarded as participants:
-
-  NOTE: The following script can be run locally (as it will use <https://laconicd.laconic.com> endpoint by default)
-
-  * Clone this repo
-
-    ```bash
-    git clone git@git.vdb.to:cerc-io/testnet-laconicd-stack.git
-    ```
-
-  * Build
-
-    ```bash
-    # Change to scripts dir
-    cd testnet-laconicd-stack/scripts
-
-    # Install dependencies and build
-    yarn && yarn build
-    ```
-
-  * Run script
-    
-    ```bash
-    # subscribers.csv is the CSV from beehiv
-    yarn map-subscribers-to-participants --subscribers-csv /path/to/subscribers.csv --output subscribed-participants.csv
-
-    # This should create the CSV file at subscribed-participants.csv
-    # The participants in CSV can be filtered further
-    ```
-  
-  * Copy over the file to deployments VM
-    
-    ```bash
-    scp ./subscribed-participants.csv dev@<deployments-server-hostname>:/srv/laconicd/csvs/subscribed-participants.csv
-    ```
-
 ## Halt stage0
 
 * Confirm the the currently running node is for stage0 chain:
@@ -86,12 +30,6 @@ Once all the participants have completed their onboarding, stage0 laconicd chain
 
 ## Start stage1
 
-* Fetch updated scripts in fixturenet-laconicd-stack:
-
-  ```bash
-  laconic-so fetch-stack git.vdb.to/cerc-io/fixturenet-laconicd-stack --pull
-  ```
-
 * Rebuild laconicd container with `>=v0.1.7` to enable `slashing` module:
 
   ```bash
@@ -110,42 +48,11 @@ Once all the participants have completed their onboarding, stage0 laconicd chain
   laconic-so --stack ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd build-containers --force-rebuild
   ```
 
-* Use the scripts in fixturenet-laconicd-stack to generate genesis file for stage1 using CSV with subscribed participants with token allocations:
+* Fetch the generated genesis file with stage1 participants and token allocations:
 
   ```bash
-  cd /srv/laconicd
+  # Place in stage1 deployment directory
-
+  wget -O /srv/laconicd/stage1-deployment/data/genesis-config/genesis.json https://git.vdb.to/cerc-io/testnet-laconicd-stack/raw/branch/main/ops/stage1/genesis-accounts.json
-  # Set current working dir path in a variable
-  DEPLOYMENTS_DIR=$(pwd)
-
-  cd ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd
-
-  # Generate the genesis file
-  # Participant allocation: 1000000000000 (10^12)
-  # Validator allocation:   2000000000000000 (10^15)
-  # These amounts are used only if funding amount is not present in the input CSV
-  ./scripts/generate-stage1-genesis-from-csv.sh /srv/laconicd/csvs/subscribed-participants.csv 1000000000000 2000000000000000
-
-  # If you see the error "Error: genesis.json file already exists: /root/.laconicd/config/genesis.json",
-  # remove the temporary data directory and try again
-  sudo rm -rf stage1-genesis
-
-  # Expected output:
-  # Genesis file for stage1 written to output/genesis.json
-
-  # Remove the temporary data directory
-  sudo rm -rf stage1-genesis
-
-  # Go back to the directory where deployments are created
-  cd $DEPLOYMENTS_DIR
-  ```
-
-* Copy over the generated genesis file (`.json`) containing the onboarding module state with funded participants to data directory in stage1 deployment (`stage1-deployment/data/genesis-config`):
-
-  ```bash
-  cd /srv/laconicd
-
-  cp ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd/output/genesis.json stage1-deployment/data/genesis-config/genesis.json
   ```
 
 * Start the stage1 deployment:
@@ -187,3 +94,78 @@ Once all the participants have completed their onboarding, stage0 laconicd chain
     ```
 
 * Now users can follow the steps to [Join as a validator on stage1](https://git.vdb.to/cerc-io/testnet-laconicd-stack/src/branch/main/testnet-onboarding-validator.md#join-as-a-validator-on-stage1)
+
+## Bank Transfer
+
+* Transfer tokens to an address:
+
+  ```bash
+  cd /srv/laconicd
+
+  RECEIVER_ADDRESS=
+  AMOUNT=
+
+  laconic-so deployment --dir stage1-deployment exec laconicd "laconicd tx bank send alice ${RECEIVER_ADDRESS} ${AMOUNT}alnt --from alice --fees 1000000alnt"
+  ```
+
+* Check balance:
+
+  ```bash
+  laconic-so deployment --dir stage1-deployment exec laconicd "laconicd query bank balances ${RECEIVER_ADDRESS}"
+  ```
+
+---
+
+## Generating stage1 genesis
+
+* Following steps to be run on a local machine
+
+* Clone repos:
+
+  ```bash
+  git clone git@git.vdb.to:cerc-io/testnet-laconicd-stack.git
+
+  git clone git@git.vdb.to:cerc-io/fixturenet-laconicd-stack.git
+  ```
+
+* Create stage1 participants and allocations using provided validators list:
+
+  * Prerequisite: `validators.csv` file with list of laconic addresses, example:
+
+    ```csv
+    laconic13ftz0c6cg6ttfda7ct4r6pf2j976zsey7l4wmj
+    laconic1he4wjpfm5atwfvqurpg57ctp8chmxt9swf02dx
+    laconic1wpsdkwz0t4ejdm7gcl7kn8989z88dd6wwy04np
+    ...
+    ```
+
+  * Build
+
+    ```bash
+    # Change to scripts dir
+    cd testnet-laconicd-stack/scripts
+
+    # Install dependencies and build
+    yarn && yarn build
+    ```
+
+  * Run script
+
+    ```bash
+    yarn participants-with-filtered-validators --validators-csv ./validators.csv --participant-alloc 200000000000 --validator-alloc 1000200000000000 --output stage1-participants-$(date +"%Y-%m-%dT%H%M%S").json --output-allocs stage1-allocs-$(date +"%Y-%m-%dT%H%M%S").json
+
+    # This should create two json files with stage1 participants and allocations
+    ```
+
+* Create stage1 genesis file:
+
+  ```bash
+  # Change to fixturenet-laconicd stack dir
+  cd fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd
+
+  # Generate genesis file
+  # Provide absolute paths to generated stage1-participants and stage1-allocs files
+  ./scripts/generate-stage1-genesis-from-json.sh /path/to/testnet-laconicd-stack/scripts/stage1-participants.json /path/to/testnet-laconicd-stack/scripts/stage1-allocs.json
+
+  # This should generate the required genesis file at output/genesis.json
+  ```

scripts/README.md

@@ -26,11 +26,20 @@
   cp .env.example .env
   ```
 
+- Generate a list of onboarded participants and allocations with given list of validators:
+
+  ```bash
+  yarn participants-with-filtered-validators --validators-csv <validators-csv-file> --participant-alloc <participant-alloc-amount> --validator-alloc <validator-alloc-amount> --output <output-json-file> --output-allocs <output-allocs-json-file>
+
+  # Example:
+  # yarn participants-with-filtered-validators --validators-csv ./validators.csv --participant-alloc 200000000000 --validator-alloc 1000200000000000 --output stage1-participants-$(date +"%Y-%m-%dT%H%M%S").json --output-allocs stage1-allocs-$(date +"%Y-%m-%dT%H%M%S").json
+  ```
+
 - Map subscribers to onboarded participants:
 
   ```bash
   yarn map-subscribers-to-participants --subscribers-csv <subscribers-csv-file> --output <output-csv-file>
 
   # Example:
-  # yarn map-subscribers-to-participants --subscribers-csv subscribers.csv --output result-$(date +"%Y-%m-%d%Y%m%d").csv
+  # yarn map-subscribers-to-participants --subscribers-csv subscribers.csv --output result-$(date +"%Y-%m-%dT%H%M%S").csv
   ```

scripts/src/participants-with-filtered-validators.ts

@@ -0,0 +1,114 @@
+import * as fs from 'fs';
+import * as path from 'path';
+import yargs from 'yargs';
+import { hideBin } from 'yargs/helpers';
+import dotenv from 'dotenv';
+
+import { Registry } from '@cerc-io/registry-sdk';
+
+dotenv.config();
+
+const LACONICD_GQL_ENDPOINT = process.env.LACONICD_GQL_ENDPOINT || 'https://laconicd.laconic.com/api';
+const LACONICD_RPC_ENDPOINT = process.env.LACONICD_RPC_ENDPOINT || 'https://laconicd.laconic.com';
+const LACONICD_CHAIN_ID = process.env.LACONICD_CHAIN_ID || 'laconic_9000-1';
+
+async function main(): Promise<void> {
+  const argv = _getArgv();
+
+  const registry = new Registry(LACONICD_GQL_ENDPOINT, LACONICD_RPC_ENDPOINT, LACONICD_CHAIN_ID);
+
+  console.time('time_taken_getParticipants');
+  const participants = await registry.getParticipants();
+  console.log('Fetched participants, count:', participants.length);
+  console.timeEnd('time_taken_getParticipants');
+
+  let validators: Array<string> = await readValidators(argv.validatorsCsv);
+  console.log('Read validators, count:', validators.length);
+
+  let stage1Allocations: Array<{ 'cosmos_address': string, balance: string }> = [];
+
+  const stage1Participants = participants.map((participant: any) => {
+    const outputParticipant: any = {
+      'cosmos_address': participant.cosmosAddress,
+      'nitro_address': participant.nitroAddress,
+      'kyc_id': participant.kycId
+    };
+
+    if (validators.includes(participant.cosmosAddress)) {
+      outputParticipant.role = 'validator';
+
+      stage1Allocations.push({
+        cosmos_address: participant.cosmosAddress,
+        balance: argv.validatorAlloc
+      });
+
+      // Remove processed participant from validators list
+      validators = validators.filter(val => val !== participant.cosmosAddress);
+    } else {
+      outputParticipant.role = 'participant';
+
+      stage1Allocations.push({
+        cosmos_address: participant.cosmosAddress,
+        balance: argv.participantAlloc
+      });
+    }
+
+    return outputParticipant;
+  });
+
+  // Provide allocs for remaining validators
+  validators.forEach(val => {
+    stage1Allocations.push({
+      cosmos_address: val,
+      balance: argv.validatorAlloc
+    });
+  });
+
+  const participantsOutputFilePath = path.resolve(argv.output);
+  fs.writeFileSync(participantsOutputFilePath, JSON.stringify(stage1Participants, null, 2));
+  console.log(`Onboarded participants with filtered validators written to ${participantsOutputFilePath}`);
+
+  const allocsOutputFilePath = path.resolve(argv.outputAllocs);
+  fs.writeFileSync(allocsOutputFilePath, JSON.stringify(stage1Allocations, null, 2));
+  console.log(`Stage1 allocations written to ${allocsOutputFilePath}`);
+}
+
+async function readValidators(subscribersCsvPath: string): Promise<any> {
+  const fileContent = fs.readFileSync(path.resolve(subscribersCsvPath), { encoding: 'utf-8' });
+  return fileContent.split('\r\n').map(address => address.trim());
+}
+
+function _getArgv (): any {
+  return yargs(hideBin(process.argv))
+  .option('validatorsCsv', {
+    type: 'string',
+    demandOption: true,
+    describe: 'Path to a CSV file with validators list',
+  })
+  .option('participantAlloc', {
+    type: 'string',
+    demandOption: true,
+    describe: 'Participant stage1 balance allocation',
+  })
+  .option('validatorAlloc', {
+    type: 'string',
+    demandOption: true,
+    describe: 'Validator stage1 balance allocation',
+  })
+  .option('output', {
+    type: 'string',
+    demandOption: true,
+    describe: 'Path to the output JSON file',
+  })
+  .option('outputAllocs', {
+    type: 'string',
+    demandOption: true,
+    describe: 'Path to the output JSON file with allocs',
+  })
+  .help()
+  .argv;
+}
+
+main().catch(err => {
+  console.log(err);
+});

stack-orchestrator/config/laconic-console/cli/create-config.sh

@@ -13,7 +13,7 @@ services:
   registry:
     rpcEndpoint: ${CERC_LACONICD_RPC_ENDPOINT}
     gqlEndpoint: ${CERC_LACONICD_GQL_ENDPOINT}
-    userKey: ${CERC_LACONICD_USER_KEY}
+    userKey: "${CERC_LACONICD_USER_KEY}"
     bondId: ${CERC_LACONICD_BOND_ID}
     chainId: ${CERC_LACONICD_CHAIN_ID}
     gas: ${CERC_LACONICD_GAS}

stack-orchestrator/stacks/testnet-laconicd/README.md

@@ -266,7 +266,7 @@ laconic-so deployment --dir laconic-console-deployment start
   # services:
   # registry:
   #   ...
-  #   userKey: <your-private-key>
+  #   userKey: "<your-private-key>"
   #   ...
 
   # Note: any changes made to the config will be lost when the cli Docker container is brought down
@@ -277,7 +277,7 @@ laconic-so deployment --dir laconic-console-deployment start
 
   ```bash
   # Example
-  laconic-so deployment --dir laconic-console-deployment exec cli "laconic registry bond create --type photon --quantity 1000000000000"
+  laconic-so deployment --dir laconic-console-deployment exec cli "laconic registry bond create --type alnt --quantity 1000000000000"
   ```
 
 ## Clean up