History

realbigsean adbb62f7f3 Devnet6 (#4404 ) * some blob reprocessing work * remove ForceBlockLookup * reorder enum match arms in sync manager * a lot more reprocessing work * impl logic for triggerng blob lookups along with block lookups * deal with rpc blobs in groups per block in the da checker. don't cache missing blob ids in the da checker. * make single block lookup generic * more work * add delayed processing logic and combine some requests * start fixing some compile errors * fix compilation in main block lookup mod * much work * get things compiling * parent blob lookups * fix compile * revert red/stevie changes * fix up sync manager delay message logic * add peer usefulness enum * should remove lookup refactor * consolidate retry error handling * improve peer scoring during certain failures in parent lookups * improve retry code * drop parent lookup if either req has a peer disconnect during download * refactor single block processed method * processing peer refactor * smol bugfix * fix some todos * fix lints * fix lints * fix compile in lookup tests * fix lints * fix lints * fix existing block lookup tests * renamings * fix after merge * cargo fmt * compilation fix in beacon chain tests * fix * refactor lookup tests to work with multiple forks and response types * make tests into macros * wrap availability check error * fix compile after merge * add random blobs * start fixing up lookup verify error handling * some bug fixes and the start of deneb only tests * make tests work for all forks * track information about peer source * error refactoring * improve peer scoring * fix test compilation * make sure blobs are sent for processing after stream termination, delete copied tests * add some tests and fix a bug * smol bugfixes and moar tests * add tests and fix some things * compile after merge * lots of refactoring * retry on invalid block/blob * merge unknown parent messages before current slot lookup * get tests compiling * penalize blob peer on invalid blobs * Check disk on in-memory cache miss * Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs * Update beacon_node/network/src/sync/network_context.rs Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com> * fix bug in matching blocks and blobs in range sync * pr feedback * fix conflicts * upgrade logs from warn to crit when we receive incorrect response in range * synced_and_connected_within_tolerance -> should_search_for_block * remove todo * add data gas used and update excess data gas to u64 * Fix Broken Overflow Tests * payload verification with commitments * fix merge conflicts * restore payload file * Restore payload file * remove todo * add max blob commitments per block * c-kzg lib update * Fix ef tests * Abstract over minimal/mainnet spec in kzg crate * Start integrating new KZG * checkpoint sync without alignment * checkpoint sync without alignment * add import * add import * query for checkpoint state by slot rather than state root (teku doesn't serve by state root) * query for checkpoint state by slot rather than state root (teku doesn't serve by state root) * loosen check * get state first and query by most recent block root * Revert "loosen check" This reverts commit 069d13dd63aa794a3505db9f17bd1a6b73f0be81. * get state first and query by most recent block root * merge max blobs change * simplify delay logic * rename unknown parent sync message variants * rename parameter, block_slot -> slot * add some docs to the lookup module * use interval instead of sleep * drop request if blocks and blobs requests both return `None` for `Id` * clean up `find_single_lookup` logic * add lookup source enum * clean up `find_single_lookup` logic * add docs to find_single_lookup_request * move LookupSource our of param where unnecessary * remove unnecessary todo * query for block by `state.latest_block_header.slot` * fix lint * fix merge transition ef tests * fix test * fix test * fix observed blob sidecars test * Add some metrics (#33) * fix protocol limits for blobs by root * Update Engine API for 1:1 Structure Method * make beacon chain tests to fix devnet 6 changes * get ckzg working and fix some tests * fix remaining tests * fix lints * Fix KZG linking issues * remove unused dep * lockfile * test fixes * remove dbgs * remove unwrap * cleanup tx generator * small fixes * fixing fixes * more self reivew * more self review * refactor genesis header initialization * refactor mock el instantiations * fix compile * fix network test, make sure they run for each fork * pr feedback * fix last test (hopefully) --------- Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com> Co-authored-by: Mark Mackey <mark@sigmaprime.io> Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io>		2023-06-29 15:35:43 -04:00
..
anvil_test_node.sh	Replace ganache-cli with anvil (#3555 )	2023-05-15 07:22:02 +00:00
beacon_node.sh	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
bootnode.sh	Fix errors from local testnet scripts on MacOS (#2919 )	2022-01-26 23:14:20 +00:00
clean.sh	Use #!/usr/bin/env everywhere for local testnets (#3606 )	2022-09-29 06:13:30 +00:00
dump_logs.sh	Use #!/usr/bin/env everywhere for local testnets (#3606 )	2022-09-29 06:13:30 +00:00
el_bootnode.sh	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
genesis.json	Merge branch 'unstable' of https://github.com/sigp/lighthouse into deneb-free-blobs	2023-06-02 11:57:15 -04:00
geth.sh	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
kill_processes.sh	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
README.md	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
reset_genesis_time.sh	Fix errors from local testnet scripts on MacOS (#2919 )	2022-01-26 23:14:20 +00:00
setup.sh	Merge branch 'unstable' into deneb-free-blobs	2023-05-30 22:44:05 +10:00
start_local_testnet.sh	Devnet6 (#4404 )	2023-06-29 15:35:43 -04:00
stop_local_testnet.sh	Fix errors from local testnet scripts on MacOS (#2919 )	2022-01-26 23:14:20 +00:00
validator_client.sh	Post merge local testnets (#3807 )	2023-05-17 05:51:54 +00:00
vars.env	Update testnet ETH1_BLOCK_HASH value	2023-05-31 11:54:56 +10:00

README.md

Simple Local Testnet

These scripts allow for running a small local testnet with multiple beacon nodes and validator clients and a geth execution client. This setup can be useful for testing and development.

Requirements

The scripts require lcli, lighthouse, geth, bootnode to be installed on PATH.

MacOS users need to install GNU sed and GNU grep, and add them both to PATH as well.

From the root of this repository, run:

make
make install-lcli

Starting the testnet

Modify vars.env as desired.

The testnet starts with a post-merge genesis state. Start a consensus layer and execution layer boot node along with BN_COUNT number of beacon nodes each connected to a geth execution client and VC_COUNT validator clients.

The start_local_testnet.sh script takes four options -v VC_COUNT, -d DEBUG_LEVEL, -p to enable builder proposals and -h for help. It also takes a mandatory GENESIS_FILE for initialising geth's state. A sample genesis.json is provided in this directory.

The ETH1_BLOCK_HASH environment variable is set to the block_hash of the genesis execution layer block which depends on the contents of genesis.json. Users of these scripts need to ensure that the ETH1_BLOCK_HASH variable is updated if genesis file is modified.

The options may be in any order or absent in which case they take the default value specified.

VC_COUNT: the number of validator clients to create, default: BN_COUNT
DEBUG_LEVEL: one of { error, warn, info, debug, trace }, default: info

./start_local_testnet.sh genesis.json

Stopping the testnet

This is not necessary before start_local_testnet.sh as it invokes stop_local_testnet.sh automatically.

./stop_local_testnet.sh

Manual creation of local testnet

These scripts are used by ./start_local_testnet.sh and may be used to manually

Assuming you are happy with the configuration in vars.env, create the testnet directory, genesis state with embedded validators and validator keys with:

./setup.sh

Note: The generated genesis validators are embedded into the genesis state as genesis validators and hence do not require manual deposits to activate.

Generate bootnode enr and start an EL and CL bootnode so that multiple nodes can find each other

./bootnode.sh
./el_bootnode.sh

Start a geth node:

./geth.sh <DATADIR> <NETWORK-PORT> <HTTP-PORT> <AUTH-HTTP-PORT> <GENESIS_FILE>

e.g.

./geth.sh $HOME/.lighthouse/local-testnet/geth_1 5000 6000 7000 genesis.json

Start a beacon node:

./beacon_node.sh <DATADIR> <NETWORK-PORT> <HTTP-PORT> <EXECUTION-ENDPOINT> <EXECUTION-JWT-PATH> <OPTIONAL-DEBUG-LEVEL>

e.g.

./beacon_node.sh $HOME/.lighthouse/local-testnet/node_1 9000 8000 http://localhost:6000 ~/.lighthouse/local-testnet/geth_1/geth/jwtsecret

In a new terminal, start the validator client which will attach to the first beacon node:

./validator_client.sh <DATADIR> <BEACON-NODE-HTTP> <OPTIONAL-DEBUG-LEVEL>

e.g. to attach to the above created beacon node

./validator_client.sh $HOME/.lighthouse/local-testnet/node_1 http://localhost:8000

You can create additional beacon node and validator client instances with appropriate parameters.

Additional Info

Adjusting number and distribution of validators

The VALIDATOR_COUNT parameter is used to specify the number of insecure validator keystores to generate and make deposits for. The BN_COUNT parameter is used to adjust the division of these generated keys among separate validator client instances. For e.g. for VALIDATOR_COUNT=80 and BN_COUNT=4, the validator keys are distributed over 4 datadirs with 20 keystores per datadir. The datadirs are located in $DATADIR/node_{i} which can be passed to separate validator client instances using the --datadir parameter.

Starting fresh

Delete the current testnet and all related files using. Generally not necessary as start_local_test.sh does this each time it starts.

./clean.sh

Updating the genesis time of the beacon state

If it's been a while since you ran ./setup then the genesis time of the genesis state will be far in the future, causing lots of skip slots.

Update the genesis time to now using:

./reset_genesis_time.sh

Note: you probably want to just rerun ./start_local_testnet.sh to start over but this is another option.