24fc6358d6
* Set starting slot and improve error gap capturing * Set starting slot and improve error gap capturing * Tests + Significant Refactor The code for historical processing has been significantly refactored to use a context to signify a shutdown. There have also been many tests added for historical and knownGaps processing. * Update MhKeys in test * Update correct values * Update Max Retry Genesis is not working as expected. * Ensure we release locks properly * Add ordered testing * Include system tests * Update workflow calls * Add secrets * Add required secrets * update path * Try using the absolute path * Remove volumes at the end. * Update system-tests.yml * Update system-tests.yml * Update test err * Update and test the shutdown * rename ethcl --> eth-beacon * Try forcing /bin/bash for docker-compose * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Use single quote cron * Dont run generic on schedule |
||
---|---|---|
.github | ||
cmd | ||
internal | ||
pkg | ||
.gitignore | ||
application_component.md | ||
Dockerfile | ||
entrypoint.sh | ||
example.ipld-eth-beacon-indexer-config.json | ||
go.mod | ||
go.sum | ||
ipld-eth-beacon-indexer.log | ||
LICENSE | ||
main.go | ||
Makefile | ||
README.md |
Table of contents generated with markdown-toc
ipld-eth-beacon-indexer
This application will capture all the BeaconState
's and SignedBeaconBlock
's from the consensus chain on Ethereum. This application is going to connect to the lighthouse client, but hypothetically speaking, it should be interchangeable with any eth2 beacon node.
To learn more about the applications individual components, please read the application components.
Quick Start
Running the Application
To run the application, do as follows:
-
Setup the prerequisite applications. a. Run a beacon client (such as lighthouse). b. Run a postgres DB for eth-beacon. c. You can utilize the
stack-orchestrator
repository../wrapper.sh -e skip \ -d ../docker/local/docker-compose-eth-beacon-db.yml \ -d ../docker/latest/docker-compose-lighthouse.yml \ -v remove \ -p ../local-config.sh
-
Run the start up command.
go run -race main.go capture historic --config ./example.ipld-eth-beacon-indexer-config.json
Running Tests
To run tests, you will need to clone another repository which contains all the ssz files.
git clone git@github.com:vulcanize/ssz-data.git pkg/beaconclient/ssz-data
- To run unit tests, make sure you have a DB running:
make unit-test-local
- To run integration tests, make sure you have a lighthouse client and a DB running:
make integration-test-local-no-race
.
Development Patterns
This section will cover some generic development patterns utilizes.
Logging
For logging, please keep the following in mind:
- Utilize logrus.
- Use
log.Debug
to highlight that you are about to do something. - Use
log.Info-Fatal
when the thing you were about to do has been completed, along with the result.
log.Debug("Adding 1 + 2")
a := 1 + 2
log.Info("1 + 2 successfully Added, outcome is: ", a)
loghelper.LogError(err)
is a pretty wrapper to output errors.
Testing
This project utilizes ginkgo
for testing. A few notes on testing:
- All tests within this code base will test public methods only.
- All test packages are named
{base_package}_test
. This ensures we only test the public methods. - If there is a need to test a private method, please include why in the testing file.
- Unit tests must contain the
Label("unit")
. - Unit tests should not rely on any running service (except for a postgres DB). If a running service is needed. Utilize an integration test.
- Integration tests must contain the
Label("integration")
.
Understanding Testing Components
A few notes about the testing components.
- The
TestEvents
map contains several events for testers to leverage when testing. - Any object ending in
-dummy
is not a real object. You will also notice it has a present field calledMimicConfig
. This object will use an existing SSZ object, and update the parameters from theHead
andMimicConfig
.- This is done because creating an empty or minimal
SignedBeaconBlock
andBeaconState
is fairly challenging. - By slightly modifying an existing object, we can test re-org, malformed objects, and other negative conditions.
- This is done because creating an empty or minimal
Contribution
If you want to contribute please make sure you do the following:
- Create a Github issue before starting your work.
- Follow the branching structure.
- Delete your branch once it has been merged.
- Do not delete the
develop
branch. We can add branch protection once we make the branch public.
- Do not delete the
Branching Structure
The branching structure is as follows: main
<-- develop
<-- your-branch
.
It is adviced that your-branch
follows the following structure: {type}/{issue-number}-{description}
.
type
- This can be anything identifying the reason for this PR, for example:bug
,feature
,release
.issue-number
- This is the issue number of the GitHub issue. It will help users easily find a full description of the issue you are trying to solve.description
- A few words to identify your issue.