Validate Ethereum IPLD objects in a Postgres database
d111dd85db
* Upgrade Geth, Go and add CICD * update ipld-eth-server version * Track validation progress on a channel * Add integration tests * Setup validator config and update instructions to run tests locally * Update readme and tests * Update test to use v4 Infrastructure * Inlcude the env file * Fix config file write * Update DB configuration Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com> Co-authored-by: Ashwin Phatak <ashwinpphatak@gmail.com> |
||
---|---|---|
.github/workflows | ||
cmd | ||
environments | ||
pkg/validator | ||
scripts | ||
test | ||
validator_test | ||
docker-compose.yml | ||
go.mod | ||
go.sum | ||
LICENSE | ||
main.go | ||
Makefile | ||
README.md |
- Validator-README
- Overview
- Intention for the Validator
- Instructions for Testing
- Code Overview
- Known Bugs
- Tests on 03/03/22
Table of contents generated with markdown-toc
Overview
This repository contains the validator. The purpose of the validator is to ensure that the data in the Core Postgres database match the data on the blockchain.
Intention for the Validator
The perfect scenario for the validator is as follows:
- The validator will have the capacity to perform historical checks for the Core Postgres database. Users can contain these historical checks to specified configurations (block range).
- The validator will validate a certain number of trailing blocks,
t
, trailing the head,n
. Therefore the validator will constantly perform real-time validation starting atn
and ending atn - t
. - The validator validates the IPLD blocks in the Core Database; it will update the core database to indicate that the validator validated it.
Edge Cases
We must consider the following edge cases for the validator.
- There are three different data types that the validator must account for.
Instructions for Testing
Follow steps in test/README.md
Code Overview
This section will provide some insight into specific files and their purpose.
validator_test/chain_maker.go
- This file contains the code for creating a “test” blockchain.validator_test/validator_test.go
- This file contains testing to validate the validator. It leverageschain_maker.go
to create a blockchain to validate.pkg/validator/validator.go
- This file contains most of the core logic for the validator.
Known Bugs
- The validator is improperly handling missing headers from the database.
- Scenario
- The IPLD blocks from the mock blockchain are inserted into the Postgres Data.
- The validator runs, and all tests pass.
- Users manually remove the last few rows from the database.
- The validator runs, and all tests pass - This behavior is neither expected nor wanted.
- Scenario
Tests on 03/03/22
The tests highlighted below were conducted to validate the initial behavior of the validator.
Set Up
Below are the steps utilized to set up the test environment.
- Run the
scripts/run_integration_test.sh
script.- First comment outline 130 to 133 from
validator_test/validator_test.go
- First comment outline 130 to 133 from
- Once the code has completed running, comment out lines 55 to 126, 38 to 40, and 42 to 44.
- Make the following change
db, err = setupDB() --> db, _ = setupDB()
- Make the following change
- Run the following command:
ginkgo -r validator_test/ -v
- All tests should pass
Testing Failures
Once we had populated the database, we tested for failures.
- Removing a Transaction from
transaction_cids
- If we removed a transaction from the database and ran the test, the test would fail. This is the expected behavior. - Removing Headers from
eth.header_cids
- If we removed a header block sandwiched between two header blocks, the test would fail (For example, we removed the entry for block 4, and the block range is 1-10). This is the expected behavior.
- If we removed the tail block(s) from the table, the test would pass (For example, we remove the entry for blocks 8, 9, 10, and the block range is 1-10). This is not the expected behavior.