Feature/known gaps #217

Merged
abdulrabbani00 merged 15 commits from feature/known-gaps into v1.10.16-statediff-v3 2022-03-31 18:38:59 +00:00

15 Commits

Author SHA1 Message Date
Abdul Rabbani
d24a49ef03 Add Workflow dispatch for docker builds 2022-03-31 14:14:34 -04:00
Abdul Rabbani
102f874c63 Update known_gaps table name 2022-03-31 14:11:57 -04:00
Abdul Rabbani
359799dee2 Update docker compose 2022-03-31 14:02:30 -04:00
Abdul Rabbani
2e836e869a PR Updates as per Ian's request 2022-03-31 11:37:01 -04:00
Abdul Rabbani
a60d5a016d Code Clean Up 2022-03-30 17:30:19 -04:00
Abdul Rabbani
fc6a9379e1 Add Prometheus metrics and write SQL file if it has data. 2022-03-30 12:53:44 -04:00
Abdul Rabbani
b960661807 Refactor: Decouple knownGaps and Indexer
This commit decouples knownGaps and Indexer. All knownGaps logic is in its own file. This makes testing and maintainability easier.

We have also removed all efforts to check the `lastProcessedblock` - This is because we won't ever run into this issue (hyptothetically), because geth won't let it happen.
2022-03-29 16:45:26 -04:00
Abdul Rabbani
fc341a90fc ** Write to file if you can't write to DB
This commit contains many new features:

1. Write the insert statement to file if there is an error when trying to write to DB.
2. Test the write to DB and write to file features in unit tests. They work as expected.
3. Insert logic for calling the knownGaps function (untested).

This PR makes a major change to the way statediff is started. Unless it is started in `file` mode, it will create a `file` indexer. This indexer will be used for writing errors to disk.
2022-03-28 15:00:29 -04:00
Abdul Rabbani
f566aa780c % Checkpoint - Added the feature to write to File if writing to DB errors out. NOT TESTED 2022-03-25 16:50:43 -04:00
Abdul Rabbani
f6ff20eb0e Handle All Gaps within Geth
Including an updated doc which keeps track of events in this PR.
2022-03-25 09:54:36 -04:00
Abdul Rabbani
1a3a63d00e Find Gaps and at them to known_gaps
This contains all the logic needed to add a gap to the `known_gaps` table.

We can now add this code to various parts of the application to calculate if a gap has occured.
2022-03-23 14:03:35 -04:00
Abdul Rabbani
ce66532fcf Improve Testing for Insert, add a query function 2022-03-22 15:49:30 -04:00
Abdul Rabbani
73f46a59ef Remove the need to pass a transaction when upserting to known_gaps
Thanks Ian :D
2022-03-22 11:52:35 -04:00
Abdul Rabbani
bca12f9cdf Delete geth-linux-amd64 2022-03-21 17:43:15 -04:00
Abdul Rabbani
921538ba64 Function To Add entry to known_gaps
This commit contains all the code needed to update the `known_gaps` table. It uses a generic function `PushKnownGaps` to insert gaps to the database.

The current test is very "basic." All it does it insert a dummy gap into the database. As long as their is no error it will return succesfully.

I have also removed the foundry subfolder. I am using the `related-repository` setup for testing.
2022-03-21 16:28:13 -04:00