cerc-io/ipld-eth-beacon-indexer
19
0
Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases 9 Wiki Activity

Include all changes needed for historical and knownGaps processing #59

Merged
abdulrabbani00 merged 11 commits from develop into main 2022-06-10 13:45:42 +00:00
Conversation 0 Commits 11 Files Changed 50 +3105 -980

11 Commits

Author SHA1 Message Date
Abdul Rabbani
72eea46bab
Update README and condition for cleanup (#58) 
* Update README and condition for cleanup

* Utilize config file for ipld-eth-beacon

* Update CICD to use config file on container build.

* Add env file for build cp

* Add a sleep before running the test
 2022-06-10 09:30:52 -04:00
Abdul Rabbani
3c6a9787fd Merge branch 'main' of github.com:vulcanize/ipld-ethcl-indexer into develop 2022-06-09 17:37:30 -04:00
Abdul Rabbani
24fc6358d6
Testing for Batch Processing (#56) 
* Set starting slot and improve error gap capturing

* Set starting slot and improve error gap capturing

* Tests + Significant Refactor

The code for historical processing has been significantly refactored to use a context to signify a shutdown.

There have also been many tests added for historical and knownGaps processing.

* Update MhKeys in test

* Update correct values

* Update Max Retry

Genesis is not working as expected.

* Ensure we release locks properly

* Add ordered testing

* Include system tests

* Update workflow calls

* Add secrets

* Add required secrets

* update path

* Try using the absolute path

* Remove volumes at the end.

* Update system-tests.yml

* Update system-tests.yml

* Update test err

* Update and test the shutdown

* rename ethcl --> eth-beacon

* Try forcing /bin/bash for docker-compose

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Use single quote cron

* Dont run generic on schedule
 2022-06-09 17:32:46 -04:00
Abdul Rabbani
d674df1ef0
Merge pull request #55 from vulcanize/ian/update_multihash_prefix 
update multihash prefix
 2022-06-06 15:52:12 -04:00
i-norden
87aa87cb51 update multihash prefix 2022-06-06 13:10:09 -05:00
Abdul Rabbani
4d7781e2b0
Use a transaction for writing, knowngaps, and reorgs (#53) 
* Use a transaction for writing, knowngaps, and reorgs

* Checkpoint - Check the DB before writing

* Ensure an error exists when writing a transaction to knowngaps

* Update unneccesary parameter
 2022-06-06 09:02:43 -04:00
Abdul Rabbani
e3b4fad3c7
Capture the unique identifier everywhere its needed. (#51) 
* Capture the unique identifier everywhere its needed.

* Handle graceful shutdown for historical processing

* Update flags and minor shutdown updates

* Update checkout statement for historic

* update cicd pipeline

* Add secret

* Use deploy-key instead of personal PAT

* update variables
 2022-06-03 12:47:13 -04:00
Abdul Rabbani
9160dded11
Add Prometheus Metrics (#49) 
* Add Prometheus Metrics

* Fix Prometheus duplicate error

* Handle duplicate registrations
 2022-05-25 10:19:29 -04:00
Abdul Rabbani
347984a547
Feature/44 read write historic slots (#46) 
* Update boot to set processing type.

* Handle entries from the historic_process table.

* Update on-pr.yml

* Fix head processing Error

* Update names and debug

* Seperate checking for new entries and locking them

* Application can process historic and known gaps

Untested

* Handle genesis

* Update remove entry for knownGaps, viper

* Disregard unused code from linter
 2022-05-24 16:18:55 -04:00
Abdul Rabbani
041276da81
Update the boot process (#45) 
* Update the boot process

* Update the CI/CD reference for stack-orchestrator
 2022-05-19 09:46:38 -04:00
Abdul Rabbani
891f4c7ef3 Update Vulcanize License in a .go files 2022-05-18 12:12:54 -04:00