Commit Graph

9 Commits

Author SHA1 Message Date
Abdul Rabbani
16d034c844 Close idle connections and chan's properly. 2022-06-23 09:17:46 -04:00
Abdul Rabbani
041f862c11 Make channels buffered 2022-06-23 07:50:36 -04:00
Abdul Rabbani
2299e54197 Use context to end goroutine 2022-06-22 14:44:26 -04:00
Abdul Rabbani
117a2deea9
Slight Performance improvements with Logging (#67)
* Slight Performance improvements with Logging

* Use exec in docker container

* Update packages for dockerfile

* Add flakey test and boot command specific logic

* Remove excess logging and add config val
2022-06-15 11:49:30 -04:00
Abdul Rabbani
cdeb43177c Send slots to processedCh and remove 2022-06-14 09:27:59 -04:00
Abdul Rabbani
24fc6358d6
Testing for Batch Processing (#56)
* Set starting slot and improve error gap capturing

* Set starting slot and improve error gap capturing

* Tests + Significant Refactor

The code for historical processing has been significantly refactored to use a context to signify a shutdown.

There have also been many tests added for historical and knownGaps processing.

* Update MhKeys in test

* Update correct values

* Update Max Retry

Genesis is not working as expected.

* Ensure we release locks properly

* Add ordered testing

* Include system tests

* Update workflow calls

* Add secrets

* Add required secrets

* update path

* Try using the absolute path

* Remove volumes at the end.

* Update system-tests.yml

* Update system-tests.yml

* Update test err

* Update and test the shutdown

* rename ethcl --> eth-beacon

* Try forcing /bin/bash for docker-compose

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Update system-tests.yml

* Use single quote cron

* Dont run generic on schedule
2022-06-09 17:32:46 -04:00
Abdul Rabbani
4d7781e2b0
Use a transaction for writing, knowngaps, and reorgs (#53)
* Use a transaction for writing, knowngaps, and reorgs

* Checkpoint - Check the DB before writing

* Ensure an error exists when writing a transaction to knowngaps

* Update unneccesary parameter
2022-06-06 09:02:43 -04:00
Abdul Rabbani
e3b4fad3c7
Capture the unique identifier everywhere its needed. (#51)
* Capture the unique identifier everywhere its needed.

* Handle graceful shutdown for historical processing

* Update flags and minor shutdown updates

* Update checkout statement for historic

* update cicd pipeline

* Add secret

* Use deploy-key instead of personal PAT

* update variables
2022-06-03 12:47:13 -04:00
Abdul Rabbani
347984a547
Feature/44 read write historic slots (#46)
* Update boot to set processing type.

* Handle entries from the historic_process table.

* Update on-pr.yml

* Fix head processing Error

* Update names and debug

* Seperate checking for new entries and locking them

* Application can process historic and known gaps

Untested

* Handle genesis

* Update remove entry for knownGaps, viper

* Disregard unused code from linter
2022-05-24 16:18:55 -04:00