Commit Graph

9 Commits

Author SHA1 Message Date
Abdul Rabbani
60855d2823 Tests + Significant Refactor
The code for historical processing has been significantly refactored to use a context to signify a shutdown.

There have also been many tests added for historical and knownGaps processing.
2022-06-08 10:26:27 -04:00
Abdul Rabbani
4d7781e2b0
Use a transaction for writing, knowngaps, and reorgs (#53)
* Use a transaction for writing, knowngaps, and reorgs

* Checkpoint - Check the DB before writing

* Ensure an error exists when writing a transaction to knowngaps

* Update unneccesary parameter
2022-06-06 09:02:43 -04:00
Abdul Rabbani
e3b4fad3c7
Capture the unique identifier everywhere its needed. (#51)
* Capture the unique identifier everywhere its needed.

* Handle graceful shutdown for historical processing

* Update flags and minor shutdown updates

* Update checkout statement for historic

* update cicd pipeline

* Add secret

* Use deploy-key instead of personal PAT

* update variables
2022-06-03 12:47:13 -04:00
Abdul Rabbani
9160dded11
Add Prometheus Metrics (#49)
* Add Prometheus Metrics

* Fix Prometheus duplicate error

* Handle duplicate registrations
2022-05-25 10:19:29 -04:00
Abdul Rabbani
347984a547
Feature/44 read write historic slots (#46)
* Update boot to set processing type.

* Handle entries from the historic_process table.

* Update on-pr.yml

* Fix head processing Error

* Update names and debug

* Seperate checking for new entries and locking them

* Application can process historic and known gaps

Untested

* Handle genesis

* Update remove entry for knownGaps, viper

* Disregard unused code from linter
2022-05-24 16:18:55 -04:00
Abdul Rabbani
041276da81
Update the boot process (#45)
* Update the boot process

* Update the CI/CD reference for stack-orchestrator
2022-05-19 09:46:38 -04:00
Abdul Rabbani
891f4c7ef3 Update Vulcanize License in a .go files 2022-05-18 12:12:54 -04:00
Abdul Rabbani
11a8a0f99c
Testing, CI/CD and DB connections (#10)
* Update cobra to require `head` or `historic` when using `capture`.

* A very generic package for implementing PGX driver

I copied most of the code from the `statediff` service from within geth. The idea is that I can create formal DB packages, that can be utilized in other projects down the road.

* Put PGX and other future Postgres Drivers behind SQL package

This PR makes allows users to provide a config, along with a driver string. It will then provide the respective driver.

* Add DB Connection and Logging

* Utilize LogRus
* Create a DB connection using PGX.
* Create an internal boot package for starting the application.

* Code clean up + Beacon Chain Connection

This concludes all the code needed to connect to the DB and beacon node. We will no longer reference the lighthouse client because this application should work interchangeably with any beacon node.

I have also standardized logging.

* Last second clean ups

* Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9)

* Utilize Ginkgo and replace `testing` library.

* Add TOC to docs

* Add Docker specific files

* Remove -e

* Update on-pr-manual.yml

* Add depth

* Add repositories

* Remove $ from path

* Update path for make

* Setup Go and Ginkgo

* Use go mod download

* Use go install

* Update on-pr-manual.yml

* Use latest

* Remove install of GINKGO

* Add explicit gopath

* Explicitly specify the gopath

* Update on-pr-manual.yml

* Update on-pr-manual.yml

* Update on-pr-manual.yml

* Update on-pr-manual.yml

* Update on-pr-manual.yml

* Update on-pr-manual.yml

* Use which ginkgo

* Try with make now

* Final working Make
2022-04-22 08:31:57 -04:00
Abdul Rabbani
daf99c7279 Utilize Cobra 2022-04-19 17:09:59 -04:00