ipld-eth-beacon-indexer/application_component.md

55 lines
2.3 KiB
Markdown
Raw Permalink Normal View History

Testing, CI/CD and DB connections (#10) * Update cobra to require `head` or `historic` when using `capture`. * A very generic package for implementing PGX driver I copied most of the code from the `statediff` service from within geth. The idea is that I can create formal DB packages, that can be utilized in other projects down the road. * Put PGX and other future Postgres Drivers behind SQL package This PR makes allows users to provide a config, along with a driver string. It will then provide the respective driver. * Add DB Connection and Logging * Utilize LogRus * Create a DB connection using PGX. * Create an internal boot package for starting the application. * Code clean up + Beacon Chain Connection This concludes all the code needed to connect to the DB and beacon node. We will no longer reference the lighthouse client because this application should work interchangeably with any beacon node. I have also standardized logging. * Last second clean ups * Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9) * Utilize Ginkgo and replace `testing` library. * Add TOC to docs * Add Docker specific files * Remove -e * Update on-pr-manual.yml * Add depth * Add repositories * Remove $ from path * Update path for make * Setup Go and Ginkgo * Use go mod download * Use go install * Update on-pr-manual.yml * Use latest * Remove install of GINKGO * Add explicit gopath * Explicitly specify the gopath * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Use which ginkgo * Try with make now * Final working Make
2022-04-22 12:31:57 +00:00
- [Overview](#overview)
- [Components](#components)
- [Boot](#boot)
<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>
# Overview
This document will go through various application components
# Components
## `internal/boot`
Testing, CI/CD and DB connections (#10) * Update cobra to require `head` or `historic` when using `capture`. * A very generic package for implementing PGX driver I copied most of the code from the `statediff` service from within geth. The idea is that I can create formal DB packages, that can be utilized in other projects down the road. * Put PGX and other future Postgres Drivers behind SQL package This PR makes allows users to provide a config, along with a driver string. It will then provide the respective driver. * Add DB Connection and Logging * Utilize LogRus * Create a DB connection using PGX. * Create an internal boot package for starting the application. * Code clean up + Beacon Chain Connection This concludes all the code needed to connect to the DB and beacon node. We will no longer reference the lighthouse client because this application should work interchangeably with any beacon node. I have also standardized logging. * Last second clean ups * Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9) * Utilize Ginkgo and replace `testing` library. * Add TOC to docs * Add Docker specific files * Remove -e * Update on-pr-manual.yml * Add depth * Add repositories * Remove $ from path * Update path for make * Setup Go and Ginkgo * Use go mod download * Use go install * Update on-pr-manual.yml * Use latest * Remove install of GINKGO * Add explicit gopath * Explicitly specify the gopath * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Use which ginkgo * Try with make now * Final working Make
2022-04-22 12:31:57 +00:00
The boot package in `internal` is utilized to start the application. Everything in the boot process must complete successfully for the application to start. If it does not, the application will not start.
## `pkg/database`
The `database` package allows us to interact with a postgres DB. We utilize the interface to ensure we can interact with any `sql` database as well. I copied most of the code here from `vulcanize/go-ethereum`. Down the road, internal teams should be able to reference the package instead of copy pasting it and re-implementing it.
## `pkg/beaconclient`
This package will contain code to interact with the beacon client.
2022-05-12 19:44:05 +00:00
### Known Gaps
Known Gaps tracking is handled within this package. The columns are as follows:
- StartSlot - The start slot for known_gaps, inclusive.
- EndSlot - The end slot for known_gaps, inclusive.
- CheckedOut - Indicates if any process is currently processing this entry.
- ErrorMessage - Captures any error message that might have occurred when previously processing this entry.
- EntryTime - The time this range was added to the DB. This can help us catch ranges that have not been processed for a long time due to some error.
- EntryProcess - The entry process that added this process. Potential options are StartUp, Error, HeadGap.
- This can help us understand how a specific entry was added. It can be useful for debugging the application.
- StartUp - Gaps found when we started the application.
- Error - Indicates that the entry was added due to an error with processing.
- HeadGap - Indicates that gaps where found when keeping up with Head.
## `pkg/version`
A generic package which can be utilized to easily version our applications.
## `pkg/gracefulshutdown`
A generic package that can be used to shutdown various services within an application.
## `pkg/loghelper`
This package contains useful functions for logging.
## `internal/shutdown`
This package is used to shutdown the `ipld-eth-beacon-indexer`. It calls the `pkg/gracefulshutdown` package.