Patch for Performance and Performance Logging #66

New Issue

2022-06-15T12:32:33Z

abdulrabbani00 commented

2022-06-15 12:32:33 +00:00

(Migrated from github.com)

Overview
- Description
Dependencies
Context
Issues
Acceptance criteria
- Proof

Table of contents generated with markdown-toc

Overview

Owner: @abdulrabbani00
Stakeholder Point of Contact: @abdulrabbani00

Description

Currently, the eth-beacon-indexer keeps stopping after a few hours. It consumes all of the system's memory and shut down abruptly. We want to resolve this performance issue.

Dependencies

Context

We should also consider that while it might be an application memory issue, there are other factors that can negatively affect the applications performance, such as DB and Lighthouse node reachability.

Issues

https://github.com/vulcanize/ipld-eth-beacon-indexer/issues/62
https://github.com/vulcanize/ipld-eth-beacon-indexer/issues/65
Add clearer performance logging for each slot processed, highlighting the time it took for each major step.
Implement longer pauses between infinite for loop retries.
Leverage pointers everywhere the beaconState is referenced.
Review compile flags and options.
Properly increment KnownGap and Historical Processing Metrics.
Run the application in prod and study the behavior.

Acceptance criteria

We need to see significant performance improvements when running the application and ensure it does not crash. This EPIC will likely be a fewfold:

Deploy with clearer metrics capturing/logging binary.
Study the application, determine the issue.
Create new tickets and resolve the issue.
Re-deploy (repeat the process as necessary).

Proof

- [Overview](#overview) - [Description](#description) - [Dependencies](#dependencies) - [Context](#context) - [Issues](#issues) - [Acceptance criteria](#acceptance-criteria) - [Proof](#proof) <small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small> # Overview - **Owner**: @abdulrabbani00 - **Stakeholder Point of Contact**: @abdulrabbani00 ## Description  Currently, the eth-beacon-indexer keeps stopping after a few hours. It consumes all of the system's memory and shut down abruptly. We want to resolve this performance issue. # Dependencies  # Context  We should also consider that while it might be an application memory issue, there are other factors that can negatively affect the applications performance, such as DB and Lighthouse node reachability. # Issues  - [x] https://github.com/vulcanize/ipld-eth-beacon-indexer/issues/62 - [x] https://github.com/vulcanize/ipld-eth-beacon-indexer/issues/65 - [x] Add clearer performance logging for each slot processed, highlighting the time it took for each major step. - [x] Implement longer pauses between infinite for loop retries. - [x] Leverage pointers everywhere the beaconState is referenced. - [x] Review compile flags and options. - [x] Properly increment KnownGap and Historical Processing Metrics. - [ ] Run the application in prod and study the behavior. # Acceptance criteria  We need to see significant performance improvements when running the application and ensure it does not crash. This EPIC will likely be a fewfold: 1. Deploy with clearer metrics capturing/logging binary. 2. Study the application, determine the issue. 3. Create new tickets and resolve the issue. 4. Re-deploy (repeat the process as necessary). ## Proof

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/ipld-eth-beacon-indexer#66