Backlog Collection #8

New Issue

2022-04-21T12:02:35Z

abdulrabbani00 commented

2022-04-21 12:02:35 +00:00

(Migrated from github.com)

Overview

This issue will capture all backlog topics we hope to implement down the road. When the time for implementation arises, we can remove them from the backlog and create tickets for the specified items.

Items

Add sqlx driver support.
Add the ability to write to a file instead of directly to the DB.
Treat the application as a Linux service, and provide start and stop commands.
When performing any sort of batch processing (historic or known gaps), we should keep track of the blocks that have been proceeded in real time. This way, when the application shuts down, we can update the "start" column. This will avoid having to reprocess slots that have already been processed.
Further test shutdown. As of right now it will make sure to revert the DB and checkout any locks. But I am noticing an issue with the dw.Tx.Rollback which makes sense. We need to figure out a better way to rollback when we cancel the context.
Add a delay when checking the DB for values after updating the DB. There seems to be some sort of "lag" that randomly causes tests to fail. It seems like we are checking the DB too quickly, before it has had time to be properly updated.
At start up, check to see if known_gaps or historic_process have any tables checked out for the current nodeIdentifier. If they do, "un-check" them out.

# Overview This issue will capture all backlog topics we hope to implement down the road. When the time for implementation arises, we can remove them from the backlog and create tickets for the specified items. # Items * Add `sqlx` driver support. * Add the ability to write to a file instead of directly to the DB. * Treat the application as a Linux service, and provide start and stop commands. * When performing any sort of batch processing (historic or known gaps), we should keep track of the blocks that have been proceeded in real time. This way, when the application shuts down, we can update the "start" column. This will avoid having to reprocess slots that have already been processed. * Further test shutdown. As of right now it will make sure to revert the DB and checkout any locks. But I am noticing an issue with the `dw.Tx.Rollback` which makes sense. We need to figure out a better way to rollback when we cancel the context. * Add a delay when checking the DB for values after updating the DB. There seems to be some sort of "lag" that randomly causes tests to fail. It seems like we are checking the DB too quickly, before it has had time to be properly updated. * At start up, check to see if known_gaps or historic_process have any tables checked out for the current nodeIdentifier. If they do, "un-check" them out.

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/ipld-eth-beacon-indexer#8