debug_traceCall and debug_traceBlock #261

Open

i-norden wants to merge 16 commits from ian/tracing into v5

pull from: ian/tracing

merge into: :v5

:v5

:fix/pin-eth-typing

:fix/pin-dlv-version

:ian/tracing

:payments

:v5-go-upgrade

:v4

:release-v4.1.7-alpha

:ian_v4

:release-v4.1.4-alpha

:release-v4.1.2-alpha

:sharding

:release-v4.1.0-alpha

:master

Conversation 8 Commits 16 Files Changed 14

+1144 -13

i-norden commented 2023-11-01 20:23:51 +00:00

Member

Copy Link

Copy Source

Implement the debug_traceCall and debug_traceBlock methods using the direct state/storage access pattern supported by ipld-eth-statedb. For #260. Relies on cerc-io/ipld-eth-statedb#4.

In geth, it is possible to perform these traces on top of state which does not exist in the underlying DB by regenerating the state from the last known state below where the requested state exists. It does this by loading that state and then re-executing blocks until it regenerates the state expected for the trace. This feature makes sense in geth where the goal may be to operate without a complete archive.

I removed this capability here as it simplifies things and because I don't think this issue should ever be fixed this way. Our goal is to have a complete archive, if we are missing state in the database that is a problem that needs to be fixed at a different level. Rather than ephemerally regenerating the missing state to use in a single query/trace, which could be quite expensive, the effort should be spent on properly entering and indexing the missing data into the underlying database.

Implement the `debug_traceCall` and `debug_traceBlock` methods using the direct state/storage access pattern supported by ipld-eth-statedb. For https://git.vdb.to/cerc-io/ipld-eth-server/issues/260. Relies on https://git.vdb.to/cerc-io/ipld-eth-statedb/pulls/4. In geth, it is possible to perform these traces on top of state which does not exist in the underlying DB by regenerating the state from the last known state below where the requested state exists. It does this by loading that state and then re-executing blocks until it regenerates the state expected for the trace. This feature makes sense in geth where the goal may be to operate without a complete archive. I removed this capability here as it simplifies things and because I don't think this issue should ever be fixed this way. Our goal is to have a complete archive, if we are missing state in the database that is a problem that needs to be fixed at a different level. Rather than ephemerally regenerating the missing state to use in a single query/trace, which could be quite expensive, the effort should be spent on properly entering and indexing the missing data into the underlying database.

i-norden added 8 commits 2023-11-01 20:23:52 +00:00

BlockOverride required by tracing endpoints 74f5bdda0d

ToMessage method for transaction_arguments f275264147

tracing API for traceBlock and traceCall 800bc2c37a

update config 8985110b6c

update top-level service 0ea4d50856

go fmt 969e650f67

temporary go mod using replace directive bae020d7ef

add ability to fall through to another RPC provider b092154e33

i-norden added 1 commit 2023-11-01 21:09:04 +00:00

go mod bab45dfda7

i-norden added 3 commits 2023-11-02 13:22:23 +00:00

debug_traceCall test b1f3ec613e

TraceBlockByHash and TraceBlockByNumber methods 7de639f3fb

debug_traceBlock test 7e8b372a56

i-norden commented 2023-11-02 14:53:38 +00:00

Author

Member

Copy Link

Copy Source

Hmm I'm wondering where our CI testing went for this repo...

Hmm I'm wondering where our CI testing went for this repo...

i-norden force-pushed ian/tracing from 7e8b372a56 to 1d578b3bf9 2023-11-02 18:03:27 +00:00 Compare

i-norden added 1 commit 2023-11-06 13:27:07 +00:00

go mod 0b53b52b3f

i-norden force-pushed ian/tracing from 0b53b52b3f to cae2a3e843 2023-11-06 13:47:54 +00:00 Compare

i-norden force-pushed ian/tracing from cae2a3e843 to 51747c01a4 2023-11-06 13:49:23 +00:00 Compare

i-norden force-pushed ian/tracing from 51747c01a4 to 8ff7b4077b 2023-11-06 13:57:58 +00:00 Compare

i-norden force-pushed ian/tracing from 8ff7b4077b to 8dfdfcc484 2023-11-06 14:10:28 +00:00 Compare

i-norden force-pushed ian/tracing from 8dfdfcc484 to 877b731340 2023-11-06 14:25:15 +00:00 Compare

i-norden added 1 commit 2023-11-08 14:19:40 +00:00

fix CI errors a2fe12334c

i-norden force-pushed ian/tracing from a2fe12334c to fbcc9664da 2023-11-08 14:44:47 +00:00 Compare

i-norden force-pushed ian/tracing from fbcc9664da to ec49318294 2023-11-08 14:54:41 +00:00 Compare

i-norden commented 2023-11-08 15:24:51 +00:00

Author

Member

Copy Link

Copy Source

Hey @roysc @telackey curious if you have seen this error that is popping up when trying to use stack-orchestrator to run the testnet tests (it can't find ipld-eth-db v5.0.5-alpha)?

Hey @roysc @telackey curious if you have seen this error that is popping up when trying to use stack-orchestrator to run the testnet tests (it can't find ipld-eth-db v5.0.5-alpha)?

dboreham commented 2023-11-08 16:15:32 +00:00

First-time contributor

Copy Link

Copy Source

I'm not sure of the whole context, but that branch (or tag) doesn't exist in this repo: https://github.com/cerc-io/ipld-eth-db
Should it be pulling the Gitea hosted version of the repo? (but I don't see that branch there either). I'm actually not sure where
v5.0.5-alpha came from because in the log it said it was looking for branch v5.

I'm not sure of the whole context, but that branch (or tag) doesn't exist in this repo: https://github.com/cerc-io/ipld-eth-db Should it be pulling the Gitea hosted version of the repo? (but I don't see that branch there either). I'm actually not sure where v5.0.5-alpha came from because in the log it said it was looking for branch v5.

👀 1

i-norden commented 2023-11-08 16:31:20 +00:00

Author

Member

Copy Link

Copy Source

@dboreham none of the v5 tags appear to be in that repo any longer if you view the releases here https://github.com/cerc-io/ipld-eth-db/releases or tags here https://github.com/cerc-io/ipld-eth-db/tags, but you can still see the v5.0.5-alpha release here: https://github.com/cerc-io/ipld-eth-db/releases/tag/v5.0.5-alpha and v5.0.5-alpha is the version in use across most our repos currently. Something weird has happened to our tagged releases...

@dboreham none of the v5 tags appear to be in that repo any longer if you view the releases here https://github.com/cerc-io/ipld-eth-db/releases or tags here https://github.com/cerc-io/ipld-eth-db/tags, but you can still see the v5.0.5-alpha release here: https://github.com/cerc-io/ipld-eth-db/releases/tag/v5.0.5-alpha and v5.0.5-alpha is the version in use across most our repos currently. Something weird has happened to our tagged releases...

i-norden commented 2023-11-08 16:42:05 +00:00

Author

Member

Copy Link

Copy Source

And it looks like all the v5 releases are gone in the gitea repo too: https://git.vdb.to/cerc-io/ipld-eth-db/tags

And it looks like all the v5 releases are gone in the gitea repo too: https://git.vdb.to/cerc-io/ipld-eth-db/tags

dboreham commented 2023-11-08 16:43:48 +00:00

First-time contributor

Copy Link

Copy Source

The relationship between releases and tags is odd, iirc.
Probably the thing to do is to trigger a new release on the Gitea side.
We can't use releases from Github (as far as I know) because the sync from Gitea will blow them away.

The relationship between releases and tags is odd, iirc. Probably the thing to do is to trigger a new release on the Gitea side. We can't use releases from Github (as far as I know) because the sync from Gitea will blow them away.

i-norden commented 2023-11-08 16:50:08 +00:00

Author

Member

Copy Link

Copy Source

Inconsistencies in the releases and tags seem to be present in many of the repos that have been ported over to gitea

Inconsistencies in the releases and tags seem to be present in many of the repos that have been ported over to gitea

i-norden added 1 commit 2023-11-16 14:39:38 +00:00

update ipld-eth-db ref c7f0c9a1a4

i-norden force-pushed ian/tracing from c7f0c9a1a4 to b9983b8c55 2023-11-20 13:32:43 +00:00 Compare

i-norden force-pushed ian/tracing from b9983b8c55 to ec49318294 2023-11-20 13:50:30 +00:00 Compare

i-norden added 2 commits 2023-12-07 06:41:19 +00:00

update ipld-eth-db ref c7f0c9a1a4

CICD_REPO_TOKEN => CICD_PUBLISH_TOKEN 6b271ce5f3

i-norden force-pushed ian/tracing from 6b271ce5f3 to c7f0c9a1a4 2023-12-07 07:15:50 +00:00 Compare

i-norden commented 2024-01-11 15:21:12 +00:00

Author

Member

Copy Link

Copy Source

I'd like to merge this, the tests pass, but there is an issue with the ipld-eth-db image not being published https://git.vdb.to/cerc-io/ipld-eth-db/actions/runs/1

I'd like to merge this, the tests pass, but there is an issue with the ipld-eth-db image not being published https://git.vdb.to/cerc-io/ipld-eth-db/actions/runs/1

i-norden added 1 commit 2024-01-22 20:11:12 +00:00

bump ipld-eth-db to v5.2.1-alpha 3399e4aa3c

i-norden force-pushed ian/tracing from 3399e4aa3c to 2009cecefd 2024-01-22 20:16:36 +00:00 Compare

i-norden force-pushed ian/tracing from 2009cecefd to 29c1523eb8 2024-01-22 20:17:37 +00:00 Compare

i-norden added 1 commit 2024-01-24 17:11:23 +00:00

update SO version 86e9d79d47

i-norden force-pushed ian/tracing from 86e9d79d47 to a8d6ca8305 2024-01-25 15:58:30 +00:00 Compare

i-norden force-pushed ian/tracing from a8d6ca8305 to 8ec25764cd 2024-01-25 20:44:40 +00:00 Compare

i-norden force-pushed ian/tracing from 8ec25764cd to 402a6e183b 2024-01-29 14:30:35 +00:00 Compare

Some required checks failed

Hide all checks

Test the stack. / Run unit tests (pull_request) Successful in 5m4s

Details

Test the stack. / Run integration tests (pull_request) Failing after 5m37s

Details

This pull request has changes conflicting with the target branch.

• .github/workflows/tests.yaml
• go.mod
• go.sum
• pkg/eth/api.go
• pkg/eth/test_helpers/chain_indexer.go
• test/compose-db.yml
• test/stack-refs.txt

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin ian/tracing:ian/tracing

git checkout ian/tracing

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

Epic

Integration tests

bug

Something isn't working

critical

documentation

Improvements or additions to documentation

duplicate

This issue or pull request already exists

enhancement

New feature or request

good first issue

Good for newcomers

help wanted

Extra attention is needed

invalid

This doesn't seem right

question

Further information is requested

v5

wontfix

This will not be worked on

Copied from Github

An issue or PR manually copied from GitHub.

Kind/Breaking

Breaking change that won't be backward compatible

Kind/Bug

Something is not working

Kind/Documentation

Documentation changes

Kind/Enhancement

Improve existing functionality

Kind/Feature

New functionality

Kind/Security

This is security issue

Kind/Testing

Issue or pull request related to testing

Priority

Critical

The priority is critical

Priority

High

The priority is high

Priority

Low

The priority is low

Priority

Medium

The priority is medium

Reviewed

Confirmed

Issue has been confirmed

Reviewed

Duplicate

This issue or pull request already exists

Reviewed

Invalid

Invalid issue

Reviewed

Won't Fix

This issue won't be fixed

Status

Abandoned

Somebody has started to work on this but abandoned work

Status

Blocked

Something is blocking this issue or pull request

Status

Need More Info

Feedback is required to reproduce issue or to continue work

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

AFDudley Alex ashwin cerccicd ericlewis i-norden (Ian Norden) prathamesh (Prathamesh Musale) rachmaninovquar rjk-laconic roysc (Roy Crihfield) srw zramsay

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/ipld-eth-server#261