cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity

fix encoding when storage is empty #94

Merged
n0cte merged 14 commits from empty_data_encoder into master 2021-09-28 05:17:43 +00:00
Conversation 3 Commits 14 Files Changed 11 +132 -29
n0cte commented 2021-08-19 08:50:57 +00:00 (Migrated from github.com)
Copy Link
No description provided.
ashwinphatak (Migrated from github.com) reviewed 2021-08-19 08:50:57 +00:00
i-norden requested changes 2021-08-20 02:22:31 +00:00
i-norden left a comment
Member
Copy Link

Looks good, just a few questions/comments.

Looks good, just a few questions/comments.
pkg/eth/api.go
i-norden commented 2021-08-20 02:20:48 +00:00
Member
Copy Link

So just to double check and make sure I follow, an empty 32 byte slice is what geth would return here?

So just to double check and make sure I follow, an empty 32 byte slice is what geth would return here?
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-08-20 02:16:39 +00:00
Member
Copy Link

Just to note, we should only ever see a zero value for a node that is not "Removed" if we are at a block height prior to EIP-158 activation (which is when zeroed value pruning begun). Does this make sense, and align with what we are seeing?

Just to note, we should only ever see a zero value for a node that is not "Removed" if we are at a block height prior to EIP-158 activation (which is when zeroed value pruning begun). Does this make sense, and align with what we are seeing?
i-norden commented 2021-08-20 02:19:14 +00:00
Member
Copy Link

I think we need the analogous check in the account retrieval methods above (RetrieveAccountByAddressAndBlockNumber and RetrieveAccountByAddressAndBlockHash) and the other storage retrieval method below (RetrieveStorageAtByAddressAndStorageKeyAndBlockNumber).

I think we need the analogous check in the account retrieval methods above (`RetrieveAccountByAddressAndBlockNumber` and `RetrieveAccountByAddressAndBlockHash`) and the other storage retrieval method below (`RetrieveStorageAtByAddressAndStorageKeyAndBlockNumber`).
ashwinphatak (Migrated from github.com) reviewed 2021-08-20 05:13:34 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
ashwinphatak (Migrated from github.com) commented 2021-08-20 05:13:34 +00:00
Copy Link

We're running a local/private chain for which the value in genesis is "eip158Block": 0,, so this isn't consistent with what we expect?

We're running a local/private chain for which the value in genesis is `"eip158Block": 0,`, so this isn't consistent with what we expect?
ramilexe (Migrated from github.com) reviewed 2021-08-20 07:23:35 +00:00
pkg/eth/api.go
ramilexe (Migrated from github.com) commented 2021-08-20 07:23:34 +00:00
Copy Link

yes

yes
ramilexe (Migrated from github.com) reviewed 2021-08-20 07:33:14 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
ramilexe (Migrated from github.com) commented 2021-08-20 07:33:14 +00:00
Copy Link

Where is RetrieveAccountByAddressAndBlockNumber function called? I couldn't find it. And how to test account deleting?

Where is `RetrieveAccountByAddressAndBlockNumber` function called? I couldn't find it. And how to test account deleting?
ramilexe (Migrated from github.com) reviewed 2021-08-20 07:40:15 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
ramilexe (Migrated from github.com) commented 2021-08-20 07:40:15 +00:00
Copy Link

Does this make sense, and align with what we are seeing?

It makes sense but doesn't align with current behavior. Isn't this misalignment was introduced here in this PR https://github.com/vulcanize/go-ethereum/pull/58 ?

> Does this make sense, and align with what we are seeing? It makes sense but doesn't align with current behavior. Isn't this misalignment was introduced here in this PR https://github.com/vulcanize/go-ethereum/pull/58 ?
i-norden reviewed 2021-08-23 13:26:18 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-08-23 13:26:18 +00:00
Member
Copy Link

Hmm I need to think about this some more, sorry, let me get back.

Hmm I need to think about this some more, sorry, let me get back.
i-norden reviewed 2021-08-23 13:26:37 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-08-23 13:26:37 +00:00
Member
Copy Link

I think we want to fix the function even if it isn't currently called. We could test account deleting similar to how we do it here: https://github.com/vulcanize/go-ethereum/blob/v1.10.7-statediff/statediff/builder_test.go#L1318

I think we want to fix the function even if it isn't currently called. We could test account deleting similar to how we do it here: https://github.com/vulcanize/go-ethereum/blob/v1.10.7-statediff/statediff/builder_test.go#L1318
i-norden reviewed 2021-08-31 18:18:26 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-08-31 18:18:25 +00:00
Member
Copy Link

Does this make sense, and align with what we are seeing?

It makes sense but doesn't align with current behavior. Isn't this misalignment was introduced here in this PR vulcanize/go-ethereum#58 ?

If that is the cause of the misalignment then all we need to do is check that the node isn't a "Removed" type when we query for the latest value at a given path/key, which I think we are already doing?

> > Does this make sense, and align with what we are seeing? > > It makes sense but doesn't align with current behavior. Isn't this misalignment was introduced here in this PR [vulcanize/go-ethereum#58](https://github.com/vulcanize/go-ethereum/pull/58) ? If that is the cause of the misalignment then all we need to do is check that the node isn't a "Removed" type when we query for the latest value at a given path/key, which I think we are already doing?
i-norden reviewed 2021-08-31 18:27:14 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-08-31 18:27:14 +00:00
Member
Copy Link

@arijitAD is taking over here, to echo what I told him in slack:

First thing to do is verify that we are excluding "Removed" type nodes when we perform those queries.

If we arent doing that then we are actually just seeing null values where they are supposed to be, and need to add a check to filter those out

But I think we are already filtering out "Removed" nodes in those queries, and if we are already doing that then there is some deeper reason as to why we are seeing null values for nodes that are not of the "Removed" type

Another possible lead is how we (and geth) are distinguishing (or not) between true nulls and legitimate zero values

@arijitAD is taking over here, to echo what I told him in slack: First thing to do is verify that we are excluding "Removed" type nodes when we perform those queries. If we arent doing that then we are actually just seeing null values where they are supposed to be, and need to add a check to filter those out But I think we are already filtering out "Removed" nodes in those queries, and if we are already doing that then there is some deeper reason as to why we are seeing null values for nodes that are not of the "Removed" type Another possible lead is how we (and geth) are distinguishing (or not) between true nulls and legitimate zero values
i-norden reviewed 2021-09-06 18:34:59 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-09-06 18:34:59 +00:00
Member
Copy Link

On the other side of this coin, if 0 values are not showing up when they are supposed to it is likely because the query being made against the database does not properly consider the "Removed" node type entries as indicating a 0 value and instead the query finds the last entry where the node is not "Removed" and returns that latest non-zero value (which is the behavior we want in some cases, but not here).

On the other side of this coin, if 0 values are not showing up when they are supposed to it is likely because the query being made against the database does not properly consider the "Removed" node type entries as indicating a 0 value and instead the query finds the last entry where the node is not "Removed" and returns that latest non-zero value (which is the behavior we want in some cases, but not here).
i-norden reviewed 2021-09-13 13:43:55 +00:00
pkg/eth/ipld_retriever.go
@ -438,9 +441,11 @@ func (r *IPLDRetriever) RetrieveAccountByAddressAndBlockHash(address common.Addr
if err := r.db.Get(accountResult, RetrieveAccountByLeafKeyAndBlockHashPgStr, leafKey.Hex(), hash.Hex()); err != nil {
i-norden commented 2021-09-13 13:43:55 +00:00
Member
Copy Link

Please see https://github.com/vulcanize/ipld-eth-server/issues/95#issuecomment-918202095 there is a much simpler fix now that we have leaf_key for "Removed" nodes

Please see https://github.com/vulcanize/ipld-eth-server/issues/95#issuecomment-918202095 there is a much simpler fix now that we have leaf_key for "Removed" nodes
arijitAD commented 2021-09-24 12:20:13 +00:00 (Migrated from github.com)
Copy Link

Added migration https://github.com/vulcanize/statediff-migrations/pull/19

Added migration https://github.com/vulcanize/statediff-migrations/pull/19
i-norden reviewed 2021-09-24 22:04:15 +00:00
i-norden left a comment
Member
Copy Link

Looks good but see https://github.com/vulcanize/statediff-migrations/pull/19#discussion_r715925284, we can improve the stored function we are using to determine if a contract was destroyed or not.

Looks good but see https://github.com/vulcanize/statediff-migrations/pull/19#discussion_r715925284, we can improve the stored function we are using to determine if a contract was destroyed or not.
arijitAD commented 2021-09-27 12:31:39 +00:00 (Migrated from github.com)
Copy Link

Looks good but see vulcanize/statediff-migrations#19 (comment), we can improve the stored function we are using to determine if a contract was destroyed or not.

Done

> Looks good but see [vulcanize/statediff-migrations#19 (comment)](https://github.com/vulcanize/statediff-migrations/pull/19#discussion_r715925284), we can improve the stored function we are using to determine if a contract was destroyed or not. Done
i-norden approved these changes 2021-09-27 15:47:25 +00:00
i-norden left a comment
Member
Copy Link

Do the unit tests here test the new function in both contexts (when the contract has been destroyed at that height and when it hasn't)? If so, good to go, if not let's add that.

Do the unit tests here test the new function in both contexts (when the contract has been destroyed at that height and when it hasn't)? If so, good to go, if not let's add that.
arijitAD commented 2021-09-27 16:08:18 +00:00 (Migrated from github.com)
Copy Link

Do the unit tests here test the new function in both contexts (when the contract has been destroyed at that height and when it hasn't)? If so, good to go, if not let's add that.

Yes, It checks before destroying the contract and after it is destroyed.

> Do the unit tests here test the new function in both contexts (when the contract has been destroyed at that height and when it hasn't)? If so, good to go, if not let's add that. Yes, It checks before destroying the contract and after it is destroyed.
Sign in to join this conversation.
Reviewers
No reviewers
i-norden
ashwinphatak
Labels
Clear labels   
bug

Something isn't working

  
critical

   
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
enhancement

New feature or request

  
Epic

   
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
Integration tests

   
invalid

This doesn't seem right

  
question

Further information is requested

  
v5

   
wontfix

This will not be worked on

  
Copied from Github

An issue or PR manually copied from GitHub.

  
Kind/Breaking

Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No Label
bug
critical
documentation
duplicate
enhancement
Epic
good first issue
help wanted
Integration tests
invalid
question
v5
wontfix
Copied from Github
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/ipld-eth-server#94
No description provided.
Delete Branch "empty_data_encoder"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?