Commit Graph

393 Commits

Author SHA1 Message Date
aaronbuchwald
56d2366699
core/state/snapshot: replace diffToDisk ideal batch size with 64MB (#27977) 2023-08-25 15:48:10 +03:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
Martin Holst Swende
eb6cbe37e1
core/state: remove public method ForEachStorage (#27986)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 21:53:31 +02:00
Péter Szilágyi
0c6bbeb423
core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
rjl493456442
503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Guillaume Ballet
67979022aa
core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-08-09 18:02:45 +02:00
rjl493456442
7de748d3f6
all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
rjl493456442
817553cc28
core, trie: track state change set with account address (#27815) 2023-07-31 15:07:51 +03:00
rjl493456442
88f3d61468
all: expose block number information to statedb (#27753)
* core/state: clean up

* all: add block number infomration to statedb

* core, trie: rename blockNumber to block
2023-07-24 13:22:09 +03:00
jwasinger
988d84aa7c
core/state, core/vm: implement EIP 6780 (#27189)
EIP-6780: SELFDESTRUCT only in same transaction

>     SELFDESTRUCT will recover all funds to the caller but not delete the account, except when called in the same transaction as creation

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-07-17 13:02:18 -04:00
jwasinger
d233b6b23a
core: replace instances of 'suicide' with 'selfdestruct' to improve code consistency. (#27716)
---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
2023-07-15 10:35:30 -04:00
rjl493456442
4b06e4f25e
core/state: value diff tracking in StateDB (#27349)
This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.
2023-07-11 15:43:23 +02:00
rjl493456442
59f7b289c3
cmd, core, eth, graphql, trie: no persisted clean trie cache file (#27525)
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.
2023-07-04 10:21:06 +03:00
Guillaume Ballet
8bbb16b70e
core/state, light, les: make signature of ContractCode hash-independent (#27209)
* core/state, light, les: make signature of ContractCode hash-independent

* push current state for feedback

* les: fix unit test

* core, les, light: fix les unittests

* core/state, trie, les, light: fix state iterator

* core, les: address comments

* les: fix lint

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-06-28 12:11:02 +03:00
Guillaume Ballet
c7b099b2ea
trie, core/state: revert error removal in (*state.Trie).Commit (#27544)
* trie, core/state: revert error removal in (*state.Trie).Commit

* Gary's nitpick :)

Co-Authored-By:  rjl493456442 <garyrong0905@gmail.com>

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2023-06-27 15:36:38 +03:00
Guillaume Ballet
699243f8ae
core/state, light, trie: add UpdateContractCode to the Trie interface (#27476)
Verkle trees store the code inside the trie. This PR changes the interface to pass the code, as well as the dirty flag to tell the trie package if the code is dirty and needs to be updated. This is a no-op for the MPT and the odr trie.
2023-06-22 08:52:52 -04:00
rjl493456442
6d2aeb43d5
cmd, core/state, eth, tests, trie: improve state reader (#27428)
The state availability is checked during the creation of a state reader.

-    In hash-based database, if the specified root node does not exist on disk disk, then
    the state reader won't be created and an error will be returned.

-    In path-based database, if the specified state layer is not available, then the
    state reader won't be created and an error will be returned.

This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.
2023-06-20 15:31:45 -04:00
Dan Laine
154b016b6c
core: use slices package for sorting (#27489)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-20 11:58:47 +02:00
rjl493456442
ceca4578ca
trie: remove parameter 'fromLevel' in Prove (#27512)
This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth 
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.
2023-06-19 16:28:40 +02:00
rjl493456442
0e5d2c7c53
core/state/snapshot, core/types, eth: move account definition to type (#27323)
* core/state/snapshot, core/types, eth: move account definition to type

* core, eth: revert snapshot Account API change
2023-06-06 11:17:39 +03:00
rjl493456442
380fb4e249
core/state: clear out cached state data when reset occurs (#27376)
* core/state: remove cached snap data if reset occurs

* core/state: address comment from peter

* core/state: skip revert in case data is nil
2023-06-05 16:25:57 +03:00
Péter Szilágyi
c7c84ca16c
all: remove the Rinkeby testnet (#27406) 2023-06-02 14:03:21 +03:00
rjl493456442
15bd21f3c8
core/state: mark account as dirty when resetObject occurs (#27339)
This changes the journal logic to mark the state object dirty immediately when it
is reset. 

We're mostly adding this change to appease the fuzzer. Marking it dirty immediately
makes no difference in practice because accounts will always be modified by EVM
right after creation.
2023-06-01 11:09:32 +02:00
Guillaume Ballet
45a3ab42aa
core/state: move slot RLP encoding into the MPT implementation (#27000)
Continuing with a series of PRs to make the Trie interface more generic, this PR moves
the RLP encoding of storage slots inside the StateTrie and light.Trie implementations,
as other types of tries don't use RLP.
2023-06-01 10:29:41 +02:00
Delweng
6c732766c8
core,console: replace noarg fmt.Errorf with errors.New (#27332)
* core: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* console: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* core: go autoimport

Signed-off-by: jsvisa <delweng@gmail.com>

* core: dry

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-25 08:24:09 -04:00
Park Changwan
bfded65ed8
core/state: do not ignore null addr while iterative dump (#27320)
fixes bug which caused the zero-address to be ignored during an iterative state-dump.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-05-23 06:10:26 -04:00
John Chase
2f2959d003
core/state/pruner: remove unused error-return (#27273) 2023-05-17 04:23:06 -04:00
Martin Holst Swende
eb83e7c540
core/state/snapshot: check difflayer staleness early (#27255)
This PR adds a staleness-check to AccountRLP, before checking the bloom-filter and potentially going directly into the disklayer.

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2023-05-16 09:18:39 -04:00
rjl493456442
a14301823e
all: new empty trie with types.EmptyRootHash instead of null (#27230) 2023-05-11 10:19:42 +03:00
Guillaume Ballet
7577b9c28f
core/state: unexport NodeIterator (#27239) 2023-05-11 10:15:44 +03:00
rjl493456442
5021d36d35
all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
ucwong
cc8d40c65f
core/state: initialize maps with known size (#27222)
* core/state : fix map size avoid resizing

* core/state : fixed size
2023-05-08 09:59:14 +03:00
rjl493456442
99f81d2724
all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Anusha
cb66eba85a
core: fix comment to reflect function name (#27070) 2023-04-17 11:02:31 -04:00
s7v7nislands
fb8a3aaf1e
core/state: use atomic.Bool (#26992) 2023-03-28 03:06:50 -04:00
Guillaume Ballet
41f89ca944
core/state, trie: remove Try prefix in Trie accessors (#26975)
This change renames StateTrie methods to remove the Try* prefix. 

We added the Trie methods with prefix 'Try' a long time ago, working
around the problem that most existing methods of Trie did not return the
database error. This weird naming convention has persisted until now.

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-03-27 10:48:46 +02:00
Guillaume Ballet
8990c92aea
core/state: add account address to Trie slot accessors (#26934)
This changes the Trie interface to add the plain account address as a
parameter to all storage-related methods.

After the introduction of the TryAccount* functions, TryGet, TryUpdate and
TryDelete are now only meant to read an account's storage. In their current
form, they assume that an account storage is stored in a separate trie, and
that the hashing of the slot is independent of its account's address.

The proposed structure for a stateless storage breaks these two
assumptions: the hashing of a slot key requires the address and all slots
and accounts are stored in a single trie.

This PR therefore adds an address parameter to the interface. It is ignored
in the MPT version, so this change has no functional impact, however it
will reduce the diff size when merging verkle trees.
2023-03-23 11:52:22 +01:00
rjl493456442
bba2a1bac5
core: show db error-info in case of mismatched hash root (#26870)
When a database failure occurs, bubble it up a into statedb, and report it in suitable places, such as during a 'bad block' report.
2023-03-16 03:12:34 -04:00
rjl493456442
c8a6b7100c
core/state, trie: port changes from PBSS (#26763) 2023-03-14 04:50:53 -04:00
rjl493456442
fe01a2f63b
all: use unified emptyRootHash and emptyCodeHash (#26718)
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
2023-02-21 06:12:27 -05:00
rjl493456442
13ef21d467
Revert "core/trie: remove trie tracer (#26665)" (#26732)
This reverts commit 7c749c947a.
2023-02-20 09:54:52 -05:00
Martin Holst Swende
7c749c947a
core/trie: remove trie tracer (#26665)
This PR contains a small portion of the full pbss PR, namely

    Remove the tracer from trie (and comitter), and instead using an accessList.
    Related changes to the Nodeset.


---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-19 14:10:19 -05:00
Martin Holst Swende
4d3525610e
all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Martin Holst Swende
22c3ad1d12
core/state, trie: remove unused error-return from trie Commit operation (#26641) 2023-02-09 08:56:59 -05:00
rjl493456442
9842301376
all: remove database commit callback, rework noderesolver (#26637)
This change ports some changes from the main PBSS PR:

  - get rid of callback function in `trie.Database.Commit` which is not required anymore
  - rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
  - some other shallow changes in tests and typo-fixes
2023-02-08 06:14:34 -05:00
Martin Holst Swende
8860b39754
all: prepare for path-based trie storage (#26603)
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.

This PR is intended to be a no-functionality-change prep PR for #25963 .

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-06 10:28:40 -05:00
Martin Holst Swende
7a489623ac
core/state: remove notion of fake storage (#24916)
This PR removes the notion of fakeStorage from the state objects, and instead, for any state modifications that are needed, it simply makes the changes.
2023-01-10 08:24:30 -05:00
rjl493456442
6c149fd4ad
core, eth, trie, light: clean up trie interface (#26388)
* all: cleanup trie interface

* eth, trie: address comments
2023-01-03 15:41:40 +02:00
rjl493456442
c87f321b8f
core/state: maintain destruction flag by default (#26371)
This changes moves the tracking of "deleted in this block" out from snap-only domain, so that it happens regardless of whether the execution is snapshot-backed or trie-backed.
2022-12-28 08:53:43 -05:00