Commit Graph
340 Commits
Author SHA1 Message Date
philip-morlier f912a2192d Merge commit '0d45d72d7' into merge/geth-v1.13.3 2023-10-12 07:16:53 -07:00
Martin Holst Swende 8976a0c97a trie: remove owner and binary marshaling from stacktrie (#28291)
This change
  - Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
  - Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
2023-10-11 06:12:45 +02:00
Brandon LiuandMartin Holst Swende 6505297456 trie: fix a typo, use correct docstrings (#28302)
* fix a typo

* trie: additional fixes to docstrings

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-10-10 10:32:14 +02:00
Martin Holst Swende 6b1e4f4211 all: move light.NodeSet to trienode.ProofSet (#28287)
This is a minor refactor in preparation of changes to range verifier. This PR contains no intentional functional changes but moves (and renames) the light.NodeSet
2023-10-10 10:30:47 +02:00
Martin Holst SwendeandGary Rong 08326794e8 trie: refactor stacktrie (#28233)
This change refactors stacktrie to separate the stacktrie itself from the
internal representation of nodes: a stacktrie is not a recursive structure
of stacktries, rather, a framework for representing and operating upon a set of nodes.

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-10-10 08:28:56 +02:00
Chirag GargandMartin Holst Swende 2091ebdf5e trie: fix benchmark by ensuring key immutability (#28221)
This change fixes the bug in a benchmark, where the input to the trie is reused in a way which is not correct. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-10-03 07:46:22 -04:00
philip-morlier 18be67b3a3 Merge commit 'dc34fe829' into merge/geth-v1.13.2 2023-09-28 08:41:54 -07:00
rjl493456442 73f5bcb75b core, accounts, eth, trie: handle genesis state missing (#28171)
* core, accounts, eth, trie: handle genesis state missing

* core, eth, trie: polish

* core: manage txpool subscription in mainpool

* eth/backend: fix test

* cmd, eth: fix test

* core/rawdb, trie/triedb/pathdb: address comments

* eth, trie: address comments

* eth: inline the function

* eth: use synced flag

* core/txpool: revert changes in txpool

* core, eth, trie: rename functions
2023-09-28 10:00:53 +03:00
rjl493456442 03c2176a1d trie/triedb/pathdb: improve error log (#28177) 2023-09-22 09:33:17 +03:00
rjl493456442 4773dcbc81 trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-09-22 09:31:10 +03:00
Guillaume Ballet 7ed5bc021a trie: add getter for preimage store in trie.Database (#28155) 2023-09-19 08:47:24 -04:00
philip-morlier bdc42a9128 Merge commit '7371b3817' into merge/geth-v1.13.0 2023-09-12 12:34:24 -07:00
rjl493456442 a7842c9cae core, trie: cleanup trie database (#28062) 2023-09-07 21:17:14 +08:00
Péter Szilágyi 0c6bbeb423 core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
Marius van der Wijden 5976e58415 trie: reduce allocs in recHash (#27770) 2023-08-18 22:41:19 +02:00
Paweł Bylica ab28680e66 trie: add tests for "short" nodes in StackTrie (#27932) 2023-08-15 14:16:00 -04:00
Péter Szilágyi be65b47645 all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
Péter Szilágyi 0ce331f56a trie/triedb/pathdb: make shutdown journal log friendlier (#27905) 2023-08-11 17:05:35 +03:00
503f1f7ada all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
7de748d3f6 all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
rjl493456442 817553cc28 core, trie: track state change set with account address (#27815) 2023-07-31 15:07:51 +03:00
rjl493456442 88f3d61468 all: expose block number information to statedb (#27753)
* core/state: clean up

* all: add block number infomration to statedb

* core, trie: rename blockNumber to block
2023-07-24 13:22:09 +03:00
philip-morlier e9630a85f4 This is a functional build a dev tagged release of foundation plugeth
This brings the project up to geth v1.12.0

The plugeth test plugin has been added but is failing on the live tracer hooks as of this commit
2023-07-13 13:10:19 -07:00
philip-morlier dc56f2a361 incremental commit, manual changes to support merge 2023-07-11 17:01:10 -07:00
philip-morlier e516d5842c automated merge 2023-07-11 16:01:17 -07:00
rjl493456442 4b06e4f25e core/state: value diff tracking in StateDB (#27349)
This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.
2023-07-11 15:43:23 +02:00
rjl493456442 59f7b289c3 cmd, core, eth, graphql, trie: no persisted clean trie cache file (#27525)
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.
2023-07-04 10:21:06 +03:00
Guillaume BalletandGary Rong 8bbb16b70e core/state, light, les: make signature of ContractCode hash-independent (#27209)
* core/state, light, les: make signature of ContractCode hash-independent

* push current state for feedback

* les: fix unit test

* core, les, light: fix les unittests

* core/state, trie, les, light: fix state iterator

* core, les: address comments

* les: fix lint

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-06-28 12:11:02 +03:00
Guillaume Balletandrjl493456442 c7b099b2ea trie, core/state: revert error removal in (*state.Trie).Commit (#27544)
* trie, core/state: revert error removal in (*state.Trie).Commit

* Gary's nitpick :)

Co-Authored-By:  rjl493456442 <garyrong0905@gmail.com>

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2023-06-27 15:36:38 +03:00
Guillaume Ballet 699243f8ae core/state, light, trie: add UpdateContractCode to the Trie interface (#27476)
Verkle trees store the code inside the trie. This PR changes the interface to pass the code, as well as the dirty flag to tell the trie package if the code is dirty and needs to be updated. This is a no-op for the MPT and the odr trie.
2023-06-22 08:52:52 -04:00
rjl493456442 6d2aeb43d5 cmd, core/state, eth, tests, trie: improve state reader (#27428)
The state availability is checked during the creation of a state reader.

-    In hash-based database, if the specified root node does not exist on disk disk, then
    the state reader won't be created and an error will be returned.

-    In path-based database, if the specified state layer is not available, then the
    state reader won't be created and an error will be returned.

This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.
2023-06-20 15:31:45 -04:00
rjl493456442 ceca4578ca trie: remove parameter 'fromLevel' in Prove (#27512)
This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth 
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.
2023-06-19 16:28:40 +02:00
Dan LaineandFelix Lange 50ecb16de0 tests, trie: use slices package for sorting (#27496)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-19 11:41:31 +02:00
Guillaume Ballet 45a3ab42aa core/state: move slot RLP encoding into the MPT implementation (#27000)
Continuing with a series of PRs to make the Trie interface more generic, this PR moves
the RLP encoding of storage slots inside the StateTrie and light.Trie implementations,
as other types of tries don't use RLP.
2023-06-01 10:29:41 +02:00
rjl493456442 a14301823e all: new empty trie with types.EmptyRootHash instead of null (#27230) 2023-05-11 10:19:42 +03:00
rjl493456442 5021d36d35 all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
rjl493456442 5d3f5805d5 trie: add node type common package (#27160)
* trie: add node type common package

In trie/types package, a few node wrappers are defined, which will be used
in both trie package, trie/snap package, etc. Therefore, a standalone common
package is created to put these stuffs.

* trie: rename trie/types to trie/trienode
2023-04-26 09:01:54 +03:00
rjl493456442 bbcb5ea37b core, trie: rework trie database (#26813)
* core, trie: rework trie database

* trie: fix comment
2023-04-24 10:38:52 +03:00
philip-morlier e986bbfa6f Initial commit of merge geth v1.11.5 2023-04-21 13:24:40 -07:00
rjl493456442 99f81d2724 all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Guillaume BalletandGary Rong 41f89ca944 core/state, trie: remove Try prefix in Trie accessors (#26975)
This change renames StateTrie methods to remove the Try* prefix. 

We added the Trie methods with prefix 'Try' a long time ago, working
around the problem that most existing methods of Trie did not return the
database error. This weird naming convention has persisted until now.

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-03-27 10:48:46 +02:00
Guillaume Ballet 8990c92aea core/state: add account address to Trie slot accessors (#26934)
This changes the Trie interface to add the plain account address as a
parameter to all storage-related methods.

After the introduction of the TryAccount* functions, TryGet, TryUpdate and
TryDelete are now only meant to read an account's storage. In their current
form, they assume that an account storage is stored in a separate trie, and
that the hashing of the slot is independent of its account's address.

The proposed structure for a stateless storage breaks these two
assumptions: the hashing of a slot key requires the address and all slots
and accounts are stored in a single trie.

This PR therefore adds an address parameter to the interface. It is ignored
in the MPT version, so this change has no functional impact, however it
will reduce the diff size when merging verkle trees.
2023-03-23 11:52:22 +01:00
philip-morlier f3e558d4e0 Merge commit 'a38f41085' into merge/geth-v1.11.5
Manual touches in core/state_transtion.go and go.mod
2023-03-21 08:39:16 -07:00
Marius van der Wijden 81b0aa0cc7 trie: reduce unit test time (#26918) 2023-03-20 04:09:35 -04:00
Darioush Jalali b7bfbc1e64 trie, accounts/abi: add error-checks (#26914) 2023-03-17 06:19:51 -04:00
rjl493456442 c8a6b7100c core/state, trie: port changes from PBSS (#26763) 2023-03-14 04:50:53 -04:00
philip-morlier 34191cb50a Removed unnecessary comments from trie/plugin_hooks.go 2023-03-08 14:51:21 -08:00
philip-morlier 3527a74333 type correction in trei hoppks, adjusting length condition in core hooks 2023-03-08 11:37:28 -08:00
philip-morlier e2c33f60c4 Added missing import to trie/plugin_hooks 2023-03-08 10:10:52 -08:00
philip-morlier 27810f60ad Added hook into trie.Database.Commit() in the trie package.
The hook enables a pre and post commit plugin to report on the status of the commit and expose this information to plugins.
2023-03-08 09:59:47 -08:00