Commit Graph

586 Commits

Author SHA1 Message Date
Martin Holst Swende
c5b7cfa9c3
tests: skip tests on windows 32bit CI (#28521)
tests: skip half the blockchain- and state-tests on windows 32bit CI-tests
2023-11-14 18:40:16 +01:00
Martin Holst Swende
2391fbc676
tests/fuzzers: move fuzzers into native packages (#28467)
This PR moves our fuzzers from tests/fuzzers into whatever their respective 'native' package is.

The historical reason why they were placed in an external location, is that when they were based on go-fuzz, they could not be "hidden" via the _test.go prefix. So in order to shove them away from the go-ethereum "production code", they were put aside.

But now we've rewritten them to be based on golang testing, and thus can be brought back. I've left (in tests/) the ones that are not production (bls128381), require non-standard imports (secp requires btcec, bn256 requires gnark/google/cloudflare deps).

This PR also adds a fuzzer for precompiled contracts, because why not.

This PR utilizes a newly rewritten replacement for go-118-fuzz-build, namely gofuzz-shim, which utilises the inputs from the fuzzing engine better.
2023-11-14 14:34:29 +01:00
Martin Holst Swende
c1d5a012ea
core/state, tests: fix memory leak via fastcache (#28387)
This change fixes a memory leak, when running either state-tests or blockchain-tests, we allocate a `1MB` fastcache during snapshot generation. `fastcache` is a bit special, and requires a `Reset()` (it has it's own memory allocator). 

The `1MB` was hidden [here](https://github.com/ethereum/go-ethereum/blob/master/tests/state_test_util.go#L333) and [here](https://github.com/ethereum/go-ethereum/blob/master/tests/block_test_util.go#L146) respectively.
2023-10-20 13:35:49 +02:00
Marius van der Wijden
d10a2f6ab7
tests/fuzzers: update fuzzers to be based on go-native fuzzing (#28352)
This change modifies the fuzzers to use the native golang fuzzing framework instead of go-fuzz
2023-10-18 15:01:16 +02:00
Marius van der Wijden
d782dc2341
tests/fuzzers/les: fix crash in fuzzer (#28362) 2023-10-17 15:20:22 +02:00
rjl493456442
1b1611b8d0
core, trie, eth: refactor stacktrie constructor (#28350)
This change enhances the stacktrie constructor by introducing an option struct. It also simplifies the `Hash` and `Commit` operations, getting rid of the special handling round root node.
2023-10-17 14:09:25 +02:00
Martin Holst Swende
4632b7b31e
tests: update execution-spec-tests to 1.0.5 (#28337)
Updates execution-spec-tests to 1.0.5: https://github.com/ethereum/execution-spec-tests/releases/tag/v1.0.5, switching to develop which contains Cancun tests (which are also enabled in this change).
2023-10-16 20:10:05 +02:00
Martin Holst Swende
f62c58f8de
trie: make rhs-proof align with last key in range proofs (#28311)
During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.

Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.

So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.

On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).

Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
2023-10-13 16:05:29 +02:00
Martin Holst Swende
8976a0c97a
trie: remove owner and binary marshaling from stacktrie (#28291)
This change
  - Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
  - Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
2023-10-11 06:12:45 +02:00
Péter Szilágyi
8afbcf4713
eth: enforce announcement metadatas and drop peers violating the protocol (#28261)
* eth: enforce announcement metadatas and drop peers violating the protocol

* eth/fetcher: relax eth/68 validation a bit for flakey clients

* tests/fuzzers/txfetcher: pull in suggestion from Marius

* eth/fetcher: add tests for peer dropping

* eth/fetcher: linter linter linter linter linter
2023-10-10 11:35:51 +03:00
Martin Holst Swende
b8adb4cb0c
tests: use 'sender' in state tests if present (#28023)
A while back, statetests started coming with sender baked in, which at least 
evmone makes use of. Let's make use of that too, and save some cycles.
2023-08-29 04:36:10 +02:00
Marius van der Wijden
f174ddba7a
build, tests: add execution-spec-tests (#26985)
This makes it possible to run the execution-spec-tests (a.k.a. pyspec) in CI.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-26 15:42:27 +02:00
Felix Lange
2a6beb6a39
core/types: support for optional blob sidecar in BlobTx (#27841)
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.

However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:

- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
  - Note there is a special case here: when transactions are re-added because of a chain
    reorg, we cannot use the transactions gathered from the old chain blocks as-is,
    because they will be missing their blobs. This was previously handled by storing the
    blobs into the 'blobpool limbo'. The code has now changed to store the full
    transaction in the limbo instead, but it might be confusing for code readers why we're
    not simply adding the types.Transaction we already have.

Code changes summary:

- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
2023-08-14 10:13:34 +02:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
rjl493456442
503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Péter Szilágyi
57268f7e6c
all: rename dataGas to blobGas (#27789) 2023-07-27 16:53:28 +03:00
rjl493456442
88f3d61468
all: expose block number information to statedb (#27753)
* core/state: clean up

* all: add block number infomration to statedb

* core, trie: rename blockNumber to block
2023-07-24 13:22:09 +03:00
Martin Holst Swende
b058cf454b
core, tests: EIP-4844 transaction processing logic (#27721)
This updates the reference tests to the latest version and also adds logic
to process EIP-4844 blob transactions into the state transition. We are now
passing most Cancun fork tests.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-07-15 23:27:36 +02:00
jwasinger
d233b6b23a
core: replace instances of 'suicide' with 'selfdestruct' to improve code consistency. (#27716)
---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
2023-07-15 10:35:30 -04:00
rjl493456442
4b06e4f25e
core/state: value diff tracking in StateDB (#27349)
This change makes the StateDB track the state key value diff of a block transition.
We already tracked current account and storage values for the purpose of updating
the state snapshot. With this PR, we now also track the original (pre-transition) values
of accounts and storage slots.
2023-07-11 15:43:23 +02:00
Martin Holst Swende
af8b138c1a
core/vm: define cancun + enable 1153 (tstore/tload) in cancun (#27663) 2023-07-10 14:40:31 -04:00
rjl493456442
59f7b289c3
cmd, core, eth, graphql, trie: no persisted clean trie cache file (#27525)
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.
2023-07-04 10:21:06 +03:00
Guillaume Ballet
8bbb16b70e
core/state, light, les: make signature of ContractCode hash-independent (#27209)
* core/state, light, les: make signature of ContractCode hash-independent

* push current state for feedback

* les: fix unit test

* core, les, light: fix les unittests

* core/state, trie, les, light: fix state iterator

* core, les: address comments

* les: fix lint

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-06-28 12:11:02 +03:00
Guillaume Ballet
c7b099b2ea
trie, core/state: revert error removal in (*state.Trie).Commit (#27544)
* trie, core/state: revert error removal in (*state.Trie).Commit

* Gary's nitpick :)

Co-Authored-By:  rjl493456442 <garyrong0905@gmail.com>

---------

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2023-06-27 15:36:38 +03:00
rjl493456442
6d2aeb43d5
cmd, core/state, eth, tests, trie: improve state reader (#27428)
The state availability is checked during the creation of a state reader.

-    In hash-based database, if the specified root node does not exist on disk disk, then
    the state reader won't be created and an error will be returned.

-    In path-based database, if the specified state layer is not available, then the
    state reader won't be created and an error will be returned.

This change also contains a stricter semantics regarding the `Commit` operation: once it has been performed, the trie is no longer usable, and certain operations will return an error.
2023-06-20 15:31:45 -04:00
rjl493456442
ceca4578ca
trie: remove parameter 'fromLevel' in Prove (#27512)
This removes the feature where top nodes of the proof can be elided.
It was intended to be used by the LES server, to save bandwidth 
when the client had already fetched parts of the state and only needed
some extra nodes to complete the proof. Alas, it never got implemented
in the client.
2023-06-19 16:28:40 +02:00
Dan Laine
50ecb16de0
tests, trie: use slices package for sorting (#27496)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-19 11:41:31 +02:00
Péter Szilágyi
d40a255e97
all: move main transaction pool into a subpool (#27463)
* all: move main transaction pool into a subpool

* go.mod: remove superfluous updates

* core/txpool: review fixes, handle txs rejected by all subpools

* core/txpool: typos
2023-06-16 15:29:40 +03:00
Péter Szilágyi
950d5643b1
core/txpool: make transaction validation reusable across packages (pools) (#27429)
* core/txpool: abstraction prep work for secondary pools (blob pool)

* core/txpool: leave subpool concepts to a followup pr

* les: fix tests using hard coded errors

* core/txpool: use bitmaps instead of maps for tx type filtering
2023-06-06 12:53:29 +03:00
Martin Holst Swende
8eb0c2de76
cmd/evm: make evm blocktest output logs if so instructed (#27396)
* cmd/evm: make evm blocktest output logs if so instructed

* Apply suggestions from code review

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2023-06-06 04:27:40 -04:00
Delweng
b21ba668e6
internal,tests: replace noarg fmt.Errorf with errors.New (#27335)
* internal: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* tests: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* tests: go autoimport

Signed-off-by: jsvisa <delweng@gmail.com>

* tests: go autoimport

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-25 02:54:28 -04:00
Marius van der Wijden
d46f69dc7a
tests/fuzzers/bn256: add PairingCheck fuzzer (#27252)
* tests/fuzzers/bn256: scale gnark result by constant

* tests/fuzzers/bn256: scale gnark result by constant
2023-05-16 07:27:54 -04:00
rjl493456442
a14301823e
all: new empty trie with types.EmptyRootHash instead of null (#27230) 2023-05-11 10:19:42 +03:00
rjl493456442
5021d36d35
all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
Péter Szilágyi
dde2da0efb
all: remove ethash pow, only retain shims needed for consensus and tests (#27178)
* all: remove ethash pow, only retain shims needed for consensus and tests

* all: thank you linter

* all: disallow launching Geth in legacy PoW mode

* cmd/env/internal/t8ntool: remove dangling ethash flag
2023-05-03 12:58:39 +03:00
rjl493456442
99f81d2724
all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Delweng
5e4d726e2a
params: remove EIP150Hash from chainconfig (#27087)
The EIP150Hash was an idea where, after the fork, we hardcoded the forked hash as an extra defensive mechanism. It wasn't really used, since forks weren't contentious and for all the various testnets and private networks it's been a hassle to have around. 

This change removes that config field. 

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-04-18 03:49:09 -04:00
Martin Holst Swende
ab1a404b01
all: remove debug-field from vm config (#27048)
This PR removes the Debug field from vmconfig, making it so that if a tracer is set, debug=true is implied.

---------

Co-authored-by: 0xTylerHolmes <tyler@ethereum.org>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
2023-04-04 09:50:13 -04:00
Martin Holst Swende
a25dd8064e
test/fuzzers: fuzz rlp handling of big.Int and uint256.Int (#26917)
test/fuzzers: fuzz rlp handling of big.Lnt and uint256.Int
2023-04-03 05:42:34 -04:00
lightclient
37ecff0967
cmd/evm, tests: record preimages if dump is expected (#26955)
With #25287 we made it so that preimages were not recorded by default. This had the side effect that the evm command is no longer able to dump state since it does a preimage lookup to determine the address represented by a key.

This change enables the recording of preimages when the dump command is given.
2023-03-23 05:15:40 -04:00
Guruprasad Kamath
789de23d16
tests: define MuirGlacier fork (#26856)
add muir glacier to t8n
2023-03-10 15:47:05 -05:00
Roberto Bayardo
67ac5f0ae7
core, core/types: plain Message struct (#25977)
Here, the core.Message interface turns into a plain struct and
types.Message gets removed.

This is a breaking change to packages core and core/types. While we do
not promise API stability for package core, we do for core/types. An
exception can be made for types.Message, since it doesn't have any
purpose apart from invoking the state transition in package core.
types.Message was also marked deprecated by the same commit it
got added in, 4dca5d4db7 (November 2016).

The core.Message interface was added in December 2014, in commit
db494170dc, for the purpose of 'testing' state transitions. It's the
same change that made transaction struct fields private. Before that,
the state transition used *types.Transaction directly.

Over time, multiple implementations of the interface accrued across
different packages, since constructing a Message is required whenever
one wants to invoke the state transition. These implementations all
looked very similar, a struct with private fields exposing the fields
as accessor methods.

By changing Message into a struct with public fields we can remove all
these useless interface implementations. It will also hopefully
simplify future changes to the type with less updates to apply across
all of go-ethereum when a field is added to Message.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-09 14:19:12 +01:00
Martin Holst Swende
b80f05bde2
core/vm: use golang native big.Int (#26834)
reverts #26021, to use the upstream bigint instead.
2023-03-08 13:12:53 -05:00
Péter Szilágyi
cd31f2dee2
all: change chain head markers from block to header (#26777) 2023-03-02 08:29:15 +02:00
Martin Holst Swende
03585ed7a9
tests/fuzzers: supply gnark multiexp config, fixes #26669 (#26670)
This change fixes a fuzzer which broke when we updated the gnark dependency earlier.
2023-02-13 08:14:34 -05:00
Martin Holst Swende
22c3ad1d12
core/state, trie: remove unused error-return from trie Commit operation (#26641) 2023-02-09 08:56:59 -05:00
Péter Szilágyi
095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
rjl493456442
9842301376
all: remove database commit callback, rework noderesolver (#26637)
This change ports some changes from the main PBSS PR:

  - get rid of callback function in `trie.Database.Commit` which is not required anymore
  - rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
  - some other shallow changes in tests and typo-fixes
2023-02-08 06:14:34 -05:00
Martin Holst Swende
91cb6f863a
tests: update (#26548)
This updates the reference tests to the latest version.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-06 14:52:51 -05:00
Martin Holst Swende
8860b39754
all: prepare for path-based trie storage (#26603)
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.

This PR is intended to be a no-functionality-change prep PR for #25963 .

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-02-06 10:28:40 -05:00