Commit Graph

449 Commits

Author SHA1 Message Date
gary rong
f566dd305e
all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-02-08 13:16:30 +02:00
Marius van der Wijden
54735a6723
fuzzers: added consensys/gurvy library to bn256 differential fuzzer (#21812)
This pr adds consensys' gurvy bn256 variant into the code for differential fuzzing.
2021-02-03 15:04:28 +01:00
Sina Mahmoodi
eb21c652c0
cmd,core,eth,params,tests: define yolov3 + enable EIP-2565 (#22213)
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3. 
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
2021-01-28 21:19:07 +01:00
Marius van der Wijden
7202b410b0
tests/fuzzers/abi: fixed one-off panic with int.Min64 value (#22233)
* tests/fuzzers/abi: fixed one-off panic with int.Min64 value

* tests/fuzzers/abi: fixed one-off panic with int.Min64 value
2021-01-25 21:40:14 +01:00
Marius van der Wijden
398182284c
tests/fuzzers/abi: better test generation (#22158)
* tests/fuzzers/abi: better test generation

* tests/fuzzers/abi: fixed packing issue

* oss-fuzz: enable abi fuzzer
2021-01-18 14:33:15 +01:00
Martin Holst Swende
c94081774f
tests: update the reference tests (#22009) 2021-01-13 11:29:28 +01:00
Martin Holst Swende
b13e9c4e3d
tests/fuzzers: fix false positive in bitutil fuzzer (#22076) 2020-12-27 21:58:39 +01:00
Martin Holst Swende
b9012a039b
common,crypto: move fuzzers out of core (#22029)
* common,crypto: move fuzzers out of core

* fuzzers: move vm fuzzer out from core

* fuzzing: rework cover package logic

* fuzzers: lint
2020-12-23 17:44:45 +01:00
Péter Szilágyi
017831dd5b
core, eth: split eth package, implement snap protocol (#21482)
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default. 

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-12-14 10:27:15 +01:00
Martin Holst Swende
efe6dd2904
consensus/ethash: implement faster difficulty calculators (#21976)
This PR adds re-written difficulty calculators, which are based on uint256. It also adds a fuzzer + oss-fuzz integration for the new fuzzer. It does differential fuzzing between the new and old calculators.

Note: this PR does not actually enable the new calculators.
2020-12-11 11:06:44 +01:00
Martin Holst Swende
6104ab6b6d
tests/fuzzers/bls1381: add bls fuzzer (#21796)
* added bls fuzzer

* crypto/bls12381: revert bls-changes, fixup fuzzer tests

* fuzzers: split bls fuzzing into 8 different units

* fuzzers/bls: remove (now stale) corpus

* crypto/bls12381: added blsfuzz corpus

* fuzzers/bls12381: fix the bls corpus

* fuzzers: fix oss-fuzz script

* tests/fuzzers: fixups on bls corpus

* test/fuzzers: remove leftover corpus

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-11-23 15:49:16 +01:00
Marius van der Wijden
2045a2bba3
core, all: split vm.Context into BlockContext and TxContext (#21672)
* all: core: split vm.Config into BlockConfig and TxConfig

* core: core/vm: reset EVM between tx in block instead of creating new

* core/vm: added docs
2020-11-13 13:42:19 +01:00
Martin Holst Swende
0703c91fba
tests/fuzzers: improve the fuzzers (#21829)
* tests/fuzzers, common/bitutil: make fuzzers use correct returnvalues + remove output

* tests/fuzzers/stacktrie: fix duplicate-key insertion in stacktrie (false positive)

* tests/fuzzers/stacktrie: fix compilation error

* tests/fuzzers: linter nits
2020-11-13 12:36:38 +01:00
Martin Holst Swende
81678971db
trie, tests/fuzzers: implement a stacktrie fuzzer + stacktrie fixes (#21799)
* trie: fix error in stacktrie not committing small roots

* fuzzers: make trie-fuzzer use correct returnvalues

* trie: improved tests

* tests/fuzzers: fuzzer for stacktrie vs regular trie

* test/fuzzers: make stacktrie fuzzer use 32-byte keys

* trie: fix error in stacktrie with small nodes

* trie: add (skipped) testcase for stacktrie

* tests/fuzzers: address review comments for stacktrie fuzzer

* trie: fix docs in stacktrie
2020-11-09 15:08:12 +01:00
gary rong
b63e3c37a6
core: improve snapshot journal recovery (#21594)
* core/state/snapshot: introduce snapshot journal version

* core: update the disk layer in an atomic way

* core: persist the disk layer generator periodically

* core/state/snapshot: improve logging

* core/state/snapshot: forcibly ensure the legacy snapshot is matched

* core/state/snapshot: add debug logs

* core, tests: fix tests and special recovery case

* core: polish

* core: add more blockchain tests for snapshot recovery

* core/state: fix comment

* core: add recovery flag for snapshot

* core: add restart after start-after-crash tests

* core/rawdb: fix imports

* core: fix tests

* core: remove log

* core/state/snapshot: fix snapshot

* core: avoid callbacks in SetHead

* core: fix setHead cornercase where the threshold root has state

* core: small docs for the test cases

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-10-29 21:01:58 +02:00
Martin Holst Swende
6487c002f6
all: implement EIP-2929 (gas cost increases for state access opcodes) + yolo-v2 (#21509)
* core/vm, core/state: implement EIP-2929 + YOLOv2

* core/state, core/vm: fix some review concerns

* core/state, core/vm: address review concerns

* core/vm: address review concerns

* core/vm: better documentation

* core/vm: unify sload cost as fully dynamic

* core/vm: fix typo

* core/vm/runtime: fix compilation flaw

* core/vm/runtime: fix renaming-err leftovers

* core/vm: renaming

* params/config: use correct yolov2 chainid for config

* core, params: use a proper new genesis for yolov2

* core/state/tests: golinter nitpicks
2020-10-23 08:26:57 +02:00
Marius van der Wijden
420b78659b
accounts/abi: ABI explicit difference between Unpack and UnpackIntoInterface (#21091)
* accounts/abi: refactored abi.Unpack

* accounts/abi/bind: fixed error

* accounts/abi/bind: modified template

* accounts/abi/bind: added ToStruct for conversion

* accounts/abi: reenabled tests

* accounts/abi: fixed tests

* accounts/abi: fixed tests for packing/unpacking

* accounts/abi: fixed tests

* accounts/abi: added more logic to ToStruct

* accounts/abi/bind: fixed template

* accounts/abi/bind: fixed ToStruct conversion

* accounts/abi/: removed unused code

* accounts/abi: updated template

* accounts/abi: refactored unused code

* contracts/checkpointoracle: updated contracts to sol ^0.6.0

* accounts/abi: refactored reflection logic

* accounts/abi: less code duplication in Unpack*

* accounts/abi: fixed rebasing bug

* fix a few typos in comments

* rebase on master

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2020-09-28 14:10:26 +02:00
Marius van der Wijden
89884dc353
tests/fuzzers/abi: add fuzzer for fuzzing package accounts/abi (#21217)
* tests/fuzzers/abi: added abi fuzzer

* accounts/abi: fixed issues found by fuzzing

* tests/fuzzers/abi: update fuzzers, added repro test

* tests/fuzzers/abi: renamed abi_fuzzer to abifuzzer

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: updated abi fuzzer

* accounts/abi: minor style fix

* go.mod: added go-fuzz dependency

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: comment out false positives
2020-09-16 13:15:22 +02:00
Guillaume Ballet
d54f2f2e5e
whisper: remove whisper (#21487)
* whisper: remove whisper

* Update cmd/geth/config.go

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* cmd/geth: warn on enabling whisper + remove more whisper deps

* mobile: remove all whisper references

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-09-08 11:47:48 +03:00
Martin Holst Swende
c28fd9c079
tests: add Berlin-definition identical to YOLOv1 (#21435) 2020-08-10 21:06:14 +02:00
Martin Holst Swende
e376d2fb31
cmd/evm: add state transition tool for testing (#20958)
This PR implements the EVM state transition tool, which is intended
to be the replacement for our retesteth client implementation.
Documentation is present in the cmd/evm/README.md file.

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-06-30 10:12:51 +02:00
Martin Holst Swende
890757f03a
cmd, core, params: inital support for yolo-v1 testnet (#21154)
* core,params,puppeth: inital support for yolo-v1 testnet

* cmd/geth, core: add yolov1 console flag

* cmd, core, params: YoloV1 bakein fixups

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-06-03 12:05:15 +03:00
Martin Michlmayr
4f2784b38f
all: fix typos in comments (#21118) 2020-05-25 10:21:28 +02:00
Martin Holst Swende
4535230059
cmd, core, eth: background transaction indexing (#20302)
* cmd, core, eth: init tx lookup in background

* core/rawdb: tiny log fixes to make it clearer what's happening

* core, eth: fix rebase errors

* core/rawdb: make reindexing less generic, but more optimal

* rlp: implement rlp list iterator

* core/rawdb: new implementation of tx indexing/unindex using generic tx iterator and hashing rlp-data

* core/rawdb, cmd/utils: fix review concerns

* cmd/utils: fix merge issue

* core/rawdb: add some log formatting polishes

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-05-11 18:58:43 +03:00
Péter Szilágyi
4515772993
tests: skip consensus test using 1GB RAM 2020-05-05 12:27:09 +03:00
Péter Szilágyi
65cd28aa0e
tests: cleanup snapshot generator goroutine leak 2020-05-04 12:10:02 +03:00
gary rong
26d271dfbb
core/state/snapshot: implement storage iterator (#20971)
* core/state/snapshot: implement storage iterator

* core/state/snapshot, tests: implement helper function

* core/state/snapshot: fix storage issue

If an account is deleted in the tx_1 but recreated in the tx_2,
the it can happen that in this diff layer, both destructedSet
and storageData records this account. In this case, the storage
iterator should be able to iterate the slots belong to new account
but disable further iteration in deeper layers(belong to old account)

* core/state/snapshot: address peter and martin's comment

* core/state: address comments

* core/state/snapshot: fix test
2020-04-29 12:53:08 +03:00
gary rong
b9df7ecdc3
all: seperate consensus error and evm internal error (#20830)
* all: seperate consensus error and evm internal error

There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.

This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.

And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails

* all: polish

* accounts/abi/bind/backends: add tests

* accounts/abi/bind/backends, internal: cleanup error message

* all: address comments

* core: fix lint

* accounts, core, eth, internal: address comments

* accounts, internal: resolve revert reason if possible

* accounts, internal: address comments
2020-04-22 11:25:36 +03:00
Raw Pong Ghmoa
15540ae992
cmd: deprecate --testnet, use named networks instead (#20852)
* cmd/utils: make goerli the default testnet

* cmd/geth: explicitly rename testnet to ropsten

* core: explicitly rename testnet to ropsten

* params: explicitly rename testnet to ropsten

* cmd: explicitly rename testnet to ropsten

* miner: explicitly rename testnet to ropsten

* mobile: allow for returning the goerli spec

* tests: explicitly rename testnet to ropsten

* docs: update readme to reflect changes to the default testnet

* mobile: allow for configuring goerli and rinkeby nodes

* cmd/geth: revert --testnet back to ropsten and mark as legacy

* cmd/util: mark --testnet flag as deprecated

* docs: update readme to properly reflect the 3 testnets

* cmd/utils: add an explicit deprecation warning on startup

* cmd/utils: swap goerli and ropsten in usage

* cmd/geth: swap goerli and ropsten in usage

* cmd/geth: if running a known preset, log it for convenience

* docs: improve readme on usage of ropsten's testnet datadir

* cmd/utils: check if legacy `testnet` datadir exists for ropsten

* cmd/geth: check for legacy testnet path in console command

* cmd/geth: use switch statement for complex conditions in main

* cmd/geth: move known preset log statement to the very top

* cmd/utils: create new ropsten configurations in the ropsten datadir

* cmd/utils: makedatadir should check for existing testnet dir

* cmd/geth: add legacy testnet flag to the copy db command

* cmd/geth: add legacy testnet flag to the inspect command
2020-04-09 12:09:58 +03:00
Martin Holst Swende
76eed9e50d
snapshotter/tests: verify snapdb post-state against trie (#20812)
* core/state/snapshot: basic trie-to-hash implementation

* tests: validate snapshot after test

* core/state/snapshot: fix review concerns
2020-03-31 10:25:41 +02:00
Péter Szilágyi
613af7ceea
Merge pull request #20152 from karalabe/snapshot-5
Dynamic state snapshots
2020-03-23 12:57:31 +02:00
meowsbits
0bdb21f0cb
tests: update tests/testdata@develop, include EIP2384 config (#20746)
Includes difficulty tests for EIP2384 aka MuirGlacier.
2020-03-10 10:55:27 +01:00
Péter Szilágyi
6e05ccd845
core/state/snapshot, tests: sync snap gen + snaps in consensus tests 2020-03-03 09:17:13 +02:00
Péter Szilágyi
542df8898e
core: initial version of state snapshots 2020-02-25 12:51:04 +02:00
Péter Szilágyi
9938d954c8
eth: rework tx fetcher to use O(1) ops + manage network requests 2020-02-13 15:27:15 +02:00
Gerald Nash
a67fe48b43 Change file extension of the ./tests/fuzzers README (#20474) 2019-12-20 13:19:01 +01:00
Martin Holst Swende
cecc7230c0 tests/fuzzers: fuzzbuzz fuzzers for keystore, rlp, trie, whisper (#19910)
* fuzzers: fuzzers for keystore, rlp, trie, whisper (cred to @guidovranken)

* fuzzers: move fuzzers to testdata

* testdata/fuzzers: documentation

* testdata/fuzzers: corpus for rlp

* tests/fuzzers: fixup
2019-12-10 11:57:37 +02:00
Guillaume Ballet
5d21667587 tests, signer: remove staticcheck warnings (#20364) 2019-11-23 12:51:37 +01:00
Péter Szilágyi
c5b46a79c1
Merge pull request #20338 from etclabscore/feat/statetests-dedupe-walk-refactor
tests: refactor TestState to dedupe walk callback
2019-11-20 16:24:03 +02:00
meowsbits
b4bc3b3c35 tests: enable TransactionTests Istanbul case (#20337) 2019-11-20 15:08:07 +01:00
meows
b8ced9e00b
tests: refactor TestState to dedupe walk callback
Minor refactoring.
2019-11-20 07:54:18 -05:00
Martin Holst Swende
7f5f62aaa0 tests: update test suite for istanbul (#20082)
* update tests for istanbul

* tests: updated blockchaintests, see https://github.com/ethereum/tests/issues/637

* tests: update again, hopefully fixed this time

* tests: skip time consuming, run legacy tests

* tests: update again

* build: disable long-running tests on travis

* tests: fix formatting nits

* tests: I hate github's editor
2019-10-02 11:33:51 +03:00
Lucas Hendren
62391ddbeb tests/solidity: add contract to test every opcode (#19283)
Fixes #18210
2019-09-26 10:30:33 +02:00
Felix Lange
39b0b1a1a6
all: make unit tests work with Go 1.13 (#20053)
Most of these changes are related to the Go 1.13 changes to test binary
flag handling. 

* cmd/geth: make attach tests more reliable

This makes the test wait for the endpoint to come up by polling
it instead of waiting for two seconds.

* tests: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* crypto/ecies: remove useless -dump flag in tests

* p2p/simulations: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* build: remove workaround for ./... vendor matching

This workaround was necessary for Go 1.8. The Go 1.9 release changed
the expansion rules to exclude vendored packages.

* Makefile: use relative path for GOBIN

This makes the "Run ./build/bin/..." line look nicer.

* les: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
2019-09-11 14:41:22 +02:00
Martin Holst Swende
91b7349509 tests: expose internal RunNoVerify method (#20051) 2019-09-11 13:46:14 +02:00
Martin Holst Swende
4aeeddc658
tests: implement Istanbul support 2019-08-23 10:01:09 +02:00
Martin Holst Swende
dbb03fe989 tests: update from ethereum/tests (#19945) 2019-08-15 14:59:46 +02:00
gary rong
c2c4c9f1e5 core, light, params: implement eip2028 (#19931)
* core, light, params: implement eip2028

* core, light: address comments

* core: address comments

* tests: disable Istanbul tx tests (until updated)

* core: address comment
2019-08-14 15:53:21 +03:00
Martin Holst Swende
3e993ff64a Eip 1884 v3 (#19743)
* core/vm, tests: implement EIP 1884, add support for feature-tests

* core/vm: 1884-changes to extcodehash, move selfbalance opcode

* tests: fix statetests

* core/vm: move constants, address review concerns

* core/vm: word formatting

Co-Authored-By: Péter Szilágyi <peterke@gmail.com>
2019-08-08 12:07:23 +03:00
Martin Holst Swende
2cd6059e51 tests: make transaction tests run again, fix #19033 (#19529)
* tests: make transaction tests run again, fix #19033

* tests: refactor transaction tests
2019-05-21 14:58:06 +03:00