Commit Graph

53 Commits

Author SHA1 Message Date
rjl493456442
bff84a99fe
cmd, core, eth, les, light: track deleted nodes (#25757)
* cmd, core, eth, les, light: track deleted nodes

* trie: add docs

* trie: address comments

* cmd, core, eth, les, light, trie: trie id

* trie: add tests

* trie, core: updates

* trie: fix imports

* trie: add utility print-method for nodeset

* trie: import err

* trie: fix go vet warnings

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-27 10:01:02 +02:00
Felix Lange
b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
rjl493456442
dea1fb3cfc
all: cleanup tests (#25641)
Follow-up to PR #25523 to cleanup all relevant tests.
2022-09-07 20:21:59 +02:00
rjl493456442
d10c280309
all: move genesis initialization to blockchain (#25523)
* all: move genesis initialization to blockchain

* core: add one more check

* core: fix tests
2022-08-30 18:22:28 +02:00
rjl493456442
8b53b92eb4
core, trie: rework trie committer (#25320)
* all: rework trie and trie committer

* all: get rid of internal cache in trie

* all: fixes

* trie: polish

* core, trie: address comments

* trie: fix imports

* core/state: address comments

* core/state/snapshot: polish

* trie: remove unused code

* trie: update tests

* trie: don't set db as nil

* trie: address comments

* trie: unskip test
2022-08-04 11:03:20 +03:00
Delweng
b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
Seungbae.yu
2697e44d81
all: change format 0x%x to %#x (#25221) 2022-07-04 11:03:32 +03:00
Martin Holst Swende
01e5e9c2c3
tests/fuzzers/rlp: avoid very large input (#25109)
The oss-fuzz engine crashes due to stack overflow decoding a large nested
structure into a interface{}. This PR limits the size of the input data, so
should avoid such crashes.
2022-06-17 00:44:30 +02:00
Martin Holst Swende
a907d7e81a
all: more linters (#24783)
This enables the following linters

- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec

WIth a few exceptions.

- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
2022-06-13 16:24:45 +02:00
Marius van der Wijden
5e8fa1da70
tests/fuzzers/bls12381: fix blst pairing (#25037)
* tests/fuzzers/bls12381: fix blst pairing

* tests/fuzzers/bls12381: only build on gofuzz

* tests/fuzzers/bls12381: remove unused code

* tests/fuzzers/bls12381: remove unused code
2022-06-07 09:06:34 +02:00
rjl493456442
22defa5af7
all: introduce trie owner notion (#24750)
* cmd, core/state, light, trie, eth: add trie owner notion

* all: refactor

* tests: fix goimports

* core/state/snapshot: fix ineffasigns

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-06 17:14:55 +02:00
Marius van der Wijden
d6b55749e6
tests/fuzzers/bls12381: fix blst deserializing (#25036)
* tests/fuzzers/bls12381: fix blst deserializing

* tests/fuzzers/bls12381: fix blst deserializing
2022-06-06 17:01:59 +02:00
Marius van der Wijden
3c6d6f7ee8
tests/fuzzers/bls12381: Add BLST to fuzzing support (#24249)
* tests/fuzzers/bls12381: added blst library

* go.mod: added blst dependency

* tests/fuzzers/bls12381: stuff

* tests/fuzzers/bls12381: added blst to pairing fuzzer
2022-06-02 13:13:28 +02:00
Felix Lange
9244d5cd61
all: update license headers and AUTHORS from git history (#24947) 2022-05-24 20:39:40 +02:00
Håvard Anda Estensen
07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
John Difool
a52bcccfe1
consensus/ethash: fix typos in var names (#24745) 2022-04-26 10:16:57 +02:00
rjl493456442
da16d089c0
trie, les, tests, core: implement trie tracer (#24403)
Trie tracer is an auxiliary tool to capture all deleted nodes
which can't be captured by trie.Committer. The deleted nodes
can be removed from the disk later.
2022-03-31 09:28:32 +02:00
Jonathan Chappelow
830231c1c4
crypto: use btcec/v2 for no-cgo (#24533)
This updates the no-cgo implementations in the crypto package to use
the github.com/btcsuite/btcd/btcec/v2 module instead of the older btcec
package that was part of the main github.com/btcsuite/btcd module.

name                   old time/op  new time/op  delta
EcrecoverSignature-32   198µs ± 0%   144µs ± 0%  -27.11%
VerifySignature-32      177µs ± 0%   128µs ± 0%  -27.44%
DecompressPubkey-32    20.9µs ± 0%  10.1µs ± 0%  -51.51%

Use (*ModNScalar).IsOverHalfOrder instead of math/big.Int when checking
for malleable signatures.
2022-03-16 14:23:14 +01:00
rjl493456442
7a80cf6543
les, tests: fix vflux fuzzer by removing unnecessary panic (#24537) 2022-03-16 11:13:10 +01:00
rjl493456442
8c8a9e5ca1
core, ethdb, tests, trie: introduce database snapshot (#24486) 2022-03-10 09:35:22 +01:00
rjl493456442
4d086430bd
core, ethdb, tests, trie: implement NewBatchWithSize API for batcher (#24392)
This PR adds an addtional API called `NewBatchWithSize` for db
batcher. It turns out that leveldb batch memory allocation is
super inefficient. The main reason is the allocation step of
leveldb Batch is too small when the batch size is large. It can
take a few second to build a leveldb batch with 100MB size.

Luckily, leveldb also offers another API called MakeBatch which can
pre-allocate the memory area. So if the approximate size of batch is
known in advance, this API can be used in this case.

It's needed in new state scheme PR which needs to commit a batch of
trie nodes in a single batch. Implement the feature in a seperate PR.
2022-02-15 15:15:13 +02:00
Martin Holst Swende
1988b47e02
tests/fuzzzers. eth/protocols/snap: add snap protocol fuzzers (#23957) 2021-12-01 10:17:18 +01:00
Felix Lange
8a134014b4
all: add go:build lines (#23468)
Generated by go1.17 fmt ./...
2021-08-25 18:46:29 +02:00
gary rong
a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
baptiste-b-pegasys
5cee33eb72
tests/fuzzers: fix go vet warning about ReadByte (#23380) 2021-08-24 14:27:15 +02:00
Marius van der Wijden
0703ef62d3
crypto/secp256k1: fix undefined behavior in BitCurve.Add (#22621)
This commit changes the behavior of BitCurve.Add to be more inline
with btcd. It fixes two different bugs:

1) When adding a point at infinity to another point, the other point
   should be returned. While this is undefined behavior, it is better
   to be more inline with the go standard library.
   Thus (0,0) + (a, b) = (a,b)

2) Adding the same point to itself produced the point at infinity.
   This is incorrect, now doubleJacobian is used to correctly calculate it.
   Thus (a,b) + (a,b) == 2* (a,b) and not (0,0) anymore.

The change also adds a differential fuzzer for Add, testing it against btcd.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-27 13:30:25 +02:00
ucwong
017cf71fbd
rlp, tests/fuzzers/bls12381: gofmt (#22937) 2021-05-25 10:14:39 +02:00
Péter Szilágyi
fae165a5de
core, eth, ethdb, trie: simplify range proofs 2021-04-29 10:59:08 +03:00
Martin Holst Swende
a81cf0d2b3
trie: remove redundant returns + use stacktrie where applicable (#22760)
* trie: add benchmark for proofless range

* trie: remove unused returns + use stacktrie
2021-04-28 22:47:48 +03:00
gary rong
6d7c9566df
les, tests: fix les clientpool (#22756)
* les, tests: fix les clientpool

* tests: disable debug mode

* les: polish code
2021-04-28 14:18:25 +02:00
Gautam Botrel
9e5bb84c0e
tests/fuzzers: crypto/bn256 and crypto/bls12381 tests against gnark-crypto (#22755)
Add more cross-fuzzers to fuzz bls with gnark versus geth's own bls12-381 library
2021-04-28 12:04:25 +02:00
Péter Szilágyi
caea6c4661
eth/protocols/snap: generate storage trie from full dirty snap data (#22668)
* eth/protocols/snap: generate storage trie from full dirty snap data

* eth/protocols/snap: get rid of some more dead code

* eth/protocols/snap: less frequent logs, also log during trie generation

* eth/protocols/snap: implement dirty account range stack-hashing

* eth/protocols/snap: don't loop on account trie generation

* eth/protocols/snap: fix account format in trie

* core, eth, ethdb: glue snap packets together, but not chunks

* eth/protocols/snap: print completion log for snap phase

* eth/protocols/snap: extended tests

* eth/protocols/snap: make testcase pass

* eth/protocols/snap: fix account stacktrie commit without defer

* ethdb: fix key counts on reset

* eth/protocols: fix typos

* eth/protocols/snap: make better use of delivered data (#44)

* eth/protocols/snap: make better use of delivered data

* squashme

* eth/protocols/snap: reduce chunking

* squashme

* eth/protocols/snap: reduce chunking further

* eth/protocols/snap: break out hash range calculations

* eth/protocols/snap: use sort.Search instead of looping

* eth/protocols/snap: prevent crash on storage response with no keys

* eth/protocols/snap: nitpicks all around

* eth/protocols/snap: clear heal need on 1-chunk storage completion

* eth/protocols/snap: fix range chunker, add tests

Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* trie: fix test API error

* eth/protocols/snap: fix some further liter issues

* eth/protocols/snap: fix accidental batch reuse

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-27 17:19:59 +03:00
Felföldi Zsolt
2d89fe0883
les: move client pool to les/vflux/server (#22495)
* les: move client pool to les/vflux/server

* les/vflux/server: un-expose NodeBalance, remove unused fn, fix bugs

* tests/fuzzers/vflux: add ClientPool fuzzer

* les/vflux/server: fixed balance tests

* les: rebase fix

* les/vflux/server: fixed more bugs

* les/vflux/server: unexported NodeStateMachine fields and flags

* les/vflux/server: unexport all internal components and functions

* les/vflux/server: fixed priorityPool test

* les/vflux/server: polish balance

* les/vflux/server: fixed mutex locking error

* les/vflux/server: priorityPool bug fixed

* common/prque: make Prque wrap-around priority handling optional

* les/vflux/server: rename funcs, small optimizations

* les/vflux/server: fixed timeUntil

* les/vflux/server: separated balance.posValue and negValue

* les/vflux/server: polish setup

* les/vflux/server: enforce capacity curve monotonicity

* les/vflux/server: simplified requestCapacity

* les/vflux/server: requestCapacity with target range, no iterations in SetCapacity

* les/vflux/server: minor changes

* les/vflux/server: moved default factors to balanceTracker

* les/vflux/server: set inactiveFlag in priorityPool

* les/vflux/server: moved related metrics to vfs package

* les/vflux/client: make priorityPool temp state logic cleaner

* les/vflux/server: changed log.Crit to log.Error

* add vflux fuzzer to oss-fuzz

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-04-06 20:42:50 +02:00
Martin Holst Swende
bc47993692
tests/fuzzers: fix goroutine leak in les fuzzer (#22455)
The oss-fuzz fuzzer has been reporting some failing testcases for les. They're all spurious, and cannot reliably be reproduced. However, running them showed that there was a goroutine leak: the tests created a lot of new clients, which started an exec queue that was never torn down.

This PR fixes the goroutine leak, and also a log message which was erroneously formatted.
2021-03-16 09:43:33 +01:00
Felföldi Zsolt
8f03e3b107
tests/fuzzers/les: add fuzzer for les server handler (#22282)
* les: refactored server handler

* tests/fuzzers/les: add fuzzer for les server handler

* tests, les: update les fuzzer

tests: update les fuzzer

tests/fuzzer/les: release resources

tests/fuzzer/les: pre-initialize all resources

* les: refactored server handler and fuzzer

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-02-20 10:40:38 +01:00
Marius van der Wijden
54735a6723
fuzzers: added consensys/gurvy library to bn256 differential fuzzer (#21812)
This pr adds consensys' gurvy bn256 variant into the code for differential fuzzing.
2021-02-03 15:04:28 +01:00
Sina Mahmoodi
eb21c652c0
cmd,core,eth,params,tests: define yolov3 + enable EIP-2565 (#22213)
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3. 
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
2021-01-28 21:19:07 +01:00
Marius van der Wijden
7202b410b0
tests/fuzzers/abi: fixed one-off panic with int.Min64 value (#22233)
* tests/fuzzers/abi: fixed one-off panic with int.Min64 value

* tests/fuzzers/abi: fixed one-off panic with int.Min64 value
2021-01-25 21:40:14 +01:00
Marius van der Wijden
398182284c
tests/fuzzers/abi: better test generation (#22158)
* tests/fuzzers/abi: better test generation

* tests/fuzzers/abi: fixed packing issue

* oss-fuzz: enable abi fuzzer
2021-01-18 14:33:15 +01:00
Martin Holst Swende
b13e9c4e3d
tests/fuzzers: fix false positive in bitutil fuzzer (#22076) 2020-12-27 21:58:39 +01:00
Martin Holst Swende
b9012a039b
common,crypto: move fuzzers out of core (#22029)
* common,crypto: move fuzzers out of core

* fuzzers: move vm fuzzer out from core

* fuzzing: rework cover package logic

* fuzzers: lint
2020-12-23 17:44:45 +01:00
Péter Szilágyi
017831dd5b
core, eth: split eth package, implement snap protocol (#21482)
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default. 

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-12-14 10:27:15 +01:00
Martin Holst Swende
efe6dd2904
consensus/ethash: implement faster difficulty calculators (#21976)
This PR adds re-written difficulty calculators, which are based on uint256. It also adds a fuzzer + oss-fuzz integration for the new fuzzer. It does differential fuzzing between the new and old calculators.

Note: this PR does not actually enable the new calculators.
2020-12-11 11:06:44 +01:00
Martin Holst Swende
6104ab6b6d
tests/fuzzers/bls1381: add bls fuzzer (#21796)
* added bls fuzzer

* crypto/bls12381: revert bls-changes, fixup fuzzer tests

* fuzzers: split bls fuzzing into 8 different units

* fuzzers/bls: remove (now stale) corpus

* crypto/bls12381: added blsfuzz corpus

* fuzzers/bls12381: fix the bls corpus

* fuzzers: fix oss-fuzz script

* tests/fuzzers: fixups on bls corpus

* test/fuzzers: remove leftover corpus

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-11-23 15:49:16 +01:00
Martin Holst Swende
0703c91fba
tests/fuzzers: improve the fuzzers (#21829)
* tests/fuzzers, common/bitutil: make fuzzers use correct returnvalues + remove output

* tests/fuzzers/stacktrie: fix duplicate-key insertion in stacktrie (false positive)

* tests/fuzzers/stacktrie: fix compilation error

* tests/fuzzers: linter nits
2020-11-13 12:36:38 +01:00
Martin Holst Swende
81678971db
trie, tests/fuzzers: implement a stacktrie fuzzer + stacktrie fixes (#21799)
* trie: fix error in stacktrie not committing small roots

* fuzzers: make trie-fuzzer use correct returnvalues

* trie: improved tests

* tests/fuzzers: fuzzer for stacktrie vs regular trie

* test/fuzzers: make stacktrie fuzzer use 32-byte keys

* trie: fix error in stacktrie with small nodes

* trie: add (skipped) testcase for stacktrie

* tests/fuzzers: address review comments for stacktrie fuzzer

* trie: fix docs in stacktrie
2020-11-09 15:08:12 +01:00
Marius van der Wijden
420b78659b
accounts/abi: ABI explicit difference between Unpack and UnpackIntoInterface (#21091)
* accounts/abi: refactored abi.Unpack

* accounts/abi/bind: fixed error

* accounts/abi/bind: modified template

* accounts/abi/bind: added ToStruct for conversion

* accounts/abi: reenabled tests

* accounts/abi: fixed tests

* accounts/abi: fixed tests for packing/unpacking

* accounts/abi: fixed tests

* accounts/abi: added more logic to ToStruct

* accounts/abi/bind: fixed template

* accounts/abi/bind: fixed ToStruct conversion

* accounts/abi/: removed unused code

* accounts/abi: updated template

* accounts/abi: refactored unused code

* contracts/checkpointoracle: updated contracts to sol ^0.6.0

* accounts/abi: refactored reflection logic

* accounts/abi: less code duplication in Unpack*

* accounts/abi: fixed rebasing bug

* fix a few typos in comments

* rebase on master

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2020-09-28 14:10:26 +02:00
Marius van der Wijden
89884dc353
tests/fuzzers/abi: add fuzzer for fuzzing package accounts/abi (#21217)
* tests/fuzzers/abi: added abi fuzzer

* accounts/abi: fixed issues found by fuzzing

* tests/fuzzers/abi: update fuzzers, added repro test

* tests/fuzzers/abi: renamed abi_fuzzer to abifuzzer

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: updated abi fuzzer

* accounts/abi: minor style fix

* go.mod: added go-fuzz dependency

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: comment out false positives
2020-09-16 13:15:22 +02:00
Guillaume Ballet
d54f2f2e5e
whisper: remove whisper (#21487)
* whisper: remove whisper

* Update cmd/geth/config.go

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

* cmd/geth: warn on enabling whisper + remove more whisper deps

* mobile: remove all whisper references

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-09-08 11:47:48 +03:00
Martin Michlmayr
4f2784b38f
all: fix typos in comments (#21118) 2020-05-25 10:21:28 +02:00