Commit Graph

1682 Commits

Author SHA1 Message Date
Marius van der Wijden
e6fa102eb0
core, eth, internal, rpc: implement final block (#24282)
* eth: core: implement finalized block

* eth/catalyst: fix final block

* eth/catalyst: update finalized head gauge

* internal/jsre/deps: updated web3.js to allow for finalized block

* eth/catalyst: make sure only one thread can call fcu

* eth/catalyst: nitpicks

* eth/catalyst: use plain mutex

* eth: nitpicks
2022-05-18 17:30:42 +03:00
Håvard Anda Estensen
4b309c7006
all: replace non-trivial uses of package ioutil with os (#24886)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-17 14:27:04 +02:00
Marius van der Wijden
381c66caf0
eth/catalyst: set the correct LatestValidHash (#24855)
* eth/catalyst: set the correct LatestValidHash

* eth/catalyst: core: return LVH during reorg, rework invalid teminal block

* eth/catalyst: nitpicks
2022-05-17 12:32:55 +03:00
rjl493456442
39fb82bcfb
eth: fix flaky test, don't attach empty slots/proofs (#24885)
* eth/protocols/snap: don't include empty snapshot slot slice

This PR fixes the snapshot storage serving handler. In snap protocol
the response is capped by the response size. Server can cutdown the
response if the accumulated byte size exceeds the local hard limit.

It means we can meet a special scenario that there is no storage slot
included for a requested account, but we attach the proof for this
account by mistake.

So in the prover side, when it meets a empty storage response but with
a valid proof proves there are some more slots left in the trie, then
requestor will reject this response and disconnect with server.

In this PR, if there is no storage slot served for the requested account,
then no proof should be attached as well.

* eth/protocols/snap: loosen restrictions for flaky tests

* eth/catalyst: fix flaky test in catalyst
2022-05-17 11:19:51 +03:00
Håvard Anda Estensen
07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
Martin Holst Swende
646503208e
eth/protocols/snap: sort trienode heal requests by path (#24779)
* sort snap trienode heal requests

* eth/protocols/snap: remove debug code

* eth/protocols/snap: simplify sort, generate pathsets later

* eth/protocols/snap: review concern

* eth/protocols/snap: renamings

* eth/protocols/snap: add comments in Merge

* eth/protocols/snap: remove variable 'last' in Merge

* eth/protocols/snap: fix lint flaws in test

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-05-10 17:37:24 +03:00
s7v7nislands
7caa2d8163
all: replace strings.Replace with string.ReplaceAll (#24835) 2022-05-09 13:13:23 +03:00
Marius van der Wijden
86d5477079
core/state/snapshot: fix race condition (#24685)
Fixes three race conditions found through fuzzing by David Theodore
2022-05-06 17:20:41 +02:00
Martin Holst Swende
7175f82495
eth/fetcher: avoid hang in tests (partial fix for #23331) (#23351)
* eth/fetcher: fix test to avoid hanging. Partial fix for #23331

* eth/filters: avoid dangling goroutines

* eth/fetcher: revert closing of proceed
2022-05-06 15:23:43 +02:00
Mateusz Morusiewicz
cef1a86df2
miner: discard interrupted blocks (#24638)
During mining, when a new head arrives and interrupts the block building, the block being built should not be commited (but discarded). Committing the interrupted block introduces unnecessary delay, and possibly causes miner to mine on the previous head, which could result in higher uncle rate.
2022-05-06 11:19:30 +02:00
EXEC
256aae0bfa
eth/filters: remove explicit continue label in filterLogs (#24795)
The loop label can be removed because this 'continue' statement
is not in a nested loop.
2022-05-05 17:58:43 +02:00
s7v7nislands
1c90d97c1e
eth/protocols/eth: fix godoc comments (#24810)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-05-05 17:35:36 +02:00
rjl493456442
7f6f01d46f
core: recover state when beacon sets canonical head if it's missing (#24613)
* core: recover the state in SetChainHead if the head state is missing

* core: disable test logging

* core: address comment from martin

* core: improve log level in case state is recovered

* core, eth, les, light: rename SetChainHead to SetCanonical
2022-05-05 10:36:26 +03:00
Péter Szilágyi
ecae8e4f65
cmd, eth: fix required blocks regression 2022-05-04 19:55:17 +03:00
EXEC
d6b77f661c
eth/filters: fix code comment (#24799) 2022-05-04 15:32:51 +02:00
hero5512
f94e23ca66
eth/filters: remove unused struct fields (#24782) 2022-04-29 10:07:52 +02:00
s7v7nislands
7ab15490e9
all: use 'embed' instead of go-bindata (#24744) 2022-04-25 11:15:14 +02:00
Emmanuel T Odeke
a8bb49b8ea
eth/tracers/logger: remove unnecessary comparisons in accessList.equal (#24663)
This change removes extraneous/unnecessary checks for equality
when comparing 2 accessList values A and B. Given that we validate that
their lengths of A and B are equal, if so and if every element in A is
in B, reflexively every element in B is already in A. If that weren't
the case and an element g existed in A but not in B, that would mean
that there is an extra element and hence a mathematical contradiction.

Fixes #24658
2022-04-22 01:30:15 +02:00
Ikko Ashimine
40cfe71002
eth/downloader: fix typo in downloader.go (#24704)
synchornization -> synchronization
2022-04-20 16:17:29 +02:00
rjl493456442
f0328f241b
eth/downloader: resolve local header by hash for beacon sync (#24691)
* eth/downlaoder: resolve local header by hash for beacon sync

* eth/downloader: fix error message

* eth/downloader: cap the reverse header resolving

* eth/downloader: re-enable tests

* eth/downloader: add warning logs
2022-04-14 09:49:23 +03:00
Péter Szilágyi
86216189a5
eth/downloader: remove stale beacon headers as backfilling progresses (#24670)
* eth/downloader: remove stale beacon headers as backfilling progresses

* eth/downloader: remove leftover from a previous design

* eth/downloader: do partial beacon cleanups if chain is large

* eth/downloader: linter != heart
2022-04-13 20:31:08 +03:00
Sina Mahmoodi
9c82c646e4
eth/tracers: make txhash blockhash accessible to native tracers (#24679) 2022-04-12 21:09:27 +02:00
ucwong
d4d288e3f1
build: add imports for go generate tools (#24682)
This adds a tools.go file to import all command packages used for
go:generate. Doing so makes it possible to execute go-based code
generators using 'go run', locking in the tool version using go.mod.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-04-12 20:24:02 +02:00
Marius van der Wijden
c40943a167
cmd: set DefaultGasLimit to 30M (#24680)
* cmd: set DefaultGasLimit to 30M, rem deprec. Flag

* cmd: revert flag deprecation
2022-04-12 08:48:03 +03:00
Eng Zer Jun
8d066f1f42
all: use T.TempDir to create temporary test directories (#24633)
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.

	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}

is also tedious, but t.TempDir handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-08 15:44:55 +02:00
Péter Szilágyi
111a1b73cf
Merge pull request #24652 from karalabe/block-fetcher-timeouts
eth/fetcher: if peers never respond, drop them
2022-04-06 10:43:18 +03:00
Sina Mahmoodi
fb3a081c7e
eth/tracers: refactor traceTx to separate out struct logging (#24326)
* eth/tracers: refactor traceTx to separate out struct logging

review fix

Update eth/tracers/api.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

Mv ExecutionResult type to logger package

review fix

impl GetResult for StructLogger

make formatLogs private

confused exit and end..

account for intrinsicGas in structlogger, fix TraceCall test

Add Stop method to logger

Simplify traceTx

Fix test

rm logger from blockchain test

account for refund in structLogger

* use tx hooks in struct logger

* minor

* avoid executionResult in struct logger

* revert blockchain test changes
2022-04-06 09:34:18 +02:00
Péter Szilágyi
7e2bbb9cbb
eth/fetcher: if peers never respond, drop them 2022-04-06 10:18:57 +03:00
rjl493456442
28ec26094b
eth/downloader: retrieve pivot header from local chain if necessary (#24610)
* eth/downloader: retrieve pivot header from local chain if necessary

* eth/downloader: improve readability

* eth/downloader: update fix

* eth/downloader: add beacon sync tests

* eth/downloader: remove duplicated code
2022-04-04 10:10:16 +03:00
Martin Holst Swende
1e973a96b4
eth: clarify the error string on getlogs failure (#24617)
This PR makes the errors we spit out a bit more clear about what block is problematic.
2022-03-31 21:16:03 +02:00
Sina Mahmoodi
3fd16af5a9
core,eth: implement tx-level hooks for tracers (#24510)
* core,eth: add empty tx logger hooks

* core,eth: add initial and remaining gas to tx hooks

* store tx gasLimit in js tracer

* use gasLimit to compute intrinsic cost for js tracer

* re-use rules in transitiondb

* rm logs

* rm logs

* Mv some fields from Start to TxStart

* simplify sender lookup in prestate tracer

* mv env to TxStart

* Revert "mv env to TxStart"

This reverts commit 656939634b9aff19f55a1cd167345faf8b1ec310.

* Revert "simplify sender lookup in prestate tracer"

This reverts commit ab65bce48007cab99e68232e7aac2fe008338d50.

* Revert "Mv some fields from Start to TxStart"

This reverts commit aa50d3d9b2559addc80df966111ef5fb5d0c1b6b.

* fix intrinsic gas for prestate tracer

* add comments

* refactor

* fix test case

* simplify consumedGas calc in prestate tracer
2022-03-31 11:51:44 +02:00
Nic Jansma
127dc5982e
eth: change snapshot extension registration failure to warning instead of error (#24475)
* core: Change Snapshot extension registration failed to Debug

* Update eth/handler.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-30 10:28:27 +02:00
Martin Holst Swende
67c070c379
eth/tracers/logger: use omitempty to reduce log bloat (#24547)
Makes the evm json output less verbose: omitting output of `memory` and `returndata` in case they are empty.
2022-03-29 22:36:55 +02:00
Marius van der Wijden
de6a113f84
eth/catalyst: only apply block if we actually have the state (#24598)
* eth/catalyst: only apply block if we actually have the state

* add header to payload queue

* Update cmd/geth/dbcmd.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-03-29 18:26:18 +02:00
Marius van der Wijden
535f25d65f
eth/catalyst: fix log message (#24574) 2022-03-23 21:00:26 +01:00
Martin Holst Swende
f252154599
eth/downloader: fix flakey test (#24576) 2022-03-23 20:57:53 +01:00
rjl493456442
e0e8bf31c5
eth/downloader: ignore zero size header batch for importing (#24569)
* eth/downloader: ignore zero size header batch for importing

* core, light: reject empty header batch for importing
2022-03-22 18:58:05 +01:00
Péter Szilágyi
51de2bc9dc
eth/downloader: terminate beacon sync early when linked to local chain (#24550)
* eth/downloader: terminate beacon sync early when linked to local chain

* eth/downloader: fix backfiller resume on early beacon termination
2022-03-21 10:43:54 +02:00
Marius van der Wijden
4f4622bc8b
core: eth: implement Kiln-v2 spec (#24506)
* core/beacon: eth/catalyst: updated engine api to new version

* core: implement exchangeTransitionConfig

* core/beacon: prevRandao instead of Random

* eth/catalyst: Fix ExchangeTransitionConfig, add test

* eth/catalyst: stop external miners on TTD reached

* node: implement --authrpc.vhosts flag

* core: allow for config override on non-mainnet networks

* eth/catalyst: fix peters comments

* eth/catalyst: make stop remote sealer more explicit

* eth/catalyst: add log output

* cmd/utils: rename authrpc.host to authrpc.addr

* eth/catalyst: disable the disabling of the miner

* eth: core: remove notion of terminal pow block

* eth: les: more of peters nitpicks
2022-03-17 17:20:03 +02:00
Péter Szilágyi
6b8718c374
eth/downloader: fix off-by-one error in test causing 50% fails 2022-03-16 08:57:00 +02:00
Ryan Schneider
dbfd397262
cmd/geth: rename --whitelist to --eth.requiredblocks (#24505)
* cmd, eth: Rename whitelist argument to peer.requiredblocks

* eth/ethconfig: document PeerRequiredBlocks better

* cmd/utils: rename new flag to --eth.requiredblocks

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-15 12:20:03 +01:00
Guifel
85042b7090
eth/tracers: use hex encoding for debug_traceBlock argument (#24517) 2022-03-15 10:27:30 +01:00
Péter Szilágyi
8f66ea3786
eth/downloader: implement beacon sync (#23982)
* eth/downloader: implement beacon sync

* eth/downloader: fix a crash if the beacon chain is reduced in length

* eth/downloader: fix beacon sync start/stop thrashing data race

* eth/downloader: use a non-nil pivot even in degenerate sync requests

* eth/downloader: don't touch internal state on beacon Head retrieval

* eth/downloader: fix spelling mistakes

* eth/downloader: fix some typos

* eth: integrate legacy/beacon sync switchover and UX

* eth: handle UX wise being stuck on post-merge TTD

* core, eth: integrate the beacon client with the beacon sync

* eth/catalyst: make some warning messages nicer

* eth/downloader: remove Ethereum 1&2 notions in favor of merge

* core/beacon, eth: clean up engine API returns a bit

* eth/downloader: add skeleton extension tests

* eth/catalyst: keep non-kiln spec, handle mining on ttd

* eth/downloader: add beacon header retrieval tests

* eth: fixed spelling, commented failing tests out

* eth/downloader: review fixes

* eth/downloader: drop peers failing to deliver beacon headers

* core/rawdb: track beacon sync data in db inspect

* eth: fix review concerns

* internal/web3ext: nit

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-11 14:14:45 +02:00
Péter Szilágyi
1b58e42802
Merge pull request #24529 from holiman/fix_inf
eth/protocols/snap: avoid estimating infinite percentage
2022-03-11 10:40:55 +02:00
沉风
7d3ecca451
eth/protocols/snap: a little typo Merkel -> Merkle (#24530) 2022-03-11 10:32:08 +02:00
Sina Mahmoodi
57cec89253
graphql: fee history fields (#24452)
This PR adds the `NextBaseFeePerGas` to `Block` and `EffectiveTip` to `Transaction` to make it easier for clients to compute fee history themselves via graphql queries.
2022-03-10 13:59:22 +01:00
Martin Holst Swende
658415960e
eth/protocols/snap: avoid estimating infinite percentage 2022-03-10 12:46:48 +01:00
Martin Holst Swende
4860e50e05
cmd/geth, node, rpc: implement jwt tokens (#24364)
* rpc, node: refactor request validation and add jwt validation

* node, rpc: fix error message, ignore engine api in RegisterAPIs

* node: make authenticated port configurable

* eth/catalyst: enable unauthenticated version of engine api

* node: rework obtainjwtsecret (backport later)

* cmd/geth: added auth port flag

* node: happy lint, happy life

* node: refactor authenticated api

Modifies the authentication mechanism to use default values

* node: trim spaces and newline away from secret

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-07 09:30:27 +02:00
Felix Lange
d6f49bf764
core/types: faster RLP encoding of Header, StateAcccount, ReceiptForStorage (#24420)
This change makes use of the new code generator rlp/rlpgen to improve the
performance of RLP encoding for Header and StateAccount. It also speeds up
encoding of ReceiptForStorage using the new rlp.EncoderBuffer API.

The change is much less transparent than I wanted it to be, because Header and
StateAccount now have an EncodeRLP method defined with pointer receiver. It
used to be possible to encode non-pointer values of these types, but the new
method prevents that and attempting to encode unadressable values (even if
part of another value) will return an error. The error can be surprising and may
pop up in places that previously didn't expect any errors.

To make things work, I also needed to update all code paths (mostly in unit tests)
that lead to encoding of non-pointer values, and pass a pointer instead.

Benchmark results:

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           328ns ± 0%     237ns ± 1%   -27.63%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8           353ns ± 0%     247ns ± 1%   -30.06%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     237ns ± 0%     123ns ± 0%   -47.86%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8            297ns ± 0%     301ns ± 1%    +1.39%  (p=0.000 n=8+8)

    name                             old speed      new speed      delta
    EncodeRLP/legacy-header-8        1.66GB/s ± 0%  2.29GB/s ± 1%   +38.19%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8        1.55GB/s ± 0%  2.22GB/s ± 1%   +42.99%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8  38.0MB/s ± 0%  64.8MB/s ± 0%   +70.48%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8          910MB/s ± 0%   897MB/s ± 1%    -1.37%  (p=0.000 n=8+8)

    name                             old alloc/op   new alloc/op   delta
    EncodeRLP/legacy-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/london-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/receipt-for-storage-8     64.0B ± 0%      0.0B       -100.00%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8             320B ± 0%      320B ± 0%      ~     (all equal)
2022-02-18 08:10:26 +01:00
Martin Holst Swende
6ce4670bc0
cmd/devp2p: implement snap protocol testing (#24276)
This also contains some changes to the protocol handler to
make the tests pass.
2022-02-04 15:24:32 +01:00
Sina Mahmoodi
c5436c8eb7
eth/tracers: clean-up tracer collection (#24320)
* eth/tracers: clean-up tracer collection

* Rm test for dropped tracer
2022-02-01 10:44:44 +01:00
rjl493456442
9da25c5db7
all: separate catalyst package (#24280)
* all: seperate catalyst package

* eth/catalyst: moved some methods, added docs

* eth/catalyst, les/catalyst: add method docs

* core, eth, les, miner: move common function to beacon package

* eth/catalyst: goimported

* cmd/utils, miner/stress/beacon: naming nitpicks

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-31 14:22:35 +02:00
Sina Mahmoodi
cac09a3823
eth/tracers: native prestate tracer (#24268)
* eth/tracers: add initial native prestate tracer

* fix balance hex

* handle prestate for tx from and to

* drop created contract from prestate

* fix sender balance

* use switch instead

Co-authored-by: Martin Holst Swende <martin@swende.se>

* minor fix

* lookup create2 account

* mv code around a bit

* check stackLen for create2

* fix transfer tx for js prestate tracer

* fix create2 addr

* track extcodehash in js prestate tracer

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-31 09:51:17 +01:00
Martin Holst Swende
015fde9a2c
eth/tracers: avoid using blockCtx concurrently (#24286) 2022-01-25 14:45:30 +01:00
rjl493456442
78636ee568
eth, miner: use miner for post-merge block production (#23256)
* eth, miner: remove duplicated code

* eth/catalyst: remove unneeded code

* miner: keep update pending state even the Merge is happened

* eth, miner: rebase

* miner: fix tests

* eth, miner: address comments from marius

* miner: use empty zero randomness for pending blocks after the merge

* eth/catalyst: gofmt

* miner: add warning log for state recovery

* miner: ignore uncles for post-merge blocks

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-24 09:19:52 +02:00
Shihao Xia
eef7a33135
core, miner, rpc, eth: fix goroutine leaks in tests (#24211)
* fix blocking and non-blocking issues

* core: revert change in blockchain.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-21 12:22:44 +01:00
Diederik Loerakker
514ae7cfa3
eth/catalyst: evict old payloads, type PayloadID (#24236)
* eth/catalyst: evict old payloads, type PayloadID

* eth/catalyst: added tracing info to engine api

* eth/catalyst: add test for create payload timestamps

* catalyst: better logs

* eth/catalyst: computePayloadId return style

* catalyst: add queue for payloads

* eth/catalyst: nitpicks

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-20 13:29:06 +02:00
Marius van der Wijden
b1e72f7ea9
core/evm: RANDOM opcode (EIP-4399) (#24141)
* core: implement eip-4399 random opcode

* core: make vmconfig threadsafe

* core: miner: pass vmConfig by value not reference

* all: enable 4399 by Rules

* core: remove diff (f)

* tests: set proper difficulty (f)

* smaller diff (f)

* eth/catalyst: nit

* core: make RANDOM a pointer which is only set post-merge

* cmd/evm/internal/t8ntool: fix t8n tracing of 4399

* tests: set difficulty

* cmd/evm/internal/t8ntool: check that baserules are london before applying the merge chainrules
2022-01-10 09:44:21 +02:00
Martin Holst Swende
0dec47b5c0
eth: continue after whitelist check 2022-01-07 14:19:00 +01:00
Péter Szilágyi
af2ca5a654
Merge pull request #24117 from holiman/db_has
trie, core, eth: use db.has over db.get where possible
2022-01-06 11:30:11 +02:00
Marius van der Wijden
2295640ebd
eth/catalyst: implement kintsugi-spec v3 (#24067) 2021-12-17 15:38:58 +01:00
Sina Mahmoodi
ada9c774e9
eth, les: update unclean shutdown markers regularly (#24077)
Fixes #22580

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-17 15:18:51 +01:00
Andrei Maiboroda
3e47e38a4e
core/vm: Make INVALID a defined opcode (#24017)
* core/vm: Define 0xfe opcode as INVALID

* core/vm: Remove opInvalid as opUndefined handles it

Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>
2021-12-17 13:44:05 +01:00
Martin Holst Swende
893502e561
trie, core, eth: use db.has over db.get where possible 2021-12-15 16:16:45 +01:00
Martin Holst Swende
db03faa10d
core, eth: improve delivery speed on header requests (#23105)
This PR reduces the amount of work we do when answering header queries, e.g. when a peer
is syncing from us.

For some items, e.g block bodies, when we read the rlp-data from database, we plug it
directly into the response package. We didn't do that for headers, but instead read
headers-rlp, decode to types.Header, and re-encode to rlp. This PR changes that to keep it
in RLP-form as much as possible. When a node is syncing from us, it typically requests 192
contiguous headers. On master it has the following effect:

- For headers not in ancient: 2 db lookups. One for translating hash->number (even though
  the request is by number), and another for reading by hash (this latter one is sometimes
  cached).
  
- For headers in ancient: 1 file lookup/syscall for translating hash->number (even though
  the request is by number), and another for reading the header itself. After this, it
  also performes a hashing of the header, to ensure that the hash is what it expected. In
  this PR, I instead move the logic for "give me a sequence of blocks" into the lower
  layers, where the database can determine how and what to read from leveldb and/or
  ancients.

There are basically four types of requests; three of them are improved this way. The
fourth, by hash going backwards, is more tricky to optimize. However, since we know that
the gap is 0, we can look up by the parentHash, and stlil shave off all the number->hash
lookups.

The gapped collection can be optimized similarly, as a follow-up, at least in three out of
four cases.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-07 17:50:58 +01:00
aaronbuchwald
d78590560d
eth/filters: remove unused field 'chaindb' in PublicFilterAPI (#24072) 2021-12-07 02:36:01 +01:00
Marius van der Wijden
93f196c4b0
eth/catalyst: implement kintsugi spec v1.0.0-alpha.4 (#23984)
This PR implements the new kintsugi specification which can be found here: https://github.com/ethereum/execution-apis/blob/v1.0.0-alpha.4/src/engine/specification.md
2021-12-03 16:26:28 +01:00
Péter Szilágyi
58d1988349
core, eth, les, trie: remove the sync bloom, used by fast sync 2021-12-03 12:32:41 +02:00
Péter Szilágyi
5e78fc034b
Merge pull request #24032 from karalabe/downloader-response-preprocess
eth: pre-process downloader responses on the peer reader thread
2021-12-03 10:34:25 +02:00
Taeik Lim
85064ed09b
all: fix 'the the' in comments (#24036) 2021-12-02 15:42:09 +01:00
Péter Szilágyi
c893488349
eth: pre-process downloader responses on the peer reader thread 2021-12-01 20:18:12 +02:00
Alex Beregszaszi
721c5723c0
eth/tracers/js: add support for REVERT/SELFDESTRUCT in evmdis_tracer (#24016)
* eth/tracers: Add support for REVERT in evmdis_tracer

* evm/tracers: Fix evmdis_tracer to use SELFDESTRUCT instead of SUICIDE

* eth/tracers: Regenerate tracer library
2021-12-01 10:34:52 +01:00
Alex Beregszaszi
9393d1fb5d
core/vm: Move interpreter.ReadOnly check into the opcode implementations (#23970)
* core/vm: Move interpreter.ReadOnly check into the opcode implementations

Also remove the same check from the interpreter inner loop.

* core/vm: Remove obsolete operation.writes flag

* core/vm: Capture fault states in logger

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/vm: Remove panic added for testing

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-12-01 10:21:21 +01:00
Martin Holst Swende
1988b47e02
tests/fuzzzers. eth/protocols/snap: add snap protocol fuzzers (#23957) 2021-12-01 10:17:18 +01:00
Péter Szilágyi
c10a0a62c3
eth: request id dispatcher and direct req/reply APIs (#23576)
* eth: request ID based message dispatcher

* eth: fix dispatcher cancellation, rework fetchers idleness tracker

* eth/downloader: drop peers who refuse to serve advertised chains
2021-11-26 13:26:03 +02:00
Marius van der Wijden
3038e480f5
all: core rework for the merge transition (#23761)
* all: work for eth1/2 transtition

* consensus/beacon, eth: change beacon difficulty to 0

* eth: updates

* all: add terminalBlockDifficulty config, fix rebasing issues

* eth: implemented merge interop spec

* internal/ethapi: update to v1.0.0.alpha.2

                                                                 This commit updates the code to the new spec, moving payloadId into
                                                                 it's own object. It also fixes an issue with finalizing an empty blockhash.
                                                                 It also properly sets the basefee

* all: sync polishes, other fixes + refactors

* core, eth: correct semantics for LeavePoW, EnterPoS

* core: fixed rebasing artifacts

* core: light: performance improvements

* core: use keyed field (f)

* core: eth: fix compilation issues + tests

* eth/catalyst: dbetter error codes

* all: move Merger to consensus/, remove reliance on it in bc

* all: renamed EnterPoS and LeavePoW to ReachTDD and FinalizePoS

* core: make mergelogs a function

* core: use InsertChain instead of InsertBlock

* les: drop merger from lightchain object

* consensus: add merger

* core: recoverAncestors in catalyst mode

* core: fix nitpick

* all: removed merger from beacon, use TTD, nitpicks

* consensus: eth: add docstring, removed unnecessary code duplication

* consensus/beacon: better comment

* all: easy to fix nitpicks by karalabe

* consensus/beacon: verify known headers to be sure

* core: comments

* core: eth: don't drop peers who advertise blocks, nitpicks

* core: never add beacon blocks to the future queue

* core: fixed nitpicks

* consensus/beacon: simplify IsTTDReached check

* consensus/beacon: correct IsTTDReached check

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-11-26 13:23:02 +02:00
Sina Mahmoodi
1876cb443b
all: move loggers to eth/tracers (#23892)
* all: mv loggers to eth/tracers

* core/vm: minor

* eth/tracers: tmp comment out testStoreCapture

* eth/tracers: uncomment and fix logger test

* eth/tracers: simplify test

* core/vm: re-add license

* core/vm: minor

* rename LogConfig to Config
2021-11-25 14:17:09 +02:00
Péter Szilágyi
d9c13d407f
core, eth/downloader: fix resetting below freezer threshold 2021-11-22 11:12:51 +02:00
Anatole
e0761432a4
eth: fix typo in comment (#23941) 2021-11-22 02:53:16 +01:00
courtier
c52def7f11
eth/gasprice: sanitize max header and block history (#23886)
Fixes #23452
2021-11-18 19:20:36 +01:00
Sina Mahmoodi
05acc272b5
eth/tracers: make native 4byte default, remove js version (#23916) 2021-11-16 08:44:57 +01:00
Ward Bradt
e9294a7fe9
eth/tracers: add golang 4byte tracer (#23882)
* native 4byte tracer

* Update eth/tracers/native/4byte.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* Update eth/tracers/native/4byte.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* goimports

* eth/tracers: make 4byte tracer not care about create

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-11-11 20:20:46 +01:00
Sina Mahmoodi
c57df9ca28
core/rawdb: add slow path for getting legacy logs (#23879)
* eth/tracers: add slow path for getting legacy logs

* core/rawdb: fix test
2021-11-11 15:04:06 +01:00
Martin Holst Swende
6b9c77f060
eth/tracers: package restructuring (#23857)
* eth/tracers: restructure tracer package

* core/vm/runtime: load js tracers

* eth/tracers: mv bigint js code to own file

* eth/tracers: add method docs for native tracers

* eth/tracers: minor doc fix

* core,eth: cancel evm on nativecalltracer stop

* core/vm: fix failing test

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2021-11-09 12:09:35 +01:00
Sina Mahmoodi
787a3b185c
eth/tracers: make native calltracer default (#23867) 2021-11-08 14:08:12 +01:00
Sina Mahmoodi
8d7e6062ec
eth/tracers: support for golang tracers + add golang callTracer (#23708)
* eth/tracers: add basic native loader

* eth/tracers: add GetResult to tracer interface

* eth/tracers: add native call tracer

* eth/tracers: fix call tracer json result

* eth/tracers: minor fix

* eth/tracers: fix

* eth/tracers: fix benchTracer

* eth/tracers: test native call tracer

* eth/tracers: fix

* eth/tracers: rm extra make

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/tracers: rm extra make

* eth/tracers: make callFrame private

* eth/tracers: clean-up and comments

* eth/tracers: add license

* eth/tracers: rework the model a bit

* eth/tracers: move tracecall tests to subpackage

* cmd/geth: load native tracers

* eth/tracers: minor fix

* eth/tracers: impl stop

* eth/tracers: add native noop tracer

* renamings

Co-authored-by: Martin Holst Swende <martin@swende.se>

* eth/tracers: more renamings

* eth/tracers: make jstracer non-exported, avoid cast

* eth/tracers, core/vm: rename vm.Tracer to vm.EVMLogger for clarity

* eth/tracers: minor comment fix

* eth/tracers/testing: lint nitpicks

* core,eth: cancel evm on nativecalltracer stop

* Revert "core,eth: cancel evm on nativecalltracer stop"

This reverts commit 01bb908790a369c1bb9d3937df9325c6857bf855.

* eth/tracers: linter nits

* eth/tracers: fix output on err

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-11-05 11:48:21 +01:00
Martin Holst Swende
3bbeb94c1c
eth: make traceChain avoid OOM on long-running tracing (#23736)
This PR changes long-running chain tracing, so that it at some points releases the memory trie db, and switch over to a fresh disk-backed trie.
2021-11-04 18:54:00 +01:00
Martin Holst Swende
551bd6e721
eth/tracers: invoke enter/exit on 0-value calls to inex accounts (#23828) 2021-11-01 19:06:33 +01:00
XuDon9 Liu AKA Rapper Of CN background diablo & revelations, a man optimistic, negative, silent, active, Taoism
410e731bea
optimize the judge of invalide notification.number (#22658)
Don't bother fetching genesis

Co-authored-by: wuff1996 <33193253+wuff1996@users.noreply.github.com>
2021-10-31 19:38:48 +01:00
Martin Holst Swende
31870a59ff
eth/filters, p2p/simulations: fix benchmarks (#23806)
Some benchmarks in eth/filters were not good: they weren't reproducible, relying on geth chaindata to be present.

Another one was rejected because the receipt was lacking a backing transcation.

The p2p simulation benchmark had a lot of the warnings below, due to the framework calling both
Stop() and Close(). Apparently, the simulated adapter is the only implementation which has a Close(),
and there is no need to call both Stop and Close on it.
2021-10-29 14:37:00 +02:00
Martin Holst Swende
32150f8aa9
cmd/geth, cmd/evm, params: implement Arrow Glacier (EIP 4345) (#23810)
This PR adds support for ArrowGlacier, as defined by

    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md
    https://eips.ethereum.org/EIPS/eip-4345

> Starting with FORK_BLOCK_NUMBER the client will calculate the difficulty based on a fake block number suggesting to the client that the difficulty bomb is adjusting 10,700,000 blocks later than the actual block number.

This also adds support for evm t8n to return the calculated difficulty, so it can be used to construct test.
2021-10-28 22:18:14 +02:00
Preston Van Loon
b97f57882c
eth/fetcher: remove superfluous nilness-check (#23739)
* eth/fetcher: fix nilness check https://github.com/ethereum/go-ethereum/issues/23738

* eth/fetcher: Use errors.Is. PR feedback from @holiman.
2021-10-18 22:34:20 +02:00
jwasinger
011fe3eb5e
core: remove unused error from TxPool.Pending (#23720) 2021-10-13 23:00:45 +02:00
Iskander (Alex) Sharipov
778ff94794
all: fix some go-critic linter warnings (#23709)
This doesn't fix all go-critic warnings, just the most serious ones.

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-10-13 17:31:02 +02:00
noam-alchemy
633e7ef478
eth,rpc: allow for flag configured timeouts for eth_call (#23645)
* eth,rpc: allow for flag configured timeouts for eth_call

* lint: account for package-local import order

* cr: rename `rpc.calltimeout` to `rpc.evmtimeout`
2021-10-12 10:46:04 +02:00
Martin Holst Swende
6289137827
consensus/clique, core: API cleanup (#23100)
This removes some code:

- The clique engine calculated the snapshot twice when verifying headers/blocks.

- The method GetBlockHashesFromHash in Header/Block/Lightchain was only used by tests. It
  is now removed from the API.
  
- The method GetTdByHash internally looked up the number before calling GetTd(hash, num).
  In many cases, callers already had the number, and used this method just because it has a
  shorter name. I have removed the method to make the API surface smaller.
2021-10-11 23:16:46 +02:00
Marius van der Wijden
53b1420ede
params: changed CatalystBlock to TerminalTotalDifficulty (#23700)
* params: changed CatalystBlock to TerminalTotalDifficulty

* eth/catalyst: comment out unused code
2021-10-11 18:58:11 +03:00
aaronbuchwald
bcbd700367
eth/gasprice: avoid modifying TestChainConfig (#23204)
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-11 01:52:30 +02:00
Marius van der Wijden
ee120ef865
miner: fix data race during shutdown (#23435)
This fixes a data race on worker.current by moving the call to StopPrefetcher
into the main loop.

The commit also contains fixes for two other races in unit tests of unrelated packages.
2021-10-08 20:12:52 +02:00
Martin Holst Swende
28d30b51f8
eth: close miner on exit (instead of just stopping) (#21992)
This ensures that all miner goroutines have exited before stopping the blockchain. 

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-08 18:36:58 +02:00
Sina Mahmoodi
48496e0675
internal/ethapi: use correct signer when serving old blocks (#23683)
Fixes #23681

After the fix I get the address 0x6d6d02e83c4ced98204e20126acf27e9d87b8af2 for the
tx mentioned in the ticket, which agrees with etherscan.
2021-10-07 15:51:14 +02:00
Martin Holst Swende
307156cc46
eth/api: add rpc method to obtain which states are accessible (#23646)
This PR adds a method to the debug namespace, to iterate over the blocks and check where we have the roots on disk.
2021-10-05 08:13:00 +02:00
Ferran Borreguero
0dbb3b1601
eth/protocols/eth: replace array with counter in txn broadcaster (#23656) 2021-10-04 16:10:51 +02:00
Miro
5a0e1d88f4
eth/filters: fix TestPendingLogsSubscription (#23619)
The test did not synchronize with per-case goroutines, and thus didn't notice
that some tests were just hanging. This change adds missing synchronization
and fixes the broken tests.
2021-10-04 14:09:51 +02:00
Sina Mahmoodi
b522f5e091
eth/tracers: fix callTracer fault handling (#23667)
* eth/tracers: fix calltracer fault handling

* eth/tracers: fix calltracer indentation
2021-10-01 13:03:24 +02:00
Martin Holst Swende
3a6fe69f23
eth/protocols/snap, trie: better error-handling (#23657) 2021-09-29 15:19:40 +02:00
Ferran Borreguero
a541fbea18
eth/protocols/eth: simplify peer known block/txs caches (#23649)
* Simplify peer known block/txns cache

* Address minor changes

* Add more minor comments

* Minor changes from review
2021-09-28 13:44:07 +02:00
Martin Holst Swende
3531ca2246
eth/tracers: avoid unsyncronized mutations on trie database (#23632)
This PR fixes an issue in traceChain, where the statedb Commit operation was performed asynchronously with dereference-operations agains the underlying trie.Database instance. Due to how the reference counting works within the trie database (where parent count is recursively updated when new parents are added), doing dereferencing in the middle of Commit can cause the refcount to become wrong, leading to an inconsistent state. 

This was fixed by doing Commit/Deref from the same routine.
2021-09-28 13:06:19 +02:00
Sina Mahmoodi
783e97ef1f
core/rawdb: avoid unnecessary receipt processing for log filtering (#23147)
* core/types: rm extranous check in test

* core/rawdb: add lightweight types for block logs

* core/rawdb,eth: use lightweight accessor for log filtering

* core/rawdb: add bench for decoding into rlpLogs
2021-09-28 12:54:49 +02:00
Martin Holst Swende
ab2caaee11
eth/tracers: implement debug.intermediateRoots (#23594)
This PR implements a new debug method, which I've talked briefly about to some other client developers. It allows the caller to obtain the intermediate state roots for a block (which might be either a canon block or a 'bad' block).
2021-09-28 12:53:11 +02:00
Guillaume Ballet
443afc975c
core/state: move state account to core/types + abstracted "write account to trie" (#23567)
* core/state: abstracted "write account to trie" method

* fix appveyor build

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

* review feedback

* core/state/accounts: move Account to core/types

* core/types: rename Account -> StateAccount

* core/state: restore EncodeRLP for stateObject

* core/types: add the missing file

* more review feedback

* more review feedback

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-28 10:48:07 +02:00
Sina Mahmoodi
ac7baeab57
eth/tracers: re-write of 4byte tracer using enter/exit (#23622)
* eth/tracers: add re-write of 4byte tracer using enter/exit

* eth/tracers: fix 4byte indent
2021-09-27 18:44:16 +02:00
Sina Mahmoodi
401354976b
core,eth: call frame tracing (#23087)
This change introduces 2 new optional methods; `enter()` and `exit()` for js tracers, and makes `step()` optiona. The two new methods are invoked when entering and exiting a call frame (but not invoked for the outermost scope, which has it's own methods). Currently these are the data fields passed to each of them:

    enter: type (opcode), from, to, input, gas, value
    exit: output, gasUsed, error

The PR also comes with a re-write of the callTracer. As a backup we keep the previous tracing script under the name `callTracerLegacy`. Behaviour of both tracers are equivalent for the most part, although there are some small differences (improvements), where the new tracer is more correct / has more information.
2021-09-17 09:31:22 +02:00
Martin Holst Swende
7ada89d4e6
eth/tracers: abort evm execution when trace is aborted (#23580) 2021-09-16 13:45:16 +02:00
Marius van der Wijden
b8d7c662cd
core, rpc: disable memory output by default in traces (#23558)
* core: cmd: invert disableMemory

* core: fix missed inversion

* cmd/evm: preserve Flags but change default value

* Apply suggestions from code review

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-09-13 18:59:52 +02:00
Péter Szilágyi
9e17648d8c
les: duplicate downloader and fetcher to allow progressive refactoring 2021-09-10 11:11:59 +03:00
Martin Holst Swende
62ad17fb00
Revert "eth, internal/ethapi: make RPC block miner field show block sealer correctly (#23312)" (#23466)
This reverts commit 57feabea66.
2021-08-26 08:35:37 +02:00
Péter Szilágyi
0db0b27754
Revert "Revert "eth: drop eth/65, the last non-reqid protocol version" (#23426)" (#23456)
This reverts commit c368f728c1.
2021-08-24 21:52:58 +03:00
Felföldi Zsolt
f38abc55f1
eth/gasprice: feeHistory improvements (#23422)
* eth/gasprice: cache feeHistory results

* eth/gasprice: changed feeHistory block count limitation

* eth/gasprice: do not use embedded struct in blockFees

* eth/gasprice: fee processing logic cleanup

* eth/gasprice: purge feeHistory cache at chain reorgs
2021-08-23 23:50:24 +02:00
Péter Szilágyi
c368f728c1
Revert "eth: drop eth/65, the last non-reqid protocol version" (#23426) 2021-08-20 15:14:21 +03:00
chuwt
5566e5d152
eth/downloader: fix typo in comment (#23413) 2021-08-18 13:03:41 +03:00
陈佳
57feabea66
eth, internal/ethapi: make RPC block miner field show block sealer correctly (#23312)
Makes the RPC block return the POA sealer for clique blocks on the 'miner' field (was previously zeroes)
2021-08-17 18:55:18 +02:00
Péter Szilágyi
fd604becbb
Merge pull request #23120 from karalabe/drop-eth-65
eth: drop eth/65, the last non-reqid protocol version
2021-08-13 11:52:47 +03:00
gary rong
6d175460df
cmd, core, eth, miner: deprecate miner.gastarget flag (#23213) 2021-08-10 11:28:33 +03:00
Martin Holst Swende
0658712f65
core: check if sender is EOA (#23303)
This adds a check to verify that a sender-account does not have code, which means that the codehash is either `emptyCodeHash` _OR_ not present. The latter occurs IFF the sender did not previously exist, a situation which can only occur with zero cost gasprices.
2021-08-07 19:38:18 +02:00
lightclient
bbfa6488ac
Use hexutil.Uint for blockCount parameter in feeHistory method (#23239)
* internal/ethapi/api: use hexutil.uint for blockCount parameter instead of int for feeHistory

* return hex value for oldestBlock instead of number

* return uint64 from oracle.resolveBlockRange

* eth/gasprice: fixed test

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
2021-07-27 05:27:28 +02:00
Marius van der Wijden
97aacd9b35
core: fix pre-check for account balance under EIP-1559 (#23244)
When processing a transaction with London fork rules, EIP-1559 mandates
checking that the sender must have sufficient balance to cover gas * gasFeeCap.

In the EIP's pseudocode, this check happens after the value transferred by the
transaction has already been deducted. However, in go-ethereum, the balance
has not yet been updated when the check happens, and therefore needs to be
added explicitly.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-07-22 15:39:40 +02:00
aaronbuchwald
a5e3aa693c
eth/tracers: fix typo in test name (#23218) 2021-07-15 21:23:16 +03:00
Mark
c49e065fea
internal: get pending and queued transaction by address (#22992)
* core, eth, internal, les, light: get pending and queued transaction by address

* core: tiny nitpick fixes

* light: tiny nitpick

Co-authored-by: mark <mark@amis.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-07-13 13:40:58 +03:00
Péter Szilágyi
dea71556cc
eth/gasprice, internal/ethapi, miner: minor feehistory fixes 2021-07-08 21:50:35 +03:00
Sina Mahmoodi
5441a8fa47
all: remove noop vm config flags (#23111)
* all: rm external interpreter and ewasm config

* core/vm: rm Interpreter interface

* cmd/geth: deprecate interpreter config fields
2021-07-06 22:03:09 +02:00
Martin Holst Swende
d21a069619
eth, miner: add RPC method to modify miner gaslimit (pre london: ceiling) (#23134) 2021-07-06 10:35:39 +02:00
ucwong
3b05318525
cmd/evm, eth/ethconfig: regenerate struct codecs (#23140) 2021-07-02 11:08:53 +02:00
Martin Holst Swende
f5f906dd0d
eth/tracers: improve tracing performance (#23016)
Improves the performance of debug.traceTransaction
2021-07-01 09:15:04 +02:00
Martin Holst Swende
c131e812ae
eth/fetcher, trie: unit test reliability fixes (#23020)
Some tests take quite some time during exit, which I think causes
some appveyor fails like this:

    https://ci.appveyor.com/project/ethereum/go-ethereum/builds/39511210/job/xhom84eg2e4uulq3

One of the things that seem to take time during exit is waiting
(up to 100ms) for the syncbloom to close. This PR changes it to use
a channel, instead of looping with a 100ms wait.

This also includes some unrelated changes improving the reliability of
eth/fetcher tests, which fail a lot because they are time-dependent.
2021-06-30 22:24:17 +02:00
Marius van der Wijden
686b2884ee
all: removed blockhash from statedb (#23126)
This PR removes the blockhash from the statedb
2021-06-30 15:17:01 +02:00
Péter Szilágyi
d3f018fde8
eth: drop eth/65, the last non-reqid protocol version 2021-06-29 12:31:30 +03:00
Felföldi Zsolt
35dbf7a8a3
eth/gasprice: implement feeHistory API (#23033)
* eth/gasprice: implement feeHistory API

* eth/gasprice: factored out resolveBlockRange

* eth/gasprice: add sanity check for missing block

* eth/gasprice: fetch actual gas used from receipts

* miner, eth/gasprice: add PendingBlockAndReceipts

* internal/ethapi: use hexutil.Big

* eth/gasprice: return error when requesting beyond head block

* eth/gasprice: fixed tests and return errors correctly

* eth/gasprice: rename receiver name

* eth/gasprice: return directly if blockCount == 0

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-06-28 16:16:32 +02:00
Sina Mahmoodi
1b5582acf7
core, eth: fix precompile addresses for tracers (#23097)
* core,eth/tracers: make isPrecompiled dependent on HF

* eth/tracers: use keys when constructing chain config struct

* eth/tracers: dont initialize activePrecompiles with random value
2021-06-28 14:13:27 +02:00
Martin Holst Swende
2d4eff21ca
eth/downloader: increase downloader block body allowance (#23074)
This change increases the cache size from 64 to 256 Mb for block bodies.
Benchmarks have shown this to be one bottleneck when trying to achieve
higher download speeds.

The commit also includes a minor optimization for header inserts in package
core: previously, the presence of headers in the database was checked for
every header before writing it. With the change, if one header fails the
presence check, all subsequent headers are also assumed to be missing.
This is an improvement because in practice, the headers are almost always
missing during sync.
2021-06-25 14:53:22 +02:00
Li Dongwei
bca8c03e57
core/state: remove unused methods ReturnGas, GetStorageProofByHash (#23092)
Co-authored-by: lidongwei <lidongwei@huobi.com>
2021-06-25 14:34:09 +02:00
haryu703
c07918e7d8
eth/gasprice: fix typo in comment (#22998) 2021-06-25 12:48:06 +02:00
williamberman
948a600ed5
eth/tracers: convert int/hash values from context into js object (#23108)
* Convert int/hash values from context into js object

* Use js fixed buffer

Co-authored-by: William <william.berman@coinbase.com>
2021-06-25 09:02:15 +03:00
Péter Szilágyi
29905d86ae
eth/tracers: expose contextual infos (block hash, tx hash, tx index) 2021-06-24 12:46:26 +03:00
Péter Szilágyi
7a7abe3de8
accounts/abi/bind: fix bounded contracts and sim backend for 1559 (#23038)
* accounts/abi/bind: fix bounded contracts and sim backend for 1559

* accounts/abi/bind, ethclient: don't rely on chain config for gas prices

* all: enable London for all internal tests

* les: get receipt type info in les tests

* les: fix weird test

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-06-15 13:56:14 +03:00
Péter Szilágyi
be1267ced5
eth/ethconfig: bump the RPC gas cap to 50M, since 1559 exceeds 25 2021-06-10 09:07:03 +03:00
Péter Szilágyi
c503f98f6d
all: rename internal 1559 gas fields, add support for graphql (#23010)
* all: rename internal 1559 gas fields, add support for graphql

* cmd/evm/testdata, core: use public 1559 gas names on API surfaces
2021-06-08 12:05:41 +02:00
Péter Szilágyi
7e915ee379
core, eth, miner: enforce configured mining reward post 1559 too 2021-06-04 10:18:37 +03:00
gary rong
3094e7f3b8
catalyst: runs every transaction in a snapshot in assembleBlock handler (#7) (#22989)
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
2021-06-03 17:12:47 +02:00
Péter Szilágyi
7760a60794
Merge pull request #22973 from karalabe/the-switch
eth/ethconfig: flip the default from fast to snap sync
2021-06-03 12:05:08 +03:00
Martin Holst Swende
5cff9754d7
core, eth, internal, les: RPC methods and fields for EIP 1559 (#22964)
* internal/ethapi: add baseFee to RPCMarshalHeader

* internal/ethapi: add FeeCap, Tip and correct GasPrice to EIP-1559 RPCTransaction results

* core,eth,les,internal: add support for tip estimation in gas price oracle

* internal/ethapi,eth/gasprice: don't suggest tip larger than fee cap

* core/types,internal: use correct eip1559 terminology for json marshalling

* eth, internal/ethapi: fix rebase problems

* internal/ethapi: fix rpc name of basefee

* internal/ethapi: address review concerns

* core, eth, internal, les: simplify gasprice oracle (#25)

* core, eth, internal, les: simplify gasprice oracle

* eth/gasprice: fix typo

* internal/ethapi: minor tweak in tx args

* internal/ethapi: calculate basefee for pending block

* internal/ethapi: fix panic

* internal/ethapi, eth/tracers: simplify txargs ToMessage

* internal/ethapi: remove unused param

* core, eth, internal: fix regressions wrt effective gas price in the evm

* eth/gasprice: drop weird debug println

* internal/jsre/deps: hack in 1559 gas conversions into embedded web3

* internal/jsre/deps: hack basFee to decimal conversion

* internal/ethapi: init feecap and tipcap for legacy txs too

* eth, graphql, internal, les: fix gas price suggestion on all combos

* internal/jsre/deps: handle decimal tipcap and feecap

* eth, internal: minor review fixes

* graphql, internal: export max fee cap RPC endpoint

* internal/ethapi: fix crash in transaction_args

* internal/ethapi: minor refactor to make the code safer

Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: gary rong <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-06-02 16:13:10 +03:00
Péter Szilágyi
9aaa4208a8
eth/ethconfig: flip the default from fast to snap sync 2021-05-31 10:21:48 +03:00
Felföldi Zsolt
966ee3ae6d
all: EIP-1559 tx pool support (#22898)
This pull request implements EIP-1559 compatible transaction pool with dual heap eviction ordering.
It is based on #22791
The eviction ordering scheme and the reasoning behind it is described here: https://gist.github.com/zsfelfoldi/9607ad248707a925b701f49787904fd6
2021-05-28 10:28:07 +02:00
Felix Lange
427175153c
p2p/msgrate: return capacity as integer, clamp to max uint32 (#22943)
* p2p/msgrate: return capacity as integer

* eth/protocols/snap: remove conversions

* p2p/msgrate: add overflow test

* p2p/msgrate: make the capacity overflow test actually overflow

* p2p/msgrate: clamp capacity to max int32

* p2p/msgrate: fix min/max confusion
2021-05-27 19:43:55 +03:00
gary rong
51b32cc7e4
internal/ethapi: merge CallArgs and SendTxArgs (#22718)
There are two transaction parameter structures defined in
the codebase, although for different purposes. But most of
the parameters are shared. So it's nice to reduce the code
duplication by merging them together.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-05-25 22:30:21 +02:00
Felix Lange
836c647bdd
eth: unregister peer only when handler exits (#22908)
This removes the error log message that says 

    Ethereum peer removal failed ... err="peer not registered"

The error happened because removePeer was called multiple
times: once to disconnect the peer, and another time when the
handler exited. With this change, removePeer now has the sole
purpose of disconnecting the peer. Unregistering happens exactly
once, when the handler exits.
2021-05-25 22:20:36 +02:00
Martin Holst Swende
a6c462781f
EIP-1559: miner changes (#22896)
* core/types, miner: create TxWithMinerFee wrapper, add EIP-1559 support to TransactionsByMinerFeeAndNonce

miner: set base fee when creating a new header, handle gas limit, log miner fees

* all: rename to NewTransactionsByPriceAndNonce

* core/types, miner: rename to NewTransactionsByPriceAndNonce + EffectiveTip

miner: activate 1559 for testGenerateBlockAndImport tests

* core,miner: revert naming to TransactionsByPriceAndTime

* core/types/transaction: update effective tip calculation logic

* miner: update aleut to london

* core/types/transaction_test: use correct signer for 1559 txs + add back sender check

* miner/worker: calculate gas target from gas limit

* core, miner: fix block  gas limits for 1559

Co-authored-by: Ansgar Dietrichs <adietrichs@gmail.com>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
2021-05-21 09:59:26 +02:00
Péter Szilágyi
3e795881ea
eth, p2p/msgrate: move peer QoS tracking to its own package and use it for snap (#22876)
This change extracts the peer QoS tracking logic from eth/downloader, moving
it into the new package p2p/msgrate. The job of msgrate.Tracker is determining
suitable timeout values and request sizes per peer.

The snap sync scheduler now uses msgrate.Tracker instead of the hard-coded 15s
timeout. This should make the sync work better on network links with high latency.
2021-05-19 14:09:03 +02:00
Martin Holst Swende
94451c2788
all: implement EIP-1559 (#22837)
This is the initial implementation of EIP-1559 in packages core/types and core.
Mining, RPC, etc. will be added in subsequent commits.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-05-17 15:13:22 +02:00
Martin Holst Swende
addd8824cf
cmd/geth, eth, core: snapshot dump + unify with trie dump (#22795)
* cmd/geth, eth, core: snapshot dump + unify with trie dump

* cmd/evm: dump API fixes

* cmd/geth, core, eth: fix some remaining errors

* cmd/evm: dump - add limit, support address startkey, address review concerns

* cmd, core/state, eth: minor polishes, fix snap dump crash, unify format

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-05-12 11:05:39 +03:00
Péter Szilágyi
0524cede37
eth/tracers: do the JSON serialization via .js to capture C faults 2021-05-11 16:23:54 +03:00
Ryan Schneider
ca98080798
cmd/geth, eth/gasprice: add configurable threshold to gas price oracle (#22752)
This adds a cmd line parameter `--gpo.ignoreprice`, to make the gas price oracle ignore transactions below the given threshold.
2021-05-11 11:25:51 +02:00
Felix Lange
e536bb52ff
eth/protocols/snap: adapt to uint256 API changes (#22851) 2021-05-10 13:35:07 +02:00
Felix Lange
c0e201b690
eth/protocols/eth, les: avoid Raw() when decoding HashOrNumber (#22841)
Getting the raw value is not necessary to decode this type, and
decoding it directly from the stream is faster.
2021-05-10 12:38:54 +02:00
Martin Holst Swende
cc606be74c
all: define London+baikal, undefine yolov3, add london override flag (#22822)
* all: define London+baikal, undefine yolov3, add london override flag

* cmd, core, params: add baikal genesis definition
2021-05-06 12:07:42 +03:00
Felix Lange
b8040a430e
cmd/utils: use eth DNS tree for snap discovery (#22808)
This removes auto-configuration of the snap.*.ethdisco.net DNS discovery tree.
Since measurements have shown that > 75% of nodes in all.*.ethdisco.net support
snap, we have decided to retire the dedicated index for snap and just use the eth
tree instead.

The dial iterators of eth and snap now use the same DNS tree in the default configuration,
so both iterators should use the same DNS discovery client instance. This ensures that
the record cache and rate limit are shared. Records will not be requested multiple times.

While testing the change, I noticed that duplicate DNS requests do happen even
when the client instance is shared. This is because the two iterators request the tree
root, link tree root, and first levels of the tree in lockstep. To avoid this problem, the
change also adds a singleflight.Group instance in the client. When one iterator
attempts to resolve an entry which is already being resolved, the singleflight object
waits for the existing resolve call to finish and returns the entry to both places.
2021-05-04 11:29:32 +02:00
Péter Szilágyi
856c379626
eth: don't print db upgrade warning on db init 2021-05-03 15:42:43 +03:00
Péter Szilágyi
52b5d2d869
eth/protocols/snap: use storage batch, not account batch in st task 2021-04-30 18:24:34 +03:00
Péter Szilágyi
8681a2536c
Merge pull request #22777 from karalabe/snapshots-abort-resume-on-sync
core, eth: abort snapshot generation on snap sync and resume later
2021-04-30 17:04:05 +03:00
Péter Szilágyi
745757ac6b
core, eth: abort snapshot generation on snap sync and resume later 2021-04-30 17:03:10 +03:00
aaronbuchwald
f66f1a16b3
eth/filters: fix comment on PublicFilterAPI timeoutLoop (#22782) 2021-04-30 13:00:48 +02:00
Diederik Loerakker
c7d07294a6
catalyst: check if block exists in assemble-block call with unknown parent-hash (#22770) 2021-04-29 16:42:21 +02:00
Péter Szilágyi
06f44c0fd4
eth: restore eth_hashrate API endpoint 2021-04-29 12:02:30 +03:00
Péter Szilágyi
fae165a5de
core, eth, ethdb, trie: simplify range proofs 2021-04-29 10:59:08 +03:00
Martin Holst Swende
a81cf0d2b3
trie: remove redundant returns + use stacktrie where applicable (#22760)
* trie: add benchmark for proofless range

* trie: remove unused returns + use stacktrie
2021-04-28 22:47:48 +03:00
Péter Szilágyi
558bff4008
eth/protocols/snap: lower the packet size to avoid overloading link 2021-04-28 21:40:06 +03:00
Gregory Markou
256c5d68b2
eth/gasprice: improve stability of estimated price (#22722)
This PR makes the gas price oracle ignore transactions priced at `<=1 wei`.
2021-04-28 09:06:34 +02:00
Martin Holst Swende
0c99868416
cmd/devp2p, eth/protocols/eth: fix tests + make sanity checks earlier (#22749) 2021-04-28 08:48:07 +02:00
Péter Szilágyi
caea6c4661
eth/protocols/snap: generate storage trie from full dirty snap data (#22668)
* eth/protocols/snap: generate storage trie from full dirty snap data

* eth/protocols/snap: get rid of some more dead code

* eth/protocols/snap: less frequent logs, also log during trie generation

* eth/protocols/snap: implement dirty account range stack-hashing

* eth/protocols/snap: don't loop on account trie generation

* eth/protocols/snap: fix account format in trie

* core, eth, ethdb: glue snap packets together, but not chunks

* eth/protocols/snap: print completion log for snap phase

* eth/protocols/snap: extended tests

* eth/protocols/snap: make testcase pass

* eth/protocols/snap: fix account stacktrie commit without defer

* ethdb: fix key counts on reset

* eth/protocols: fix typos

* eth/protocols/snap: make better use of delivered data (#44)

* eth/protocols/snap: make better use of delivered data

* squashme

* eth/protocols/snap: reduce chunking

* squashme

* eth/protocols/snap: reduce chunking further

* eth/protocols/snap: break out hash range calculations

* eth/protocols/snap: use sort.Search instead of looping

* eth/protocols/snap: prevent crash on storage response with no keys

* eth/protocols/snap: nitpicks all around

* eth/protocols/snap: clear heal need on 1-chunk storage completion

* eth/protocols/snap: fix range chunker, add tests

Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* trie: fix test API error

* eth/protocols/snap: fix some further liter issues

* eth/protocols/snap: fix accidental batch reuse

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-27 17:19:59 +03:00
Felix Lange
a3f0da1ac4
build: upgrade to golangci-lint v1.39.0 (#22696)
* build: upgrade to golangci-lint v1.39.0

* consensus/ethash: fix go vet warning regarding reflect.SliceHeader

* eth/catalyst: fix lint issue

* consensus/ethash: fix bug in memoryMapFile
2021-04-27 11:49:06 +03:00
Péter Szilágyi
1fb9a6dd32
eth/protocols, prp/tracker: add support for req/rep rtt tracking (#22608)
* eth/protocols, prp/tracker: add support for req/rep rtt tracking

* p2p/tracker: sanity cap the number of pending requests

* pap/tracker: linter <3

* p2p/tracker: disable entire tracker if no metrics are enabled
2021-04-22 11:42:46 +03:00
gary rong
96828c90f5
eth/tracers, internal/ethapi: fix typos causing lint issue (#22711) 2021-04-21 10:18:27 +02:00
gary rong
dd9c3225cf
eth, internal: extend the TraceCall API (#22245)
Adds an an optional parameter `overrides *map[common.Address]account` to the `eth_call` API in order for the caller to  can customize the state.
2021-04-21 09:21:22 +02:00
Guillaume Ballet
f79cce5de9
eth/catalyst: add catalyst API prototype (#22641)
This change adds the --catalyst flag, enabling an RPC API for eth2 integration.
In this initial version, catalyst mode also disables all peer-to-peer networking.

Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-16 21:29:22 +02:00
Péter Szilágyi
9553c98de8
eth/protocols/snap: use ephemeral channels to avoid cross-sync delveries 2021-04-15 21:16:54 +03:00
Péter Szilágyi
1e207342b5
all: make logs a bit easier on the eye to digest (#22665)
* all: add thousandths separators for big numbers on log messages

* p2p/sentry: drop accidental file

* common, log: add fast number formatter

* common, eth/protocols/snap: simplifty fancy num types

* log: handle nil big ints
2021-04-15 20:35:00 +03:00
gary rong
7088f1e814
core, eth: faster snapshot generation (#22504)
* eth/protocols: persist received state segments

* core: initial implementation

* core/state/snapshot: add tests

* core, eth: updates

* eth/protocols/snapshot: count flat state size

* core/state: add metrics

* core/state/snapshot: skip unnecessary deletion

* core/state/snapshot: rename

* core/state/snapshot: use the global batch

* core/state/snapshot: add logs and fix wiping

* core/state/snapshot: fix

* core/state/snapshot: save generation progress even if the batch is empty

* core/state/snapshot: fixes

* core/state/snapshot: fix initial account range length

* core/state/snapshot: fix initial account range

* eth/protocols/snap: store flat states during the healing

* eth/protocols/snap: print logs

* core/state/snapshot: refactor (#4)

* core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* core, eth: fixes

* core, eth: fix healing writer

* core, trie, eth: fix paths

* eth/protocols/snap: fix encoding

* eth, core: add debug log

* core/state/generate: release iterator asap (#5)

core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish

* core/state/snapshot: optimize stats counter

* core, eth: add metric

* core/state/snapshot: update comments

* core/state/snapshot: improve tests

* core/state/snapshot: replace secure trie with standard trie

* core/state/snapshot: wrap return as the struct

* core/state/snapshot: skip wiping correct states

* core/state/snapshot: updates

* core/state/snapshot: fixes

* core/state/snapshot: fix panic due to reference flaw in closure

* core/state/snapshot: fix errors in state generation logic + fix log output

* core/state/snapshot: remove an error case

* core/state/snapshot: fix condition-check for exhausted snap state

* core/state/snapshot: use stackTrie for small tries

* core/state/snapshot: don't resolve small storage tries in vain

* core/state/snapshot: properly clean up storage of deleted accounts

* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

* core/state/snapshot: fix error (+testcase)

* core/state/snapshot: clean up tests a bit

* core/state/snapshot: work in progress on better tests

* core/state/snapshot: polish code

* core/state/snapshot: fix trie iteration abortion trigger

* core/state/snapshot: fixes flaws

* core/state/snapshot: remove panic

* core/state/snapshot: fix abort

* core/state/snapshot: more tests (plus failing testcase)

* core/state/snapshot: more testcases + fix for failing test

* core/state/snapshot: testcase for malformed data

* core/state/snapshot: some test nitpicks

* core/state/snapshot: improvements to logging

* core/state/snapshot: testcase to demo error in abortion

* core/state/snapshot: fix abortion

* cmd/geth: make verify-state report the root

* trie: fix failing test

* core/state/snapshot: add timer metrics

* core/state/snapshot: fix metrics

* core/state/snapshot: udpate tests

* eth/protocols/snap: write snapshot account even if code or state is needed

* core/state/snapshot: fix diskmore check

* core/state/snapshot: review fixes

* core/state/snapshot: improve error message

* cmd/geth: rename 'error' to 'err' in logs

* core/state/snapshot: fix some review concerns

* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync

* core: add error log

* core/state/snapshot: use proper timers for metrics collection

* core/state/snapshot: address some review concerns

* eth/protocols/snap: improved log message

* eth/protocols/snap: fix heal logs to condense infos

* core/state/snapshot: wait for generator termination before restarting

* core/state/snapshot: revert timers to counters to track total time

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-14 23:23:11 +03:00
xD AKA Rapper King Of cn background diablo & revelations
a50251e6cb
eth/fetcher: avoid spurious timer events at startup (#22652)
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-14 12:44:32 +02:00
Péter Szilágyi
fe1586b094
eth, les: drop support for eth/64, fix eth/66 tests 2021-04-09 10:39:45 +03:00
Péter Szilágyi
2a7c4b62a9
eth: fix tracing state retrieval if requesting the non-dirty genesis 2021-04-07 18:35:25 +03:00
Marius van der Wijden
9d10856e84
core, eth, internal/ethapi: create access list RPC API (#22550)
* core/vm: implement AccessListTracer

* eth: implement debug.createAccessList

* core/vm: fixed nil panics in accessListTracer

* eth: better error messages for createAccessList

* eth: some fixes on CreateAccessList

* eth: allow for provided accesslists

* eth: pass accesslist by value

* eth: remove created acocunt from accesslist

* core/vm: simplify access list tracer

* core/vm: unexport accessListTracer

* eth: return best guess if al iteration times out

* eth: return best guess if al iteration times out

* core: docstring, unexport methods

* eth: typo

* internal/ethapi: move createAccessList to eth package

* internal/ethapi: remove reexec from createAccessList

* internal/ethapi: break if al is equal to last run, not if gas is equal

* internal/web3ext: fixed arguments

* core/types: fixed equality check for accesslist

* core/types: no hardcoded vals

* core, internal: simplify access list generation, make it precise

* core/vm: fix typo

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-07 17:54:31 +03:00
gary rong
a600dab7e5
eth, les: fix tracers (#22473)
* eth, les: fix tracer

* eth: isolate live trie database in tracer

* eth: fix nil

* eth: fix

* eth, les: add checkLive param

* eth/tracer: fix
2021-04-07 10:30:26 +03:00
piersy
706683ea72
internal/ethapi: fix eth_chainId method (#22243)
This removes the duplicated definition of eth_chainID
in package eth and updates the definition in internal/ethapi
to treat chain ID as a bigint.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-06 12:24:39 +02:00
Péter Szilágyi
55300d4fdb
all: fix miner hashRate -> hashrate on API calls 2021-03-31 10:56:51 +03:00
Martin Holst Swende
7644795950
eth/protocols/snap: try to prevent requests timing out 2021-03-29 14:17:35 +02:00
Péter Szilágyi
099be04100
eth/protocols/snap: add peer id and req id to the timeout logs 2021-03-26 22:29:22 +02:00
Felix Lange
cae6b5527e
cmd/geth, consensus/ethash: add support for --miner.notify.full flag (#22558)
The PR implements the --miner.notify.full flag that enables full pending block
notifications. When this flag is used, the block notifications sent to mining
endpoints contain the complete block header JSON instead of a work package
array.

Co-authored-by: AlexSSD7 <alexandersadovskyi7@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-03-26 18:30:10 +01:00
Zou Guangxian
955727181b
eth: fix corner case in sync head determination (#21695)
This avoids synchronisation failures when the local header is ahead of
the local full block.
2021-03-26 18:06:25 +01:00
Péter Szilágyi
2550e46269
eth/protocols, metrics: use resetting histograms for rare packets 2021-03-26 16:14:12 +02:00
Péter Szilágyi
6d7ff6acea
eth/protocols, metrics, p2p: add handler performance metrics 2021-03-26 14:00:06 +02:00
Marius van der Wijden
54c0d573d7
eth: dump rpc gas cap and tx fee cap (#22574) 2021-03-25 16:37:51 +02:00
Martin Holst Swende
0fda25e471
eth/tracers, core: use scopecontext in tracers, provide statedb in capturestart (#22333)
Fixes the CaptureStart api to include the EVM, thus being able to set the statedb early on. This pr also exposes the struct we used internally in the interpreter to encapsulate the contract, mem, stack, rstack, so we pass it as a single struct to the tracer, and removes the error returns on the capture methods.
2021-03-25 10:13:14 +01:00
gary rong
c5df05b9a9
eth/protocols/snap: fix the flaws in the snap sync (#22553)
* eth/protocols/snap: fix snap sync

* eth/protocols/snap: fix tests

* eth: fix tiny

* eth: update tests

* eth: update tests

* core/state/snapshot: testcase for #22534

* eth/protocols/snap: fix boundary loss on full-but-proven range

* core/state/snapshot: lintfix

* eth: address comment

* eth: fix handler

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-03-24 16:33:34 +02:00
Péter Szilágyi
477ec75323
Merge pull request #22513 from holiman/no_snap_bloom
eth/protocols/snap, eth/downloader: don't use bloom filter in snap sync
2021-03-23 11:37:32 +02:00
gary rong
0c70b83e00
all: add read-only option to database (#22407)
* all: add read-only option to database

* all: fixes tests

* cmd/geth: migrate flags

* cmd/geth: fix the compact

* cmd/geth: fix the format

* cmd/geth: fix log

* cmd: add chain-readonly

* core: add readonly notion to freezer

* core/rawdb: add log

* core/rawdb: fix freezer close

* cmd: fix

* cmd, core: construct db

* core: update tests
2021-03-22 19:06:30 +01:00
wuff1996
117fa7d4a1
eth/protocols/snap: fix typo (#22530) 2021-03-19 08:49:24 +01:00
Martin Holst Swende
410089afea
eth/protocols/snap, eth/downloader: don't use bloom filter in snap sync 2021-03-17 09:36:34 +01:00
Martin Holst Swende
faacc8e0fa
cmd/geth, eth/downloader: remove copydb command (#22501)
* cmd/geth: remove copydb command

* eth/downloader: remove fakepeer
2021-03-15 16:25:45 +02:00
Martin Holst Swende
c6d45009f1
eth, les: properly init statedb accesslist during tracing (#22480)
* eth/state, les/state: properly init statedb accesslist when tracing, fixes #22475

* eth: review comments

* eth/tracers: fix compilation err

* eth/tracers: apply @karalabe's suggested fix
2021-03-14 17:13:25 +02:00
Péter Szilágyi
7834e4a278
core, eth: unship EIP 2315 2021-03-02 23:40:57 +02:00
Péter Szilágyi
0928562670
all: define Berlin hard fork spec 2021-02-26 14:24:07 +02:00
lightclient
bbfb1e4008
all: add support for EIP-2718, EIP-2930 transactions (#21502)
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.

There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.

The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID. 

This change also adds support for the YoloV3 testnet.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
2021-02-25 15:26:57 +01:00
gary rong
378e961d85
cmd, eth, les: enable serving light clients when non-synced (#22250)
This PR adds a more CLI flag, so that the les-server can serve light clients even the local node is not synced yet.

This functionality is needed in some testing environments(e.g. hive). After launching the les server, no more blocks will be imported so the node is always marked as "non-synced".
2021-02-25 13:55:07 +01:00
Péter Szilágyi
b2b5c82aca
eth/protocols/snap: lower abortion and resumption logs to debug 2021-02-25 12:56:18 +02:00
Marius van der Wijden
142fbcfd6f
internal/ethapi: reject non-replay-protected txs over RPC (#22339)
This PR prevents users from submitting transactions without EIP-155 enabled. This behaviour can be overridden by specifying the flag --rpc.allow-unprotected-txs=true.
2021-02-23 13:09:19 +01:00
Martin Holst Swende
6ec1561044
eth: implement eth66 (#22241)
* eth/protocols/eth: split up the eth protocol handlers

* eth/protocols/eth: define eth-66 protocol messages

* eth/protocols/eth: poc implement getblockheaders on eth/66

* eth/protocols/eth: implement remaining eth-66 handlers

* eth/protocols: define handler map for eth 66

* eth/downloader: use protocol constants from eth package

* eth/protocols/eth: add ETH66 capability

* eth/downloader: tests for eth66

* eth/downloader: fix error in tests

* eth/protocols/eth: use eth66 for outgoing requests

* eth/protocols/eth: remove unused error type

* eth/protocols/eth: define protocol length

* eth/protocols/eth: fix pooled tx over eth66

* protocols/eth/handlers: revert behavioural change which caused tests to fail

* eth/downloader: fix failing test

* eth/protocols/eth: add testcases + fix flaw with header requests

* eth/protocols: change comments

* eth/protocols/eth: review fixes + fixed flaw in RequestOneHeader

* eth/protocols: documentation

* eth/protocols/eth: review concerns about types
2021-02-18 18:54:29 +02:00
Martin Holst Swende
e01096f531
eth/handler, broadcast: optimize tx broadcast mechanism (#22176)
This PR optimizes the broadcast loop. Instead of iterating twice through a given set of transactions to weed out which peers have and which do not have a tx, to send/announce transactions, we do it only once.
2021-02-17 14:59:00 +01:00
Péter Szilágyi
bfdff4c5b8
eth: fix snap sync cancellation 2021-02-16 16:11:33 +02:00
Péter Szilágyi
409b16e5ab
cmd/utils, eth/ethconfig: unindex txs older than ~1 year 2021-02-10 16:01:37 +02:00
gary rong
f566dd305e
all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-02-08 13:16:30 +02:00
Martin Holst Swende
7ed860d4f1
eth: don't wait for snap registration if we're not running snap (#22272)
Prevents a situation where we (not running snap) connects with a peer running snap, and get stalled waiting for snap registration to succeed (which will never happen), which cause a waitgroup wait to halt shutdown
2021-02-05 14:15:22 +01:00
gary rong
098a2b6e26
eth: move eth.Config to a common package (#22205)
This moves the eth config definition into a separate package, eth/ethconfig. 
Packages eth and les can now import this common package instead of
importing eth from les, reducing dependencies.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-02-05 13:51:15 +01:00
ucwong
83e4c49e2b
trie : use trie.NewStackTrie instead of new(trie.Trie) (#22246)
The PR makes use of the stacktrie, which is is more lenient on resource consumption, than the regular trie, in cases where we only need it for DeriveSha
2021-02-02 13:09:23 +01:00
Alex Prut
ef84da8481
all: remove unneeded parentheses (#21921)
* remove uneeded convertion type

* remove redundant type in composite literal

* omit explicit type where implicit

* remove unused redundant parenthesis

* remove redundant import alias duktape
2021-02-02 11:32:44 +02:00
Péter Szilágyi
e3430ac7df
eth: check snap satelliteness, delegate drop to eth (#22235)
* eth: check snap satelliteness, delegate drop to eth

* eth: better handle eth/snap satellite relation, merge reg/unreg paths
2021-02-02 10:44:36 +02:00
Sina Mahmoodi
3c728fb129
eth/tracers: fix unigram tracer (#22248) 2021-02-01 14:41:43 +01:00
Sina Mahmoodi
eb21c652c0
cmd,core,eth,params,tests: define yolov3 + enable EIP-2565 (#22213)
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3. 
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
2021-01-28 21:19:07 +01:00
Martin Holst Swende
d2779ed7ac
eth, p2p: reserve half peer slots for snap peers during snap sync (#22171)
* eth, p2p: reserve half peer slots for snap peers during snap sync

* eth: less logging

* eth: rework the eth/snap peer reservation logic

* eth: rework the eth/snap peer reservation logic (again)
2021-01-25 20:06:52 +02:00
gary rong
adf130def8
eth/tracers: move tracing APIs into eth/tracers (#22161)
This moves the tracing RPC API implementation to package eth/tracers.
By doing so, package eth no longer depends on tracing and the duktape JS engine.

The change also enables tracing using the light client. All tracing methods work with the
light client, but it's a lot slower compared to using a full node.
2021-01-25 14:36:39 +01:00
Martin Holst Swende
797b0812ab
eth/protocols/snap: snap sync testing (#22179)
* eth/protocols/snap: make timeout configurable

* eth/protocols/snap: snap sync testing

* eth/protocols/snap: test to trigger panic

* eth/protocols/snap: fix race condition on timeouts

* eth/protocols/snap: return error on cancelled sync

* squashme: updates + test causing panic + properly serve accounts in order

* eth/protocols/snap: revert failing storage response

* eth/protocols/snap: revert on bad responses (storage, code)

* eth/protocols/snap: fix account handling stall

* eth/protocols/snap: fix remaining revertal-issues

* eth/protocols/snap: timeouthandler for bytecode requests

* eth/protocols/snap: debugging + fix log message

* eth/protocols/snap: fix misspelliings in docs

* eth/protocols/snap: fix race in bytecode handling

* eth/protocols/snap: undo deduplication of storage roots

* synctests: refactor + minify panic testcase

* eth/protocols/snap: minor polishes

* eth: minor polishes to make logs more useful

* eth/protocols/snap: remove excessive logs from the test runs

* eth/protocols/snap: stress tests with concurrency

* eth/protocols/snap: further fixes to test cancel channel handling

* eth/protocols/snap: extend test timeouts on CI

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-01-25 08:17:05 +02:00
Sina Mahmoodi
c4307a9339
eth/filters: fix potential deadlock in filter timeout loop (#22178)
This fixes #22131 and adds a test reproducing the issue.
2021-01-21 12:17:10 +01:00
Péter Szilágyi
ddadc3d273
Merge pull request #21047 from holiman/improve_updates_2
core: improve trie updates (part 2)
2021-01-21 01:48:08 +02:00
Péter Szilágyi
42f9f1f073
core/state: convert prefetcher to concurrent per-trie loader 2021-01-21 01:47:14 +02:00
meowsbits
81bf9f97c9
downloader: extract findAncestor search functions (#21744)
This is a simple refactoring, extracting common ancestor
negotiation logic to named function
2021-01-20 22:45:01 +01:00
gary rong
45cb1a580a
eth, les: add new config field SyncFromCheckpoint (#22123)
This PR introduces a new config field SyncFromCheckpoint for light client.

In some special scenarios, it's required to start synchronization from some
arbitrary checkpoint or even from the scratch. So this PR offers this
flexibility to users so that the synchronization start point can be configured.

There are two relevant configs: SyncFromCheckpoint and Checkpoint.

- If the SyncFromCheckpoint is true, the light client will try to sync from the
  specified checkpoint.

- If the Checkpoint is not configured, then the light client will sync from the
  scratch(from the latest header if the database is not empty)

Additional notes: these two configs are not visible in the CLI flags but only
accessable in the config file.

Example Usage:

[Eth]
SyncFromCheckpoint = true

[Eth.Checkpoint]
SectionIndex = 100
SectionHead = "0xabc"
CHTRoot = "0xabc"
BloomRoot = "0xabc"

PS. Historical checkpoint can be retrieved from the synced full node or light
client via les_getCheckpoint API.
2021-01-19 10:52:45 +01:00
Dan DeGreef
c76573a97b
eth/protocols/eth: fix slice resize flaw (#22181) 2021-01-16 18:15:18 +01:00
rene
984e752ce5
eth: return error from eth_chainID during sync before EIP-155 activates (#21686)
This changes the chainID RPC method to return an error when EIP-155 is not yet
active at the current block height. It used to simply return zero in this case, but
that's confusing.
2021-01-12 10:52:13 +01:00
Marius van der Wijden
49c2816d54
eth: improve log message (#22146)
* eth: fixed typos

* eth: fixed log message
2021-01-11 12:53:13 +01:00
gary rong
5a1b384352
core: persist bad blocks (#21827)
* core: persist bad blocks

* core, eth, internal: address comments

* core/rawdb: add badblocks to inspector

* core, eth: update

* internal: revert

* core, eth: only save 10 bad blocks

* core/rawdb: address comments

* core/rawdb: fix

* core: address comments
2021-01-10 12:54:15 +01:00
Chris Ziogas
89030ec0b4
eth/downloader: fix race condition in tests (#22140)
* downloader: fix race condition in tests

* eth/downloader: fix race condition in tests

* Revert "downloader: fix race condition in tests"

This reverts commit 108033ebc6985de83791d375b6e6647a77d28d5a.
2021-01-09 17:29:19 +01:00
Martin Holst Swende
4bb5c6ca7a
eth/protocols/snap: speed up hash checks (#22023)
* eth/protocols/snap: speed up hash checks

* eth/protocols/snap: nit fix

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-01-07 18:12:41 +02:00
Martin Holst Swende
58b9db5f7c
eth/protocols/snap: track reverts when peer rejects request (#22016)
* eth/protocols/snap: reschedule missed deliveries

* eth/protocols/snap: clarify log message

* eth/protocols/snap: revert failures async and update runloop

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-01-07 12:58:07 +02:00
Péter Szilágyi
83d317cff9
cmd/utils, eth/downloader: minor snap nitpicks 2021-01-06 08:37:45 +02:00
jk-jeongkyun
618454214b
eth/downloader: enhanced test cases for downloader queue (#22114) 2021-01-05 14:56:01 +01:00
ucwong
a425a47ddc
core/rawdb, eth/protocols : Method name typo fix (#22026) 2020-12-27 22:38:16 +01:00
jk-jeongkyun
653e8b9dd9
eth/downloader: remove unnecessary condition (#22052) 2020-12-27 22:26:42 +01:00
Suriyaa Sundararuban
0a09a39325
eth/filters: replace wiki links with new doc pages (#22070) 2020-12-27 22:09:05 +01:00
Sina Mahmoodi
9c6b5b904a
eth, eth/tracers: include intrinsic gas in calltracer, expose for all tracers (#22038)
* eth/tracers: share tx gas price with js tracer

* eth/tracers: use `go generate`

* eth/tracers: try with another version of go-bindata

* eth/tracers: export txGas

* eth, eth/tracers: pass intrinsic gas to js tracers

eth/tracers: include tx gas in tracers usedGas

eth/tracers: fix prestate tracer's sender balance

eth/tracers: rm unnecessary import

eth/tracers: pass intrinsicGas separately to tracer

eth/tracers: fix tests broken by lack of txdata

eth, eth/tracers: minor fix

* eth/tracers: regenerate assets + unexport test-struct + add testcase

* eth/tracers: simplify tests + make table-driven

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-12-27 21:57:19 +01:00
Marius van der Wijden
158f72cc0c
internal/ethapi: restore net_version RPC method (#22061)
During the snap and eth refactor, the net_version rpc call was falsely deprecated.
This restores the net_version RPC handler as most eth2 nodes and other software
depend on it.
2020-12-23 13:43:22 +01:00
ucwong
61469cfeaf
eth/downloader: fix typo in comment (#22019) 2020-12-21 15:39:58 +01:00