Commit Graph

1951 Commits

Author SHA1 Message Date
Marius van der Wijden
667e1c038e
core, params: add kiln flag (#24548) 2022-03-17 17:34:22 +02:00
Marius van der Wijden
4f4622bc8b
core: eth: implement Kiln-v2 spec (#24506)
* core/beacon: eth/catalyst: updated engine api to new version

* core: implement exchangeTransitionConfig

* core/beacon: prevRandao instead of Random

* eth/catalyst: Fix ExchangeTransitionConfig, add test

* eth/catalyst: stop external miners on TTD reached

* node: implement --authrpc.vhosts flag

* core: allow for config override on non-mainnet networks

* eth/catalyst: fix peters comments

* eth/catalyst: make stop remote sealer more explicit

* eth/catalyst: add log output

* cmd/utils: rename authrpc.host to authrpc.addr

* eth/catalyst: disable the disabling of the miner

* eth: core: remove notion of terminal pow block

* eth: les: more of peters nitpicks
2022-03-17 17:20:03 +02:00
aaronbuchwald
c3701b265e
core/rawdb: add specified key length iterator (#24535) 2022-03-15 10:28:26 +02:00
Péter Szilágyi
70da74e73a
Merge pull request #24515 from karalabe/pending-statedb-accesslist-init
core/statedb: always clear out access list when setting a new one
2022-03-14 10:51:32 +02:00
Péter Szilágyi
8f66ea3786
eth/downloader: implement beacon sync (#23982)
* eth/downloader: implement beacon sync

* eth/downloader: fix a crash if the beacon chain is reduced in length

* eth/downloader: fix beacon sync start/stop thrashing data race

* eth/downloader: use a non-nil pivot even in degenerate sync requests

* eth/downloader: don't touch internal state on beacon Head retrieval

* eth/downloader: fix spelling mistakes

* eth/downloader: fix some typos

* eth: integrate legacy/beacon sync switchover and UX

* eth: handle UX wise being stuck on post-merge TTD

* core, eth: integrate the beacon client with the beacon sync

* eth/catalyst: make some warning messages nicer

* eth/downloader: remove Ethereum 1&2 notions in favor of merge

* core/beacon, eth: clean up engine API returns a bit

* eth/downloader: add skeleton extension tests

* eth/catalyst: keep non-kiln spec, handle mining on ttd

* eth/downloader: add beacon header retrieval tests

* eth: fixed spelling, commented failing tests out

* eth/downloader: review fixes

* eth/downloader: drop peers failing to deliver beacon headers

* core/rawdb: track beacon sync data in db inspect

* eth: fix review concerns

* internal/web3ext: nit

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-11 14:14:45 +02:00
rjl493456442
538a868384
core/rawdb, cmd, ethdb, eth: implement freezer tail deletion (#23954)
* core/rawdb, cmd, ethdb, eth: implement freezer tail deletion

* core/rawdb: address comments from martin and sina

* core/rawdb: fixes cornercase in tail deletion

* core/rawdb: separate metadata into a standalone file

* core/rawdb: remove unused code

* core/rawdb: add random test

* core/rawdb: polish code

* core/rawdb: fsync meta file before manipulating the index

* core/rawdb: fix typo

* core/rawdb: address comments
2022-03-10 09:37:23 +01:00
rjl493456442
8c8a9e5ca1
core, ethdb, tests, trie: introduce database snapshot (#24486) 2022-03-10 09:35:22 +01:00
Justus
d1f6a9f544
core/types: improve error for too short transaction / receipt encoding (#24256)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 10:44:53 +01:00
Ceyhun Onur
8401e4277a
core/rawdb: add HasCode, HashTrieNode and use them where possible (#24454) 2022-03-09 00:39:34 +01:00
Péter Szilágyi
48605b5f61
core/statedb: always clear out access list when setting a new one 2022-03-08 12:00:29 +02:00
Felix Lange
d6f49bf764
core/types: faster RLP encoding of Header, StateAcccount, ReceiptForStorage (#24420)
This change makes use of the new code generator rlp/rlpgen to improve the
performance of RLP encoding for Header and StateAccount. It also speeds up
encoding of ReceiptForStorage using the new rlp.EncoderBuffer API.

The change is much less transparent than I wanted it to be, because Header and
StateAccount now have an EncodeRLP method defined with pointer receiver. It
used to be possible to encode non-pointer values of these types, but the new
method prevents that and attempting to encode unadressable values (even if
part of another value) will return an error. The error can be surprising and may
pop up in places that previously didn't expect any errors.

To make things work, I also needed to update all code paths (mostly in unit tests)
that lead to encoding of non-pointer values, and pass a pointer instead.

Benchmark results:

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           328ns ± 0%     237ns ± 1%   -27.63%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8           353ns ± 0%     247ns ± 1%   -30.06%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     237ns ± 0%     123ns ± 0%   -47.86%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8            297ns ± 0%     301ns ± 1%    +1.39%  (p=0.000 n=8+8)

    name                             old speed      new speed      delta
    EncodeRLP/legacy-header-8        1.66GB/s ± 0%  2.29GB/s ± 1%   +38.19%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8        1.55GB/s ± 0%  2.22GB/s ± 1%   +42.99%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8  38.0MB/s ± 0%  64.8MB/s ± 0%   +70.48%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8          910MB/s ± 0%   897MB/s ± 1%    -1.37%  (p=0.000 n=8+8)

    name                             old alloc/op   new alloc/op   delta
    EncodeRLP/legacy-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/london-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/receipt-for-storage-8     64.0B ± 0%      0.0B       -100.00%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8             320B ± 0%      320B ± 0%      ~     (all equal)
2022-02-18 08:10:26 +01:00
rjl493456442
4d086430bd
core, ethdb, tests, trie: implement NewBatchWithSize API for batcher (#24392)
This PR adds an addtional API called `NewBatchWithSize` for db
batcher. It turns out that leveldb batch memory allocation is
super inefficient. The main reason is the allocation step of
leveldb Batch is too small when the batch size is large. It can
take a few second to build a leveldb batch with 100MB size.

Luckily, leveldb also offers another API called MakeBatch which can
pre-allocate the memory area. So if the approximate size of batch is
known in advance, this API can be used in this case.

It's needed in new state scheme PR which needs to commit a batch of
trie nodes in a single batch. Implement the feature in a seperate PR.
2022-02-15 15:15:13 +02:00
Martin Holst Swende
51e7968b8b
core/state: fix read-meters + simplify code (#24304) 2022-02-14 09:22:57 +01:00
rjl493456442
9da25c5db7
all: separate catalyst package (#24280)
* all: seperate catalyst package

* eth/catalyst: moved some methods, added docs

* eth/catalyst, les/catalyst: add method docs

* core, eth, les, miner: move common function to beacon package

* eth/catalyst: goimported

* cmd/utils, miner/stress/beacon: naming nitpicks

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-31 14:22:35 +02:00
Marius van der Wijden
29cb5deea3
core/rawdb: fix typo (#24289) 2022-01-25 11:36:51 +01:00
Martin Holst Swende
0e35192797
core/rawdb: do prefixed lookup first 2022-01-25 10:51:18 +01:00
Shihao Xia
eef7a33135
core, miner, rpc, eth: fix goroutine leaks in tests (#24211)
* fix blocking and non-blocking issues

* core: revert change in blockchain.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-21 12:22:44 +01:00
Di Peng
c029cdc90b
core: fix typo in blockchain test (#24263) 2022-01-21 09:12:40 +01:00
Martin Holst Swende
51eb5f8ca8
cmd/geth: add db cmd to show metadata (#23900)
* cmd/geth: add db cmd to show metadata

* cmd/geth: better output generator status

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* cmd: minor

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2022-01-18 12:30:41 +02:00
Sina Mahmoodi
4aab440ee2
core/rawdb: enforce readonly in freezer instantiation (#24119)
* freezer: add readonly flag to table

* freezer: enforce readonly in table repair

* freezer: enforce readonly in newFreezer

* minor fix

* minor

* core/rawdb: test that writing during readonly fails

* rm unused log

* check readonly on batch append

* minor

* Revert "check readonly on batch append"

This reverts commit 2ddb5ec4ba7534bf6edbdfec158ea99a2eed5036.

* review fixes

* minor test refactor

* attempt at fixing windows issue

* add comment re windows sync issue

* k->kind

* open readonly db for genesis check

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-18 11:29:38 +02:00
Péter Szilágyi
acd7b36999
Merge pull request #24197 from rjl493456442/periodically-flush-batch
core: periodically flush the transaction indexes
2022-01-11 11:32:43 +02:00
Marius van der Wijden
b1e72f7ea9
core/evm: RANDOM opcode (EIP-4399) (#24141)
* core: implement eip-4399 random opcode

* core: make vmconfig threadsafe

* core: miner: pass vmConfig by value not reference

* all: enable 4399 by Rules

* core: remove diff (f)

* tests: set proper difficulty (f)

* smaller diff (f)

* eth/catalyst: nit

* core: make RANDOM a pointer which is only set post-merge

* cmd/evm/internal/t8ntool: fix t8n tracing of 4399

* tests: set difficulty

* cmd/evm/internal/t8ntool: check that baserules are london before applying the merge chainrules
2022-01-10 09:44:21 +02:00
Sam
7403a38ab7
core: fix a typo (#24198) 2022-01-06 16:02:23 +02:00
Péter Szilágyi
af2ca5a654
Merge pull request #24117 from holiman/db_has
trie, core, eth: use db.has over db.get where possible
2022-01-06 11:30:11 +02:00
Gary Rong
4bd2d0eccf core: periodically flush the transaction indexes 2022-01-05 15:00:03 +08:00
Martin Holst Swende
66a908c5e8
core/rawdb: fix double-lock causing hang (#24189)
Fixes #24159

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-01-04 19:02:37 +01:00
zgfzgf
356bbe343a
core/asm: change order of items in stringtokenTypes (#24153)
This orders the items in slice definition same as the enum values.
2021-12-26 14:58:17 +01:00
Felix Lange
980b7682b4
core/types: document JSON field name equivalents of DynamicFeeTx (#24143) 2021-12-22 15:32:17 +01:00
xwjack
8bbf83e7a4
core: ignore basefee when comparing with pool gasprice in txpool (#24080)
This reverts commit 9489853321.
2021-12-17 15:44:57 +01:00
Marius van der Wijden
2295640ebd
eth/catalyst: implement kintsugi-spec v3 (#24067) 2021-12-17 15:38:58 +01:00
Sina Mahmoodi
ada9c774e9
eth, les: update unclean shutdown markers regularly (#24077)
Fixes #22580

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-17 15:18:51 +01:00
Andrei Maiboroda
3e47e38a4e
core/vm: Make INVALID a defined opcode (#24017)
* core/vm: Define 0xfe opcode as INVALID

* core/vm: Remove opInvalid as opUndefined handles it

Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>
2021-12-17 13:44:05 +01:00
Paweł Bylica
81ec6b1d4c
core/vm: reverse bit order in bytes of code bitmap (#24120)
* core/vm: reverse bit order in bytes of code bitmap

This bit order is more natural for bit manipulation operations and we
can eliminate some small number of CPU instructions.

* core/vm: drop lookup table
2021-12-17 10:32:00 +01:00
Martin Holst Swende
893502e561
trie, core, eth: use db.has over db.get where possible 2021-12-15 16:16:45 +01:00
Martin Holst Swende
155795be99
core/vm: avoid memory expansion check for trivial ops (#24048) 2021-12-14 11:30:20 +01:00
ucwong
adec878c1d
core/rawdb: fix error message fields in ReadLogs (#24104) 2021-12-14 11:11:05 +01:00
Alex Beregszaszi
ae8ff2661d
core/vm: remove unused code (IsStaticJump) (#24085) 2021-12-09 13:55:06 +01:00
Martin Holst Swende
db03faa10d
core, eth: improve delivery speed on header requests (#23105)
This PR reduces the amount of work we do when answering header queries, e.g. when a peer
is syncing from us.

For some items, e.g block bodies, when we read the rlp-data from database, we plug it
directly into the response package. We didn't do that for headers, but instead read
headers-rlp, decode to types.Header, and re-encode to rlp. This PR changes that to keep it
in RLP-form as much as possible. When a node is syncing from us, it typically requests 192
contiguous headers. On master it has the following effect:

- For headers not in ancient: 2 db lookups. One for translating hash->number (even though
  the request is by number), and another for reading by hash (this latter one is sometimes
  cached).
  
- For headers in ancient: 1 file lookup/syscall for translating hash->number (even though
  the request is by number), and another for reading the header itself. After this, it
  also performes a hashing of the header, to ensure that the hash is what it expected. In
  this PR, I instead move the logic for "give me a sequence of blocks" into the lower
  layers, where the database can determine how and what to read from leveldb and/or
  ancients.

There are basically four types of requests; three of them are improved this way. The
fourth, by hash going backwards, is more tricky to optimize. However, since we know that
the gap is 0, we can look up by the parentHash, and stlil shave off all the number->hash
lookups.

The gapped collection can be optimized similarly, as a follow-up, at least in three out of
four cases.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-07 17:50:58 +01:00
Péter Szilágyi
a25906e4c0
Merge pull request #24066 from holiman/no_no_recursion
core/vm: remove no-recursion option from config
2021-12-06 12:47:25 +02:00
rjl493456442
69686fa328
core, core/rawdb: fix transaction indexing (#24024)
This PR fixes a special corner case in transaction indexing.

When the chain is rewound by SetHead to a historical point which is even lower than the transaction indexes tail, then system will report Failed to decode block body error all the time, because the relevant blocks are already deleted.

In order to avoid this "non-critical-but-annoying" issue, we can recap the indexing target to head+1(to is excluded, so it means indexing transactions from 0 to head).
2021-12-06 11:26:03 +01:00
Martin Holst Swende
a95675d50f
core/vm: remove no-recursion option from config 2021-12-06 09:55:50 +01:00
Péter Szilágyi
cca482b4b1
Merge pull request #24047 from karalabe/delete-dead-sync-bloom
core, eth, les, trie: remove the sync bloom, used by fast sync
2021-12-03 13:55:17 +02:00
Péter Szilágyi
58d1988349
core, eth, les, trie: remove the sync bloom, used by fast sync 2021-12-03 12:32:41 +02:00
Andrei Maiboroda
b02fe5317f
core/vm: move interpreter interruption check to jump instructions (#24026)
* core/vm: Remove interpreter loop interruption check

* core/vm: Unit test for interpreter loop interruption

* core/vm: Check for interpreter loop abort on every jump
2021-12-03 11:10:26 +01:00
Paweł Bylica
9331fe28e8
core/vm: fill gaps in jump table with opUndefined (#24031) 2021-12-03 11:04:54 +01:00
Alex Beregszaszi
a0f7771962
core/vm: remove stack.pushN (#24040) 2021-12-03 10:16:49 +01:00
Taeik Lim
85064ed09b
all: fix 'the the' in comments (#24036) 2021-12-02 15:42:09 +01:00
Alex Beregszaszi
2be129b5cf
core/vm: rename opSuicide to opSelfdestruct (#24022)
The opcode was renamed in the codebase in 2017, but the functions were kept unchanged.
2021-12-01 10:33:29 +01:00
Alex Beregszaszi
9393d1fb5d
core/vm: Move interpreter.ReadOnly check into the opcode implementations (#23970)
* core/vm: Move interpreter.ReadOnly check into the opcode implementations

Also remove the same check from the interpreter inner loop.

* core/vm: Remove obsolete operation.writes flag

* core/vm: Capture fault states in logger

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core/vm: Remove panic added for testing

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-12-01 10:21:21 +01:00
Andrei Maiboroda
163f1665dd
core/vm: don't copy JumpTable when no EIP mods are needed (#23977) 2021-11-30 14:21:40 +02:00