Commit Graph

2324 Commits

Author SHA1 Message Date
rjl493456442
a7842c9cae
core, trie: cleanup trie database (#28062) 2023-09-07 21:17:14 +08:00
Delweng
2e02c1ffd9
core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
lightclient
25733a4aad
params: update 4844 parameters (#28026)
On ACD 163, it was agreed to bump the target and max blob values from `2/4` to `3/6` for future devnets until we could decide on final mainnet number. This change contains said update, making master pass all the hive tests. The final decision for mainnet cancun is still to be made.
---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-09-05 04:49:51 -04:00
lightclient
eff7c3bda0
core/forkid: skip genesis forks by time (#28034)
* core/forkid: skip genesis forks by time

* core/forkid: add comment about skipping non-zero fork times

* core/forkid: skip all time based forks in genesis using loop

* core/forkid: simplify logic for dropping time-based forks
2023-09-04 09:32:14 -04:00
rjl493456442
0acc0a1f86
core/state: simplify storage trie update and commit (#28030)
This change improves function description and simplifies logic in statedb update and commit operations.
2023-08-31 14:33:18 -04:00
Jorge
53f3c2ae65
metrics, cmd/geth: informational metrics (prometheus, influxdb, opentsb) (#24877)
This chang creates a GaugeInfo metrics type for registering informational (textual) metrics, e.g. geth version number. It also improves the testing for backend-exporters, and uses a shared subpackage in 'internal' to provide sample datasets and ordered registry. 

Implements #21783

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-31 13:37:17 -04:00
lightclient
41ee96fdfe
core/txpool/blobpool: fix rlp decoding flaw during offload (#28027) 2023-08-30 03:28:03 -04:00
Martin Holst Swende
d4e345c7d4
core/state: fix missing import (#28010) 2023-08-26 04:43:36 -04:00
rjl493456442
3ff6b3c31e
core/state: implement fast storage deletion (#27955)
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema. 

For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-26 04:13:22 -04:00
Martin Holst Swende
6aa88ccdd2
beacon/engine, eth/catalyst, miner: EIP-4788 CL/EL protocol updates (#27872)
This PR makes EIP-4788 work in the engine API and miner. It also fixes some bugs related to 
EIP-4844 block processing and mining. Changes in detail:

- Header.BeaconRoot has been renamed to ParentBeaconRoot.
- The engine API now implements forkchoiceUpdatedV3
- newPayloadV3 method has been updated with the parentBeaconBlockRoot parameter
- beacon root is now applied to new blocks in miner
- For EIP-4844, block creation now updates the blobGasUsed field of the header
2023-08-26 04:52:12 +02:00
Felix Lange
9bbb9df185
core/types: transaction and receipt encoding/decoding optimizations (#27976)
Just some minor optimizations I figured out a while ago. By using ReadBytes instead of
Bytes on the rlp stream, we can save the allocation of a temporary buffer for the typed tx
payload.

If kind == rlp.Byte, the size reported by Stream.Kind will be zero, but we need a buffer
of size 1 for ReadBytes. Since typed txs always have to be longer than 1 byte, we can just
return an error for kind == rlp.Byte.

There is a also a small change for Log: since the first three fields of Log are the ones that 
should appear in the canon encoding, we can simply ignore the remaining fields via 
struct tag. Doing this removes an indirection through the rlpLog type.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-25 20:03:41 +02:00
Péter Szilágyi
6b98d18789
cmd, core, params: add support for the Holesky testnet (#28007)
* cmd, core, params: add support for the Holesky testnet

* cmd/devp2p: add support for holesky for the dns crawler
2023-08-25 18:11:40 +03:00
Felix Lange
6dc9cdf15b
core: support null balance in genesis (#28005) 2023-08-25 15:59:40 +02:00
aaronbuchwald
56d2366699
core/state/snapshot: replace diffToDisk ideal batch size with 64MB (#27977) 2023-08-25 15:48:10 +03:00
Guido Vranken
0ba2d3cfa4
core/vm/runtime: Add Random field to config (#28001) 2023-08-25 04:05:33 -04:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
lightclient
feb8f416ac
miner: add to build block with EIP-4844 blobs (#27875)
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 23:16:14 +02:00
Felix Lange
d1f6735171
core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Martin Holst Swende
eb6cbe37e1
core/state: remove public method ForEachStorage (#27986)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 21:53:31 +02:00
Marius van der Wijden
2f4dbb4f90
core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende
f0f8703bf2
core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi
00fead91c4
cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
Péter Szilágyi
0c6bbeb423
core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
Felix Lange
4c7053baf1
core: add BeaconRoot to header in GenerateChain (#27974)
Fixes an error in tests of internal/ethapi.
2023-08-22 18:29:17 +02:00
Martin Holst Swende
b8d38e76ef
core, params, beacon/engine: implement EIP 4788 BeaconRoot (#27849)
This change implements "EIP 4788 : Beacon block root in the EVM". It implements version-2 of EPI-4788, main difference being that the contract is an actual contract rather than a precompile, as in #27289.
2023-08-22 08:33:39 -04:00
Delweng
7c0d90c8c9
core: ensure txindex will be triggered at least once (#27847)
Currently, we trigger the logic to (un)index transactions when the node receives a new
block. However, in some cases the node may not receive new blocks (eg, when the Geth node
is configured without peer discovery, or when it acts as an RPC node for historical-only
data).

In these situations, the Geth node user may not have previously configured txlookuplimit
(i.e. the default of around one year), but later realizes they need to index all
historical blocks. However, adding txlookuplimit=0 and restarting geth has no effect. This
change makes it check for required indexing work once, on startup, to fix the issue.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:58:20 +02:00
Delweng
9f4a528793
internal/ethapi: add testcases for blobTx (#27818)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:53:29 +02:00
Martin Holst Swende
1aa5520d75
core/txpool/legacypool: protect cache with mutex (#27898)
This change fixes the a potential race by using mutexes when the m.cache is read or modified.
2023-08-17 05:22:18 -04:00
lightclient
32fde3f838
core/forkid: correctly compute forkid when timestamp fork is activated in genesis (#27895)
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:

> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
2023-08-16 23:31:02 +02:00
lightclient
a3e35414b7
core/types: fix unmarshalling of BlobTx values (#27939)
FromBig returns true *when overflow occurs*
2023-08-16 23:22:15 +02:00
Felix Lange
2a6beb6a39
core/types: support for optional blob sidecar in BlobTx (#27841)
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.

However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:

- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
  - Note there is a special case here: when transactions are re-added because of a chain
    reorg, we cannot use the transactions gathered from the old chain blocks as-is,
    because they will be missing their blobs. This was previously handled by storing the
    blobs into the 'blobpool limbo'. The code has now changed to store the full
    transaction in the limbo instead, but it might be confusing for code readers why we're
    not simply adding the types.Transaction we already have.

Code changes summary:

- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
2023-08-14 10:13:34 +02:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
DongXi Huang
80b76a9527
core/types: remove duplication in eip2930 signer (#27860)
Remove duplication in signer
---------

Co-authored-by: GDdark <huangdongxi@minijoy.work>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-11 09:50:57 -04:00
imulmat4
6ddb92cac3
core/txpool/blobpool: fix metrics name for prometheus export (#27901) 2023-08-11 10:12:26 +03:00
rjl493456442
503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Martin Holst Swende
8d1db1601d
eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-08-10 06:49:05 -04:00
Guillaume Ballet
67979022aa
core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-08-09 18:02:45 +02:00
Martin Holst Swende
e13fa32cea
core/vm: update 4844 - point evaluation precompile address (#27874) 2023-08-08 15:54:19 +03:00
Felix Lange
bb148dd342
core/types: support yParity field in JSON transactions (#27744)
This adds support for the "yParity" field in transaction objects returned by RPC
APIs. We somehow forgot to add this field even though it has been in the spec for
a long time.
2023-08-05 00:54:55 +02:00
Darioush Jalali
57cdbaef30
all: remove trailing whitespace (#27741) 2023-08-05 00:24:32 +02:00
Felix Lange
df544350bc
core/types: fix immutability guarantees in Block (#27844)
This change rearranges the accessor methods in block.go and fixes some minor issues with
the copy-on-write logic of block data. Fixed issues:

- Block.WithWithdrawals did not create a shallow copy of the block.

- Block.WithBody copied the header unnecessarily, and did not preserve withdrawals.

However, the bugs did not affect any code in go-ethereum because blocks are *always*
created using NewBlockWithHeader().WithBody().WithWithdrawals()
2023-08-04 14:16:23 +02:00
rjl493456442
7de748d3f6
all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
Delweng
7c95ebd63d
consensus/misc: move eip1559 into a package (#27828)
* consensus/misc: move eip1559 as a sub directory

Signed-off-by: jsvisa <delweng@gmail.com>

* consensus/misc: package name

Signed-off-by: jsvisa <delweng@gmail.com>

* all: eip1559

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-08-01 12:58:45 +03:00
rjl493456442
2fd77a6a7e
core/txpool/blobpool: remove error log when finalized block is nil (#27822)
* core/txpool/blobpool: remove error log when finalized block is nil

* core/txpool/blobpool: take peter's suggestion
2023-08-01 10:09:20 +03:00
Delweng
3ca92f70e5
all: replace data gas to blob gas in comments (#27825)
* eth: excessDataGas -> excessBlobGas

* consensus: data gas -> blob gas

* core: data gas -> blob gas

* params: data gas -> blob gas
2023-08-01 10:07:25 +03:00
rjl493456442
817553cc28
core, trie: track state change set with account address (#27815) 2023-07-31 15:07:51 +03:00
Delweng
43a1a48ee2
core: remove pointless check for excessBlobGas nilness (#27797)
* core: check excessBlobGas in front

Signed-off-by: jsvisa <delweng@gmail.com>

* core: no need to manual panic

Signed-off-by: jsvisa <delweng@gmail.com>

* core: no comment

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-07-31 10:54:26 +03:00
Mario Vega
8f2ae29b8f
core/types: fix receipt blob fields marshaling (#27793) 2023-07-28 10:08:38 +03:00
Marius van der Wijden
d9556533c3
core/types: put header fields in correct order (#27791) 2023-07-27 17:15:08 +03:00
Péter Szilágyi
57268f7e6c
all: rename dataGas to blobGas (#27789) 2023-07-27 16:53:28 +03:00