Commit Graph

225 Commits

Author SHA1 Message Date
philip-morlier
bdc42a9128 Merge commit '7371b3817' into merge/geth-v1.13.0 2023-09-12 12:34:24 -07:00
Delweng
2e02c1ffd9
core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
Felix Lange
d1f6735171
core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Marius van der Wijden
2f4dbb4f90
core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende
f0f8703bf2
core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi
00fead91c4
cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
rjl493456442
503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Martin Holst Swende
8d1db1601d
eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-08-10 06:49:05 -04:00
rjl493456442
7de748d3f6
all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
Delweng
3ca92f70e5
all: replace data gas to blob gas in comments (#27825)
* eth: excessDataGas -> excessBlobGas

* consensus: data gas -> blob gas

* core: data gas -> blob gas

* params: data gas -> blob gas
2023-08-01 10:07:25 +03:00
Péter Szilágyi
57268f7e6c
all: rename dataGas to blobGas (#27789) 2023-07-27 16:53:28 +03:00
Felix Lange
393d4db18c
core/types: add 4844 data gas fields in Receipt (#27743)
* core/types: add data gas fields in Receipt

* core/types: use BlobGas method of tx

* core: fix test

* core/types: fix receipt tests, add data gas used field test

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-07-27 14:11:09 +03:00
philip-morlier
6ed9390f9b Merge remote-tracking branch 'origin/feature/plugeth-test-plugin' into merge/geth-v1.12.0 2023-07-12 13:47:51 -07:00
rjl493456442
0b1f97e151
core/rawdb: support freezer batch read with no size limit (#27687)
This change adds the ability to perform reads from freezer without size limitation. This can be useful in cases where callers are certain that out-of-memory will not happen (e.g. reading only a few elements). 

The previous API was designed to behave both optimally and secure while servicing a request from a peer, whereas this change should _not_ be used when an untrusted peer can influence the query size.
2023-07-12 03:19:01 -04:00
philip-morlier
e516d5842c automated merge 2023-07-11 16:01:17 -07:00
Delweng
aecf3f9579
internal/blocktest: add package for shared test code (#27270) 2023-07-11 14:57:02 +02:00
philip-morlier
4ca9210d59 Modified injection test in /core/rawdb/ 2023-06-22 13:00:05 -07:00
Dan Laine
154b016b6c
core: use slices package for sorting (#27489)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-20 11:58:47 +02:00
philip-morlier
1bf1a5634e Merge remote-tracking branch 'origin/develop' into feature/old-consensus-engine 2023-06-13 22:52:01 -07:00
philip-morlier
5dc7541135 Functional stand alone test for pluginBlockProcessingError 2023-06-02 09:45:48 -07:00
Péter Szilágyi
c7c84ca16c
all: remove the Rinkeby testnet (#27406) 2023-06-02 14:03:21 +03:00
philip-morlier
679d028484 Functional test for injections in core/rawdb 2023-06-01 13:14:08 -07:00
Delweng
6c732766c8
core,console: replace noarg fmt.Errorf with errors.New (#27332)
* core: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* console: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* core: go autoimport

Signed-off-by: jsvisa <delweng@gmail.com>

* core: dry

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-25 08:24:09 -04:00
rjl493456442
5021d36d35
all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
philip-morlier
c3e91ce1af These files were changed by hand to support the merge
The previous commit message was made in error, it should say v1.11.6 not v1.11.5. Fixing the commit message is causing us to have to fix confilicts and we are going to leave it alone.
The project builds but doesnt load the required pluggins. More work needs to be done acrross the various projects.
2023-04-26 14:32:55 -07:00
philip-morlier
e986bbfa6f Initial commit of merge geth v1.11.5 2023-04-21 13:24:40 -07:00
Péter Szilágyi
d3ece3a07c
cmd/utils, node: switch to Pebble as the default db if none exists (#27136)
* cmd/utils, node: switch to Pebble as the default db if none exists

* node: fall back to LevelDB on platforms not supporting Pebble

* core/rawdb, node: default to Pebble at the node level

* cmd/geth: fix some tests explicitly using leveldb

* ethdb/pebble: allow double closes, makes tests simpler
2023-04-21 19:24:18 +03:00
Péter Szilágyi
bbc565ab05
core/types, params: add blob transaction type, RLP encoded for now (#27049)
* core/types, params: add blob transaction type, RLP encoded for now

* all: integrate Cancun (and timestamp based forks) into MakeSigner

* core/types: fix 2 back-and-forth type refactors

* core: fix review comment

* core/types: swap blob tx type id to 0x03
2023-04-21 12:52:02 +03:00
rjl493456442
99f81d2724
all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Delweng
7f3fc15a8b
core/rawdb: update freezertable read meter (#26946)
The meter for "for measuring the effective amount of data read" within the freezertable was never updated. This change remedies that. 
---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-03-23 03:34:40 -04:00
philip-morlier
f3e558d4e0 Merge commit 'a38f41085' into merge/geth-v1.11.5
Manual touches in core/state_transtion.go and go.mod
2023-03-21 08:39:16 -07:00
s7v7nislands
905a723fae
core/rawdb: use atomic int added in go1.19 (#26935) 2023-03-21 07:10:23 -04:00
rjl493456442
f7661a662a
core/rawdb: find smallest block stored in key-value store when chain gapped (#26719)
This change prints out more information about the problem, in the case where geth detects a gap between leveldb and ancients, so we can determine more exactly where the gap is (what the first missing is). Also prints out more metadata. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-03-08 02:39:13 -05:00
philip-morlier
6e40cd7920 Merge commit '5ed08c473' into merge/geth-v1.11.3 2023-03-07 10:04:53 -08:00
Felix Lange
87186148e0
core/types: add EffectiveGasPrice in Receipt (#26713)
This change adds a struct field EffectiveGasPrice in types.Receipt. The field is present
in RPC responses, but not in the Go struct, and thus can't easily be accessed via ethclient.

Co-authored-by: PulsarAI <dev@pulsar-systems.fi>
2023-03-06 17:19:41 +01:00
Péter Szilágyi
19f74fa3c0
core/rawdb, ethdb/pebble: disable pebble on openbsd (#26801) 2023-03-03 12:05:00 +02:00
Martin Holst Swende
09a9ccdbce
core/rawdb, node: use standalone flock dependency (#26633) 2023-02-23 09:11:50 +02:00
philip-morlier
7bf0c614f6 Modified chain freezer map keys to Public to match v1.11.2 2023-02-22 10:01:43 -08:00
philip-morlier
4415e7c7b8 Merge commit '73b01f40c' geth v1.11.2 into merge/geth-v1.11.2 2023-02-22 08:13:38 -08:00
philip-morlier
78e00bf68d Manual touches to preserve compatability of other networks with utils as well as fix a failing test in core/rawdb. 2023-02-21 10:29:25 -08:00
rjl493456442
fe01a2f63b
all: use unified emptyRootHash and emptyCodeHash (#26718)
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
2023-02-21 06:12:27 -05:00
Péter Szilágyi
2f20fd31ee
core/rawdb: expose chain freezer constructor without internals (#26748) 2023-02-21 13:10:01 +02:00
Martin Holst Swende
6d2d126100
core: fix accessor mismatch for genesis state (#26747) 2023-02-21 12:18:33 +02:00
Péter Szilágyi
90d25514af
core, eth: merge snap-sync chain download progress logs (#26676) 2023-02-21 12:17:34 +02:00
philip-morlier
2f6b8d86f1 manual geth v1.11.1 merge 2023-02-17 07:53:01 -08:00
Martin Holst Swende
4d3525610e
all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Sina Mahmoodi
2def62b99b
eth/filters: avoid block body retrieval when no matching logs (#25199)
Logs stored on disk have minimal information. Contextual information such as block
number, index of log in block, index of transaction in block are filled in upon request.
We can fill in all these fields only having the block header and list of receipts.
But determining the transaction hash of a log requires the block body.

The goal of this PR is postponing this retrieval until we are sure we the transaction hash.
It happens often that the header bloom filter signals there might be matches in a block,
but after actually checking them reveals the logs do not match. We want to avoid fetching
the body in this case.

Note that this changes the semantics of Backend.GetLogs. Downstream callers of
GetLogs now assume log context fields have not been derived, and need to call
DeriveFields on the logs if necessary.
2023-02-13 10:59:27 +01:00
Péter Szilágyi
bf1798e04e
common/prque: generic priority queue (#26290)
* common, core, eth, les, trie: make prque generic

* les/vflux/server: fixed issues in priorityPool

* common, core, eth, les, trie: make priority also generic in prque

* les/flowcontrol: add test case for priority accumulator overflow

* les/flowcontrol: avoid priority value overflow

* common/prque: use int priority in some tests

No need to convert to int64 when we can just change the type used by the
queue.

* common/prque: remove comment about int64 range

---------

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-09 13:03:54 +02:00