Commit Graph
225 Commits
Author SHA1 Message Date
philip-morlier bdc42a9128 Merge commit '7371b3817' into merge/geth-v1.13.0 2023-09-12 12:34:24 -07:00
Delweng 2e02c1ffd9 core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
Felix Lange 9b46986edc all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
Felix Lange d1f6735171 core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Marius van der Wijden 2f4dbb4f90 core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende f0f8703bf2 core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi 00fead91c4 cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
Péter Szilágyi be65b47645 all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
503f1f7ada all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Martin Holst Swende 8d1db1601d eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-08-10 06:49:05 -04:00
7de748d3f6 all: implement path-based state scheme (#25963)
* all: implement path-based state scheme

* all: edits from review

* core/rawdb, trie/triedb/pathdb: review changes

* core, light, trie, eth, tests: reimplement pbss history

* core, trie/triedb/pathdb: track block number in state history

* trie/triedb/pathdb: add history documentation

* core, trie/triedb/pathdb: address comments from Peter's review

Important changes to list:

- Cache trie nodes by path in clean cache
- Remove root->id mappings when history is truncated

* trie/triedb/pathdb: fallback to disk if unexpect node in clean cache

* core/rawdb: fix tests

* trie/triedb/pathdb: rename metrics, change clean cache key

* trie/triedb: manage the clean cache inside of disk layer

* trie/triedb/pathdb: move journal function

* trie/triedb/path: fix tests

* trie/triedb/pathdb: fix journal

* trie/triedb/pathdb: fix history

* trie/triedb/pathdb: try to fix tests on windows

* core, trie: address comments

* trie/triedb/pathdb: fix test issues

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-01 15:17:32 +03:00
Delweng 3ca92f70e5 all: replace data gas to blob gas in comments (#27825)
* eth: excessDataGas -> excessBlobGas

* consensus: data gas -> blob gas

* core: data gas -> blob gas

* params: data gas -> blob gas
2023-08-01 10:07:25 +03:00
Péter Szilágyi 57268f7e6c all: rename dataGas to blobGas (#27789) 2023-07-27 16:53:28 +03:00
Felix LangeandPéter Szilágyi 393d4db18c core/types: add 4844 data gas fields in Receipt (#27743)
* core/types: add data gas fields in Receipt

* core/types: use BlobGas method of tx

* core: fix test

* core/types: fix receipt tests, add data gas used field test

---------

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-07-27 14:11:09 +03:00
philip-morlier 6ed9390f9b Merge remote-tracking branch 'origin/feature/plugeth-test-plugin' into merge/geth-v1.12.0 2023-07-12 13:47:51 -07:00
rjl493456442 0b1f97e151 core/rawdb: support freezer batch read with no size limit (#27687)
This change adds the ability to perform reads from freezer without size limitation. This can be useful in cases where callers are certain that out-of-memory will not happen (e.g. reading only a few elements). 

The previous API was designed to behave both optimally and secure while servicing a request from a peer, whereas this change should _not_ be used when an untrusted peer can influence the query size.
2023-07-12 03:19:01 -04:00
philip-morlier e516d5842c automated merge 2023-07-11 16:01:17 -07:00
Delweng aecf3f9579 internal/blocktest: add package for shared test code (#27270) 2023-07-11 14:57:02 +02:00
philip-morlier 4ca9210d59 Modified injection test in /core/rawdb/ 2023-06-22 13:00:05 -07:00
Dan LaineandFelix Lange 154b016b6c core: use slices package for sorting (#27489)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-20 11:58:47 +02:00
philip-morlier 1bf1a5634e Merge remote-tracking branch 'origin/develop' into feature/old-consensus-engine 2023-06-13 22:52:01 -07:00
philip-morlier 5dc7541135 Functional stand alone test for pluginBlockProcessingError 2023-06-02 09:45:48 -07:00
Péter Szilágyi c7c84ca16c all: remove the Rinkeby testnet (#27406) 2023-06-02 14:03:21 +03:00
philip-morlier 679d028484 Functional test for injections in core/rawdb 2023-06-01 13:14:08 -07:00
Delweng 6c732766c8 core,console: replace noarg fmt.Errorf with errors.New (#27332)
* core: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* console: replace noarg fmt.Errorf with errors.New

Signed-off-by: jsvisa <delweng@gmail.com>

* core: go autoimport

Signed-off-by: jsvisa <delweng@gmail.com>

* core: dry

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-25 08:24:09 -04:00
rjl493456442 5021d36d35 all: port boring changes from pbss (#27176)
* all: port boring changes from pbss

* core, trie: address comments from martin

* trie: minor fixes

* core/rawdb: update comment

* core, eth, tests, trie: address comments

* tests, trie: add extra check when update trie database

* trie/triedb/hashdb: degrade the error to warning
2023-05-09 10:11:04 +03:00
philip-morlier c3e91ce1af These files were changed by hand to support the merge
The previous commit message was made in error, it should say v1.11.6 not v1.11.5. Fixing the commit message is causing us to have to fix confilicts and we are going to leave it alone.
The project builds but doesnt load the required pluggins. More work needs to be done acrross the various projects.
2023-04-26 14:32:55 -07:00
philip-morlier e986bbfa6f Initial commit of merge geth v1.11.5 2023-04-21 13:24:40 -07:00
Péter Szilágyi d3ece3a07c cmd/utils, node: switch to Pebble as the default db if none exists (#27136)
* cmd/utils, node: switch to Pebble as the default db if none exists

* node: fall back to LevelDB on platforms not supporting Pebble

* core/rawdb, node: default to Pebble at the node level

* cmd/geth: fix some tests explicitly using leveldb

* ethdb/pebble: allow double closes, makes tests simpler
2023-04-21 19:24:18 +03:00
Péter Szilágyi bbc565ab05 core/types, params: add blob transaction type, RLP encoded for now (#27049)
* core/types, params: add blob transaction type, RLP encoded for now

* all: integrate Cancun (and timestamp based forks) into MakeSigner

* core/types: fix 2 back-and-forth type refactors

* core: fix review comment

* core/types: swap blob tx type id to 0x03
2023-04-21 12:52:02 +03:00
rjl493456442 99f81d2724 all: refactor trie API (#26995)
In this PR, all TryXXX(e.g. TryGet) APIs of trie are renamed to XXX(e.g. Get) with an error returned.

The original XXX(e.g. Get) APIs are renamed to MustXXX(e.g. MustGet) and does not return any error -- they print a log output. A future PR will change the behaviour to panic on errorrs.
2023-04-20 06:57:24 -04:00
Delweng 7f3fc15a8b core/rawdb: update freezertable read meter (#26946)
The meter for "for measuring the effective amount of data read" within the freezertable was never updated. This change remedies that. 
---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-03-23 03:34:40 -04:00
philip-morlier f3e558d4e0 Merge commit 'a38f41085' into merge/geth-v1.11.5
Manual touches in core/state_transtion.go and go.mod
2023-03-21 08:39:16 -07:00
s7v7nislands 905a723fae core/rawdb: use atomic int added in go1.19 (#26935) 2023-03-21 07:10:23 -04:00
rjl493456442andMartin Holst Swende f7661a662a core/rawdb: find smallest block stored in key-value store when chain gapped (#26719)
This change prints out more information about the problem, in the case where geth detects a gap between leveldb and ancients, so we can determine more exactly where the gap is (what the first missing is). Also prints out more metadata. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-03-08 02:39:13 -05:00
philip-morlier 6e40cd7920 Merge commit '5ed08c473' into merge/geth-v1.11.3 2023-03-07 10:04:53 -08:00
Felix LangeandPulsarAI 87186148e0 core/types: add EffectiveGasPrice in Receipt (#26713)
This change adds a struct field EffectiveGasPrice in types.Receipt. The field is present
in RPC responses, but not in the Go struct, and thus can't easily be accessed via ethclient.

Co-authored-by: PulsarAI <dev@pulsar-systems.fi>
2023-03-06 17:19:41 +01:00
Péter Szilágyi 19f74fa3c0 core/rawdb, ethdb/pebble: disable pebble on openbsd (#26801) 2023-03-03 12:05:00 +02:00
Martin Holst Swende 09a9ccdbce core/rawdb, node: use standalone flock dependency (#26633) 2023-02-23 09:11:50 +02:00
philip-morlier 7bf0c614f6 Modified chain freezer map keys to Public to match v1.11.2 2023-02-22 10:01:43 -08:00
philip-morlier 4415e7c7b8 Merge commit '73b01f40c' geth v1.11.2 into merge/geth-v1.11.2 2023-02-22 08:13:38 -08:00
philip-morlier 78e00bf68d Manual touches to preserve compatability of other networks with utils as well as fix a failing test in core/rawdb. 2023-02-21 10:29:25 -08:00
rjl493456442 fe01a2f63b all: use unified emptyRootHash and emptyCodeHash (#26718)
The EmptyRootHash and EmptyCodeHash are defined everywhere in the codebase, this PR replaces all of them with unified one defined in core/types package, and also defines constants for TxRoot, WithdrawalsRoot and UncleRoot
2023-02-21 06:12:27 -05:00
Péter Szilágyi 2f20fd31ee core/rawdb: expose chain freezer constructor without internals (#26748) 2023-02-21 13:10:01 +02:00
Martin Holst Swende 6d2d126100 core: fix accessor mismatch for genesis state (#26747) 2023-02-21 12:18:33 +02:00
Péter Szilágyi 90d25514af core, eth: merge snap-sync chain download progress logs (#26676) 2023-02-21 12:17:34 +02:00
philip-morlier 2f6b8d86f1 manual geth v1.11.1 merge 2023-02-17 07:53:01 -08:00
Martin Holst Swende 4d3525610e all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Sina Mahmoodi 2def62b99b eth/filters: avoid block body retrieval when no matching logs (#25199)
Logs stored on disk have minimal information. Contextual information such as block
number, index of log in block, index of transaction in block are filled in upon request.
We can fill in all these fields only having the block header and list of receipts.
But determining the transaction hash of a log requires the block body.

The goal of this PR is postponing this retrieval until we are sure we the transaction hash.
It happens often that the header bloom filter signals there might be matches in a block,
but after actually checking them reveals the logs do not match. We want to avoid fetching
the body in this case.

Note that this changes the semantics of Backend.GetLogs. Downstream callers of
GetLogs now assume log context fields have not been derived, and need to call
DeriveFields on the logs if necessary.
2023-02-13 10:59:27 +01:00
bf1798e04e common/prque: generic priority queue (#26290)
* common, core, eth, les, trie: make prque generic

* les/vflux/server: fixed issues in priorityPool

* common, core, eth, les, trie: make priority also generic in prque

* les/flowcontrol: add test case for priority accumulator overflow

* les/flowcontrol: avoid priority value overflow

* common/prque: use int priority in some tests

No need to convert to int64 when we can just change the type used by the
queue.

* common/prque: remove comment about int64 range

---------

Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-02-09 13:03:54 +02:00