Commit Graph

517 Commits

Author SHA1 Message Date
rjl493456442
59f7b289c3
cmd, core, eth, graphql, trie: no persisted clean trie cache file (#27525)
The clean trie cache is persisted periodically, therefore Geth can
quickly warmup the cache in next restart.

However it will reduce the robustness of system. The assumption is
held in Geth that if the parent trie node is present, then the entire
sub-trie associated with the parent are all prensent.

Imagine the scenario that Geth rewinds itself to a past block and
restart, but Geth finds the root node of "future state" in clean
cache then regard this state is present in disk, while is not in fact.

Another example is offline pruning tool. Whenever an offline pruning
is performed, the clean cache file has to be removed to aviod hitting
the root node of "deleted states" in clean cache.

All in all, compare with the minor performance gain, system robustness
is something we care more.
2023-07-04 10:21:06 +03:00
Curith
6ca3ef9a7b
node: fix listening on IPv6 address (#27628) (#27635) 2023-07-02 13:21:16 +02:00
Guillaume Ballet
85b8d1c06c
params, trie: add verkle fork management + upgrade go-verkle (#27464)
* params, trie: add verkle fork management + upgrade go-verkle

* remove the two verkle files

* core, eth, params: add missing function

* Gary's feedback

* remove trie/utils/verkle.go

* add verkle block override

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-06-28 12:08:48 +03:00
Péter Szilágyi
d40a255e97
all: move main transaction pool into a subpool (#27463)
* all: move main transaction pool into a subpool

* go.mod: remove superfluous updates

* core/txpool: review fixes, handle txs rejected by all subpools

* core/txpool: typos
2023-06-16 15:29:40 +03:00
mmsqe
f3314bb6df
rpc: add limit for batch request items and response size (#26681)
This PR adds server-side limits for JSON-RPC batch requests. Before this change, batches
were limited only by processing time. The server would pick calls from the batch and
answer them until the response timeout occurred, then stop processing the remaining batch
items.

Here, we are adding two additional limits which can be configured:

- the 'item limit': batches can have at most N items
- the 'response size limit': batches can contain at most X response bytes

These limits are optional in package rpc. In Geth, we set a default limit of 1000 items
and 25MB response size.

When a batch goes over the limit, an error response is returned to the client. However,
doing this correctly isn't always possible. In JSON-RPC, only method calls with a valid
`id` can be responded to. Since batches may also contain non-call messages or
notifications, the best effort thing we can do to report an error with the batch itself is
reporting the limit violation as an error for the first method call in the batch. If a batch is
too large, but contains only notifications and responses, the error will be reported with
a null `id`.

The RPC client was also changed so it can deal with errors resulting from too large
batches. An older client connected to the server code in this PR could get stuck
until the request timeout occurred when the batch is too large. **Upgrading to a version
of the RPC client containing this change is strongly recommended to avoid timeout issues.**

For some weird reason, when writing the original client implementation, @fjl worked off of
the assumption that responses could be distributed across batches arbitrarily. So for a
batch request containing requests `[A B C]`, the server could respond with `[A B C]` but
also with `[A B] [C]` or even `[A] [B] [C]` and it wouldn't make a difference to the
client.

So in the implementation of BatchCallContext, the client waited for all requests in the
batch individually. If the server didn't respond to some of the requests in the batch, the
client would eventually just time out (if a context was used).

With the addition of batch limits into the server, we anticipate that people will hit this
kind of error way more often. To handle this properly, the client now waits for a single
response batch and expects it to contain all responses to the requests.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-06-13 13:38:58 +02:00
Péter Szilágyi
c7c84ca16c
all: remove the Rinkeby testnet (#27406) 2023-06-02 14:03:21 +03:00
Péter Szilágyi
2169fa343a
crypto/kzg4844: pull in the C and Go libs for KZG cryptography (#27155)
* cryto/kzg4844: pull in the C and Go libs for KZG cryptography

* go.mod: pull in the KZG libraries

* crypto/kzg4844: add basic becnhmarks for ballpark numbers

* cmd, crypto: integrate both CKZG and GoKZG all the time, add flag

* cmd/utils, crypto/kzg4844: run library init on startup

* crypto/kzg4844: make linter happy

* crypto/kzg4844: push missing file

* crypto/kzg4844: fully disable CKZG but leave in the sources

* build, crypto/kzg4844, internal: link CKZG by default and with portable mode

* crypto/kzg4844: drop verifying the trusted setup in gokzg

* internal/build: yolo until it works?

* cmd/utils: make flag description friendlier

Co-authored-by: Martin Holst Swende <martin@swende.se>

* crypto/ckzg: no need for double availability check

* build: tiny flag cleanup nitpick

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-05-10 14:54:14 +03:00
Péter Szilágyi
dde2da0efb
all: remove ethash pow, only retain shims needed for consensus and tests (#27178)
* all: remove ethash pow, only retain shims needed for consensus and tests

* all: thank you linter

* all: disallow launching Geth in legacy PoW mode

* cmd/env/internal/t8ntool: remove dangling ethash flag
2023-05-03 12:58:39 +03:00
Péter Szilágyi
66c0c4e517
cmd, eth, les: replace Shanghai override flag with Cancun (#27171) 2023-04-26 18:17:37 +03:00
Péter Szilágyi
d3ece3a07c
cmd/utils, node: switch to Pebble as the default db if none exists (#27136)
* cmd/utils, node: switch to Pebble as the default db if none exists

* node: fall back to LevelDB on platforms not supporting Pebble

* core/rawdb, node: default to Pebble at the node level

* cmd/geth: fix some tests explicitly using leveldb

* ethdb/pebble: allow double closes, makes tests simpler
2023-04-21 19:24:18 +03:00
Martin Holst Swende
ed51b8c5d3
ethdb: pebble backend (64bit platforms only) (#26517)
* ethdb: use pebble

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

foo

update

* apply suggested changes

* flags: go format

node: fix ddir lookup mistake

accounts/abi/bind: fix go.mod replacement for generated binding

deps: update pebble + with fix 32-bit build

* ethdb/pebble: respect max memtable size

* core/rawdb, ethdb: enable pebble on non-32bit platforms only

* core/rawdb: fix build tags, fix some review concerns

* core/rawdb: refactor methods for database opening

* core/rawdb: remove erroneous build tag

* cmd/geth: fix the flag default handling + testcase

* cmd/geth: improve testing regarding custom backends

* ethdb/pebble, deps: update pebble dependency

* core/rawdb: replace method with Open

* ethdb/pebble: several updates for pebble (#49)

* ethdb/pebble: fix size count in batch

* ethdb/pebble: disable seek compaction

* ethdb/pebble: more fixes

* ethdb, core, cmd: polish and fixes (#50)

* cmd/utils, core/rawdb, ethdb/pebble: address some review concerns

* Update flags.go

* ethdb/pebble: minor refactors

* ethdb/pebble: avoid copy on batch replay

* ethdb: fix compilation flaw

* cmd: fix test fail due to mismatching error message

* cmd/geth, node: rename backingdb to db.engine

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-02-09 10:48:34 +02:00
Péter Szilágyi
095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
Marius van der Wijden
d0a4989a8d
cmd, eth, node: deprecate personal namespace (#26390)
* eth: cmd: deprecate personal namespace

* eth: cmd: move deprecation to node

* node: disable toml of enablepersonal

* node: disable personal on ipc as well

* Update node/node.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* console: error -> warn

* node: less roulette

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-02-02 13:52:19 +02:00
Felix Lange
55f41d198c
cmd/utils: fix error at geth startup in --dev mode (#26550)
This fixes a regression in #26541 where we turned the miner address
being missing into a startup error. The address was not configured in
--dev mode.
2023-01-26 10:07:20 +01:00
Martin Holst Swende
2b57a27d9e
all: make timestamp-based fork checks based on uint64 (#26474)
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-01-25 12:12:28 +01:00
Felix Lange
59a48e0289
cmd/utils: improve parsing of --miner.etherbase address (#26541)
This fixes a regression where the flag did not accept values without
the 0x prefix anymore. What's worse, if an invalid value was passed,
the client would just log an INFO level message and continue.
2023-01-24 11:11:33 +01:00
rjl493456442
2b44ef5f93
miner, cmd, eth: require explicit etherbase address (#26413)
This change introduces a breaking change to miner.etherbase is configured.

Previously, users did not need to explicitly set the  etherbase address via flag, since 'first' local account was used as etherbase automatically. This change removes the  "default first account" feature.

In Proof-of-stake world, the fee recipient address is provided by CL, and not configured in Geth any more - meaning that miner.etherbase is mostly for legacy networks(pow, clique networks etc).
2023-01-20 11:26:01 -05:00
Zachinquarantine
690338f0fa
all: remove Kiln testnet (#26522)
Kiln was deprecated after the merge.
2023-01-19 10:49:48 +01:00
Seungbae Yu
d345a4a3c6
cmd/bootnode, cmd/utils: add 'pmp:<IP>' to --nat flag description (#26381) 2023-01-16 11:21:41 +01:00
Zachinquarantine
554c8d77c5
cmd/utils: update Rinkeby deprecation message (#26496) 2023-01-16 10:57:35 +01:00
Péter Szilágyi
b56c796220
cmd, core, eth, els, params: disallow setheads below genesis, tweaks 2023-01-06 15:29:58 +02:00
rjl493456442
890e2efca2
eth, cmd: remove syncTarget from eth config (#26330)
--syncTarget is a feature for development purpose in post-merge world. Previously
it's added into eth.Config. But it turns out that's a stupid idea.

- syncTarget is a block object, which is hard to be put in config file(large)
- syncTarget is just a dev feature, doesn't make too much sense to add it in config file

So I remove it from the eth config object. And it also fixes the #26328
2022-12-08 14:40:43 +01:00
Sina Mahmoodi
10347c6b54
core: drop legacy receipt types (#26225)
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028.

As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
2022-12-03 21:42:11 +01:00
Felix Lange
c5dc61c62d cmd/utils: gofmt 2022-11-24 13:03:38 +01:00
Felix Lange
8846c07d04
cmd/utils: print warning when --metrics.port set without --metrics.addr (#26248) 2022-11-24 11:37:58 +01:00
rjl493456442
2c1af8b1ec
cmd, eth: implement full-sync tester (#26035)
This PR adds a parameter to startup, --synctarget. The synctarget flag is a developer-flag, that can be useful in some scenarios as a replacement for a CL node. It defines a fixed block sync target:

geth --syncmode=full --synctarget=./block_15816882.hex_rlp 

The --synctarget is only made available during syncmode=full
2022-10-28 14:48:08 +02:00
Péter Szilágyi
a6dda03644
all: refactor txpool into it's own package in prep for 4844 2022-10-24 16:35:53 +03:00
Martin Holst Swende
6069d8294e
cmd/utils: enable snapshot generation in import-mode (#25990)
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode.  It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
2022-10-19 08:20:39 +02:00
Felix Lange
564751668a
cmd/geth: fix help output issues (#25941)
This update resolves an issue where StringSliceFlag would not be
rendered correctly in help output + mention that -H can be used multiple times

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-06 19:59:23 +02:00
rjl493456442
1913b50111
eth, miner: add timeout for building sealing block (#25407)
* eth, miner: add timeout for building sealing block

* eth, cmd, miner: add newpayloadtimeout flag

* eth, miner, cmd: address comments

* eth, miner: minor fixes

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-10-03 14:10:00 +02:00
Martin Holst Swende
ea26fc8a6c
cmd/geth, cmd/utils: geth attach with custom headers (#25829)
This PR makes it possible to set custom headers, in particular for two scenarios: 

- geth attach
- geth commands which can use --remotedb, e..g geth db inspect

The ability to use custom headers is typically useful for connecting to cloud-apis, e.g. providing an infura- or alchemy key, or for that matter access-keys for environments behind cloudflare.  

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-30 13:50:25 +02:00
jwasinger
7227c9ef07
cmd/geth: make dumpgenesis load genesis datadir if it exists (#25135)
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.


Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-09-26 13:55:18 +02:00
Zachinquarantine
15b4a4bf2e
cmd/puppeth, cmd/utils: finalize removal of gasTarget flag (#24370)
This PR fully removes the --miner.gastarget flag, as previously it was only hidden from the geth --help command, but could still be used.
2022-09-23 22:32:10 +02:00
rjl493456442
3da42f85d9
all: clean up the configs for pruner and snapshotter (#22396)
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.

And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
2022-09-23 20:20:36 +02:00
Delweng
6215b92523
cmd/utils: make --authrpc.jwtsecret a DirectoryFlag (#25849)
Signed-off-by: Delweng <delweng@gmail.com>
2022-09-22 15:43:47 +02:00
rjl493456442
d10c280309
all: move genesis initialization to blockchain (#25523)
* all: move genesis initialization to blockchain

* core: add one more check

* core: fix tests
2022-08-30 18:22:28 +02:00
Sina Mahmoodi
36874b63a1
eth/filters: add global block logs cache (#25459)
This adds a cache for block logs which is shared by all filters. The cache
size of is configurable using the `--cache.blocklogs` flag.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-08-19 11:14:59 +02:00
rjl493456442
e44d6551c3
cmd, core, ethdb, node: move chain freezer one folder deeper (#25487)
* cmd, core, ethdb, node: create chain freezer in a sub folder

* core/rawdb: remove unused code

* core, ethdb, node: add AncientDatadir API back

* cmd, core: extend freezer info dump for sub-ancient-store

* core/rawdb: rework freezer inspector

* core/rawdb: address comments from Peter

* core/rawdb: fix build issue
2022-08-08 12:08:36 +03:00
Péter Szilágyi
6fd06ab075
cmd, core, eth, les, params: add merge-passed chain config (#24538)
* cmd, core, eth, les, params: add merge-passed chain config

* eth/catalyst, params: add various warning on malfunctioning beacons

* eth/catalyst: fix warning for beacons without transition exchanges
2022-08-01 15:13:25 +03:00
rjl493456442
1ed8b7d24f
cmd: use flags.Merge for grouping flags (#25392) 2022-07-25 12:07:44 +03:00
Marius van der Wijden
3b2a6b34d9
cmd/geth: eth/catalyst: enable authrpc by default (#25152)
* cmd/geth: eth/catalyst: enable authrpc by default

* eth/catalyst: rename catalyst -> Engine API in logs

* eth/catalyst: don't panic
2022-07-23 09:56:02 +02:00
Andre Patta
9ecf8a97a9
cmd/utils: fix applying bootstrap nodes from config file (#25174) 2022-06-29 11:57:12 +02:00
zeim839
9a5c1000c7
cmd/geth, p2p: add support for custom discovery UDP port (#24979)
This adds a new flag to set the discovery port to be different from
the TCP listener port.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-06-28 17:25:47 +02:00
willian.eth
52ed3570c4
cmd: migrate to urfave/cli/v2 (#24751)
This change updates our urfave/cli dependency to the v2 branch of the library.
There are some Go API changes in cli v2:

- Flag values can now be accessed using the methods ctx.Bool,
  ctx.Int, ctx.String, ... regardless of whether the flag is 'local' or
  'global'.

- v2 has built-in support for flag categories. Our home-grown category
  system is removed and the categories of flags are assigned as part of
  the flag definition.

For users, there is only one observable difference with cli v2: flags must now
strictly appear before regular arguments. For example, the following command is
now invalid:

   geth account import mykey.json --password file.txt

Instead, the command must be invoked as follows:

   geth account import --password file.txt mykey.json
2022-06-27 18:22:36 +02:00
Sina Mahmoodi
f20a569265
cmd/geth: drop geth js command (#25000)
* cmd/geth: drop js command

* cmd: simplify ipc path determination for attach

* Add deprecation warning for js

* rm testdata for exec

* fix account unlock test cases

* Update cmd/geth/consolecmd.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* fix

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-06-21 12:16:05 +03:00
Marius van der Wijden
d8f963811d
cmd, params: implement Gray Glacier hard-fork (EIP-5133) (#25088)
* cmd/geth, params: implement Gray Glacier (EIP-5133)

* cmd/evm: add gray glacier tests

* params: nitpicks

* params: fixes
2022-06-15 14:10:38 +03:00
Ivan Aracki
f74bb3a3bf
cmd/utils: update --ropsten description (#25078) 2022-06-14 10:24:29 +02:00
Sina Mahmoodi
0559a9a61e
cmd/geth: exit when freezer has legacy receipts (#24943)
In #24028 we flagged a warning when finding legacy receipts in the freezer. This PR nudges users a bit more strongly by preventing geth from starting in this case until receipts have been migrated.

It also adds a flag --ignore-legacy-receipts which when present allows geth to start normally.
2022-05-26 09:22:10 +02:00
Martin Holst Swende
57192bd0dc
ethdb/remotedb, cmd: add support for remote (readonly) databases (#24836)
* ethdb/remotedb, cmd: add support for remote (readonly) databases

* ethdb/remotedb: minor changes

* ethdb/remotedb: close the conn

* cmd, ethdb: add rpc accessor for ancient data

* internal/ethapi: license

* ethdb/remotedb: linter fixes
2022-05-18 17:27:17 +03:00
Håvard Anda Estensen
07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00