Commit Graph

2434 Commits

Author SHA1 Message Date
jwasinger
9e018ce3a5
cmd/geth: update log test data (#28780)
update logger test data
2024-01-09 15:35:49 +01:00
Martin HS
1010a79c7c
cmd/geth: make it possible to autopilot removedb (#28725)
When managing geth, it is sometimes desirable to do a partial wipe; deleting state but retaining freezer data. A partial wipe can be somewhat tricky to accomplish. 

This change implements the ability to perform partial wipe by making it possible to run geth removedb non-interactive, using command line options instead.
2024-01-09 08:56:01 +01:00
vuittont60
f29520ffdf
cmd/devp2p/internal/ethtest: fix typos in comments (#28772) 2024-01-08 20:31:22 +01:00
Marius Kjærstad
99eb49e601
internal/flags: update copyright year to 2024 (#28760)
Co-authored-by: Felix Lange <fjl@twurst.com>
2024-01-04 15:03:58 +01:00
Mario Vega
0b471c312a
cmd/evm: Fix blob-gas-used on invalid transactions in t8n (#28734)
cmd/evm: fixes the blob gas calculation if a transaction is invalid
2024-01-03 16:12:20 +01:00
ddl
33c94ef083
cmd/evm: fix link in README.md (#28755) 2024-01-02 11:37:22 +01:00
rjl493456442
cca94792a4
core, cmd, trie: fix the condition of pathdb initialization (#28718)
Original problem was caused by #28595, where we made it so that as soon as we start to sync, the root of the disk layer is deleted. That is not wrong per se, but another part of the code uses the "presence of the root" as an init-check for the pathdb. And, since the init-check now failed, the code tried to re-initialize it which failed since a sync was already ongoing.

The total impact being: after a state-sync has begun, if the node for some reason is is shut down, it will refuse to start up again, with the error message: `Fatal: Failed to register the Ethereum service: waiting for sync.`. 

This change also modifies how `geth removedb` works, so that the user is prompted for two things: `state data` and `ancient chain`. The former includes both the chaindb aswell as any state history stored in ancients. 

---------

Co-authored-by: Martin HS <martin@swende.se>
2023-12-21 20:28:32 +01:00
lightclient
577be37e0e
cmd/devp2p: update eth/snap protocol test suites for PoS (#28340)
Here we update the eth and snap protocol test suites with a new test chain,
created by the hivechain tool. The new test chain uses proof-of-stake. As such,
tests using PoW block propagation in the eth protocol are removed. The test suite
now connects to the node under test using the engine API in order to make it
accept transactions. 

The snap protocol test suite has been rewritten to output test descriptions and
log requests more verbosely.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-20 17:23:48 +01:00
Martin HS
d3452a22cc
accounts: properly close managed wallets when closing manager (#28710) 2023-12-20 13:41:40 +01:00
Delweng
c18c5c3d92
cmd/evm: t8n support custom tracers (#28557)
This change implements ability for the `evm t8n` tool to use custom tracers; either 'native' golang tracers or javascript tracers.
2023-12-18 15:16:25 +01:00
Martin HS
553bafc127
cmd/evm, cmd/clef, cmd/bootnode: fix / unify logging (#28696)
This change fixes a problem with our non-core binaries: evm, clef, bootnode.

First of all, they failed to convert from legacy loglevels 1 to 5, to the new slog loglevels -4 to 4.

Secondly, the logging was actually setup in the init phase, and then overridden in the main. This is not needed for evm, since it used the same flag name as the main geth verbosity. Better to let the flags/internal handle the logging init.
2023-12-18 14:11:27 +01:00
jwasinger
05bbc56677
cmd/evm: default to mirror mainnet forks enabled (#28691)
cmd/evm:  default to using dev chain config (all mainnet HFs activated at block/timestamp 0
2023-12-18 13:56:27 +01:00
alex
edc864f9ba
all: fix typos in comments (#28682)
chore(core,eth):fix a couple of typos
2023-12-18 09:35:12 +01:00
Elias Rad
0f74aad641
all: fix typos in comments (#28662)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-13 23:33:46 +01:00
ucwong
81fd1b3cf9
core/txpool : small cleanup refactors (#28654) 2023-12-12 17:23:36 +02:00
Ng Wei Han
a3ca1b2818
cmd/utils: fix HTTPHost, WSHost flag priority (#28669)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-12 14:40:50 +01:00
Péter Szilágyi
d98d70f670
cmd/utils, eth: disallow invalid snap sync / snapshot flag combos (#28657)
* eth: prevent startup in snap mode without snapshots

* cmd/utils: try to fix bad flag combos wrt snap sync and snapshot generation
2023-12-08 15:16:04 +02:00
Martin HS
1048e2d6a3
cmd/evm: fix dump after state-test exec (#28650)
The dump after state-test didn't work, the problem was an error, "Already committed", which was silently ignored. 

This change re-initialises the state, so the dumping works again.
2023-12-08 11:06:01 +01:00
Martin Holst Swende
b8d44ed98b
log: remove lazy, remove unused interfaces, unexport methods (#28622)
This change 

- Removes interface `log.Format`, 
- Removes method `log.FormatFunc`, 
- unexports `TerminalHandler.TerminalFormat` formatting methods (renamed to `TerminalHandler.format`)
- removes the notion of `log.Lazy` values


The lazy handler was useful in the old log package, since it
could defer the evaluation of costly attributes until later in the
log pipeline: thus, if the logging was done at 'Trace', we could
skip evaluation if logging only was set to 'Info'.

With the move to slog, this way of deferring evaluation is no longer
needed, since slog introduced 'Enabled': the caller can thus do
the evaluate-or-not decision at the callsite, which is much more
straight-forward than dealing with lazy reflect-based evaluation.

Also, lazy evaluation would not work with 'native' slog, as in, these
two statements would be evaluated differently:

```golang
  log.Info("foo", "my lazy", lazyObj)
  slog.Info("foo", "my lazy", lazyObj)
```
2023-12-05 11:54:44 +01:00
ddl
6e488c2449
cmd/evm: fix Env struct json tag (#28635) 2023-12-04 11:52:55 +02:00
jwasinger
28e7371701
all: replace log15 with slog (#28187)
This PR replaces Geth's logger package (a fork of [log15](https://github.com/inconshreveable/log15)) with an implementation using slog, a logging library included as part of the Go standard library as of Go1.21.

Main changes are as follows:
* removes any log handlers that were unused in the Geth codebase.
* Json, logfmt, and terminal formatters are now slog handlers.
* Verbosity level constants are changed to match slog constant values.  Internal translation is done to make this opaque to the user and backwards compatible with existing `--verbosity` and `--vmodule` options.
* `--log.backtraceat` and `--log.debug` are removed.

The external-facing API is largely the same as the existing Geth logger.  Logger method signatures remain unchanged.

A small semantic difference is that a `Handler` can only be set once per `Logger` and not changed dynamically.  This just means that a new logger must be instantiated every time the handler of the root logger is changed.

----
For users of the `go-ethereum/log` module. If you were using this module for your own project, you will need to change the initialization. If you previously did 
```golang
log.Root().SetHandler(log.LvlFilterHandler(log.LvlInfo, log.StreamHandler(os.Stderr, log.TerminalFormat(true))))
```
You now instead need to do 
```golang
log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelInfo, true)))
```
See more about reasoning here: https://github.com/ethereum/go-ethereum/issues/28558#issuecomment-1820606613
2023-11-29 08:33:50 +01:00
Martin Holst Swende
63979bc9cc
cmd/evm, core/state: fix post-exec dump of state (statetests, blockchaintests) (#28504)
There were several problems related to dumping state. 

- If a preimage was missing, even if we had set the `OnlyWithAddresses` to `false`, to export them anyway, the way the mapping was constructed (using `common.Address` as key) made the entries get lost anyway. Concerns both state- and blockchain tests. 
- Blockchain test execution was not configured to store preimages.

This changes makes it so that the block test executor takes a callback, just like the state test executor already does. This callback can be used to examine the post-execution state, e.g. to aid debugging of test failures.
2023-11-28 13:54:17 +01:00
Marius van der Wijden
bdf5e388ca
cmd, les, tests: remove light client code (#28586)
* cmd, les, tests: remove light client code

This commit removes the light client (LES) code.
Since the merge the light client has been broken and
it is hard to maintain it alongside the normal client.
We decided it would be best to remove it for now and
maybe rework and reintroduce it in the future.

* cmd, eth: remove some more mentions of light mode

* cmd: re-add flags and mark as deprecated

* cmd: warn the user about deprecated flags

* eth: better error message
2023-11-23 16:28:26 +02:00
Martin Holst Swende
eec37e3b71
cmd/devp2p/internal/ethtest: undo debug-hack (#28588)
cmd/devp2p/internal/ethtest: remove a debug-hack flaw which prevented certain tests from running
2023-11-23 09:22:09 +01:00
Felix Lange
63127f5443
cmd/geth: fix build error (#28585) 2023-11-22 16:32:43 +01:00
Guillaume Ballet
d468c333a7
cmd/{geth,utils}: add cmd to export preimages in snap enumeration order (#28256)
Adds a subcommand: `geth snapshot export-preimages`, to export preimages of every hash found during a snapshot enumeration: that is, it exports _only the active state_, and not _all_ preimages that have been used but are no longer part of the state. 

This tool is needed for the verkle transition, in order to distribute the preimages needed for the conversion. Since only the 'active' preimages are exported, the output is shrunk from ~70GB to ~4GB.

The order of the output is the order used by the snapshot enumeration, which avoids database thrashing. However, it also means that storage-slot preimages are not deduplicated.
2023-11-22 14:48:25 +01:00
jwasinger
3cfcd252db
cmd/geth: add support for --dev flag in dumpgenesis (#28463)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
2023-11-22 12:08:39 +01:00
jwasinger
104dbf7821
cmd/utils: validate pre-existing genesis in --dev mode (#28468)
geth --dev can be used with an existing data directory and genesis block. Since
dev mode only works with PoS, we need to verify that the merge has happened.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-22 12:01:38 +01:00
Haotian
ad16f11f84
cmd/evm: capitalize evm commands (#28569)
* standard:fix for a unified standard

* standard:fix more as a complements

---------

Co-authored-by: haotian <haotian@haotiandeMacBook-Air.local>
2023-11-21 09:56:23 +02:00
Håvard Anda Estensen
460cc1673e
cmd: run tests in parallel (#28546) 2023-11-20 10:52:14 +01:00
Håvard Anda Estensen
a75a2d6db6
all: replace some cases of strings.SplitN with strings.Cut (#28446) 2023-11-15 14:42:33 +01:00
Zoro
2814ee0547
accounts,cmd,console,les,metrics: refactor some errors checked by (ST1005) go-staticcheck (#28532)
fix: fix some (ST1005)go-staticcheck
2023-11-15 14:36:57 +02:00
jwasinger
984f82629c
cmd/geth: more special cases logging tests (#28527)
adds logging tests for errors and custom fmt.Stringer-types which output strings that needs to be quoted/escaped.
2023-11-15 09:54:35 +01:00
Guillaume Ballet
fa8d39807d
cmd, core, trie: verkle-capable geth init (#28270)
This change allows the creation of a genesis block for verkle testnets. This makes for a chunk of code that is easier to review and still touches many discussion points.
2023-11-14 13:09:40 +01:00
Delweng
f265cc24b4
cmd/geth: remove some whitespace in code and comments (#28148)
This changes just removes some whitespace
2023-11-10 13:26:13 +01:00
Wei Tang
2f4833b828
cmd/evm: allow state dump regardless if test passes in statetest (#28484)
This change makes it so that when executing state tess, state is always dumped out if the corresponding flag is set.
2023-11-10 12:21:51 +01:00
Martin Holst Swende
b77a9b127c
cmd/geth: more testcases for logging (#28501)
* cmd/geth: more testcases for logging

This adds more edgecases around logging, particularly around handling of different types of nil-values
as concrete types and within interfaces.

Also adds tests with 'reserved' values which breaks json/logfmt formats. The json output is checked in,
but not actively used by any testcase at the moment.

* cmd/geth/testdata: remove timestamps
2023-11-09 10:46:48 +01:00
Mario Vega
285202aae2
cmd/evm: add --run option to blocktest command (#28421)
Co-authored-by: lightclient <lightclient@protonmail.com>
2023-10-31 15:23:51 +01:00
Martin Holst Swende
233db64cc1
all: make vendored copy of reexec (#28382)
a little copying is better than a little dependency

-- go proverb

We have this dependency on docker, a.k.a moby: a gigantic library, and we only need ~70 LOC,
so here I tried moving it inline instead.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-10-28 00:14:43 +02:00
Guillaume Ballet
4cbca5178a
core, cmd/geth: add --override.* flags to geth init (#28407)
* core, cmd/geth: add --override.* flags to geth init

* also apply overrides before genesis commit with new block

* review feedback
2023-10-27 10:15:14 +08:00
Martin Holst Swende
58ae1df684
cmd/geth: test for logging-output (#28373)
This PR is a bit in preparation for the slog work in #28187 .

Our current test re logging mostly test the internals, but we have no real end-to-end test of the logging output. This PR introduces a simple reexec-based log tester. This also relies upon a special mode in geth, which can be made to eject a set of predefined log messages (only available if the build-tag `integrationtests` is used

e.g. go run --tags=integrationtests ./cmd/geth --log.format terminal logtest

While working on this, I also noticed a quirk in the setup: when geth was configured to use a file output, then two separate handlers were used (one handler for the file, one handler for the console). Using two separate handlers means that two formatters are used, thus the formatting of any/all records happened twice. This PR changes the mechanism to use two separate io.Writers instead, which is both more optimal and fixes a bug which occurs due to a global statefulness in the formatter.
2023-10-25 17:57:12 +02:00
Martin Holst Swende
300df874d7
cmd/evm: make t8ntool handle transaction decoding errors better (#28397)
This change closes https://github.com/ethereum/go-ethereum/issues/27730 . By using an iterator instead of a slice of transactions, we can better handle the case when an individual transaction (within an otherwise well-formed RLP-list) cannot be decoded.
2023-10-25 14:08:52 +02:00
xiaolou86
00c63830e4
cmd: fix typos (#28323) 2023-10-17 15:53:10 +08:00
Martin Holst Swende
4632b7b31e
tests: update execution-spec-tests to 1.0.5 (#28337)
Updates execution-spec-tests to 1.0.5: https://github.com/ethereum/execution-spec-tests/releases/tag/v1.0.5, switching to develop which contains Cancun tests (which are also enabled in this change).
2023-10-16 20:10:05 +02:00
Péter Szilágyi
509a64ffb9
cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too (#28335)
* cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too

* ethdb/pebble: use Pebble's internal constant calculation
2023-10-13 22:50:20 +03:00
Péter Szilágyi
425cb6f65d
go.mod: pull in the latest cloudflare API libs (#28336) 2023-10-13 17:08:38 +03:00
Martin Holst Swende
f62c58f8de
trie: make rhs-proof align with last key in range proofs (#28311)
During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.

Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.

So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.

On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).

Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
2023-10-13 16:05:29 +02:00
rjl493456442
1cb3b6aee4
eth/protocols/snap: fix snap sync failure on empty storage range (#28306)
This change addresses an issue in snap sync, specifically when the entire sync process can be halted due to an encountered empty storage range.

Currently, on the snap sync client side, the response to an empty (partial) storage range is discarded as a non-delivery. However, this response can be a valid response, when the particular range requested does not contain any slots.

For instance, consider a large contract where the entire key space is divided into 16 chunks, and there are no available slots in the last chunk [0xf] -> [end]. When the node receives a request for this particular range, the response includes:

    The proof with origin [0xf]
    A nil storage slot set

If we simply discard this response, the finalization of the last range will be skipped, halting the entire sync process indefinitely. The test case TestSyncWithUnevenStorage can reproduce the scenario described above.

In addition, this change also defines the common variables MaxAddress and MaxHash.
2023-10-13 09:08:26 +02:00
Martin Holst Swende
d2c0bed9d5
cmd/geth: fix failing test (#28322) 2023-10-12 09:54:00 +02:00
vuittont60
0004c6b229
accounts, cmd: fix typos (#28300) 2023-10-11 11:29:10 +03:00