Commit Graph

124 Commits

Author SHA1 Message Date
lightclient
1f50aa7631
cmd,internal/era: implement export-history subcommand (#26621)
* all: implement era format, add history importer/export

* internal/era/e2store: refactor e2store to provide ReadAt interface

* internal/era/e2store: export HeaderSize

* internal/era: refactor era to use ReadAt interface

* internal/era: elevate anonymous func to named

* cmd/utils: don't store entire era file in-memory during import / export

* internal/era: better abstraction between era and e2store

* cmd/era: properly close era files

* cmd/era: don't let defers stack

* cmd/geth: add description for import-history

* cmd/utils: better bytes buffer

* internal/era: error if accumulator has more records than max allowed

* internal/era: better doc comment

* internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer

* internal/era: avoid some repetition

* internal/era: simplify clauses

* internal/era: unexport things

* internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries

* cmd/utils: better defer handling in history test

* internal/era,cmd: add number method to era iterator to get the current block number

* internal/era/e2store: avoid double allocation during write

* internal/era,cmd/utils: fix lint issues

* internal/era: add ReaderAt func so entry value can be read lazily

Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>

* internal/era: improve iterator interface

* internal/era: fix rlp decode of header and correctly read total difficulty

* cmd/era: fix rebase errors

* cmd/era: clearer comments

* cmd,internal: fix comment typos

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2024-02-07 09:18:27 -07:00
Guillaume Ballet
d468c333a7
cmd/{geth,utils}: add cmd to export preimages in snap enumeration order (#28256)
Adds a subcommand: `geth snapshot export-preimages`, to export preimages of every hash found during a snapshot enumeration: that is, it exports _only the active state_, and not _all_ preimages that have been used but are no longer part of the state. 

This tool is needed for the verkle transition, in order to distribute the preimages needed for the conversion. Since only the 'active' preimages are exported, the output is shrunk from ~70GB to ~4GB.

The order of the output is the order used by the snapshot enumeration, which avoids database thrashing. However, it also means that storage-slot preimages are not deduplicated.
2023-11-22 14:48:25 +01:00
Zoro
2814ee0547
accounts,cmd,console,les,metrics: refactor some errors checked by (ST1005) go-staticcheck (#28532)
fix: fix some (ST1005)go-staticcheck
2023-11-15 14:36:57 +02:00
Delweng
e9c3183c52
cmd: use errrors.New instead of empty fmt.Errorf (#27329)
Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-24 12:21:29 +02:00
Shude Li
3223950a5d
cmd/utils: do not check free disk space in dev mode (#27281) 2023-05-19 08:38:21 -04:00
Guillaume Ballet
c62da24dce
cmd/utils: report the blocknumber when block import fails (#27213)
When block import fails, the error displays the number of the first block past the import batch, not the number of the failing block. This change fixes this problem by identifying which blocks fails and reporting its number.
2023-05-09 03:57:42 -04:00
Péter Szilágyi
cd31f2dee2
all: change chain head markers from block to header (#26777) 2023-03-02 08:29:15 +02:00
kumavis
64dccf7aa4
cmd/utils: log path used when checking disk space (#26212)
This change logs the path checked when encountering low disk space.
2022-11-21 10:18:18 +01:00
willian.eth
52ed3570c4
cmd: migrate to urfave/cli/v2 (#24751)
This change updates our urfave/cli dependency to the v2 branch of the library.
There are some Go API changes in cli v2:

- Flag values can now be accessed using the methods ctx.Bool,
  ctx.Int, ctx.String, ... regardless of whether the flag is 'local' or
  'global'.

- v2 has built-in support for flag categories. Our home-grown category
  system is removed and the categories of flags are assigned as part of
  the flag definition.

For users, there is only one observable difference with cli v2: flags must now
strictly appear before regular arguments. For example, the following command is
now invalid:

   geth account import mykey.json --password file.txt

Instead, the command must be invoked as follows:

   geth account import --password file.txt mykey.json
2022-06-27 18:22:36 +02:00
Martin Holst Swende
53304ff6c7
cmd/utils: double limit on free-disk monitor (#24781)
This PR doubles the limit on which to trigger automatic shutdown, and also changes the timer to run once every 30s instead of 60s.
2022-05-03 12:12:40 +02:00
Martin Holst Swende
72c2c0ae7e
cmd/geth, console: support interrupting the js console (#23387)
Previously, Ctrl-C (SIGINT) was ignored during JS execution, so it was not
possible to get out of infinite loops in the console. With this change,
Ctrl-C now interrupts JS.

Fixes #23344

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-12-11 16:51:05 +01:00
rjl493456442
2e8b58f076
cmd/geth: implement data import and export (#22931)
This PR offers two more database sub commands for exporting and importing data.
Two exporters are implemented: preimage and snapshot data respectively. 
The import command is generic, it can take any data export and import into leveldb. 
The data format has a 'magic' for disambiguation, and a version field for future compatibility.
2021-11-02 11:31:45 +01:00
gary rong
098a2b6e26
eth: move eth.Config to a common package (#22205)
This moves the eth config definition into a separate package, eth/ethconfig. 
Packages eth and les can now import this common package instead of
importing eth from les, reducing dependencies.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-02-05 13:51:15 +01:00
Alex Mazalov
24c1e3053b
cmd/geth: graceful shutdown if disk is full (#22103)
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check. 

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-01-19 09:26:42 +01:00
rene
c0c01612e9
node: refactor package node (#21105)
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.

There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.

There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.

The --graphql.port and --graphql.addr flags are no longer available.
2020-08-03 19:40:46 +02:00
Martin Holst Swende
6402c42b67
all: simplify and fix database iteration with prefix/start (#20808)
* core/state/snapshot: start fixing disk iterator seek

* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start

* les, core/state/snapshot: iterator fixes

* all: remove two iterator methods

* all: rename Iteratee.NewIteratorWith -> NewIterator

* ethdb: fix review concerns
2020-04-15 14:08:53 +03:00
gary rong
37d280da41
core, cmd, vendor: fixes and database inspection tool (#15)
* core, eth: some fixes for freezer

* vendor, core/rawdb, cmd/geth: add db inspector

* core, cmd/utils: check ancient store path forceily

* cmd/geth, common, core/rawdb: a few fixes

* cmd/geth: support windows file rename and fix rename error

* core: support ancient plugin

* core, cmd: streaming file copy

* cmd, consensus, core, tests: keep genesis in leveldb

* core: write txlookup during ancient init

* core: bump database version
2019-05-16 10:39:34 +03:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Corey Lin
1ff152f3a4 rawdb: remove unused parameter for WritePreimages func (#18059)
* rawdb: remove unused parameter for WritePreimages func and modify a
spelling mistake

* rawdb: update the doc for function WritePreimages
2018-11-09 12:51:07 +02:00
Péter Szilágyi
6cf0ab38bd
core/rawdb: separate raw database access to own package (#16666) 2018-05-07 14:35:06 +03:00
Péter Szilágyi
495bdb0c71
cmd: export preimages in RLP, support GZIP, uniform with block export 2018-03-26 14:08:01 +03:00
Péter Szilágyi
01507d9b9d
cmd, console: support all termination signals 2018-02-21 15:23:10 +02:00
Péter Szilágyi
55599ee95d core, trie: intermediate mempool between trie and database (#15857)
This commit reduces database I/O by not writing every state trie to disk.
2018-02-05 17:40:32 +01:00
Felix Lange
10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Péter Szilágyi
9184249b39 Logger updates 3 (#3730)
* accounts, cmd, eth, ethdb: port logs over to new system

* ethdb: drop concept of cache distribution between dbs

* eth: fix some log nitpicks to make them nicer
2017-03-02 14:06:16 +01:00
Felix Lange
5c8fe28b72 common: move big integer math to common/math (#3699)
* common: remove CurrencyToString

Move denomination values to params instead.

* common: delete dead code

* common: move big integer operations to common/math

This commit consolidates all big integer operations into common/math and
adds tests and documentation.

There should be no change in semantics for BigPow, BigMin, BigMax, S256,
U256, Exp and their behaviour is now locked in by tests.

The BigD, BytesToBig and Bytes2Big functions don't provide additional
value, all uses are replaced by new(big.Int).SetBytes().

BigToBytes is now called PaddedBigBytes, its minimum output size
parameter is now specified as the number of bytes instead of bits. The
single use of this function is in the EVM's MSTORE instruction.

Big and String2Big are replaced by ParseBig, which is slightly stricter.
It previously accepted leading zeros for hexadecimal inputs but treated
decimal inputs as octal if a leading zero digit was present.

ParseUint64 is used in places where String2Big was used to decode a
uint64.

The new functions MustParseBig and MustParseUint64 are now used in many
places where parsing errors were previously ignored.

* common: delete unused big integer variables

* accounts/abi: replace uses of BytesToBig with use of encoding/binary

* common: remove BytesToBig

* common: remove Bytes2Big

* common: remove BigTrue

* cmd/utils: add BigFlag and use it for error-checked integer flags

While here, remove environment variable processing for DirectoryFlag
because we don't use it.

* core: add missing error checks in genesis block parser

* common: remove String2Big

* cmd/evm: use utils.BigFlag

* common/math: check for 256 bit overflow in ParseBig

This is supposed to prevent silent overflow/truncation of values in the
genesis block JSON. Without this check, a genesis block that set a
balance larger than 256 bits would lead to weird behaviour in the VM.

* cmd/utils: fixup import
2017-02-26 22:21:51 +01:00
Péter Szilágyi
1ca20a2697
cmd, whisper/mailserver: revert to utils.Fatalf 2017-02-23 12:16:46 +02:00
Péter Szilágyi
23a5d64fd0
accounts, cmd: port packages over to the new logging system 2017-02-23 12:16:46 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Felix Lange
13e3b2f433 logger, pow/dagger, pow/ezp: delete dead code 2017-01-06 18:18:07 +01:00
Felix Lange
f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Nick Johnson
9ba9fe818d cmd/utils, eth: Add gzip support for chain dump and restore 2016-12-14 08:59:55 +00:00
Felix Lange
b04219fdbb cmd/utils: don't check for stderr redirect on windows
The redirect check did not work on Go 1.6 and below because Stat
returned an error for stdout and stderr. In Go 1.7 Stat works on stdout
but doesn't return anything meaningful, causing cmd/geth test failures
because the message is printed to stderr only. Fix it by printing to
stdout only.
2016-09-26 17:35:19 +02:00
Tosh Camille
ac66d96c5a cmd/utils: add space between "to" and filename 2016-06-15 11:02:22 +03:00
Felix Lange
130bccc763 cmd/utils: flush trace and CPU profile data when force-qutting
Also reduce log messages a little bit.
2016-05-06 11:16:47 +02:00
Felix Lange
dff9b4246f cmd/geth, cmd/utils: improve input handling
These changes make prompting behave consistently on all platforms:

* The input buffer is now global.
  Buffering was previously set up for each prompt, which can cause weird
  behaviour, e.g. when running "geth account update <input.txt" where
  input.txt contains three lines. In this case, the first password
  prompt would fill up the buffer with all lines and then use only the
  first one.

* Print the "unsupported terminal" warning only once.
  Now that stdin prompting has global state, we can use it to track
  the warning there.

* Work around small liner issues, particularly on Windows.
  Prompting didn't work under most of the third-party terminal emulators
  on Windows because liner assumes line editing is always available.
2016-04-12 15:56:49 +02:00
Ramesh Nair
6f30034413 cmd/utils: removed password line endings when not using liner. 2016-03-23 22:53:20 +08:00
Felix Lange
c1343c8872 cmd/utils, internal/debug: show all stacks for 10x Ctrl-C induced panic
Go 1.6 only prints stacks for the current goroutine by default,
but for this panic we want to see all of them.
2016-03-12 00:59:37 +01:00
Péter Szilágyi
1e806c4c77 cmd, common, core, eth, node, rpc, tests, whisper, xeth: use protocol stacks 2015-11-27 11:06:12 +02:00
Jeffrey Wilcke
1c63d08ed1 cmd/geth, cmd/utils: removed legalese
Removed the legalese confirmation dialog. This closes #1992
2015-11-19 12:03:33 +01:00
zelig
8b81ad1fc4 console:
* lines with leading space are ommitted from history
* exit processed even with whitespace around
* all whitespace lines (not only empty ones) are ignored

add 7 missing commands to admin api autocomplete

registrar: methods now return proper error if reg addresses are not set. fixes #1457

rpc/console: fix personal.newAccount() regression. Now all comms accept interactive password

registrar: add registrar tests for errors

crypto: catch AES decryption error on presale wallet import + fix error msg format. fixes #1580

CLI: improve error message when starting a second instance of geth. fixes #1564

cli/accounts: unlock multiple accounts. fixes #1785
* make unlocking multiple accounts work with inline <() fd
* passwdfile now correctly read only once
* improve logs
* fix CLI help text for unlocking

fix regression with docRoot / admin API
* docRoot/jspath passed to rpc/api ParseApis, which passes onto adminApi
* docRoot field for JS console in order to pass when RPC is (re)started
* improve flag desc for jspath

common/docserver: catch http errors from response

fix rpc/api tests

common/natspec: fix end to end test (skipped because takes 8s)

registrar: fix major regression:
* deploy registrars on frontier
* register HashsReg and UrlHint in GlobalRegistrar.
* set all 3 contract addresses in code
* zero out addresses first in tests
2015-10-22 00:22:39 +02:00
Péter Szilágyi
2547c9c9b7 cmd: properly initialize Olympic for all subcommands 2015-10-07 18:25:33 +03:00
Jeffrey Wilcke
7c7692933c cmd/geth, cmd/utils, core, rpc: renamed to blockchain
* Renamed ChainManager to BlockChain
* Checkpointing is no longer required and never really properly worked
when the state was corrupted.
2015-10-04 01:13:56 +02:00
Jeffrey Wilcke
74f6d90153 cmd/utils, core: disable exp diff for olympic net 2015-08-06 13:29:06 +02:00
Jeffrey Wilcke
dcdb7059cc cmd, core, eth: support for the olympic network
Added a --olympic flag which initialiser the olympic protocol settings
2015-08-03 18:15:48 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Taylor Gerring
cf5313f13e Update disclaimer 2015-07-20 18:49:25 +02:00
Taylor Gerring
618991f136 Move text to separate file 2015-07-16 13:34:34 +02:00
Taylor Gerring
1909d26fe2 Prompt user to accept legalese when datadir doesn't exist 2015-07-16 12:27:42 +02:00
Felix Lange
bdae4fd573 all: add some godoc synopsis comments 2015-07-07 14:12:45 +02:00