Commit Graph

66 Commits

Author SHA1 Message Date
Felix Lange
12df45662a
all: remove remaining uses of untyped golang-lru (#26194) 2022-11-17 11:15:28 +01:00
Martin Holst Swende
ee301c750b
all: fix docstrings 2022-10-04 09:18:02 +02:00
Justin Traglia
2c5648d891
all: fix some typos (#25551)
* Fix some typos

* Fix some mistakes

* Revert 4byte.json

* Fix an incorrect fix

* Change files to fails
2022-08-19 09:00:21 +03:00
Martin Holst Swende
a907d7e81a
all: more linters (#24783)
This enables the following linters

- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec

WIth a few exceptions.

- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
2022-06-13 16:24:45 +02:00
rjl493456442
7f6f01d46f
core: recover state when beacon sets canonical head if it's missing (#24613)
* core: recover the state in SetChainHead if the head state is missing

* core: disable test logging

* core: address comment from martin

* core: improve log level in case state is recovered

* core, eth, les, light: rename SetChainHead to SetCanonical
2022-05-05 10:36:26 +03:00
rjl493456442
e0e8bf31c5
eth/downloader: ignore zero size header batch for importing (#24569)
* eth/downloader: ignore zero size header batch for importing

* core, light: reject empty header batch for importing
2022-03-22 18:58:05 +01:00
Marius van der Wijden
3038e480f5
all: core rework for the merge transition (#23761)
* all: work for eth1/2 transtition

* consensus/beacon, eth: change beacon difficulty to 0

* eth: updates

* all: add terminalBlockDifficulty config, fix rebasing issues

* eth: implemented merge interop spec

* internal/ethapi: update to v1.0.0.alpha.2

                                                                 This commit updates the code to the new spec, moving payloadId into
                                                                 it's own object. It also fixes an issue with finalizing an empty blockhash.
                                                                 It also properly sets the basefee

* all: sync polishes, other fixes + refactors

* core, eth: correct semantics for LeavePoW, EnterPoS

* core: fixed rebasing artifacts

* core: light: performance improvements

* core: use keyed field (f)

* core: eth: fix compilation issues + tests

* eth/catalyst: dbetter error codes

* all: move Merger to consensus/, remove reliance on it in bc

* all: renamed EnterPoS and LeavePoW to ReachTDD and FinalizePoS

* core: make mergelogs a function

* core: use InsertChain instead of InsertBlock

* les: drop merger from lightchain object

* consensus: add merger

* core: recoverAncestors in catalyst mode

* core: fix nitpick

* all: removed merger from beacon, use TTD, nitpicks

* consensus: eth: add docstring, removed unnecessary code duplication

* consensus/beacon: better comment

* all: easy to fix nitpicks by karalabe

* consensus/beacon: verify known headers to be sure

* core: comments

* core: eth: don't drop peers who advertise blocks, nitpicks

* core: never add beacon blocks to the future queue

* core: fixed nitpicks

* consensus/beacon: simplify IsTTDReached check

* consensus/beacon: correct IsTTDReached check

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-11-26 13:23:02 +02:00
Martin Holst Swende
6289137827
consensus/clique, core: API cleanup (#23100)
This removes some code:

- The clique engine calculated the snapshot twice when verifying headers/blocks.

- The method GetBlockHashesFromHash in Header/Block/Lightchain was only used by tests. It
  is now removed from the API.
  
- The method GetTdByHash internally looked up the number before calling GetTd(hash, num).
  In many cases, callers already had the number, and used this method just because it has a
  shorter name. I have removed the method to make the API surface smaller.
2021-10-11 23:16:46 +02:00
Martin Holst Swende
40b6ccf383
core,les: headerchain import in batches (#21471)
* core: add test for headerchain inserts

* core, light: write headerchains in batches

* core: change to one callback per batch of inserted headers + review concerns

* core: error-check on batch write

* core: unexport writeHeaders

* core: remove callback parameter in InsertHeaderChain

The semantics of InsertHeaderChain are now much simpler: it is now an
all-or-nothing operation. The new WriteStatus return value allows
callers to check for the canonicality of the insertion. This change
simplifies use of HeaderChain in package les, where the callback was
previously used to post chain events.

* core: skip some hashing when writing headers

* core: less hashing in header validation

* core: fix headerchain flaw regarding blacklisted hashes

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-12-09 11:13:02 +01:00
gary rong
6eef141aef
les: historical data garbage collection (#19570)
This change introduces garbage collection for the light client. Historical
chain data is deleted periodically. If you want to disable the GC, use
the --light.nopruning flag.
2020-07-13 11:02:54 +02:00
Felix Lange
9219e0fba4
eth: interrupt chain insertion on shutdown (#21114)
This adds a new API method on core.BlockChain to allow interrupting
running data inserts, and calls the method before shutting down the
downloader.

The BlockChain interrupt checks are now done through a method instead
of inlining the atomic load everywhere. There is no loss of efficiency from
this and it makes the interrupt protocol a lot clearer because the check is
defined next to the method that sets the flag.
2020-05-26 21:37:37 +02:00
gary rong
770316dc20 core, light: write chain data in atomic way (#20287)
* core: write chain data in atomic way

* core, light: address comments

* core, light: fix linter

* core, light: address comments
2020-01-17 12:49:32 +02:00
Felix Lange
9c6cf960b4
internal/web3ext, les: update clique JS and make it work with the light client (#20318)
Also fix the input formatter on clique_getSnapshot and clique_getSigners
so that integers as well as hex number strings are accepted.
2019-11-19 18:22:04 +01:00
Ryan Schneider
ad03d9801c internal/ethapi: support block number or hash on state-related methods (#19491)
This change adds support for EIP-1898.
2019-09-26 10:47:31 +02:00
Felföldi Zsolt
0ac9bbba6c les: multiple server bugfixes (#20079)
* les: detailed relative cost metrics

* les: filter txpool relative request statistic

* les: initialize price factors

* les: increased connected bias to lower churn rate

* les: fixed clientPool.setLimits

* core: do not use mutex in GetAncestor

* les: bump factor db version again

* les: add metrics

* les, light: minor fixes
2019-09-17 16:28:41 +03:00
gary rong
f7cdea2bdc all: on-chain oracle checkpoint syncing (#19543)
* all: implement simple checkpoint syncing

cmd, les, node: remove callback mechanism

cmd, node: remove callback definition

les: simplify the registrar

les: expose checkpoint rpc services in the light client

les, light: don't store untrusted receipt

cmd, contracts, les: discard stale checkpoint

cmd, contracts/registrar: loose restriction of registeration

cmd, contracts: add replay-protection

all: off-chain multi-signature contract

params: deploy checkpoint contract for rinkeby

cmd/registrar: add raw signing mode for registrar

cmd/registrar, contracts/registrar, les: fixed messages

* cmd/registrar, contracts/registrar: fix lints

* accounts/abi/bind, les: address comments

* cmd, contracts, les, light, params: minor checkpoint sync cleanups

* cmd, eth, les, light: move checkpoint config to config file

* cmd, eth, les, params: address comments

* eth, les, params: address comments

* cmd: polish up the checkpoint admin CLI

* cmd, contracts, params: deploy new version contract

* cmd/checkpoint-admin: add another flag for clef mode signing

* cmd, contracts, les: rename and regen checkpoint oracle with abigen
2019-06-28 10:34:02 +03:00
gary rong
80469bea0c
all: integrate the freezer with fast sync
* all: freezer style syncing

core, eth, les, light: clean up freezer relative APIs

core, eth, les, trie, ethdb, light: clean a bit

core, eth, les, light: add unit tests

core, light: rewrite setHead function

core, eth: fix downloader unit tests

core: add receipt chain insertion test

core: use constant instead of hardcoding table name

core: fix rollback

core: fix setHead

core/rawdb: remove canonical block first and then iterate side chain

core/rawdb, ethdb: add hasAncient interface

eth/downloader: calculate ancient limit via cht first

core, eth, ethdb: lots of fixes

* eth/downloader: print ancient disable log only for fast sync
2019-05-16 10:39:32 +03:00
Péter Szilágyi
38f6b85638
eth, les, light: enforce CHT checkpoints on fast-sync too 2019-04-17 13:16:15 +03:00
Martin Holst Swende
0b4fe8d192 all: simplify timestamps to uint64 (#19372)
* all: simplify timestamps to uint64

* tests: update definitions

* clef, faucet, mobile: leftover uint64 fixups

* ethash: fix tests

* graphql: update schema for timestamp

* ethash: remove unused variable
2019-04-02 23:28:48 +03:00
Péter Szilágyi
211ec46284
les, light: verbose errors on state retrieval issues 2019-03-18 13:19:40 +02:00
Felföldi Zsolt
c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
Matthew Halpern
6f714ed73e light: make chain receiver names consistent (#18997) 2019-02-07 12:53:45 +02:00
Felix Lange
684facedb8
light: fix disableCheckFreq locking (#18515)
This change unbreaks the build and removes racy access to
disableCheckFreq. Even though the field is set while holding
the lock, it was read outside of the protected section.
2019-01-24 13:40:37 +01:00
b00ris
769657060e les: implement ultralight client (#16904)
For more information about this light client mode, read
https://hackmd.io/s/HJy7jjZpm
2019-01-24 12:18:26 +01:00
Péter Szilágyi
f25f776c9f
core, light: get rid of the dual mutexes, hard to reason with 2019-01-11 15:27:47 +02:00
Péter Szilágyi
af89093116
les, light, params: update light client CHTs 2018-09-20 14:14:48 +03:00
Péter Szilágyi
0f2ba07c41
common, core, light: add block age into info logs 2018-09-20 12:56:35 +03:00
Felföldi Zsolt
63352bf424 core: safe indexer operation when syncing starts before the checkpoint (#17511) 2018-08-28 10:31:34 +03:00
gary rong
b69476b372 all: make indexer configurable (#17188) 2018-08-28 10:08:16 +03:00
Péter Szilágyi
9f036647e4
consensus/clique, light: light client snapshots on Rinkeby 2018-08-21 15:21:59 +03:00
Felföldi Zsolt
2cdf6ee7e0 light: CHT and bloom trie indexers working in light mode (#16534)
This PR enables the indexers to work in light client mode by
downloading a part of these tries (the Merkle proofs of the last
values of the last known section) in order to be able to add new
values and recalculate subsequent hashes. It also adds CHT data to
NodeInfo.
2018-08-15 22:25:46 +02:00
Felföldi Zsolt
049f5b3572 core, eth, les: more efficient hash-based header chain retrieval (#16946) 2018-06-12 16:52:54 +03:00
Péter Szilágyi
6cf0ab38bd
core/rawdb: separate raw database access to own package (#16666) 2018-05-07 14:35:06 +03:00
Martin Holst Swende
f83237573f core: make current*Block atomic, and accessor functions mutex-free (#16171)
* core: make current*Block atomic, and accessor functions mutex-free

* core: fix review concerns

* core: fix error in atomic assignment

* core/light: implement atomic getter/setter for headerchain
2018-02-26 11:53:10 +02:00
Péter Szilágyi
7a0019c63b
les, light: fix CHT trie retrievals (#16039)
* les, light: fix CHT trie retrievals

* les, light: minor polishes, test remote CHT retrievals

* les, light: deterministic nodeset rlp, bloombits test skeleton

* les: add an event emission to the les bloombits test

* les: drop dead tester code
2018-02-11 14:57:46 +02:00
Péter Szilágyi
eb07dbb079
eth, light: minor light client startup cleanups 2018-02-08 07:49:23 +02:00
Péter Szilágyi
55599ee95d core, trie: intermediate mempool between trie and database (#15857)
This commit reduces database I/O by not writing every state trie to disk.
2018-02-05 17:40:32 +01:00
Péter Szilágyi
566d5c0777
core, eth, les, light: get rid of redundant methods 2018-01-30 18:42:00 +02:00
Felföldi Zsolt
ec96216d16 Chain indexer fix + new CHT (#15934)
* core, light: fix chain indexer bug

* light: add new CHT
2018-01-23 13:10:49 +02:00
Péter Szilágyi
6f69cdd109
all: switch gas limits from big.Int to uint64 2018-01-03 14:45:35 +02:00
Péter Szilágyi
f7ca03ae87 eth, les, light: expose chain config in les node info too (#15732) 2017-12-28 14:18:34 +01:00
Felföldi Zsolt
ca376ead88 les, light: LES/2 protocol version (#14970)
This PR implements the new LES protocol version extensions:

* new and more efficient Merkle proofs reply format (when replying to
  a multiple Merkle proofs request, we just send a single set of trie
  nodes containing all necessary nodes)
* BBT (BloomBitsTrie) works similarly to the existing CHT and contains
  the bloombits search data to speed up log searches
* GetTxStatusMsg returns the inclusion position or the
  pending/queued/unknown state of a transaction referenced by hash
* an optional signature of new block data (number/hash/td) can be
  included in AnnounceMsg to provide an option for "very light
  clients" (mobile/embedded devices) to skip expensive Ethash check
  and accept multiple signatures of somewhat trusted servers (still a
  lot better than trusting a single server completely and retrieving
  everything through RPC). The new client mode is not implemented in
  this PR, just the protocol extension.
2017-10-24 15:19:09 +02:00
Zsolt Felfoldi
7a7f6a4f29 light: new CHTs for mainnet and ropsten 2017-09-11 23:36:16 +02:00
Felix Lange
10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Miya Chen
bf1e263128 core, light: send chain events using event.Feed (#14865) 2017-08-18 12:58:36 +02:00
Egon Elbre
971079822e light: fix megacheck warnings (#14920) 2017-08-07 17:25:18 +02:00
Péter Szilágyi
8f12d76a47
params: remove redundant consts, disable metro on AllProtocolChanges 2017-07-04 12:28:58 +03:00
Felix Lange
9e5f03b6c4 core/state: access trie through Database interface, track errors (#14589)
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.

Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
2017-06-27 15:57:06 +02:00
Felix Lange
7b2fc0643f core, light: delete SplitStatTy, ChainSplitEvent (unused) 2017-04-21 18:56:00 +02:00
Péter Szilágyi
a7b9e484d0 consensus, core, ethstats: use engine specific block beneficiary (#14318)
* consensus, core, ethstats: use engine specific block beneficiary

* core, eth, les, miner: use explicit beneficiary during mining
2017-04-12 16:38:31 +03:00