Commit Graph

174 Commits

Author SHA1 Message Date
Péter Szilágyi
fc85777a21
core: concurrent database reinit from freezer dump
* core: reinit chain from freezer in batches

* core/rawdb: concurrent database reinit from freezer dump

* core/rawdb: reinit from freezer in sequential order
2019-05-27 15:48:30 +03:00
gary rong
611113e967 core: never delete genesis block (#19617) 2019-05-27 12:05:45 +03:00
Péter Szilágyi
536b3b416c
cosensus, core, eth, params, trie: fixes + clique history cap 2019-05-16 10:39:35 +03:00
gary rong
37d280da41
core, cmd, vendor: fixes and database inspection tool (#15)
* core, eth: some fixes for freezer

* vendor, core/rawdb, cmd/geth: add db inspector

* core, cmd/utils: check ancient store path forceily

* cmd/geth, common, core/rawdb: a few fixes

* cmd/geth: support windows file rename and fix rename error

* core: support ancient plugin

* core, cmd: streaming file copy

* cmd, consensus, core, tests: keep genesis in leveldb

* core: write txlookup during ancient init

* core: bump database version
2019-05-16 10:39:34 +03:00
gary rong
80469bea0c
all: integrate the freezer with fast sync
* all: freezer style syncing

core, eth, les, light: clean up freezer relative APIs

core, eth, les, trie, ethdb, light: clean a bit

core, eth, les, light: add unit tests

core, light: rewrite setHead function

core, eth: fix downloader unit tests

core: add receipt chain insertion test

core: use constant instead of hardcoding table name

core: fix rollback

core: fix setHead

core/rawdb: remove canonical block first and then iterate side chain

core/rawdb, ethdb: add hasAncient interface

eth/downloader: calculate ancient limit via cht first

core, eth, ethdb: lots of fixes

* eth/downloader: print ancient disable log only for fast sync
2019-05-16 10:39:32 +03:00
Péter Szilágyi
751effa35e
core: fix formatting error (trailing whitepace) 2019-05-13 14:07:55 +03:00
PilkyuJung
ec2131c8d3 core: move error variable to error.go (#19560)
* move error variable to error.go

* Update error.go

Edit "Genesis" to "genesis"
2019-05-13 13:23:32 +03:00
Péter Szilágyi
6ec6b29051
core: fix import errors on clique crashes + empty blocks (#19544)
* core: fix import errors on clique crashes + empty blocks

* cosensus/clique, core: add test for the mirrored state issue

* core: address todo question wrt log count

* core: raise a louder warning for non-clique known blocks
2019-05-10 17:04:10 +03:00
gary rong
c113723fdb core: handle importing known blocks more gracefully (#19417)
* core: import known blocks if they can be inserted as canonical blocks

* core: insert knowns blocks

* core: remove useless

* core: doesn't process head block in reorg function
2019-05-08 14:30:36 +03:00
Martin Holst Swende
c8cf360f29
core: fix canonicality confusion (#19514)
* core: add tests for canonicality confusion

* core: delete stale future canon number mappings during reorg to shorter+heavier chain
2019-05-07 14:26:00 +02:00
Matthew Halpern
937417527c core: lookup txs by block number instead of block hash (#19431)
* core: lookup txs by block number instead of block hash

Transaction hashes now store a reference to their corresponding
block number as opposed to their hash. In benchmarks this was
shown to reduce storage by over 12 GB.

The main limitation of this approach is that transactions on
non-canonical blocks could never be looked up, however that is
currently not supported.

The database version has been upgraded to version 5 and the
transaction lookup process is backwards-compatible with the
prior two transaction lookup formats prexisting in the
database instance. Tests have been added to ensure this.

* core/rawdb: tiny review nit fixes
2019-04-25 17:24:55 +03:00
Péter Szilágyi
7221cb1434
core, eth, les, light: scope receipt functionality a bit cleaner 2019-04-15 13:42:26 +03:00
Matthew Halpern
6b0ddd141e
core, eth, les, light: store transaction receipts without txHash and gasCost 2019-04-15 13:15:39 +03:00
Péter Szilágyi
43631aa1d6
core: minor code polishes + rebase fixes 2019-04-04 16:29:25 +03:00
rjl493456442
690bd8a417
core: re-omit new log event when logs rebirth 2019-04-04 14:17:43 +03:00
Martin Holst Swende
0b4fe8d192 all: simplify timestamps to uint64 (#19372)
* all: simplify timestamps to uint64

* tests: update definitions

* clef, faucet, mobile: leftover uint64 fixups

* ethash: fix tests

* graphql: update schema for timestamp

* ethash: remove unused variable
2019-04-02 23:28:48 +03:00
Péter Szilágyi
e14f8a408c
Merge pull request #19328 from karalabe/preload
core: prefetch next block state concurrently
2019-04-02 17:03:12 +03:00
Runchao Han
c4109d790f core: fix typo in insertChain method doc (#19371) 2019-04-02 13:01:02 +03:00
Péter Szilágyi
ed34a5e08a
cmd, core, eth: support disabling the concurrent state prefetcher 2019-04-01 11:52:11 +03:00
Péter Szilágyi
bb9631c399
core: prefetch next block state concurrently 2019-04-01 11:06:15 +03:00
Brent
7fb89697fd core/types: add block location fields to receipt (#17662)
Solves #15210 without changing consensus, in a backwards compatible way,
by adding tx inclusion information to the Receipt struct.
2019-03-27 13:39:25 +01:00
Péter Szilágyi
a732c93309
core: 3rd try on splitting the trie metrics correctly 2019-03-27 13:02:04 +02:00
Martin Holst Swende
59e1953246 core, ethdb, trie: mode dirty data to clean cache on flush (#19307)
This PR is a more advanced form of the dirty-to-clean cacher (#18995),
where we reuse previous database write batches as datasets to uncache,
saving a dirty-trie-iteration and a dirty-trie-rlp-reencoding per block.
2019-03-26 15:48:31 +01:00
Péter Szilágyi
435020f9b3
core: split trie op metrics from the correct chain metrics 2019-03-25 16:27:46 +02:00
Péter Szilágyi
86989e3fcd
core: split out detailed trie access metrics from insertion time (#19316)
* core: split out detailed trie access metrics from insertion time

* cmd, core, metrics: support expensive optional metrics
2019-03-25 10:01:18 +02:00
Péter Szilágyi
91eec1251c
cmd, core, eth, trie: get rid of trie cache generations (#19262)
* cmd, core, eth, trie: get rid of trie cache generations

* core, trie: get rid of remainder of cache gen boilerplate
2019-03-14 15:25:12 +02:00
Péter Szilágyi
4f457859a2
core: use headers only where blocks are unnecessary 2019-03-13 12:32:47 +02:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Felföldi Zsolt
c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
gary rong
7fd0ccaa68 core: remove unnecessary fields in logs, receipts and tx lookups (#17106)
* core: remove unnecessary fields in log

* core: bump blockchain database version

* core, les: remove unnecessary fields in txlookup

* eth: print db version explicitly

* core/rawdb: drop txlookup entry struct wrapper
2019-02-21 15:14:35 +02:00
Martin Holst Swende
8577b5b020 core: more tests for sidechain import, fixes #19105 (#19113)
* blockchain: more tests for sidechain import, fixes #19105

* core/blockchain: rework import of pruned canon blocks and canon-prepended sidechains

* core/blockchain: minor clarity change

* core/blockchain: remove unused method
2019-02-21 12:36:49 +02:00
Matthew Halpern
bf42535d31 core: remove redundant parentheses (#19106) 2019-02-19 13:25:42 +02:00
Péter Szilágyi
940e317094
core: fix pruner panic when importing low-diff-large-sidechain 2019-02-08 11:56:25 +02:00
Péter Szilágyi
d6225ab846
cmd/utils, eth: relinquish GC cache to read cache in archive mode 2019-02-07 14:36:25 +02:00
Martin Holst Swende
ec3432bccb
core: fix error in block iterator (#18986) 2019-02-04 13:30:19 +01:00
silence
66f0c464cc core: only cache non-nil receipts from the database (#18447)
receipts may be null for very short time in some condition. For this case, we should not add the null value into cache. Because you will not get the right result if you keep requesting that receipt.
2019-01-17 17:49:12 +02:00
Péter Szilágyi
f25f776c9f
core, light: get rid of the dual mutexes, hard to reason with 2019-01-11 15:27:47 +02:00
gary rong
d5cad488be core, eth: fix database version (#18429)
* core, eth: fix database version

* eth: polish error message
2019-01-11 13:49:12 +02:00
Paweł Bylica
de39513ced core, internal, eth, miner, les: Take VM config from BlockChain (#17955)
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.

This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.

Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
2018-12-06 14:34:49 +01:00
Martin Holst Swende
bfed28a421 core: more detailed metrics for block processing (#18119) 2018-11-28 10:29:05 +02:00
mr_franklin
2a113f6d72 core: return error if repair block failed (#18126)
* core: return error if repair block failed

* make error a bit shorter
2018-11-23 11:16:14 +02:00
Péter Szilágyi
3ba0418a9a
Merge pull request #17973 from holiman/splitter2
core: better side-chain importing
2018-11-22 15:01:10 +02:00
Martin Holst Swende
e0d091e090 core: better printout of receipts in bad block reports (#18156)
* core/blockchain: better printout of receipts in bad block reports

* fix splleing
2018-11-22 11:00:16 +02:00
Péter Szilágyi
333b5fb123
core: polish side chain importer a bit 2018-11-21 13:19:56 +02:00
Martin Holst Swende
493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
Péter Szilágyi
accc0fab4f
core, eth/downloader: fix ancestor lookup for fast sync 2018-11-16 13:21:20 +02:00
Péter Szilágyi
434dd5bc00
cmd, core, eth, light, trie: add trie read caching layer 2018-11-15 12:22:13 +02:00
gary rong
1212c7b844 core: fix default trie cache limit (#17860) 2018-11-12 18:06:34 +02:00
Corey Lin
1ff152f3a4 rawdb: remove unused parameter for WritePreimages func (#18059)
* rawdb: remove unused parameter for WritePreimages func and modify a
spelling mistake

* rawdb: update the doc for function WritePreimages
2018-11-09 12:51:07 +02:00
Ryan Schneider
b69942befe core, internal/ethapi: add and use LRU cache for receipts (#17610) 2018-09-29 22:53:31 +02:00