Commit Graph

459 Commits

Author SHA1 Message Date
Péter Szilágyi
f30179d62e
eth: disable fast sync after pivot is committed 2017-09-06 15:02:44 +03:00
rjl493456442
02aa86e659 eth/downloader: exit loop when there is no more available task 2017-08-14 13:51:37 +08:00
Ivan Daniluk
17ce0a37de eth/downloader: fix race in downloadTesterPeer (#14942)
* eth/downloader: fix race in downloadTesterPeer

Signed-off-by: Ivan Daniluk <ivan.daniluk@gmail.com>

* eth/downloader: minor datarace fix cleanup
2017-08-08 20:10:09 +03:00
Egon Elbre
8f06b7980d eth: fix megacheck warnings 2017-08-07 19:54:20 +03:00
Nick Johnson
fe13949d9d eth/downloader: Doc fixes 2017-07-05 11:13:16 +01:00
Nick Johnson
be8f8409bc eth/downloader, les, light: Changes in response to review 2017-07-03 15:17:12 +01:00
Nick Johnson
ae11545bc5 eth, les: Refactor downloader peer to use structs 2017-06-29 12:49:18 +01:00
Nick Johnson
0550957989 eth, les, light: Refactor downloader to use blockchain interface 2017-06-28 15:58:41 +01:00
Felix Lange
9e5f03b6c4 core/state: access trie through Database interface, track errors (#14589)
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.

Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
2017-06-27 15:57:06 +02:00
Felix Lange
0042f13d47 eth/downloader: separate state sync from queue (#14460)
* eth/downloader: separate state sync from queue

Scheduling of state node downloads hogged the downloader queue lock when
new requests were scheduled. This caused timeouts for other requests.
With this change, state sync is fully independent of all other downloads
and doesn't involve the queue at all.

State sync is started and checked on in processContent. This is slightly
awkward because processContent doesn't have a select loop. Instead, the
queue is closed by an auxiliary goroutine when state sync fails. We
tried several alternatives to this but settled on the current approach
because it's the least amount of change overall.

Handling of the pivot block has changed slightly: the queue previously
prevented import of pivot block receipts before the state of the pivot
block was available. In this commit, the receipt will be imported before
the state. This causes an annoyance where the pivot block is committed
as fast block head even when state downloads fail. Stay tuned for more
updates in this area ;)

* eth/downloader: remove cancelTimeout channel

* eth/downloader: retry state requests on timeout

* eth/downloader: improve comment

* eth/downloader: mark peers idle when state sync is done

* eth/downloader: move pivot block splitting to processContent

This change also ensures that pivot block receipts aren't imported
before the pivot block itself.

* eth/downloader: limit state node retries

* eth/downloader: improve state node error handling and retry check

* eth/downloader: remove maxStateNodeRetries

It fails the sync too much.

* eth/downloader: remove last use of cancelCh in statesync.go

Fixes TestDeliverHeadersHang*Fast and (hopefully)
the weird cancellation behaviour at the end of fast sync.

* eth/downloader: fix leak in runStateSync

* eth/downloader: don't run processFullSyncContent in LightSync mode

* eth/downloader: improve comments

* eth/downloader: fix vet, megacheck

* eth/downloader: remove unrequested tasks anyway

* eth/downloader, trie: various polishes around duplicate items

This commit explicitly tracks duplicate and unexpected state
delieveries done against a trie Sync structure, also adding
there to import info logs.

The commit moves the db batch used to commit trie changes one
level deeper so its flushed after every node insertion. This
is needed to avoid a lot of duplicate retrievals caused by
inconsistencies between Sync internals and database. A better
approach is to track not-yet-written states in trie.Sync and
flush on commit, but I'm focuing on correctness first now.

The commit fixes a regression around pivot block fail count.
The counter previously was reset to 1 if and only if a sync
cycle progressed (inserted at least 1 entry to the database).
The current code reset it already if a node was delivered,
which is not stong enough, because unless it ends up written
to disk, an attacker can just loop and attack ad infinitum.

The commit also fixes a regression around state deliveries
and timeouts. The old downloader tracked if a delivery is
stale (none of the deliveries were requestedt), in which
case it didn't mark the node idle and did not send further
requests, since it signals a past timeout. The current code
did mark it idle even on stale deliveries, which eventually
caused two requests to be in flight at the same time, making
the deliveries always stale and mass duplicating retrievals
between multiple peers.

* eth/downloader: fix state request leak

This commit fixes the hang seen sometimes while doing the state
sync. The cause of the hang was a rare combination of events:
request state data from peer, peer drops and reconnects almost
immediately. This caused a new download task to be assigned to
the peer, overwriting the old one still waiting for a timeout,
which in turned leaked the requests out, never to be retried.
The fix is to ensure that a task assignment moves any pending
one back into the retry queue.

The commit also fixes a regression with peer dropping due to
stalls. The current code considered a peer stalling if they
timed out delivering 1 item. However, the downloader never
requests only one, the minimum is 2 (attempt to fine tune
estimated latency/bandwidth). The fix is simply to drop if
a timeout is detected at 2 items.

Apart from the above bugfixes, the commit contains some code
polishes I made while debugging the hang.

* core, eth, trie: support batched trie sync db writes

* trie: rename SyncMemCache to syncMemBatch
2017-06-22 15:26:03 +03:00
Péter Szilágyi
96c2ab22e0
eth/downloader: fix a potential issue against future refactors 2017-05-02 16:14:35 +03:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Felix Lange
c213fd1fd8 all: import "context" instead of "golang.org/x/net/context"
There is no need to depend on the old context package now that the
minimum Go version is 1.7. The move to "context" eliminates our weird
vendoring setup. Some vendored code still uses golang.org/x/net/context
and it is now vendored in the normal way.

This change triggered new vet checks around context.WithTimeout which
didn't fire with golang.org/x/net/context.
2017-03-22 20:49:15 +01:00
Péter Szilágyi
9b84caf3a5 core, eth, les: support resuming fast sync on heavy rollback (#3743) 2017-03-22 01:37:24 +01:00
Péter Szilágyi
9184249b39 Logger updates 3 (#3730)
* accounts, cmd, eth, ethdb: port logs over to new system

* ethdb: drop concept of cache distribution between dbs

* eth: fix some log nitpicks to make them nicer
2017-03-02 14:06:16 +01:00
Péter Szilágyi
2f28a12cdb
common, eth/downloader, log: support terminal log formatting 2017-02-27 19:15:18 +02:00
Péter Szilágyi
8676aeb798
eth/downloader: review fixes 2017-02-27 13:22:33 +02:00
Péter Szilágyi
46eea4d105
accounts, eth/downloader: use "err" instead of "error" in logs 2017-02-27 13:17:58 +02:00
Péter Szilágyi
0a63c3e362
eth/downloader: port over old logs from glog to log15 2017-02-27 13:16:40 +02:00
Péter Szilágyi
61e6bb1247
eth, les, swarm: fix go vet issues sufraced by log15 2017-02-23 12:16:45 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Jeffrey Wilcke
c12f4df910 params: core, core/vm, miner: 64bit gas instructions
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-13 21:44:25 +01:00
Jeffrey Wilcke
57f4e90257 Revert "params: core, core/vm, miner: 64bit gas instructions (#3514)"
This reverts commit 8b57c49490.
2017-02-13 15:15:12 +01:00
Jeffrey Wilcke
8b57c49490 params: core, core/vm, miner: 64bit gas instructions (#3514)
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-02 15:25:42 +01:00
Felix Lange
f1069a30b9 eth/downloader: improve deliverNodeData (#3588)
Commit d3b751e accidentally deleted a crucial 'return' statement,
leading to a crash in case of an issue with node data. This change
improves the fix in PR #3591 by removing the lock entirely.
2017-01-24 13:20:37 +01:00
Péter Szilágyi
ebc3d232f4 eth/downloader: fix mutex regression causing panics on fail (#3591) 2017-01-20 01:12:14 +01:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Péter Szilágyi
59b8245bbc Merge pull request #3516 from fjl/types-drop-sign-ecdsa
core/types: remove redundant SignECDSA wrappers, rename to SignTx
2017-01-06 15:55:55 +02:00
Felix Lange
d3b751e4d9 trie: remove dependency on ethdb
This removes the core/types -> leveldb dependency.
2017-01-06 14:15:22 +01:00
Felix Lange
e171bf74f8 core/types: remove redundant SignECDSA wrappers, rename to SignTx 2017-01-05 12:59:17 +01:00
Jeffrey Wilcke
4dca5d4db7 core/types, params: EIP#155 2016-11-13 14:55:30 +01:00
Jeffrey Wilcke
445feaeef5 core, core/state, trie: EIP158, reprice & skip empty account write
This commit implements EIP158 part 1, 2, 3 & 4

1. If an account is empty it's no longer written to the trie. An empty
  account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
  reation costs.

params: moved core/config to params

Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
2016-11-13 10:44:04 +01:00
zsfelfoldi
7db7109a5b cmd, eth: added light client and light server modes 2016-11-09 02:12:53 +01:00
Péter Szilágyi
e1b4acfb6e
eth/downloader: clear pending requests when switching trie root 2016-11-01 12:53:33 +02:00
Péter Szilágyi
8ecee175f2
eth/downloader: lower a state sync log level (users freak out) 2016-10-31 18:28:03 +02:00
Péter Szilágyi
8b10617bba
eth/downloader: reduce fast sync block requirements, fix test 2016-10-31 17:26:44 +02:00
Péter Szilágyi
90b16a3e85
core/state, eth/downloader, trie: reset fast-failure on progress 2016-10-31 14:19:14 +02:00
Péter Szilágyi
53db80da89
eth/downloader: fix a data race in a log output 2016-10-21 13:09:27 +03:00
Péter Szilágyi
1291778032
cmd/geth, code, eth/downloader: tune import logs and mem stats 2016-10-21 12:23:39 +03:00
Péter Szilágyi
64500ab0fa
common, core, eth/downloader: adjust import log formatting 2016-10-18 13:16:36 +03:00
Felix Lange
40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Péter Szilágyi
2924fdfcf7 ethereum, ethclient: add SyncProgress API endpoint 2016-09-06 13:41:43 +03:00
Bas van Kervel
47ff813012 rpc: refactor subscriptions and filters 2016-08-17 12:59:58 +02:00
Péter Szilágyi
89a3fbc0fb Merge pull request #2866 from karalabe/downloader-future-ancestors
eth/downloader: fewer headers and futures too in ancestor lookup
2016-08-16 10:01:52 +03:00
Péter Szilágyi
b46b36729f Merge pull request #2868 from karalabe/downloader-abort-master-drop
eth/downloader: abort sync if master drops (timeout prev)
2016-08-09 11:39:24 +03:00
Péter Szilágyi
893fabd336 Merge pull request #2867 from karalabe/dao-challenge-finish
eth, eth/downloader: don't forward the DAO challenge header
2016-08-09 11:39:13 +03:00
Péter Szilágyi
44ea0da2b0 Merge pull request #2861 from karalabe/track-peer-heads-properly
eth, eth/downloader: better remote head tracking
2016-08-09 11:38:34 +03:00
Péter Szilágyi
8f0a4a25f8 eth/downloader: abort sync if master drops (timeout prev) 2016-07-26 13:07:12 +03:00
Péter Szilágyi
071af57bcf eth, eth/downloader: don't forward the DAO challenge header 2016-07-26 12:26:41 +03:00
Péter Szilágyi
d68865f3b1 eth/downloader: fewer headers and futures too un ancestor lookup 2016-07-26 11:15:38 +03:00
Péter Szilágyi
1dd272080d eth, eth/downloader: better remote head tracking 2016-07-25 15:14:14 +03:00
Felix Lange
3e3a79ea13 Merge pull request #2855 from karalabe/downloader-fix-stall-drop
eth/downloader: fix the stall checks/drops during sync
2016-07-25 12:56:11 +02:00
Péter Szilágyi
c7c82f1b44 eth/downloader: fix the stall checks/drops during sync 2016-07-22 17:55:46 +03:00
Felix Lange
016007bd25 eth, eth/downloader, eth/fetcher: delete eth/61 code
The eth/61 protocol was disabled in #2776, this commit removes its
message handlers and hash-chain sync logic.
2016-07-22 13:17:19 +02:00
Péter Szilágyi
3291235711 accounts, core, eth: pass chain config for chain maker to test DAO 2016-07-15 16:52:55 +03:00
Péter Szilágyi
a691aa2a13 eth/downloader: return invalid chain (peer drop) on import fails 2016-07-11 19:00:49 +03:00
Péter Szilágyi
783289068a eth/downloader: fix occasional fast sync critical section test fails 2016-06-13 13:04:19 +03:00
Péter Szilágyi
88f174a014 eth/downloader: adaptive quality of service tuning 2016-06-06 14:20:57 +03:00
Péter Szilágyi
61ee9f299d eth/downloader: make fast sync resilient to critical section fails 2016-06-02 12:37:14 +03:00
Péter Szilágyi
16a23ff740 Merge pull request #2637 from karalabe/downloader-always-cancel
eth/downloader: ensure cancel channel is closed post sync
2016-05-31 14:48:52 +03:00
Péter Szilágyi
4496a44f68 eth/downloader: ensure cancel channel is closed post sync 2016-05-30 12:01:50 +03:00
Péter Szilágyi
4f1d92b332 eth/downloader, trie: pull head state concurrently with chain 2016-05-27 14:26:00 +03:00
Péter Szilágyi
8906b2fe09 eth/downloader: fix reviewer comments 2016-05-20 10:56:01 +03:00
Péter Szilágyi
e86619e75d eth/downloader: stream partial skeleton filling to processor 2016-05-17 10:03:34 +03:00
Péter Szilágyi
b40dc8a1da eth/downloader: implement concurrent header downloads 2016-05-17 10:03:34 +03:00
Péter Szilágyi
39ce85cf5d eth/downloader: bound fork ancestry and allow heavy short forks 2016-05-13 12:12:13 +02:00
Felix Lange
a40e61b4ac rpc: remove NotifierContextKey
Context keys must have a unique type in order to prevent
any unintented clashes. The code used int(1) as key.

Fix it by implementing the pattern recommended by package context.
2016-04-15 18:10:52 +02:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Felix Lange
d04a2e7557 all: update license information 2016-04-15 09:48:05 +02:00
Bas van Kervel
f7328c5ecb rpc: add pub/sub support 2016-04-01 18:26:35 +02:00
Leif Jurvetson
434e4b31d8 core, eth: replace reorganiz with reorganis 2016-03-15 12:03:17 -07:00
Leif Jurvetson
bbbe2360d0 eth: various typos 2016-03-15 11:27:49 -07:00
Péter Szilágyi
e3f2b541f2 eth/downloader: fix a throughput estimation data race 2016-03-10 12:58:46 +02:00
Péter Szilágyi
64ee5763ee eth/downloader: fix premature exit before notifying all part fetchers 2016-02-29 14:22:28 +02:00
Felix Lange
26e72b2ccd eth/downloader: fix header download limiting
Fixes #2201
2016-02-26 15:24:51 +01:00
Péter Szilágyi
66b148dd8f Merge pull request #2251 from karalabe/increase-downlaoder-queue
eth/downloader: bump the download queue size to prevent starvation
2016-02-24 15:55:21 +02:00
Jeffrey Wilcke
483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Péter Szilágyi
93c0012000 eth/downloader: bump the download queue size to prevent starvation 2016-02-23 13:12:11 +02:00
Ricardo Catalinas Jiménez
436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Péter Szilágyi
216729009b eth/downloader: fix partial rollback and ancestor lookup 2016-02-19 11:17:02 +02:00
Péter Szilágyi
59cd60b266 eth, eth/downloader, jsre: surface state sync progress through the API 2016-02-10 14:30:41 +02:00
Péter Szilágyi
f1ec226d80 eth/downloader: raise pending state limit that prevented concurrency 2016-02-08 14:57:02 +02:00
Bas van Kervel
19b2640e89 rpc: migrated the RPC insterface to a new reflection based RPC layer 2016-01-26 13:51:50 +01:00
Péter Szilágyi
9e011ff1cd core, eth/downloader: ensure state presence in ancestor lookup 2016-01-04 16:27:23 +02:00
Péter Szilágyi
1f39746886 eth/downloader: throttling tests are time-sensitive, don't run parallel 2015-12-30 14:42:21 +02:00
Felix Lange
9be5d5cd90 eth/downloader: fix negative balance issue in tests
The test chain generated by makeChainFork included invalid uncle
headers, crashing the generator during the state commit.

The headers were invalid because they used the iteration counter as the
block number, even though makeChainFork uses a block with number > 0 as
the parent. Fix this by introducing BlockGen.Number, which allows
accessing the actual number of the block being generated.
2015-12-18 12:09:10 +01:00
Bas van Kervel
eae81465c1 rpc: new RPC implementation with pub/sub support 2015-12-14 16:34:05 +01:00
Péter Szilágyi
b6f5523bdc eth/downloader: fetch data proportionally to peer capacity 2015-11-19 17:01:39 +02:00
Felix Lange
dd09af27af eth/downloader: run tests in parallel 2015-11-19 14:18:35 +01:00
Felix Lange
b7b62d4b3c eth/downloader: also drain stateCh, receiptCh in eth/61 mode
State and receipt deliveries from a previous eth/62+ sync can hang if
the downloader has moved on to syncing with eth/61. Fix this by also
draining the eth/63 channels while waiting for eth/61 data.

A nicer solution would be to take care of the channels in a central
place, but that would involve a major rewrite.
2015-11-19 14:18:35 +01:00
Felix Lange
900da3d800 eth/downloader: don't hang for spurious deliveries
Unexpected deliveries could block indefinitely if they arrived at the
right time. The fix is to ensure that the cancellation channel is
always closed when the sync ends, unblocking any deliveries. Also remove
the atomic check for whether a sync is currently running because it
doesn't help and can be misleading.

Cancelling always seems to break the tests though. The downloader
spawned d.process whenever new data arrived, making it somewhat hard to
track when block processing was actually done. Fix this by running
d.process in a dedicated goroutine that is tied to the lifecycle of the
sync. d.process gets notified of new work by the queue instead of being
invoked all the time. This removes a ton of weird workaround code,
including a hairy use of atomic CAS.
2015-11-19 14:18:34 +01:00
Péter Szilágyi
b658a73ed5 eth/downloader: fix dysfunctional ignore list hidden by generic set 2015-11-04 13:11:52 +02:00
Péter Szilágyi
2019ed71b4 eth: don't block sync goroutines that short circuit 2015-10-28 16:41:01 +02:00
Péter Szilágyi
5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi
aa0538db0b eth: clean out light node notions from eth 2015-10-19 10:03:10 +03:00
Péter Szilágyi
a9d8dfc8e7 core, eth: roll back uncertain headers in failed fast syncs 2015-10-19 10:03:10 +03:00
Péter Szilágyi
b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Péter Szilágyi
ab27bee25a core, eth, trie: direct state trie synchronization 2015-10-19 10:03:09 +03:00
Péter Szilágyi
832b37c822 core, eth: receipt chain reconstruction 2015-10-19 10:03:09 +03:00
Péter Szilágyi
f186b39018 eth/downloader: add fast and light sync strategies 2015-10-19 10:03:09 +03:00
Péter Szilágyi
47f62a67aa eth/downloader: match capabilities when querying idle peers 2015-10-02 13:20:41 +03:00
Péter Szilágyi
f459a3f0ae eth/downloader: always send termination wakes, clean leftover 2015-09-23 12:39:17 +03:00
Péter Szilágyi
99b62f36b6 eth/downloader: header-chain order and ancestry check 2015-09-15 14:45:53 +03:00
Péter Szilágyi
0a7d059b6a eth, rpc: standardize the chain sync progress counters 2015-09-15 14:45:53 +03:00
Péter Szilágyi
cdc2662c40 core: split out TD from database and all internals 2015-09-11 17:42:25 +03:00
Péter Szilágyi
1f1d73ab74 eth/downloader: fix race causing occasional test failure 2015-09-01 16:11:14 +03:00
Péter Szilágyi
17f65cd1e5 eth: update metrics collection to handle eth/62 algos 2015-08-25 17:48:47 +03:00
Péter Szilágyi
47a7fe5d22 eth: port the synchronisation algo to eth/62 2015-08-25 17:48:47 +03:00
Péter Szilágyi
ca88e18f59 eth: kill off protocol eth/60 in preparation for eth/62 2015-08-24 17:57:28 +03:00
Péter Szilágyi
42f44dda54 eth, eth/downloader: handle header requests, table driven proto tests 2015-08-24 17:57:28 +03:00
Péter Szilágyi
c51e153b5c eth, metrics, p2p: prepare metrics and net packets to eth/62 2015-08-21 10:30:57 +03:00
Péter Szilágyi
6adbaabc65 eth, eth/downloader: don't report stall if fetcher filled the block 2015-07-29 15:39:08 +03:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Péter Szilágyi
492d5454b1 eth/downloader: drop peer if advertised TD but won't delvier 2015-07-09 14:40:18 +03:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Péter Szilágyi
f857fb7600 eth/downloader: fix a rare test race on the OSX CI 2015-07-03 13:01:22 +03:00
Péter Szilágyi
d6f2c0a76f eth, eth/downloader: fix #1231, DOS vulnerability in hash queueing 2015-07-01 15:21:35 +03:00
Péter Szilágyi
60454da650 eth/downloader: reduce hash fetches in prep for eth/61 2015-07-01 01:20:49 +03:00
Péter Szilágyi
f43c07cb3c eth, eth/downloader: transition to eth 61 2015-06-30 19:05:06 +03:00
Péter Szilágyi
af51dc4d63 eth, eth/downloader: pass the eth protocol version through 2015-06-30 19:00:01 +03:00
Felix Lange
e0e5f74776 eth/downloader, eth/fetcher: use core.GenerateChain in tests
TestMadeupParentBlockChainAttack has been deleted because it was too
hard to port and the attack that it checks the prevention of is being
averted in a different way (through a protocol change).
2015-06-29 18:51:47 +02:00
Felix Lange
1d42888d30 core/types: make blocks immutable 2015-06-29 18:51:47 +02:00
Péter Szilágyi
5ec6ecc511 eth, eth/fetcher: move propagated block import into fetcher 2015-06-18 15:56:08 +03:00
Péter Szilágyi
2a1b722d04 eth/fetcher: fix timer reset bug, add initial tests 2015-06-18 15:56:07 +03:00
Péter Szilágyi
7c2af1c117 eth, eth/fetcher: separate notification sync mechanism 2015-06-18 15:56:07 +03:00
Péter Szilágyi
4365668462 eth/downloader: extend slow test to fix even slower CI server... 2015-06-18 00:42:02 +03:00
Péter Szilágyi
55dd8fd621 eth/downloader: always reenter processing if not exiting 2015-06-18 00:26:54 +03:00
Péter Szilágyi
2f4cbe22f5 eth, eth/downloader: fix processing interrupt caused by temp cancel 2015-06-18 00:04:57 +03:00
Péter Szilágyi
4a1e82cf3f eth/downloader: fix #1280, overlapping (good/bad) delivery hang 2015-06-17 12:03:16 +03:00
Péter Szilágyi
cf7c44a7f6 eth/downloader: detailed comment for the race corner case 2015-06-15 15:18:04 +03:00
Péter Szilágyi
9c03c374e3 eth/downloader: fix import statistic reset, fetch hashes async 2015-06-15 13:05:01 +03:00
Péter Szilágyi
b240983e2b eth, eth/downloader: do async block fetches, add dl tests 2015-06-15 12:26:05 +03:00
Péter Szilágyi
30a9939388 eth/downloader: sanity test for multi peer syncs 2015-06-15 09:22:37 +03:00
Péter Szilágyi
fc7abd9886 eth, eth/downloader: move block processing into the downlaoder 2015-06-15 09:22:37 +03:00
Péter Szilágyi
0fc71877a7 eth/downloader: add valid peer during attacks (check interference) 2015-06-15 09:22:37 +03:00
Péter Szilágyi
80833f8137 eth/downloader: instreument and test the sync peer drop 2015-06-15 09:22:37 +03:00
Péter Szilágyi
2dd6a62f67 eth/downloader: support individual peers in the test suite 2015-06-15 09:22:36 +03:00
Péter Szilágyi
2937903299 eth/downloader: remove uneeded testing functions 2015-06-15 09:22:36 +03:00
Péter Szilágyi
66d3dc8690 eth, eth/downloader: move peer removal into downloader 2015-06-15 09:22:36 +03:00
Péter Szilágyi
3c1cccc801 eth/downloader: fetch the block hashes on the fly, when needed 2015-06-10 20:12:22 +03:00
Péter Szilágyi
271fb20ecb cmd/geth, eth/downloader: rough guess at the import eta 2015-06-10 18:01:05 +03:00
Péter Szilágyi
b3d5ce7d48 cmd/geth, eth/downloader: collect and report import progress too 2015-06-10 01:20:35 +03:00
Felix Lange
8dc3048f65 eth/downloader: fix hash fetch timeout handling
Fixes #1206
2015-06-09 17:07:10 +02:00
Péter Szilágyi
4ed3509a02 eth/downloader: test registration rejection on head ban 2015-06-08 15:02:52 +03:00
Péter Szilágyi
c4f224932f eth/downloader: reject peer registration if head is banned 2015-06-08 14:46:31 +03:00
Péter Szilágyi
63c6cedb14 eth/downloader: cap the hash ban set, add test for it 2015-06-08 14:12:00 +03:00
Péter Szilágyi
4b2dd44711 eth/downloader: fix throttling test to be less timing dependent 2015-06-08 13:23:58 +03:00
Péter Szilágyi
2d627995cf eth/downloader: fix another rebase error 2015-06-08 13:23:58 +03:00
Péter Szilágyi
b40c796ff7 eth/downloader: preallocate the block cache 2015-06-08 13:23:58 +03:00
Péter Szilágyi
1d7bf3d39f eth/downloader: fix merge compile error 2015-06-08 13:23:58 +03:00
Péter Szilágyi
6d497f61c6 eth/downloader: don't block hash deliveries while pulling blocks 2015-06-08 13:23:58 +03:00
Péter Szilágyi
9da0232eef eth/downloader: update test for shitty travis 2015-06-08 13:23:58 +03:00
Péter Szilágyi
0275fcb3d3 eth/downloader: clean up and simplify the code a bit 2015-06-08 13:23:58 +03:00
Péter Szilágyi
abdfcda4dd eth/downloader: short circuit sync if head hash is banned 2015-06-08 13:23:58 +03:00
Péter Szilágyi
84bc93d8cb eth/downloader: accumulating hash bans for reconnecting attackers 2015-06-08 13:23:58 +03:00
Péter Szilágyi
eedb25b22a eth/downloader: clean up tests and unused variables 2015-06-08 13:23:57 +03:00
Jeffrey Wilcke
b94a76d17e Merge pull request #1189 from karalabe/downloader-polishes
eth/downloader: handle timeouts more gracefully
2015-06-05 08:31:57 -07:00
Péter Szilágyi
94e525ae12 eth, eth/downloader: fix #1098, elevate empty hash errors to peer drops 2015-06-05 12:52:48 +03:00
Péter Szilágyi
328ef60b85 eth/downloader: differentiate stale and nonexistent deliveries 2015-06-05 12:37:48 +03:00
Péter Szilágyi
94e4aa6ea9 eth/downloader: log hard timeouts and reset capacity 2015-06-05 11:53:46 +03:00
Péter Szilágyi
d754c25cc8 eth/downloader: drop log entry from peer, it's covered already 2015-06-04 16:22:55 +03:00
Péter Szilágyi
24cca2f18d eth/downloader: log after state updates, easier to debug 2015-06-04 15:10:43 +03:00
Péter Szilágyi
28c32d1b1b eth/downloader: fix #1178, don't request blocks beyond the cache bounds 2015-06-04 14:51:14 +03:00
Péter Szilágyi
3ec159ab6b eth/downloader: demote peers if they exceed the soft limits at 1 blocks already 2015-06-03 15:43:12 +03:00
Péter Szilágyi
c9a546c310 eth/downloader: add a basic block download congestion control 2015-06-03 14:40:11 +03:00
Péter Szilágyi
29b0480cfb core, eth/downloader: expose the bad hashes, check in downloader 2015-05-28 14:03:10 +03:00
Péter Szilágyi
3083ec5e32 eth/downloader: silence "Added N blocks from..." if N == 0 2015-05-26 16:10:28 +03:00
Péter Szilágyi
eafdc1f8e3 eth, eth/downloader: surface downloaded block origin, drop on error 2015-05-26 14:00:21 +03:00
Jeffrey Wilcke
af28736bd0 Merge pull request #1064 from karalabe/downloader-attacks
Fix two additional download vulnerabilities
2015-05-21 09:00:12 -07:00
Péter Szilágyi
06a041589f eth, eth/downloader: remove duplicate consts, bump hash fetch to 2K 2015-05-21 18:16:04 +03:00
Péter Szilágyi
52db6d8be5 eth/downloader: circumvent a forged block chain with known parent attack 2015-05-21 08:37:27 +03:00
Péter Szilágyi
e8b22b9253 eth/downloader: prevent a peer from dripping bad hashes 2015-05-21 08:07:58 +03:00
Péter Szilágyi
6f54eb6d9a eth/downloader: fix test to it doesn't time out on a slow machine 2015-05-20 10:15:42 +03:00
obscuren
c2ef8682fe eth/downloader: moved start event
Start event has moved because it could possibly could stall the miner
2015-05-16 13:02:30 +02:00
Jeffrey Wilcke
82c0780f81 Merge pull request #996 from karalabe/fix-potential-crosscheck-race
eth/downloader: circumvent download race between crosscheck and hashes
2015-05-15 11:23:44 -07:00
Péter Szilágyi
412cf98bbc eth/downloader: fix #992, where tests may time out on a slow machine 2015-05-15 21:16:42 +03:00
Péter Szilágyi
f3ae8f50a5 eth/downloader: circumvent download race between crosscheck and hashes 2015-05-15 20:54:10 +03:00
Péter Szilágyi
4f0d88cb02 eth/downloader: fix cancel channel double close 2015-05-15 19:43:42 +03:00
Péter Szilágyi
5c1a7b965c eth/downloader: circumvent a fake blockchain attack 2015-05-15 15:04:15 +03:00
Péter Szilágyi
b517967f86 eth/downloader: don't penalize for stale cross checks 2015-05-15 15:03:08 +03:00
Péter Szilágyi
83226762c2 eth, eth/downloader: detect and handle madeup hash attacks 2015-05-15 15:01:58 +03:00
Péter Szilágyi
72411eb24c eth/downloader: circumvent hash reordering attacks 2015-05-15 15:01:58 +03:00
Péter Szilágyi
bcc2980179 eth/downloader: check sync after failed attacks 2015-05-15 15:01:58 +03:00
Péter Szilágyi
366e9627e8 eth/downloader: add a test for detecting missing blocks 2015-05-15 15:01:58 +03:00
Péter Szilágyi
cd2fb09051 eth, eth/downloader: prevent hash repeater attack 2015-05-15 15:01:58 +03:00
obscuren
cfb2b51b74 eth/downloader: fixed test 2015-05-15 12:47:32 +02:00
obscuren
b71091e337 eth, eth/downloader, miner: use download events to check miner start 2015-05-15 00:43:10 +02:00
Jeffrey Wilcke
060a07cf69 Merge pull request #974 from karalabe/downloader-fix-unknown-parent-attack
eth, eth/downloader: handle a potential unknown parent attack
2015-05-14 08:16:30 -07:00
Péter Szilágyi
fe87feccb1 eth/downloader: add a small additional check to the test 2015-05-14 15:44:54 +03:00
Péter Szilágyi
ebf1eb9359 eth/downloader: remove a previous leftover 2015-05-14 15:40:28 +03:00
Péter Szilágyi
3eda70c64c eth, eth/downloader: remove parent verification from the downlaoder 2015-05-14 15:38:49 +03:00
Péter Szilágyi
a4246c2da6 eth, eth/downloader: handle a potential unknown parent attack 2015-05-14 15:24:18 +03:00
Jeffrey Wilcke
2b06fe3eff Merge pull request #950 from karalabe/downloader-cancel-sync
eth/downloader: fix deliveries to check for sync cancels
2015-05-14 04:05:13 -07:00
Péter Szilágyi
910a6d4e46 eth/downloader: fix nil panic caused by wrong variable use 2015-05-13 17:25:01 +03:00
Péter Szilágyi
de3a71cafd eth/downloader: remove a redundant sync progress check 2015-05-13 16:56:01 +03:00
Péter Szilágyi
48ee0777a5 eth/downloader: add a user sync notificaton 2015-05-13 16:03:05 +03:00
Péter Szilágyi
ec57aa64cd eth/downloader: sync the cancel channel during cancel too 2015-05-13 14:01:08 +03:00
Péter Szilágyi
ee0c892303 eth/downloader: fix deliveries to check for sync cancels 2015-05-13 13:47:21 +03:00
Péter Szilágyi
d2d5dbc6fb eth/downloader: fix active peer shadowing, polish func names 2015-05-13 13:13:43 +03:00
obscuren
48bd48876c eth, eth/downloader: moved pending queue error message to debug 2015-05-11 17:27:34 +02:00
Péter Szilágyi
064cf16099 eth/downloader: use count instead of peers, clearer 2015-05-11 17:21:22 +02:00
Péter Szilágyi
d37a2559b9 eth/downloader: revert to demotion, use harsher penalty 2015-05-11 17:21:22 +02:00
Péter Szilágyi
70c65835f4 eth/downloader: fix #910, thread safe peers & polishes 2015-05-11 17:21:22 +02:00
obscuren
05715f27cf eth: added a cancel method for the downloader
Added a cancel method to the downloader which gracefully shuts down any
active syncing process (hash fetching or block downloading) and resets
the queue and remove any pending blocks.

Issue with the downloader which would stall because of an active ongoing
process when an invalid block was found.
2015-05-10 00:34:07 +02:00
Péter Szilágyi
edad47bf0e eth/downloader: fix leftover state between syncs 2015-05-08 17:39:03 +03:00
Péter Szilágyi
bd5720f480 eth, eth/downloader: handle sync errors a bit more gracefully 2015-05-08 15:22:48 +03:00
Péter Szilágyi
9d188f73b5 eth, eth/downloader: make synchronize thread safe 2015-05-07 21:07:20 +03:00
Péter Szilágyi
43901c9282 eth/downloader: fix priority queue reset, add throttling test 2015-05-07 14:40:50 +03:00
Péter Szilágyi
45f8304f3c eth/downloader: fix expiration not running while fetching 2015-05-07 12:59:19 +03:00
Péter Szilágyi
4800c94392 eth/downloader: prioritize block fetch based on chain position, cap memory use 2015-05-07 10:57:34 +03:00
obscuren
1470b22e90 downloader: hash downloading recovery
If a peer fails to respond (disconnect, etc) during hash downloading
switch to a different peer which has it's current_hash in the queue's
peer set.
2015-05-03 16:27:03 +02:00
obscuren
f2a2b2ac70 eth/downloader: put back hashes on block overflow error 2015-05-03 14:11:00 +02:00
obscuren
493181ea09 eth/downloader: changed has to blockHashes for chain linking 2015-05-03 14:10:31 +02:00
obscuren
25bc88113f eth/downloader: added additional tests 2015-05-03 14:09:33 +02:00
obscuren
016f152b36 eth, eth/downloader: Moved block processing & graceful shutdown
The downloader is no longer responsible for processing blocks. The
eth-protocol handler now takes care of this instead.

Added graceful shutdown during block processing. Closes #846
2015-05-01 15:58:44 +02:00
obscuren
8595198c1b eth/downloader: delete blocks from queue 2015-04-30 17:51:47 +02:00
obscuren
9b6e8f6195 eth, eth/downloader: remove bad peers from peer set
Peers in the eth protocol handler are now being ignored for catch up.
2015-04-30 12:38:16 +02:00
obscuren
f8c27d7159 eth/downloader: drop block 2015-04-29 19:55:04 +02:00
obscuren
c930045834 core: fixed tetst to reflect (int, error) return by insertChain 2015-04-29 15:27:04 +02:00
obscuren
dfbf580354 eth/downloader: ignore orphan blocks in the downloader.
When blocks have been sorted and are being processed, orphan blocks
should be ignored and thrown out. The protocol handler is responsible
for downloading blocks which have missing parents.
2015-04-29 14:49:37 +02:00
obscuren
735b029db9 core: return the index of the block that failed when inserting a chain 2015-04-29 14:00:24 +02:00
obscuren
13364a2dcf eth/downloader: reset the chain when parent is missing during processing 2015-04-29 13:40:39 +02:00
obscuren
4b7bdc3766 eth: check if downloader is busy before showing log message 2015-04-29 12:51:05 +02:00
obscuren
7ce3d06402 eth/downloader: removed update loop and synch channel 2015-04-24 18:28:46 +02:00
obscuren
a9e4b96573 eth/downloader: fixed tests 2015-04-24 15:56:18 +02:00
obscuren
d84c2202e7 eth, eth/downloader: simplified synchronisation process 2015-04-24 15:38:10 +02:00
obscuren
bd9c76097d eth/downloader: removed peer td management and best peer selection 2015-04-24 15:37:07 +02:00
obscuren
31f82eb334 eth, eth/downloader: don't require td on downloader. Fixed tests 2015-04-24 15:04:58 +02:00
obscuren
b86e7526e1 eth, eth/downloader: moved peer selection to protocol handler 2015-04-24 14:40:32 +02:00
obscuren
45da3e17e2 core: added chain head reset to known block 2015-04-20 12:29:02 +02:00
obscuren
5dd56bb474 geth: admin download status 2015-04-19 21:46:19 +02:00
obscuren
ed07ffcde5 downloader: fixed tests with low ttl 2015-04-19 17:37:01 +02:00
obscuren
dcf1a1988a moved 2015-04-19 17:14:15 +02:00
obscuren
cce4158cfc Removed debug println 2015-04-19 13:33:14 +02:00
obscuren
c58918c84a downloader: moved chunk ignoring. Fixes issue with catching up 2015-04-19 13:30:34 +02:00
obscuren
5c59d95532 downloader: defer peer reset after download 2015-04-19 01:39:37 +02:00
obscuren
86ecdcd5ff downloader: reset hashTtl on receive 2015-04-19 00:15:05 +02:00
obscuren
50e096e627 downloader: don't remove peers. keep them around 2015-04-18 23:56:08 +02:00
obscuren
a1d97ea4db typo 2015-04-18 20:35:49 +02:00
obscuren
6830ddb659 downloader: free up peers from work when the downloader resets 2015-04-18 20:25:55 +02:00
obscuren
7c5d50f627 downloader: throw an error if there are no peers available for download
If all peers have been tried during the block download process and some
hashes are unfetchable (available peers > 0 and fetching == 0) throw an
error so the process can be aborted.
2015-04-18 19:30:29 +02:00
obscuren
78e37e98e7 downloader: fixed a race condition for download status 2015-04-18 19:14:25 +02:00
obscuren
0d536734fe eth: adapted to new synchronous api of downloader's AddBlock 2015-04-18 18:57:59 +02:00
obscuren
c2c24b3bb4 downloader: improved downloading and synchronisation
* Downloader's peers keeps track of peer's previously requested hashes
  so that we don't have to re-request
* Changed `AddBlock` to be fully synchronous
2015-04-18 18:55:13 +02:00
obscuren
60613b57d1 downloader: make sure that hashes are only accepted from the active peer 2015-04-18 17:35:03 +02:00
obscuren
8244825bbf downloader: reset the queue if a peer response with an empty hash set 2015-04-18 15:14:12 +02:00
obscuren
8f873b762b downloader: all handlers check for isBusy 2015-04-18 03:15:26 +02:00