Commit Graph

330 Commits

Author SHA1 Message Date
Martin Holst Swende
39f502329f
internal/ethapi: don't set sender-balance to maxuint, fixes #16999 (#20783)
Prior to this change, eth_call changed the balance of the sender account in the
EVM environment to 2^256 wei to cover the gas cost of the call execution.
We've had this behavior for a long time even though it's super confusing.

This commit sets the default call gasprice to zero instead of updating the balance,
which is better because it makes eth_call semantics less surprising. Removing
the built-in balance assignment also makes balance overrides work as expected.
2020-03-23 18:21:23 +01:00
Péter Szilágyi
613af7ceea
Merge pull request #20152 from karalabe/snapshot-5
Dynamic state snapshots
2020-03-23 12:57:31 +02:00
gary rong
92f3405dae
eth, les: fix time sensitive unit tests (#20741) 2020-03-12 11:25:52 +01:00
gary rong
4fabd9cbd2
les: separate peer into clientPeer and serverPeer (#19991)
* les: separate peer into clientPeer and serverPeer

* les: address comments
2020-02-26 11:41:24 +02:00
Péter Szilágyi
06d4470b41
core: fix broken tests due to API changes + linter 2020-02-25 12:51:16 +02:00
gary rong
2e1ecc02bd
les, miner, accounts/abi/bind: fix load-sensitive unit tests (#20698) 2020-02-20 13:05:54 +01:00
Felix Lange
dae3aee5ff
les: add bootstrap nodes as initial discoveries (#20688) 2020-02-18 13:24:05 +02:00
Adam Schmideg
8045504abf
les: log disconnect reason when light server is not synced (#20643)
Co-authored-by: ligi <ligi@ligi.de>
2020-02-11 16:46:32 +01:00
gary rong
4d663d57d6 les: fix request serving metrics (#20507) 2020-01-08 15:08:56 +02:00
gary rong
c6285e6437 les/checkpointoracle: move oracle into its own package (#20508)
* les: move the checkpoint oracle into its own package

It's first step of refactor LES package. LES package
basically can be divided into LES client and LES server.
However both sides will use checkpoint package for
status retrieval and verification. So this PR moves
checkpoint oracle into a separate package

* les: address comments
2020-01-07 11:24:21 +02:00
Felföldi Zsolt
3bb6815fc1
les: do not disconnect another server (#20453) 2019-12-25 02:06:00 +01:00
Felix Lange
d90d1db609
eth/filters: remove use of event.TypeMux for pending logs (#20312) 2019-12-10 12:39:14 +01:00
Martin Holst Swende
bc01593afb consensus/ethash, params: eip-2384: bump difficulty bomb (#20347)
* consensus/ethash, params: implement eip-2384: bump difficulty bomb

* params: EIP 2384 compat checks

* consensus, params: add Muir Glacier block number (mainnet,ropsten) + official name

* core/forkid: forkid tests for muir glacier

* params/config: address review concerns

* params, core/forkid: review nitpicks

* cmd/geth,eth,les: add override option for muir glacier

* params: nit fix
2019-12-06 11:36:40 +02:00
Guillaume Ballet
c4844e9ee2
les: fix staticcheck warnings (#20371) 2019-11-27 09:49:41 +01:00
Felföldi Zsolt
878e35bfde les: fix clientInfo deadlock (#20395) 2019-11-26 12:17:15 +02:00
Felix Lange
9c6cf960b4
internal/web3ext, les: update clique JS and make it work with the light client (#20318)
Also fix the input formatter on clique_getSnapshot and clique_getSigners
so that integers as well as hex number strings are accepted.
2019-11-19 18:22:04 +01:00
Felföldi Zsolt
422604b438 les: rename UpdateBalance to AddBalance and simplify return format (#20304) 2019-11-18 12:42:49 +01:00
Felix Lange
689486449d build: use golangci-lint (#20295)
* build: use golangci-lint

This changes build/ci.go to download and run golangci-lint instead
of gometalinter.

* core/state: fix unnecessary conversion

* p2p/simulations: fix lock copying (found by go vet)

* signer/core: fix unnecessary conversions

* crypto/ecies: remove unused function cmpPublic

* core/rawdb: remove unused function print

* core/state: remove unused function xTestFuzzCutter

* core/vm: disable TestWriteExpectedValues in a different way

* core/forkid: remove unused function checksum

* les: remove unused type proofsData

* cmd/utils: remove unused functions prefixedNames, prefixFor

* crypto/bn256: run goimports

* p2p/nat: fix goimports lint issue

* cmd/clef: avoid using unkeyed struct fields

* les: cancel context in testRequest

* rlp: delete unreachable code

* core: gofmt

* internal/build: simplify DownloadFile for Go 1.11 compatibility

* build: remove go test --short flag

* .travis.yml: disable build cache

* whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement

* .golangci.yml: enable goconst and ineffassign linters

* build: print message when there are no lint issues

* internal/build: refactor download a bit
2019-11-18 10:49:17 +02:00
Felföldi Zsolt
bf5c6b29fa les: implement server priority API (#20070)
This PR implements the LES server RPC API. Methods for server
capacity, client balance and client priority management are provided.
2019-11-13 23:47:03 +01:00
gary rong
b9bac1f384 les: fix and slim the unit tests of les (#20247)
* les: loose restriction of unit tests

* les: update unit tests

* les, light: slim the unit tests
2019-11-06 22:09:37 +01:00
gary rong
0ce5e113be les: rework clientpool (#20077)
* les: rework clientpool
2019-11-02 13:02:35 +01:00
Felix Lange
6e730915bd les: add empty "les" ENR entry for servers (#20145) 2019-10-02 14:14:27 +03:00
Ryan Schneider
ad03d9801c internal/ethapi: support block number or hash on state-related methods (#19491)
This change adds support for EIP-1898.
2019-09-26 10:47:31 +02:00
gary rong
32b07e8b1f les: fix checkpoint sync (#20120) 2019-09-25 10:05:15 +02:00
Felföldi Zsolt
0ac9bbba6c les: multiple server bugfixes (#20079)
* les: detailed relative cost metrics

* les: filter txpool relative request statistic

* les: initialize price factors

* les: increased connected bias to lower churn rate

* les: fixed clientPool.setLimits

* core: do not use mutex in GetAncestor

* les: bump factor db version again

* les: add metrics

* les, light: minor fixes
2019-09-17 16:28:41 +03:00
Felix Lange
b1c3010bf2
common/mclock: clean up AfterFunc support (#20054)
This change adds tests for the virtual clock and aligns the interface
with the time package by renaming Cancel to Stop. It also removes the
binary search from Stop because it complicates the code unnecessarily.
2019-09-16 11:16:30 +02:00
Felix Lange
39b0b1a1a6
all: make unit tests work with Go 1.13 (#20053)
Most of these changes are related to the Go 1.13 changes to test binary
flag handling. 

* cmd/geth: make attach tests more reliable

This makes the test wait for the endpoint to come up by polling
it instead of waiting for two seconds.

* tests: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* crypto/ecies: remove useless -dump flag in tests

* p2p/simulations: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* build: remove workaround for ./... vendor matching

This workaround was necessary for Go 1.8. The Go 1.9 release changed
the expansion rules to exclude vendored packages.

* Makefile: use relative path for GOBIN

This makes the "Run ./build/bin/..." line look nicer.

* les: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
2019-09-11 14:41:22 +02:00
gary rong
396f1dd87b les: fix panic (#20013) 2019-08-27 15:29:00 +02:00
gary rong
68502595f6 les: wait for all task goroutines before dropping the peer (#20010)
* les: wait all task routines before drop the peer

* les: address comments

* les: fix issue
2019-08-27 14:07:25 +03:00
Péter Szilágyi
e39b2a2bde
acmd, core, eth, les: support --override.istanbul 2019-08-23 12:09:27 +03:00
gary rong
2ed729d38e les: handler separation (#19639)
les: handler separation
2019-08-21 11:29:34 +02:00
gary rong
8657a0d6b5 les: degrade the log level (#19939) 2019-08-12 14:19:53 +03:00
Felföldi Zsolt
a7de796840
les: implement new client pool (#19745) 2019-08-03 14:36:10 +02:00
Péter Szilágyi
140a7e9177
accounts/abi/bind: support closing a simulated backend 2019-07-31 11:35:57 +03:00
gary rong
8812c4d3f9 eth, graphql, internal/ethapi, les: polish and improve graphql (#19886) 2019-07-25 09:29:53 +03:00
gary rong
530f78e22d eth, internal, les: add getHeaderBy* APIs (#19669)
* eth, interal, les: add getHeaderBy* APIs

* internal: address the comment

* eth, internal, les: getHeader nits, missing TD, console callable
2019-07-23 16:52:24 +03:00
gary rong
f08eb04896 les: get rid of testing tx journal (#19876) 2019-07-23 12:53:35 +03:00
gary rong
b80c840af3 core, les: fix les unit tests (#19823) 2019-07-22 15:45:40 +03:00
Péter Szilágyi
1a83114c74
all: update author list and licenses 2019-07-22 12:17:27 +03:00
大彬
4ac04ae0fe all: replace fmt.Print* calls with t.Log* in tests (#19670) 2019-07-17 13:20:24 +02:00
Péter Szilágyi
c705aac826
cmd, eth, les: make les flags conform to dotted style 2019-07-10 09:12:07 +03:00
Péter Szilágyi
213690cdfd
cmd, eth, les: fix up ultra light config integration 2019-07-09 20:34:42 +03:00
Martin Holst Swende
cdfe9a3a2a eth, les: add sanity checks for unbounded block fields (#19573)
This PR adds some hardening in the lower levels of the protocol stack, to bail early on invalid data. Primarily, attacks that this PR protects against are on the "annoyance"-level, which would otherwise write a couple of megabytes of data into the log output, which is a bit resource intensive.
2019-07-08 11:42:22 +02:00
gary rong
6814797173 accounts, cmd, contracts, les: integrate clef for transaction signing (#19783)
* accounts, cmd, contracts, les: integrate clef for transaction signing

* accounts, cmd/checkpoint-admin, signer/core: minor fixups
2019-07-03 22:54:59 +03:00
gary rong
59a3198382 les: remove half-finished priority pool APIs (#19780)
* les: remove half-finish APIs

* les: remove half-finish APIs
2019-07-03 21:23:06 +03:00
Péter Szilágyi
22411919da
cmd, eth, les, param: drop --override.constantinople 2019-07-02 14:14:59 +03:00
gary rong
f7cdea2bdc all: on-chain oracle checkpoint syncing (#19543)
* all: implement simple checkpoint syncing

cmd, les, node: remove callback mechanism

cmd, node: remove callback definition

les: simplify the registrar

les: expose checkpoint rpc services in the light client

les, light: don't store untrusted receipt

cmd, contracts, les: discard stale checkpoint

cmd, contracts/registrar: loose restriction of registeration

cmd, contracts: add replay-protection

all: off-chain multi-signature contract

params: deploy checkpoint contract for rinkeby

cmd/registrar: add raw signing mode for registrar

cmd/registrar, contracts/registrar, les: fixed messages

* cmd/registrar, contracts/registrar: fix lints

* accounts/abi/bind, les: address comments

* cmd, contracts, les, light, params: minor checkpoint sync cleanups

* cmd, eth, les, light: move checkpoint config to config file

* cmd, eth, les, params: address comments

* eth, les, params: address comments

* cmd: polish up the checkpoint admin CLI

* cmd, contracts, params: deploy new version contract

* cmd/checkpoint-admin: add another flag for clef mode signing

* cmd, contracts, les: rename and regen checkpoint oracle with abigen
2019-06-28 10:34:02 +03:00
Matthew Halpern
702f52fb99 les: prefer nil slices over zero-length slices (#19081) 2019-06-27 12:03:56 +03:00
gary rong
c8c3ebd593 les: reject client if it makes too many invalid requests (#19691)
* les: reject client connection if it makes too much invalid req

* les: address comments

* les: use uint32

* les: fix variable name

* les: add invalid counter for duplicate invalid req
2019-06-12 14:09:40 +03:00
Martin Holst Swende
3675b8545d
les/handler: avoid lookup missing state 2019-06-12 11:18:49 +02:00
gary rong
c0a034ec89 eth, les: reject stale request (#19689)
* eth, les: reject stale request

* les: reuse local head number
2019-06-11 10:40:32 +03:00
Felix Lange
e83c3ccc47
p2p/enode: improve IPv6 support, add ENR text representation (#19663)
* p2p/enr: add entries for for IPv4/IPv6 separation

This adds entry types for "ip6", "udp6", "tcp6" keys. The IP type stays
around because removing it would break a lot of code and force everyone
to care about the distinction.

* p2p/enode: track IPv4 and IPv6 address separately

LocalNode predicts the local node's UDP endpoint and updates the record.
This change makes it predict IPv4 and IPv6 endpoints separately since
they can now be in the record at the same time.

* p2p/enode: implement base64 text format
* all: switch to enode.Parse(...)

This allows passing base64-encoded node records to all the places that
previously accepted enode:// URLs. The URL format is still supported.

* cmd/bootnode, p2p: log node URL instead of ENR

...and return the base64 record in NodeInfo.
2019-06-07 15:31:00 +02:00
gary rong
f2612ac948 les: short circuit in the unregister if peer is not registered (#19644) 2019-05-31 10:54:50 +03:00
Felföldi Zsolt
58497f46bd
les, les/flowcontrol: implement LES/3 (#19329)
les, les/flowcontrol: implement LES/3
2019-05-30 20:51:13 +02:00
gary rong
4e0c1a1a6b eth, les: reject light client connection is server is not synced (#19616)
* eth, les: reject light client connection is server is not synced

* eth, les: rename function and variables

* les: format
2019-05-26 19:15:05 +03:00
Felföldi Zsolt
97d3615612
les: avoid fetcher deadlock on requestChn (#19571)
* les: avoid fetcher deadlock on requestChn
2019-05-17 20:39:39 +02:00
gary rong
80469bea0c
all: integrate the freezer with fast sync
* all: freezer style syncing

core, eth, les, light: clean up freezer relative APIs

core, eth, les, trie, ethdb, light: clean a bit

core, eth, les, light: add unit tests

core, light: rewrite setHead function

core, eth: fix downloader unit tests

core: add receipt chain insertion test

core: use constant instead of hardcoding table name

core: fix rollback

core: fix setHead

core/rawdb: remove canonical block first and then iterate side chain

core/rawdb, ethdb: add hasAncient interface

eth/downloader: calculate ancient limit via cht first

core, eth, ethdb: lots of fixes

* eth/downloader: print ancient disable log only for fast sync
2019-05-16 10:39:32 +03:00
Péter Szilágyi
9effd64290
core, eth, trie: bloom filter for trie node dedup during fast sync (#19489)
* core, eth, trie: bloom filter for trie node dedup during fast sync

* eth/downloader, trie: address review comments

* core, ethdb, trie: restart fast-sync bloom construction now and again

* eth/downloader: initialize fast sync bloom on startup

* eth: reenable eth/62 until we properly remove it
2019-05-13 15:28:01 +03:00
Felföldi Zsolt
40cdcf8c47 les, light: implement ODR transaction lookup by hash (#19069)
* les, light: implement ODR transaction lookup by hash

* les: delete useless file

* internal/ethapi: always use backend to find transaction

* les, eth, internal/ethapi: renamed GetCanonicalTransaction to GetTransaction

* light: add canonical header verification to GetTransaction
2019-05-13 14:41:10 +03:00
Felföldi Zsolt
f4fb1a1801 les: fixed cost table update (#19546) 2019-05-13 14:26:47 +03:00
Martin Holst Swende
5036992b06 eth, les: add error when accessing missing block state (#18346)
This change makes getBalance, getCode, getStorageAt, getProof,
call, getTransactionCount return an error if the block number in
the request doesn't exist. getHeaderByNumber still returns null
for missing headers.
2019-05-02 14:50:23 +02:00
gary rong
6269e5574c miner: polish miner configuration (#19480)
* cmd, eth, miner: disable advance sealing if user require

* cmd, console, miner, les, eth: wrap the miner config

* eth: remove todo

* cmd, miner: revert noadvance flag

The reason for this is: if the transaction execution is even longer
than block time, then this kind of transactions is DoS attack.
2019-04-23 10:08:51 +03:00
Péter Szilágyi
f496927a93
Merge pull request #19468 from karalabe/enforce-fastsync-checkpoints
eth, les, light: enforce CHT checkpoints on fast-sync too
2019-04-17 14:50:51 +03:00
Péter Szilágyi
38f6b85638
eth, les, light: enforce CHT checkpoints on fast-sync too 2019-04-17 13:16:15 +03:00
Felföldi Zsolt
921b3160db les: fix p2p.Protocol.PeerInfo (#19472) 2019-04-17 10:57:53 +03:00
Felföldi Zsolt
85b6823d16 les: check required message types in cost table (#19454) 2019-04-16 14:30:47 +03:00
Péter Szilágyi
7221cb1434
core, eth, les, light: scope receipt functionality a bit cleaner 2019-04-15 13:42:26 +03:00
Matthew Halpern
6b0ddd141e
core, eth, les, light: store transaction receipts without txHash and gasCost 2019-04-15 13:15:39 +03:00
Felföldi Zsolt
7c08e48141
Merge pull request #19403 from zsfelfoldi/remove-les1
les: remove support for LES/1
2019-04-08 14:04:14 +02:00
Martin Holst Swende
e2f3465e83 eth, les, geth: implement cli-configurable global gas cap for RPC calls (#19401)
* eth, les, geth: implement cli-configurable global gas cap for RPC calls

* graphql, ethapi: place gas cap in DoCall

* ethapi: reformat log message
2019-04-08 14:49:52 +03:00
Zsolt Felfoldi
64f9c1ea09 les, light: remove support for les/1 4096 block CHT sections 2019-04-08 13:17:24 +02:00
Zsolt Felfoldi
5515f364ae les: removed les/1 protocol messages 2019-04-08 13:17:24 +02:00
gary rong
d5cae48bae accounts, cmd, internal: disable unlock account on open HTTP (#17037)
* cmd, accounts, internal, node, rpc, signer: insecure unlock protect

* all: strict unlock API by rpc

* cmd/geth: check before printing warning log

* accounts, cmd/geth, internal: tiny polishes
2019-04-04 14:03:10 +03:00
Felföldi Zsolt
5164274872 les: extend error message for coinbase API calls (#19380) 2019-04-03 10:15:15 +03:00
Felföldi Zsolt
e852505ace les: fix block announcements (#19322) 2019-03-25 09:17:55 +02:00
Felföldi Zsolt
c53c5e616f les: fix peer id and reply error handling (#19289)
* les: fixed peer id format

* les: fixed peer reply error handling
2019-03-20 10:35:05 +02:00
Péter Szilágyi
211ec46284
les, light: verbose errors on state retrieval issues 2019-03-18 13:19:40 +02:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Felföldi Zsolt
c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
Matthew Halpern
badaf43019 les: remove redundant type specifiers (#19091) 2019-02-25 12:49:49 +02:00
gary rong
7fd0ccaa68 core: remove unnecessary fields in logs, receipts and tx lookups (#17106)
* core: remove unnecessary fields in log

* core: bump blockchain database version

* core, les: remove unnecessary fields in txlookup

* eth: print db version explicitly

* core/rawdb: drop txlookup entry struct wrapper
2019-02-21 15:14:35 +02:00
b00ris
769657060e les: implement ultralight client (#16904)
For more information about this light client mode, read
https://hackmd.io/s/HJy7jjZpm
2019-01-24 12:18:26 +01:00
Martin Holst Swende
83a9a73b89 cmd/geth, core, eth: implement Constantinople override flag (#18273)
* geth/core/eth: implement constantinople override flag

* les: implemnent constantinople override flag for les clients

* cmd/geth, eth, les: fix typo, move flag to experimentals
2018-12-11 14:19:03 +02:00
Paweł Bylica
de39513ced core, internal, eth, miner, les: Take VM config from BlockChain (#17955)
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.

This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.

Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
2018-12-06 14:34:49 +01:00
zah
55a4ff806f remove a no-op line in the code (#17760) 2018-11-29 10:56:59 +01:00
Felföldi Zsolt
f0515800e6 les: fix fetcher syncing logic (#18072) 2018-11-26 13:34:33 +02:00
Sheldon
b8a2ac3fcf les: fix pubkey index typo (#18093) 2018-11-15 11:10:45 +02:00
gary rong
c5d34fc94e les, light: reduce les testing stress (#17867) 2018-10-08 16:52:23 +03:00
Felföldi Zsolt
b7bbe66b19 les: limit state ODR retrievals to the last 100 blocks (#17744) 2018-10-01 15:14:53 +02:00
Felix Lange
30cd5c1854
all: new p2p node representation (#17643)
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.

Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.

The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.

* p2p/discover: port to p2p/enode

This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:

  - Table.Lookup is not available anymore. It used to take a public key
    as argument because v4 protocol requires one. Its replacement is
    LookupRandom.
  - Table.Resolve takes *enode.Node instead of NodeID. This is also for
    v4 protocol compatibility because nodes cannot be looked up by ID
    alone.
  - Types Node and NodeID are gone. Further commits in the series will be
    fixes all over the the codebase to deal with those removals.

* p2p: port to p2p/enode and discovery changes

This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.

New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.

* p2p/simulations, p2p/testing: port to p2p/enode

No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:

 - testing.ProtocolSession tracks complete nodes, not just their IDs.
 - adapters.NodeConfig has a new method to create a complete node.

These changes were needed to make swarm tests work.

Note that the NodeID change makes the code incompatible with old
simulation snapshots.

* whisper/whisperv5, whisper/whisperv6: port to p2p/enode

This port was easy because whisper uses []byte for node IDs and
URL strings in the API.

* eth: port to p2p/enode

Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.

* les: port to p2p/enode

Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.

* node: port to p2p/enode

This change simply replaces discover.Node and discover.NodeID with their
new equivalents.

* swarm/network: port to p2p/enode

Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).

There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.

Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
2018-09-25 00:59:00 +02:00
Felföldi Zsolt
ab13cd9924 les: fix invalid delivery handling in retriever (#17727) 2018-09-21 10:59:21 +03:00
Péter Szilágyi
f55c26ae6d
Merge pull request #17719 from karalabe/update-chts
les, light, params: update light client CHTs
2018-09-20 15:10:04 +03:00
gary rong
d6254f827b all: protect self-mined block during reorg (#17656) 2018-09-20 15:09:30 +03:00
Péter Szilágyi
af89093116
les, light, params: update light client CHTs 2018-09-20 14:14:48 +03:00
Felföldi Zsolt
c37238cae9 les: fix retriever logic (#17705) 2018-09-20 10:46:39 +03:00
Felföldi Zsolt
d4a28a13ca les: fix distReq.sentChn double close bug (#17639) 2018-09-14 22:14:29 +02:00
gary rong
c1c003e4ff consensus, miner: stale block mining support (#17506)
* consensus, miner: stale block supporting

* consensus, miner: refactor seal signature

* cmd, consensus, eth: add miner noverify flag

* cmd, consensus, miner: polish
2018-08-28 16:59:05 +03:00
gary rong
b69476b372 all: make indexer configurable (#17188) 2018-08-28 10:08:16 +03:00
gary rong
b2c644ffb5 cmd, eth, miner: make recommit configurable (#17444)
* cmd, eth, miner: make recommit configurable

* cmd, eth, les, miner: polish a bit

* miner: filter duplicate sealing work

* cmd: remove uncessary conversion

* miner: avoid microptimization in favor of cleaner code
2018-08-21 22:56:54 +03:00
Felföldi Zsolt
355fc47d39 les: fix CHT field in nodeInfo (#17465) 2018-08-21 14:58:10 +03:00
Felix Lange
2695fa2213 les: fix crasher in NodeInfo when running as server (#17419)
* les: fix crasher in NodeInfo when running as server

The ProtocolManager computes CHT and Bloom trie roots by asking the
indexers for their current head. It tried to get the indexers from
LesOdr, but no LesOdr instance is created in server mode.

Attempt to fix this by moving the indexers, protocol creation and
NodeInfo to a new lesCommons struct which is embedded into both server
and client.

All this setup code should really be cleaned up, but this is just a
hotfix so we have to do that some other time.

* les: fix commons protocol maker
2018-08-17 13:21:53 +03:00
Felföldi Zsolt
2cdf6ee7e0 light: CHT and bloom trie indexers working in light mode (#16534)
This PR enables the indexers to work in light client mode by
downloading a part of these tries (the Merkle proofs of the last
values of the last known section) in order to be able to add new
values and recalculate subsequent hashes. It also adds CHT data to
NodeInfo.
2018-08-15 22:25:46 +02:00
Felföldi Zsolt
b2ddb1fcbf les: implement client connection logic (#16899)
This PR implements les.freeClientPool. It also adds a simulated clock
in common/mclock, which enables time-sensitive tests to run quickly
and still produce accurate results, and package common/prque which is
a generalised variant of prque that enables removing elements other
than the top one from the queue.

les.freeClientPool implements a client database that limits the
connection time of each client and manages accepting/rejecting
incoming connections and even kicking out some connected clients. The
pool calculates recent usage time for each known client (a value that
increases linearly when the client is connected and decreases
exponentially when not connected). Clients with lower recent usage are
preferred, unknown nodes have the highest priority. Already connected
nodes receive a small bias in their favor in order to avoid accepting
and instantly kicking out clients.

Note: the pool can use any string for client identification. Using
signature keys for that purpose would not make sense when being known
has a negative value for the client. Currently the LES protocol
manager uses IP addresses (without port address) to identify clients.
2018-08-14 22:44:46 +02:00
Péter Szilágyi
f0998415ba
cmd, consensus/ethash, eth: miner push notifications 2018-08-10 09:06:59 +03:00
gary rong
51db5975cc consensus/ethash: move remote agent logic to ethash internal (#15853)
* consensus/ethash: start remote ggoroutine to handle remote mining

* consensus/ethash: expose remote miner api

* consensus/ethash: expose submitHashrate api

* miner, ethash: push empty block to sealer without waiting execution

* consensus, internal: add getHashrate API for ethash

* consensus: add three method for consensus interface

* miner: expose consensus engine running status to miner

* eth, miner: specify etherbase when miner created

* miner: commit new work when consensus engine is started

* consensus, miner: fix some logics

* all: delete useless interfaces

* consensus: polish a bit
2018-08-03 11:33:37 +03:00
Oleg Kovalov
d42ce0f2c1 all: simplify switches (#17267)
* all: simplify switches

* silly mistake
2018-07-30 12:30:09 +03:00
Péter Szilágyi
e1f1d3085c
accounts, eth, les: blockhash based filtering on all code paths 2018-07-12 18:16:54 +03:00
Felföldi Zsolt
51df1c1f20 les: add announcement safety check to light fetcher (#17034) 2018-07-04 13:40:20 +03:00
gary rong
4895665670 les: handle conn/disc/reg logic in the eventloop (#16981)
* les: handle conn/disc/reg logic in the eventloop

* les: try to dial before start eventloop

* les: handle disconnect logic more safely

* les: grammar fix
2018-06-25 11:52:24 +03:00
Felföldi Zsolt
25982375a8 les: fix retriever logic (#16776)
This PR fixes a retriever logic bug. When a peer had a soft timeout
and then a response arrived, it always assumed it was the same peer
even though it could have been a later requested one that did not time
out at all yet. In this case the logic went to an illegal state and
deadlocked, causing a goroutine leak.

Fixes #16243 and replaces #16359.
Thanks to @riceke for finding the bug in the logic.
2018-06-12 15:58:47 +02:00
Felföldi Zsolt
049f5b3572 core, eth, les: more efficient hash-based header chain retrieval (#16946) 2018-06-12 16:52:54 +03:00
Felföldi Zsolt
546d42179e les: pass server pool to protocol manager (#16947) 2018-06-12 14:00:52 +03:00
Felföldi Zsolt
a5237a27ea les: add Skip overflow check to GetBlockHeadersMsg handler (#16891) 2018-06-05 10:23:00 +03:00
kiel barry
0fe47e98c4 trie: fixes to comply with golint (#16771) 2018-05-21 23:41:31 +03:00
Péter Szilágyi
49719e21bc
core, eth: minor txpool event cleanups 2018-05-18 12:08:24 +03:00
rjl493456442
a2e43d28d0
all: collate new transaction events together 2018-05-18 11:46:44 +03:00
gary rong
7beccb29be all: get rid of error when creating memory database (#16716)
* all: get rid of error when create mdb

* core: clean up variables definition

* all: inline mdb definition
2018-05-09 15:24:25 +03:00
Péter Szilágyi
6cf0ab38bd
core/rawdb: separate raw database access to own package (#16666) 2018-05-07 14:35:06 +03:00
GagziW
7c02933275 les: changed if-else blocks to conform with golint (#16658) 2018-05-03 11:35:06 +03:00
thomasmodeneis
ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
cpusoft
e7cc5b4160 les: add ps.lock.Unlock() before return (#16360) 2018-04-11 11:02:33 +03:00
Andrey Petrov
2e9c8fd4fb eth, les: allow exceeding maxPeers for trusted peers (#16189)
Fixes #3326, #14472
2018-02-27 12:52:59 +02:00
Anton Evangelatov
ae9f97221a metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.

* vendor: change nonsense/go-metrics to ethersphere/go-metrics

* go-metrics: add tests. move ResettingTimer logic from reporter to type.

* all, metrics: pull in metrics package in go-ethereum

* metrics/test: make sure metrics are enabled for tests

* metrics: apply gosimple rules

* metrics/exp, internal/debug: init expvar endpoint when starting pprof server

* internal/debug: tiny comment formatting fix
2018-02-23 11:56:08 +02:00
Péter Szilágyi
5cf1d35470
eth, les, light: filter on logs only, derive receipts on demand 2018-02-22 19:12:43 +02:00
Felföldi Zsolt
1bdde620da les: fix light fetcher database race (#16103)
* les: fix light fetcher database race

* les: lightFetcher comments
2018-02-19 10:41:30 +02:00
Péter Szilágyi
dfc5842a89
les: add missing lock around peer access 2018-02-14 21:09:20 +02:00
Felix Lange
a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Péter Szilágyi
7a0019c63b
les, light: fix CHT trie retrievals (#16039)
* les, light: fix CHT trie retrievals

* les, light: minor polishes, test remote CHT retrievals

* les, light: deterministic nodeset rlp, bloombits test skeleton

* les: add an event emission to the les bloombits test

* les: drop dead tester code
2018-02-11 14:57:46 +02:00
Felföldi Zsolt
2f849ade82 les: fix server panic when discovery disabled (#16055) 2018-02-10 14:33:52 +02:00
Péter Szilágyi
55599ee95d core, trie: intermediate mempool between trie and database (#15857)
This commit reduces database I/O by not writing every state trie to disk.
2018-02-05 17:40:32 +01:00
Felföldi Zsolt
c3f238dd53 les: limit LES peer count and improve peer configuration logic (#16010)
* les: limit number of LES connections

* eth, cmd/utils: light vs max peer configuration logic
2018-02-05 15:41:53 +02:00
Péter Szilágyi
566d5c0777
core, eth, les, light: get rid of redundant methods 2018-01-30 18:42:00 +02:00
Felföldi Zsolt
23bca0f374 les: fix TxStatusMsg RLP coding (#15974) 2018-01-26 19:30:45 +02:00
Felföldi Zsolt
92580d69d3 p2p, p2p/discover, p2p/discv5: implement UDP port sharing (#15200)
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.

DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
2018-01-22 13:38:34 +01:00
Felföldi Zsolt
83d1657444 les: fix les/1 CHT compatibility issue (#15692) 2018-01-09 11:41:59 +01:00
Péter Szilágyi
6f69cdd109
all: switch gas limits from big.Int to uint64 2018-01-03 14:45:35 +02:00
Furkan KAMACI
b8caba9709 various: remove redundant parentheses (#15793) 2018-01-03 14:14:47 +02:00
Péter Szilágyi
f7ca03ae87 eth, les, light: expose chain config in les node info too (#15732) 2017-12-28 14:18:34 +01:00
gary rong
5f8888e116 accounts, consensus, core, eth: make chain maker consensus agnostic (#15497)
* accounts, consensus, core, eth: make chain maker consensus agnostic

* consensus, core: move CalcDifficulty to Engine interface

* consensus: add docs for calcDifficulty function

* consensus, core: minor comment fixups
2017-12-22 14:37:50 +02:00
gary rong
f14047dae5 cmd, consensus, eth: split ethash related config to it own (#15520)
* cmd, consensus, eth: split ethash related config to it own

* eth, consensus: minor polish

* eth, consenus, console: compress pow testing config field to single one

* consensus, eth: document pow mode
2017-11-24 16:10:27 +02:00
rjl493456442
41ef34ae40 les: use modified default txpool config to avoid creating journal file 2017-11-21 22:13:57 +08:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
b00ris
7ace023981 les: fix channel assignment data race (#15441) 2017-11-09 10:43:37 +01:00
Evgeny Danilenko
0914d4e0d2 les: fix misuse of WaitGroup (#15365) 2017-11-09 10:34:35 +01:00
ferhat elmas
9619a61024 all: gofmt -w -s (#15419) 2017-11-08 11:45:52 +01:00
Felföldi Zsolt
8d434f6a6f les, core/bloombits: post-LES/2 fixes (#15391)
* les: fix topic ID

* core/bloombits: fix interface conversion
2017-10-27 17:18:53 +03:00
Péter Szilágyi
0095531a58 core, eth, les: fix messy code (#15367)
* core, eth, les: fix messy code

* les: fixed tx status test and rlp encoding

* core: add a workaround for light sync
2017-10-25 12:18:44 +03:00
Felföldi Zsolt
ca376ead88 les, light: LES/2 protocol version (#14970)
This PR implements the new LES protocol version extensions:

* new and more efficient Merkle proofs reply format (when replying to
  a multiple Merkle proofs request, we just send a single set of trie
  nodes containing all necessary nodes)
* BBT (BloomBitsTrie) works similarly to the existing CHT and contains
  the bloombits search data to speed up log searches
* GetTxStatusMsg returns the inclusion position or the
  pending/queued/unknown state of a transaction referenced by hash
* an optional signature of new block data (number/hash/td) can be
  included in AnnounceMsg to provide an option for "very light
  clients" (mobile/embedded devices) to skip expensive Ethash check
  and accept multiple signatures of somewhat trusted servers (still a
  lot better than trusting a single server completely and retrieving
  everything through RPC). The new client mode is not implemented in
  this PR, just the protocol extension.
2017-10-24 15:19:09 +02:00
Felix Lange
10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Péter Szilágyi
f585f9eee8
core, eth: clean up bloom filtering, add some tests 2017-09-06 11:14:19 +03:00