Commit Graph

443 Commits

Author SHA1 Message Date
Felföldi Zsolt
b4a2681120
les, les/lespay: implement new server pool (#20758)
This PR reimplements the light client server pool. It is also a first step
to move certain logic into a new lespay package. This package will contain
the implementation of the lespay token sale functions, the token buying and
selling logic and other components related to peer selection/prioritization
and service quality evaluation. Over the long term this package will be
reusable for incentivizing future protocols.

Since the LES peer logic is now based on enode.Iterator, it can now use
DNS-based fallback discovery to find servers.

This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4
2020-05-22 13:46:34 +02:00
gary rong
53cac027d0
les: drop the message if the entire p2p connection is stuck (#21033)
* les: drop the message if the entire p2p connection is stuck

* les: fix lint
2020-05-12 11:02:15 +03:00
Boqin Qin
c43be6cf87
les: remove invalid use of t.Fatal in TestHandshake (#21012) 2020-05-01 13:48:52 +02:00
gary rong
44ff3f3dc9
trie: initial implementation for range proof (#20908)
* trie: initial implementation for range proof

* trie: add benchmark

* trie: fix lint

* trie: fix minor issue

* trie: unset the edge valuenode as well

* trie: unset the edge valuenode as nilValuenode
2020-04-24 14:37:56 +03:00
gary rong
b9df7ecdc3
all: seperate consensus error and evm internal error (#20830)
* all: seperate consensus error and evm internal error

There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.

This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.

And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails

* all: polish

* accounts/abi/bind/backends: add tests

* accounts/abi/bind/backends, internal: cleanup error message

* all: address comments

* core: fix lint

* accounts, core, eth, internal: address comments

* accounts, internal: resolve revert reason if possible

* accounts, internal: address comments
2020-04-22 11:25:36 +03:00
rene
648b0cb714
cmd, core: remove override muir glacier and override istanbul (#20942) 2020-04-20 12:46:38 +03:00
Martin Holst Swende
6402c42b67
all: simplify and fix database iteration with prefix/start (#20808)
* core/state/snapshot: start fixing disk iterator seek

* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start

* les, core/state/snapshot: iterator fixes

* all: remove two iterator methods

* all: rename Iteratee.NewIteratorWith -> NewIterator

* ethdb: fix review concerns
2020-04-15 14:08:53 +03:00
gary rong
00064ddcfb
accounts/abi: implement new fallback functions (#20764)
* accounts/abi: implement new fackball functions

In Solidity v0.6.0, the original fallback is separated
into two different sub types: fallback and receive.

This PR addes the support for parsing new format abi
and the relevant abigen functionalities.

* accounts/abi: fix unit tests

* accounts/abi: minor fixes

* accounts/abi, mobile: support jave binding

* accounts/abi: address marius's comment

* accounts/abi: Work around the uin64 conversion issue

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
2020-04-15 09:23:58 +02:00
Felföldi Zsolt
0851646e48
les, les/lespay/client: add service value statistics and API (#20837)
This PR adds service value measurement statistics to the light client. It
also adds a private API that makes these statistics accessible. A follow-up
PR will add the new server pool which uses these statistics to select
servers with good performance.

This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2020-04-09 11:55:32 +02:00
gary rong
f0b5eb09eb
eth, les: fix flaky tests (#20897)
* les: fix flaky test

* eth: fix flaky test
2020-04-07 09:16:21 +03:00
ucwong
4d891f23b5
les: add missing Ticker.Stop call (#20864) 2020-04-02 15:54:59 +02:00
gary rong
f78ffc0545
les: create utilities as common package (#20509)
* les: move execqueue into utilities package

execqueue is a util for executing queued functions
in a serial order which is used by both les server
and les client. Move it to common package.

* les: move randselect to utilities package

weighted_random_selector is a helpful tool for randomly select
items maintained in a set but based on the item weight.

It's used anywhere is LES package, mainly by les client but will
be used in les server with very high chance. So move it into a
common package as the second step for les separation.

* les: rename to utils
2020-03-31 17:17:24 +02:00
gary rong
62cd943c7b
les: fix dead lock (#20828) 2020-03-27 17:21:58 +02:00
Martin Holst Swende
39f502329f
internal/ethapi: don't set sender-balance to maxuint, fixes #16999 (#20783)
Prior to this change, eth_call changed the balance of the sender account in the
EVM environment to 2^256 wei to cover the gas cost of the call execution.
We've had this behavior for a long time even though it's super confusing.

This commit sets the default call gasprice to zero instead of updating the balance,
which is better because it makes eth_call semantics less surprising. Removing
the built-in balance assignment also makes balance overrides work as expected.
2020-03-23 18:21:23 +01:00
Péter Szilágyi
613af7ceea
Merge pull request #20152 from karalabe/snapshot-5
Dynamic state snapshots
2020-03-23 12:57:31 +02:00
gary rong
92f3405dae
eth, les: fix time sensitive unit tests (#20741) 2020-03-12 11:25:52 +01:00
gary rong
4fabd9cbd2
les: separate peer into clientPeer and serverPeer (#19991)
* les: separate peer into clientPeer and serverPeer

* les: address comments
2020-02-26 11:41:24 +02:00
Péter Szilágyi
06d4470b41
core: fix broken tests due to API changes + linter 2020-02-25 12:51:16 +02:00
gary rong
2e1ecc02bd
les, miner, accounts/abi/bind: fix load-sensitive unit tests (#20698) 2020-02-20 13:05:54 +01:00
Felix Lange
dae3aee5ff
les: add bootstrap nodes as initial discoveries (#20688) 2020-02-18 13:24:05 +02:00
Adam Schmideg
8045504abf
les: log disconnect reason when light server is not synced (#20643)
Co-authored-by: ligi <ligi@ligi.de>
2020-02-11 16:46:32 +01:00
gary rong
4d663d57d6 les: fix request serving metrics (#20507) 2020-01-08 15:08:56 +02:00
gary rong
c6285e6437 les/checkpointoracle: move oracle into its own package (#20508)
* les: move the checkpoint oracle into its own package

It's first step of refactor LES package. LES package
basically can be divided into LES client and LES server.
However both sides will use checkpoint package for
status retrieval and verification. So this PR moves
checkpoint oracle into a separate package

* les: address comments
2020-01-07 11:24:21 +02:00
Felföldi Zsolt
3bb6815fc1
les: do not disconnect another server (#20453) 2019-12-25 02:06:00 +01:00
Felix Lange
d90d1db609
eth/filters: remove use of event.TypeMux for pending logs (#20312) 2019-12-10 12:39:14 +01:00
Martin Holst Swende
bc01593afb consensus/ethash, params: eip-2384: bump difficulty bomb (#20347)
* consensus/ethash, params: implement eip-2384: bump difficulty bomb

* params: EIP 2384 compat checks

* consensus, params: add Muir Glacier block number (mainnet,ropsten) + official name

* core/forkid: forkid tests for muir glacier

* params/config: address review concerns

* params, core/forkid: review nitpicks

* cmd/geth,eth,les: add override option for muir glacier

* params: nit fix
2019-12-06 11:36:40 +02:00
Guillaume Ballet
c4844e9ee2
les: fix staticcheck warnings (#20371) 2019-11-27 09:49:41 +01:00
Felföldi Zsolt
878e35bfde les: fix clientInfo deadlock (#20395) 2019-11-26 12:17:15 +02:00
Felix Lange
9c6cf960b4
internal/web3ext, les: update clique JS and make it work with the light client (#20318)
Also fix the input formatter on clique_getSnapshot and clique_getSigners
so that integers as well as hex number strings are accepted.
2019-11-19 18:22:04 +01:00
Felföldi Zsolt
422604b438 les: rename UpdateBalance to AddBalance and simplify return format (#20304) 2019-11-18 12:42:49 +01:00
Felix Lange
689486449d build: use golangci-lint (#20295)
* build: use golangci-lint

This changes build/ci.go to download and run golangci-lint instead
of gometalinter.

* core/state: fix unnecessary conversion

* p2p/simulations: fix lock copying (found by go vet)

* signer/core: fix unnecessary conversions

* crypto/ecies: remove unused function cmpPublic

* core/rawdb: remove unused function print

* core/state: remove unused function xTestFuzzCutter

* core/vm: disable TestWriteExpectedValues in a different way

* core/forkid: remove unused function checksum

* les: remove unused type proofsData

* cmd/utils: remove unused functions prefixedNames, prefixFor

* crypto/bn256: run goimports

* p2p/nat: fix goimports lint issue

* cmd/clef: avoid using unkeyed struct fields

* les: cancel context in testRequest

* rlp: delete unreachable code

* core: gofmt

* internal/build: simplify DownloadFile for Go 1.11 compatibility

* build: remove go test --short flag

* .travis.yml: disable build cache

* whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement

* .golangci.yml: enable goconst and ineffassign linters

* build: print message when there are no lint issues

* internal/build: refactor download a bit
2019-11-18 10:49:17 +02:00
Felföldi Zsolt
bf5c6b29fa les: implement server priority API (#20070)
This PR implements the LES server RPC API. Methods for server
capacity, client balance and client priority management are provided.
2019-11-13 23:47:03 +01:00
gary rong
b9bac1f384 les: fix and slim the unit tests of les (#20247)
* les: loose restriction of unit tests

* les: update unit tests

* les, light: slim the unit tests
2019-11-06 22:09:37 +01:00
gary rong
0ce5e113be les: rework clientpool (#20077)
* les: rework clientpool
2019-11-02 13:02:35 +01:00
Felix Lange
6e730915bd les: add empty "les" ENR entry for servers (#20145) 2019-10-02 14:14:27 +03:00
Ryan Schneider
ad03d9801c internal/ethapi: support block number or hash on state-related methods (#19491)
This change adds support for EIP-1898.
2019-09-26 10:47:31 +02:00
gary rong
32b07e8b1f les: fix checkpoint sync (#20120) 2019-09-25 10:05:15 +02:00
Felföldi Zsolt
0ac9bbba6c les: multiple server bugfixes (#20079)
* les: detailed relative cost metrics

* les: filter txpool relative request statistic

* les: initialize price factors

* les: increased connected bias to lower churn rate

* les: fixed clientPool.setLimits

* core: do not use mutex in GetAncestor

* les: bump factor db version again

* les: add metrics

* les, light: minor fixes
2019-09-17 16:28:41 +03:00
Felix Lange
b1c3010bf2
common/mclock: clean up AfterFunc support (#20054)
This change adds tests for the virtual clock and aligns the interface
with the time package by renaming Cancel to Stop. It also removes the
binary search from Stop because it complicates the code unnecessarily.
2019-09-16 11:16:30 +02:00
Felix Lange
39b0b1a1a6
all: make unit tests work with Go 1.13 (#20053)
Most of these changes are related to the Go 1.13 changes to test binary
flag handling. 

* cmd/geth: make attach tests more reliable

This makes the test wait for the endpoint to come up by polling
it instead of waiting for two seconds.

* tests: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* crypto/ecies: remove useless -dump flag in tests

* p2p/simulations: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.

* build: remove workaround for ./... vendor matching

This workaround was necessary for Go 1.8. The Go 1.9 release changed
the expansion rules to exclude vendored packages.

* Makefile: use relative path for GOBIN

This makes the "Run ./build/bin/..." line look nicer.

* les: fix test binary flags for Go 1.13

Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
2019-09-11 14:41:22 +02:00
gary rong
396f1dd87b les: fix panic (#20013) 2019-08-27 15:29:00 +02:00
gary rong
68502595f6 les: wait for all task goroutines before dropping the peer (#20010)
* les: wait all task routines before drop the peer

* les: address comments

* les: fix issue
2019-08-27 14:07:25 +03:00
Péter Szilágyi
e39b2a2bde
acmd, core, eth, les: support --override.istanbul 2019-08-23 12:09:27 +03:00
gary rong
2ed729d38e les: handler separation (#19639)
les: handler separation
2019-08-21 11:29:34 +02:00
gary rong
8657a0d6b5 les: degrade the log level (#19939) 2019-08-12 14:19:53 +03:00
Felföldi Zsolt
a7de796840
les: implement new client pool (#19745) 2019-08-03 14:36:10 +02:00
Péter Szilágyi
140a7e9177
accounts/abi/bind: support closing a simulated backend 2019-07-31 11:35:57 +03:00
gary rong
8812c4d3f9 eth, graphql, internal/ethapi, les: polish and improve graphql (#19886) 2019-07-25 09:29:53 +03:00
gary rong
530f78e22d eth, internal, les: add getHeaderBy* APIs (#19669)
* eth, interal, les: add getHeaderBy* APIs

* internal: address the comment

* eth, internal, les: getHeader nits, missing TD, console callable
2019-07-23 16:52:24 +03:00
gary rong
f08eb04896 les: get rid of testing tx journal (#19876) 2019-07-23 12:53:35 +03:00
gary rong
b80c840af3 core, les: fix les unit tests (#19823) 2019-07-22 15:45:40 +03:00
Péter Szilágyi
1a83114c74
all: update author list and licenses 2019-07-22 12:17:27 +03:00
大彬
4ac04ae0fe all: replace fmt.Print* calls with t.Log* in tests (#19670) 2019-07-17 13:20:24 +02:00
Péter Szilágyi
c705aac826
cmd, eth, les: make les flags conform to dotted style 2019-07-10 09:12:07 +03:00
Péter Szilágyi
213690cdfd
cmd, eth, les: fix up ultra light config integration 2019-07-09 20:34:42 +03:00
Martin Holst Swende
cdfe9a3a2a eth, les: add sanity checks for unbounded block fields (#19573)
This PR adds some hardening in the lower levels of the protocol stack, to bail early on invalid data. Primarily, attacks that this PR protects against are on the "annoyance"-level, which would otherwise write a couple of megabytes of data into the log output, which is a bit resource intensive.
2019-07-08 11:42:22 +02:00
gary rong
6814797173 accounts, cmd, contracts, les: integrate clef for transaction signing (#19783)
* accounts, cmd, contracts, les: integrate clef for transaction signing

* accounts, cmd/checkpoint-admin, signer/core: minor fixups
2019-07-03 22:54:59 +03:00
gary rong
59a3198382 les: remove half-finished priority pool APIs (#19780)
* les: remove half-finish APIs

* les: remove half-finish APIs
2019-07-03 21:23:06 +03:00
Péter Szilágyi
22411919da
cmd, eth, les, param: drop --override.constantinople 2019-07-02 14:14:59 +03:00
gary rong
f7cdea2bdc all: on-chain oracle checkpoint syncing (#19543)
* all: implement simple checkpoint syncing

cmd, les, node: remove callback mechanism

cmd, node: remove callback definition

les: simplify the registrar

les: expose checkpoint rpc services in the light client

les, light: don't store untrusted receipt

cmd, contracts, les: discard stale checkpoint

cmd, contracts/registrar: loose restriction of registeration

cmd, contracts: add replay-protection

all: off-chain multi-signature contract

params: deploy checkpoint contract for rinkeby

cmd/registrar: add raw signing mode for registrar

cmd/registrar, contracts/registrar, les: fixed messages

* cmd/registrar, contracts/registrar: fix lints

* accounts/abi/bind, les: address comments

* cmd, contracts, les, light, params: minor checkpoint sync cleanups

* cmd, eth, les, light: move checkpoint config to config file

* cmd, eth, les, params: address comments

* eth, les, params: address comments

* cmd: polish up the checkpoint admin CLI

* cmd, contracts, params: deploy new version contract

* cmd/checkpoint-admin: add another flag for clef mode signing

* cmd, contracts, les: rename and regen checkpoint oracle with abigen
2019-06-28 10:34:02 +03:00
Matthew Halpern
702f52fb99 les: prefer nil slices over zero-length slices (#19081) 2019-06-27 12:03:56 +03:00
gary rong
c8c3ebd593 les: reject client if it makes too many invalid requests (#19691)
* les: reject client connection if it makes too much invalid req

* les: address comments

* les: use uint32

* les: fix variable name

* les: add invalid counter for duplicate invalid req
2019-06-12 14:09:40 +03:00
Martin Holst Swende
3675b8545d
les/handler: avoid lookup missing state 2019-06-12 11:18:49 +02:00
gary rong
c0a034ec89 eth, les: reject stale request (#19689)
* eth, les: reject stale request

* les: reuse local head number
2019-06-11 10:40:32 +03:00
Felix Lange
e83c3ccc47
p2p/enode: improve IPv6 support, add ENR text representation (#19663)
* p2p/enr: add entries for for IPv4/IPv6 separation

This adds entry types for "ip6", "udp6", "tcp6" keys. The IP type stays
around because removing it would break a lot of code and force everyone
to care about the distinction.

* p2p/enode: track IPv4 and IPv6 address separately

LocalNode predicts the local node's UDP endpoint and updates the record.
This change makes it predict IPv4 and IPv6 endpoints separately since
they can now be in the record at the same time.

* p2p/enode: implement base64 text format
* all: switch to enode.Parse(...)

This allows passing base64-encoded node records to all the places that
previously accepted enode:// URLs. The URL format is still supported.

* cmd/bootnode, p2p: log node URL instead of ENR

...and return the base64 record in NodeInfo.
2019-06-07 15:31:00 +02:00
gary rong
f2612ac948 les: short circuit in the unregister if peer is not registered (#19644) 2019-05-31 10:54:50 +03:00
Felföldi Zsolt
58497f46bd
les, les/flowcontrol: implement LES/3 (#19329)
les, les/flowcontrol: implement LES/3
2019-05-30 20:51:13 +02:00
gary rong
4e0c1a1a6b eth, les: reject light client connection is server is not synced (#19616)
* eth, les: reject light client connection is server is not synced

* eth, les: rename function and variables

* les: format
2019-05-26 19:15:05 +03:00
Felföldi Zsolt
97d3615612
les: avoid fetcher deadlock on requestChn (#19571)
* les: avoid fetcher deadlock on requestChn
2019-05-17 20:39:39 +02:00
gary rong
80469bea0c
all: integrate the freezer with fast sync
* all: freezer style syncing

core, eth, les, light: clean up freezer relative APIs

core, eth, les, trie, ethdb, light: clean a bit

core, eth, les, light: add unit tests

core, light: rewrite setHead function

core, eth: fix downloader unit tests

core: add receipt chain insertion test

core: use constant instead of hardcoding table name

core: fix rollback

core: fix setHead

core/rawdb: remove canonical block first and then iterate side chain

core/rawdb, ethdb: add hasAncient interface

eth/downloader: calculate ancient limit via cht first

core, eth, ethdb: lots of fixes

* eth/downloader: print ancient disable log only for fast sync
2019-05-16 10:39:32 +03:00
Péter Szilágyi
9effd64290
core, eth, trie: bloom filter for trie node dedup during fast sync (#19489)
* core, eth, trie: bloom filter for trie node dedup during fast sync

* eth/downloader, trie: address review comments

* core, ethdb, trie: restart fast-sync bloom construction now and again

* eth/downloader: initialize fast sync bloom on startup

* eth: reenable eth/62 until we properly remove it
2019-05-13 15:28:01 +03:00
Felföldi Zsolt
40cdcf8c47 les, light: implement ODR transaction lookup by hash (#19069)
* les, light: implement ODR transaction lookup by hash

* les: delete useless file

* internal/ethapi: always use backend to find transaction

* les, eth, internal/ethapi: renamed GetCanonicalTransaction to GetTransaction

* light: add canonical header verification to GetTransaction
2019-05-13 14:41:10 +03:00
Felföldi Zsolt
f4fb1a1801 les: fixed cost table update (#19546) 2019-05-13 14:26:47 +03:00
Martin Holst Swende
5036992b06 eth, les: add error when accessing missing block state (#18346)
This change makes getBalance, getCode, getStorageAt, getProof,
call, getTransactionCount return an error if the block number in
the request doesn't exist. getHeaderByNumber still returns null
for missing headers.
2019-05-02 14:50:23 +02:00
gary rong
6269e5574c miner: polish miner configuration (#19480)
* cmd, eth, miner: disable advance sealing if user require

* cmd, console, miner, les, eth: wrap the miner config

* eth: remove todo

* cmd, miner: revert noadvance flag

The reason for this is: if the transaction execution is even longer
than block time, then this kind of transactions is DoS attack.
2019-04-23 10:08:51 +03:00
Péter Szilágyi
f496927a93
Merge pull request #19468 from karalabe/enforce-fastsync-checkpoints
eth, les, light: enforce CHT checkpoints on fast-sync too
2019-04-17 14:50:51 +03:00
Péter Szilágyi
38f6b85638
eth, les, light: enforce CHT checkpoints on fast-sync too 2019-04-17 13:16:15 +03:00
Felföldi Zsolt
921b3160db les: fix p2p.Protocol.PeerInfo (#19472) 2019-04-17 10:57:53 +03:00
Felföldi Zsolt
85b6823d16 les: check required message types in cost table (#19454) 2019-04-16 14:30:47 +03:00
Péter Szilágyi
7221cb1434
core, eth, les, light: scope receipt functionality a bit cleaner 2019-04-15 13:42:26 +03:00
Matthew Halpern
6b0ddd141e
core, eth, les, light: store transaction receipts without txHash and gasCost 2019-04-15 13:15:39 +03:00
Felföldi Zsolt
7c08e48141
Merge pull request #19403 from zsfelfoldi/remove-les1
les: remove support for LES/1
2019-04-08 14:04:14 +02:00
Martin Holst Swende
e2f3465e83 eth, les, geth: implement cli-configurable global gas cap for RPC calls (#19401)
* eth, les, geth: implement cli-configurable global gas cap for RPC calls

* graphql, ethapi: place gas cap in DoCall

* ethapi: reformat log message
2019-04-08 14:49:52 +03:00
Zsolt Felfoldi
64f9c1ea09 les, light: remove support for les/1 4096 block CHT sections 2019-04-08 13:17:24 +02:00
Zsolt Felfoldi
5515f364ae les: removed les/1 protocol messages 2019-04-08 13:17:24 +02:00
gary rong
d5cae48bae accounts, cmd, internal: disable unlock account on open HTTP (#17037)
* cmd, accounts, internal, node, rpc, signer: insecure unlock protect

* all: strict unlock API by rpc

* cmd/geth: check before printing warning log

* accounts, cmd/geth, internal: tiny polishes
2019-04-04 14:03:10 +03:00
Felföldi Zsolt
5164274872 les: extend error message for coinbase API calls (#19380) 2019-04-03 10:15:15 +03:00
Felföldi Zsolt
e852505ace les: fix block announcements (#19322) 2019-03-25 09:17:55 +02:00
Felföldi Zsolt
c53c5e616f les: fix peer id and reply error handling (#19289)
* les: fixed peer id format

* les: fixed peer reply error handling
2019-03-20 10:35:05 +02:00
Péter Szilágyi
211ec46284
les, light: verbose errors on state retrieval issues 2019-03-18 13:19:40 +02:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Felföldi Zsolt
c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
Matthew Halpern
badaf43019 les: remove redundant type specifiers (#19091) 2019-02-25 12:49:49 +02:00
gary rong
7fd0ccaa68 core: remove unnecessary fields in logs, receipts and tx lookups (#17106)
* core: remove unnecessary fields in log

* core: bump blockchain database version

* core, les: remove unnecessary fields in txlookup

* eth: print db version explicitly

* core/rawdb: drop txlookup entry struct wrapper
2019-02-21 15:14:35 +02:00
b00ris
769657060e les: implement ultralight client (#16904)
For more information about this light client mode, read
https://hackmd.io/s/HJy7jjZpm
2019-01-24 12:18:26 +01:00
Martin Holst Swende
83a9a73b89 cmd/geth, core, eth: implement Constantinople override flag (#18273)
* geth/core/eth: implement constantinople override flag

* les: implemnent constantinople override flag for les clients

* cmd/geth, eth, les: fix typo, move flag to experimentals
2018-12-11 14:19:03 +02:00
Paweł Bylica
de39513ced core, internal, eth, miner, les: Take VM config from BlockChain (#17955)
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.

This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.

Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
2018-12-06 14:34:49 +01:00
zah
55a4ff806f remove a no-op line in the code (#17760) 2018-11-29 10:56:59 +01:00
Felföldi Zsolt
f0515800e6 les: fix fetcher syncing logic (#18072) 2018-11-26 13:34:33 +02:00
Sheldon
b8a2ac3fcf les: fix pubkey index typo (#18093) 2018-11-15 11:10:45 +02:00
gary rong
c5d34fc94e les, light: reduce les testing stress (#17867) 2018-10-08 16:52:23 +03:00
Felföldi Zsolt
b7bbe66b19 les: limit state ODR retrievals to the last 100 blocks (#17744) 2018-10-01 15:14:53 +02:00
Felix Lange
30cd5c1854
all: new p2p node representation (#17643)
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.

Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.

The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.

* p2p/discover: port to p2p/enode

This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:

  - Table.Lookup is not available anymore. It used to take a public key
    as argument because v4 protocol requires one. Its replacement is
    LookupRandom.
  - Table.Resolve takes *enode.Node instead of NodeID. This is also for
    v4 protocol compatibility because nodes cannot be looked up by ID
    alone.
  - Types Node and NodeID are gone. Further commits in the series will be
    fixes all over the the codebase to deal with those removals.

* p2p: port to p2p/enode and discovery changes

This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.

New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.

* p2p/simulations, p2p/testing: port to p2p/enode

No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:

 - testing.ProtocolSession tracks complete nodes, not just their IDs.
 - adapters.NodeConfig has a new method to create a complete node.

These changes were needed to make swarm tests work.

Note that the NodeID change makes the code incompatible with old
simulation snapshots.

* whisper/whisperv5, whisper/whisperv6: port to p2p/enode

This port was easy because whisper uses []byte for node IDs and
URL strings in the API.

* eth: port to p2p/enode

Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.

* les: port to p2p/enode

Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.

* node: port to p2p/enode

This change simply replaces discover.Node and discover.NodeID with their
new equivalents.

* swarm/network: port to p2p/enode

Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).

There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.

Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
2018-09-25 00:59:00 +02:00
Felföldi Zsolt
ab13cd9924 les: fix invalid delivery handling in retriever (#17727) 2018-09-21 10:59:21 +03:00
Péter Szilágyi
f55c26ae6d
Merge pull request #17719 from karalabe/update-chts
les, light, params: update light client CHTs
2018-09-20 15:10:04 +03:00
gary rong
d6254f827b all: protect self-mined block during reorg (#17656) 2018-09-20 15:09:30 +03:00
Péter Szilágyi
af89093116
les, light, params: update light client CHTs 2018-09-20 14:14:48 +03:00
Felföldi Zsolt
c37238cae9 les: fix retriever logic (#17705) 2018-09-20 10:46:39 +03:00
Felföldi Zsolt
d4a28a13ca les: fix distReq.sentChn double close bug (#17639) 2018-09-14 22:14:29 +02:00
gary rong
c1c003e4ff consensus, miner: stale block mining support (#17506)
* consensus, miner: stale block supporting

* consensus, miner: refactor seal signature

* cmd, consensus, eth: add miner noverify flag

* cmd, consensus, miner: polish
2018-08-28 16:59:05 +03:00
gary rong
b69476b372 all: make indexer configurable (#17188) 2018-08-28 10:08:16 +03:00
gary rong
b2c644ffb5 cmd, eth, miner: make recommit configurable (#17444)
* cmd, eth, miner: make recommit configurable

* cmd, eth, les, miner: polish a bit

* miner: filter duplicate sealing work

* cmd: remove uncessary conversion

* miner: avoid microptimization in favor of cleaner code
2018-08-21 22:56:54 +03:00
Felföldi Zsolt
355fc47d39 les: fix CHT field in nodeInfo (#17465) 2018-08-21 14:58:10 +03:00
Felix Lange
2695fa2213 les: fix crasher in NodeInfo when running as server (#17419)
* les: fix crasher in NodeInfo when running as server

The ProtocolManager computes CHT and Bloom trie roots by asking the
indexers for their current head. It tried to get the indexers from
LesOdr, but no LesOdr instance is created in server mode.

Attempt to fix this by moving the indexers, protocol creation and
NodeInfo to a new lesCommons struct which is embedded into both server
and client.

All this setup code should really be cleaned up, but this is just a
hotfix so we have to do that some other time.

* les: fix commons protocol maker
2018-08-17 13:21:53 +03:00
Felföldi Zsolt
2cdf6ee7e0 light: CHT and bloom trie indexers working in light mode (#16534)
This PR enables the indexers to work in light client mode by
downloading a part of these tries (the Merkle proofs of the last
values of the last known section) in order to be able to add new
values and recalculate subsequent hashes. It also adds CHT data to
NodeInfo.
2018-08-15 22:25:46 +02:00
Felföldi Zsolt
b2ddb1fcbf les: implement client connection logic (#16899)
This PR implements les.freeClientPool. It also adds a simulated clock
in common/mclock, which enables time-sensitive tests to run quickly
and still produce accurate results, and package common/prque which is
a generalised variant of prque that enables removing elements other
than the top one from the queue.

les.freeClientPool implements a client database that limits the
connection time of each client and manages accepting/rejecting
incoming connections and even kicking out some connected clients. The
pool calculates recent usage time for each known client (a value that
increases linearly when the client is connected and decreases
exponentially when not connected). Clients with lower recent usage are
preferred, unknown nodes have the highest priority. Already connected
nodes receive a small bias in their favor in order to avoid accepting
and instantly kicking out clients.

Note: the pool can use any string for client identification. Using
signature keys for that purpose would not make sense when being known
has a negative value for the client. Currently the LES protocol
manager uses IP addresses (without port address) to identify clients.
2018-08-14 22:44:46 +02:00
Péter Szilágyi
f0998415ba
cmd, consensus/ethash, eth: miner push notifications 2018-08-10 09:06:59 +03:00
gary rong
51db5975cc consensus/ethash: move remote agent logic to ethash internal (#15853)
* consensus/ethash: start remote ggoroutine to handle remote mining

* consensus/ethash: expose remote miner api

* consensus/ethash: expose submitHashrate api

* miner, ethash: push empty block to sealer without waiting execution

* consensus, internal: add getHashrate API for ethash

* consensus: add three method for consensus interface

* miner: expose consensus engine running status to miner

* eth, miner: specify etherbase when miner created

* miner: commit new work when consensus engine is started

* consensus, miner: fix some logics

* all: delete useless interfaces

* consensus: polish a bit
2018-08-03 11:33:37 +03:00
Oleg Kovalov
d42ce0f2c1 all: simplify switches (#17267)
* all: simplify switches

* silly mistake
2018-07-30 12:30:09 +03:00
Péter Szilágyi
e1f1d3085c
accounts, eth, les: blockhash based filtering on all code paths 2018-07-12 18:16:54 +03:00
Felföldi Zsolt
51df1c1f20 les: add announcement safety check to light fetcher (#17034) 2018-07-04 13:40:20 +03:00
gary rong
4895665670 les: handle conn/disc/reg logic in the eventloop (#16981)
* les: handle conn/disc/reg logic in the eventloop

* les: try to dial before start eventloop

* les: handle disconnect logic more safely

* les: grammar fix
2018-06-25 11:52:24 +03:00
Felföldi Zsolt
25982375a8 les: fix retriever logic (#16776)
This PR fixes a retriever logic bug. When a peer had a soft timeout
and then a response arrived, it always assumed it was the same peer
even though it could have been a later requested one that did not time
out at all yet. In this case the logic went to an illegal state and
deadlocked, causing a goroutine leak.

Fixes #16243 and replaces #16359.
Thanks to @riceke for finding the bug in the logic.
2018-06-12 15:58:47 +02:00
Felföldi Zsolt
049f5b3572 core, eth, les: more efficient hash-based header chain retrieval (#16946) 2018-06-12 16:52:54 +03:00
Felföldi Zsolt
546d42179e les: pass server pool to protocol manager (#16947) 2018-06-12 14:00:52 +03:00
Felföldi Zsolt
a5237a27ea les: add Skip overflow check to GetBlockHeadersMsg handler (#16891) 2018-06-05 10:23:00 +03:00
kiel barry
0fe47e98c4 trie: fixes to comply with golint (#16771) 2018-05-21 23:41:31 +03:00
Péter Szilágyi
49719e21bc
core, eth: minor txpool event cleanups 2018-05-18 12:08:24 +03:00
rjl493456442
a2e43d28d0
all: collate new transaction events together 2018-05-18 11:46:44 +03:00
gary rong
7beccb29be all: get rid of error when creating memory database (#16716)
* all: get rid of error when create mdb

* core: clean up variables definition

* all: inline mdb definition
2018-05-09 15:24:25 +03:00
Péter Szilágyi
6cf0ab38bd
core/rawdb: separate raw database access to own package (#16666) 2018-05-07 14:35:06 +03:00
GagziW
7c02933275 les: changed if-else blocks to conform with golint (#16658) 2018-05-03 11:35:06 +03:00
thomasmodeneis
ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
cpusoft
e7cc5b4160 les: add ps.lock.Unlock() before return (#16360) 2018-04-11 11:02:33 +03:00
Andrey Petrov
2e9c8fd4fb eth, les: allow exceeding maxPeers for trusted peers (#16189)
Fixes #3326, #14472
2018-02-27 12:52:59 +02:00
Anton Evangelatov
ae9f97221a metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.

* vendor: change nonsense/go-metrics to ethersphere/go-metrics

* go-metrics: add tests. move ResettingTimer logic from reporter to type.

* all, metrics: pull in metrics package in go-ethereum

* metrics/test: make sure metrics are enabled for tests

* metrics: apply gosimple rules

* metrics/exp, internal/debug: init expvar endpoint when starting pprof server

* internal/debug: tiny comment formatting fix
2018-02-23 11:56:08 +02:00
Péter Szilágyi
5cf1d35470
eth, les, light: filter on logs only, derive receipts on demand 2018-02-22 19:12:43 +02:00
Felföldi Zsolt
1bdde620da les: fix light fetcher database race (#16103)
* les: fix light fetcher database race

* les: lightFetcher comments
2018-02-19 10:41:30 +02:00
Péter Szilágyi
dfc5842a89
les: add missing lock around peer access 2018-02-14 21:09:20 +02:00
Felix Lange
a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Péter Szilágyi
7a0019c63b
les, light: fix CHT trie retrievals (#16039)
* les, light: fix CHT trie retrievals

* les, light: minor polishes, test remote CHT retrievals

* les, light: deterministic nodeset rlp, bloombits test skeleton

* les: add an event emission to the les bloombits test

* les: drop dead tester code
2018-02-11 14:57:46 +02:00
Felföldi Zsolt
2f849ade82 les: fix server panic when discovery disabled (#16055) 2018-02-10 14:33:52 +02:00
Péter Szilágyi
55599ee95d core, trie: intermediate mempool between trie and database (#15857)
This commit reduces database I/O by not writing every state trie to disk.
2018-02-05 17:40:32 +01:00
Felföldi Zsolt
c3f238dd53 les: limit LES peer count and improve peer configuration logic (#16010)
* les: limit number of LES connections

* eth, cmd/utils: light vs max peer configuration logic
2018-02-05 15:41:53 +02:00
Péter Szilágyi
566d5c0777
core, eth, les, light: get rid of redundant methods 2018-01-30 18:42:00 +02:00
Felföldi Zsolt
23bca0f374 les: fix TxStatusMsg RLP coding (#15974) 2018-01-26 19:30:45 +02:00
Felföldi Zsolt
92580d69d3 p2p, p2p/discover, p2p/discv5: implement UDP port sharing (#15200)
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.

DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
2018-01-22 13:38:34 +01:00
Felföldi Zsolt
83d1657444 les: fix les/1 CHT compatibility issue (#15692) 2018-01-09 11:41:59 +01:00
Péter Szilágyi
6f69cdd109
all: switch gas limits from big.Int to uint64 2018-01-03 14:45:35 +02:00
Furkan KAMACI
b8caba9709 various: remove redundant parentheses (#15793) 2018-01-03 14:14:47 +02:00
Péter Szilágyi
f7ca03ae87 eth, les, light: expose chain config in les node info too (#15732) 2017-12-28 14:18:34 +01:00
gary rong
5f8888e116 accounts, consensus, core, eth: make chain maker consensus agnostic (#15497)
* accounts, consensus, core, eth: make chain maker consensus agnostic

* consensus, core: move CalcDifficulty to Engine interface

* consensus: add docs for calcDifficulty function

* consensus, core: minor comment fixups
2017-12-22 14:37:50 +02:00
gary rong
f14047dae5 cmd, consensus, eth: split ethash related config to it own (#15520)
* cmd, consensus, eth: split ethash related config to it own

* eth, consensus: minor polish

* eth, consenus, console: compress pow testing config field to single one

* consensus, eth: document pow mode
2017-11-24 16:10:27 +02:00
rjl493456442
41ef34ae40 les: use modified default txpool config to avoid creating journal file 2017-11-21 22:13:57 +08:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
b00ris
7ace023981 les: fix channel assignment data race (#15441) 2017-11-09 10:43:37 +01:00
Evgeny Danilenko
0914d4e0d2 les: fix misuse of WaitGroup (#15365) 2017-11-09 10:34:35 +01:00
ferhat elmas
9619a61024 all: gofmt -w -s (#15419) 2017-11-08 11:45:52 +01:00
Felföldi Zsolt
8d434f6a6f les, core/bloombits: post-LES/2 fixes (#15391)
* les: fix topic ID

* core/bloombits: fix interface conversion
2017-10-27 17:18:53 +03:00
Péter Szilágyi
0095531a58 core, eth, les: fix messy code (#15367)
* core, eth, les: fix messy code

* les: fixed tx status test and rlp encoding

* core: add a workaround for light sync
2017-10-25 12:18:44 +03:00
Felföldi Zsolt
ca376ead88 les, light: LES/2 protocol version (#14970)
This PR implements the new LES protocol version extensions:

* new and more efficient Merkle proofs reply format (when replying to
  a multiple Merkle proofs request, we just send a single set of trie
  nodes containing all necessary nodes)
* BBT (BloomBitsTrie) works similarly to the existing CHT and contains
  the bloombits search data to speed up log searches
* GetTxStatusMsg returns the inclusion position or the
  pending/queued/unknown state of a transaction referenced by hash
* an optional signature of new block data (number/hash/td) can be
  included in AnnounceMsg to provide an option for "very light
  clients" (mobile/embedded devices) to skip expensive Ethash check
  and accept multiple signatures of somewhat trusted servers (still a
  lot better than trusting a single server completely and retrieving
  everything through RPC). The new client mode is not implemented in
  this PR, just the protocol extension.
2017-10-24 15:19:09 +02:00
Felix Lange
10181b57a9 core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method

* ethdb: improve docs and add IdealBatchSize

* ethdb: remove memory batch lock

Batches are not safe for concurrent use.

* core: use ethdb.Putter for Write* functions

This covers the easy cases.

* core/state: simplify StateSync

* trie: optimize local node check

* ethdb: add ValueSize to Batch

* core: optimize HasHeader check

This avoids one random database read get the block number. For many uses
of HasHeader, the expectation is that it's actually there. Using Has
avoids a load + decode of the value.

* core: write fast sync block data in batches

Collect writes into batches up to the ideal size instead of issuing many
small, concurrent writes.

* eth/downloader: commit larger state batches

Collect nodes into a batch up to the ideal size instead of committing
whenever a node is received.

* core: optimize HasBlock check

This avoids a random database read to get the number.

* core: use numberCache in HasHeader

numberCache has higher capacity, increasing the odds of finding the
header without a database lookup.

* core: write imported block data using a batch

Restore batch writes of state and add blocks, tx entries, receipts to
the same batch. The change also simplifies the miner.

This commit also removes posting of logs when a forked block is imported.

* core: fix DB write error handling

* ethdb: use RLock for Has

* core: fix HasBlock comment
2017-09-09 19:03:07 +03:00
Péter Szilágyi
f585f9eee8
core, eth: clean up bloom filtering, add some tests 2017-09-06 11:14:19 +03:00
Zsolt Felfoldi
4ea4d2dc34
core, eth: add bloombit indexer, filter based on it 2017-09-06 11:13:13 +03:00
rjl493456442
28aea46ac0
core: implement Metropolis EIP 658, receipt status byte 2017-08-22 18:35:17 +03:00
Miya Chen
bf1e263128 core, light: send chain events using event.Feed (#14865) 2017-08-18 12:58:36 +02:00
Felföldi Zsolt
fff6e03a79 les: fix megacheck warnings (#14941)
* les: fix megacheck warnings

* les: fixed testGetProofs
2017-08-08 19:31:08 +03:00
Péter Szilágyi
4f7a38001f Merge pull request #14737 from holiman/txpool_localaccounts
Txpool localaccounts
2017-07-10 12:43:23 +03:00
Péter Szilágyi
48ee7f9de7
core, eth, les: polish txpool API around local/remote txs 2017-07-05 16:51:55 +03:00
Nick Johnson
be8f8409bc eth/downloader, les, light: Changes in response to review 2017-07-03 15:17:12 +01:00
Nick Johnson
ae11545bc5 eth, les: Refactor downloader peer to use structs 2017-06-29 12:49:18 +01:00
Nick Johnson
0550957989 eth, les, light: Refactor downloader to use blockchain interface 2017-06-28 15:58:41 +01:00
Felix Lange
9e5f03b6c4 core/state: access trie through Database interface, track errors (#14589)
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.

Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
2017-06-27 15:57:06 +02:00
Felföldi Zsolt
a5d08c893d les: code refactoring (#14416)
This commit does various code refactorings:

- generalizes and moves the request retrieval/timeout/resend logic out of LesOdr
  (will be used by a subsequent PR)
- reworks the peer management logic so that all services can register with
  peerSet to get notified about added/dropped peers (also gets rid of the ugly
  getAllPeers callback in requestDistributor)
- moves peerSet, LesOdr, requestDistributor and retrieveManager initialization
  out of ProtocolManager because I believe they do not really belong there and the
  whole init process was ugly and ad-hoc
2017-06-21 12:27:38 +02:00
Felix Lange
e20158176d les: fix goroutine leak in execQueue (#14480)
execQueue used an atomic counter to track whether the queue had been
closed, but the checking the counter didn't happen because the queue was
blocked on its channel.

Fix it by using a condition variable instead of sync/atomic. I tried an
implementation based on channels first, but it was hard to make it
reliable.

quit now waits for the queue loop to exit.
2017-05-16 20:56:02 +02:00
Péter Szilágyi
e61035c5a3
cmd, eth, les, mobile: make networkid uint64 everywhere 2017-04-25 14:53:50 +03:00
Felix Lange
0cc492f815 all: update license information 2017-04-14 10:29:00 +02:00
bas-vk
5e29f4be93 cmd/utils, node: remove unused solc references and improve RPC config (#14324)
Currently http cors and websocket origins are a comma separated string in the
config object. These are replaced with string arrays that are more expressive in
case of a config file.
2017-04-12 23:04:14 +02:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Péter Szilágyi
a7b9e484d0 consensus, core, ethstats: use engine specific block beneficiary (#14318)
* consensus, core, ethstats: use engine specific block beneficiary

* core, eth, les, miner: use explicit beneficiary during mining
2017-04-12 16:38:31 +03:00
Felföldi Zsolt
18bbe12425 les: allow LES connection to other servers (#13889) 2017-04-11 01:23:39 +02:00
Péter Szilágyi
cc13d576f0 Merge pull request #13870 from karalabe/miners-fixes
all: clean up various error handling in core and the miner
2017-04-07 11:03:11 +03:00
Péter Szilágyi
158d603528
consensus, core: drop all the legacy custom core error types 2017-04-06 17:34:19 +03:00
Felföldi Zsolt
9aca9e6deb cmd, les, eth, eth/gasprice: using new gas price oracle (#13853)
* cmd, les, eth, eth/gasprice: using new gas price oracle

* eth/gasprice: renamed source file

* eth/gasprice: added security checks for gpo params

* eth/gasprice: fixed naming issues

* eth/gasprice: max limit, maxEmpty
2017-04-06 17:20:42 +03:00
Péter Szilágyi
09777952ee core, consensus: pluggable consensus engines (#3817)
This commit adds pluggable consensus engines to go-ethereum. In short, it
introduces a generic consensus interface, and refactors the entire codebase to
use this interface.
2017-04-05 00:16:29 +02:00
Felix Lange
37dd9086ec core: refactor genesis handling
This commit solves several issues concerning the genesis block:

* Genesis/ChainConfig loading was handled by cmd/geth code. This left
  library users in the cold. They could specify a JSON-encoded
  string and overwrite the config, but didn't get any of the additional
  checks performed by geth.
* Decoding and writing of genesis JSON was conflated in
  WriteGenesisBlock. This made it a lot harder to embed the genesis
  block into the forthcoming config file loader. This commit changes
  things so there is a single Genesis type that represents genesis
  blocks. All uses of Write*Genesis* are changed to use the new type
  instead.
* If the chain config supplied by the user was incompatible with the
  current chain (i.e. the chain had already advanced beyond a scheduled
  fork), it got overwritten. This is not an issue in practice because
  previous forks have always had the highest total difficulty. It might
  matter in the future though. The new code reverts the local chain to
  the point of the fork when upgrading configuration.

The change to genesis block data removes compression library
dependencies from package core.
2017-03-23 15:58:43 +01:00
Felix Lange
c213fd1fd8 all: import "context" instead of "golang.org/x/net/context"
There is no need to depend on the old context package now that the
minimum Go version is 1.7. The move to "context" eliminates our weird
vendoring setup. Some vendored code still uses golang.org/x/net/context
and it is now vendored in the normal way.

This change triggered new vet checks around context.WithTimeout which
didn't fire with golang.org/x/net/context.
2017-03-22 20:49:15 +01:00
Felföldi Zsolt
525116dbff les: implement request distributor, fix blocking issues (#3660)
* les: implement request distributor, fix blocking issues
* core: moved header validation before chain mutex lock
2017-03-22 20:44:22 +01:00
Péter Szilágyi
9b84caf3a5 core, eth, les: support resuming fast sync on heavy rollback (#3743) 2017-03-22 01:37:24 +01:00
Péter Szilágyi
023670f6ba cmd, eth, les, node, pow: disk caching and progress reports 2017-03-09 15:50:14 +01:00
Péter Szilágyi
567d41d936 all: swap out the C++ ethash to the pure Go one (mining todo) 2017-03-09 15:50:14 +01:00
Péter Szilágyi
e7030c4bf5
all: update light logs (and a few others) to the new model 2017-03-03 11:41:52 +02:00
Felix Lange
35e8308bf7 eth, les: shorten genesis block mismatch error message 2017-02-27 17:19:41 +01:00
Felix Lange
5c8fe28b72 common: move big integer math to common/math (#3699)
* common: remove CurrencyToString

Move denomination values to params instead.

* common: delete dead code

* common: move big integer operations to common/math

This commit consolidates all big integer operations into common/math and
adds tests and documentation.

There should be no change in semantics for BigPow, BigMin, BigMax, S256,
U256, Exp and their behaviour is now locked in by tests.

The BigD, BytesToBig and Bytes2Big functions don't provide additional
value, all uses are replaced by new(big.Int).SetBytes().

BigToBytes is now called PaddedBigBytes, its minimum output size
parameter is now specified as the number of bytes instead of bits. The
single use of this function is in the EVM's MSTORE instruction.

Big and String2Big are replaced by ParseBig, which is slightly stricter.
It previously accepted leading zeros for hexadecimal inputs but treated
decimal inputs as octal if a leading zero digit was present.

ParseUint64 is used in places where String2Big was used to decode a
uint64.

The new functions MustParseBig and MustParseUint64 are now used in many
places where parsing errors were previously ignored.

* common: delete unused big integer variables

* accounts/abi: replace uses of BytesToBig with use of encoding/binary

* common: remove BytesToBig

* common: remove Bytes2Big

* common: remove BigTrue

* cmd/utils: add BigFlag and use it for error-checked integer flags

While here, remove environment variable processing for DirectoryFlag
because we don't use it.

* core: add missing error checks in genesis block parser

* common: remove String2Big

* cmd/evm: use utils.BigFlag

* common/math: check for 256 bit overflow in ParseBig

This is supposed to prevent silent overflow/truncation of values in the
genesis block JSON. Without this check, a genesis block that set a
balance larger than 256 bits would lead to weird behaviour in the VM.

* cmd/utils: fixup import
2017-02-26 22:21:51 +01:00
Péter Szilágyi
61e6bb1247
eth, les, swarm: fix go vet issues sufraced by log15 2017-02-23 12:16:45 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Jeffrey Wilcke
c12f4df910 params: core, core/vm, miner: 64bit gas instructions
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-13 21:44:25 +01:00
Jeffrey Wilcke
57f4e90257 Revert "params: core, core/vm, miner: 64bit gas instructions (#3514)"
This reverts commit 8b57c49490.
2017-02-13 15:15:12 +01:00
Jeffrey Wilcke
8b57c49490 params: core, core/vm, miner: 64bit gas instructions (#3514)
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.

In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.

* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
2017-02-02 15:25:42 +01:00
Zsolt Felfoldi
a390ca5f30 les, cmd/util: disable topic discovery with --nodiscover 2017-01-27 02:52:45 +01:00
Zsolt Felfoldi
12379c697a les: remove delayed les server starting 2017-01-26 04:23:53 +01:00
Zsolt Felfoldi
f5348e17f8 les: add unknown peers to server pool instead of rejecting them 2017-01-26 04:23:49 +01:00
Péter Szilágyi
d63752ef4d Merge pull request #3579 from bas-vk/natspec
cmd,eth,les,internal: remove natspec support
2017-01-17 14:38:57 +02:00
Nick Johnson
17d92233d9 cmd/geth, core: add support for recording SHA3 preimages (#3543) 2017-01-17 12:19:50 +01:00
Bas van Kervel
54a65e6d87
cmd,eth,les,internal: remove natspec support 2017-01-17 12:13:50 +01:00
Felföldi Zsolt
0fa9a8929c les: fixed transaction sending deadlock (#3568) 2017-01-16 10:51:29 +01:00
Péter Szilágyi
02b67558e8 Merge pull request #3535 from fjl/all-ineffassign
all: fix ineffectual assignments
2017-01-09 23:53:17 +02:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Péter Szilágyi
681b51aac4 Merge pull request #3519 from zsfelfoldi/light-topic5
les: fixed selectPeer deadlock, improved request distribution
2017-01-09 16:58:23 +02:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Péter Szilágyi
59b8245bbc Merge pull request #3516 from fjl/types-drop-sign-ecdsa
core/types: remove redundant SignECDSA wrappers, rename to SignTx
2017-01-06 15:55:55 +02:00
Felix Lange
7731061903 core/vm: move Log to core/types
This significantly reduces the dependency closure of ethclient, which no
longer depends on core/vm as of this change.

All uses of vm.Logs are replaced by []*types.Log. NewLog is gone too,
the constructor simply returned a literal.
2017-01-06 14:15:22 +01:00
Zsolt Felfoldi
93f9c023cc les: fixed selectPeer deadlock, improved request distribution
les/flowcontrol: using proper types for relative and absolute times
2017-01-06 04:34:31 +01:00
Felix Lange
e171bf74f8 core/types: remove redundant SignECDSA wrappers, rename to SignTx 2017-01-05 12:59:17 +01:00
Jeffrey Wilcke
bbc4ea4ae8 core/vm: improved EVM run loop & instruction calling (#3378)
The run loop, which previously contained custom opcode executes have been
removed and has been simplified to a few checks.

Each operation consists of 4 elements: execution function, gas cost function,
stack validation function and memory size function. The execution function
implements the operation's runtime behaviour, the gas cost function implements
the operation gas costs function and greatly depends on the memory and stack,
the stack validation function validates the stack and makes sure that enough
items can be popped off and pushed on and the memory size function calculates
the memory required for the operation and returns it.

This commit also allows the EVM to go unmetered. This is helpful for offline
operations such as contract calls.
2017-01-05 11:52:10 +01:00
Péter Szilágyi
2ce30382d9
cmd/utils, eth, les: bubble --fakepow flag into eth/les too 2017-01-04 10:42:41 +02:00
Felix Lange
cf71f5cd60 rpc: remove HexNumber, replace all uses with hexutil types
This change couldn't be automated because HexNumber was used for numbers
of all sizes.
2016-12-20 14:41:58 +01:00
Felix Lange
a98e8c0889 Merge pull request #3413 from zsfelfoldi/light-topic4
les, p2p/discv5: implement server pool, improve peer selection, light fetcher and topic searching
2016-12-12 20:46:15 +01:00
bas-vk
4e36b1e3da core: bugfix state change race condition in txpool (#3412)
The transaction pool keeps track of the current nonce in its local pendingState. When a
new block comes in the pendingState is reset. During the reset it fetches multiple times
the current state through the use of the currentState callback. When a second block comes
in during the reset its possible that the state changes during the reset. If that block
holds transactions that are currently in the pool the local pendingState that is used to
determine nonces can get out of sync.
2016-12-10 23:54:58 +01:00
Zsolt Felfoldi
f12f8a6c14 les, light: add block availability check for ODR requests 2016-12-10 09:53:25 +01:00
Zsolt Felfoldi
c57c54ce96 eth, les: defer starting LES service until ETH initial sync is finished 2016-12-10 09:53:25 +01:00
Zsolt Felfoldi
c8130df1d9 les: using random request IDs 2016-12-10 09:53:25 +01:00
Zsolt Felfoldi
af8a742d00 les: improved header fetcher and server statistics 2016-12-10 09:53:08 +01:00
Zsolt Felfoldi
e67500aa15 les: fixed light fetcher request ID matching 2016-12-08 13:38:15 +01:00
Zsolt Felfoldi
3e617f3cd6 les: implement light server pool 2016-12-08 13:38:15 +01:00
Jeffrey Wilcke
3fc7c97827 core, core/vm: implemented a generic environment (#3348)
Environment is now a struct (not an interface). This
reduces a lot of tech-debt throughout the codebase where a virtual
machine environment had to be implemented in order to test or run it.

The new environment is suitable to be used en the json tests, core
consensus and light client.
2016-12-06 02:16:03 +01:00
Péter Szilágyi
4c8c5e2f74 cmd, ethstats, les, mobile, params: native netstats (#3336) 2016-11-25 16:55:06 +01:00
Jeffrey Wilcke
67e0894d9e common/httpclient, les: removed httpclient 2016-11-25 13:10:44 +01:00
Jeffrey Wilcke
2d4bd3b3ad les: fixed logging 2016-11-15 22:48:00 +01:00
Zsolt Felfoldi
b10bcd924b core/types: turn off nonce checking for Call messages 2016-11-14 14:16:06 +01:00
Zsolt Felfoldi
e33e57684f p2p/discv5: fixed bootnode connect issues 2016-11-14 13:22:19 +01:00
Zsolt Felfoldi
a0c6649960 les/flowcontrol: fixed locking scheme 2016-11-14 13:22:19 +01:00
Felix Lange
ca73dea3b9 Merge pull request #3179 from obscuren/eip-158
EIP158 & 160 Hardfork
2016-11-14 10:52:02 +01:00
Jeffrey Wilcke
4dca5d4db7 core/types, params: EIP#155 2016-11-13 14:55:30 +01:00
Jeffrey Wilcke
21701190ac les: cleaned up logging (#3256) 2016-11-13 12:34:50 +01:00
Jeffrey Wilcke
445feaeef5 core, core/state, trie: EIP158, reprice & skip empty account write
This commit implements EIP158 part 1, 2, 3 & 4

1. If an account is empty it's no longer written to the trie. An empty
  account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
  reation costs.

params: moved core/config to params

Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
2016-11-13 10:44:04 +01:00
Zsolt Felfoldi
323c2d6775 les: add missing nil check and handle quit in findServers 2016-11-11 03:16:46 +01:00
Felix Lange
bbb5e5d56a les: print 'experimental feature' warning on startup 2016-11-09 02:51:34 +01:00
Felix Lange
b8bd9a71c8 all: update license information 2016-11-09 02:51:34 +01:00
Felix Lange
be3865211c core/types: remove header accessors
These accessors were introduced by light client changes, but
the only method that is actually used is GetNumberU64. This
commit replaces all uses of .GetNumberU64 with .Number.Uint64.
2016-11-09 02:19:07 +01:00
Zsolt Felfoldi
49da42983a p2p/discv5: added new topic discovery package 2016-11-09 02:12:53 +01:00
Zsolt Felfoldi
9f8d192991 les: light client protocol and API 2016-11-09 02:12:53 +01:00