Commit Graph

636 Commits

Author SHA1 Message Date
Liang Ma
8b9f469419 p2p/protocols: fix minor comments typo (#18185) 2018-11-27 12:52:30 +02:00
holisticode
bba5fd8192 Accounting metrics reporter (#18136) 2018-11-26 17:05:18 +01:00
Martin Holst Swende
493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
lash
201a0bf181 p2p/simulations, swarm/network: Custom services in snapshot (#17991)
* p2p/simulations: Add custom services to simnodes + remove sim down conn objs

* p2p/simulation, swarm/network: Add selective services to discovery sim

* p2p/simulations, swarm/network: Remove useless comments

* p2p/simulations, swarm/network: Clean up mess from rebase

* p2p/simulation: Add sleep to prevent connect flakiness in http test

* p2p/simulations: added concurrent goroutines to prevent sleeps on simulation connect/disconnect

* p2p/simulations, swarm/network/simulations: address pr comments

* reinstated dummy service

* fixed http snapshot test
2018-11-12 14:57:17 +01:00
Kurkó Mihály
f574c4e74b metrics, p2p: add ephemeral registry (#18067)
* metrics, p2p: add ephemeral registry

* metrics: fix linter issue
2018-11-09 10:20:51 +01:00
Corey Lin
212bf266c5 eth, p2p: fix comment typos (#18014) 2018-11-08 12:25:14 +01:00
Liang Ma
c71e4fc4d5 p2p: fix comment typo (#18027) 2018-11-08 12:22:28 +01:00
Kurkó Mihály
503993c819 p2p: use enode.ID type in metered connection (#17933)
Change the type of the metered connection's id field from string to enode.ID.
2018-11-08 12:11:20 +01:00
Corey Lin
0fe0b8f7b9 p2p/protocols: use keyed fields for struct instantiation (#18017) 2018-11-07 13:22:31 +02:00
holisticode
8ed4739176 p2p accounting (#17951)
* p2p/protocols: introduced protocol accounting

* p2p/protocols: added TestExchange simulation

* p2p/protocols: add accounting simulation

* p2p/protocols: remove unnecessary tests

* p2p/protocols: comments for accounting simulation

* p2p/protocols: addressed PR comments

* p2p/protocols: finalized accounting implementation

* p2p/protocols: removed unused code

* p2p/protocols: addressed @nonsense PR comments
2018-10-26 00:26:31 +02:00
Kurkó Mihály
16e4d0e005 p2p: meter peer traffic, emit metered peer events (#17695)
This change extends the peer metrics collection:

- traces the life-cycle of the peers
- meters the peer traffic separately for every peer
- creates event feed for the peer events
- emits the peer events
2018-10-16 00:40:51 +02:00
Felix Lange
6f607de5d5
p2p, p2p/discover: add signed ENR generation (#17753)
This PR adds enode.LocalNode and integrates it into the p2p
subsystem. This new object is the keeper of the local node
record. For now, a new version of the record is produced every
time the client restarts. We'll make it smarter to avoid that in
the future.

There are a couple of other changes in this commit: discovery now
waits for all of its goroutines at shutdown and the p2p server
now closes the node database after discovery has shut down. This
fixes a leveldb crash in tests. p2p server startup is faster
because it doesn't need to wait for the external IP query
anymore.
2018-10-12 11:47:24 +02:00
Felix Lange
dcae0d348b
p2p/simulations: fix a deadlock and clean up adapters (#17891)
This fixes a rare deadlock with the inproc adapter:

- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
  waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
  which waits for Network.lock.

The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.

Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.

In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.

A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
2018-10-11 20:32:14 +02:00
holisticode
11d0ff6578 Fix retrieval tests and simulation backends (#17723)
* swarm/network/stream: introduced visualized snapshot sync test

* swarm/network/stream: non-existing hash visualization sim

* swarm/network/stream: fixed retrieval tests; new backend for visualization

* swarm/network/stream: cleanup of visualized_snapshot_sync_sim_test.go

* swarm/network/stream: rebased PR on master

* swarm/network/stream: fixed loop logic in retrieval tests

* swarm/network/stream: fixed iterations for snapshot tests

* swarm/network/stream: address PR comments

* swarm/network/stream: addressed PR comments
2018-10-08 20:28:44 +02:00
Felix Lange
1895059119 p2p: add enode URL to PeerInfo (#17838) 2018-10-04 18:13:21 +03:00
Liang ZOU
6663e5da10 all: fix various comment typos (#17748) 2018-09-25 12:26:35 +02:00
Felix Lange
30cd5c1854
all: new p2p node representation (#17643)
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.

Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.

The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.

* p2p/discover: port to p2p/enode

This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:

  - Table.Lookup is not available anymore. It used to take a public key
    as argument because v4 protocol requires one. Its replacement is
    LookupRandom.
  - Table.Resolve takes *enode.Node instead of NodeID. This is also for
    v4 protocol compatibility because nodes cannot be looked up by ID
    alone.
  - Types Node and NodeID are gone. Further commits in the series will be
    fixes all over the the codebase to deal with those removals.

* p2p: port to p2p/enode and discovery changes

This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.

New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.

* p2p/simulations, p2p/testing: port to p2p/enode

No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:

 - testing.ProtocolSession tracks complete nodes, not just their IDs.
 - adapters.NodeConfig has a new method to create a complete node.

These changes were needed to make swarm tests work.

Note that the NodeID change makes the code incompatible with old
simulation snapshots.

* whisper/whisperv5, whisper/whisperv6: port to p2p/enode

This port was easy because whisper uses []byte for node IDs and
URL strings in the API.

* eth: port to p2p/enode

Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.

* les: port to p2p/enode

Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.

* node: port to p2p/enode

This change simply replaces discover.Node and discover.NodeID with their
new equivalents.

* swarm/network: port to p2p/enode

Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).

There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.

Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
2018-09-25 00:59:00 +02:00
Emil
86a03f97d3 all: simplify s[:] to s where s is a slice (#17673) 2018-09-14 22:07:13 +02:00
HAOYUatHZ
5c0954afff p2p/discv5: make idx bounds checking more sound (#17571) 2018-09-03 16:47:20 +02:00
Wenbiao Zheng
d1aa605f1e all: remove the duplicate 'the' in annotations (#17509) 2018-08-27 11:49:29 +03:00
Mymskmkt
1df1187d83 p2p: fix comment typo (#17491) 2018-08-23 11:47:43 +03:00
Wuxiang
d3488c1aff p2p: fix typo (#17446) 2018-08-20 15:07:21 +03:00
Felföldi Zsolt
b52bb31b76 p2p/discv5: add delay to refresh cycle when no seed nodes are found (#16994) 2018-08-14 22:59:18 +02:00
libotony
834057592f p2p/discv5: fix negative index after uint convert to int (#17274) 2018-08-09 10:03:42 +03:00
Oleg Kovalov
cf05ef9106 p2p, swarm, trie: avoid copying slices in loops (#17265) 2018-08-07 13:56:40 +03:00
Felföldi Zsolt
eef65b20fc p2p: use safe atomic operations when changing connFlags (#17325) 2018-08-06 15:46:30 +03:00
Felföldi Zsolt
c4df67461f
Merge pull request #16333 from shazow/addremovetrustedpeer
rpc: Add admin_addTrustedPeer and admin_removeTrustedPeer.
2018-08-06 13:30:04 +02:00
Janoš Guljaš
8f4c4fea20 p2p: fix rare deadlock in Stop (#17260) 2018-07-30 12:44:17 +03:00
Oleg Kovalov
d42ce0f2c1 all: simplify switches (#17267)
* all: simplify switches

* silly mistake
2018-07-30 12:30:09 +03:00
Viktor Trón
b536460f8e
Merge pull request #17231 from ethersphere/develop
swarm: client-side MRU signatures ; BMT fixes ; network simulation tests
2018-07-24 08:44:43 +02:00
Wenbiao Zheng
fe6a9473dc p2p: token is useless in xxxEncHandshake (#17230) 2018-07-23 17:36:08 +02:00
Janoš Guljaš
dcaaa3c804 swarm: network simulation for swarm tests (#769)
* cmd/swarm: minor cli flag text adjustments

* cmd/swarm, swarm/storage, swarm: fix  mingw on windows test issues

* cmd/swarm: support for smoke tests on the production swarm cluster

* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion

* changed colour of landing page

* landing page reacts to enter keypress

* swarm/api/http: sticky footer for swarm landing page using flex

* swarm/api/http: sticky footer for error pages and fix for multiple choices

* swarm: propagate ctx to internal apis (#754)

* swarm/simnet: add basic node/service functions

* swarm/netsim: add buckets for global state and kademlia health check

* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...

* swarm, swarm/netsim: adjust SwarmNetworkTest

* swarm/netsim: fix tests

* swarm: added visualization option to sim net redesign

* swarm/netsim: support multiple services per node

* swarm/netsim: remove redundant return statement

* swarm/netsim: add comments

* swarm: shutdown HTTP in Simulation.Close

* swarm: sim HTTP server timeout

* swarm/netsim: add more simulation methods and peer events examples

* swarm/netsim: add WaitKademlia example

* swarm/netsim: fix comments

* swarm/netsim: terminate peer events goroutines on simulation done

* swarm, swarm/netsim: naming updates

* swarm/netsim: return not healthy kademlias on WaitTillHealthy

* swarm: fix WaitTillHealthy call in testSwarmNetwork

* swarm/netsim: allow bucket to have any type for a key

* swarm: Added snapshots to new netsim

* swarm/netsim: add more tests for bucket

* swarm/netsim: move http related things into separate files

* swarm/netsim: add AddNodeWithService option

* swarm/netsim: add more tests and Start* methods

* swarm/netsim: add peer events and kademlia tests

* swarm/netsim: fix some tests flakiness

* swarm/netsim: improve random nodes selection, fix TestStartStop* tests

* swarm/netsim: remove time measurement from TestClose to avoid flakiness

* swarm/netsim: builder pattern for netsim HTTP server (#773)

* swarm/netsim: add connect related tests

* swarm/netsim: add comment for TestPeerEvents

* swarm: rename netsim package to network/simulation
2018-07-23 15:33:25 +02:00
jkcomment
65c91ad5e7 p2p: correct comments typo (#17184) 2018-07-18 10:41:18 +03:00
Anton Evangelatov
7c9314f231 swarm: integrate OpenTracing; propagate ctx to internal APIs (#17169)
* swarm: propagate ctx, enable opentracing

* swarm/tracing: log error when tracing is misconfigured
2018-07-13 17:40:28 +02:00
Felix Lange
c73b654fd1 p2p/discover: move bond logic from table to transport (#17048)
* p2p/discover: move bond logic from table to transport

This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.

It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.

The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.

The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.

* p2p/discover: reduce findnode failure counter on success

* p2p/discover: remove unused parameter of loadSeedNodes

* p2p/discover: improve ping-back check and comments

* p2p/discover: add neighbors reply nodes always, not just during init
2018-07-03 16:24:12 +03:00
ethersphere
e187711c65 swarm: network rewrite merge 2018-06-21 21:10:31 +02:00
Andrey Petrov
6209545083 p2p: Wrap conn.flags ops with atomic.Load/Store 2018-06-21 12:22:47 -04:00
Andrey Petrov
193a402cc0 p2p: Test for peer.rw.flags race conditions 2018-06-21 12:22:47 -04:00
Andrey Petrov
dcca66bce8 p2p: Cache inbound flag on Peer.isInbound to avoid a race 2018-06-21 12:22:47 -04:00
Andrey Petrov
399aa710d5 p2p: Attempt to race check peer.Inbound() in TestServerDial 2018-06-21 12:22:47 -04:00
Andrey Petrov
699794d88d p2p: More tests for AddTrustedPeer/RemoveTrustedPeer 2018-06-21 12:22:47 -04:00
Andrey Petrov
773857a524 p2p: Test for MaxPeers=0 and TrustedPeer override 2018-06-21 12:21:48 -04:00
Andrey Petrov
2a75fe3308 rpc: Add admin_addTrustedPeer and admin_removeTrustedPeer.
These RPC calls are analogous to Parity's parity_addReservedPeer and
parity_removeReservedPeer.

They are useful for adjusting the trusted peer set during runtime,
without requiring restarting the server.
2018-06-21 12:21:48 -04:00
Elad
1836366ac1 all: library changes for swarm-network-rewrite (#16898)
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:

- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
2018-06-14 11:21:17 +02:00
Felix Lange
0255951587 crypto: replace ToECDSAPub with error-checking func UnmarshalPubkey (#16932)
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
2018-06-12 15:26:08 +02:00
Dmitry Shulyak
7677ec1f34 p2p/discv5: add egress/ingress traffic metrics to discv5 udp transport (#16369) 2018-05-29 13:46:09 +02:00
Felix Lange
6286c255f1
p2p/enr: updates for discovery v4 compatibility (#16679)
This applies spec changes from ethereum/EIPs#1049 and adds support for
pluggable identity schemes.

Some care has been taken to make the "v4" scheme standalone. It uses
public APIs only and could be moved out of package enr at any time.

A couple of minor changes were needed to make identity schemes work:

- The sequence number is now updated in Set instead of when signing.
- Record is now copy-safe, i.e. calling Set on a shallow copy doesn't
  modify the record it was copied from.
2018-05-17 15:11:27 +02:00
Guilherme Salgado
c60f6f6214 p2p: don't discard reason set by Disconnect (#16559)
Peer.run was discarding the reason for disconnection sent to the disc
channel by Disconnect.
2018-05-09 01:20:20 +02:00
Ivan Daniluk
c4a4613d95 p2p/simulations/adapters: fix websocket log line parsing in exec adapter (#16667) 2018-05-08 17:05:27 +02:00
kiel barry
864e80a48f p2p: fix some golint warnings (#16577) 2018-05-08 13:08:43 +02:00
GagziW
f2447bd4c3 p2p: changed if-else blocks to conform with golint (#16660) 2018-05-03 11:33:39 +03:00
thomasmodeneis
ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
David Huie
23ac783332 ecies: drop randomness parameter from PrivateKey.Decrypt (#16374)
The parameter `rand` is unused in `PrivateKey.Decrypt`. Decryption in
the ECIES encryption scheme is deterministic, so randomness isn't
needed.
2018-03-26 13:46:18 +03:00
JU HYEONG PARK
61c9730b2d p2p: fix doEncHandshake documentation (#16184) 2018-02-26 17:22:46 +01:00
Anton Evangelatov
ae9f97221a metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.

* vendor: change nonsense/go-metrics to ethersphere/go-metrics

* go-metrics: add tests. move ResettingTimer logic from reporter to type.

* all, metrics: pull in metrics package in go-ethereum

* metrics/test: make sure metrics are enabled for tests

* metrics: apply gosimple rules

* metrics/exp, internal/debug: init expvar endpoint when starting pprof server

* internal/debug: tiny comment formatting fix
2018-02-23 11:56:08 +02:00
Ivan Daniluk
8522b31221 p2p: remove unused code (#16158)
* p2p: remove unused code

* p2p: remove unused imports
2018-02-22 19:20:28 +02:00
Balint Gabor
221486a291
Merge pull request #15919 from ethersphere/p2p-protocols-pr
p2p/protocols, p2p/testing: protocol abstraction and testing
2018-02-22 15:02:51 +01:00
Anton Evangelatov
1e457b6599 p2p: don't send DiscReason when using net.Pipe (#16004) 2018-02-22 11:41:06 +01:00
Felix Lange
28b20cff4b p2p/protocols: gofmt -w -s 2018-02-22 11:37:57 +01:00
Dmitry Shulyak
14c76371ba p2p: when peer is removed remove it also from dial history (#16060)
This change removes a peer information from dialing history
when peer is removed from static list. It allows to force a
server to re-dial concrete peer if it is needed.

In our case we are running geth node on mobile devices, and
it is common for a network connection to flap on mobile.
Almost every time it flaps or network connection is changed
from cellular to wifi peers are disconnected with read
timeout. And usually it takes 30 seconds (default expiration
timeout) to recover connection with static peers after
connectivity is restored.

This change allows us to reconnect with peers almost
immediately and it seems harmless enough.
2018-02-21 15:03:26 +01:00
Janos Guljas
e07603bbc4 p2p/testing: check for all expectations in TestExchanges
Handle all expectations in ProtocolSession.TestExchanges in any
order that are received.
2018-02-17 23:42:28 +01:00
Felix Lange
aeedec4078 p2p/discover: s/lastPong/bondTime/, update TestUDP_findnode
I forgot to change the check in udp.go when I changed Table.bond to be
based on lastPong instead of node presence in db. Rename lastPong to
bondTime and add hasBond so it's clearer what this DB key is used for
now.
2018-02-16 21:29:20 +01:00
Péter Szilágyi
32301a4d6b
p2p/discover: validate bond against lastpong, not db presence 2018-02-16 17:05:08 +02:00
Felix Lange
a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Péter Szilágyi
20797348ca
p2p/discover: fix out-of-bounds issue 2018-02-13 20:59:43 +02:00
Martin Holst Swende
589b603a9b rpc: dns rebind protection (#15962)
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks

* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d

* rpc: add wildcard support for rpcallowedhosts + go fmt

* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check

* http, rpc, utils: make vhosts into map, address review concerns

* node: change log messages to use geth standard (not sprintf)

* rpc: fix spelling
2018-02-12 14:52:07 +02:00
Felix Lange
9123eceb0f p2p, p2p/discover: misc connectivity improvements (#16069)
* p2p: add DialRatio for configuration of inbound vs. dialed connections

* p2p: add connection flags to PeerInfo

* p2p/netutil: add SameNet, DistinctNetSet

* p2p/discover: improve revalidation and seeding

This changes node revalidation to be periodic instead of on-demand. This
should prevent issues where dead nodes get stuck in closer buckets
because no other node will ever come along to replace them.

Every 5 seconds (on average), the last node in a random bucket is
checked and moved to the front of the bucket if it is still responding.
If revalidation fails, the last node is replaced by an entry of the
'replacement list' containing recently-seen nodes.

Most close buckets are removed because it's very unlikely we'll ever
encounter a node that would fall into any of those buckets.

Table seeding is also improved: we now require a few minutes of table
membership before considering a node as a potential seed node. This
should make it less likely to store short-lived nodes as potential
seeds.

* p2p/discover: fix nits in UDP transport

We would skip sending neighbors replies if there were fewer than
maxNeighbors results and CheckRelayIP returned an error for the last
one. While here, also resolve a TODO about pong reply tokens.
2018-02-12 14:36:09 +02:00
Felföldi Zsolt
c4712bf96b p2p/discv5: fix multiple discovery issues (#16036)
* p2p/discv5: add query delay, fix node address update logic, retry refresh if empty

* p2p/discv5: remove unnecessary ping before topic query

* p2p/discv5: do not filter local address from topicNodes

* p2p/discv5: remove canQuery()

* p2p/discv5: gofmt
2018-02-08 19:06:31 +02:00
Felföldi Zsolt
6198c53e28 p2p/discv5: fix removeTicketRef cached ticket removal (#15995) 2018-01-30 18:01:22 +02:00
Felföldi Zsolt
397c6cde1e p2p/discv5: fix topic register panic at shutdown (#15946) 2018-01-23 12:53:09 +02:00
Martin Holst Swende
48641d7308
p2p/discv5: logs info about discv5 node info at bind time 2018-01-23 08:50:11 +01:00
Felföldi Zsolt
92580d69d3 p2p, p2p/discover, p2p/discv5: implement UDP port sharing (#15200)
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.

DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
2018-01-22 13:38:34 +01:00
zelig
407339085f p2p/protocols, p2p/testing: protocol abstraction and testing 2018-01-18 10:53:47 +01:00
Felix Lange
5c2f1e0014 all: update generated code (#15808)
* core/types, core/vm, eth, tests: regenerate gencodec files

* Makefile: update devtools target

Install protoc-gen-go and print reminders about npm, solc and protoc.
Also switch to github.com/kevinburke/go-bindata because it's more
maintained.

* contracts/ens: update contracts and regenerate with solidity v0.4.19

The newer upstream version of the FIFSRegistrar contract doesn't set the
resolver anymore. The resolver is now deployed separately.

* contracts/release: regenerate with solidity v0.4.19

* contracts/chequebook: fix fallback and regenerate with solidity v0.4.19

The contract didn't have a fallback function, payments would be rejected
when compiled with newer solidity. References to 'mortal' and 'owned'
use the local file system so we can compile without network access.

* p2p/discv5: regenerate with recent stringer

* cmd/faucet: regenerate

* dashboard: regenerate

* eth/tracers: regenerate

* internal/jsre/deps: regenerate

* dashboard: avoid sed -i because it's not portable

* accounts/usbwallet/internal/trezor: fix go generate warnings
2018-01-08 14:15:57 +02:00
ferhat elmas
5866626b08 core, p2p/discv5: use time.NewTicker instead of time.Tick (#15747) 2018-01-02 12:50:46 +01:00
Anton Evangelatov
36a10875c8 p2p/enr: initial implementation (#15585)
Initial implementation of ENR according to ethereum/EIPs#778
2017-12-29 21:18:51 +01:00
Péter Szilágyi
c15d76a40f p2p/discv5: fix reg lookup, polish code, use logger (#15737) 2017-12-28 14:17:03 +01:00
ferhat elmas
afa3c72c40 p2p/discover: fix leaked goroutine in data expiration 2017-12-18 09:16:54 +01:00
Felix Lange
3654aeaa4f
p2p/simulations: fix gosimple nit (#15661) 2017-12-13 03:15:27 +01:00
holisticode
fd777bb210 p2p/simulations: add mocker functionality (#15207)
This commit adds mocker functionality to p2p/simulations. A
mocker allows to starting/stopping of nodes via the HTTP API.
2017-12-12 19:10:41 +01:00
Zach
3da1bf8ca1 all: use gometalinter.v2, fix new gosimple issues (#15650) 2017-12-12 19:05:47 +01:00
ferhat elmas
1d06e41f04 p2p, swarm/network/kademlia: use IsZero to check for zero time (#15603) 2017-12-04 11:07:10 +01:00
Lewis Marshall
54aeb8e4c0 p2p/simulations: various stability fixes (#15198)
p2p/simulations: introduce dialBan

- Refactor simulations/network connection getters to support
  avoiding simultaneous dials between two peers If two peers dial
  simultaneously, the connection will be dropped to help avoid
  that, we essentially lock the connection object with a
  timestamp which serves as a ban on dialing for a period of time
  (dialBanTimeout).

- The connection getter InitConn can be wrapped and passed to the
  nodes via adapters.NodeConfig#Reachable field and then used by
  the respective services when they initiate connections. This
  massively stablise the emerging connectivity when running with
  hundreds of nodes bootstrapping a network.

p2p: add Inbound public method to p2p.Peer

p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.

p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
2017-12-01 12:49:04 +01:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
Darrel Herbst
d54e3539d4 p2p/nat: delete port mapping before adding (#15222)
Fixes #1024
2017-10-06 13:39:47 +02:00
Péter Szilágyi
2ee885958b p2p: snappy encoding for devp2p (version bump to 5) (#15106)
* p2p: snappy encoding for devp2p (version bump to 5)

* p2p: remove lazy decompression, enforce 16MB limit
2017-09-26 16:54:49 +03:00
Lewis Marshall
9feec51e2d p2p: add network simulation framework (#14982)
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
2017-09-25 10:08:07 +02:00
Martin Holst Swende
dc92779c0a p2p: change ping ticker to timer (#15071)
Using a Timer over Ticker seems to be a lot better, though I cannot fully
account for why that it behaves so (since Ticker should be more bursty, but not
necessarily more active over time, but that may depend on how long window it
uses to decide on when to tick next)
2017-09-04 09:24:52 +02:00
Ali Hajimirza
33b158e0ed discover: Changed Logging from Debug to Info (#14485) 2017-05-20 13:10:59 +02:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Péter Szilágyi
04fcae207d p2p: if no nodes are connected, attempt dialing bootnodes (#13874) 2017-04-10 18:33:41 +02:00
Felix Lange
96ae35e2ac p2p, p2p/discover, p2p/nat: rework logging using context keys 2017-02-28 10:20:29 +01:00
Felix Lange
d0eba23af3 all: disable log message colors outside of geth
Also tweak behaviour so colors are only enabled when stderr is a terminal.
2017-02-27 15:33:12 +01:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Péter Szilágyi
189dee26c6
p2p: remove trailing newlines from log messages 2017-02-23 12:00:04 +02:00
Felix Lange
9b0af51386 crypto: add btcec fallback for sign/recover without cgo (#3680)
* vendor: add github.com/btcsuite/btcd/btcec

* crypto: add btcec fallback for sign/recover without cgo

This commit adds a non-cgo fallback implementation of secp256k1
operations.

* crypto, core/vm: remove wrappers for sha256, ripemd160
2017-02-18 09:24:12 +01:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
13e3b2f433 logger, pow/dagger, pow/ezp: delete dead code 2017-01-06 18:18:07 +01:00
Felix Lange
f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Péter Szilágyi
21fd9f037e
p2p/nat: fix a bytes based net.IP comparison 2016-12-15 16:28:31 +02:00
Péter Szilágyi
2573094df2
p2p/discover, p2p/discv5: use flexible comparison for IPs 2016-12-14 18:40:49 +02:00
Zsolt Felfoldi
a6d3bf6fc3 p2p/discv5: search and lookup improvement 2016-12-08 13:38:15 +01:00
Felix Lange
a47341cf96 p2p, p2p/discover, p2p/discv5: add IP network restriction feature
The p2p packages can now be configured to restrict all communication to
a certain subset of IP networks. This feature is meant to be used for
private networks.
2016-11-22 22:21:18 +01:00
Felix Lange
a98d1d67d6 p2p/discover, p2p/discv5: prevent relay of invalid IPs and low ports
The discovery DHT contains a number of hosts with LAN and loopback IPs.
These get relayed because some implementations do not perform any checks
on the IP.

go-ethereum already prevented relay in most cases because it verifies
that the host actually exists before adding it to the local table. But
this verification causes other issues. We have received several reports
where people's VPSs got shut down by hosting providers because sending
packets to random LAN hosts is indistinguishable from a slow port scan.

The new check prevents sending random packets to LAN by discarding LAN
IPs sent by Internet hosts (and loopback IPs from LAN and Internet
hosts). The new check also blacklists almost all currently registered
special-purpose networks assigned by IANA to avoid inciting random
responses from services in the LAN.

As another precaution against abuse of the DHT, ports below 1024 are now
considered invalid.
2016-11-22 22:21:18 +01:00
Felix Lange
ba2884f343 p2p/discover, p2p/discv5: use netutil.IsTemporaryError 2016-11-22 22:21:15 +01:00
Felix Lange
1d80155d5e p2p/netutil: new package for network utilities
The new package contains three things for now:

- IP network list parsing and matching
- The WSAEMSGSIZE workaround, which is duplicated in p2p/discover and
  p2p/discv5.
2016-11-22 22:20:38 +01:00
Péter Szilágyi
b61f48e5aa
cmd, mobile, node, p2p: surface the discovery V5 bootnodes 2016-11-14 18:00:14 +02:00
Péter Szilágyi
178da7c6a9
mobile: initial wrappers for mobile support 2016-11-14 17:56:58 +02:00
Zsolt Felfoldi
1f70b279ba p2p/discv5: added new bootnodes 2016-11-14 13:22:19 +01:00
Zsolt Felfoldi
e33e57684f p2p/discv5: fixed bootnode connect issues 2016-11-14 13:22:19 +01:00
Zsolt Felfoldi
09baeec0da discv5: fixed state machine lockup bug 2016-11-11 03:10:46 +01:00
Felix Lange
b8bd9a71c8 all: update license information 2016-11-09 02:51:34 +01:00
Felix Lange
0f19cbc6e5 p2p/discv5: fix build with Go 1.5, delete package testimg 2016-11-09 02:18:48 +01:00
Zsolt Felfoldi
49da42983a p2p/discv5: added new topic discovery package 2016-11-09 02:12:53 +01:00
Felix Lange
44bc2e80dd Merge pull request #2914 from fjl/node-coinhabit
cmd/utils, node: make datadir reusable for bzzd
2016-09-29 14:24:37 +02:00
ken10100147
afecb93e2e p2p/nat: fix parameter order for AddMapping 2016-09-28 12:11:58 +02:00
Felix Lange
f5c432bcab p2p/nat: delay auto discovery until first use
Port mapper auto discovery used to run immediately after parsing the
--nat flag, giving it a slight performance boost. But this is becoming
inconvenient because we create node.Node for all geth operations
including account management and bare chain interaction. Delay
autodiscovery until the first use instead, which avoids any network
interaction until the node is actually started.
2016-09-16 11:06:52 +02:00
Felix Lange
b8ba80bff7 Merge pull request #2740 from Firescar96/removepeer
node, p2p, internal: Add ability to remove peers via admin interface
2016-07-29 12:34:28 +02:00
Firescar96
4c3da0f2e1 node, p2p, internal: Add ability to remove peers via admin interface 2016-07-14 18:51:41 -04:00
villesundell
c7442ef0d2 ethdb, p2p/discover: replace "alloted" with "allotted" (#2785) 2016-07-12 11:42:47 +02:00
Felix Lange
542b839ec7 node, p2p: move network config out of Server
This silences a go vet message about copying p2p.Server in package node.
2016-05-18 12:19:04 +02:00
Felix Lange
8110671960 p2p/discover: prevent bonding self 2016-05-02 18:50:16 +02:00
Felix Lange
32bb280179 p2p: improve readability of dial task scheduling code 2016-05-02 18:50:15 +02:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Felix Lange
659c0cb9e8 p2p: enable EIP-8 handshake sending
With the Ethereum Homestead fork is now behind us, we can
assume that everyone runs an EIP-8 capable client.
2016-04-03 23:10:11 +02:00
Péter Szilágyi
64c8e2f2ca p2p/nat: fix #2291, NAT discovery did't abort on failure 2016-03-14 17:08:15 +02:00
Jeffrey Wilcke
483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Péter Szilágyi
ac954f48bd p2p/discover: emphasize warning, add 10 min cooldown 2016-02-24 12:16:28 +02:00
Péter Szilágyi
b1908f6a16 psp/discovery: NTP sanity check clock drift in case of expirations 2016-02-24 12:14:15 +02:00
Ricardo Catalinas Jiménez
436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Felix Lange
7d8155714b p2p: EIP-8 changes 2016-02-19 11:14:48 +01:00
Felix Lange
ee1debda53 p2p/discover: EIP-8 changes 2016-02-19 11:14:42 +01:00
Felix Lange
2871781f64 p2p/discover: fix Windows-specific issue for larger-than-buffer packets
On Windows, UDPConn.ReadFrom returns an error for packets larger
than the receive buffer. The error is not marked temporary, causing
our loop to exit when the first oversized packet arrived. The fix
is to treat this particular error as temporary.

Fixes: #1579, #2087
Updates: #2082
2016-01-22 23:44:25 +01:00
Felix Lange
b3f1f4c673 p2p/discover: attempt to deflake TestUDP_responseTimeouts
The test expected the timeout to fire after a matcher for the response
was added, but the timeout is random and fired sooner sometimes.
2015-12-17 23:44:56 +01:00
Felix Lange
6c41e675ec p2p: resolve incomplete dial targets
This change makes it possible to add peers without providing their IP
address. The endpoint of the target node is resolved using the discovery
protocol.
2015-12-17 23:39:49 +01:00
Felix Lange
04c6369a09 p2p, p2p/discover: track bootstrap state in p2p/discover
This change simplifies the dial scheduling logic because it
no longer needs to track whether the discovery table has been
bootstrapped.
2015-12-17 23:38:54 +01:00
Felix Lange
d1f507b7f1 p2p/discover: support incomplete node URLs, add Resolve 2015-12-17 23:38:54 +01:00
Péter Szilágyi
abb53644c6 p2p: always allow dynamic dials if network not disabled 2015-12-03 11:45:35 +02:00
Gustav Simonsson
c8ad64f33c crypto, crypto/ecies, crypto/secp256k1: libsecp256k1 scalar mult
thanks to Felix Lange (fjl) for help with design & impl
2015-11-30 13:43:32 +01:00
Péter Szilágyi
9e1d9bff3b node: customizable protocol and service stacks 2015-11-27 11:06:12 +02:00
Jeffrey Wilcke
e165c2d23c Merge pull request #1934 from karalabe/polish-protocol-infos
eth, p2p, rpc/api: polish protocol info gathering
2015-11-04 11:59:31 +01:00
Felix Lange
f570b68ed1 p2p/nat: add docs for discover 2015-10-29 22:54:44 +01:00
Felix Lange
bf11a47f22 Godeps: upgrade github.com/huin/goupnp to 90f71cb5 2015-10-29 22:53:59 +01:00
Péter Szilágyi
e46ab3bdcd eth, p2p, rpc/api: polish protocol info gathering 2015-10-28 12:44:15 +02:00
Felix Lange
32dda97602 p2p/discover: ignore packet version numbers
The strict matching can get in the way of protocol upgrades.
2015-09-30 16:23:03 +02:00
Felix Lange
631bf36102 p2p/discover: remove unused lastLookup field 2015-09-30 16:23:03 +02:00
Felix Lange
b4374436f3 p2p/discover: fix race involving the seed node iterator
nodeDB.querySeeds was not safe for concurrent use but could be called
concurrenty on multiple goroutines in the following case:

- the table was empty
- a timed refresh started
- a lookup was started and initiated refresh

These conditions are unlikely to coincide during normal use, but are
much more likely to occur all at once when the user's machine just woke
from sleep. The root cause of the issue is that querySeeds reused the
same leveldb iterator until it was exhausted.

This commit moves the refresh scheduling logic into its own goroutine
(so only one refresh is ever active) and changes querySeeds to not use
a persistent iterator. The seed node selection is now more random and
ignores nodes that have not been contacted in the last 5 days.
2015-09-30 16:23:03 +02:00
Péter Szilágyi
c51e153b5c eth, metrics, p2p: prepare metrics and net packets to eth/62 2015-08-21 10:30:57 +03:00
Jeffrey Wilcke
e2d44814a5 Merge pull request #1694 from obscuren/hide-fdtrack
fdtrack: hide message
2015-08-19 13:50:54 -07:00
Jeffrey Wilcke
269c5c7107 Revert "fdtrack: temporary hack for tracking file descriptor usage"
This reverts commit 5c949d3b3b.
2015-08-19 21:46:01 +02:00
Felix Lange
dd54fef898 p2p/discover: don't attempt to replace nodes that are being replaced
PR #1621 changed Table locking so the mutex is not held while a
contested node is being pinged. If multiple nodes ping the local node
during this time window, multiple ping packets will be sent to the
contested node. The changes in this commit prevent multiple packets by
tracking whether the node is being replaced.
2015-08-19 14:57:16 +02:00
Felix Lange
edccc7ae34 p2p: continue listening after temporary errors 2015-08-19 14:39:04 +02:00
Felix Lange
7d5ff770e2 p2p/discover: continue reading after temporary errors
Might solve #1579
2015-08-19 14:38:55 +02:00
Felix Lange
a89cfe92cc Merge pull request #1470 from ebuchman/encHandshake
p2p: validate recovered ephemeral pubkey
2015-08-13 11:59:27 +02:00
Felix Lange
0d10d5a0a5 p2p: fix value of DiscSubprotocolError
We had the wrong value (12) since forever.
2015-08-12 14:15:54 +02:00
Felix Lange
590c99a98f p2p/discover: fix UDP reply packet timeout handling
If the timeout fired (even just nanoseconds) before the deadline of the
next pending reply, the timer was not rescheduled. The timer would've
been rescheduled anyway once the next packet was sent, but there were
cases where no next packet could ever be sent due to the locking issue
fixed in the previous commit.

As timing-related bugs go, this issue had been present for a long time
and I could never reproduce it. The test added in this commit did
reproduce the issue on about one out of 15 runs.
2015-08-11 11:42:17 +02:00
Felix Lange
01ed3fa1a9 p2p/discover: unlock the table during ping replacement
Table.mutex was being held while waiting for a reply packet, which
effectively made many parts of the whole stack block on that packet,
including the net_peerCount RPC call.
2015-08-11 11:42:17 +02:00
Felix Lange
6ee908848c p2p/nat: disable UPnP test on windows 2015-08-06 17:18:59 +02:00
Felix Lange
b23b4dbd79 p2p/discover: close Table during testing
Not closing the table used to be fine, but now the table has a database.
2015-08-06 12:27:59 +02:00
Felix Lange
5c949d3b3b fdtrack: temporary hack for tracking file descriptor usage
Package fdtrack logs statistics about open file descriptors.
This should help identify the source of #1549.
2015-08-04 03:10:27 +02:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Ethan Buchman
37efd08b42 p2p: validate recovered ephemeral pubkey against checksum in decodeAuthMsg 2015-07-14 03:06:44 +00:00
Felix Lange
bdae4fd573 all: add some godoc synopsis comments 2015-07-07 14:12:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Péter Szilágyi
01fe972113 cmd, core, eth, metrics, p2p: require enabling metrics 2015-06-30 00:51:46 +02:00
Péter Szilágyi
216fc267fa p2p: fix local/remote cap/protocol mixup 2015-06-26 20:45:13 +03:00
Péter Szilágyi
d84638bd31 p2p: support protocol version negotiation 2015-06-26 15:48:50 +03:00
Péter Szilágyi
6994a3daaa p2p: instrument P2P networking layer 2015-06-24 18:33:33 +03:00
Felix Lange
6fb810adaa p2p: throttle all discovery lookups
Lookup calls would spin out of control when network connectivity was
lost. The throttling that was in place only took effect when the table
returned zero results, which doesn't happen very often.

The new throttling should not have a negative impact when the host is
online. Lookups against the network take some time and dials for all
results must complete or hit the cache before a new one is started. This
usually takes longer than four seconds, leaving online lookups
unaffected.

Fixes #1296
2015-06-22 01:07:58 +02:00
Felix Lange
70da79f04c p2p: improve disconnect logging 2015-06-15 15:03:46 +02:00
Felix Lange
8dcbdcad0a p2p: track write errors and prevent writes during shutdown
As of this commit, we no longer rely on the protocol handler to report
write errors in a timely fashion. When a write fails, shutdown is
initiated immediately and no new writes can start. This will also
prevent new writes from starting after Server.Stop has been called.
2015-06-15 15:03:46 +02:00
Felix Lange
a8e4cb6dfe p2p/discover: use separate rand.Source instances in tests
rand.Source isn't safe for concurrent use.
2015-06-10 15:18:01 +02:00
Felix Lange
261a8077c4 p2p/discover: deflake TestUDP_successfulPing 2015-06-10 13:08:21 +02:00
Péter Szilágyi
1cbbfbe7fa p2p: fix a close race in the dial test 2015-06-09 22:26:26 +03:00
Felix Lange
3239aca69b p2p: bump global write timeout to 20s
The previous value of 5 seconds causes timeouts for legitimate messages
if large messages are sent.
2015-06-09 17:07:10 +02:00
Péter Szilágyi
ff84352fb7 p2p: fix close data race 2015-06-09 16:12:24 +03:00
Felix Lange
fc6a5ae3ec p2p/nat: add timeout for UPnP SOAP requests 2015-06-04 22:25:43 +02:00
Felix Lange
8e4512a5e7 p2p/nat: bump timeout in TestAutoDiscRace 2015-05-28 01:09:26 +02:00
Péter Szilágyi
612f01400f p2p/discover: bond with seed nodes too (runs only if findnode failed) 2015-05-26 23:30:41 +02:00
Péter Szilágyi
3630432dfb p2p/discovery: fix a cornercase loop if no seeds or bootnodes are known 2015-05-26 23:30:40 +02:00
Péter Szilágyi
f539ed1e66 p2p/discover: force refresh if the table is empty 2015-05-26 23:30:40 +02:00
Péter Szilágyi
5076170f34 p2p/discover: permit temporary bond failures for previously known nodes 2015-05-26 23:30:40 +02:00
Péter Szilágyi
6078aa08eb p2p/discover: watch find failures, evacuate on too many, rebond if failed 2015-05-26 23:30:40 +02:00
Péter Szilágyi
64174f196f p2p/discover: add support for counting findnode failures 2015-05-26 23:30:40 +02:00
Péter Szilágyi
68898a4d6b p2p: fix Self() panic if listening is disabled 2015-05-26 19:16:05 +03:00
Péter Szilágyi
e1a0ee8fc5 cmd/geth, cmd/utils, eth, p2p: pass and honor a no discovery flag 2015-05-26 19:07:24 +03:00
Péter Szilágyi
278183c7e7 eth, p2p: start the p2p server even if maxpeers == 0 2015-05-26 17:49:37 +03:00
Felix Lange
9e1fd70b50 p2p: decrease frameReadTimeout to 30s
This detects hanging connections sooner. We send a ping every 15s and
other implementation have similar limits.
2015-05-25 01:17:14 +02:00
Felix Lange
1440f9a37a p2p: new dialer, peer management without locks
The most visible change is event-based dialing, which should be an
improvement over the timer-based system that we have at the moment.
The dialer gets a chance to compute new tasks whenever peers change or
dials complete. This is better than checking peers on a timer because
dials happen faster. The dialer can now make more precise decisions
about whom to dial based on the peer set and we can test those
decisions without actually opening any sockets.

Peer management is easier to test because the tests can inject
connections at checkpoints (after enc handshake, after protocol
handshake).

Most of the handshake stuff is now part of the RLPx code. It could be
exported or move to its own package because it is no longer entangled
with Server logic.
2015-05-25 01:17:14 +02:00
Felix Lange
9f38ef5d97 p2p/discover: add ReadRandomNodes 2015-05-25 01:17:14 +02:00
Felix Lange
64564da20b p2p: decrease maximum message size for devp2p to 1kB
The previous limit was 10MB which is unacceptable for all kinds
of reasons, the most important one being that we don't want to
allow the remote side to make us allocate 10MB at handshake time.
2015-05-25 01:17:14 +02:00
Felix Lange
dbdc5fd4b3 p2p: delete Server.Broadcast 2015-05-25 01:17:14 +02:00
Péter Szilágyi
cbd3ae6906 p2p/discover: fix #838, evacuate self entries from the node db 2015-05-21 19:41:46 +03:00
Péter Szilágyi
af24c271c7 p2p/discover: fix database presistency test folder 2015-05-21 19:28:10 +03:00
Jeffrey Wilcke
90b94e64fc Merge pull request #971 from fjl/p2p-limit-tweaks
p2p: tweak connection limits
2015-05-14 08:15:51 -07:00
Felix Lange
d2f119cf9b p2p/discover: limit open files for node database 2015-05-14 15:01:13 +02:00
Felix Lange
206fe25971 p2p: remove testlog 2015-05-14 14:56:34 +02:00
Felix Lange
7fa2607bd1 p2p/discover: bump maxBondingPingPongs to 16
This should increase the speed a bit because all findnode
results (up to 16) can be verified at the same time.
2015-05-14 14:53:29 +02:00
Felix Lange
691cb90284 p2p: log remote reason when disconnect is requested
The returned reason is currently not used except for the log
message. This change makes the log messages a bit more useful.
The handshake code also returns the remote reason.
2015-05-14 14:53:29 +02:00
Felix Lange
c14de2e973 p2p/nat: tweak port mapping log messages and levels
People stil get confused about the messages. This commit changes
the levels so that the only thing printed at the default level (info)
is a successful mapping.
2015-05-14 12:54:59 +02:00
Felix Lange
663d4e0aff p2p/nat: add test for UPnP auto discovery via SSDP
The test listens for multicast UDP packets on the default interface
because I couldn't get it to work reliably on loopback without massive
changes to goupnp. This means that the test might fail when there is a
UPnP-enabled router attached on that interface. I checked that locally
by looping the test and it passes reliably because the local SSDP server
always responds faster.
2015-05-14 12:13:19 +02:00
Felix Lange
983f5a717a p2p/nat: fix concurrent access to autodisc Interface
Concurrent calls to Interface methods on autodisc could return a "not
discovered" error if the discovery did not finish before the call.
autodisc.wait expected the done channel to carry the found Interface
but it was closed instead.

The fix is to use sync.Once for now, which is easier to get right.
And there is a test. Finally.

This will have to change again when we introduce re-discovery.
2015-05-14 03:53:11 +02:00
Felix Lange
7efeb4bd96 p2p: bump maxAcceptConns and defaultDialTimout
On the test network, we've seen that it becomes harder to connect
if the queues are so short.
2015-05-14 03:48:28 +02:00
Felix Lange
251846d65a p2p/discover: fix out-of-bounds slicing for chunked neighbors packets
The code assumed that Table.closest always returns at least 13 nodes.
This is not true for small tables (e.g. during bootstrap).
2015-05-13 21:49:04 +02:00
subtly
8eef2b765a fix test. 2015-05-13 20:15:01 +02:00
subtly
a32693770c Manual send of multiple neighbours packets. Test receiving multiple neighbours packets. 2015-05-13 20:03:17 +02:00
subtly
7473c93668 UDP Interop. Limit datagrams to 1280bytes.
We don't have a UDP which specifies any messages that will be 4KB. Aside from being implemented for months and a necessity for encryption and piggy-backing packets, 1280bytes is ideal, and, means this TODO can be completed!

Why 1280 bytes?
* It's less than the default MTU for most WAN/LAN networks. That means fewer fragmented datagrams (esp on well-connected networks).
* Fragmented datagrams and dropped packets suck and add latency while OS waits for a dropped fragment to never arrive (blocking readLoop())
* Most of our packets are < 1280 bytes.
* 1280 bytes is minimum datagram size and MTU for IPv6 -- on IPv6, a datagram < 1280bytes will *never* be fragmented.

UDP datagrams are dropped. A lot! And fragmented datagrams are worse. If a datagram has a 30% chance of being dropped, then a fragmented datagram has a 60% chance of being dropped. More importantly, we have signed packets and can't do anything with a packet unless we receive the entire datagram because the signature can't be verified. The same is true when we have encrypted packets.

So the solution here to picking an ideal buffer size for receiving datagrams is a number under 1400bytes. And the lower-bound value for IPv6 of 1280 bytes make's it a non-decision. On IPv4 most ISPs and 3g/4g/let networks have an MTU just over 1400 -- and *never* over 1500. Never -- that means packets over 1500 (in reality: ~1450) bytes are fragmented. And probably dropped a lot.

Just to prove the point, here are pings sending non-fragmented packets over wifi/ISP, and a second set of pings via cell-phone tethering. It's important to note that, if *any* router between my system and the EC2 node has a lower MTU, the message would not go through:

On wifi w/normal ISP:
localhost:Debug $ ping -D -s 1450 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1450 data bytes
1458 bytes from 52.6.250.242: icmp_seq=0 ttl=42 time=104.831 ms
1458 bytes from 52.6.250.242: icmp_seq=1 ttl=42 time=119.004 ms
^C
--- 52.6.250.242 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 104.831/111.918/119.004/7.087 ms
localhost:Debug $ ping -D -s 1480 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1480 data bytes
ping: sendto: Message too long
ping: sendto: Message too long
Request timeout for icmp_seq 0
ping: sendto: Message too long
Request timeout for icmp_seq 1


Tethering to O2:
localhost:Debug $ ping -D -s 1480 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1480 data bytes
ping: sendto: Message too long
ping: sendto: Message too long
Request timeout for icmp_seq 0
^C
--- 52.6.250.242 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
localhost:Debug $ ping -D -s 1450 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1450 data bytes
1458 bytes from 52.6.250.242: icmp_seq=0 ttl=42 time=107.844 ms
1458 bytes from 52.6.250.242: icmp_seq=1 ttl=42 time=105.127 ms
1458 bytes from 52.6.250.242: icmp_seq=2 ttl=42 time=120.483 ms
1458 bytes from 52.6.250.242: icmp_seq=3 ttl=42 time=102.136 ms
2015-05-13 19:03:00 +02:00
Bas van Kervel
95773b9673 removed redundant newlines in import block 2015-05-12 15:20:53 +02:00
Bas van Kervel
b79dd188d9 replaced several path.* with filepath.* which is platform independent 2015-05-12 14:24:11 +02:00
Felix Lange
d4f0a67323 p2p: drop connections with no matching protocols 2015-05-08 16:09:55 +02:00
Felix Lange
9c0f36c46d p2p: use maxDialingConns instead of maxAcceptConns as dial limit 2015-05-08 16:09:55 +02:00
Felix Lange
914e57e49b p2p: fix disconnect at capacity
With the introduction of static/trusted nodes, the peer count
can go above MaxPeers. Update the capacity check to handle this.
While here, decouple the trusted nodes check from the handshake
by passing a function instead.
2015-05-08 16:09:54 +02:00
Péter Szilágyi
8735e5addd p2p: increase the handshake timeout in the tests 2015-05-07 15:30:56 +03:00
Péter Szilágyi
4d5a719f25 cmd, eth, p2p: introduce pending peer cli arg, add tests 2015-05-07 15:30:56 +03:00
Péter Szilágyi
af93217775 p2p: reduce the concurrent handshakes to 10/10 in/out 2015-05-07 15:22:09 +03:00
Péter Szilágyi
2060bc8bac p2p: fix dial throttling race condition 2015-05-07 15:22:08 +03:00
Péter Szilágyi
29fef349ef p2p: fix a dialing race in the throttler 2015-05-07 15:22:08 +03:00
Péter Szilágyi
3953bf0031 p2p: limit the outbound dialing too 2015-05-07 15:22:08 +03:00
Jeffrey Wilcke
a0cb1945ae Merge pull request #866 from fjl/p2p-last-minute
Last minute p2p fixes
2015-05-06 14:49:52 -07:00
Felix Lange
3e2a928caa p2p: stop dialing at half the maximum peer count 2015-05-06 23:44:51 +02:00
Felix Lange
6a2fec5309 p2p, whisper: use glog for peer-level logging 2015-05-06 23:19:14 +02:00
Felix Lange
bcfd788661 p2p/discover: bump packet timeouts to 500ms 2015-05-06 22:59:00 +02:00
Felix Lange
fd4b75cfa8 p2p/nat: less confusing error logging 2015-05-06 22:58:03 +02:00
obscuren
062fa049d0 fixed merge issue 2015-05-06 22:54:21 +02:00
Felix Lange
2adcc31bb4 p2p/discover: new distance metric based on sha3(id)
The previous metric was pubkey1^pubkey2, as specified in the Kademlia
paper. We missed that EC public keys are not uniformly distributed.
Using the hash of the public keys addresses that. It also makes it
a bit harder to generate node IDs that are close to a particular node.
2015-05-06 16:10:41 +02:00
Péter Szilágyi
4accc187d5 eth, p2p: add trusted node list beside static list 2015-05-04 13:59:51 +03:00
Péter Szilágyi
54db54931e p2p: add static node dialing test 2015-05-04 13:08:42 +03:00
Péter Szilágyi
e82ddd9198 p2p: correct a leftover trusted -> static 2015-04-30 19:34:33 +03:00
Péter Szilágyi
413ace37d3 eth, p2p: rename trusted nodes to static, drop inbound extra slots 2015-04-30 19:32:48 +03:00
Péter Szilágyi
701591b403 cmd, eth, p2p: fix review issues enumerated by Felix 2015-04-30 16:15:29 +03:00
Péter Szilágyi
1528dbc171 p2p: add trust check to handshake, test privileged connectivity
Conflicts:
	p2p/server_test.go
2015-04-30 16:06:47 +03:00
Péter Szilágyi
14f32a0c3a p2p: reduce the severity of a debug log 2015-04-30 16:04:09 +03:00
Péter Szilágyi
de0549fabb cmd/geth, cmd/mist, cmd/utils, eth, p2p: support trusted peers 2015-04-30 16:03:10 +03:00
Felix Lange
72ab6d3255 p2p/discover: track sha3(ID) in Node 2015-04-30 15:02:23 +02:00
Felix Lange
b34a8ef624 p2p, p2p/discover: protocol version 4 2015-04-30 14:57:34 +02:00
Felix Lange
fc747ef4a6 p2p/discover: new endpoint format
This commit changes the discovery protocol to use the new "v4" endpoint
format, which allows for separate UDP and TCP ports and makes it
possible to discover the UDP address after NAT.
2015-04-30 14:57:33 +02:00
obscuren
01e3d694a6 p2p: added received at to peer message
p2p.Msg.ReceivedAt can be used for determining block propagation from
begining to end.
2015-04-29 22:49:58 +02:00
Péter Szilágyi
b569550a39 p2p/discover: fix api issues caused by leveldb update 2015-04-28 13:57:57 +03:00
Péter Szilágyi
4992765032 p2p/discover: fix goroutine leak due to blocking on sync.Once 2015-04-28 10:28:04 +03:00
Péter Szilágyi
437cf4b3ac p2p/discover: add node expirer and related tests 2015-04-27 17:38:28 +03:00
Péter Szilágyi
a136e2bb22 p2p/discover: parametrize nodedb version, add persistency tests 2015-04-27 15:28:17 +03:00
Péter Szilágyi
75fd738dea p2p/discover: drop a superfluous warning 2015-04-27 15:06:31 +03:00
Péter Szilágyi
706da56f75 p2p/discover: wrap the pinger to update the node db too 2015-04-27 14:56:42 +03:00
Péter Szilágyi
85b4b44235 p2p/discover: use iterator based seeding, drop old protocol test 2015-04-27 14:45:35 +03:00
Péter Szilágyi
8de8f61d36 p2p/discover: write the basic tests, catch RLP bug 2015-04-27 12:33:06 +03:00
Péter Szilágyi
0201c04b95 p2p/discovery: fix issues raised in the nodeDb PR 2015-04-27 10:19:16 +03:00
Péter Szilágyi
8646365b42 cmd/bootnode, eth, p2p, p2p/discover: use a fancier db design 2015-04-24 18:04:41 +03:00
Péter Szilágyi
6def110c37 cmd/bootnode, eth, p2p, p2p/discover: clean up the seeder and mesh into eth. 2015-04-24 11:33:55 +03:00
Péter Szilágyi
971702e7a1 p2p/discovery: fix broken tests due to API update 2015-04-24 11:23:20 +03:00
Péter Szilágyi
af923c965f p2p/discovery: use the seed table for finding nodes, auto drop stale ones 2015-04-24 11:23:20 +03:00
Péter Szilágyi
5f735d6fce cmd, eth, p2p, p2p/discover: init and clean up the seed cache 2015-04-24 11:23:20 +03:00
Felix Lange
936c8e19ff p2p/discover: store nodes in leveldb 2015-04-24 11:23:20 +03:00
Felix Lange
635b66acdc p2p: return zero node from Self if the server is not running
This helps with fixing the tests for cmd/geth to run without networking.
2015-04-22 12:31:19 +02:00
Felix Lange
9c7281c17e p2p: make DiscReason bigger than byte
We decode into [1]DiscReason in a few places. That doesn't work anymore
because package rlp no longer accepts RLP lists for byte arrays.
2015-04-17 14:45:10 +02:00
Felix Lange
eedbb1ee9a p2p/discover: use rlp.DecodeBytes 2015-04-17 14:45:09 +02:00
Felix Lange
56a48101dc cmd/rlpdump, cmd/utils, eth, p2p, whisper: use rlp input limit 2015-04-17 14:45:09 +02:00
Felix Lange
5528abc795 p2p: fix the dial timer
The dial timer was not reset properly when the peer count reached
MaxPeers.
2015-04-17 08:17:01 +02:00
obscuren
474aa924ca p2p: added limiter function to limit package broadcasting 2015-04-14 12:47:31 +02:00
Felix Lange
0217652d1b p2p/discover: improve timer handling for reply timeouts 2015-04-13 18:08:11 +02:00
Felix Lange
b8aeb04f6f p2p/discover: remove unused field Node.activeStamp 2015-04-13 17:44:14 +02:00
Felix Lange
b9929d289d p2p: fix unsynchronized map access during Server shutdown
removePeer can be called even after listenLoop and dialLoop have returned.
2015-04-13 17:37:32 +02:00
Felix Lange
995fab2ebc p2p: fix yet another disconnect hang
Peer.readLoop will only terminate if the connection is closed. Fix the
hang by closing the connection before waiting for readLoop to terminate.

This also removes the british disconnect procedure where we're waiting
for the remote end to close the connection. I have confirmed with
@subtly that cpp-ethereum doesn't adhere to it either.
2015-04-13 17:34:08 +02:00
Felix Lange
79a6782c1c p2p: fix goroutine leak when handshake read fails
This regression was introduced in b3c058a9e4.
2015-04-13 17:06:19 +02:00
Felix Lange
c5332537f5 p2p: limit number of lingering inbound pre-handshake connections
This is supposed to apply some back pressure so Server is not accepting
more connections than it can actually handle. The current limit is 50.
This doesn't really need to be configurable, but we'll see how it
behaves in our test nodes and adjust accordingly.
2015-04-10 17:24:41 +02:00
Felix Lange
56977c225e p2p: use RLock instead of Lock for pre-dial checks 2015-04-10 17:23:09 +02:00
Felix Lange
b3c058a9e4 p2p: improve disconnect signaling at handshake time
As of this commit, p2p will disconnect nodes directly after the
encryption handshake if too many peer connections are active.
Errors in the protocol handshake packet are now handled more politely
by sending a disconnect packet before closing the connection.
2015-04-10 16:57:56 +02:00
Felix Lange
99a1db2d40 p2p: don't mess with the socket deadline in Peer.readLoop
netWrapper already sets a read deadline in ReadMsg.
2015-04-10 13:26:28 +02:00
Felix Lange
145330fdf2 p2p: properly decrement peer wait group counter for setup errors 2015-04-10 13:26:27 +02:00
Felix Lange
f1d710af00 p2p: fix Peer shutdown deadlocks
There were multiple synchronization issues in the disconnect handling,
all caused by the odd special-casing of Peer.readLoop errors. Remove the
special handling of read errors and make readLoop part of the Peer
WaitGroup.

Thanks to @Gustav-Simonsson for pointing at arrows in a diagram
and playing rubber-duck.
2015-04-10 13:26:27 +02:00
Felix Lange
22d1f0faf1 p2p: improve peer selection logic
This commit introduces a new (temporary) peer selection
strategy based on random lookups.

While we're here, also implement the TODOs in dialLoop.
2015-04-10 13:26:27 +02:00
Felix Lange
7be05b4b9d p2p/discover: don't log packet content 2015-04-10 13:26:27 +02:00
Felix Lange
9cd8c96157 p2p/discover: make packet processing less concurrent 2015-04-10 13:26:27 +02:00
obscuren
688d118c7e Updated logging 2015-04-07 14:57:04 +02:00
Felix Lange
a77c431e37 p2p/discover: fix off by one error causing buckets to contain duplicates 2015-04-01 17:00:12 +02:00
Felix Lange
de7af720d6 p2p/discover: implement node bonding
This a fix for an attack vector where the discovery protocol could be
used to amplify traffic in a DDOS attack. A malicious actor would send a
findnode request with the IP address and UDP port of the target as the
source address. The recipient of the findnode packet would then send a
neighbors packet (which is 16x the size of findnode) to the victim.

Our solution is to require a 'bond' with the sender of findnode. If no
bond exists, the findnode packet is not processed. A bond between nodes
α and β is created when α replies to a ping from β.

This (initial) version of the bonding implementation might still be
vulnerable against replay attacks during the expiration time window.
We will add stricter source address validation later.
2015-04-01 17:00:12 +02:00
Felix Lange
92928309b2 p2p/discover: add version number to ping packet
The primary motivation for doing this right now is that old PoC 8
nodes and newer PoC 9 nodes keep discovering each other, causing
handshake failures.
2015-04-01 15:53:04 +02:00
zelig
9880f99ccb fix p2p/testlog_test 2015-03-23 16:41:41 +00:00
obscuren
a7ad9c309b Merge branch 'develop' into conversion 2015-03-21 14:52:42 +01:00
obscuren
4f5b362bda %#x => %x 2015-03-20 12:52:03 +01:00
Felix Lange
b9e0b11e7d p2p: interrupt MsgPipe payload read/write
This is better because protocols might not actually read the payload for
some errors (msg too big, etc.) which can be a pain to test with the old
behaviour.
2015-03-19 15:16:06 +01:00
Felix Lange
a7bced779a p2p: log disconnect requests
This helps a lot with debugging.
2015-03-19 15:15:07 +01:00
Felix Lange
5ba51594c7 p2p: use package rlp to encode messages
Message encoding functions have been renamed to catch any uses.
The switch to the new encoder can cause subtle incompatibilities.
If there are any users outside of our tree, they will at least be
alerted that there was a change.

NewMsg no longer exists. The replacements for EncodeMsg are called
Send and SendItems.
2015-03-19 15:11:02 +01:00
Felix Lange
4811f460e7 p2p: export ExpectMsg (for eth protocol testing) 2015-03-19 15:08:04 +01:00
zelig
b3e133dd15 Merge branch 'frontier/js' into frontier/nodeadmin.js 2015-03-16 22:50:29 +07:00
obscuren
b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
zelig
e150832734 p2p: server>discover table Self=Node exported 2015-03-15 13:38:41 +07:00
Taylor Gerring
485e37e889 Move MakeName to ethutil 2015-03-11 14:29:07 -05:00
Felix Lange
215c763d53 eth, p2p: delete p2p.Blacklist
It is unused and untested right now. We can
bring it back later if required.
2015-03-04 16:54:36 +01:00
Felix Lange
429828cd92 p2p: reject messages that cannot be written as simple RLPx frames
Until chunked frames are implemented we cannot send messages
with a size overflowing uint24.
2015-03-04 16:42:01 +01:00
Felix Lange
22659a7fea p2p: restore read/write timeouts
They got lost in the transition to rlpxFrameRW.
2015-03-04 16:42:00 +01:00
Felix Lange
7964f30dcb p2p: msg.Payload contains list data
With RLPx frames, the message code is contained in the
frame and is no longer part of the encoded data.

EncodeMsg, Msg.Decode have been updated to match.
Code that decodes RLP directly from Msg.Payload will need
to change.
2015-03-04 12:27:24 +01:00
Felix Lange
21649100b1 p2p: verify protocol handshake node ID 2015-03-04 12:27:24 +01:00
Felix Lange
7d39fd6678 p2p: make encryption handshake code easier to follow
This mostly changes how information is passed around.
Instead of using many function parameters and return values,
put the entire state in a struct and pass that.

This also adds back derivation of ecdhe-shared-secret. I deleted
it by accident in a previous refactoring.
2015-03-04 12:27:24 +01:00
Felix Lange
2c505efd1e p2p/discover: add NodeID.Pubkey 2015-03-04 12:27:24 +01:00
Felix Lange
d344054e5a p2p: make RLPx frame MAC 16 bytes as defined in the spec 2015-03-04 12:27:24 +01:00
Felix Lange
d084aed5e9 p2p: delete frameRW 2015-03-04 12:27:24 +01:00
Felix Lange
736e632215 p2p: use RLPx frames for messaging 2015-03-04 12:27:23 +01:00