Commit Graph

176 Commits

Author SHA1 Message Date
Felix Lange
dba1750eda p2p/discover: split out discv4 code
This change restructures the internals of p2p/discover to make room for
the discv5 code which will soon be added to this package.

- packet type names now have a "V4" suffix.
- ListenUDP returns *UDPv4 instead of *Table. This technically breaks
  the API but the only caller in go-ethereum is package p2p, which uses
  a compatible interface and doesn't need changes.
- The internal transport interface is changed to make Table reusable for v5.
- The 'lookup' code moves from table to transport. This required
  updating the lookup unit test to use udpTest instead of a custom transport.
2019-04-30 13:13:22 +02:00
Felix Lange
ed97517ff4 p2p/discover: bump failure counter only if no nodes were provided (#19362)
This resolves a minor issue where neighbors responses containing less
than 16 nodes would bump the failure counter, removing the node. One
situation where this can happen is a private deployment where the total
number of extant nodes is less than 16.

Issue found by @jsying.
2019-04-08 14:35:11 +03:00
Péter Szilágyi
dac7cbcf21
p2p/enode: use localItemKey for local sequence number (#19131)
* p2p/discover: remove unused function

* p2p/enode: use localItemKey for local sequence number

I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
2019-02-28 13:14:45 +02:00
Felix Lange
cf147c71d5 p2p/discover: remove unused function 2019-02-19 13:29:19 +01:00
Matthew Halpern
f1537b774c p2p/discover: make maximum packet size a constant (#19061) 2019-02-19 12:27:29 +01:00
Felix Lange
a89170cfb2
p2p/discover: improve table addition code (#18974)
This change clears up confusion around the two ways in which nodes
can be added to the table.

When a neighbors packet is received as a reply to findnode, the nodes
contained in the reply are added as 'seen' entries if sufficient space
is available.

When a ping is received and the endpoint verification has taken place,
the remote node is added as a 'verified' entry or moved to the front of
the bucket if present. This also updates the node's IP address and port
if they have changed.
2019-01-31 11:48:54 +01:00
Felix Lange
f0c6f92140
p2p/discover, p2p/enode: rework endpoint proof handling, packet logging (#18963)
This change resolves multiple issues around handling of endpoint proofs.
The proof is now done separately for each IP and completing the proof
requires a matching ping hash.

Also remove waitping because it's equivalent to sleep. waitping was
slightly more efficient, but that may cause issues with findnode if
packets are reordered and the remote end sees findnode before pong.

Logging of received packets was hitherto done after handling the packet,
which meant that sent replies were logged before the packet that
generated them. This change splits up packet handling into 'preverify'
and 'handle'. The error from 'preverify' is logged, but 'handle' happens
after the message is logged. This fixes the order. Packet logs now
contain the node ID.
2019-01-29 17:39:20 +01:00
Martin Holst Swende
493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
Felix Lange
6f607de5d5
p2p, p2p/discover: add signed ENR generation (#17753)
This PR adds enode.LocalNode and integrates it into the p2p
subsystem. This new object is the keeper of the local node
record. For now, a new version of the record is produced every
time the client restarts. We'll make it smarter to avoid that in
the future.

There are a couple of other changes in this commit: discovery now
waits for all of its goroutines at shutdown and the p2p server
now closes the node database after discovery has shut down. This
fixes a leveldb crash in tests. p2p server startup is faster
because it doesn't need to wait for the external IP query
anymore.
2018-10-12 11:47:24 +02:00
Felix Lange
30cd5c1854
all: new p2p node representation (#17643)
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.

Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.

The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.

* p2p/discover: port to p2p/enode

This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:

  - Table.Lookup is not available anymore. It used to take a public key
    as argument because v4 protocol requires one. Its replacement is
    LookupRandom.
  - Table.Resolve takes *enode.Node instead of NodeID. This is also for
    v4 protocol compatibility because nodes cannot be looked up by ID
    alone.
  - Types Node and NodeID are gone. Further commits in the series will be
    fixes all over the the codebase to deal with those removals.

* p2p: port to p2p/enode and discovery changes

This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.

New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.

* p2p/simulations, p2p/testing: port to p2p/enode

No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:

 - testing.ProtocolSession tracks complete nodes, not just their IDs.
 - adapters.NodeConfig has a new method to create a complete node.

These changes were needed to make swarm tests work.

Note that the NodeID change makes the code incompatible with old
simulation snapshots.

* whisper/whisperv5, whisper/whisperv6: port to p2p/enode

This port was easy because whisper uses []byte for node IDs and
URL strings in the API.

* eth: port to p2p/enode

Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.

* les: port to p2p/enode

Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.

* node: port to p2p/enode

This change simply replaces discover.Node and discover.NodeID with their
new equivalents.

* swarm/network: port to p2p/enode

Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).

There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.

Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
2018-09-25 00:59:00 +02:00
Emil
86a03f97d3 all: simplify s[:] to s where s is a slice (#17673) 2018-09-14 22:07:13 +02:00
Oleg Kovalov
cf05ef9106 p2p, swarm, trie: avoid copying slices in loops (#17265) 2018-08-07 13:56:40 +03:00
Felix Lange
c73b654fd1 p2p/discover: move bond logic from table to transport (#17048)
* p2p/discover: move bond logic from table to transport

This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.

It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.

The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.

The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.

* p2p/discover: reduce findnode failure counter on success

* p2p/discover: remove unused parameter of loadSeedNodes

* p2p/discover: improve ping-back check and comments

* p2p/discover: add neighbors reply nodes always, not just during init
2018-07-03 16:24:12 +03:00
Elad
1836366ac1 all: library changes for swarm-network-rewrite (#16898)
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:

- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
2018-06-14 11:21:17 +02:00
kiel barry
864e80a48f p2p: fix some golint warnings (#16577) 2018-05-08 13:08:43 +02:00
thomasmodeneis
ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
Felix Lange
aeedec4078 p2p/discover: s/lastPong/bondTime/, update TestUDP_findnode
I forgot to change the check in udp.go when I changed Table.bond to be
based on lastPong instead of node presence in db. Rename lastPong to
bondTime and add hasBond so it's clearer what this DB key is used for
now.
2018-02-16 21:29:20 +01:00
Péter Szilágyi
32301a4d6b
p2p/discover: validate bond against lastpong, not db presence 2018-02-16 17:05:08 +02:00
Péter Szilágyi
20797348ca
p2p/discover: fix out-of-bounds issue 2018-02-13 20:59:43 +02:00
Felix Lange
9123eceb0f p2p, p2p/discover: misc connectivity improvements (#16069)
* p2p: add DialRatio for configuration of inbound vs. dialed connections

* p2p: add connection flags to PeerInfo

* p2p/netutil: add SameNet, DistinctNetSet

* p2p/discover: improve revalidation and seeding

This changes node revalidation to be periodic instead of on-demand. This
should prevent issues where dead nodes get stuck in closer buckets
because no other node will ever come along to replace them.

Every 5 seconds (on average), the last node in a random bucket is
checked and moved to the front of the bucket if it is still responding.
If revalidation fails, the last node is replaced by an entry of the
'replacement list' containing recently-seen nodes.

Most close buckets are removed because it's very unlikely we'll ever
encounter a node that would fall into any of those buckets.

Table seeding is also improved: we now require a few minutes of table
membership before considering a node as a potential seed node. This
should make it less likely to store short-lived nodes as potential
seeds.

* p2p/discover: fix nits in UDP transport

We would skip sending neighbors replies if there were fewer than
maxNeighbors results and CheckRelayIP returned an error for the last
one. While here, also resolve a TODO about pong reply tokens.
2018-02-12 14:36:09 +02:00
Felföldi Zsolt
92580d69d3 p2p, p2p/discover, p2p/discv5: implement UDP port sharing (#15200)
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.

DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
2018-01-22 13:38:34 +01:00
ferhat elmas
afa3c72c40 p2p/discover: fix leaked goroutine in data expiration 2017-12-18 09:16:54 +01:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
Lewis Marshall
9feec51e2d p2p: add network simulation framework (#14982)
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
2017-09-25 10:08:07 +02:00
Ali Hajimirza
33b158e0ed discover: Changed Logging from Debug to Info (#14485) 2017-05-20 13:10:59 +02:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Felix Lange
96ae35e2ac p2p, p2p/discover, p2p/nat: rework logging using context keys 2017-02-28 10:20:29 +01:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Péter Szilágyi
189dee26c6
p2p: remove trailing newlines from log messages 2017-02-23 12:00:04 +02:00
Felix Lange
9b0af51386 crypto: add btcec fallback for sign/recover without cgo (#3680)
* vendor: add github.com/btcsuite/btcd/btcec

* crypto: add btcec fallback for sign/recover without cgo

This commit adds a non-cgo fallback implementation of secp256k1
operations.

* crypto, core/vm: remove wrappers for sha256, ripemd160
2017-02-18 09:24:12 +01:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Péter Szilágyi
2573094df2
p2p/discover, p2p/discv5: use flexible comparison for IPs 2016-12-14 18:40:49 +02:00
Felix Lange
a47341cf96 p2p, p2p/discover, p2p/discv5: add IP network restriction feature
The p2p packages can now be configured to restrict all communication to
a certain subset of IP networks. This feature is meant to be used for
private networks.
2016-11-22 22:21:18 +01:00
Felix Lange
a98d1d67d6 p2p/discover, p2p/discv5: prevent relay of invalid IPs and low ports
The discovery DHT contains a number of hosts with LAN and loopback IPs.
These get relayed because some implementations do not perform any checks
on the IP.

go-ethereum already prevented relay in most cases because it verifies
that the host actually exists before adding it to the local table. But
this verification causes other issues. We have received several reports
where people's VPSs got shut down by hosting providers because sending
packets to random LAN hosts is indistinguishable from a slow port scan.

The new check prevents sending random packets to LAN by discarding LAN
IPs sent by Internet hosts (and loopback IPs from LAN and Internet
hosts). The new check also blacklists almost all currently registered
special-purpose networks assigned by IANA to avoid inciting random
responses from services in the LAN.

As another precaution against abuse of the DHT, ports below 1024 are now
considered invalid.
2016-11-22 22:21:18 +01:00
Felix Lange
ba2884f343 p2p/discover, p2p/discv5: use netutil.IsTemporaryError 2016-11-22 22:21:15 +01:00
Péter Szilágyi
178da7c6a9
mobile: initial wrappers for mobile support 2016-11-14 17:56:58 +02:00
villesundell
c7442ef0d2 ethdb, p2p/discover: replace "alloted" with "allotted" (#2785) 2016-07-12 11:42:47 +02:00
Felix Lange
8110671960 p2p/discover: prevent bonding self 2016-05-02 18:50:16 +02:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Jeffrey Wilcke
483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Péter Szilágyi
ac954f48bd p2p/discover: emphasize warning, add 10 min cooldown 2016-02-24 12:16:28 +02:00
Péter Szilágyi
b1908f6a16 psp/discovery: NTP sanity check clock drift in case of expirations 2016-02-24 12:14:15 +02:00
Ricardo Catalinas Jiménez
436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Felix Lange
ee1debda53 p2p/discover: EIP-8 changes 2016-02-19 11:14:42 +01:00
Felix Lange
2871781f64 p2p/discover: fix Windows-specific issue for larger-than-buffer packets
On Windows, UDPConn.ReadFrom returns an error for packets larger
than the receive buffer. The error is not marked temporary, causing
our loop to exit when the first oversized packet arrived. The fix
is to treat this particular error as temporary.

Fixes: #1579, #2087
Updates: #2082
2016-01-22 23:44:25 +01:00
Felix Lange
b3f1f4c673 p2p/discover: attempt to deflake TestUDP_responseTimeouts
The test expected the timeout to fire after a matcher for the response
was added, but the timeout is random and fired sooner sometimes.
2015-12-17 23:44:56 +01:00
Felix Lange
04c6369a09 p2p, p2p/discover: track bootstrap state in p2p/discover
This change simplifies the dial scheduling logic because it
no longer needs to track whether the discovery table has been
bootstrapped.
2015-12-17 23:38:54 +01:00
Felix Lange
d1f507b7f1 p2p/discover: support incomplete node URLs, add Resolve 2015-12-17 23:38:54 +01:00
Gustav Simonsson
c8ad64f33c crypto, crypto/ecies, crypto/secp256k1: libsecp256k1 scalar mult
thanks to Felix Lange (fjl) for help with design & impl
2015-11-30 13:43:32 +01:00
Péter Szilágyi
9e1d9bff3b node: customizable protocol and service stacks 2015-11-27 11:06:12 +02:00
Felix Lange
32dda97602 p2p/discover: ignore packet version numbers
The strict matching can get in the way of protocol upgrades.
2015-09-30 16:23:03 +02:00
Felix Lange
631bf36102 p2p/discover: remove unused lastLookup field 2015-09-30 16:23:03 +02:00
Felix Lange
b4374436f3 p2p/discover: fix race involving the seed node iterator
nodeDB.querySeeds was not safe for concurrent use but could be called
concurrenty on multiple goroutines in the following case:

- the table was empty
- a timed refresh started
- a lookup was started and initiated refresh

These conditions are unlikely to coincide during normal use, but are
much more likely to occur all at once when the user's machine just woke
from sleep. The root cause of the issue is that querySeeds reused the
same leveldb iterator until it was exhausted.

This commit moves the refresh scheduling logic into its own goroutine
(so only one refresh is ever active) and changes querySeeds to not use
a persistent iterator. The seed node selection is now more random and
ignores nodes that have not been contacted in the last 5 days.
2015-09-30 16:23:03 +02:00
Jeffrey Wilcke
e2d44814a5 Merge pull request #1694 from obscuren/hide-fdtrack
fdtrack: hide message
2015-08-19 13:50:54 -07:00
Jeffrey Wilcke
269c5c7107 Revert "fdtrack: temporary hack for tracking file descriptor usage"
This reverts commit 5c949d3b3b.
2015-08-19 21:46:01 +02:00
Felix Lange
dd54fef898 p2p/discover: don't attempt to replace nodes that are being replaced
PR #1621 changed Table locking so the mutex is not held while a
contested node is being pinged. If multiple nodes ping the local node
during this time window, multiple ping packets will be sent to the
contested node. The changes in this commit prevent multiple packets by
tracking whether the node is being replaced.
2015-08-19 14:57:16 +02:00
Felix Lange
7d5ff770e2 p2p/discover: continue reading after temporary errors
Might solve #1579
2015-08-19 14:38:55 +02:00
Felix Lange
590c99a98f p2p/discover: fix UDP reply packet timeout handling
If the timeout fired (even just nanoseconds) before the deadline of the
next pending reply, the timer was not rescheduled. The timer would've
been rescheduled anyway once the next packet was sent, but there were
cases where no next packet could ever be sent due to the locking issue
fixed in the previous commit.

As timing-related bugs go, this issue had been present for a long time
and I could never reproduce it. The test added in this commit did
reproduce the issue on about one out of 15 runs.
2015-08-11 11:42:17 +02:00
Felix Lange
01ed3fa1a9 p2p/discover: unlock the table during ping replacement
Table.mutex was being held while waiting for a reply packet, which
effectively made many parts of the whole stack block on that packet,
including the net_peerCount RPC call.
2015-08-11 11:42:17 +02:00
Felix Lange
b23b4dbd79 p2p/discover: close Table during testing
Not closing the table used to be fine, but now the table has a database.
2015-08-06 12:27:59 +02:00
Felix Lange
5c949d3b3b fdtrack: temporary hack for tracking file descriptor usage
Package fdtrack logs statistics about open file descriptors.
This should help identify the source of #1549.
2015-08-04 03:10:27 +02:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Felix Lange
a8e4cb6dfe p2p/discover: use separate rand.Source instances in tests
rand.Source isn't safe for concurrent use.
2015-06-10 15:18:01 +02:00
Felix Lange
261a8077c4 p2p/discover: deflake TestUDP_successfulPing 2015-06-10 13:08:21 +02:00
Péter Szilágyi
612f01400f p2p/discover: bond with seed nodes too (runs only if findnode failed) 2015-05-26 23:30:41 +02:00
Péter Szilágyi
3630432dfb p2p/discovery: fix a cornercase loop if no seeds or bootnodes are known 2015-05-26 23:30:40 +02:00
Péter Szilágyi
f539ed1e66 p2p/discover: force refresh if the table is empty 2015-05-26 23:30:40 +02:00
Péter Szilágyi
5076170f34 p2p/discover: permit temporary bond failures for previously known nodes 2015-05-26 23:30:40 +02:00
Péter Szilágyi
6078aa08eb p2p/discover: watch find failures, evacuate on too many, rebond if failed 2015-05-26 23:30:40 +02:00
Péter Szilágyi
64174f196f p2p/discover: add support for counting findnode failures 2015-05-26 23:30:40 +02:00
Felix Lange
9f38ef5d97 p2p/discover: add ReadRandomNodes 2015-05-25 01:17:14 +02:00
Péter Szilágyi
cbd3ae6906 p2p/discover: fix #838, evacuate self entries from the node db 2015-05-21 19:41:46 +03:00
Péter Szilágyi
af24c271c7 p2p/discover: fix database presistency test folder 2015-05-21 19:28:10 +03:00
Felix Lange
d2f119cf9b p2p/discover: limit open files for node database 2015-05-14 15:01:13 +02:00
Felix Lange
7fa2607bd1 p2p/discover: bump maxBondingPingPongs to 16
This should increase the speed a bit because all findnode
results (up to 16) can be verified at the same time.
2015-05-14 14:53:29 +02:00
Felix Lange
251846d65a p2p/discover: fix out-of-bounds slicing for chunked neighbors packets
The code assumed that Table.closest always returns at least 13 nodes.
This is not true for small tables (e.g. during bootstrap).
2015-05-13 21:49:04 +02:00
subtly
8eef2b765a fix test. 2015-05-13 20:15:01 +02:00
subtly
a32693770c Manual send of multiple neighbours packets. Test receiving multiple neighbours packets. 2015-05-13 20:03:17 +02:00
subtly
7473c93668 UDP Interop. Limit datagrams to 1280bytes.
We don't have a UDP which specifies any messages that will be 4KB. Aside from being implemented for months and a necessity for encryption and piggy-backing packets, 1280bytes is ideal, and, means this TODO can be completed!

Why 1280 bytes?
* It's less than the default MTU for most WAN/LAN networks. That means fewer fragmented datagrams (esp on well-connected networks).
* Fragmented datagrams and dropped packets suck and add latency while OS waits for a dropped fragment to never arrive (blocking readLoop())
* Most of our packets are < 1280 bytes.
* 1280 bytes is minimum datagram size and MTU for IPv6 -- on IPv6, a datagram < 1280bytes will *never* be fragmented.

UDP datagrams are dropped. A lot! And fragmented datagrams are worse. If a datagram has a 30% chance of being dropped, then a fragmented datagram has a 60% chance of being dropped. More importantly, we have signed packets and can't do anything with a packet unless we receive the entire datagram because the signature can't be verified. The same is true when we have encrypted packets.

So the solution here to picking an ideal buffer size for receiving datagrams is a number under 1400bytes. And the lower-bound value for IPv6 of 1280 bytes make's it a non-decision. On IPv4 most ISPs and 3g/4g/let networks have an MTU just over 1400 -- and *never* over 1500. Never -- that means packets over 1500 (in reality: ~1450) bytes are fragmented. And probably dropped a lot.

Just to prove the point, here are pings sending non-fragmented packets over wifi/ISP, and a second set of pings via cell-phone tethering. It's important to note that, if *any* router between my system and the EC2 node has a lower MTU, the message would not go through:

On wifi w/normal ISP:
localhost:Debug $ ping -D -s 1450 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1450 data bytes
1458 bytes from 52.6.250.242: icmp_seq=0 ttl=42 time=104.831 ms
1458 bytes from 52.6.250.242: icmp_seq=1 ttl=42 time=119.004 ms
^C
--- 52.6.250.242 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 104.831/111.918/119.004/7.087 ms
localhost:Debug $ ping -D -s 1480 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1480 data bytes
ping: sendto: Message too long
ping: sendto: Message too long
Request timeout for icmp_seq 0
ping: sendto: Message too long
Request timeout for icmp_seq 1


Tethering to O2:
localhost:Debug $ ping -D -s 1480 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1480 data bytes
ping: sendto: Message too long
ping: sendto: Message too long
Request timeout for icmp_seq 0
^C
--- 52.6.250.242 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
localhost:Debug $ ping -D -s 1450 52.6.250.242
PING 52.6.250.242 (52.6.250.242): 1450 data bytes
1458 bytes from 52.6.250.242: icmp_seq=0 ttl=42 time=107.844 ms
1458 bytes from 52.6.250.242: icmp_seq=1 ttl=42 time=105.127 ms
1458 bytes from 52.6.250.242: icmp_seq=2 ttl=42 time=120.483 ms
1458 bytes from 52.6.250.242: icmp_seq=3 ttl=42 time=102.136 ms
2015-05-13 19:03:00 +02:00
Bas van Kervel
95773b9673 removed redundant newlines in import block 2015-05-12 15:20:53 +02:00
Bas van Kervel
b79dd188d9 replaced several path.* with filepath.* which is platform independent 2015-05-12 14:24:11 +02:00
Felix Lange
bcfd788661 p2p/discover: bump packet timeouts to 500ms 2015-05-06 22:59:00 +02:00
Felix Lange
2adcc31bb4 p2p/discover: new distance metric based on sha3(id)
The previous metric was pubkey1^pubkey2, as specified in the Kademlia
paper. We missed that EC public keys are not uniformly distributed.
Using the hash of the public keys addresses that. It also makes it
a bit harder to generate node IDs that are close to a particular node.
2015-05-06 16:10:41 +02:00
Felix Lange
72ab6d3255 p2p/discover: track sha3(ID) in Node 2015-04-30 15:02:23 +02:00
Felix Lange
b34a8ef624 p2p, p2p/discover: protocol version 4 2015-04-30 14:57:34 +02:00
Felix Lange
fc747ef4a6 p2p/discover: new endpoint format
This commit changes the discovery protocol to use the new "v4" endpoint
format, which allows for separate UDP and TCP ports and makes it
possible to discover the UDP address after NAT.
2015-04-30 14:57:33 +02:00
Péter Szilágyi
b569550a39 p2p/discover: fix api issues caused by leveldb update 2015-04-28 13:57:57 +03:00
Péter Szilágyi
4992765032 p2p/discover: fix goroutine leak due to blocking on sync.Once 2015-04-28 10:28:04 +03:00
Péter Szilágyi
437cf4b3ac p2p/discover: add node expirer and related tests 2015-04-27 17:38:28 +03:00
Péter Szilágyi
a136e2bb22 p2p/discover: parametrize nodedb version, add persistency tests 2015-04-27 15:28:17 +03:00
Péter Szilágyi
75fd738dea p2p/discover: drop a superfluous warning 2015-04-27 15:06:31 +03:00
Péter Szilágyi
706da56f75 p2p/discover: wrap the pinger to update the node db too 2015-04-27 14:56:42 +03:00
Péter Szilágyi
85b4b44235 p2p/discover: use iterator based seeding, drop old protocol test 2015-04-27 14:45:35 +03:00
Péter Szilágyi
8de8f61d36 p2p/discover: write the basic tests, catch RLP bug 2015-04-27 12:33:06 +03:00
Péter Szilágyi
0201c04b95 p2p/discovery: fix issues raised in the nodeDb PR 2015-04-27 10:19:16 +03:00
Péter Szilágyi
8646365b42 cmd/bootnode, eth, p2p, p2p/discover: use a fancier db design 2015-04-24 18:04:41 +03:00