Commit Graph

566 Commits

Author SHA1 Message Date
ImanSharaf
ca8e2f1ecf
p2p/simulations: escape mockerType value from request (#24822)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-05-05 19:44:36 +02:00
henopied
63972e7548
p2p: fix type of DiscSubprotocolError (#24747)
It was 'int' accidentally, should be DiscReason instead.
2022-04-25 11:05:47 +02:00
Eng Zer Jun
8d066f1f42
all: use T.TempDir to create temporary test directories (#24633)
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.

	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}

is also tedious, but t.TempDir handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-08 15:44:55 +02:00
Felix Lange
870b4505a0
p2p: define DiscReason as uint8 (#24507)
All other implementations store disconnect reasons as a single byte,
so go-ethereum should do it too.
2022-03-07 18:25:45 +01:00
Felix Lange
d6f49bf764
core/types: faster RLP encoding of Header, StateAcccount, ReceiptForStorage (#24420)
This change makes use of the new code generator rlp/rlpgen to improve the
performance of RLP encoding for Header and StateAccount. It also speeds up
encoding of ReceiptForStorage using the new rlp.EncoderBuffer API.

The change is much less transparent than I wanted it to be, because Header and
StateAccount now have an EncodeRLP method defined with pointer receiver. It
used to be possible to encode non-pointer values of these types, but the new
method prevents that and attempting to encode unadressable values (even if
part of another value) will return an error. The error can be surprising and may
pop up in places that previously didn't expect any errors.

To make things work, I also needed to update all code paths (mostly in unit tests)
that lead to encoding of non-pointer values, and pass a pointer instead.

Benchmark results:

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           328ns ± 0%     237ns ± 1%   -27.63%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8           353ns ± 0%     247ns ± 1%   -30.06%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     237ns ± 0%     123ns ± 0%   -47.86%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8            297ns ± 0%     301ns ± 1%    +1.39%  (p=0.000 n=8+8)

    name                             old speed      new speed      delta
    EncodeRLP/legacy-header-8        1.66GB/s ± 0%  2.29GB/s ± 1%   +38.19%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8        1.55GB/s ± 0%  2.22GB/s ± 1%   +42.99%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8  38.0MB/s ± 0%  64.8MB/s ± 0%   +70.48%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8          910MB/s ± 0%   897MB/s ± 1%    -1.37%  (p=0.000 n=8+8)

    name                             old alloc/op   new alloc/op   delta
    EncodeRLP/legacy-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/london-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/receipt-for-storage-8     64.0B ± 0%      0.0B       -100.00%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8             320B ± 0%      320B ± 0%      ~     (all equal)
2022-02-18 08:10:26 +01:00
zhiqiangxu
6c3513c077
p2p: reduce the scope of variable dialPubkey (#24385)
dialPubkey isn't used anywhere else after dialDest.Load, so it should be safe to
restrict its scope to the if clause.
2022-02-14 12:05:48 +01:00
Felföldi Zsolt
fc01a7ce8e
les/vflux/client, p2p/nodestate: fix data races (#24058)
Fixes #23848
2021-12-14 11:34:50 +01:00
Arvid Hast
acb0f7a67b
p2p/simulations: improve README language (#24051) 2021-12-06 12:25:42 +01:00
Taeik Lim
85064ed09b
all: fix 'the the' in comments (#24036) 2021-12-02 15:42:09 +01:00
Evolution404
8fbe0b9b68
p2p/enr: reduce allocation in Record.encode (#24034) 2021-12-02 10:55:01 +01:00
Péter Szilágyi
c10a0a62c3
eth: request id dispatcher and direct req/reply APIs (#23576)
* eth: request ID based message dispatcher

* eth: fix dispatcher cancellation, rework fetchers idleness tracker

* eth/downloader: drop peers who refuse to serve advertised chains
2021-11-26 13:26:03 +02:00
Serhat Şevki Dinçer
d15e423562
p2p/enode: store local port number as uint16 (#23926) 2021-11-23 15:14:08 +01:00
Ikko Ashimine
fb7da82dde
p2p: fix typo in v4wire.go
Neigbors -> Neighbors
2021-11-10 22:18:12 +09:00
Martin Holst Swende
31870a59ff
eth/filters, p2p/simulations: fix benchmarks (#23806)
Some benchmarks in eth/filters were not good: they weren't reproducible, relying on geth chaindata to be present.

Another one was rejected because the receipt was lacking a backing transcation.

The p2p simulation benchmark had a lot of the warnings below, due to the framework calling both
Stop() and Close(). Apparently, the simulated adapter is the only implementation which has a Close(),
and there is no need to call both Stop and Close on it.
2021-10-29 14:37:00 +02:00
Guillaume Ballet
0183256e7f
all: fix warning flagging the use of DeepEqual on error (#23624)
* core: fix warning flagging the use of DeepEqual on error

* apply the same change everywhere possible

* revert change that was committed by mistake

* fix build error

* Update config.go

* revert changes to ConfigCompatError

* review feedback

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-10-19 17:06:47 +02:00
Péter Szilágyi
6ef3a16869
p2p/enode: use unix timestamp as base ENR sequence number (#19903)
This PR ensures that wiping all data associated with a node (apart from its nodekey)
will not generate already used sequence number for the ENRs, since all remote nodes
would reject them until they out-number the previously published largest one.

The big complication with this scheme is that every local update to the ENR can
potentially bump the sequence number by one. In order to ensure that local updates
do not outrun the clock, the sequence number is a millisecond-precision timestamp,
and updates are throttled to occur at most once per millisecond.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-09-07 12:36:48 +02:00
Felix Lange
8a134014b4
all: add go:build lines (#23468)
Generated by go1.17 fmt ./...
2021-08-25 18:46:29 +02:00
Marius van der Wijden
8dbf261fd9
p2p, p2p/enode: fix data races (#23434)
In p2p/dial.go, conn.flags was accessed without using sync/atomic.
This race is fixed by removing the access.

In p2p/enode/iter_test.go, a similar race is resolved by writing the field atomically.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-08-24 12:22:56 +02:00
baptiste-b-pegasys
860184d542
p2p: remove term "whitelist" (#23295)
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-07-29 17:50:18 +02:00
Evolution404
8f11d279d2
p2p/simulations: fix unlikely crash in probabilistic connect (#23200)
When the nodeCount is less than 10, it will panic with the out of bound error.
How about we just skip this round, when rand1 and rand2 are equal?
2021-07-29 16:03:50 +02:00
Evolution404
5afc82de6e
p2p: fix array out of bounds issue (#23165) 2021-07-06 09:33:51 +02:00
Ahyun
4fcc93d922
p2p/server: fix method name in comment (#23123) 2021-06-29 12:14:47 +03:00
ForLina
dde6f1e92d
p2p/enode: fix method doc (#23115)
This is an obvious spelling error

Co-authored-by: liuyaxiong <liuyaxiong@inspur.com>
2021-06-28 10:48:17 +03:00
Felix Lange
427175153c
p2p/msgrate: return capacity as integer, clamp to max uint32 (#22943)
* p2p/msgrate: return capacity as integer

* eth/protocols/snap: remove conversions

* p2p/msgrate: add overflow test

* p2p/msgrate: make the capacity overflow test actually overflow

* p2p/msgrate: clamp capacity to max int32

* p2p/msgrate: fix min/max confusion
2021-05-27 19:43:55 +03:00
Felix Lange
7194c847b6
p2p/rlpx: reduce allocation and syscalls (#22899)
This change significantly improves the performance of RLPx message reads
and writes. In the previous implementation, reading and writing of
message frames performed multiple reads and writes on the underlying
network connection, and allocated a new []byte buffer for every read.

In the new implementation, reads and writes re-use buffers, and perform
much fewer system calls on the underlying connection. This doubles the
theoretically achievable throughput on a single connection, as shown by
the benchmark result:

    name             old speed      new speed       delta
    Throughput-8     70.3MB/s ± 0%  155.4MB/s ± 0%  +121.11%  (p=0.000 n=9+8)

The change also removes support for the legacy, pre-EIP-8 handshake encoding.
As of May 2021, no actively maintained client sends this format.
2021-05-27 10:19:13 +02:00
meowsbits
750115ff39
p2p/nat: skip TestUPNP in non-CI environments if discover fails (#22877)
Fixes #21476
2021-05-25 22:37:30 +02:00
Felix Lange
836c647bdd
eth: unregister peer only when handler exits (#22908)
This removes the error log message that says 

    Ethereum peer removal failed ... err="peer not registered"

The error happened because removePeer was called multiple
times: once to disconnect the peer, and another time when the
handler exited. With this change, removePeer now has the sole
purpose of disconnecting the peer. Unregistering happens exactly
once, when the handler exits.
2021-05-25 22:20:36 +02:00
Felix Lange
16bc57438b
p2p/dnsdisc: fix crash when iterator closed before first call to Next (#22906) 2021-05-20 09:24:41 +02:00
Péter Szilágyi
3e795881ea
eth, p2p/msgrate: move peer QoS tracking to its own package and use it for snap (#22876)
This change extracts the peer QoS tracking logic from eth/downloader, moving
it into the new package p2p/msgrate. The job of msgrate.Tracker is determining
suitable timeout values and request sizes per peer.

The snap sync scheduler now uses msgrate.Tracker instead of the hard-coded 15s
timeout. This should make the sync work better on network links with high latency.
2021-05-19 14:09:03 +02:00
Felix Lange
3e6f46caec
p2p/discover/v4wire: use optional RLP field for EIP-868 seq (#22842)
This changes the definitions of Ping and Pong, adding an optional field
for the sequence number. This field was previously encoded/decoded using
the "tail" struct tag, but using "optional" is much nicer.
2021-05-18 11:48:41 +02:00
Felix Lange
b8040a430e
cmd/utils: use eth DNS tree for snap discovery (#22808)
This removes auto-configuration of the snap.*.ethdisco.net DNS discovery tree.
Since measurements have shown that > 75% of nodes in all.*.ethdisco.net support
snap, we have decided to retire the dedicated index for snap and just use the eth
tree instead.

The dial iterators of eth and snap now use the same DNS tree in the default configuration,
so both iterators should use the same DNS discovery client instance. This ensures that
the record cache and rate limit are shared. Records will not be requested multiple times.

While testing the change, I noticed that duplicate DNS requests do happen even
when the client instance is shared. This is because the two iterators request the tree
root, link tree root, and first levels of the tree in lockstep. To avoid this problem, the
change also adds a singleflight.Group instance in the client. When one iterator
attempts to resolve an entry which is already being resolved, the singleflight object
waits for the existing resolve call to finish and returns the entry to both places.
2021-05-04 11:29:32 +02:00
Péter Szilágyi
ff3535e8e0
p2p/tracker: only reschedule wake if previous didn't run 2021-04-27 21:47:59 +03:00
Martin Holst Swende
45fca44c24
p2p/tracker: properly clean up fulfilled requests 2021-04-27 18:09:34 +02:00
Nishant Das
34f3c9539b
p2p/discover: improve discv5 handling of IPv4-in-IPv6 addresses (#22703)
When receiving PING from an IPv4 address over IPv6, the implementation sent
back a IPv4-in-IPv6 address. This change makes it reflect the IPv4 address.
2021-04-23 18:18:10 +02:00
Péter Szilágyi
1fb9a6dd32
eth/protocols, prp/tracker: add support for req/rep rtt tracking (#22608)
* eth/protocols, prp/tracker: add support for req/rep rtt tracking

* p2p/tracker: sanity cap the number of pending requests

* pap/tracker: linter <3

* p2p/tracker: disable entire tracker if no metrics are enabled
2021-04-22 11:42:46 +03:00
Felföldi Zsolt
2d89fe0883
les: move client pool to les/vflux/server (#22495)
* les: move client pool to les/vflux/server

* les/vflux/server: un-expose NodeBalance, remove unused fn, fix bugs

* tests/fuzzers/vflux: add ClientPool fuzzer

* les/vflux/server: fixed balance tests

* les: rebase fix

* les/vflux/server: fixed more bugs

* les/vflux/server: unexported NodeStateMachine fields and flags

* les/vflux/server: unexport all internal components and functions

* les/vflux/server: fixed priorityPool test

* les/vflux/server: polish balance

* les/vflux/server: fixed mutex locking error

* les/vflux/server: priorityPool bug fixed

* common/prque: make Prque wrap-around priority handling optional

* les/vflux/server: rename funcs, small optimizations

* les/vflux/server: fixed timeUntil

* les/vflux/server: separated balance.posValue and negValue

* les/vflux/server: polish setup

* les/vflux/server: enforce capacity curve monotonicity

* les/vflux/server: simplified requestCapacity

* les/vflux/server: requestCapacity with target range, no iterations in SetCapacity

* les/vflux/server: minor changes

* les/vflux/server: moved default factors to balanceTracker

* les/vflux/server: set inactiveFlag in priorityPool

* les/vflux/server: moved related metrics to vfs package

* les/vflux/client: make priorityPool temp state logic cleaner

* les/vflux/server: changed log.Crit to log.Error

* add vflux fuzzer to oss-fuzz

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-04-06 20:42:50 +02:00
Péter Szilágyi
6d7ff6acea
eth/protocols, metrics, p2p: add handler performance metrics 2021-03-26 14:00:06 +02:00
Felix Lange
ab8fd4d005
p2p/dnsdisc: rate limit resolving before checking cache (#22566)
This makes the rate limit apply regardless of whether the node is
already cached.
2021-03-24 14:37:20 +02:00
Chen Quan
15e6c27f8b
p2p: fix minor typo and remove fd parameter in checkInboundConn (#22547) 2021-03-24 13:18:29 +01:00
Martin Holst Swende
5bc0343ed3
p2p/dnsdisc: fix flaw in dns size calculation (#22533)
This fixes the calculation of the tree branch factor. With the new
formula, we now creat at most 13 children instead of 30, ensuring
the TXT record size will be below 370 bytes.
2021-03-19 13:20:27 +01:00
Felix Lange
aae7660410
p2p/enr: fix decoding of incomplete lists (#22484)
Given a list of less than two elements DecodeRLP returned rlp.EOL,
leading to issues in outer decoders.
2021-03-11 15:09:25 +01:00
Felföldi Zsolt
d96870428f
les: UDP pre-negotiation of available server capacity (#22183)
This PR implements the first one of the "lespay" UDP queries which
is already useful in itself: the capacity query. The server pool is making
use of this query by doing a cheap UDP query to determine whether it is
worth starting the more expensive TCP connection process.
2021-03-01 10:24:20 +01:00
Felix Lange
d36276d85e
p2p/dnsdisc: fix hot-spin when all trees are empty (#22313)
In the random sync algorithm used by the DNS node iterator, we first pick a random
tree and then perform one sync action on that tree. This happens in a loop until any
node is found. If no trees contain any nodes, the iterator will enter a hot loop spinning
at 100% CPU.

The fix is complicated. The iterator now checks if a meaningful sync action can
be performed on any tree. If there is nothing to do, it waits for the next root record
recheck time to arrive and then tries again.

Fixes #22306
2021-02-19 09:54:46 +01:00
Martin Holst Swende
6ec1561044
eth: implement eth66 (#22241)
* eth/protocols/eth: split up the eth protocol handlers

* eth/protocols/eth: define eth-66 protocol messages

* eth/protocols/eth: poc implement getblockheaders on eth/66

* eth/protocols/eth: implement remaining eth-66 handlers

* eth/protocols: define handler map for eth 66

* eth/downloader: use protocol constants from eth package

* eth/protocols/eth: add ETH66 capability

* eth/downloader: tests for eth66

* eth/downloader: fix error in tests

* eth/protocols/eth: use eth66 for outgoing requests

* eth/protocols/eth: remove unused error type

* eth/protocols/eth: define protocol length

* eth/protocols/eth: fix pooled tx over eth66

* protocols/eth/handlers: revert behavioural change which caused tests to fail

* eth/downloader: fix failing test

* eth/protocols/eth: add testcases + fix flaw with header requests

* eth/protocols: change comments

* eth/protocols/eth: review fixes + fixed flaw in RequestOneHeader

* eth/protocols: documentation

* eth/protocols/eth: review concerns about types
2021-02-18 18:54:29 +02:00
Martin Holst Swende
7ed860d4f1
eth: don't wait for snap registration if we're not running snap (#22272)
Prevents a situation where we (not running snap) connects with a peer running snap, and get stalled waiting for snap registration to succeed (which will never happen), which cause a waitgroup wait to halt shutdown
2021-02-05 14:15:22 +01:00
Péter Szilágyi
e3430ac7df
eth: check snap satelliteness, delegate drop to eth (#22235)
* eth: check snap satelliteness, delegate drop to eth

* eth: better handle eth/snap satellite relation, merge reg/unreg paths
2021-02-02 10:44:36 +02:00
Felföldi Zsolt
a72fa88a0d
les: switch to new discv5 (#21940)
This PR enables running the new discv5 protocol in both LES client
and server mode. In client mode it mixes discv5 and dnsdisc iterators
(if both are enabled) and filters incoming ENRs for "les" tag and fork ID.
The old p2p/discv5 package and all references to it are removed.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-01-26 21:41:35 +01:00
Martin Holst Swende
d2779ed7ac
eth, p2p: reserve half peer slots for snap peers during snap sync (#22171)
* eth, p2p: reserve half peer slots for snap peers during snap sync

* eth: less logging

* eth: rework the eth/snap peer reservation logic

* eth: rework the eth/snap peer reservation logic (again)
2021-01-25 20:06:52 +02:00
Guillaume Ballet
c7a6be163f
cmd/utils: don't enumerate USB unless --usb is set (#22130)
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
2021-01-13 11:14:36 +01:00
Felix Lange
817a3fb562
p2p/enode: avoid crashing for invalid IP (#21981)
The database panicked for invalid IPs. This is usually no problem
because all code paths leading to node DB access verify the IP, but it's
dangerous because improper validation can turn this panic into a DoS
vulnerability. The quick fix here is to just turn database accesses
using invalid IP into a noop. This isn't great, but I'm planning to
remove the node DB for discv5 long-term, so it should be fine to have
this quick fix for half a year.

Fixes #21849
2020-12-09 20:21:31 +01:00