Commit Graph

635 Commits

Author SHA1 Message Date
Dimitris Apostolou
8fd43c8013
all: fix typos in comments (#28881) 2024-02-05 22:16:32 +01:00
Chris Ziogas
19af9008f1
p2p: fix accidental termination of portMappingLoop (#28911) 2024-02-05 22:00:46 +01:00
ddl
1335ba5f28
p2p/dnsdisc: use strings.Cut over strings.IndexByte (#28787) 2024-01-12 19:57:47 +01:00
Felix Lange
5b22a472d6
p2p/discover: add liveness check in collectTableNodes (#28686)
* p2p/discover: add liveness check in collectTableNodes

* p2p/discover: fix test

* p2p/discover: rename to appendLiveNodes

* p2p/discover: add dedup logic back

* p2p/discover: simplify

* p2p/discover: fix issue found by test
2023-12-18 10:47:21 +01:00
alex
edc864f9ba
all: fix typos in comments (#28682)
chore(core,eth):fix a couple of typos
2023-12-18 09:35:12 +01:00
Martin Holst Swende
b8d44ed98b
log: remove lazy, remove unused interfaces, unexport methods (#28622)
This change 

- Removes interface `log.Format`, 
- Removes method `log.FormatFunc`, 
- unexports `TerminalHandler.TerminalFormat` formatting methods (renamed to `TerminalHandler.format`)
- removes the notion of `log.Lazy` values


The lazy handler was useful in the old log package, since it
could defer the evaluation of costly attributes until later in the
log pipeline: thus, if the logging was done at 'Trace', we could
skip evaluation if logging only was set to 'Info'.

With the move to slog, this way of deferring evaluation is no longer
needed, since slog introduced 'Enabled': the caller can thus do
the evaluate-or-not decision at the callsite, which is much more
straight-forward than dealing with lazy reflect-based evaluation.

Also, lazy evaluation would not work with 'native' slog, as in, these
two statements would be evaluated differently:

```golang
  log.Info("foo", "my lazy", lazyObj)
  slog.Info("foo", "my lazy", lazyObj)
```
2023-12-05 11:54:44 +01:00
jwasinger
28e7371701
all: replace log15 with slog (#28187)
This PR replaces Geth's logger package (a fork of [log15](https://github.com/inconshreveable/log15)) with an implementation using slog, a logging library included as part of the Go standard library as of Go1.21.

Main changes are as follows:
* removes any log handlers that were unused in the Geth codebase.
* Json, logfmt, and terminal formatters are now slog handlers.
* Verbosity level constants are changed to match slog constant values.  Internal translation is done to make this opaque to the user and backwards compatible with existing `--verbosity` and `--vmodule` options.
* `--log.backtraceat` and `--log.debug` are removed.

The external-facing API is largely the same as the existing Geth logger.  Logger method signatures remain unchanged.

A small semantic difference is that a `Handler` can only be set once per `Logger` and not changed dynamically.  This just means that a new logger must be instantiated every time the handler of the root logger is changed.

----
For users of the `go-ethereum/log` module. If you were using this module for your own project, you will need to change the initialization. If you previously did 
```golang
log.Root().SetHandler(log.LvlFilterHandler(log.LvlInfo, log.StreamHandler(os.Stderr, log.TerminalFormat(true))))
```
You now instead need to do 
```golang
log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelInfo, true)))
```
See more about reasoning here: https://github.com/ethereum/go-ethereum/issues/28558#issuecomment-1820606613
2023-11-29 08:33:50 +01:00
Håvard Anda Estensen
a75a2d6db6
all: replace some cases of strings.SplitN with strings.Cut (#28446) 2023-11-15 14:42:33 +01:00
Martin Holst Swende
233db64cc1
all: make vendored copy of reexec (#28382)
a little copying is better than a little dependency

-- go proverb

We have this dependency on docker, a.k.a moby: a gigantic library, and we only need ~70 LOC,
so here I tried moving it inline instead.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-10-28 00:14:43 +02:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
Felix Lange
7ec60d5f0c
p2p: move ping handling into pingLoop goroutine (#27887)
Moving the response sending there allows tracking all peer goroutines
in the peer WaitGroup.
2023-08-09 16:00:31 +02:00
ucwong
6d2bcb911a
p2p/simulations/examples: use atomic.Int64 (#27861) 2023-08-07 18:30:05 +02:00
ucwong
d04bde0a20
p2p: use atomic types (#27764)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-01 23:20:52 +02:00
Seungbae Yu
60ecf48dd4
cmd/bootnode, p2p: support for alternate mapped ports (#26359)
This changes the port mapping procedure such that, when the requested port is unavailable
an alternative port suggested by the router is used instead.

We now also repeatedly request the external IP from the router in order to catch any IP changes.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-07-14 14:58:27 +02:00
Felix Lange
a426999fc9
p2p/discover: filter bootnodes by netrestrict (#27701)
This prevents an issue where the node would attempt to contact the bootstrap
nodes even if they weren't contained in the netrestrict list.
2023-07-12 12:01:38 +02:00
lightclient
645b0db98e
cmd/utils, p2p: clean up discovery setup (#27518)
This simplifies the code that initializes the discovery a bit, and
adds new flags for enabling/disabling discv4 and discv5 separately.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-07-11 21:21:32 +02:00
lightclient
cbf2579691
p2p, p2p/discover: add dial metrics (#27621)
This PR adds metrics for p2p dialing, which gives us visibility into the quality of the dial 
candidates  returned by our discovery methods.
2023-07-06 16:20:31 +02:00
Exca-DK
5520cd97a1
p2p/discover: swap verification order in discv4 ping handler (#27532)
In all other UDPv4 methods, the deadline is checked first. It seems weird to me that ping is an exception. Deadline comparison is also less resource intensive.

Co-authored-by: Exca-DK <Exca-DK@users.noreply.github.com>
2023-06-22 10:06:14 +02:00
Dan Laine
289c6c3b15
p2p: use slices package for sorting (#27494)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-19 07:48:12 +02:00
Sanghee Choi
acc2a2ac61
node: remove unused error return from Attach (#27450)
node: Delete the unused error from return parameters of Node.Attach() func
2023-06-14 08:24:47 -04:00
Felix Lange
ac86547b01
p2p/discover: add Table configuration and Nodes method (#27387)
* p2p/discover: remove ReadRandomNodes

Even though it's public, this method is not callable by code outside of
package p2p/discover because one can't get a valid instance of Table.

* p2p/discover: add Table.Nodes

* p2p/discover: make Table settings configurable

In unit tests and externally developed cmd/devp2p test runs, it can be
useful to tune the timer intervals used by Table.
2023-05-31 13:37:10 +02:00
ucwong
ae7db289b8
p2p: initialize maps with known size (#27229) 2023-05-10 10:52:26 +02:00
Martin Holst Swende
7d1ebe51b7
p2p/discover: fix lint nit (#27206) 2023-05-02 04:36:11 -04:00
ucwong
a865e28f28
p2p/discover : typo (#27193) 2023-04-28 21:34:08 +03:00
Felix Lange
47cdea5ac5
p2p/discover: concurrent TALKREQ handling (#27112)
This changes TALKREQ message processing to run the handler on separate goroutine,
instead of running on the main discv5 dispatcher goroutine. It's better this way because
it allows the handler to perform blocking actions.

I'm also adding a new method TalkRequestToID here. The method allows implementing
a request flow where one node A sends TALKREQ to another node B, and node B later
sends a TALKREQ back. With TalkRequestToID, node B does not need the ENR of A to
send its request.
2023-04-28 11:03:43 +02:00
Exca-DK
f8f95346f9
p2p/discover: add traffic metrics (#27008)
Co-authored-by: Exca-DK <dev@DESKTOP-RI45P4J.localdomain>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2023-04-25 12:12:34 +02:00
joohhnnn
b1972627d9
p2p: access embedded fields of Server directly (#27078) 2023-04-18 11:57:08 +02:00
lightclient
2c5798464e
common: delete MakeName (#27023)
common,p2p: remove unused function MakeName
2023-04-03 03:51:31 -04:00
Delweng
117530b0e6
metrics/librato: ensure resp.body closed (#26969)
This change ensures that we call Close on a http response body, in various places in the source code (mostly tests)
2023-03-27 07:44:41 -04:00
Martin Holst Swende
eca3d39c31
p2p/discover: pass invalid discv5 packets to Unhandled channel (#26699)
This makes it possible to run another protocol alongside discv5, by reading 
unhandled packets from the channel.
2023-03-14 12:40:40 +01:00
panicalways
051493d9bf
p2p: small comment typo (#26850)
Update server.go
2023-03-10 11:45:49 +02:00
Martin Holst Swende
4d3525610e
all: remove deprecated uses of math.rand (#26710)
This PR is a (superior) alternative to https://github.com/ethereum/go-ethereum/pull/26708, it handles deprecation, primarily two specific cases. 

`rand.Seed` is typically used in two ways
- `rand.Seed(time.Now().UnixNano())` -- we seed it, just to be sure to get some random, and not always get the same thing on every run. This is not needed, with global seeding, so those are just removed. 
- `rand.Seed(1)` this is typically done to ensure we have a stable test. If we rely on this, we need to fix up the tests to use a deterministic prng-source. A few occurrences like this has been replaced with a proper custom source. 

`rand.Read` has been replaced by `crypto/rand`.`Read` in this PR.
2023-02-16 14:36:58 -05:00
Felix Lange
101587b3f4
p2p/dnsdisc: fix tests with Go 1.20 (#26690) 2023-02-14 15:58:43 +02:00
Shude Li
163e996d0e
all: use http package to replace http method names (#26535) 2023-01-24 11:12:25 +02:00
Felix Lange
a251bca67c
p2p/discover: add more packet information in logs (#26307)
* p2p/discover: add more packet information in logs

This adds more fields to discv5 packet logs. These can be useful when
debugging multi-packet interactions.

The FINDNODE message also gets an additional field, OpID for debugging
purposes. This field is not encoded onto the wire.

I'm also removing topic system related message types in this change.
These will come back in the future, where support for them will be
guarded by a config flag.

* p2p/discover/v5wire: rename 'Total' to 'RespCount'

The new name captures the meaning of this field better.
2023-01-03 12:36:38 +01:00
Felix Lange
9e6a1c3834
common/mclock: add Alarm (#26333)
Alarm is a timer utility that simplifies code where a timer needs to be rescheduled over
and over. Doing this can be tricky with time.Timer or time.AfterFunc because the channel
requires draining in some cases.

Alarm is optimized for use cases where items are tracked in a heap according to their expiry
time, and a goroutine with a for/select loop wants to be woken up whenever the next item expires.
In this application, the timer needs to be rescheduled when an item is added or removed
from the heap. Using a timer naively, these updates will always require synchronization
with the global runtime timer datastructure to update the timer using Reset. Alarm avoids
this by tracking the next expiry time and only modifies the timer if it would need to fire earlier
than already scheduled.

As an example use, I have converted p2p.dialScheduler to use Alarm instead of AfterFunc.
2023-01-03 12:10:48 +01:00
zhiqiangxu
a9dfac0332
p2p/discover: improve nodesByDistance.push code (#26019)
This improves readability of function 'push'.

sort.Search(N, ...) will at most return N when no match, so ix should be compared
with N. The previous version would compare ix with N+1 in case an additional item
was appended. No bug resulted from this comparison, but it's not easy to understand
why.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-12-07 23:31:47 +01:00
Seungbae Yu
42212808f0
p2p/nat: handle responses with alternative port in NAT-PMP (#26321)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-12-07 16:22:04 +01:00
Sina Mahmoodi
f20eba426a
graphql, node, rpc: improve HTTP write timeout handling (#25457)
Here we add special handling for sending an error response when the write timeout of the
HTTP server is just about to expire. This is surprisingly difficult to get right, since is
must be ensured that all output is fully flushed in time, which needs support from
multiple levels of the RPC handler stack:

The timeout response can't use chunked transfer-encoding because there is no way to write
the final terminating chunk. net/http writes it when the topmost handler returns, but the
timeout will already be over by the time that happens. We decided to disable chunked
encoding by setting content-length explicitly.

Gzip compression must also be disabled for timeout responses because we don't know the
true content-length before compressing all output, i.e. compression would reintroduce
chunked transfer-encoding.
2022-12-07 14:02:14 +01:00
Håvard Anda Estensen
06632da2bb
all: assign zero after resize in implementations of heap.Interface (#26296)
This changes the Pop method to assign the zero value before
reducing slice size. Doing so ensures the backing array does not
reference removed item values.
2022-12-05 13:49:54 +01:00
RichΛrd
c1aa1db69e
p2p/discover: add config option for discv5 protocol ID (#26041)
This option is occasionally useful for advanced uses of the discv5 protocol.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-30 22:03:34 +01:00
Seungbae Yu
53d1ae096a
p2p/nat: use IP.IsPrivate (#26252) 2022-11-25 16:30:07 +01:00
Felix Lange
9afc6816d2
common/lru: add generic LRU implementation (#26162)
It seems there is no fully typed library implementation of an LRU cache.
So I wrote one. Method names are the same as github.com/hashicorp/golang-lru,
and the new type can be used as a drop-in replacement.

Two reasons to do this:

- It's much easier to understand what a cache is for when the types are right there.
- Performance: the new implementation is slightly faster and performs zero memory
   allocations in Add when the cache is at capacity. Overall, memory usage of the cache
   is much reduced because keys are values are no longer wrapped in interface.
2022-11-14 15:41:56 +01:00
zhiqiangxu
4cb1fca43d
p2p/enode: implement per-source timeout in FairMix (#25962)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-08 12:12:36 +01:00
Felix Lange
9027ee0b45
p2p/discover: improve discv5 NODES response packing (#26033)
Instead of using a limit of three nodes per message, we can pack more nodes
into each message based on ENR size. In my testing, this halves the number
of sent NODES messages, because ENR size is usually < 300 bytes.

This also adds RLP helper functions that compute the encoded size of
[]byte and string.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-07 19:19:02 +01:00
Justin Traglia
621b423ac1
p2p/discover: fix handling of distance 256 in lookupDistances (#26087)
Noticed that lookupDistances for FINDNODE requests didn't consider 256 a valid
distance. This is actually part of the example in the comment above the
function, surprised that wasn't tested before.
2022-11-02 14:50:07 +01:00
Martin Holst Swende
5a02b2d6d0
all: fix spelling mistakes (#25961) 2022-10-11 09:37:00 +02:00
Martin Holst Swende
ee301c750b
all: fix docstrings 2022-10-04 09:18:02 +02:00
Boqin@MetaSecureLabs
e6d4aedb8c
p2p: add channel buffers to avoid goroutine leaks in tests (#24929) 2022-09-23 10:51:12 +02:00