Commit Graph

923 Commits

Author SHA1 Message Date
philip-morlier
bdc42a9128 Merge commit '7371b3817' into merge/geth-v1.13.0 2023-09-12 12:34:24 -07:00
Martin Holst Swende
16946d218a
rpc: use go-winio for named pipes (#27972)
We're trying a new named pipe library, which should hopefully fix some occasional failures in CI.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 04:39:01 +02:00
Felix Lange
8f8ef2bc0c
rpc: attempt to fix ping/pong logic race (#27733)
This should fix #27726. With enough load, it might happen that the SetPongHandler 
callback gets invoked before the call to SetReadDeadline is made in pingLoop. When 
this occurs, the socket will end up with a 30s read deadline even though it got the pong,
which will lead to a timeout.

The fix here is processing the pong on pingLoop, synchronizing with the code that 
sends the ping.
2023-08-11 15:16:45 +02:00
philip-morlier
6ed9390f9b Merge remote-tracking branch 'origin/feature/plugeth-test-plugin' into merge/geth-v1.12.0 2023-07-12 13:47:51 -07:00
philip-morlier
dc56f2a361 incremental commit, manual changes to support merge 2023-07-11 17:01:10 -07:00
philip-morlier
e516d5842c automated merge 2023-07-11 16:01:17 -07:00
philip-morlier
b3f1d35171 functional test plugin.
This PR will also include the stand alone tests.
2023-06-22 01:05:10 -07:00
Koichi Shiraishi
f0b5af74a3
rpc: avoid use of cgo by hard-coding maxPathSize (#27447)
Package rpc uses cgo to find the maximum UNIX domain socket path 
length. If exceeded, a warning is printed. This is the only use of cgo in this
package. It seems excessive to depend on cgo just for this warning, so
we now hard-code the usual limit for Linux instead.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-19 08:06:58 +02:00
philip-morlier
fee25089a9 Added HookTester back into main. 2023-06-14 22:24:08 -07:00
zhiqiangxu
6f08c2f3f1
rpc: add method to test for subscription support (#25942)
This adds two ways to check for subscription support. First, one can now check
whether the transport method (HTTP/WS/etc.) is capable of subscriptions using
the new Client.SupportsSubscriptions method.

Second, the error returned by Subscribe can now reliably be tested using this
pattern:
    
    sub, err := client.Subscribe(...)
    if errors.Is(err, rpc.ErrNotificationsUnsupported) {
        // no subscription support
    }

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-06-14 14:04:41 +02:00
mmsqe
f3314bb6df
rpc: add limit for batch request items and response size (#26681)
This PR adds server-side limits for JSON-RPC batch requests. Before this change, batches
were limited only by processing time. The server would pick calls from the batch and
answer them until the response timeout occurred, then stop processing the remaining batch
items.

Here, we are adding two additional limits which can be configured:

- the 'item limit': batches can have at most N items
- the 'response size limit': batches can contain at most X response bytes

These limits are optional in package rpc. In Geth, we set a default limit of 1000 items
and 25MB response size.

When a batch goes over the limit, an error response is returned to the client. However,
doing this correctly isn't always possible. In JSON-RPC, only method calls with a valid
`id` can be responded to. Since batches may also contain non-call messages or
notifications, the best effort thing we can do to report an error with the batch itself is
reporting the limit violation as an error for the first method call in the batch. If a batch is
too large, but contains only notifications and responses, the error will be reported with
a null `id`.

The RPC client was also changed so it can deal with errors resulting from too large
batches. An older client connected to the server code in this PR could get stuck
until the request timeout occurred when the batch is too large. **Upgrading to a version
of the RPC client containing this change is strongly recommended to avoid timeout issues.**

For some weird reason, when writing the original client implementation, @fjl worked off of
the assumption that responses could be distributed across batches arbitrarily. So for a
batch request containing requests `[A B C]`, the server could respond with `[A B C]` but
also with `[A B] [C]` or even `[A] [B] [C]` and it wouldn't make a difference to the
client.

So in the implementation of BatchCallContext, the client waited for all requests in the
batch individually. If the server didn't respond to some of the requests in the batch, the
client would eventually just time out (if a context was used).

With the addition of batch limits into the server, we anticipate that people will hit this
kind of error way more often. To handle this properly, the client now waits for a single
response batch and expects it to contain all responses to the requests.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-06-13 13:38:58 +02:00
f7ff01fd54 fix plugin subscription method registration 2023-06-10 01:51:39 +08:00
Martin Holst Swende
9231770811
rpc: change BlockNumber constant values to match ethclient (#27219)
ethclient accepts certain negative block number values as specifiers for the "pending",
"safe" and "finalized" block. In case of "pending", the value accepted by ethclient (-1)
did not match rpc.PendingBlockNumber (-2).

This wasn't really a problem, but other values accepted by ethclient did match the
definitions in package rpc, and it's weird to have this one special case where they don't.

To fix it, we decided to change the values of the constants rather than changing ethclient.
The constant values are not otherwise significant. This is a breaking API change, but we
believe not a dangerous one.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-05-23 13:18:38 +02:00
Delweng
6fe0252571
rpc, internal/cmdtest: increase timeout in tests (#27083)
This change gives the cmd-tests have a bit more time to finish before getting forcibly torn down.
2023-05-22 08:13:03 -04:00
Stephen Guo
84c3799e21
rpc: more accurate checking of handler method signatures (#27287)
This changes the RPC server to ignore methods using *context.Context as parameter
and *error as return value type. Methods with such types would crash the server when
called.
2023-05-17 12:27:51 +02:00
oseau
7369752999
rpc: websocket should respect the "HTTP_PROXY" by default (#27264)
rpc: the default dialer for websocket should respect the proxy environment variables like "HTTP_PROXY"
2023-05-15 08:48:06 +03:00
s7v7nislands
ffda2c64c4
rpc: use atomic types (#27214)
rpc: use atomic type
2023-05-04 04:54:45 -04:00
philip-morlier
e986bbfa6f Initial commit of merge geth v1.11.5 2023-04-21 13:24:40 -07:00
Delweng
117530b0e6
metrics/librato: ensure resp.body closed (#26969)
This change ensures that we call Close on a http response body, in various places in the source code (mostly tests)
2023-03-27 07:44:41 -04:00
philip-morlier
f3e558d4e0 Merge commit 'a38f41085' into merge/geth-v1.11.5
Manual touches in core/state_transtion.go and go.mod
2023-03-21 08:39:16 -07:00
Jonathan Otto
6bc68f8d94
Increase websocket frame size (from erigon rpc client) (#26883)
This increases the maximum allowed message size to 32MB.

Originally submitted at https://github.com/ledgerwatch/erigon/pull/2739

example block failure: https://etherscan.io/tx/0x1317d973a55cedf9b0f2df6ea48e8077dd176f5444a3423368a46d6e4db89982#internal
2023-03-14 18:41:28 +01:00
philip-morlier
4415e7c7b8 Merge commit '73b01f40c' geth v1.11.2 into merge/geth-v1.11.2 2023-02-22 08:13:38 -08:00
Felix Lange
1db978ca6b
rpc: fix unmarshaling of null result in CallContext (#26723)
The change fixes unmarshaling of JSON null results into json.RawMessage.

---------

Co-authored-by: Jason Yuan <jason.yuan@curvegrid.com>
Co-authored-by: Jason Yuan <jason.yuan869@gmail.com>
2023-02-19 14:23:18 -05:00
philip-morlier
2f6b8d86f1 manual geth v1.11.1 merge 2023-02-17 07:53:01 -08:00
Felix Lange
31d401ea68
rpc: remove DecimalOrHex type (#26629)
It's the same as math.HexOrDecimal64, which has more uses across the codebase.
2023-02-07 15:44:27 +01:00
Martin Holst Swende
8e92881a3d
rpc: fix off-by-one in ipc endpoint length check (#26614)
This change fixes a minor flaw in the check for ipc endpoint length. The max_path_size is the max path that an ipc endpoint can have, which is 208. However, that size concerns the null-terminated pathname, so we need to account for an extra null-character too.
2023-02-06 07:25:42 -05:00
Shude Li
163e996d0e
all: use http package to replace http method names (#26535) 2023-01-24 11:12:25 +02:00
lightclient
6f858fa806
rpc: fix setting client in DialHTTPWithClient (#26470) 2023-01-12 11:04:34 +01:00
Sina Mahmoodi
f20eba426a
graphql, node, rpc: improve HTTP write timeout handling (#25457)
Here we add special handling for sending an error response when the write timeout of the
HTTP server is just about to expire. This is surprisingly difficult to get right, since is
must be ensured that all output is fully flushed in time, which needs support from
multiple levels of the RPC handler stack:

The timeout response can't use chunked transfer-encoding because there is no way to write
the final terminating chunk. net/http writes it when the topmost handler returns, but the
timeout will already be over by the time that happens. We decided to disable chunked
encoding by setting content-length explicitly.

Gzip compression must also be disabled for timeout responses because we don't know the
true content-length before compressing all output, i.e. compression would reintroduce
chunked transfer-encoding.
2022-12-07 14:02:14 +01:00
Wihan de Beer
8c6e74eed4
rpc: decrease websocket ping interval (#26253)
This is to cater for more node providers.
2022-11-25 16:29:38 +01:00
Martin Holst Swende
97c563e055
rpc, internal/guide: speed up tests a bit (#26193)
This removes an RPC test which takes > 90s to execute, and updates the
internal/guide tests to use lighter scrypt parameters.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-17 15:30:53 +01:00
storyicon
add337e0f7
rpc: support injecting HTTP headers through context (#26023)
This adds a way to specify HTTP headers per request.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-16 15:22:12 +01:00
Felix Lange
ae42148093
rpc: fix connection tracking set in Server (#26180)
rpc: fix connection tracking in Server

When upgrading to mapset/v2 with generics, the set element type used in
rpc.Server had to be changed to *ServerCodec because ServerCodec is not
'comparable'. While the distinction is technically correct, we know all
possible ServerCodec types, and all of them are comparable. So just use
a map instead.
2022-11-15 14:05:16 +01:00
Jolly Zhao
f58ebd9696
all: use github.com/deckarep/golang-set/v2 (generic set) (#26159)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-11-14 15:16:52 +01:00
philip-morlier
db52b5e61d Merge tag 'v1.10.26' into feature/merge-v1.10.26 2022-11-03 09:06:21 -07:00
Jordan Krage
211dbb7197 rpc: handle wrong HTTP batch response length (#26064) 2022-11-03 01:15:36 +01:00
Jordan Krage
05037eaffc
rpc: handle wrong HTTP batch response length (#26064) 2022-11-02 15:29:33 +01:00
s7v7nislands
fb4a97f33f
rpc: update docstring (#26072)
Co-authored-by: seven <seven@nodereal.io>
2022-10-31 09:42:25 +01:00
Felix Lange
b628d72766
build: upgrade to go 1.19 (#25726)
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.

In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.

With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2022-09-10 13:25:40 +02:00
Nicholas
610cf02c4a
rpc: improve error codes for internal server errors (#25678)
This changes the error code returned by the RPC server in certain situations:

- handler panic: code -32603
- result marshaling error: code -32603
- attempt to subscribe via HTTP: code -32001

In all of the above cases, the server previously returned the default error
code -32000.

Co-authored-by: Nicholas Zhao <nicholas.zhao@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-09 14:03:23 +02:00
Seungbae Yu
38e002f464
rpc: check that "version" is "2.0" in request objects (#25570)
The JSON-RPC spec requires the "version" field to be exactly "2.0",
so we should verify that. This change is not backwards-compatible with
sloppy client implementations, but I decided to go ahead with it anyway
because the failure will be caught via the returned error.
2022-09-02 17:47:29 +02:00
protolambda
90711efb0a
node, rpc: add JWT auth support in client (#24911)
This adds a generic mechanism for 'dial options' in the RPC client,
and also implements a specific dial option for the JWT authentication
mechanism used by the engine API. Some real tests for the server-side
authentication handling are also added.

Co-authored-by: Joshua Gutow <jgutow@optimism.io>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-02 17:40:41 +02:00
philip-morlier
59a823409e Merge tag 'v1.10.22' into feature/merge-v1.10.22 2022-08-22 10:48:40 -07:00
Justin Traglia
2c5648d891
all: fix some typos (#25551)
* Fix some typos

* Fix some mistakes

* Revert 4byte.json

* Fix an incorrect fix

* Change files to fails
2022-08-19 09:00:21 +03:00
Tristan-Wilson
9244f87dc1
node, rpc: add ReadHeaderTimeout config option (#25338)
This change makes http.Server.ReadHeaderTimeout configurable separately
from ReadTimeout for RPC servers. The default is set to the same as
ReadTimeout, which in order to cause no change in existing deployments.
2022-08-03 16:50:12 +02:00
philip-morlier
81a8106bc6 Merge Geth v1.10.21 as well as update to Plugeth-Utils v0.0.18 2022-07-27 10:38:42 -07:00
Marius van der Wijden
c6dcd018d2
core: eth: rpc: implement safe rpc block (#25165)
* core: eth: rpc: implement safe rpc block

* core: fix setHead, panics
2022-07-25 18:42:05 +03:00
lightclient@protonmail.com
0c6f81f888
all: remove version field from rpc.API 2022-06-27 12:39:46 +02:00
lightclient
119f955686
all: remove public field from rpc.API (#25059)
all: remove public field from rpc.API
2022-06-27 13:33:13 +03:00
Zachinquarantine
f49e298330
README,rpc: remove mention of "shh" RPC API (#25137) 2022-06-21 11:59:02 +02:00