Commit Graph

84 Commits

Author SHA1 Message Date
mmsqe
f3314bb6df
rpc: add limit for batch request items and response size (#26681)
This PR adds server-side limits for JSON-RPC batch requests. Before this change, batches
were limited only by processing time. The server would pick calls from the batch and
answer them until the response timeout occurred, then stop processing the remaining batch
items.

Here, we are adding two additional limits which can be configured:

- the 'item limit': batches can have at most N items
- the 'response size limit': batches can contain at most X response bytes

These limits are optional in package rpc. In Geth, we set a default limit of 1000 items
and 25MB response size.

When a batch goes over the limit, an error response is returned to the client. However,
doing this correctly isn't always possible. In JSON-RPC, only method calls with a valid
`id` can be responded to. Since batches may also contain non-call messages or
notifications, the best effort thing we can do to report an error with the batch itself is
reporting the limit violation as an error for the first method call in the batch. If a batch is
too large, but contains only notifications and responses, the error will be reported with
a null `id`.

The RPC client was also changed so it can deal with errors resulting from too large
batches. An older client connected to the server code in this PR could get stuck
until the request timeout occurred when the batch is too large. **Upgrading to a version
of the RPC client containing this change is strongly recommended to avoid timeout issues.**

For some weird reason, when writing the original client implementation, @fjl worked off of
the assumption that responses could be distributed across batches arbitrarily. So for a
batch request containing requests `[A B C]`, the server could respond with `[A B C]` but
also with `[A B] [C]` or even `[A] [B] [C]` and it wouldn't make a difference to the
client.

So in the implementation of BatchCallContext, the client waited for all requests in the
batch individually. If the server didn't respond to some of the requests in the batch, the
client would eventually just time out (if a context was used).

With the addition of batch limits into the server, we anticipate that people will hit this
kind of error way more often. To handle this properly, the client now waits for a single
response batch and expects it to contain all responses to the requests.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-06-13 13:38:58 +02:00
Péter Szilágyi
c7c84ca16c
all: remove the Rinkeby testnet (#27406) 2023-06-02 14:03:21 +03:00
Delweng
e9c3183c52
cmd: use errrors.New instead of empty fmt.Errorf (#27329)
Signed-off-by: jsvisa <delweng@gmail.com>
2023-05-24 12:21:29 +02:00
Péter Szilágyi
095e365fac
all: remove support for Ropsten (#26644) 2023-02-09 10:03:00 +02:00
Joseph Cook
55a92fa0a4
cmd/clef: list accounts at startup (#26082)
Reports accounts known to Clef during startup, after master seed is provided by the user.
2022-11-07 16:41:36 +01:00
Martin Holst Swende
17744639da
cmd/clef: add importraw feature to clef (#26058)
This adds a subcommand that imports a raw secp256k1 key
into the keystore managed by clef.
2022-11-06 13:02:49 +01:00
Joseph Cook
f3a005f176
cmd/clef: add list-accounts and list-wallets to CLI (#26080)
This commit adds support for two new commands to clef, making it possible to list accounts / wallets from the command-line-interface. 

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-11-02 19:02:32 +01:00
Felix Lange
65f3c1b46f
internal/version: use gitCommit injection in version handling code (#25851)
This changes the CI build to store the git commit and date into package
internal/version instead of package main. Doing this essentially merges our
two ways of tracking the go-ethereum version into a single place, achieving
two objectives:

- Bad block reports, which use version.Info(), will now have the git commit
  information even when geth is built in an environment such as
  launchpad.net where git access is unavailable.

- For geth builds created by `go build ./cmd/geth` (i.e. not using `go run
  build/ci.go install`), git information stored by the go tool is now used
  in the p2p node name as well as in `geth version` and `geth
  version-check`.
2022-09-23 14:08:25 +02:00
Delweng
b196ad1c16
all: add whitespace linter (#25312)
* golangci: typo

Signed-off-by: Delweng <delweng@gmail.com>

* golangci: add whietspace

Signed-off-by: Delweng <delweng@gmail.com>

* *: rm whitesapce using golangci-lint

Signed-off-by: Delweng <delweng@gmail.com>

* cmd/puppeth: revert accidental resurrection

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-07-25 13:14:03 +03:00
Péter Szilágyi
b53d38246e
Merge pull request #25096 from lightclient/remove-version-field
all: remove version field
2022-07-14 12:13:59 +03:00
willian.eth
52ed3570c4
cmd: migrate to urfave/cli/v2 (#24751)
This change updates our urfave/cli dependency to the v2 branch of the library.
There are some Go API changes in cli v2:

- Flag values can now be accessed using the methods ctx.Bool,
  ctx.Int, ctx.String, ... regardless of whether the flag is 'local' or
  'global'.

- v2 has built-in support for flag categories. Our home-grown category
  system is removed and the categories of flags are assigned as part of
  the flag definition.

For users, there is only one observable difference with cli v2: flags must now
strictly appear before regular arguments. For example, the following command is
now invalid:

   geth account import mykey.json --password file.txt

Instead, the command must be invoked as follows:

   geth account import --password file.txt mykey.json
2022-06-27 18:22:36 +02:00
lightclient@protonmail.com
0c6f81f888
all: remove version field from rpc.API 2022-06-27 12:39:46 +02:00
lightclient
119f955686
all: remove public field from rpc.API (#25059)
all: remove public field from rpc.API
2022-06-27 13:33:13 +03:00
Håvard Anda Estensen
07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
Martin Holst Swende
559a174899
cmd/clef: fixups to the python clef poc (#24440)
This PR fixes up the example python clef wrapper. The poc is intended to demonstrate how to wite a UI for clef, and had severely bitrotted.
With these changes, it "works" in the sense that all the built-in tests triggers the intended python callbacks (no errors about method not found). It does not "work" in the sense that the wrapper can be used as an actual UI. It will auto-reject any signing requests, for example.
2022-05-02 14:28:11 +02:00
Martin Holst Swende
4860e50e05
cmd/geth, node, rpc: implement jwt tokens (#24364)
* rpc, node: refactor request validation and add jwt validation

* node, rpc: fix error message, ignore engine api in RegisterAPIs

* node: make authenticated port configurable

* eth/catalyst: enable unauthenticated version of engine api

* node: rework obtainjwtsecret (backport later)

* cmd/geth: added auth port flag

* node: happy lint, happy life

* node: refactor authenticated api

Modifies the authentication mechanism to use default values

* node: trim spaces and newline away from secret

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-07 09:30:27 +02:00
Martin Holst Swende
619a3e7085
signer/core: move EIP-712 types to package apitypes (#24029)
Fixes #23972
2021-12-05 14:31:41 +01:00
baptiste-b-pegasys
95bbd46eab
node, cmd/clef: remove term "whitelist" (#23296)
* node: remove term "whitelist"

* include cmd/clef
2021-08-02 15:43:01 +02:00
Martin Holst Swende
295bc35ecf
signer/core: move API JSON types to separate package (#23275)
This PR moves (some) account types into a standalone package, to avoid
depending on signer/core from accounts/external.
2021-07-29 16:06:44 +02:00
Martin Holst Swende
7a00378e2b
cmd/clef, signer: support for eip-1559 txs in clef (#22966) 2021-06-09 13:48:47 +02:00
ligi
7cbf1d70a7
cmd/clef (docs): fix image background (#22399)
Flatten the image so we do not have dark text on dark background
2021-03-16 09:55:03 +01:00
ligi
7076e8e42f
cmd/clef: docs - link to ethereum org repo (#22400) 2021-03-16 09:53:43 +01:00
lightclient
bbfb1e4008
all: add support for EIP-2718, EIP-2930 transactions (#21502)
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.

There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.

The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID. 

This change also adds support for the YoloV3 testnet.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
2021-02-25 15:26:57 +01:00
rene
8e547eecd5
cmd/utils: remove deprecated command line flags (#22263)
This removes support for all deprecated flags except --rpc*.
2021-02-24 14:07:58 +01:00
Martin Holst Swende
f25b437b70
cmd/clef: don't check file permissions on windows (#22251)
Fixes #20123
2021-01-29 16:53:44 +01:00
Alex Prut
810f9e057d
all: remove redundant conversions and import names (#21903) 2020-11-25 21:00:23 +01:00
Alex Prut
c92faee66e
all: simplify nested complexity and if blocks ending with a return statement (#21854)
Changes:

    Simplify nested complexity
    If an if blocks ends with a return statement then remove the else nesting.

Most of the changes has also been reported in golint https://goreportcard.com/report/github.com/ethereum/go-ethereum#golint
2020-11-25 09:24:50 +01:00
Martin Holst Swende
dad26582b6
accounts, signer: implement gnosis safe support (#21593)
* accounts, signer: implement gnosis safe support

* common/math: add type for marshalling big to dec

* accounts, signer: properly sign gnosis requests

* signer, clef: implement account_signGnosisTx

* signer: fix auditlog print, change rpc-name (signGnosisTx to signGnosisSafeTx)

* signer: pass validation-messages/warnings to the UI for gnonsis-safe txs

* signer/core: minor change to validationmessages of typed data
2020-09-29 17:40:08 +02:00
Binacs
fdb742419e
cmd/clef, cmd/geth: use SplitAndTrim from cmd/utils (#21579) 2020-09-22 23:22:54 +02:00
6543
79ce5537ab
signer/storage: fix a badly ordered error check (#21379) 2020-07-28 12:47:05 +03:00
rene
5b081ab214
cmd/clef: change --rpcport to --http.port and update flags in docs (#21318) 2020-07-14 10:35:32 +02:00
chris-j-h
c16967c267
cmd/clef: Fix broken link in README and other minor fixes (#21303) 2020-07-07 22:23:23 +02:00
chris-j-h
e5871b928f
cmd/clef: Update README with external v6.0.0 & internal v7.0.1 APIs (#21298)
Changes include:
* Updates response docs for `account_new`, `account_list`, `account_signTransaction`
* Removes `account_import`, `account_export` docs
* Adds `account_version` docs
* Updates request docs for `ui_approveListing`, `ui_approveSignData`, `ui_showInfo`, `ui_showError`, `ui_onApprovedTx`
* Adds `ui_approveNewAccount`, `ui_onInputRequired` docs
2020-07-07 11:12:38 +02:00
Binacs
dd91c7ce6a
cmd: abstract getPassPhrase functions into one (#21219)
* [cmd] Abstract `getPassPhrase` functions into one.

* cmd/ethkey: fix compilation failure

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2020-06-30 09:56:40 +02:00
Martin Michlmayr
4f2784b38f
all: fix typos in comments (#21118) 2020-05-25 10:21:28 +02:00
Martin Holst Swende
e0987f67e0
cmd/clef, signer/core: password input fixes (#20960)
* cmd/clef, signer/core: use better terminal input for passwords, make it possible to avoid boot-up warning

* all: move commonly used prompter to isolated (small) package

* cmd/clef: Add new --acceptWarn to clef README

* cmd/clef: rename flag 'acceptWarn' to 'suppress-bootwarn'

Co-authored-by: ligi <ligi@ligi.de>
2020-05-19 10:44:46 +02:00
Satpal
677724af0c
cmd: fix log contexts (#21077) 2020-05-13 18:34:24 +03:00
rene
c989bca173
cmd/utils: renames flags related to http-rpc server (#20935)
* rpc flags related to starting http server renamed to http

* old rpc flags aliased and still functional

* pprof flags fixed

* renames gpo related flags

* linted

* renamed rpc flags for consistency and clarity

* added warn logs

* added more warn logs for all deprecated flags for consistency

* moves legacy flags to separate file, hides older flags under show-deprecated-flags command

* legacy prefix and moved some more legacy flags to legacy file

* fixed circular import

* added docs

* fixed imports lint error

* added notes about when flags were deprecated

* cmd/utils: group flags by deprecation date + reorder by date,

* modified deprecated comments for consistency, added warn log for --rpc

* making sure deprecated flags are still functional

* show-deprecated-flags command cleaned up

* fixed lint errors

* corrected merge conflict

* IsSet --> GlobalIsSet

* uncategorized flags, if not deprecated, displayed under misc

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-05-05 11:19:17 +03:00
Steven E. Harris
40283d0522
node: shut down all node-related HTTP servers gracefully (#20956)
Rather than just closing the underlying network listener to stop our
HTTP servers, use the graceful shutdown procedure, waiting for any
in-process requests to finish.
2020-04-27 11:16:00 +02:00
rene
07d909ff32
node: allow websocket and HTTP on the same port (#20810)
This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546. 

    geth --rpc --rpcport 8545 --ws --wsport 8545

This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.
2020-04-08 13:33:12 +02:00
rene
671f22be38
couple of fixes to docs in clef (#20900) 2020-04-07 14:37:24 +02:00
Martin Holst Swende
c56f4fa808
cmd/clef: add newaccount command (#20782)
* cmd/clef: add newaccount command

* cmd/clef: document clef_New, update API versioning

* Update cmd/clef/intapi_changelog.md

Co-Authored-By: ligi <ligi@ligi.de>

* Update signer/core/uiapi.go

Co-Authored-By: ligi <ligi@ligi.de>

Co-authored-by: ligi <ligi@ligi.de>
2020-03-31 12:03:48 +02:00
Martin Holst Swende
0734c4b820
node, cmd/clef: report actual port used for http rpc (#20789) 2020-03-23 16:26:56 +01:00
Chris Chinchilla
6ffee2afd6
docs: correct clef typo (#20705) 2020-02-21 16:30:21 +01:00
zaccoding
23c8c74131 cmd: fix command help messages in modules (#20203) 2019-11-26 11:46:39 +01:00
Felix Lange
a3d263dd3a
cmd/clef: fix staticcheck warnings (#20314) 2019-11-18 16:38:54 +01:00
Felix Lange
689486449d build: use golangci-lint (#20295)
* build: use golangci-lint

This changes build/ci.go to download and run golangci-lint instead
of gometalinter.

* core/state: fix unnecessary conversion

* p2p/simulations: fix lock copying (found by go vet)

* signer/core: fix unnecessary conversions

* crypto/ecies: remove unused function cmpPublic

* core/rawdb: remove unused function print

* core/state: remove unused function xTestFuzzCutter

* core/vm: disable TestWriteExpectedValues in a different way

* core/forkid: remove unused function checksum

* les: remove unused type proofsData

* cmd/utils: remove unused functions prefixedNames, prefixFor

* crypto/bn256: run goimports

* p2p/nat: fix goimports lint issue

* cmd/clef: avoid using unkeyed struct fields

* les: cancel context in testRequest

* rlp: delete unreachable code

* core: gofmt

* internal/build: simplify DownloadFile for Go 1.11 compatibility

* build: remove go test --short flag

* .travis.yml: disable build cache

* whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement

* .golangci.yml: enable goconst and ineffassign linters

* build: print message when there are no lint issues

* internal/build: refactor download a bit
2019-11-18 10:49:17 +02:00
Martin Holst Swende
db79143a13
clef: resolve windows pipes, fixes #20121 (#20166) 2019-10-24 10:45:07 +02:00
Ben Holden-Crowther
85d2b0d654 cmd/clef: spelling correction (#19973) 2019-08-19 13:31:45 +03:00
ligi
9b8d727655 cmd/clef: fix typo introduced in #19932 (#19946) 2019-08-12 17:39:15 +03:00