Commit Graph

74 Commits

Author SHA1 Message Date
Martin Holst Swende
ed51b8c5d3
ethdb: pebble backend (64bit platforms only) (#26517)
* ethdb: use pebble

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

foo

update

* apply suggested changes

* flags: go format

node: fix ddir lookup mistake

accounts/abi/bind: fix go.mod replacement for generated binding

deps: update pebble + with fix 32-bit build

* ethdb/pebble: respect max memtable size

* core/rawdb, ethdb: enable pebble on non-32bit platforms only

* core/rawdb: fix build tags, fix some review concerns

* core/rawdb: refactor methods for database opening

* core/rawdb: remove erroneous build tag

* cmd/geth: fix the flag default handling + testcase

* cmd/geth: improve testing regarding custom backends

* ethdb/pebble, deps: update pebble dependency

* core/rawdb: replace method with Open

* ethdb/pebble: several updates for pebble (#49)

* ethdb/pebble: fix size count in batch

* ethdb/pebble: disable seek compaction

* ethdb/pebble: more fixes

* ethdb, core, cmd: polish and fixes (#50)

* cmd/utils, core/rawdb, ethdb/pebble: address some review concerns

* Update flags.go

* ethdb/pebble: minor refactors

* ethdb/pebble: avoid copy on batch replay

* ethdb: fix compilation flaw

* cmd: fix test fail due to mismatching error message

* cmd/geth, node: rename backingdb to db.engine

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-02-09 10:48:34 +02:00
Marius van der Wijden
d0a4989a8d
cmd, eth, node: deprecate personal namespace (#26390)
* eth: cmd: deprecate personal namespace

* eth: cmd: move deprecation to node

* node: disable toml of enablepersonal

* node: disable personal on ipc as well

* Update node/node.go

Co-authored-by: Martin Holst Swende <martin@swende.se>

* console: error -> warn

* node: less roulette

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-02-02 13:52:19 +02:00
Sina Mahmoodi
3630cafb34
node: drop support for static & trusted node list files (#25610)
This changes the node setup to ignore datadir files

    static-nodes.json
    trusted-nodes.json

When these files are present, it an error will be printed to the log.
2022-10-12 10:50:12 +02:00
Felix Lange
9cddfe92a3
node: prevent exposing engine API on unauthenticated endpoint (#25939)
* node: prevent exposing engine API on unauthenticated endpoint

* node: improve RPC setup
2022-10-06 14:01:04 +02:00
protolambda
90711efb0a
node, rpc: add JWT auth support in client (#24911)
This adds a generic mechanism for 'dial options' in the RPC client,
and also implements a specific dial option for the JWT authentication
mechanism used by the engine API. Some real tests for the server-side
authentication handling are also added.

Co-authored-by: Joshua Gutow <jgutow@optimism.io>
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-09-02 17:40:41 +02:00
rjl493456442
e44d6551c3
cmd, core, ethdb, node: move chain freezer one folder deeper (#25487)
* cmd, core, ethdb, node: create chain freezer in a sub folder

* core/rawdb: remove unused code

* core, ethdb, node: add AncientDatadir API back

* cmd, core: extend freezer info dump for sub-ancient-store

* core/rawdb: rework freezer inspector

* core/rawdb: address comments from Peter

* core/rawdb: fix build issue
2022-08-08 12:08:36 +03:00
Marius van der Wijden
d78d302f3d
node: add info message when JWT secret is loaded (#25095)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-06-15 14:35:53 +02:00
Marius van der Wijden
a8040bc2c5
node: allow JWT pass by file only (#24579) 2022-03-24 15:04:47 +01:00
Marius van der Wijden
4f4622bc8b
core: eth: implement Kiln-v2 spec (#24506)
* core/beacon: eth/catalyst: updated engine api to new version

* core: implement exchangeTransitionConfig

* core/beacon: prevRandao instead of Random

* eth/catalyst: Fix ExchangeTransitionConfig, add test

* eth/catalyst: stop external miners on TTD reached

* node: implement --authrpc.vhosts flag

* core: allow for config override on non-mainnet networks

* eth/catalyst: fix peters comments

* eth/catalyst: make stop remote sealer more explicit

* eth/catalyst: add log output

* cmd/utils: rename authrpc.host to authrpc.addr

* eth/catalyst: disable the disabling of the miner

* eth: core: remove notion of terminal pow block

* eth: les: more of peters nitpicks
2022-03-17 17:20:03 +02:00
Felix Lange
5079e3c6e5
cmd/geth: make authrpc listening address settable from command line (#24522)
The default listening address "localhost" is not sufficient when running
geth in Docker.
2022-03-09 14:45:39 +01:00
Martin Holst Swende
4860e50e05
cmd/geth, node, rpc: implement jwt tokens (#24364)
* rpc, node: refactor request validation and add jwt validation

* node, rpc: fix error message, ignore engine api in RegisterAPIs

* node: make authenticated port configurable

* eth/catalyst: enable unauthenticated version of engine api

* node: rework obtainjwtsecret (backport later)

* cmd/geth: added auth port flag

* node: happy lint, happy life

* node: refactor authenticated api

Modifies the authentication mechanism to use default values

* node: trim spaces and newline away from secret

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-07 09:30:27 +02:00
Sina Mahmoodi
108eec3fee
node: remove dependency on wallet backend packages (#23019)
* accounts: new AddBackends method in manager

* node,cmd/geth: mv accman backend init to cmd/geth

* node,cmd/geth: mv scrypt config downstreawm from node

* accounts: use static buffer size for accman sub chan

minor fix

* accounts,cmd/geth: update accman backends through its event loop

* accounts,node: add comments

* accounts: un-export newBackendEvent

* accounts: use chan instead of wg in newBlockEvent

* node: rename isKeyDirEphem

* accounts,cmd: AddBackends->AddBackend

* accounts: fix potential blocking when adding backend
2021-08-25 22:34:22 +02:00
gary rong
0c70b83e00
all: add read-only option to database (#22407)
* all: add read-only option to database

* all: fixes tests

* cmd/geth: migrate flags

* cmd/geth: fix the compact

* cmd/geth: fix the format

* cmd/geth: fix log

* cmd: add chain-readonly

* core: add readonly notion to freezer

* core/rawdb: add log

* core/rawdb: fix freezer close

* cmd: fix

* cmd, core: construct db

* core: update tests
2021-03-22 19:06:30 +01:00
rene
4eae0c6b6f
cmd/geth, node: allow configuring JSON-RPC on custom path prefix (#22184)
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.

Fixes #21826

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-02-02 10:05:46 +01:00
rene
5c2a7ce2cc
node: rename startNetworking to openEndpoints (#22105) 2021-01-04 12:39:25 +01:00
rene
c0c01612e9
node: refactor package node (#21105)
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.

There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.

There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.

The --graphql.port and --graphql.addr flags are no longer available.
2020-08-03 19:40:46 +02:00
Steven E. Harris
40283d0522
node: shut down all node-related HTTP servers gracefully (#20956)
Rather than just closing the underlying network listener to stop our
HTTP servers, use the graceful shutdown procedure, waiting for any
in-process requests to finish.
2020-04-27 11:16:00 +02:00
rene
07d909ff32
node: allow websocket and HTTP on the same port (#20810)
This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546. 

    geth --rpc --rpcport 8545 --ws --wsport 8545

This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.
2020-04-08 13:33:12 +02:00
rene
d7851e6359
graphql, node, rpc: fix typos in comments (#20824) 2020-03-27 13:52:53 +01:00
Martin Holst Swende
0734c4b820
node, cmd/clef: report actual port used for http rpc (#20789) 2020-03-23 16:26:56 +01:00
Felix Lange
38d1b0cba2
cmd/geth: enable DNS discovery by default (#20660)
* node: expose config in service context

* eth: integrate p2p/dnsdisc

* cmd/geth: add some DNS flags

* eth: remove DNS URLs

* cmd/utils: configure DNS names for testnets

* params: update DNS URLs

* cmd/geth: configure mainnet DNS

* cmd/utils: rename DNS flag and fix flag processing

* cmd/utils: remove debug print

* node: fix test
2020-02-13 15:38:30 +02:00
gary rong
f7cdea2bdc all: on-chain oracle checkpoint syncing (#19543)
* all: implement simple checkpoint syncing

cmd, les, node: remove callback mechanism

cmd, node: remove callback definition

les: simplify the registrar

les: expose checkpoint rpc services in the light client

les, light: don't store untrusted receipt

cmd, contracts, les: discard stale checkpoint

cmd, contracts/registrar: loose restriction of registeration

cmd, contracts: add replay-protection

all: off-chain multi-signature contract

params: deploy checkpoint contract for rinkeby

cmd/registrar: add raw signing mode for registrar

cmd/registrar, contracts/registrar, les: fixed messages

* cmd/registrar, contracts/registrar: fix lints

* accounts/abi/bind, les: address comments

* cmd, contracts, les, light, params: minor checkpoint sync cleanups

* cmd, eth, les, light: move checkpoint config to config file

* cmd, eth, les, params: address comments

* eth, les, params: address comments

* cmd: polish up the checkpoint admin CLI

* cmd, contracts, params: deploy new version contract

* cmd/checkpoint-admin: add another flag for clef mode signing

* cmd, contracts, les: rename and regen checkpoint oracle with abigen
2019-06-28 10:34:02 +03:00
Péter Szilágyi
006c21efc7
cmd, core, eth, les, node: chain freezer on top of db rework 2019-05-16 10:39:29 +03:00
Péter Szilágyi
29bc982d75
cmd/geth, internal, node, vendor: nuke geth monitor 2019-04-05 12:13:56 +03:00
Steve Ruckdashel
a8dd1f93c6 node: switching prometheus flock location to tsdb (#19376)
* node: switching prometheus flock location to tsdb

* rookie mistake
2019-04-04 16:59:18 +03:00
gary rong
d5cae48bae accounts, cmd, internal: disable unlock account on open HTTP (#17037)
* cmd, accounts, internal, node, rpc, signer: insecure unlock protect

* all: strict unlock API by rpc

* cmd/geth: check before printing warning log

* accounts, cmd/geth, internal: tiny polishes
2019-04-04 14:03:10 +03:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Matthew Halpern
7b3e3d549e node: prefer nil slices over zero-length slices (#19083) 2019-02-18 13:31:22 +02:00
Janoš Guljaš
26aea73673 cmd, node, p2p/simulations: fix node account manager leak (#19004)
* node: close AccountsManager in new Close method

* p2p/simulations, p2p/simulations/adapters: handle node close on shutdown

* node: move node ephemeralKeystore cleanup to stop method

* node: call Stop in Node.Close method

* cmd/geth: close node.Node created with makeFullNode in cli commands

* node: close Node instances in tests

* cmd/geth, node: minor code style fixes

* cmd, console, miner, mobile: proper node Close() termination
2019-02-07 12:40:36 +02:00
Péter Szilágyi
af8daf91a6
node, rpc: log cleanups in ipc listener function (#18124)
node,rpc: remove unused log in ipc listener function
2018-12-10 13:15:03 +02:00
Franklin
21dd59bd04 . 2018-11-20 22:16:40 +08:00
Martin Holst Swende
493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
Felix Lange
dcae0d348b
p2p/simulations: fix a deadlock and clean up adapters (#17891)
This fixes a rare deadlock with the inproc adapter:

- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
  waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
  which waits for Network.lock.

The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.

Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.

In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.

A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
2018-10-11 20:32:14 +02:00
Ryan Schneider
5d7e18539e rpc: make HTTP RPC timeouts configurable, raise defaults (#17240)
* rpc: Make HTTP server timeout values configurable

* rpc: Remove flags for setting HTTP Timeouts, configuring via .toml is sufficient.

* rpc: Replace separate constants with a single default struct.

* rpc: Update HTTP Server Read and Write Timeouts to 30s.

* rpc: Remove redundant NewDefaultHTTPTimeouts function.

* rpc: document HTTPTimeouts.

* rpc: sanitize timeout values for library use
2018-07-31 12:16:14 +03:00
Kurkó Mihály
a9835c1816 cmd, dashboard, log: log collection and exploration (#17097)
* cmd, dashboard, internal, log, node: logging feature

* cmd, dashboard, internal, log: requested changes

* dashboard, vendor: gofmt, govendor, use vendored file watcher

* dashboard, log: gofmt -s -w, goimports

* dashboard, log: gosimple
2018-07-11 10:59:04 +03:00
gary rong
7beccb29be all: get rid of error when creating memory database (#16716)
* all: get rid of error when create mdb

* core: clean up variables definition

* all: inline mdb definition
2018-05-09 15:24:25 +03:00
Felix Lange
52b046c9b6
rpc: clean up IPC handler (#16524)
This avoids logging accept errors on shutdown and removes
a bit of duplication. It also fixes some goimports lint warnings.
2018-04-18 12:27:20 +02:00
Martin Holst Swende
ec3db0f56c cmd/clef, signer: initial poc of the standalone signer (#16154)
* signer: introduce external signer command

* cmd/signer, rpc: Implement new signer. Add info about remote user to Context

* signer: refactored request/response, made use of urfave.cli

* cmd/signer: Use common flags

* cmd/signer: methods to validate calldata against abi

* cmd/signer: work on abi parser

* signer: add mutex around UI

* cmd/signer: add json 4byte directory, remove passwords from api

* cmd/signer: minor changes

* cmd/signer: Use ErrRequestDenied, enable lightkdf

* cmd/signer: implement tests

* cmd/signer: made possible for UI to modify tx parameters

* cmd/signer: refactors, removed channels in ui comms, added UI-api via stdin/out

* cmd/signer: Made lowercase json-definitions, added UI-signer test functionality

* cmd/signer: update documentation

* cmd/signer: fix bugs, improve abi detection, abi argument display

* cmd/signer: minor change in json format

* cmd/signer: rework json communication

* cmd/signer: implement mixcase addresses in API, fix json id bug

* cmd/signer: rename fromaccount, update pythonpoc with new json encoding format

* cmd/signer: make use of new abi interface

* signer: documentation

* signer/main: remove redundant  option

* signer: implement audit logging

* signer: create package 'signer', minor changes

* common: add 0x-prefix to mixcaseaddress in json marshalling + validation

* signer, rules, storage: implement rules + ephemeral storage for signer rules

* signer: implement OnApprovedTx, change signing response (API BREAKAGE)

* signer: refactoring + documentation

* signer/rules: implement dispatching to next handler

* signer: docs

* signer/rules: hide json-conversion from users, ensure context is cleaned

* signer: docs

* signer: implement validation rules, change signature of call_info

* signer: fix log flaw with string pointer

* signer: implement custom 4byte databsae that saves submitted signatures

* signer/storage: implement aes-gcm-backed credential storage

* accounts: implement json unmarshalling of url

* signer: fix listresponse, fix gas->uint64

* node: make http/ipc start methods public

* signer: add ipc capability+review concerns

* accounts: correct docstring

* signer: address review concerns

* rpc: go fmt -s

* signer: review concerns+ baptize Clef

* signer,node: move Start-functions to separate file

* signer: formatting
2018-04-16 15:04:32 +03:00
GuiltyMorishita
e2f2bb3e2e node: fix typo hvosts -> vhosts (#16096) 2018-02-15 12:38:39 +02:00
Martin Holst Swende
6c6247a690
node, rpc: fix linter issues 2018-02-12 14:12:55 +01:00
Martin Holst Swende
589b603a9b rpc: dns rebind protection (#15962)
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks

* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d

* rpc: add wildcard support for rpcallowedhosts + go fmt

* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check

* http, rpc, utils: make vhosts into map, address review concerns

* node: change log messages to use geth standard (not sprintf)

* rpc: fix spelling
2018-02-12 14:52:07 +02:00
Lewis Marshall
54aeb8e4c0 p2p/simulations: various stability fixes (#15198)
p2p/simulations: introduce dialBan

- Refactor simulations/network connection getters to support
  avoiding simultaneous dials between two peers If two peers dial
  simultaneously, the connection will be dropped to help avoid
  that, we essentially lock the connection object with a
  timestamp which serves as a ban on dialing for a period of time
  (dialBanTimeout).

- The connection getter InitConn can be wrapped and passed to the
  nodes via adapters.NodeConfig#Reachable field and then used by
  the respective services when they initiate connections. This
  massively stablise the emerging connectivity when running with
  hundreds of nodes bootstrapping a network.

p2p: add Inbound public method to p2p.Peer

p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.

p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
2017-12-01 12:49:04 +01:00
Lewis Marshall
9feec51e2d p2p: add network simulation framework (#14982)
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
2017-09-25 10:08:07 +02:00
Felix Lange
7e57fee355 node: fix instance dir locking and improve error message
The lock file was ineffective because opening leveldb storage in
read-only mode doesn't really take the lock. Fix it by including a
dedicated flock library (which is actually split out of goleveldb).
2017-08-18 12:14:00 +02:00
Chase Wright
1802682f65 node: Rename TrusterNodes (#14827)
* node: Rename TrusterNodes

* node: Rename TrusterNodes
2017-07-18 11:58:46 +03:00
Victor Farazdagi
5f7eb78918 node: fixes deadlock on Wait() 2017-04-25 18:04:02 +03:00
bas-vk
5e29f4be93 cmd/utils, node: remove unused solc references and improve RPC config (#14324)
Currently http cors and websocket origins are a comma separated string in the
config object. These are replaced with string arrays that are more expressive in
case of a config file.
2017-04-12 23:04:14 +02:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Péter Szilágyi
24b9860c1b cmd/geth, node: surface geth architecture into version (#13866) 2017-04-05 21:51:01 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00