Commit Graph

13482 Commits

Author SHA1 Message Date
rjl493456442
538a868384
core/rawdb, cmd, ethdb, eth: implement freezer tail deletion (#23954)
* core/rawdb, cmd, ethdb, eth: implement freezer tail deletion

* core/rawdb: address comments from martin and sina

* core/rawdb: fixes cornercase in tail deletion

* core/rawdb: separate metadata into a standalone file

* core/rawdb: remove unused code

* core/rawdb: add random test

* core/rawdb: polish code

* core/rawdb: fsync meta file before manipulating the index

* core/rawdb: fix typo

* core/rawdb: address comments
2022-03-10 09:37:23 +01:00
rjl493456442
8c8a9e5ca1
core, ethdb, tests, trie: introduce database snapshot (#24486) 2022-03-10 09:35:22 +01:00
Felix Lange
5079e3c6e5
cmd/geth: make authrpc listening address settable from command line (#24522)
The default listening address "localhost" is not sufficient when running
geth in Docker.
2022-03-09 14:45:39 +01:00
Qian Bin
65ed1a6871
rlp, trie: faster trie node encoding (#24126)
This change speeds up trie hashing and all other activities that require
RLP encoding of trie nodes by approximately 20%. The speedup is achieved by
avoiding reflection overhead during node encoding.

The interface type trie.node now contains a method 'encode' that works with
rlp.EncoderBuffer. Management of EncoderBuffers is left to calling code.
trie.hasher, which is pooled to avoid allocations, now maintains an
EncoderBuffer. This means memory resources related to trie node encoding
are tied to the hasher pool.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 14:45:17 +01:00
Justus
d1f6a9f544
core/types: improve error for too short transaction / receipt encoding (#24256)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 10:44:53 +01:00
Zachinquarantine
19c2c60bbe
mobile: remove deprecated Stop function (#24369) 2022-03-09 00:42:14 +01:00
Ceyhun Onur
8401e4277a
core/rawdb: add HasCode, HashTrieNode and use them where possible (#24454) 2022-03-09 00:39:34 +01:00
uji
ec64358ac9
crypto/bn256/cloudflare: fix asm for dynamic linking (#24476)
When using -buildmode=shared, R15 is clobbered by a global variable
access; use a different register instead.

Fixes: #24439
2022-03-09 00:23:13 +01:00
Péter Szilágyi
48605b5f61
core/statedb: always clear out access list when setting a new one 2022-03-08 12:00:29 +02:00
Ikko Ashimine
0a4ec1dde5
crypto/bls12381: fix typo in comment (#24509) 2022-03-08 10:48:25 +01:00
Felix Lange
870b4505a0
p2p: define DiscReason as uint8 (#24507)
All other implementations store disconnect reasons as a single byte,
so go-ethereum should do it too.
2022-03-07 18:25:45 +01:00
Olivier H
a79afd9ac3
cmd: allow file descriptor limit to be set via CLI (#24477)
* eth, cmd: allow FdLimit to be set in config/command line (#24148)

* eth/ethconfig: format code

* cmd, eth/ethconfig: simplify fdlimit arg, disallow toml

* cnd/utils: make fdlimit setting nicer on the logs

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-03-07 10:21:06 +02:00
Martin Holst Swende
4860e50e05
cmd/geth, node, rpc: implement jwt tokens (#24364)
* rpc, node: refactor request validation and add jwt validation

* node, rpc: fix error message, ignore engine api in RegisterAPIs

* node: make authenticated port configurable

* eth/catalyst: enable unauthenticated version of engine api

* node: rework obtainjwtsecret (backport later)

* cmd/geth: added auth port flag

* node: happy lint, happy life

* node: refactor authenticated api

Modifies the authentication mechanism to use default values

* node: trim spaces and newline away from secret

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-03-07 09:30:27 +02:00
Daniel Perez
37f9d25ba0
signer/fourbyte: add support for nested types in selectors (#24407)
This replaces the simple selector parser in signer/fourbyte with one that
can actually handle most types. The new parser is added in accounts/abi
to also make it useable elsewhere.
2022-03-04 13:39:09 +01:00
uji
8fddf27a98
internal/build: switch azure sdk from Azure/azure-storage-blob-go to Azure/azure-sdk-for-go/sdk/storage/azblob. (#24473)
* go.mod: update azure-storage-blob-go

update Azure/azure-storage-blob-go from v0.7.0 to v0.14.0.
relation #24396.

* internal/build: fix for breaking changes of azure-storage-blob-go

fix for breaking changes of update Azure/azure-storage-blob-go from v0.7.0 to v0.14.0.
relation #24396.

* internal/build: switch azure sdk from Azure/azure-storage-blob-go to Azure/azure-sdk-for-go/sdk/storage/azblob.

* internal/build refactor appending BlobItems

* internal/build: fix azure blobstore client to include container id

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-03-02 13:05:14 +02:00
s7v7nislands
f4ff4268f7
rpc: fix godoc (#24488)
Co-authored-by: seven <seven@nodereal.io>
2022-03-02 02:51:55 +01:00
Yash Atwal
7307d97ae1
README.md: add hardware requirements (#24395)
Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-01 11:00:44 +01:00
Zachinquarantine
6662c78ec0
internal/flags: update copyright year (#24415) 2022-03-01 10:53:19 +01:00
Felföldi Zsolt
7033724522
les/vflux/client: fix goroutine leak in testIter (#24449) 2022-03-01 10:36:42 +01:00
Guruprasad Kamath
03b7de28b2
appveyor.yml: fetch sub-modules recursively (#24451) 2022-03-01 10:36:11 +01:00
ucwong
687e4dc855
rlp: add WriteString method on EncoderBuffer (#24425)
rlpgen outputs calls to this method for values of type string.
2022-02-24 22:55:45 +01:00
Martin Holst Swende
0cb4d65f8d
params: replace 2 bootnodes (#24432) 2022-02-23 11:33:14 +01:00
Sina Mahmoodi
862f8e98bc
graphql: fix nonce for pending accounts (#24443) 2022-02-22 10:45:11 +01:00
Felix Lange
d6f49bf764
core/types: faster RLP encoding of Header, StateAcccount, ReceiptForStorage (#24420)
This change makes use of the new code generator rlp/rlpgen to improve the
performance of RLP encoding for Header and StateAccount. It also speeds up
encoding of ReceiptForStorage using the new rlp.EncoderBuffer API.

The change is much less transparent than I wanted it to be, because Header and
StateAccount now have an EncodeRLP method defined with pointer receiver. It
used to be possible to encode non-pointer values of these types, but the new
method prevents that and attempting to encode unadressable values (even if
part of another value) will return an error. The error can be surprising and may
pop up in places that previously didn't expect any errors.

To make things work, I also needed to update all code paths (mostly in unit tests)
that lead to encoding of non-pointer values, and pass a pointer instead.

Benchmark results:

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           328ns ± 0%     237ns ± 1%   -27.63%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8           353ns ± 0%     247ns ± 1%   -30.06%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     237ns ± 0%     123ns ± 0%   -47.86%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8            297ns ± 0%     301ns ± 1%    +1.39%  (p=0.000 n=8+8)

    name                             old speed      new speed      delta
    EncodeRLP/legacy-header-8        1.66GB/s ± 0%  2.29GB/s ± 1%   +38.19%  (p=0.000 n=8+8)
    EncodeRLP/london-header-8        1.55GB/s ± 0%  2.22GB/s ± 1%   +42.99%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8  38.0MB/s ± 0%  64.8MB/s ± 0%   +70.48%  (p=0.000 n=8+7)
    EncodeRLP/receipt-full-8          910MB/s ± 0%   897MB/s ± 1%    -1.37%  (p=0.000 n=8+8)

    name                             old alloc/op   new alloc/op   delta
    EncodeRLP/legacy-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/london-header-8           0.00B          0.00B           ~     (all equal)
    EncodeRLP/receipt-for-storage-8     64.0B ± 0%      0.0B       -100.00%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8             320B ± 0%      320B ± 0%      ~     (all equal)
2022-02-18 08:10:26 +01:00
Felix Lange
06aaeed1a6
build: add Ubuntu 21.10 PPA target (#24418)
Also specify EOL dates of all listed releases.
2022-02-17 18:35:33 +01:00
Felix Lange
9b93564e21
rlp/rlpgen: RLP encoder code generator (#24251)
This change adds a code generator tool for creating EncodeRLP method
implementations. The generated methods will behave identically to the
reflect-based encoder, but run faster because there is no reflection overhead.

Package rlp now provides the EncoderBuffer type for incremental encoding. This
is used by generated code, but the new methods can also be useful for
hand-written encoders.

There is also experimental support for generating DecodeRLP, and some new
methods have been added to the existing Stream type to support this. Creating
decoders with rlpgen is not recommended at this time because the generated
methods create very poor error reporting.

More detail about package rlp changes:

* rlp: externalize struct field processing / validation

This adds a new package, rlp/internal/rlpstruct, in preparation for the
RLP encoder generator.

I think the struct field rules are subtle enough to warrant extracting
this into their own package, even though it means that a bunch of
adapter code is needed for converting to/from rlpstruct.Type.

* rlp: add more decoder methods (for rlpgen)

This adds new methods on rlp.Stream:

- Uint64, Uint32, Uint16, Uint8, BigInt
- ReadBytes for decoding into []byte
- MoreDataInList - useful for optional list elements

* rlp: expose encoder buffer (for rlpgen)

This exposes the internal encoder buffer type for use in EncodeRLP
implementations.

The new EncoderBuffer type is a sort-of 'opaque handle' for a pointer to
encBuffer. It is implemented this way to ensure the global encBuffer pool
is handled correctly.
2022-02-16 18:14:12 +01:00
Martin Holst Swende
4335bbbf0a
build: disable Ubuntu Hirsuite (#24408)
This Ubuntu release has reached EOL and Launchpad does not
accept uploads for it anymore.
2022-02-16 17:38:21 +01:00
Péter Szilágyi
fc8ad1b70d
Merge pull request #24391 from rjl493456442/trie-iterator
trie: implement NodeBlob API for trie iterator
2022-02-15 15:35:47 +02:00
rjl493456442
4d086430bd
core, ethdb, tests, trie: implement NewBatchWithSize API for batcher (#24392)
This PR adds an addtional API called `NewBatchWithSize` for db
batcher. It turns out that leveldb batch memory allocation is
super inefficient. The main reason is the allocation step of
leveldb Batch is too small when the batch size is large. It can
take a few second to build a leveldb batch with 100MB size.

Luckily, leveldb also offers another API called MakeBatch which can
pre-allocate the memory area. So if the approximate size of batch is
known in advance, this API can be used in this case.

It's needed in new state scheme PR which needs to commit a batch of
trie nodes in a single batch. Implement the feature in a seperate PR.
2022-02-15 15:15:13 +02:00
Martin Holst Swende
2056e596f2
params: begin v1.10.17 release cycle 2022-02-15 13:34:12 +01:00
Martin Holst Swende
20356e57b1
params: go-ethereum v1.10.16 stable 2022-02-15 13:32:24 +01:00
zhiqiangxu
e98114da4f
ethclient: add CallContractAtHash (#24355)
* add CallContractAtHash to ethclient

* add docstring and test

* optimize test

* ethclient: nits

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-02-15 11:55:55 +02:00
b1ackd0t
f01e2fab07
internal/ethapi: fix incorrect type on empty slice (#24372)
* Fixes #24368

Signed-off-by: 0x6f736f646f <blackd0t@protonmail.com>

* Update internal/ethapi/api.go

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-02-15 10:44:22 +01:00
Gary Rong
55430b6ea2 trie: implement NodeBlob API for trie iterator
This functionality is needed in new path-based storage scheme, but
can be implemented in a seperate PR though.

When an account is deleted, then all the storage slots should be
nuked out from the disk as well. In hash-based storage scheme they
are still left in the disk but in new scheme, they will be iterated
and marked as deleted.

But why the NodeBlob API is needed in this scenario? Because when
the node is marked deleted, the previous value is also required to
be recorded to construct the reverse diff.
2022-02-15 16:12:17 +08:00
zhiqiangxu
6c3513c077
p2p: reduce the scope of variable dialPubkey (#24385)
dialPubkey isn't used anywhere else after dialDest.Load, so it should be safe to
restrict its scope to the if clause.
2022-02-14 12:05:48 +01:00
Martin Holst Swende
51e7968b8b
core/state: fix read-meters + simplify code (#24304) 2022-02-14 09:22:57 +01:00
Felix Lange
fb3a6528cf
go.mod: upgrade to github.com/karalabe/usb v0.0.2 (#24356)
This upgrade resolves a build issue on Android 7+ and removes
a build warning on macOS >= 12.0.
2022-02-08 19:58:27 +01:00
Martin Holst Swende
5a0d487c3b
signer/core: fix complex typed data sign (EIP712) (#24220)
Co-authored-by: specerxi <xhxpecer@gmail.com>
2022-02-08 14:40:00 +01:00
rjl493456442
2d20fed893
miner: avoid data race in miner (#24349) 2022-02-07 18:34:13 +01:00
Martin Holst Swende
6ce4670bc0
cmd/devp2p: implement snap protocol testing (#24276)
This also contains some changes to the protocol handler to
make the tests pass.
2022-02-04 15:24:32 +01:00
aaronbuchwald
aaca58a7a1
go.mode: bump graphql-go dependency to v1.3.0 (#24324) 2022-02-02 18:21:25 +02:00
Péter Szilágyi
1a7e345af4
Merge pull request #24328 from karalabe/uke-catalyst
cmd: auto-enable beacon APIs if TTD is defined
2022-02-02 18:15:38 +02:00
Péter Szilágyi
d99e759e76
cmd: auto-enable beacon APIs if TTD is defined 2022-02-02 17:57:34 +02:00
komika
afe344bcf3
accounts/abi/bind: improve WaitMined error handling (#24321)
This change makes it so WaitMined no longer logs an error when the receipt
is unavailable. It also changes the simulated backend to return NotFound for
unavailable receipts, just like ethclient does.
2022-02-01 16:42:51 +01:00
Sina Mahmoodi
c5436c8eb7
eth/tracers: clean-up tracer collection (#24320)
* eth/tracers: clean-up tracer collection

* Rm test for dropped tracer
2022-02-01 10:44:44 +01:00
Brandon Harden
b868ca1790
accounts: correct spelling mistake (#24323)
I believe the sentence is attempting to explain that the URL is "[used] by upper layers to define a sorting order over all wallets from multiple backends."
2022-02-01 11:36:39 +02:00
rjl493456442
9da25c5db7
all: separate catalyst package (#24280)
* all: seperate catalyst package

* eth/catalyst: moved some methods, added docs

* eth/catalyst, les/catalyst: add method docs

* core, eth, les, miner: move common function to beacon package

* eth/catalyst: goimported

* cmd/utils, miner/stress/beacon: naming nitpicks

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2022-01-31 14:22:35 +02:00
ligi
a5c0cfb451
build: fix lint on ARM (#24311)
* build: append GOARM to arm lint download URL

otherwise it fails with:

downloading from https://github.com/golangci/golangci-lint/releases/download/v1.42.0/golangci-lint-1.42.0-linux-arm.tar.gz
ci.go:347: download error: status 404

* build: increase timeout for lint

Otherwise it times out on a pi

* Increase timeout even further

saw longer build times
2022-01-31 14:17:18 +02:00
Sina Mahmoodi
cac09a3823
eth/tracers: native prestate tracer (#24268)
* eth/tracers: add initial native prestate tracer

* fix balance hex

* handle prestate for tx from and to

* drop created contract from prestate

* fix sender balance

* use switch instead

Co-authored-by: Martin Holst Swende <martin@swende.se>

* minor fix

* lookup create2 account

* mv code around a bit

* check stackLen for create2

* fix transfer tx for js prestate tracer

* fix create2 addr

* track extcodehash in js prestate tracer

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-31 09:51:17 +01:00
ligi
0c1bd22ec0
cmd/geth: make test pass on a pi4 by using lightkdf (#24314) 2022-01-31 09:50:10 +01:00