Commit Graph

32 Commits

Author SHA1 Message Date
Martin Holst Swende
58d0f6440b
rlp: support for uint256 (#26898)
This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 06:51:55 -04:00
Roberto Bayardo
4a81e5afea
rlp: add more tests for nil pointer / optional field encoding (#26077) 2022-11-02 14:50:48 +01:00
lmittmann
bc013bc42e
all: prefer new(big.Int) over big.NewInt(0) (#25087)
minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 15:09:48 +03:00
Håvard Anda Estensen
07508ac0e9
all: replace uses of ioutil with io and os (#24869) 2022-05-16 11:59:35 +02:00
Qian Bin
65ed1a6871
rlp, trie: faster trie node encoding (#24126)
This change speeds up trie hashing and all other activities that require
RLP encoding of trie nodes by approximately 20%. The speedup is achieved by
avoiding reflection overhead during node encoding.

The interface type trie.node now contains a method 'encode' that works with
rlp.EncoderBuffer. Management of EncoderBuffers is left to calling code.
trie.hasher, which is pooled to avoid allocations, now maintains an
EncoderBuffer. This means memory resources related to trie node encoding
are tied to the hasher pool.

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 14:45:17 +01:00
Felix Lange
9b93564e21
rlp/rlpgen: RLP encoder code generator (#24251)
This change adds a code generator tool for creating EncodeRLP method
implementations. The generated methods will behave identically to the
reflect-based encoder, but run faster because there is no reflection overhead.

Package rlp now provides the EncoderBuffer type for incremental encoding. This
is used by generated code, but the new methods can also be useful for
hand-written encoders.

There is also experimental support for generating DecodeRLP, and some new
methods have been added to the existing Stream type to support this. Creating
decoders with rlpgen is not recommended at this time because the generated
methods create very poor error reporting.

More detail about package rlp changes:

* rlp: externalize struct field processing / validation

This adds a new package, rlp/internal/rlpstruct, in preparation for the
RLP encoder generator.

I think the struct field rules are subtle enough to warrant extracting
this into their own package, even though it means that a bunch of
adapter code is needed for converting to/from rlpstruct.Type.

* rlp: add more decoder methods (for rlpgen)

This adds new methods on rlp.Stream:

- Uint64, Uint32, Uint16, Uint8, BigInt
- ReadBytes for decoding into []byte
- MoreDataInList - useful for optional list elements

* rlp: expose encoder buffer (for rlpgen)

This exposes the internal encoder buffer type for use in EncodeRLP
implementations.

The new EncoderBuffer type is a sort-of 'opaque handle' for a pointer to
encBuffer. It is implemented this way to ensure the global encBuffer pool
is handled correctly.
2022-02-16 18:14:12 +01:00
Felix Lange
32c576bd3c
rlp: minor optimizations for slice/array encoding (#23467)
As per benchmark results below, these changes speed up encoding/decoding of
consensus objects a bit.

    name                             old time/op    new time/op    delta
    EncodeRLP/legacy-header-8           384ns ± 1%     331ns ± 3%  -13.83%  (p=0.000 n=7+8)
    EncodeRLP/london-header-8           411ns ± 1%     359ns ± 2%  -12.53%  (p=0.000 n=8+8)
    EncodeRLP/receipt-for-storage-8     251ns ± 0%     239ns ± 0%   -4.97%  (p=0.000 n=8+8)
    EncodeRLP/receipt-full-8            319ns ± 0%     300ns ± 0%   -5.89%  (p=0.000 n=8+7)
    EncodeRLP/legacy-transaction-8      389ns ± 1%     387ns ± 1%     ~     (p=0.099 n=8+8)
    EncodeRLP/access-transaction-8      607ns ± 0%     581ns ± 0%   -4.26%  (p=0.000 n=8+8)
    EncodeRLP/1559-transaction-8        627ns ± 0%     606ns ± 1%   -3.44%  (p=0.000 n=8+8)
    DecodeRLP/legacy-header-8           831ns ± 1%     813ns ± 1%   -2.20%  (p=0.000 n=8+8)
    DecodeRLP/london-header-8           824ns ± 0%     804ns ± 1%   -2.44%  (p=0.000 n=8+7)

* rlp: pass length to byteArrayBytes

This makes it possible to inline byteArrayBytes. For arrays, the length is known
at encoder construction time, so the call to v.Len() can be avoided.

* rlp: avoid IsNil for pointer encoding

It's actually cheaper to use Elem first, because it performs less checks
on the value. If the pointer was nil, the result of Elem is 'invalid'.

* rlp: minor optimizations for slice/array encoding

For empty slices/arrays, we can avoid storing a list header entry in the
encoder buffer. Also avoid doing the tail check at encoding time because
it is already known at encoder construction time.
2021-08-25 19:01:10 +02:00
Felix Lange
4d33de9b49
rlp: optimize big.Int decoding for size <= 32 bytes (#22927)
This change grows the static integer buffer in Stream to 32 bytes,
making it possible to decode 256bit integers without allocating a
temporary buffer.

In the recent commit 088da24, Stream struct size decreased from 120
bytes down to 88 bytes. This commit grows the struct to 112 bytes again,
but the size change will not degrade performance because Stream
instances are internally cached in sync.Pool.

    name             old time/op    new time/op    delta
    DecodeBigInts-8    12.2µs ± 0%     8.6µs ± 4%  -29.58%  (p=0.000 n=9+10)

    name             old speed      new speed      delta
    DecodeBigInts-8   230MB/s ± 0%   326MB/s ± 4%  +42.04%  (p=0.000 n=9+10)
2021-05-25 21:56:25 +02:00
Felix Lange
154ca32a8a
rlp: optimize byte array handling (#22924)
This change improves the performance of encoding/decoding [N]byte.

    name                     old time/op    new time/op    delta
    DecodeByteArrayStruct-8     336ns ± 0%     246ns ± 0%  -26.98%  (p=0.000 n=9+10)
    EncodeByteArrayStruct-8     225ns ± 1%     148ns ± 1%  -34.12%  (p=0.000 n=10+10)

    name                     old alloc/op   new alloc/op   delta
    DecodeByteArrayStruct-8      120B ± 0%       48B ± 0%  -60.00%  (p=0.000 n=10+10)
    EncodeByteArrayStruct-8     0.00B          0.00B          ~     (all equal)
2021-05-22 15:10:16 +02:00
Felix Lange
0d076d92db
rlp: use atomic.Value for type cache (#22902)
All encoding/decoding operations read the type cache to find the
writer/decoder function responsible for a type. When analyzing CPU
profiles of geth during sync, I found that the use of sync.RWMutex in
cache lookups appears in the profiles. It seems we are running into
CPU cache contention problems when package rlp is heavily used
on all CPU cores during sync.

This change makes it use atomic.Value + a writer lock instead of
sync.RWMutex. In the common case where the typeinfo entry is present in
the cache, we simply fetch the map and lookup the type.
2021-05-22 13:34:29 +02:00
Felix Lange
700df1442d
rlp: add support for optional struct fields (#22832)
This adds support for a new struct tag "optional". Using this tag, structs used
for RLP encoding/decoding can be extended in a backwards-compatible way,
by adding new fields at the end.
2021-05-07 14:37:13 +02:00
Alex Prut
c92faee66e
all: simplify nested complexity and if blocks ending with a return statement (#21854)
Changes:

    Simplify nested complexity
    If an if blocks ends with a return statement then remove the else nesting.

Most of the changes has also been reported in golint https://goreportcard.com/report/github.com/ethereum/go-ethereum#golint
2020-11-25 09:24:50 +01:00
Felix Lange
6315b6fcc0
rlp: reduce allocations for big.Int and byte array encoding (#21291)
This change further improves the performance of RLP encoding by removing
allocations for big.Int and [...]byte types. I have added a new benchmark
that measures RLP encoding of types.Block to verify that performance is
improved.
2020-07-06 11:17:09 +02:00
Felix Lange
96fb839133
rlp: improve nil pointer handling (#20064)
* rlp: improve nil pointer handling

In both encoder and decoder, the rules for encoding nil pointers were a
bit hard to understand, and didn't leave much choice. Since RLP allows
two empty values (empty list, empty string), any protocol built on RLP
must choose either of these values to represent the null value in a
certain context.

This change adds choice in the form of two new struct tags, "nilString"
and "nilList". These can be used to specify how a nil pointer value is
encoded. The "nil" tag still exists, but its implementation is now
explicit and defines exactly how nil pointers are handled in a single
place.

Another important change in this commit is how nil pointers and the
Encoder interface interact. The EncodeRLP method was previously called
even on nil values, which was supposed to give users a choice of how
their value would be handled when nil. It turns out this is a stupid
idea. If you create a network protocol containing an object defined in
another package, it's better to be able to say that the object should be
a list or string when nil in the definition of the protocol message
rather than defining the encoding of nil on the object itself.

As of this commit, the encoding rules for pointers now take precedence
over the Encoder interface rule. I think the "nil" tag will work fine
for most cases. For special kinds of objects which are a struct in Go
but strings in RLP, code using the object can specify the desired
encoding of nil using the "nilString" and "nilList" tags.

* rlp: propagate struct field type errors

If a struct contained fields of undecodable type, the encoder and
decoder would panic instead of returning an error. Fix this by
propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 11:10:57 +02:00
Felix Lange
8deec2e45a
rlp: fixes for two corner cases and documentation (#19527)
These changes fix two corner cases related to internal handling of types
in package rlp: The "tail" struct tag can only be applied to the last field.
The check for this was wrong and didn't allow for private fields after the
field with the tag. Unsupported types (e.g. structs containing int) which
implement either the Encoder or Decoder interface but not both 
couldn't be encoded/decoded.

Also fixes #19367
2019-05-14 15:09:56 +02:00
Felix Lange
eee96a5bb7 rlp: add support for "-" struct tag 2017-03-07 12:45:12 +01:00
Felix Lange
bb07ce3eed rlp: add "tail" struct tag 2016-02-19 11:14:29 +01:00
Felix Lange
24bb68e7cf rlp: add RawValue 2015-09-10 19:41:51 +02:00
Felix Lange
ac32f52ca6 rlp: fix encReader returning nil buffers to the pool
The bug can cause crashes if Read is called after EOF has been returned.
No code performs such calls right now, but hitting the bug gets more
likely as rlp.EncodeToReader gets used in more places.
2015-09-10 19:12:32 +02:00
Péter Szilágyi
1d2420323c rlp: add support for boolean encoding/decoding 2015-08-13 12:05:39 +03:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Felix Lange
a0566c1058 rlp: remove Flat 2015-06-29 18:51:47 +02:00
Felix Lange
181a21c67c rlp: encode nil array pointers as empty list or string 2015-03-25 16:46:29 +01:00
Felix Lange
965c9babe3 rlp: fix encoding of one element strings and byte slices
The encoder was missing a special case for one element strings whose
element is below 0x7f. Such strings must be encoded as a single byte
without a string header.
2015-03-19 12:15:43 +01:00
Felix Lange
cb009a5c4d rlp: don't panic for nil *big.Int
All other pointer types can handle nil just fine.
2015-03-17 23:49:49 +01:00
obscuren
76fa75b394 wip 2015-02-13 15:35:54 +01:00
Felix Lange
b94f85de22 rlp: add Flat 2015-02-11 19:28:56 +01:00
Felix Lange
8c3095faf0 rlp: fix encoding of arrays with byte element type 2015-02-06 00:00:36 +01:00
Felix Lange
fc92abec2c rlp: allow encoding non-empty interface values
This needs to be supported because []someInterface does occur sometimes.

Funny enough, the fix involves changes to the decoder. makeDecoder
cannot return an error for non-empty interfaces anymore because the type
cache builds both decoder and writer. Do the check at 'runtime' instead.
2015-01-15 23:35:26 +01:00
Felix Lange
552f5b2693 rlp: add functions for encoding
I'm reasonably confident that the encoding matches the output of
ethutil.Encode for values that it supports. Some of the tests have been
adpated from the Ethereum testing repository.

There are still TODOs in the code.
2015-01-15 11:00:19 +01:00