Commit Graph

104 Commits

Author SHA1 Message Date
Péter Szilágyi
d8da0b3d81
core/state, eth, trie: stabilize memory use, fix memory leak 2020-08-26 13:05:06 +03:00
gary rong
6eef141aef
les: historical data garbage collection (#19570)
This change introduces garbage collection for the light client. Historical
chain data is deleted periodically. If you want to disable the GC, use
the --light.nopruning flag.
2020-07-13 11:02:54 +02:00
Martin Holst Swende
9b09c0fc83 * trie: utilize callbacks instead of amassing lists in ref/unref (#20529)
* trie/tests: add benchmarks and update trie tests

* trie: update benchmark tests

* trie: utilize callbacks instead of amassing lists of hashes in database ref/unref

* trie: replace remaining non-callback based accesses
2020-01-17 13:59:45 +02:00
Felix Lange
0abcf03fde trie: remove unused code (#20366) 2019-11-22 17:24:48 +02:00
Christian Muehlhaeuser
5183483c53 core/state, p2p/discover, trie, whisper: avoid unnecessary conversions (#19870)
No need to convert these types.
2019-07-22 10:30:09 +03:00
Péter Szilágyi
91eec1251c
cmd, core, eth, trie: get rid of trie cache generations (#19262)
* cmd, core, eth, trie: get rid of trie cache generations

* core, trie: get rid of remainder of cache gen boilerplate
2019-03-14 15:25:12 +02:00
Péter Szilágyi
8111b9dda5
ethdb, trie: tiny API tidy-up from the database rework pr 2019-03-12 12:32:02 +02:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Martin Holst Swende
4f85c2b88b
trie: fix error in node decoding (#19111) 2019-02-16 16:16:12 +01:00
Péter Szilágyi
434dd5bc00
cmd, core, eth, light, trie: add trie read caching layer 2018-11-15 12:22:13 +02:00
gary rong
7beccb29be all: get rid of error when creating memory database (#16716)
* all: get rid of error when create mdb

* core: clean up variables definition

* all: inline mdb definition
2018-05-09 15:24:25 +03:00
Péter Szilágyi
55599ee95d core, trie: intermediate mempool between trie and database (#15857)
This commit reduces database I/O by not writing every state trie to disk.
2018-02-05 17:40:32 +01:00
Péter Szilágyi
a5fcaa55ac
trie: make hasher benchmark meaningful post-caches 2017-10-13 11:22:38 +03:00
Felix Lange
693d9ccbfb trie: more node iterator improvements (#14615)
* ethdb: remove Set

Set deadlocks immediately and isn't part of the Database interface.

* trie: add Err to Iterator

This is useful for testing because the underlying NodeIterator doesn't
need to be kept in a separate variable just to get the error.

* trie: add LeafKey to iterator, panic when not at leaf

LeafKey is useful for callers that can't interpret Path.

* trie: retry failed seek/peek in iterator Next

Instead of failing iteration irrecoverably, make it so Next retries the
pending seek or peek every time.

Smaller changes in this commit make this easier to test:

* The iterator previously returned from Next on encountering a hash
  node. This caused it to visit the same path twice.
* Path returned nibbles with terminator symbol for valueNode attached
  to fullNode, but removed it for valueNode attached to shortNode. Now
  the terminator is always present. This makes Path unique to each node
  and simplifies Leaf.

* trie: add Path to MissingNodeError

The light client trie iterator needs to know the path of the node that's
missing so it can retrieve a proof for it. NodeIterator.Path is not
sufficient because it is updated when the node is resolved and actually
visited by the iterator.

Also remove unused fields. They were added a long time ago before we
knew which fields would be needed for the light client.
2017-06-20 18:26:09 +02:00
Felix Lange
4047ccad2f trie: add start key to NodeIterator constructors
The 'step' method is split into two parts, 'peek' and 'push'. peek
returns the next state but doesn't make it current.

The end of iteration was previously tracked by setting 'trie' to nil.
End of iteration is now tracked using the 'iteratorEnd' error, which is
slightly cleaner and requires less code.
2017-04-25 02:14:31 +02:00
Felix Lange
a13e920af0 trie: clean up iterator constructors
Make it so each iterator has exactly one public constructor:

- NodeIterators can be created through a method.
- Iterators can be created through NewIterator on any NodeIterator.
2017-04-25 02:14:31 +02:00
Felix Lange
f1534f5797 trie, whisper/whisperv5: use math/rand Read function 2017-03-22 20:49:15 +01:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Felix Lange
8d56bf5ceb trie: ensure dirty flag is unset for embedded child nodes
This was caught by the new invariant check.
2016-10-18 04:57:47 +02:00
Felix Lange
177cab5fe7 trie: ensure resolved nodes stay loaded
Commit 40cdcf1183 broke the optimisation which kept nodes resolved
during Get in the trie. The decoder assigned cache generation 0
unconditionally, causing resolved nodes to get flushed on Commit.

This commit fixes it and adds two tests.
2016-10-18 04:57:47 +02:00
Felix Lange
40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Felix Lange
c3a77d6268 trie: fix delete bug for values contained in fullNode
Delete crashed if a fullNode contained a valueNode directly. This bug is
very unlikely to occur with SecureTrie, but can happen with regular
tries. This commit also introduces a randomised test which triggers all
trie operations, which should prevent such bugs in the future.

Credit for finding this bug goes to Github user @rjl493456442.
2016-09-29 22:53:59 +02:00
Felix Lange
cd791bd855 core, trie: replace state caches with trie journal 2016-09-28 11:27:28 +03:00
Péter Szilágyi
748d1c171d core, core/state, trie: enterprise hand-tuned multi-level caching 2016-05-26 16:33:09 +03:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Péter Szilágyi
e90958cd29 cmd, eth, ethdb, node: prioritise chaindata for resources, bump cache 2016-03-09 10:33:39 +02:00
zsfelfoldi
52904ae32f trie: added error handling
Created alternate versions of Trie and SecureTrie functions that can return a MissingNodeError (used by ODR services)
2015-12-01 01:38:32 +01:00
Felix Lange
565d9f2306 core, trie: new trie 2015-09-22 22:53:49 +02:00
Felix Lange
6b91a4abe5 trie: improve benchmarks 2015-09-22 22:49:27 +02:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Jeffrey Wilcke
0a1ff68c11 trie: dirty tracking 2015-07-04 02:51:36 +02:00
obscuren
a40a91d60f trie: fixed tests 2015-06-21 17:08:47 +02:00
obscuren
d7ab716eea Fixed mkdnode & added some tests 2015-03-19 10:57:02 +01:00
obscuren
b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
obscuren
d44fe4ec62 Merge branch 'develop' into poc-9 2015-03-05 17:46:40 +01:00
obscuren
357d17ae58 Start at 0 2015-03-05 17:46:00 +01:00
obscuren
bf5365b317 Delet => Delete 2015-03-03 12:15:58 +01:00
obscuren
ae45a39dc1 Secure trie 2015-02-28 20:52:29 +01:00
obscuren
99ebb869bf Fixed iterator 2015-02-07 17:03:22 +01:00
obscuren
db4aaedcbd Moved ptrie => trie. Removed old trie 2015-01-08 11:47:04 +01:00
obscuren
fed3e6a808 Refactored ethutil.Config.Db out 2015-01-07 13:17:48 +01:00
obscuren
93e693be72 Fixed tests for 'types' 2014-11-18 19:52:45 +01:00
obscuren
675ba4d7eb Merge branch 'tests' into poc8 2014-11-18 19:48:35 +01:00
obscuren
62cd9946ee j => i 2014-11-18 12:20:14 +01:00
obscuren
e08aba5dd9 added output test 2014-11-18 12:18:27 +01:00
obscuren
3220a32ff0 Added some comparison tests for the new ptrie 2014-11-18 12:03:09 +01:00
Taylor Gerring
ddca18638f update imports order per goimports 2014-11-15 20:21:55 -06:00
Taylor Gerring
e6e468ef59 Merge branch 'develop' of https://github.com/tgerring/go-ethereum into tests
Conflicts:
	p2p/messenger_test.go
	trie/trie_test.go
2014-11-14 15:19:50 -06:00
Taylor Gerring
2a9fc7baa9 Merge branch 'develop' of https://github.com/tgerring/go-ethereum 2014-11-14 15:01:52 -06:00
Taylor Gerring
00878e5b6e Convert trie tests to gocheck 2014-11-12 21:43:25 +01:00
obscuren
60cdb1148c Transaction execution fixes 2014-11-12 01:36:36 +01:00
obscuren
af34749a6b ethtrie => trie 2014-10-31 14:45:03 +01:00