Commit Graph

271 Commits

Author SHA1 Message Date
Felix Lange
40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Felix Lange
c3a77d6268 trie: fix delete bug for values contained in fullNode
Delete crashed if a fullNode contained a valueNode directly. This bug is
very unlikely to occur with SecureTrie, but can happen with regular
tries. This commit also introduces a randomised test which triggers all
trie operations, which should prevent such bugs in the future.

Credit for finding this bug goes to Github user @rjl493456442.
2016-09-29 22:53:59 +02:00
Felix Lange
ba8c4c6b1a trie: remove dependency on package crypto
Package crypto needs cgo, which is inconvenient for some build
configurations.
2016-09-29 22:23:24 +02:00
Péter Szilágyi
710435b51b core, eth, trie: reuse trie journals in all our code 2016-09-28 11:27:31 +03:00
Felix Lange
cd791bd855 core, trie: replace state caches with trie journal 2016-09-28 11:27:28 +03:00
Péter Szilágyi
1d5d21726a Merge pull request #2627 from karalabe/concurrent-head-sync
eth/downloader, trie: pull head state concurrently with chain
2016-05-31 11:52:16 +03:00
Péter Szilágyi
4f1d92b332 eth/downloader, trie: pull head state concurrently with chain 2016-05-27 14:26:00 +03:00
Péter Szilágyi
748d1c171d core, core/state, trie: enterprise hand-tuned multi-level caching 2016-05-26 16:33:09 +03:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Felix Lange
d04a2e7557 all: update license information 2016-04-15 09:48:05 +02:00
Péter Szilágyi
e90958cd29 cmd, eth, ethdb, node: prioritise chaindata for resources, bump cache 2016-03-09 10:33:39 +02:00
Jeffrey Wilcke
483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Ricardo Catalinas Jiménez
436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Péter Szilágyi
b8d59d9c98 core/state, trie: switch iterator panics to error fields 2016-02-16 12:37:00 +02:00
Péter Szilágyi
151c7bef41 core/state, trie: node iterator reports parent hashes too 2016-02-16 12:21:08 +02:00
Péter Szilágyi
5a057a8ded core/state, trie: surface iterator entry hashes 2016-02-16 12:21:08 +02:00
Péter Szilágyi
7e29b0b5b4 core/state, trie: add node iterator, test state/trie sync consistency 2016-02-16 12:21:08 +02:00
Péter Szilágyi
f3d4ce0d16 core/state, ethdb, trie: test intermediate secure key leak, fix memdb bug 2016-01-20 16:06:28 +02:00
Péter Szilágyi
886478b18b core/state, trie: don't leak database writes before commit 2016-01-20 12:09:24 +02:00
zsfelfoldi
ef422ee1e1 light: implemented odr-capable trie and state structures 2015-12-17 16:07:54 +01:00
zsfelfoldi
52904ae32f trie: added error handling
Created alternate versions of Trie and SecureTrie functions that can return a MissingNodeError (used by ODR services)
2015-12-01 01:38:32 +01:00
Péter Szilágyi
5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi
b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Péter Szilágyi
ab27bee25a core, eth, trie: direct state trie synchronization 2015-10-19 10:03:09 +03:00
Felix Lange
c1a352c108 trie: add merkle proof functions 2015-09-22 22:57:37 +02:00
Felix Lange
565d9f2306 core, trie: new trie 2015-09-22 22:53:49 +02:00
Felix Lange
6b91a4abe5 trie: improve benchmarks 2015-09-22 22:49:27 +02:00
Jeffrey Wilcke
b8ca0a830e eth, trie: removed key prefixing from state entries & merge db fix
Fixed database merge strategy to use the correct database. Due to a copy
paste fail when doing type evaluation the same database was being
iterated (chain), all others were ignored.

Removed state prefixing because {H(code): code} is stored in the same
database as the rest of the state.
2015-08-13 20:44:03 +02:00
Jeffrey Wilcke
a23478c0be core, eth, trie, xeth: merged state, chain, extra databases in one 2015-08-07 22:29:02 +02:00
Ethan Buchman
46c9594081 trie: run codec tests, add benchmarks, faster 2015-08-06 14:04:16 -04:00
Ethan Buchman
c1d516546d faster hex-prefix codec and string -> []byte 2015-08-06 03:17:59 -04:00
Jeffrey Wilcke
1fad8798ec Merge pull request #1515 from fjl/license-fixes
all: fix license headers one more time
2015-07-28 04:29:42 -07:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Jeffrey Wilcke
8f56eea77d ethdb, trie: removed RLE compression 2015-07-23 12:57:33 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange
bdae4fd573 all: add some godoc synopsis comments 2015-07-07 14:12:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Jeffrey Wilcke
47460b3b4a trie: removed shallow copies (thanks to @fjl) 2015-07-04 02:51:37 +02:00
Jeffrey Wilcke
0a1ff68c11 trie: dirty tracking 2015-07-04 02:51:36 +02:00
obscuren
c850c41ec1 trie: Implemented a batch write approach for flushing 2015-06-29 18:51:48 +02:00
obscuren
a40a91d60f trie: fixed tests 2015-06-21 17:08:47 +02:00
obscuren
c590b505ed core, ethdb, trie: validate database errors 2015-06-21 16:59:15 +02:00
obscuren
bbe795455a Secure trie shakey / key matching 2015-03-24 15:14:03 +01:00
obscuren
cf45b939a0 fixed tests 2015-03-19 14:31:14 +01:00
obscuren
d7ab716eea Fixed mkdnode & added some tests 2015-03-19 10:57:02 +01:00
obscuren
b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
obscuren
d44fe4ec62 Merge branch 'develop' into poc-9 2015-03-05 17:46:40 +01:00
obscuren
357d17ae58 Start at 0 2015-03-05 17:46:00 +01:00
=
a6fe92194f Fixed iterator bug which skips the 0x000...000 key 2015-03-05 01:47:49 +00:00
obscuren
b909a5ec18 Trie => SecureTrie 2015-03-03 12:25:44 +01:00
obscuren
bf5365b317 Delet => Delete 2015-03-03 12:15:58 +01:00
obscuren
ae45a39dc1 Secure trie 2015-02-28 20:52:29 +01:00
obscuren
99ebb869bf Fixed iterator 2015-02-07 17:03:22 +01:00
obscuren
3c7181d28f Fixed a copy issue in the trie which could cause a consensus failure 2015-02-02 19:58:34 -08:00
obscuren
705cf6113d debug err removed 2015-01-29 23:18:46 +01:00
obscuren
9022f5034f default values removed 2015-01-29 23:17:43 +01:00
obscuren
db4aaedcbd Moved ptrie => trie. Removed old trie 2015-01-08 11:47:04 +01:00
obscuren
fed3e6a808 Refactored ethutil.Config.Db out 2015-01-07 13:17:48 +01:00
obscuren
93e693be72 Fixed tests for 'types' 2014-11-18 19:52:45 +01:00
obscuren
675ba4d7eb Merge branch 'tests' into poc8 2014-11-18 19:48:35 +01:00
obscuren
62cd9946ee j => i 2014-11-18 12:20:14 +01:00
obscuren
e08aba5dd9 added output test 2014-11-18 12:18:27 +01:00
obscuren
3220a32ff0 Added some comparison tests for the new ptrie 2014-11-18 12:03:09 +01:00
Taylor Gerring
ddca18638f update imports order per goimports 2014-11-15 20:21:55 -06:00
Taylor Gerring
e6e468ef59 Merge branch 'develop' of https://github.com/tgerring/go-ethereum into tests
Conflicts:
	p2p/messenger_test.go
	trie/trie_test.go
2014-11-14 15:19:50 -06:00
Taylor Gerring
2a9fc7baa9 Merge branch 'develop' of https://github.com/tgerring/go-ethereum 2014-11-14 15:01:52 -06:00
Taylor Gerring
313cfba7d4 convert trie encoding tests to checker 2014-11-13 04:40:21 +01:00
Taylor Gerring
00878e5b6e Convert trie tests to gocheck 2014-11-12 21:43:25 +01:00
obscuren
60cdb1148c Transaction execution fixes 2014-11-12 01:36:36 +01:00
obscuren
a82b89e2d5 Added storage root to dump 2014-11-03 23:45:44 +01:00
obscuren
af34749a6b ethtrie => trie 2014-10-31 14:45:03 +01:00