Commit Graph

338 Commits

Author SHA1 Message Date
Nick Johnson
555273495b trie: add difference iterator (#3637)
This PR implements a differenceIterator, which allows iterating over trie nodes
that exist in one trie but not in another. This is a prerequisite for most GC
strategies, in order to find obsolete nodes.
2017-02-22 23:49:34 +01:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Felix Lange
d3b751e4d9 trie: remove dependency on ethdb
This removes the core/types -> leveldb dependency.
2017-01-06 14:15:22 +01:00
Péter Szilágyi
90b16a3e85
core/state, eth/downloader, trie: reset fast-failure on progress 2016-10-31 14:19:14 +02:00
Péter Szilágyi
f8608a5228 trie: while fast syncing, don't keep trie nodes in memory (#3186) 2016-10-21 17:34:17 +02:00
Péter Szilágyi
88a593d559
cmd/geth, trie: report on trie cache unloads, also add debug log 2016-10-19 17:31:19 +03:00
Felix Lange
5d9bb0a050 trie: add metric to track cache unloading (#3175) 2016-10-19 16:07:19 +02:00
Felix Lange
d48e6ae66f trie: make the cache miss counter a real metric 2016-10-19 15:22:37 +02:00
Jeffrey Wilcke
25ac04a444 Merge pull request #3153 from fjl/trie-unload-fix
trie: improve cache unloading mechanism
2016-10-19 13:35:49 +02:00
Péter Szilágyi
2bb5ec1e41
cmd/geth, trie: track and report trie cache misses 2016-10-19 14:12:35 +03:00
Felix Lange
8d56bf5ceb trie: ensure dirty flag is unset for embedded child nodes
This was caught by the new invariant check.
2016-10-18 04:57:47 +02:00
Felix Lange
177cab5fe7 trie: ensure resolved nodes stay loaded
Commit 40cdcf1183 broke the optimisation which kept nodes resolved
during Get in the trie. The decoder assigned cache generation 0
unconditionally, causing resolved nodes to get flushed on Commit.

This commit fixes it and adds two tests.
2016-10-18 04:57:47 +02:00
Felix Lange
187d6a66a5 trie: avoid loading the root node twice
New checks whether the root node is present by loading it from the
database. Keep the node around instead of discarding it.
2016-10-18 04:57:47 +02:00
Felix Lange
2cd7a0395d trie: fix regression that linked all downloaded nodes together
The trie sync code links subtries using pointers into node structs.
Since commit 40cdcf1183 nodes are no longer copied when unpacking from
an interface value, causing all nodes to get linked up as the sync
progresses. Fix it by breaking the pointer chain with an explicit copy.
2016-10-15 00:42:52 +02:00
Felix Lange
40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Felix Lange
c3a77d6268 trie: fix delete bug for values contained in fullNode
Delete crashed if a fullNode contained a valueNode directly. This bug is
very unlikely to occur with SecureTrie, but can happen with regular
tries. This commit also introduces a randomised test which triggers all
trie operations, which should prevent such bugs in the future.

Credit for finding this bug goes to Github user @rjl493456442.
2016-09-29 22:53:59 +02:00
Felix Lange
ba8c4c6b1a trie: remove dependency on package crypto
Package crypto needs cgo, which is inconvenient for some build
configurations.
2016-09-29 22:23:24 +02:00
Péter Szilágyi
710435b51b core, eth, trie: reuse trie journals in all our code 2016-09-28 11:27:31 +03:00
Felix Lange
cd791bd855 core, trie: replace state caches with trie journal 2016-09-28 11:27:28 +03:00
Péter Szilágyi
1d5d21726a Merge pull request #2627 from karalabe/concurrent-head-sync
eth/downloader, trie: pull head state concurrently with chain
2016-05-31 11:52:16 +03:00
Péter Szilágyi
4f1d92b332 eth/downloader, trie: pull head state concurrently with chain 2016-05-27 14:26:00 +03:00
Péter Szilágyi
748d1c171d core, core/state, trie: enterprise hand-tuned multi-level caching 2016-05-26 16:33:09 +03:00
Felix Lange
6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Felix Lange
d04a2e7557 all: update license information 2016-04-15 09:48:05 +02:00
Péter Szilágyi
e90958cd29 cmd, eth, ethdb, node: prioritise chaindata for resources, bump cache 2016-03-09 10:33:39 +02:00
Jeffrey Wilcke
483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Ricardo Catalinas Jiménez
436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Péter Szilágyi
b8d59d9c98 core/state, trie: switch iterator panics to error fields 2016-02-16 12:37:00 +02:00
Péter Szilágyi
151c7bef41 core/state, trie: node iterator reports parent hashes too 2016-02-16 12:21:08 +02:00
Péter Szilágyi
5a057a8ded core/state, trie: surface iterator entry hashes 2016-02-16 12:21:08 +02:00
Péter Szilágyi
7e29b0b5b4 core/state, trie: add node iterator, test state/trie sync consistency 2016-02-16 12:21:08 +02:00
Péter Szilágyi
f3d4ce0d16 core/state, ethdb, trie: test intermediate secure key leak, fix memdb bug 2016-01-20 16:06:28 +02:00
Péter Szilágyi
886478b18b core/state, trie: don't leak database writes before commit 2016-01-20 12:09:24 +02:00
zsfelfoldi
ef422ee1e1 light: implemented odr-capable trie and state structures 2015-12-17 16:07:54 +01:00
zsfelfoldi
52904ae32f trie: added error handling
Created alternate versions of Trie and SecureTrie functions that can return a MissingNodeError (used by ODR services)
2015-12-01 01:38:32 +01:00
Péter Szilágyi
5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi
b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Péter Szilágyi
ab27bee25a core, eth, trie: direct state trie synchronization 2015-10-19 10:03:09 +03:00
Felix Lange
c1a352c108 trie: add merkle proof functions 2015-09-22 22:57:37 +02:00
Felix Lange
565d9f2306 core, trie: new trie 2015-09-22 22:53:49 +02:00
Felix Lange
6b91a4abe5 trie: improve benchmarks 2015-09-22 22:49:27 +02:00
Jeffrey Wilcke
b8ca0a830e eth, trie: removed key prefixing from state entries & merge db fix
Fixed database merge strategy to use the correct database. Due to a copy
paste fail when doing type evaluation the same database was being
iterated (chain), all others were ignored.

Removed state prefixing because {H(code): code} is stored in the same
database as the rest of the state.
2015-08-13 20:44:03 +02:00
Jeffrey Wilcke
a23478c0be core, eth, trie, xeth: merged state, chain, extra databases in one 2015-08-07 22:29:02 +02:00
Ethan Buchman
46c9594081 trie: run codec tests, add benchmarks, faster 2015-08-06 14:04:16 -04:00
Ethan Buchman
c1d516546d faster hex-prefix codec and string -> []byte 2015-08-06 03:17:59 -04:00
Jeffrey Wilcke
1fad8798ec Merge pull request #1515 from fjl/license-fixes
all: fix license headers one more time
2015-07-28 04:29:42 -07:00
Felix Lange
bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Jeffrey Wilcke
8f56eea77d ethdb, trie: removed RLE compression 2015-07-23 12:57:33 +02:00
Felix Lange
3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange
bdae4fd573 all: add some godoc synopsis comments 2015-07-07 14:12:45 +02:00
Felix Lange
ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Jeffrey Wilcke
47460b3b4a trie: removed shallow copies (thanks to @fjl) 2015-07-04 02:51:37 +02:00
Jeffrey Wilcke
0a1ff68c11 trie: dirty tracking 2015-07-04 02:51:36 +02:00
obscuren
c850c41ec1 trie: Implemented a batch write approach for flushing 2015-06-29 18:51:48 +02:00
obscuren
a40a91d60f trie: fixed tests 2015-06-21 17:08:47 +02:00
obscuren
c590b505ed core, ethdb, trie: validate database errors 2015-06-21 16:59:15 +02:00
obscuren
bbe795455a Secure trie shakey / key matching 2015-03-24 15:14:03 +01:00
obscuren
cf45b939a0 fixed tests 2015-03-19 14:31:14 +01:00
obscuren
d7ab716eea Fixed mkdnode & added some tests 2015-03-19 10:57:02 +01:00
obscuren
b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
obscuren
d44fe4ec62 Merge branch 'develop' into poc-9 2015-03-05 17:46:40 +01:00
obscuren
357d17ae58 Start at 0 2015-03-05 17:46:00 +01:00
=
a6fe92194f Fixed iterator bug which skips the 0x000...000 key 2015-03-05 01:47:49 +00:00
obscuren
b909a5ec18 Trie => SecureTrie 2015-03-03 12:25:44 +01:00
obscuren
bf5365b317 Delet => Delete 2015-03-03 12:15:58 +01:00
obscuren
ae45a39dc1 Secure trie 2015-02-28 20:52:29 +01:00
obscuren
99ebb869bf Fixed iterator 2015-02-07 17:03:22 +01:00
obscuren
3c7181d28f Fixed a copy issue in the trie which could cause a consensus failure 2015-02-02 19:58:34 -08:00
obscuren
705cf6113d debug err removed 2015-01-29 23:18:46 +01:00
obscuren
9022f5034f default values removed 2015-01-29 23:17:43 +01:00
obscuren
db4aaedcbd Moved ptrie => trie. Removed old trie 2015-01-08 11:47:04 +01:00
obscuren
fed3e6a808 Refactored ethutil.Config.Db out 2015-01-07 13:17:48 +01:00
obscuren
93e693be72 Fixed tests for 'types' 2014-11-18 19:52:45 +01:00
obscuren
675ba4d7eb Merge branch 'tests' into poc8 2014-11-18 19:48:35 +01:00
obscuren
62cd9946ee j => i 2014-11-18 12:20:14 +01:00
obscuren
e08aba5dd9 added output test 2014-11-18 12:18:27 +01:00
obscuren
3220a32ff0 Added some comparison tests for the new ptrie 2014-11-18 12:03:09 +01:00
Taylor Gerring
ddca18638f update imports order per goimports 2014-11-15 20:21:55 -06:00
Taylor Gerring
e6e468ef59 Merge branch 'develop' of https://github.com/tgerring/go-ethereum into tests
Conflicts:
	p2p/messenger_test.go
	trie/trie_test.go
2014-11-14 15:19:50 -06:00
Taylor Gerring
2a9fc7baa9 Merge branch 'develop' of https://github.com/tgerring/go-ethereum 2014-11-14 15:01:52 -06:00
Taylor Gerring
313cfba7d4 convert trie encoding tests to checker 2014-11-13 04:40:21 +01:00
Taylor Gerring
00878e5b6e Convert trie tests to gocheck 2014-11-12 21:43:25 +01:00
obscuren
60cdb1148c Transaction execution fixes 2014-11-12 01:36:36 +01:00
obscuren
a82b89e2d5 Added storage root to dump 2014-11-03 23:45:44 +01:00
obscuren
af34749a6b ethtrie => trie 2014-10-31 14:45:03 +01:00