vyzo
f33d4e79aa
simplify transactional protection logic
...
Now that we delete objects heaviest first, we don't have to do deep walk and rescan gymnastics.
2021-07-04 20:49:39 +03:00
vyzo
40c271cda1
sort cold objects before deleting
...
so that we can't shoot ourselves in the foot by deleting the constituents of a DAG while it is
still in the hotstore.
2021-07-04 20:17:07 +03:00
vyzo
13d612f72f
smarter trackTxnRefMany
2021-07-04 19:33:49 +03:00
vyzo
f124389b66
recursively protect all references
2021-07-04 19:21:00 +03:00
vyzo
4d286da593
fix error message
2021-07-04 18:58:39 +03:00
vyzo
680af8eb09
use deep object walking for more robust handling of transactional references
2021-07-04 18:38:28 +03:00
vyzo
1f02428225
fix lint
2021-07-04 18:38:28 +03:00
vyzo
2c7a89a1db
short-circuit rescanning on block headers
2021-07-04 18:38:28 +03:00
vyzo
8e56fffb33
walkChain should visit the genesis state root
2021-07-04 18:38:28 +03:00
vyzo
190cb18ab0
housekeeping
...
- remove defunct tracking store implementations
- update splitstore node config
- use mark set type config option (defaulting to mapts); a memory constrained node
may want to use an on-disk one
2021-07-04 18:38:28 +03:00
vyzo
19d1b1f532
deal with partially written objects
2021-07-04 18:38:28 +03:00
vyzo
0a1d7b3732
fix log
2021-07-04 18:38:28 +03:00
vyzo
eafffc1634
more efficient trackTxnRefMany
2021-07-04 18:38:28 +03:00
vyzo
36f93649ef
fix panic from concurrent map writes in txnRefs
2021-07-04 18:38:28 +03:00
vyzo
6fa2cd232d
simplify compaction model
2021-07-04 18:38:28 +03:00
vyzo
1f2b604c07
RIP tracking store
2021-07-04 18:38:28 +03:00
vyzo
68a83500bc
fix bug that turned candidate filtering to dead code
2021-07-04 18:38:28 +03:00
vyzo
642f0e4740
deal with memory pressure, don't walk under the boundary
2021-07-04 18:38:28 +03:00
vyzo
c5cf8e226b
remove unnecessary code
2021-07-04 18:38:28 +03:00
vyzo
d79e4da7aa
more accurate stats about mark set updates
2021-07-04 18:38:28 +03:00
vyzo
6f58fdcb22
remove vm copy context detection hack
...
stack tracing is slow.
2021-07-04 18:38:28 +03:00
vyzo
2b03316cd9
fix log message
2021-07-04 18:38:28 +03:00
vyzo
184d3802b6
remove dead code
2021-07-04 18:38:28 +03:00
vyzo
228a435ba7
rework tracking logic; do it lazily and far more efficiently
2021-07-04 18:38:28 +03:00
vyzo
9d6cabd18a
if it's not a dag, it's not a block
2021-07-04 18:38:28 +03:00
vyzo
8157f889ce
short-circuit marking walks when encountering a block and more efficient walking
2021-07-04 18:38:28 +03:00
vyzo
736d6a3c19
only treat Has as an implicit write within vm.Copy context
2021-07-04 18:38:28 +03:00
vyzo
39723bbe60
use a single map for tracking pending writes, properly track implicits
2021-07-04 18:38:28 +03:00
vyzo
5834231e58
create the transactional protect filter before walking
2021-07-04 18:38:28 +03:00
vyzo
e4bb4be855
fix some residual purge races
2021-07-04 18:38:28 +03:00
vyzo
68bc5d2291
skip moving cold blocks when running with a noop coldstore
...
it is a noop but it still takes (a lot of) time because it has to read all the cold blocks.
2021-07-04 18:38:28 +03:00
vyzo
b87295db93
bubble up dependent txn ref errors
...
This cause Has to return false if it fails to traverse/protect all links, which would cause
the vm to recompute.
2021-07-04 18:38:28 +03:00
vyzo
637fbf6c5b
fix faulty if/else logic for implicit txn protection
2021-07-04 18:38:28 +03:00
vyzo
9d6bcd7705
avoid clown shoes: only walk links for tracking in implicit writes/refs
2021-07-04 18:38:28 +03:00
vyzo
484dfaebce
reused cidset across all walks when flushing pending writes
2021-07-04 18:38:28 +03:00
vyzo
1d41e1544a
optimize transitive write tracking a bit
2021-07-04 18:38:28 +03:00
vyzo
da00fc66ee
downgrade a couple of logs to warnings
2021-07-04 18:38:28 +03:00
vyzo
4071488ef2
first write, then track
2021-07-04 18:38:28 +03:00
vyzo
bd92c230da
refactor txn reference tracking, do deep marking of DAGs
2021-07-04 18:38:28 +03:00
vyzo
a98a062347
do the dag walk for deep write tracking during flush
...
avoid crawling everything to a halt
2021-07-04 18:38:28 +03:00
vyzo
13a674330f
add pending write check before tracking the object in Has
2021-07-04 18:38:28 +03:00
vyzo
982867317e
transitively track dags from implicit writes in Has
2021-07-04 18:38:28 +03:00
vyzo
4de0cd9fcb
move write log back to flush so that we don't crawl to a halt
2021-07-04 18:38:28 +03:00
vyzo
2faa4aa993
debug log writes at track so that we get correct stack traces
2021-07-04 18:38:28 +03:00
vyzo
aeaa59d4b5
move comments about tracking perf issues into a more pertinent place
2021-07-04 18:38:28 +03:00
vyzo
3e8e9273ca
track all writes using async batching, not just implicit ones
2021-07-04 18:38:28 +03:00
vyzo
d0bfe421b5
flush implicit writes at the right time before starting compaction to avoid races
2021-07-04 18:38:28 +03:00
vyzo
7f473f56eb
flush implicit writes before starting compaction
2021-07-04 18:38:28 +03:00
vyzo
a29947d47c
flush implicit writes in all paths in updateWriteEpoch
2021-07-04 18:38:28 +03:00
vyzo
be6cc2c3e6
batch implicit write tracking
...
bolt performance leaves something to be desired; doing a single Put takes 10ms, about the same time
as batching thousands of them.
2021-07-04 18:38:28 +03:00
