vyzo
b87295db93
bubble up dependent txn ref errors
...
This cause Has to return false if it fails to traverse/protect all links, which would cause
the vm to recompute.
2021-07-04 18:38:28 +03:00
vyzo
637fbf6c5b
fix faulty if/else logic for implicit txn protection
2021-07-04 18:38:28 +03:00
vyzo
9d6bcd7705
avoid clown shoes: only walk links for tracking in implicit writes/refs
2021-07-04 18:38:28 +03:00
vyzo
484dfaebce
reused cidset across all walks when flushing pending writes
2021-07-04 18:38:28 +03:00
vyzo
1d41e1544a
optimize transitive write tracking a bit
2021-07-04 18:38:28 +03:00
vyzo
da00fc66ee
downgrade a couple of logs to warnings
2021-07-04 18:38:28 +03:00
vyzo
4071488ef2
first write, then track
2021-07-04 18:38:28 +03:00
vyzo
bd92c230da
refactor txn reference tracking, do deep marking of DAGs
2021-07-04 18:38:28 +03:00
vyzo
a98a062347
do the dag walk for deep write tracking during flush
...
avoid crawling everything to a halt
2021-07-04 18:38:28 +03:00
vyzo
13a674330f
add pending write check before tracking the object in Has
2021-07-04 18:38:28 +03:00
vyzo
982867317e
transitively track dags from implicit writes in Has
2021-07-04 18:38:28 +03:00
vyzo
4de0cd9fcb
move write log back to flush so that we don't crawl to a halt
2021-07-04 18:38:28 +03:00
vyzo
2faa4aa993
debug log writes at track so that we get correct stack traces
2021-07-04 18:38:28 +03:00
vyzo
aeaa59d4b5
move comments about tracking perf issues into a more pertinent place
2021-07-04 18:38:28 +03:00
vyzo
3e8e9273ca
track all writes using async batching, not just implicit ones
2021-07-04 18:38:28 +03:00
vyzo
d0bfe421b5
flush implicit writes at the right time before starting compaction to avoid races
2021-07-04 18:38:28 +03:00
vyzo
7f473f56eb
flush implicit writes before starting compaction
2021-07-04 18:38:28 +03:00
vyzo
a29947d47c
flush implicit writes in all paths in updateWriteEpoch
2021-07-04 18:38:28 +03:00
vyzo
be6cc2c3e6
batch implicit write tracking
...
bolt performance leaves something to be desired; doing a single Put takes 10ms, about the same time
as batching thousands of them.
2021-07-04 18:38:28 +03:00
vyzo
e472cacb3e
add missing return
2021-07-04 18:38:28 +03:00
vyzo
6a3cbea790
treat Has as an implicit Write
...
Rationale: the VM uses the Has check to avoid issuing a duplicate Write in the blockstore.
This means that live objects that would be otherwise written are not actually written, resulting
in the first write epoch being considered the write epoch.
2021-07-04 18:38:28 +03:00
vyzo
90dc274113
better logging for chain walk
2021-07-04 18:38:28 +03:00
vyzo
40f42db7fa
walk tweaks
2021-07-04 18:38:28 +03:00
vyzo
09efed50fd
check for lookback references to block headers in walk
2021-07-04 18:38:28 +03:00
vyzo
7de0771883
count txn live objects explicitly for logging
2021-07-04 18:38:28 +03:00
vyzo
e29b64c5de
check both markset and txn liveset before declaring an object cold
2021-07-04 18:38:28 +03:00
vyzo
4bed3161f0
fix broken purge count log
2021-07-04 18:38:28 +03:00
vyzo
dec61fa333
deduplicate stack logs and optionally trace write stacks
2021-07-04 18:38:28 +03:00
vyzo
7ebef6d838
better log message
2021-07-04 18:38:28 +03:00
vyzo
40ff5bf164
log put errors in splitstore log
2021-07-04 18:38:28 +03:00
vyzo
9fda61abec
fix error check for unreachable cids
2021-07-04 18:38:28 +03:00
vyzo
4a71c68e06
move code around for better readability
2021-07-04 18:38:28 +03:00
vyzo
31497f4bd3
use internal get during walk to avoid blowing the compaction txn
...
otherwise the walk itself precludes purge... duh!
2021-07-04 18:38:28 +03:00
vyzo
6af3a23dd4
use a map for txn protection mark set
2021-07-04 18:38:28 +03:00
vyzo
65ccc99e79
minor tweaks in purge
...
- allocate once
- log purge count
2021-07-04 18:38:28 +03:00
vyzo
cb665d07e0
fix transactional race during compaction
...
It is possible for an object to be written or recreated (and checked with Has)
after the mark completes and during the purge; if this happens we will purge
a live block.
2021-07-04 18:38:28 +03:00
vyzo
50ebaf25aa
don't log read misses before warmup
2021-07-04 18:38:28 +03:00
vyzo
b187b5c301
fix lint
2021-07-04 18:38:28 +03:00
vyzo
a53c4e1597
implement debug log
2021-07-04 18:38:28 +03:00
vyzo
fce7b8dc9b
flush move log when cold collection is done
2021-07-04 18:38:28 +03:00
vyzo
fc247e4223
add debug log skeleton
2021-07-04 18:38:28 +03:00
vyzo
0390285c4e
always do full walks, not only when there is a sync gap
2021-07-04 18:38:28 +03:00
vyzo
30dbe4978b
adjust compaction range
2021-07-04 18:38:28 +03:00
vyzo
a21f55919b
CompactionThreshold should be 4 finalities
...
otherwise we'll wear clown shoes with the slack and end up in continuous compaction.
2021-07-04 18:38:28 +03:00
vyzo
a25ac80777
reintroduce compaction slack
2021-07-04 18:38:28 +03:00
vyzo
c4d95de987
coalesce back-to-back compactions
...
get rid of the CompactionCold construct, run a single compaction on catch up
2021-07-04 18:38:28 +03:00
vyzo
b7897595eb
augment current epoch by +1
...
to account for off by one conditions
2021-07-04 18:38:28 +03:00
vyzo
933c786421
update write epoch in the background every second
2021-07-04 18:38:28 +03:00
vyzo
66f1630f14
fix lint issue
2021-07-04 18:38:28 +03:00
vyzo
bb17608ae0
track writeEpoch relative to current wall clock time
...
The issue: head change notifications are not emitted until after catching up,
which results in all writes during a catch up period being tracked at the base epoch.
2021-07-04 18:38:28 +03:00
