vyzo
|
9dbb2e0abd
|
don't leak tracking errors through the API
|
2021-07-07 16:34:02 +03:00 |
|
vyzo
|
83c30dc4c0
|
protect assignment of warmup epoch with the mutex
|
2021-07-07 11:31:27 +03:00 |
|
vyzo
|
6cc2112749
|
remove the curTs state variable; we don't need it
|
2021-07-07 09:55:25 +03:00 |
|
vyzo
|
05dbbe9681
|
rename som Txn methods for better readability
|
2021-07-07 09:52:31 +03:00 |
|
vyzo
|
90da6227b3
|
transactional protect incoming tipsets
|
2021-07-07 02:11:37 +03:00 |
|
vyzo
|
0e2af11f6a
|
prepare the transaction before launching the compaction goroutine
|
2021-07-07 01:39:58 +03:00 |
|
vyzo
|
f2f4af669d
|
clean up: simplify debug log, get rid of ugly debug log
|
2021-07-06 17:13:38 +03:00 |
|
vyzo
|
c1c25868cc
|
improve comments
|
2021-07-06 15:09:04 +03:00 |
|
vyzo
|
fdff1bebc9
|
move map markset implementation to its own file
|
2021-07-06 14:44:40 +03:00 |
|
vyzo
|
5c514504f7
|
remove unused GetGenesis method from ChainAccessor interface
|
2021-07-06 14:41:41 +03:00 |
|
vyzo
|
dc8139a1d2
|
add some comments for debug only code
|
2021-07-06 13:23:12 +03:00 |
|
vyzo
|
c4ae3e0c3d
|
minor tweak
|
2021-07-06 09:17:35 +03:00 |
|
vyzo
|
169ab262f5
|
really optimize computing object weights
sort is still taking a long time, this should be as fast as it gets.
|
2021-07-06 09:02:44 +03:00 |
|
vyzo
|
55a9e0ccd1
|
short-circuit block headers on sort weight computation
|
2021-07-06 08:22:43 +03:00 |
|
vyzo
|
bf7aeb3167
|
optimize sort a tad
it's taking a long time to compute weights...
|
2021-07-06 08:10:57 +03:00 |
|
vyzo
|
0659235e21
|
cache cid strings in sort
so as to avoid making a gazillion of strings
|
2021-07-06 07:26:13 +03:00 |
|
vyzo
|
525a2c71dd
|
use hashes as keys in weight map to avoid duplicate work
otherwise the root object will be raw, but internal references will be dag; duplicate work.
|
2021-07-06 01:27:56 +03:00 |
|
vyzo
|
c6ad8fdaed
|
use walkObjectRaw for computing object weights
cids that come out of the hotstore with ForEach are raw.
|
2021-07-06 01:08:44 +03:00 |
|
vyzo
|
2cbd3faf5a
|
make sure to nil everything in txnEndProtect
|
2021-07-05 23:56:31 +03:00 |
|
vyzo
|
51ab891d5c
|
quiet linter
it's a false positive, function doesn't escape.
|
2021-07-05 23:53:45 +03:00 |
|
vyzo
|
bd436ab9de
|
make endTxnProtect idempotent
|
2021-07-05 23:51:10 +03:00 |
|
vyzo
|
e859942fa4
|
code cleanup: refactor txn state code into their own functions
|
2021-07-05 23:31:37 +03:00 |
|
vyzo
|
3477d265c6
|
unify the two marksets
really, it's concurrent marking and there is no reason to have two different marksets
|
2021-07-05 20:10:47 +03:00 |
|
vyzo
|
73d07999bf
|
dont needlessly wait 1 min in first retry for missing refs
|
2021-07-05 18:24:48 +03:00 |
|
vyzo
|
af8cf712be
|
handle all missing refs together
so that we wait 6min at most, not 12.
|
2021-07-05 18:16:54 +03:00 |
|
vyzo
|
5a099b7d05
|
more commentary on the missing refs situation
|
2021-07-05 16:12:17 +03:00 |
|
vyzo
|
59639a0788
|
reinstate some better code for handling missing references.
|
2021-07-05 16:08:08 +03:00 |
|
vyzo
|
fa195bede2
|
get rid of ugly missing reference handling code
those missing objects don't seem to ever get there, are they from an abandoned fork?
|
2021-07-05 14:29:55 +03:00 |
|
vyzo
|
59936ef468
|
fix log
|
2021-07-05 13:30:31 +03:00 |
|
vyzo
|
0b7153be86
|
use internal version of has for occurs checks
|
2021-07-05 12:41:11 +03:00 |
|
vyzo
|
d8b8d75e0f
|
readd minute delay before trying for missing objects
|
2021-07-05 12:38:09 +03:00 |
|
vyzo
|
d7709deb2b
|
reduce memory pressure from marksets when the size is decreased
|
2021-07-05 11:51:22 +03:00 |
|
vyzo
|
3ec834b2e3
|
improve logs and error messages
|
2021-07-05 11:41:09 +03:00 |
|
vyzo
|
918a7ec749
|
a bit more fil commitment short-circuiting
|
2021-07-05 11:38:53 +03:00 |
|
vyzo
|
2ea2abc07d
|
short-circuit fil commitments
they don't make it to the blockstore anyway
|
2021-07-05 11:32:52 +03:00 |
|
vyzo
|
839f7bd2b5
|
only occur check for DAGs
|
2021-07-05 11:11:08 +03:00 |
|
vyzo
|
c81ae5fc20
|
add some comments about the missing business and anothre log
|
2021-07-05 10:42:14 +03:00 |
|
vyzo
|
4c41f52828
|
add warning for missing objects for marking for debug purposes
|
2021-07-05 10:35:04 +03:00 |
|
vyzo
|
3597192d58
|
remove the sleeps and busy loop more times when waiting for missing objects
|
2021-07-05 10:31:47 +03:00 |
|
vyzo
|
1726eb993c
|
deal with incomplete objects that need to be marked and protected
seems that something is writing DAGs before its consituents, which causes problems.
|
2021-07-05 10:22:52 +03:00 |
|
vyzo
|
db53859e7a
|
reduce CompactionThreshold to 5 finalities
so that we run compaction every finality, once we've first compacted
|
2021-07-04 22:12:51 +03:00 |
|
vyzo
|
b08e0b7102
|
fix lint
|
2021-07-04 21:24:15 +03:00 |
|
vyzo
|
94efae419e
|
reduce length of critical section
Just the purge; the rest is not critical -- e.g. it's ok if we do some duplicate copies
to the coldstore, we'll have gc soon.
|
2021-07-04 21:21:53 +03:00 |
|
vyzo
|
f33d4e79aa
|
simplify transactional protection logic
Now that we delete objects heaviest first, we don't have to do deep walk and rescan gymnastics.
|
2021-07-04 20:49:39 +03:00 |
|
vyzo
|
40c271cda1
|
sort cold objects before deleting
so that we can't shoot ourselves in the foot by deleting the constituents of a DAG while it is
still in the hotstore.
|
2021-07-04 20:17:07 +03:00 |
|
vyzo
|
13d612f72f
|
smarter trackTxnRefMany
|
2021-07-04 19:33:49 +03:00 |
|
vyzo
|
f124389b66
|
recursively protect all references
|
2021-07-04 19:21:00 +03:00 |
|
vyzo
|
4d286da593
|
fix error message
|
2021-07-04 18:58:39 +03:00 |
|
vyzo
|
680af8eb09
|
use deep object walking for more robust handling of transactional references
|
2021-07-04 18:38:28 +03:00 |
|
vyzo
|
1f02428225
|
fix lint
|
2021-07-04 18:38:28 +03:00 |
|