Commit Graph

13260 Commits

Author SHA1 Message Date
vyzo
fce7b8dc9b flush move log when cold collection is done 2021-07-04 18:38:28 +03:00
vyzo
fc247e4223 add debug log skeleton 2021-07-04 18:38:28 +03:00
vyzo
0390285c4e always do full walks, not only when there is a sync gap 2021-07-04 18:38:28 +03:00
vyzo
30dbe4978b adjust compaction range 2021-07-04 18:38:28 +03:00
vyzo
a21f55919b CompactionThreshold should be 4 finalities
otherwise we'll wear clown shoes with the slack and end up in continuous compaction.
2021-07-04 18:38:28 +03:00
vyzo
79d21489df fix test 2021-07-04 18:38:28 +03:00
vyzo
a25ac80777 reintroduce compaction slack 2021-07-04 18:38:28 +03:00
vyzo
a178c1fb93 fix test 2021-07-04 18:38:28 +03:00
vyzo
c4d95de987 coalesce back-to-back compactions
get rid of the CompactionCold construct, run a single compaction on catch up
2021-07-04 18:38:28 +03:00
vyzo
b7897595eb augment current epoch by +1
to account for off by one conditions
2021-07-04 18:38:28 +03:00
vyzo
933c786421 update write epoch in the background every second 2021-07-04 18:38:28 +03:00
vyzo
66f1630f14 fix lint issue 2021-07-04 18:38:28 +03:00
vyzo
bb17608ae0 track writeEpoch relative to current wall clock time
The issue: head change notifications are not emitted until after catching up,
which results in all writes during a catch up period being tracked at the base epoch.
2021-07-04 18:38:28 +03:00
vyzo
421f05eab9 save the warm up epoch only if successful in warming up 2021-07-04 18:38:28 +03:00
vyzo
9b6448518c refactor warmup to trigger at startup and not wait for sync 2021-07-04 18:38:28 +03:00
vyzo
3fe4261f12 don't attempt compaction while still syncing 2021-07-04 18:38:28 +03:00
vyzo
7b02673620 don't try to visit genesis parent blocks 2021-07-04 18:38:28 +03:00
vyzo
997f2c098b keep headers hot when running with a noop splitstore 2021-07-04 18:38:28 +03:00
vyzo
7c814cd2e3 refactor genesis state loading code into its own method 2021-07-04 18:38:28 +03:00
vyzo
41573f1fb2 also walk parent message receipts when including messages in the walk 2021-07-04 18:38:28 +03:00
vyzo
fa6481401d reduce SyncGapTime to 1 minute
for maximal safety.
2021-07-04 18:38:28 +03:00
vyzo
fda291b876 fix test 2021-07-04 18:38:28 +03:00
vyzo
d33a44e67f first visit the cid, then short-circuit non dagcbor objects 2021-07-04 18:38:28 +03:00
vyzo
bdb97d6186 more robust handling of sync gap walks 2021-07-04 18:38:28 +03:00
vyzo
7cf75e667d keep genesis-linked state hot 2021-07-04 18:38:28 +03:00
vyzo
e9f531b4aa don't open bolt tracking store with NoSync, it might get corrupted 2021-07-04 18:38:28 +03:00
Raúl Kripalani
b2b7eb2ded metrics: increment misses in View(). 2021-07-04 18:38:28 +03:00
vyzo
3a9b7c592d mark from current epoch to boundary epoch when necessary
this is necessary to avoid wearing clown shoes when the node stays
offline for an extended period of time (more than 1 finality).

Basically it gets quite slow if we do the full 2 finality walk, so we
try to avoid it unless necessary.
The conditions under which a full walk is necessary is if there is a
sync gap (most likely because the node was offline) during which the
tracking of writes is inaccurate because we have not yet delivered the
HeadChange notification.  In this case, it is possible to have
actually hot blocks to be tracked before the boundary and fail to mark
them accordingly.  So when we detect a sync gap, we do the full walk;
if there is no sync gap, we can just use the much faster boundary
epoch walk.
2021-07-04 18:38:28 +03:00
vyzo
d7ceef104e decrease CompactionThreshold to 3 finalities 2021-07-04 18:38:28 +03:00
vyzo
e3cbeec6ee implement chain walking 2021-07-04 18:38:28 +03:00
vyzo
04f2e102a1 kill full splitstore compaction, simplify splitstore configuration 2021-07-04 18:38:28 +03:00
vyzo
5cca29d1db hook noop blockstore for splitstore in DI 2021-07-04 18:38:28 +03:00
vyzo
4d3c73f4ca noop blockstore 2021-07-04 18:38:28 +03:00
Łukasz Magiera
a577bf340e
Merge pull request #6675 from filecoin-project/chore/remove_vestigial_replace
Remove leftover from the great de-submoduling
2021-07-04 12:11:54 +02:00
Peter Rabbitson
5c643d6141 Remove leftover from the great de-submoduling
This is no longer used anywhere in lotus
2021-07-04 09:40:47 +02:00
Aayush Rajasekaran
069fd0a244 Add helptext to lotus chain export 2021-07-03 12:41:43 -04:00
aarshkshah1992
de13b6edc5 dont upgrade multihash 2021-07-03 14:26:03 +05:30
aarshkshah1992
303d5d6ff6 removed combined blockstore 2021-07-03 14:12:08 +05:30
aarshkshah1992
40b30d1fb1 remove multistore from Lotus 2021-07-03 14:12:08 +05:30
aarshkshah1992
dae6c757b5 lotus changes for the DAG Store 2021-07-03 14:12:08 +05:30
aarshkshah1992
31ecce3a72 draft lotus storage client CARv2 changes 2021-07-03 14:12:08 +05:30
Cory Schwartz
5a7743214a Merge branch 'master' into feat/docker-compose 2021-07-02 17:00:13 -07:00
Łukasz Magiera
13d4563c25
Merge pull request #6663 from filecoin-project/raulk/test-incremental-nonce
add an incremental nonce itest.
2021-07-02 23:33:52 +02:00
Jennifer
0519cd371e
Merge pull request #6665 from filecoin-project/backports/v1.11.0-rc2
Backports for v1.11.0-rc2
2021-07-02 17:09:33 -04:00
Łukasz Magiera
acc4888b13
Merge pull request #6667 from filecoin-project/jen/changelogv11
bump version to v1.11.0-rc2
2021-07-02 22:28:43 +02:00
Łukasz Magiera
83114546c6
Merge branch 'backports/v1.11.0-rc2' into jen/changelogv11 2021-07-02 22:28:24 +02:00
Jennifer Wang
68bde33977 update changelog and bump version to v1.11.0-rc2 2021-07-02 16:01:51 -04:00
Jennifer Wang
0b5d1551fe format fix and small clarification 2021-07-02 15:53:29 -04:00
Jennifer Wang
a06ecde580 recommend to update 2021-07-02 15:33:58 -04:00
Jennifer Wang
94574447e6 Add v1.10.1-rc1 changelog 2021-07-02 15:30:58 -04:00