Commit Graph

13285 Commits

Author SHA1 Message Date
vyzo
bf7aeb3167 optimize sort a tad
it's taking a long time to compute weights...
2021-07-06 08:10:57 +03:00
vyzo
0659235e21 cache cid strings in sort
so as to avoid making a gazillion of strings
2021-07-06 07:26:13 +03:00
aarshkshah1992
a07a0935fe fix bug in CAR generation 2021-07-06 09:31:52 +05:30
vyzo
525a2c71dd use hashes as keys in weight map to avoid duplicate work
otherwise the root object will be raw, but internal references will be dag; duplicate work.
2021-07-06 01:27:56 +03:00
vyzo
c6ad8fdaed use walkObjectRaw for computing object weights
cids that come out of the hotstore with ForEach are raw.
2021-07-06 01:08:44 +03:00
vyzo
2cbd3faf5a make sure to nil everything in txnEndProtect 2021-07-05 23:56:31 +03:00
vyzo
51ab891d5c quiet linter
it's a false positive, function doesn't escape.
2021-07-05 23:53:45 +03:00
vyzo
bd436ab9de make endTxnProtect idempotent 2021-07-05 23:51:10 +03:00
vyzo
e859942fa4 code cleanup: refactor txn state code into their own functions 2021-07-05 23:31:37 +03:00
vyzo
3477d265c6 unify the two marksets
really, it's concurrent marking and there is no reason to have two different marksets
2021-07-05 20:10:47 +03:00
Łukasz Magiera
97207df705
Merge pull request #6639 from filecoin-project/frrist/context-in-statetree-diff
polish(statetree): accept a context in statetree diff for timeouts
2021-07-05 17:30:31 +02:00
vyzo
73d07999bf dont needlessly wait 1 min in first retry for missing refs 2021-07-05 18:24:48 +03:00
vyzo
af8cf712be handle all missing refs together
so that we wait 6min at most, not 12.
2021-07-05 18:16:54 +03:00
Jim Pick
17c0b9be44
Spelling fix
Change 'actorr' to 'actor'
2021-07-05 07:44:06 -07:00
Łukasz Magiera
d76cbf805a
Merge pull request #6672 from filecoin-project/asr/nit
Add helptext to lotus chain export
2021-07-05 16:22:16 +02:00
Łukasz Magiera
5e5de9b41b
Merge pull request #6679 from filecoin-project/fix/get-retrieval-pricing-input-error
Get retrieval pricing input should not error out on a deal state fetch
2021-07-05 16:18:48 +02:00
Łukasz Magiera
a56b6fea99
Merge pull request #6680 from filecoin-project/chore/double-encoded-cid
Fix more CID double-encoding as hex
2021-07-05 16:14:52 +02:00
vyzo
5a099b7d05 more commentary on the missing refs situation 2021-07-05 16:12:17 +03:00
vyzo
59639a0788 reinstate some better code for handling missing references. 2021-07-05 16:08:08 +03:00
Anton Evangelatov
5a87f04e71 fix finalize sectors early test 2021-07-05 13:36:23 +02:00
vyzo
fa195bede2 get rid of ugly missing reference handling code
those missing objects don't seem to ever get there, are they from an abandoned fork?
2021-07-05 14:29:55 +03:00
Anton Evangelatov
4be0a7a215 resolve conflicts with master 2021-07-05 13:13:32 +02:00
Peter Rabbitson
a544fa0f97 Fix more CID doucle-encoding as hex 2021-07-05 12:51:15 +02:00
aarshkshah1992
add113f585 use multierror 2021-07-05 16:08:51 +05:30
vyzo
59936ef468 fix log 2021-07-05 13:30:31 +03:00
aarshkshah1992
44985722d2 changes as per review 2021-07-05 15:53:41 +05:30
vyzo
0b7153be86 use internal version of has for occurs checks 2021-07-05 12:41:11 +03:00
vyzo
d8b8d75e0f readd minute delay before trying for missing objects 2021-07-05 12:38:09 +03:00
vyzo
d7709deb2b reduce memory pressure from marksets when the size is decreased 2021-07-05 11:51:22 +03:00
vyzo
3ec834b2e3 improve logs and error messages 2021-07-05 11:41:09 +03:00
vyzo
918a7ec749 a bit more fil commitment short-circuiting 2021-07-05 11:38:53 +03:00
vyzo
2ea2abc07d short-circuit fil commitments
they don't make it to the blockstore anyway
2021-07-05 11:32:52 +03:00
vyzo
839f7bd2b5 only occur check for DAGs 2021-07-05 11:11:08 +03:00
vyzo
c81ae5fc20 add some comments about the missing business and anothre log 2021-07-05 10:42:14 +03:00
vyzo
4c41f52828 add warning for missing objects for marking for debug purposes 2021-07-05 10:35:04 +03:00
vyzo
3597192d58 remove the sleeps and busy loop more times when waiting for missing objects 2021-07-05 10:31:47 +03:00
vyzo
1726eb993c deal with incomplete objects that need to be marked and protected
seems that something is writing DAGs before its consituents, which causes problems.
2021-07-05 10:22:52 +03:00
aarshkshah1992
8ecdc929b4 get retrieval pricing input should not error out on a deal state fetch 2021-07-05 09:34:56 +05:30
vyzo
db53859e7a reduce CompactionThreshold to 5 finalities
so that we run compaction every finality, once we've first compacted
2021-07-04 22:12:51 +03:00
vyzo
b08e0b7102 fix lint 2021-07-04 21:24:15 +03:00
vyzo
94efae419e reduce length of critical section
Just the purge; the rest is not critical -- e.g. it's ok if we do some duplicate copies
to the coldstore, we'll have gc soon.
2021-07-04 21:21:53 +03:00
vyzo
f33d4e79aa simplify transactional protection logic
Now that we delete objects heaviest first, we don't have to do deep walk and rescan gymnastics.
2021-07-04 20:49:39 +03:00
vyzo
40c271cda1 sort cold objects before deleting
so that we can't shoot ourselves in the foot by deleting the constituents of a DAG while it is
still in the hotstore.
2021-07-04 20:17:07 +03:00
vyzo
13d612f72f smarter trackTxnRefMany 2021-07-04 19:33:49 +03:00
vyzo
f124389b66 recursively protect all references 2021-07-04 19:21:00 +03:00
vyzo
4d286da593 fix error message 2021-07-04 18:58:39 +03:00
vyzo
680af8eb09 use deep object walking for more robust handling of transactional references 2021-07-04 18:38:28 +03:00
vyzo
1f02428225 fix lint 2021-07-04 18:38:28 +03:00
vyzo
2c7a89a1db short-circuit rescanning on block headers 2021-07-04 18:38:28 +03:00
vyzo
028a5c4942 make test do something useful again 2021-07-04 18:38:28 +03:00