Łukasz Magiera
5e5de9b41b
Merge pull request #6679 from filecoin-project/fix/get-retrieval-pricing-input-error
...
Get retrieval pricing input should not error out on a deal state fetch
2021-07-05 16:18:48 +02:00
Łukasz Magiera
a56b6fea99
Merge pull request #6680 from filecoin-project/chore/double-encoded-cid
...
Fix more CID double-encoding as hex
2021-07-05 16:14:52 +02:00
vyzo
5a099b7d05
more commentary on the missing refs situation
2021-07-05 16:12:17 +03:00
vyzo
59639a0788
reinstate some better code for handling missing references.
2021-07-05 16:08:08 +03:00
Anton Evangelatov
5a87f04e71
fix finalize sectors early test
2021-07-05 13:36:23 +02:00
vyzo
fa195bede2
get rid of ugly missing reference handling code
...
those missing objects don't seem to ever get there, are they from an abandoned fork?
2021-07-05 14:29:55 +03:00
Anton Evangelatov
4be0a7a215
resolve conflicts with master
2021-07-05 13:13:32 +02:00
Peter Rabbitson
a544fa0f97
Fix more CID doucle-encoding as hex
2021-07-05 12:51:15 +02:00
aarshkshah1992
add113f585
use multierror
2021-07-05 16:08:51 +05:30
vyzo
59936ef468
fix log
2021-07-05 13:30:31 +03:00
aarshkshah1992
44985722d2
changes as per review
2021-07-05 15:53:41 +05:30
vyzo
0b7153be86
use internal version of has for occurs checks
2021-07-05 12:41:11 +03:00
vyzo
d8b8d75e0f
readd minute delay before trying for missing objects
2021-07-05 12:38:09 +03:00
vyzo
d7709deb2b
reduce memory pressure from marksets when the size is decreased
2021-07-05 11:51:22 +03:00
vyzo
3ec834b2e3
improve logs and error messages
2021-07-05 11:41:09 +03:00
vyzo
918a7ec749
a bit more fil commitment short-circuiting
2021-07-05 11:38:53 +03:00
vyzo
2ea2abc07d
short-circuit fil commitments
...
they don't make it to the blockstore anyway
2021-07-05 11:32:52 +03:00
vyzo
839f7bd2b5
only occur check for DAGs
2021-07-05 11:11:08 +03:00
vyzo
c81ae5fc20
add some comments about the missing business and anothre log
2021-07-05 10:42:14 +03:00
vyzo
4c41f52828
add warning for missing objects for marking for debug purposes
2021-07-05 10:35:04 +03:00
vyzo
3597192d58
remove the sleeps and busy loop more times when waiting for missing objects
2021-07-05 10:31:47 +03:00
vyzo
1726eb993c
deal with incomplete objects that need to be marked and protected
...
seems that something is writing DAGs before its consituents, which causes problems.
2021-07-05 10:22:52 +03:00
aarshkshah1992
8ecdc929b4
get retrieval pricing input should not error out on a deal state fetch
2021-07-05 09:34:56 +05:30
vyzo
db53859e7a
reduce CompactionThreshold to 5 finalities
...
so that we run compaction every finality, once we've first compacted
2021-07-04 22:12:51 +03:00
vyzo
b08e0b7102
fix lint
2021-07-04 21:24:15 +03:00
vyzo
94efae419e
reduce length of critical section
...
Just the purge; the rest is not critical -- e.g. it's ok if we do some duplicate copies
to the coldstore, we'll have gc soon.
2021-07-04 21:21:53 +03:00
vyzo
f33d4e79aa
simplify transactional protection logic
...
Now that we delete objects heaviest first, we don't have to do deep walk and rescan gymnastics.
2021-07-04 20:49:39 +03:00
vyzo
40c271cda1
sort cold objects before deleting
...
so that we can't shoot ourselves in the foot by deleting the constituents of a DAG while it is
still in the hotstore.
2021-07-04 20:17:07 +03:00
vyzo
13d612f72f
smarter trackTxnRefMany
2021-07-04 19:33:49 +03:00
vyzo
f124389b66
recursively protect all references
2021-07-04 19:21:00 +03:00
vyzo
4d286da593
fix error message
2021-07-04 18:58:39 +03:00
vyzo
680af8eb09
use deep object walking for more robust handling of transactional references
2021-07-04 18:38:28 +03:00
vyzo
1f02428225
fix lint
2021-07-04 18:38:28 +03:00
vyzo
2c7a89a1db
short-circuit rescanning on block headers
2021-07-04 18:38:28 +03:00
vyzo
028a5c4942
make test do something useful again
2021-07-04 18:38:28 +03:00
vyzo
8e56fffb33
walkChain should visit the genesis state root
2021-07-04 18:38:28 +03:00
vyzo
95c3aaec9a
fix test
2021-07-04 18:38:28 +03:00
vyzo
190cb18ab0
housekeeping
...
- remove defunct tracking store implementations
- update splitstore node config
- use mark set type config option (defaulting to mapts); a memory constrained node
may want to use an on-disk one
2021-07-04 18:38:28 +03:00
vyzo
19d1b1f532
deal with partially written objects
2021-07-04 18:38:28 +03:00
vyzo
0a1d7b3732
fix log
2021-07-04 18:38:28 +03:00
vyzo
08cad30be2
reuse key buffer in badger ForEachKey
...
cid copies the bytes so it's safe
2021-07-04 18:38:28 +03:00
vyzo
eafffc1634
more efficient trackTxnRefMany
2021-07-04 18:38:28 +03:00
vyzo
36f93649ef
fix panic from concurrent map writes in txnRefs
2021-07-04 18:38:28 +03:00
vyzo
6fa2cd232d
simplify compaction model
2021-07-04 18:38:28 +03:00
vyzo
5f7ae1f489
update splistore DI constructor
2021-07-04 18:38:28 +03:00
vyzo
1f2b604c07
RIP tracking store
2021-07-04 18:38:28 +03:00
vyzo
d476a3db2c
BlockstoreIterator trait with implementation for badger
2021-07-04 18:38:28 +03:00
vyzo
68a83500bc
fix bug that turned candidate filtering to dead code
2021-07-04 18:38:28 +03:00
vyzo
00fcf6dd72
add staging cache to bolt tracking store
2021-07-04 18:38:28 +03:00
vyzo
642f0e4740
deal with memory pressure, don't walk under the boundary
2021-07-04 18:38:28 +03:00
