cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
12,413 Commits 24 Branches 342 Tags 314 MiB
30dbe4978b
Commit Graph

100 Commits

Author SHA1 Message Date
vyzo
0a2f2cf00d use the right condition for triggering the miss metric 2021-03-05 14:48:59 +02:00
vyzo
2b32c2e597 add some metrics 2021-03-05 14:48:57 +02:00
vyzo
99d21573da remove DEBUG log spam 2021-03-05 14:46:18 +02:00
vyzo
c58df3f079 don't panic on compaction errors 2021-03-05 14:46:18 +02:00
vyzo
9bd009d795 use atomics to demarkate critical section and limit close delay 2021-03-05 14:46:18 +02:00
vyzo
17be7d3919 save markSetSize 2021-03-05 14:46:18 +02:00
vyzo
aff0f1ed4c deduplicate code for batch deletion 2021-03-05 14:46:18 +02:00
vyzo
5fb6a907cb fix loop condition in batch deletion 2021-03-05 14:46:18 +02:00
vyzo
98a7b884fe implement DeleteMany in union blockstore 2021-03-05 14:46:18 +02:00
vyzo
fdd877534f walk at boundary epoch, 2 finalities from current epoch, to find live objects 
objects written after that are retained anyway.
 2021-03-05 14:46:18 +02:00
vyzo
508fcb9d26 properly close snoop at shutdown 2021-03-05 14:46:18 +02:00
vyzo
47d8c87486 fix log 2021-03-05 14:46:18 +02:00
vyzo
11b2f41804 overestimate markSetSize a bit 2021-03-05 14:46:18 +02:00
vyzo
6b680d112b do tracker purge in smaller batches 2021-03-05 14:46:18 +02:00
vyzo
d2d0980532 don't delete in one giant batch, use smaller chunks of batchSize 2021-03-05 14:46:18 +02:00
vyzo
70ebb2ad8d improve startup log 2021-03-05 14:46:18 +02:00
vyzo
006c55a7c9 add startup log 2021-03-05 14:46:18 +02:00
vyzo
06d8ea10b1 batch delete during the cold purge 2021-03-05 14:46:18 +02:00
vyzo
4c05ec28ba fix FromDatastore to not do double adapting 2021-03-05 14:46:18 +02:00
vyzo
ab52e34e6a add comment 
Co-authored-by: raulk <raul@protocol.ai>
 2021-03-05 14:46:18 +02:00
vyzo
86fdad2e31 fix typo 
Co-authored-by: raulk <raul@protocol.ai>
 2021-03-05 14:46:18 +02:00
vyzo
2ff5aec80e satisfy linter, use Prefix for common path of non inline CIDs 2021-03-05 14:46:18 +02:00
vyzo
8a55b73146 fix the situation with WrapIDStore 2021-03-05 14:46:18 +02:00
vyzo
86b73d651e add DeleteMany to Blockstore interface 2021-03-05 14:46:18 +02:00
vyzo
c762536dcb deduplicate code 2021-03-05 14:46:18 +02:00
vyzo
5184bc5c40 log consistency for full compaction 2021-03-05 14:46:18 +02:00
vyzo
68213a92cb use ioutil.TempDir for test directories 2021-03-05 14:46:18 +02:00
vyzo
35d466d847 use sha256 for bloom key rehashing 2021-03-05 14:46:18 +02:00
vyzo
f651f43c5e improve comment accuracy 2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52 rename liveset => markset; rename snoop => tracking store; docs. 2021-03-05 14:46:18 +02:00
vyzo
48f253328d increase batch size to 16K 2021-03-05 14:46:18 +02:00
vyzo
ce68b9b229 batch writes during warm up 2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092 renames and polish. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f remove dependency from blockstore/splitstore => chain/store. 2021-03-05 14:46:18 +02:00
vyzo
b9400c590f use crypto/rand for bloom salt 2021-03-05 14:46:18 +02:00
vyzo
e612fff1fe also estimate liveset size during warm up 2021-03-05 14:46:18 +02:00
vyzo
748dd962d8 snake current tipset from head change notification 2021-03-05 14:46:18 +02:00
vyzo
cb36d5b6a4 warm up splitstore at first head change notification 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec move splitstore into blockstore package. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
68b8e8e9cb implement unionBlockstore#HashOnRead. 2021-03-02 21:22:24 +00:00
Raúl Kripalani
2047a74958 implement blockstore.Union, a union blockstore. 
The union blockstore takes a list of blockstores. It returns the first
satisfying read, and broadcasts writes to all stores.

It can be used for operations that require reading from any two blockstores,
for example WalkSnapshot.
 2021-03-02 17:03:11 +00:00
Raúl Kripalani
b34b4e0374 fix test compilation error. 2021-02-28 23:10:01 +00:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Raúl Kripalani
853de3daf7 fix TimedCacheBlockstore#View. 2021-02-28 22:39:00 +00:00
Raúl Kripalani
45a650c012 remove unnecessary View casting. 2021-02-28 22:20:29 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Raúl Kripalani
8601e5da3a address review comments. 2021-02-28 19:44:02 +00:00
Raúl Kripalani
35d1e3d1e0 refine docs. 2021-01-29 23:24:44 +00:00
Raúl Kripalani
d1104fec4c rename blockstores for consistency. 2021-01-29 23:17:25 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00