cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
19,307 Commits 24 Branches 342 Tags 314 MiB
fa8792a65e
Commit Graph

36 Commits

Author SHA1 Message Date
zenground0
bd0c010be6 Configur buffer and threshold 2023-03-09 08:40:14 -07:00
zenground0
a994153e27 GC respects target for max hotstore space 2023-03-07 07:39:17 -07:00
ZenGround0
4ffded6fef
feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571) 
* begin

* rough draft -- this should probably actually work?

* WIP

* Start testing

* message mode

* Fix tests, make gen

* Better default

* docsgen-cli

* Review Response

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-11-07 16:31:12 -05:00
ZenGround0
8b7be6d47e
feat:chain:splitstore auto prune (#9123) 
Auto Prune

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-08-08 16:06:32 -04:00
vyzo
6bd2d89e30 use memory blockstore for miner manifest loading 2022-04-14 19:34:26 +03:00
vyzo
9d25464703 use functional options for hotstore gc, rename MovingGC to FullGC 2021-07-27 09:53:22 +03:00
vyzo
aa0bd51b2c thread GCFrequency option into the splitstore config 2021-07-26 15:05:57 +03:00
Łukasz Magiera
da5aeda197
Merge branch 'master' into feat/splitstore-hot-messages 2021-07-22 12:58:06 +02:00
vyzo
ebbaf23af8 support out-of-chain reference protection 2021-07-20 09:02:40 +03:00
vyzo
1b77361301 add option for hotstore message retention 2021-07-17 08:35:35 +03:00
vyzo
e003203bea implement exposed splitstore 2021-07-15 13:12:10 +03:00
vyzo
c0a1cfffa1 rename noopstore to discardstore 2021-07-09 19:19:37 +03:00
vyzo
190cb18ab0 housekeeping 
- remove defunct tracking store implementations
- update splitstore node config
- use mark set type config option (defaulting to mapts); a memory constrained node
  may want to use an on-disk one
 2021-07-04 18:38:28 +03:00
vyzo
5f7ae1f489 update splistore DI constructor 2021-07-04 18:38:28 +03:00
vyzo
68bc5d2291 skip moving cold blocks when running with a noop coldstore 
it is a noop but it still takes (a lot of) time because it has to read all the cold blocks.
 2021-07-04 18:38:28 +03:00
vyzo
649b7dd162 add config option for hot headers 2021-07-04 18:38:28 +03:00
vyzo
997f2c098b keep headers hot when running with a noop splitstore 2021-07-04 18:38:28 +03:00
vyzo
04f2e102a1 kill full splitstore compaction, simplify splitstore configuration 2021-07-04 18:38:28 +03:00
vyzo
5cca29d1db hook noop blockstore for splitstore in DI 2021-07-04 18:38:28 +03:00
Łukasz Magiera
7e4cb9da84 Fix fallback chainstore 2021-04-09 16:11:02 +02:00
vyzo
6b8c60a659 don't ID wrap the hotstore 2021-03-05 14:46:18 +02:00
vyzo
dd0c308427 move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType 2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52 rename liveset => markset; rename snoop => tracking store; docs. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092 renames and polish. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f remove dependency from blockstore/splitstore => chain/store. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec move splitstore into blockstore package. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1b51c10d78 split off lmdb support to a different branch. 2021-03-05 14:46:18 +02:00
vyzo
0fc2f3a26f fix post-rebase compilation errors 2021-03-05 14:46:18 +02:00
vyzo
97abbe1eca add (salted) bloom filter liveset 2021-03-05 14:46:17 +02:00
vyzo
73259aa350 add configuration for splitstore and default to a simple compaction algorithm 2021-03-05 14:46:17 +02:00
vyzo
b83994797a separate LMDB options for hotstore and tracking stores 2021-03-05 14:46:17 +02:00
vyzo
68b6f913c7 propagate useLMDB option to splitstore through DI 2021-03-05 14:46:17 +02:00
vyzo
99c7d8e3eb more informative names for the hotstore directories 2021-03-05 14:46:17 +02:00
vyzo
a586d42c3b make hot store DI injectable in the split store, default to badger. 2021-03-05 14:46:17 +02:00
vyzo
622b4f7d9d hook splitstore into DI 2021-03-05 14:46:17 +02:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00