Raúl Kripalani
3795cc2bd2
segregate chain and state blockstores.
...
This paves the way for better object lifetime management.
Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
sizing caches differently.
- specifying different retention policies for chain and state data.
This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
store, and only retain state objects reachable from the last finality
in the state store.
2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36
Merge branch 'master' into refactor/lib/blockstore
2021-02-28 19:55:23 +00:00
Raúl Kripalani
d1104fec4c
rename blockstores for consistency.
2021-01-29 23:17:25 +00:00
Raúl Kripalani
b0cbc932bd
consolidate all blockstores in blockstore package.
2021-01-29 20:01:00 +00:00
Raúl Kripalani
a1da1dab85
add context to LockedRepo#Datastore().
2021-01-26 11:01:43 +00:00
Peter Rabbitson
47141042b6
Add comment
...
Co-authored-by: raulk <raul@protocol.ai>
2021-01-12 18:11:18 +01:00
Peter Rabbitson
9334e73396
Snake a context through the Chain-blockstore creation
2021-01-04 15:27:52 +01:00
Raúl Kripalani
2ef8acdfc8
back out index cache usage in badger blockstore.
2020-12-02 21:35:13 +00:00
Raúl Kripalani
e8a5e0d2fe
integrate memory watchdog; impose limits on badger caches.
2020-12-02 16:40:28 +00:00
Raúl Kripalani
44040f0f3f
badger options: enable IndexCache and CompactL0OnClose.
2020-11-20 00:33:43 +00:00
Raúl Kripalani
55061ae6fb
fix blockstore directory not created automatically.
2020-11-19 15:55:02 +00:00
Peter Rabbitson
7be7a9b5e3
Stop referring to github.com/ipfs/go-ipfs-blockstore outside of lib
2020-11-12 06:01:35 +01:00
Raúl Kripalani
3577300aee
Merge branch 'master' into badger-viewable
2020-11-06 19:34:25 +00:00
Raúl Kripalani
0b2a02c8ea
remove redundant import.
2020-11-06 18:57:04 +00:00
Raúl Kripalani
85e37e4b70
make the value log loading mode mmap, to leverage zero-copy access.
2020-11-06 18:39:08 +00:00
Raúl Kripalani
0b8a21ee16
badger: restore prev. max table size (64MiB; default).
2020-11-01 17:26:37 +00:00
Raúl Kripalani
72e573d98d
fix lint.
2020-11-01 17:09:14 +00:00
Raúl Kripalani
5a98660d1b
make repo.Blockstore() idempotent; wrap in IDStore.
2020-11-01 16:56:20 +00:00
Raúl Kripalani
d8d85373f5
fix lotus-shed datastore commands.
2020-11-01 13:50:41 +00:00
Raúl Kripalani
a16d7f221e
Merge branch 'master' into badger-viewable
2020-11-01 13:10:56 +00:00
Raúl Kripalani
d2e2322fd2
make the lotus node use the new native badger blockstore.
2020-11-01 13:01:26 +00:00
Jakub Sztandera
0297be4b9a
Add lotus-shed datastore rewrite command
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-28 23:24:45 +01:00
Jakub Sztandera
ecc1d94b3a
Reduce badger ValueTreshold to 128
...
It should significntly size of the LSM index, and thus increase the
performance with bigger datastores
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-28 22:17:01 +01:00
Łukasz Magiera
8783c7434e
Merge remote-tracking branch 'origin/master' into feat/signing-backends
2020-10-09 00:50:41 +02:00
Łukasz Magiera
9b5a0815fc
backup: open datastores in readonly in offline mode
2020-10-01 17:55:47 +02:00
Aayush Rajasekaran
6abccc4d5e
Add an option to set config
2020-09-30 03:19:04 -04:00
Frank
dcae513487
make nested dir
2020-09-14 16:49:35 +08:00
Łukasz Magiera
780f6dba34
Remote wallet backends
2020-09-05 21:39:09 +02:00
Jakub Sztandera
997d49b63f
Disable GC on chain badger datastore
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-01 20:39:58 +02:00
Raúl Kripalani
efdc428d5d
keep storage-fsm (renamed to storage-sealing) and sector-storage in extern.
2020-08-17 14:26:18 +01:00
Raúl Kripalani
3c17cd655e
integrate extern/sector-storage into lotus proper.
2020-08-16 11:09:58 +01:00
Łukasz Magiera
73b8868046
gofmt
2020-08-06 03:32:59 +02:00
Łukasz Magiera
f8b8ecc0c3
Consensus filter
2020-08-06 03:16:39 +02:00
hannahhoward
8e6862f7ca
fix(lint): fix lint errors
2020-07-31 14:19:40 -07:00
hannahhoward
3cac1080cc
feat(retrievalstoremgr): add retrievalstoremgr
...
add manager for retrievals to handle different cases for IPFS/non-ipfs
2020-07-31 14:16:18 -07:00
Łukasz Magiera
1fd23248ff
gofmt
2020-07-28 16:38:26 +02:00
Łukasz Magiera
09b90773d8
Merge remote-tracking branch 'origin/next' into feat/cid-builder
2020-07-28 16:06:06 +02:00
hannahhoward
5a623cccb5
feat(markets): update markets mulitple deal stores
2020-07-27 23:13:28 -07:00
hannahhoward
dd885b7302
feat(multistore): extract multistore code to repo
...
Extract multistore + multiread blockstore to a seperate repo
2020-07-24 14:47:22 -07:00
Steven Allen
b7a4dbb07f
Support inline CIDs
...
And use the new CidBuilder from the spec actors.
This patch does not switch over to inline CIDs by default, but paves the way.
2020-07-23 23:12:32 -07:00
Frank
c38b8b1265
close file
2020-07-23 18:21:13 +08:00
Jakub Sztandera
0f5e56289e
Update stores list when deleting
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-17 22:44:16 +02:00
Jakub Sztandera
f2ebeaa7e4
Fix lint
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-17 22:29:39 +02:00
Jakub Sztandera
cc1d23a94c
Use single multi ds
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-17 22:14:37 +02:00
Łukasz Magiera
ede57c0b12
Fix retrieval datastore-key-not-found error
2020-07-10 18:33:26 +02:00
Łukasz Magiera
5a117d8edf
gofmt
2020-07-08 23:05:54 +02:00
Łukasz Magiera
66237415c5
Merge remote-tracking branch 'origin/next' into feat/client-multi-bstore
2020-07-08 22:48:47 +02:00
Łukasz Magiera
5ca1be567d
Merge pull request #2323 from filecoin-project/feat/fastretrieval
...
Support fast-retrieval deals
2020-07-08 21:54:46 +02:00
Jakub Sztandera
d3d4612575
Re-eanble compact on close
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-08 20:59:46 +02:00
Łukasz Magiera
99796220f9
Support fast-retrieval deals
2020-07-08 20:42:07 +02:00
