cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
13,010 Commits 24 Branches 342 Tags 314 MiB
bb2d99908c
Commit Graph

42 Commits

Author SHA1 Message Date
vyzo
bb2d99908c make state constants typed 2021-07-27 12:03:26 +03:00
vyzo
b82f953fd5 use the badger streaming interface in doCopy 2021-07-27 11:46:35 +03:00
vyzo
59aebba0d9 use a slab allocator for the copy 2021-07-27 11:35:21 +03:00
vyzo
0baeec0686 remove filter from doCopy; it's not used 
it was there to support a potential CopyTo interface; but we'll cross that bridge when we get there.
 2021-07-27 11:18:24 +03:00
vyzo
649fc62863 panic if we fail to correctly setup the db paths. 
we can't really continue and leave a ticking bomb for the next restart; the user might not see it.
 2021-07-27 10:27:16 +03:00
vyzo
d6ace68540 extend test to do a double move and check symlink following 2021-07-27 10:16:50 +03:00
vyzo
cbaffab9dd use EvalSymlinks in deleteDB 2021-07-27 10:08:07 +03:00
vyzo
c03859c1b5 resolve symlinks when constructing the new db path 
so that the new path is adjacent to the old path, allowing the user to symlink the db in
a different file system.
 2021-07-27 10:05:35 +03:00
vyzo
96c1123c33 use functional options in the BlockstoreGC interface 2021-07-27 09:52:06 +03:00
vyzo
938330e6c3 moveMx is not an RWMutex, just a regular mutex 2021-07-26 15:09:31 +03:00
vyzo
21e7c188da use CollectGarbage in blockstore move test, as it is the real interface 2021-07-26 15:05:57 +03:00
vyzo
c747f2f1e2 do moving GC if the user asks for it 2021-07-26 15:05:57 +03:00
vyzo
a843665132 add options to BlockstoreGC trait 2021-07-26 15:05:57 +03:00
vyzo
94509968a0 make moveTo a private method 2021-07-26 15:05:20 +03:00
vyzo
5cf6fdf81d don't heap allocate the cond, just set L 2021-07-26 15:04:46 +03:00
vyzo
524564e2cf add some more logging around move 2021-07-26 15:04:46 +03:00
vyzo
f2c7b08be5 follow symbolic links when deleting old dbs 2021-07-26 15:04:46 +03:00
vyzo
608a9f84d2 fix copy: flush discards the transaction 2021-07-26 15:04:46 +03:00
vyzo
4b0b37a4ef fix lint 
the great spellchecker strikes again
 2021-07-26 15:04:46 +03:00
vyzo
4715b1f436 fix lotus-shed 2021-07-26 15:04:46 +03:00
vyzo
aec126879e add MoveTo test 2021-07-26 15:04:46 +03:00
vyzo
001c04f2dd use pooled slices for the copy 2021-07-26 15:04:46 +03:00
vyzo
b741d61b20 implement BlockstoreMover in badger 2021-07-26 15:04:46 +03:00
vyzo
2cfd73c879 manually compute size when badger is being stupid 2021-07-26 09:46:21 +03:00
vyzo
30e4b405b7 implement BlockstoreSize for badger 2021-07-26 08:43:09 +03:00
vyzo
c93328b036 use the new traits for hotstore gc 2021-07-14 22:59:52 +03:00
vyzo
35180b4761 merge Compact and CollectGarbage in badger 2021-07-14 22:59:52 +03:00
vyzo
909f7039d4 make badger Close-safe 2021-07-09 09:54:12 +03:00
vyzo
08cad30be2 reuse key buffer in badger ForEachKey 
cid copies the bytes so it's safe
 2021-07-04 18:38:28 +03:00
vyzo
d476a3db2c BlockstoreIterator trait with implementation for badger 2021-07-04 18:38:28 +03:00
Łukasz Magiera
5f80869fe0
Merge pull request #5794 from filecoin-project/fix/atomic-first 
fix: make sure atomic 64bit fields are 64bit aligned
 2021-03-25 13:29:59 +01:00
Łukasz Magiera
c69b26cfc6
Merge pull request #5778 from filecoin-project/feat/splitstore-compact-hotstore 
splitstore: compact hotstore prior to garbage collection
 2021-03-12 16:30:05 +01:00
Steven Allen
bba71da401 fix: return buffers after canceling badger operation 
In theory, Delete/Put could fail. If it does, we'll return the buffers
to the pool before we're really done with them.

In practice, this is almost certainly not an issue as badger shouldn't
_use_ the buffer unless we flush. But I feel slightly safer this way.
 2021-03-11 20:30:43 -08:00
Steven Allen
1c490f3fda fix: make sure atomic 64bit fields are 64bit aligned 
Otherwise, this won't work on 32bit ARM.
 2021-03-11 20:10:39 -08:00
vyzo
01ce9b5c44 add Compact to badger blockstore 2021-03-11 11:45:05 +02:00
vyzo
90741da019 tune badger gc to repeated gc the value log until there is no rewrite 2021-03-08 21:46:44 +02:00
vyzo
3d1b855f20 rename GC to CollectGarbage, ignore badger.ErrNoRewrite 2021-03-08 19:42:38 +02:00
vyzo
8562a9bb82 garbage collect hotstore after compaction 2021-03-08 18:12:09 +02:00
vyzo
86b73d651e add DeleteMany to Blockstore interface 2021-03-05 14:46:18 +02:00
Raúl Kripalani
b34b4e0374 fix test compilation error. 2021-02-28 23:10:01 +00:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00