Steven Allen
9872ca7f12
correctness fixes for the autobatch blockstore
...
1. Simplify shutdown and make it idempotent by using a context.
2. Make sure `Flush` actually _fully_ flushes if the previous flush failed.
3. Don't clear the flush batch if flushing fails.
2022-01-14 17:17:51 -05:00
Aayush Rajasekaran
107f80274c
Don't lock in Get
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
4ec8fbf1ff
Address review part 2
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
5b38419c5f
Address review
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
dafd1f77fa
Support faster Get, retry flushes on error
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
ce6e328d52
Use channels to trigger flushes in a dedicated goroutine
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
2bc4b35f51
implement stubs
2022-01-14 17:15:44 -05:00
Aayush Rajasekaran
031b06be77
cache added cids
2022-01-14 17:15:43 -05:00
Aayush Rajasekaran
b9474cf0a2
Implement an autobatcher
2022-01-14 17:15:20 -05:00
vyzo
0ca95a5ea0
satisfy the spellchecker that masquarades as a linter
2022-01-14 17:14:33 -05:00
vyzo
6cfee12f31
add unit test for compaction supression
2022-01-14 17:14:32 -05:00
vyzo
6ac3d529ef
add logic for supressing compaction near upgrade boundaries
2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
b3966ab4eb
Fix typo in comment
2021-08-24 20:39:28 -04:00
Steven Allen
cbf69f6ded
fix: make TestTimedCacheBlockstoreSimple pass reliably
2021-08-24 11:35:56 -07:00
vyzo
eb0a62e9e9
RIP bloom filter markset; you weren't used anyway.
2021-08-10 10:47:01 +03:00
vyzo
742c85bf81
add SupportsVisitor in the markset env interface
2021-08-10 10:47:01 +03:00
vyzo
79f348a01c
make the write lock scope limited within a function
...
Also avoid removing the writing set if there was an error while writing.
2021-08-10 10:47:01 +03:00
vyzo
a9403b42f0
deduplicate put code
2021-08-10 10:47:01 +03:00
vyzo
1a59b73adc
add .tmp suffix to transient db names
2021-08-10 10:47:01 +03:00
vyzo
26a5832f92
simplify and deduplicate Has/Visit using helper methods tryPending and tryDB.
2021-08-10 10:47:01 +03:00
vyzo
380e16d465
remove redundant writers state variable
2021-08-10 10:47:01 +03:00
vyzo
b83b5405c6
widen MarkSetVisitor interface and get rid of the casts
2021-08-10 10:47:01 +03:00
vyzo
57c984cea1
improve concurrency properties of Visit with optimistic concurrency
2021-08-10 10:47:01 +03:00
vyzo
563fa1e31a
take advantage of MarkSet/Visitor unification to atomically check+mark
2021-08-10 10:47:01 +03:00
vyzo
cb3c53664d
unify marksets and visitors
2021-08-10 10:47:01 +03:00
vyzo
3c994d94aa
deduplicate some code in markset_badger
2021-08-10 10:47:01 +03:00
vyzo
32d94d4e85
reset walked set as epoch boundaries are crossed
...
the walk is BFS, so we can do this!
2021-08-10 10:47:01 +03:00
vyzo
49346f5679
use visitors instead of cidsets in walks
2021-08-10 10:47:01 +03:00
vyzo
1323d8fb20
markset-backed visitors
2021-08-10 10:47:01 +03:00
vyzo
6f22cffb6b
object visitor interface
2021-08-10 10:47:01 +03:00
vyzo
a768dd84fa
document splitstore clear shed command
2021-08-01 19:18:34 +03:00
vyzo
5e8b2c7860
make symlink helper freestanding
2021-07-29 08:35:53 +03:00
vyzo
b75ff37448
fix typo
...
Co-authored-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-07-28 17:11:04 +03:00
vyzo
c65d72fbca
fix format specifier
2021-07-28 16:20:25 +03:00
vyzo
fff1c0ae57
improve detection of relative links
2021-07-28 16:15:39 +03:00
vyzo
297c9e2f7a
extend test to check the validity of relative links
2021-07-28 11:56:35 +03:00
vyzo
0740274b7c
make relative links when the canonical and new db paths are in the same directory
2021-07-28 11:56:23 +03:00
vyzo
950ded454d
code cosmetics: rename variables for better readability and some comments
2021-07-28 11:49:42 +03:00
vyzo
da3e36acdc
Merge branch 'master' into feat/moving-gc
2021-07-27 12:12:38 +03:00
vyzo
8809707158
update README
2021-07-27 12:08:39 +03:00
vyzo
c21c413695
remove db2 to dbNext
2021-07-27 12:06:40 +03:00
vyzo
bb2d99908c
make state constants typed
2021-07-27 12:03:26 +03:00
vyzo
b82f953fd5
use the badger streaming interface in doCopy
2021-07-27 11:46:35 +03:00
vyzo
59aebba0d9
use a slab allocator for the copy
2021-07-27 11:35:21 +03:00
vyzo
0baeec0686
remove filter from doCopy; it's not used
...
it was there to support a potential CopyTo interface; but we'll cross that bridge when we get there.
2021-07-27 11:18:24 +03:00
vyzo
649fc62863
panic if we fail to correctly setup the db paths.
...
we can't really continue and leave a ticking bomb for the next restart; the user might not see it.
2021-07-27 10:27:16 +03:00
vyzo
d6ace68540
extend test to do a double move and check symlink following
2021-07-27 10:16:50 +03:00
vyzo
cbaffab9dd
use EvalSymlinks in deleteDB
2021-07-27 10:08:07 +03:00
vyzo
c03859c1b5
resolve symlinks when constructing the new db path
...
so that the new path is adjacent to the old path, allowing the user to symlink the db in
a different file system.
2021-07-27 10:05:35 +03:00
vyzo
9d25464703
use functional options for hotstore gc, rename MovingGC to FullGC
2021-07-27 09:53:22 +03:00
