cerc-io/lotus
15
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
21,262 Commits 24 Branches 342 Tags 314 MiB
62961c3eee
Commit Graph

73 Commits

Author SHA1 Message Date
Jorropo
6c01310728
chore: migrate to boxo 
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
 2023-06-19 14:45:05 -07:00
Aayush Rajasekaran
7903224ae5
Merge pull request #10680 from filecoin-project/steb/blockstore-has-check 
feat: badger: add a has check before writing to reduce duplicates
 2023-04-26 16:17:16 -04:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo 
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
 2023-04-18 17:22:18 +02:00
Steven Allen
0cff56a16d feat: badger: add a has check before writing to reduce duplicates 2023-04-17 10:57:41 -07:00
zenground0
d77ac5ddcb Stop swallowing errors 2023-04-10 17:16:13 +02:00
zenground0
0ce9ae4809 Implement yield friendly online GC 2023-04-10 17:16:01 +02:00
Phi-rjan
b240031605
Limit moving gc threads (#10623) 
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2023-04-05 11:21:32 -04:00
zenground0
c281d053d2 Limit moving gc threads 2023-04-04 14:07:36 -06:00
Peter Rabbitson
6afa99d0a1 Plumb through a proper Flush() method on all blockstores 2023-03-14 02:30:15 +01:00
zenground0
71b21db0d9 chain prune hot -- hotstore online gc 2023-03-03 11:14:52 -05:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format 
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
 2023-01-26 17:03:18 +01:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Masih H. Derkani
6f686cae11 Upgrade to go-blockstore 1.2.0 and go-cid 0.2.0 
Upgrade required dependencies and fix breaking changes.
 2022-06-29 14:55:22 -04:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir 2022-03-17 12:06:52 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272) 
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
 2022-03-16 18:37:34 +01:00
Eng Zer Jun
c229842f57
test: use T.TempDir to create temporary test directory 
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-03-15 00:21:31 +08:00
vyzo
976a3a5e7b fix blockstore test contexts 2021-12-17 13:44:05 +02:00
vyzo
dd327f0b22 plumb more contexts 2021-12-17 11:42:09 +02:00
Aayush Rajasekaran
dfb65ed89f Plumb contexts through 2021-12-11 17:04:00 -05:00
whyrusleeping
072297e661 WIP: updating to new datastore/blockstore code with contexts 2021-12-09 14:12:25 +02:00
vyzo
5e8b2c7860 make symlink helper freestanding 2021-07-29 08:35:53 +03:00
vyzo
b75ff37448
fix typo 
Co-authored-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-07-28 17:11:04 +03:00
vyzo
c65d72fbca fix format specifier 2021-07-28 16:20:25 +03:00
vyzo
fff1c0ae57 improve detection of relative links 2021-07-28 16:15:39 +03:00
vyzo
297c9e2f7a extend test to check the validity of relative links 2021-07-28 11:56:35 +03:00
vyzo
0740274b7c make relative links when the canonical and new db paths are in the same directory 2021-07-28 11:56:23 +03:00
vyzo
950ded454d code cosmetics: rename variables for better readability and some comments 2021-07-28 11:49:42 +03:00
vyzo
c21c413695 remove db2 to dbNext 2021-07-27 12:06:40 +03:00
vyzo
bb2d99908c make state constants typed 2021-07-27 12:03:26 +03:00
vyzo
b82f953fd5 use the badger streaming interface in doCopy 2021-07-27 11:46:35 +03:00
vyzo
59aebba0d9 use a slab allocator for the copy 2021-07-27 11:35:21 +03:00
vyzo
0baeec0686 remove filter from doCopy; it's not used 
it was there to support a potential CopyTo interface; but we'll cross that bridge when we get there.
 2021-07-27 11:18:24 +03:00
vyzo
649fc62863 panic if we fail to correctly setup the db paths. 
we can't really continue and leave a ticking bomb for the next restart; the user might not see it.
 2021-07-27 10:27:16 +03:00
vyzo
d6ace68540 extend test to do a double move and check symlink following 2021-07-27 10:16:50 +03:00
vyzo
cbaffab9dd use EvalSymlinks in deleteDB 2021-07-27 10:08:07 +03:00
vyzo
c03859c1b5 resolve symlinks when constructing the new db path 
so that the new path is adjacent to the old path, allowing the user to symlink the db in
a different file system.
 2021-07-27 10:05:35 +03:00
vyzo
96c1123c33 use functional options in the BlockstoreGC interface 2021-07-27 09:52:06 +03:00
vyzo
938330e6c3 moveMx is not an RWMutex, just a regular mutex 2021-07-26 15:09:31 +03:00
vyzo
21e7c188da use CollectGarbage in blockstore move test, as it is the real interface 2021-07-26 15:05:57 +03:00
vyzo
c747f2f1e2 do moving GC if the user asks for it 2021-07-26 15:05:57 +03:00
vyzo
a843665132 add options to BlockstoreGC trait 2021-07-26 15:05:57 +03:00
vyzo
94509968a0 make moveTo a private method 2021-07-26 15:05:20 +03:00
vyzo
5cf6fdf81d don't heap allocate the cond, just set L 2021-07-26 15:04:46 +03:00
vyzo
524564e2cf add some more logging around move 2021-07-26 15:04:46 +03:00
vyzo
f2c7b08be5 follow symbolic links when deleting old dbs 2021-07-26 15:04:46 +03:00
vyzo
608a9f84d2 fix copy: flush discards the transaction 2021-07-26 15:04:46 +03:00
vyzo
4b0b37a4ef fix lint 
the great spellchecker strikes again
 2021-07-26 15:04:46 +03:00