Commit Graph

244 Commits

Author SHA1 Message Date
Jorropo
6c01310728
chore: migrate to boxo
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
2023-06-19 14:45:05 -07:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
2023-04-18 17:22:18 +02:00
Marten Seemann
db6efe6512 use go-libipfs/files instead of ipfs/go-ipfs-files 2023-03-08 17:50:56 +13:00
hannahhoward
16dad6342c
feat(deps): update to go-data-transfer v2, still wip 2023-03-07 11:44:29 -08:00
Łukasz Magiera
401359646a netbs: Address review 2022-11-08 09:37:43 +00:00
Łukasz Magiera
73a515b93f make lint happy 2022-11-08 09:37:43 +00:00
Łukasz Magiera
2c89b3240f retrieval: Support retrievals into remote stores 2022-11-08 09:37:34 +00:00
Aayush
c0b7343e60 Merge branch 'release/v1.18.0' into asr/merge-release-into-master 2022-11-06 14:40:13 -05:00
Geoff Stuart
b4c04ad927 update markets 2022-10-06 11:06:21 -04:00
Geoff Stuart
a1e08e5664 Fix a calculation error in price-per-byte in retrievals 2022-09-21 15:35:52 -04:00
Łukasz Magiera
2086b219d2 Don't use go-libp2p-core 2022-08-25 14:20:41 -04:00
Dirk McCormick
2d2b06464c feat: update go-fil-markets to v1.23.0 2022-07-06 11:43:06 -04:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
Aayush
339b7db2b7 Integrate FIP 0027 2022-04-14 19:32:56 +03:00
Łukasz Magiera
6bef1aeb82 feat: cli: lotus client list-asks --protocols 2022-04-11 19:49:52 +02:00
Anton Evangelatov
0ed8647b6f extract unixfs filestore into lib 2022-03-21 10:48:52 +01:00
Łukasz Magiera
0d9e34d651 client impl: Fix typo in ClientCalcCommP CopyBuffer error 2022-03-10 22:41:45 +01:00
Łukasz Magiera
a715fb3063 feat: Make ClientCalcCommP multithreaded 2022-03-09 23:25:29 +01:00
Aayush
fd8cb44e67 fix: client: calculate commps for pieces bigger than 32GB 2022-02-24 10:27:37 -05:00
Dirk McCormick
b1734f84b3 feat: retrieval ask CLI command 2021-12-17 15:55:12 +01:00
Aayush Rajasekaran
dfb65ed89f Plumb contexts through 2021-12-11 17:04:00 -05:00
Łukasz Magiera
d019853687
review: Cleanup some comments
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2021-11-29 21:29:00 +01:00
Łukasz Magiera
5b5e6b9e44 retrieval: DagSpec.MatchPath -> ExportMerkleProof 2021-11-29 21:14:00 +01:00
Łukasz Magiera
61791b90ea retrieval: Only output matching nodes, MatchPath dagspec 2021-11-29 20:40:55 +01:00
Peter Rabbitson
d5b4d93575 Cleanup partial retrieval codepaths ( zero functional changes )
- Adjust the ExportRef comment to reflect change from 
- Simplify control flow in parseDagSpec() ( read diff under -w )
2021-11-25 10:35:05 +01:00
Łukasz Magiera
0c884734a3
Merge pull request from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars
Chore/dm level tests plus merkle proof cars
2021-11-24 20:43:39 +01:00
Łukasz Magiera
bd4927b494 retrieval: Cleanup some comments 2021-11-24 20:13:49 +01:00
Peter Rabbitson
46937a1b9f Add tests cementing the DM-level behavior on simple paths 2021-11-24 15:31:15 +01:00
Łukasz Magiera
78949d6c05 retrieval: Fix deadlock in ClientGetRetrievalUpdates 2021-11-23 18:32:56 +01:00
Łukasz Magiera
407c2ed114 retrieval: Drop the RootSelector hack 2021-11-23 17:42:43 +01:00
Łukasz Magiera
25e89d3a7a retrieval: require unixfs exports to be aligned on block boundary 2021-11-22 13:29:09 +01:00
Łukasz Magiera
2c583b03ff retrieval: Support DM-paths in ls 2021-11-22 12:46:29 +01:00
Łukasz Magiera
74f645a098 retrieval: Fix parseDagSpec root check 2021-11-22 12:46:29 +01:00
Łukasz Magiera
597b72e286 retrieval: Fix lint, cli docsgen 2021-11-22 12:46:29 +01:00
Łukasz Magiera
9c119bfdad retrieval: Make the ls command work 2021-11-22 12:46:17 +01:00
Łukasz Magiera
450d0687da retrieval: REST export endpoint 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b83a9b902a gofmt 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b26906963b retrieval: Support multi-root export 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b9bd061bdd fix unixfs selector root node selection 2021-11-22 12:46:04 +01:00
Łukasz Magiera
b868769ec8 more retrieval api work 2021-11-22 12:46:02 +01:00
Łukasz Magiera
89138bab4d Simplify retrieval APIs 2021-11-22 12:45:42 +01:00
Łukasz Magiera
60ea33b1c7 client: cleanup clientRetrieve 2021-11-22 12:45:42 +01:00
Łukasz Magiera
08e297a217 client: Support json selectors in retrieval 2021-11-22 12:45:42 +01:00
Peter Rabbitson
a40aa9f90b Normlize selector use within lotus 2021-10-07 11:01:29 +02:00
Peter Rabbitson
f8ac98bf46 Adjust error wording 2021-10-05 18:16:14 +02:00
Peter Rabbitson
5e6aceeb0e Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval 2021-10-05 18:15:51 +02:00
Rod Vagg
97c2cdb8d6 traversals: limit link load count for SelectiveCar use as well 2021-10-05 10:47:49 +02:00
Peter Rabbitson
af0d9b650d Error on corner cases with corresponding negative tests 2021-10-04 23:21:24 +02:00
Peter Rabbitson
0444435589 Expose basic text-based datamodel selector on retrieval
Syntaxt of selection is located at
https://pkg.go.dev/github.com/ipld/go-ipld-selector-text-lite#SelectorSpecFromPath

Example use, assuming that:
  - The root of the deal is a plain dag-pb unixfs directory
  - The directory is not sharded
  - The user wants to retrieve the first entry in that directory

  lotus client retrieve --miner f0XXXXX --datamodel-path-selector 'Links/0/Hash' bafyROOTCID ~/output

For a much more elaborate example see the top of ./itests/deals_partial_retrieval_test.go
2021-09-10 09:44:11 +02:00