Commit Graph

387 Commits

Author SHA1 Message Date
Łukasz Magiera
9e03fcab83
feat: sealing: Support nv22 DDO features in the sealing pipeline (#11226)
* Initial work supporting DDO pieces in lotus-miner

* sealing: Update pipeline input to operate on UniversalPiece

* sealing: Update pipeline checks/sealing states to operate on UniversalPiece

* sealing: Make pipeline build with UniversalPiece

* move PieceDealInfo out of api

* make gen

* make sealing pipeline unit tests pass

* fix itest ensemble build

* don't panic in SectorsStatus with deals

* stop linter from complaining about checkPieces

* fix sector import tests

* mod tidy

* sealing: Add logic for (pre)committing DDO sectors

* sealing: state-types with method defs

* DDO non-snap pipeline works(?), DDO Itests

* DDO support in snapdeals pipeline

* make gen

* update actor bundles

* update the gst market fix

* fix: chain: use PreCommitSectorsBatch2 when setting up genesis

* some bug fixes

* integration working changes

* update actor bundles

* Make TestOnboardRawPieceSnap pass

* Appease the linter

* Make deadlines test pass with v12 actors

* Update go-state-types, abstract market DealState

* make gen

* mod tidy, lint fixes

* Fix some more tests

* Bump version in master

Bump version in master

* Make gen

Make gen

* fix sender

* fix: lotus-provider: Fix winning PoSt

* fix: sql Scan cannot write to an object

* Actually show miner-addrs in info-log

Actually show miner-addrs in lotus-provider info-log

* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* ddo is now nv22

* make gen

* temp actor bundle with ddo

* use working go-state-types

* gst with v13 market migration

* update bundle, builtin.MethodsMiner.ProveCommitSectors2 -> 3

* actually working v13 migration, v13 migration itest

* Address review

* sealing: Correct DDO snap pledge math

* itests: Mixed ddo itest

* pipeline: Fix sectorWeight

* sealing: convert market deals into PAMs in mixed sectors

* sealing: make market to ddo conversion work

* fix lint

* update gst

* Update actors and GST to lastest integ branch

* commit batcher: Update ProveCommitSectors3Params builder logic

* make gen

* use builtin-actors master

* ddo: address review

* itests: Add commd assertions to ddo tests

* make gen

* gst with fixed types

* config knobs for RequireActivationSuccess

* storage: Drop obsolete flaky tasts

---------

Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>
2024-01-25 15:15:55 +01:00
Andrew Jackson (Ajax)
2bc4951d79 Merge branch 'release/v1.24.0' into release/v1.25.0 2023-10-17 10:19:51 -05:00
Phi
024dd5e08e Merge branch 'feat/nv21' into feat/implement-SyntheticPoRep 2023-09-08 16:41:39 +02:00
Łukasz Magiera
556815f844 sealing pipeline: Set commd in precommit params 2023-08-29 13:47:20 +02:00
Anton Evangelatov
9028df409c fix: markets/dagstore: remove trace goroutine for dagstore wrapper 2023-08-21 17:55:45 +02:00
Łukasz Magiera
40cfe626b8 sealing pipeline: Set commd in precommit params 2023-08-08 15:12:50 +02:00
Andrew Jackson (Ajax)
8281b26f94 Synthetic PoRep provider interface solution 2023-07-13 15:47:59 +02:00
Andrew Jackson (Ajax)
7f306a5b02 Synthetic PoRep Impl 2023-07-13 15:47:59 +02:00
Jorropo
6c01310728
chore: migrate to boxo
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
2023-06-19 14:45:05 -07:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
2023-04-18 17:22:18 +02:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
hannahhoward
16dad6342c
feat(deps): update to go-data-transfer v2, still wip 2023-03-07 11:44:29 -08:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
2023-01-26 17:03:18 +01:00
Dirk McCormick
a02c783f0e fix: dagstore miner api test 2023-01-10 14:50:38 +01:00
LexLuthr
a425a38e29 change connection initiation direction 2022-11-10 21:51:51 +05:30
Łukasz Magiera
401359646a netbs: Address review 2022-11-08 09:37:43 +00:00
Łukasz Magiera
1513aab4c8 netbs: Add more missing locks 2022-11-08 09:37:43 +00:00
Łukasz Magiera
73a515b93f make lint happy 2022-11-08 09:37:43 +00:00
Łukasz Magiera
8c1bc10aa4 apistore: Lock accesses 2022-11-08 09:37:43 +00:00
Łukasz Magiera
2c89b3240f retrieval: Support retrievals into remote stores 2022-11-08 09:37:34 +00:00
Aayush
c0b7343e60 Merge branch 'release/v1.18.0' into asr/merge-release-into-master 2022-11-06 14:40:13 -05:00
Geoff Stuart
b4c04ad927 update markets 2022-10-06 11:06:21 -04:00
Aayush
54a4f14d09 feat: dealpublisher: check for duplicate deals before adding 2022-09-23 12:23:42 -04:00
Aayush
4382bbcc96 feat: add logs to markets event handler 2022-09-16 15:36:08 -04:00
Aayush
19c3232d2f more fixes 2022-09-10 12:37:03 -04:00
Geoff Stuart
642555d0fb Fix discrepancies, run fiximports 2022-09-09 21:40:15 -04:00
Aayush
0efca4d266 introduce v9 actors and nv17 2022-09-09 21:40:15 -04:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Łukasz Magiera
2086b219d2 Don't use go-libp2p-core 2022-08-25 14:20:41 -04:00
Łukasz Magiera
c90826e670
Merge pull request #9010 from LexLuthr/feat/dagstore-destroyshard
feat: dagstore: Add DestroyShard() in dagstore wrapper
2022-07-12 14:41:52 +02:00
LexLuthr
0e2118e12e fix lint errors 2022-07-11 17:24:53 +05:30
LexLuthr
89b3119220 mock test for DestroyShard 2022-07-11 17:11:35 +05:30
LexLuthr
2b63a9d0bd add destroyshard to wrapper 2022-07-11 15:00:12 +05:30
Łukasz Magiera
ae17b765c0 storage: Address review, no tok 2022-06-17 18:25:59 +02:00
Łukasz Magiera
58416d0881 storage: Don't depend on specs-storage, move to storiface 2022-06-17 13:36:33 +02:00
Łukasz Magiera
a084445e6c sealing pipeline: Use non-special ChainHead 2022-06-16 13:15:49 +02:00
Łukasz Magiera
9aa5659d24 sealing pipeline: Drop TipSetToken, use TipSetKey directly 2022-06-16 12:05:58 +02:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
39cede946c chore: Fix more imports with '2' suffix 2022-06-14 21:37:54 +02:00
Łukasz Magiera
a9600b8a6f storage: Move extern/sector-storage to storage/sealer 2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
717bda63b1 storage: Move control address logic to storage/ctladdr 2022-06-14 19:32:29 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Geoff Stuart
b7010c9e60 Implement function to migrate actors with only code changes 2022-06-10 15:52:32 -04:00
Jennifer Wang
2e59d0129d Merge branch 'release/v1.15.3' into jen/mergev1153to16 2022-05-31 16:33:18 -04:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
Rob Quist
767a53f8c3
Make markets logger less noise when doing retrievals
Logging each block causes a lot of useless information in the logs
2022-05-05 11:56:09 +02:00
zenground0
8562798940 Use go-fil-markets tag 2022-04-14 19:32:56 +03:00
zenground0
60d09579be More fixes 2022-04-14 19:32:56 +03:00
Aayush
339b7db2b7 Integrate FIP 0027 2022-04-14 19:32:56 +03:00