Commit Graph

2585 Commits

Author SHA1 Message Date
hannahhoward
353f93fddf fix(storagemarketadapter): minor fix for test 2020-01-10 09:49:41 -08:00
hannahhoward
65ecb33630 feat(storagemarket): extract storage market
remove all shared components, point at go-fil-components repo
2020-01-10 09:49:07 -08:00
hannahhoward
0a4eeb7cb1 refactor(retrievalmarket): switch to go-fil-markets repo
Switch go-fil-components to go-fil-markets
2020-01-10 09:13:12 -08:00
hannahhoward
ccf359d057 feat(retrieval): extract retrievalmarket
Extract retrieval market and modify shared types
2020-01-09 19:46:07 -08:00
Łukasz Magiera
d6b0648610
Merge pull request #951 from filecoin-project/feat/storage-market-skeleton
Storage Market: Draw clean interfaces and node boundaries around storage market
2020-01-10 03:58:39 +01:00
hannahhoward
da4528932a feat(storagemarket): initial extraction
Types for storage market

Modify deals.Provider to implement storagemarket.StorageProvider

Inject storagemarket.StorageProvider

Storage Provider interfaces

Storage Client interfaces

Add ValidatePublishedDeal to ClientNodeAdapter

Remove FundManager from client

Remove Wallet from client

Remove StateManager, Events, Wallet from client

Rebasing

- Copy types.BigInt, use TokenAmount/BigInt for token amounts
- Remove auto-imported log package
- Move `checkAskSignature` to a client file.
- Plumb contexts through

fix(storagemarket): use publish cids

Switch back to publish message cids to reduce the dependency surface area
2020-01-10 03:29:46 +01:00
Łukasz Magiera
5f460b380d
Merge pull request #1045 from filecoin-project/feat/mock-sectorbuilder
WIP: implement mock sectorbuilder for easier storage miner testing
2020-01-10 01:33:43 +01:00
Łukasz Magiera
55f76f19d3
Merge pull request #1051 from filecoin-project/feat/self-deal-no-sig
don't require signatures for self deals
2020-01-10 01:25:06 +01:00
whyrusleeping
6c9289df0a don't require signatures for self deals 2020-01-09 14:18:06 -08:00
Łukasz Magiera
f0f8d83980
Merge pull request #826 from filecoin-project/feat/retrieval-market-skeleton
Retrieval Market Cleanup W/ Node Interfaces
2020-01-09 22:06:46 +01:00
whyrusleeping
45e576addf mod tidy 2020-01-09 12:59:37 -08:00
whyrusleeping
7534ac1e1a add ability to control seal operations completion time 2020-01-09 12:50:45 -08:00
whyrusleeping
bda85e4097 more features, better locking 2020-01-09 11:23:42 -08:00
whyrusleeping
5e16aab51b fix tests, expand interfaces a little 2020-01-09 10:28:14 -08:00
whyrusleeping
cc56389dbb attempt to fix the DI loading, still not working 2020-01-08 20:51:18 -08:00
whyrusleeping
60969c69a7 implement a mocked out sectorbuilder, probably not useful yet 2020-01-08 20:27:14 -08:00
hannahhoward
99309ec0dc fix(retrievalmarket): add mutex for subscribers
Add a mutex to protect access to the subscriber list for retrieval market
2020-01-08 19:46:11 -08:00
whyrusleeping
d08898ef37 make storage miner acccept an interface to the sectorbuilder 2020-01-08 17:10:17 -08:00
Łukasz Magiera
b7e35ae3ed
Merge pull request #1042 from filecoin-project/feat/limit-seal-proof-lookback
Set hard limit on maximum randomness lookback for seal proof submissions
2020-01-09 01:14:46 +01:00
whyrusleeping
7ba34535b7 Set hard limit on maximum randomness lookback for seal proof submissions 2020-01-08 14:43:03 -08:00
Łukasz Magiera
8cbcbbd7a4 Merge branch 'feat/testnet2' of github.com:filecoin-project/go-lotus into feat/testnet2 2020-01-08 22:44:54 +01:00
Łukasz Magiera
36a5c7d8f5 Merge branch 'master' into feat/testnet2 2020-01-08 22:38:45 +01:00
hannahhoward
5b74a71dd3 feat(retrievalmarket): extract skeleton interfaces
Define all types to spec, modify interfaces, wrap old code

fix(builder): use client blockstore for retrieval

feat(retrieval): add node implementations

add node adapters for client & provider so that retrieval can be extracted
2020-01-08 12:45:43 -08:00
Łukasz Magiera
716b2a4bcb
Merge pull request #1041 from filecoin-project/fix/paramsjson-embed
paramfetch: bring parameters.json back to this repo
2020-01-08 21:41:32 +01:00
Łukasz Magiera
13c8fe3fb3 mod tidy 2020-01-08 21:31:44 +01:00
Łukasz Magiera
da38822dfd paramfetch: bring parameters.json back to this repo 2020-01-08 21:31:35 +01:00
Łukasz Magiera
f36d81070b mod tidy 2020-01-08 21:15:55 +01:00
Jakub Sztandera
959b725ab6
Merge pull request #1038 from filecoin-project/feat/new-golog
Update go-log to v2
2020-01-08 20:41:55 +01:00
Łukasz Magiera
3891f015d7
Merge pull request #1039 from filecoin-project/feat/secpk-default
switch default key type to secp256k1
2020-01-08 20:40:08 +01:00
whyrusleeping
6ce9a7f225 switch default key type to secp256k1 2020-01-08 11:24:18 -08:00
Jakub Sztandera
1ed62628a7
Update go-log to v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:22:31 +01:00
Jakub Sztandera
b5328d01cf
Update to go-log v1.0.1
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:20:56 +01:00
Jakub Sztandera
72a62d49b3
Merge pull request #1037 from filecoin-project/feat/badger2
Update to badger v2
2020-01-08 20:20:11 +01:00
Jakub Sztandera
886898b137
Update to badger v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 19:56:45 +01:00
Jakub Sztandera
00683549ae
Merge pull request #1036 from filecoin-project/feat/v0.1.6
v0.1.6
2020-01-08 18:44:49 +01:00
Łukasz Magiera
f32bacb32d v0.1.6 2020-01-08 18:42:19 +01:00
Łukasz Magiera
5a12d60c6f
Merge pull request #1035 from filecoin-project/fix/rle-count
Verify RLE+ counts
2020-01-08 18:41:06 +01:00
Jakub Sztandera
f627ff2383
Verify RLE+ counts
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 18:30:13 +01:00
Łukasz Magiera
f198e81af1
Merge pull request #1032 from filecoin-project/feat/drop-most-fatal-errors
reduce most vm errors to non-fatal, explicitly mark disk issues as fatal
2020-01-08 14:58:35 +01:00
Jakub Sztandera
83cbf141a8
Merge pull request #1034 from filecoin-project/feat/lotuslog
Set logging levels in a single place
2020-01-08 14:57:55 +01:00
Łukasz Magiera
9fd183e74f
Merge pull request #500 from filecoin-project/feat/dt-implementation
Extract Data Transfer Implementation
2020-01-08 14:52:01 +01:00
Łukasz Magiera
31173ff4b4 Set logging levels in a single place 2020-01-08 14:49:34 +01:00
Łukasz Magiera
bf9aa4465b mod tidy 2020-01-08 14:20:43 +01:00
hannahhoward
5312340bf9 refactor(datatransfer): use independent repo
Use independent repo for go-data-transfer
2020-01-08 14:20:31 +01:00
hannahhoward
61e2568b8d feat(datatransfer): implement and extract
feat(datatransfer): setup implementation path

Sets up a path to implementation, offering both the dagservice implementation and a future graphsync
implement, establishes message interfaces and network layer, and isolates the datatransfer module
from the app

WIP using CBOR encoding for dataxfermsg

* Bring cbor-gen stuff into datatransfer package
* make transferRequest private struct
* add transferResponse + funcs
* Rename VoucherID to VoucherType
* more tests passing

WIP trying out some stuff
* Embed request/response in message so all the interfaces work AND the CBOR unmarshaling works: this is more like the spec anyway
* get rid of pb stuff

all message tests passing, some others in datatransfer

Some cleanup for PR

Cleanup for PR, clarifying and additional comments

mod tidy

Respond to PR comments:
* Make DataTransferRequest/Response be returned in from Net
* Regenerate cbor_gen and fix the generator caller so it works better
* Please the linters

Fix tests

Initiate push and pull requests (#536)

* add issue link for data TransferID generation
* comment out failing but not relevant tests
* finish voucher rename from Identifier --> Type

tests passing

cleanup for PR

remove unused fmt import in graphsync_test

a better reflection

send data transfer response

other tests passing

feat(datatransfer): milestone 2 infrastructure

Setup test path for all tickets for milestone 2

responses alert subscribers when request is not accepted (#607)

Graphsync response is scheduled when a valid push request is received (#625)

fix(datatransfer): fix tests

fix an error with read buffers in tests

fix(deps): fix go.sum

Feat/dt graphsync pullreqs (#627)

* graphsync responses to pull requests

Feat/dt initiator cleanup (#645)

* ChannelID.To --> ChannelID.Initiator
* We now store our peer ID (from host.ID()) so it can be used when creating ChannelIDs.
* InProgressChannels returns all of impl.channels, currently just for testing
* Implements go-data-transfer issue
* Some assertions were changed based on the above.
* Renamed some variables and added some assertions based on the new understanding
* Updated SHA for graphsync module
* Updated fakeGraphSync test structs to use new interfaces from new SHA above

Techdebt/dt split graphsync impl receiver (#651)

* Split up graphsyncImpl and graphsyncReceiver
* rename graphsync to utils

DTM sends data over graphsync for validated push requests (#665)

* create channels when a request is received. register push request hook with graphsync. fix tests.
* better NewReaders
* use mutex lock around impl.channels access
* fix(datatransfer): fix test uncertainty
* fix a data race and also don't use random bytes in basic block which can fail
* privatize 3 funcs

with @hannahhoward

Feat/dt gs pullrequests (#693)

* Implements DTM Sends Data Over Graphsync For Validated Pull Requests
* rename a field in a test struct
* refactor a couple of private functions (one was refactored out of existence)

Feat/dt subscribe, file Xfer round trip (#720)

Implements the rest of Subscriber Is Notified When Request Completed #24:
* send a graphsync message within a go func and consume responses until error or transfer is complete.
* notify subscribers of results.
* Rename datatransfer.Event to EventCode.
* datatransfer.Event is now a struct that includes a message and a timestamp as well as the Event.Code int, formerly Event, update all uses
* Add extension data to graphsync request hook, gsReq
* rename sendRequest to sendDtRequest, to distinguish it from sendGsRequest, where Dt = datatransfer, Gs = graphsync
* use a mutex lock for last transfer ID
* obey the linter

Don't respond with error in gsReqRcdHook when we can't find the datatransfer extension. (#754)

* update to correct graphsync version, update tests & code to call the new graphsync hooks
* getExtensionData returns an empty struct + nil if we can't find our extension
* Don't respond with error when we can't find the extension.
* Test for same
* mod tidy

minor fix to go.sum

feat(datatransfer): switch to graphsync implementation

Move over to real graphsync implementation of data transfer, add constructors for graphsync
instances on client and miner side

fix(datatransfer): Fix validators

Validators were checking payload cid against commP -- which are not the same any more. Added a
payloadCid to client deal to maintain the record, fixed validator logic

Feat/dt extraction use go-fil-components/datatransfer (#770)

* Initial commit after changing to go-fil-components/datatransfer
* blow away the datatransfer dir
* use go-fil-components master after its PR #1 was merged
* go mod tidy

use a package

updates after rebase with master
2020-01-08 14:19:28 +01:00
Łukasz Magiera
1392dcc273 Merge remote-tracking branch 'origin/master' into feat/testnet2 2020-01-08 13:53:01 +01:00
Łukasz Magiera
117bea4e4e
Merge pull request #1030 from filecoin-project/feat/chain-list-messages
implement list-messages command
2020-01-08 13:50:55 +01:00
Łukasz Magiera
86a22a3896
Merge pull request #1029 from filecoin-project/feat/mpoolpush-cid
mpool: Return CID in MpoolPush
2020-01-08 13:43:03 +01:00
whyrusleeping
95a89ad192 implement list-messages command 2020-01-08 13:41:23 +01:00
Łukasz Magiera
5153f67a2e
Merge pull request #1033 from filecoin-project/feat/sync-mark-bad
Add command to mark given blocks as bad
2020-01-08 13:35:34 +01:00