Commit Graph

1048 Commits

Author SHA1 Message Date
Jakub Sztandera
1ed62628a7
Update go-log to v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:22:31 +01:00
Łukasz Magiera
f198e81af1
Merge pull request #1032 from filecoin-project/feat/drop-most-fatal-errors
reduce most vm errors to non-fatal, explicitly mark disk issues as fatal
2020-01-08 14:58:35 +01:00
hannahhoward
5312340bf9 refactor(datatransfer): use independent repo
Use independent repo for go-data-transfer
2020-01-08 14:20:31 +01:00
hannahhoward
61e2568b8d feat(datatransfer): implement and extract
feat(datatransfer): setup implementation path

Sets up a path to implementation, offering both the dagservice implementation and a future graphsync
implement, establishes message interfaces and network layer, and isolates the datatransfer module
from the app

WIP using CBOR encoding for dataxfermsg

* Bring cbor-gen stuff into datatransfer package
* make transferRequest private struct
* add transferResponse + funcs
* Rename VoucherID to VoucherType
* more tests passing

WIP trying out some stuff
* Embed request/response in message so all the interfaces work AND the CBOR unmarshaling works: this is more like the spec anyway
* get rid of pb stuff

all message tests passing, some others in datatransfer

Some cleanup for PR

Cleanup for PR, clarifying and additional comments

mod tidy

Respond to PR comments:
* Make DataTransferRequest/Response be returned in from Net
* Regenerate cbor_gen and fix the generator caller so it works better
* Please the linters

Fix tests

Initiate push and pull requests (#536)

* add issue link for data TransferID generation
* comment out failing but not relevant tests
* finish voucher rename from Identifier --> Type

tests passing

cleanup for PR

remove unused fmt import in graphsync_test

a better reflection

send data transfer response

other tests passing

feat(datatransfer): milestone 2 infrastructure

Setup test path for all tickets for milestone 2

responses alert subscribers when request is not accepted (#607)

Graphsync response is scheduled when a valid push request is received (#625)

fix(datatransfer): fix tests

fix an error with read buffers in tests

fix(deps): fix go.sum

Feat/dt graphsync pullreqs (#627)

* graphsync responses to pull requests

Feat/dt initiator cleanup (#645)

* ChannelID.To --> ChannelID.Initiator
* We now store our peer ID (from host.ID()) so it can be used when creating ChannelIDs.
* InProgressChannels returns all of impl.channels, currently just for testing
* Implements go-data-transfer issue
* Some assertions were changed based on the above.
* Renamed some variables and added some assertions based on the new understanding
* Updated SHA for graphsync module
* Updated fakeGraphSync test structs to use new interfaces from new SHA above

Techdebt/dt split graphsync impl receiver (#651)

* Split up graphsyncImpl and graphsyncReceiver
* rename graphsync to utils

DTM sends data over graphsync for validated push requests (#665)

* create channels when a request is received. register push request hook with graphsync. fix tests.
* better NewReaders
* use mutex lock around impl.channels access
* fix(datatransfer): fix test uncertainty
* fix a data race and also don't use random bytes in basic block which can fail
* privatize 3 funcs

with @hannahhoward

Feat/dt gs pullrequests (#693)

* Implements DTM Sends Data Over Graphsync For Validated Pull Requests
* rename a field in a test struct
* refactor a couple of private functions (one was refactored out of existence)

Feat/dt subscribe, file Xfer round trip (#720)

Implements the rest of Subscriber Is Notified When Request Completed #24:
* send a graphsync message within a go func and consume responses until error or transfer is complete.
* notify subscribers of results.
* Rename datatransfer.Event to EventCode.
* datatransfer.Event is now a struct that includes a message and a timestamp as well as the Event.Code int, formerly Event, update all uses
* Add extension data to graphsync request hook, gsReq
* rename sendRequest to sendDtRequest, to distinguish it from sendGsRequest, where Dt = datatransfer, Gs = graphsync
* use a mutex lock for last transfer ID
* obey the linter

Don't respond with error in gsReqRcdHook when we can't find the datatransfer extension. (#754)

* update to correct graphsync version, update tests & code to call the new graphsync hooks
* getExtensionData returns an empty struct + nil if we can't find our extension
* Don't respond with error when we can't find the extension.
* Test for same
* mod tidy

minor fix to go.sum

feat(datatransfer): switch to graphsync implementation

Move over to real graphsync implementation of data transfer, add constructors for graphsync
instances on client and miner side

fix(datatransfer): Fix validators

Validators were checking payload cid against commP -- which are not the same any more. Added a
payloadCid to client deal to maintain the record, fixed validator logic

Feat/dt extraction use go-fil-components/datatransfer (#770)

* Initial commit after changing to go-fil-components/datatransfer
* blow away the datatransfer dir
* use go-fil-components master after its PR #1 was merged
* go mod tidy

use a package

updates after rebase with master
2020-01-08 14:19:28 +01:00
Łukasz Magiera
1392dcc273 Merge remote-tracking branch 'origin/master' into feat/testnet2 2020-01-08 13:53:01 +01:00
Łukasz Magiera
86a22a3896
Merge pull request #1029 from filecoin-project/feat/mpoolpush-cid
mpool: Return CID in MpoolPush
2020-01-08 13:43:03 +01:00
whyrusleeping
897cf72933 Add command to mark given blocks as bad 2020-01-07 21:42:15 -08:00
Łukasz Magiera
a11e8093e5 mpool: Return CID in MpoolPush 2020-01-07 21:10:30 -08:00
whyrusleeping
3965e1053e fix build 2020-01-07 21:05:45 -08:00
whyrusleeping
e41dbdad90 reduce most vm errors to non-fatal, explicitly mark disk issues as fatal 2020-01-07 19:46:55 -08:00
Jakub Sztandera
f572cdea47
Fix logic, improve tests
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 23:03:58 +01:00
Jakub Sztandera
5e775929be
Cleanup handing and add tests
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 23:03:58 +01:00
Jakub Sztandera
65724133ea
Add tests
,
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 23:03:58 +01:00
Jakub Sztandera
a63c34a902
Handle faults in ss and pss correctly
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 23:03:18 +01:00
Jakub Sztandera
dbc729d049
Validate length of RLE
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 22:54:43 +01:00
Łukasz Magiera
7b134b8317
Merge pull request #1031 from filecoin-project/feat/block-message-limit
introduce a limit on the number of messages a block can have
2020-01-07 21:48:57 +01:00
Łukasz Magiera
22e9815928 Merge remote-tracking branch 'origin/master' into feat/testnet2 2020-01-07 21:45:30 +01:00
whyrusleeping
a109ae1490 introduce a limit on the number of messages a block can have 2020-01-07 12:41:26 -08:00
laser
841f9c3627 reorganize imports to pass linter 2020-01-07 08:23:12 -08:00
laser
20321bb476 handle rebase 2020-01-07 08:18:35 -08:00
acruikshank
7873213b11 switch sectorbuilder from lib to external module
point to correct version of sectorbuilder + ensure go-datastore stays at 0.1.1

replace local address type with go-address

consume paramfetch.GetParams instead of local paramfetch.go

remove constants now defined in sectorbuilder
2020-01-07 07:48:43 -08:00
Łukasz Magiera
97ae184d0a Merge remote-tracking branch 'origin/master' into feat/testnet2 2020-01-07 15:06:35 +01:00
Łukasz Magiera
1e7f10e387 gofmt 2020-01-07 15:00:10 +01:00
hannahhoward
718acb0a86 refactor(cborutil): extract cborutil library
Use extracted cborutil library
2020-01-07 14:59:35 +01:00
hannahhoward
14ecd1929b refactor(crypto): use extracted crypto library 2020-01-07 14:53:30 +01:00
hannahhoward
8418464d91 refactor(address): use extracted address library
Switch to using extracted address library
2020-01-07 14:53:27 +01:00
Łukasz Magiera
2635028933
Merge pull request #1024 from filecoin-project/fix/sizestr-overflow
Fix SizeStr index out of band
2020-01-07 14:30:15 +01:00
Łukasz Magiera
605562467d Flatten fork code for testnet2 2020-01-06 19:39:57 +01:00
Jakub Sztandera
2d22f9c3d3
Fix SizeStr index out of band
add test for it

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-06 15:27:19 +01:00
whyrusleeping
dafad17a9b also test fault values out of range 2019-12-20 20:04:38 -08:00
whyrusleeping
8098d23768 test to ensure that submitting bad faults doesnt cause an error 2019-12-20 20:02:18 -08:00
whyrusleeping
5d9d51ba2d don't fail execution if the fault set contains an invalid sector ID 2019-12-20 19:33:11 -08:00
Łukasz Magiera
e3a22d2027 Fix withUpdates 2019-12-20 09:31:59 +01:00
Łukasz Magiera
ed3a2b0025 Set correct head in forkNoPowerEPS 2019-12-20 08:30:12 +01:00
Whyrusleeping
41aef3d2eb
Merge pull request #984 from filecoin-project/feat/tag-good-peers
Tag peers who send us good blocks over pubsub
2019-12-19 17:58:29 -08:00
Łukasz Magiera
ef3795ae3d
Merge pull request #988 from filecoin-project/fork/fix-no-power-eps
FORK: No power EPS Fix
2019-12-19 22:53:35 +01:00
Łukasz Magiera
f7604b84f7
Merge pull request #965 from filecoin-project/feat/no-fatal-serialization
make serialization error non-fatal
2019-12-19 22:04:27 +01:00
whyrusleeping
e5899f614a Tag peers who send us good blocks over pubsub 2019-12-19 12:47:55 -08:00
Łukasz Magiera
6ce4cf12f7 Care about nullblocks in handleStateForks 2019-12-19 21:01:05 +01:00
Łukasz Magiera
d21e24c270 don't modify existing actor code 2019-12-19 20:38:50 +01:00
Łukasz Magiera
3f54cbabe0 FORK: No power EPS Fix 2019-12-19 19:47:25 +01:00
Łukasz Magiera
81233b894d Implement 'chain get' for easier ipld digging 2019-12-19 17:58:02 +01:00
Łukasz Magiera
7fd31192d2
Merge pull request #964 from filecoin-project/feat/sb-faults
Wire up faults in fPoSt
2019-12-18 17:37:27 +01:00
whyrusleeping
897d4c7d24 make serialization error non-fatal 2019-12-17 19:41:58 -08:00
whyrusleeping
fda3811dcf Update cbor-gen to latest master 2019-12-17 19:06:06 -08:00
Łukasz Magiera
2e95a53679 Wire up faults in fPoSt 2019-12-17 23:33:08 +01:00
whyrusleeping
3cb4a60db4 some log cleanup and an extra trace value 2019-12-17 14:15:51 -08:00
Whyrusleeping
31327a383f
Merge pull request #948 from filecoin-project/feat/sybill-fun
Basic sybill resistance mechanisms
2019-12-17 20:45:45 +01:00
whyrusleeping
6cb6167393 bump up filecoin related weights 2019-12-17 10:16:10 -08:00
Łukasz Magiera
4ea6305771 sync: Check bls aggregate early 2019-12-17 12:20:09 +01:00
whyrusleeping
7dcc528cd1 tag peers who propagate good blocks to us 2019-12-16 23:06:48 -08:00
whyrusleeping
022581d50b add routine to tag miners peer IDs as high value in connection manager 2019-12-16 22:51:41 -08:00
whyrusleeping
28b470cb0d blocksync tagging 2019-12-16 22:15:06 -08:00
whyrusleeping
4e4d6fb8de parallelize bls message hashing for validation 2019-12-16 19:36:32 -08:00
Łukasz Magiera
3fda442bb3
Merge pull request #934 from filecoin-project/feat/sb-diskmgr
sectorbuilder: Check free space before creating sectors
2019-12-16 21:37:18 +01:00
Łukasz Magiera
3892e94bfe
Merge pull request #938 from filecoin-project/feat/load-tipset-cache
cache load tipset
2019-12-16 21:13:03 +01:00
whyrusleeping
6e94377469 cache load tipset 2019-12-16 11:22:56 -08:00
Łukasz Magiera
f297409d6d
Merge pull request #936 from filecoin-project/feat/cleanup-logspam
clean up some log spam
2019-12-16 18:17:14 +01:00
whyrusleeping
99eae286c2 clean up some log spam 2019-12-16 09:14:21 -08:00
Łukasz Magiera
825e3a0774 sectorbuilder: check free space before creating sectors 2019-12-16 17:16:08 +01:00
Jakub Sztandera
c25f616562
Merge pull request #932 from filecoin-project/feat/cfg-uncomment-sections
config: Don't comment sections be default
2019-12-16 13:03:45 +01:00
Łukasz Magiera
d6f543ce87 gofmt 2019-12-16 13:00:43 +01:00
Łukasz Magiera
9b2eab6a3f
Merge pull request #925 from filecoin-project/fix/future-mining
dont allow submitting blocks more than a second in the future
2019-12-16 12:39:35 +01:00
Whyrusleeping
7293f7415d
Merge pull request #715 from filecoin-project/frrist/chainval-multisig
chainval | multisig tests
2019-12-16 02:58:30 +01:00
whyrusleeping
159117e720 dont allow submitting blocks more than a second in the future 2019-12-15 16:43:32 -08:00
Łukasz Magiera
1415ba4ce7
Merge pull request #898 from filecoin-project/feat/storage-market-tracing
add a bit more tracing in publish storage deal
2019-12-13 10:59:31 +01:00
whyrusleeping
28a325efe6 soft fail messages that cause precomitted sectors map to become too large 2019-12-13 00:00:20 +01:00
whyrusleeping
71cbd79290 add a bit more tracing in publish storage deal 2019-12-12 19:25:15 +01:00
Łukasz Magiera
cf9edae000
Merge pull request #878 from filecoin-project/feat/cw-sset-counts
chainwatch: Collect sector set sizes
2019-12-12 17:32:12 +01:00
Jakub Sztandera
b96da09a5d
Add additional checks
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-12 14:24:24 +01:00
Jakub Sztandera
1eed506728
Implement fork
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-12 14:16:47 +01:00
Jakub Sztandera
344dd2ca5a
Fix candidate check
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-12 14:12:43 +01:00
Łukasz Magiera
fc6073d13b chainwatch: Collect sector set sizes 2019-12-12 00:32:09 +01:00
Jakub Sztandera
ae2a694dcb
Remove sync time
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-11 22:03:53 +01:00
whyrusleeping
f141849df5 fix incoming tipset bucketing 2019-12-11 21:41:24 +01:00
Łukasz Magiera
37113aeab1 Final testnet params 2019-12-11 17:27:30 +01:00
Łukasz Magiera
62ede45dfc
Merge pull request #855 from dtynn/fix/duplicate_call_on_unsub
fix: duplicate unsub calls when client quits
2019-12-11 14:09:09 +01:00
dtynn
8fd624fd5b fix: duplicate unsub calls when client quits 2019-12-11 20:37:52 +08:00
Łukasz Magiera
544bba6126
Merge pull request #845 from filecoin-project/fix/other-gen-sync-panic
fix all panics from calling NewTipSet on an empty set of blocks
2019-12-10 22:47:21 +01:00
whyrusleeping
12eed004ae fix all panics from calling NewTipSet on an empty set of blocks 2019-12-10 22:41:02 +01:00
Łukasz Magiera
b4cf80f540 mempool: Don't crash if loading local messages fails 2019-12-10 20:42:09 +01:00
Łukasz Magiera
70c057934e
Merge pull request #836 from filecoin-project/fix/genesis-miner-power
Fix setting genesis miner power
2019-12-10 20:28:40 +01:00
Łukasz Magiera
e5dfbef83a genesis: Fix miner power setting 2019-12-10 17:55:21 +01:00
Jakub Sztandera
9d9040558f
Collect hello message latency
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-10 16:36:48 +01:00
Łukasz Magiera
fe95c59158
Merge pull request #832 from filecoin-project/feat/smrt-rbc
Smarter message rebroadcast
2019-12-10 16:18:04 +01:00
Jakub Sztandera
43bc25d04a
Fix edge case
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-10 14:45:15 +01:00
Jakub Sztandera
c37ec227e9
Fix lint warning
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-10 14:16:01 +01:00
Jakub Sztandera
44d8f4d6d4
Smarter message rebroadcast
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-10 14:13:20 +01:00
whyrusleeping
bbc616c70c fix including multiple blocks from the same miner in a tipset 2019-12-10 12:07:07 +01:00
Łukasz Magiera
852415ce04
Merge pull request #822 from filecoin-project/fix/sync-tests
fix sync tests
2019-12-09 22:18:29 +01:00
whyrusleeping
e27217c6a1 fix sync tests 2019-12-09 22:17:15 +01:00
Jakub Sztandera
1914412adf
Make it cleaner
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-09 22:11:02 +01:00
Łukasz Magiera
c816ded4e5
Merge pull request #814 from filecoin-project/feat/minimum-miner-size
implement a minimum miner size
2019-12-09 21:32:30 +01:00
whyrusleeping
d64f8a38d1
implement a minimum miner size
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-09 21:30:04 +01:00
Łukasz Magiera
d750723405 Merge remote-tracking branch 'origin/testnet-staging' into feat/cleanup-sectorstate-structs 2019-12-09 21:27:20 +01:00
whyrusleeping
d2448912a6 Implement basic fault handling
WIP: miner actor tests

working miner actor test

rebase and fix tests
2019-12-09 20:53:43 +01:00
frrist
b6b91372cb add multisig test calls and update to v0.0.3 2019-12-09 09:55:24 -08:00
Jakub Sztandera
887a34135b
Make ElectionPostChallengeCount math overflow safe
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-09 18:34:18 +01:00
Henri S
9e363f9266
fix target for ePoSt IsTicketWinner fn
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-09 18:34:15 +01:00
Łukasz Magiera
ff963dfea5
Merge pull request #780 from filecoin-project/feat/storage-deal-cleanup
Clean up storage deal structures
2019-12-09 18:31:18 +01:00