Commit Graph

228 Commits

Author SHA1 Message Date
whyrusleeping
00efd097c7 implement basic message filtering 2020-02-27 17:40:16 -08:00
Nate Walck
f9dac3a291 Swap to opencensus for publishing data to prometheus 2020-02-25 19:40:07 -05:00
Nate Walck
c0ef65f442 Enable Prometheus metrics endpoint and add lotus version 2020-02-23 17:35:29 -05:00
Whyrusleeping
6afeb2e72b
Merge pull request #1268 from filecoin-project/feat/simple-spam-protection
basic gossip spam protection logic
2020-02-18 10:39:36 -08:00
whyrusleeping
1dafcf24f1 basic gossip spam protection logic 2020-02-16 21:51:18 -08:00
Jakub Sztandera
14580b6144
Import go-systemd properly
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-02-12 20:44:35 +01:00
Łukasz Magiera
ff77198a08 Update sectorbuilder with later cache mkdir 2020-02-03 18:37:40 +01:00
Łukasz Magiera
ee9060aa14 Update sectorbuilder 2020-01-31 02:18:48 +01:00
Łukasz Magiera
5af64c53b6 initial sectorbuilder FS refactor integration 2020-01-29 00:08:02 +01:00
Łukasz Magiera
9fc5f0cd9d
Merge pull request #1124 from filecoin-project/feat/sector-recovery
sealing: Implement some common error states
2020-01-24 21:25:03 +01:00
Jakub Sztandera
23b46a05f7
Update badger
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-23 12:46:32 -08:00
Łukasz Magiera
558e4c7669 Fix tests after sectorbuilder update 2020-01-23 15:33:28 +01:00
Łukasz Magiera
c3a5da8586 storageminer: log flag for sector status 2020-01-23 15:18:05 +01:00
Jakub Sztandera
a404d535ee
Make it buld without cgo
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-22 12:03:12 -08:00
Łukasz Magiera
d2401bc339 Update cbor-gen, error on unknown fields 2020-01-21 17:28:55 +01:00
Łukasz Magiera
2076cb106d cobr-gen: Soft struct-map unmarshaling 2020-01-21 17:05:10 +01:00
whyrusleeping
955b755055 wire up importing 2020-01-20 17:53:55 -08:00
whyrusleeping
62f05d4c14 WIP: chain export 2020-01-20 15:51:28 -08:00
Jakub Sztandera
84de47377c
Intoroduce AMT v2 and amtFsck
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-16 23:21:16 +01:00
ognots
d8d8ce7526 health agent to monitor lotus
watch if chain head changes in a given window of api polls
allows setting a threshold of how many times the chain head can remain
unchanged before failing health check
also can set interval for polling chain head
on failure, restarts systemd unit
2020-01-14 12:18:45 -05:00
Łukasz Magiera
11ac026b62 Merge branch 'feat/testnet2' into feat/chainwatch-pg 2020-01-14 04:20:39 +01:00
Łukasz Magiera
69752b419d Update sectorbuilder 2020-01-14 03:09:34 +01:00
Łukasz Magiera
94a21e3aeb Update deps 2020-01-14 03:05:46 +01:00
Łukasz Magiera
a2bcc1fec2 Mostly functional mock sectorbuilder 2020-01-14 03:05:35 +01:00
Łukasz Magiera
85f0f0bf81 mod tidy 2020-01-14 00:38:08 +01:00
Łukasz Magiera
b6fda30719
Merge pull request #1057 from filecoin-project/feat/markets-extraction
Markets Extraction
2020-01-10 20:49:09 +01:00
hannahhoward
0ce1bf706b refactor(markets): reorg file structure 2020-01-10 10:33:26 -08:00
hannahhoward
65ecb33630 feat(storagemarket): extract storage market
remove all shared components, point at go-fil-components repo
2020-01-10 09:49:07 -08:00
hannahhoward
0a4eeb7cb1 refactor(retrievalmarket): switch to go-fil-markets repo
Switch go-fil-components to go-fil-markets
2020-01-10 09:13:12 -08:00
Łukasz Magiera
469136ee86
Merge pull request #1040 from filecoin-project/feat/fork-signaling
Add fork signaling to blockheader
2020-01-10 13:33:29 +01:00
hannahhoward
ccf359d057 feat(retrieval): extract retrievalmarket
Extract retrieval market and modify shared types
2020-01-09 19:46:07 -08:00
whyrusleeping
7534ac1e1a add ability to control seal operations completion time 2020-01-09 12:50:45 -08:00
Łukasz Magiera
8cbcbbd7a4 Merge branch 'feat/testnet2' of github.com:filecoin-project/go-lotus into feat/testnet2 2020-01-08 22:44:54 +01:00
Łukasz Magiera
36a5c7d8f5 Merge branch 'master' into feat/testnet2 2020-01-08 22:38:45 +01:00
Łukasz Magiera
da38822dfd paramfetch: bring parameters.json back to this repo 2020-01-08 21:31:35 +01:00
Jakub Sztandera
8cfb4aafcd
Add fork signaling to blockheader
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:31:08 +01:00
Jakub Sztandera
1ed62628a7
Update go-log to v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:22:31 +01:00
Jakub Sztandera
b5328d01cf
Update to go-log v1.0.1
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:20:56 +01:00
Jakub Sztandera
886898b137
Update to badger v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 19:56:45 +01:00
Łukasz Magiera
855fb26a81 mod tidy 2020-01-08 15:18:13 +01:00
Łukasz Magiera
ab922ed1ba Merge remote-tracking branch 'origin/master' into feat/chainwatch-pg 2020-01-08 15:08:34 +01:00
hannahhoward
5312340bf9 refactor(datatransfer): use independent repo
Use independent repo for go-data-transfer
2020-01-08 14:20:31 +01:00
hannahhoward
61e2568b8d feat(datatransfer): implement and extract
feat(datatransfer): setup implementation path

Sets up a path to implementation, offering both the dagservice implementation and a future graphsync
implement, establishes message interfaces and network layer, and isolates the datatransfer module
from the app

WIP using CBOR encoding for dataxfermsg

* Bring cbor-gen stuff into datatransfer package
* make transferRequest private struct
* add transferResponse + funcs
* Rename VoucherID to VoucherType
* more tests passing

WIP trying out some stuff
* Embed request/response in message so all the interfaces work AND the CBOR unmarshaling works: this is more like the spec anyway
* get rid of pb stuff

all message tests passing, some others in datatransfer

Some cleanup for PR

Cleanup for PR, clarifying and additional comments

mod tidy

Respond to PR comments:
* Make DataTransferRequest/Response be returned in from Net
* Regenerate cbor_gen and fix the generator caller so it works better
* Please the linters

Fix tests

Initiate push and pull requests (#536)

* add issue link for data TransferID generation
* comment out failing but not relevant tests
* finish voucher rename from Identifier --> Type

tests passing

cleanup for PR

remove unused fmt import in graphsync_test

a better reflection

send data transfer response

other tests passing

feat(datatransfer): milestone 2 infrastructure

Setup test path for all tickets for milestone 2

responses alert subscribers when request is not accepted (#607)

Graphsync response is scheduled when a valid push request is received (#625)

fix(datatransfer): fix tests

fix an error with read buffers in tests

fix(deps): fix go.sum

Feat/dt graphsync pullreqs (#627)

* graphsync responses to pull requests

Feat/dt initiator cleanup (#645)

* ChannelID.To --> ChannelID.Initiator
* We now store our peer ID (from host.ID()) so it can be used when creating ChannelIDs.
* InProgressChannels returns all of impl.channels, currently just for testing
* Implements go-data-transfer issue
* Some assertions were changed based on the above.
* Renamed some variables and added some assertions based on the new understanding
* Updated SHA for graphsync module
* Updated fakeGraphSync test structs to use new interfaces from new SHA above

Techdebt/dt split graphsync impl receiver (#651)

* Split up graphsyncImpl and graphsyncReceiver
* rename graphsync to utils

DTM sends data over graphsync for validated push requests (#665)

* create channels when a request is received. register push request hook with graphsync. fix tests.
* better NewReaders
* use mutex lock around impl.channels access
* fix(datatransfer): fix test uncertainty
* fix a data race and also don't use random bytes in basic block which can fail
* privatize 3 funcs

with @hannahhoward

Feat/dt gs pullrequests (#693)

* Implements DTM Sends Data Over Graphsync For Validated Pull Requests
* rename a field in a test struct
* refactor a couple of private functions (one was refactored out of existence)

Feat/dt subscribe, file Xfer round trip (#720)

Implements the rest of Subscriber Is Notified When Request Completed #24:
* send a graphsync message within a go func and consume responses until error or transfer is complete.
* notify subscribers of results.
* Rename datatransfer.Event to EventCode.
* datatransfer.Event is now a struct that includes a message and a timestamp as well as the Event.Code int, formerly Event, update all uses
* Add extension data to graphsync request hook, gsReq
* rename sendRequest to sendDtRequest, to distinguish it from sendGsRequest, where Dt = datatransfer, Gs = graphsync
* use a mutex lock for last transfer ID
* obey the linter

Don't respond with error in gsReqRcdHook when we can't find the datatransfer extension. (#754)

* update to correct graphsync version, update tests & code to call the new graphsync hooks
* getExtensionData returns an empty struct + nil if we can't find our extension
* Don't respond with error when we can't find the extension.
* Test for same
* mod tidy

minor fix to go.sum

feat(datatransfer): switch to graphsync implementation

Move over to real graphsync implementation of data transfer, add constructors for graphsync
instances on client and miner side

fix(datatransfer): Fix validators

Validators were checking payload cid against commP -- which are not the same any more. Added a
payloadCid to client deal to maintain the record, fixed validator logic

Feat/dt extraction use go-fil-components/datatransfer (#770)

* Initial commit after changing to go-fil-components/datatransfer
* blow away the datatransfer dir
* use go-fil-components master after its PR #1 was merged
* go mod tidy

use a package

updates after rebase with master
2020-01-08 14:19:28 +01:00
Łukasz Magiera
1392dcc273 Merge remote-tracking branch 'origin/master' into feat/testnet2 2020-01-08 13:53:01 +01:00
whyrusleeping
80c19df4bb go mod tidy 2020-01-07 21:45:10 -08:00
Jakub Sztandera
a63c34a902
Handle faults in ss and pss correctly
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-07 23:03:18 +01:00
laser
0ffec0a9ad go mod tidy 2020-01-07 08:27:08 -08:00
laser
20321bb476 handle rebase 2020-01-07 08:18:35 -08:00
laser
4bbc693d9c go mod tidy 2020-01-07 07:52:19 -08:00
laser
3ae1bd5b78 add paramfetch import to lotus-bench 2020-01-07 07:52:06 -08:00