Commit Graph

424 Commits

Author SHA1 Message Date
Whyrusleeping
83436842c5
Merge pull request from filecoin-project/feat/gfm-v0.2.7
Use latest go-fil-markets
2020-05-21 10:05:54 -07:00
vyzo
d597c3fb0d increase OpportunisticGraftThreshold to 5 2020-05-21 14:38:11 +03:00
vyzo
0cc8446a79 configure Dout=0 for bootstrappers, turn off Mesh Failure Penalty for messages as well 2020-05-21 12:11:22 +03:00
hannahhoward
3264a9faad feat(deps): update to latest go-fil-markets
update to go-fil-markets v0.2.7 with latest changes, modify constructors as needed to accomodate new
DS Prefix customization
2020-05-20 16:01:03 -07:00
shannonwells
652c257763 passing w/ new namespace wraps 2020-05-20 15:58:59 -07:00
shannonwells
e68ab286f6 upgrade lotus to go-fil-markets v0.2.5 2020-05-20 15:56:58 -07:00
Łukasz Magiera
f7325a69c1 Extract auth utils to go-jsonrpc 2020-05-20 20:37:45 +02:00
Łukasz Magiera
ae60001dcc post: Handle Recovering sectors 2020-05-20 18:41:20 +02:00
Łukasz Magiera
cea7c41023
Merge pull request from dtynn/fix/dtynn/use_abs_path_for_client_filestore
use abs path to construct FileManager
2020-05-20 12:49:53 +02:00
Łukasz Magiera
0fe7a0b2e3
Merge pull request from filecoin-project/feat/has-cache-bs
add blockstore 'Has' caching
2020-05-20 12:48:50 +02:00
Jeromy
607bcc401c very basic api documentation autogeneration 2020-05-18 17:20:54 -07:00
Jeromy
9804bd3c4c add blockstore 'Has' caching 2020-05-16 11:31:14 -07:00
dtynn
30e2642f4d use abs path to construct FileManager 2020-05-16 09:36:38 +08:00
Whyrusleeping
70657e35ba
Merge pull request from filecoin-project/feat/block-validator
Improved pubsub block validator
2020-05-15 11:26:26 -07:00
vyzo
dbe2a1e2cd turn off Mesh Delivery Failure penalty 2020-05-15 12:50:20 +03:00
vyzo
6e0dac06f4 utility to retrieve miner worker key from the chain 2020-05-14 20:48:19 +03:00
vyzo
5bd0e92068 remove source flagging from bogus CID blocks
it's ineffective.
2020-05-14 20:48:19 +03:00
vyzo
b032d3524b configure behaviour penalty for gsv1.1 2020-05-14 20:48:19 +03:00
Jeromy
d3eb808ff4 add profile for bootstrappers 2020-05-14 09:27:07 -07:00
Łukasz Magiera
e8d92a5f31 node: Use correct network name for DHT 2020-05-12 19:38:04 +02:00
hannahhoward
581e7e6fde chore(deps): update to fil-markets 0.2.1
Update fil-markets to 0.2.1 that no longer keeps streams open
2020-05-08 14:11:40 -07:00
Łukasz Magiera
523a7a9d8d Merge pull request from filecoin-project/feat/update-markets-0.2.0
Update go-fil-markets
2020-05-06 01:45:33 +02:00
Łukasz Magiera
60b78a74f7 Merge pull request from filecoin-project/cleanup/multi-addr-miner
block producers should have only one address
2020-05-05 22:12:32 +02:00
Jeromy
8b58f22c2e block producers should have only one address 2020-05-05 12:01:44 -07:00
vyzo
eb24baa70c use go-libp2p-core/peer instead of the deprecated alias, try to pacify the silly linter 2020-05-05 17:26:59 +03:00
vyzo
4e5179557d flag block source when messages cannot be retrieved 2020-05-05 16:49:43 +03:00
hannahhoward
a9866c757e feat(deps): update go-fil-markets
Update to latest 0.2.0 release of go-fil-markets with less-blocking node io
2020-05-04 18:36:51 -07:00
vyzo
e3f6e586ee use blake2b hash as the message ID function 2020-05-04 19:34:30 +03:00
vyzo
805d07bca6 turn off the mesh in bootstrappers 2020-05-04 19:04:24 +03:00
vyzo
112aca7896 pubsub-specific configuration 2020-05-04 18:30:54 +03:00
Whyrusleeping
ae420fc9f4 Merge branch 'testnet/3' into feat/gossipsub-v1.1 2020-04-30 10:19:42 -07:00
Łukasz Magiera
81e3b714a2 gofmt 2020-04-30 12:51:19 +02:00
Łukasz Magiera
5ec76d4b7b support making deals with data directly from IPFS nodes 2020-04-30 01:56:45 +02:00
vyzo
43aca717d1 gossipsubv1.1: tweak score parameters 2020-04-29 23:15:29 +03:00
vyzo
7de882b4d0 gossipsubv1.1: use score parameter helpers 2020-04-29 22:50:51 +03:00
vyzo
f8f6e28985 fix linter complaint 2020-04-29 19:01:45 +03:00
vyzo
9570ebfe0a fix comparison logic for bootstrap peers 2020-04-29 18:56:58 +03:00
vyzo
667fb17b43 gossipsubv1.1: configure larger overlay D params 2020-04-29 17:09:39 +03:00
vyzo
7d7c83ef79 gossipsubv1.1: configure scores for bootstrapper PX 2020-04-29 17:03:41 +03:00
vyzo
6b56652768 gossipsubv1.1: score parameters and flood publishing 2020-04-29 16:41:44 +03:00
Jeromy
6a27c65201 updating to specs actors v0.2.0 2020-04-28 13:52:07 -07:00
Travis Person
c807469a5a bitswap: disable provider ()
Co-authored-by: Travis Person <travisperson@users.noreply.github.com>
2020-04-27 15:54:16 -07:00
Aayush Rajasekaran
7c22dabb63 default to setting max piece size to sector size 2020-04-24 01:01:22 -04:00
Łukasz Magiera
e6ce6211de Merge remote-tracking branch 'origin/testnet/3' into feat/dht-upgrade 2020-04-24 00:50:23 +02:00
Whyrusleeping
faa9002cce working on sector pledge requirements after specs-actors 1.0.0 ()
* WIP: working on sector pledge requirements after specs-actors 1.0.0

* update storage-fsm with pledge collateral logic

* Set bigger balances for miners

* Return only additional required pledge

* split collateral methods into two methods

Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
2020-04-23 12:39:34 -07:00
Jeromy
18f4e9d828 enable autonat service 2020-04-22 15:28:01 -07:00
Adin Schmahmann
9199c03f0d disable using the DHT to find provider records or arbitrary values 2020-04-22 11:13:26 -07:00
Adin Schmahmann
744a02eb5a update DHT options and use automatic DHT mode switching 2020-04-22 11:13:25 -07:00
Łukasz Magiera
b941b29fe5 miner: Use correct randomness for sealing 2020-04-20 20:33:53 +02:00
Łukasz Magiera
93b2b6d65d Merge branch 'testnet/3' into feat/windowed-post 2020-04-17 18:38:20 +02:00
Łukasz Magiera
bab352fb44 Wireup WinningPost validation 2020-04-17 16:47:19 +02:00
Łukasz Magiera
dc2a67f13f Fix worker key resolving 2020-04-16 22:38:42 +02:00
Łukasz Magiera
afdfc8807d api: Single method for static miner info 2020-04-16 19:36:36 +02:00
Jakub Sztandera
08e74a90f9 Add missing files
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-15 16:13:20 +02:00
Jeromy
4e6839b5ef some small cleanup 2020-04-14 15:13:52 -07:00
Jeromy
674be39af1 real drand network integration 2020-04-13 20:06:10 -07:00
Łukasz Magiera
5bf80a60f1 Get proof type in NewWinningPoStProver constructor 2020-04-13 23:25:46 +02:00
Łukasz Magiera
d3ffe48fcb Get things to mostly build 2020-04-10 23:29:05 +02:00
Łukasz Magiera
e053cd6b9c Merge remote-tracking branch 'origin/testnet/3' into feat/windowed-post 2020-04-10 23:09:01 +02:00
Łukasz Magiera
613e61222b mostly wire up WindowedPoSt 2020-04-10 23:07:18 +02:00
Łukasz Magiera
3426611340 Merge pull request from filecoin-project/feat/pass-deal-schedule-for-pre-commit-expiry-scheduling
provide deal scheduling info to FSM so that it can set pre-commit expiry
2020-04-10 23:05:50 +02:00
Łukasz Magiera
072dd8b744 Merge remote-tracking branch 'origin/testnet/3' into feat/windowed-post 2020-04-10 20:38:25 +02:00
Łukasz Magiera
0f0015f45c Merge pull request from filecoin-project/feat/graphsync-chainsync-extension
Update to latest go-fil-markets w/ enhanced Graphsync
2020-04-10 20:27:05 +02:00
laser
6dffe9377f provide deal scheduling info to FSM so that it can set pre-commit expiry 2020-04-10 08:22:28 -07:00
Jeromy
ef6fb1b756 rename beacon to randombeacon 2020-04-08 16:10:08 -07:00
hannahhoward
00cd89750d feat(deps): update fil-markets, graphsync
Updates dependencies for graphsync, fil-markets, data-transfer. Moves to new graphsync blockstore
swapping capabilities, and also locks down graphsync impl so it does not accept arbitrary requests
2020-04-07 23:25:29 -07:00
Jeromy
7075eaba26 WIP: random beacon interface
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-07 15:32:23 -07:00
Łukasz Magiera
e36f356908 wip changes for windowed post 2020-04-07 21:55:34 +02:00
Łukasz Magiera
5524241645 fpost -> windowPost renaming 2020-04-07 19:41:41 +02:00
laser
110c869dd3 cut over to storage-fsm, deleting local sealing package 2020-04-07 09:34:09 -07:00
laser
650a31b050 remove all lotus types from sealing package in preparation for extraction
events adapter

implement StateWaitMsg and StateComputeDataCommitment

implement StateGetSectorPreCommitOnChainInfo

implement ChainHead and SendMsg

implement remaining methods
2020-04-06 12:57:55 -07:00
Łukasz Magiera
f05be81275 fsm: Get correct interactive randomness if it lands on a nullblock 2020-04-04 04:57:05 +02:00
Łukasz Magiera
13f2c21e1d Get network name from genesis 2020-04-01 01:15:49 +02:00
Łukasz Magiera
95ee1c1cde Extract sector-storage 2020-03-28 01:32:39 +01:00
Łukasz Magiera
c66d3dfad7 sectorstorage: Untangle from lotus deps 2020-03-27 22:35:33 +01:00
Łukasz Magiera
e050e0307e Cleanup after dropping sectorbuilder 2020-03-26 20:35:16 +01:00
Łukasz Magiera
4be54a976c Merge sectorbuilder into sectorstorage 2020-03-26 03:51:06 +01:00
Łukasz Magiera
cfc65f525a sectorstorage: wire up closing logic 2020-03-25 00:52:55 +01:00
Łukasz Magiera
5de17e903c storageminer: Config for local worker task types 2020-03-24 20:38:00 +01:00
Łukasz Magiera
66a682b55d Merge remote-tracking branch 'origin/testnet/3' into feat/new-workers 2020-03-24 19:09:02 +01:00
Łukasz Magiera
726c9c1fe7 workers: Address review 2020-03-24 19:01:04 +01:00
Łukasz Magiera
5e4a7e54df Rename agvmgr+sealmgr to sectorstorage 2020-03-23 12:47:10 +01:00
hannahhoward
34f755b2b9 feat(chainsync): fixes to make graphsync work for chain fetching
- store to chain blockstore (ok for now, since storage provider is a seperate process)
- simplify request fetching and processing
2020-03-20 21:30:24 -07:00
whyrusleeping
1076a1a89d WIP: integrate graphsync for chainsync requests 2020-03-20 21:30:24 -07:00
Łukasz Magiera
1ed7779701 Merge remote-tracking branch 'origin/testnet/3' into feat/new-workers 2020-03-19 03:28:55 +01:00
laser
366db2f38c adjust import order as per PR feedback 2020-03-18 12:43:06 -07:00
laser
f59bf6fab1 wire up dtypes.ClientDatastore 2020-03-18 11:57:22 -07:00
Łukasz Magiera
5f2caffb51 Fix tests after specs-storage changes 2020-03-18 02:08:11 +01:00
hannahhoward
f259bc6a09 feat(graphsync): unified graphsync instance
setup a single graphsync that loads from both the chainstore & client blockstore
2020-03-17 17:25:12 -07:00
Łukasz Magiera
d8ce5078d9 Use new specs-storage interface 2020-03-17 21:19:52 +01:00
Łukasz Magiera
7e997e40f3 workers: Simple storage diagnostics 2020-03-16 18:50:07 +01:00
whyrusleeping
ab51c7e2d2 Fix fallback post scheduling to hopefully do the right thing 2020-03-08 00:19:38 -08:00
whyrusleeping
24bf720a9a Merge branch 'master' into testnet/3 2020-03-07 16:46:12 -08:00
Steven Allen
6992fd4dba chore(dep): update dependencies
Updates:

* cid, multihash, and multiaddr to ensure that minimal varints are used.
* log to deduplicate some code between version 2 and version 1.
* multistream to avoid writing from multiple threads.
* badger/datastore to harden prefix handling in queries.
* libp2p to 0.5.2

BREAKING:

The new libp2p version now formats addresses with /p2p instead of /ipfs. /ipfs
multiaddrs are still supported.
2020-03-05 16:28:42 -08:00
Łukasz Magiera
1461e475da storage: Support adding paths at runtime 2020-03-05 23:02:01 +01:00
Łukasz Magiera
7db1dd52bd storagemgr: Wire up most of AddPiece logic 2020-03-05 01:38:07 +01:00
Łukasz Magiera
799ff17c7c Merge remote-tracking branch 'origin/testnet/3' into feat/4stage-seal 2020-03-04 06:37:11 +01:00
whyrusleeping
37d5baf0b2 fix blocks topic 2020-03-03 21:03:35 -08:00
Łukasz Magiera
4e46d00eb6 Merge remote-tracking branch 'origin/testnet/3' into feat/4stage-seal 2020-03-04 04:01:13 +01:00
whyrusleeping
d6723935b9 fixup to match spec, move constants to common location 2020-03-03 15:48:25 -08:00
Łukasz Magiera
a0dbb6bdd6 Storage Manager refactor 2020-03-03 23:19:22 +01:00
lanzafame
4f2baa8953 conform protocol ids to spec
Signed-off-by: lanzafame <adrianlanzafame92@gmail.com>
2020-03-03 11:34:43 +10:00
whyrusleeping
00efd097c7 implement basic message filtering 2020-02-27 17:40:16 -08:00
whyrusleeping
d8dda1ee66 debugging test failure 2020-02-27 15:14:15 -08:00
whyrusleeping
df6e3e83bf make it all build finally 2020-02-27 13:45:31 -08:00
Łukasz Magiera
d787aa5007 Merge branch 'testnet/3' into feat/specs-actors 2020-02-24 18:32:02 +01:00
Łukasz Magiera
917df9a5b8 More unified randomness handling 2020-02-23 21:00:47 +01:00
Łukasz Magiera
99c0e14fd8 Fix genesis writing 2020-02-22 13:03:41 +01:00
Łukasz Magiera
b12482df89 move peermgr, tracing into lib 2020-02-22 12:36:22 +01:00
Łukasz Magiera
111d35deef seed: New genesis creation commands 2020-02-21 21:57:00 +01:00
Łukasz Magiera
852e888232 cmd/lotus: Fix build 2020-02-21 19:00:10 +01:00
Łukasz Magiera
a6268f055d storageminer: Update api interfaces 2020-02-21 18:43:44 +01:00
Łukasz Magiera
69ec3c205a Merge remote-tracking branch 'origin/master' into testnet/3 2020-02-19 22:10:14 +01:00
Whyrusleeping
6afeb2e72b
Merge pull request from filecoin-project/feat/simple-spam-protection
basic gossip spam protection logic
2020-02-18 10:39:36 -08:00
whyrusleeping
4f6ec225cf mark peer for disconnection in connmgr 2020-02-18 09:20:17 -08:00
Aayush Rajasekaran
60970008c8 Re: : API methods should receive TipSetKeys, not TipSets, as input 2020-02-17 19:53:43 -05:00
Whyrusleeping
1c9072bf6e
Merge pull request from filecoin-project/feat/tls
Prefer tls
2020-02-17 11:26:50 -08:00
Jakub Sztandera
8af80169c0 Add graphsync
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-02-17 18:15:30 +01:00
whyrusleeping
1dafcf24f1 basic gossip spam protection logic 2020-02-16 21:51:18 -08:00
Łukasz Magiera
36aed6f871 Update cbor-gen 2020-02-13 02:37:28 +01:00
Łukasz Magiera
e1a593c22e Rename paych -> paychmgr to fix cbor-gen 2020-02-13 01:28:23 +01:00
Jakub Sztandera
49167cc55f
Expose JWT permission error
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-02-12 21:25:29 +01:00
Łukasz Magiera
63e801e00f genesis: Cleanup the structure 2020-02-11 21:48:03 +01:00
Łukasz Magiera
c72727b4fd specs-actors: Fix most compilation errors 2020-02-09 07:06:32 +01:00
Jakub Sztandera
352fb9f9ea
Prefer tls
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-02-07 14:04:47 +01:00
hannahhoward
d2c5d81e59 fix(deps): update go-fil-markets
update go-fil-markets to latest to solve an issue with retrieval
2020-02-05 18:48:47 -08:00
Łukasz Magiera
0e81c1cf7e Merge branch 'testnet/3' into feat/update-markets 2020-02-04 07:17:22 +01:00
Łukasz Magiera
90ecfbe0e4 Merge branch 'master' into feat/new-sb-fs 2020-02-04 04:15:25 +01:00
laser
d8f5330c29 don't consume short-lived context 2020-02-03 15:48:11 -08:00
laser
ea2f53745e split FPoStScheduler from Miner
Rationale: Nodes want to handle scheduling of PoSts. Splitting the 
FPoStScheduler from the Miner allows nodes to reuse the Miner w/out 
having to reuse the PoSt scheduler.
2020-02-03 15:48:11 -08:00
Łukasz Magiera
a62572bb76 Merge remote-tracking branch 'origin/master' into feat/new-sb-fs 2020-01-31 08:36:47 +01:00
Łukasz Magiera
aefc463586 build: Lazy ParametersJson 2020-01-30 23:50:24 +01:00
Łukasz Magiera
3f58ffe572 storageminer: New storage config 2020-01-29 20:53:45 +01:00
Łukasz Magiera
5af64c53b6 initial sectorbuilder FS refactor integration 2020-01-29 00:08:02 +01:00
hannahhoward
71289b58ff feat(markets): update to support car files
Upgrades lotus version of go-fil-markets to its head, which supports car files and handles tracking
of pieces, reducing the size of sectorBlocks
2020-01-28 14:08:13 -08:00
Łukasz Magiera
d4c50b8ea0 sealing: gofmt 2020-01-16 03:54:57 +01:00
Łukasz Magiera
604bf64ef6 Move miner sealing logic into a separate package 2020-01-15 21:49:11 +01:00
Łukasz Magiera
2ca0c152ba Deal tests on mock sectorbuilder 2020-01-14 03:05:46 +01:00
Łukasz Magiera
a2bcc1fec2 Mostly functional mock sectorbuilder 2020-01-14 03:05:35 +01:00
whyrusleeping
fefd21ad9c remove private network and put dht into a sub namespace 2020-01-13 15:29:06 -08:00
Łukasz Magiera
a896b30039 gofmt 2020-01-10 20:55:58 +01:00
Łukasz Magiera
50bac00329 Merge branch 'master' into feat/testnet2 2020-01-10 20:55:40 +01:00
hannahhoward
0ce1bf706b refactor(markets): reorg file structure 2020-01-10 10:33:26 -08:00
hannahhoward
56653eeca6 fix(storagemarket): fix compile errors 2020-01-10 10:01:48 -08:00
hannahhoward
65ecb33630 feat(storagemarket): extract storage market
remove all shared components, point at go-fil-components repo
2020-01-10 09:49:07 -08:00
hannahhoward
0a4eeb7cb1 refactor(retrievalmarket): switch to go-fil-markets repo
Switch go-fil-components to go-fil-markets
2020-01-10 09:13:12 -08:00
hannahhoward
ccf359d057 feat(retrieval): extract retrievalmarket
Extract retrieval market and modify shared types
2020-01-09 19:46:07 -08:00
hannahhoward
da4528932a feat(storagemarket): initial extraction
Types for storage market

Modify deals.Provider to implement storagemarket.StorageProvider

Inject storagemarket.StorageProvider

Storage Provider interfaces

Storage Client interfaces

Add ValidatePublishedDeal to ClientNodeAdapter

Remove FundManager from client

Remove Wallet from client

Remove StateManager, Events, Wallet from client

Rebasing

- Copy types.BigInt, use TokenAmount/BigInt for token amounts
- Remove auto-imported log package
- Move `checkAskSignature` to a client file.
- Plumb contexts through

fix(storagemarket): use publish cids

Switch back to publish message cids to reduce the dependency surface area
2020-01-10 03:29:46 +01:00
Łukasz Magiera
5f460b380d
Merge pull request from filecoin-project/feat/mock-sectorbuilder
WIP: implement mock sectorbuilder for easier storage miner testing
2020-01-10 01:33:43 +01:00
Łukasz Magiera
f0f8d83980
Merge pull request from filecoin-project/feat/retrieval-market-skeleton
Retrieval Market Cleanup W/ Node Interfaces
2020-01-09 22:06:46 +01:00