Commit Graph

336 Commits

Author SHA1 Message Date
whyrusleeping
96482f456a WIP: election post restructuring 2019-11-21 16:21:45 -06:00
Łukasz Magiera
b726b95298 WIP remote sector CommitSseal 2019-11-21 20:51:53 +01:00
Łukasz Magiera
d4197bbadc Working remote PreCommit 2019-11-21 19:38:50 +01:00
Łukasz Magiera
98b1de33b6 Stats for remote workers 2019-11-21 17:23:42 +01:00
Łukasz Magiera
88bbcd80ea support remote SealPreCommit 2019-11-21 17:10:09 +01:00
Łukasz Magiera
ba3ad75670 remote-worker: wire up storage miner endpoints 2019-11-21 15:10:51 +01:00
Łukasz Magiera
9725eb78bf wip remote sectorbuilder workers 2019-11-21 01:53:06 +01:00
Łukasz Magiera
ca70ac05d7 sectorbuilder: Drop some unused functions 2019-11-20 20:59:10 +01:00
Łukasz Magiera
7690c2f3b7 sectorbuilder: open FD later in AddPiece 2019-11-20 18:28:14 +01:00
Łukasz Magiera
5381d10fe3 StateAPI: account for slashing in StateMinerPower 2019-11-15 02:01:53 +01:00
Łukasz Magiera
727ca2ccb4 Skip sectorbuilder tests on slow hardware 2019-11-14 23:39:00 +01:00
Łukasz Magiera
d5c620dc44 sectorbuilder: Call AcquireSectorId in sync 2019-11-14 23:00:21 +01:00
Łukasz Magiera
b88807b947 Merge remote-tracking branch 'origin/master' into wip/sbfixes-links 2019-11-14 19:31:52 +01:00
Łukasz Magiera
a2c01defab Merge branch 'master' into wip/sbfixes-links 2019-11-14 19:27:53 +01:00
Łukasz Magiera
d64b21936f parallel sectorbuilder test 2019-11-14 19:26:09 +01:00
Jakub Sztandera
383c5c0c6f
Rename to --include-test-params
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-14 17:43:33 +01:00
Jakub Sztandera
c7b82dbc40
Skip slow tests
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-13 23:43:16 +01:00
whyrusleeping
86624c8ed4 fix tests and add some more trace attributes 2019-11-12 22:44:29 -08:00
whyrusleeping
9e464a4b90 more tracing spans 2019-11-12 12:42:22 -08:00
Łukasz Magiera
56d902c1bd storageminer: Throttle GeneratePieceCommitment in storeGarbage 2019-11-12 19:05:03 +01:00
Łukasz Magiera
73ab6c0c66 WorkerCount on storageminer config 2019-11-12 18:59:38 +01:00
Łukasz Magiera
81bd0bc517 Test to reproduce post error after restart 2019-11-09 11:42:36 +01:00
Łukasz Magiera
ed64e52e3a sectorbuilder: apply some review suggestions 2019-11-09 00:21:04 +01:00
Łukasz Magiera
3d360167df sectorbuilder: Call destroy in DI module 2019-11-08 21:30:50 +01:00
Łukasz Magiera
2c66e20a1e padreader: Some more testcases 2019-11-08 20:55:03 +01:00
Łukasz Magiera
61505b6802 Use the same dir in TestAcquireID 2019-11-08 19:55:23 +01:00
Łukasz Magiera
c729cabef1 keep track of last used sector id across restarts 2019-11-08 19:49:36 +01:00
Łukasz Magiera
071f05fa95 Storage miner API improvements 2019-11-08 19:23:58 +01:00
Łukasz Magiera
34846c538e sectorbuilder: Use StandaloneWriteWithAlignment 2019-11-07 21:43:57 +01:00
Łukasz Magiera
f6a49ab9f9 Wip fixing a thing 2019-11-07 20:54:24 +01:00
Łukasz Magiera
58c21b4a5e sectorbuilder: Always create directories 2019-11-07 19:33:46 +01:00
Łukasz Magiera
69b4bd9fb4 sectorbuilder: Also test PoSt 2019-11-07 17:43:59 +01:00
Łukasz Magiera
3e39d6e445 sectorbuilder: use standalone methods 2019-11-07 17:39:27 +01:00
Łukasz Magiera
1549269773 Make multiple deals per almost work 2019-11-07 15:45:53 +01:00
Łukasz Magiera
3fbe0abb75 cborrpc -> cborutil 2019-11-07 15:11:39 +01:00
Łukasz Magiera
8ec37e8728 deals: cleanup client state machine 2019-11-07 14:29:43 +01:00
whyrusleeping
480e7899ef fail test if deal errors 2019-11-07 00:39:34 +01:00
whyrusleeping
597dbe369c Add method to query latest deal state 2019-11-07 00:39:34 +01:00
whyrusleeping
9863942fe2 WIP: trying to write a test to reproduce the storage deal error 2019-11-07 00:39:34 +01:00
Łukasz Magiera
f7651f180b Create filler deals 2019-11-07 00:09:48 +01:00
Łukasz Magiera
68c2d4f58a deals: Set correct Refs 2019-11-06 18:38:42 +01:00
Łukasz Magiera
cb3965bcf5 Improve errors around deal handling 2019-11-06 13:04:33 +01:00
Łukasz Magiera
630134486d Mostly fix deals 2019-11-05 19:40:51 +01:00
Łukasz Magiera
3020f7a203 Fix statestore.List 2019-11-05 19:20:12 +01:00
Łukasz Magiera
ed5ccfd750 Merge remote-tracking branch 'origin/devnet/7' into feat/interactive-porep 2019-11-05 18:53:19 +01:00
Łukasz Magiera
6a802cc9db WIP fixing tests 2019-11-05 18:09:42 +01:00
whyrusleeping
4321256992 fix retrieval protocol error by wrapping stream in peeker 2019-11-05 08:38:54 -08:00
Łukasz Magiera
eb28c45c00 events: Plumb context to callbacks 2019-11-05 15:03:59 +01:00
whyrusleeping
9a398c6260 Track down all the uses of cboripld and eliminate them 2019-11-04 19:42:13 -08:00
Łukasz Magiera
c76ce2a2eb rate-limit some sectorbuilder ops 2019-11-04 18:36:29 +01:00
Łukasz Magiera
13da5a5966 Put WorkerThreads on sectorbuilder.Config 2019-11-04 17:47:08 +01:00
Łukasz Magiera
2a695611d5 Update sectorbuilder 2019-11-03 21:44:22 +01:00
Łukasz Magiera
c550e030b1 INTERACTIVE PROEP IS ALIVE 2019-11-02 16:07:26 +01:00
Łukasz Magiera
09e8cdc109
Merge pull request #504 from filcloud/fix/312-panic-close-miner
fix panic when close miner
2019-11-02 00:59:12 +01:00
Łukasz Magiera
1dcebece71 Some smaller fixes 2019-11-01 23:44:55 +01:00
Łukasz Magiera
82344649d3 Use state store for sectors 2019-11-01 15:06:49 +01:00
Łukasz Magiera
02c3be9099 statestore: More generic keys 2019-11-01 12:14:32 +01:00
Łukasz Magiera
2343ebc5b8 statestore: Use reflect for mutators 2019-11-01 12:07:05 +01:00
Łukasz Magiera
1583cf2593 Strip unused functionality from sectorstore 2019-11-01 11:05:48 +01:00
whyrusleeping
fc9091cc89 Get interactive porep sector sealing mostly working 2019-10-31 20:57:10 -07:00
Łukasz Magiera
3cde267a2a Move statestore to lib 2019-10-31 22:01:44 +01:00
Łukasz Magiera
ba937cf859 Fix sectorbuilder.VerifySeal 2019-10-31 20:03:26 +01:00
whyrusleeping
4e478330a4 more refactoring for interactive porep scheduling 2019-10-31 09:55:35 -07:00
wanghui
2aa8eebb99 move select 2019-10-31 17:39:42 +08:00
wanghui
8cb14335c3 use reflect select 2019-10-31 13:11:10 +08:00
whyrusleeping
759094198c Some more progress on interactive porep 2019-10-30 18:22:50 -07:00
Łukasz Magiera
7fa3e2db7a WIP Interactive PoRep 2019-10-30 19:10:29 +01:00
wanghui
03ca08d9bf fix panic when close miner 2019-10-30 18:23:13 +08:00
Łukasz Magiera
5a5c66600f storageminer: More correct listing of sealed sectors 2019-10-30 10:57:00 +01:00
Łukasz Magiera
922d8a90a5 storageminer: Restart sealing on restart 2019-10-29 23:19:58 +01:00
Łukasz Magiera
169c285fb3 storageminer: Handle uncommited sectors on start 2019-10-29 20:46:32 +01:00
Łukasz Magiera
02b67ea6b2 Fix TestSealAndVerify 2019-10-27 12:58:15 +01:00
Łukasz Magiera
874be79958 very basic sector seal scheduling 2019-10-27 09:56:53 +01:00
whyrusleeping
5257b1cce1 should use the new parameters 2019-10-25 18:09:11 +02:00
whyrusleeping
874ab1456c WIP: updating to new proofs code 2019-10-25 18:07:34 +02:00
Łukasz Magiera
76f1e6e207 on chain deals: Put dealIDs in CommitSector messages 2019-10-24 16:24:31 +02:00
Łukasz Magiera
fabd074165 on chain deals: Deals make it to the chain 2019-10-23 19:39:14 +02:00
Łukasz Magiera
213ac77d08 on chain deals: Expose more chain state in pond 2019-10-23 12:44:00 +02:00
Łukasz Magiera
64bfb38834 on chain deals: Get things to actually run! 2019-10-23 12:44:00 +02:00
Łukasz Magiera
aab3bd617a REST file import endpoint 2019-10-23 11:39:44 +02:00
whyrusleeping
6e90066a20 dont hang requests if websockets server shuts down 2019-10-20 15:37:51 +09:00
Jakub Sztandera
1bf713cb0a
Cleanup imports after rename
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-18 13:47:41 +09:00
whyrusleeping
ab7eafe76c remove empty const block 2019-10-15 09:56:40 +09:00
whyrusleeping
e8a1b54740 Add more info to storage-miner info command output 2019-10-14 11:32:32 +09:00
Jakub Sztandera
edd7085712
Update filebeat
Change log of rpc buffer as I want to set up alert when it goes to high

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-11 18:02:21 +02:00
Łukasz Magiera
b3a88a21ce
Merge pull request #299 from filecoin-project/feat/remove-vdfs
remove VDFs from tickets
2019-10-09 10:02:51 +02:00
whyrusleeping
19b16077fe remove VDFs from tickets 2019-10-09 13:38:59 +09:00
Łukasz Magiera
752654ab03 Devnet 3; Builtin bootstrap; NAT Port Map 2019-10-09 05:16:35 +02:00
whyrusleeping
86deb7528c fix websocket closing again 2019-10-08 17:29:37 +09:00
whyrusleeping
6dd8ed7d7a pass those tests 2019-10-04 10:06:05 -06:00
whyrusleeping
697bed27a6 fix websocket closing 2019-10-04 10:06:05 -06:00
Łukasz Magiera
2874022251 jsonrpc: Channel buffeering 2019-09-27 13:37:44 +02:00
Łukasz Magiera
fe8e1fe1e4 Almost working new post code 2019-09-27 01:07:40 +02:00
Łukasz Magiera
182b754aa2 fix tests 2019-09-23 15:19:05 +02:00
Łukasz Magiera
9a990dbd8a use new sectorbuilder file interfaces 2019-09-23 12:50:28 +02:00
Łukasz Magiera
d55e872135 Merge remote-tracking branch 'origin/master' into feat/post-schedule 2019-09-19 16:42:50 +02:00
Jakub Sztandera
9c02f7f839 Rought PoST method
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-09-18 17:10:03 +02:00
whyrusleeping
3e45088812 cleaning up a few types 2019-09-17 20:32:52 -07:00
Łukasz Magiera
5e2c100f4d storageminer: Better context handling 2019-09-17 20:36:06 +02:00
Łukasz Magiera
f2ecb772fa repo: Close datastore in Close 2019-09-17 16:34:22 +02:00
Łukasz Magiera
83f1a336a6 node: Basic graceful shutdown 2019-09-17 16:23:08 +02:00
Łukasz Magiera
5470fbeb8a jsonrpc: include method name in error log 2019-09-16 20:08:24 +02:00
Łukasz Magiera
eda72468ce Sector Commitment tracker 2019-09-16 20:08:24 +02:00
Jakub Sztandera
6eef3a6d64 Update sectorbuilder
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-09-16 14:47:55 +02:00
Łukasz Magiera
f79b755c58 retrieval: Make types more spec complaiant 2019-08-29 17:54:35 +02:00
Łukasz Magiera
e28c0c8d80 Fix sectorbuilder poRepProofPartitions 2019-08-29 17:51:06 +02:00
Łukasz Magiera
cad3efb9ba Command to list sealed blocks 2019-08-29 17:48:19 +02:00
whyrusleeping
7eb89f90d1 Replace most marshaling with codegen 2019-08-22 12:53:32 -07:00
whyrusleeping
5a7f59498e Wire up more proper ticket generation and verification logic 2019-08-16 12:39:12 -07:00
Łukasz Magiera
1579137236 storageminer: Initial PaymentVerify implementation 2019-08-15 15:52:14 +02:00
Łukasz Magiera
baf4fbe309 sectorstore: Address review feetback 2019-08-15 00:46:15 +02:00
Łukasz Magiera
e6493afd46 move poller to sector store 2019-08-14 23:34:43 +02:00
Łukasz Magiera
e430f86b69 pond: Basic message display in Block window 2019-08-09 19:32:46 +02:00
Łukasz Magiera
b2c3434bcd deals: Upstream bitswap changes 2019-08-08 19:16:41 +02:00
Łukasz Magiera
3e2d04a540 deals: Use temp files for AddPiece 2019-08-07 20:02:05 -07:00
Łukasz Magiera
c79cb7bf31 deals: Almost sealing client data 2019-08-07 20:01:51 -07:00
Łukasz Magiera
b65041cac1 deals: Sending initial proposal works 2019-08-07 20:01:22 -07:00
Łukasz Magiera
549e7db12f deals: Implement basic handshake logic 2019-08-07 20:01:22 -07:00
whyrusleeping
95e7b230ec cleanup from self review 2019-08-07 18:23:44 -07:00
whyrusleeping
ba0d648546 sealing garbage data into a sector that gets accepted by the chain works! 2019-08-07 18:16:58 -07:00
whyrusleeping
5c32725510 sector commitments make it all the way to the chain 2019-08-07 16:22:35 -07:00
whyrusleeping
b7c6333fef implement sector commitments and proof verification 2019-08-06 23:35:57 -07:00
whyrusleeping
2576853426 begin work on storage miner processes 2019-07-29 14:01:31 -07:00
whyrusleeping
b83ff6b9dc Wire up sector builder commands through the api 2019-07-29 12:08:47 -07:00
whyrusleeping
f1432826d5 start integrating sector builder 2019-07-29 12:08:05 -07:00
whyrusleeping
4058885df9 switch over to new sector builder library 2019-07-26 15:29:18 -07:00
Jakub Sztandera
81b5f46062 Implement RPC tracing
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-07-26 08:50:21 +02:00
whyrusleeping
501b2e4640 update to and use fixed cbor rpc code 2019-07-25 17:49:27 -07:00
Łukasz Magiera
6f80ff0800 pond: Connect stuff in ConnMgr 2019-07-25 23:52:15 +02:00
Łukasz Magiera
4054200bdd jsonrpc: Catch panics in jsonrpc calls 2019-07-25 16:08:41 +02:00
Łukasz Magiera
7ec9d45915 gofmt 2019-07-25 14:54:19 +02:00
Łukasz Magiera
93a8ee11db jsonrpc: Work with browsers 2019-07-24 19:09:00 +02:00
Łukasz Magiera
77bc243110 jsonrpc: Break NewMergeClient into smaller functions 2019-07-24 14:20:57 +02:00
Łukasz Magiera
81d7c30fac Couple lint fixes 2019-07-24 03:16:17 +02:00
Łukasz Magiera
f0e807dabb Fix jsonrpc client after splitting apis 2019-07-24 02:40:19 +02:00
Łukasz Magiera
0a17041221 Fix rpc tests after rebase 2019-07-24 00:39:48 +02:00
Łukasz Magiera
6554db3e5f auth: Addres review 2019-07-24 00:38:52 +02:00
Łukasz Magiera
a88be8d1df auth: Store token in repo 2019-07-24 00:38:52 +02:00
Łukasz Magiera
309ecc4052 auth: Put auth methods in API 2019-07-24 00:37:05 +02:00
Łukasz Magiera
fa4bf5178a auth: Restructure for getting tokens form keystore 2019-07-24 00:37:05 +02:00
Łukasz Magiera
c0f3bdbb61 jsonrpc: verify jwt tokens 2019-07-24 00:37:05 +02:00
Łukasz Magiera
265302f4aa Implement API-level auth 2019-07-24 00:36:19 +02:00
Łukasz Magiera
9b3ae45f61 jsonrpc: Break handleWsConn into smaller functions 2019-07-23 03:45:10 +02:00
Łukasz Magiera
2f0a088b18 jsonrpc: Move ws handler state to a struct 2019-07-23 03:20:48 +02:00
Łukasz Magiera
0d5d6cd1c2 jsonrpc: lint fixes 2019-07-23 02:40:45 +02:00
Łukasz Magiera
1b1ec2b812 jsonrpc: cleanup websocket handling logic a bit 2019-07-23 02:40:45 +02:00
Łukasz Magiera
40fa1becb5 jsonrpc: Handle closing channels 2019-07-23 02:40:45 +02:00
Łukasz Magiera
64e3272fbf jsonrpc: remove test stuff from main code 2019-07-23 02:40:45 +02:00
Łukasz Magiera
98cbfdebb4 jsonrpc: test unmarshalable responses 2019-07-23 02:40:01 +02:00
Łukasz Magiera
17c76abdb2 jsonrpc: Panic less in client 2019-07-23 02:40:01 +02:00
Łukasz Magiera
527ab7100a jsonrpc: basic channel test 2019-07-23 02:40:01 +02:00
Łukasz Magiera
dda1dfdc80 jsonrpc: don't panic on not found chan handlers 2019-07-23 02:40:01 +02:00
Łukasz Magiera
3e8afa9511 jsonrpc: fix chan handler registration race 2019-07-23 02:40:01 +02:00
Łukasz Magiera
fd40f9a82a jsonrpc: gofmt 2019-07-23 02:40:01 +02:00
Łukasz Magiera
790ed0730e jsonrpc: use testify in tests 2019-07-23 02:40:01 +02:00
Łukasz Magiera
ff4d1b5819 jsonrpc: Channel support 2019-07-23 02:40:01 +02:00
Whyrusleeping
19979a5ab7
Merge pull request #48 from filecoin-project/feat/create-miner
create miner command, stubbed out methods
2019-07-22 12:13:11 -07:00
whyrusleeping
05b8158285 implement a few of the TODOs 2019-07-20 16:37:37 -07:00
Jakub Sztandera
d745fa1af1 Switch to go-bls-sigs
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-07-19 13:56:58 +02:00
Whyrusleeping
69b8ec42af
Merge pull request #56 from filecoin-project/fix/ws-disconn-hang
jsonrpc: Don't hang when remote crashes
2019-07-18 09:51:29 -07:00
Łukasz Magiera
185db76646 jsonrpc: don't hang when remote crashes 2019-07-18 13:26:09 +02:00
Łukasz Magiera
055a98c57b jsonrpc: fix test 2019-07-18 13:09:13 +02:00
Łukasz Magiera
2b651ad04d jsonrpc: better server errors on unmarshal 2019-07-18 13:07:28 +02:00
Łukasz Magiera
e32d7aee12 jsonrpc: wrap result unmarshal errors 2019-07-18 12:46:32 +02:00
Whyrusleeping
4c2fd3b976
Merge pull request #35 from filecoin-project/feat/sectorbuilder
Import sectorbuilder bindings
2019-07-17 09:38:34 -07:00
Łukasz Magiera
19fd7d6cb6 Import sectorbuilder submodule 2019-07-17 13:08:38 +02:00
Łukasz Magiera
084c46c858 Extract sectorbuilder 2019-07-17 13:01:02 +02:00
Łukasz Magiera
2c9ad786a4 Prepare sectorbuilder for extraction 2019-07-17 00:09:40 +02:00
Łukasz Magiera
8c94758285 Improt sectorbuilder bindings 2019-07-16 17:14:16 +02:00
Łukasz Magiera
661043f5c8 Lint fixes 2019-07-15 18:34:55 +02:00
Łukasz Magiera
1153f050bb Context cancellation over websockets 2019-07-15 18:21:48 +02:00
Łukasz Magiera
b93d71e8cb jsonrpc: ws addresses in tests 2019-07-15 17:05:45 +02:00
Łukasz Magiera
96ed4225fb improve error handling in handleWsConn 2019-07-13 14:44:20 +02:00
Łukasz Magiera
df90b72500 jsonrpc: update tests 2019-07-12 19:12:51 +02:00
Łukasz Magiera
6a20d0dafe jsonrpc: Websocket server 2019-07-12 19:12:38 +02:00
Łukasz Magiera
0eb208e1d3 WS JsonRPC client 2019-07-12 17:29:41 +02:00
Whyrusleeping
b843dcb590
Merge pull request #9 from filecoin-project/feat/netcli
Network CLI
2019-07-09 12:05:56 -04:00
Łukasz Magiera
1bead4bb7e jsonrpc: client debug logging 2019-07-09 15:16:15 +02:00
whyrusleeping
e8d54839da pull buffered blockstore into the lib folder 2019-07-08 22:29:01 +02:00
Łukasz Magiera
cdca2ff2c7 Some net commands / apis 2019-07-08 21:07:16 +02:00
Łukasz Magiera
9299ba5fa6 Fix imports 2019-07-08 14:46:30 +02:00
Łukasz Magiera
4ef4721820 Move json/cbor rpc to lib 2019-07-08 13:02:02 +02:00
Jakub Sztandera
8fcda5f610 Cleanup bls-signatures
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-07-08 12:41:45 +02:00
Jakub Sztandera
6ab0017a07 Merge commit 'ec306ba24dad8f616f61e5ae89b55a723bc6e6f6' as 'lib/bls-signatures' 2019-07-08 12:31:38 +02:00
Jakub Sztandera
200d8058b8 Cleanup after subtree go-filecoin/crypto
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-07-08 12:31:00 +02:00
Jakub Sztandera
c4614713f3 Merge commit '800f835a2eff3d00e0518ae4f703731f72faf686' as 'lib/crypto' 2019-07-08 12:24:55 +02:00