Commit Graph

287 Commits

Author SHA1 Message Date
Jakub Sztandera
794490c490
Remove old bitvector
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:20 +01:00
Jakub Sztandera
01c0a6ec9f
Optimize and start wrapping it up
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:20 +01:00
Jakub Sztandera
d3dfd8a73b
Add rle encoder
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:20 +01:00
Jakub Sztandera
3d6071ed3f
Improve benchmarks and fix bitvector iterator
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:20 +01:00
Jakub Sztandera
9bf871ee53
Improve bitvector performance
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:19 +01:00
Jakub Sztandera
e5b3c4757d
More iterative algorithms
- Add RunIterator and decoder from RLE
 - Add BitIterator and BitsFromRuns
 - Add BitsFromSlice
 - Add RunsFromBits

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:19 +01:00
Jakub Sztandera
c557aa206f
Add lazy RLE+ decoding
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-12-07 15:48:19 +01:00
Łukasz Magiera
bbc61a8f86
Merge pull request #766 from filecoin-project/feat/holes
Pre-sealing holes
2019-12-07 10:29:17 +01:00
Jakub Sztandera
92e7b4afbf
Fix tests, circle and make ux nicer
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-06 22:49:44 +01:00
Łukasz Magiera
7ed63fb7f8 seed: Trim cache 2019-12-06 22:18:52 +01:00
Jakub Sztandera
80592a87cf
Pre-sealing holes
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-06 18:53:33 +01:00
Łukasz Magiera
969ef179ef Fix AddPiece with disabled local workers 2019-12-05 16:53:29 +01:00
Łukasz Magiera
2ce56dbfcb Allow no local sectorbuilder workers 2019-12-05 16:28:04 +01:00
Łukasz Magiera
6f024bee58
Merge pull request #736 from filecoin-project/feat/params-v20
Params v20
2019-12-05 14:03:37 +01:00
whyrusleeping
5aa7bf6123 there is no real correlation between challenge count and len(winners) 2019-12-05 13:13:45 +01:00
Łukasz Magiera
291fab74c1 sectorbuilder: update PoRepProofPartitions 2019-12-05 12:52:13 +01:00
whyrusleeping
b7a89b56b3 more statticcheck! 2019-12-04 21:14:19 -08:00
Łukasz Magiera
3daf9103a8 paramfetch: Only pull necessary params 2019-12-04 20:44:15 +01:00
Łukasz Magiera
b2001db9d5 worker: Fix rebaining bugs 2019-12-04 17:53:32 +01:00
Łukasz Magiera
28dde1a2d3 worker: Use system tar for moving cache around 2019-12-04 15:58:28 +01:00
Łukasz Magiera
160e11ce8c Merge branch 'master' into feat/remote-workers 2019-12-03 03:24:00 +01:00
Łukasz Magiera
1cdf6a6f8d use challangeCount as sampleRate in IsTicketWinner 2019-12-02 15:24:27 +01:00
Łukasz Magiera
d5b94884c8 Buch of lint fixes 2019-12-02 13:51:16 +01:00
Łukasz Magiera
c7cf20843e actually set unsealed path in sectorbuilder 2019-12-01 22:22:39 +01:00
Łukasz Magiera
fd8f65248b deals: Correctly set deal ID in provider states 2019-12-01 21:07:57 +01:00
Łukasz Magiera
a59d0f0f8c Fix unsealing, sector based data refs 2019-12-01 18:58:31 +01:00
whyrusleeping
422e25c8e6 self review: some cleanup 2019-11-30 22:02:52 -06:00
whyrusleeping
59d9c681ad Rework miner test setups to fix tests 2019-11-30 17:21:11 -06:00
Łukasz Magiera
89556819ae seal-worker: Handle cache 2019-11-30 14:22:50 +01:00
Łukasz Magiera
ed9279cf0c Some fixes and dev utils 2019-11-30 10:25:31 +01:00
Łukasz Magiera
8767d8c16b Merge branch 'feat/election-post' into feat/remote-workers 2019-11-30 09:42:28 +01:00
Łukasz Magiera
466f1d8cb9 Use ProvingSet for GetSectorsForElectionPost 2019-11-28 19:52:28 +01:00
Łukasz Magiera
920fd3ba9b storageminer: New fallback post scheduler 2019-11-28 18:44:57 +01:00
Łukasz Magiera
aefd432422 actors: Fallback post progress 2019-11-28 13:46:56 +01:00
whyrusleeping
45288b8810 WIP: uncomment out windowed post code, try to make it work 2019-11-27 21:36:34 -06:00
Łukasz Magiera
99b9dfe18d filecoin-ffi master 2019-11-27 23:37:02 +01:00
Łukasz Magiera
5d30106f5a More v18 updates 2019-11-27 18:10:34 +01:00
Łukasz Magiera
1ccfe7d8f7 update filecoin-ffi to v18 params 2019-11-27 16:00:04 +01:00
Łukasz Magiera
23e0008b81 Merge remote-tracking branch 'origin/master' into feat/election-post 2019-11-27 13:31:44 +01:00
laser
3623925f66 replace go-bls-sigs and go-sectorbuilder with filecoin-ffi
- remove old submodules and add new submodule
- update build tooling to consume new unified static library
- update consumers of old libs to use new package
2019-11-26 18:16:37 -08:00
whyrusleeping
249bd8389f Copy over last sector ID key when migrating sectorbuilder 2019-11-26 19:39:07 -06:00
Łukasz Magiera
407f6fa6f8 sync: correct vrfBase for ticket check 2019-11-26 23:54:11 +01:00
Łukasz Magiera
d5a632f644 sectorbuilder: Drop stateful sectorbuilder refs 2019-11-26 23:13:01 +01:00
Łukasz Magiera
871c30d2a0 WIP trying to get election post to compute 2019-11-26 16:15:41 +01:00
Łukasz Magiera
86cd28fa7e sectorbuilder: Measure thigs in TestSealAndVerify 2019-11-26 13:29:42 +01:00
whyrusleeping
a98295a747 use the correct sector sizes in places 2019-11-25 20:43:43 -06:00
Łukasz Magiera
7a3ffbf9fe WIP getting post in sectorbuilder_test to work 2019-11-26 00:32:02 +01:00
Łukasz Magiera
df0be815ae More correct challangeCount calc 2019-11-25 17:18:11 +01:00
whyrusleeping
a0588d513d Add lotus-gen, rewire genesis mining 2019-11-24 22:45:13 -06:00
Jakub Sztandera
36b7c5a32a
Swtich to xerrors
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-22 10:20:56 -06:00
Łukasz Magiera
3281e9448a fix rspco serialization 2019-11-22 16:48:02 +01:00
whyrusleeping
96482f456a WIP: election post restructuring 2019-11-21 16:21:45 -06:00
Łukasz Magiera
b726b95298 WIP remote sector CommitSseal 2019-11-21 20:51:53 +01:00
Łukasz Magiera
d4197bbadc Working remote PreCommit 2019-11-21 19:38:50 +01:00
Łukasz Magiera
98b1de33b6 Stats for remote workers 2019-11-21 17:23:42 +01:00
Łukasz Magiera
88bbcd80ea support remote SealPreCommit 2019-11-21 17:10:09 +01:00
Łukasz Magiera
ba3ad75670 remote-worker: wire up storage miner endpoints 2019-11-21 15:10:51 +01:00
Łukasz Magiera
9725eb78bf wip remote sectorbuilder workers 2019-11-21 01:53:06 +01:00
Łukasz Magiera
ca70ac05d7 sectorbuilder: Drop some unused functions 2019-11-20 20:59:10 +01:00
Łukasz Magiera
7690c2f3b7 sectorbuilder: open FD later in AddPiece 2019-11-20 18:28:14 +01:00
Łukasz Magiera
5381d10fe3 StateAPI: account for slashing in StateMinerPower 2019-11-15 02:01:53 +01:00
Łukasz Magiera
727ca2ccb4 Skip sectorbuilder tests on slow hardware 2019-11-14 23:39:00 +01:00
Łukasz Magiera
d5c620dc44 sectorbuilder: Call AcquireSectorId in sync 2019-11-14 23:00:21 +01:00
Łukasz Magiera
b88807b947 Merge remote-tracking branch 'origin/master' into wip/sbfixes-links 2019-11-14 19:31:52 +01:00
Łukasz Magiera
a2c01defab Merge branch 'master' into wip/sbfixes-links 2019-11-14 19:27:53 +01:00
Łukasz Magiera
d64b21936f parallel sectorbuilder test 2019-11-14 19:26:09 +01:00
Jakub Sztandera
383c5c0c6f
Rename to --include-test-params
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-14 17:43:33 +01:00
Jakub Sztandera
c7b82dbc40
Skip slow tests
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-13 23:43:16 +01:00
whyrusleeping
86624c8ed4 fix tests and add some more trace attributes 2019-11-12 22:44:29 -08:00
whyrusleeping
9e464a4b90 more tracing spans 2019-11-12 12:42:22 -08:00
Łukasz Magiera
56d902c1bd storageminer: Throttle GeneratePieceCommitment in storeGarbage 2019-11-12 19:05:03 +01:00
Łukasz Magiera
73ab6c0c66 WorkerCount on storageminer config 2019-11-12 18:59:38 +01:00
Łukasz Magiera
81bd0bc517 Test to reproduce post error after restart 2019-11-09 11:42:36 +01:00
Łukasz Magiera
ed64e52e3a sectorbuilder: apply some review suggestions 2019-11-09 00:21:04 +01:00
Łukasz Magiera
3d360167df sectorbuilder: Call destroy in DI module 2019-11-08 21:30:50 +01:00
Łukasz Magiera
2c66e20a1e padreader: Some more testcases 2019-11-08 20:55:03 +01:00
Łukasz Magiera
61505b6802 Use the same dir in TestAcquireID 2019-11-08 19:55:23 +01:00
Łukasz Magiera
c729cabef1 keep track of last used sector id across restarts 2019-11-08 19:49:36 +01:00
Łukasz Magiera
071f05fa95 Storage miner API improvements 2019-11-08 19:23:58 +01:00
Łukasz Magiera
34846c538e sectorbuilder: Use StandaloneWriteWithAlignment 2019-11-07 21:43:57 +01:00
Łukasz Magiera
f6a49ab9f9 Wip fixing a thing 2019-11-07 20:54:24 +01:00
Łukasz Magiera
58c21b4a5e sectorbuilder: Always create directories 2019-11-07 19:33:46 +01:00
Łukasz Magiera
69b4bd9fb4 sectorbuilder: Also test PoSt 2019-11-07 17:43:59 +01:00
Łukasz Magiera
3e39d6e445 sectorbuilder: use standalone methods 2019-11-07 17:39:27 +01:00
Łukasz Magiera
1549269773 Make multiple deals per almost work 2019-11-07 15:45:53 +01:00
Łukasz Magiera
3fbe0abb75 cborrpc -> cborutil 2019-11-07 15:11:39 +01:00
Łukasz Magiera
8ec37e8728 deals: cleanup client state machine 2019-11-07 14:29:43 +01:00
whyrusleeping
480e7899ef fail test if deal errors 2019-11-07 00:39:34 +01:00
whyrusleeping
597dbe369c Add method to query latest deal state 2019-11-07 00:39:34 +01:00
whyrusleeping
9863942fe2 WIP: trying to write a test to reproduce the storage deal error 2019-11-07 00:39:34 +01:00
Łukasz Magiera
f7651f180b Create filler deals 2019-11-07 00:09:48 +01:00
Łukasz Magiera
68c2d4f58a deals: Set correct Refs 2019-11-06 18:38:42 +01:00
Łukasz Magiera
cb3965bcf5 Improve errors around deal handling 2019-11-06 13:04:33 +01:00
Łukasz Magiera
630134486d Mostly fix deals 2019-11-05 19:40:51 +01:00
Łukasz Magiera
3020f7a203 Fix statestore.List 2019-11-05 19:20:12 +01:00
Łukasz Magiera
ed5ccfd750 Merge remote-tracking branch 'origin/devnet/7' into feat/interactive-porep 2019-11-05 18:53:19 +01:00
Łukasz Magiera
6a802cc9db WIP fixing tests 2019-11-05 18:09:42 +01:00
whyrusleeping
4321256992 fix retrieval protocol error by wrapping stream in peeker 2019-11-05 08:38:54 -08:00
Łukasz Magiera
eb28c45c00 events: Plumb context to callbacks 2019-11-05 15:03:59 +01:00
whyrusleeping
9a398c6260 Track down all the uses of cboripld and eliminate them 2019-11-04 19:42:13 -08:00