Commit Graph

1065 Commits

Author SHA1 Message Date
Łukasz Magiera
c91d970c41 storageminer: Init with default local storage 2020-03-05 02:12:52 +01:00
Łukasz Magiera
7db1dd52bd storagemgr: Wire up most of AddPiece logic 2020-03-05 01:38:07 +01:00
Łukasz Magiera
4e46d00eb6 Merge remote-tracking branch 'origin/testnet/3' into feat/4stage-seal 2020-03-04 04:01:13 +01:00
Łukasz Magiera
12d870e274 sealmgr: Read only multi-path file manager 2020-03-04 03:24:08 +01:00
whyrusleeping
7b11224130 go mod updates, and a couple small fixes 2020-03-03 16:07:27 -08:00
Łukasz Magiera
a0dbb6bdd6 Storage Manager refactor 2020-03-03 23:19:22 +01:00
Łukasz Magiera
3abb59a550 sealmgr: Interfaces; simple initial impl 2020-03-03 01:46:58 +01:00
whyrusleeping
60b19c51ef more misc fixes 2020-03-01 17:09:38 -08:00
Łukasz Magiera
f54a555aa8 Merge remote-tracking branch 'origin/testnet/3' into feat/4stage-seal 2020-02-29 03:21:01 +01:00
whyrusleeping
cf2eefdfbe Fix up client storage deal matcher 2020-02-28 16:05:56 -08:00
whyrusleeping
713059275d progress is made incrementally 2020-02-28 14:00:34 -08:00
whyrusleeping
5b657a2905 pass on proper sector types 2020-02-28 12:52:14 -08:00
whyrusleeping
0945d8725e fixing miner logic to make more tests pass 2020-02-28 11:31:28 -08:00
Łukasz Magiera
5a4f917c60 4-stage v23 lotus-bench 2020-02-28 19:06:59 +01:00
Łukasz Magiera
d5b8defcbb Begin work on integrating new sectorbuilder interfaces 2020-02-28 18:17:40 +01:00
Łukasz Magiera
58f8481a12 sbmock: Set commR with correct cid codec 2020-02-28 00:49:28 +01:00
Łukasz Magiera
f83bbc2cbe Regen cbor marshalers 2020-02-28 00:34:48 +01:00
whyrusleeping
df6e3e83bf make it all build finally 2020-02-27 13:45:31 -08:00
Łukasz Magiera
48619dc238 Merge branch 'feat/update-sectorbuilder' of github.com:filecoin-project/go-lotus into feat/update-sectorbuilder 2020-02-27 01:56:15 +01:00
Łukasz Magiera
2aeab13aae zerocomm: 'Fix' the test 2020-02-27 01:56:09 +01:00
whyrusleeping
6c12068fb7 more fixes for random garbage 2020-02-26 16:42:39 -08:00
whyrusleeping
767392332e WIP: update sectorbuilder 2020-02-26 14:54:34 -08:00
Łukasz Magiera
cb09e48e8c Some test fixes 2020-02-26 10:05:22 +01:00
Łukasz Magiera
0a9e0acc6f types: Drop some redundant alias types 2020-02-25 22:09:22 +01:00
Łukasz Magiera
ae634ef7df actors: Remove addrass aliases 2020-02-25 21:54:58 +01:00
Łukasz Magiera
cd10a3126b actors: drop a bunch of type aliases 2020-02-25 21:35:15 +01:00
Łukasz Magiera
794429dd55 post-upstream-merge fixes 2020-02-24 18:45:25 +01:00
Łukasz Magiera
d787aa5007 Merge branch 'testnet/3' into feat/specs-actors 2020-02-24 18:32:02 +01:00
Łukasz Magiera
1d417685d1 Fix Sealing in TestAPIDealFlow 2020-02-23 21:33:10 +01:00
Łukasz Magiera
917df9a5b8 More unified randomness handling 2020-02-23 21:00:47 +01:00
Łukasz Magiera
a82d156ccb Deal flow fixes 2020-02-23 16:50:36 +01:00
Łukasz Magiera
226685dc1c Implement committed capacity sectors 2020-02-23 01:47:47 +01:00
Łukasz Magiera
1381baf088 Generate zero CommP table 2020-02-22 15:06:38 +01:00
Łukasz Magiera
b8428423a1 get chain to runnable state 2020-02-22 14:10:46 +01:00
Łukasz Magiera
18627e77ff gofmt; mod tidy 2020-02-21 20:28:20 +01:00
Łukasz Magiera
4e98f2e438 some test fixes 2020-02-21 20:28:01 +01:00
Łukasz Magiera
a6268f055d storageminer: Update api interfaces 2020-02-21 18:43:44 +01:00
Łukasz Magiera
57c43183e6 update specs-actors 2020-02-21 17:57:40 +01:00
whyrusleeping
00d7bd1a77 chain/gen tests now pass! 2020-02-20 00:37:10 -08:00
Łukasz Magiera
69ec3c205a Merge remote-tracking branch 'origin/master' into testnet/3 2020-02-19 22:10:14 +01:00
whyrusleeping
6d0eec6395 working towards a working genesis generator 2020-02-17 23:15:30 -08:00
Aayush Rajasekaran
60970008c8 Re: : API methods should receive TipSetKeys, not TipSets, as input 2020-02-17 19:53:43 -05:00
Łukasz Magiera
b6354dfb74 vm: Partially fix vm runtime shim 2020-02-16 23:15:12 +01:00
Łukasz Magiera
5ced1ab905 gofmt 2020-02-14 22:38:30 +01:00
Łukasz Magiera
c544c2b5c5 Set hamt bitwidth 2020-02-14 15:14:39 +01:00
Łukasz Magiera
48c5494cf9 use specs-actors birfields 2020-02-14 01:24:24 +01:00
Łukasz Magiera
74bf9119cc spects-actors: More type propagation 2020-02-13 04:50:45 +01:00
Łukasz Magiera
36aed6f871 Update cbor-gen 2020-02-13 02:37:28 +01:00
Łukasz Magiera
e5ab64a3ab Fix client market impl, gofmt 2020-02-13 01:15:33 +01:00
Łukasz Magiera
91b2d2c2fe cbor-gen fixes 2020-02-13 01:10:07 +01:00
Łukasz Magiera
255f511abd storageminer: Updates to new types 2020-02-12 23:12:11 +01:00
Łukasz Magiera
0aaf7b25a9 Propagate spec actor types more 2020-02-12 08:44:20 +01:00
Łukasz Magiera
a4f6b739be genesis: Change template types 2020-02-12 01:58:55 +01:00
Łukasz Magiera
63e801e00f genesis: Cleanup the structure 2020-02-11 21:48:03 +01:00
Łukasz Magiera
3343e82307 wip gengen fixing 2020-02-11 04:31:28 +01:00
Łukasz Magiera
4d5c4d89ce stmgr: Update stmgr utils 2020-02-11 03:33:27 +01:00
Łukasz Magiera
07ae3e49a3 mocksb: Update to the new interface 2020-02-11 02:23:03 +01:00
Łukasz Magiera
fa2c6605c2 Fix compilation after dep updates 2020-02-11 02:11:58 +01:00
Łukasz Magiera
c0ed0dda8e specs-actors: Fis some test compilation errors 2020-02-10 20:16:36 +01:00
Łukasz Magiera
c72727b4fd specs-actors: Fix most compilation errors 2020-02-09 07:06:32 +01:00
Łukasz Magiera
41daf5ad28 gofmt 2020-02-08 03:18:37 +01:00
Łukasz Magiera
13435aebdc Spec Actors integration 2020-02-08 03:18:32 +01:00
Łukasz Magiera
72c89ab89b Merge pull request from filecoin-project/fix/squash-forks
squash forks and use correct amt library everywhere
2020-02-06 20:14:55 +01:00
Łukasz Magiera
cbf63aab1e Merge branch 'master' into testnet/3 2020-02-05 21:27:04 +01:00
whyrusleeping
faf05cafcf squash forks and use correct amt library everywhere 2020-02-04 18:26:42 -08:00
Łukasz Magiera
4bd6d7be1a storageminer: Work around broken fastPledgeCommitment 2020-02-04 21:46:31 +01:00
Łukasz Magiera
0e81c1cf7e Merge branch 'testnet/3' into feat/update-markets 2020-02-04 07:17:22 +01:00
Łukasz Magiera
90ecfbe0e4 Merge branch 'master' into feat/new-sb-fs 2020-02-04 04:15:25 +01:00
laser
ea2f53745e split FPoStScheduler from Miner
Rationale: Nodes want to handle scheduling of PoSts. Splitting the 
FPoStScheduler from the Miner allows nodes to reuse the Miner w/out 
having to reuse the PoSt scheduler.
2020-02-03 15:48:11 -08:00
Łukasz Magiera
e453a38865 sealing: fix finalize with cache only 2020-02-02 20:36:15 +01:00
Łukasz Magiera
37929ff75f sealing: more logging in pledge sector flow 2020-01-31 20:22:31 +01:00
Łukasz Magiera
b6ba0ed47c fix pledgeReader with small sectors 2020-01-31 20:07:20 +01:00
Łukasz Magiera
95f344540e Fix tests 2020-01-31 02:27:38 +01:00
Łukasz Magiera
ee9060aa14 Update sectorbuilder 2020-01-31 02:18:48 +01:00
Łukasz Magiera
db50d01db9 Merge remote-tracking branch 'origin/master' into feat/new-sb-fs 2020-01-30 07:46:47 +01:00
Łukasz Magiera
cec2085790 sealing: Fix pledgeReader 2020-01-30 07:41:30 +01:00
Łukasz Magiera
75ee71e35a Merge branch 'master' into feat/new-sb-fs 2020-01-30 07:39:52 +01:00
Łukasz Magiera
d27187b1de Fix fast pledge math 2020-01-30 02:01:10 +01:00
Łukasz Magiera
4aaa758543 fpost: better fault checks 2020-01-30 01:50:58 +01:00
Łukasz Magiera
03f07042ff fpost: print message apply result 2020-01-29 23:47:28 +01:00
Łukasz Magiera
e2b2026fa5 actually call finalizeSector 2020-01-29 23:37:31 +01:00
Łukasz Magiera
4b8b79dbe0 sealing: FinalizeSector step 2020-01-29 22:25:06 +01:00
Whyrusleeping
9e5cc46638
Merge pull request from filecoin-project/feat/par-pledge
sealing: Parallel CommP calc in pledge sector
2020-01-29 12:15:22 -08:00
Łukasz Magiera
ba542cf67a test fastPledgeCommitment 2020-01-29 21:01:20 +01:00
Łukasz Magiera
25f544f3db sealing: round parts in fastPledgeCommitment 2020-01-29 20:12:27 +01:00
Łukasz Magiera
5af64c53b6 initial sectorbuilder FS refactor integration 2020-01-29 00:08:02 +01:00
hannahhoward
00941e4461 fix(sectorblocks): minor cleanups
add error check, slice keys
2020-01-28 14:15:12 -08:00
hannahhoward
71289b58ff feat(markets): update to support car files
Upgrades lotus version of go-fil-markets to its head, which supports car files and handles tracking
of pieces, reducing the size of sectorBlocks
2020-01-28 14:08:13 -08:00
laser
1cdf29244c write basic test affirming state change 2020-01-28 12:39:07 -08:00
laser
cb665caf9e planCommitting must handle SectorCommitFailed
The SectorCommitFailed struct can be created from within Sealing#handleCommitting, and is created if actors.SerializeParams(params) produces an error or if m.api.MpoolPushMessage(ctx.Context(), msg) produces an error.
2020-01-28 11:46:26 -08:00
Łukasz Magiera
c9c437cb3b sealing: Parallel CommP calc in pledge sector 2020-01-25 12:15:28 +01:00
Łukasz Magiera
2f2e7be12a sealing: docstrings for sanity-checks 2020-01-24 21:15:02 +01:00
Łukasz Magiera
b5660b58b3 Merge remote-tracking branch 'origin/master' into feat/sector-recovery 2020-01-24 01:53:58 +01:00
Łukasz Magiera
6b2b22782b sealing: PreCommitFailed can go to SealFailed 2020-01-23 18:45:57 +01:00
Łukasz Magiera
de52d3cadd sealing: Handlef for PreCommitFailed 2020-01-23 18:34:04 +01:00
Łukasz Magiera
7e43c40529 storageminer: Use tabwriter in sectors list 2020-01-23 17:15:45 +01:00
Łukasz Magiera
df9dfa9d0c sealing: Don't infinite-loop on fatal errors 2020-01-23 17:11:58 +01:00
Łukasz Magiera
08a2e0f82c sealing: Errors are hard 2020-01-23 17:02:55 +01:00
Łukasz Magiera
01dc9c767e sealing: Error types that can actually be checked 2020-01-23 16:57:50 +01:00
Łukasz Magiera
67b441838b sealing: Actually call handleSealFailed 2020-01-23 16:47:33 +01:00
Łukasz Magiera
94f2948020 sealing: implement handler for sealFailed 2020-01-23 16:38:10 +01:00
Łukasz Magiera
558e4c7669 Fix tests after sectorbuilder update 2020-01-23 15:33:28 +01:00
Łukasz Magiera
c3a5da8586 storageminer: log flag for sector status 2020-01-23 15:18:05 +01:00
Łukasz Magiera
02c8ab7839 sealing: Check ticket expiration 2020-01-23 15:05:44 +01:00
Łukasz Magiera
26e6bc32ee sealing: check deal expiration 2020-01-23 13:17:45 +01:00
Łukasz Magiera
e67ee227a3 sealing: Nonzero exit on commit isn't fatal 2020-01-23 11:02:20 +01:00
Łukasz Magiera
7fb3360d6f storage: Fix handing stop call 2020-01-23 10:46:46 +01:00
Łukasz Magiera
e43174f2af sealing: Not getting seal ticket isn't fatal 2020-01-22 22:16:45 +01:00
Łukasz Magiera
d3a9006272 sealing: Wire up sector event log 2020-01-22 21:29:19 +01:00
Łukasz Magiera
ba5a5d1248 sealing: wire up checkPieces and checkSeal 2020-01-22 20:47:29 +01:00
Łukasz Magiera
052b090bbf sealing: Fix planOne for global events 2020-01-22 19:30:56 +01:00
Łukasz Magiera
1e877eae86 sealing: Add Log field to SectorInfo 2020-01-22 03:41:39 +01:00
Łukasz Magiera
3c934ea21f Merge branch 'master' into feat/sector-recovery 2020-01-22 03:27:11 +01:00
Łukasz Magiera
08e1b52281 Fix master build 2020-01-21 19:51:17 +01:00
Łukasz Magiera
cf153f89df
Merge pull request from filecoin-project/feat/soft-map-cbg
cbor-gen: Soft struct-map unmarshaling
2020-01-21 19:44:39 +01:00
Łukasz Magiera
d2401bc339 Update cbor-gen, error on unknown fields 2020-01-21 17:28:55 +01:00
Łukasz Magiera
fd0bfc8777
Merge pull request from filecoin-project/feat/state-call-cmd
implement state call command for easy method calling
2020-01-21 17:16:54 +01:00
Łukasz Magiera
2076cb106d cobr-gen: Soft struct-map unmarshaling 2020-01-21 17:05:10 +01:00
Łukasz Magiera
5b304fa20a fpost: More logging 2020-01-21 16:01:42 +01:00
Łukasz Magiera
e4de5c55ac sealing: Some state renaming 2020-01-20 23:04:46 +01:00
Łukasz Magiera
efaf2c720b sealing: WIP SectorInfo sanity checks 2020-01-20 23:04:27 +01:00
dtynn
b2fcca87ff small fixes 2020-01-20 16:23:56 +08:00
Łukasz Magiera
ff4b0c8f5d
Merge pull request from filecoin-project/fix/nil-newts-fpost-panic
fpost: Don't panic with nil new tipset
2020-01-19 21:17:51 +01:00
Łukasz Magiera
be2d138aac fpost: Don't panic with nil new tipset 2020-01-18 16:00:22 +01:00
Łukasz Magiera
1e782b3de2 Merge branch 'master' into feat/event-states 2020-01-18 14:40:59 +01:00
whyrusleeping
0a45e97699 implement state call command for easy method calling 2020-01-17 18:27:29 -08:00
whyrusleeping
2a7dcb6e17 fixup bitfield serialization in manual faulty sector handler 2020-01-17 00:13:00 -08:00
Łukasz Magiera
0aebec6697 Merge branch 'master' into feat/event-states 2020-01-16 03:59:14 +01:00
Łukasz Magiera
d4c50b8ea0 sealing: gofmt 2020-01-16 03:54:57 +01:00
Łukasz Magiera
cc424d64fe sealing: Handle events based on current state 2020-01-16 03:53:59 +01:00
Łukasz Magiera
ffdd436b52 sealing: Handle seed changes more correctly 2020-01-16 02:25:49 +01:00
Łukasz Magiera
e9ca6c0871 sealing: More cleanup 2020-01-15 21:53:14 +01:00
Łukasz Magiera
604bf64ef6 Move miner sealing logic into a separate package 2020-01-15 21:49:11 +01:00
Łukasz Magiera
925e057acb Some sector fsm tests 2020-01-15 21:04:02 +01:00
Łukasz Magiera
23472aa4f2 statemachine: don't force users to see reflect 2020-01-15 01:08:20 +01:00
Łukasz Magiera
8a9707788c Merge branch 'master' into feat/event-states 2020-01-14 16:31:10 +01:00
Łukasz Magiera
94a21e3aeb Update deps 2020-01-14 03:05:46 +01:00
Łukasz Magiera
2ca0c152ba Deal tests on mock sectorbuilder 2020-01-14 03:05:46 +01:00
Łukasz Magiera
a2bcc1fec2 Mostly functional mock sectorbuilder 2020-01-14 03:05:35 +01:00
Łukasz Magiera
2ef8c1ae0c statemachine: Better naming 2020-01-13 18:44:59 +01:00
Łukasz Magiera
2fea9433d8 sectors: Correctly set CommitMessage 2020-01-13 18:18:52 +01:00
Łukasz Magiera
e06c19b51c update deps 2020-01-13 17:39:54 +01:00
Łukasz Magiera
34abc3d023 Merge branch 'feat/testnet2' into feat/event-states 2020-01-13 15:52:43 +01:00
Łukasz Magiera
f6d41ee77d Port sector state maching to evtsm 2020-01-10 20:24:15 +01:00
Łukasz Magiera
469136ee86
Merge pull request from filecoin-project/feat/fork-signaling
Add fork signaling to blockheader
2020-01-10 13:33:29 +01:00
Łukasz Magiera
5f460b380d
Merge pull request from filecoin-project/feat/mock-sectorbuilder
WIP: implement mock sectorbuilder for easier storage miner testing
2020-01-10 01:33:43 +01:00
whyrusleeping
6c9289df0a don't require signatures for self deals 2020-01-09 14:18:06 -08:00
Łukasz Magiera
da2a11ed76 Event-based state machine 2020-01-09 22:23:31 +01:00
whyrusleeping
7534ac1e1a add ability to control seal operations completion time 2020-01-09 12:50:45 -08:00
whyrusleeping
bda85e4097 more features, better locking 2020-01-09 11:23:42 -08:00
whyrusleeping
5e16aab51b fix tests, expand interfaces a little 2020-01-09 10:28:14 -08:00
whyrusleeping
cc56389dbb attempt to fix the DI loading, still not working 2020-01-08 20:51:18 -08:00
whyrusleeping
60969c69a7 implement a mocked out sectorbuilder, probably not useful yet 2020-01-08 20:27:14 -08:00
whyrusleeping
d08898ef37 make storage miner acccept an interface to the sectorbuilder 2020-01-08 17:10:17 -08:00
Jakub Sztandera
8cfb4aafcd
Add fork signaling to blockheader
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:31:08 +01:00
Jakub Sztandera
1ed62628a7
Update go-log to v2
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:22:31 +01:00
laser
20321bb476 handle rebase 2020-01-07 08:18:35 -08:00
acruikshank
7873213b11 switch sectorbuilder from lib to external module
point to correct version of sectorbuilder + ensure go-datastore stays at 0.1.1

replace local address type with go-address

consume paramfetch.GetParams instead of local paramfetch.go

remove constants now defined in sectorbuilder
2020-01-07 07:48:43 -08:00
Łukasz Magiera
1e7f10e387 gofmt 2020-01-07 15:00:10 +01:00
hannahhoward
718acb0a86 refactor(cborutil): extract cborutil library
Use extracted cborutil library
2020-01-07 14:59:35 +01:00
hannahhoward
8418464d91 refactor(address): use extracted address library
Switch to using extracted address library
2020-01-07 14:53:27 +01:00
Łukasz Magiera
2e95a53679 Wire up faults in fPoSt 2019-12-17 23:33:08 +01:00
whyrusleeping
28a325efe6 soft fail messages that cause precomitted sectors map to become too large 2019-12-13 00:00:20 +01:00
Whyrusleeping
9eb1371896
Merge pull request from filecoin-project/fix/soft-sectors-load-err
sectors: Handle sector state reload errors more gracefully
2019-12-11 15:48:45 +01:00
Łukasz Magiera
c579b2c56f
Merge pull request from filecoin-project/fix/retrieval-unsealed
Don't allow retrieval of unsealed data
2019-12-11 15:28:00 +01:00
Łukasz Magiera
613bb25297 sectors: Handle sector state reload errors more gracefully 2019-12-11 15:17:38 +01:00
Łukasz Magiera
9246788bff Don't allow retrieval of unsealed data 2019-12-10 00:19:46 +01:00
Łukasz Magiera
d750723405 Merge remote-tracking branch 'origin/testnet-staging' into feat/cleanup-sectorstate-structs 2019-12-09 21:27:20 +01:00
Łukasz Magiera
2f2a5824b3
storage: Use fatal instead of .to.error 2019-12-09 21:19:46 +01:00
whyrusleeping
d2448912a6 Implement basic fault handling
WIP: miner actor tests

working miner actor test

rebase and fix tests
2019-12-09 20:53:43 +01:00
Łukasz Magiera
ff963dfea5
Merge pull request from filecoin-project/feat/storage-deal-cleanup
Clean up storage deal structures
2019-12-09 18:31:18 +01:00
Łukasz Magiera
94df2c656e Use map encoders for some structs 2019-12-09 17:40:15 +01:00
Łukasz Magiera
1d7dd4711d Cleanup sectorinfo struct 2019-12-09 16:42:47 +01:00
Łukasz Magiera
0b4db5530c
Merge pull request from filecoin-project/fix/retrieve-large
Fix retrieval of large files
2019-12-09 16:28:47 +01:00
Łukasz Magiera
e3d1f5b362 storage: Better errors when starting sectorStateLoop 2019-12-09 16:11:54 +01:00
Łukasz Magiera
570d725ce5 type-gen 2019-12-09 15:41:44 +01:00
Łukasz Magiera
f29e1eacff storage: set sector state correctly 2019-12-09 14:32:58 +01:00
whyrusleeping
eba3f4af86 address PR review 2019-12-08 23:39:08 +01:00
Łukasz Magiera
0efc625405 type-gen 2019-12-08 23:35:46 +01:00
Łukasz Magiera
ef0d45e406 storage: addres review around nonces in sector states 2019-12-08 23:35:46 +01:00
Łukasz Magiera
1766987be5 Fancy math to update nonces on precommit revert 2019-12-08 23:35:46 +01:00
Łukasz Magiera
55c1432347 Handle reverts in precommit more correctly 2019-12-08 23:35:46 +01:00
Łukasz Magiera
6d0e5997dd
Merge pull request from filecoin-project/feat/pledge-capacity
Rename pledge-sector
2019-12-08 23:22:48 +01:00
Jakub Sztandera
7f537dce2d
More renames
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-08 22:48:20 +01:00
Łukasz Magiera
716b4cc652 Restart fPoSt after fail 2019-12-08 20:48:17 +01:00
whyrusleeping
98f252da65 better error message for sector failure 2019-12-07 21:36:15 +01:00
whyrusleeping
f0653decf3 Clean up storage deal structures 2019-12-07 21:29:37 +01:00
Łukasz Magiera
83924e6b97
sectorbuilder: Allow to restrict task types
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-07 19:05:15 +01:00
Łukasz Magiera
77b5490e57
Merge pull request from filecoin-project/feat/more-sector-states
Improving storage miner sector states
2019-12-06 12:17:19 +01:00
whyrusleeping
14c7e1f0ac Fix copying partial ticket arrays into slices 2019-12-06 00:32:57 -08:00
whyrusleeping
d92ba5fbba Add separate state for waiting for commit message 2019-12-05 18:40:57 -08:00
Łukasz Magiera
3673a9110f
Merge pull request from filecoin-project/feat/static-check-2
more of that sweet staticcheck goodness
2019-12-05 12:21:43 +01:00
whyrusleeping
40be53c9f9 add command for power users to manually update sector state 2019-12-04 20:43:54 -08:00
whyrusleeping
613a0a05a0 more of that sweet staticcheck goodness 2019-12-04 18:04:09 -08:00
whyrusleeping
1b475ba6fe some fixes from staticcheck in ./chain 2019-12-04 17:18:30 -08:00
whyrusleeping
5133c357dc Better error checks 2019-12-04 14:14:50 -08:00
Łukasz Magiera
2619567dfd
Merge pull request from filecoin-project/fix/sectorstore-fails
Miner: improve sector state handling
2019-12-04 02:10:30 +01:00
Łukasz Magiera
d112fd18b1 Persist sector sealing errors 2019-12-04 01:44:29 +01:00
Łukasz Magiera
13c39452c1 storageminer: States for common failure modes 2019-12-04 00:59:40 +01:00
Łukasz Magiera
dbcb839b6f Make sector state transitions more explicit 2019-12-04 00:42:22 +01:00
Łukasz Magiera
21629d005b storageminer: separate sectorstore types 2019-12-03 23:57:47 +01:00
Łukasz Magiera
90dfb18ab8
Merge pull request from filecoin-project/fix/clean-fpost-exit
storageminer: exit fpostScheduler loop cleanly
2019-12-03 23:46:44 +01:00
Łukasz Magiera
a7738af169 storageminer: exit fpostScheduler loop cleanly 2019-12-03 23:39:45 +01:00
Łukasz Magiera
a65320c970 storageminer: Don't start fPoSt with every head change 2019-12-03 23:00:04 +01:00
whyrusleeping
4500a36ec6 some useful log messages 2019-12-02 16:08:08 -08:00
Łukasz Magiera
d5b94884c8 Buch of lint fixes 2019-12-02 13:51:16 +01:00
Łukasz Magiera
c7cf20843e actually set unsealed path in sectorbuilder 2019-12-01 22:22:39 +01:00
Łukasz Magiera
fd8f65248b deals: Correctly set deal ID in provider states 2019-12-01 21:07:57 +01:00
Łukasz Magiera
a59d0f0f8c Fix unsealing, sector based data refs 2019-12-01 18:58:31 +01:00
whyrusleeping
59d9c681ad Rework miner test setups to fix tests 2019-11-30 17:21:11 -06:00
Łukasz Magiera
ad978949db import sector/deal into state machines on init 2019-11-29 20:11:42 +01:00
Łukasz Magiera
367512d067 Some fpost fixes 2019-11-28 19:08:10 +01:00
Łukasz Magiera
920fd3ba9b storageminer: New fallback post scheduler 2019-11-28 18:44:57 +01:00
whyrusleeping
45288b8810 WIP: uncomment out windowed post code, try to make it work 2019-11-27 21:36:34 -06:00
Łukasz Magiera
d2412f4f00 Fix pond 2019-11-27 15:18:51 +01:00
Łukasz Magiera
23e0008b81 Merge remote-tracking branch 'origin/master' into feat/election-post 2019-11-27 13:31:44 +01:00
laser
3623925f66 replace go-bls-sigs and go-sectorbuilder with filecoin-ffi
- remove old submodules and add new submodule
- update build tooling to consume new unified static library
- update consumers of old libs to use new package
2019-11-26 18:16:37 -08:00
Łukasz Magiera
871c30d2a0 WIP trying to get election post to compute 2019-11-26 16:15:41 +01:00
whyrusleeping
a0588d513d Add lotus-gen, rewire genesis mining 2019-11-24 22:45:13 -06:00
Łukasz Magiera
a755ca5dae stmgr: check for duplicate nonces when looking back for msg 2019-11-24 20:16:18 +01:00
Łukasz Magiera
c7b34153fb more mpooland post sched fixes 2019-11-24 17:35:50 +01:00
Łukasz Magiera
2deae35dcc
Merge pull request from filecoin-project/feat/no-pkg-error
Swtich to xerrors
2019-11-22 12:40:59 -06:00
Jakub Sztandera
36b7c5a32a
Swtich to xerrors
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-11-22 10:20:56 -06:00
Łukasz Magiera
a33462b01a Make ppe == 0 error less scary 2019-11-22 17:15:33 +01:00
whyrusleeping
96482f456a WIP: election post restructuring 2019-11-21 16:21:45 -06:00
whyrusleeping
0e816932e3 log more information when message execution fails 2019-11-20 22:18:04 -06:00
Łukasz Magiera
174f8f3a7e sectors: Simple state restore 2019-11-20 20:58:09 +01:00
Łukasz Magiera
37792b149f sync: fix some issues 2019-11-20 20:49:29 +01:00
Łukasz Magiera
0a6a0e5e39 Address review 2019-11-20 18:00:05 +01:00
Łukasz Magiera
ddf2e05dd0 post: More correct 'wait' logic 2019-11-20 18:00:05 +01:00
whyrusleeping
f8eabd3db4 Refactor mining logic to match specs ticket chain 2019-11-20 10:52:36 -06:00
Łukasz Magiera
4f9947a27b
Merge pull request from filecoin-project/anorth/tskey
Add TipSetKey as a concatenation of block CIDs, and use if for ChainGetTipSet, ChainGetRandomness.
2019-11-13 22:07:24 +01:00
Łukasz Magiera
8f0e3ede24 Bunch of small fixes / improvements, gofmt 2019-11-13 15:48:57 +01:00
anorth
84a4766d8f Add TipSetKey as a concatenation of block CIDs, and use if for ChainGetTipSet, ChainGetRandomness. 2019-11-13 15:23:37 +11:00
Łukasz Magiera
56d902c1bd storageminer: Throttle GeneratePieceCommitment in storeGarbage 2019-11-12 19:05:03 +01:00
Łukasz Magiera
55697209a7
Merge pull request from filecoin-project/fix/post-restart
Fix PoSt after storage miner restart
2019-11-11 18:33:33 +01:00
Łukasz Magiera
b8f46bff26 storageminer: Drop sortedSectorInfo log 2019-11-11 18:32:46 +01:00
whyrusleeping
bda6d7e119 Don't try syncing to a chain if its not clearly heavier 2019-11-09 12:14:40 -08:00
Łukasz Magiera
81bd0bc517 Test to reproduce post error after restart 2019-11-09 11:42:36 +01:00
Łukasz Magiera
3b6c079014 Some final parameter setting for devent 7 2019-11-09 01:18:15 +01:00
Łukasz Magiera
ed64e52e3a sectorbuilder: apply some review suggestions 2019-11-09 00:21:04 +01:00
Łukasz Magiera
aaacfd15cd Add comments to fillersFromRem 2019-11-09 00:06:07 +01:00
Łukasz Magiera
8c39486736 some import fixes 2019-11-08 21:11:56 +01:00
Łukasz Magiera
071f05fa95 Storage miner API improvements 2019-11-08 19:23:58 +01:00
Łukasz Magiera
34846c538e sectorbuilder: Use StandaloneWriteWithAlignment 2019-11-07 21:43:57 +01:00
Łukasz Magiera
f6a49ab9f9 Wip fixing a thing 2019-11-07 20:54:24 +01:00
Łukasz Magiera
f40eb8a521 More post-refactor fixes 2019-11-07 19:43:23 +01:00
Łukasz Magiera
68adf519c7 cbor-gen 2019-11-07 19:27:23 +01:00
Łukasz Magiera
2d26a4edf7 Sector storage refactor 2019-11-07 19:22:59 +01:00
Łukasz Magiera
1549269773 Make multiple deals per almost work 2019-11-07 15:45:53 +01:00
Łukasz Magiera
3fbe0abb75 cborrpc -> cborutil 2019-11-07 15:11:39 +01:00
Łukasz Magiera
be0d07e143 pass deal IDs in SectorPreCommitInfo 2019-11-07 13:03:18 +01:00
whyrusleeping
54722a0d38 WIP: fixing the tests by waiting for chain activity 2019-11-06 23:57:10 -08:00
Łukasz Magiera
cfac6f3272 Fix TestSectorStore 2019-11-07 00:50:37 +01:00
Łukasz Magiera
74ea4d71c5 post-rebase fixes 2019-11-07 00:47:48 +01:00
whyrusleeping
41d13b12a0 test is getting close to passing 2019-11-07 00:39:34 +01:00
whyrusleeping
9863942fe2 WIP: trying to write a test to reproduce the storage deal error 2019-11-07 00:39:34 +01:00
whyrusleeping
e075eb163f Add a test for the sectorstore 2019-11-07 00:39:34 +01:00
Łukasz Magiera
f7651f180b Create filler deals 2019-11-07 00:09:48 +01:00
Łukasz Magiera
37721b2a22 Get some deals to work 2019-11-06 20:00:04 +01:00
Łukasz Magiera
68c2d4f58a deals: Set correct Refs 2019-11-06 18:38:42 +01:00
Łukasz Magiera
40b1f91843 fix sector block ref serialization 2019-11-06 13:22:08 +01:00
Łukasz Magiera
cb3965bcf5 Improve errors around deal handling 2019-11-06 13:04:33 +01:00
Łukasz Magiera
1fc0971db0 sectorstore: test computePaddedSize 2019-11-06 13:03:36 +01:00
whyrusleeping
87616e1c03 use better math 2019-11-05 15:57:46 -08:00
whyrusleeping
ba1e8c6668 add proper piece padding in AddPiece 2019-11-05 13:30:44 -08:00
Łukasz Magiera
bcf9778b18 Update cbor-gen post-merge 2019-11-05 19:02:28 +01:00
Łukasz Magiera
daabe470a4 storageminer: tracing in computePost 2019-11-05 15:52:40 +01:00
Łukasz Magiera
eb28c45c00 events: Plumb context to callbacks 2019-11-05 15:03:59 +01:00
Łukasz Magiera
2a695611d5 Update sectorbuilder 2019-11-03 21:44:22 +01:00
Łukasz Magiera
c550e030b1 INTERACTIVE PROEP IS ALIVE 2019-11-02 16:07:26 +01:00
Łukasz Magiera
be8fe54ea3 Set correct Provider addr in StoreGarbage 2019-11-02 15:13:37 +01:00
Łukasz Magiera
3bc4a5dddf More progress on fixing storing garbage data 2019-11-02 00:43:54 +01:00
Łukasz Magiera
1dcebece71 Some smaller fixes 2019-11-01 23:44:55 +01:00
whyrusleeping
2fe1b5a538 some cleanup 2019-11-01 15:05:05 -07:00
Łukasz Magiera
82344649d3 Use state store for sectors 2019-11-01 15:06:49 +01:00
Łukasz Magiera
1583cf2593 Strip unused functionality from sectorstore 2019-11-01 11:05:48 +01:00
whyrusleeping
fc9091cc89 Get interactive porep sector sealing mostly working 2019-10-31 20:57:10 -07:00
whyrusleeping
34998d9362 add file i forgot 2019-10-31 11:46:53 -07:00
whyrusleeping
4e478330a4 more refactoring for interactive porep scheduling 2019-10-31 09:55:35 -07:00
whyrusleeping
759094198c Some more progress on interactive porep 2019-10-30 18:22:50 -07:00
Łukasz Magiera
7fa3e2db7a WIP Interactive PoRep 2019-10-30 19:10:29 +01:00
Łukasz Magiera
46d782b30b storageminer: Drop commitment tracker 2019-10-30 18:37:38 +01:00
Łukasz Magiera
a4c0610cc3 actors: Rename some storageminer stuff to match the spec 2019-10-30 16:55:55 +01:00
whyrusleeping
3e5654d575 initial actor changes for interactive porep 2019-10-30 16:05:31 +01:00
Łukasz Magiera
5a5c66600f storageminer: More correct listing of sealed sectors 2019-10-30 10:57:00 +01:00
Łukasz Magiera
922d8a90a5 storageminer: Restart sealing on restart 2019-10-29 23:19:58 +01:00
Łukasz Magiera
6c25c5384d commitment tracker: prefer new commitments 2019-10-29 23:04:08 +01:00
Łukasz Magiera
b7a78670ab storageminer: Call beginPosting before trying to commit a sector 2019-10-29 22:58:41 +01:00
Łukasz Magiera
3047c9d4c2 storageminer: Check on-chain commitments 2019-10-29 22:41:28 +01:00
Łukasz Magiera
44f4ee0de1 storageminer: cmd to list sector commitments 2019-10-29 21:41:30 +01:00
Łukasz Magiera
169c285fb3 storageminer: Handle uncommited sectors on start 2019-10-29 20:46:32 +01:00
Łukasz Magiera
237f8e8018 Set proving params matching the spec more closely 2019-10-28 17:51:45 +01:00
Łukasz Magiera
117ec636c5 paramfetch: check checksums in parallel 2019-10-27 10:18:27 +01:00
Łukasz Magiera
874be79958 very basic sector seal scheduling 2019-10-27 09:56:53 +01:00
whyrusleeping
874ab1456c WIP: updating to new proofs code 2019-10-25 18:07:34 +02:00
whyrusleeping
4890f577f5 give miners the ability to select different sector sizes 2019-10-25 17:26:15 +02:00
Łukasz Magiera
76f1e6e207 on chain deals: Put dealIDs in CommitSector messages 2019-10-24 16:24:31 +02:00
Łukasz Magiera
64bfb38834 on chain deals: Get things to actually run! 2019-10-23 12:44:00 +02:00
Jakub Sztandera
1bf713cb0a
Cleanup imports after rename
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-18 13:47:41 +09:00
Jakub Sztandera
0cf45801f3
Increase gas limits
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-18 13:08:36 +09:00
Jakub Sztandera
2ed308829a
Change PoSt confidence to 3
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-17 09:42:32 +09:00
Jakub Sztandera
a38adb3b44
Skip over the bug
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-16 21:13:34 +09:00
Jakub Sztandera
e11deb9551
Add bughunt Error log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-13 01:18:32 +02:00
Jakub Sztandera
49e3fe39f8
Structure logs
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-11 02:00:04 +02:00
Jakub Sztandera
79ee959a90
Use log.Warn
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-10-09 22:23:04 +02:00
whyrusleeping
14c4a8bee6 implement looking in the past for messages 2019-10-09 09:33:18 +02:00
whyrusleeping
f9ac86ab00 the doneset is for dropping sectors 2019-10-04 12:30:30 -06:00
Łukasz Magiera
6ffa4b164e storage: set ppe in beginPosting 2019-10-01 00:56:23 +02:00
Łukasz Magiera
a3081f1e12 storageminer: fix call to ProvingPeriodEnd in beginPosting 2019-09-28 01:00:07 +02:00
Łukasz Magiera
e0632a93ab Share ProvingPeriodEnd math 2019-09-27 21:47:47 +02:00
Łukasz Magiera
ba0559ae58 events: Fix tipSetCache at higher heights 2019-09-27 14:35:09 +02:00
Łukasz Magiera
fe8e1fe1e4 Almost working new post code 2019-09-27 01:07:40 +02:00
Łukasz Magiera
1024812559 storageminer: Update post scheduling 2019-09-26 23:32:47 +02:00
Łukasz Magiera
a167b9f5db storageminer: Set Value when posting post 2019-09-26 23:32:47 +02:00
Łukasz Magiera
ebb744f42f storageminer: retry PoSt 2019-09-26 23:32:47 +02:00
Łukasz Magiera
9a990dbd8a use new sectorbuilder file interfaces 2019-09-23 12:50:28 +02:00
Łukasz Magiera
73d1182121 post: More review addressing 2019-09-19 22:34:23 +02:00
Łukasz Magiera
fdb45aeeaa pond: +20k, +200k buttons 2019-09-19 22:17:26 +02:00
Łukasz Magiera
6c48ab3be4 Compute post more than once 2019-09-19 20:38:58 +02:00
Łukasz Magiera
839597012d Fix post seed height math 2019-09-19 19:38:56 +02:00
Łukasz Magiera
c40f89f2a9 Almost working PoSt submission 2019-09-19 18:17:54 +02:00
Łukasz Magiera
d55e872135 Merge remote-tracking branch 'origin/master' into feat/post-schedule 2019-09-19 16:42:50 +02:00
whyrusleeping
9a4ffe0a3f fix error handling in track commit sector msg 2019-09-18 22:11:19 -07:00
Łukasz Magiera
689b1e5b3c events: Safer restarting in listenHeadChanges 2019-09-18 20:07:39 +02:00
Łukasz Magiera
cc82cc9675 miner: Initial event system integration 2019-09-18 15:32:21 +02:00
Łukasz Magiera
76ce3d9bb2 events: Call ChainGetBlockMessages with the right CID 2019-09-18 13:45:52 +02:00
Łukasz Magiera
fe7efa753d events: Build on chain API calls 2019-09-18 13:01:52 +02:00
whyrusleeping
3e45088812 cleaning up a few types 2019-09-17 20:32:52 -07:00
whyrusleeping
b1e77748e7 scaffolding and refactoring to make post scheduler work 2019-09-17 19:50:03 -07:00
whyrusleeping
cc877f5c04 WIP: expanding API to serve post producer needs 2019-09-17 17:08:49 -07:00
whyrusleeping
11040f105b WIP: post scheduling 2019-09-17 15:51:46 -07:00
Łukasz Magiera
b9ab33f0ae storageminer: use switch in TrackCommitSectorMsg 2019-09-17 10:40:13 +02:00
Łukasz Magiera
f8a2ecaf52 gofmt; mod tidy 2019-09-16 23:26:19 +02:00
Łukasz Magiera
133ab265d4 storageminer: Call TrackCommitSectorMsg with the right address 2019-09-16 20:08:24 +02:00
Łukasz Magiera
55014becc8 commitment tracker: Fix deadlock 2019-09-16 20:08:24 +02:00
Łukasz Magiera
eda72468ce Sector Commitment tracker 2019-09-16 20:08:24 +02:00
Łukasz Magiera
874b33781d storageminer: fix getWorkerAddr 2019-09-13 19:59:10 +02:00
whyrusleeping
ed45d1c2b4 refactor state utilities into StateManager package, implement proper election proofs 2019-09-09 19:14:00 -07:00
Łukasz Magiera
bc0bb913f1 storageminer: Fix CommitSector 2019-09-07 00:42:01 +02:00
Łukasz Magiera
711c61cba1 Merge remote-tracking branch 'origin/master' into feat/retrieval 2019-08-29 20:59:03 +02:00
Łukasz Magiera
83748b40b6 retrieval: address some review comments 2019-08-29 20:55:20 +02:00
Łukasz Magiera
d7d42416d7 retrieval: Almost working chunked files 2019-08-29 17:50:19 +02:00
Łukasz Magiera
a5d3122ba5 Make deals work with chunked files 2019-08-29 17:50:19 +02:00
Łukasz Magiera
433550e9a4 gofmt 2019-08-29 17:50:19 +02:00
Łukasz Magiera
6714240653 SectorSize const 2019-08-29 17:50:19 +02:00
Łukasz Magiera
be30bc79a5 Retrieval works! 2019-08-29 17:50:19 +02:00
Łukasz Magiera
28d3eb38eb basic retrieval content discovery 2019-08-29 17:48:19 +02:00
Łukasz Magiera
cad3efb9ba Command to list sealed blocks 2019-08-29 17:48:19 +02:00
Łukasz Magiera
e0dc17bc1a Correctly construct sealbstore 2019-08-29 17:48:19 +02:00
Łukasz Magiera
9691211df1 Sealed blockstore 2019-08-29 17:48:19 +02:00
whyrusleeping
2013db604a switch to using AMT for sector set 2019-08-28 17:01:46 -07:00
Łukasz Magiera
baf4fbe309 sectorstore: Address review feetback 2019-08-15 00:46:15 +02:00
Łukasz Magiera
e6493afd46 move poller to sector store 2019-08-14 23:34:43 +02:00
Łukasz Magiera
399f91940b Sector store 2019-08-14 22:27:32 +02:00
Łukasz Magiera
5ab1b1caaf deals: Sealing handler 2019-08-12 23:48:28 +02:00
whyrusleeping
28ef081d0e use wallet has instead of list 2019-08-08 10:56:14 -07:00
whyrusleeping
959fe371b0 move storage miner keys to full node 2019-08-08 10:56:14 -07:00
whyrusleeping
95e7b230ec cleanup from self review 2019-08-07 18:23:44 -07:00
whyrusleeping
ba0d648546 sealing garbage data into a sector that gets accepted by the chain works! 2019-08-07 18:16:58 -07:00
whyrusleeping
5c32725510 sector commitments make it all the way to the chain 2019-08-07 16:22:35 -07:00
whyrusleeping
b7c6333fef implement sector commitments and proof verification 2019-08-06 23:35:57 -07:00
Łukasz Magiera
d8bd222cb1 Optional tipset param for ChainCall 2019-07-31 18:54:28 +02:00
whyrusleeping
ee224e5b21 Implement 'storage miner' module, wire up a few bits it needs to start 2019-07-29 17:46:56 -07:00
whyrusleeping
2576853426 begin work on storage miner processes 2019-07-29 14:01:31 -07:00