Aayush Rajasekaran
ba27d45214
Split the getSectorCutoff methods between precommit and commit batchers
2021-06-08 16:46:35 -04:00
Łukasz Magiera
78171055e7
fee config for sector batching
2021-06-08 15:45:32 +02:00
Anton Evangelatov
30ba9a2751
remove double http handler decl
2021-06-08 15:36:54 +02:00
Anton Evangelatov
4dd093b160
fix piece provider test
2021-06-08 14:57:45 +02:00
Anton Evangelatov
9f3ec82cc1
initial resolution of conflicts
2021-06-08 13:01:22 +02:00
aarshkshah1992
4e9bb16532
changes as per review
2021-06-08 10:09:27 +05:30
aarshkshah1992
212d0fc264
fix remote store diff
2021-06-08 09:54:45 +05:30
aarshkshah1992
e4e60d7af4
fix mock
2021-06-08 09:37:12 +05:30
aarshkshah1992
16166504f4
merged master
2021-06-08 09:17:40 +05:30
Aayush Rajasekaran
b13169f071
Rename deadlines to cutoffs in the batchers
2021-06-07 20:20:06 -04:00
aarshkshah1992
ddd9bf610e
fix CI
2021-06-07 16:45:52 +05:30
aarshkshah1992
1a90d3bbb5
update ffi
2021-06-07 16:12:39 +05:30
Anton Evangelatov
b0cb0c1a4a
do not depend on filecoin-ffi in api package
2021-06-07 11:51:25 +02:00
aarshkshah1992
8d9cef17af
changes as per review
2021-06-07 15:03:09 +05:30
aarshkshah1992
2a134887c3
logs to debug read & unseal
2021-06-07 15:03:09 +05:30
aarshkshah1992
ec6a49693f
logs to debug read and unseal
2021-06-07 15:03:09 +05:30
aarshkshah1992
207f0d901a
integration test should remove unsealed files
2021-06-07 15:03:09 +05:30
aarshkshah1992
73f7825fbb
clean up logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
3b792a32c3
better logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
22f36483cb
more logging
2021-06-07 15:03:09 +05:30
aarshkshah1992
acfa3d7370
finish integration tests
2021-06-07 15:03:09 +05:30
aarshkshah1992
ad4b182bfe
remove read task type and run gen and docsgen
2021-06-07 15:03:06 +05:30
aarshkshah1992
35a0dbfa8c
fix go mod
2021-06-07 15:02:52 +05:30
aarshkshah1992
4efaa8d688
unit tests for the remote store Reader
2021-06-07 15:02:52 +05:30
aarshkshah1992
0d88800eb9
use mockgen
2021-06-07 15:02:52 +05:30
aarshkshah1992
77b5e8d045
use an actual worker in the integration tests
2021-06-07 15:02:52 +05:30
Dirk McCormick
6879ae9e6a
feat: TestPieceProviderReadPiece
2021-06-07 15:02:52 +05:30
aarshkshah1992
74372d3e81
fix linting problems
2021-06-07 15:02:52 +05:30
aarshkshah1992
759d8f090b
test http handler
2021-06-07 15:02:52 +05:30
aarshkshah1992
db5c88196d
address review comments
2021-06-07 15:02:52 +05:30
Aarsh Shah
c853350bdf
Apply suggestions from code review
...
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2021-06-07 15:02:52 +05:30
aarshkshah1992
65eb610ec3
docs, logs and green ci
2021-06-07 15:02:49 +05:30
aarshkshah1992
670835fca0
bypass task scheduler for reading unsealed pieces
2021-06-07 15:02:04 +05:30
Anton Evangelatov
d9a7348ae1
use masters filecoin-ffi
2021-06-04 16:20:40 +02:00
Anton Evangelatov
8bd09e39ca
resolve merge conflicts
2021-06-04 16:17:00 +02:00
Anton Evangelatov
88756f3ebf
fix TestAPIDeal tests
2021-06-04 15:41:38 +02:00
aarshkshah1992
05ba3de5cc
changes as per review
2021-06-04 10:27:43 +05:30
Łukasz Magiera
565bb4f589
mock: Log debug info on bad aggregates
2021-06-03 17:35:10 +02:00
aarshkshah1992
43453bb633
fix broken test
2021-06-03 10:15:23 +05:30
Łukasz Magiera
40cc29d723
Skip FD check in TestDownloadParams
2021-06-02 19:50:17 +02:00
Łukasz Magiera
d77d3df698
Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11
2021-06-02 15:06:50 +02:00
Łukasz Magiera
f6963523f8
Use filecoin-ffi master
2021-06-02 11:13:24 +02:00
Łukasz Magiera
dc642d0b7b
Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11
2021-06-01 21:06:58 +02:00
Łukasz Magiera
8b84e499c5
storagefsm: Handle preCommitParams errors better
2021-06-01 20:36:20 +02:00
Łukasz Magiera
482e1110c2
precommit batcher: Improve error propagation
2021-06-01 14:35:30 +02:00
Łukasz Magiera
cb4eb487f4
commit batcher: Fix min aggregate size check
2021-06-01 12:27:22 +02:00
Łukasz Magiera
084b0e7f60
Handle collateral when submitting aggregated commits
2021-06-01 12:09:15 +02:00
Łukasz Magiera
9fcb564bef
Make commit batcher more robust
2021-06-01 11:56:19 +02:00
Anton Evangelatov
0a79f49dc5
fix tests
2021-06-01 11:45:34 +02:00
Łukasz Magiera
ffa47659a1
Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11
2021-05-31 21:38:34 +02:00
Łukasz Magiera
c3e8eddb9b
Merge remote-tracking branch 'origin/master' into feat/nv13-1.11
2021-05-31 21:24:56 +02:00
Aayush Rajasekaran
61554cf3e0
Update to latest actors
2021-05-31 14:47:41 -04:00
Anton Evangelatov
d195a12565
Merge branch 'master' into nonsense/split-market-miner-processes
2021-05-31 11:26:11 +02:00
Łukasz Magiera
f3bf773152
storagefsm: Fix too-long log handling
2021-05-30 19:24:42 +02:00
Łukasz Magiera
6e1919c67f
storagefsm: Fix race spawning more than one new sector at once
2021-05-30 18:30:38 +02:00
Łukasz Magiera
9475079b97
Make batch deal input test less flaky
2021-05-30 17:46:22 +02:00
Łukasz Magiera
c8d603557b
storagefsm: Fix batch deal packing behavior
2021-05-30 17:46:22 +02:00
Łukasz Magiera
1e4456138e
Merge master into feat/nv13
2021-05-27 12:28:20 +02:00
yaohcn
d04e7d98ce
Get current seal proof when necessary
2021-05-27 11:53:33 +08:00
Łukasz Magiera
fc76a09b19
mock: Use real aggregate lengths
2021-05-26 10:06:41 +02:00
Łukasz Magiera
7bd0fcbb24
sealing: Don't start batch timers with empty batches
2021-05-25 16:47:42 +02:00
Łukasz Magiera
04658e1cae
fix lint
2021-05-25 16:26:59 +02:00
Łukasz Magiera
f5409845b5
Some review addressing
2021-05-25 16:07:45 +02:00
aarshkshah1992
0f3ff9e06a
fix typo
2021-05-24 15:22:11 +05:30
aarshkshah1992
dc6dbc9a11
dpr changes and test based on new unsealing PR
2021-05-22 22:40:21 +05:30
Łukasz Magiera
58203559e7
Merge pull request #6304 from yaohcn/fix-ticket
...
fix ticket expired
2021-05-21 16:07:28 +02:00
aarshkshah1992
50e023edd3
changes as per review
2021-05-21 19:15:05 +05:30
aarshkshah1992
2c9f5922b5
logs to debug read & unseal
2021-05-21 19:02:37 +05:30
aarshkshah1992
8ff5bce7a3
logs to debug read and unseal
2021-05-21 19:01:17 +05:30
aarshkshah1992
fb29f782df
integration test should remove unsealed files
2021-05-21 16:15:08 +05:30
aarshkshah1992
536d7c4275
clean up logging
2021-05-21 15:50:25 +05:30
aarshkshah1992
40642b2cad
better logging
2021-05-21 15:16:20 +05:30
aarshkshah1992
85f2ac8789
more logging
2021-05-21 14:56:37 +05:30
aarshkshah1992
78a0458ada
finish integration tests
2021-05-21 11:00:17 +05:30
yaohcn
00a1e2c05b
test ticket expired
2021-05-21 11:45:57 +08:00
aarshkshah1992
c17300dc1f
remove read task type and run gen and docsgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
50d7acfa0c
fix go mod
2021-05-20 15:25:46 -06:00
aarshkshah1992
bd9959070e
unit tests for the remote store Reader
2021-05-20 15:25:46 -06:00
aarshkshah1992
9b34494501
use mockgen
2021-05-20 15:25:46 -06:00
aarshkshah1992
31a5f68dfa
use an actual worker in the integration tests
2021-05-20 15:25:46 -06:00
Dirk McCormick
ec85a973b1
feat: TestPieceProviderReadPiece
2021-05-20 15:25:46 -06:00
aarshkshah1992
8c4c26ca4c
fix linting problems
2021-05-20 15:25:46 -06:00
aarshkshah1992
dd9c9fd4fd
test http handler
2021-05-20 15:25:46 -06:00
aarshkshah1992
c58048d16a
address review comments
2021-05-20 15:25:46 -06:00
Aarsh Shah
d33d426692
Apply suggestions from code review
...
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2021-05-20 15:25:46 -06:00
aarshkshah1992
73613ee883
docs, logs and green ci
2021-05-20 15:25:46 -06:00
aarshkshah1992
2a40c802ea
bypass task scheduler for reading unsealed pieces
2021-05-20 15:25:46 -06:00
Łukasz Magiera
2d4eaf08c4
Merge pull request #6233 from nonsense/fix-finalizesector-storage-req
...
consider storiface.PathStorage when calculating storage requirements
2021-05-20 21:43:55 +02:00
Łukasz Magiera
5f2e163bbb
Merge pull request #6259 from filecoin-project/raulk/docs-storage
...
`storage` module: add go docs and minor code quality refactors
2021-05-20 21:42:30 +02:00
Łukasz Magiera
e3255a06ea
sealing: Add missing states to SubmitPreCommitBatch planner
2021-05-20 18:39:59 +02:00
Łukasz Magiera
d71334cc24
Address self-review
2021-05-20 18:04:07 +02:00
yaohcn
e21d4d7a7e
fix ticket expired
2021-05-20 20:08:24 +08:00
Anton Evangelatov
9a7b0b657e
comment
2021-05-20 13:18:56 +02:00
Anton Evangelatov
19bd5beb96
update remoteGetSector
2021-05-20 13:15:57 +02:00
Anton Evangelatov
1a9b5760a7
add remoteGetAllocated http handler
2021-05-20 13:14:25 +02:00
Anton Evangelatov
90928991b3
remove DealInfo and DealSchedule from storage-sealing gen/main.go
2021-05-20 13:11:10 +02:00
Anton Evangelatov
76bb424de0
update StorageMinerAPI
2021-05-20 13:01:14 +02:00
Anton Evangelatov
cb603c62d9
update retrievaladapter ; add piece_provider
2021-05-20 12:49:53 +02:00
Anton Evangelatov
c12d802811
update reader to use ReadAtLeast
2021-05-20 12:38:38 +02:00
Anton Evangelatov
2aad7b6979
update sectorstorage.New (Manager) interface
2021-05-20 12:32:29 +02:00
Łukasz Magiera
bb889a5976
sealing: Fix max commit batch size check
2021-05-20 12:11:11 +02:00
Łukasz Magiera
7edffcd37b
Update ffi
2021-05-20 11:06:40 +02:00
Łukasz Magiera
51139361c7
sealing: Handle full batches correctly
2021-05-19 20:34:50 +02:00
Łukasz Magiera
e088c71b9a
marketadapter: Handle batch sealing messages
2021-05-19 20:07:20 +02:00
Łukasz Magiera
a5677d1b7a
ffiwrapper: Separate Prover interface
2021-05-19 15:20:23 +02:00
Anton Evangelatov
83e55dc09e
move handleProvingSector to correct file - states_proving.go
2021-05-19 13:17:59 +02:00
Anton Evangelatov
55401116bb
add UnpaddedByteIndex.Valid()
2021-05-19 13:16:29 +02:00
Anton Evangelatov
e275b54f55
export PartialFile and OpenPartialFile
2021-05-19 13:14:16 +02:00
Anton Evangelatov
a989f60e27
add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule
2021-05-19 13:05:07 +02:00
Łukasz Magiera
dd393b470f
Fix aggregation inputs
2021-05-18 20:36:47 +02:00
Łukasz Magiera
0419c64a06
CLI for precommit batching
2021-05-18 20:35:35 +02:00
Łukasz Magiera
e400bdf87a
Order proofs before aggregation
2021-05-18 18:58:41 +02:00
Łukasz Magiera
9690bc882c
Test to trigger batching logic
2021-05-18 18:41:42 +02:00
Łukasz Magiera
1946d2ffd4
Wire up Precommit Batching
2021-05-18 17:37:52 +02:00
Łukasz Magiera
f66b9c5663
Maybe working precommit batching
2021-05-18 17:21:16 +02:00
Łukasz Magiera
d92c5e1001
Missing precommit batcher
2021-05-18 16:54:55 +02:00
Łukasz Magiera
c7ba083fa4
Import precommit batcher
2021-05-18 16:51:06 +02:00
Łukasz Magiera
56145201db
Merge pull request #6282 from filecoin-project/feat/fip-0013-addenda
...
FIP 13 addenda: correctly handle commit batch timer
2021-05-18 16:23:23 +02:00
vyzo
c544f4ce44
avoid extraneous rpc call for storage start epoch
2021-05-18 16:59:11 +03:00
vyzo
6b3e04b9b1
cache sector deadlines.
2021-05-18 15:28:51 +03:00
vyzo
74bad49068
correctly handle commit batch timer
2021-05-18 14:30:47 +03:00
Łukasz Magiera
fe9311e435
update ffi
2021-05-18 13:24:10 +02:00
vyzo
357c0868b7
proper config for termination batching and commit wait
2021-05-18 12:20:19 +03:00
Łukasz Magiera
ba2032c642
Fix some aggregation bugs
2021-05-17 22:51:29 +02:00
Jennifer Wang
5af3af5a22
Merge branch 'release/v1.9.0' into releases
2021-05-17 16:16:34 -04:00
Łukasz Magiera
6278bdc69a
Make things build
2021-05-17 20:47:41 +02:00
Łukasz Magiera
8f42f375cf
Update ffi
2021-05-17 19:55:44 +02:00
Łukasz Magiera
71bd005594
wip
2021-05-17 13:31:43 -04:00
Aayush Rajasekaran
5f8c80533a
Update to latest actors and FFI
2021-05-17 13:15:07 -04:00
Raúl Kripalani
2d7f4b1c61
docs: add godocs to storage module.
2021-05-14 19:45:47 +01:00
Łukasz Magiera
506f39b294
WIP: Integrate FIP0013
2021-05-11 22:10:29 -04:00
Anton Evangelatov
eb13c74dce
panic on unknown pathType
2021-05-11 18:14:01 +02:00
Anton Evangelatov
e07438417c
consider storiface.PathStorage when calculating storage requirements
2021-05-11 13:19:26 +02:00
Jennifer Wang
b1d046ce16
Update to proof v7.0.1
2021-05-06 17:31:15 -04:00
Travis Person
323bb02ca5
Update to new proofs with gpu support enabled in release binaries
2021-05-04 20:31:26 +00:00
Łukasz Magiera
7c37618b50
Update ffi to proofs v7
2021-04-30 14:30:12 +02:00
Łukasz Magiera
19ced50d81
Update ffi to proofs v7
2021-04-30 14:28:36 +02:00
whyrusleeping
ac77c51d5e
address nit
2021-04-29 10:24:16 -07:00
whyrusleeping
48feb52cbf
add a test for adding padded pieces
2021-04-27 16:28:25 -07:00
whyrusleeping
6daaf6ac2e
attempt to do better padding on pieces being written into sectors
2021-04-27 11:39:06 -07:00
Łukasz Magiera
a24a73bb52
Merge tag 'v1.8.0' into release/v1.9.0
2021-04-27 10:30:33 +02:00
Łukasz Magiera
c4137a438d
Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11
2021-04-27 08:22:55 +02:00
Aayush Rajasekaran
f1ded63d60
v4 specs-actors integration, nv12 migration
2021-04-27 02:02:44 -04:00
Dirk McCormick
61344644a4
feat: add more debug logging for unsealing
2021-04-22 11:33:28 +02:00
Łukasz Magiera
e76935147f
Merge remote-tracking branch 'origin/master' into feat/v1-api
2021-04-13 14:24:22 +02:00
Łukasz Magiera
335a2df062
Merge pull request #6006 from filecoin-project/asr/proofs-update
...
Update to latest proofs
2021-04-13 11:27:02 +02:00
Łukasz Magiera
b54b1303f9
Merge pull request #5962 from filecoin-project/fix/storagefsm-allow-broken-global
...
storagefsm: Apply global events even in broken states
2021-04-11 22:30:56 +02:00
yaohcn
844d1d364d
fix health report
2021-04-11 14:47:04 +08:00
Aayush Rajasekaran
738cc2555c
Update FFI
2021-04-10 01:40:24 -04:00
Łukasz Magiera
9d4935a736
Merge pull request #5976 from filecoin-project/fix/storage-demote-stat-err
...
localstorage: Demote reservation stat error to debug
2021-04-09 21:08:08 +02:00
Łukasz Magiera
bed5ec81bd
localstorage: Demote reservation stat error to debug
2021-04-06 16:06:47 +02:00
Łukasz Magiera
9fb345af67
storagefsm: Test applying global events in broken state
2021-04-06 15:38:13 +02:00
Peter Rabbitson
25a77d905d
Centralize everything on ipfs/go-log/v2
...
I am not entirely sure this is right, but everything seems to build...
2021-04-06 15:04:32 +02:00
Łukasz Magiera
81bd27911f
Propagate StateMsg api changes
2021-04-05 19:56:53 +02:00
Łukasz Magiera
9a2ec07837
storagefsm: Apply global events even in broken states
2021-04-05 18:25:29 +02:00
Łukasz Magiera
f372964a60
fix lint
2021-04-01 16:40:11 +02:00
Łukasz Magiera
c7de7882a1
Merge branch 'master' of github.com:ppiaas/lotus-ppiaas into fix/stg-fetch-close
2021-04-01 16:38:17 +02:00
yaohcn
3895b2ccca
fix reservations
2021-03-31 14:30:24 +08:00
Łukasz Magiera
a7122d424b
Merge pull request #5527 from filcloud/handle-ticket-expire
...
fix precommit ticket expiration handling
2021-03-29 22:55:43 +02:00
Łukasz Magiera
1aea1ebb9d
Merge pull request #5801 from filecoin-project/fix/fsm-input-stuck
...
storagefsm: Trigger input processing when below limits
2021-03-12 19:02:15 +01:00
Łukasz Magiera
0974e02089
storagefsm: common createSector method
2021-03-12 18:42:17 +01:00
Steven Allen
b74b1658b2
test: mock proof side effects on verify as well
2021-03-12 09:06:41 -08:00
Łukasz Magiera
d9923125ff
mock sealer: Simulate randomness sideeffects
2021-03-12 17:33:43 +01:00
Łukasz Magiera
e5b5bf1c1b
storagefsm: Update sector stats atomically with sector creation
2021-03-12 17:25:24 +01:00
Łukasz Magiera
679f5f7661
storagefsm: Helper stat functions in updateSector
2021-03-12 17:00:20 +01:00
Łukasz Magiera
957d3f0636
storgefsm: Use locks correctly
2021-03-12 15:01:00 +01:00
Łukasz Magiera
5c605d99e2
storgefsm: Fix tests
2021-03-12 14:56:46 +01:00
Łukasz Magiera
f5ed25371b
storagefsm: Trigger input processing wheen below limits
2021-03-12 14:54:10 +01:00
Łukasz Magiera
d0243904f7
Merge pull request #5779 from filecoin-project/feat/storagefsm-no-restart-log
...
storagefsm: Don't log the SectorRestart event
2021-03-11 15:18:32 +01:00
Łukasz Magiera
881c722025
storagefsm: Fix double unlock with ready WaitDeals sectors
2021-03-11 15:03:51 +01:00
Łukasz Magiera
a1b6a2f011
storagefsm: Don't log the SectorRestart event
2021-03-11 12:52:34 +01:00
Łukasz Magiera
cc490b983a
Merge pull request #5761 from filecoin-project/fix/testunpadreader-go116
...
Fix TestUnpadReader on Go 1.16
2021-03-10 13:13:27 +01:00
Łukasz Magiera
7fbb4bdd3b
Fix TestUnpadReader on Go 1.16
2021-03-10 13:05:25 +01:00
Łukasz Magiera
40fdf6c180
Merge pull request #5624 from filecoin-project/feat/miner-storage-limit
...
Configurable storage path storage limit
2021-03-10 11:15:28 +01:00
Łukasz Magiera
df0a8d9507
Merge pull request #5759 from filecoin-project/asr/terminate-livecheck
...
Check liveness of sectors when processing termination batches
2021-03-10 11:14:11 +01:00
Aayush Rajasekaran
47064f987c
Check liveness of sectors when processing termination batches
2021-03-09 23:36:30 -05:00
Łukasz Magiera
fe230f901e
Collect worker task metrics
2021-03-05 12:48:17 +01:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup
...
storagefsm: Cleanup CC sector creation
2021-02-26 11:39:28 +01:00
Łukasz Magiera
e49a412f6d
Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input
...
storagefsm: Rewrite input handling
2021-02-25 14:28:04 +01:00
Łukasz Magiera
2b9ed9fa3d
Remove sectors with all deals expired in RecoverDealIDs
2021-02-22 21:03:51 +01:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map
...
Update markets, cbor-gen with soft map decoding
2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
6732758d0f
Tweak mock ProveCommits to respect proof sizes
2021-02-19 17:22:59 -05:00
Łukasz Magiera
26399dba70
Update markets, cbor-gen with soft map decoding
2021-02-19 20:11:43 +01:00
Łukasz Magiera
37b1fb97fb
Configurable storage path storage limit
2021-02-18 16:44:34 +01:00
yaohcn
3f66e731fa
fix drop sector
2021-02-18 12:00:28 +08:00
Łukasz Magiera
90b8612cf4
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-16 22:39:06 +01:00
Łukasz Magiera
9ebeafb174
Update ffi
2021-02-16 22:04:08 +01:00
Łukasz Magiera
f719765069
storageminer: exit PledgeSector after sectors enter sealing pipeline
2021-02-16 17:41:58 +01:00
Łukasz Magiera
fc5e243c92
storagefsm: Cleanup CC sector creation
2021-02-16 17:14:59 +01:00
Łukasz Magiera
dd82729f60
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-16 13:45:39 +01:00
Aayush Rajasekaran
bb7fabb439
Update FFI
2021-02-15 17:22:25 -05:00
Łukasz Magiera
6907e5879d
Fix WaitDeals sector accounting
2021-02-11 13:52:00 +01:00
Łukasz Magiera
e27a530cbc
storagefsm: cleanup openSectors better; pendingPieces by pieceCid
2021-02-09 18:44:41 +01:00
He Weidong
0e2e1125d3
fix precommit ticket expiration handling
2021-02-05 10:45:15 +08:00
Łukasz Magiera
6e7fcb7ee6
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-03 14:01:56 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg
...
Add miner config to always keep ensealed deal copies
2021-02-03 12:40:02 +01:00
Łukasz Magiera
94009f247f
Merge pull request #5411 from filecoin-project/feat/sealing-handle-batch-publish
...
Handle batch publish storage deals message in sealing recovery
2021-01-28 19:33:49 +01:00
Łukasz Magiera
aebf3cd64c
Update ffi with more portable blst
2021-01-27 22:36:51 +01:00
Łukasz Magiera
d4c7b63aac
Add miner config to always keep ensealed deal copies
2021-01-26 17:50:31 +01:00
Łukasz Magiera
fabcbb621d
Merge remote-tracking branch 'origin/master' into next
2021-01-26 00:47:22 +01:00
Łukasz Magiera
67780346e2
storagemgr: Always unseal full sectors
2021-01-25 21:39:15 +01:00
Łukasz Magiera
e4f4fa2c7d
Update FFI more, fix zero aggregate sigs
2021-01-25 20:05:03 +01:00
Dirk McCormick
1b494acc9e
feat: handle batch publish storage deals message in sealing recovery
2021-01-25 14:10:05 +01:00
Łukasz Magiera
9b62e224d5
Use filecoin-ffi master
2021-01-25 13:57:15 +01:00
Steven Allen
03cd3760bb
correctly pick the seal type based on the network version
...
Of course, we should really just run all of our tests post actors v2.
2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc
Actors update: MinerInfo.SealProofType has been removed
2021-01-21 15:21:19 -08:00
Łukasz Magiera
1070ad2289
storagefsm: Drop unused TargetWaitDealsSectors
2021-01-21 22:20:16 +01:00
Łukasz Magiera
ec4deb7e28
storagefsm: Fix unlocking in handleWaitDeals
2021-01-21 19:59:18 +01:00
Łukasz Magiera
1336d8855d
storagefsm: Drop addpiece wait after fixing storageadapter
2021-01-21 17:40:50 +01:00
Łukasz Magiera
df14f156e1
storagefsm: More logging for deal test debugging
2021-01-21 17:40:50 +01:00
Łukasz Magiera
b9a9f23204
storagefsm: Add stub AddPieceFailed state
2021-01-21 17:40:50 +01:00
Łukasz Magiera
f96f12c836
storagefsm: Add rest of checks in WaitDeals
2021-01-21 17:40:50 +01:00
Łukasz Magiera
069766ecc4
storagefsm: Don't persist piece assignment queue
2021-01-21 17:40:50 +01:00
Łukasz Magiera
270f2935a9
storagefsm: Check per-sector deal limits
2021-01-21 17:40:50 +01:00
Łukasz Magiera
fd67a41c75
storagefsm: Change sector CreationTime to unix ts
2021-01-21 17:40:50 +01:00
Łukasz Magiera
9857ad8378
storagefsm: Fix some deadlock cases
2021-01-21 17:40:50 +01:00
Łukasz Magiera
e5814dac4f
cbor-gen
2021-01-21 17:40:50 +01:00
Łukasz Magiera
542357a1df
storagefsm: Start packing correctly
2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d
storagefsm: Rewrite input handling
2021-01-21 17:40:50 +01:00
dignifiedquire
3d9eb226fa
refactor: switch to filecoin-ffi bls api for bls signatures
2021-01-20 16:19:07 +01:00
dignifiedquire
187a499cca
remove extern/blst
2021-01-20 16:19:07 +01:00
Łukasz Magiera
48c6c55f68
Merge pull request #5384 from filecoin-project/fix-storage-size-estimation
...
fix FileSize to return correct disk usage recursively
2021-01-19 19:22:47 +01:00
Łukasz Magiera
560a127ce6
fsutil: Defensive check for Stat cast
2021-01-19 18:53:37 +01:00
Anton Evangelatov
834a6148b0
use stat.Blocks and not info.Size
2021-01-19 18:07:37 +01:00
Anton Evangelatov
16d07d3f18
fix FileSize to return correct disk usage recursively
2021-01-19 15:21:31 +01:00
Łukasz Magiera
0eaa6d9983
storagefsm: Send correct event on ErrExpiredTicket in CommitFailed
2021-01-17 22:29:07 +01:00
Łukasz Magiera
80b8d4b9d7
Address review
2021-01-14 20:27:15 +01:00
Łukasz Magiera
32885e1129
termination batcher: Notify based on what was sent
2021-01-14 17:14:26 +01:00
Łukasz Magiera
4015ddbb4f
fsm: Fix panic in precommit check in handleTerminating
2021-01-14 16:13:32 +01:00
Łukasz Magiera
49abdd7d7d
Sector termination support - address review
2021-01-14 15:46:57 +01:00
Łukasz Magiera
7ddf1d1feb
Add terminating states to state lists
2021-01-14 12:44:27 +01:00
Łukasz Magiera
144b5a1350
perning termination API
2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce
Sector termination test
2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a
SectorTerminateFlush API
2021-01-13 23:32:04 +01:00
Łukasz Magiera
174c595acc
Don't declare in proving window
2021-01-13 22:19:10 +01:00
Łukasz Magiera
52cc2cd3eb
Initial sector termination support
2021-01-13 00:42:01 +01:00
Łukasz Magiera
5462ecf6da
gofmt
2021-01-11 16:44:11 +01:00
Łukasz Magiera
9e74c3c4e8
Multicore AddPiece CommP
2021-01-11 16:44:11 +01:00
Łukasz Magiera
299e72458e
storagefsm: Fix unsealedInfoMap.lk init race
2021-01-10 14:01:29 +01:00
Łukasz Magiera
2c9f4ad5ad
Merge pull request #5251 from liismn/master
...
Fix deadlock in runWorker in sched_worker.go
2021-01-05 19:35:49 +01:00
Łukasz Magiera
44a56c7e33
Merge pull request #5208 from jsign/jsign/imprreadpiece
...
sector-storage/mock: improve mocked readpiece
2021-01-05 18:37:48 +01:00
15ho
5b235db366
fix: stores.FetchHandler: defer close file and increase buffer size
2020-12-29 10:07:28 +08:00
liismn
e9cc697ee8
Update sched_worker.go
...
Avoid possible deadlock by sh.workersLk
2020-12-25 15:05:01 +08:00
quanzhan lu
d806fafee8
update weight canSeal and canStore when attach
...
update weight canSeal and canStore when attach storage
2020-12-22 19:07:17 +08:00
Ignacio Hagopian
f6cf27c781
improve mocked readpiece
...
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-12-16 10:15:38 -03:00
zzx234234
7db5e8061d
fix a coding error in truncating sector log
2020-12-11 11:31:36 +08:00
Łukasz Magiera
46a5013dc1
stores: Simplify remote copy buf size logic
2020-12-10 20:54:42 +01:00
Łukasz Magiera
53a6c4eac6
Merge branch 'iosize' of github.com:acautman/lotus into acautman-iosize
2020-12-10 20:48:04 +01:00
caoao
d4582fb32e
write size according to RegisteredSealProof
2020-12-10 11:59:09 +08:00
stirlingx
3c50baaf5b
fix curSealing out of MaxSealingSectors limit
2020-12-10 11:04:42 +08:00
beck
d3fa728894
Remove extra spaces
2020-12-09 19:22:21 +08:00
Łukasz Magiera
1502ca6460
storagefsm: Fix addr selection for commits
2020-12-03 12:30:41 +01:00
Łukasz Magiera
562f5839b7
Fix lint
2020-12-02 22:01:09 +01:00
Łukasz Magiera
2fd93a55ac
plumb AddressSelector to stoage fsm
2020-12-02 21:47:54 +01:00
Łukasz Magiera
289ef910a0
fix imports, docsgen
2020-12-02 00:39:55 +01:00
Łukasz Magiera
95eaf13b5a
sectorstorage: Fix tests
2020-12-02 00:36:32 +01:00
Łukasz Magiera
c6f85886bf
miner: Add slow mode to proving check
2020-12-02 00:32:14 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013
...
Sector check command
2020-12-01 14:59:01 +01:00
Łukasz Magiera
a4f3758f4c
worker api: better grouping
2020-11-30 23:16:30 +01:00
Łukasz Magiera
e8d2560581
Fix tests
2020-11-27 16:47:48 +01:00
Łukasz Magiera
3672053ae9
worker: Support setting task types at runtime
2020-11-26 17:33:34 +01:00
Łukasz Magiera
2304ea0077
Merge pull request #5016 from filecoin-project/fix/fsm-rm-unceroverable
...
storagefsm: Add missing planners
2020-11-26 12:22:48 +01:00
Łukasz Magiera
74f040f9d5
storagefsm: Add SubmitCommit to toStatState switch
2020-11-26 12:02:35 +01:00
Łukasz Magiera
eeb38fa649
storagefsm: Test fsmPlanners map
2020-11-26 10:58:07 +01:00
Łukasz Magiera
ba87cb4fa9
storagefsm: Add missing planners
2020-11-26 10:57:47 +01:00
Łukasz Magiera
65653a1747
storagefsm: Add GetTicket to some maps
2020-11-26 10:53:31 +01:00
s1m0n21
663b3f4f21
add a sector check command
2020-11-26 15:02:43 +08:00
Łukasz Magiera
89493fe2b8
storagefsm: newDealSector review
2020-11-25 18:55:58 +01:00
Łukasz Magiera
6166204bac
storagefsm: Improve new deal sector logic
2020-11-25 16:40:04 +01:00
Łukasz Magiera
214f46bc88
Merge pull request #4876 from filecoin-project/fix/miner-ticket-loop
...
storagefsm: Fix expired ticket retry loop
2020-11-24 19:54:41 +01:00
Łukasz Magiera
a63ef1dcd5
Merge pull request #4984 from yaohcn/fix-log-warn
...
fix log format
2020-11-24 18:01:56 +01:00
Łukasz Magiera
7c831101f3
Merge pull request #4727 from filecoin-project/nonsense/integrate-testplans-to-lotus
...
move testground/lotus-soup testplan from oni to lotus
2020-11-24 17:05:29 +01:00
Łukasz Magiera
fe92a6e2d4
Merge pull request #4024 from zgfzgf/sector-sched
...
modify for unsafe
2020-11-24 16:19:17 +01:00
yaohcn
7c0b6f41d8
fix log format
2020-11-24 19:09:48 +08:00
Anton Evangelatov
fd640864cb
Merge branch 'master' into nonsense/integrate-testplans-to-lotus
2020-11-23 20:20:44 +01:00
Łukasz Magiera
83b1009837
storagefsm: check getticket precommit expication correctly
2020-11-23 19:12:54 +01:00
Łukasz Magiera
85e592bbe2
storagefsm: Retry pushing messages when out-of-funds
2020-11-23 19:03:31 +01:00
Łukasz Magiera
e2a6b99144
storagefsm: Fix expired ticket retry loop
2020-11-23 19:03:31 +01:00
Łukasz Magiera
0e39e760e5
Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction
...
Remove all temp file generation for deals
2020-11-23 18:43:00 +01:00
Dirk McCormick
3634ca50ca
fix: sector-storage tests
2020-11-23 16:09:57 +01:00
Dirk McCormick
c965696586
fix: ffiwrapper sealer test
2020-11-23 14:21:46 +01:00
Dan Shao
f3838a47c7
Format workerID as string
2020-11-23 15:07:50 +08:00
zgfzgf
b6893b0a3f
solve merage problem
2020-11-22 16:15:30 +08:00
hannahhoward
218f7c2c5d
feat(markets): update markets, extract commp
2020-11-20 13:29:14 -08:00
Łukasz Magiera
f0eac01a3c
Merge remote-tracking branch 'origin/master' into nonsense/integrate-testplans-to-lotus
2020-11-20 15:22:10 +01:00
S1M0N21
2b611b3a99
Remove unused key
2020-11-19 23:48:28 +08:00
Aayush Rajasekaran
68a16afeda
Merge branch 'master' into release/v1.2.0
2020-11-17 19:40:05 -05:00
Łukasz Magiera
00335878ab
Merge pull request #4712 from filecoin-project/fix/commitfailed-checks
...
Check (pre)commit receipt before other checks in failed states
2020-11-18 01:26:02 +01:00
Łukasz Magiera
b242d69805
Make storiface.CallError json-friendly
2020-11-17 16:28:41 +01:00
Łukasz Magiera
b8853aa4d5
Add error codes to worker return
2020-11-17 16:17:55 +01:00
Łukasz Magiera
e0ade92b29
Fix lint warnings
2020-11-16 19:20:18 +01:00
Łukasz Magiera
444239a643
update ffi
2020-11-16 19:17:07 +01:00
Łukasz Magiera
cd0cb1407d
update ffi
2020-11-16 19:03:30 +01:00
Łukasz Magiera
915c5e0fdd
Get most tests to pass
2020-11-16 19:03:30 +01:00
Łukasz Magiera
2a7e4af5ee
Fix lotus-miner/lotus-worker build
2020-11-16 19:03:30 +01:00
Steven Allen
584907269a
make sector-storage compile
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bea9dd178
Making sealing logic work with multiple seal proof types
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bb67f2fe9
Merge pull request #4834 from filecoin-project/feat/sector-gc
...
miner: Basic storage cleanup command
2020-11-13 00:03:17 +01:00
Łukasz Magiera
408183c7a6
docsgen: Fix CI
2020-11-12 20:15:57 +01:00
Łukasz Magiera
11b041224e
fsm: Allow manually resurrecting removed sectors
2020-11-12 19:37:54 +01:00
Łukasz Magiera
8409e155d4
docsgen: Support miner/worker structs
2020-11-12 10:29:42 +01:00
Łukasz Magiera
8ac495723e
gofmt
2020-11-11 17:42:23 +01:00
Łukasz Magiera
09f9f871a3
Create a command to abort sealing calls
2020-11-11 17:39:12 +01:00
Łukasz Magiera
bd71bf2379
Expand sched-diag
2020-11-11 16:47:44 +01:00
Łukasz Magiera
18aa97f962
Merge pull request #4779 from filecoin-project/feat/work-track-hostname
...
storage: Track worker hostnames with work
2020-11-10 20:38:44 +01:00
Łukasz Magiera
5caa277261
storage: Track abandoned work more correctly
2020-11-09 23:38:20 +01:00
Łukasz Magiera
f819e71d12
storage: Separate returned jobs in jobs cli
2020-11-09 23:13:29 +01:00
Łukasz Magiera
27a9dd3bbb
storage: Track worker hostnames with work
2020-11-09 23:09:04 +01:00
Łukasz Magiera
5b8f555221
sched: Update after tasks finish preparing
2020-11-09 21:44:28 +01:00
Łukasz Magiera
fbc05fa5a8
stores: Report storage changes quickly
2020-11-09 21:43:52 +01:00
Łukasz Magiera
95d9084899
Merge pull request #4765 from zgfzgf/feat-sealing-garbage
...
optimize pledge sector
2020-11-09 21:06:33 +01:00
zgfzgf
5bcc6339b4
optimize code replace strings with constants
2020-11-09 16:21:16 +08:00
zgfzgf
3665f90795
optimize pledge sector
2020-11-08 16:24:50 +08:00
Anton Evangelatov
f7f5a27ebf
remove oni submodule
2020-11-06 21:03:56 +01:00
Łukasz Magiera
cde0e58e39
Check (pre)commit receipt before other checks in failed states
2020-11-03 18:47:29 +01:00
Łukasz Magiera
761c9ee2b6
Merge pull request #4699 from blockchain-lib/master
...
struct name incorrect
2020-11-03 11:47:18 +01:00
dignifiedquire
5c3055219c
update filecoin-ffi to master
2020-11-03 11:02:58 +01:00
stirlingx
b5d2f01993
Update selector_task.go
2020-11-03 09:20:48 +08:00
dignifiedquire
1d17d61dbc
update filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
d457b7a5e3
update filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
398a831701
update blst and filecoin-ffi
2020-11-02 17:05:52 +01:00
dignifiedquire
61fa03b1ad
use supranation/blst@0.2.0 directly
2020-11-02 17:05:52 +01:00
dignifiedquire
56980ffc1f
remove extern/fil-blst
2020-11-02 17:05:52 +01:00
Łukasz Magiera
094ea3fe97
Merge pull request #4666 from filecoin-project/fix/sched-issues
...
Fix worker reenabling, handle multiple restarts in worker
2020-10-30 21:34:07 +01:00
Łukasz Magiera
69e44ebf07
sched: re-register worker windows after re-enabling correctly
2020-10-30 18:32:59 +01:00
Łukasz Magiera
f90a387f96
sched: Print worker UUIDs in shed-diag correctly
2020-10-30 18:32:16 +01:00
Łukasz Magiera
774e2ecebf
sched: Fix worker reenabling
2020-10-30 18:01:37 +01:00
Łukasz Magiera
7fbb868513
Debug flag to force running sealing scheduler
2020-10-30 11:07:35 +01:00
Łukasz Magiera
ee814fb48e
Merge pull request #4643 from filecoin-project/fix/fsm-ticket-loop
...
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 18:25:14 +01:00
Łukasz Magiera
c6b03ce62b
sectorstorage: Missing unlock in waitWork
2020-10-29 15:18:51 +01:00
Łukasz Magiera
ea5bb5cdab
sectorstorage: Fix manager restart edge-case
2020-10-29 12:14:21 +01:00
Łukasz Magiera
33459f1b36
storagefsm: Fix GetTicket loop when the sector is already precommitted
2020-10-29 10:26:02 +01:00
Łukasz Magiera
f0f75e2d2c
Merge pull request #4627 from karalabe/fix-gpu-usage-tracking
...
extern/sector-storage: fix GPU usage overwrite bug
2020-10-29 10:13:54 +01:00
Łukasz Magiera
32ea060e99
Merge pull request #3618 from filecoin-project/feat/async-restartable-workers
...
Track sealing processes across lotus-miner restarts
2020-10-28 21:55:45 +01:00
Péter Szilágyi
5f657b4333
extern/sector-storage: fix GPU usage overwrite bug
2020-10-28 20:52:33 +02:00
Łukasz Magiera
da7ecc1527
Fix flaky sealing manager tests
2020-10-28 16:15:17 +01:00
Łukasz Magiera
4475b9ac05
Use filecoin-ffi master
2020-10-28 15:31:15 +01:00
Łukasz Magiera
4100f6eead
fix TestWDPostDoPost
2020-10-28 15:23:21 +01:00
Łukasz Magiera
ed2f81da2f
sched: Fix tests
2020-10-28 14:34:28 +01:00
Łukasz Magiera
4cf00b8b42
worker_local: address review
2020-10-28 14:29:17 +01:00
Łukasz Magiera
96c5ff7e7f
sched: use more letters for variables
2020-10-28 14:23:38 +01:00
Łukasz Magiera
8731fe9112
sched: split worker handling into more funcs
2020-10-28 14:14:50 +01:00
Łukasz Magiera
84b567c790
sched: move worker funcs to a separate file
2020-10-28 13:39:28 +01:00
Łukasz Magiera
843c0719e0
Update FFI
2020-10-27 15:37:11 +01:00
Łukasz Magiera
660236b224
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-23 23:25:35 +02:00
Łukasz Magiera
29e334de54
Merge pull request #4511 from filecoin-project/steb/generalize-window-post
...
Manage sectors by size instead of proof type.
2020-10-22 21:27:48 +02:00
Steven Allen
bcabe7b3b5
migrate methods to abstracted methods
...
Method numbers never change anyways. At worst, we'll deprecate old methods and
have to explicitly import them from the correct actors version to use them.
2020-10-21 12:18:37 -07:00
Steven Allen
4e730b5ec8
port to v2 imports
2020-10-21 12:16:23 -07:00
Steven Allen
00dcb1bce9
Manage sectors by size instead of proof type.
...
* We may have multiple sectors with the same size and different proof types, but all these management functions stay the same.
* This simplifies PoSt logic.
2020-10-20 18:30:56 -07:00
Łukasz Magiera
8c86ea6b75
localworker: Try very hard to get ruselts to manager
2020-10-18 19:45:11 +02:00
