Commit Graph

149 Commits

Author SHA1 Message Date
Łukasz Magiera
2cef55a4f6 feat: sealing: Sector upgrade queue 2022-03-16 22:29:37 +01:00
Łukasz Magiera
c6bda0e5d4 fix: storage cli: Output primary sector status correctly 2022-03-15 14:29:21 +01:00
Łukasz Magiera
5416ce5443
Merge pull request #7313 from filecoin-project/feat/cid-to-piece-idx
feat: markets: Integrate index ingest protocol and retrieve by any CID
2022-03-03 16:50:35 +00:00
zenground0
874124c4e7 Handle finalize failures in fsm 2022-03-02 09:33:33 -07:00
Masih H. Derkani
769b0f30ef
Address review comments
- Add comment to clarify the reason for loop in testkit
- Trim common prefix in state printed in CLI commands for better
  readability
- Upgrade to a tagged release of `go-fil-markets` that includes indexing
  work; see: https://github.com/filecoin-project/go-fil-markets/pull/673
- Fix typo in CLI usage.
- Add comments to note that it is safe to use fx `OnStart` context when
  starting the provider engine.
- Fix string concatenation in error message formatting.
2022-03-02 11:04:50 +00:00
zenground0
77a954c7c3 Merge branch 'master' into feat/cid-to-piece-idx 2022-02-28 12:57:11 -07:00
duzy555666
296ae4305f avoid panic 2022-02-28 16:51:15 +08:00
jennijuju
52756128da add flag usage 2022-02-25 10:49:18 -05:00
zenground0
367146f032 add rdi flag for abort upgrade 2022-02-24 15:20:21 -07:00
zenground0
abe04c33c1 Fix fault tracker to handle snap deals 2022-02-23 09:56:47 -07:00
Aarsh Shah
e3f5b7c59f update dagstore top level index db 2022-02-18 14:00:01 +04:00
Łukasz Magiera
c974872494 lotus-miner: More sector-related data in info all 2022-02-17 16:35:09 +01:00
Łukasz Magiera
7823363bba lotus-miner: Support update files in storage find 2022-02-17 16:29:27 +01:00
Łukasz Magiera
6f21a10a56
Merge pull request #8098 from filecoin-project/feat/sectorlist-pledge
feat: lotus-miner sectors list --initial-pledge
2022-02-15 16:44:49 +01:00
Łukasz Magiera
a3e785cb4a
Merge pull request #8057 from mtelka/info-noadmin
fix: state: Allow lotus-miner info to complete without admin permission
2022-02-15 15:00:29 +01:00
Łukasz Magiera
0b7addc031 lotus-miner sectors list --initial-pledge 2022-02-15 14:21:30 +01:00
Aayush
eebe784191 fix: sealer: allow enable/disabling ReplicaUpdate tasks 2022-02-14 15:08:28 -05:00
Marcel Telka
3c82ee49c0 Allow lotus-miner info to complete without admin permission 2022-02-10 12:45:57 +01:00
Jennifer Wang
de1f4792cd Merge remote-tracking branch 'origin/master' into jen/v15 2022-02-09 19:27:34 -05:00
Łukasz Magiera
947000c94f fsm: Call ReleaseSectorKey after WinningPoSt lookback period 2022-02-09 12:41:39 +01:00
Jennifer Wang
6ed97f0d04 Merge branch 'release/v1.14.0' into jen/v15 2022-02-09 03:17:38 -05:00
Aayush
1b18236f91 feat: sealer: allow users to abort in-flight snap upgrades 2022-01-25 13:01:51 -05:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
Aayush Rajasekaran
51b4458d32 Resolve conflict 2022-01-14 17:18:47 -05:00
Aayush Rajasekaran
e4cb68e05d format 2022-01-14 17:14:32 -05:00
Jennifer Wang
60cf2a71cf fix lint 2022-01-14 17:14:32 -05:00
Jennifer Wang
87babdf3e7 Disable mark-for-upgrade two days before the network v15 OhSnap upgrade to avoid unexpected edge cases that may cause deal/sector failure 2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f Snap Deals Integration
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
52411b12e3 Resolve conflict 2022-01-12 19:15:56 -05:00
Aarsh Shah
8c8652e7bb add a new cli 2022-01-12 18:06:48 +04:00
Aarsh Shah
edfc2c215c Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx 2022-01-12 12:48:54 +04:00
Aayush Rajasekaran
1e24ef4d0d format 2022-01-11 12:50:34 -05:00
Jiaying Wang
93653d9a58
Merge branch 'feat/snap-deals' into 7891 2022-01-10 22:22:46 -05:00
zenground0
33f2d24f54 Snap Deals Integration
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-10 15:39:38 +05:30
Jennifer Wang
bf23e59900 fix lint 2022-01-07 18:24:57 -05:00
Jennifer Wang
c578efaabd Disable mark-for-upgrade two days before the network v15 OhSnap upgrade to avoid unexpected edge cases that may cause deal/sector failure 2022-01-07 18:05:36 -05:00
zenground0
61e173215b Snap Deals Integration
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-05 03:10:31 -05:00
hannahhoward
ced22687eb fix(cmd): fix cli peer decoding 2021-12-22 13:41:56 -08:00
hannahhoward
cddf63efe9 feat(storageminer): add api for transfer diagnostics
Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer
2021-12-22 13:41:29 -08:00
Aayush Rajasekaran
dd20829dbe
Merge pull request #7762 from filecoin-project/feat/sectors-update-state-guard
sectors update-state checks if sector exists before changing its state
2021-12-17 19:21:26 -05:00
gstuart
27e48d73dd Check if sector exists before changing its state 2021-12-17 14:32:18 -05:00
vyzo
dd327f0b22 plumb more contexts 2021-12-17 11:42:09 +02:00
Aayush Rajasekaran
0e2278cc76 Merge branch 'master' into deps/update-ctx-dsbs 2021-12-14 18:08:29 -05:00
vyzo
2e057b8f52 plumb contexts in lotus-miner 2021-12-13 14:28:29 +02:00
gstuart
21e6c963ab Sort lotus-miner retrieval-deals by dealId 2021-12-09 19:53:13 -05:00
Łukasz Magiera
727765b248 Command to list active sector locks 2021-12-03 12:33:23 +01:00
hannahhoward
bfd1099ef3 fix(main): fix bug in startup when tracing is not present 2021-12-01 18:53:00 -08:00
hannahhoward
f3b691d618 feat(tracing): switch to OpenTelemetry
Switch the underlying tracing library to OpenTelemetry, with a bridge to OpenCensus for
compatibility
2021-12-01 16:19:47 -08:00
Clint Armstrong
93e4656a27 Use a float to represent GPU utilization
Before this change workers can only be allocated one GPU task,
regardless of how much of the GPU resources that task uses, or how many
GPUs are in the system.

This makes GPUUtilization a float which can represent that a task needs
a portion, or multiple GPUs. GPUs are accounted for like RAM and CPUs so
that workers with more GPUs can be allocated more tasks.

A known issue is that PC2 cannot use multiple GPUs. And even if the
worker has multiple GPUs and is allocated multiple PC2 tasks, those
tasks will only run on the first GPU.

This could result in unexpected behavior when a worker with multiple
GPUs is assigned multiple PC2 tasks. But this should not suprise any
existing users who upgrade, as any existing users who run workers with
multiple GPUs should already know this and be running a worker per GPU
for PC2. But now those users have the freedom to customize the GPU
utilization of PC2 to be less than one and effectively run multiple PC2
processes in a single worker.

C2 is capable of utilizing multiple GPUs, and now workers can be
customized for C2 accordingly.
2021-11-30 02:06:58 +01:00
Clint Armstrong
c4f46171ae Report memory used and swap used in worker res
Attempting to report "memory used by other processes" in the MemReserved
field fails to take into account the fact that the system's memory used
includes memory used by ongoing tasks.

To properly account for this, worker should report the memory and swap
used, then the scheduler that is aware of the memory requirements for a
task can determine if there is sufficient memory available for a task.
2021-11-30 02:06:58 +01:00