Commit Graph

98 Commits

Author SHA1 Message Date
Andrew Jackson (Ajax)
6b3e9b109f
feat: curio: sectors UI (#11869)
* cfg edit 1

* jsonschema deps

* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* part 2 of last

* message

* merge addtl

* fixing guided setup for myself

* warn-on-no-post

* EditorLoads

* cleanups and styles

* create info

* fix tests

* make gen

* sector early bird

* sectors v2

* sector termination v1

* terminate2

* mjs

* minor things

* flag bad sectors

* fix errors

* add dealweight and deals

* change column width

* refactor sql, handle sealing sectors

* fix estimates

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
2024-04-18 14:57:29 -05:00
Łukasz Magiera
c785e59371
feat: curio: storage index gc task (#11884)
* curio storage path gc: lay out the structure

* curio gc: Implement storage metadata gc

* move bored singleton task impl to harmonytask

* curio: run storage gc task on storage node

* make gen
2024-04-16 16:34:48 -05:00
LexLuthr
3ebb92a976
fix: curio: fix incorrect null check for varchar column (#11881)
* err check empty string

* log skipped tasks
2024-04-15 19:27:31 -05:00
Łukasz Magiera
bc43bd6d69
fix: local storage reservations fixes (#11866)
* paths: Debugging local storage reservations

* paths: Log when individual reservation is less than on-disk space

* paths: fix debug reservations print

* paths: More reserve logs

* paths: More more reserve logs

* paths: add stacks to duplicate done call log

* curio: task storage: Release storage at most once

* curio: cleanup before restarting sdr

* address review

* paths: Simplify reservation release logic
2024-04-15 17:33:06 -05:00
Łukasz Magiera
3cc62e04e9
feat: curio: Cleanup data copies after seal ops (#11847)
* feat: curio: Cleanup data copies after seal ops

* curio: make ensureOneCopy safer

* curio: Don't randomly remove cache files

* Address review
2024-04-11 11:35:09 -05:00
Łukasz Magiera
d7d849cf20
feat: curioweb: Sector info page (#11846)
* feat: curioweb: Sector info page

* Address review
2024-04-11 11:30:53 -05:00
Łukasz Magiera
fd64e38b39
fix: paths/local: Fix on-disk storage accounting in new reservations (#11825) 2024-04-04 11:17:04 -05:00
LexLuthr
a24028c3c0 apply suggestions 2024-04-04 18:06:19 +02:00
LexLuthr
243bf1a0b3 refactor MinerFilter func, update comments 2024-04-04 18:06:19 +02:00
LexLuthr
d1a52cf04d refactor filter decision code 2024-04-04 18:06:19 +02:00
LexLuthr
2951d038a8 add miner fliter 2024-04-04 18:06:19 +02:00
Łukasz Magiera
2ccc0db1cc feat: curio: Storage reservations in MoveStorage 2024-04-03 21:40:06 +02:00
Łukasz Magiera
021385116a paths: Fix local existing allocate requests in Finalize 2024-03-26 20:32:46 +01:00
Łukasz Magiera
c7ca271507 rebuild storage mocks 2024-03-26 20:32:46 +01:00
Łukasz Magiera
7b00cc5827 curio: Add task storage to SDRTrees 2024-03-26 20:32:46 +01:00
Łukasz Magiera
c7b64bd6a9 curio: Storage reservations when fetching 2024-03-26 20:32:46 +01:00
Łukasz Magiera
e86d81b26a fix: curio seal: Fix partial Finalize retry 2024-03-22 16:05:48 +01:00
Łukasz Magiera
6ca55d18a7 address review 2024-03-15 14:10:48 +01:00
Łukasz Magiera
b90cf19604 lpseal: PiecePark in SDRTrees 2024-03-14 09:36:51 +01:00
Łukasz Magiera
0800e6e5a7 make gen 2024-03-14 09:36:47 +01:00
Łukasz Magiera
e060cd2f37 lppiece: Implement Piece Park 2024-03-14 09:36:03 +01:00
Łukasz Magiera
5ab75d35f9 fix mac build 2024-02-22 14:00:47 +01:00
Łukasz Magiera
0634dfce07 Fix lint 2024-02-22 11:39:17 +01:00
Łukasz Magiera
4ae73f059c provider: Fix sql query capitalization 2024-02-21 14:19:48 +01:00
Łukasz Magiera
91d3edecfd lp: Address review 2024-02-21 13:28:49 +01:00
Łukasz Magiera
9f9e2f7fe4 make gen 2024-02-17 18:13:30 +01:00
Łukasz Magiera
c293bcebb6 storage: Make remote Generate Vanilla more robust 2024-02-17 11:20:49 +01:00
Łukasz Magiera
e22ab4c8fe lp: makePhase1Out for large sectors 2024-02-17 01:40:01 +01:00
Łukasz Magiera
06a2f8c670 paths: Fix dbindex heartbeat 2024-02-17 00:57:27 +01:00
Łukasz Magiera
bc8517baa0 paths: Support multiple endpoints in FsStat 2024-02-16 21:23:21 +01:00
Łukasz Magiera
9ddfe23702 lp: Small migration improvements 2024-02-16 21:23:19 +01:00
Łukasz Magiera
4c2235a686 make gen, mod tidy 2024-02-16 21:21:25 +01:00
Łukasz Magiera
fe7558368a storage: Support C1 directly from sector storage 2024-02-16 21:21:25 +01:00
Łukasz Magiera
725460bc45 dbindex: Fix heartbeat checks 2024-02-16 21:21:25 +01:00
Łukasz Magiera
47672e0933 storage: Fix dbindex heartbeat 2024-02-16 21:20:13 +01:00
Andrew Jackson (Ajax)
96353e63ea exp backoff, short stack err 2023-12-11 10:50:49 -06:00
Andrew Jackson (Ajax)
0e49673c49 fix: db serialize txn - retry 2023-12-07 15:32:35 -06:00
Andrew Jackson (Ajax)
81e59b7fce fix:sector_filetype undef 2023-11-28 09:05:59 -06:00
Łukasz Magiera
f0a931edc9 fix: miner: Don't require db config when it's not used 2023-11-23 16:28:39 +01:00
Andrew Jackson (Ajax)
848c20d15f Merge branch 'master' into feat/sturdypost 2023-11-06 16:10:57 -06:00
Andrew Jackson (Ajax)
43680400c3 lp runs. todo ds-sectorInfo move & cleanups 2023-10-24 19:26:13 -05:00
Łukasz Magiera
d981baa362 storage: Return soft err when sector alloc fails in acquire 2023-10-17 11:42:41 +02:00
Andrew Jackson (Ajax)
a520ee85d6 master-to-sturdy 2023-09-21 10:37:02 -05:00
Łukasz Magiera
bcc2d7fd10 ffiwrapper: Fix RegenerateSectorKey 2023-08-31 09:46:10 -04:00
Shrenuj Bansal
15247486ae anotha one 2023-08-22 12:19:18 -04:00
Shrenuj Bansal
068416ecfc experiment 2023-08-22 11:48:38 -04:00
Shrenuj Bansal
459255d1f0 make linter happy 2023-08-22 11:06:16 -04:00
Shrenuj Bansal
a51e55390a address moar comments 2023-08-22 10:43:50 -04:00
Shrenuj Bansal
95709a0583 fix a couple of bugs and address comments 2023-08-21 11:54:51 -04:00
Shrenuj Bansal
9262eb4fc9 change table names and address other comments 2023-08-17 13:23:03 -04:00