Commit Graph

110 Commits

Author SHA1 Message Date
Łukasz Magiera
7dae7ddc23
fix: harmonydb: Use timestampz instead of timestamp across the schema (#12030)
* fix: harmonydb: Use timestampz instead of timestamp across the schema

* ci: Try YB 2.21

* try smaller alters
2024-05-22 21:41:02 +02:00
LexLuthr
7e3846c669
feat: curio: allow multiple pieces per sector (#11935)
* multiple piece per sector, DDO deals

* in memory to DB

* sql parser

* add seal command

* multi piece TreeD

* redo filler pieces

* remove psql exception handling

* fix deal sectors porep

* fix tests

* ddo deals

* lower SDR cpu for test

* devnet cpu 0

* get params for itest

* fix itest sector size

* revert sdr devnet cpu

* improve SectorStatus API

* account for verified constraints
2024-05-22 10:08:32 +02:00
Steven Allen
967524aa83
chore: update yugabyte deps (#12022)
And handle a breaking change to the Rows interface.
2024-05-21 14:07:07 -07:00
Łukasz Magiera
e14a85bcd3
fix: curio: Drop FKs from pipeline to fix retry loops (#11973) 2024-05-15 17:40:40 +02:00
Łukasz Magiera
01ec229749
feat: curio: Keep more sector metadata in the DB long-term (#11933)
* feat: curio: Migrate lotus-miner sector metadata into Curio DB

* curio seal: Transfer seal pipeline entries to long-term sector metadata table

* curio: Only open db in sectors migrate cmd

* curio: Add an envvar to force migration

* curio: Debugging sector migration

* curio: Fix typo in table name

* curio: Plumb KeepUnsealed into the sealing pipeline

* Don't add redundant keep_data to sdr pipeline pieces table

* fix transferFinalizedSectorData where check

* ui for sector fail

---------

Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
2024-05-07 13:03:51 +02:00
Łukasz Magiera
ecc82d4526
feat: curioweb: Show piece info on the sector page (#11955) 2024-05-02 10:26:35 +02:00
Łukasz Magiera
c2dd67403b feat: curioweb: Improve task_history indexes (#11911) 2024-04-30 20:43:56 +02:00
Andrew Jackson (Ajax)
b9c06b64d7 1 2024-04-18 12:27:37 +02:00
Łukasz Magiera
c785e59371
feat: curio: storage index gc task (#11884)
* curio storage path gc: lay out the structure

* curio gc: Implement storage metadata gc

* move bored singleton task impl to harmonytask

* curio: run storage gc task on storage node

* make gen
2024-04-16 16:34:48 -05:00
Andrew Jackson (Ajax)
fd7f1a95e2
feat: curio: web based config edit (#11822)
* cfg edit 1

* jsonschema deps

* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* part 2 of last

* message

* merge addtl

* fixing guided setup for myself

* warn-on-no-post

* EditorLoads

* cleanups and styles

* create info

* fix tests

* make gen

* change layout, add help button

* Duration custom json

* mjs naming

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
2024-04-16 09:30:27 -05:00
LexLuthr
243bf1a0b3 refactor MinerFilter func, update comments 2024-04-04 18:06:19 +02:00
LexLuthr
2951d038a8 add miner fliter 2024-04-04 18:06:19 +02:00
Łukasz Magiera
23a3c74720
feat: curio: Add schemas for DDO deal support (#11805) 2024-04-02 11:16:59 -05:00
Łukasz Magiera
ea19e1df6b chore: revert accidental direct commits to master 2024-03-21 13:51:35 +01:00
Andrew Jackson (Ajax)
99158dfd87 Merge branch 'master' of https://github.com/filecoin-project/lotus 2024-03-20 18:11:08 -05:00
Andrew Jackson (Ajax)
32efba72cf harmonydb safety 2024-03-20 18:11:05 -05:00
Łukasz Magiera
51b59e3b81 curio web: fix capitalization in 20240317-web-summary-index.sql 2024-03-19 09:30:25 +01:00
Łukasz Magiera
49712a6e88 curio web: Show SP wins in actor summary 2024-03-19 09:30:25 +01:00
Łukasz Magiera
25711c5ce4 curio web: Task summaries in Cluster machines category 2024-03-19 09:30:25 +01:00
Łukasz Magiera
3c2be70d59 Merge remote-tracking branch 'origin/master' into feat/lpdeal-cache 2024-03-17 17:31:26 +01:00
Andrew Jackson (Ajax)
81ba6ab6f0
feat: Curio - Easy Migration (#11617)
* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* review fixes

* fix backup path

* fixes1

* fixes2

* fixes 3

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
2024-03-15 16:38:13 -05:00
Łukasz Magiera
5dfec4ab36 lppiece: Fix piece cleanup task 2024-03-14 21:30:38 +01:00
Łukasz Magiera
465ec58a7c lpdeal: Fix adapter deadlock with duplicate pieces 2024-03-14 13:03:14 +01:00
Łukasz Magiera
27317a9489 fix 20240228-piece-park.sql 2024-03-14 12:24:00 +01:00
Łukasz Magiera
e93a3e0d4a lppiece: GC task 2024-03-14 09:36:51 +01:00
Łukasz Magiera
1bb228898e Integrate PiecePark into boost-proxy 2024-03-14 09:36:51 +01:00
Łukasz Magiera
0800e6e5a7 make gen 2024-03-14 09:36:47 +01:00
Łukasz Magiera
e060cd2f37 lppiece: Implement Piece Park 2024-03-14 09:36:03 +01:00
Łukasz Magiera
4ae73f059c provider: Fix sql query capitalization 2024-02-21 14:19:48 +01:00
Łukasz Magiera
94a31c6db6 harmony: BeginTransaction Retry Serialization option 2024-02-20 13:14:58 +01:00
Łukasz Magiera
4f8d6add1d lpseal: Make tasks return to poller when out of retries 2024-02-16 21:22:15 +01:00
Łukasz Magiera
d820e76beb lpseal: MoveStorage 2024-02-16 21:21:38 +01:00
Łukasz Magiera
4649d3b8d0 lpseal: Finalize 2024-02-16 21:21:38 +01:00
Łukasz Magiera
12db86c0aa lpseal: Seal sectors with data 2024-02-16 21:21:38 +01:00
Łukasz Magiera
0337d0198f lpseal: Basic deal-accepting logic 2024-02-16 21:21:36 +01:00
Łukasz Magiera
f2e1b5a35c lpseal: Poller for PoRep 2024-02-16 21:21:25 +01:00
Łukasz Magiera
0244c7df50 lpseal: Wire up SubmitPrecommit 2024-02-16 21:21:21 +01:00
Łukasz Magiera
2f45646a92 lpseal: write precommit message to message_waits 2024-02-16 21:20:14 +01:00
Łukasz Magiera
e743db5fee lpmessage: Scaffold waiter 2024-02-16 21:20:13 +01:00
Łukasz Magiera
04f5a595e4 lpseal: Fix poller 2024-02-16 21:20:13 +01:00
Łukasz Magiera
44b441d7ad lpseal: Implement SDR for CC sectors 2024-02-16 21:20:13 +01:00
Łukasz Magiera
e89fff2308 lpseal: Sector number allocator, pipeline start cli 2024-02-16 21:20:13 +01:00
Łukasz Magiera
67fb3bd27f begin drafting lotus-provider SDR pipeline 2024-02-16 21:20:13 +01:00
Andrew Jackson (Ajax)
1f3abf4995 also-sql naming 2024-01-29 18:22:50 -06:00
Andrew Jackson (Ajax)
de38e77cfc tx-detector use atomic 2023-12-11 11:31:38 -06:00
Andrew Jackson (Ajax)
96353e63ea exp backoff, short stack err 2023-12-11 10:50:49 -06:00
Andrew Jackson (Ajax)
9dca4346b1 fix: lint 2023-12-09 11:20:41 -06:00
Andrew Jackson (Ajax)
1e09e1e966 detect unsafe code uses 2023-12-07 16:01:28 -06:00
Andrew Jackson (Ajax)
0e49673c49 fix: db serialize txn - retry 2023-12-07 15:32:35 -06:00
Łukasz Magiera
c113dda590 harmony: fix alter sector_location 2023-12-01 23:12:53 +01:00