Commit Graph

21322 Commits

Author SHA1 Message Date
Aarsh Shah
6b353e56f3
add ETH addrs API to Gateway (#11979) 2024-05-08 15:35:00 +02:00
LexLuthr
22ccaf91ef
feat: curio: alertManager (#11926)
* add alert task

* maxbalance config, storage alert

* docs gen

* improve storage alert

* fix empty alert
2024-05-08 15:08:15 +02:00
Łukasz Magiera
33876bf81c
fix: curio/lmrpc: Check ParkPiece success before creating sectors (#11975) 2024-05-08 13:22:23 +02:00
LexLuthr
ebf1e00846
feat: curio: docker devnet (#11954)
* curio docker devnet

* fix UI, improve init, compose

* fix TreeRC

* bump FFI
2024-05-07 17:20:00 +02:00
Łukasz Magiera
01ec229749
feat: curio: Keep more sector metadata in the DB long-term (#11933)
* feat: curio: Migrate lotus-miner sector metadata into Curio DB

* curio seal: Transfer seal pipeline entries to long-term sector metadata table

* curio: Only open db in sectors migrate cmd

* curio: Add an envvar to force migration

* curio: Debugging sector migration

* curio: Fix typo in table name

* curio: Plumb KeepUnsealed into the sealing pipeline

* Don't add redundant keep_data to sdr pipeline pieces table

* fix transferFinalizedSectorData where check

* ui for sector fail

---------

Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
2024-05-07 13:03:51 +02:00
Phi-rjan
2d57bfe273
feat: Skeleton for nv23 (#11964)
* Use local GST

Use local GST

* Import actors and define upgrade heights

Creatin a mock actor-bundle and define upgrade heights

* Generate adapters

Updates gen/inlinegen-data.json, and runs make actors-gen

* Add schedule and migration

Add schedule and migration

* Add upgrade and network version fields/params

Add upgrade and network version fields/params

* Run make gen / make docsgen-cli

Run make gen / make docsgen-cli

* Update filecoin-ffi

Update filecoin-ffi to the v1.28.0-dev tag, which supports the nv23 skeleton.

* Update GST to v0.14.0-dev

Update GST to v0.14.0-dev, which includes the nv23 skeleton

* Add support for actor version 14 in actor registry

Add support for actor version 14 in actor registry
2024-05-07 09:40:49 +02:00
Phi-rjan
d6d4fee4b6
Bump pubsub-dep (#11966)
Bump pubsub-dep
2024-05-06 15:27:54 +02:00
Phi-rjan
1b2dde1e65
chore: docs: Update skeleton guide (#11960)
* Updates to nv-skeleton guide

Updates to nv-skeleton guide

* Add link to FVM crates checklist

Add link to FVM crates checklist
2024-05-04 10:52:58 +02:00
Andrew Jackson (Ajax)
441f5995d3
fix: curio: node UI & darwin gpu count (#11950)
* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* 1

* relatable

* add and delete layer

* chore: curio: merge master (#11956)

* Fixing dead links (#11907)

* ci: ci: create gh workflow that updates sorted pr checks (#11861)

* ci: create gh workflow that updates sorted pr checks

* ci: use grouped_by_result template for pr checks sticky comment

* chore: apply pr review suggestion

* Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Fix mismatched method names in comments (#11913)

Signed-off-by: forcedebug <forcedebug@outlook.com>

* release: v1.26.3 (#11908) (#11915) (#11922)

* deps: update dependencies to address migration memory bloat

to address memory concerns during a heavy migration

Ref: https://github.com/filecoin-project/go-state-types/pull/260
Ref: https://github.com/whyrusleeping/cbor-gen/pull/96
Ref: https://github.com/filecoin-project/go-amt-ipld/pull/90

* release: prep v1.26.3 patch

Prep v1.26.3 patch release:
- Update changelog, version and make gen + make docsgen-cli

* deps: update cbor-gen to tagged version

deps: update cbor-gen to tagged version

* deps: update go-state-types to tagged version

deps: update go-state-types to tagged version v0.13.2

* chore: deps: update go-state-types to v0.13.3

Fixes a panic when we have fewer than 1k proposals.

---------

Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>

* Refactor `LookupID*` APIs in `StateManager` and `StateTree`

The naming of `LookupID` can cause confusion when resolving actor IDs vs
 ID addresses. To avoid this:

* Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it
returns ID address.
* Refactor `StateManager` `LookupID` to
`LookupIDAddress` because it also returns ID address via a chain call to
`StateTree`.
* Introduce a new API `StateManager` dedicated to resolving address to
actor ID, called `LookupID` which returns `abi.ActorID`.

For context, see:
 * https://github.com/filecoin-project/lotus/pull/11723#discussion_r1534728607

* Add v13 support to invariants-checker (#11931)

Add v13 support to invariants-checker

* chore: docs: nv-skeleton documentation (#11065)

* nv-skeleton documentation

Add a tutorial for how one can create a nv-skeleton in Lotus

* Add footnote for `Add migration` step

Add footnote for `Add migration` step

* Indent migration-code

Indent migration-code to make it show properly as a footnote.

* Add ref-fvm and filecoin-ffi checklist

Add ref-fvm and filecoin-ffi checklist

* Add Filecoin-FFI steps

Add Filecoin-FFI steps

* Add step to params_butterfly.go

Add step to params_butterfly.go

* Fix typo

Fix typo

* Add links to reference PRs

Add links to reference PRs

* Update ref-fvm list

Update ref-fvm list

* feat: curio: add StorageInit api (#11918)

* feat: add StorageInit api

* remove unused variables

* fix gen check

* feat: curio: simpler reservation release logic (#11900)

* simpler release logic

* oops, plus simpler

* simpler

* fix NewLine (#11893)

* fix(events): check for sync-in-progress (#11932)

* feat(events): adjust indexes in event index db to match query patterns

Introduces a v4 migration that just adjusts indexes.

Copies some improvements from https://github.com/filecoin-project/lotus/pull/11723

Closes: https://github.com/filecoin-project/lotus/issues/11909

* fix(pipeline): should return if error occurred when get network version (#11902)

* fix(events): correct log msg for v4 events index db migration

* chore: remove duplicate words in strings and comments

* fix(events): register events index db migration v4

* fix: curio seal: Failed commit retry strategy (#11870)

* ffi: improved-error-handling

* curio seal: Failed commit retry strategy

* use master ffi

* mod tidy

* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* mod tidy

* Event index should be unique for tipsets (#11952)

* event index should be unique for tipsets

* fix formatting

* migrate to version 5

* chore: bump build version in master (#11946)

* Bump version

Bump version in master branch in preperation for cutting v1.27.0-rc1

* chore: bump build-version

chore: bump build-version

* feat: curioweb: Show piece info on the sector page (#11955)

* curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)

* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>

* linter oops

* gen cleanup

* fix

* named returns are confusing

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2024-05-04 10:35:30 +02:00
Piotr Galar
b50049a387
fix: ci: use filecoin-ffi hash to cache make deps outputs (#11961) 2024-05-03 18:31:52 +02:00
LexLuthr
00edad4e4d
curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)
* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release
2024-05-02 11:17:33 +02:00
Łukasz Magiera
ecc82d4526
feat: curioweb: Show piece info on the sector page (#11955) 2024-05-02 10:26:35 +02:00
Phi-rjan
ed9b1b1bba
chore: bump build version in master (#11946)
* Bump version

Bump version in master branch in preperation for cutting v1.27.0-rc1

* chore: bump build-version

chore: bump build-version
2024-05-02 07:26:04 +02:00
Aarsh Shah
6bbe090411
Event index should be unique for tipsets (#11952)
* event index should be unique for tipsets

* fix formatting

* migrate to version 5
2024-05-01 13:20:41 +04:00
Łukasz Magiera
10f7b6ed5f mod tidy 2024-04-30 20:43:56 +02:00
Łukasz Magiera
c2dd67403b feat: curioweb: Improve task_history indexes (#11911) 2024-04-30 20:43:56 +02:00
Łukasz Magiera
1b6bffd339 fix: curio: Update pgx imports, fix db_storage alloc 2024-04-30 20:43:56 +02:00
Łukasz Magiera
914a65cea6
fix: curio seal: Failed commit retry strategy (#11870)
* ffi: improved-error-handling

* curio seal: Failed commit retry strategy

* use master ffi

* mod tidy
2024-04-30 11:16:44 +02:00
Rod Vagg
46992f0fd2 fix(events): register events index db migration v4 2024-04-30 17:29:23 +10:00
Rod Vagg
cee77aa415 chore: remove duplicate words in strings and comments 2024-04-30 16:04:38 +10:00
Rod Vagg
aa76a45086 fix(events): correct log msg for v4 events index db migration 2024-04-29 20:21:56 +10:00
0x5459
15f5f4745d
fix(pipeline): should return if error occurred when get network version (#11902) 2024-04-29 12:21:40 +02:00
Rod Vagg
10877d2e66 feat(events): adjust indexes in event index db to match query patterns
Introduces a v4 migration that just adjusts indexes.

Copies some improvements from https://github.com/filecoin-project/lotus/pull/11723

Closes: https://github.com/filecoin-project/lotus/issues/11909
2024-04-29 13:21:35 +10:00
Rod Vagg
ae2cc67f96
fix(events): check for sync-in-progress (#11932) 2024-04-29 12:19:58 +10:00
beck
0a28ce471e
fix NewLine (#11893) 2024-04-27 11:17:00 +02:00
Andrew Jackson (Ajax)
5e1d8f661e
feat: curio: simpler reservation release logic (#11900)
* simpler release logic

* oops, plus simpler

* simpler
2024-04-26 11:11:42 +02:00
Lee
0f7c74e499
feat: curio: add StorageInit api (#11918)
* feat: add StorageInit api

* remove unused variables

* fix gen check
2024-04-26 10:26:57 +02:00
Phi-rjan
ec6d3e1483
chore: docs: nv-skeleton documentation (#11065)
* nv-skeleton documentation

Add a tutorial for how one can create a nv-skeleton in Lotus

* Add footnote for `Add migration` step

Add footnote for `Add migration` step

* Indent migration-code

Indent migration-code to make it show properly as a footnote.

* Add ref-fvm and filecoin-ffi checklist

Add ref-fvm and filecoin-ffi checklist

* Add Filecoin-FFI steps

Add Filecoin-FFI steps

* Add step to params_butterfly.go

Add step to params_butterfly.go

* Fix typo

Fix typo

* Add links to reference PRs

Add links to reference PRs

* Update ref-fvm list

Update ref-fvm list
2024-04-25 15:46:13 -04:00
Phi-rjan
65edab42bc
Add v13 support to invariants-checker (#11931)
Add v13 support to invariants-checker
2024-04-25 17:13:18 +02:00
Masih H. Derkani
f907354300 Refactor LookupID* APIs in StateManager and StateTree
The naming of `LookupID` can cause confusion when resolving actor IDs vs
 ID addresses. To avoid this:

* Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it
returns ID address.
* Refactor `StateManager` `LookupID` to
`LookupIDAddress` because it also returns ID address via a chain call to
`StateTree`.
* Introduce a new API `StateManager` dedicated to resolving address to
actor ID, called `LookupID` which returns `abi.ActorID`.

For context, see:
 * https://github.com/filecoin-project/lotus/pull/11723#discussion_r1534728607
2024-04-24 15:25:48 +01:00
Jiaying Wang
d23ea7665b
release: v1.26.3 (#11908) (#11915) (#11922)
* deps: update dependencies to address migration memory bloat

to address memory concerns during a heavy migration

Ref: https://github.com/filecoin-project/go-state-types/pull/260
Ref: https://github.com/whyrusleeping/cbor-gen/pull/96
Ref: https://github.com/filecoin-project/go-amt-ipld/pull/90

* release: prep v1.26.3 patch

Prep v1.26.3 patch release:
- Update changelog, version and make gen + make docsgen-cli

* deps: update cbor-gen to tagged version

deps: update cbor-gen to tagged version

* deps: update go-state-types to tagged version

deps: update go-state-types to tagged version v0.13.2

* chore: deps: update go-state-types to v0.13.3

Fixes a panic when we have fewer than 1k proposals.

---------

Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
2024-04-23 23:04:32 -04:00
forcedebug
772f6a38c5
Fix mismatched method names in comments (#11913)
Signed-off-by: forcedebug <forcedebug@outlook.com>
2024-04-23 02:56:43 +02:00
ZenGround0
95b6483ad9
Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2024-04-22 20:01:26 -04:00
Piotr Galar
70e0767acf
ci: ci: create gh workflow that updates sorted pr checks (#11861)
* ci: create gh workflow that updates sorted pr checks

* ci: use grouped_by_result template for pr checks sticky comment

* chore: apply pr review suggestion
2024-04-22 11:50:55 +02:00
parthshah1
06761db7bf
Fixing dead links (#11907) 2024-04-20 09:58:17 +02:00
Andrew Jackson (Ajax)
1a789d3acb
feat: curioweb: Improve UX, add top menu (#11901)
* cfg edit 1

* jsonschema deps

* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* part 2 of last

* message

* merge addtl

* fixing guided setup for myself

* warn-on-no-post

* EditorLoads

* cleanups and styles

* create info

* fix tests

* make gen

* sector early bird

* sectors v2

* sector termination v1

* terminate2

* mjs

* minor things

* flag bad sectors

* fix errors

* add dealweight and deals

* change column width

* ui looking better

* cleanups

* fix pipeline page

* comments

* curioweb: Add missing sector info file

* curioweb: fix hapi root template

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
2024-04-19 19:08:21 +04:00
Łukasz Magiera
445f1d4b33 Bump storage endpoint GC interval 2024-04-19 16:13:09 +02:00
Łukasz Magiera
be00ba996e Cleanup sector_path_url_liveness when not removing the whole path 2024-04-19 16:13:09 +02:00
Łukasz Magiera
7c292905a8 curio: Make sector index GC work 2024-04-19 16:13:09 +02:00
LexLuthr
103a7e0697 apply suggestions 2024-04-19 16:07:01 +02:00
LexLuthr
72fac27a4d account for cc sectors 2024-04-19 16:07:01 +02:00
LexLuthr
c8dec8de65 add allocation check for commit 2024-04-19 16:07:01 +02:00
Jon
031802eb08
fix(conformance): pass Timestamp through ExecuteMessageParams (#11891) 2024-04-19 12:06:41 +10:00
fuyangpengqi
5f7aaaa8f9
docs: fix function names in comments (#11898)
Signed-off-by: fuyangpengqi <995764973@qq.com>
2024-04-19 12:00:02 +10:00
Andrew Jackson (Ajax)
6b3e9b109f
feat: curio: sectors UI (#11869)
* cfg edit 1

* jsonschema deps

* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* part 2 of last

* message

* merge addtl

* fixing guided setup for myself

* warn-on-no-post

* EditorLoads

* cleanups and styles

* create info

* fix tests

* make gen

* sector early bird

* sectors v2

* sector termination v1

* terminate2

* mjs

* minor things

* flag bad sectors

* fix errors

* add dealweight and deals

* change column width

* refactor sql, handle sealing sectors

* fix estimates

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
2024-04-18 14:57:29 -05:00
Andrew Jackson (Ajax)
edd9c82bc1 mod tidy 2024-04-18 12:27:37 +02:00
Andrew Jackson (Ajax)
b9c06b64d7 1 2024-04-18 12:27:37 +02:00
writegr
52a190aa7b
chore: fix some typos in comments (#11892)
Signed-off-by: writegr <wellweek@outlook.com>
2024-04-18 10:34:20 +02:00
Andrew Jackson (Ajax)
1cdca19a85 oops 2024-04-17 12:01:01 +02:00
Łukasz Magiera
c785e59371
feat: curio: storage index gc task (#11884)
* curio storage path gc: lay out the structure

* curio gc: Implement storage metadata gc

* move bored singleton task impl to harmonytask

* curio: run storage gc task on storage node

* make gen
2024-04-16 16:34:48 -05:00