Commit Graph

21361 Commits

Author SHA1 Message Date
Łukasz Magiera
9851d35a38
feat: curio: jsonrpc in webui (#11904)
* curioweb: Setup jsonrpc communication

* curioweb: Move chain state from SSE to rpc

* appease the linter
2024-05-24 17:48:43 +02:00
Łukasz Magiera
759709b768
fix: itests: Fix flaky curio itest (#12037)
* fix: itests: Fix flaky curio itest

* fix curio test with ffiselect

* try bigger instance for curio itest

* curio itest: reduce block time
2024-05-24 16:35:00 +02:00
LexLuthr
803acdfd16
feat: curio: wdPost and wnPost alerts (#12029)
* post alerts

* check missed posts

* fix tasks

* fix typo
2024-05-24 12:02:11 +02:00
yumeiyin
4088706697
chore: fix some function names (#12031) 2024-05-24 11:34:03 +10:00
LexLuthr
9983c466ec
fix logs (#12036) 2024-05-23 13:04:28 +02:00
Andrew Jackson (Ajax)
c1f99c58c5
feat: curio ffiselect: Isolate gpu calls in a subprocess (#11994)
* started

* so far so good

* builds and looks good

* changing level of abstration. some work remains

* it builds

* betterment

* import order

* 2

* stupid linter - you can cast a nil

* build commit and date

* nicer

* tmp and nide makefile

* comments handled

* oops

* added debug and reg

* ffiselect: change err encode to strings, fix some bugs

* ffiselect: Wrap rust logs into go-log

* ffiselect: Make the linter happy

* verification tests

* ffiselect: Fix startup

---------

Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
2024-05-23 13:03:59 +02:00
Peter Rabbitson
939154879b
src: chain: remove C dependency from builtin types (#12015)
Without this change the followung test fails:

CGO_ENABLED=0 go test -count=1 ./chain/types_test.go
2024-05-23 19:36:43 +10:00
Phi-rjan
1d220f24b5
Bump go-jsonrpc to v0.4.0 (#12034)
Bump go-jsonrpc to v0.4.0
2024-05-23 10:38:46 +02:00
Steven Allen
081efa47e7
fix: hello: avoid dialing when fetching hello tipset (#12032)
We should already be connected to this peer and we're only fetching this
tipset opportunistically. If we've already disconnected from them, move
on.
2024-05-23 08:44:52 +04:00
Łukasz Magiera
7dae7ddc23
fix: harmonydb: Use timestampz instead of timestamp across the schema (#12030)
* fix: harmonydb: Use timestampz instead of timestamp across the schema

* ci: Try YB 2.21

* try smaller alters
2024-05-22 21:41:02 +02:00
Piotr Galar
cfbe59d182
fix: ci: keep lotus checkout clean in the release workflow (#12028)
* ci: keep lotus checkout clean in the release workflow

* ci: allow providing custom ref to the release workflow

* ci: fix version check performed during the release

* ci: fix install go step of the release workflow

* ci: fix the working directory for the install go step in release workflow

* ci: provide github ref to lotus scripts explicitly

* ci: use actions from the chosen ref in release workflow

* ci: fix install go in release workflow

* ci: fix artifact upload in release workflow

* ci: set INPUTS_REF variable in release workflow

* ci: fix publish checksums script

* ci: allow releasing docker from an arbitrary ref

* ci: fix docker channel discovery
2024-05-22 14:47:29 +02:00
Andrew Jackson (Ajax)
75ca4dec4e
gui-listen fix (#12013)
* gui-listen fix

* fix test

---------

Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
2024-05-22 11:54:51 +02:00
LexLuthr
7e3846c669
feat: curio: allow multiple pieces per sector (#11935)
* multiple piece per sector, DDO deals

* in memory to DB

* sql parser

* add seal command

* multi piece TreeD

* redo filler pieces

* remove psql exception handling

* fix deal sectors porep

* fix tests

* ddo deals

* lower SDR cpu for test

* devnet cpu 0

* get params for itest

* fix itest sector size

* revert sdr devnet cpu

* improve SectorStatus API

* account for verified constraints
2024-05-22 10:08:32 +02:00
Steven Allen
967524aa83
chore: update yugabyte deps (#12022)
And handle a breaking change to the Rows interface.
2024-05-21 14:07:07 -07:00
Steven Allen
04ae41628b
chore: update drand (#12021) 2024-05-20 19:43:25 -07:00
Łukasz Magiera
db105f1cb6
fix: curio taskstorage: Don't try to free reservations by nulled TaskID (#12018) 2024-05-20 13:04:37 +02:00
LexLuthr
1afe58dcad
fix actor string (#12019) 2024-05-20 10:50:24 +00:00
LexLuthr
0da9646418
fix sector UI (#12016) 2024-05-20 07:56:46 +00:00
LexLuthr
bedd69602c
fix non existing sector (#12012) 2024-05-17 09:08:37 -05:00
Rod Vagg
6dc4ebd2ad doc: eth: restore comment lost in linter cleanup
Ref: https://github.com/filecoin-project/lotus/pull/11968
2024-05-17 16:19:17 +10:00
Rod Vagg
f098fc2a66 feat: api: add SectorNumber to MarketDealState (nv22) 2024-05-17 11:08:45 +10:00
Piotr Galar
3ee09115e6
ci: deprecate circle ci in favour of github actions (#11786) 2024-05-16 13:31:08 +02:00
Rod Vagg
3d6ae227ef chore: update go-data-transfer and go-graphsync 2024-05-16 18:08:27 +10:00
Łukasz Magiera
e14a85bcd3
fix: curio: Drop FKs from pipeline to fix retry loops (#11973) 2024-05-15 17:40:40 +02:00
Rod Vagg
b38dadb684 fix: copy Flags field from SectorOnChainInfo
Fixes: https://github.com/filecoin-project/lotus/issues/11962
2024-05-15 15:43:28 +10:00
Andrew Jackson (Ajax)
672864ad17
Remove "provider" relics (#11992)
* really get rid of provider this time

* one more
2024-05-14 14:50:31 -05:00
Aarsh Shah
af31126ea6
feat: libp2p: Lotus stream cleanup (#11993)
* set stream deadlines in Lotus

* reduce timeout

* whitelist bootstrappers

* fix tests
2024-05-14 14:12:03 +04:00
galargh
d0bbb0b20a ci: fix when sorted pr checks workflow is executed 2024-05-14 13:42:58 +10:00
Phi-rjan
74172ba8f8
Revert "Update SupportedProofTypes (#11988)" (#11990)
This reverts commit 7e005be751.
2024-05-14 03:05:42 +02:00
ZenGround0
84b08fc25f
feat:state:Ignore market balance after nv23 (#11976)
* Ignore market balance after nv23

* Review Response and failing test

* Fixes

* Test passing

* Bump up threshold

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2024-05-13 17:53:20 -04:00
Andrew Jackson (Ajax)
f5dd7665f6
Produce DEB files for amd64 for openCL and cuda (#11885)
* service changes

* better postinst

* better postinst

* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* ready to test deb making

* deb files build

* it works

* fullname

* last bit of cleanup

* doc update

* future plans

* lint

* code comments

---------

Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2024-05-13 14:18:50 -05:00
Phi-rjan
7e005be751
Update SupportedProofTypes (#11988)
Update SupportedProofTypes to the correct ones.
2024-05-13 09:43:43 +02:00
Phi-rjan
af4b68023a
chore: remove unmaintained bootstrappers (#11983)
* chore: remove unmaintained bootstrappers

chore: remove unmaintained bootstrappers

* Update mainnet.pi fixing typoed domain

fixing typo for 1475.io 'bootstarp' -> 'bootstrap'

* Update mainnet.pi

apparently the actual hostname is typoed. so bootstarp it is.

---------

Co-authored-by: smagdali <stefan@fil.org>
2024-05-13 07:24:26 +02:00
Rod Vagg
1882525578 chore: lint: address feedback from reviews 2024-05-13 14:52:09 +10:00
Rod Vagg
26d3fd2ecc chore: lint: fix lint errors with new linting config
Ref: https://github.com/filecoin-project/lotus/issues/11967
2024-05-13 14:52:09 +10:00
Rod Vagg
6a8110322d chore: lint: update golangci lint config
* remove and replace some linters
* remove some exclusions
* make all exclusions more explicit matches
2024-05-13 14:52:09 +10:00
Andrew Jackson (Ajax)
f5b6c4edc7
curio cfg edit: ux cleanups (#11985)
* fixed popper and bools

* local fonts

* dark select boxes
2024-05-10 11:24:14 -05:00
Rod Vagg
4f70204342 docs: rpc: document go-jsonrpc behaviour change 2024-05-09 15:46:02 +10:00
Hubert Bugaj
6fa6a0458a fix: update go-jsonrpc to v0.3.2 2024-05-09 10:09:09 +10:00
Aarsh Shah
6b353e56f3
add ETH addrs API to Gateway (#11979) 2024-05-08 15:35:00 +02:00
LexLuthr
22ccaf91ef
feat: curio: alertManager (#11926)
* add alert task

* maxbalance config, storage alert

* docs gen

* improve storage alert

* fix empty alert
2024-05-08 15:08:15 +02:00
Łukasz Magiera
33876bf81c
fix: curio/lmrpc: Check ParkPiece success before creating sectors (#11975) 2024-05-08 13:22:23 +02:00
LexLuthr
ebf1e00846
feat: curio: docker devnet (#11954)
* curio docker devnet

* fix UI, improve init, compose

* fix TreeRC

* bump FFI
2024-05-07 17:20:00 +02:00
Łukasz Magiera
01ec229749
feat: curio: Keep more sector metadata in the DB long-term (#11933)
* feat: curio: Migrate lotus-miner sector metadata into Curio DB

* curio seal: Transfer seal pipeline entries to long-term sector metadata table

* curio: Only open db in sectors migrate cmd

* curio: Add an envvar to force migration

* curio: Debugging sector migration

* curio: Fix typo in table name

* curio: Plumb KeepUnsealed into the sealing pipeline

* Don't add redundant keep_data to sdr pipeline pieces table

* fix transferFinalizedSectorData where check

* ui for sector fail

---------

Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
2024-05-07 13:03:51 +02:00
Phi-rjan
2d57bfe273
feat: Skeleton for nv23 (#11964)
* Use local GST

Use local GST

* Import actors and define upgrade heights

Creatin a mock actor-bundle and define upgrade heights

* Generate adapters

Updates gen/inlinegen-data.json, and runs make actors-gen

* Add schedule and migration

Add schedule and migration

* Add upgrade and network version fields/params

Add upgrade and network version fields/params

* Run make gen / make docsgen-cli

Run make gen / make docsgen-cli

* Update filecoin-ffi

Update filecoin-ffi to the v1.28.0-dev tag, which supports the nv23 skeleton.

* Update GST to v0.14.0-dev

Update GST to v0.14.0-dev, which includes the nv23 skeleton

* Add support for actor version 14 in actor registry

Add support for actor version 14 in actor registry
2024-05-07 09:40:49 +02:00
Phi-rjan
d6d4fee4b6
Bump pubsub-dep (#11966)
Bump pubsub-dep
2024-05-06 15:27:54 +02:00
Phi-rjan
1b2dde1e65
chore: docs: Update skeleton guide (#11960)
* Updates to nv-skeleton guide

Updates to nv-skeleton guide

* Add link to FVM crates checklist

Add link to FVM crates checklist
2024-05-04 10:52:58 +02:00
Andrew Jackson (Ajax)
441f5995d3
fix: curio: node UI & darwin gpu count (#11950)
* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* 1

* relatable

* add and delete layer

* chore: curio: merge master (#11956)

* Fixing dead links (#11907)

* ci: ci: create gh workflow that updates sorted pr checks (#11861)

* ci: create gh workflow that updates sorted pr checks

* ci: use grouped_by_result template for pr checks sticky comment

* chore: apply pr review suggestion

* Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Fix mismatched method names in comments (#11913)

Signed-off-by: forcedebug <forcedebug@outlook.com>

* release: v1.26.3 (#11908) (#11915) (#11922)

* deps: update dependencies to address migration memory bloat

to address memory concerns during a heavy migration

Ref: https://github.com/filecoin-project/go-state-types/pull/260
Ref: https://github.com/whyrusleeping/cbor-gen/pull/96
Ref: https://github.com/filecoin-project/go-amt-ipld/pull/90

* release: prep v1.26.3 patch

Prep v1.26.3 patch release:
- Update changelog, version and make gen + make docsgen-cli

* deps: update cbor-gen to tagged version

deps: update cbor-gen to tagged version

* deps: update go-state-types to tagged version

deps: update go-state-types to tagged version v0.13.2

* chore: deps: update go-state-types to v0.13.3

Fixes a panic when we have fewer than 1k proposals.

---------

Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>

* Refactor `LookupID*` APIs in `StateManager` and `StateTree`

The naming of `LookupID` can cause confusion when resolving actor IDs vs
 ID addresses. To avoid this:

* Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it
returns ID address.
* Refactor `StateManager` `LookupID` to
`LookupIDAddress` because it also returns ID address via a chain call to
`StateTree`.
* Introduce a new API `StateManager` dedicated to resolving address to
actor ID, called `LookupID` which returns `abi.ActorID`.

For context, see:
 * https://github.com/filecoin-project/lotus/pull/11723#discussion_r1534728607

* Add v13 support to invariants-checker (#11931)

Add v13 support to invariants-checker

* chore: docs: nv-skeleton documentation (#11065)

* nv-skeleton documentation

Add a tutorial for how one can create a nv-skeleton in Lotus

* Add footnote for `Add migration` step

Add footnote for `Add migration` step

* Indent migration-code

Indent migration-code to make it show properly as a footnote.

* Add ref-fvm and filecoin-ffi checklist

Add ref-fvm and filecoin-ffi checklist

* Add Filecoin-FFI steps

Add Filecoin-FFI steps

* Add step to params_butterfly.go

Add step to params_butterfly.go

* Fix typo

Fix typo

* Add links to reference PRs

Add links to reference PRs

* Update ref-fvm list

Update ref-fvm list

* feat: curio: add StorageInit api (#11918)

* feat: add StorageInit api

* remove unused variables

* fix gen check

* feat: curio: simpler reservation release logic (#11900)

* simpler release logic

* oops, plus simpler

* simpler

* fix NewLine (#11893)

* fix(events): check for sync-in-progress (#11932)

* feat(events): adjust indexes in event index db to match query patterns

Introduces a v4 migration that just adjusts indexes.

Copies some improvements from https://github.com/filecoin-project/lotus/pull/11723

Closes: https://github.com/filecoin-project/lotus/issues/11909

* fix(pipeline): should return if error occurred when get network version (#11902)

* fix(events): correct log msg for v4 events index db migration

* chore: remove duplicate words in strings and comments

* fix(events): register events index db migration v4

* fix: curio seal: Failed commit retry strategy (#11870)

* ffi: improved-error-handling

* curio seal: Failed commit retry strategy

* use master ffi

* mod tidy

* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* mod tidy

* Event index should be unique for tipsets (#11952)

* event index should be unique for tipsets

* fix formatting

* migrate to version 5

* chore: bump build version in master (#11946)

* Bump version

Bump version in master branch in preperation for cutting v1.27.0-rc1

* chore: bump build-version

chore: bump build-version

* feat: curioweb: Show piece info on the sector page (#11955)

* curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)

* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>

* linter oops

* gen cleanup

* fix

* named returns are confusing

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2024-05-04 10:35:30 +02:00
Piotr Galar
b50049a387
fix: ci: use filecoin-ffi hash to cache make deps outputs (#11961) 2024-05-03 18:31:52 +02:00
LexLuthr
00edad4e4d
curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)
* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release
2024-05-02 11:17:33 +02:00