Commit Graph

129 Commits

Author SHA1 Message Date
Aarsh Shah
48287574d8
fix: ETH RPC API: ETH Call should use the parent state root of the subsequent tipset (#11905)
* fix eth call

* tests

* changes as per review

* changes as per review

* Update node/impl/full/eth.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix as per review

---------

Co-authored-by: Rod Vagg <rod@vagg.org>
2024-06-11 19:07:17 +04:00
Aayush Rajasekaran
277cbf9229
fix: stmgr: do not use cached migration results if absent from the blockstore (#11663) 2024-03-14 11:04:13 -07:00
Phi-rjan
6f7498b622
chore: Merge nv22 into master (#11699)
* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* feat: drand: refactor round verification

* feat: sealing: Support nv22 DDO features in the sealing pipeline (#11226)

* Initial work supporting DDO pieces in lotus-miner

* sealing: Update pipeline input to operate on UniversalPiece

* sealing: Update pipeline checks/sealing states to operate on UniversalPiece

* sealing: Make pipeline build with UniversalPiece

* move PieceDealInfo out of api

* make gen

* make sealing pipeline unit tests pass

* fix itest ensemble build

* don't panic in SectorsStatus with deals

* stop linter from complaining about checkPieces

* fix sector import tests

* mod tidy

* sealing: Add logic for (pre)committing DDO sectors

* sealing: state-types with method defs

* DDO non-snap pipeline works(?), DDO Itests

* DDO support in snapdeals pipeline

* make gen

* update actor bundles

* update the gst market fix

* fix: chain: use PreCommitSectorsBatch2 when setting up genesis

* some bug fixes

* integration working changes

* update actor bundles

* Make TestOnboardRawPieceSnap pass

* Appease the linter

* Make deadlines test pass with v12 actors

* Update go-state-types, abstract market DealState

* make gen

* mod tidy, lint fixes

* Fix some more tests

* Bump version in master

Bump version in master

* Make gen

Make gen

* fix sender

* fix: lotus-provider: Fix winning PoSt

* fix: sql Scan cannot write to an object

* Actually show miner-addrs in info-log

Actually show miner-addrs in lotus-provider info-log

* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* ddo is now nv22

* make gen

* temp actor bundle with ddo

* use working go-state-types

* gst with v13 market migration

* update bundle, builtin.MethodsMiner.ProveCommitSectors2 -> 3

* actually working v13 migration, v13 migration itest

* Address review

* sealing: Correct DDO snap pledge math

* itests: Mixed ddo itest

* pipeline: Fix sectorWeight

* sealing: convert market deals into PAMs in mixed sectors

* sealing: make market to ddo conversion work

* fix lint

* update gst

* Update actors and GST to lastest integ branch

* commit batcher: Update ProveCommitSectors3Params builder logic

* make gen

* use builtin-actors master

* ddo: address review

* itests: Add commd assertions to ddo tests

* make gen

* gst with fixed types

* config knobs for RequireActivationSuccess

* storage: Drop obsolete flaky tasts

---------

Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>

* feat: implement FIP-0063

* chore: deps: update to go-multiaddr v0.12.2 (#11602)

* feat: fvm: update the FVM/FFI to v4.1 (#11608) (#11612)

This:

1. Adds nv22 support.
2. Updates the message tracing format.

Co-authored-by: Steven Allen <steven@stebalien.com>

* AggregateProofType nil when doing batch updates

Use latest nv22 go-state-types version with matching update

* Update to v13.0.0-rc.2 bundle

* chore: Upgrade heights and codename

Update upgrade heights

Co-Authored-By: Steven Allen <steven@stebalien.com>

* Update epoch after nv22 DRAND switch

Update epoch after nv22 DRAND switch

* Update Mango codename to Phoneix

Make the codename for the Drand-change inline with Dragon style.

* Add UpgradePhoenixHeight to API params

* set UpgradePhoenixHeight to be one hour after Dragon

* Make gen

Make gen and UpgradePhoenixHeight in butterfly and local devnet to be in line with Calibration and Mainnet

* Update epoch heights (#11637)

Update epoch heights

* new: add forest bootstrap nodes (#11636)

Signed-off-by: samuelarogbonlo <sbayo971@gmail.com>

* Merge pull request #11491 from filecoin-project/fix/remove-decommissioned-pl-bootstrap-nodes

Remove PL operated bootstrap nodes from mainnet.pi

* feat: api: new verified registry methods to get all allocations and claims (#11631)

* new verireg methods

* update changelog and add itest

* update itest and cli

* update new method's support till v9

* remove gateway APIs

* fix cli internal var names

* chore:: backport #11609 to the feat/nv22 branch (#11644)

* feat: api: improve the correctness of Eth's trace_block (#11609)

* Improve the correctness of Eth's trace_block

- Improve encoding/decoding of parameters and return values:
  - Encode "native" parameters and return values with Solidity ABI.
  - Correctly decode parameters to "create" calls.
  - Use the correct (ish) output for "create" calls.
  - Handle all forms of "create".
- Make robust with respect to reverts:
  - Use the actor ID/address from the trace instead of looking it up in
    the state-tree (may not exist in the state-tree due to a revert).
  - Gracefully handle failed actor/contract creation.
- Improve performance:
  - We avoid looking anything up in the state-tree when translating the
    trace, which should significantly improve performance.
- Improve code readability:
  - Remove all "backtracking" logic.
  - Use an "environment" struct to store temporary state instead of
    attaching it to the trace.
- Fix random bugs:
  - Fix an allocation bug in the "address" logic (need to set the
    capacity before modifying the slice).
  - Improved error checking/handling.
- Use correct types for `trace_block` action/results (create, call, etc.).
  - And use the correct types for Result/Action structs instead of reusing the same "Call" action every time.
- Improve error messages.

* Make gen

Make gen

---------

Co-authored-by: Steven Allen <steven@stebalien.com>

* fix: add UpgradePhoenixHeight to StateGetNetworkParams (#11648)

* chore: deps: update to go-state-types v13.0.0-rc.1

* do NOT update the cache when running the real migration

* Merge pull request #11632 from hanabi1224/hm/drand-test

feat: drand quicknet: allow scheduling drand quicknet upgrade before nv22 on 2k devnet

* chore: deps: update to go-state-types v13.0.0-rc.2

chore: deps: update to go-state-types v13.0.0-rc.2

* feat: set migration config UpgradeEpoch for v13 actors upgrade

* Built-in actor events first draft

* itest for DDO non-market verified data w/ builtin actor events

* Tests for builtin actor events API

* Clean up DDO+Events tests, add lots of explainer comments

* Minor tweaks to events types

* Avoid duplicate messages when looking for receipts

* Rename internal events modules for clarity

* Adjust actor event API after review

* s/ActorEvents/Events/g in global config

* Manage event sending rate for SubscribeActorEvents

* Terminate SubscribeActorEvents chan when at max height

* Document future API changes

* More clarity in actor event API docs

* More post-review changes, lots of tests for SubscribeActorEvents

Use BlockDelay as the window for receiving events on the SubscribeActorEvents
channel. We expect the user to have received the initial batch of historical
events (if any) in one block's time. For real-time events we expect them to
not fall behind by roughly one block's time.

* Remove duplicate code from actor event type marshalling tests

Reduce verbosity and remove duplicate test logic from actor event types
JSON marshalling tests.

* Rename actor events test to follow go convention

Add missing `s` to `actor_events` test file to follow golang convention
used across the repo.

* Run actor events table tests in deterministic order

Refactor `map` usage for actor event table tests to ensure deterministic
test execution order, making debugging potential issues easier. If
non-determinism is a target, leverage Go's built-in parallel testing
capabilities.

* Reduce scope for filter removal failure when getting actor events

Use a fresh context to remove the temporary filter installed solely to
get the actor events. This should reduce chances of failure in a case
where the original context may be expired/cancelled.

Refactor removal into a `defer` statement for a more readable, concise
return statement.

* Use fixed RNG seed for actor event tests

Improve determinism in actor event tests by using a fixed RNG seed. This
makes up a more reproducible test suit.

* Use provided libraries to assert eventual conditions

Use the functionalities already provided by `testify` to assert eventual
conditions, and remove the use of `time.Sleep`.

Remove duplicate code in utility functions that are already defined.

Refactor assertion helper functions to use consistent terminology:
"require" implies fatal error, whereas "assert" implies error where the
test may proceed executing.

* Update changelog for actor events APIs

* Fix concerns and docs identified by review

* Update actor bundle to v13.0.0-rc3

Update actor bundle to v13.0.0-rc3

* Prep Lotus v1.26.0-rc1

- For sanity reverting the mainnet upgrade epoch to 99999999, and then only set it when cutting the final release

-Update Calibnet CIDs to v13.0.0-rc3

- Add GetActorEvents, SubscribeActorEvents, GetAllClaims and GetAllAllocations methods to the changelog

Co-Authored-By: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Masih H. Derkani <m@derkani.org>

* Make gen

Make gen

* fix: beacon: validate drand change at nv16 correctly

* bump to v1.26.0-rc2

* test: cleanup ddo verified itest, extract steps to functions

also add allocation-removed event case

* test: extract verified DDO test to separate file, add more checks

* test: add additional actor events checks

* Add verification for "deal-activated" actor event

* docs(drand): document the meaning of "IsChained" (#11692)

* Resolve conflicts

I encountered multiple issues when trying to run make gen. And these changes fixed a couple of them:
- go mod tidy
- Remove RaftState/RaftLeader
- Revert `if ts.Height() > claim.TermMax+claim.TermStart || !cctx.IsSet("expired")` to the what is in the release/v1.26.0: `if tsHeight > val.TermMax || !expired`

* fixup imports, make jen

* Update version

Update version in master to v1.27.0-dev

* Update node/impl/full/dummy.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* Adjust ListClaimsCmd

Adjust ListClaimsCmd according to review

---------

Signed-off-by: samuelarogbonlo <sbayo971@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Samuel Arogbonlo <47984109+samuelarogbonlo@users.noreply.github.com>
Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: tom123222 <160735201+tom123222@users.noreply.github.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
2024-03-12 10:33:58 +01:00
Steven Allen
edb2b9584b
fix: stmgr: remove watermelon migration hack (#11623)
We did this to avoid using cached migrations in the borked watermelon
upgrade on calibrationnet, but we don't need to keep this in the
code (even for users syncing from scratch).
2024-02-21 10:32:22 -08:00
Aayush
5b5dd525f5 skip migration cache correctly 2023-11-09 14:14:46 -05:00
Aayush
410bdc389e fix: nv21fix migration: correctly upgrade system actor 2023-11-09 14:14:41 -05:00
Mikers
5e5a81bf23
add go linter - "unused" (#11235)
* add go linter - "unused"

* use _ to name unused but needed padding variable

* remove unused code

* add queue test to appease unused linter

* remove unused code in test

* remove unused func

* remove unused struct identified by linter

* remove unused variable

* remove unused code

* remove unused file

* remove unused struct

* remove unused function

* remove unused observe peers function in raft

* remove unused declareFaults function

* annotate nolint:unused on needed methods
2023-09-05 12:19:43 -10:00
Aayush
e17af657da chore: stmgr: migrations: do not log noisily on cache misses 2023-07-18 10:22:30 -04:00
Travis Person
f641139bf2
Merge pull request #10282 from filecoin-project/feat/cache-migrated-stateroot
feat: stmgr: cache migrated stateroots
2023-03-09 11:47:43 -08:00
Travis Person
b4e589a0f7 review comments 2023-03-09 18:54:29 +00:00
Steven Allen
dbbcf4b2ee
feat: vm: switch to the new exec trace format (#10372)
This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).
2023-03-01 16:02:18 -08:00
Travis Person
814c146626 feat: stmgr: cache migrated stateroots 2023-03-01 16:17:34 +00:00
Travis Person
2dcaddf61d feat: stmgr: add env to disable premigrations
Setting the environment variable `LOTUS_DISABLE_PRE_MIGRATIONS=1` will
discard premigrations for all upgrade.
2023-02-17 23:12:26 +00:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
vyzo
2690b69831 fix rebase issues 2022-04-14 19:49:08 +03:00
vyzo
3b1964c507 remove manifest from upgrade schedule, get it inside the migration instead 2022-04-14 19:33:24 +03:00
vyzo
77613b57a9 fix lint 2022-04-14 19:31:34 +03:00
vyzo
a9ae38a1ce wire in nv16 migration signature and manifests 2022-04-14 19:29:22 +03:00
Aayush
0bfbcfb5e9 stmgr: GetNtwkVersion: return genesis network version before the first upgrade 2022-03-25 15:09:14 -04:00
Aayush Rajasekaran
8aabe1b488 Fast migration for v15 2022-01-11 17:19:38 -05:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Steven Allen
91da70fb7d fix: comment spelling 2021-08-31 10:37:51 -07:00
Steven Allen
e6902304d5 fix: address expensive fork review 2021-08-30 16:20:23 -07:00
Steven Allen
165735d01a fix: correctly handle null blocks when detecting an expensive fork
Also improve/fix documentation to reflect the _actual_ tipset that's
passed into upgrades.

And update some comments.

fixes #7192
2021-08-27 15:04:01 -07:00
Łukasz Magiera
50b217817e stmgr: Split upgrade code from upgrade runtime 2021-07-27 14:48:30 +02:00
ZenGround0
4cd05fa39c Lint 2021-07-22 09:49:47 -04:00
ZenGround0
ed844c5283 Use current ntwk version in mpool message check 2021-07-22 09:49:47 -04:00
ZenGround0
47b5afa84b Add version six and a half 2021-07-22 09:49:47 -04:00
frrist
0519c77c24 polish(stmgr): define ExecMonitor for message applicaiton callback 2021-06-07 11:03:18 -07:00
Łukasz Magiera
2afe725933 Get PreCommitting to work 2021-05-17 22:02:23 +02:00
Aayush Rajasekaran
b5da2655dc Introduce v5 actors 2021-05-10 19:44:28 -04:00
Aayush Rajasekaran
f1ded63d60 v4 specs-actors integration, nv12 migration 2021-04-27 02:02:44 -04:00
Aayush Rajasekaran
cea46e5f53 Upgrade to version 11 at Norwegian height 2021-03-29 18:50:13 -04:00
Peter Rabbitson
c340750672 Nerpa reset backport/cherry-pick from 73eabc310f 2021-03-18 20:55:15 +01:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores.
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Aayush Rajasekaran
60b7f56297 Code review 2021-02-19 15:10:56 -05:00
Aayush Rajasekaran
96b4b828f7 Remove the null bls actor from the init actor state 2021-02-18 19:01:28 -05:00
Aayush Rajasekaran
a0fedf8bdb Only delete the zero BLS actor on mainnet 2021-02-18 19:01:21 -05:00
Aayush Rajasekaran
5aff54161a Delete the null-bls actor 2021-02-18 19:01:03 -05:00
Raúl Kripalani
d1104fec4c rename blockstores for consistency. 2021-01-29 23:17:25 +00:00
Raúl Kripalani
e02fdf5064 Merge branch 'master' into refactor/lib/blockstore 2021-01-29 21:21:31 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Steven Allen
53271729b6 tweak migration perf 2021-01-28 16:15:23 -08:00
Steven Allen
2f3c912ad9 improve upgrade logging 2021-01-28 16:15:23 -08:00
Steven Allen
39d4f6780d pre-migration: refactor timing specification
There are now three times:

1. StartWithin: start within X epochs of the upgrade.
2. DontStartWithin: don't start within X epochs of the upgrade.
3. StopWithin: stop within X epochs of the upgrade.
2021-01-27 16:09:07 -08:00
Steven Allen
bceb246080 set worker counts for pre-migrations 2021-01-27 15:47:51 -08:00
Steven Allen
9574db6d0a log in migration 2021-01-27 15:05:00 -08:00
Steven Allen
9a03c003fe fix sorted test 2021-01-27 13:05:14 -08:00