Commit Graph

686 Commits

Author SHA1 Message Date
Łukasz Magiera
9d814d0fdc feat: shed: Add v13 migration to migrate-state (#11601)
* shed: Add v13 migration to migrate-state

* shed: some ADL tools, update GST

* shed: diff hamt address mode

* shed migration debug tooling

* shed migration: market diff on error

* shed: Fix cached migration diff

* shed: Diff deal states on bad migration

* shed: Use std json

* shed: Drill in the migration diff more

* shed: Show proposals in migration market diff

* shed: Show added provider sectors diff

* shed: hamts are hard to use

* update  gst

* update gst

* update gst

* update GST with fixed invartiant

* go mod tidy
2024-03-22 07:00:28 +01:00
Aarsh Shah
0ca169d252 Tests for builtin actor events API 2024-03-22 07:00:28 +01:00
Rod Vagg
d8fb28e36d AggregateProofType nil when doing batch updates
Use latest nv22 go-state-types version with matching update
2024-03-22 07:00:28 +01:00
Łukasz Magiera
6a0f16b084 feat: sealing: Support nv22 DDO features in the sealing pipeline (#11226)
* Initial work supporting DDO pieces in lotus-miner

* sealing: Update pipeline input to operate on UniversalPiece

* sealing: Update pipeline checks/sealing states to operate on UniversalPiece

* sealing: Make pipeline build with UniversalPiece

* move PieceDealInfo out of api

* make gen

* make sealing pipeline unit tests pass

* fix itest ensemble build

* don't panic in SectorsStatus with deals

* stop linter from complaining about checkPieces

* fix sector import tests

* mod tidy

* sealing: Add logic for (pre)committing DDO sectors

* sealing: state-types with method defs

* DDO non-snap pipeline works(?), DDO Itests

* DDO support in snapdeals pipeline

* make gen

* update actor bundles

* update the gst market fix

* fix: chain: use PreCommitSectorsBatch2 when setting up genesis

* some bug fixes

* integration working changes

* update actor bundles

* Make TestOnboardRawPieceSnap pass

* Appease the linter

* Make deadlines test pass with v12 actors

* Update go-state-types, abstract market DealState

* make gen

* mod tidy, lint fixes

* Fix some more tests

* Bump version in master

Bump version in master

* Make gen

Make gen

* fix sender

* fix: lotus-provider: Fix winning PoSt

* fix: sql Scan cannot write to an object

* Actually show miner-addrs in info-log

Actually show miner-addrs in lotus-provider info-log

* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* ddo is now nv22

* make gen

* temp actor bundle with ddo

* use working go-state-types

* gst with v13 market migration

* update bundle, builtin.MethodsMiner.ProveCommitSectors2 -> 3

* actually working v13 migration, v13 migration itest

* Address review

* sealing: Correct DDO snap pledge math

* itests: Mixed ddo itest

* pipeline: Fix sectorWeight

* sealing: convert market deals into PAMs in mixed sectors

* sealing: make market to ddo conversion work

* fix lint

* update gst

* Update actors and GST to lastest integ branch

* commit batcher: Update ProveCommitSectors3Params builder logic

* make gen

* use builtin-actors master

* ddo: address review

* itests: Add commd assertions to ddo tests

* make gen

* gst with fixed types

* config knobs for RequireActivationSuccess

* storage: Drop obsolete flaky tasts

---------

Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>
2024-03-22 07:00:28 +01:00
Andrew Jackson (Ajax)
5ea06f4e2d
Merge pull request #11405 from filecoin-project/feat/sturdypost
feat: sturdypost work branch
2023-12-04 18:18:04 -06:00
Aayush Rajasekaran
9a37ce0fc8
Merge pull request #11391 from filecoin-project/asr/fix-invariants-calib
fix: shed: make invariants checker work with splitstore
2023-12-04 15:32:32 -05:00
Andrew Jackson (Ajax)
e102e4f31f
Merge branch 'master' into feat/sturdypost 2023-11-29 17:15:42 -06:00
Łukasz Magiera
231b2070d4
feat: chain import: don't walk to genesis - 2-3x faster snapshot import (#11446)
* chain import: don't walk to genesis

* fix daemon build

* fast snapshot genesis: address review
2023-11-25 13:29:57 -08:00
Andrew Jackson (Ajax)
4b8aa53806 lints and non parallel 2023-11-14 22:58:43 -06:00
Aayush
a0026a0ecd fix: shed: make invariants checker work with splitstore 2023-11-08 11:55:56 -05:00
Mikers
72c5a9a574
add support for v12 check invariants and also a default case to reduce future confusion (#11371) 2023-11-01 09:36:00 -07:00
Steven Allen
87801b5855
feat: shed: fix blockstore prune (#11197)
1. Implement a pass-through for garbage collection on the idstore.
2. Fix the `lotus-shed state-prune` command.

NOTE: the new performance of running a full prune will be significantly
less as this version doesn't batch. However, it should now actually
_work_ and most users will be using the splitstore anyways.
2023-10-27 11:33:18 -07:00
Aayush
5e76b05b17 Merge branch 'feat/nv21' into asr/merge-feat-nv21 2023-10-16 11:13:33 -04:00
Steven Allen
c6debaadf0 make gen 2023-09-23 17:54:44 -04:00
Phi
d771aa353e Update comment
Update comment
2023-09-07 11:27:28 +02:00
Phi
7c61e890c5 Add timing in miner-select-messages
Add timing in `lotus-shed mpool miner-select-messages` cmd. And printing amount of message in the mpool
2023-09-07 11:19:10 +02:00
Phi
825064669d Continue if msg not found on-chain
Continue if msg not found on-chain
2023-08-22 10:56:50 +02:00
Phi
e1ed24ddd0 Put exec-trace behind flag
Put exec-trace behind flag
2023-08-21 13:55:49 +02:00
Phi
539a524271 Make exec-trace human readable
Make exec-trace human readable
2023-08-20 20:11:27 +02:00
Phi
c1231125e9 Print receipt
Print receipt
2023-08-20 19:54:17 +02:00
Phi
925498f938 Add exec-trace and receipt
Add exec-trace and receipt to the lotus-shed msg command
2023-08-20 10:06:00 +02:00
Aayush
20e831b9ea refactor: drop unused param from IsRoundWinner 2023-08-11 09:48:09 -04:00
Mikers
4f0e2f3276 Added CPU profiling option to lotus-shed
This commit includes the following updates:

- Added a new 'pprof' flag to the command-line arguments in main.go. This flag allows the user to specify a file name for writing CPU profile information, enabling performance analysis.

- Implemented functionality to start CPU profiling in the 'Before' function. If a file name is provided via the 'pprof' flag, the function creates the specified file and begins CPU profiling.

- Added an 'After' function to stop CPU profiling once the program execution is finished.

These changes provide a way to analyze the CPU usage of the 'lotus-shed' application, which can be valuable for performance tuning and optimization.
2023-08-02 01:20:06 +00:00
Mikers
1eca520076 Added support for Filecoin network version 21 (v12 state types)
This commit includes the following updates:

- Imported v12 of the filecoin state types in migrations.go. This allows the system to interpret and operate on the latest structure of filecoin state information, ensuring compatibility with the latest version of the Filecoin network.

- Updated the 'getMigrationFuncsForNetwork' function to include a case for network.Version21, enabling the use of the UpgradeActorsV12 and PreUpgradeActorsV12 functions for network migration. These changes ensure the system correctly handles migrations when the network upgrades to version 21.

- Implemented a new 'checkNv21Invariants' function to check the invariants for version 21 of the Filecoin network. This function:

  1. Loads the new state root from the actor store.
  2. Retrieves the actor code IDs for the current state version (v12).
  3. Loads the actor tree from the state root and checks state invariants using the actor code IDs.
  4. Logs any error messages generated during the invariant check.
  5. Finally, prints a message with the time taken to complete the checks.

By checking these invariants, we can ensure the system's state is consistent and valid after migrating to network version 21.
2023-08-01 14:23:55 -10:00
ZenGround0
d900acc506
Merge pull request #11078 from filecoin-project/shed/zip-partition-info
feat:shed:gather partition metadata
2023-07-31 18:18:16 -04:00
zenground0
482c90412d Lint 2023-07-31 11:12:56 -06:00
zenground0
6a07def34a gen 2023-07-31 10:53:07 -06:00
Fridrik Asmundsson
719215122b Address PR feedback and move each epoch inside own tx 2023-07-24 14:18:31 +00:00
Fridrik Asmundsson
8a5950b3e9 Add new lotus-shed command for backfillling actor events 2023-07-24 11:37:30 +00:00
zenground0
df17f709a4 Record faulty status of sectors 2023-07-21 13:07:25 -06:00
zenground0
8c4e705df3 Count all precommit sector numbers that we are checking 2023-07-21 09:30:45 -06:00
zenground0
2776e29042 Gather info about overhead operations in deadline summary 2023-07-21 08:53:24 -06:00
Aayush
09b566dde1 fix: daemon: set real beacon schedule when importing chain 2023-07-17 09:36:41 -04:00
zenground0
9bd0e9ed58 Fix classic bug 2023-07-10 16:32:37 -06:00
zenground0
3f4429a5ef match json parsing 2023-07-10 16:00:03 -06:00
zenground0
921bdb85ab correct json parsing of addr 2023-07-10 15:54:07 -06:00
zenground0
921e5520ea debug 2023-07-10 15:32:19 -06:00
zenground0
cff7716707 Tool for measuring partition summaries 2023-07-10 15:28:14 -06:00
ZenGround0
1358d70128
Merge pull request #11012 from filecoin-project/feat/stat-snapshot
feat:profiling:state summary and visualization
2023-07-06 16:28:05 -04:00
Łukasz Magiera
e30f8af2d1 make gen 2023-07-06 09:35:10 +02:00
Łukasz Magiera
7b21420831 feat: shed: command for decoding block headers 2023-06-30 14:24:07 +02:00
zenground0
ab72f2e230 Lint 2023-06-28 22:35:28 -06:00
mike seiler
a1bfdfbc04 Refactor state invariant checks for different actor versions
This commit refactors the code in `invariants.go` to support state invariant checks for different actor versions. The code now includes checks for versions 10 and 11 of the actors. The `CheckStateInvariants` function is invoked based on the actor version, passing the appropriate parameters. This change ensures that state invariant checks are performed correctly for different actor versions.

The changes made are as follows:
- Added imports for actor versions 10 and 11.
- Added `case` statements for versions 10 and 11 in the switch statement.
- Invoked `CheckStateInvariants` function with the correct parameters for each version.
- Handled errors returned by the `CheckStateInvariants` function.

These changes enhance the flexibility and compatibility of the code with different actor versions, improving the accuracy of state invariant checks.
2023-06-28 12:50:15 -07:00
zenground0
d2b2fba799 Remove debug pprof serving 2023-06-28 10:11:47 -06:00
zenground0
3cacbdfa18 Lint fixes 2023-06-28 09:52:47 -06:00
zenground0
016661b657 More cleanup 2023-06-28 09:17:57 -06:00
zenground0
3897bf14d1 Cleanup 2023-06-28 09:15:38 -06:00
zenground0
0cfdc9b5b6 Refactor for cleanup + measure top level HAMT churn 2023-06-27 17:57:00 -06:00
zenground0
77ea7ef90d Tweak path stuff 2023-06-27 11:29:14 -06:00
zenground0
4aa977f119 Touch up pathing 2023-06-27 11:00:20 -06:00