lotus/itests/manual_onboarding_test.go

119 lines
5.0 KiB
Go
Raw Normal View History

test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
package itests
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/network"
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
"github.com/filecoin-project/lotus/itests/kit"
)
// Manually onboard CC sectors, bypassing lotus-miner onboarding pathways
func TestManualSectorOnboarding(t *testing.T) {
req := require.New(t)
const defaultSectorSize = abi.SectorSize(2 << 10) // 2KiB
sealProofType, err := miner.SealProofTypeFromSectorSize(defaultSectorSize, network.Version23, miner.SealProofVariant_Standard)
req.NoError(err)
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
for _, withMockProofs := range []bool{true, false} {
testName := "WithRealProofs"
if withMockProofs {
testName = "WithMockProofs"
}
t.Run(testName, func(t *testing.T) {
if !withMockProofs {
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
kit.Expensive(t)
}
kit.QuietMiningLogs()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var (
// need to pick a balance value so that the test is not racy on CI by running through it's WindowPostDeadlines too fast
blocktime = 2 * time.Millisecond
client kit.TestFullNode
minerA kit.TestMiner // A is a standard genesis miner
)
// Setup and begin mining with a single miner (A)
// Miner A will only be a genesis Miner with power allocated in the genesis block and will not onboard any sectors from here on
ens := kit.NewEnsemble(t, kit.MockProofs(withMockProofs)).
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
FullNode(&client, kit.SectorSize(defaultSectorSize)).
// preseal more than the default number of sectors to ensure that the genesis miner has power
// because our unmanaged miners won't produce blocks so we may get null rounds
Miner(&minerA, &client, kit.PresealSectors(5), kit.SectorSize(defaultSectorSize), kit.WithAllSubsystems()).
Start().
InterconnectAll()
blockMiners := ens.BeginMiningMustPost(blocktime)
req.Len(blockMiners, 1)
blockMiner := blockMiners[0]
// Instantiate MinerB to manually handle sector onboarding and power acquisition through sector activation.
// Unlike other miners managed by the Lotus Miner storage infrastructure, MinerB operates independently,
// performing all related tasks manually. Managed by the TestKit, MinerB has the capability to utilize actual proofs
// for the processes of sector onboarding and activation.
nodeOpts := []kit.NodeOpt{kit.SectorSize(defaultSectorSize), kit.OwnerAddr(client.DefaultKey)}
minerB, ens := ens.UnmanagedMiner(&client, nodeOpts...)
// MinerC is similar to MinerB, but onboards pieces instead of a pure CC sector
minerC, ens := ens.UnmanagedMiner(&client, nodeOpts...)
ens.Start()
build.Clock.Sleep(time.Second)
t.Log("Checking initial power ...")
// Miner A should have power as it has already onboarded sectors in the genesis block
head, err := client.ChainHead(ctx)
req.NoError(err)
p, err := client.StateMinerPower(ctx, minerA.ActorAddr, head.Key())
req.NoError(err)
t.Logf("MinerA RBP: %v, QaP: %v", p.MinerPower.QualityAdjPower.String(), p.MinerPower.RawBytePower.String())
// Miner B should have no power as it has yet to onboard and activate any sectors
minerB.AssertNoPower(ctx)
// Miner C should have no power as it has yet to onboard and activate any sectors
minerC.AssertNoPower(ctx)
// ---- Miner B onboards a CC sector
var bSectorNum abi.SectorNumber
var bRespCh chan kit.WindowPostResp
var bWdPostCancelF context.CancelFunc
bSectorNum, bRespCh, bWdPostCancelF = minerB.OnboardCCSector(ctx, sealProofType)
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
// Miner B should still not have power as power can only be gained after sector is activated i.e. the first WindowPost is submitted for it
minerB.AssertNoPower(ctx)
// Ensure that the block miner checks for and waits for posts during the appropriate proving window from our new miner with a sector
blockMiner.WatchMinerForPost(minerB.ActorAddr)
// --- Miner C onboards sector with data/pieces
var cSectorNum abi.SectorNumber
var cRespCh chan kit.WindowPostResp
cSectorNum, cRespCh, _ = minerC.OnboardSectorWithPieces(ctx, kit.TestSpt)
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
// Miner C should still not have power as power can only be gained after sector is activated i.e. the first WindowPost is submitted for it
minerC.AssertNoPower(ctx)
// Ensure that the block miner checks for and waits for posts during the appropriate proving window from our new miner with a sector
blockMiner.WatchMinerForPost(minerC.ActorAddr)
// Wait till both miners' sectors have had their first post and are activated and check that this is reflected in miner power
minerB.WaitTillActivatedAndAssertPower(ctx, bRespCh, bSectorNum)
minerC.WaitTillActivatedAndAssertPower(ctx, cRespCh, cSectorNum)
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
// Miner B has activated the CC sector -> upgrade it with snapdeals
_ = minerB.SnapDeal(ctx, kit.TestSpt, bSectorNum)
// cancel the WdPost for the CC sector as the corresponding CommR is no longer valid
bWdPostCancelF()
test: actors: manual CC onboarding and proving integration test (#12017) * remove client CLI * remove markets CLI from miner * remove markets from all CLI * remove client API * update go mod * remove EnableMarkets flag * remove market subsystem * remove dagstore * remove index provider * remove graphsync and data-transfer * remove markets * go mod tidy * fix cbor gen deps * remove deal making from config * remove eol alert * go mod tidy * changes as per review * make jen * changes as per review * test: actors: manual CC onboarding and proving integration test * test: actors: manual CC onboarding itest with real proofs * test: actors: fix lint issue, require proofs in CI * test: actors: rename real proofs test, fix dispute window wait * feat: add TestUnmanagedMiner in the itest kit for non-storage managed miners * feat: test: improve UnmanagedMiner test harness * feat: test: MineBlocksMustPost can watch for >1 miners (#12063) * feat: test: MineBlocksMustPost can watch for >1 miners * feat: test: wait for both sectors at the end of test * feat: test: minor manual onboarding test fixups and speed up * feat: test: handle case where miners have close deadline ends * Implement snap deals test for manual sector onboarding (#12066) * changes as per review * thread safety * test for snap deals * remove extraneous change * Apply suggestions from code review Co-authored-by: Rod Vagg <rod@vagg.org> * cancel CC Post after snap deals --------- Co-authored-by: Rod Vagg <rod@vagg.org> * fix config --------- Co-authored-by: aarshkshah1992 <aarshkshah1992@gmail.com>
2024-06-06 04:47:25 +00:00
})
}
}