cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity

Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers

Browse Source
This commit is contained in:
Łukasz Magiera 2020-09-10 17:30:54 +02:00
commit 5f08fe7ead
292 changed files with 4979 additions and 1760 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
.circleci/config.yml
View File

@ -225,12 +225,17 @@ jobs:
cd extern/test-vectors
git fetch
git checkout origin/<< parameters.vectors-branch >>
- run:
name: go get vectors branch
command: go get github.com/filecoin-project/test-vectors@<< parameters.vectors-branch >>
- go/install-gotestsum:
gobin: $HOME/.local/bin
version: 0.5.2
- run:
name: install statediff globally
command: |
## statediff is optional; we succeed even if compilation fails.
mkdir -p /tmp/statediff
git clone https://github.com/filecoin-project/statediff.git /tmp/statediff
cd /tmp/statediff
go install ./cmd/statediff || exit 0
- run:
name: go test
environment:

15
.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1,15 @@
## filecoin-project/lotus CODEOWNERS
## Refer to https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners.
##
## These users or groups will be automatically assigned as reviewers every time
## a PR is submitted that modifies code in the specified locations.
##
## The Lotus repo configuration requires that at least ONE codeowner approves
## the PR before merging.
### Global owners.
* @magik6k @whyrusleeping @Kubuxu
### Conformance testing.
conformance/ @raulk
extern/test-vectors @raulk

115
CHANGELOG.md
View File

@ -1,5 +1,120 @@
# Lotus changelog
# 0.6.2 / 2020-09-09
This release introduces some critical fixes to message selection and gas estimation logic. It also adds the ability for nodes to mark a certain tipset as checkpointed, as well as various minor improvements and bugfixes.
## Changes
#### Messagepool
- Warn when optimal selection fails to pack a block and we fall back to random selection (https://github.com/filecoin-project/lotus/pull/3708)
- Add basic command for printing gas performance of messages in the mpool (https://github.com/filecoin-project/lotus/pull/3701)
- Adjust optimal selection to always try to fill blocks (https://github.com/filecoin-project/lotus/pull/3685)
- Fix very minor bug in repub baseFeeLowerBound (https://github.com/filecoin-project/lotus/pull/3663)
- Add an auto flag to mpool replace (https://github.com/filecoin-project/lotus/pull/3676)
- Fix mpool optimal selection packing failure (https://github.com/filecoin-project/lotus/pull/3698)
#### Core Lotus
- Don't use latency as initital estimate for blocksync (https://github.com/filecoin-project/lotus/pull/3648)
- Add niceSleep 1 second when drand errors (https://github.com/filecoin-project/lotus/pull/3664)
- Fix isChainNearSync check in block validator (https://github.com/filecoin-project/lotus/pull/3650)
- Add peer to peer manager before fetching the tipset (https://github.com/filecoin-project/lotus/pull/3667)
- Add StageFetchingMessages to sync status (https://github.com/filecoin-project/lotus/pull/3668)
- Pass tipset through upgrade logic (https://github.com/filecoin-project/lotus/pull/3673)
- Allow nodes to mark tipsets as checkpointed (https://github.com/filecoin-project/lotus/pull/3680)
- Remove hard-coded late-fee in window PoSt (https://github.com/filecoin-project/lotus/pull/3702)
- Gas: Fix median calc (https://github.com/filecoin-project/lotus/pull/3686)
#### Storage
- Storage manager: bail out with an error if unsealed cid is undefined (https://github.com/filecoin-project/lotus/pull/3655)
- Storage: return true from Sealer.ReadPiece() on success (https://github.com/filecoin-project/lotus/pull/3657)
#### Maintenance
- Resolve lotus, test-vectors, statediff dependency cycle (https://github.com/filecoin-project/lotus/pull/3688)
- Paych: add docs on how to use paych status (https://github.com/filecoin-project/lotus/pull/3690)
- Initial CODEOWNERS (https://github.com/filecoin-project/lotus/pull/3691)
# 0.6.1 / 2020-09-08
This optional release introduces a minor improvement to the sync process, ensuring nodes don't fall behind and then resync.
## Changes
- Update `test-vectors` (https://github.com/filecoin-project/lotus/pull/3645)
- Revert "only subscribe to pubsub topics once we are synced" (https://github.com/filecoin-project/lotus/pull/3643)
# 0.6.0 / 2020-09-07
This consensus-breaking release of Lotus is designed to test a network upgrade on the space race testnet. The changes that break consensus are:
- Tweaking of some cryptoecon parameters in specs-actors 0.9.7 (https://github.com/filecoin-project/specs-actors/releases/tag/v0.9.7)
- Rebalancing FIL distribution to make testnet FIL scarce, which prevents base fee spikes and sets better expectations for mainnet
This release also introduces many improvements to Lotus! Among them are a new version of go-fil-markets that supports non-blocking retrieval, various spam reduction measures in the messagepool and p2p logic, and UX improvements to payment channels, dealmaking, and state inspection.
## Changes
#### Core Lotus and dependencies
- Implement faucet funds reallocation logic (https://github.com/filecoin-project/lotus/pull/3632)
- Network upgrade: Upgrade to correct fork threshold (https://github.com/filecoin-project/lotus/pull/3628)
- Update to specs 0.9.7 and markets 0.6.0 (https://github.com/filecoin-project/lotus/pull/3627)
- Network upgrade: Perform base fee tamping (https://github.com/filecoin-project/lotus/pull/3623)
- Chain events: if cache best() is nil, return chain head (https://github.com/filecoin-project/lotus/pull/3611)
- Update to specs actors v0.9.6 (https://github.com/filecoin-project/lotus/pull/3603)
#### Messagepool
- Temporarily allow negative chains (https://github.com/filecoin-project/lotus/pull/3625)
- Improve publish/republish logic (https://github.com/filecoin-project/lotus/pull/3592)
- Fix selection bug; priority messages were not included if other chains were negative (https://github.com/filecoin-project/lotus/pull/3580)
- Add defensive check for minimum GasFeeCap for inclusion within the next 20 blocks (https://github.com/filecoin-project/lotus/pull/3579)
- Add additional info about gas premium (https://github.com/filecoin-project/lotus/pull/3578)
- Fix GasPremium capping logic (https://github.com/filecoin-project/lotus/pull/3552)
#### Payment channels
- Get available funds by address or by from/to (https://github.com/filecoin-project/lotus/pull/3547)
- Create `lotus paych status` command (https://github.com/filecoin-project/lotus/pull/3523)
- Rename CLI command from "paych get" to "paych add-funds" (https://github.com/filecoin-project/lotus/pull/3520)
#### Peer-to-peer
- Only subscribe to pubsub topics once we are synced (https://github.com/filecoin-project/lotus/pull/3602)
- Reduce mpool add failure log spam (https://github.com/filecoin-project/lotus/pull/3562)
- Republish messages even if the chains have negative performance(https://github.com/filecoin-project/lotus/pull/3557)
- Adjust gossipsub gossip factor (https://github.com/filecoin-project/lotus/pull/3556)
- Integrate pubsub Random Early Drop (https://github.com/filecoin-project/lotus/pull/3518)
#### Miscellaneous
- Fix panic in OnDealExpiredSlashed (https://github.com/filecoin-project/lotus/pull/3553)
- Robustify state manager against holes in actor method numbers (https://github.com/filecoin-project/lotus/pull/3538)
#### UX
- VM: Fix an error message (https://github.com/filecoin-project/lotus/pull/3608)
- Documentation: Batch replacement,update lotus-storage-miner to lotus-miner (https://github.com/filecoin-project/lotus/pull/3571)
- CLI: Robust actor lookup (https://github.com/filecoin-project/lotus/pull/3535)
- Add agent flag to net peers (https://github.com/filecoin-project/lotus/pull/3534)
- Add watch option to storage-deals list (https://github.com/filecoin-project/lotus/pull/3527)
#### Testing & tooling
- Decommission chain-validation (https://github.com/filecoin-project/lotus/pull/3606)
- Metrics: add expected height metric (https://github.com/filecoin-project/lotus/pull/3586)
- PCR: Use current tipset during refund (https://github.com/filecoin-project/lotus/pull/3570)
- Lotus-shed: Add math command (https://github.com/filecoin-project/lotus/pull/3568)
- PCR: Add tipset aggergation (https://github.com/filecoin-project/lotus/pull/3565)- Fix broken paych tests (https://github.com/filecoin-project/lotus/pull/3551)
- Make chain export ~1000x times faster (https://github.com/filecoin-project/lotus/pull/3533)
- Chainwatch: Stop SyncIncomingBlocks from leaking into chainwatch processing; No panics during processing (https://github.com/filecoin-project/lotus/pull/3526)
- Conformance: various changes (https://github.com/filecoin-project/lotus/pull/3521)
# 0.5.10 / 2020-09-03
This patch includes a crucial fix to the message pool selection logic, strongly disfavouring messages that might cause a miner penalty.

70
api/api_full.go
View File

@ -2,23 +2,27 @@ package api
import (
"context"
"fmt"
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/lotus/chain/types"
marketevents "github.com/filecoin-project/lotus/markets/loggers"
@ -153,10 +157,16 @@ type FullNode interface {
// yet synced block headers.
SyncIncomingBlocks(ctx context.Context) (<-chan *types.BlockHeader, error)
// SyncCheckpoint marks a blocks as checkpointed, meaning that it won't ever fork away from it.
SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) error
// SyncMarkBad marks a blocks as bad, meaning that it won't ever by synced.
// Use with extreme caution.
SyncMarkBad(ctx context.Context, bcid cid.Cid) error
// SyncUnmarkBad unmarks a blocks as bad, making it possible to be validated and synced again.
SyncUnmarkBad(ctx context.Context, bcid cid.Cid) error
// SyncCheckBad checks if a block was marked as bad, and if it was, returns
// the reason.
SyncCheckBad(ctx context.Context, bcid cid.Cid) (string, error)
@ -270,6 +280,9 @@ type FullNode interface {
// ClientListTransfers returns the status of all ongoing transfers of data
ClientListDataTransfers(ctx context.Context) ([]DataTransferChannel, error)
ClientDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error)
// ClientRetrieveTryRestartInsufficientFunds attempts to restart stalled retrievals on a given payment channel
// which are stuck due to insufficient funds
ClientRetrieveTryRestartInsufficientFunds(ctx context.Context, paymentChannel address.Address) error
// ClientUnimport removes references to the specified file from filestore
//ClientUnimport(path string)
@ -300,7 +313,7 @@ type FullNode interface {
// StateMinerSectors returns info about the given miner's sectors. If the filter bitfield is nil, all sectors are included.
// If the filterOut boolean is set to true, any sectors in the filter are excluded.
// If false, only those sectors in the filter are included.
StateMinerSectors(context.Context, address.Address, *abi.BitField, bool, types.TipSetKey) ([]*ChainSectorInfo, error)
StateMinerSectors(context.Context, address.Address, *bitfield.BitField, bool, types.TipSetKey) ([]*ChainSectorInfo, error)
// StateMinerActiveSectors returns info about sectors that a given miner is actively proving.
StateMinerActiveSectors(context.Context, address.Address, types.TipSetKey) ([]*ChainSectorInfo, error)
// StateMinerProvingDeadline calculates the deadline at some epoch for a proving period
@ -315,11 +328,11 @@ type FullNode interface {
// StateMinerPartitions loads miner partitions for the specified miner/deadline
StateMinerPartitions(context.Context, address.Address, uint64, types.TipSetKey) ([]*miner.Partition, error)
// StateMinerFaults returns a bitfield indicating the faulty sectors of the given miner
StateMinerFaults(context.Context, address.Address, types.TipSetKey) (abi.BitField, error)
StateMinerFaults(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error)
// StateAllMinerFaults returns all non-expired Faults that occur within lookback epochs of the given tipset
StateAllMinerFaults(ctx context.Context, lookback abi.ChainEpoch, ts types.TipSetKey) ([]*Fault, error)
// StateMinerRecoveries returns a bitfield indicating the recovering sectors of the given miner
StateMinerRecoveries(context.Context, address.Address, types.TipSetKey) (abi.BitField, error)
StateMinerRecoveries(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error)
// StateMinerInitialPledgeCollateral returns the precommit deposit for the specified miner's sector
StateMinerPreCommitDepositForPower(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error)
// StateMinerInitialPledgeCollateral returns the initial pledge collateral for the specified miner's sector
@ -384,6 +397,9 @@ type FullNode interface {
// MsigGetAvailableBalance returns the portion of a multisig's balance that can be withdrawn or spent
MsigGetAvailableBalance(context.Context, address.Address, types.TipSetKey) (types.BigInt, error)
// MsigGetVested returns the amount of FIL that vested in a multisig in a certain period.
// It takes the following params: <multisig address>, <start epoch>, <end epoch>
MsigGetVested(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error)
// MsigCreate creates a multisig wallet
// It takes the following params: <required number of senders>, <approving addresses>, <unlock duration>
//<initial balance>, <sender address of the create msg>, <gas price>
@ -400,17 +416,29 @@ type FullNode interface {
// It takes the following params: <multisig address>, <proposed message ID>, <recipient address>, <value to transfer>,
// <sender address of the cancel msg>, <method to call in the proposed message>, <params to include in the proposed message>
MsigCancel(context.Context, address.Address, uint64, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error)
// MsigAddPropose proposes adding a signer in the multisig
// It takes the following params: <multisig address>, <sender address of the propose msg>,
// <new signer>, <whether the number of required signers should be increased>
MsigAddPropose(context.Context, address.Address, address.Address, address.Address, bool) (cid.Cid, error)
// MsigAddApprove approves a previously proposed AddSigner message
// It takes the following params: <multisig address>, <sender address of the approve msg>, <proposed message ID>,
// <proposer address>, <new signer>, <whether the number of required signers should be increased>
MsigAddApprove(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, bool) (cid.Cid, error)
// MsigAddCancel cancels a previously proposed AddSigner message
// It takes the following params: <multisig address>, <sender address of the cancel msg>, <proposed message ID>,
// <new signer>, <whether the number of required signers should be increased>
MsigAddCancel(context.Context, address.Address, address.Address, uint64, address.Address, bool) (cid.Cid, error)
// MsigSwapPropose proposes swapping 2 signers in the multisig
// It takes the following params: <multisig address>, <sender address of the propose msg>,
// <old signer> <new signer>
// <old signer>, <new signer>
MsigSwapPropose(context.Context, address.Address, address.Address, address.Address, address.Address) (cid.Cid, error)
// MsigSwapApprove approves a previously proposed SwapSigner
// It takes the following params: <multisig address>, <sender address of the approve msg>, <proposed message ID>,
// <proposer address>, <old signer> <new signer>
// <proposer address>, <old signer>, <new signer>
MsigSwapApprove(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, address.Address) (cid.Cid, error)
// MsigSwapCancel cancels a previously proposed SwapSigner message
// It takes the following params: <multisig address>, <sender address of the cancel msg>, <proposed message ID>,
// <old signer> <new signer>
// <old signer>, <new signer>
MsigSwapCancel(context.Context, address.Address, address.Address, uint64, address.Address, address.Address) (cid.Cid, error)
MarketEnsureAvailable(context.Context, address.Address, address.Address, types.BigInt) (cid.Cid, error)
@ -703,8 +731,28 @@ const (
StageMessages
StageSyncComplete
StageSyncErrored
StageFetchingMessages
)
func (v SyncStateStage) String() string {
switch v {
case StageHeaders:
return "header sync"
case StagePersistHeaders:
return "persisting headers"
case StageMessages:
return "message sync"
case StageSyncComplete:
return "complete"
case StageSyncErrored:
return "error"
case StageFetchingMessages:
return "fetching messages"
default:
return fmt.Sprintf("<unknown: %d>", v)
}
}
type MpoolChange int
const (
@ -738,7 +786,7 @@ type CirculatingSupply struct {
type MiningBaseInfo struct {
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []abi.SectorInfo
Sectors []proof.SectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry
@ -755,7 +803,7 @@ type BlockTemplate struct {
Messages []*types.SignedMessage
Epoch abi.ChainEpoch
Timestamp uint64
WinningPoStProof []abi.PoStProof
WinningPoStProof []proof.PoStProof
}
type DataSize struct {

2
api/api_storage.go
View File

@ -11,11 +11,11 @@ import (
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/extern/sector-storage/fsutil"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"
"github.com/filecoin-project/specs-actors/actors/abi"
)
// StorageMiner is a low-level interface to the Filecoin network storage miner node

3
api/api_worker.go
View File

@ -3,8 +3,7 @@ package api
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/sealtasks"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"

90
api/apistruct/struct.go
View File

@ -12,22 +12,23 @@ import (
protocol "github.com/libp2p/go-libp2p-core/protocol"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/fsutil"
"github.com/filecoin-project/lotus/extern/sector-storage/sealtasks"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"
marketevents "github.com/filecoin-project/lotus/markets/loggers"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-storage/storage"
"github.com/filecoin-project/lotus/api"
@ -104,7 +105,9 @@ type FullNodeStruct struct {
SyncState func(context.Context) (*api.SyncState, error) `perm:"read"`
SyncSubmitBlock func(ctx context.Context, blk *types.BlockMsg) error `perm:"write"`
SyncIncomingBlocks func(ctx context.Context) (<-chan *types.BlockHeader, error) `perm:"read"`
SyncCheckpoint func(ctx context.Context, key types.TipSetKey) error `perm:"admin"`
SyncMarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncUnmarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncCheckBad func(ctx context.Context, bcid cid.Cid) (string, error) `perm:"read"`
MpoolGetConfig func(context.Context) (*types.MpoolConfig, error) `perm:"read"`
@ -136,36 +139,37 @@ type FullNodeStruct struct {
WalletImport func(context.Context, *types.KeyInfo) (address.Address, error) `perm:"admin"`
WalletDelete func(context.Context, address.Address) error `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientGetDealUpdates func(ctx context.Context) (<-chan api.DealInfo, error) `perm:"read"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
ClientDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientGetDealUpdates func(ctx context.Context) (<-chan api.DealInfo, error) `perm:"read"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
ClientDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
ClientRetrieveTryRestartInsufficientFunds func(ctx context.Context, paymentChannel address.Address) error `perm:"write"`
StateNetworkName func(context.Context) (dtypes.NetworkName, error) `perm:"read"`
StateMinerSectors func(context.Context, address.Address, *abi.BitField, bool, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerSectors func(context.Context, address.Address, *bitfield.BitField, bool, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerActiveSectors func(context.Context, address.Address, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerProvingDeadline func(context.Context, address.Address, types.TipSetKey) (*miner.DeadlineInfo, error) `perm:"read"`
StateMinerPower func(context.Context, address.Address, types.TipSetKey) (*api.MinerPower, error) `perm:"read"`
StateMinerInfo func(context.Context, address.Address, types.TipSetKey) (api.MinerInfo, error) `perm:"read"`
StateMinerDeadlines func(context.Context, address.Address, types.TipSetKey) ([]*miner.Deadline, error) `perm:"read"`
StateMinerPartitions func(context.Context, address.Address, uint64, types.TipSetKey) ([]*miner.Partition, error) `perm:"read"`
StateMinerFaults func(context.Context, address.Address, types.TipSetKey) (abi.BitField, error) `perm:"read"`
StateMinerFaults func(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error) `perm:"read"`
StateAllMinerFaults func(context.Context, abi.ChainEpoch, types.TipSetKey) ([]*api.Fault, error) `perm:"read"`
StateMinerRecoveries func(context.Context, address.Address, types.TipSetKey) (abi.BitField, error) `perm:"read"`
StateMinerRecoveries func(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error) `perm:"read"`
StateMinerPreCommitDepositForPower func(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error) `perm:"read"`
StateMinerInitialPledgeCollateral func(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error) `perm:"read"`
StateMinerAvailableBalance func(context.Context, address.Address, types.TipSetKey) (types.BigInt, error) `perm:"read"`
@ -197,10 +201,14 @@ type FullNodeStruct struct {
StateCirculatingSupply func(context.Context, types.TipSetKey) (api.CirculatingSupply, error) `perm:"read"`
MsigGetAvailableBalance func(context.Context, address.Address, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigGetVested func(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigCreate func(context.Context, uint64, []address.Address, abi.ChainEpoch, types.BigInt, address.Address, types.BigInt) (cid.Cid, error) `perm:"sign"`
MsigPropose func(context.Context, address.Address, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigApprove func(context.Context, address.Address, uint64, address.Address, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigCancel func(context.Context, address.Address, uint64, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigAddPropose func(context.Context, address.Address, address.Address, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigAddApprove func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigAddCancel func(context.Context, address.Address, address.Address, uint64, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigSwapPropose func(context.Context, address.Address, address.Address, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
MsigSwapApprove func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
MsigSwapCancel func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
@ -505,6 +513,10 @@ func (c *FullNodeStruct) ClientDataTransferUpdates(ctx context.Context) (<-chan
return c.Internal.ClientDataTransferUpdates(ctx)
}
func (c *FullNodeStruct) ClientRetrieveTryRestartInsufficientFunds(ctx context.Context, paymentChannel address.Address) error {
return c.Internal.ClientRetrieveTryRestartInsufficientFunds(ctx, paymentChannel)
}
func (c *FullNodeStruct) GasEstimateGasPremium(ctx context.Context, nblocksincl uint64, sender address.Address, gaslimit int64, tsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.GasEstimateGasPremium(ctx, nblocksincl, sender, gaslimit, tsk)
}
@ -709,10 +721,18 @@ func (c *FullNodeStruct) SyncIncomingBlocks(ctx context.Context) (<-chan *types.
return c.Internal.SyncIncomingBlocks(ctx)
}
func (c *FullNodeStruct) SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) error {
return c.Internal.SyncCheckpoint(ctx, tsk)
}
func (c *FullNodeStruct) SyncMarkBad(ctx context.Context, bcid cid.Cid) error {
return c.Internal.SyncMarkBad(ctx, bcid)
}
func (c *FullNodeStruct) SyncUnmarkBad(ctx context.Context, bcid cid.Cid) error {
return c.Internal.SyncUnmarkBad(ctx, bcid)
}
func (c *FullNodeStruct) SyncCheckBad(ctx context.Context, bcid cid.Cid) (string, error) {
return c.Internal.SyncCheckBad(ctx, bcid)
}
@ -721,7 +741,7 @@ func (c *FullNodeStruct) StateNetworkName(ctx context.Context) (dtypes.NetworkNa
return c.Internal.StateNetworkName(ctx)
}
func (c *FullNodeStruct) StateMinerSectors(ctx context.Context, addr address.Address, filter *abi.BitField, filterOut bool, tsk types.TipSetKey) ([]*api.ChainSectorInfo, error) {
func (c *FullNodeStruct) StateMinerSectors(ctx context.Context, addr address.Address, filter *bitfield.BitField, filterOut bool, tsk types.TipSetKey) ([]*api.ChainSectorInfo, error) {
return c.Internal.StateMinerSectors(ctx, addr, filter, filterOut, tsk)
}
@ -749,7 +769,7 @@ func (c *FullNodeStruct) StateMinerPartitions(ctx context.Context, m address.Add
return c.Internal.StateMinerPartitions(ctx, m, dlIdx, tsk)
}
func (c *FullNodeStruct) StateMinerFaults(ctx context.Context, actor address.Address, tsk types.TipSetKey) (abi.BitField, error) {
func (c *FullNodeStruct) StateMinerFaults(ctx context.Context, actor address.Address, tsk types.TipSetKey) (bitfield.BitField, error) {
return c.Internal.StateMinerFaults(ctx, actor, tsk)
}
@ -757,7 +777,7 @@ func (c *FullNodeStruct) StateAllMinerFaults(ctx context.Context, cutoff abi.Cha
return c.Internal.StateAllMinerFaults(ctx, cutoff, endTsk)
}
func (c *FullNodeStruct) StateMinerRecoveries(ctx context.Context, actor address.Address, tsk types.TipSetKey) (abi.BitField, error) {
func (c *FullNodeStruct) StateMinerRecoveries(ctx context.Context, actor address.Address, tsk types.TipSetKey) (bitfield.BitField, error) {
return c.Internal.StateMinerRecoveries(ctx, actor, tsk)
}
@ -877,6 +897,10 @@ func (c *FullNodeStruct) MsigGetAvailableBalance(ctx context.Context, a address.
return c.Internal.MsigGetAvailableBalance(ctx, a, tsk)
}
func (c *FullNodeStruct) MsigGetVested(ctx context.Context, a address.Address, sTsk types.TipSetKey, eTsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.MsigGetVested(ctx, a, sTsk, eTsk)
}
func (c *FullNodeStruct) MsigCreate(ctx context.Context, req uint64, addrs []address.Address, duration abi.ChainEpoch, val types.BigInt, src address.Address, gp types.BigInt) (cid.Cid, error) {
return c.Internal.MsigCreate(ctx, req, addrs, duration, val, src, gp)
}
@ -893,6 +917,18 @@ func (c *FullNodeStruct) MsigCancel(ctx context.Context, msig address.Address, t
return c.Internal.MsigCancel(ctx, msig, txID, to, amt, src, method, params)
}
func (c *FullNodeStruct) MsigAddPropose(ctx context.Context, msig address.Address, src address.Address, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddPropose(ctx, msig, src, newAdd, inc)
}
func (c *FullNodeStruct) MsigAddApprove(ctx context.Context, msig address.Address, src address.Address, txID uint64, proposer address.Address, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddApprove(ctx, msig, src, txID, proposer, newAdd, inc)
}
func (c *FullNodeStruct) MsigAddCancel(ctx context.Context, msig address.Address, src address.Address, txID uint64, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddCancel(ctx, msig, src, txID, newAdd, inc)
}
func (c *FullNodeStruct) MsigSwapPropose(ctx context.Context, msig address.Address, src address.Address, oldAdd address.Address, newAdd address.Address) (cid.Cid, error) {
return c.Internal.MsigSwapPropose(ctx, msig, src, oldAdd, newAdd)
}

2
api/cbor_gen.go
View File

@ -6,7 +6,7 @@ import (
"fmt"
"io"
abi "github.com/filecoin-project/specs-actors/actors/abi"
abi "github.com/filecoin-project/go-state-types/abi"
paych "github.com/filecoin-project/specs-actors/actors/builtin/paych"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"

8
api/docgen/docgen.go
View File

@ -28,9 +28,9 @@ import (
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/go-state-types/exitcode"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apistruct"
@ -105,7 +105,7 @@ func init() {
addExample(network.Connected)
addExample(dtypes.NetworkName("lotus"))
addExample(api.SyncStateStage(1))
addExample(build.APIVersion)
addExample(build.FullAPIVersion)
addExample(api.PCHInbound)
addExample(time.Minute)
addExample(datatransfer.TransferID(3))

2
api/test/blockminer.go
View File

@ -7,8 +7,8 @@ import (
"testing"
"time"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/specs-actors/actors/abi"
)
type BlockMiner struct {

2
api/test/ccupgrade.go
View File

@ -10,7 +10,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/impl"

2
api/test/deals.go
View File

@ -20,11 +20,11 @@ import (
"github.com/ipld/go-car"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/specs-actors/actors/abi"
dag "github.com/ipfs/go-merkledag"
dstest "github.com/ipfs/go-merkledag/test"
unixfile "github.com/ipfs/go-unixfs/file"

2
api/test/mining.go
View File

@ -12,7 +12,7 @@ import (
logging "github.com/ipfs/go-log/v2"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/lotus/build"

4
api/test/paych.go
View File

@ -10,8 +10,8 @@ import (
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/ipfs/go-cid"

2
api/test/test.go
View File

@ -65,6 +65,8 @@ func TestApis(t *testing.T, b APIBuilder) {
var OneMiner = []StorageMiner{{Full: 0, Preseal: PresealGenesis}}
func (ts *testSuite) testVersion(t *testing.T) {
build.RunningNodeType = build.NodeFull
ctx := context.Background()
apis, _ := ts.makeNodes(t, 1, OneMiner)
api := apis[0]

2
api/test/util.go
View File

@ -5,7 +5,7 @@ import (
"testing"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"

2
api/test/window_post.go
View File

@ -12,9 +12,9 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/mock"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/specs-actors/actors/abi"
miner2 "github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/lotus/api"

2
api/types.go
View File

@ -6,8 +6,8 @@ import (
"github.com/filecoin-project/go-address"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/ipfs/go-cid"

2
api/utils.go
View File

@ -4,7 +4,7 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
)
type SignFunc = func(context.Context, []byte) (*crypto.Signature, error)

7
build/params_2k.go
View File

@ -3,13 +3,16 @@
package build
import (
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
)
const UpgradeBreezeHeight = -1
const BreezeGasTampingDuration = 0
func init() {
power.ConsensusMinerMinPower = big.NewInt(2048)
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{

2
build/params_shared_funcs.go
View File

@ -5,7 +5,7 @@ import (
"github.com/libp2p/go-libp2p-core/protocol"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/lotus/node/modules/dtypes"

2
build/params_shared_vals.go
View File

@ -5,7 +5,7 @@ package build
import (
"math/big"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
)

5
build/params_testground.go
View File

@ -10,7 +10,7 @@ package build
import (
"math/big"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
)
@ -69,4 +69,7 @@ var (
PackingEfficiencyNum int64 = 4
PackingEfficiencyDenom int64 = 5
UpgradeBreezeHeight abi.ChainEpoch = 0
BreezeGasTampingDuration abi.ChainEpoch = 0
)

7
build/params_testnet.go
View File

@ -5,13 +5,16 @@
package build
import (
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
)
const UpgradeBreezeHeight = 41280
const BreezeGasTampingDuration = 120
func init() {
power.ConsensusMinerMinPower = big.NewInt(10 << 40)
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{

41
build/version.go
View File

@ -1,6 +1,10 @@
package build
import "fmt"
import (
"fmt"
"golang.org/x/xerrors"
)
var CurrentCommit string
var BuildType int
@ -25,7 +29,7 @@ func buildType() string {
}
// BuildVersion is the local build version, set by build system
const BuildVersion = "0.5.10"
const BuildVersion = "0.6.2"
func UserVersion() string {
return BuildVersion + buildType() + CurrentCommit
@ -52,8 +56,37 @@ func (ve Version) EqMajorMinor(v2 Version) bool {
return ve&minorMask == v2&minorMask
}
// APIVersion is a semver version of the rpc api exposed
var APIVersion Version = newVer(0, 15, 0)
type NodeType int
const (
NodeUnknown NodeType = iota
NodeFull
NodeMiner
NodeWorker
)
var RunningNodeType NodeType
func VersionForType(nodeType NodeType) (Version, error) {
switch nodeType {
case NodeFull:
return FullAPIVersion, nil
case NodeMiner:
return MinerAPIVersion, nil
case NodeWorker:
return WorkerAPIVersion, nil
default:
return Version(0), xerrors.Errorf("unknown node type %d", nodeType)
}
}
// semver versions of the rpc api exposed
var (
FullAPIVersion = newVer(0, 14, 0)
MinerAPIVersion = newVer(0, 15, 0)
WorkerAPIVersion = newVer(0, 15, 0)
)
//nolint:varcheck,deadcode
const (

2
chain/actors/aerrors/error.go
View File

@ -3,7 +3,7 @@ package aerrors
import (
"fmt"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
"golang.org/x/xerrors"
)

2
chain/actors/aerrors/error_test.go
View File

@ -3,8 +3,8 @@ package aerrors_test
import (
"testing"
"github.com/filecoin-project/go-state-types/exitcode"
. "github.com/filecoin-project/lotus/chain/actors/aerrors"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/stretchr/testify/assert"
"golang.org/x/xerrors"

2
chain/actors/aerrors/wrap.go
View File

@ -4,7 +4,7 @@ import (
"errors"
"fmt"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
)

4
chain/badtscache.go
View File

@ -56,6 +56,10 @@ func (bts *BadBlockCache) Add(c cid.Cid, bbr BadBlockReason) {
bts.badBlocks.Add(c, bbr)
}
func (bts *BadBlockCache) Remove(c cid.Cid) {
bts.badBlocks.Remove(c)
}
func (bts *BadBlockCache) Has(c cid.Cid) (BadBlockReason, bool) {
rval, ok := bts.badBlocks.Get(c)
if !ok {

2
chain/beacon/beacon.go
View File

@ -3,7 +3,7 @@ package beacon
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
logging "github.com/ipfs/go-log"
"golang.org/x/xerrors"

2
chain/beacon/drand/drand.go
View File

@ -19,7 +19,7 @@ import (
logging "github.com/ipfs/go-log"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/beacon"

2
chain/beacon/mock.go
View File

@ -6,8 +6,8 @@ import (
"encoding/binary"
"time"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/minio/blake2b-simd"
"golang.org/x/xerrors"
)

81
chain/checkpoint.go Normal file
View File

@ -0,0 +1,81 @@
package chain
import (
"encoding/json"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/ipfs/go-datastore"
"golang.org/x/xerrors"
)
var CheckpointKey = datastore.NewKey("/chain/checks")
func loadCheckpoint(ds dtypes.MetadataDS) (types.TipSetKey, error) {
haveChks, err := ds.Has(CheckpointKey)
if err != nil {
return types.EmptyTSK, err
}
if !haveChks {
return types.EmptyTSK, nil
}
tskBytes, err := ds.Get(CheckpointKey)
if err != nil {
return types.EmptyTSK, err
}
var tsk types.TipSetKey
err = json.Unmarshal(tskBytes, &tsk)
if err != nil {
return types.EmptyTSK, err
}
return tsk, err
}
func (syncer *Syncer) SetCheckpoint(tsk types.TipSetKey) error {
if tsk == types.EmptyTSK {
return xerrors.Errorf("called with empty tsk")
}
syncer.checkptLk.Lock()
defer syncer.checkptLk.Unlock()
ts, err := syncer.ChainStore().LoadTipSet(tsk)
if err != nil {
return xerrors.Errorf("cannot find tipset: %w", err)
}
hts := syncer.ChainStore().GetHeaviestTipSet()
anc, err := syncer.ChainStore().IsAncestorOf(ts, hts)
if err != nil {
return xerrors.Errorf("cannot determine whether checkpoint tipset is in main-chain: %w", err)
}
if !hts.Equals(ts) && !anc {
return xerrors.Errorf("cannot mark tipset as checkpoint, since it isn't in the main-chain: %w", err)
}
tskBytes, err := json.Marshal(tsk)
if err != nil {
return err
}
err = syncer.ds.Put(CheckpointKey, tskBytes)
if err != nil {
return err
}
syncer.checkpt = tsk
return nil
}
func (syncer *Syncer) GetCheckpoint() types.TipSetKey {
syncer.checkptLk.Lock()
defer syncer.checkptLk.Unlock()
return syncer.checkpt
}

5
chain/events/events.go
View File

@ -5,7 +5,7 @@ import (
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
"golang.org/x/xerrors"
@ -35,6 +35,7 @@ type eventAPI interface {
ChainNotify(context.Context) (<-chan []*api.HeadChange, error)
ChainGetBlockMessages(context.Context, cid.Cid) (*api.BlockMessages, error)
ChainGetTipSetByHeight(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
ChainHead(context.Context) (*types.TipSet, error)
StateGetReceipt(context.Context, cid.Cid, types.TipSetKey) (*types.MessageReceipt, error)
ChainGetTipSet(context.Context, types.TipSetKey) (*types.TipSet, error)
@ -57,7 +58,7 @@ type Events struct {
func NewEvents(ctx context.Context, api eventAPI) *Events {
gcConfidence := 2 * build.ForkLengthThreshold
tsc := newTSCache(gcConfidence, api.ChainGetTipSetByHeight)
tsc := newTSCache(gcConfidence, api)
e := &Events{
api: api,

7
chain/events/events_called.go
View File

@ -5,7 +5,7 @@ import (
"math"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
"golang.org/x/xerrors"
@ -307,7 +307,10 @@ func (e *hcEvents) onHeadChanged(check CheckFunc, hnd EventHandler, rev RevertHa
defer e.lk.Unlock()
// Check if the event has already occurred
ts := e.tsc.best()
ts, err := e.tsc.best()
if err != nil {
return 0, xerrors.Errorf("error getting best tipset: %w", err)
}
done, more, err := check(ts)
if err != nil {
return 0, xerrors.Errorf("called check error (h: %d): %w", ts.Height(), err)

15
chain/events/events_height.go
View File

@ -4,8 +4,9 @@ import (
"context"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"go.opencensus.io/trace"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/types"
)
@ -152,8 +153,12 @@ func (e *heightEvents) ChainAt(hnd HeightHandler, rev RevertHandler, confidence
e.lk.Lock() // Tricky locking, check your locks if you modify this function!
bestH := e.tsc.best().Height()
best, err := e.tsc.best()
if err != nil {
return xerrors.Errorf("error getting best tipset: %w", err)
}
bestH := best.Height()
if bestH >= h+abi.ChainEpoch(confidence) {
ts, err := e.tsc.getNonNull(h)
if err != nil {
@ -172,7 +177,11 @@ func (e *heightEvents) ChainAt(hnd HeightHandler, rev RevertHandler, confidence
}
e.lk.Lock()
bestH = e.tsc.best().Height()
best, err = e.tsc.best()
if err != nil {
return xerrors.Errorf("error getting best tipset: %w", err)
}
bestH = best.Height()
}
defer e.lk.Unlock()

21
chain/events/events_test.go
View File

@ -11,8 +11,8 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
@ -46,6 +46,10 @@ type fakeCS struct {
sub func(rev, app []*types.TipSet)
}
func (fcs *fakeCS) ChainHead(ctx context.Context) (*types.TipSet, error) {
panic("implement me")
}
func (fcs *fakeCS) ChainGetTipSet(ctx context.Context, key types.TipSetKey) (*types.TipSet, error) {
return fcs.tipsets[key], nil
}
@ -110,7 +114,11 @@ func (fcs *fakeCS) makeTs(t *testing.T, parents []cid.Cid, h abi.ChainEpoch, msg
func (fcs *fakeCS) ChainNotify(context.Context) (<-chan []*api.HeadChange, error) {
out := make(chan []*api.HeadChange, 1)
out <- []*api.HeadChange{{Type: store.HCCurrent, Val: fcs.tsc.best()}}
best, err := fcs.tsc.best()
if err != nil {
return nil, err
}
out <- []*api.HeadChange{{Type: store.HCCurrent, Val: best}}
fcs.sub = func(rev, app []*types.TipSet) {
notif := make([]*api.HeadChange, len(rev)+len(app))
@ -174,7 +182,8 @@ func (fcs *fakeCS) advance(rev, app int, msgs map[int]cid.Cid, nulls ...int) { /
var revs []*types.TipSet
for i := 0; i < rev; i++ {
ts := fcs.tsc.best()
ts, err := fcs.tsc.best()
require.NoError(fcs.t, err)
if _, ok := nullm[int(ts.Height())]; !ok {
revs = append(revs, ts)
@ -196,7 +205,9 @@ func (fcs *fakeCS) advance(rev, app int, msgs map[int]cid.Cid, nulls ...int) { /
continue
}
ts := fcs.makeTs(fcs.t, fcs.tsc.best().Key().Cids(), fcs.h, mc)
best, err := fcs.tsc.best()
require.NoError(fcs.t, err)
ts := fcs.makeTs(fcs.t, best.Key().Cids(), fcs.h, mc)
require.NoError(fcs.t, fcs.tsc.add(ts))
if hasMsgs {

4
chain/events/state/predicates.go
View File

@ -5,8 +5,8 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
init_ "github.com/filecoin-project/specs-actors/actors/builtin/init"
"github.com/filecoin-project/specs-actors/actors/builtin/market"

6
chain/events/state/predicates_test.go
View File

@ -13,11 +13,11 @@ import (
cbornode "github.com/ipfs/go-ipld-cbor"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
tutils "github.com/filecoin-project/specs-actors/support/testing"

23
chain/events/tscache.go
View File

@ -3,13 +3,16 @@ package events
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/types"
)
type tsByHFunc func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
type tsCacheAPI interface {
ChainGetTipSetByHeight(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
ChainHead(context.Context) (*types.TipSet, error)
}
// tipSetCache implements a simple ring-buffer cache to keep track of recent
// tipsets
@ -18,10 +21,10 @@ type tipSetCache struct {
start int
len int
storage tsByHFunc
storage tsCacheAPI
}
func newTSCache(cap abi.ChainEpoch, storage tsByHFunc) *tipSetCache {
func newTSCache(cap abi.ChainEpoch, storage tsCacheAPI) *tipSetCache {
return &tipSetCache{
cache: make([]*types.TipSet, cap),
start: 0,
@ -94,7 +97,7 @@ func (tsc *tipSetCache) getNonNull(height abi.ChainEpoch) (*types.TipSet, error)
func (tsc *tipSetCache) get(height abi.ChainEpoch) (*types.TipSet, error) {
if tsc.len == 0 {
log.Warnf("tipSetCache.get: cache is empty, requesting from storage (h=%d)", height)
return tsc.storage(context.TODO(), height, types.EmptyTSK)
return tsc.storage.ChainGetTipSetByHeight(context.TODO(), height, types.EmptyTSK)
}
headH := tsc.cache[tsc.start].Height()
@ -114,14 +117,18 @@ func (tsc *tipSetCache) get(height abi.ChainEpoch) (*types.TipSet, error) {
if height < tail.Height() {
log.Warnf("tipSetCache.get: requested tipset not in cache, requesting from storage (h=%d; tail=%d)", height, tail.Height())
return tsc.storage(context.TODO(), height, tail.Key())
return tsc.storage.ChainGetTipSetByHeight(context.TODO(), height, tail.Key())
}
return tsc.cache[normalModulo(tsc.start-int(headH-height), clen)], nil
}
func (tsc *tipSetCache) best() *types.TipSet {
return tsc.cache[tsc.start]
func (tsc *tipSetCache) best() (*types.TipSet, error) {
best := tsc.cache[tsc.start]
if best == nil {
return tsc.storage.ChainHead(context.TODO())
}
return best, nil
}
func normalModulo(n, m int) int {

76
chain/events/tscache_test.go
View File

@ -4,8 +4,8 @@ import (
"context"
"testing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
@ -13,10 +13,7 @@ import (
)
func TestTsCache(t *testing.T) {
tsc := newTSCache(50, func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error) {
t.Fatal("storage call")
return &types.TipSet{}, nil
})
tsc := newTSCache(50, &tsCacheAPIFailOnStorageCall{t: t})
h := abi.ChainEpoch(75)
@ -43,7 +40,12 @@ func TestTsCache(t *testing.T) {
for i := 0; i < 9000; i++ {
if i%90 > 60 {
if err := tsc.revert(tsc.best()); err != nil {
best, err := tsc.best()
if err != nil {
t.Fatal(err, "; i:", i)
return
}
if err := tsc.revert(best); err != nil {
t.Fatal(err, "; i:", i)
return
}
@ -55,11 +57,21 @@ func TestTsCache(t *testing.T) {
}
type tsCacheAPIFailOnStorageCall struct {
t *testing.T
}
func (tc *tsCacheAPIFailOnStorageCall) ChainGetTipSetByHeight(ctx context.Context, epoch abi.ChainEpoch, key types.TipSetKey) (*types.TipSet, error) {
tc.t.Fatal("storage call")
return &types.TipSet{}, nil
}
func (tc *tsCacheAPIFailOnStorageCall) ChainHead(ctx context.Context) (*types.TipSet, error) {
tc.t.Fatal("storage call")
return &types.TipSet{}, nil
}
func TestTsCacheNulls(t *testing.T) {
tsc := newTSCache(50, func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error) {
t.Fatal("storage call")
return &types.TipSet{}, nil
})
tsc := newTSCache(50, &tsCacheAPIFailOnStorageCall{t: t})
h := abi.ChainEpoch(75)
@ -91,7 +103,9 @@ func TestTsCacheNulls(t *testing.T) {
add()
add()
require.Equal(t, h-1, tsc.best().Height())
best, err := tsc.best()
require.NoError(t, err)
require.Equal(t, h-1, best.Height())
ts, err := tsc.get(h - 1)
require.NoError(t, err)
@ -109,9 +123,17 @@ func TestTsCacheNulls(t *testing.T) {
require.NoError(t, err)
require.Equal(t, h-8, ts.Height())
require.NoError(t, tsc.revert(tsc.best()))
require.NoError(t, tsc.revert(tsc.best()))
require.Equal(t, h-8, tsc.best().Height())
best, err = tsc.best()
require.NoError(t, err)
require.NoError(t, tsc.revert(best))
best, err = tsc.best()
require.NoError(t, err)
require.NoError(t, tsc.revert(best))
best, err = tsc.best()
require.NoError(t, err)
require.Equal(t, h-8, best.Height())
h += 50
add()
@ -120,3 +142,27 @@ func TestTsCacheNulls(t *testing.T) {
require.NoError(t, err)
require.Equal(t, h-1, ts.Height())
}
type tsCacheAPIStorageCallCounter struct {
t *testing.T
chainGetTipSetByHeight int
chainHead int
}
func (tc *tsCacheAPIStorageCallCounter) ChainGetTipSetByHeight(ctx context.Context, epoch abi.ChainEpoch, key types.TipSetKey) (*types.TipSet, error) {
tc.chainGetTipSetByHeight++
return &types.TipSet{}, nil
}
func (tc *tsCacheAPIStorageCallCounter) ChainHead(ctx context.Context) (*types.TipSet, error) {
tc.chainHead++
return &types.TipSet{}, nil
}
func TestTsCacheEmpty(t *testing.T) {
// Calling best on an empty cache should just call out to the chain API
callCounter := &tsCacheAPIStorageCallCounter{t: t}
tsc := newTSCache(50, callCounter)
_, err := tsc.best()
require.NoError(t, err)
require.Equal(t, 1, callCounter.chainHead)
}

2
chain/blocksync/cbor_gen.go → chain/exchange/cbor_gen.go
View File

@ -1,6 +1,6 @@
// Code generated by github.com/whyrusleeping/cbor-gen. DO NOT EDIT.
package blocksync
package exchange
import (
"fmt"

140
chain/blocksync/client.go → chain/exchange/client.go
View File

@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"bufio"
@ -7,13 +7,16 @@ import (
"math/rand"
"time"
host "github.com/libp2p/go-libp2p-core/host"
inet "github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
"go.opencensus.io/trace"
"go.uber.org/fx"
"golang.org/x/xerrors"
cborutil "github.com/filecoin-project/go-cbor-util"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
@ -21,11 +24,9 @@ import (
"github.com/filecoin-project/lotus/lib/peermgr"
)
// Protocol client.
// FIXME: Rename to just `Client`. Not done at the moment to avoid
// disrupting too much of the consumer code, should be done along
// https://github.com/filecoin-project/lotus/issues/2612.
type BlockSync struct {
// client implements exchange.Client, using the libp2p ChainExchange protocol
// as the fetching mechanism.
type client struct {
// Connection manager used to contact the server.
// FIXME: We should have a reduced interface here, initialized
// just with our protocol ID, we shouldn't be able to open *any*
@ -35,13 +36,14 @@ type BlockSync struct {
peerTracker *bsPeerTracker
}
func NewClient(
host host.Host,
pmgr peermgr.MaybePeerMgr,
) *BlockSync {
return &BlockSync{
var _ Client = (*client)(nil)
// NewClient creates a new libp2p-based exchange.Client that uses the libp2p
// ChainExhange protocol as the fetching mechanism.
func NewClient(lc fx.Lifecycle, host host.Host, pmgr peermgr.MaybePeerMgr) Client {
return &client{
host: host,
peerTracker: newPeerTracker(pmgr.Mgr),
peerTracker: newPeerTracker(lc, host, pmgr.Mgr),
}
}
@ -62,11 +64,7 @@ func NewClient(
// request options without disrupting external calls. In the future the
// consumers should be forced to use a more standardized service and
// adhere to a single API derived from this function.
func (client *BlockSync) doRequest(
ctx context.Context,
req *Request,
singlePeer *peer.ID,
) (*validatedResponse, error) {
func (c *client) doRequest(ctx context.Context, req *Request, singlePeer *peer.ID) (*validatedResponse, error) {
// Validate request.
if req.Length == 0 {
return nil, xerrors.Errorf("invalid request of length 0")
@ -86,7 +84,7 @@ func (client *BlockSync) doRequest(
if singlePeer != nil {
peers = []peer.ID{*singlePeer}
} else {
peers = client.getShuffledPeers()
peers = c.getShuffledPeers()
if len(peers) == 0 {
return nil, xerrors.Errorf("no peers available")
}
@ -107,9 +105,9 @@ func (client *BlockSync) doRequest(
}
// Send request, read response.
res, err := client.sendRequestToPeer(ctx, peer, req)
res, err := c.sendRequestToPeer(ctx, peer, req)
if err != nil {
if !xerrors.Is(err, inet.ErrNoConn) {
if !xerrors.Is(err, network.ErrNoConn) {
log.Warnf("could not connect to peer %s: %s",
peer.String(), err)
}
@ -117,15 +115,15 @@ func (client *BlockSync) doRequest(
}
// Process and validate response.
validRes, err := client.processResponse(req, res)
validRes, err := c.processResponse(req, res)
if err != nil {
log.Warnf("processing peer %s response failed: %s",
peer.String(), err)
continue
}
client.peerTracker.logGlobalSuccess(build.Clock.Since(globalTime))
client.host.ConnManager().TagPeer(peer, "bsync", SUCCESS_PEER_TAG_VALUE)
c.peerTracker.logGlobalSuccess(build.Clock.Since(globalTime))
c.host.ConnManager().TagPeer(peer, "bsync", SuccessPeerTagValue)
return validRes, nil
}
@ -144,11 +142,8 @@ func (client *BlockSync) doRequest(
// We are conflating in the single error returned both status and validation
// errors. Peer penalization should happen here then, before returning, so
// we can apply the correct penalties depending on the cause of the error.
func (client *BlockSync) processResponse(
req *Request,
res *Response,
// FIXME: Add the `peer` as argument once we implement penalties.
) (*validatedResponse, error) {
// FIXME: Add the `peer` as argument once we implement penalties.
func (c *client) processResponse(req *Request, res *Response) (*validatedResponse, error) {
err := res.statusToError()
if err != nil {
return nil, xerrors.Errorf("status error: %s", err)
@ -246,16 +241,8 @@ func (client *BlockSync) processResponse(
return validRes, nil
}
// GetBlocks fetches count blocks from the network, from the provided tipset
// *backwards*, returning as many tipsets as count.
//
// {hint/usage}: This is used by the Syncer during normal chain syncing and when
// resolving forks.
func (client *BlockSync) GetBlocks(
ctx context.Context,
tsk types.TipSetKey,
count int,
) ([]*types.TipSet, error) {
// GetBlocks implements Client.GetBlocks(). Refer to the godocs there.
func (c *client) GetBlocks(ctx context.Context, tsk types.TipSetKey, count int) ([]*types.TipSet, error) {
ctx, span := trace.StartSpan(ctx, "bsync.GetBlocks")
defer span.End()
if span.IsRecordingEvents() {
@ -271,7 +258,7 @@ func (client *BlockSync) GetBlocks(
Options: Headers,
}
validRes, err := client.doRequest(ctx, req, nil)
validRes, err := c.doRequest(ctx, req, nil)
if err != nil {
return nil, err
}
@ -279,11 +266,8 @@ func (client *BlockSync) GetBlocks(
return validRes.tipsets, nil
}
func (client *BlockSync) GetFullTipSet(
ctx context.Context,
peer peer.ID,
tsk types.TipSetKey,
) (*store.FullTipSet, error) {
// GetFullTipSet implements Client.GetFullTipSet(). Refer to the godocs there.
func (c *client) GetFullTipSet(ctx context.Context, peer peer.ID, tsk types.TipSetKey) (*store.FullTipSet, error) {
// TODO: round robin through these peers on error
req := &Request{
@ -292,7 +276,7 @@ func (client *BlockSync) GetFullTipSet(
Options: Headers | Messages,
}
validRes, err := client.doRequest(ctx, req, &peer)
validRes, err := c.doRequest(ctx, req, &peer)
if err != nil {
return nil, err
}
@ -302,11 +286,8 @@ func (client *BlockSync) GetFullTipSet(
// *one* tipset here, so it's safe to index directly.
}
func (client *BlockSync) GetChainMessages(
ctx context.Context,
head *types.TipSet,
length uint64,
) ([]*CompactedMessages, error) {
// GetChainMessages implements Client.GetChainMessages(). Refer to the godocs there.
func (c *client) GetChainMessages(ctx context.Context, head *types.TipSet, length uint64) ([]*CompactedMessages, error) {
ctx, span := trace.StartSpan(ctx, "GetChainMessages")
if span.IsRecordingEvents() {
span.AddAttributes(
@ -322,7 +303,7 @@ func (client *BlockSync) GetChainMessages(
Options: Messages,
}
validRes, err := client.doRequest(ctx, req, nil)
validRes, err := c.doRequest(ctx, req, nil)
if err != nil {
return nil, err
}
@ -333,11 +314,7 @@ func (client *BlockSync) GetChainMessages(
// Send a request to a peer. Write request in the stream and read the
// response back. We do not do any processing of the request/response
// here.
func (client *BlockSync) sendRequestToPeer(
ctx context.Context,
peer peer.ID,
req *Request,
) (_ *Response, err error) {
func (c *client) sendRequestToPeer(ctx context.Context, peer peer.ID, req *Request) (_ *Response, err error) {
// Trace code.
ctx, span := trace.StartSpan(ctx, "sendRequestToPeer")
defer span.End()
@ -358,34 +335,33 @@ func (client *BlockSync) sendRequestToPeer(
}()
// -- TRACE --
supported, err := client.host.Peerstore().SupportsProtocols(peer, BlockSyncProtocolID)
supported, err := c.host.Peerstore().SupportsProtocols(peer, BlockSyncProtocolID, ChainExchangeProtocolID)
if err != nil {
c.RemovePeer(peer)
return nil, xerrors.Errorf("failed to get protocols for peer: %w", err)
}
if len(supported) == 0 || supported[0] != BlockSyncProtocolID {
return nil, xerrors.Errorf("peer %s does not support protocol %s",
peer, BlockSyncProtocolID)
// FIXME: `ProtoBook` should support a *single* protocol check that returns
// a bool instead of a list.
if len(supported) == 0 || (supported[0] != BlockSyncProtocolID && supported[0] != ChainExchangeProtocolID) {
return nil, xerrors.Errorf("peer %s does not support protocols %s",
peer, []string{BlockSyncProtocolID, ChainExchangeProtocolID})
}
connectionStart := build.Clock.Now()
// Open stream to peer.
stream, err := client.host.NewStream(
inet.WithNoDial(ctx, "should already have connection"),
stream, err := c.host.NewStream(
network.WithNoDial(ctx, "should already have connection"),
peer,
BlockSyncProtocolID)
ChainExchangeProtocolID, BlockSyncProtocolID)
if err != nil {
client.RemovePeer(peer)
c.RemovePeer(peer)
return nil, xerrors.Errorf("failed to open stream to peer: %w", err)
}
// Write request.
_ = stream.SetWriteDeadline(time.Now().Add(WRITE_REQ_DEADLINE))
_ = stream.SetWriteDeadline(time.Now().Add(WriteReqDeadline))
if err := cborutil.WriteCborRPC(stream, req); err != nil {
_ = stream.SetWriteDeadline(time.Time{})
client.peerTracker.logFailure(peer, build.Clock.Since(connectionStart))
c.peerTracker.logFailure(peer, build.Clock.Since(connectionStart), req.Length)
// FIXME: Should we also remove peer here?
return nil, err
}
@ -395,11 +371,11 @@ func (client *BlockSync) sendRequestToPeer(
// Read response.
var res Response
err = cborutil.ReadCborRPC(
bufio.NewReader(incrt.New(stream, READ_RES_MIN_SPEED, READ_RES_DEADLINE)),
bufio.NewReader(incrt.New(stream, ReadResMinSpeed, ReadResDeadline)),
&res)
if err != nil {
client.peerTracker.logFailure(peer, build.Clock.Since(connectionStart))
return nil, xerrors.Errorf("failed to read blocksync response: %w", err)
c.peerTracker.logFailure(peer, build.Clock.Since(connectionStart), req.Length)
return nil, xerrors.Errorf("failed to read chainxchg response: %w", err)
}
// FIXME: Move all this together at the top using a defer as done elsewhere.
@ -412,32 +388,34 @@ func (client *BlockSync) sendRequestToPeer(
)
}
client.peerTracker.logSuccess(peer, build.Clock.Since(connectionStart))
c.peerTracker.logSuccess(peer, build.Clock.Since(connectionStart), uint64(len(res.Chain)))
// FIXME: We should really log a success only after we validate the response.
// It might be a bit hard to do.
return &res, nil
}
func (client *BlockSync) AddPeer(p peer.ID) {
client.peerTracker.addPeer(p)
// AddPeer implements Client.AddPeer(). Refer to the godocs there.
func (c *client) AddPeer(p peer.ID) {
c.peerTracker.addPeer(p)
}
func (client *BlockSync) RemovePeer(p peer.ID) {
client.peerTracker.removePeer(p)
// RemovePeer implements Client.RemovePeer(). Refer to the godocs there.
func (c *client) RemovePeer(p peer.ID) {
c.peerTracker.removePeer(p)
}
// getShuffledPeers returns a preference-sorted set of peers (by latency
// and failure counting), shuffling the first few peers so we don't always
// pick the same peer.
// FIXME: Consider merging with `shufflePrefix()s`.
func (client *BlockSync) getShuffledPeers() []peer.ID {
peers := client.peerTracker.prefSortedPeers()
func (c *client) getShuffledPeers() []peer.ID {
peers := c.peerTracker.prefSortedPeers()
shufflePrefix(peers)
return peers
}
func shufflePrefix(peers []peer.ID) {
prefix := SHUFFLE_PEERS_PREFIX
prefix := ShufflePeersPrefix
if len(peers) < prefix {
prefix = len(peers)
}

19
chain/exchange/doc.go Normal file
View File

@ -0,0 +1,19 @@
// Package exchange contains the ChainExchange server and client components.
//
// ChainExchange is the basic chain synchronization protocol of Filecoin.
// ChainExchange is an RPC-oriented protocol, with a single operation to
// request blocks for now.
//
// A request contains a start anchor block (referred to with a CID), and a
// amount of blocks requested beyond the anchor (including the anchor itself).
//
// A client can also pass options, encoded as a 64-bit bitfield. Lotus supports
// two options at the moment:
//
// - include block contents
// - include block messages
//
// The response will include a status code, an optional message, and the
// response payload in case of success. The payload is a slice of serialized
// tipsets.
package exchange

51
chain/exchange/interfaces.go Normal file
View File

@ -0,0 +1,51 @@
package exchange
import (
"context"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
)
// Server is the responder side of the ChainExchange protocol. It accepts
// requests from clients and services them by returning the requested
// chain data.
type Server interface {
// HandleStream is the protocol handler to be registered on a libp2p
// protocol router.
//
// In the current version of the protocol, streams are single-use. The
// server will read a single Request, and will respond with a single
// Response. It will dispose of the stream straight after.
HandleStream(stream network.Stream)
}
// Client is the requesting side of the ChainExchange protocol. It acts as
// a proxy for other components to request chain data from peers. It is chiefly
// used by the Syncer.
type Client interface {
// GetBlocks fetches block headers from the network, from the provided
// tipset *backwards*, returning as many tipsets as the count parameter,
// or less.
GetBlocks(ctx context.Context, tsk types.TipSetKey, count int) ([]*types.TipSet, error)
// GetChainMessages fetches messages from the network, from the provided
// tipset *backwards*, returning the messages from as many tipsets as the
// count parameter, or less.
GetChainMessages(ctx context.Context, head *types.TipSet, length uint64) ([]*CompactedMessages, error)
// GetFullTipSet fetches a full tipset from a given peer. If successful,
// the fetched object contains block headers and all messages in full form.
GetFullTipSet(ctx context.Context, peer peer.ID, tsk types.TipSetKey) (*store.FullTipSet, error)
// AddPeer adds a peer to the pool of peers that the Client requests
// data from.
AddPeer(peer peer.ID)
// RemovePeer removes a peer from the pool of peers that the Client
// requests data from.
RemovePeer(peer peer.ID)
}

57
chain/blocksync/peer_tracker.go → chain/exchange/peer_tracker.go
View File

@ -1,13 +1,16 @@
package blocksync
package exchange
// FIXME: This needs to be reviewed.
import (
"context"
"sort"
"sync"
"time"
host "github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/peer"
"go.uber.org/fx"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/lib/peermgr"
@ -29,11 +32,30 @@ type bsPeerTracker struct {
pmgr *peermgr.PeerMgr
}
func newPeerTracker(pmgr *peermgr.PeerMgr) *bsPeerTracker {
return &bsPeerTracker{
func newPeerTracker(lc fx.Lifecycle, h host.Host, pmgr *peermgr.PeerMgr) *bsPeerTracker {
bsPt := &bsPeerTracker{
peers: make(map[peer.ID]*peerStats),
pmgr: pmgr,
}
sub, err := h.EventBus().Subscribe(new(peermgr.NewFilPeer))
if err != nil {
panic(err)
}
go func() {
for newPeer := range sub.Out() {
bsPt.addPeer(newPeer.(peermgr.NewFilPeer).Id)
}
}()
lc.Append(fx.Hook{
OnStop: func(ctx context.Context) error {
return sub.Close()
},
})
return bsPt
}
func (bpt *bsPeerTracker) addPeer(p peer.ID) {
@ -72,16 +94,7 @@ func (bpt *bsPeerTracker) prefSortedPeers() []peer.ID {
var costI, costJ float64
getPeerInitLat := func(p peer.ID) float64 {
var res float64
if bpt.pmgr != nil {
if lat, ok := bpt.pmgr.GetPeerLatency(p); ok {
res = float64(lat)
}
}
if res == 0 {
res = float64(bpt.avgGlobalTime)
}
return res * newPeerMul
return float64(bpt.avgGlobalTime) * newPeerMul
}
if pi.successes+pi.failures > 0 {
@ -107,8 +120,8 @@ func (bpt *bsPeerTracker) prefSortedPeers() []peer.ID {
const (
// xInvAlpha = (N+1)/2
localInvAlpha = 5 // 86% of the value is the last 9
globalInvAlpha = 20 // 86% of the value is the last 39
localInvAlpha = 10 // 86% of the value is the last 19
globalInvAlpha = 25 // 86% of the value is the last 49
)
func (bpt *bsPeerTracker) logGlobalSuccess(dur time.Duration) {
@ -133,7 +146,7 @@ func logTime(pi *peerStats, dur time.Duration) {
}
func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration) {
func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration, reqSize uint64) {
bpt.lk.Lock()
defer bpt.lk.Unlock()
@ -145,10 +158,13 @@ func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration) {
}
pi.successes++
logTime(pi, dur)
if reqSize == 0 {
reqSize = 1
}
logTime(pi, dur/time.Duration(reqSize))
}
func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration) {
func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration, reqSize uint64) {
bpt.lk.Lock()
defer bpt.lk.Unlock()
@ -160,7 +176,10 @@ func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration) {
}
pi.failures++
logTime(pi, dur)
if reqSize == 0 {
reqSize = 1
}
logTime(pi, dur/time.Duration(reqSize))
}
func (bpt *bsPeerTracker) removePeer(p peer.ID) {

34
chain/blocksync/protocol.go → chain/exchange/protocol.go
View File

@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"time"
@ -13,9 +13,17 @@ import (
"github.com/filecoin-project/lotus/chain/types"
)
var log = logging.Logger("blocksync")
var log = logging.Logger("chainxchg")
const BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
const (
// BlockSyncProtocolID is the protocol ID of the former blocksync protocol.
// Deprecated.
BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
// ChainExchangeProtocolID is the protocol ID of the chain exchange
// protocol.
ChainExchangeProtocolID = "/fil/chain/xchg/0.0.1"
)
// FIXME: Bumped from original 800 to this to accommodate `syncFork()`
// use of `GetBlocks()`. It seems the expectation of that API is to
@ -25,14 +33,16 @@ const BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
// qualifier to avoid "const initializer [...] is not a constant" error.)
var MaxRequestLength = uint64(build.ForkLengthThreshold)
// Extracted constants from the code.
// FIXME: Should be reviewed and confirmed.
const SUCCESS_PEER_TAG_VALUE = 25
const WRITE_REQ_DEADLINE = 5 * time.Second
const READ_RES_DEADLINE = WRITE_REQ_DEADLINE
const READ_RES_MIN_SPEED = 50 << 10
const SHUFFLE_PEERS_PREFIX = 5
const WRITE_RES_DEADLINE = 60 * time.Second
const (
// Extracted constants from the code.
// FIXME: Should be reviewed and confirmed.
SuccessPeerTagValue = 25
WriteReqDeadline = 5 * time.Second
ReadResDeadline = WriteReqDeadline
ReadResMinSpeed = 50 << 10
ShufflePeersPrefix = 5
WriteResDeadline = 60 * time.Second
)
// FIXME: Rename. Make private.
type Request struct {
@ -117,7 +127,7 @@ func (res *Response) statusToError() error {
case NotFound:
return xerrors.Errorf("not found")
case GoAway:
return xerrors.Errorf("not handling 'go away' blocksync responses yet")
return xerrors.Errorf("not handling 'go away' chainxchg responses yet")
case InternalError:
return xerrors.Errorf("block sync peer errored: %s", res.ErrorMessage)
case BadRequest:

71
chain/blocksync/server.go → chain/exchange/server.go
View File

@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"bufio"
@ -18,38 +18,25 @@ import (
inet "github.com/libp2p/go-libp2p-core/network"
)
// BlockSyncService is the component that services BlockSync requests from
// peers.
//
// BlockSync is the basic chain synchronization protocol of Filecoin. BlockSync
// is an RPC-oriented protocol, with a single operation to request blocks.
//
// A request contains a start anchor block (referred to with a CID), and a
// amount of blocks requested beyond the anchor (including the anchor itself).
//
// A client can also pass options, encoded as a 64-bit bitfield. Lotus supports
// two options at the moment:
//
// - include block contents
// - include block messages
//
// The response will include a status code, an optional message, and the
// response payload in case of success. The payload is a slice of serialized
// tipsets.
// FIXME: Rename to just `Server` (will be done later, see note on `BlockSync`).
type BlockSyncService struct {
// server implements exchange.Server. It services requests for the
// libp2p ChainExchange protocol.
type server struct {
cs *store.ChainStore
}
func NewBlockSyncService(cs *store.ChainStore) *BlockSyncService {
return &BlockSyncService{
var _ Server = (*server)(nil)
// NewServer creates a new libp2p-based exchange.Server. It services requests
// for the libp2p ChainExchange protocol.
func NewServer(cs *store.ChainStore) Server {
return &server{
cs: cs,
}
}
// Entry point of the service, handles `Request`s.
func (server *BlockSyncService) HandleStream(stream inet.Stream) {
ctx, span := trace.StartSpan(context.Background(), "blocksync.HandleStream")
// HandleStream implements Server.HandleStream. Refer to the godocs there.
func (s *server) HandleStream(stream inet.Stream) {
ctx, span := trace.StartSpan(context.Background(), "chainxchg.HandleStream")
defer span.End()
defer stream.Close() //nolint:errcheck
@ -62,13 +49,13 @@ func (server *BlockSyncService) HandleStream(stream inet.Stream) {
log.Infow("block sync request",
"start", req.Head, "len", req.Length)
resp, err := server.processRequest(ctx, &req)
resp, err := s.processRequest(ctx, &req)
if err != nil {
log.Warn("failed to process request: ", err)
return
}
_ = stream.SetDeadline(time.Now().Add(WRITE_RES_DEADLINE))
_ = stream.SetDeadline(time.Now().Add(WriteResDeadline))
if err := cborutil.WriteCborRPC(stream, resp); err != nil {
_ = stream.SetDeadline(time.Time{})
log.Warnw("failed to write back response for handle stream",
@ -80,10 +67,7 @@ func (server *BlockSyncService) HandleStream(stream inet.Stream) {
// Validate and service the request. We return either a protocol
// response or an internal error.
func (server *BlockSyncService) processRequest(
ctx context.Context,
req *Request,
) (*Response, error) {
func (s *server) processRequest(ctx context.Context, req *Request) (*Response, error) {
validReq, errResponse := validateRequest(ctx, req)
if errResponse != nil {
// The request did not pass validation, return the response
@ -91,17 +75,14 @@ func (server *BlockSyncService) processRequest(
return errResponse, nil
}
return server.serviceRequest(ctx, validReq)
return s.serviceRequest(ctx, validReq)
}
// Validate request. We either return a `validatedRequest`, or an error
// `Response` indicating why we can't process it. We do not return any
// internal errors here, we just signal protocol ones.
func validateRequest(
ctx context.Context,
req *Request,
) (*validatedRequest, *Response) {
_, span := trace.StartSpan(ctx, "blocksync.ValidateRequest")
func validateRequest(ctx context.Context, req *Request) (*validatedRequest, *Response) {
_, span := trace.StartSpan(ctx, "chainxchg.ValidateRequest")
defer span.End()
validReq := validatedRequest{}
@ -147,14 +128,11 @@ func validateRequest(
return &validReq, nil
}
func (server *BlockSyncService) serviceRequest(
ctx context.Context,
req *validatedRequest,
) (*Response, error) {
_, span := trace.StartSpan(ctx, "blocksync.ServiceRequest")
func (s *server) serviceRequest(ctx context.Context, req *validatedRequest) (*Response, error) {
_, span := trace.StartSpan(ctx, "chainxchg.ServiceRequest")
defer span.End()
chain, err := collectChainSegment(server.cs, req)
chain, err := collectChainSegment(s.cs, req)
if err != nil {
log.Warn("block sync request: collectChainSegment failed: ", err)
return &Response{
@ -174,10 +152,7 @@ func (server *BlockSyncService) serviceRequest(
}, nil
}
func collectChainSegment(
cs *store.ChainStore,
req *validatedRequest,
) ([]*BSTipSet, error) {
func collectChainSegment(cs *store.ChainStore, req *validatedRequest) ([]*BSTipSet, error) {
var bstips []*BSTipSet
cur := req.head

22
chain/gen/gen.go
View File

@ -8,11 +8,13 @@ import (
"sync/atomic"
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
saminer "github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/crypto"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-blockservice"
"github.com/ipfs/go-cid"
@ -46,7 +48,7 @@ const msgsPerBlock = 20
//nolint:deadcode,varcheck
var log = logging.Logger("gen")
var ValidWpostForTesting = []abi.PoStProof{{
var ValidWpostForTesting = []proof.PoStProof{{
ProofBytes: []byte("valid proof"),
}}
@ -457,7 +459,7 @@ func (cg *ChainGen) NextTipSetFromMinersWithMessages(base *types.TipSet, miners
func (cg *ChainGen) makeBlock(parents *types.TipSet, m address.Address, vrfticket *types.Ticket,
eticket *types.ElectionProof, bvals []types.BeaconEntry, height abi.ChainEpoch,
wpost []abi.PoStProof, msgs []*types.SignedMessage) (*types.FullBlock, error) {
wpost []proof.PoStProof, msgs []*types.SignedMessage) (*types.FullBlock, error) {
var ts uint64
if cg.Timestamper != nil {
@ -588,7 +590,7 @@ func (mca mca) WalletSign(ctx context.Context, a address.Address, v []byte) (*cr
type WinningPoStProver interface {
GenerateCandidates(context.Context, abi.PoStRandomness, uint64) ([]uint64, error)
ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error)
ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error)
}
type wppProvider struct{}
@ -597,7 +599,7 @@ func (wpp *wppProvider) GenerateCandidates(ctx context.Context, _ abi.PoStRandom
return []uint64{0}, nil
}
func (wpp *wppProvider) ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error) {
func (wpp *wppProvider) ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error) {
return ValidWpostForTesting, nil
}
@ -664,15 +666,15 @@ type genFakeVerifier struct{}
var _ ffiwrapper.Verifier = (*genFakeVerifier)(nil)
func (m genFakeVerifier) VerifySeal(svi abi.SealVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifySeal(svi proof.SealVerifyInfo) (bool, error) {
return true, nil
}
func (m genFakeVerifier) VerifyWinningPoSt(ctx context.Context, info abi.WinningPoStVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifyWinningPoSt(ctx context.Context, info proof.WinningPoStVerifyInfo) (bool, error) {
panic("not supported")
}
func (m genFakeVerifier) VerifyWindowPoSt(ctx context.Context, info abi.WindowPoStVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifyWindowPoSt(ctx context.Context, info proof.WindowPoStVerifyInfo) (bool, error) {
panic("not supported")
}

4
chain/gen/gen_test.go
View File

@ -3,8 +3,8 @@ package gen
import (
"testing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"

13
chain/gen/genesis/genesis.go
View File

@ -6,6 +6,8 @@ import (
"encoding/json"
"fmt"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
@ -14,13 +16,13 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/account"
"github.com/filecoin-project/specs-actors/actors/builtin/multisig"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/build"
@ -404,6 +406,10 @@ func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot ci
verifNeeds := make(map[address.Address]abi.PaddedPieceSize)
var sum abi.PaddedPieceSize
nwv := func(context.Context, abi.ChainEpoch) runtime.NetworkVersion {
return runtime.NetworkVersion1
}
vmopt := vm.VMOpts{
StateBase: stateroot,
Epoch: 0,
@ -411,6 +417,7 @@ func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot ci
Bstore: cs.Blockstore(),
Syscalls: mkFakedSigSyscalls(cs.VMSys()),
CircSupplyCalc: nil,
NtwkVersion: nwv,
BaseFee: types.NewInt(0),
}
vm, err := vm.NewVM(&vmopt)

11
chain/gen/genesis/miners.go
View File

@ -14,15 +14,15 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/lotus/chain/store"
@ -61,6 +61,10 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
return big.Zero(), nil
}
nwv := func(context.Context, abi.ChainEpoch) runtime.NetworkVersion {
return runtime.NetworkVersion1
}
vmopt := &vm.VMOpts{
StateBase: sroot,
Epoch: 0,
@ -68,6 +72,7 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
Bstore: cs.Blockstore(),
Syscalls: mkFakedSigSyscalls(cs.VMSys()),
CircSupplyCalc: csc,
NtwkVersion: nwv,
BaseFee: types.NewInt(0),
}

2
chain/gen/genesis/t02_reward.go
View File

@ -3,7 +3,7 @@ package genesis
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"

2
chain/gen/genesis/util.go
View File

@ -4,7 +4,7 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"

2
chain/gen/mining.go
View File

@ -3,7 +3,7 @@ package gen
import (
"context"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
cid "github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"

2
chain/gen/slashfilter/slashfilter.go
View File

@ -9,8 +9,8 @@ import (
ds "github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/namespace"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
)
type SlashFilter struct {

4
chain/market/fundmgr.go
View File

@ -4,8 +4,8 @@ import (
"context"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"go.uber.org/fx"
"github.com/filecoin-project/specs-actors/actors/builtin"

4
chain/market/fundmgr_test.go
View File

@ -10,9 +10,9 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
tutils "github.com/filecoin-project/specs-actors/support/testing"
"github.com/filecoin-project/lotus/api"

2
chain/messagepool/gasguess/guessgas.go
View File

@ -9,7 +9,7 @@ import (
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
)

111
chain/messagepool/messagepool.go
View File

@ -11,8 +11,8 @@ import (
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/hashicorp/go-multierror"
lru "github.com/hashicorp/golang-lru"
"github.com/ipfs/go-cid"
@ -46,9 +46,10 @@ var rbfDenomBig = types.NewInt(RbfDenom)
const RbfDenom = 256
var RepublishInterval = pubsub.TimeCacheDuration + time.Duration(5*build.BlockDelaySecs+build.PropagationDelaySecs)*time.Second
var RepublishInterval = time.Duration(10*build.BlockDelaySecs+build.PropagationDelaySecs) * time.Second
var minimumBaseFee = types.NewInt(uint64(build.MinimumBaseFee))
var baseFeeLowerBoundFactor = types.NewInt(10)
var MaxActorPendingMessages = 1000
@ -81,6 +82,18 @@ const (
localUpdates = "update"
)
// this is *temporary* mutilation until we have implemented uncapped miner penalties -- it will go
// away in the next fork.
var strictBaseFeeValidation = false
func init() {
// if the republish interval is too short compared to the pubsub timecache, adjust it
minInterval := pubsub.TimeCacheDuration + time.Duration(build.PropagationDelaySecs)
if RepublishInterval < minInterval {
RepublishInterval = minInterval
}
}
type MessagePool struct {
lk sync.Mutex
@ -142,6 +155,11 @@ func newMsgSet(nonce uint64) *msgSet {
}
}
func ComputeMinRBF(curPrem abi.TokenAmount) abi.TokenAmount {
minPrice := types.BigAdd(curPrem, types.BigDiv(types.BigMul(curPrem, rbfNumBig), rbfDenomBig))
return types.BigAdd(minPrice, types.NewInt(1))
}
func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict bool) (bool, error) {
nextNonce := ms.nextNonce
nonceGap := false
@ -169,9 +187,7 @@ func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict bool) (boo
if m.Cid() != exms.Cid() {
// check if RBF passes
minPrice := exms.Message.GasPremium
minPrice = types.BigAdd(minPrice, types.BigDiv(types.BigMul(minPrice, rbfNumBig), rbfDenomBig))
minPrice = types.BigAdd(minPrice, types.NewInt(1))
minPrice := ComputeMinRBF(exms.Message.GasPremium)
if types.BigCmp(m.Message.GasPremium, minPrice) >= 0 {
log.Infow("add with RBF", "oldpremium", exms.Message.GasPremium,
"newpremium", m.Message.GasPremium, "addr", m.Message.From, "nonce", m.Message.Nonce)
@ -355,13 +371,48 @@ func (mp *MessagePool) addLocal(m *types.SignedMessage, msgb []byte) error {
return nil
}
func (mp *MessagePool) verifyMsgBeforeAdd(m *types.SignedMessage, epoch abi.ChainEpoch) error {
// verifyMsgBeforeAdd verifies that the message meets the minimum criteria for block inclusio
// and whether the message has enough funds to be included in the next 20 blocks.
// If the message is not valid for block inclusion, it returns an error.
// For local messages, if the message can be included in the next 20 blocks, it returns true to
// signal that it should be immediately published. If the message cannot be included in the next 20
// blocks, it returns false so that the message doesn't immediately get published (and ignored by our
// peers); instead it will be published through the republish loop, once the base fee has fallen
// sufficiently.
// For non local messages, if the message cannot be included in the next 20 blocks it returns
// a (soft) validation error.
func (mp *MessagePool) verifyMsgBeforeAdd(m *types.SignedMessage, curTs *types.TipSet, local bool) (bool, error) {
epoch := curTs.Height()
minGas := vm.PricelistByEpoch(epoch).OnChainMessage(m.ChainLength())
if err := m.VMMessage().ValidForBlockInclusion(minGas.Total()); err != nil {
return xerrors.Errorf("message will not be included in a block: %w", err)
return false, xerrors.Errorf("message will not be included in a block: %w", err)
}
return nil
// this checks if the GasFeeCap is suffisciently high for inclusion in the next 20 blocks
// if the GasFeeCap is too low, we soft reject the message (Ignore in pubsub) and rely
// on republish to push it through later, if the baseFee has fallen.
// this is a defensive check that stops minimum baseFee spam attacks from overloading validation
// queues.
// Note that for local messages, we always add them so that they can be accepted and republished
// automatically.
publish := local
if strictBaseFeeValidation && len(curTs.Blocks()) > 0 {
baseFee := curTs.Blocks()[0].ParentBaseFee
baseFeeLowerBound := types.BigDiv(baseFee, baseFeeLowerBoundFactor)
if m.Message.GasFeeCap.LessThan(baseFeeLowerBound) {
if local {
log.Warnf("local message will not be immediately published because GasFeeCap doesn't meet the lower bound for inclusion in the next 20 blocks (GasFeeCap: %s, baseFeeLowerBound: %s)",
m.Message.GasFeeCap, baseFeeLowerBound)
publish = false
} else {
return false, xerrors.Errorf("GasFeeCap doesn't meet base fee lower bound for inclusion in the next 20 blocks (GasFeeCap: %s, baseFeeLowerBound: %s): %w",
m.Message.GasFeeCap, baseFeeLowerBound, ErrSoftValidationFailure)
}
}
}
return publish, nil
}
func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
@ -382,7 +433,8 @@ func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
}
mp.curTsLk.Lock()
if err := mp.addTs(m, mp.curTs); err != nil {
publish, err := mp.addTs(m, mp.curTs, true)
if err != nil {
mp.curTsLk.Unlock()
return cid.Undef, err
}
@ -395,7 +447,11 @@ func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
}
mp.lk.Unlock()
return m.Cid(), mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
if publish {
err = mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
}
return m.Cid(), err
}
func (mp *MessagePool) checkMessage(m *types.SignedMessage) error {
@ -443,7 +499,9 @@ func (mp *MessagePool) Add(m *types.SignedMessage) error {
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
return mp.addTs(m, mp.curTs)
_, err = mp.addTs(m, mp.curTs, false)
return err
}
func sigCacheKey(m *types.SignedMessage) (string, error) {
@ -510,28 +568,29 @@ func (mp *MessagePool) checkBalance(m *types.SignedMessage, curTs *types.TipSet)
return nil
}
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet) error {
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet, local bool) (bool, error) {
snonce, err := mp.getStateNonce(m.Message.From, curTs)
if err != nil {
return xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrSoftValidationFailure)
return false, xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrSoftValidationFailure)
}
if snonce > m.Message.Nonce {
return xerrors.Errorf("minimum expected nonce is %d: %w", snonce, ErrNonceTooLow)
return false, xerrors.Errorf("minimum expected nonce is %d: %w", snonce, ErrNonceTooLow)
}
mp.lk.Lock()
defer mp.lk.Unlock()
if err := mp.verifyMsgBeforeAdd(m, curTs.Height()); err != nil {
return err
publish, err := mp.verifyMsgBeforeAdd(m, curTs, local)
if err != nil {
return false, err
}
if err := mp.checkBalance(m, curTs); err != nil {
return err
return false, err
}
return mp.addLocked(m, true)
return publish, mp.addLocked(m, !local)
}
func (mp *MessagePool) addLoaded(m *types.SignedMessage) error {
@ -557,7 +616,8 @@ func (mp *MessagePool) addLoaded(m *types.SignedMessage) error {
mp.lk.Lock()
defer mp.lk.Unlock()
if err := mp.verifyMsgBeforeAdd(m, curTs.Height()); err != nil {
_, err = mp.verifyMsgBeforeAdd(m, curTs, true)
if err != nil {
return err
}
@ -743,7 +803,8 @@ func (mp *MessagePool) PushWithNonce(ctx context.Context, addr address.Address,
return nil, ErrTryAgain
}
if err := mp.verifyMsgBeforeAdd(msg, curTs.Height()); err != nil {
publish, err := mp.verifyMsgBeforeAdd(msg, curTs, true)
if err != nil {
return nil, err
}
@ -751,14 +812,18 @@ func (mp *MessagePool) PushWithNonce(ctx context.Context, addr address.Address,
return nil, err
}
if err := mp.addLocked(msg, true); err != nil {
if err := mp.addLocked(msg, false); err != nil {
return nil, xerrors.Errorf("add locked failed: %w", err)
}
if err := mp.addLocal(msg, msgb); err != nil {
log.Errorf("addLocal failed: %+v", err)
}
return msg, mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
if publish {
err = mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
}
return msg, err
}
func (mp *MessagePool) Remove(from address.Address, nonce uint64, applied bool) {

10
chain/messagepool/messagepool_test.go
View File

@ -7,6 +7,8 @@ import (
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
@ -14,7 +16,6 @@ import (
_ "github.com/filecoin-project/lotus/lib/sigs/bls"
_ "github.com/filecoin-project/lotus/lib/sigs/secp"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
logging "github.com/ipfs/go-log/v2"
@ -56,6 +57,13 @@ func (tma *testMpoolAPI) nextBlock() *types.BlockHeader {
return newBlk
}
func (tma *testMpoolAPI) nextBlockWithHeight(height uint64) *types.BlockHeader {
newBlk := mock.MkBlock(tma.tipsets[len(tma.tipsets)-1], 1, 1)
newBlk.Height = abi.ChainEpoch(height)
tma.tipsets = append(tma.tipsets, mock.TipSet(newBlk))
return newBlk
}
func (tma *testMpoolAPI) applyBlock(t *testing.T, b *types.BlockHeader) {
t.Helper()
if err := tma.cb(nil, []*types.TipSet{mock.TipSet(b)}); err != nil {

28
chain/messagepool/repub.go
View File

@ -16,7 +16,7 @@ import (
const repubMsgLimit = 30
var RepublishBatchDelay = 200 * time.Millisecond
var RepublishBatchDelay = 100 * time.Millisecond
func (mp *MessagePool) republishPendingMessages() error {
mp.curTsLk.Lock()
@ -28,6 +28,11 @@ func (mp *MessagePool) republishPendingMessages() error {
return xerrors.Errorf("computing basefee: %w", err)
}
baseFeeLowerBound := types.BigDiv(baseFee, baseFeeLowerBoundFactor)
if baseFeeLowerBoundFactor.LessThan(minimumBaseFee) {
baseFeeLowerBound = minimumBaseFee
}
pending := make(map[address.Address]map[uint64]*types.SignedMessage)
mp.lk.Lock()
mp.republished = nil // clear this to avoid races triggering an early republish
@ -55,7 +60,11 @@ func (mp *MessagePool) republishPendingMessages() error {
var chains []*msgChain
for actor, mset := range pending {
next := mp.createMessageChains(actor, mset, baseFee, ts)
// We use the baseFee lower bound for createChange so that we optimistically include
// chains that might become profitable in the next 20 blocks.
// We still check the lowerBound condition for individual messages so that we don't send
// messages that will be rejected by the mpool spam protector, so this is safe to do.
next := mp.createMessageChains(actor, mset, baseFeeLowerBound, ts)
chains = append(chains, next...)
}
@ -70,6 +79,7 @@ func (mp *MessagePool) republishPendingMessages() error {
gasLimit := int64(build.BlockGasLimit)
minGas := int64(gasguess.MinGas)
var msgs []*types.SignedMessage
loop:
for i := 0; i < len(chains); {
chain := chains[i]
@ -91,8 +101,18 @@ func (mp *MessagePool) republishPendingMessages() error {
// does it fit in a block?
if chain.gasLimit <= gasLimit {
gasLimit -= chain.gasLimit
msgs = append(msgs, chain.msgs...)
// check the baseFee lower bound -- only republish messages that can be included in the chain
// within the next 20 blocks.
for _, m := range chain.msgs {
if !allowNegativeChains(ts.Height()) && m.Message.GasFeeCap.LessThan(baseFeeLowerBound) {
chain.Invalidate()
continue loop
}
gasLimit -= m.Message.GasLimit
msgs = append(msgs, m)
}
// we processed the whole chain, advance
i++
continue
}

2
chain/messagepool/repub_test.go
View File

@ -4,10 +4,10 @@ import (
"testing"
"time"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-datastore"
)

126
chain/messagepool/selection.go
View File

@ -3,21 +3,29 @@ package messagepool
import (
"context"
"math/big"
"math/rand"
"sort"
"time"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
tbig "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
abig "github.com/filecoin-project/specs-actors/actors/abi/big"
)
var bigBlockGasLimit = big.NewInt(build.BlockGasLimit)
// this is *temporary* mutilation until we have implemented uncapped miner penalties -- it will go
// away in the next fork.
func allowNegativeChains(epoch abi.ChainEpoch) bool {
return epoch < build.UpgradeBreezeHeight+5
}
const MaxBlocks = 15
type msgChain struct {
@ -100,7 +108,7 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
return chains[i].Before(chains[j])
})
if len(chains) != 0 && chains[0].gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all messages in mpool have non-positive gas performance", "bestGasPerf", chains[0].gasPerf)
return result, nil
}
@ -153,7 +161,7 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
last := len(chains)
for i, chain := range chains {
// did we run out of performing chains?
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break
}
@ -191,9 +199,11 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
gasLimit -= chainGasLimit
// resort to account for already merged chains and effective performance adjustments
sort.Slice(chains[i+1:], func(i, j int) bool {
// the sort *must* be stable or we end up getting negative gasPerfs pushed up.
sort.SliceStable(chains[i+1:], func(i, j int) bool {
return chains[i].BeforeEffective(chains[j])
})
continue
}
@ -217,7 +227,7 @@ tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim if necessary
if chains[last].gasLimit > gasLimit {
chains[last].Trim(gasLimit, mp, baseFee, false)
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
}
// push down if it hasn't been invalidated
@ -243,7 +253,7 @@ tailLoop:
}
// if gasPerf < 0 we have no more profitable chains
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break tailLoop
}
@ -284,7 +294,7 @@ tailLoop:
}
// dependencies fit, just trim it
chain.Trim(gasLimit-depGasLimit, mp, baseFee, false)
chain.Trim(gasLimit-depGasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
last += i
continue tailLoop
}
@ -297,6 +307,79 @@ tailLoop:
log.Infow("pack tail chains done", "took", dt)
}
// if we have gasLimit to spare, pick some random (non-negative) chains to fill the block
// we pick randomly so that we minimize the probability of duplication among all miners
if gasLimit >= minGas {
randomCount := 0
startRandom := time.Now()
shuffleChains(chains)
for _, chain := range chains {
// have we filled the block
if gasLimit < minGas {
break
}
// has it been merged or invalidated?
if chain.merged || !chain.valid {
continue
}
// is it negative?
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
continue
}
// compute the dependencies that must be merged and the gas limit including deps
chainGasLimit := chain.gasLimit
depGasLimit := int64(0)
var chainDeps []*msgChain
for curChain := chain.prev; curChain != nil && !curChain.merged; curChain = curChain.prev {
chainDeps = append(chainDeps, curChain)
chainGasLimit += curChain.gasLimit
depGasLimit += curChain.gasLimit
}
// do the deps fit? if the deps won't fit, invalidate the chain
if depGasLimit > gasLimit {
chain.Invalidate()
continue
}
// do they fit as is? if it doesn't, trim to make it fit if possible
if chainGasLimit > gasLimit {
chain.Trim(gasLimit-depGasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
if !chain.valid {
continue
}
}
// include it together with all dependencies
for i := len(chainDeps) - 1; i >= 0; i-- {
curChain := chainDeps[i]
curChain.merged = true
result = append(result, curChain.msgs...)
randomCount += len(curChain.msgs)
}
chain.merged = true
result = append(result, chain.msgs...)
randomCount += len(chain.msgs)
gasLimit -= chainGasLimit
}
if dt := time.Since(startRandom); dt > time.Millisecond {
log.Infow("pack random tail chains done", "took", dt)
}
if randomCount > 0 {
log.Warnf("optimal selection failed to pack a block; picked %d messages with random selection",
randomCount)
}
}
return result, nil
}
@ -349,7 +432,7 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
return chains[i].Before(chains[j])
})
if len(chains) != 0 && chains[0].gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all messages in mpool have non-positive gas performance", "bestGasPerf", chains[0].gasPerf)
return result, nil
}
@ -360,7 +443,7 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
last := len(chains)
for i, chain := range chains {
// did we run out of performing chains?
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break
}
@ -389,7 +472,7 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim
chains[last].Trim(gasLimit, mp, baseFee, false)
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
// push down if it hasn't been invalidated
if chains[last].valid {
@ -409,7 +492,7 @@ tailLoop:
}
// if gasPerf < 0 we have no more profitable chains
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break tailLoop
}
@ -471,7 +554,7 @@ func (mp *MessagePool) selectPriorityMessages(pending map[address.Address]map[ui
return chains[i].Before(chains[j])
})
if len(chains) != 0 && chains[0].gasPerf < 0 {
if !allowNegativeChains(ts.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all priority messages in mpool have negative gas performance", "bestGasPerf", chains[0].gasPerf)
return nil, gasLimit
}
@ -479,7 +562,7 @@ func (mp *MessagePool) selectPriorityMessages(pending map[address.Address]map[ui
// 3. Merge chains until the block limit, as long as they have non-negative gas performance
last := len(chains)
for i, chain := range chains {
if chain.gasPerf < 0 {
if !allowNegativeChains(ts.Height()) && chain.gasPerf < 0 {
break
}
@ -497,7 +580,7 @@ func (mp *MessagePool) selectPriorityMessages(pending map[address.Address]map[ui
tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim, discarding negative performing messages
chains[last].Trim(gasLimit, mp, baseFee, false)
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(ts.Height()))
// push down if it hasn't been invalidated
if chains[last].valid {
@ -517,7 +600,7 @@ tailLoop:
}
// if gasPerf < 0 we have no more profitable chains
if chain.gasPerf < 0 {
if !allowNegativeChains(ts.Height()) && chain.gasPerf < 0 {
break tailLoop
}
@ -592,7 +675,7 @@ func (*MessagePool) getGasReward(msg *types.SignedMessage, baseFee types.BigInt)
maxPremium = msg.Message.GasPremium
}
gasReward := abig.Mul(maxPremium, types.NewInt(uint64(msg.Message.GasLimit)))
gasReward := tbig.Mul(maxPremium, types.NewInt(uint64(msg.Message.GasLimit)))
return gasReward.Int
}
@ -841,7 +924,16 @@ func (mc *msgChain) SetNullEffectivePerf() {
func (mc *msgChain) BeforeEffective(other *msgChain) bool {
// move merged chains to the front so we can discard them earlier
return (mc.merged && !other.merged) || mc.effPerf > other.effPerf ||
return (mc.merged && !other.merged) ||
(mc.gasPerf >= 0 && other.gasPerf < 0) ||
mc.effPerf > other.effPerf ||
(mc.effPerf == other.effPerf && mc.gasPerf > other.gasPerf) ||
(mc.effPerf == other.effPerf && mc.gasPerf == other.gasPerf && mc.gasReward.Cmp(other.gasReward) > 0)
}
func shuffleChains(lst []*msgChain) {
for i := range lst {
j := rand.Intn(i + 1)
lst[i], lst[j] = lst[j], lst[i]
}
}

183
chain/messagepool/selection_test.go
View File

@ -1,21 +1,26 @@
package messagepool
import (
"compress/gzip"
"context"
"encoding/json"
"fmt"
"io"
"math"
"math/big"
"math/rand"
"os"
"sort"
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
@ -729,6 +734,8 @@ func TestPriorityMessageSelection2(t *testing.T) {
}
func TestPriorityMessageSelection3(t *testing.T) {
t.Skip("reenable after removing allow negative")
mp, tma := makeTestMpool()
// the actors
@ -1279,3 +1286,177 @@ func TestGasReward(t *testing.T) {
})
}
}
func TestRealWorldSelection(t *testing.T) {
// load test-messages.json.gz and rewrite the messages so that
// 1) we map each real actor to a test actor so that we can sign the messages
// 2) adjust the nonces so that they start from 0
file, err := os.Open("test-messages.json.gz")
if err != nil {
t.Fatal(err)
}
gzr, err := gzip.NewReader(file)
if err != nil {
t.Fatal(err)
}
dec := json.NewDecoder(gzr)
var msgs []*types.SignedMessage
baseNonces := make(map[address.Address]uint64)
readLoop:
for {
m := new(types.SignedMessage)
err := dec.Decode(m)
switch err {
case nil:
msgs = append(msgs, m)
nonce, ok := baseNonces[m.Message.From]
if !ok || m.Message.Nonce < nonce {
baseNonces[m.Message.From] = m.Message.Nonce
}
case io.EOF:
break readLoop
default:
t.Fatal(err)
}
}
actorMap := make(map[address.Address]address.Address)
actorWallets := make(map[address.Address]*wallet.Wallet)
for _, m := range msgs {
baseNonce := baseNonces[m.Message.From]
localActor, ok := actorMap[m.Message.From]
if !ok {
w, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a, err := w.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
actorMap[m.Message.From] = a
actorWallets[a] = w
localActor = a
}
w, ok := actorWallets[localActor]
if !ok {
t.Fatalf("failed to lookup wallet for actor %s", localActor)
}
m.Message.From = localActor
m.Message.Nonce -= baseNonce
sig, err := w.Sign(context.TODO(), localActor, m.Message.Cid().Bytes())
if err != nil {
t.Fatal(err)
}
m.Signature = *sig
}
mp, tma := makeTestMpool()
block := tma.nextBlockWithHeight(build.UpgradeBreezeHeight + 10)
ts := mock.TipSet(block)
tma.applyBlock(t, block)
for _, a := range actorMap {
tma.setBalance(a, 1000000)
}
tma.baseFee = types.NewInt(800_000_000)
sort.Slice(msgs, func(i, j int) bool {
return msgs[i].Message.Nonce < msgs[j].Message.Nonce
})
// add the messages
for _, m := range msgs {
mustAdd(t, mp, m)
}
// do message selection and check block packing
minGasLimit := int64(0.9 * float64(build.BlockGasLimit))
// greedy first
selected, err := mp.SelectMessages(ts, 1.0)
if err != nil {
t.Fatal(err)
}
gasLimit := int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=1.0; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// high quality ticket
selected, err = mp.SelectMessages(ts, .8)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.8; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// mid quality ticket
selected, err = mp.SelectMessages(ts, .4)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.4; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// low quality ticket
selected, err = mp.SelectMessages(ts, .1)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.1; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// very low quality ticket
selected, err = mp.SelectMessages(ts, .01)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.01; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
}

BIN
chain/messagepool/test-messages.json.gz Normal file
View File

Binary file not shown.

2
chain/metrics/consensus.go
View File

@ -4,7 +4,7 @@ import (
"context"
"encoding/json"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
pubsub "github.com/libp2p/go-libp2p-pubsub"

6
chain/stmgr/call.go
View File

@ -5,8 +5,8 @@ import (
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/ipfs/go-cid"
"go.opencensus.io/trace"
"golang.org/x/xerrors"
@ -29,6 +29,7 @@ func (sm *StateManager) CallRaw(ctx context.Context, msg *types.Message, bstate
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: types.NewInt(0),
}
@ -130,6 +131,7 @@ func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, pri
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := vm.NewVM(vmopt)

308
chain/stmgr/forks.go
View File

@ -3,16 +3,28 @@ package stmgr
import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/util/adt"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
)
var ForksAtHeight = map[abi.ChainEpoch]func(context.Context, *StateManager, types.StateTree) error{}
var ForksAtHeight = map[abi.ChainEpoch]func(context.Context, *StateManager, types.StateTree, *types.TipSet) error{
build.UpgradeBreezeHeight: UpgradeFaucetBurnRecovery,
}
func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree, height abi.ChainEpoch) (err error) {
func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree, height abi.ChainEpoch, ts *types.TipSet) (err error) {
f, ok := ForksAtHeight[height]
if ok {
err := f(ctx, sm, st)
err := f(ctx, sm, st, ts)
if err != nil {
return err
}
@ -20,3 +32,291 @@ func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree
return nil
}
type forEachTree interface {
ForEach(func(address.Address, *types.Actor) error) error
}
func doTransfer(tree types.StateTree, from, to address.Address, amt abi.TokenAmount) error {
fromAct, err := tree.GetActor(from)
if err != nil {
return xerrors.Errorf("failed to get 'from' actor for transfer: %w", err)
}
fromAct.Balance = types.BigSub(fromAct.Balance, amt)
if fromAct.Balance.Sign() < 0 {
return xerrors.Errorf("(sanity) deducted more funds from target account than it had (%s, %s)", from, types.FIL(amt))
}
if err := tree.SetActor(from, fromAct); err != nil {
return xerrors.Errorf("failed to persist from actor: %w", err)
}
toAct, err := tree.GetActor(to)
if err != nil {
return xerrors.Errorf("failed to get 'to' actor for transfer: %w", err)
}
toAct.Balance = types.BigAdd(toAct.Balance, amt)
if err := tree.SetActor(to, toAct); err != nil {
return xerrors.Errorf("failed to persist to actor: %w", err)
}
return nil
}
func UpgradeFaucetBurnRecovery(ctx context.Context, sm *StateManager, tree types.StateTree, ts *types.TipSet) error {
// Some initial parameters
FundsForMiners := types.FromFil(1_000_000)
LookbackEpoch := abi.ChainEpoch(32000)
AccountCap := types.FromFil(0)
BaseMinerBalance := types.FromFil(20)
DesiredReimbursementBalance := types.FromFil(5_000_000)
isSystemAccount := func(addr address.Address) (bool, error) {
id, err := address.IDFromAddress(addr)
if err != nil {
return false, xerrors.Errorf("id address: %w", err)
}
if id < 1000 {
return true, nil
}
return false, nil
}
minerFundsAlloc := func(pow, tpow abi.StoragePower) abi.TokenAmount {
return types.BigDiv(types.BigMul(pow, FundsForMiners), tpow)
}
// Grab lookback state for account checks
lbts, err := sm.ChainStore().GetTipsetByHeight(ctx, LookbackEpoch, ts, false)
if err != nil {
return xerrors.Errorf("failed to get tipset at lookback height: %w", err)
}
var lbtree *state.StateTree
if err = sm.WithStateTree(lbts.ParentState(), func(state *state.StateTree) error {
lbtree = state
return nil
}); err != nil {
return xerrors.Errorf("loading state tree failed: %w", err)
}
ReserveAddress, err := address.NewFromString("t090")
if err != nil {
return xerrors.Errorf("failed to parse reserve address: %w", err)
}
fetree, ok := tree.(forEachTree)
if !ok {
return xerrors.Errorf("fork transition state tree doesnt support ForEach (%T)", tree)
}
type transfer struct {
From address.Address
To address.Address
Amt abi.TokenAmount
}
var transfers []transfer
// Take all excess funds away, put them into the reserve account
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
switch act.Code {
case builtin.AccountActorCodeID, builtin.MultisigActorCodeID, builtin.PaymentChannelActorCodeID:
sysAcc, err := isSystemAccount(addr)
if err != nil {
return xerrors.Errorf("checking system account: %w", err)
}
if !sysAcc {
transfers = append(transfers, transfer{
From: addr,
To: ReserveAddress,
Amt: act.Balance,
})
}
case builtin.StorageMinerActorCodeID:
var st miner.State
if err := sm.WithActorState(ctx, &st)(act); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
var available abi.TokenAmount
{
defer func() {
if err := recover(); err != nil {
log.Warnf("Get available balance failed (%s, %s, %s): %s", addr, act.Head, act.Balance, err)
}
available = abi.NewTokenAmount(0)
}()
// this panics if the miner doesnt have enough funds to cover their locked pledge
available = st.GetAvailableBalance(act.Balance)
}
transfers = append(transfers, transfer{
From: addr,
To: ReserveAddress,
Amt: available,
})
}
return nil
})
if err != nil {
return xerrors.Errorf("foreach over state tree failed: %w", err)
}
// Execute transfers from previous step
for _, t := range transfers {
if err := doTransfer(tree, t.From, t.To, t.Amt); err != nil {
return xerrors.Errorf("transfer %s %s->%s failed: %w", t.Amt, t.From, t.To, err)
}
}
// pull up power table to give miners back some funds proportional to their power
var ps power.State
powAct, err := tree.GetActor(builtin.StoragePowerActorAddr)
if err != nil {
return xerrors.Errorf("failed to load power actor: %w", err)
}
cst := cbor.NewCborStore(sm.ChainStore().Blockstore())
if err := cst.Get(ctx, powAct.Head, &ps); err != nil {
return xerrors.Errorf("failed to get power actor state: %w", err)
}
totalPower := ps.TotalBytesCommitted
var transfersBack []transfer
// Now, we return some funds to places where they are needed
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
lbact, err := lbtree.GetActor(addr)
if err != nil {
if !xerrors.Is(err, types.ErrActorNotFound) {
return xerrors.Errorf("failed to get actor in lookback state")
}
}
prevBalance := abi.NewTokenAmount(0)
if lbact != nil {
prevBalance = lbact.Balance
}
switch act.Code {
case builtin.AccountActorCodeID, builtin.MultisigActorCodeID, builtin.PaymentChannelActorCodeID:
nbalance := big.Min(prevBalance, AccountCap)
if nbalance.Sign() != 0 {
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: addr,
Amt: nbalance,
})
}
case builtin.StorageMinerActorCodeID:
var st miner.State
if err := sm.WithActorState(ctx, &st)(act); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
var minfo miner.MinerInfo
if err := cst.Get(ctx, st.Info, &minfo); err != nil {
return xerrors.Errorf("failed to get miner info: %w", err)
}
sectorsArr, err := adt.AsArray(sm.ChainStore().Store(ctx), st.Sectors)
if err != nil {
return xerrors.Errorf("failed to load sectors array: %w", err)
}
slen := sectorsArr.Length()
power := types.BigMul(types.NewInt(slen), types.NewInt(uint64(minfo.SectorSize)))
mfunds := minerFundsAlloc(power, totalPower)
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: minfo.Worker,
Amt: mfunds,
})
// Now make sure to give each miner who had power at the lookback some FIL
lbact, err := lbtree.GetActor(addr)
if err == nil {
var lbst miner.State
if err := sm.WithActorState(ctx, &lbst)(lbact); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
lbsectors, err := adt.AsArray(sm.ChainStore().Store(ctx), lbst.Sectors)
if err != nil {
return xerrors.Errorf("failed to load lb sectors array: %w", err)
}
if lbsectors.Length() > 0 {
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: minfo.Worker,
Amt: BaseMinerBalance,
})
}
} else {
log.Warnf("failed to get miner in lookback state: %s", err)
}
}
return nil
})
if err != nil {
return xerrors.Errorf("foreach over state tree failed: %w", err)
}
for _, t := range transfersBack {
if err := doTransfer(tree, t.From, t.To, t.Amt); err != nil {
return xerrors.Errorf("transfer %s %s->%s failed: %w", t.Amt, t.From, t.To, err)
}
}
// transfer all burnt funds back to the reserve account
burntAct, err := tree.GetActor(builtin.BurntFundsActorAddr)
if err != nil {
return xerrors.Errorf("failed to load burnt funds actor: %w", err)
}
if err := doTransfer(tree, builtin.BurntFundsActorAddr, ReserveAddress, burntAct.Balance); err != nil {
return xerrors.Errorf("failed to unburn funds: %w", err)
}
// Top up the reimbursement service
reimbAddr, err := address.NewFromString("t0111")
if err != nil {
return xerrors.Errorf("failed to parse reimbursement service address")
}
reimb, err := tree.GetActor(reimbAddr)
if err != nil {
return xerrors.Errorf("failed to load reimbursement account actor: %w", err)
}
difference := types.BigSub(DesiredReimbursementBalance, reimb.Balance)
if err := doTransfer(tree, ReserveAddress, reimbAddr, difference); err != nil {
return xerrors.Errorf("failed to top up reimbursement account: %w", err)
}
// Now, a final sanity check to make sure the balances all check out
total := abi.NewTokenAmount(0)
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
total = types.BigAdd(total, act.Balance)
return nil
})
if err != nil {
return xerrors.Errorf("checking final state balance failed: %w", err)
}
exp := types.FromFil(build.FilBase)
if !exp.Equals(total) {
return xerrors.Errorf("resultant state tree account balance was not correct: %s", total)
}
return nil
}

6
chain/stmgr/forks_test.go
View File

@ -7,8 +7,8 @@ import (
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
init_ "github.com/filecoin-project/specs-actors/actors/builtin/init"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
@ -119,7 +119,7 @@ func TestForkHeightTriggers(t *testing.T) {
t.Fatal(err)
}
stmgr.ForksAtHeight[testForkHeight] = func(ctx context.Context, sm *StateManager, st types.StateTree) error {
stmgr.ForksAtHeight[testForkHeight] = func(ctx context.Context, sm *StateManager, st types.StateTree, ts *types.TipSet) error {
cst := cbor.NewCborStore(sm.ChainStore().Blockstore())
act, err := st.GetActor(taddr)

25
chain/stmgr/stmgr.go
View File

@ -5,6 +5,8 @@ import (
"fmt"
"sync"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/multisig"
@ -18,8 +20,8 @@ import (
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
@ -145,7 +147,7 @@ func (sm *StateManager) ExecutionTrace(ctx context.Context, ts *types.TipSet) (c
type ExecCallback func(cid.Cid, *types.Message, *vm.ApplyRet) error
func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEpoch, pstate cid.Cid, bms []store.BlockMessages, epoch abi.ChainEpoch, r vm.Rand, cb ExecCallback, baseFee abi.TokenAmount) (cid.Cid, cid.Cid, error) {
func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEpoch, pstate cid.Cid, bms []store.BlockMessages, epoch abi.ChainEpoch, r vm.Rand, cb ExecCallback, baseFee abi.TokenAmount, ts *types.TipSet) (cid.Cid, cid.Cid, error) {
vmopt := &vm.VMOpts{
StateBase: pstate,
@ -154,6 +156,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: baseFee,
}
@ -198,7 +201,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
for i := parentEpoch; i < epoch; i++ {
// handle state forks
err = sm.handleStateForks(ctx, vmi.StateTree(), i)
err = sm.handleStateForks(ctx, vmi.StateTree(), i, ts)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("error handling state forks: %w", err)
}
@ -347,7 +350,7 @@ func (sm *StateManager) computeTipSetState(ctx context.Context, ts *types.TipSet
baseFee := blks[0].ParentBaseFee
return sm.ApplyBlocks(ctx, parentEpoch, pstate, blkmsgs, blks[0].Height, r, cb, baseFee)
return sm.ApplyBlocks(ctx, parentEpoch, pstate, blkmsgs, blks[0].Height, r, cb, baseFee, ts)
}
func (sm *StateManager) parentState(ts *types.TipSet) cid.Cid {
@ -1120,3 +1123,15 @@ func (sm *StateManager) GetCirculatingSupply(ctx context.Context, height abi.Cha
return csi.FilCirculating, nil
}
func (sm *StateManager) GetNtwkVersion(ctx context.Context, height abi.ChainEpoch) runtime.NetworkVersion {
if build.UpgradeBreezeHeight == 0 {
return runtime.NetworkVersion1
}
if height <= build.UpgradeBreezeHeight {
return runtime.NetworkVersion0
}
return runtime.NetworkVersion1
}

24
chain/stmgr/utils.go
View File

@ -9,6 +9,9 @@ import (
"runtime"
"strings"
saruntime "github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
cbg "github.com/whyrusleeping/cbor-gen"
@ -16,8 +19,9 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/account"
"github.com/filecoin-project/specs-actors/actors/builtin/cron"
@ -29,7 +33,6 @@ import (
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/api"
@ -166,7 +169,7 @@ func MinerSectorInfo(ctx context.Context, sm *StateManager, maddr address.Addres
return sectorInfo, nil
}
func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet, maddr address.Address, filter *abi.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet, maddr address.Address, filter *bitfield.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
var mas miner.State
_, err := sm.LoadActorState(ctx, maddr, &mas, ts)
if err != nil {
@ -176,8 +179,8 @@ func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet,
return LoadSectorsFromSet(ctx, sm.ChainStore().Blockstore(), mas.Sectors, filter, filterOut)
}
func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *StateManager, st cid.Cid, maddr address.Address, rand abi.PoStRandomness) ([]abi.SectorInfo, error) {
var partsProving []abi.BitField
func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *StateManager, st cid.Cid, maddr address.Address, rand abi.PoStRandomness) ([]proof.SectorInfo, error) {
var partsProving []bitfield.BitField
var mas *miner.State
var info *miner.MinerInfo
@ -264,7 +267,7 @@ func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *S
return nil, xerrors.Errorf("failed to load sectors amt: %w", err)
}
out := make([]abi.SectorInfo, len(ids))
out := make([]proof.SectorInfo, len(ids))
for i, n := range ids {
sid := sectors[n]
@ -275,7 +278,7 @@ func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *S
return nil, xerrors.Errorf("failed to find sector %d", sid)
}
out[i] = abi.SectorInfo{
out[i] = proof.SectorInfo{
SealProof: spt,
SectorNumber: sinfo.SectorNumber,
SealedCID: sinfo.SealedCID,
@ -390,7 +393,7 @@ func ListMinerActors(ctx context.Context, sm *StateManager, ts *types.TipSet) ([
return miners, nil
}
func LoadSectorsFromSet(ctx context.Context, bs blockstore.Blockstore, ssc cid.Cid, filter *abi.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
func LoadSectorsFromSet(ctx context.Context, bs blockstore.Blockstore, ssc cid.Cid, filter *bitfield.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
a, err := adt.AsArray(store.ActorStore(ctx, bs), ssc)
if err != nil {
return nil, err
@ -443,6 +446,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := vm.NewVM(vmopt)
@ -452,7 +456,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
for i := ts.Height(); i < height; i++ {
// handle state forks
err = sm.handleStateForks(ctx, vmi.StateTree(), i)
err = sm.handleStateForks(ctx, vmi.StateTree(), i, ts)
if err != nil {
return cid.Undef, nil, xerrors.Errorf("error handling state forks: %w", err)
}
@ -614,7 +618,7 @@ func init() {
}
for c, m := range cidToMethods {
exports := m[1].(abi.Invokee).Exports()
exports := m[1].(saruntime.Invokee).Exports()
methods := make(map[abi.MethodNum]MethodMeta, len(exports))
// Explicitly add send, it's special.

8
chain/store/basefee.go
View File

@ -3,10 +3,10 @@ package store
import (
"context"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/ipfs/go-cid"
"golang.org/x/xerrors"
)
@ -40,6 +40,10 @@ func computeNextBaseFee(baseFee types.BigInt, gasLimitUsed int64, noOfBlocks int
}
func (cs *ChainStore) ComputeBaseFee(ctx context.Context, ts *types.TipSet) (abi.TokenAmount, error) {
if ts.Height() > build.UpgradeBreezeHeight && ts.Height() < build.UpgradeBreezeHeight+build.BreezeGasTampingDuration {
return abi.NewTokenAmount(100), nil
}
zero := abi.NewTokenAmount(0)
// totalLimit is sum of GasLimits of unique messages in a tipset

2
chain/store/index.go
View File

@ -5,8 +5,8 @@ import (
"os"
"strconv"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
lru "github.com/hashicorp/golang-lru"
"golang.org/x/xerrors"
)

2
chain/store/index_test.go
View File

@ -5,11 +5,11 @@ import (
"context"
"testing"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/specs-actors/actors/abi"
datastore "github.com/ipfs/go-datastore"
syncds "github.com/ipfs/go-datastore/sync"
"github.com/stretchr/testify/assert"

6
chain/store/store.go
View File

@ -10,11 +10,11 @@ import (
"strconv"
"sync"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/api"
@ -471,7 +471,7 @@ func (cs *ChainStore) IsAncestorOf(a, b *types.TipSet) (bool, error) {
cur := b
for !a.Equals(cur) && cur.Height() > a.Height() {
next, err := cs.LoadTipSet(b.Parents())
next, err := cs.LoadTipSet(cur.Parents())
if err != nil {
return false, err
}

6
chain/store/store_test.go
View File

@ -7,12 +7,12 @@ import (
datastore "github.com/ipfs/go-datastore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/store"

2
chain/store/weight.go
View File

@ -4,10 +4,10 @@ import (
"context"
"math/big"
big2 "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/types"
big2 "github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
cbor "github.com/ipfs/go-ipld-cbor"

5
chain/sub/incoming.go
View File

@ -369,9 +369,8 @@ func (bv *BlockValidator) decodeAndCheckBlock(msg *pubsub.Message) (*types.Block
func (bv *BlockValidator) isChainNearSynced() bool {
ts := bv.chain.GetHeaviestTipSet()
timestamp := ts.MinTimestamp()
now := build.Clock.Now().UnixNano()
cutoff := uint64(now) - uint64(6*time.Hour)
return timestamp > cutoff
timestampTime := time.Unix(int64(timestamp), 0)
return build.Clock.Since(timestampTime) < 6*time.Hour
}
func (bv *BlockValidator) validateMsgMeta(ctx context.Context, msg *types.BlockMsg) error {

115
chain/sync.go
View File

@ -9,8 +9,13 @@ import (
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/Gurpartap/async"
"github.com/hashicorp/go-multierror"
"github.com/ipfs/go-cid"
@ -25,18 +30,18 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
blst "github.com/supranational/blst/bindings/go"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/beacon"
"github.com/filecoin-project/lotus/chain/blocksync"
"github.com/filecoin-project/lotus/chain/exchange"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/stmgr"
@ -50,7 +55,7 @@ import (
)
// Blocks that are more than MaxHeightDrift epochs above
//the theoretical max height based on systime are quickly rejected
// the theoretical max height based on systime are quickly rejected
const MaxHeightDrift = 5
var defaultMessageFetchWindowSize = 200
@ -87,7 +92,7 @@ var LocalIncoming = "incoming"
// The Syncer does not run workers itself. It's mainly concerned with
// ensuring a consistent state of chain consensus. The reactive and network-
// interfacing processes are part of other components, such as the SyncManager
// (which owns the sync scheduler and sync workers), BlockSync, the HELLO
// (which owns the sync scheduler and sync workers), ChainExchange, the HELLO
// protocol, and the gossipsub block propagation layer.
//
// {hint/concept} The fork-choice rule as it currently stands is: "pick the
@ -110,7 +115,7 @@ type Syncer struct {
bad *BadBlockCache
// handle to the block sync service
Bsync *blocksync.BlockSync
Exchange exchange.Client
self peer.ID
@ -125,10 +130,18 @@ type Syncer struct {
verifier ffiwrapper.Verifier
windowSize int
tickerCtxCancel context.CancelFunc
checkptLk sync.Mutex
checkpt types.TipSetKey
ds dtypes.MetadataDS
}
// NewSyncer creates a new Syncer object.
func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connmgr.ConnManager, self peer.ID, beacon beacon.RandomBeacon, verifier ffiwrapper.Verifier) (*Syncer, error) {
func NewSyncer(ds dtypes.MetadataDS, sm *stmgr.StateManager, exchange exchange.Client, connmgr connmgr.ConnManager, self peer.ID, beacon beacon.RandomBeacon, verifier ffiwrapper.Verifier) (*Syncer, error) {
gen, err := sm.ChainStore().GetGenesis()
if err != nil {
return nil, xerrors.Errorf("getting genesis block: %w", err)
@ -139,11 +152,18 @@ func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connm
return nil, err
}
cp, err := loadCheckpoint(ds)
if err != nil {
return nil, xerrors.Errorf("error loading mpool config: %w", err)
}
s := &Syncer{
ds: ds,
checkpt: cp,
beacon: beacon,
bad: NewBadBlockCache(),
Genesis: gent,
Bsync: bsync,
Exchange: exchange,
store: sm.ChainStore(),
sm: sm,
self: self,
@ -166,11 +186,35 @@ func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connm
}
func (syncer *Syncer) Start() {
tickerCtx, tickerCtxCancel := context.WithCancel(context.Background())
syncer.syncmgr.Start()
syncer.tickerCtxCancel = tickerCtxCancel
go syncer.runMetricsTricker(tickerCtx)
}
func (syncer *Syncer) runMetricsTricker(tickerCtx context.Context) {
genesisTime := time.Unix(int64(syncer.Genesis.MinTimestamp()), 0)
ticker := build.Clock.Ticker(time.Duration(build.BlockDelaySecs) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
sinceGenesis := build.Clock.Now().Sub(genesisTime)
expectedHeight := int64(sinceGenesis.Seconds()) / int64(build.BlockDelaySecs)
stats.Record(tickerCtx, metrics.ChainNodeHeightExpected.M(expectedHeight))
case <-tickerCtx.Done():
return
}
}
}
func (syncer *Syncer) Stop() {
syncer.syncmgr.Stop()
syncer.tickerCtxCancel()
}
// InformNewHead informs the syncer about a new potential tipset
@ -220,7 +264,7 @@ func (syncer *Syncer) InformNewHead(from peer.ID, fts *store.FullTipSet) bool {
return false
}
syncer.Bsync.AddPeer(from)
syncer.Exchange.AddPeer(from)
bestPweight := syncer.store.GetHeaviestTipSet().ParentWeight()
targetWeight := fts.TipSet().ParentWeight()
@ -451,7 +495,7 @@ func computeMsgMeta(bs cbor.IpldStore, bmsgCids, smsgCids []cid.Cid) (cid.Cid, e
}
// FetchTipSet tries to load the provided tipset from the store, and falls back
// to the network (BlockSync) by querying the supplied peer if not found
// to the network (client) by querying the supplied peer if not found
// locally.
//
// {hint/usage} This is used from the HELLO protocol, to fetch the greeting
@ -462,7 +506,7 @@ func (syncer *Syncer) FetchTipSet(ctx context.Context, p peer.ID, tsk types.TipS
}
// fall back to the network.
return syncer.Bsync.GetFullTipSet(ctx, p, tsk)
return syncer.Exchange.GetFullTipSet(ctx, p, tsk)
}
// tryLoadFullTipSet queries the tipset in the ChainStore, and returns a full
@ -953,7 +997,7 @@ func (syncer *Syncer) VerifyWinningPoStProof(ctx context.Context, h *types.Block
return xerrors.Errorf("getting winning post sector set: %w", err)
}
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, abi.WinningPoStVerifyInfo{
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, proof.WinningPoStVerifyInfo{
Randomness: rand,
Proofs: h.WinPoStProof,
ChallengedSectors: sectors,
@ -1164,7 +1208,7 @@ func extractSyncState(ctx context.Context) *SyncerState {
// total equality of the BeaconEntries in each block.
// 3. Traverse the chain backwards, for each tipset:
// 3a. Load it from the chainstore; if found, it move on to its parent.
// 3b. Query our peers via BlockSync in batches, requesting up to a
// 3b. Query our peers via client in batches, requesting up to a
// maximum of 500 tipsets every time.
//
// Once we've concluded, if we find a mismatching tipset at the height where the
@ -1265,7 +1309,7 @@ loop:
if gap := int(blockSet[len(blockSet)-1].Height() - untilHeight); gap < window {
window = gap
}
blks, err := syncer.Bsync.GetBlocks(ctx, at, window)
blks, err := syncer.Exchange.GetBlocks(ctx, at, window)
if err != nil {
// Most likely our peers aren't fully synced yet, but forwarded
// new block message (ideally we'd find better peers)
@ -1283,7 +1327,7 @@ loop:
// have. Since we fetch from the head backwards our reassembled chain
// is sorted in reverse here: we have a child -> parent order, our last
// tipset then should be child of the first tipset retrieved.
// FIXME: The reassembly logic should be part of the `BlockSync`
// FIXME: The reassembly logic should be part of the `client`
// service, the consumer should not be concerned with the
// `MaxRequestLength` limitation, it should just be able to request
// an segment of arbitrary length. The same burden is put on
@ -1333,7 +1377,7 @@ loop:
log.Warnf("(fork detected) synced header chain (%s - %d) does not link to our best block (%s - %d)", incoming.Cids(), incoming.Height(), known.Cids(), known.Height())
fork, err := syncer.syncFork(ctx, base, known)
if err != nil {
if xerrors.Is(err, ErrForkTooLong) {
if xerrors.Is(err, ErrForkTooLong) || xerrors.Is(err, ErrForkCheckpoint) {
// TODO: we're marking this block bad in the same way that we mark invalid blocks bad. Maybe distinguish?
log.Warn("adding forked chain to our bad tipset cache")
for _, b := range incoming.Blocks() {
@ -1349,15 +1393,24 @@ loop:
}
var ErrForkTooLong = fmt.Errorf("fork longer than threshold")
var ErrForkCheckpoint = fmt.Errorf("fork would require us to diverge from checkpointed block")
// syncFork tries to obtain the chain fragment that links a fork into a common
// ancestor in our view of the chain.
//
// If the fork is too long (build.ForkLengthThreshold), we add the entire subchain to the
// denylist. Else, we find the common ancestor, and add the missing chain
// If the fork is too long (build.ForkLengthThreshold), or would cause us to diverge from the checkpoint (ErrForkCheckpoint),
// we add the entire subchain to the denylist. Else, we find the common ancestor, and add the missing chain
// fragment until the fork point to the returned []TipSet.
func (syncer *Syncer) syncFork(ctx context.Context, incoming *types.TipSet, known *types.TipSet) ([]*types.TipSet, error) {
tips, err := syncer.Bsync.GetBlocks(ctx, incoming.Parents(), int(build.ForkLengthThreshold))
chkpt := syncer.GetCheckpoint()
if known.Key() == chkpt {
return nil, ErrForkCheckpoint
}
// TODO: Does this mean we always ask for ForkLengthThreshold blocks from the network, even if we just need, like, 2?
// Would it not be better to ask in smaller chunks, given that an ~ForkLengthThreshold is very rare?
tips, err := syncer.Exchange.GetBlocks(ctx, incoming.Parents(), int(build.ForkLengthThreshold))
if err != nil {
return nil, err
}
@ -1382,12 +1435,18 @@ func (syncer *Syncer) syncFork(ctx context.Context, incoming *types.TipSet, know
if nts.Height() < tips[cur].Height() {
cur++
} else {
// We will be forking away from nts, check that it isn't checkpointed
if nts.Key() == chkpt {
return nil, ErrForkCheckpoint
}
nts, err = syncer.store.LoadTipSet(nts.Parents())
if err != nil {
return nil, xerrors.Errorf("loading next local tipset: %w", err)
}
}
}
return nil, ErrForkTooLong
}
@ -1411,6 +1470,7 @@ func (syncer *Syncer) syncMessagesAndCheckState(ctx context.Context, headers []*
// fills out each of the given tipsets with messages and calls the callback with it
func (syncer *Syncer) iterFullTipsets(ctx context.Context, headers []*types.TipSet, cb func(context.Context, *store.FullTipSet) error) error {
ss := extractSyncState(ctx)
ctx, span := trace.StartSpan(ctx, "iterFullTipsets")
defer span.End()
@ -1438,12 +1498,13 @@ mainLoop:
nextI := (i + 1) - batchSize // want to fetch batchSize values, 'i' points to last one we want to fetch, so its 'inclusive' of our request, thus we need to add one to our request start index
var bstout []*blocksync.CompactedMessages
ss.SetStage(api.StageFetchingMessages)
var bstout []*exchange.CompactedMessages
for len(bstout) < batchSize {
next := headers[nextI]
nreq := batchSize - len(bstout)
bstips, err := syncer.Bsync.GetChainMessages(ctx, next, uint64(nreq))
bstips, err := syncer.Exchange.GetChainMessages(ctx, next, uint64(nreq))
if err != nil {
// TODO check errors for temporary nature
if windowSize > 1 {
@ -1457,6 +1518,7 @@ mainLoop:
bstout = append(bstout, bstips...)
nextI += len(bstips)
}
ss.SetStage(api.StageMessages)
for bsi := 0; bsi < len(bstout); bsi++ {
// temp storage so we don't persist data we dont want to
@ -1488,8 +1550,8 @@ mainLoop:
if i >= windowSize {
newWindowSize := windowSize + 10
if newWindowSize > int(blocksync.MaxRequestLength) {
newWindowSize = int(blocksync.MaxRequestLength)
if newWindowSize > int(exchange.MaxRequestLength) {
newWindowSize = int(exchange.MaxRequestLength)
}
if newWindowSize > windowSize {
windowSize = newWindowSize
@ -1506,7 +1568,7 @@ mainLoop:
return nil
}
func persistMessages(bs bstore.Blockstore, bst *blocksync.CompactedMessages) error {
func persistMessages(bs bstore.Blockstore, bst *exchange.CompactedMessages) error {
for _, m := range bst.Bls {
//log.Infof("putting BLS message: %s", m.Cid())
if _, err := store.PutMessage(bs, m); err != nil {
@ -1613,6 +1675,11 @@ func (syncer *Syncer) MarkBad(blk cid.Cid) {
syncer.bad.Add(blk, NewBadBlockReason([]cid.Cid{blk}, "manually marked bad"))
}
// UnmarkBad manually adds a block to the "bad blocks" cache.
func (syncer *Syncer) UnmarkBad(blk cid.Cid) {
syncer.bad.Remove(blk)
}
func (syncer *Syncer) CheckBadBlockCache(blk cid.Cid) (string, bool) {
bbr, ok := syncer.bad.Has(blk)
return bbr.String(), ok

124
chain/sync_test.go
View File

@ -7,6 +7,8 @@ import (
"testing"
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
@ -16,8 +18,8 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
@ -331,6 +333,36 @@ func (tu *syncTestUtil) compareSourceState(with int) {
}
}
func (tu *syncTestUtil) assertBad(node int, ts *types.TipSet) {
for _, blk := range ts.Cids() {
rsn, err := tu.nds[node].SyncCheckBad(context.TODO(), blk)
require.NoError(tu.t, err)
require.True(tu.t, len(rsn) != 0)
}
}
func (tu *syncTestUtil) getHead(node int) *types.TipSet {
ts, err := tu.nds[node].ChainHead(context.TODO())
require.NoError(tu.t, err)
return ts
}
func (tu *syncTestUtil) checkpointTs(node int, tsk types.TipSetKey) {
require.NoError(tu.t, tu.nds[node].SyncCheckpoint(context.TODO(), tsk))
}
func (tu *syncTestUtil) waitUntilNodeHasTs(node int, tsk types.TipSetKey) {
for {
_, err := tu.nds[node].ChainGetTipSet(context.TODO(), tsk)
if err != nil {
break
}
}
// Time to allow for syncing and validation
time.Sleep(2 * time.Second)
}
func (tu *syncTestUtil) waitUntilSync(from, to int) {
target, err := tu.nds[from].ChainHead(tu.ctx)
if err != nil {
@ -442,8 +474,8 @@ func (wpp badWpp) GenerateCandidates(context.Context, abi.PoStRandomness, uint64
return []uint64{1}, nil
}
func (wpp badWpp) ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error) {
return []abi.PoStProof{
func (wpp badWpp) ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error) {
return []proof.PoStProof{
{
PoStProof: abi.RegisteredPoStProof_StackedDrgWinning2KiBV1,
ProofBytes: []byte("evil"),
@ -676,3 +708,87 @@ func TestSyncInputs(t *testing.T) {
t.Fatal("should error on block with nil election proof")
}
}
func TestSyncCheckpointHead(t *testing.T) {
H := 10
tu := prepSyncTest(t, H)
p1 := tu.addClientNode()
p2 := tu.addClientNode()
fmt.Println("GENESIS: ", tu.g.Genesis().Cid())
tu.loadChainToNode(p1)
tu.loadChainToNode(p2)
base := tu.g.CurTipset
fmt.Println("Mining base: ", base.TipSet().Cids(), base.TipSet().Height())
// The two nodes fork at this point into 'a' and 'b'
a1 := tu.mineOnBlock(base, p1, []int{0}, true, false, nil)
a := tu.mineOnBlock(a1, p1, []int{0}, true, false, nil)
a = tu.mineOnBlock(a, p1, []int{0}, true, false, nil)
tu.waitUntilSyncTarget(p1, a.TipSet())
tu.checkpointTs(p1, a.TipSet().Key())
require.NoError(t, tu.g.ResyncBankerNonce(a1.TipSet()))
// chain B will now be heaviest
b := tu.mineOnBlock(base, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
fmt.Println("A: ", a.Cids(), a.TipSet().Height())
fmt.Println("B: ", b.Cids(), b.TipSet().Height())
// Now for the fun part!! p1 should mark p2's head as BAD.
require.NoError(t, tu.mn.LinkAll())
tu.connect(p1, p2)
tu.waitUntilNodeHasTs(p1, b.TipSet().Key())
p1Head := tu.getHead(p1)
require.Equal(tu.t, p1Head, a.TipSet())
tu.assertBad(p1, b.TipSet())
}
func TestSyncCheckpointEarlierThanHead(t *testing.T) {
H := 10
tu := prepSyncTest(t, H)
p1 := tu.addClientNode()
p2 := tu.addClientNode()
fmt.Println("GENESIS: ", tu.g.Genesis().Cid())
tu.loadChainToNode(p1)
tu.loadChainToNode(p2)
base := tu.g.CurTipset
fmt.Println("Mining base: ", base.TipSet().Cids(), base.TipSet().Height())
// The two nodes fork at this point into 'a' and 'b'
a1 := tu.mineOnBlock(base, p1, []int{0}, true, false, nil)
a := tu.mineOnBlock(a1, p1, []int{0}, true, false, nil)
a = tu.mineOnBlock(a, p1, []int{0}, true, false, nil)
tu.waitUntilSyncTarget(p1, a.TipSet())
tu.checkpointTs(p1, a1.TipSet().Key())
require.NoError(t, tu.g.ResyncBankerNonce(a1.TipSet()))
// chain B will now be heaviest
b := tu.mineOnBlock(base, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
fmt.Println("A: ", a.Cids(), a.TipSet().Height())
fmt.Println("B: ", b.Cids(), b.TipSet().Height())
// Now for the fun part!! p1 should mark p2's head as BAD.
require.NoError(t, tu.mn.LinkAll())
tu.connect(p1, p2)
tu.waitUntilNodeHasTs(p1, b.TipSet().Key())
p1Head := tu.getHead(p1)
require.Equal(tu.t, p1Head, a.TipSet())
tu.assertBad(p1, b.TipSet())
}

20
chain/syncstate.go
View File

@ -1,34 +1,16 @@
package chain
import (
"fmt"
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
)
func SyncStageString(v api.SyncStateStage) string {
switch v {
case api.StageHeaders:
return "header sync"
case api.StagePersistHeaders:
return "persisting headers"
case api.StageMessages:
return "message sync"
case api.StageSyncComplete:
return "complete"
case api.StageSyncErrored:
return "error"
default:
return fmt.Sprintf("<unknown: %d>", v)
}
}
type SyncerState struct {
lk sync.Mutex
Target *types.TipSet

2
chain/types/bigint.go
View File

@ -4,7 +4,7 @@ import (
"fmt"
"math/big"
big2 "github.com/filecoin-project/specs-actors/actors/abi/big"
big2 "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
)

8
chain/types/blockheader.go
View File

@ -4,10 +4,12 @@ import (
"bytes"
"math/big"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
@ -53,7 +55,7 @@ type BlockHeader struct {
BeaconEntries []BeaconEntry // 3
WinPoStProof []abi.PoStProof // 4
WinPoStProof []proof.PoStProof // 4
Parents []cid.Cid // 5

8
chain/types/blockheader_test.go
View File

@ -7,12 +7,14 @@ import (
"reflect"
"testing"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
)
func testBlockHeader(t testing.TB) *BlockHeader {
@ -80,7 +82,7 @@ func TestInteropBH(t *testing.T) {
t.Fatal(err)
}
posts := []abi.PoStProof{
posts := []proof.PoStProof{
{PoStProof: abi.RegisteredPoStProof_StackedDrgWinning2KiBV1, ProofBytes: []byte{0x07}},
}

15
chain/types/cbor_gen.go
View File

@ -6,9 +6,10 @@ import (
"fmt"
"io"
abi "github.com/filecoin-project/specs-actors/actors/abi"
crypto "github.com/filecoin-project/specs-actors/actors/crypto"
exitcode "github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
abi "github.com/filecoin-project/go-state-types/abi"
crypto "github.com/filecoin-project/go-state-types/crypto"
exitcode "github.com/filecoin-project/go-state-types/exitcode"
proof "github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
@ -58,7 +59,7 @@ func (t *BlockHeader) MarshalCBOR(w io.Writer) error {
}
}
// t.WinPoStProof ([]abi.PoStProof) (slice)
// t.WinPoStProof ([]proof.PoStProof) (slice)
if len(t.WinPoStProof) > cbg.MaxLength {
return xerrors.Errorf("Slice value in field t.WinPoStProof was too long")
}
@ -243,7 +244,7 @@ func (t *BlockHeader) UnmarshalCBOR(r io.Reader) error {
t.BeaconEntries[i] = v
}
// t.WinPoStProof ([]abi.PoStProof) (slice)
// t.WinPoStProof ([]proof.PoStProof) (slice)
maj, extra, err = cbg.CborReadHeaderBuf(br, scratch)
if err != nil {
@ -259,12 +260,12 @@ func (t *BlockHeader) UnmarshalCBOR(r io.Reader) error {
}
if extra > 0 {
t.WinPoStProof = make([]abi.PoStProof, extra)
t.WinPoStProof = make([]proof.PoStProof, extra)
}
for i := 0; i < int(extra); i++ {
var v abi.PoStProof
var v proof.PoStProof
if err := v.UnmarshalCBOR(br); err != nil {
return err
}

4
chain/types/message.go
View File

@ -4,9 +4,9 @@ import (
"bytes"
"fmt"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
xerrors "golang.org/x/xerrors"

2
chain/types/message_receipt.go
View File

@ -3,7 +3,7 @@ package types
import (
"bytes"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
)
type MessageReceipt struct {

2
chain/types/message_test.go
View File

@ -5,7 +5,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
)

6
chain/types/mock/chain.go
View File

@ -5,10 +5,11 @@ import (
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
)
@ -80,6 +81,7 @@ func MkBlock(parents *types.TipSet, weightInc uint64, ticketNonce uint64) *types
Height: height,
ParentStateRoot: pstateRoot,
BlockSig: &crypto.Signature{Type: crypto.SigTypeBLS, Data: []byte("boo! im a signature")},
ParentBaseFee: types.NewInt(uint64(build.MinimumBaseFee)),
}
}

2
chain/types/signature_test.go
View File

@ -4,7 +4,7 @@ import (
"bytes"
"testing"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
)
func TestSignatureSerializeRoundTrip(t *testing.T) {

4
chain/types/signedmessage.go
View File

@ -3,8 +3,8 @@ package types
import (
"bytes"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
)

2
chain/types/tipset.go
View File

@ -7,7 +7,7 @@ import (
"io"
"sort"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
"github.com/minio/blake2b-simd"

2
chain/types/tipset_key.go
View File

@ -5,7 +5,7 @@ import (
"encoding/json"
"strings"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
)

215
chain/validation/applier.go
View File

@ -1,215 +0,0 @@
package validation
import (
"context"
"golang.org/x/xerrors"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/puppet"
"github.com/ipfs/go-cid"
vtypes "github.com/filecoin-project/chain-validation/chain/types"
vstate "github.com/filecoin-project/chain-validation/state"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
)
// Applier applies messages to state trees and storage.
type Applier struct {
stateWrapper *StateWrapper
syscalls vm.SyscallBuilder
}
var _ vstate.Applier = &Applier{}
func NewApplier(sw *StateWrapper, syscalls vm.SyscallBuilder) *Applier {
return &Applier{sw, syscalls}
}
func (a *Applier) ApplyMessage(epoch abi.ChainEpoch, message *vtypes.Message) (vtypes.ApplyMessageResult, error) {
lm := toLotusMsg(message)
receipt, penalty, reward, err := a.applyMessage(epoch, lm)
return vtypes.ApplyMessageResult{
Msg: *message,
Receipt: receipt,
Penalty: penalty,
Reward: reward,
Root: a.stateWrapper.Root().String(),
}, err
}
func (a *Applier) ApplySignedMessage(epoch abi.ChainEpoch, msg *vtypes.SignedMessage) (vtypes.ApplyMessageResult, error) {
var lm types.ChainMsg
switch msg.Signature.Type {
case crypto.SigTypeSecp256k1:
lm = toLotusSignedMsg(msg)
case crypto.SigTypeBLS:
lm = toLotusMsg(&msg.Message)
default:
return vtypes.ApplyMessageResult{}, xerrors.New("Unknown signature type")
}
// TODO: Validate the sig first
receipt, penalty, reward, err := a.applyMessage(epoch, lm)
return vtypes.ApplyMessageResult{
Msg: msg.Message,
Receipt: receipt,
Penalty: penalty,
Reward: reward,
Root: a.stateWrapper.Root().String(),
}, err
}
func (a *Applier) ApplyTipSetMessages(epoch abi.ChainEpoch, blocks []vtypes.BlockMessagesInfo, rnd vstate.RandomnessSource) (vtypes.ApplyTipSetResult, error) {
cs := store.NewChainStore(a.stateWrapper.bs, a.stateWrapper.ds, a.syscalls)
sm := stmgr.NewStateManager(cs)
var bms []store.BlockMessages
for _, b := range blocks {
bm := store.BlockMessages{
Miner: b.Miner,
WinCount: 1,
}
for _, m := range b.BLSMessages {
bm.BlsMessages = append(bm.BlsMessages, toLotusMsg(m))
}
for _, m := range b.SECPMessages {
bm.SecpkMessages = append(bm.SecpkMessages, toLotusSignedMsg(m))
}
bms = append(bms, bm)
}
var receipts []vtypes.MessageReceipt
// TODO: base fee
sroot, _, err := sm.ApplyBlocks(context.TODO(), epoch-1, a.stateWrapper.Root(), bms, epoch, &randWrapper{rnd}, func(c cid.Cid, msg *types.Message, ret *vm.ApplyRet) error {
if msg.From == builtin.SystemActorAddr {
return nil // ignore reward and cron calls
}
rval := ret.Return
if rval == nil {
rval = []byte{} // chain validation tests expect empty arrays to not be nil...
}
receipts = append(receipts, vtypes.MessageReceipt{
ExitCode: ret.ExitCode,
ReturnValue: rval,
GasUsed: vtypes.GasUnits(ret.GasUsed),
})
return nil
}, abi.NewTokenAmount(100))
if err != nil {
return vtypes.ApplyTipSetResult{}, err
}
a.stateWrapper.stateRoot = sroot
return vtypes.ApplyTipSetResult{
Receipts: receipts,
Root: a.stateWrapper.Root().String(),
}, nil
}
type randWrapper struct {
rand vstate.RandomnessSource
}
// TODO: these should really be two different randomness sources
func (w *randWrapper) GetChainRandomness(ctx context.Context, pers crypto.DomainSeparationTag, round abi.ChainEpoch, entropy []byte) ([]byte, error) {
return w.rand.Randomness(ctx, pers, round, entropy)
}
func (w *randWrapper) GetBeaconRandomness(ctx context.Context, pers crypto.DomainSeparationTag, round abi.ChainEpoch, entropy []byte) ([]byte, error) {
return w.rand.Randomness(ctx, pers, round, entropy)
}
type vmRand struct {
}
func (*vmRand) GetChainRandomness(ctx context.Context, dst crypto.DomainSeparationTag, h abi.ChainEpoch, input []byte) ([]byte, error) {
panic("implement me")
}
func (*vmRand) GetBeaconRandomness(ctx context.Context, dst crypto.DomainSeparationTag, h abi.ChainEpoch, input []byte) ([]byte, error) {
panic("implement me")
}
func (a *Applier) applyMessage(epoch abi.ChainEpoch, lm types.ChainMsg) (vtypes.MessageReceipt, abi.TokenAmount, abi.TokenAmount, error) {
ctx := context.TODO()
base := a.stateWrapper.Root()
vmopt := &vm.VMOpts{
StateBase: base,
Epoch: epoch,
Rand: &vmRand{},
Bstore: a.stateWrapper.bs,
Syscalls: a.syscalls,
CircSupplyCalc: nil,
BaseFee: abi.NewTokenAmount(100),
}
lotusVM, err := vm.NewVM(vmopt)
// need to modify the VM invoker to add the puppet actor
chainValInvoker := vm.NewInvoker()
chainValInvoker.Register(puppet.PuppetActorCodeID, puppet.Actor{}, puppet.State{})
lotusVM.SetInvoker(chainValInvoker)
if err != nil {
return vtypes.MessageReceipt{}, big.Zero(), big.Zero(), err
}
ret, err := lotusVM.ApplyMessage(ctx, lm)
if err != nil {
return vtypes.MessageReceipt{}, big.Zero(), big.Zero(), err
}
rval := ret.Return
if rval == nil {
rval = []byte{}
}
a.stateWrapper.stateRoot, err = lotusVM.Flush(ctx)
if err != nil {
return vtypes.MessageReceipt{}, big.Zero(), big.Zero(), err
}
mr := vtypes.MessageReceipt{
ExitCode: ret.ExitCode,
ReturnValue: rval,
GasUsed: vtypes.GasUnits(ret.GasUsed),
}
return mr, ret.Penalty, abi.NewTokenAmount(ret.GasUsed), nil
}
func toLotusMsg(msg *vtypes.Message) *types.Message {
return &types.Message{
To: msg.To,
From: msg.From,
Nonce: msg.CallSeqNum,
Method: msg.Method,
Value: msg.Value,
GasLimit: msg.GasLimit,
GasFeeCap: msg.GasFeeCap,
GasPremium: msg.GasPremium,
Params: msg.Params,
}
}
func toLotusSignedMsg(msg *vtypes.SignedMessage) *types.SignedMessage {
return &types.SignedMessage{
Message: *toLotusMsg(&msg.Message),
Signature: msg.Signature,
}
}

37
chain/validation/config.go
View File

@ -1,37 +0,0 @@
package validation
//
// Config
//
type Config struct {
trackGas bool
checkExitCode bool
checkReturnValue bool
checkState bool
}
func NewConfig(gas, exit, ret, state bool) *Config {
return &Config{
trackGas: gas,
checkExitCode: exit,
checkReturnValue: ret,
checkState: state,
}
}
func (v Config) ValidateGas() bool {
return v.trackGas
}
func (v Config) ValidateExitCode() bool {
return v.checkExitCode
}
func (v Config) ValidateReturnValue() bool {
return v.checkReturnValue
}
func (v Config) ValidateStateRoot() bool {
return v.checkState
}

40
chain/validation/factories.go
View File

@ -1,40 +0,0 @@
package validation
import (
"context"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/specs-actors/actors/runtime"
cbor "github.com/ipfs/go-ipld-cbor"
vstate "github.com/filecoin-project/chain-validation/state"
)
type Factories struct {
*Applier
}
var _ vstate.Factories = &Factories{}
func NewFactories() *Factories {
return &Factories{}
}
func (f *Factories) NewStateAndApplier(syscalls runtime.Syscalls) (vstate.VMWrapper, vstate.Applier) {
st := NewState()
return st, NewApplier(st, func(ctx context.Context, cstate *state.StateTree, cst cbor.IpldStore) runtime.Syscalls {
return syscalls
})
}
func (f *Factories) NewKeyManager() vstate.KeyManager {
return newKeyManager()
}
func (f *Factories) NewValidationConfig() vstate.ValidationConfig {
trackGas := true
checkExit := true
checkRet := true
checkState := true
return NewConfig(trackGas, checkExit, checkRet, checkState)
}

104
chain/validation/keymanager.go
View File

@ -1,104 +0,0 @@
package validation
import (
"fmt"
"math/rand"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-crypto"
acrypto "github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
)
type KeyManager struct {
// Private keys by address
keys map[address.Address]*wallet.Key
// Seed for deterministic secp key generation.
secpSeed int64
// Seed for deterministic bls key generation.
blsSeed int64 // nolint: structcheck
}
func newKeyManager() *KeyManager {
return &KeyManager{
keys: make(map[address.Address]*wallet.Key),
secpSeed: 0,
}
}
func (k *KeyManager) NewSECP256k1AccountAddress() address.Address {
secpKey := k.newSecp256k1Key()
k.keys[secpKey.Address] = secpKey
return secpKey.Address
}
func (k *KeyManager) NewBLSAccountAddress() address.Address {
blsKey := k.newBLSKey()
k.keys[blsKey.Address] = blsKey
return blsKey.Address
}
func (k *KeyManager) Sign(addr address.Address, data []byte) (acrypto.Signature, error) {
ki, ok := k.keys[addr]
if !ok {
return acrypto.Signature{}, fmt.Errorf("unknown address %v", addr)
}
var sigType acrypto.SigType
if ki.Type == wallet.KTSecp256k1 {
sigType = acrypto.SigTypeBLS
hashed := blake2b.Sum256(data)
sig, err := crypto.Sign(ki.PrivateKey, hashed[:])
if err != nil {
return acrypto.Signature{}, err
}
return acrypto.Signature{
Type: sigType,
Data: sig,
}, nil
} else if ki.Type == wallet.KTBLS {
panic("lotus validator cannot sign BLS messages")
} else {
panic("unknown signature type")
}
}
func (k *KeyManager) newSecp256k1Key() *wallet.Key {
randSrc := rand.New(rand.NewSource(k.secpSeed)) // nolint
prv, err := crypto.GenerateKeyFromSeed(randSrc)
if err != nil {
panic(err)
}
k.secpSeed++
key, err := wallet.NewKey(types.KeyInfo{
Type: wallet.KTSecp256k1,
PrivateKey: prv,
})
if err != nil {
panic(err)
}
return key
}
func (k *KeyManager) newBLSKey() *wallet.Key {
// FIXME: bls needs deterministic key generation
//sk := ffi.PrivateKeyGenerate(s.blsSeed)
// s.blsSeed++
sk := [32]byte{}
sk[0] = uint8(k.blsSeed) // hack to keep gas values determinist
k.blsSeed++
key, err := wallet.NewKey(types.KeyInfo{
Type: wallet.KTBLS,
PrivateKey: sk[:],
})
if err != nil {
panic(err)
}
return key
}

217
chain/validation/state.go
View File

@ -1,217 +0,0 @@
package validation
import (
"context"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
vstate "github.com/filecoin-project/chain-validation/state"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/lib/blockstore"
)
var _ vstate.VMWrapper = &StateWrapper{}
type StateWrapper struct {
// The blockstore underlying the state tree and storage.
bs blockstore.Blockstore
ds datastore.Batching
// HAMT-CBOR store on top of the blockstore.
cst cbor.IpldStore
// CID of the root of the state tree.
stateRoot cid.Cid
}
func NewState() *StateWrapper {
bs := blockstore.NewTemporary()
cst := cbor.NewCborStore(bs)
// Put EmptyObjectCid value in the store. When an actor is initially created its Head is set to this value.
_, err := cst.Put(context.TODO(), map[string]string{})
if err != nil {
panic(err)
}
treeImpl, err := state.NewStateTree(cst)
if err != nil {
panic(err) // Never returns error, the error return should be removed.
}
root, err := treeImpl.Flush(context.TODO())
if err != nil {
panic(err)
}
return &StateWrapper{
bs: bs,
ds: datastore.NewMapDatastore(),
cst: cst,
stateRoot: root,
}
}
func (s *StateWrapper) NewVM() {
return
}
func (s *StateWrapper) Root() cid.Cid {
return s.stateRoot
}
// StoreGet the value at key from vm store
func (s *StateWrapper) StoreGet(key cid.Cid, out runtime.CBORUnmarshaler) error {
tree, err := state.LoadStateTree(s.cst, s.stateRoot)
if err != nil {
return err
}
return tree.Store.Get(context.Background(), key, out)
}
// StorePut `value` into vm store
func (s *StateWrapper) StorePut(value runtime.CBORMarshaler) (cid.Cid, error) {
tree, err := state.LoadStateTree(s.cst, s.stateRoot)
if err != nil {
return cid.Undef, err
}
return tree.Store.Put(context.Background(), value)
}
func (s *StateWrapper) Actor(addr address.Address) (vstate.Actor, error) {
tree, err := state.LoadStateTree(s.cst, s.stateRoot)
if err != nil {
return nil, err
}
fcActor, err := tree.GetActor(addr)
if err != nil {
return nil, err
}
return &actorWrapper{*fcActor}, nil
}
func (s *StateWrapper) SetActorState(addr address.Address, balance abi.TokenAmount, actorState runtime.CBORMarshaler) (vstate.Actor, error) {
tree, err := state.LoadStateTree(s.cst, s.stateRoot)
if err != nil {
return nil, err
}
// actor should exist
act, err := tree.GetActor(addr)
if err != nil {
return nil, err
}
// add the state to the store and get a new head cid
actHead, err := tree.Store.Put(context.Background(), actorState)
if err != nil {
return nil, err
}
// update the actor object with new head and balance parameter
actr := &actorWrapper{types.Actor{
Code: act.Code,
Nonce: act.Nonce,
// updates
Head: actHead,
Balance: balance,
}}
if err := tree.SetActor(addr, &actr.Actor); err != nil {
return nil, err
}
return actr, s.flush(tree)
}
func (s *StateWrapper) CreateActor(code cid.Cid, addr address.Address, balance abi.TokenAmount, actorState runtime.CBORMarshaler) (vstate.Actor, address.Address, error) {
idAddr := addr
tree, err := state.LoadStateTree(s.cst, s.stateRoot)
if err != nil {
return nil, address.Undef, err
}
if addr.Protocol() != address.ID {
actHead, err := tree.Store.Put(context.Background(), actorState)
if err != nil {
return nil, address.Undef, err
}
actr := &actorWrapper{types.Actor{
Code: code,
Head: actHead,
Balance: balance,
}}
idAddr, err = tree.RegisterNewAddress(addr)
if err != nil {
return nil, address.Undef, xerrors.Errorf("register new address for actor: %w", err)
}
if err := tree.SetActor(addr, &actr.Actor); err != nil {
return nil, address.Undef, xerrors.Errorf("setting new actor for actor: %w", err)
}
}
// store newState
head, err := tree.Store.Put(context.Background(), actorState)
if err != nil {
return nil, address.Undef, err
}
// create and store actor object
a := types.Actor{
Code: code,
Head: head,
Balance: balance,
}
if err := tree.SetActor(idAddr, &a); err != nil {
return nil, address.Undef, err
}
return &actorWrapper{a}, idAddr, s.flush(tree)
}
// Flushes a state tree to storage and sets this state's root to that tree's root CID.
func (s *StateWrapper) flush(tree *state.StateTree) (err error) {
s.stateRoot, err = tree.Flush(context.TODO())
return
}
//
// Actor Wrapper
//
type actorWrapper struct {
types.Actor
}
func (a *actorWrapper) Code() cid.Cid {
return a.Actor.Code
}
func (a *actorWrapper) Head() cid.Cid {
return a.Actor.Head
}
func (a *actorWrapper) CallSeqNum() uint64 {
return a.Actor.Nonce
}
func (a *actorWrapper) Balance() big.Int {
return a.Actor.Balance
}
//
// Storage
//
type contextStore struct {
cbor.IpldStore
ctx context.Context
}
func (s *contextStore) Context() context.Context {
return s.ctx
}

9
chain/vectors/gen/main.go
View File

@ -6,19 +6,20 @@ import (
"math/rand"
"os"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/go-address"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/chain/vectors"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
_ "github.com/filecoin-project/lotus/lib/sigs/bls"
_ "github.com/filecoin-project/lotus/lib/sigs/secp"

Some files were not shown because too many files have changed in this diff Show More