Compare commits

..
Author SHA1 Message Date
Travis Person be4006378d Genesis 2020-09-30 01:55:00 +00:00
Travis Person 1a2026eea8 New butterfly network info 2020-09-29 23:25:29 +00:00
230 changed files with 4829 additions and 12038 deletions
-1
View File
@@ -13,4 +13,3 @@
### Conformance testing.
conformance/ @raulk
extern/test-vectors @raulk
cmd/tvx @raulk
+1 -160
View File
@@ -1,164 +1,5 @@
# Lotus changelog
# 0.9.0 / 2020-10-07
This consensus-breaking release of Lotus upgrades the actors version to v2.0.0. This requires migrating actor state from v0 to v2. The changes that break consensus are:
- Introducing v2 actors and its migration (https://github.com/filecoin-project/lotus/pull/3936)
- Runtime's Receiver() should only return ID addresses (https://github.com/filecoin-project/lotus/pull/3589)
- Update miner eligibility checks for v2 actors (https://github.com/filecoin-project/lotus/pull/4188)
- Add funds that have left FilReserve to circ supply (https://github.com/filecoin-project/lotus/pull/4160)
- Set WinningPoStSectorSetLookback to finality post-v2 actors (https://github.com/filecoin-project/lotus/pull/4190)
- fix: error when actor panics directly (https://github.com/filecoin-project/lotus/pull/3697)
## Changes
#### Dependencies
- Update go-bitfield (https://github.com/filecoin-project/lotus/pull/4171)
- update the AMT implementation (https://github.com/filecoin-project/lotus/pull/4194)
- Update to actors v0.2.1 (https://github.com/filecoin-project/lotus/pull/4199)
#### Core Lotus
- Paych: fix voucher amount verification (https://github.com/filecoin-project/lotus/pull/3821)
- Cap market provider messages (https://github.com/filecoin-project/lotus/pull/4141)
- Run fork function after cron for null block safety (https://github.com/filecoin-project/lotus/pull/4114)
- use bitswap sessions when fetching messages, and cancel them (https://github.com/filecoin-project/lotus/pull/4142)
- relax pubsub IPColocationFactorThreshold to 5 (https://github.com/filecoin-project/lotus/pull/4183)
- Support addresses with mainnet prefixes (https://github.com/filecoin-project/lotus/pull/4186)
- fix: make message signer nonce generation transactional (https://github.com/filecoin-project/lotus/pull/4165)
- build: Env var to keep test address output (https://github.com/filecoin-project/lotus/pull/4213)
- make vm.EnableGasTracing public (https://github.com/filecoin-project/lotus/pull/4214)
- introduce separate state-tree versions (https://github.com/filecoin-project/lotus/pull/4197)
- reject explicit "calls" at the upgrade height (https://github.com/filecoin-project/lotus/pull/4231)
- return an illegal actor error when we see an unsupported actor version (https://github.com/filecoin-project/lotus/pull/4232)
- Set head should unmark blocks as valid (https://gist.github.com/travisperson/3c7cddd77a33979a519ccef4e6515f20)
#### Mining
- Increased ExpectedSealDuration and and WaitDealsDelay (https://github.com/filecoin-project/lotus/pull/3743)
- Miner backup/restore commands (https://github.com/filecoin-project/lotus/pull/4133)
- lotus-miner: add more help text to storage / attach (https://github.com/filecoin-project/lotus/pull/3961)
- Reject deals that are > 7 days in the future in the BasicDealFilter (https://github.com/filecoin-project/lotus/pull/4173)
- feat(miner): add miner deadline diffing logic (https://github.com/filecoin-project/lotus/pull/4178)
#### UX
- Improve the UX for replacing messages (https://github.com/filecoin-project/lotus/pull/4134)
- Add verified flag to interactive deal creation (https://github.com/filecoin-project/lotus/pull/4145)
- Add command to (slowly) prune lotus chain datastore (https://github.com/filecoin-project/lotus/pull/3876)
- Some helpers for verifreg work (https://github.com/filecoin-project/lotus/pull/4124)
- Always use default 720h for setask duration and hide the duration param option (https://github.com/filecoin-project/lotus/pull/4077)
- Convert ID addresses to key addresses before checking wallet (https://github.com/filecoin-project/lotus/pull/4122)
- add a command to view block space utilization (https://github.com/filecoin-project/lotus/pull/4176)
- allow usage inspection on a chain segment (https://github.com/filecoin-project/lotus/pull/4177)
- Add mpool stats for base fee (https://github.com/filecoin-project/lotus/pull/4170)
- Add verified status to api.DealInfo (https://github.com/filecoin-project/lotus/pull/4153)
- Add a CLI command to set a miner's owner address (https://github.com/filecoin-project/lotus/pull/4189)
#### Tooling and validation
- Lotus-pcr: add recover-miners command (https://github.com/filecoin-project/lotus/pull/3714)
- MpoolPushUntrusted API for gateway (https://github.com/filecoin-project/lotus/pull/3915)
- Test lotus-miner info all (https://github.com/filecoin-project/lotus/pull/4166)
- chain export: Error with unfinished exports (https://github.com/filecoin-project/lotus/pull/4179)
- add printf in TestWindowPost (https://github.com/filecoin-project/lotus/pull/4043)
- add trace wdpost (https://github.com/filecoin-project/lotus/pull/4020)
- Fix noncefix (https://github.com/filecoin-project/lotus/pull/4202)
- Lotus-pcr: Limit the fee cap of messages we will process, refund gas fees for windowed post and storage deals (https://github.com/filecoin-project/lotus/pull/4198)
- Fix pond (https://github.com/filecoin-project/lotus/pull/4203)
- allow manual setting of noncefix fee cap (https://github.com/filecoin-project/lotus/pull/4205)
- implement command to get execution traces of any message (https://github.com/filecoin-project/lotus/pull/4200)
- conformance: minor driver refactors (https://github.com/filecoin-project/lotus/pull/4211)
- lotus-pcr: ignore all other messages (https://github.com/filecoin-project/lotus/pull/4218)
- lotus-pcr: zero refund (https://github.com/filecoin-project/lotus/pull/4229)
## Contributors
The following contributors had 5 or more commits go into this release.
We are grateful for every contribution!
| Contributor | Commits | Lines ± |
|--------------------|---------|---------------|
| Stebalien | 84 | +3425/-2287 |
| magik6k | 41 | +2121/-506 |
| arajasek | 39 | +2467/-424 |
| Kubuxu | 25 | +2344/-775 |
| raulk | 21 | +287/-196 |
| whyrusleeping | 13 | +727/-71 |
| hsanjuan | 13 | +5886/-7956 |
| dirkmc | 11 | +2634/-576 |
| travisperson | 8 | +923/-202 |
| ribasushi | 6 | +188/-128 |
| zgfzgf | 5 | +21/-17 |
# 0.8.1 / 2020-09-30
This optional release of Lotus introduces a new version of markets which switches to CBOR-map encodings, and allows datastore migrations. The release also introduces several improvements to the mining process, a few performance optimizations, and a battery of UX additions and enhancements.
## Changes
#### Dependencies
- Markets 0.7.0 with updated data stores (https://github.com/filecoin-project/lotus/pull/4089)
- Update ffi to code with blst fixes (https://github.com/filecoin-project/lotus/pull/3998)
#### Core Lotus
- Fix GetPower with no miner address (https://github.com/filecoin-project/lotus/pull/4049)
- Refactor: Move nonce generation out of mpool (https://github.com/filecoin-project/lotus/pull/3970)
#### Performance
- Implement caching syscalls for import-bench (https://github.com/filecoin-project/lotus/pull/3888)
- Fetch tipset blocks in parallel (https://github.com/filecoin-project/lotus/pull/4074)
- Optimize Tipset equals() (https://github.com/filecoin-project/lotus/pull/4056)
- Make state transition in validation async (https://github.com/filecoin-project/lotus/pull/3868)
#### Mining
- Add trace window post (https://github.com/filecoin-project/lotus/pull/4020)
- Use abstract types for Dont recompute post on revert (https://github.com/filecoin-project/lotus/pull/4022)
- Fix injectNulls logic in test miner (https://github.com/filecoin-project/lotus/pull/4058)
- Fix potential panic in FinalizeSector (https://github.com/filecoin-project/lotus/pull/4092)
- Don't recompute post on revert (https://github.com/filecoin-project/lotus/pull/3924)
- Fix some failed precommit handling (https://github.com/filecoin-project/lotus/pull/3445)
- Add --no-swap flag for worker (https://github.com/filecoin-project/lotus/pull/4107)
- Allow some single-thread tasks to run in parallel with PC2/C2 (https://github.com/filecoin-project/lotus/pull/4116)
#### UX
- Add an envvar to set address network version (https://github.com/filecoin-project/lotus/pull/4028)
- Add logging to chain export (https://github.com/filecoin-project/lotus/pull/4030)
- Add JSON output to state compute (https://github.com/filecoin-project/lotus/pull/4038)
- Wallet list CLI: Print balances/nonces (https://github.com/filecoin-project/lotus/pull/4088)
- Added an option to show or not show sector info for `lotus-miner info` (https://github.com/filecoin-project/lotus/pull/4003)
- Add a command to import an ipld object into the chainstore (https://github.com/filecoin-project/lotus/pull/3434)
- Improve the lotus-shed dealtracker (https://github.com/filecoin-project/lotus/pull/4051)
- Docs review and re-organization (https://github.com/filecoin-project/lotus/pull/3431)
- Fix wallet list (https://github.com/filecoin-project/lotus/pull/4104)
- Add an endpoint to validate whether a string is a well-formed address (https://github.com/filecoin-project/lotus/pull/4106)
- Add an option to set config path (https://github.com/filecoin-project/lotus/pull/4103)
- Add printf in TestWindowPost (https://github.com/filecoin-project/lotus/pull/4043)
- Improve miner sectors list UX (https://github.com/filecoin-project/lotus/pull/4108)
#### Tooling
- Move policy change to seal bench (https://github.com/filecoin-project/lotus/pull/4032)
- Add back network power to stats (https://github.com/filecoin-project/lotus/pull/4050)
- Conformance: Record and feed circulating supply (https://github.com/filecoin-project/lotus/pull/4078)
- Snapshot import progress bar, add HTTP support (https://github.com/filecoin-project/lotus/pull/4070)
- Add lotus shed util to validate a tipset (https://github.com/filecoin-project/lotus/pull/4065)
- tvx: a test vector extraction and execution tool (https://github.com/filecoin-project/lotus/pull/4064)
#### Bootstrap
- Add new bootstrappers (https://github.com/filecoin-project/lotus/pull/4007)
- Add Glif node to bootstrap peers (https://github.com/filecoin-project/lotus/pull/4004)
- Add one more node located in China (https://github.com/filecoin-project/lotus/pull/4041)
- Add ipfsmain bootstrapper (https://github.com/filecoin-project/lotus/pull/4067)
# 0.8.0 / 2020-09-26
This consensus-breaking release of Lotus introduces an upgrade to the network. The changes that break consensus are:
@@ -210,7 +51,7 @@ We are grateful for every contribution!
| vyzo | 22 | +287/-196 |
| alanshaw | 15 | +761/-146 |
| whyrusleeping | 15 | +736/-52 |
| hannahhoward | 14 | +1237/-837 |
| hannahhoward | 14 | +1237/837- |
| anton | 6 | +32/-8 |
| travisperson | 5 | +502/-6 |
| Frank | 5 | +78/-39 |
+2 -2
View File
@@ -1,5 +1,5 @@
<p align="center">
<a href="https://docs.filecoin.io/" title="Filecoin Docs">
<a href="https://lotu.sh/" title="Lotus Docs">
<img src="documentation/images/lotus_logo_h.png" alt="Project Lotus Logo" width="244" />
</a>
</p>
@@ -18,7 +18,7 @@ Lotus is an implementation of the Filecoin Distributed Storage Network. For more
## Building & Documentation
For instructions on how to build, install and setup lotus, please visit [https://docs.filecoin.io/get-started/lotus](https://docs.filecoin.io/get-started/lotus/).
For instructions on how to build lotus from source, please visit [Lotus build and setup instruction](https://docs.filecoin.io/get-started/lotus/installation/#minimal-requirements) or read the source [here](https://github.com/filecoin-project/lotus/tree/master/documentation).
## Reporting a Vulnerability
+9 -45
View File
@@ -176,9 +176,6 @@ type FullNode interface {
// the reason.
SyncCheckBad(ctx context.Context, bcid cid.Cid) (string, error)
// SyncValidateTipset indicates whether the provided tipset is valid or not
SyncValidateTipset(ctx context.Context, tsk types.TipSetKey) (bool, error)
// MethodGroup: Mpool
// The Mpool methods are for interacting with the message pool. The message pool
// manages all incoming and outgoing 'messages' going over the network.
@@ -192,9 +189,6 @@ type FullNode interface {
// MpoolPush pushes a signed message to mempool.
MpoolPush(context.Context, *types.SignedMessage) (cid.Cid, error)
// MpoolPushUntrusted pushes a signed message to mempool from untrusted sources.
MpoolPushUntrusted(context.Context, *types.SignedMessage) (cid.Cid, error)
// MpoolPushMessage atomically assigns a nonce, signs, and pushes a message
// to mempool.
// maxFee is only used when GasFeeCap/GasPremium fields aren't specified
@@ -250,8 +244,6 @@ type FullNode interface {
WalletImport(context.Context, *types.KeyInfo) (address.Address, error)
// WalletDelete deletes an address from the wallet.
WalletDelete(context.Context, address.Address) error
// WalletValidateAddress validates whether a given string can be decoded as a well-formed address
WalletValidateAddress(context.Context, string) (address.Address, error)
// Other
@@ -283,7 +275,7 @@ type FullNode interface {
// of status updates.
ClientRetrieveWithEvents(ctx context.Context, order RetrievalOrder, ref *FileRef) (<-chan marketevents.RetrievalEvent, error)
// ClientQueryAsk returns a signed StorageAsk from the specified miner.
ClientQueryAsk(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.StorageAsk, error)
ClientQueryAsk(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error)
// ClientCalcCommP calculates the CommP for a specified file
ClientCalcCommP(ctx context.Context, inpath string) (*CommPRet, error)
// ClientGenCar generates a CAR file for the specified file.
@@ -393,16 +385,10 @@ type FullNode interface {
// StateCompute is a flexible command that applies the given messages on the given tipset.
// The messages are run as though the VM were at the provided height.
StateCompute(context.Context, abi.ChainEpoch, []*types.Message, types.TipSetKey) (*ComputeStateOutput, error)
// StateVerifierStatus returns the data cap for the given address.
// Returns nil if there is no entry in the data cap table for the
// address.
StateVerifierStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)
// StateVerifiedClientStatus returns the data cap for the given address.
// Returns nil if there is no entry in the data cap table for the
// address.
StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)
// StateVerifiedClientStatus returns the address of the Verified Registry's root key
StateVerifiedRegistryRootKey(ctx context.Context, tsk types.TipSetKey) (address.Address, error)
// StateDealProviderCollateralBounds returns the min and max collateral a storage provider
// can issue. It takes the deal size and verified status as parameters.
StateDealProviderCollateralBounds(context.Context, abi.PaddedPieceSize, bool, types.TipSetKey) (DealCollateralBounds, error)
@@ -418,9 +404,6 @@ type FullNode interface {
// MsigGetAvailableBalance returns the portion of a multisig's balance that can be withdrawn or spent
MsigGetAvailableBalance(context.Context, address.Address, types.TipSetKey) (types.BigInt, error)
// MsigGetLockedBalance returns the locked balance of an msig at a vien epoch.
// The return may be greater than the multisig actor's actual balance.
MsigGetVestingSchedule(context.Context, address.Address, types.TipSetKey) (MsigVesting, error)
// MsigGetVested returns the amount of FIL that vested in a multisig in a certain period.
// It takes the following params: <multisig address>, <start epoch>, <end epoch>
MsigGetVested(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error)
@@ -487,12 +470,6 @@ type FullNode interface {
PaychVoucherAdd(context.Context, address.Address, *paych.SignedVoucher, []byte, types.BigInt) (types.BigInt, error)
PaychVoucherList(context.Context, address.Address) ([]*paych.SignedVoucher, error)
PaychVoucherSubmit(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (cid.Cid, error)
// CreateBackup creates node backup onder the specified file name. The
// method requires that the lotus daemon is running with the
// LOTUS_BACKUP_BASE_PATH environment variable set to some path, and that
// the path specified when calling CreateBackup is within the base path
CreateBackup(ctx context.Context, fpath string) error
}
type FileRef struct {
@@ -539,7 +516,6 @@ type DealInfo struct {
DealID abi.DealID
CreationTime time.Time
Verified bool
}
type MsgLookup struct {
@@ -815,14 +791,14 @@ type CirculatingSupply struct {
}
type MiningBaseInfo struct {
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []builtin.SectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry
BeaconEntries []types.BeaconEntry
EligibleForMining bool
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []builtin.SectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry
BeaconEntries []types.BeaconEntry
HasMinPower bool
}
type BlockTemplate struct {
@@ -874,15 +850,3 @@ type Fault struct {
Miner address.Address
Epoch abi.ChainEpoch
}
var EmptyVesting = MsigVesting{
InitialBalance: types.EmptyInt,
StartEpoch: -1,
UnlockDuration: -1,
}
type MsigVesting struct {
InitialBalance abi.TokenAmount
StartEpoch abi.ChainEpoch
UnlockDuration abi.ChainEpoch
}
-6
View File
@@ -101,12 +101,6 @@ type StorageMiner interface {
PiecesListCidInfos(ctx context.Context) ([]cid.Cid, error)
PiecesGetPieceInfo(ctx context.Context, pieceCid cid.Cid) (*piecestore.PieceInfo, error)
PiecesGetCIDInfo(ctx context.Context, payloadCid cid.Cid) (*piecestore.CIDInfo, error)
// CreateBackup creates node backup onder the specified file name. The
// method requires that the lotus-miner is running with the
// LOTUS_BACKUP_BASE_PATH environment variable set to some path, and that
// the path specified when calling CreateBackup is within the base path
CreateBackup(ctx context.Context, fpath string) error
}
type SealRes struct {
+15 -58
View File
@@ -112,7 +112,6 @@ type FullNodeStruct struct {
SyncMarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncUnmarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncCheckBad func(ctx context.Context, bcid cid.Cid) (string, error) `perm:"read"`
SyncValidateTipset func(ctx context.Context, tsk types.TipSetKey) (bool, error) `perm:"read"`
MpoolGetConfig func(context.Context) (*types.MpoolConfig, error) `perm:"read"`
MpoolSetConfig func(context.Context, *types.MpoolConfig) error `perm:"write"`
@@ -122,9 +121,7 @@ type FullNodeStruct struct {
MpoolPending func(context.Context, types.TipSetKey) ([]*types.SignedMessage, error) `perm:"read"`
MpoolClear func(context.Context, bool) error `perm:"write"`
MpoolPush func(context.Context, *types.SignedMessage) (cid.Cid, error) `perm:"write"`
MpoolPushUntrusted func(context.Context, *types.SignedMessage) (cid.Cid, error) `perm:"write"`
MpoolPush func(context.Context, *types.SignedMessage) (cid.Cid, error) `perm:"write"`
MpoolPushMessage func(context.Context, *types.Message, *api.MessageSendSpec) (*types.SignedMessage, error) `perm:"sign"`
MpoolGetNonce func(context.Context, address.Address) (uint64, error) `perm:"read"`
MpoolSub func(context.Context) (<-chan api.MpoolUpdate, error) `perm:"read"`
@@ -132,19 +129,18 @@ type FullNodeStruct struct {
MinerGetBaseInfo func(context.Context, address.Address, abi.ChainEpoch, types.TipSetKey) (*api.MiningBaseInfo, error) `perm:"read"`
MinerCreateBlock func(context.Context, *api.BlockTemplate) (*types.BlockMsg, error) `perm:"write"`
WalletNew func(context.Context, crypto.SigType) (address.Address, error) `perm:"write"`
WalletHas func(context.Context, address.Address) (bool, error) `perm:"write"`
WalletList func(context.Context) ([]address.Address, error) `perm:"write"`
WalletBalance func(context.Context, address.Address) (types.BigInt, error) `perm:"read"`
WalletSign func(context.Context, address.Address, []byte) (*crypto.Signature, error) `perm:"sign"`
WalletSignMessage func(context.Context, address.Address, *types.Message) (*types.SignedMessage, error) `perm:"sign"`
WalletVerify func(context.Context, address.Address, []byte, *crypto.Signature) (bool, error) `perm:"read"`
WalletDefaultAddress func(context.Context) (address.Address, error) `perm:"write"`
WalletSetDefault func(context.Context, address.Address) error `perm:"admin"`
WalletExport func(context.Context, address.Address) (*types.KeyInfo, error) `perm:"admin"`
WalletImport func(context.Context, *types.KeyInfo) (address.Address, error) `perm:"admin"`
WalletDelete func(context.Context, address.Address) error `perm:"write"`
WalletValidateAddress func(context.Context, string) (address.Address, error) `perm:"read"`
WalletNew func(context.Context, crypto.SigType) (address.Address, error) `perm:"write"`
WalletHas func(context.Context, address.Address) (bool, error) `perm:"write"`
WalletList func(context.Context) ([]address.Address, error) `perm:"write"`
WalletBalance func(context.Context, address.Address) (types.BigInt, error) `perm:"read"`
WalletSign func(context.Context, address.Address, []byte) (*crypto.Signature, error) `perm:"sign"`
WalletSignMessage func(context.Context, address.Address, *types.Message) (*types.SignedMessage, error) `perm:"sign"`
WalletVerify func(context.Context, address.Address, []byte, *crypto.Signature) (bool, error) `perm:"read"`
WalletDefaultAddress func(context.Context) (address.Address, error) `perm:"write"`
WalletSetDefault func(context.Context, address.Address) error `perm:"admin"`
WalletExport func(context.Context, address.Address) (*types.KeyInfo, error) `perm:"admin"`
WalletImport func(context.Context, *types.KeyInfo) (address.Address, error) `perm:"admin"`
WalletDelete func(context.Context, address.Address) error `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
@@ -158,7 +154,7 @@ type FullNodeStruct struct {
ClientGetDealUpdates func(ctx context.Context) (<-chan api.DealInfo, error) `perm:"read"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.StorageAsk, error) `perm:"read"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
@@ -204,15 +200,12 @@ type FullNodeStruct struct {
StateMinerSectorCount func(context.Context, address.Address, types.TipSetKey) (api.MinerSectors, error) `perm:"read"`
StateListMessages func(ctx context.Context, match *types.Message, tsk types.TipSetKey, toht abi.ChainEpoch) ([]cid.Cid, error) `perm:"read"`
StateCompute func(context.Context, abi.ChainEpoch, []*types.Message, types.TipSetKey) (*api.ComputeStateOutput, error) `perm:"read"`
StateVerifierStatus func(context.Context, address.Address, types.TipSetKey) (*abi.StoragePower, error) `perm:"read"`
StateVerifiedClientStatus func(context.Context, address.Address, types.TipSetKey) (*abi.StoragePower, error) `perm:"read"`
StateVerifiedRegistryRootKey func(ctx context.Context, tsk types.TipSetKey) (address.Address, error) `perm:"read"`
StateDealProviderCollateralBounds func(context.Context, abi.PaddedPieceSize, bool, types.TipSetKey) (api.DealCollateralBounds, error) `perm:"read"`
StateCirculatingSupply func(context.Context, types.TipSetKey) (api.CirculatingSupply, error) `perm:"read"`
StateNetworkVersion func(context.Context, types.TipSetKey) (stnetwork.Version, error) `perm:"read"`
MsigGetAvailableBalance func(context.Context, address.Address, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigGetVestingSchedule func(context.Context, address.Address, types.TipSetKey) (api.MsigVesting, error) `perm:"read"`
MsigGetVested func(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigCreate func(context.Context, uint64, []address.Address, abi.ChainEpoch, types.BigInt, address.Address, types.BigInt) (cid.Cid, error) `perm:"sign"`
MsigPropose func(context.Context, address.Address, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
@@ -244,8 +237,6 @@ type FullNodeStruct struct {
PaychVoucherCreate func(context.Context, address.Address, big.Int, uint64) (*api.VoucherCreateResult, error) `perm:"sign"`
PaychVoucherList func(context.Context, address.Address) ([]*paych.SignedVoucher, error) `perm:"write"`
PaychVoucherSubmit func(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (cid.Cid, error) `perm:"sign"`
CreateBackup func(ctx context.Context, fpath string) error `perm:"admin"`
}
}
@@ -326,8 +317,6 @@ type StorageMinerStruct struct {
PiecesListCidInfos func(ctx context.Context) ([]cid.Cid, error) `perm:"read"`
PiecesGetPieceInfo func(ctx context.Context, pieceCid cid.Cid) (*piecestore.PieceInfo, error) `perm:"read"`
PiecesGetCIDInfo func(ctx context.Context, payloadCid cid.Cid) (*piecestore.CIDInfo, error) `perm:"read"`
CreateBackup func(ctx context.Context, fpath string) error `perm:"admin"`
}
}
@@ -495,7 +484,7 @@ func (c *FullNodeStruct) ClientRetrieveWithEvents(ctx context.Context, order api
return c.Internal.ClientRetrieveWithEvents(ctx, order, ref)
}
func (c *FullNodeStruct) ClientQueryAsk(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.StorageAsk, error) {
func (c *FullNodeStruct) ClientQueryAsk(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) {
return c.Internal.ClientQueryAsk(ctx, p, miner)
}
func (c *FullNodeStruct) ClientCalcCommP(ctx context.Context, inpath string) (*api.CommPRet, error) {
@@ -562,10 +551,6 @@ func (c *FullNodeStruct) MpoolPush(ctx context.Context, smsg *types.SignedMessag
return c.Internal.MpoolPush(ctx, smsg)
}
func (c *FullNodeStruct) MpoolPushUntrusted(ctx context.Context, smsg *types.SignedMessage) (cid.Cid, error) {
return c.Internal.MpoolPushUntrusted(ctx, smsg)
}
func (c *FullNodeStruct) MpoolPushMessage(ctx context.Context, msg *types.Message, spec *api.MessageSendSpec) (*types.SignedMessage, error) {
return c.Internal.MpoolPushMessage(ctx, msg, spec)
}
@@ -646,10 +631,6 @@ func (c *FullNodeStruct) WalletDelete(ctx context.Context, addr address.Address)
return c.Internal.WalletDelete(ctx, addr)
}
func (c *FullNodeStruct) WalletValidateAddress(ctx context.Context, str string) (address.Address, error) {
return c.Internal.WalletValidateAddress(ctx, str)
}
func (c *FullNodeStruct) MpoolGetNonce(ctx context.Context, addr address.Address) (uint64, error) {
return c.Internal.MpoolGetNonce(ctx, addr)
}
@@ -754,10 +735,6 @@ func (c *FullNodeStruct) SyncCheckBad(ctx context.Context, bcid cid.Cid) (string
return c.Internal.SyncCheckBad(ctx, bcid)
}
func (c *FullNodeStruct) SyncValidateTipset(ctx context.Context, tsk types.TipSetKey) (bool, error) {
return c.Internal.SyncValidateTipset(ctx, tsk)
}
func (c *FullNodeStruct) StateNetworkName(ctx context.Context) (dtypes.NetworkName, error) {
return c.Internal.StateNetworkName(ctx)
}
@@ -906,18 +883,10 @@ func (c *FullNodeStruct) StateCompute(ctx context.Context, height abi.ChainEpoch
return c.Internal.StateCompute(ctx, height, msgs, tsk)
}
func (c *FullNodeStruct) StateVerifierStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error) {
return c.Internal.StateVerifierStatus(ctx, addr, tsk)
}
func (c *FullNodeStruct) StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error) {
return c.Internal.StateVerifiedClientStatus(ctx, addr, tsk)
}
func (c *FullNodeStruct) StateVerifiedRegistryRootKey(ctx context.Context, tsk types.TipSetKey) (address.Address, error) {
return c.Internal.StateVerifiedRegistryRootKey(ctx, tsk)
}
func (c *FullNodeStruct) StateDealProviderCollateralBounds(ctx context.Context, size abi.PaddedPieceSize, verified bool, tsk types.TipSetKey) (api.DealCollateralBounds, error) {
return c.Internal.StateDealProviderCollateralBounds(ctx, size, verified, tsk)
}
@@ -934,10 +903,6 @@ func (c *FullNodeStruct) MsigGetAvailableBalance(ctx context.Context, a address.
return c.Internal.MsigGetAvailableBalance(ctx, a, tsk)
}
func (c *FullNodeStruct) MsigGetVestingSchedule(ctx context.Context, a address.Address, tsk types.TipSetKey) (api.MsigVesting, error) {
return c.Internal.MsigGetVestingSchedule(ctx, a, tsk)
}
func (c *FullNodeStruct) MsigGetVested(ctx context.Context, a address.Address, sTsk types.TipSetKey, eTsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.MsigGetVested(ctx, a, sTsk, eTsk)
}
@@ -1050,10 +1015,6 @@ func (c *FullNodeStruct) PaychVoucherSubmit(ctx context.Context, ch address.Addr
return c.Internal.PaychVoucherSubmit(ctx, ch, sv, secret, proof)
}
func (c *FullNodeStruct) CreateBackup(ctx context.Context, fpath string) error {
return c.Internal.CreateBackup(ctx, fpath)
}
// StorageMinerStruct
func (c *StorageMinerStruct) ActorAddress(ctx context.Context) (address.Address, error) {
@@ -1294,10 +1255,6 @@ func (c *StorageMinerStruct) PiecesGetCIDInfo(ctx context.Context, payloadCid ci
return c.Internal.PiecesGetCIDInfo(ctx, payloadCid)
}
func (c *StorageMinerStruct) CreateBackup(ctx context.Context, fpath string) error {
return c.Internal.CreateBackup(ctx, fpath)
}
// WorkerStruct
func (w *WorkerStruct) Version(ctx context.Context) (build.Version, error) {
+1 -22
View File
@@ -12,36 +12,15 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node"
"github.com/filecoin-project/lotus/node/impl"
)
func TestCCUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
for _, height := range []abi.ChainEpoch{
1, // before
162, // while sealing
520, // after upgrade deal
5000, // after
} {
height := height // make linters happy by copying
t.Run(fmt.Sprintf("upgrade-%d", height), func(t *testing.T) {
testCCUpgrade(t, b, blocktime, height)
})
}
}
func testCCUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration, upgradeHeight abi.ChainEpoch) {
ctx := context.Background()
n, sn := b(t, 1, OneMiner, node.Override(new(stmgr.UpgradeSchedule), stmgr.UpgradeSchedule{{
Network: build.ActorUpgradeNetworkVersion,
Height: upgradeHeight,
Migration: stmgr.UpgradeActorsV2,
}}))
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
+1 -4
View File
@@ -402,12 +402,9 @@ func testRetrieval(t *testing.T, ctx context.Context, client *impl.FullNodeAPI,
IsCAR: carExport,
}
updates, err := client.ClientRetrieveWithEvents(ctx, offers[0].Order(caddr), ref)
if err != nil {
t.Fatal(err)
}
for update := range updates {
if update.Err != "" {
t.Fatalf("retrieval failed: %s", update.Err)
t.Fatalf("%v", err)
}
}
+1 -46
View File
@@ -71,7 +71,7 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
t.Fatal(err)
}
channelAmt := int64(7000)
channelAmt := int64(100000)
channelInfo, err := paymentCreator.PaychGet(ctx, createrAddr, receiverAddr, abi.NewTokenAmount(channelAmt))
if err != nil {
t.Fatal(err)
@@ -182,51 +182,6 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
t.Fatal("Timed out waiting for receiver to submit vouchers")
}
// Create a new voucher now that some vouchers have already been submitted
vouchRes, err := paymentCreator.PaychVoucherCreate(ctx, channel, abi.NewTokenAmount(1000), 3)
if err != nil {
t.Fatal(err)
}
if vouchRes.Voucher == nil {
t.Fatal(fmt.Errorf("Not enough funds to create voucher: missing %d", vouchRes.Shortfall))
}
vdelta, err := paymentReceiver.PaychVoucherAdd(ctx, channel, vouchRes.Voucher, nil, abi.NewTokenAmount(1000))
if err != nil {
t.Fatal(err)
}
if !vdelta.Equals(abi.NewTokenAmount(1000)) {
t.Fatal("voucher didn't have the right amount")
}
// Create a new voucher whose value would exceed the channel balance
excessAmt := abi.NewTokenAmount(1000)
vouchRes, err = paymentCreator.PaychVoucherCreate(ctx, channel, excessAmt, 4)
if err != nil {
t.Fatal(err)
}
if vouchRes.Voucher != nil {
t.Fatal("Expected not to be able to create voucher whose value would exceed channel balance")
}
if !vouchRes.Shortfall.Equals(excessAmt) {
t.Fatal(fmt.Errorf("Expected voucher shortfall of %d, got %d", excessAmt, vouchRes.Shortfall))
}
// Add a voucher whose value would exceed the channel balance
vouch := &paych.SignedVoucher{ChannelAddr: channel, Amount: excessAmt, Lane: 4, Nonce: 1}
vb, err := vouch.SigningBytes()
if err != nil {
t.Fatal(err)
}
sig, err := paymentCreator.WalletSign(ctx, createrAddr, vb)
if err != nil {
t.Fatal(err)
}
vouch.Signature = sig
_, err = paymentReceiver.PaychVoucherAdd(ctx, channel, vouch, nil, abi.NewTokenAmount(1000))
if err == nil {
t.Fatal(fmt.Errorf("Expected shortfall error of %d", excessAmt))
}
// wait for the settlement period to pass before collecting
waitForBlocks(ctx, t, bm, paymentReceiver, receiverAddr, paych0.SettleDelay)
+1 -2
View File
@@ -12,7 +12,6 @@ import (
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/lotus/node"
)
type TestNode struct {
@@ -45,7 +44,7 @@ type StorageMiner struct {
//
// storage array defines storage nodes, numbers in the array specify full node
// index the storage node 'belongs' to
type APIBuilder func(t *testing.T, nFull int, storage []StorageMiner, opts ...node.Option) ([]TestNode, []TestStorageNode)
type APIBuilder func(t *testing.T, nFull int, storage []StorageMiner) ([]TestNode, []TestStorageNode)
type testSuite struct {
makeNodes APIBuilder
}
+24 -47
View File
@@ -15,11 +15,9 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/mock"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/lotus/node"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
bminer "github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/lotus/node/impl"
@@ -116,29 +114,8 @@ func pledgeSectors(t *testing.T, ctx context.Context, miner TestStorageNode, n,
}
func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int) {
for _, height := range []abi.ChainEpoch{
1, // before
162, // while sealing
5000, // while proving
} {
height := height // copy to satisfy lints
t.Run(fmt.Sprintf("upgrade-%d", height), func(t *testing.T) {
testWindowPostUpgrade(t, b, blocktime, nSectors, height)
})
}
}
func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int,
upgradeHeight abi.ChainEpoch) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
n, sn := b(t, 1, OneMiner, node.Override(new(stmgr.UpgradeSchedule), stmgr.UpgradeSchedule{{
Network: build.ActorUpgradeNetworkVersion,
Height: upgradeHeight,
Migration: stmgr.UpgradeActorsV2,
}}))
ctx := context.Background()
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@@ -152,24 +129,17 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
}
build.Clock.Sleep(time.Second)
mine := true
done := make(chan struct{})
go func() {
defer close(done)
for ctx.Err() == nil {
for mine {
build.Clock.Sleep(blocktime)
if err := sn[0].MineOne(ctx, MineNext); err != nil {
if ctx.Err() != nil {
// context was canceled, ignore the error.
return
}
t.Error(err)
}
}
}()
defer func() {
cancel()
<-done
}()
pledgeSectors(t, ctx, miner, nSectors, 0, nil)
@@ -183,16 +153,18 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
require.NoError(t, err)
fmt.Printf("Running one proving period\n")
fmt.Printf("End for head.Height > %d\n", di.PeriodStart+di.WPoStProvingPeriod+2)
for {
head, err := client.ChainHead(ctx)
require.NoError(t, err)
if head.Height() > di.PeriodStart+di.WPoStProvingPeriod+2 {
fmt.Printf("Now head.Height = %d\n", head.Height())
if head.Height() > di.PeriodStart+(di.WPoStProvingPeriod)+2 {
break
}
if head.Height()%100 == 0 {
fmt.Printf("@%d\n", head.Height())
}
build.Clock.Sleep(blocktime)
}
@@ -214,6 +186,7 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
require.Greater(t, len(parts), 0)
secs := parts[0].AllSectors
require.NoError(t, err)
n, err := secs.Count()
require.NoError(t, err)
require.Equal(t, uint64(2), n)
@@ -237,6 +210,7 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
require.Greater(t, len(parts), 0)
secs := parts[0].AllSectors
require.NoError(t, err)
n, err := secs.Count()
require.NoError(t, err)
require.Equal(t, uint64(2), n)
@@ -262,17 +236,18 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
require.NoError(t, err)
fmt.Printf("Go through another PP, wait for sectors to become faulty\n")
fmt.Printf("End for head.Height > %d\n", di.PeriodStart+di.WPoStProvingPeriod+2)
for {
head, err := client.ChainHead(ctx)
require.NoError(t, err)
if head.Height() > di.PeriodStart+(di.WPoStProvingPeriod)+2 {
fmt.Printf("Now head.Height = %d\n", head.Height())
break
}
if head.Height()%100 == 0 {
fmt.Printf("@%d\n", head.Height())
}
build.Clock.Sleep(blocktime)
}
@@ -292,17 +267,17 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
di, err = client.StateMinerProvingDeadline(ctx, maddr, types.EmptyTSK)
require.NoError(t, err)
fmt.Printf("End for head.Height > %d\n", di.PeriodStart+di.WPoStProvingPeriod+2)
for {
head, err := client.ChainHead(ctx)
require.NoError(t, err)
if head.Height() > di.PeriodStart+di.WPoStProvingPeriod+2 {
fmt.Printf("Now head.Height = %d\n", head.Height())
break
}
if head.Height()%100 == 0 {
fmt.Printf("@%d\n", head.Height())
}
build.Clock.Sleep(blocktime)
}
@@ -319,19 +294,18 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
pledgeSectors(t, ctx, miner, 1, nSectors, nil)
{
// Wait until proven.
di, err = client.StateMinerProvingDeadline(ctx, maddr, types.EmptyTSK)
// wait a bit more
head, err := client.ChainHead(ctx)
require.NoError(t, err)
waitUntil := di.PeriodStart + di.WPoStProvingPeriod + 2
fmt.Printf("End for head.Height > %d\n", waitUntil)
waitUntil := head.Height() + 10
for {
head, err := client.ChainHead(ctx)
require.NoError(t, err)
if head.Height() > waitUntil {
fmt.Printf("Now head.Height = %d\n", head.Height())
break
}
}
@@ -344,4 +318,7 @@ func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration,
sectors = p.MinerPower.RawBytePower.Uint64() / uint64(ssz)
require.Equal(t, nSectors+GenesisPreseals-2+1, int(sectors)) // -2 not recovered sectors + 1 just pledged
mine = false
<-done
}
+2 -12
View File
@@ -1,12 +1,2 @@
/dns4/bootstrap-0.testnet.fildev.network/tcp/1347/p2p/12D3KooWJTUBUjtzWJGWU1XSiY21CwmHaCNLNYn2E7jqHEHyZaP7
/dns4/bootstrap-1.testnet.fildev.network/tcp/1347/p2p/12D3KooW9yeKXha4hdrJKq74zEo99T8DhriQdWNoojWnnQbsgB3v
/dns4/bootstrap-2.testnet.fildev.network/tcp/1347/p2p/12D3KooWCrx8yVG9U9Kf7w8KLN3Edkj5ZKDhgCaeMqQbcQUoB6CT
/dns4/bootstrap-4.testnet.fildev.network/tcp/1347/p2p/12D3KooWPkL9LrKRQgHtq7kn9ecNhGU9QaziG8R5tX8v9v7t3h34
/dns4/bootstrap-3.testnet.fildev.network/tcp/1347/p2p/12D3KooWKYSsbpgZ3HAjax5M1BXCwXLa6gVkUARciz7uN3FNtr7T
/dns4/bootstrap-5.testnet.fildev.network/tcp/1347/p2p/12D3KooWQYzqnLASJAabyMpPb1GcWZvNSe7JDcRuhdRqonFoiK9W
/dns4/lotus-bootstrap.forceup.cn/tcp/41778/p2p/12D3KooWFQsv3nRMUevZNWWsY1Wu6NUzUbawnWU5NcRhgKuJA37C
/dns4/bootstrap-0.starpool.in/tcp/12757/p2p/12D3KooWGHpBMeZbestVEWkfdnC9u7p6uFHXL1n7m1ZBqsEmiUzz
/dns4/bootstrap-1.starpool.in/tcp/12757/p2p/12D3KooWQZrGH1PxSNZPum99M1zNvjNFM33d1AAu5DcvdHptuU7u
/dns4/node.glif.io/tcp/1235/p2p/12D3KooWBF8cpp65hp2u9LK5mh19x67ftAam84z9LsfaquTDSBpt
/dns4/bootstrap-0.ipfsmain.cn/tcp/34721/p2p/12D3KooWQnwEGNqcM2nAcPtRR9rAX8Hrg4k9kJLCHoTR5chJfz6d
/dns4/bootstrap-1.ipfsmain.cn/tcp/34723/p2p/12D3KooWMKxMkD5DMpSWsW7dBddKxKT7L2GgbNuckz9otxvkvByP
/dns4/bootstrap-0.butterfly.fildev.network/tcp/1347/p2p/12D3KooWK3WNs8LcC5bNVRmcfM6QMNV8btBxYwNhxaP1NCv4ueGw
/dns4/bootstrap-1.butterfly.fildev.network/tcp/1347/p2p/12D3KooWEq5DpaYddHQ1AyxCUamFa5y6kkWZ7mjHL5zG1jvU1GeM
Binary file not shown.
+2 -12
View File
@@ -3,9 +3,6 @@
package build
import (
"math"
"os"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/actors/policy"
@@ -16,10 +13,8 @@ const BreezeGasTampingDuration = 0
const UpgradeSmokeHeight = -1
const UpgradeIgnitionHeight = -2
const UpgradeRefuelHeight = -3
var UpgradeActorsV2Height = abi.ChainEpoch(10)
var UpgradeLiftoffHeight = abi.ChainEpoch(-4)
const UpgradeLiftoffHeight = -3
const UpgradeActorsV2 = 10
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
@@ -30,11 +25,6 @@ func init() {
policy.SetConsensusMinerMinPower(abi.NewStoragePower(2048))
policy.SetMinVerifiedDealSize(abi.NewStoragePower(256))
if os.Getenv("LOTUS_DISABLE_V2_ACTOR_MIGRATION") == "1" {
UpgradeActorsV2Height = math.MaxInt64
UpgradeLiftoffHeight = 11
}
BuildType |= Build2k
}
+1 -1
View File
@@ -41,7 +41,7 @@ func DhtProtocolName(netName dtypes.NetworkName) protocol.ID {
func UseNewestNetwork() bool {
// TODO: Put these in a container we can iterate over
if UpgradeBreezeHeight <= 0 && UpgradeSmokeHeight <= 0 && UpgradeActorsV2Height <= 0 {
if UpgradeBreezeHeight <= 0 && UpgradeSmokeHeight <= 0 && UpgradeActorsV2 <= 0 {
return true
}
return false
+2 -5
View File
@@ -56,6 +56,8 @@ const SealRandomnessLookback = policy.SealRandomnessLookback
// Epochs
const TicketRandomnessLookback = abi.ChainEpoch(1)
const WinningPoStSectorSetLookback = abi.ChainEpoch(10)
// /////
// Address
@@ -70,10 +72,8 @@ const FilBase = uint64(2_000_000_000)
const FilAllocStorageMining = uint64(1_100_000_000)
const FilecoinPrecision = uint64(1_000_000_000_000_000_000)
const FilReserved = uint64(300_000_000)
var InitialRewardBalance *big.Int
var InitialFilReserved *big.Int
// TODO: Move other important consts here
@@ -81,9 +81,6 @@ func init() {
InitialRewardBalance = big.NewInt(int64(FilAllocStorageMining))
InitialRewardBalance = InitialRewardBalance.Mul(InitialRewardBalance, big.NewInt(int64(FilecoinPrecision)))
InitialFilReserved = big.NewInt(int64(FilReserved))
InitialFilReserved = InitialFilReserved.Mul(InitialFilReserved, big.NewInt(int64(FilecoinPrecision)))
if os.Getenv("LOTUS_ADDRESS_TYPE") == AddressMainnetEnvVar {
SetAddressNetwork(address.Mainnet)
}
+6 -14
View File
@@ -50,11 +50,11 @@ var (
SealRandomnessLookback = policy.SealRandomnessLookback
TicketRandomnessLookback = abi.ChainEpoch(1)
TicketRandomnessLookback = abi.ChainEpoch(1)
WinningPoStSectorSetLookback = abi.ChainEpoch(10)
FilBase uint64 = 2_000_000_000
FilAllocStorageMining uint64 = 1_400_000_000
FilReserved uint64 = 300_000_000
FilecoinPrecision uint64 = 1_000_000_000_000_000_000
@@ -63,13 +63,6 @@ var (
v = v.Mul(v, big.NewInt(int64(FilecoinPrecision)))
return v
}()
InitialFilReserved = func() *big.Int {
v := big.NewInt(int64(FilReserved))
v = v.Mul(v, big.NewInt(int64(FilecoinPrecision)))
return v
}()
// Actor consts
// TODO: Pull from actors when its made not private
MinDealDuration = abi.ChainEpoch(180 * builtin.EpochsInDay)
@@ -82,16 +75,15 @@ var (
UpgradeSmokeHeight abi.ChainEpoch = -1
UpgradeIgnitionHeight abi.ChainEpoch = -2
UpgradeRefuelHeight abi.ChainEpoch = -3
UpgradeActorsV2Height abi.ChainEpoch = 10
UpgradeLiftoffHeight abi.ChainEpoch = -4
UpgradeLiftoffHeight abi.ChainEpoch = -3
UpgradeActorsV2 abi.ChainEpoch = 10
DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}
NewestNetworkVersion = network.Version4
ActorUpgradeNetworkVersion = network.Version4
NewestNetworkVersion = network.Version2
ActorUpgradeNetworkVersion = network.Version3
Devnet = true
)
+7 -21
View File
@@ -5,10 +5,6 @@
package build
import (
"math"
"os"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/actors/policy"
@@ -16,19 +12,17 @@ import (
)
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandIncentinet,
UpgradeSmokeHeight: DrandMainnet,
0: DrandMainnet,
}
const UpgradeBreezeHeight = 41280
const UpgradeBreezeHeight = -1
const BreezeGasTampingDuration = 120
const UpgradeSmokeHeight = 51000
const UpgradeSmokeHeight = -2
const UpgradeIgnitionHeight = 94000
const UpgradeRefuelHeight = 130800
const UpgradeIgnitionHeight = -3
var UpgradeActorsV2Height = abi.ChainEpoch(138720)
const UpgradeActorsV2 = 500
// This signals our tentative epoch for mainnet launch. Can make it later, but not earlier.
// Miners, clients, developers, custodians all need time to prepare.
@@ -36,20 +30,12 @@ var UpgradeActorsV2Height = abi.ChainEpoch(138720)
const UpgradeLiftoffHeight = 148888
func init() {
policy.SetConsensusMinerMinPower(abi.NewStoragePower(10 << 40))
policy.SetConsensusMinerMinPower(abi.NewStoragePower(1 << 30))
policy.SetSupportedProofTypes(
abi.RegisteredSealProof_StackedDrg512MiBV1,
abi.RegisteredSealProof_StackedDrg32GiBV1,
abi.RegisteredSealProof_StackedDrg64GiBV1,
)
if os.Getenv("LOTUS_USE_TEST_ADDRESSES") != "1" {
SetAddressNetwork(address.Mainnet)
}
if os.Getenv("LOTUS_DISABLE_V2_ACTOR_MIGRATION") == "1" {
UpgradeActorsV2Height = math.MaxInt64
}
Devnet = false
}
+1 -1
View File
@@ -29,7 +29,7 @@ func buildType() string {
}
// BuildVersion is the local build version, set by build system
const BuildVersion = "0.9.0"
const BuildVersion = "0.8.0"
func UserVersion() string {
return BuildVersion + buildType() + CurrentCommit
-12
View File
@@ -1,7 +1,6 @@
package builtin
import (
"github.com/filecoin-project/go-address"
"github.com/ipfs/go-cid"
"golang.org/x/xerrors"
@@ -21,8 +20,6 @@ import (
var SystemActorAddr = builtin0.SystemActorAddr
var BurntFundsActorAddr = builtin0.BurntFundsActorAddr
var ReserveAddress = makeAddress("t090")
var RootVerifierAddress = makeAddress("t080")
// TODO: Why does actors have 2 different versions of this?
type SectorInfo = proof0.SectorInfo
@@ -89,12 +86,3 @@ func IsMultisigActor(c cid.Cid) bool {
func IsPaymentChannelActor(c cid.Cid) bool {
return c == builtin0.PaymentChannelActorCodeID || c == builtin2.PaymentChannelActorCodeID
}
func makeAddress(addr string) address.Address {
ret, err := address.NewFromString(addr)
if err != nil {
panic(err)
}
return ret
}
@@ -1,180 +0,0 @@
package miner
import (
"errors"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-state-types/exitcode"
)
type DeadlinesDiff map[uint64]*DeadlineDiff
func DiffDeadlines(pre, cur State) (*DeadlinesDiff, error) {
changed, err := pre.DeadlinesChanged(cur)
if err != nil {
return nil, err
}
if !changed {
return nil, nil
}
numDl, err := pre.NumDeadlines()
if err != nil {
return nil, err
}
dlDiff := make(DeadlinesDiff, numDl)
if err := pre.ForEachDeadline(func(idx uint64, preDl Deadline) error {
curDl, err := cur.LoadDeadline(idx)
if err != nil {
return err
}
diff, err := DiffDeadline(preDl, curDl)
if err != nil {
return err
}
dlDiff[idx] = diff
return nil
}); err != nil {
return nil, err
}
return &dlDiff, nil
}
type DeadlineDiff map[uint64]*PartitionDiff
func DiffDeadline(pre, cur Deadline) (*DeadlineDiff, error) {
changed, err := pre.PartitionsChanged(cur)
if err != nil {
return nil, err
}
if !changed {
return nil, nil
}
partDiff := make(DeadlineDiff)
if err := pre.ForEachPartition(func(idx uint64, prePart Partition) error {
// try loading current partition at this index
curPart, err := cur.LoadPartition(idx)
if err != nil {
if errors.Is(err, exitcode.ErrNotFound) {
// TODO correctness?
return nil // the partition was removed.
}
return err
}
// compare it with the previous partition
diff, err := DiffPartition(prePart, curPart)
if err != nil {
return err
}
partDiff[idx] = diff
return nil
}); err != nil {
return nil, err
}
// all previous partitions have been walked.
// all partitions in cur and not in prev are new... can they be faulty already?
// TODO is this correct?
if err := cur.ForEachPartition(func(idx uint64, curPart Partition) error {
if _, found := partDiff[idx]; found {
return nil
}
faults, err := curPart.FaultySectors()
if err != nil {
return err
}
recovering, err := curPart.RecoveringSectors()
if err != nil {
return err
}
partDiff[idx] = &PartitionDiff{
Removed: bitfield.New(),
Recovered: bitfield.New(),
Faulted: faults,
Recovering: recovering,
}
return nil
}); err != nil {
return nil, err
}
return &partDiff, nil
}
type PartitionDiff struct {
Removed bitfield.BitField
Recovered bitfield.BitField
Faulted bitfield.BitField
Recovering bitfield.BitField
}
func DiffPartition(pre, cur Partition) (*PartitionDiff, error) {
prevLiveSectors, err := pre.LiveSectors()
if err != nil {
return nil, err
}
curLiveSectors, err := cur.LiveSectors()
if err != nil {
return nil, err
}
removed, err := bitfield.SubtractBitField(prevLiveSectors, curLiveSectors)
if err != nil {
return nil, err
}
prevRecoveries, err := pre.RecoveringSectors()
if err != nil {
return nil, err
}
curRecoveries, err := cur.RecoveringSectors()
if err != nil {
return nil, err
}
recovering, err := bitfield.SubtractBitField(curRecoveries, prevRecoveries)
if err != nil {
return nil, err
}
prevFaults, err := pre.FaultySectors()
if err != nil {
return nil, err
}
curFaults, err := cur.FaultySectors()
if err != nil {
return nil, err
}
faulted, err := bitfield.SubtractBitField(curFaults, prevFaults)
if err != nil {
return nil, err
}
// all current good sectors
curActiveSectors, err := cur.ActiveSectors()
if err != nil {
return nil, err
}
// sectors that were previously fault and are now currently active are considered recovered.
recovered, err := bitfield.IntersectBitField(prevFaults, curActiveSectors)
if err != nil {
return nil, err
}
return &PartitionDiff{
Removed: removed,
Recovered: recovered,
Faulted: faulted,
Recovering: recovering,
}, nil
}
-6
View File
@@ -32,10 +32,6 @@ func init() {
// Unchanged between v0 and v2 actors
var WPoStProvingPeriod = miner0.WPoStProvingPeriod
var WPoStPeriodDeadlines = miner0.WPoStPeriodDeadlines
var WPoStChallengeWindow = miner0.WPoStChallengeWindow
var WPoStChallengeLookback = miner0.WPoStChallengeLookback
var FaultDeclarationCutoff = miner0.FaultDeclarationCutoff
const MinSectorExpiration = miner0.MinSectorExpiration
@@ -58,7 +54,6 @@ type State interface {
VestedFunds(abi.ChainEpoch) (abi.TokenAmount, error)
// Funds locked for various reasons.
LockedFunds() (LockedFunds, error)
FeeDebt() (abi.TokenAmount, error)
GetSector(abi.SectorNumber) (*SectorOnChainInfo, error)
FindSector(abi.SectorNumber) (*SectorLocation, error)
@@ -145,7 +140,6 @@ type MinerInfo struct {
SealProofType abi.RegisteredSealProof
SectorSize abi.SectorSize
WindowPoStPartitionSectors uint64
ConsensusFaultElapsed abi.ChainEpoch
}
type SectorExpiration struct {
-7
View File
@@ -4,8 +4,6 @@ import (
"bytes"
"errors"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-state-types/abi"
@@ -63,10 +61,6 @@ func (s *state0) LockedFunds() (LockedFunds, error) {
}, nil
}
func (s *state0) FeeDebt() (abi.TokenAmount, error) {
return big.Zero(), nil
}
func (s *state0) InitialPledge() (abi.TokenAmount, error) {
return s.State.InitialPledgeRequirement, nil
}
@@ -293,7 +287,6 @@ func (s *state0) Info() (MinerInfo, error) {
SealProofType: info.SealProofType,
SectorSize: info.SectorSize,
WindowPoStPartitionSectors: info.WindowPoStPartitionSectors,
ConsensusFaultElapsed: -1,
}
if info.PendingWorkerKey != nil {
+5 -9
View File
@@ -11,7 +11,6 @@ import (
"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors/adt"
@@ -46,7 +45,7 @@ type partition2 struct {
}
func (s *state2) AvailableBalance(bal abi.TokenAmount) (abi.TokenAmount, error) {
return s.GetAvailableBalance(bal)
return s.GetAvailableBalance(bal), nil
}
func (s *state2) VestedFunds(epoch abi.ChainEpoch) (abi.TokenAmount, error) {
@@ -61,10 +60,6 @@ func (s *state2) LockedFunds() (LockedFunds, error) {
}, nil
}
func (s *state2) FeeDebt() (abi.TokenAmount, error) {
return s.State.FeeDebt, nil
}
func (s *state2) InitialPledge() (abi.TokenAmount, error) {
return s.State.InitialPledge, nil
}
@@ -111,7 +106,9 @@ func (s *state2) NumLiveSectors() (uint64, error) {
// GetSectorExpiration returns the effective expiration of the given sector.
//
// If the sector does not expire early, the Early expiration field is 0.
// If the sector isn't found or has already been terminated, this method returns
// nil and no error. If the sector does not expire early, the Early expiration
// field is 0.
func (s *state2) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, error) {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
@@ -174,7 +171,7 @@ func (s *state2) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
return nil, err
}
if out.Early == 0 && out.OnTime == 0 {
return nil, xerrors.Errorf("failed to find sector %d", num)
return nil, nil
}
return &out, nil
}
@@ -292,7 +289,6 @@ func (s *state2) Info() (MinerInfo, error) {
SealProofType: info.SealProofType,
SectorSize: info.SectorSize,
WindowPoStPartitionSectors: info.WindowPoStPartitionSectors,
ConsensusFaultElapsed: info.ConsensusFaultElapsed,
}
if info.PendingWorkerKey != nil {
-70
View File
@@ -1,70 +0,0 @@
package multisig
import (
"fmt"
"github.com/minio/blake2b-simd"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
multisig2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/types"
)
func Message(version actors.Version, from address.Address) MessageBuilder {
switch version {
case actors.Version0:
return message0{from}
case actors.Version2:
return message2{message0{from}}
default:
panic(fmt.Sprintf("unsupported actors version: %d", version))
}
}
type MessageBuilder interface {
// Create a new multisig with the specified parameters.
Create(signers []address.Address, threshold uint64,
vestingStart, vestingDuration abi.ChainEpoch,
initialAmount abi.TokenAmount) (*types.Message, error)
// Propose a transaction to the given multisig.
Propose(msig, target address.Address, amt abi.TokenAmount,
method abi.MethodNum, params []byte) (*types.Message, error)
// Approve a multisig transaction. The "hash" is optional.
Approve(msig address.Address, txID uint64, hash *ProposalHashData) (*types.Message, error)
// Cancel a multisig transaction. The "hash" is optional.
Cancel(msig address.Address, txID uint64, hash *ProposalHashData) (*types.Message, error)
}
// this type is the same between v0 and v2
type ProposalHashData = multisig2.ProposalHashData
func txnParams(id uint64, data *ProposalHashData) ([]byte, error) {
params := multisig2.TxnIDParams{ID: multisig2.TxnID(id)}
if data != nil {
if data.Requester.Protocol() != address.ID {
return nil, xerrors.Errorf("proposer address must be an ID address, was %s", data.Requester)
}
if data.Value.Sign() == -1 {
return nil, xerrors.Errorf("proposal value must be non-negative, was %s", data.Value)
}
if data.To == address.Undef {
return nil, xerrors.Errorf("proposed destination address must be set")
}
pser, err := data.Serialize()
if err != nil {
return nil, err
}
hash := blake2b.Sum256(pser)
params.ProposalHash = hash[:]
}
return actors.SerializeParams(&params)
}
-142
View File
@@ -1,142 +0,0 @@
package multisig
import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
init0 "github.com/filecoin-project/specs-actors/actors/builtin/init"
multisig0 "github.com/filecoin-project/specs-actors/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message0 struct{ from address.Address }
func (m message0) Create(
signers []address.Address, threshold uint64,
unlockStart, unlockDuration abi.ChainEpoch,
initialAmount abi.TokenAmount,
) (*types.Message, error) {
lenAddrs := uint64(len(signers))
if lenAddrs < threshold {
return nil, xerrors.Errorf("cannot require signing of more addresses than provided for multisig")
}
if threshold == 0 {
threshold = lenAddrs
}
if m.from == address.Undef {
return nil, xerrors.Errorf("must provide source address")
}
if unlockStart != 0 {
return nil, xerrors.Errorf("actors v0 does not support a non-zero vesting start time")
}
// Set up constructor parameters for multisig
msigParams := &multisig0.ConstructorParams{
Signers: signers,
NumApprovalsThreshold: threshold,
UnlockDuration: unlockDuration,
}
enc, actErr := actors.SerializeParams(msigParams)
if actErr != nil {
return nil, actErr
}
// new actors are created by invoking 'exec' on the init actor with the constructor params
execParams := &init0.ExecParams{
CodeCID: builtin0.MultisigActorCodeID,
ConstructorParams: enc,
}
enc, actErr = actors.SerializeParams(execParams)
if actErr != nil {
return nil, actErr
}
return &types.Message{
To: init_.Address,
From: m.from,
Method: builtin0.MethodsInit.Exec,
Params: enc,
Value: initialAmount,
}, nil
}
func (m message0) Propose(msig, to address.Address, amt abi.TokenAmount,
method abi.MethodNum, params []byte) (*types.Message, error) {
if msig == address.Undef {
return nil, xerrors.Errorf("must provide a multisig address for proposal")
}
if to == address.Undef {
return nil, xerrors.Errorf("must provide a target address for proposal")
}
if amt.Sign() == -1 {
return nil, xerrors.Errorf("must provide a non-negative amount for proposed send")
}
if m.from == address.Undef {
return nil, xerrors.Errorf("must provide source address")
}
enc, actErr := actors.SerializeParams(&multisig0.ProposeParams{
To: to,
Value: amt,
Method: method,
Params: params,
})
if actErr != nil {
return nil, xerrors.Errorf("failed to serialize parameters: %w", actErr)
}
return &types.Message{
To: msig,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin0.MethodsMultisig.Propose,
Params: enc,
}, nil
}
func (m message0) Approve(msig address.Address, txID uint64, hashData *ProposalHashData) (*types.Message, error) {
enc, err := txnParams(txID, hashData)
if err != nil {
return nil, err
}
return &types.Message{
To: msig,
From: m.from,
Value: types.NewInt(0),
Method: builtin0.MethodsMultisig.Approve,
Params: enc,
}, nil
}
func (m message0) Cancel(msig address.Address, txID uint64, hashData *ProposalHashData) (*types.Message, error) {
enc, err := txnParams(txID, hashData)
if err != nil {
return nil, err
}
return &types.Message{
To: msig,
From: m.from,
Value: types.NewInt(0),
Method: builtin0.MethodsMultisig.Cancel,
Params: enc,
}, nil
}
-71
View File
@@ -1,71 +0,0 @@
package multisig
import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin2 "github.com/filecoin-project/specs-actors/v2/actors/builtin"
init2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/init"
multisig2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message2 struct{ message0 }
func (m message2) Create(
signers []address.Address, threshold uint64,
unlockStart, unlockDuration abi.ChainEpoch,
initialAmount abi.TokenAmount,
) (*types.Message, error) {
lenAddrs := uint64(len(signers))
if lenAddrs < threshold {
return nil, xerrors.Errorf("cannot require signing of more addresses than provided for multisig")
}
if threshold == 0 {
threshold = lenAddrs
}
if m.from == address.Undef {
return nil, xerrors.Errorf("must provide source address")
}
// Set up constructor parameters for multisig
msigParams := &multisig2.ConstructorParams{
Signers: signers,
NumApprovalsThreshold: threshold,
UnlockDuration: unlockDuration,
StartEpoch: unlockStart,
}
enc, actErr := actors.SerializeParams(msigParams)
if actErr != nil {
return nil, actErr
}
// new actors are created by invoking 'exec' on the init actor with the constructor params
execParams := &init2.ExecParams{
CodeCID: builtin2.MultisigActorCodeID,
ConstructorParams: enc,
}
enc, actErr = actors.SerializeParams(execParams)
if actErr != nil {
return nil, actErr
}
return &types.Message{
To: init_.Address,
From: m.from,
Method: builtin2.MethodsInit.Exec,
Params: enc,
Value: initialAmount,
}, nil
}
-28
View File
@@ -1,28 +0,0 @@
package paych
import (
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/types"
)
func Message(version actors.Version, from address.Address) MessageBuilder {
switch version {
case actors.Version0:
return message0{from}
case actors.Version2:
return message2{from}
default:
panic(fmt.Sprintf("unsupported actors version: %d", version))
}
}
type MessageBuilder interface {
Create(to address.Address, initialAmount abi.TokenAmount) (*types.Message, error)
Update(paych address.Address, voucher *SignedVoucher, secret []byte) (*types.Message, error)
Settle(paych address.Address) (*types.Message, error)
Collect(paych address.Address) (*types.Message, error)
}
-74
View File
@@ -1,74 +0,0 @@
package paych
import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
init0 "github.com/filecoin-project/specs-actors/actors/builtin/init"
paych0 "github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message0 struct{ from address.Address }
func (m message0) Create(to address.Address, initialAmount abi.TokenAmount) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych0.ConstructorParams{From: m.from, To: to})
if aerr != nil {
return nil, aerr
}
enc, aerr := actors.SerializeParams(&init0.ExecParams{
CodeCID: builtin0.PaymentChannelActorCodeID,
ConstructorParams: params,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: init_.Address,
From: m.from,
Value: initialAmount,
Method: builtin0.MethodsInit.Exec,
Params: enc,
}, nil
}
func (m message0) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych0.UpdateChannelStateParams{
Sv: *sv,
Secret: secret,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin0.MethodsPaych.UpdateChannelState,
Params: params,
}, nil
}
func (m message0) Settle(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin0.MethodsPaych.Settle,
}, nil
}
func (m message0) Collect(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin0.MethodsPaych.Collect,
}, nil
}
-74
View File
@@ -1,74 +0,0 @@
package paych
import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin2 "github.com/filecoin-project/specs-actors/v2/actors/builtin"
init2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/init"
paych2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/paych"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message2 struct{ from address.Address }
func (m message2) Create(to address.Address, initialAmount abi.TokenAmount) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych2.ConstructorParams{From: m.from, To: to})
if aerr != nil {
return nil, aerr
}
enc, aerr := actors.SerializeParams(&init2.ExecParams{
CodeCID: builtin2.PaymentChannelActorCodeID,
ConstructorParams: params,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: init_.Address,
From: m.from,
Value: initialAmount,
Method: builtin2.MethodsInit.Exec,
Params: enc,
}, nil
}
func (m message2) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych2.UpdateChannelStateParams{
Sv: *sv,
Secret: secret,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin2.MethodsPaych.UpdateChannelState,
Params: params,
}, nil
}
func (m message2) Settle(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin2.MethodsPaych.Settle,
}, nil
}
func (m message2) Collect(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin2.MethodsPaych.Collect,
}, nil
}
+2 -1
View File
@@ -27,9 +27,10 @@ type mockLaneState struct {
func NewMockPayChState(from address.Address,
to address.Address,
settlingAt abi.ChainEpoch,
toSend abi.TokenAmount,
lanes map[uint64]paych.LaneState,
) paych.State {
return &mockState{from: from, to: to, settlingAt: settlingAt, toSend: big.NewInt(0), lanes: lanes}
return &mockState{from, to, settlingAt, toSend, lanes}
}
// NewMockLaneState constructs a state for a payment channel lane with the set fixed values
-4
View File
@@ -27,10 +27,6 @@ type state0 struct {
store adt.Store
}
func (s *state0) RootKey() (address.Address, error) {
return s.State.RootKey, nil
}
func (s *state0) VerifiedClientDataCap(addr address.Address) (bool, abi.StoragePower, error) {
return getDataCap(s.store, actors.Version0, s.State.VerifiedClients, addr)
}
-4
View File
@@ -27,10 +27,6 @@ type state2 struct {
store adt.Store
}
func (s *state2) RootKey() (address.Address, error) {
return s.State.RootKey, nil
}
func (s *state2) VerifiedClientDataCap(addr address.Address) (bool, abi.StoragePower, error) {
return getDataCap(s.store, actors.Version2, s.State.VerifiedClients, addr)
}
@@ -39,7 +39,6 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
type State interface {
cbor.Marshaler
RootKey() (address.Address, error)
VerifiedClientDataCap(address.Address) (bool, abi.StoragePower, error)
VerifierDataCap(address.Address) (bool, abi.StoragePower, error)
ForEachVerifier(func(addr address.Address, dcap abi.StoragePower) error) error
+1 -36
View File
@@ -2,14 +2,12 @@ package policy
import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/chain/actors"
market0 "github.com/filecoin-project/specs-actors/actors/builtin/market"
miner0 "github.com/filecoin-project/specs-actors/actors/builtin/miner"
power0 "github.com/filecoin-project/specs-actors/actors/builtin/power"
verifreg0 "github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
builtin2 "github.com/filecoin-project/specs-actors/v2/actors/builtin"
market2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/market"
miner2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/miner"
verifreg2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/verifreg"
)
@@ -81,36 +79,3 @@ func GetMaxProveCommitDuration(ver actors.Version, t abi.RegisteredSealProof) ab
panic("unsupported actors version")
}
}
func DealProviderCollateralBounds(
size abi.PaddedPieceSize, verified bool,
rawBytePower, qaPower, baselinePower abi.StoragePower,
circulatingFil abi.TokenAmount, nwVer network.Version,
) (min, max abi.TokenAmount) {
switch actors.VersionForNetwork(nwVer) {
case actors.Version0:
return market0.DealProviderCollateralBounds(size, verified, rawBytePower, qaPower, baselinePower, circulatingFil, nwVer)
case actors.Version2:
return market2.DealProviderCollateralBounds(size, verified, rawBytePower, qaPower, baselinePower, circulatingFil)
default:
panic("unsupported network version")
}
}
// Sets the challenge window and scales the proving period to match (such that
// there are always 48 challenge windows in a proving period).
func SetWPoStChallengeWindow(period abi.ChainEpoch) {
miner0.WPoStChallengeWindow = period
miner0.WPoStProvingPeriod = period * abi.ChainEpoch(miner0.WPoStPeriodDeadlines)
miner2.WPoStChallengeWindow = period
miner2.WPoStProvingPeriod = period * abi.ChainEpoch(miner2.WPoStPeriodDeadlines)
}
func GetWinningPoStSectorSetLookback(nwVer network.Version) abi.ChainEpoch {
if nwVer <= network.Version3 {
return 10
}
return ChainFinality
}
-3
View File
@@ -43,8 +43,5 @@ func TestAssumptions(t *testing.T) {
require.EqualValues(t, miner0.SupportedProofTypes, miner2.SupportedProofTypes)
require.Equal(t, miner0.PreCommitChallengeDelay, miner2.PreCommitChallengeDelay)
require.Equal(t, miner0.ChainFinality, miner2.ChainFinality)
require.Equal(t, miner0.WPoStChallengeWindow, miner2.WPoStChallengeWindow)
require.Equal(t, miner0.WPoStProvingPeriod, miner2.WPoStProvingPeriod)
require.Equal(t, miner0.WPoStPeriodDeadlines, miner2.WPoStPeriodDeadlines)
require.True(t, verifreg0.MinVerifiedDealSize.Equals(verifreg2.MinVerifiedDealSize))
}
+2 -2
View File
@@ -9,8 +9,8 @@ import (
type Version int
const (
Version0 Version = 0
Version2 Version = 2
Version0 = 0
Version2 = 2
)
// Converts a network version into an actors adt version.
+1 -1
View File
@@ -117,7 +117,7 @@ func (c *client) doRequest(
res, err := c.sendRequestToPeer(ctx, peer, req)
if err != nil {
if !xerrors.Is(err, network.ErrNoConn) {
log.Warnf("could not send request to peer %s: %s",
log.Warnf("could not connect to peer %s: %s",
peer.String(), err)
}
continue
+1 -2
View File
@@ -9,7 +9,6 @@ import (
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/google/uuid"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
@@ -225,7 +224,7 @@ func NewGeneratorWithSectors(numSectors int) (*ChainGen, error) {
},
VerifregRootKey: DefaultVerifregRootkeyActor,
RemainderAccount: DefaultRemainderAccountActor,
NetworkName: uuid.New().String(),
NetworkName: "",
Timestamp: uint64(build.Clock.Now().Add(-500 * time.Duration(build.BlockDelaySecs) * time.Second).Unix()),
}
+8 -4
View File
@@ -6,8 +6,6 @@ import (
"encoding/json"
"fmt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
@@ -26,6 +24,7 @@ import (
adt0 "github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
@@ -116,7 +115,7 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, xerrors.Errorf("putting empty object: %w", err)
}
state, err := state.NewStateTree(cst, types.StateTreeVersion0)
state, err := state.NewStateTree(cst, actors.Version0)
if err != nil {
return nil, nil, xerrors.Errorf("making new state tree: %w", err)
}
@@ -297,9 +296,14 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, xerrors.Errorf("somehow overallocated filecoin (allocated = %s)", types.FIL(totalFilAllocated))
}
remAccKey, err := address.NewIDAddress(90)
if err != nil {
return nil, nil, err
}
template.RemainderAccount.Balance = remainingFil
if err := createMultisigAccount(ctx, bs, cst, state, builtin.ReserveAddress, template.RemainderAccount, keyIDs); err != nil {
if err := createMultisigAccount(ctx, bs, cst, state, remAccKey, template.RemainderAccount, keyIDs); err != nil {
return nil, nil, xerrors.Errorf("failed to set up remainder account: %w", err)
}
-4
View File
@@ -105,10 +105,6 @@ func checkFault(t ds.Datastore, key ds.Key, bh *types.BlockHeader, faultType str
return err
}
if other == bh.Cid() {
return nil
}
return xerrors.Errorf("produced block would trigger '%s' consensus fault; miner: %s; bh: %s, other: %s", faultType, bh.Miner, bh.Cid(), other)
}
-1
View File
@@ -42,7 +42,6 @@ var Costs = map[CostKey]int64{
{builtin0.StorageMinerActorCodeID, 16}: 5325185,
{builtin0.StorageMinerActorCodeID, 18}: 2328637,
{builtin0.StoragePowerActorCodeID, 2}: 23600956,
// TODO: Just reuse v0 values for now, this isn't actually used
{builtin2.InitActorCodeID, 2}: 8916753,
{builtin2.StorageMarketActorCodeID, 2}: 6955002,
{builtin2.StorageMarketActorCodeID, 4}: 245436108,
+83 -42
View File
@@ -55,7 +55,6 @@ var baseFeeLowerBoundFactor = types.NewInt(10)
var baseFeeLowerBoundFactorConservative = types.NewInt(100)
var MaxActorPendingMessages = 1000
var MaxUntrustedActorPendingMessages = 10
var MaxNonceGap = uint64(4)
@@ -76,6 +75,8 @@ var (
ErrRBFTooLowPremium = errors.New("replace by fee has too low GasPremium")
ErrTooManyPendingMessages = errors.New("too many pending messages for actor")
ErrNonceGap = errors.New("unfulfilled nonce gap")
ErrTryAgain = errors.New("state inconsistency while pushing message; please try again")
)
const (
@@ -196,17 +197,9 @@ func CapGasFee(msg *types.Message, maxFee abi.TokenAmount) {
msg.GasPremium = big.Min(msg.GasFeeCap, msg.GasPremium) // cap premium at FeeCap
}
func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict, untrusted bool) (bool, error) {
func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict bool) (bool, error) {
nextNonce := ms.nextNonce
nonceGap := false
maxNonceGap := MaxNonceGap
maxActorPendingMessages := MaxActorPendingMessages
if untrusted {
maxNonceGap = 0
maxActorPendingMessages = MaxUntrustedActorPendingMessages
}
switch {
case m.Message.Nonce == nextNonce:
nextNonce++
@@ -215,7 +208,7 @@ func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict, untrusted
nextNonce++
}
case strict && m.Message.Nonce > nextNonce+maxNonceGap:
case strict && m.Message.Nonce > nextNonce+MaxNonceGap:
return false, xerrors.Errorf("message nonce has too big a gap from expected nonce (Nonce: %d, nextNonce: %d): %w", m.Message.Nonce, nextNonce, ErrNonceGap)
case m.Message.Nonce > nextNonce:
@@ -251,7 +244,7 @@ func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict, untrusted
//ms.requiredFunds.Sub(ms.requiredFunds, exms.Message.Value.Int)
}
if !has && strict && len(ms.msgs) >= maxActorPendingMessages {
if !has && strict && len(ms.msgs) > MaxActorPendingMessages {
log.Errorf("too many pending messages from actor %s", m.Message.From)
return false, ErrTooManyPendingMessages
}
@@ -493,7 +486,7 @@ func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
}
mp.curTsLk.Lock()
publish, err := mp.addTs(m, mp.curTs, true, false)
publish, err := mp.addTs(m, mp.curTs, true)
if err != nil {
mp.curTsLk.Unlock()
return cid.Undef, err
@@ -560,7 +553,7 @@ func (mp *MessagePool) Add(m *types.SignedMessage) error {
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
_, err = mp.addTs(m, mp.curTs, false, false)
_, err = mp.addTs(m, mp.curTs, false)
return err
}
@@ -628,7 +621,7 @@ func (mp *MessagePool) checkBalance(m *types.SignedMessage, curTs *types.TipSet)
return nil
}
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet, local, untrusted bool) (bool, error) {
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet, local bool) (bool, error) {
snonce, err := mp.getStateNonce(m.Message.From, curTs)
if err != nil {
return false, xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrSoftValidationFailure)
@@ -650,7 +643,7 @@ func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet, local,
return false, err
}
return publish, mp.addLocked(m, !local, untrusted)
return publish, mp.addLocked(m, !local)
}
func (mp *MessagePool) addLoaded(m *types.SignedMessage) error {
@@ -685,17 +678,17 @@ func (mp *MessagePool) addLoaded(m *types.SignedMessage) error {
return err
}
return mp.addLocked(m, false, false)
return mp.addLocked(m, false)
}
func (mp *MessagePool) addSkipChecks(m *types.SignedMessage) error {
mp.lk.Lock()
defer mp.lk.Unlock()
return mp.addLocked(m, false, false)
return mp.addLocked(m, false)
}
func (mp *MessagePool) addLocked(m *types.SignedMessage, strict, untrusted bool) error {
func (mp *MessagePool) addLocked(m *types.SignedMessage, strict bool) error {
log.Debugf("mpooladd: %s %d", m.Message.From, m.Message.Nonce)
if m.Signature.Type == crypto.SigTypeBLS {
mp.blsSigCache.Add(m.Cid(), m.Signature)
@@ -722,7 +715,7 @@ func (mp *MessagePool) addLocked(m *types.SignedMessage, strict, untrusted bool)
mp.pending[m.Message.From] = mset
}
incr, err := mset.add(m, mp, strict, untrusted)
incr, err := mset.add(m, mp, strict)
if err != nil {
log.Debug(err)
return err
@@ -802,48 +795,96 @@ func (mp *MessagePool) getStateBalance(addr address.Address, ts *types.TipSet) (
return act.Balance, nil
}
// this method is provided for the gateway to push messages.
// differences from Push:
// - strict checks are enabled
// - extra strict add checks are used when adding the messages to the msgSet
// that means: no nonce gaps, at most 10 pending messages for the actor
func (mp *MessagePool) PushUntrusted(m *types.SignedMessage) (cid.Cid, error) {
err := mp.checkMessage(m)
if err != nil {
return cid.Undef, err
}
func (mp *MessagePool) PushWithNonce(ctx context.Context, addr address.Address, cb func(address.Address, uint64) (*types.SignedMessage, error)) (*types.SignedMessage, error) {
// serialize push access to reduce lock contention
mp.addSema <- struct{}{}
defer func() {
<-mp.addSema
}()
msgb, err := m.Serialize()
if err != nil {
return cid.Undef, err
mp.curTsLk.Lock()
mp.lk.Lock()
curTs := mp.curTs
fromKey := addr
if fromKey.Protocol() == address.ID {
var err error
fromKey, err = mp.api.StateAccountKey(ctx, fromKey, mp.curTs)
if err != nil {
mp.lk.Unlock()
mp.curTsLk.Unlock()
return nil, xerrors.Errorf("resolving sender key: %w", err)
}
}
mp.curTsLk.Lock()
publish, err := mp.addTs(m, mp.curTs, false, true)
nonce, err := mp.getNonceLocked(fromKey, mp.curTs)
if err != nil {
mp.lk.Unlock()
mp.curTsLk.Unlock()
return cid.Undef, err
return nil, xerrors.Errorf("get nonce locked failed: %w", err)
}
// release the locks for signing
mp.lk.Unlock()
mp.curTsLk.Unlock()
msg, err := cb(fromKey, nonce)
if err != nil {
return nil, err
}
err = mp.checkMessage(msg)
if err != nil {
return nil, err
}
msgb, err := msg.Serialize()
if err != nil {
return nil, err
}
// reacquire the locks and check state for consistency
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
if mp.curTs != curTs {
return nil, ErrTryAgain
}
mp.lk.Lock()
if err := mp.addLocal(m, msgb); err != nil {
mp.lk.Unlock()
return cid.Undef, err
defer mp.lk.Unlock()
nonce2, err := mp.getNonceLocked(fromKey, mp.curTs)
if err != nil {
return nil, xerrors.Errorf("get nonce locked failed: %w", err)
}
if nonce2 != nonce {
return nil, ErrTryAgain
}
publish, err := mp.verifyMsgBeforeAdd(msg, curTs, true)
if err != nil {
return nil, err
}
if err := mp.checkBalance(msg, curTs); err != nil {
return nil, err
}
if err := mp.addLocked(msg, false); err != nil {
return nil, xerrors.Errorf("add locked failed: %w", err)
}
if err := mp.addLocal(msg, msgb); err != nil {
log.Errorf("addLocal failed: %+v", err)
}
mp.lk.Unlock()
if publish {
err = mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
}
return m.Cid(), err
return msg, err
}
func (mp *MessagePool) Remove(from address.Address, nonce uint64, applied bool) {
+3 -3
View File
@@ -149,13 +149,13 @@ loop:
if len(msgs) > 0 {
journal.J.RecordEvent(mp.evtTypes[evtTypeMpoolRepub], func() interface{} {
msgsEv := make([]MessagePoolEvtMessage, 0, len(msgs))
msgs := make([]MessagePoolEvtMessage, 0, len(msgs))
for _, m := range msgs {
msgsEv = append(msgsEv, MessagePoolEvtMessage{Message: m.Message, CID: m.Cid()})
msgs = append(msgs, MessagePoolEvtMessage{Message: m.Message, CID: m.Cid()})
}
return MessagePoolEvt{
Action: "repub",
Messages: msgsEv,
Messages: msgs,
}
})
}
-155
View File
@@ -1,155 +0,0 @@
package messagesigner
import (
"bytes"
"context"
"sync"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/messagepool"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/namespace"
logging "github.com/ipfs/go-log/v2"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
)
const dsKeyActorNonce = "ActorNextNonce"
var log = logging.Logger("messagesigner")
type mpoolAPI interface {
GetNonce(address.Address) (uint64, error)
}
// MessageSigner keeps track of nonces per address, and increments the nonce
// when signing a message
type MessageSigner struct {
wallet *wallet.Wallet
lk sync.Mutex
mpool mpoolAPI
ds datastore.Batching
}
func NewMessageSigner(wallet *wallet.Wallet, mpool *messagepool.MessagePool, ds dtypes.MetadataDS) *MessageSigner {
return newMessageSigner(wallet, mpool, ds)
}
func newMessageSigner(wallet *wallet.Wallet, mpool mpoolAPI, ds dtypes.MetadataDS) *MessageSigner {
ds = namespace.Wrap(ds, datastore.NewKey("/message-signer/"))
return &MessageSigner{
wallet: wallet,
mpool: mpool,
ds: ds,
}
}
// SignMessage increments the nonce for the message From address, and signs
// the message
func (ms *MessageSigner) SignMessage(ctx context.Context, msg *types.Message, cb func(*types.SignedMessage) error) (*types.SignedMessage, error) {
ms.lk.Lock()
defer ms.lk.Unlock()
// Get the next message nonce
nonce, err := ms.nextNonce(msg.From)
if err != nil {
return nil, xerrors.Errorf("failed to create nonce: %w", err)
}
// Sign the message with the nonce
msg.Nonce = nonce
sig, err := ms.wallet.Sign(ctx, msg.From, msg.Cid().Bytes())
if err != nil {
return nil, xerrors.Errorf("failed to sign message: %w", err)
}
// Callback with the signed message
smsg := &types.SignedMessage{
Message: *msg,
Signature: *sig,
}
err = cb(smsg)
if err != nil {
return nil, err
}
// If the callback executed successfully, write the nonce to the datastore
if err := ms.saveNonce(msg.From, nonce); err != nil {
return nil, xerrors.Errorf("failed to save nonce: %w", err)
}
return smsg, nil
}
// nextNonce gets the next nonce for the given address.
// If there is no nonce in the datastore, gets the nonce from the message pool.
func (ms *MessageSigner) nextNonce(addr address.Address) (uint64, error) {
// Nonces used to be created by the mempool and we need to support nodes
// that have mempool nonces, so first check the mempool for a nonce for
// this address. Note that the mempool returns the actor state's nonce
// by default.
nonce, err := ms.mpool.GetNonce(addr)
if err != nil {
return 0, xerrors.Errorf("failed to get nonce from mempool: %w", err)
}
// Get the next nonce for this address from the datastore
addrNonceKey := ms.dstoreKey(addr)
dsNonceBytes, err := ms.ds.Get(addrNonceKey)
switch {
case xerrors.Is(err, datastore.ErrNotFound):
// If a nonce for this address hasn't yet been created in the
// datastore, just use the nonce from the mempool
return nonce, nil
case err != nil:
return 0, xerrors.Errorf("failed to get nonce from datastore: %w", err)
default:
// There is a nonce in the datastore, so unmarshall it
maj, dsNonce, err := cbg.CborReadHeader(bytes.NewReader(dsNonceBytes))
if err != nil {
return 0, xerrors.Errorf("failed to parse nonce from datastore: %w", err)
}
if maj != cbg.MajUnsignedInt {
return 0, xerrors.Errorf("bad cbor type parsing nonce from datastore")
}
// The message pool nonce should be <= than the datastore nonce
if nonce <= dsNonce {
nonce = dsNonce
} else {
log.Warnf("mempool nonce was larger than datastore nonce (%d > %d)", nonce, dsNonce)
}
return nonce, nil
}
}
// saveNonce increments the nonce for this address and writes it to the
// datastore
func (ms *MessageSigner) saveNonce(addr address.Address, nonce uint64) error {
// Increment the nonce
nonce++
// Write the nonce to the datastore
addrNonceKey := ms.dstoreKey(addr)
buf := bytes.Buffer{}
_, err := buf.Write(cbg.CborEncodeMajorType(cbg.MajUnsignedInt, nonce))
if err != nil {
return xerrors.Errorf("failed to marshall nonce: %w", err)
}
err = ms.ds.Put(addrNonceKey, buf.Bytes())
if err != nil {
return xerrors.Errorf("failed to write nonce to datastore: %w", err)
}
return nil
}
func (ms *MessageSigner) dstoreKey(addr address.Address) datastore.Key {
return datastore.KeyWithNamespaces([]string{dsKeyActorNonce, addr.String()})
}
-201
View File
@@ -1,201 +0,0 @@
package messagesigner
import (
"context"
"sync"
"testing"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/stretchr/testify/require"
ds_sync "github.com/ipfs/go-datastore/sync"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"
"github.com/ipfs/go-datastore"
)
type mockMpool struct {
lk sync.RWMutex
nonces map[address.Address]uint64
}
func newMockMpool() *mockMpool {
return &mockMpool{nonces: make(map[address.Address]uint64)}
}
func (mp *mockMpool) setNonce(addr address.Address, nonce uint64) {
mp.lk.Lock()
defer mp.lk.Unlock()
mp.nonces[addr] = nonce
}
func (mp *mockMpool) GetNonce(addr address.Address) (uint64, error) {
mp.lk.RLock()
defer mp.lk.RUnlock()
return mp.nonces[addr], nil
}
func TestMessageSignerSignMessage(t *testing.T) {
ctx := context.Background()
w, _ := wallet.NewWallet(wallet.NewMemKeyStore())
from1, err := w.GenerateKey(crypto.SigTypeSecp256k1)
require.NoError(t, err)
from2, err := w.GenerateKey(crypto.SigTypeSecp256k1)
require.NoError(t, err)
to1, err := w.GenerateKey(crypto.SigTypeSecp256k1)
require.NoError(t, err)
to2, err := w.GenerateKey(crypto.SigTypeSecp256k1)
require.NoError(t, err)
type msgSpec struct {
msg *types.Message
mpoolNonce [1]uint64
expNonce uint64
cbErr error
}
tests := []struct {
name string
msgs []msgSpec
}{{
// No nonce yet in datastore
name: "no nonce yet",
msgs: []msgSpec{{
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 0,
}},
}, {
// Get nonce value of zero from mpool
name: "mpool nonce zero",
msgs: []msgSpec{{
msg: &types.Message{
To: to1,
From: from1,
},
mpoolNonce: [1]uint64{0},
expNonce: 0,
}},
}, {
// Get non-zero nonce value from mpool
name: "mpool nonce set",
msgs: []msgSpec{{
msg: &types.Message{
To: to1,
From: from1,
},
mpoolNonce: [1]uint64{5},
expNonce: 5,
}, {
msg: &types.Message{
To: to1,
From: from1,
},
// Should adjust datastore nonce because mpool nonce is higher
mpoolNonce: [1]uint64{10},
expNonce: 10,
}},
}, {
// Nonce should increment independently for each address
name: "nonce increments per address",
msgs: []msgSpec{{
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 0,
}, {
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 1,
}, {
msg: &types.Message{
To: to2,
From: from2,
},
mpoolNonce: [1]uint64{5},
expNonce: 5,
}, {
msg: &types.Message{
To: to2,
From: from2,
},
expNonce: 6,
}, {
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 2,
}},
}, {
name: "recover from callback error",
msgs: []msgSpec{{
// No nonce yet in datastore
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 0,
}, {
// Increment nonce
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 1,
}, {
// Callback returns error
msg: &types.Message{
To: to1,
From: from1,
},
cbErr: xerrors.Errorf("err"),
}, {
// Callback successful, should increment nonce in datastore
msg: &types.Message{
To: to1,
From: from1,
},
expNonce: 2,
}},
}}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
mpool := newMockMpool()
ds := ds_sync.MutexWrap(datastore.NewMapDatastore())
ms := newMessageSigner(w, mpool, ds)
for _, m := range tt.msgs {
if len(m.mpoolNonce) == 1 {
mpool.setNonce(m.msg.From, m.mpoolNonce[0])
}
merr := m.cbErr
smsg, err := ms.SignMessage(ctx, m.msg, func(message *types.SignedMessage) error {
return merr
})
if m.cbErr != nil {
require.Error(t, err)
require.Nil(t, smsg)
} else {
require.NoError(t, err)
require.Equal(t, m.expNonce, smsg.Message.Nonce)
}
}
})
}
}
+16 -40
View File
@@ -13,7 +13,6 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
cbg "github.com/whyrusleeping/cbor-gen"
@@ -27,7 +26,7 @@ var log = logging.Logger("statetree")
// StateTree stores actors state by their ID.
type StateTree struct {
root adt.Map
version types.StateTreeVersion
version actors.Version // TODO
info cid.Cid
Store cbor.IpldStore
@@ -121,41 +120,21 @@ func (ss *stateSnaps) deleteActor(addr address.Address) {
ss.layers[len(ss.layers)-1].actors[addr] = streeOp{Delete: true}
}
// VersionForNetwork returns the state tree version for the given network
// version.
func VersionForNetwork(ver network.Version) types.StateTreeVersion {
if actors.VersionForNetwork(ver) == actors.Version0 {
return types.StateTreeVersion0
}
return types.StateTreeVersion1
}
func adtForSTVersion(ver types.StateTreeVersion) actors.Version {
switch ver {
case types.StateTreeVersion0:
return actors.Version0
case types.StateTreeVersion1:
return actors.Version2
default:
panic("unhandled state tree version")
}
}
func NewStateTree(cst cbor.IpldStore, ver types.StateTreeVersion) (*StateTree, error) {
func NewStateTree(cst cbor.IpldStore, version actors.Version) (*StateTree, error) {
var info cid.Cid
switch ver {
case types.StateTreeVersion0:
switch version {
case actors.Version0:
// info is undefined
case types.StateTreeVersion1:
case actors.Version2:
var err error
info, err = cst.Put(context.TODO(), new(types.StateInfo0))
info, err = cst.Put(context.TODO(), new(types.StateInfo))
if err != nil {
return nil, err
}
default:
return nil, xerrors.Errorf("unsupported state tree version: %d", ver)
return nil, xerrors.Errorf("unsupported state tree version: %d", version)
}
root, err := adt.NewMap(adt.WrapStore(context.TODO(), cst), adtForSTVersion(ver))
root, err := adt.NewMap(adt.WrapStore(context.TODO(), cst), version)
if err != nil {
return nil, err
}
@@ -163,7 +142,7 @@ func NewStateTree(cst cbor.IpldStore, ver types.StateTreeVersion) (*StateTree, e
return &StateTree{
root: root,
info: info,
version: ver,
version: version,
Store: cst,
snaps: newStateSnaps(),
}, nil
@@ -175,16 +154,13 @@ func LoadStateTree(cst cbor.IpldStore, c cid.Cid) (*StateTree, error) {
if err := cst.Get(context.TODO(), c, &root); err != nil {
// We failed to decode as the new version, must be an old version.
root.Actors = c
root.Version = types.StateTreeVersion0
root.Version = actors.Version0
}
switch root.Version {
case types.StateTreeVersion0, types.StateTreeVersion1:
case actors.Version0, actors.Version2:
// Load the actual state-tree HAMT.
nd, err := adt.AsMap(
adt.WrapStore(context.TODO(), cst), root.Actors,
adtForSTVersion(root.Version),
)
nd, err := adt.AsMap(adt.WrapStore(context.TODO(), cst), root.Actors, actors.Version(root.Version))
if err != nil {
log.Errorf("loading hamt node %s failed: %s", c, err)
return nil, err
@@ -193,7 +169,7 @@ func LoadStateTree(cst cbor.IpldStore, c cid.Cid) (*StateTree, error) {
return &StateTree{
root: nd,
info: root.Info,
version: root.Version,
version: actors.Version(root.Version),
Store: cst,
snaps: newStateSnaps(),
}, nil
@@ -333,11 +309,11 @@ func (st *StateTree) Flush(ctx context.Context) (cid.Cid, error) {
return cid.Undef, xerrors.Errorf("failed to flush state-tree hamt: %w", err)
}
// If we're version 0, return a raw tree.
if st.version == types.StateTreeVersion0 {
if st.version == actors.Version0 {
return root, nil
}
// Otherwise, return a versioned tree.
return st.Store.Put(ctx, &types.StateRoot{Version: st.version, Actors: root, Info: st.info})
return st.Store.Put(ctx, &types.StateRoot{Version: uint64(st.version), Actors: root, Info: st.info})
}
func (st *StateTree) Snapshot(ctx context.Context) error {
@@ -424,7 +400,7 @@ func (st *StateTree) ForEach(f func(address.Address, *types.Actor) error) error
}
// Version returns the version of the StateTree data structure in use.
func (st *StateTree) Version() types.StateTreeVersion {
func (st *StateTree) Version() actors.Version {
return st.version
}
+7 -6
View File
@@ -13,12 +13,13 @@ import (
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/types"
)
func BenchmarkStateTreeSet(b *testing.B) {
cst := cbor.NewMemCborStore()
st, err := NewStateTree(cst, VersionForNetwork(build.NewestNetworkVersion))
st, err := NewStateTree(cst, actors.VersionForNetwork(build.NewestNetworkVersion))
if err != nil {
b.Fatal(err)
}
@@ -45,7 +46,7 @@ func BenchmarkStateTreeSet(b *testing.B) {
func BenchmarkStateTreeSetFlush(b *testing.B) {
cst := cbor.NewMemCborStore()
st, err := NewStateTree(cst, VersionForNetwork(build.NewestNetworkVersion))
st, err := NewStateTree(cst, actors.VersionForNetwork(build.NewestNetworkVersion))
if err != nil {
b.Fatal(err)
}
@@ -75,7 +76,7 @@ func BenchmarkStateTreeSetFlush(b *testing.B) {
func BenchmarkStateTree10kGetActor(b *testing.B) {
cst := cbor.NewMemCborStore()
st, err := NewStateTree(cst, VersionForNetwork(build.NewestNetworkVersion))
st, err := NewStateTree(cst, actors.VersionForNetwork(build.NewestNetworkVersion))
if err != nil {
b.Fatal(err)
}
@@ -117,7 +118,7 @@ func BenchmarkStateTree10kGetActor(b *testing.B) {
func TestSetCache(t *testing.T) {
cst := cbor.NewMemCborStore()
st, err := NewStateTree(cst, VersionForNetwork(build.NewestNetworkVersion))
st, err := NewStateTree(cst, actors.VersionForNetwork(build.NewestNetworkVersion))
if err != nil {
t.Fatal(err)
}
@@ -154,7 +155,7 @@ func TestSetCache(t *testing.T) {
func TestSnapshots(t *testing.T) {
ctx := context.Background()
cst := cbor.NewMemCborStore()
st, err := NewStateTree(cst, VersionForNetwork(build.NewestNetworkVersion))
st, err := NewStateTree(cst, actors.VersionForNetwork(build.NewestNetworkVersion))
if err != nil {
t.Fatal(err)
}
@@ -238,7 +239,7 @@ func assertNotHas(t *testing.T, st *StateTree, addr address.Address) {
func TestStateTreeConsistency(t *testing.T) {
cst := cbor.NewMemCborStore()
// TODO: ActorUpgrade: this test tests pre actors v2
st, err := NewStateTree(cst, VersionForNetwork(network.Version3))
st, err := NewStateTree(cst, actors.VersionForNetwork(network.Version3))
if err != nil {
t.Fatal(err)
}
+18 -62
View File
@@ -2,10 +2,10 @@ package stmgr
import (
"context"
"errors"
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/ipfs/go-cid"
"go.opencensus.io/trace"
@@ -18,47 +18,14 @@ import (
"github.com/filecoin-project/lotus/chain/vm"
)
var ErrExpensiveFork = errors.New("refusing explicit call due to state fork at epoch")
func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.TipSet) (*api.InvocResult, error) {
ctx, span := trace.StartSpan(ctx, "statemanager.Call")
func (sm *StateManager) CallRaw(ctx context.Context, msg *types.Message, bstate cid.Cid, r vm.Rand, bheight abi.ChainEpoch) (*api.InvocResult, error) {
ctx, span := trace.StartSpan(ctx, "statemanager.CallRaw")
defer span.End()
// If no tipset is provided, try to find one without a fork.
if ts == nil {
ts = sm.cs.GetHeaviestTipSet()
// Search back till we find a height with no fork, or we reach the beginning.
for ts.Height() > 0 && sm.hasExpensiveFork(ctx, ts.Height()-1) {
var err error
ts, err = sm.cs.GetTipSetFromKey(ts.Parents())
if err != nil {
return nil, xerrors.Errorf("failed to find a non-forking epoch: %w", err)
}
}
}
bstate := ts.ParentState()
bheight := ts.Height()
// If we have to run an expensive migration, and we're not at genesis,
// return an error because the migration will take too long.
//
// We allow this at height 0 for at-genesis migrations (for testing).
if bheight-1 > 0 && sm.hasExpensiveFork(ctx, bheight-1) {
return nil, ErrExpensiveFork
}
// Run the (not expensive) migration.
bstate, err := sm.handleStateForks(ctx, bstate, bheight-1, nil, ts)
if err != nil {
return nil, fmt.Errorf("failed to handle fork: %w", err)
}
vmopt := &vm.VMOpts{
StateBase: bstate,
Epoch: bheight,
Rand: store.NewChainRand(sm.cs, ts.Cids()),
Rand: r,
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
@@ -66,7 +33,7 @@ func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.
BaseFee: types.NewInt(0),
}
vmi, err := sm.newVM(ctx, vmopt)
vmi, err := vm.NewVM(ctx, vmopt)
if err != nil {
return nil, xerrors.Errorf("failed to set up vm: %w", err)
}
@@ -122,30 +89,24 @@ func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.
}
func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.TipSet) (*api.InvocResult, error) {
if ts == nil {
ts = sm.cs.GetHeaviestTipSet()
}
state := ts.ParentState()
r := store.NewChainRand(sm.cs, ts.Cids())
return sm.CallRaw(ctx, msg, state, r, ts.Height())
}
func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, priorMsgs []types.ChainMsg, ts *types.TipSet) (*api.InvocResult, error) {
ctx, span := trace.StartSpan(ctx, "statemanager.CallWithGas")
defer span.End()
if ts == nil {
ts = sm.cs.GetHeaviestTipSet()
// Search back till we find a height with no fork, or we reach the beginning.
// We need the _previous_ height to have no fork, because we'll
// run the fork logic in `sm.TipSetState`. We need the _current_
// height to have no fork, because we'll run it inside this
// function before executing the given message.
for ts.Height() > 0 && (sm.hasExpensiveFork(ctx, ts.Height()) || sm.hasExpensiveFork(ctx, ts.Height()-1)) {
var err error
ts, err = sm.cs.GetTipSetFromKey(ts.Parents())
if err != nil {
return nil, xerrors.Errorf("failed to find a non-forking epoch: %w", err)
}
}
}
// When we're not at the genesis block, make sure we don't have an expensive migration.
if ts.Height() > 0 && (sm.hasExpensiveFork(ctx, ts.Height()) || sm.hasExpensiveFork(ctx, ts.Height()-1)) {
return nil, ErrExpensiveFork
}
state, _, err := sm.TipSetState(ctx, ts)
@@ -153,11 +114,6 @@ func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, pri
return nil, xerrors.Errorf("computing tipset state: %w", err)
}
state, err = sm.handleStateForks(ctx, state, ts.Height(), nil, ts)
if err != nil {
return nil, fmt.Errorf("failed to handle fork: %w", err)
}
r := store.NewChainRand(sm.cs, ts.Cids())
if span.IsRecordingEvents() {
@@ -178,7 +134,7 @@ func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, pri
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := sm.newVM(ctx, vmopt)
vmi, err := vm.NewVM(ctx, vmopt)
if err != nil {
return nil, xerrors.Errorf("failed to set up vm: %w", err)
}
+21 -209
View File
@@ -6,12 +6,9 @@ import (
"encoding/binary"
"math"
"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/network"
"github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
@@ -27,132 +24,28 @@ import (
states2 "github.com/filecoin-project/specs-actors/v2/actors/states"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
bstore "github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/lotus/lib/bufbstore"
)
// UpgradeFunc is a migration function run at every upgrade.
//
// - The oldState is the state produced by the upgrade epoch.
// - The returned newState is the new state that will be used by the next epoch.
// - The height is the upgrade epoch height (already executed).
// - The tipset is the tipset for the last non-null block before the upgrade. Do
// not assume that ts.Height() is the upgrade height.
type UpgradeFunc func(ctx context.Context, sm *StateManager, cb ExecCallback, oldState cid.Cid, height abi.ChainEpoch, ts *types.TipSet) (newState cid.Cid, err error)
type Upgrade struct {
Height abi.ChainEpoch
Network network.Version
Expensive bool
Migration UpgradeFunc
}
type UpgradeSchedule []Upgrade
func DefaultUpgradeSchedule() UpgradeSchedule {
var us UpgradeSchedule
updates := []Upgrade{{
Height: build.UpgradeBreezeHeight,
Network: network.Version1,
Migration: UpgradeFaucetBurnRecovery,
}, {
Height: build.UpgradeSmokeHeight,
Network: network.Version2,
Migration: nil,
}, {
Height: build.UpgradeIgnitionHeight,
Network: network.Version3,
Migration: UpgradeIgnition,
}, {
Height: build.UpgradeRefuelHeight,
Network: network.Version3,
Migration: UpgradeRefuel,
}, {
Height: build.UpgradeActorsV2Height,
Network: network.Version4,
Expensive: true,
Migration: UpgradeActorsV2,
}, {
Height: build.UpgradeLiftoffHeight,
Network: network.Version4,
Migration: UpgradeLiftoff,
}}
if build.UpgradeActorsV2Height == math.MaxInt64 { // disable actors upgrade
updates = []Upgrade{{
Height: build.UpgradeBreezeHeight,
Network: network.Version1,
Migration: UpgradeFaucetBurnRecovery,
}, {
Height: build.UpgradeSmokeHeight,
Network: network.Version2,
Migration: nil,
}, {
Height: build.UpgradeIgnitionHeight,
Network: network.Version3,
Migration: UpgradeIgnition,
}, {
Height: build.UpgradeRefuelHeight,
Network: network.Version3,
Migration: UpgradeRefuel,
}, {
Height: build.UpgradeLiftoffHeight,
Network: network.Version3,
Migration: UpgradeLiftoff,
}}
}
for _, u := range updates {
if u.Height < 0 {
// upgrade disabled
continue
}
us = append(us, u)
}
return us
}
func (us UpgradeSchedule) Validate() error {
// Make sure we're not trying to upgrade to version 0.
for _, u := range us {
if u.Network <= 0 {
return xerrors.Errorf("cannot upgrade to version <= 0: %d", u.Network)
}
}
// Make sure all the upgrades make sense.
for i := 1; i < len(us); i++ {
prev := &us[i-1]
curr := &us[i]
if !(prev.Network <= curr.Network) {
return xerrors.Errorf("cannot downgrade from version %d to version %d", prev.Network, curr.Network)
}
// Make sure the heights make sense.
if prev.Height < 0 {
// Previous upgrade was disabled.
continue
}
if !(prev.Height < curr.Height) {
return xerrors.Errorf("upgrade heights must be strictly increasing: upgrade %d was at height %d, followed by upgrade %d at height %d", i-1, prev.Height, i, curr.Height)
}
}
return nil
var ForksAtHeight = map[abi.ChainEpoch]func(context.Context, *StateManager, ExecCallback, cid.Cid, *types.TipSet) (cid.Cid, error){
build.UpgradeBreezeHeight: UpgradeFaucetBurnRecovery,
build.UpgradeIgnitionHeight: UpgradeIgnition,
build.UpgradeActorsV2: UpgradeActorsV2,
build.UpgradeLiftoffHeight: UpgradeLiftoff,
}
func (sm *StateManager) handleStateForks(ctx context.Context, root cid.Cid, height abi.ChainEpoch, cb ExecCallback, ts *types.TipSet) (cid.Cid, error) {
retCid := root
var err error
f, ok := sm.stateMigrations[height]
f, ok := ForksAtHeight[height]
if ok {
retCid, err = f(ctx, sm, cb, root, height, ts)
retCid, err = f(ctx, sm, cb, root, ts)
if err != nil {
return cid.Undef, err
}
@@ -161,11 +54,6 @@ func (sm *StateManager) handleStateForks(ctx context.Context, root cid.Cid, heig
return retCid, nil
}
func (sm *StateManager) hasExpensiveFork(ctx context.Context, height abi.ChainEpoch) bool {
_, ok := sm.expensiveUpgrades[height]
return ok
}
func doTransfer(cb ExecCallback, tree types.StateTree, from, to address.Address, amt abi.TokenAmount) error {
fromAct, err := tree.GetActor(from)
if err != nil {
@@ -228,7 +116,7 @@ func doTransfer(cb ExecCallback, tree types.StateTree, from, to address.Address,
return nil
}
func UpgradeFaucetBurnRecovery(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
func UpgradeFaucetBurnRecovery(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, ts *types.TipSet) (cid.Cid, error) {
// Some initial parameters
FundsForMiners := types.FromFil(1_000_000)
LookbackEpoch := abi.ChainEpoch(32000)
@@ -480,14 +368,10 @@ func UpgradeFaucetBurnRecovery(ctx context.Context, sm *StateManager, cb ExecCal
return tree.Flush(ctx)
}
func UpgradeIgnition(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
func UpgradeIgnition(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, ts *types.TipSet) (cid.Cid, error) {
store := sm.cs.Store(ctx)
if build.UpgradeLiftoffHeight <= epoch {
return cid.Undef, xerrors.Errorf("liftoff height must be beyond ignition height")
}
nst, err := nv3.MigrateStateTree(ctx, store, root, epoch)
nst, err := nv3.MigrateStateTree(ctx, store, root, build.UpgradeIgnitionHeight)
if err != nil {
return cid.Undef, xerrors.Errorf("migrating actors state: %w", err)
}
@@ -512,7 +396,7 @@ func UpgradeIgnition(ctx context.Context, sm *StateManager, cb ExecCallback, roo
return cid.Undef, xerrors.Errorf("second split address: %w", err)
}
err = resetGenesisMsigs(ctx, sm, store, tree, build.UpgradeLiftoffHeight)
err = resetGenesisMsigs(ctx, sm, store, tree)
if err != nil {
return cid.Undef, xerrors.Errorf("resetting genesis msig start epochs: %w", err)
}
@@ -527,7 +411,7 @@ func UpgradeIgnition(ctx context.Context, sm *StateManager, cb ExecCallback, roo
return cid.Undef, xerrors.Errorf("splitting second msig: %w", err)
}
err = nv3.CheckStateTree(ctx, store, nst, epoch, builtin0.TotalFilecoin)
err = nv3.CheckStateTree(ctx, store, nst, build.UpgradeIgnitionHeight, builtin0.TotalFilecoin)
if err != nil {
return cid.Undef, xerrors.Errorf("sanity check after ignition upgrade failed: %w", err)
}
@@ -535,47 +419,15 @@ func UpgradeIgnition(ctx context.Context, sm *StateManager, cb ExecCallback, roo
return tree.Flush(ctx)
}
func UpgradeRefuel(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
func UpgradeActorsV2(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, ts *types.TipSet) (cid.Cid, error) {
store := sm.cs.Store(ctx)
tree, err := sm.StateTree(root)
if err != nil {
return cid.Undef, xerrors.Errorf("getting state tree: %w", err)
}
addr, err := address.NewFromString("t0122")
if err != nil {
return cid.Undef, xerrors.Errorf("getting address: %w", err)
}
err = resetMultisigVesting(ctx, store, tree, addr, 0, 0, big.Zero())
if err != nil {
return cid.Undef, xerrors.Errorf("tweaking msig vesting: %w", err)
}
err = resetMultisigVesting(ctx, store, tree, builtin.ReserveAddress, 0, 0, big.Zero())
if err != nil {
return cid.Undef, xerrors.Errorf("tweaking msig vesting: %w", err)
}
err = resetMultisigVesting(ctx, store, tree, builtin.RootVerifierAddress, 0, 0, big.Zero())
if err != nil {
return cid.Undef, xerrors.Errorf("tweaking msig vesting: %w", err)
}
return tree.Flush(ctx)
}
func UpgradeActorsV2(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
buf := bufbstore.NewTieredBstore(sm.cs.Blockstore(), bstore.NewTemporarySync())
store := store.ActorStore(ctx, buf)
info, err := store.Put(ctx, new(types.StateInfo0))
info, err := store.Put(ctx, new(types.StateInfo))
if err != nil {
return cid.Undef, xerrors.Errorf("failed to create new state info for actors v2: %w", err)
}
newHamtRoot, err := m2.MigrateStateTree(ctx, store, root, epoch, m2.DefaultConfig())
newHamtRoot, err := m2.MigrateStateTree(ctx, store, root)
if err != nil {
return cid.Undef, xerrors.Errorf("upgrading to actors v2: %w", err)
}
@@ -594,7 +446,8 @@ func UpgradeActorsV2(ctx context.Context, sm *StateManager, cb ExecCallback, roo
}
newRoot, err := store.Put(ctx, &types.StateRoot{
Version: types.StateTreeVersion1,
// TODO: ActorUpgrade: should be state-tree specific, not just the actors version.
Version: actors.Version2,
Actors: newHamtRoot,
Info: info,
})
@@ -613,19 +466,10 @@ func UpgradeActorsV2(ctx context.Context, sm *StateManager, cb ExecCallback, roo
return cid.Undef, xerrors.Errorf("failed to load init actor after upgrade: %w", err)
}
{
from := buf
to := buf.Read()
if err := vm.Copy(ctx, from, to, newRoot); err != nil {
return cid.Undef, xerrors.Errorf("copying migrated tree: %w", err)
}
}
return newRoot, nil
}
func UpgradeLiftoff(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
func UpgradeLiftoff(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, ts *types.TipSet) (cid.Cid, error) {
tree, err := sm.StateTree(root)
if err != nil {
return cid.Undef, xerrors.Errorf("getting state tree: %w", err)
@@ -783,8 +627,7 @@ func makeKeyAddr(splitAddr address.Address, count uint64) (address.Address, erro
return addr, nil
}
// TODO: After the Liftoff epoch, refactor this to use resetMultisigVesting
func resetGenesisMsigs(ctx context.Context, sm *StateManager, store adt0.Store, tree *state.StateTree, startEpoch abi.ChainEpoch) error {
func resetGenesisMsigs(ctx context.Context, sm *StateManager, store adt0.Store, tree *state.StateTree) error {
gb, err := sm.cs.GetGenesis()
if err != nil {
return xerrors.Errorf("getting genesis block: %w", err)
@@ -813,7 +656,7 @@ func resetGenesisMsigs(ctx context.Context, sm *StateManager, store adt0.Store,
return xerrors.Errorf("reading multisig state: %w", err)
}
currState.StartEpoch = startEpoch
currState.StartEpoch = build.UpgradeLiftoffHeight
currActor.Head, err = store.Put(ctx, &currState)
if err != nil {
@@ -833,34 +676,3 @@ func resetGenesisMsigs(ctx context.Context, sm *StateManager, store adt0.Store,
return nil
}
func resetMultisigVesting(ctx context.Context, store adt0.Store, tree *state.StateTree, addr address.Address, startEpoch abi.ChainEpoch, duration abi.ChainEpoch, balance abi.TokenAmount) error {
act, err := tree.GetActor(addr)
if err != nil {
return xerrors.Errorf("getting actor: %w", err)
}
if !builtin.IsMultisigActor(act.Code) {
return xerrors.Errorf("actor wasn't msig: %w", err)
}
var msigState multisig0.State
if err := store.Get(ctx, act.Head, &msigState); err != nil {
return xerrors.Errorf("reading multisig state: %w", err)
}
msigState.StartEpoch = startEpoch
msigState.UnlockDuration = duration
msigState.InitialBalance = balance
act.Head, err = store.Put(ctx, &msigState)
if err != nil {
return xerrors.Errorf("writing new multisig state: %w", err)
}
if err := tree.SetActor(addr, act); err != nil {
return xerrors.Errorf("setting multisig actor: %w", err)
}
return nil
}
+32 -119
View File
@@ -10,9 +10,8 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/cbor"
"github.com/filecoin-project/specs-actors/actors/builtin"
init0 "github.com/filecoin-project/specs-actors/actors/builtin/init"
init_ "github.com/filecoin-project/specs-actors/actors/builtin/init"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors"
@@ -20,6 +19,7 @@ import (
lotusinit "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/actors/policy"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/stmgr"
. "github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
@@ -77,7 +77,7 @@ func (ta testActor) Exports() []interface{} {
func (ta *testActor) Constructor(rt runtime.Runtime, params *abi.EmptyValue) *abi.EmptyValue {
rt.ValidateImmediateCallerAcceptAny()
rt.StateCreate(&testActorState{11})
//fmt.Println("NEW ACTOR ADDRESS IS: ", rt.Receiver())
fmt.Println("NEW ACTOR ADDRESS IS: ", rt.Receiver())
return abi.Empty
}
@@ -110,57 +110,51 @@ func TestForkHeightTriggers(t *testing.T) {
t.Fatal(err)
}
sm := NewStateManager(cg.ChainStore())
inv := vm.NewActorRegistry()
// predicting the address here... may break if other assumptions change
taddr, err := address.NewIDAddress(1002)
if err != nil {
t.Fatal(err)
}
sm, err := NewStateManagerWithUpgradeSchedule(
cg.ChainStore(), UpgradeSchedule{{
Network: 1,
Height: testForkHeight,
Migration: func(ctx context.Context, sm *StateManager, cb ExecCallback,
root cid.Cid, height abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
cst := ipldcbor.NewCborStore(sm.ChainStore().Blockstore())
stmgr.ForksAtHeight[testForkHeight] = func(ctx context.Context, sm *StateManager, cb ExecCallback, root cid.Cid, ts *types.TipSet) (cid.Cid, error) {
cst := ipldcbor.NewCborStore(sm.ChainStore().Blockstore())
st, err := sm.StateTree(root)
if err != nil {
return cid.Undef, xerrors.Errorf("getting state tree: %w", err)
}
st, err := sm.StateTree(root)
if err != nil {
return cid.Undef, xerrors.Errorf("getting state tree: %w", err)
}
act, err := st.GetActor(taddr)
if err != nil {
return cid.Undef, err
}
act, err := st.GetActor(taddr)
if err != nil {
return cid.Undef, err
}
var tas testActorState
if err := cst.Get(ctx, act.Head, &tas); err != nil {
return cid.Undef, xerrors.Errorf("in fork handler, failed to run get: %w", err)
}
var tas testActorState
if err := cst.Get(ctx, act.Head, &tas); err != nil {
return cid.Undef, xerrors.Errorf("in fork handler, failed to run get: %w", err)
}
tas.HasUpgraded = 55
tas.HasUpgraded = 55
ns, err := cst.Put(ctx, &tas)
if err != nil {
return cid.Undef, err
}
ns, err := cst.Put(ctx, &tas)
if err != nil {
return cid.Undef, err
}
act.Head = ns
act.Head = ns
if err := st.SetActor(taddr, act); err != nil {
return cid.Undef, err
}
if err := st.SetActor(taddr, act); err != nil {
return cid.Undef, err
}
return st.Flush(ctx)
}}})
if err != nil {
t.Fatal(err)
return st.Flush(ctx)
}
inv := vm.NewActorRegistry()
inv.Register(nil, testActor{})
sm.SetVMConstructor(func(ctx context.Context, vmopt *vm.VMOpts) (*vm.VM, error) {
nvm, err := vm.NewVM(ctx, vmopt)
if err != nil {
@@ -174,7 +168,7 @@ func TestForkHeightTriggers(t *testing.T) {
var msgs []*types.SignedMessage
enc, err := actors.SerializeParams(&init0.ExecParams{CodeCID: (testActor{}).Code()})
enc, err := actors.SerializeParams(&init_.ExecParams{CodeCID: (testActor{}).Code()})
if err != nil {
t.Fatal(err)
}
@@ -234,84 +228,3 @@ func TestForkHeightTriggers(t *testing.T) {
}
}
}
func TestForkRefuseCall(t *testing.T) {
logging.SetAllLoggers(logging.LevelInfo)
ctx := context.TODO()
cg, err := gen.NewGenerator()
if err != nil {
t.Fatal(err)
}
sm, err := NewStateManagerWithUpgradeSchedule(
cg.ChainStore(), UpgradeSchedule{{
Network: 1,
Expensive: true,
Height: testForkHeight,
Migration: func(ctx context.Context, sm *StateManager, cb ExecCallback,
root cid.Cid, height abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
return root, nil
}}})
if err != nil {
t.Fatal(err)
}
inv := vm.NewActorRegistry()
inv.Register(nil, testActor{})
sm.SetVMConstructor(func(ctx context.Context, vmopt *vm.VMOpts) (*vm.VM, error) {
nvm, err := vm.NewVM(ctx, vmopt)
if err != nil {
return nil, err
}
nvm.SetInvoker(inv)
return nvm, nil
})
cg.SetStateManager(sm)
enc, err := actors.SerializeParams(&init0.ExecParams{CodeCID: (testActor{}).Code()})
if err != nil {
t.Fatal(err)
}
m := &types.Message{
From: cg.Banker(),
To: lotusinit.Address,
Method: builtin.MethodsInit.Exec,
Params: enc,
GasLimit: types.TestGasLimit,
Value: types.NewInt(0),
GasPremium: types.NewInt(0),
GasFeeCap: types.NewInt(0),
}
for i := 0; i < 50; i++ {
ts, err := cg.NextTipSet()
if err != nil {
t.Fatal(err)
}
ret, err := sm.CallWithGas(ctx, m, nil, ts.TipSet.TipSet())
switch ts.TipSet.TipSet().Height() {
case testForkHeight, testForkHeight + 1:
// If I had a fork, or I _will_ have a fork, it should fail.
require.Equal(t, ErrExpensiveFork, err)
default:
require.NoError(t, err)
require.True(t, ret.MsgRct.ExitCode.IsSuccess())
}
// Call just runs on the parent state for a tipset, so we only
// expect an error at the fork height.
ret, err = sm.Call(ctx, m, ts.TipSet.TipSet())
switch ts.TipSet.TipSet().Height() {
case testForkHeight + 1:
require.Equal(t, ErrExpensiveFork, err)
default:
require.NoError(t, err)
require.True(t, ret.MsgRct.ExitCode.IsSuccess())
}
}
}
+69 -143
View File
@@ -2,7 +2,6 @@ package stmgr
import (
"context"
"errors"
"fmt"
"sync"
@@ -40,25 +39,9 @@ import (
var log = logging.Logger("statemgr")
type versionSpec struct {
networkVersion network.Version
atOrBelow abi.ChainEpoch
}
type StateManager struct {
cs *store.ChainStore
// Determines the network version at any given epoch.
networkVersions []versionSpec
latestVersion network.Version
// Maps chain epochs to upgrade functions.
stateMigrations map[abi.ChainEpoch]UpgradeFunc
// A set of potentially expensive/time consuming upgrades. Explicit
// calls for, e.g., gas estimation fail against this epoch with
// ErrExpensiveFork.
expensiveUpgrades map[abi.ChainEpoch]struct{}
stCache map[string][]cid.Cid
compWait map[string]chan struct{}
stlk sync.Mutex
@@ -69,54 +52,12 @@ type StateManager struct {
}
func NewStateManager(cs *store.ChainStore) *StateManager {
sm, err := NewStateManagerWithUpgradeSchedule(cs, DefaultUpgradeSchedule())
if err != nil {
panic(fmt.Sprintf("default upgrade schedule is invalid: %s", err))
}
return sm
}
func NewStateManagerWithUpgradeSchedule(cs *store.ChainStore, us UpgradeSchedule) (*StateManager, error) {
// If we have upgrades, make sure they're in-order and make sense.
if err := us.Validate(); err != nil {
return nil, err
}
stateMigrations := make(map[abi.ChainEpoch]UpgradeFunc, len(us))
expensiveUpgrades := make(map[abi.ChainEpoch]struct{}, len(us))
var networkVersions []versionSpec
lastVersion := network.Version0
if len(us) > 0 {
// If we have any upgrades, process them and create a version
// schedule.
for _, upgrade := range us {
if upgrade.Migration != nil {
stateMigrations[upgrade.Height] = upgrade.Migration
}
if upgrade.Expensive {
expensiveUpgrades[upgrade.Height] = struct{}{}
}
networkVersions = append(networkVersions, versionSpec{
networkVersion: lastVersion,
atOrBelow: upgrade.Height,
})
lastVersion = upgrade.Network
}
} else {
// Otherwise, go directly to the latest version.
lastVersion = build.NewestNetworkVersion
}
return &StateManager{
networkVersions: networkVersions,
latestVersion: lastVersion,
stateMigrations: stateMigrations,
expensiveUpgrades: expensiveUpgrades,
newVM: vm.NewVM,
cs: cs,
stCache: make(map[string][]cid.Cid),
compWait: make(map[string]chan struct{}),
}, nil
newVM: vm.NewVM,
cs: cs,
stCache: make(map[string][]cid.Cid),
compWait: make(map[string]chan struct{}),
}
}
func cidsToKey(cids []cid.Cid) string {
@@ -269,18 +210,6 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
}
for i := parentEpoch; i < epoch; i++ {
if i > parentEpoch {
// run cron for null rounds if any
if err := runCron(); err != nil {
return cid.Undef, cid.Undef, err
}
pstate, err = vmi.Flush(ctx)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("flushing vm: %w", err)
}
}
// handle state forks
// XXX: The state tree
newState, err := sm.handleStateForks(ctx, pstate, i, cb, ts)
@@ -295,6 +224,18 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
}
}
if i > parentEpoch {
// run cron for null rounds if any
if err := runCron(); err != nil {
return cid.Cid{}, cid.Cid{}, err
}
newState, err = vmi.Flush(ctx)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("flushing vm: %w", err)
}
}
vmi.SetBlockHeight(i + 1)
pstate = newState
}
@@ -339,7 +280,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
sysAct, actErr := vmi.StateTree().GetActor(builtin0.SystemActorAddr)
if actErr != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("failed to get system actor: %w", actErr)
return cid.Undef, cid.Undef, xerrors.Errorf("failed to get system actor: %w", err)
}
rwMsg := &types.Message{
@@ -355,7 +296,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
}
ret, actErr := vmi.ApplyImplicitMessage(ctx, rwMsg)
if actErr != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("failed to apply reward message for miner %s: %w", b.Miner, actErr)
return cid.Undef, cid.Undef, xerrors.Errorf("failed to apply reward message for miner %s: %w", b.Miner, err)
}
if cb != nil {
if err := cb(rwMsg.Cid(), rwMsg, ret); err != nil {
@@ -508,7 +449,16 @@ func (sm *StateManager) GetReceipt(ctx context.Context, msg cid.Cid, ts *types.T
return nil, fmt.Errorf("failed to load message: %w", err)
}
_, r, _, err := sm.searchBackForMsg(ctx, ts, m)
r, _, err := sm.tipsetExecutedMessage(ts, msg, m.VMMessage())
if err != nil {
return nil, err
}
if r != nil {
return r, nil
}
_, r, _, err = sm.searchBackForMsg(ctx, ts, m)
if err != nil {
return nil, fmt.Errorf("failed to look back through chain for message: %w", err)
}
@@ -666,18 +616,6 @@ func (sm *StateManager) SearchForMessage(ctx context.Context, mcid cid.Cid) (*ty
func (sm *StateManager) searchBackForMsg(ctx context.Context, from *types.TipSet, m types.ChainMsg) (*types.TipSet, *types.MessageReceipt, cid.Cid, error) {
cur := from
curActor, err := sm.LoadActor(ctx, m.VMMessage().From, cur)
if err != nil {
return nil, nil, cid.Undef, xerrors.Errorf("failed to load initital tipset")
}
mFromId, err := sm.LookupID(ctx, m.VMMessage().From, from)
if err != nil {
return nil, nil, cid.Undef, xerrors.Errorf("looking up From id address: %w", err)
}
mNonce := m.VMMessage().Nonce
for {
if cur.Height() == 0 {
// it ain't here!
@@ -690,37 +628,32 @@ func (sm *StateManager) searchBackForMsg(ctx context.Context, from *types.TipSet
default:
}
act, err := sm.LoadActor(ctx, m.VMMessage().From, cur)
if err != nil {
return nil, nil, cid.Cid{}, err
}
// we either have no messages from the sender, or the latest message we found has a lower nonce than the one being searched for,
// either way, no reason to lookback, it ain't there
if curActor == nil || curActor.Nonce == 0 || curActor.Nonce < mNonce {
if act.Nonce == 0 || act.Nonce < m.VMMessage().Nonce {
return nil, nil, cid.Undef, nil
}
pts, err := sm.cs.LoadTipSet(cur.Parents())
ts, err := sm.cs.LoadTipSet(cur.Parents())
if err != nil {
return nil, nil, cid.Undef, xerrors.Errorf("failed to load tipset during msg wait searchback: %w", err)
return nil, nil, cid.Undef, fmt.Errorf("failed to load tipset during msg wait searchback: %w", err)
}
act, err := sm.LoadActor(ctx, mFromId, pts)
actorNoExist := errors.Is(err, types.ErrActorNotFound)
if err != nil && !actorNoExist {
return nil, nil, cid.Cid{}, xerrors.Errorf("failed to load the actor: %w", err)
r, foundMsg, err := sm.tipsetExecutedMessage(ts, m.Cid(), m.VMMessage())
if err != nil {
return nil, nil, cid.Undef, fmt.Errorf("checking for message execution during lookback: %w", err)
}
// check that between cur and parent tipset the nonce fell into range of our message
if actorNoExist || (curActor.Nonce > mNonce && act.Nonce <= mNonce) {
r, foundMsg, err := sm.tipsetExecutedMessage(cur, m.Cid(), m.VMMessage())
if err != nil {
return nil, nil, cid.Undef, xerrors.Errorf("checking for message execution during lookback: %w", err)
}
if r != nil {
return pts, r, foundMsg, nil
}
if r != nil {
return ts, r, foundMsg, nil
}
cur = pts
curActor = act
cur = ts
}
}
@@ -1189,27 +1122,14 @@ func (sm *StateManager) GetFilVested(ctx context.Context, height abi.ChainEpoch,
}
}
// After UpgradeActorsV2Height these funds are accounted for in GetFilReserveDisbursed
if height <= build.UpgradeActorsV2Height {
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.preIgnitionGenInfos.genesisPledge)
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.preIgnitionGenInfos.genesisMarketFunds)
}
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.preIgnitionGenInfos.genesisPledge)
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.preIgnitionGenInfos.genesisMarketFunds)
return vf, nil
}
func GetFilReserveDisbursed(ctx context.Context, st *state.StateTree) (abi.TokenAmount, error) {
ract, err := st.GetActor(builtin.ReserveAddress)
if err != nil {
return big.Zero(), xerrors.Errorf("failed to get reserve actor: %w", err)
}
// If money enters the reserve actor, this could lead to a negative term
return big.Sub(big.NewFromGo(build.InitialFilReserved), ract.Balance), nil
}
func GetFilMined(ctx context.Context, st *state.StateTree) (abi.TokenAmount, error) {
ractor, err := st.GetActor(reward.Address)
if err != nil {
@@ -1297,14 +1217,6 @@ func (sm *StateManager) GetCirculatingSupplyDetailed(ctx context.Context, height
return api.CirculatingSupply{}, xerrors.Errorf("failed to calculate filVested: %w", err)
}
filReserveDisbursed := big.Zero()
if height > build.UpgradeActorsV2Height {
filReserveDisbursed, err = GetFilReserveDisbursed(ctx, st)
if err != nil {
return api.CirculatingSupply{}, xerrors.Errorf("failed to calculate filReserveDisbursed: %w", err)
}
}
filMined, err := GetFilMined(ctx, st)
if err != nil {
return api.CirculatingSupply{}, xerrors.Errorf("failed to calculate filMined: %w", err)
@@ -1321,7 +1233,6 @@ func (sm *StateManager) GetCirculatingSupplyDetailed(ctx context.Context, height
}
ret := types.BigAdd(filVested, filMined)
ret = types.BigAdd(ret, filReserveDisbursed)
ret = types.BigSub(ret, filBurnt)
ret = types.BigSub(ret, filLocked)
@@ -1348,14 +1259,29 @@ func (sm *StateManager) GetCirculatingSupply(ctx context.Context, height abi.Cha
}
func (sm *StateManager) GetNtwkVersion(ctx context.Context, height abi.ChainEpoch) network.Version {
// The epochs here are the _last_ epoch for every version, or -1 if the
// version is disabled.
for _, spec := range sm.networkVersions {
if height <= spec.atOrBelow {
return spec.networkVersion
}
// TODO: move hard fork epoch checks to a schedule defined in build/
if build.UseNewestNetwork() {
return build.NewestNetworkVersion
}
return sm.latestVersion
if height <= build.UpgradeBreezeHeight {
return network.Version0
}
if height <= build.UpgradeSmokeHeight {
return network.Version1
}
if height <= build.UpgradeIgnitionHeight {
return network.Version2
}
if height <= build.UpgradeActorsV2 {
return network.Version3
}
return build.NewestNetworkVersion
}
func (sm *StateManager) GetPaychState(ctx context.Context, addr address.Address, ts *types.TipSet) (*types.Actor, paych.State, error) {
+15 -90
View File
@@ -9,11 +9,6 @@ import (
"runtime"
"strings"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/chain/actors/policy"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
@@ -26,10 +21,11 @@ import (
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
exported0 "github.com/filecoin-project/specs-actors/actors/builtin/exported"
exported2 "github.com/filecoin-project/specs-actors/actors/builtin/exported"
proof0 "github.com/filecoin-project/specs-actors/actors/runtime/proof"
exported2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/exported"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/actors/builtin/market"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
@@ -387,7 +383,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := sm.newVM(ctx, vmopt)
vmi, err := vm.NewVM(ctx, vmopt)
if err != nil {
return cid.Undef, nil, err
}
@@ -413,9 +409,8 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
func GetLookbackTipSetForRound(ctx context.Context, sm *StateManager, ts *types.TipSet, round abi.ChainEpoch) (*types.TipSet, error) {
var lbr abi.ChainEpoch
lb := policy.GetWinningPoStSectorSetLookback(sm.GetNtwkVersion(ctx, round))
if round > lb {
lbr = round - lb
if round > build.WinningPoStSectorSetLookback {
lbr = round - build.WinningPoStSectorSetLookback
}
// more null blocks than our lookback
@@ -495,7 +490,7 @@ func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcs beacon.Schedule
return nil, nil
}
mpow, tpow, _, err := GetPowerRaw(ctx, sm, lbst, maddr)
mpow, tpow, hmp, err := GetPowerRaw(ctx, sm, lbst, maddr)
if err != nil {
return nil, xerrors.Errorf("failed to get power: %w", err)
}
@@ -510,21 +505,15 @@ func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcs beacon.Schedule
return nil, xerrors.Errorf("resolving worker address: %w", err)
}
// TODO: Not ideal performance...This method reloads miner and power state (already looked up here and in GetPowerRaw)
eligible, err := MinerEligibleToMine(ctx, sm, maddr, ts, lbts)
if err != nil {
return nil, xerrors.Errorf("determining miner eligibility: %w", err)
}
return &api.MiningBaseInfo{
MinerPower: mpow.QualityAdjPower,
NetworkPower: tpow.QualityAdjPower,
Sectors: sectors,
WorkerKey: worker,
SectorSize: info.SectorSize,
PrevBeaconEntry: *prev,
BeaconEntries: entries,
EligibleForMining: eligible,
MinerPower: mpow.QualityAdjPower,
NetworkPower: tpow.QualityAdjPower,
Sectors: sectors,
WorkerKey: worker,
SectorSize: info.SectorSize,
PrevBeaconEntry: *prev,
BeaconEntries: entries,
HasMinPower: hmp,
}, nil
}
@@ -606,7 +595,7 @@ func GetReturnType(ctx context.Context, sm *StateManager, to address.Address, me
return reflect.New(m.Ret.Elem()).Interface().(cbg.CBORUnmarshaler), nil
}
func minerHasMinPower(ctx context.Context, sm *StateManager, addr address.Address, ts *types.TipSet) (bool, error) {
func MinerHasMinPower(ctx context.Context, sm *StateManager, addr address.Address, ts *types.TipSet) (bool, error) {
pact, err := sm.LoadActor(ctx, power.Address, ts)
if err != nil {
return false, xerrors.Errorf("loading power actor state: %w", err)
@@ -620,70 +609,6 @@ func minerHasMinPower(ctx context.Context, sm *StateManager, addr address.Addres
return ps.MinerNominalPowerMeetsConsensusMinimum(addr)
}
func MinerEligibleToMine(ctx context.Context, sm *StateManager, addr address.Address, baseTs *types.TipSet, lookbackTs *types.TipSet) (bool, error) {
hmp, err := minerHasMinPower(ctx, sm, addr, lookbackTs)
// TODO: We're blurring the lines between a "runtime network version" and a "Lotus upgrade epoch", is that unavoidable?
if sm.GetNtwkVersion(ctx, baseTs.Height()) <= network.Version3 {
return hmp, err
}
if err != nil {
return false, err
}
if !hmp {
return false, nil
}
// Post actors v2, also check MinerEligibleForElection with base ts
pact, err := sm.LoadActor(ctx, power.Address, baseTs)
if err != nil {
return false, xerrors.Errorf("loading power actor state: %w", err)
}
pstate, err := power.Load(sm.cs.Store(ctx), pact)
if err != nil {
return false, err
}
mact, err := sm.LoadActor(ctx, addr, baseTs)
if err != nil {
return false, xerrors.Errorf("loading miner actor state: %w", err)
}
mstate, err := miner.Load(sm.cs.Store(ctx), mact)
if err != nil {
return false, err
}
// Non-empty power claim.
if claim, found, err := pstate.MinerPower(addr); err != nil {
return false, err
} else if !found {
return false, err
} else if claim.QualityAdjPower.LessThanEqual(big.Zero()) {
return false, err
}
// No fee debt.
if debt, err := mstate.FeeDebt(); err != nil {
return false, err
} else if !debt.IsZero() {
return false, err
}
// No active consensus faults.
if mInfo, err := mstate.Info(); err != nil {
return false, err
} else if baseTs.Height() <= mInfo.ConsensusFaultElapsed {
return false, nil
}
return true, nil
}
func CheckTotalFIL(ctx context.Context, sm *StateManager, ts *types.TipSet) (abi.TokenAmount, error) {
str, err := state.LoadStateTree(sm.ChainStore().Store(ctx), ts.ParentState())
if err != nil {
+1 -1
View File
@@ -46,7 +46,7 @@ func ComputeNextBaseFee(baseFee types.BigInt, gasLimitUsed int64, noOfBlocks int
}
func (cs *ChainStore) ComputeBaseFee(ctx context.Context, ts *types.TipSet) (abi.TokenAmount, error) {
if build.UpgradeBreezeHeight >= 0 && ts.Height() > build.UpgradeBreezeHeight && ts.Height() < build.UpgradeBreezeHeight+build.BreezeGasTampingDuration {
if ts.Height() > build.UpgradeBreezeHeight && ts.Height() < build.UpgradeBreezeHeight+build.BreezeGasTampingDuration {
return abi.NewTokenAmount(100), nil
}
+12 -15
View File
@@ -11,27 +11,24 @@ import (
func TestBaseFee(t *testing.T) {
tests := []struct {
basefee uint64
limitUsed int64
noOfBlocks int
preSmoke, postSmoke uint64
basefee uint64
limitUsed int64
noOfBlocks int
output uint64
}{
{100e6, 0, 1, 87.5e6, 87.5e6},
{100e6, 0, 5, 87.5e6, 87.5e6},
{100e6, build.BlockGasTarget, 1, 103.125e6, 100e6},
{100e6, build.BlockGasTarget * 2, 2, 103.125e6, 100e6},
{100e6, build.BlockGasLimit * 2, 2, 112.5e6, 112.5e6},
{100e6, build.BlockGasLimit * 1.5, 2, 110937500, 106.250e6},
{100e6, 0, 1, 87.5e6},
{100e6, 0, 5, 87.5e6},
{100e6, build.BlockGasTarget, 1, 103.125e6},
{100e6, build.BlockGasTarget * 2, 2, 103.125e6},
{100e6, build.BlockGasLimit * 2, 2, 112.5e6},
{100e6, build.BlockGasLimit * 1.5, 2, 110937500},
}
for _, test := range tests {
test := test
t.Run(fmt.Sprintf("%v", test), func(t *testing.T) {
preSmoke := ComputeNextBaseFee(types.NewInt(test.basefee), test.limitUsed, test.noOfBlocks, build.UpgradeSmokeHeight-1)
assert.Equal(t, fmt.Sprintf("%d", test.preSmoke), preSmoke.String())
postSmoke := ComputeNextBaseFee(types.NewInt(test.basefee), test.limitUsed, test.noOfBlocks, build.UpgradeSmokeHeight+1)
assert.Equal(t, fmt.Sprintf("%d", test.postSmoke), postSmoke.String())
output := ComputeNextBaseFee(types.NewInt(test.basefee), test.limitUsed, test.noOfBlocks, 0)
assert.Equal(t, fmt.Sprintf("%d", test.output), output.String())
})
}
}
+17 -39
View File
@@ -288,16 +288,6 @@ func (cs *ChainStore) MarkBlockAsValidated(ctx context.Context, blkid cid.Cid) e
return nil
}
func (cs *ChainStore) UnmarkBlockAsValidated(ctx context.Context, blkid cid.Cid) error {
key := blockValidationCacheKeyPrefix.Instance(blkid.String())
if err := cs.ds.Delete(key); err != nil {
return xerrors.Errorf("removing from valid block cache: %w", err)
}
return nil
}
func (cs *ChainStore) SetGenesis(b *types.BlockHeader) error {
ts, err := types.NewTipSet([]*types.BlockHeader{b})
if err != nil {
@@ -1195,6 +1185,13 @@ func recurseLinks(bs bstore.Blockstore, walked *cid.Set, root cid.Cid, in []cid.
}
func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, inclRecentRoots abi.ChainEpoch, skipOldMsgs bool, w io.Writer) error {
if ts == nil {
ts = cs.GetHeaviestTipSet()
}
seen := cid.NewSet()
walked := cid.NewSet()
h := &car.CarHeader{
Roots: ts.Cids(),
Version: 1,
@@ -1204,28 +1201,6 @@ func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, inclRecentRo
return xerrors.Errorf("failed to write car header: %s", err)
}
return cs.WalkSnapshot(ctx, ts, inclRecentRoots, skipOldMsgs, func(c cid.Cid) error {
blk, err := cs.bs.Get(c)
if err != nil {
return xerrors.Errorf("writing object to car, bs.Get: %w", err)
}
if err := carutil.LdWrite(w, c.Bytes(), blk.RawData()); err != nil {
return xerrors.Errorf("failed to write block to car output: %w", err)
}
return nil
})
}
func (cs *ChainStore) WalkSnapshot(ctx context.Context, ts *types.TipSet, inclRecentRoots abi.ChainEpoch, skipOldMsgs bool, cb func(cid.Cid) error) error {
if ts == nil {
ts = cs.GetHeaviestTipSet()
}
seen := cid.NewSet()
walked := cid.NewSet()
blocksToWalk := ts.Cids()
currentMinHeight := ts.Height()
@@ -1234,15 +1209,15 @@ func (cs *ChainStore) WalkSnapshot(ctx context.Context, ts *types.TipSet, inclRe
return nil
}
if err := cb(blk); err != nil {
return err
}
data, err := cs.bs.Get(blk)
if err != nil {
return xerrors.Errorf("getting block: %w", err)
}
if err := carutil.LdWrite(w, blk.Bytes(), data.RawData()); err != nil {
return xerrors.Errorf("failed to write block to car output: %w", err)
}
var b types.BlockHeader
if err := b.UnmarshalCBOR(bytes.NewBuffer(data.RawData())); err != nil {
return xerrors.Errorf("unmarshaling block header (cid=%s): %w", blk, err)
@@ -1289,11 +1264,14 @@ func (cs *ChainStore) WalkSnapshot(ctx context.Context, ts *types.TipSet, inclRe
if c.Prefix().Codec != cid.DagCBOR {
continue
}
if err := cb(c); err != nil {
return err
data, err := cs.bs.Get(c)
if err != nil {
return xerrors.Errorf("writing object to car (get %s): %w", c, err)
}
if err := carutil.LdWrite(w, c.Bytes(), data.RawData()); err != nil {
return xerrors.Errorf("failed to write out car object: %w", err)
}
}
}
+19 -28
View File
@@ -44,11 +44,7 @@ var log = logging.Logger("sub")
var ErrSoftFailure = errors.New("soft validation failure")
var ErrInsufficientPower = errors.New("incoming block's miner does not have minimum power")
func HandleIncomingBlocks(ctx context.Context, bsub *pubsub.Subscription, s *chain.Syncer, bs bserv.BlockService, cmgr connmgr.ConnManager) {
// Timeout after (block time + propagation delay). This is useless at
// this point.
timeout := time.Duration(build.BlockDelaySecs+build.PropagationDelaySecs) * time.Second
func HandleIncomingBlocks(ctx context.Context, bsub *pubsub.Subscription, s *chain.Syncer, bserv bserv.BlockService, cmgr connmgr.ConnManager) {
for {
msg, err := bsub.Next(ctx)
if err != nil {
@@ -69,22 +65,15 @@ func HandleIncomingBlocks(ctx context.Context, bsub *pubsub.Subscription, s *cha
src := msg.GetFrom()
go func() {
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
// NOTE: we could also share a single session between
// all requests but that may have other consequences.
ses := bserv.NewSession(ctx, bs)
start := build.Clock.Now()
log.Debug("about to fetch messages for block from pubsub")
bmsgs, err := FetchMessagesByCids(ctx, ses, blk.BlsMessages)
bmsgs, err := FetchMessagesByCids(context.TODO(), bserv, blk.BlsMessages)
if err != nil {
log.Errorf("failed to fetch all bls messages for block received over pubusb: %s; source: %s", err, src)
return
}
smsgs, err := FetchSignedMessagesByCids(ctx, ses, blk.SecpkMessages)
smsgs, err := FetchSignedMessagesByCids(context.TODO(), bserv, blk.SecpkMessages)
if err != nil {
log.Errorf("failed to fetch all secpk messages for block received over pubusb: %s; source: %s", err, src)
return
@@ -109,7 +98,7 @@ func HandleIncomingBlocks(ctx context.Context, bsub *pubsub.Subscription, s *cha
func FetchMessagesByCids(
ctx context.Context,
bserv bserv.BlockGetter,
bserv bserv.BlockService,
cids []cid.Cid,
) ([]*types.Message, error) {
out := make([]*types.Message, len(cids))
@@ -138,7 +127,7 @@ func FetchMessagesByCids(
// FIXME: Duplicate of above.
func FetchSignedMessagesByCids(
ctx context.Context,
bserv bserv.BlockGetter,
bserv bserv.BlockService,
cids []cid.Cid,
) ([]*types.SignedMessage, error) {
out := make([]*types.SignedMessage, len(cids))
@@ -168,28 +157,30 @@ func FetchSignedMessagesByCids(
// blocks we did not request.
func fetchCids(
ctx context.Context,
bserv bserv.BlockGetter,
bserv bserv.BlockService,
cids []cid.Cid,
cb func(int, blocks.Block) error,
) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// FIXME: Why don't we use the context here?
fetchedBlocks := bserv.GetBlocks(context.TODO(), cids)
cidIndex := make(map[cid.Cid]int)
for i, c := range cids {
cidIndex[c] = i
}
if len(cids) != len(cidIndex) {
return fmt.Errorf("duplicate CIDs in fetchCids input")
}
fetchedBlocks := bserv.GetBlocks(ctx, cids)
for i := 0; i < len(cids); i++ {
select {
case block, ok := <-fetchedBlocks:
if !ok {
// Closed channel, no more blocks fetched, check if we have all
// of the CIDs requested.
// FIXME: Review this check. We don't call the callback on the
// last index?
if i == len(cids)-1 {
break
}
return fmt.Errorf("failed to fetch all messages")
}
@@ -484,14 +475,14 @@ func (bv *BlockValidator) checkPowerAndGetWorkerKey(ctx context.Context, bh *typ
return address.Undef, ErrSoftFailure
}
eligible, err := stmgr.MinerEligibleToMine(ctx, bv.stmgr, bh.Miner, baseTs, lbts)
hmp, err := stmgr.MinerHasMinPower(ctx, bv.stmgr, bh.Miner, lbts)
if err != nil {
log.Warnf("failed to determine if incoming block's miner has minimum power: %s", err)
return address.Undef, ErrSoftFailure
}
if !eligible {
log.Warnf("incoming block's miner is ineligible")
if !hmp {
log.Warnf("incoming block's miner does not have minimum power")
return address.Undef, ErrInsufficientPower
}
-63
View File
@@ -1,63 +0,0 @@
package sub
import (
"context"
"testing"
address "github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"
blocks "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
)
type getter struct {
msgs []*types.Message
}
func (g *getter) GetBlock(ctx context.Context, c cid.Cid) (blocks.Block, error) { panic("NYI") }
func (g *getter) GetBlocks(ctx context.Context, ks []cid.Cid) <-chan blocks.Block {
ch := make(chan blocks.Block, len(g.msgs))
for _, m := range g.msgs {
by, err := m.Serialize()
if err != nil {
panic(err)
}
b, err := blocks.NewBlockWithCid(by, m.Cid())
if err != nil {
panic(err)
}
ch <- b
}
close(ch)
return ch
}
func TestFetchCidsWithDedup(t *testing.T) {
msgs := []*types.Message{}
for i := 0; i < 10; i++ {
msgs = append(msgs, &types.Message{
From: address.TestAddress,
To: address.TestAddress,
Nonce: uint64(i),
})
}
cids := []cid.Cid{}
for _, m := range msgs {
cids = append(cids, m.Cid())
}
g := &getter{msgs}
// the cids have a duplicate
res, err := FetchMessagesByCids(context.TODO(), g, append(cids, cids[0]))
t.Logf("err: %+v", err)
t.Logf("res: %+v", res)
if err == nil {
t.Errorf("there should be an error")
}
if err == nil && (res[0] == nil || res[len(res)-1] == nil) {
t.Fatalf("there is a nil message: first %p, last %p", res[0], res[len(res)-1])
}
}
+40 -61
View File
@@ -217,12 +217,6 @@ func (syncer *Syncer) Stop() {
// This should be called when connecting to new peers, and additionally
// when receiving new blocks from the network
func (syncer *Syncer) InformNewHead(from peer.ID, fts *store.FullTipSet) bool {
defer func() {
if err := recover(); err != nil {
log.Errorf("panic in InformNewHead: ", err)
}
}()
ctx := context.Background()
if fts == nil {
log.Errorf("got nil tipset in InformNewHead")
@@ -607,7 +601,7 @@ func isPermanent(err error) bool {
return !errors.Is(err, ErrTemporal)
}
func (syncer *Syncer) ValidateTipSet(ctx context.Context, fts *store.FullTipSet, useCache bool) error {
func (syncer *Syncer) ValidateTipSet(ctx context.Context, fts *store.FullTipSet) error {
ctx, span := trace.StartSpan(ctx, "validateTipSet")
defer span.End()
@@ -623,7 +617,7 @@ func (syncer *Syncer) ValidateTipSet(ctx context.Context, fts *store.FullTipSet,
b := b // rebind to a scoped variable
futures = append(futures, async.Err(func() error {
if err := syncer.ValidateBlock(ctx, b, useCache); err != nil {
if err := syncer.ValidateBlock(ctx, b); err != nil {
if isPermanent(err) {
syncer.bad.Add(b.Cid(), NewBadBlockReason([]cid.Cid{b.Cid()}, err.Error()))
}
@@ -690,7 +684,7 @@ func blockSanityChecks(h *types.BlockHeader) error {
}
// ValidateBlock should match up with 'Semantical Validation' in validation.md in the spec
func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, useCache bool) (err error) {
func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock) (err error) {
defer func() {
// b.Cid() could panic for empty blocks that are used in tests.
if rerr := recover(); rerr != nil {
@@ -699,15 +693,13 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
}
}()
if useCache {
isValidated, err := syncer.store.IsBlockValidated(ctx, b.Cid())
if err != nil {
return xerrors.Errorf("check block validation cache %s: %w", b.Cid(), err)
}
isValidated, err := syncer.store.IsBlockValidated(ctx, b.Cid())
if err != nil {
return xerrors.Errorf("check block validation cache %s: %w", b.Cid(), err)
}
if isValidated {
return nil
}
if isValidated {
return nil
}
validationStart := build.Clock.Now()
@@ -737,7 +729,7 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
lbst, _, err := syncer.sm.TipSetState(ctx, lbts)
if err != nil {
return xerrors.Errorf("failed to compute lookback tipset state (epoch %d): %w", lbts.Height(), err)
return xerrors.Errorf("failed to compute lookback tipset state: %w", err)
}
prevBeacon, err := syncer.store.GetLatestBeaconEntry(baseTs)
@@ -794,35 +786,31 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
b.Header.ParentWeight, pweight)
}
stateRootCheck := async.Err(func() error {
stateroot, precp, err := syncer.sm.TipSetState(ctx, baseTs)
// Stuff that needs stateroot / worker address
stateroot, precp, err := syncer.sm.TipSetState(ctx, baseTs)
if err != nil {
return xerrors.Errorf("get tipsetstate(%d, %s) failed: %w", h.Height, h.Parents, err)
}
if stateroot != h.ParentStateRoot {
msgs, err := syncer.store.MessagesForTipset(baseTs)
if err != nil {
return xerrors.Errorf("get tipsetstate(%d, %s) failed: %w", h.Height, h.Parents, err)
}
if stateroot != h.ParentStateRoot {
msgs, err := syncer.store.MessagesForTipset(baseTs)
if err != nil {
log.Error("failed to load messages for tipset during tipset state mismatch error: ", err)
} else {
log.Warn("Messages for tipset with mismatching state:")
for i, m := range msgs {
mm := m.VMMessage()
log.Warnf("Message[%d]: from=%s to=%s method=%d params=%x", i, mm.From, mm.To, mm.Method, mm.Params)
}
log.Error("failed to load messages for tipset during tipset state mismatch error: ", err)
} else {
log.Warn("Messages for tipset with mismatching state:")
for i, m := range msgs {
mm := m.VMMessage()
log.Warnf("Message[%d]: from=%s to=%s method=%d params=%x", i, mm.From, mm.To, mm.Method, mm.Params)
}
return xerrors.Errorf("parent state root did not match computed state (%s != %s)", stateroot, h.ParentStateRoot)
}
if precp != h.ParentMessageReceipts {
return xerrors.Errorf("parent receipts root did not match computed value (%s != %s)", precp, h.ParentMessageReceipts)
}
return xerrors.Errorf("parent state root did not match computed state (%s != %s)", stateroot, h.ParentStateRoot)
}
return nil
})
if precp != h.ParentMessageReceipts {
return xerrors.Errorf("parent receipts root did not match computed value (%s != %s)", precp, h.ParentMessageReceipts)
}
// Stuff that needs worker address
waddr, err := stmgr.GetMinerWorkerRaw(ctx, syncer.sm, lbst, h.Miner)
if err != nil {
return xerrors.Errorf("GetMinerWorkerRaw failed: %w", err)
@@ -833,13 +821,13 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
return xerrors.Errorf("block is not claiming to be a winner")
}
eligible, err := stmgr.MinerEligibleToMine(ctx, syncer.sm, h.Miner, baseTs, lbts)
hp, err := stmgr.MinerHasMinPower(ctx, syncer.sm, h.Miner, lbts)
if err != nil {
return xerrors.Errorf("determining if miner has min power failed: %w", err)
}
if !eligible {
return xerrors.New("block's miner is ineligible to mine")
if !hp {
return xerrors.New("block's miner does not meet minimum power threshold")
}
rBeacon := *prevBeacon
@@ -943,7 +931,6 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
winnerCheck,
msgsCheck,
baseFeeCheck,
stateRootCheck,
}
var merr error
@@ -971,10 +958,8 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock, use
return mulErr
}
if useCache {
if err := syncer.store.MarkBlockAsValidated(ctx, b.Cid()); err != nil {
return xerrors.Errorf("caching block validation %s: %w", b.Cid(), err)
}
if err := syncer.store.MarkBlockAsValidated(ctx, b.Cid()); err != nil {
return xerrors.Errorf("caching block validation %s: %w", b.Cid(), err)
}
return nil
@@ -1287,11 +1272,9 @@ func (syncer *Syncer) collectHeaders(ctx context.Context, incoming *types.TipSet
blockSet := []*types.TipSet{incoming}
// Parent of the new (possibly better) tipset that we need to fetch next.
at := incoming.Parents()
// we want to sync all the blocks until the height above our
// best tipset so far
// we want to sync all the blocks until the height above the block we have
untilHeight := known.Height() + 1
ss.SetHeight(blockSet[len(blockSet)-1].Height())
@@ -1385,17 +1368,13 @@ loop:
}
base := blockSet[len(blockSet)-1]
if base.IsChildOf(known) {
// common case: receiving blocks that are building on top of our best tipset
if base.Parents() == known.Parents() {
// common case: receiving a block thats potentially part of the same tipset as our best block
return blockSet, nil
}
knownParent, err := syncer.store.LoadTipSet(known.Parents())
if err != nil {
return nil, xerrors.Errorf("failed to load next local tipset: %w", err)
}
if base.IsChildOf(knownParent) {
// common case: receiving a block thats potentially part of the same tipset as our best block
if types.CidArrsEqual(base.Parents().Cids(), known.Cids()) {
// common case: receiving blocks that are building on top of our best tipset
return blockSet, nil
}
@@ -1482,7 +1461,7 @@ func (syncer *Syncer) syncMessagesAndCheckState(ctx context.Context, headers []*
return syncer.iterFullTipsets(ctx, headers, func(ctx context.Context, fts *store.FullTipSet) error {
log.Debugw("validating tipset", "height", fts.TipSet().Height(), "size", len(fts.TipSet().Cids()))
if err := syncer.ValidateTipSet(ctx, fts, true); err != nil {
if err := syncer.ValidateTipSet(ctx, fts); err != nil {
log.Errorf("failed to validate tipset: %+v", err)
return xerrors.Errorf("message processing failed: %w", err)
}
+2 -2
View File
@@ -732,7 +732,7 @@ func TestSyncInputs(t *testing.T) {
err := s.ValidateBlock(context.TODO(), &types.FullBlock{
Header: &types.BlockHeader{},
}, false)
})
if err == nil {
t.Fatal("should error on empty block")
}
@@ -741,7 +741,7 @@ func TestSyncInputs(t *testing.T) {
h.ElectionProof = nil
err = s.ValidateBlock(context.TODO(), &types.FullBlock{Header: h}, false)
err = s.ValidateBlock(context.TODO(), &types.FullBlock{Header: h})
if err == nil {
t.Fatal("should error on block with nil election proof")
}
+8 -8
View File
@@ -1648,7 +1648,7 @@ func (t *StateRoot) MarshalCBOR(w io.Writer) error {
scratch := make([]byte, 9)
// t.Version (types.StateTreeVersion) (uint64)
// t.Version (uint64) (uint64)
if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajUnsignedInt, uint64(t.Version)); err != nil {
return err
@@ -1687,7 +1687,7 @@ func (t *StateRoot) UnmarshalCBOR(r io.Reader) error {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Version (types.StateTreeVersion) (uint64)
// t.Version (uint64) (uint64)
{
@@ -1698,7 +1698,7 @@ func (t *StateRoot) UnmarshalCBOR(r io.Reader) error {
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.Version = StateTreeVersion(extra)
t.Version = uint64(extra)
}
// t.Actors (cid.Cid) (struct)
@@ -1728,22 +1728,22 @@ func (t *StateRoot) UnmarshalCBOR(r io.Reader) error {
return nil
}
var lengthBufStateInfo0 = []byte{128}
var lengthBufStateInfo = []byte{128}
func (t *StateInfo0) MarshalCBOR(w io.Writer) error {
func (t *StateInfo) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
if _, err := w.Write(lengthBufStateInfo0); err != nil {
if _, err := w.Write(lengthBufStateInfo); err != nil {
return err
}
return nil
}
func (t *StateInfo0) UnmarshalCBOR(r io.Reader) error {
*t = StateInfo0{}
func (t *StateInfo) UnmarshalCBOR(r io.Reader) error {
*t = StateInfo{}
br := cbg.GetPeeker(r)
scratch := make([]byte, 8)
+2 -6
View File
@@ -12,15 +12,11 @@ import (
type FIL BigInt
func (f FIL) String() string {
return f.Unitless() + " FIL"
}
func (f FIL) Unitless() string {
r := new(big.Rat).SetFrac(f.Int, big.NewInt(int64(build.FilecoinPrecision)))
if r.Sign() == 0 {
return "0"
return "0 FIL"
}
return strings.TrimRight(strings.TrimRight(r.FloatString(18), "0"), ".")
return strings.TrimRight(strings.TrimRight(r.FloatString(18), "0"), ".") + " FIL"
}
func (f FIL) Format(s fmt.State, ch rune) {
+3 -14
View File
@@ -2,20 +2,9 @@ package types
import "github.com/ipfs/go-cid"
// StateTreeVersion is the version of the state tree itself, independent of the
// network version or the actors version.
type StateTreeVersion uint64
const (
// StateTreeVersion0 corresponds to actors < v2.
StateTreeVersion0 StateTreeVersion = iota
// StateTreeVersion1 corresponds to actors >= v2.
StateTreeVersion1
)
type StateRoot struct {
// State tree version.
Version StateTreeVersion
// State root version. Versioned along with actors (for now).
Version uint64
// Actors tree. The structure depends on the state root version.
Actors cid.Cid
// Info. The structure depends on the state root version.
@@ -23,4 +12,4 @@ type StateRoot struct {
}
// TODO: version this.
type StateInfo0 struct{}
type StateInfo struct{}
+3 -3
View File
@@ -70,12 +70,12 @@ func (ar *ActorRegistry) Invoke(codeCid cid.Cid, rt vmr.Runtime, method abi.Meth
log.Errorf("no code for actor %s (Addr: %s)", codeCid, rt.Receiver())
return nil, aerrors.Newf(exitcode.SysErrorIllegalActor, "no code for actor %s(%d)(%s)", codeCid, method, hex.EncodeToString(params))
}
if err := act.predicate(rt, act.vmActor); err != nil {
return nil, aerrors.Newf(exitcode.SysErrorIllegalActor, "unsupported actor: %s", err)
}
if method >= abi.MethodNum(len(act.methods)) || act.methods[method] == nil {
return nil, aerrors.Newf(exitcode.SysErrInvalidMethod, "no method %d on actor", method)
}
if err := act.predicate(rt, act.vmActor); err != nil {
return nil, aerrors.Newf(exitcode.SysErrInvalidMethod, "unsupported actor: %s", err)
}
return act.methods[method](rt, params)
}
+9 -9
View File
@@ -31,41 +31,41 @@ func init() {
var EmptyObjectCid cid.Cid
// TryCreateAccountActor creates account actors from only BLS/SECP256K1 addresses.
func TryCreateAccountActor(rt *Runtime, addr address.Address) (*types.Actor, address.Address, aerrors.ActorError) {
func TryCreateAccountActor(rt *Runtime, addr address.Address) (*types.Actor, aerrors.ActorError) {
if err := rt.chargeGasSafe(PricelistByEpoch(rt.height).OnCreateActor()); err != nil {
return nil, address.Undef, err
return nil, err
}
addrID, err := rt.state.RegisterNewAddress(addr)
if err != nil {
return nil, address.Undef, aerrors.Escalate(err, "registering actor address")
return nil, aerrors.Escalate(err, "registering actor address")
}
act, aerr := makeActor(actors.VersionForNetwork(rt.NetworkVersion()), addr)
if aerr != nil {
return nil, address.Undef, aerr
return nil, aerr
}
if err := rt.state.SetActor(addrID, act); err != nil {
return nil, address.Undef, aerrors.Escalate(err, "creating new actor failed")
return nil, aerrors.Escalate(err, "creating new actor failed")
}
p, err := actors.SerializeParams(&addr)
if err != nil {
return nil, address.Undef, aerrors.Escalate(err, "couldn't serialize params for actor construction")
return nil, aerrors.Escalate(err, "couldn't serialize params for actor construction")
}
// call constructor on account
_, aerr = rt.internalSend(builtin0.SystemActorAddr, addrID, builtin0.MethodsAccount.Constructor, big.Zero(), p)
if aerr != nil {
return nil, address.Undef, aerrors.Wrap(aerr, "failed to invoke account constructor")
return nil, aerrors.Wrap(aerr, "failed to invoke account constructor")
}
act, err = rt.state.GetActor(addrID)
if err != nil {
return nil, address.Undef, aerrors.Escalate(err, "loading newly created actor failed")
return nil, aerrors.Escalate(err, "loading newly created actor failed")
}
return act, addrID, nil
return act, nil
}
func makeActor(ver actors.Version, addr address.Address) (*types.Actor, aerrors.ActorError) {
+28 -63
View File
@@ -5,6 +5,7 @@ import (
"context"
"encoding/binary"
"fmt"
gruntime "runtime"
"time"
"github.com/filecoin-project/go-address"
@@ -15,7 +16,6 @@ import (
"github.com/filecoin-project/go-state-types/network"
rtt "github.com/filecoin-project/go-state-types/rt"
rt0 "github.com/filecoin-project/specs-actors/actors/runtime"
rt2 "github.com/filecoin-project/specs-actors/v2/actors/runtime"
"github.com/ipfs/go-cid"
ipldcbor "github.com/ipfs/go-ipld-cbor"
"go.opencensus.io/trace"
@@ -27,33 +27,8 @@ import (
"github.com/filecoin-project/lotus/chain/types"
)
type Message struct {
msg types.Message
}
func (m *Message) Caller() address.Address {
if m.msg.From.Protocol() != address.ID {
panic("runtime message has a non-ID caller")
}
return m.msg.From
}
func (m *Message) Receiver() address.Address {
if m.msg.To != address.Undef && m.msg.To.Protocol() != address.ID {
panic("runtime message has a non-ID receiver")
}
return m.msg.To
}
func (m *Message) ValueReceived() abi.TokenAmount {
return m.msg.Value
}
// EnableGasTracing, if true, outputs gas tracing in execution traces.
var EnableGasTracing = false
type Runtime struct {
rt0.Message
types.Message
rt0.Syscalls
ctx context.Context
@@ -134,7 +109,6 @@ func (rt *Runtime) StorePut(x cbor.Marshaler) cid.Cid {
}
var _ rt0.Runtime = (*Runtime)(nil)
var _ rt2.Runtime = (*Runtime)(nil)
func (rt *Runtime) shimCall(f func() interface{}) (rval []byte, aerr aerrors.ActorError) {
defer func() {
@@ -147,11 +121,7 @@ func (rt *Runtime) shimCall(f func() interface{}) (rval []byte, aerr aerrors.Act
//log.Desugar().WithOptions(zap.AddStacktrace(zapcore.ErrorLevel)).
//Sugar().Errorf("spec actors failure: %s", r)
log.Errorf("spec actors failure: %s", r)
if rt.NetworkVersion() <= network.Version3 {
aerr = aerrors.Newf(1, "spec actors failure: %s", r)
} else {
aerr = aerrors.Newf(exitcode.SysErrReserved1, "spec actors failure: %s", r)
}
aerr = aerrors.Newf(1, "spec actors failure: %s", r)
}
}()
@@ -480,10 +450,8 @@ func (rt *Runtime) stateCommit(oldh, newh cid.Cid) aerrors.ActorError {
}
func (rt *Runtime) finilizeGasTracing() {
if EnableGasTracing {
if rt.lastGasCharge != nil {
rt.lastGasCharge.TimeTaken = time.Since(rt.lastGasChargeTime)
}
if rt.lastGasCharge != nil {
rt.lastGasCharge.TimeTaken = time.Since(rt.lastGasChargeTime)
}
}
@@ -514,35 +482,32 @@ func (rt *Runtime) chargeGasFunc(skip int) func(GasCharge) {
func (rt *Runtime) chargeGasInternal(gas GasCharge, skip int) aerrors.ActorError {
toUse := gas.Total()
if EnableGasTracing {
var callers [10]uintptr
var callers [10]uintptr
cout := gruntime.Callers(2+skip, callers[:])
cout := 0 //gruntime.Callers(2+skip, callers[:])
now := build.Clock.Now()
if rt.lastGasCharge != nil {
rt.lastGasCharge.TimeTaken = now.Sub(rt.lastGasChargeTime)
}
gasTrace := types.GasTrace{
Name: gas.Name,
Extra: gas.Extra,
TotalGas: toUse,
ComputeGas: gas.ComputeGas,
StorageGas: gas.StorageGas,
TotalVirtualGas: gas.VirtualCompute*GasComputeMulti + gas.VirtualStorage*GasStorageMulti,
VirtualComputeGas: gas.VirtualCompute,
VirtualStorageGas: gas.VirtualStorage,
Callers: callers[:cout],
}
rt.executionTrace.GasCharges = append(rt.executionTrace.GasCharges, &gasTrace)
rt.lastGasChargeTime = now
rt.lastGasCharge = &gasTrace
now := build.Clock.Now()
if rt.lastGasCharge != nil {
rt.lastGasCharge.TimeTaken = now.Sub(rt.lastGasChargeTime)
}
gasTrace := types.GasTrace{
Name: gas.Name,
Extra: gas.Extra,
TotalGas: toUse,
ComputeGas: gas.ComputeGas,
StorageGas: gas.StorageGas,
TotalVirtualGas: gas.VirtualCompute*GasComputeMulti + gas.VirtualStorage*GasStorageMulti,
VirtualComputeGas: gas.VirtualCompute,
VirtualStorageGas: gas.VirtualStorage,
Callers: callers[:cout],
}
rt.executionTrace.GasCharges = append(rt.executionTrace.GasCharges, &gasTrace)
rt.lastGasChargeTime = now
rt.lastGasCharge = &gasTrace
// overflow safe
if rt.gasUsed > rt.gasAvailable-toUse {
rt.gasUsed = rt.gasAvailable
-20
View File
@@ -45,23 +45,3 @@ func TestRuntimePutErrors(t *testing.T) {
rt.StorePut(&NotAVeryGoodMarshaler{})
t.Error("expected panic")
}
func BenchmarkRuntime_CreateRuntimeChargeGas_TracingDisabled(b *testing.B) {
var (
cst = cbor.NewCborStore(nil)
gch = newGasCharge("foo", 1000, 1000)
)
b.ResetTimer()
EnableGasTracing = false
noop := func() bool { return EnableGasTracing }
for n := 0; n < b.N; n++ {
// flip the value and access it to make sure
// the compiler doesn't optimize away
EnableGasTracing = true
_ = noop()
EnableGasTracing = false
_ = (&Runtime{cst: cst}).chargeGasInternal(gch, 0)
}
}
+7 -35
View File
@@ -131,15 +131,7 @@ func (vm *VM) makeRuntime(ctx context.Context, msg *types.Message, origin addres
rt.Abortf(exitcode.SysErrInvalidReceiver, "resolve msg.From address failed")
}
vmm.From = resF
if vm.ntwkVersion(ctx, vm.blockHeight) <= network.Version3 {
rt.Message = &vmm
} else {
resT, _ := rt.ResolveAddress(msg.To)
// may be set to undef if recipient doesn't exist yet
vmm.To = resT
rt.Message = &Message{msg: vmm}
}
rt.Message = vmm
return rt
}
@@ -236,21 +228,14 @@ func (vm *VM) send(ctx context.Context, msg *types.Message, parent *Runtime,
}
rt := vm.makeRuntime(ctx, msg, origin, on, gasUsed, nac)
if EnableGasTracing {
rt.lastGasChargeTime = start
if parent != nil {
rt.lastGasChargeTime = parent.lastGasChargeTime
rt.lastGasCharge = parent.lastGasCharge
defer func() {
parent.lastGasChargeTime = rt.lastGasChargeTime
parent.lastGasCharge = rt.lastGasCharge
}()
}
}
rt.lastGasChargeTime = start
if parent != nil {
rt.lastGasChargeTime = parent.lastGasChargeTime
rt.lastGasCharge = parent.lastGasCharge
defer func() {
parent.gasUsed = rt.gasUsed
parent.lastGasChargeTime = rt.lastGasChargeTime
parent.lastGasCharge = rt.lastGasCharge
}()
}
if gasCharge != nil {
@@ -265,24 +250,11 @@ func (vm *VM) send(ctx context.Context, msg *types.Message, parent *Runtime,
toActor, err := st.GetActor(msg.To)
if err != nil {
if xerrors.Is(err, types.ErrActorNotFound) {
a, aid, err := TryCreateAccountActor(rt, msg.To)
a, err := TryCreateAccountActor(rt, msg.To)
if err != nil {
return nil, aerrors.Wrapf(err, "could not create account")
}
toActor = a
if vm.ntwkVersion(ctx, vm.blockHeight) <= network.Version3 {
// Leave the rt.Message as is
} else {
nmsg := Message{
msg: types.Message{
To: aid,
From: rt.Message.Caller(),
Value: rt.Message.ValueReceived(),
},
}
rt.Message = &nmsg
}
} else {
return nil, aerrors.Escalate(err, "getting actor")
}
+1 -47
View File
@@ -81,7 +81,7 @@ func (w *Wallet) findKey(addr address.Address) (*Key, error) {
return nil, nil
}
ki, err := w.tryFind(addr)
ki, err := w.keystore.Get(KNamePrefix + addr.String())
if err != nil {
if xerrors.Is(err, types.ErrKeyInfoNotFound) {
return nil, nil
@@ -96,42 +96,6 @@ func (w *Wallet) findKey(addr address.Address) (*Key, error) {
return k, nil
}
func (w *Wallet) tryFind(addr address.Address) (types.KeyInfo, error) {
ki, err := w.keystore.Get(KNamePrefix + addr.String())
if err == nil {
return ki, err
}
if !xerrors.Is(err, types.ErrKeyInfoNotFound) {
return types.KeyInfo{}, err
}
// We got an ErrKeyInfoNotFound error
// Try again, this time with the testnet prefix
aChars := []rune(addr.String())
prefixRunes := []rune(address.TestnetPrefix)
if len(prefixRunes) != 1 {
return types.KeyInfo{}, xerrors.Errorf("unexpected prefix length: %d", len(prefixRunes))
}
aChars[0] = prefixRunes[0]
ki, err = w.keystore.Get(KNamePrefix + string(aChars))
if err != nil {
return types.KeyInfo{}, err
}
// We found it with the testnet prefix
// Add this KeyInfo with the mainnet prefix address string
err = w.keystore.Put(KNamePrefix+addr.String(), ki)
if err != nil {
return types.KeyInfo{}, err
}
return ki, nil
}
func (w *Wallet) Export(addr address.Address) (*types.KeyInfo, error) {
k, err := w.findKey(addr)
if err != nil {
@@ -165,7 +129,6 @@ func (w *Wallet) ListAddrs() ([]address.Address, error) {
sort.Strings(all)
seen := map[address.Address]struct{}{}
out := make([]address.Address, 0, len(all))
for _, a := range all {
if strings.HasPrefix(a, KNamePrefix) {
@@ -174,19 +137,10 @@ func (w *Wallet) ListAddrs() ([]address.Address, error) {
if err != nil {
return nil, xerrors.Errorf("converting name to address: %w", err)
}
if _, ok := seen[addr]; ok {
continue // got duplicate with a different prefix
}
seen[addr] = struct{}{}
out = append(out, addr)
}
}
sort.Slice(out, func(i, j int) bool {
return out[i].String() < out[j].String()
})
return out, nil
}
-125
View File
@@ -1,125 +0,0 @@
package cli
import (
"context"
"fmt"
"os"
logging "github.com/ipfs/go-log/v2"
"github.com/mitchellh/go-homedir"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-jsonrpc"
"github.com/filecoin-project/lotus/lib/backupds"
"github.com/filecoin-project/lotus/node/repo"
)
type BackupAPI interface {
CreateBackup(ctx context.Context, fpath string) error
}
type BackupApiFn func(ctx *cli.Context) (BackupAPI, jsonrpc.ClientCloser, error)
func BackupCmd(repoFlag string, rt repo.RepoType, getApi BackupApiFn) *cli.Command {
var offlineBackup = func(cctx *cli.Context) error {
logging.SetLogLevel("badger", "ERROR") // nolint:errcheck
repoPath := cctx.String(repoFlag)
r, err := repo.NewFS(repoPath)
if err != nil {
return err
}
ok, err := r.Exists()
if err != nil {
return err
}
if !ok {
return xerrors.Errorf("repo at '%s' is not initialized", cctx.String(repoFlag))
}
lr, err := r.LockRO(rt)
if err != nil {
return xerrors.Errorf("locking repo: %w", err)
}
defer lr.Close() // nolint:errcheck
mds, err := lr.Datastore("/metadata")
if err != nil {
return xerrors.Errorf("getting metadata datastore: %w", err)
}
bds := backupds.Wrap(mds)
fpath, err := homedir.Expand(cctx.Args().First())
if err != nil {
return xerrors.Errorf("expanding file path: %w", err)
}
out, err := os.OpenFile(fpath, os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return xerrors.Errorf("opening backup file %s: %w", fpath, err)
}
if err := bds.Backup(out); err != nil {
if cerr := out.Close(); cerr != nil {
log.Errorw("error closing backup file while handling backup error", "closeErr", cerr, "backupErr", err)
}
return xerrors.Errorf("backup error: %w", err)
}
if err := out.Close(); err != nil {
return xerrors.Errorf("closing backup file: %w", err)
}
return nil
}
var onlineBackup = func(cctx *cli.Context) error {
api, closer, err := getApi(cctx)
if err != nil {
return xerrors.Errorf("getting api: %w (if the node isn't running you can use the --offline flag)", err)
}
defer closer()
err = api.CreateBackup(ReqContext(cctx), cctx.Args().First())
if err != nil {
return err
}
fmt.Println("Success")
return nil
}
return &cli.Command{
Name: "backup",
Usage: "Create node metadata backup",
Description: `The backup command writes a copy of node metadata under the specified path
Online backups:
For security reasons, the daemon must be have LOTUS_BACKUP_BASE_PATH env var set
to a path where backup files are supposed to be saved, and the path specified in
this command must be within this base path`,
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "offline",
Usage: "create backup without the node running",
},
},
ArgsUsage: "[backup file path]",
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() != 1 {
return xerrors.Errorf("expected 1 argument")
}
if cctx.Bool("offline") {
return offlineBackup(cctx)
}
return onlineBackup(cctx)
},
}
}
+6 -158
View File
@@ -8,7 +8,6 @@ import (
"os"
"os/exec"
"path"
"sort"
"strconv"
"strings"
"time"
@@ -28,10 +27,9 @@ import (
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
lapi "github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/stmgr"
types "github.com/filecoin-project/lotus/chain/types"
)
@@ -52,7 +50,6 @@ var chainCmd = &cli.Command{
chainExportCmd,
slashConsensusFault,
chainGasPriceCmd,
chainInspectUsage,
},
}
@@ -162,7 +159,7 @@ var chainGetBlock = &cli.Command{
},
}
func apiMsgCids(in []lapi.Message) []cid.Cid {
func apiMsgCids(in []api.Message) []cid.Cid {
out := make([]cid.Cid, len(in))
for k, v := range in {
out[k] = v.Cid
@@ -378,148 +375,6 @@ var chainSetHeadCmd = &cli.Command{
},
}
var chainInspectUsage = &cli.Command{
Name: "inspect-usage",
Usage: "Inspect block space usage of a given tipset",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "tipset",
Usage: "specify tipset to view block space usage of",
Value: "@head",
},
&cli.IntFlag{
Name: "length",
Usage: "length of chain to inspect block space usage for",
Value: 1,
},
&cli.IntFlag{
Name: "num-results",
Usage: "number of results to print per category",
Value: 10,
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer closer()
ctx := ReqContext(cctx)
ts, err := LoadTipSet(ctx, cctx, api)
if err != nil {
return err
}
cur := ts
var msgs []lapi.Message
for i := 0; i < cctx.Int("length"); i++ {
pmsgs, err := api.ChainGetParentMessages(ctx, cur.Blocks()[0].Cid())
if err != nil {
return err
}
msgs = append(msgs, pmsgs...)
next, err := api.ChainGetTipSet(ctx, cur.Parents())
if err != nil {
return err
}
cur = next
}
codeCache := make(map[address.Address]cid.Cid)
lookupActorCode := func(a address.Address) (cid.Cid, error) {
c, ok := codeCache[a]
if ok {
return c, nil
}
act, err := api.StateGetActor(ctx, a, ts.Key())
if err != nil {
return cid.Undef, err
}
codeCache[a] = act.Code
return act.Code, nil
}
bySender := make(map[string]int64)
byDest := make(map[string]int64)
byMethod := make(map[string]int64)
bySenderC := make(map[string]int64)
byDestC := make(map[string]int64)
byMethodC := make(map[string]int64)
var sum int64
for _, m := range msgs {
bySender[m.Message.From.String()] += m.Message.GasLimit
bySenderC[m.Message.From.String()]++
byDest[m.Message.To.String()] += m.Message.GasLimit
byDestC[m.Message.To.String()]++
sum += m.Message.GasLimit
code, err := lookupActorCode(m.Message.To)
if err != nil {
return err
}
mm := stmgr.MethodsMap[code][m.Message.Method]
byMethod[mm.Name] += m.Message.GasLimit
byMethodC[mm.Name]++
}
type keyGasPair struct {
Key string
Gas int64
}
mapToSortedKvs := func(m map[string]int64) []keyGasPair {
var vals []keyGasPair
for k, v := range m {
vals = append(vals, keyGasPair{
Key: k,
Gas: v,
})
}
sort.Slice(vals, func(i, j int) bool {
return vals[i].Gas > vals[j].Gas
})
return vals
}
senderVals := mapToSortedKvs(bySender)
destVals := mapToSortedKvs(byDest)
methodVals := mapToSortedKvs(byMethod)
numRes := cctx.Int("num-results")
fmt.Printf("Total Gas Limit: %d\n", sum)
fmt.Printf("By Sender:\n")
for i := 0; i < numRes && i < len(senderVals); i++ {
sv := senderVals[i]
fmt.Printf("%s\t%0.2f%%\t(total: %d, count: %d)\n", sv.Key, (100*float64(sv.Gas))/float64(sum), sv.Gas, bySenderC[sv.Key])
}
fmt.Println()
fmt.Printf("By Receiver:\n")
for i := 0; i < numRes && i < len(destVals); i++ {
sv := destVals[i]
fmt.Printf("%s\t%0.2f%%\t(total: %d, count: %d)\n", sv.Key, (100*float64(sv.Gas))/float64(sum), sv.Gas, byDestC[sv.Key])
}
fmt.Println()
fmt.Printf("By Method:\n")
for i := 0; i < numRes && i < len(methodVals); i++ {
sv := methodVals[i]
fmt.Printf("%s\t%0.2f%%\t(total: %d, count: %d)\n", sv.Key, (100*float64(sv.Gas))/float64(sum), sv.Gas, byMethodC[sv.Key])
}
return nil
},
}
var chainListCmd = &cli.Command{
Name: "list",
Usage: "View a segment of the chain",
@@ -793,7 +648,7 @@ var chainGetCmd = &cli.Command{
type apiIpldStore struct {
ctx context.Context
api lapi.FullNode
api api.FullNode
}
func (ht *apiIpldStore) Context() context.Context {
@@ -821,7 +676,7 @@ func (ht *apiIpldStore) Put(ctx context.Context, v interface{}) (cid.Cid, error)
panic("No mutations allowed")
}
func handleAmt(ctx context.Context, api lapi.FullNode, r cid.Cid) error {
func handleAmt(ctx context.Context, api api.FullNode, r cid.Cid) error {
s := &apiIpldStore{ctx, api}
mp, err := adt.AsArray(s, r)
if err != nil {
@@ -834,7 +689,7 @@ func handleAmt(ctx context.Context, api lapi.FullNode, r cid.Cid) error {
})
}
func handleHamtEpoch(ctx context.Context, api lapi.FullNode, r cid.Cid) error {
func handleHamtEpoch(ctx context.Context, api api.FullNode, r cid.Cid) error {
s := &apiIpldStore{ctx, api}
mp, err := adt.AsMap(s, r)
if err != nil {
@@ -852,7 +707,7 @@ func handleHamtEpoch(ctx context.Context, api lapi.FullNode, r cid.Cid) error {
})
}
func handleHamtAddress(ctx context.Context, api lapi.FullNode, r cid.Cid) error {
func handleHamtAddress(ctx context.Context, api api.FullNode, r cid.Cid) error {
s := &apiIpldStore{ctx, api}
mp, err := adt.AsMap(s, r)
if err != nil {
@@ -1075,20 +930,13 @@ var chainExportCmd = &cli.Command{
return err
}
var last bool
for b := range stream {
last = len(b) == 0
_, err := fi.Write(b)
if err != nil {
return err
}
}
if !last {
return xerrors.Errorf("incomplete export (remote connection lost?)")
}
return nil
},
}
+17 -79
View File
@@ -12,12 +12,13 @@ import (
"text/tabwriter"
"time"
"github.com/filecoin-project/specs-actors/actors/builtin"
tm "github.com/buger/goterm"
"github.com/docker/go-units"
"github.com/fatih/color"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-cidutil/cidenc"
"github.com/libp2p/go-libp2p-core/peer"
@@ -475,7 +476,6 @@ func interactiveDeal(cctx *cli.Context) error {
var ask storagemarket.StorageAsk
var epochPrice big.Int
var epochs abi.ChainEpoch
var verified bool
var a address.Address
if from := cctx.String("from"); from != "" {
@@ -536,7 +536,7 @@ func interactiveDeal(cctx *cli.Context) error {
state = "miner"
case "miner":
fmt.Print("Miner Address (f0..): ")
fmt.Print("Miner Address (t0..): ")
var maddrStr string
_, err := fmt.Scan(&maddrStr)
@@ -569,56 +569,9 @@ func interactiveDeal(cctx *cli.Context) error {
continue
}
ask = *a
ask = *a.Ask
// TODO: run more validation
state = "verified"
case "verified":
ts, err := api.ChainHead(ctx)
if err != nil {
return err
}
dcap, err := api.StateVerifiedClientStatus(ctx, a, ts.Key())
if err != nil {
return err
}
if dcap == nil {
state = "confirm"
continue
}
color.Blue(".. checking verified deal eligibility\n")
ds, err := api.ClientDealSize(ctx, data)
if err != nil {
return err
}
if dcap.Uint64() < uint64(ds.PieceSize) {
color.Yellow(".. not enough DataCap available for a verified deal\n")
state = "confirm"
continue
}
fmt.Print("\nMake this a verified deal? (yes/no): ")
var yn string
_, err = fmt.Scan(&yn)
if err != nil {
return err
}
switch yn {
case "yes":
verified = true
case "no":
verified = false
default:
fmt.Println("Type in full 'yes' or 'no'")
continue
}
state = "confirm"
case "confirm":
fromBal, err := api.WalletBalance(ctx, a)
@@ -637,15 +590,10 @@ func interactiveDeal(cctx *cli.Context) error {
epochs = abi.ChainEpoch(dur / (time.Duration(build.BlockDelaySecs) * time.Second))
// TODO: do some more or epochs math (round to miner PP, deal start buffer)
pricePerGib := ask.Price
if verified {
pricePerGib = ask.VerifiedPrice
}
gib := types.NewInt(1 << 30)
// TODO: price is based on PaddedPieceSize, right?
epochPrice = types.BigDiv(types.BigMul(pricePerGib, types.NewInt(uint64(ds.PieceSize))), gib)
epochPrice = types.BigDiv(types.BigMul(ask.Price, types.NewInt(uint64(ds.PieceSize))), gib)
totalPrice := types.BigMul(epochPrice, types.NewInt(uint64(epochs)))
fmt.Printf("-----\n")
@@ -655,7 +603,6 @@ func interactiveDeal(cctx *cli.Context) error {
fmt.Printf("Piece size: %s (Payload size: %s)\n", units.BytesSize(float64(ds.PieceSize)), units.BytesSize(float64(ds.PayloadSize)))
fmt.Printf("Duration: %s\n", dur)
fmt.Printf("Total price: ~%s (%s per epoch)\n", types.FIL(totalPrice), types.FIL(epochPrice))
fmt.Printf("Verified: %v\n", verified)
state = "accept"
case "accept":
@@ -690,7 +637,7 @@ func interactiveDeal(cctx *cli.Context) error {
MinBlocksDuration: uint64(epochs),
DealStartEpoch: abi.ChainEpoch(cctx.Int64("start-epoch")),
FastRetrieval: cctx.Bool("fast-retrieval"),
VerifiedDeal: verified,
VerifiedDeal: false, // TODO: Allow setting
})
if err != nil {
return err
@@ -1004,15 +951,15 @@ var clientQueryAskCmd = &cli.Command{
}
fmt.Printf("Ask: %s\n", maddr)
fmt.Printf("Price per GiB: %s\n", types.FIL(ask.Price))
fmt.Printf("Verified Price per GiB: %s\n", types.FIL(ask.VerifiedPrice))
fmt.Printf("Max Piece size: %s\n", types.SizeStr(types.NewInt(uint64(ask.MaxPieceSize))))
fmt.Printf("Price per GiB: %s\n", types.FIL(ask.Ask.Price))
fmt.Printf("Verified Price per GiB: %s\n", types.FIL(ask.Ask.VerifiedPrice))
fmt.Printf("Max Piece size: %s\n", types.SizeStr(types.NewInt(uint64(ask.Ask.MaxPieceSize))))
size := cctx.Int64("size")
if size == 0 {
return nil
}
perEpoch := types.BigDiv(types.BigMul(ask.Price, types.NewInt(uint64(size))), types.NewInt(1<<30))
perEpoch := types.BigDiv(types.BigMul(ask.Ask.Price, types.NewInt(uint64(size))), types.NewInt(1<<30))
fmt.Printf("Price per Block: %s\n", types.FIL(perEpoch))
duration := cctx.Int64("duration")
@@ -1039,10 +986,6 @@ var clientListDeals = &cli.Command{
Usage: "use color in display output",
Value: true,
},
&cli.BoolFlag{
Name: "show-failed",
Usage: "show failed/failing deals",
},
&cli.BoolFlag{
Name: "watch",
Usage: "watch deal updates in real-time, rather than a one time list",
@@ -1059,7 +1002,6 @@ var clientListDeals = &cli.Command{
verbose := cctx.Bool("verbose")
color := cctx.Bool("color")
watch := cctx.Bool("watch")
showFailed := cctx.Bool("show-failed")
localDeals, err := api.ClientListDeals(ctx)
if err != nil {
@@ -1076,7 +1018,7 @@ var clientListDeals = &cli.Command{
tm.Clear()
tm.MoveCursor(1, 1)
err = outputStorageDeals(ctx, tm.Screen, api, localDeals, verbose, color, showFailed)
err = outputStorageDeals(ctx, tm.Screen, api, localDeals, verbose, color)
if err != nil {
return err
}
@@ -1102,7 +1044,7 @@ var clientListDeals = &cli.Command{
}
}
return outputStorageDeals(ctx, os.Stdout, api, localDeals, cctx.Bool("verbose"), cctx.Bool("color"), showFailed)
return outputStorageDeals(ctx, os.Stdout, api, localDeals, cctx.Bool("verbose"), cctx.Bool("color"))
},
}
@@ -1125,7 +1067,7 @@ func dealFromDealInfo(ctx context.Context, full api.FullNode, head *types.TipSet
}
}
func outputStorageDeals(ctx context.Context, out io.Writer, full lapi.FullNode, localDeals []lapi.DealInfo, verbose bool, color bool, showFailed bool) error {
func outputStorageDeals(ctx context.Context, out io.Writer, full api.FullNode, localDeals []api.DealInfo, verbose bool, color bool) error {
sort.Slice(localDeals, func(i, j int) bool {
return localDeals[i].CreationTime.Before(localDeals[j].CreationTime)
})
@@ -1137,14 +1079,12 @@ func outputStorageDeals(ctx context.Context, out io.Writer, full lapi.FullNode,
var deals []deal
for _, localDeal := range localDeals {
if showFailed || localDeal.State != storagemarket.StorageDealError {
deals = append(deals, dealFromDealInfo(ctx, full, head, localDeal))
}
deals = append(deals, dealFromDealInfo(ctx, full, head, localDeal))
}
if verbose {
w := tabwriter.NewWriter(out, 2, 4, 2, ' ', 0)
fmt.Fprintf(w, "Created\tDealCid\tDealId\tProvider\tState\tOn Chain?\tSlashed?\tPieceCID\tSize\tPrice\tDuration\tVerified\tMessage\n")
fmt.Fprintf(w, "Created\tDealCid\tDealId\tProvider\tState\tOn Chain?\tSlashed?\tPieceCID\tSize\tPrice\tDuration\tMessage\n")
for _, d := range deals {
onChain := "N"
if d.OnChainDealState.SectorStartEpoch != -1 {
@@ -1157,7 +1097,7 @@ func outputStorageDeals(ctx context.Context, out io.Writer, full lapi.FullNode,
}
price := types.FIL(types.BigMul(d.LocalDeal.PricePerEpoch, types.NewInt(d.LocalDeal.Duration)))
fmt.Fprintf(w, "%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%v\t%s\n", d.LocalDeal.CreationTime.Format(time.Stamp), d.LocalDeal.ProposalCid, d.LocalDeal.DealID, d.LocalDeal.Provider, dealStateString(color, d.LocalDeal.State), onChain, slashed, d.LocalDeal.PieceCID, types.SizeStr(types.NewInt(d.LocalDeal.Size)), price, d.LocalDeal.Duration, d.LocalDeal.Verified, d.LocalDeal.Message)
fmt.Fprintf(w, "%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%s\n", d.LocalDeal.CreationTime.Format(time.Stamp), d.LocalDeal.ProposalCid, d.LocalDeal.DealID, d.LocalDeal.Provider, dealStateString(color, d.LocalDeal.State), onChain, slashed, d.LocalDeal.PieceCID, types.SizeStr(types.NewInt(d.LocalDeal.Size)), price, d.LocalDeal.Duration, d.LocalDeal.Message)
}
return w.Flush()
}
@@ -1172,7 +1112,6 @@ func outputStorageDeals(ctx context.Context, out io.Writer, full lapi.FullNode,
tablewriter.Col("Size"),
tablewriter.Col("Price"),
tablewriter.Col("Duration"),
tablewriter.Col("Verified"),
tablewriter.NewLineCol("Message"))
for _, d := range deals {
@@ -1202,7 +1141,6 @@ func outputStorageDeals(ctx context.Context, out io.Writer, full lapi.FullNode,
"PieceCID": piece,
"Size": types.SizeStr(types.NewInt(d.LocalDeal.Size)),
"Price": price,
"Verified": d.LocalDeal.Verified,
"Duration": d.LocalDeal.Duration,
"Message": d.LocalDeal.Message,
})
@@ -1485,7 +1423,7 @@ func toChannelOutput(useColor bool, otherPartyColumn string, channel lapi.DataTr
otherPartyColumn: otherParty,
"Root Cid": rootCid,
"Initiated?": initiated,
"Transferred": units.BytesSize(float64(channel.Transferred)),
"Transferred": channel.Transferred,
"Voucher": voucher,
"Message": channel.Message,
}
+17 -55
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"net/http"
"net/url"
"os"
"os/signal"
"strings"
@@ -48,44 +47,14 @@ func NewCliError(s string) error {
type ApiConnector func() api.FullNode
type APIInfo struct {
Addr string
Addr multiaddr.Multiaddr
Token []byte
}
func (a APIInfo) DialArgs() (string, error) {
ma, err := multiaddr.NewMultiaddr(a.Addr)
if err == nil {
_, addr, err := manet.DialArgs(ma)
if err != nil {
return "", err
}
_, addr, err := manet.DialArgs(a.Addr)
return "ws://" + addr + "/rpc/v0", nil
}
_, err = url.Parse(a.Addr)
if err != nil {
return "", err
}
return a.Addr + "/rpc/v0", nil
}
func (a APIInfo) Host() (string, error) {
ma, err := multiaddr.NewMultiaddr(a.Addr)
if err == nil {
_, addr, err := manet.DialArgs(ma)
if err != nil {
return "", err
}
return addr, nil
}
spec, err := url.Parse(a.Addr)
if err != nil {
return "", err
}
return spec.Host, nil
return "ws://" + addr + "/rpc/v0", err
}
func (a APIInfo) AuthHeader() http.Header {
@@ -103,11 +72,11 @@ func (a APIInfo) AuthHeader() http.Header {
func flagForAPI(t repo.RepoType) string {
switch t {
case repo.FullNode:
return "api-url"
return "api"
case repo.StorageMiner:
return "miner-api-url"
return "miner-api"
case repo.Worker:
return "worker-api-url"
return "worker-api"
default:
panic(fmt.Sprintf("Unknown repo type: %v", t))
}
@@ -161,7 +130,11 @@ func GetAPIInfo(ctx *cli.Context, t repo.RepoType) (APIInfo, error) {
strma := ctx.String(apiFlag)
strma = strings.TrimSpace(strma)
return APIInfo{Addr: strma}, nil
apima, err := multiaddr.NewMultiaddr(strma)
if err != nil {
return APIInfo{}, err
}
return APIInfo{Addr: apima}, nil
}
envKey := envForRepo(t)
@@ -179,8 +152,12 @@ func GetAPIInfo(ctx *cli.Context, t repo.RepoType) (APIInfo, error) {
if len(sp) != 2 {
log.Warnf("invalid env(%s) value, missing token or address", envKey)
} else {
ma, err := multiaddr.NewMultiaddr(sp[1])
if err != nil {
return APIInfo{}, xerrors.Errorf("could not parse multiaddr from env(%s): %w", envKey, err)
}
return APIInfo{
Addr: sp[1],
Addr: ma,
Token: []byte(sp[0]),
}, nil
}
@@ -209,7 +186,7 @@ func GetAPIInfo(ctx *cli.Context, t repo.RepoType) (APIInfo, error) {
}
return APIInfo{
Addr: ma.String(),
Addr: ma,
Token: token,
}, nil
}
@@ -239,13 +216,6 @@ func GetAPI(ctx *cli.Context) (api.Common, jsonrpc.ClientCloser, error) {
log.Errorf("repoType type does not match the type of repo.RepoType")
}
if tn, ok := ctx.App.Metadata["testnode-storage"]; ok {
return tn.(api.StorageMiner), func() {}, nil
}
if tn, ok := ctx.App.Metadata["testnode-full"]; ok {
return tn.(api.FullNode), func() {}, nil
}
addr, headers, err := GetRawAPI(ctx, t)
if err != nil {
return nil, nil, err
@@ -255,10 +225,6 @@ func GetAPI(ctx *cli.Context) (api.Common, jsonrpc.ClientCloser, error) {
}
func GetFullNodeAPI(ctx *cli.Context) (api.FullNode, jsonrpc.ClientCloser, error) {
if tn, ok := ctx.App.Metadata["testnode-full"]; ok {
return tn.(api.FullNode), func() {}, nil
}
addr, headers, err := GetRawAPI(ctx, repo.FullNode)
if err != nil {
return nil, nil, err
@@ -268,10 +234,6 @@ func GetFullNodeAPI(ctx *cli.Context) (api.FullNode, jsonrpc.ClientCloser, error
}
func GetStorageMinerAPI(ctx *cli.Context, opts ...jsonrpc.Option) (api.StorageMiner, jsonrpc.ClientCloser, error) {
if tn, ok := ctx.App.Metadata["testnode-storage"]; ok {
return tn.(api.StorageMiner), func() {}, nil
}
addr, headers, err := GetRawAPI(ctx, repo.StorageMiner)
if err != nil {
return nil, nil, err
+40 -94
View File
@@ -7,7 +7,6 @@ import (
"sort"
"strconv"
cid "github.com/ipfs/go-cid"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
@@ -44,10 +43,6 @@ var mpoolPending = &cli.Command{
Name: "local",
Usage: "print pending messages for addresses in local wallet only",
},
&cli.BoolFlag{
Name: "cids",
Usage: "only print cids of messages in output",
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
@@ -84,15 +79,11 @@ var mpoolPending = &cli.Command{
}
}
if cctx.Bool("cids") {
fmt.Println(msg.Cid())
} else {
out, err := json.MarshalIndent(msg, "", " ")
if err != nil {
return err
}
fmt.Println(string(out))
out, err := json.MarshalIndent(msg, "", " ")
if err != nil {
return err
}
fmt.Println(string(out))
}
return nil
@@ -164,6 +155,14 @@ var mpoolSub = &cli.Command{
},
}
type statBucket struct {
msgs map[uint64]*types.SignedMessage
}
type mpStat struct {
addr string
past, cur, future uint64
}
var mpoolStat = &cli.Command{
Name: "stat",
Usage: "print mempool stats",
@@ -172,11 +171,6 @@ var mpoolStat = &cli.Command{
Name: "local",
Usage: "print stats for addresses in local wallet only",
},
&cli.IntFlag{
Name: "basefee-lookback",
Usage: "number of blocks to look back for minimum basefee",
Value: 60,
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
@@ -191,20 +185,6 @@ var mpoolStat = &cli.Command{
if err != nil {
return xerrors.Errorf("getting chain head: %w", err)
}
currBF := ts.Blocks()[0].ParentBaseFee
minBF := currBF
{
currTs := ts
for i := 0; i < cctx.Int("basefee-lookback"); i++ {
currTs, err = api.ChainGetTipSet(ctx, currTs.Parents())
if err != nil {
return xerrors.Errorf("walking chain: %w", err)
}
if newBF := currTs.Blocks()[0].ParentBaseFee; newBF.LessThan(minBF) {
minBF = newBF
}
}
}
var filter map[address.Address]struct{}
if cctx.Bool("local") {
@@ -225,16 +205,8 @@ var mpoolStat = &cli.Command{
return err
}
type statBucket struct {
msgs map[uint64]*types.SignedMessage
}
type mpStat struct {
addr string
past, cur, future uint64
belowCurr, belowPast uint64
}
buckets := map[address.Address]*statBucket{}
for _, v := range msgs {
if filter != nil {
if _, has := filter[v.Message.From]; !has {
@@ -271,27 +243,23 @@ var mpoolStat = &cli.Command{
cur++
}
var s mpStat
s.addr = a.String()
past := uint64(0)
future := uint64(0)
for _, m := range bkt.msgs {
if m.Message.Nonce < act.Nonce {
s.past++
} else if m.Message.Nonce > cur {
s.future++
} else {
s.cur++
past++
}
if m.Message.GasFeeCap.LessThan(currBF) {
s.belowCurr++
}
if m.Message.GasFeeCap.LessThan(minBF) {
s.belowPast++
if m.Message.Nonce > cur {
future++
}
}
out = append(out, s)
out = append(out, mpStat{
addr: a.String(),
past: past,
cur: cur - act.Nonce,
future: future,
})
}
sort.Slice(out, func(i, j int) bool {
@@ -304,14 +272,12 @@ var mpoolStat = &cli.Command{
total.past += stat.past
total.cur += stat.cur
total.future += stat.future
total.belowCurr += stat.belowCurr
total.belowPast += stat.belowPast
fmt.Printf("%s: Nonce past: %d, cur: %d, future: %d; FeeCap cur: %d, min-%d: %d \n", stat.addr, stat.past, stat.cur, stat.future, stat.belowCurr, cctx.Int("basefee-lookback"), stat.belowPast)
fmt.Printf("%s: past: %d, cur: %d, future: %d\n", stat.addr, stat.past, stat.cur, stat.future)
}
fmt.Println("-----")
fmt.Printf("total: Nonce past: %d, cur: %d, future: %d; FeeCap cur: %d, min-%d: %d \n", total.past, total.cur, total.future, total.belowCurr, cctx.Int("basefee-lookback"), total.belowPast)
fmt.Printf("total: past: %d, cur: %d, future: %d\n", total.past, total.cur, total.future)
return nil
},
@@ -342,8 +308,21 @@ var mpoolReplaceCmd = &cli.Command{
Usage: "Spend up to X FIL for this message (applicable for auto mode)",
},
},
ArgsUsage: "<from nonce> | <message-cid>",
ArgsUsage: "[from] [nonce]",
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() < 2 {
return cli.ShowCommandHelp(cctx, cctx.Command.Name)
}
from, err := address.NewFromString(cctx.Args().Get(0))
if err != nil {
return err
}
nonce, err := strconv.ParseUint(cctx.Args().Get(1), 10, 64)
if err != nil {
return err
}
api, closer, err := GetFullNodeAPI(cctx)
if err != nil {
@@ -353,39 +332,6 @@ var mpoolReplaceCmd = &cli.Command{
ctx := ReqContext(cctx)
var from address.Address
var nonce uint64
switch cctx.Args().Len() {
case 1:
mcid, err := cid.Decode(cctx.Args().First())
if err != nil {
return err
}
msg, err := api.ChainGetMessage(ctx, mcid)
if err != nil {
return fmt.Errorf("could not find referenced message: %w", err)
}
from = msg.From
nonce = msg.Nonce
case 2:
f, err := address.NewFromString(cctx.Args().Get(0))
if err != nil {
return err
}
n, err := strconv.ParseUint(cctx.Args().Get(1), 10, 64)
if err != nil {
return err
}
from = f
nonce = n
default:
return cli.ShowCommandHelp(cctx, cctx.Command.Name)
}
ts, err := api.ChainHead(ctx)
if err != nil {
return xerrors.Errorf("getting chain head: %w", err)
+5 -5
View File
@@ -439,12 +439,12 @@ type mockCLI struct {
}
func newMockCLI(t *testing.T) *mockCLI {
// Create a CLI App with an --api-url flag so that we can specify which node
// Create a CLI App with an --api flag so that we can specify which node
// the command should be executed against
app := cli.NewApp()
app.Flags = []cli.Flag{
&cli.StringFlag{
Name: "api-url",
Name: "api",
Hidden: true,
},
}
@@ -476,8 +476,8 @@ func (c *mockCLIClient) runCmd(cmd *cli.Command, input []string) string {
}
func (c *mockCLIClient) runCmdRaw(cmd *cli.Command, input []string) (string, error) {
// prepend --api-url=<node api listener address>
apiFlag := "--api-url=" + c.addr.String()
// prepend --api=<node api listener address>
apiFlag := "--api=" + c.addr.String()
input = append([]string{apiFlag}, input...)
fs := c.flagSet(cmd)
@@ -493,7 +493,7 @@ func (c *mockCLIClient) runCmdRaw(cmd *cli.Command, input []string) (string, err
}
func (c *mockCLIClient) flagSet(cmd *cli.Command) *flag.FlagSet {
// Apply app level flags (so we can process --api-url flag)
// Apply app level flags (so we can process --api flag)
fs := &flag.FlagSet{}
for _, f := range c.cctx.App.Flags {
err := f.Apply(fs)
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/node/repo"
manet "github.com/multiformats/go-multiaddr/net"
)
var pprofCmd = &cli.Command{
@@ -36,7 +37,7 @@ var PprofGoroutines = &cli.Command{
if err != nil {
return xerrors.Errorf("could not get API info: %w", err)
}
addr, err := ainfo.Host()
_, addr, err := manet.DialArgs(ainfo.Addr)
if err != nil {
return err
}
+3 -95
View File
@@ -7,7 +7,6 @@ import (
"fmt"
"html/template"
"io"
"io/ioutil"
"os"
"reflect"
"sort"
@@ -33,7 +32,6 @@ import (
"github.com/filecoin-project/go-state-types/exitcode"
"github.com/filecoin-project/lotus/api"
lapi "github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apibstore"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
@@ -72,7 +70,6 @@ var stateCmd = &cli.Command{
stateMsgCostCmd,
stateMinerInfo,
stateMarketCmd,
stateExecTraceCmd,
},
}
@@ -316,74 +313,6 @@ var stateActiveSectorsCmd = &cli.Command{
},
}
var stateExecTraceCmd = &cli.Command{
Name: "exec-trace",
Usage: "Get the execution trace of a given message",
ArgsUsage: "<messageCid>",
Action: func(cctx *cli.Context) error {
if !cctx.Args().Present() {
return ShowHelp(cctx, fmt.Errorf("must pass message cid"))
}
mcid, err := cid.Decode(cctx.Args().First())
if err != nil {
return fmt.Errorf("message cid was invalid: %s", err)
}
capi, closer, err := GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer closer()
ctx := ReqContext(cctx)
msg, err := capi.ChainGetMessage(ctx, mcid)
if err != nil {
return err
}
lookup, err := capi.StateSearchMsg(ctx, mcid)
if err != nil {
return err
}
ts, err := capi.ChainGetTipSet(ctx, lookup.TipSet)
if err != nil {
return err
}
pts, err := capi.ChainGetTipSet(ctx, ts.Parents())
if err != nil {
return err
}
cso, err := capi.StateCompute(ctx, pts.Height(), nil, pts.Key())
if err != nil {
return err
}
var trace *api.InvocResult
for _, t := range cso.Trace {
if t.Msg.From == msg.From && t.Msg.Nonce == msg.Nonce {
trace = t
break
}
}
if trace == nil {
return fmt.Errorf("failed to find message in tipset trace output")
}
out, err := json.MarshalIndent(trace, "", " ")
if err != nil {
return err
}
fmt.Println(string(out))
return nil
},
}
var stateReplaySetCmd = &cli.Command{
Name: "replay",
Usage: "Replay a particular message within a tipset",
@@ -894,10 +823,6 @@ var stateComputeStateCmd = &cli.Command{
Name: "json",
Usage: "generate json output",
},
&cli.StringFlag{
Name: "compute-state-output",
Usage: "a json file containing pre-existing compute-state output, to generate html reports without rerunning state changes",
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
@@ -937,26 +862,9 @@ var stateComputeStateCmd = &cli.Command{
}
}
var stout *lapi.ComputeStateOutput
if csofile := cctx.String("compute-state-output"); csofile != "" {
data, err := ioutil.ReadFile(csofile)
if err != nil {
return err
}
var o lapi.ComputeStateOutput
if err := json.Unmarshal(data, &o); err != nil {
return err
}
stout = &o
} else {
o, err := api.StateCompute(ctx, h, msgs, ts.Key())
if err != nil {
return err
}
stout = o
stout, err := api.StateCompute(ctx, h, msgs, ts.Key())
if err != nil {
return err
}
if cctx.Bool("json") {
+3 -9
View File
@@ -233,13 +233,7 @@ func SyncWait(ctx context.Context, napi api.FullNode) error {
samples := 8
i := 0
var firstApp, app, lastApp uint64
state, err := napi.SyncState(ctx)
if err != nil {
return err
}
firstApp = state.VMApplied
var app, lastApp uint64
for {
state, err := napi.SyncState(ctx)
@@ -292,10 +286,10 @@ func SyncWait(ctx context.Context, napi api.FullNode) error {
if i%samples == 0 {
lastApp = app
app = state.VMApplied - firstApp
app = state.VMApplied
}
if i > 0 {
fmt.Printf("Validated %d messages (%d per second)\n", state.VMApplied-firstApp, (app-lastApp)*uint64(time.Second/tick)/uint64(samples))
fmt.Printf("Validated %d messages (%d per second)\n", state.VMApplied, (app-lastApp)*uint64(time.Second/tick)/uint64(samples))
lastLines++
}
+2 -3
View File
@@ -5,7 +5,6 @@ import (
"fmt"
"time"
"github.com/hako/durafmt"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/go-state-types/abi"
@@ -37,11 +36,11 @@ func parseTipSet(ctx context.Context, api api.FullNode, vals []string) (*types.T
func EpochTime(curr, e abi.ChainEpoch) string {
switch {
case curr > e:
return fmt.Sprintf("%d (%s ago)", e, durafmt.Parse(time.Second*time.Duration(int64(build.BlockDelaySecs)*int64(curr-e))).LimitFirstN(2))
return fmt.Sprintf("%d (%s ago)", e, time.Second*time.Duration(int64(build.BlockDelaySecs)*int64(curr-e)))
case curr == e:
return fmt.Sprintf("%d (now)", e)
case curr < e:
return fmt.Sprintf("%d (in %s)", e, durafmt.Parse(time.Second*time.Duration(int64(build.BlockDelaySecs)*int64(e-curr))).LimitFirstN(2))
return fmt.Sprintf("%d (in %s)", e, time.Second*time.Duration(int64(build.BlockDelaySecs)*int64(e-curr)))
}
panic("math broke")
+4 -57
View File
@@ -9,16 +9,13 @@ import (
"os"
"strings"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
types "github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/lotus/lib/tablewriter"
"golang.org/x/xerrors"
"github.com/urfave/cli/v2"
)
var walletCmd = &cli.Command{
@@ -69,13 +66,6 @@ var walletNew = &cli.Command{
var walletList = &cli.Command{
Name: "list",
Usage: "List wallet address",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "addr-only",
Usage: "Only print addresses",
Aliases: []string{"a"},
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
if err != nil {
@@ -89,52 +79,9 @@ var walletList = &cli.Command{
return err
}
// Assume an error means no default key is set
def, _ := api.WalletDefaultAddress(ctx)
tw := tablewriter.New(
tablewriter.Col("Address"),
tablewriter.Col("Balance"),
tablewriter.Col("Nonce"),
tablewriter.Col("Default"),
tablewriter.NewLineCol("Error"))
for _, addr := range addrs {
if cctx.Bool("addr-only") {
fmt.Println(addr.String())
} else {
a, err := api.StateGetActor(ctx, addr, types.EmptyTSK)
if err != nil {
if !strings.Contains(err.Error(), "actor not found") {
tw.Write(map[string]interface{}{
"Address": addr,
"Error": err,
})
continue
}
a = &types.Actor{
Balance: big.Zero(),
}
}
row := map[string]interface{}{
"Address": addr,
"Balance": types.FIL(a.Balance),
"Nonce": a.Nonce,
}
if addr == def {
row["Default"] = "X"
}
tw.Write(row)
}
fmt.Println(addr.String())
}
if !cctx.Bool("addr-only") {
return tw.Flush(os.Stdout)
}
return nil
},
}
-98
View File
@@ -1,98 +0,0 @@
package main
import (
"bufio"
"context"
"errors"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/ipfs/go-datastore"
"github.com/minio/blake2b-simd"
cbg "github.com/whyrusleeping/cbor-gen"
)
type cachingVerifier struct {
ds datastore.Datastore
backend ffiwrapper.Verifier
}
const bufsize = 128
func (cv cachingVerifier) withCache(execute func() (bool, error), param cbg.CBORMarshaler) (bool, error) {
hasher := blake2b.New256()
wr := bufio.NewWriterSize(hasher, bufsize)
err := param.MarshalCBOR(wr)
if err != nil {
log.Errorf("could not marshal call info: %+v", err)
return execute()
}
err = wr.Flush()
if err != nil {
log.Errorf("could not flush: %+v", err)
return execute()
}
hash := hasher.Sum(nil)
key := datastore.NewKey(string(hash))
fromDs, err := cv.ds.Get(key)
if err == nil {
switch fromDs[0] {
case 's':
return true, nil
case 'f':
return false, nil
case 'e':
return false, errors.New(string(fromDs[1:]))
default:
log.Errorf("bad cached result in cache %s(%x)", fromDs[0], fromDs[0])
return execute()
}
} else if errors.Is(err, datastore.ErrNotFound) {
// recalc
ok, err := execute()
var save []byte
if err != nil {
if ok {
log.Errorf("success with an error: %+v", err)
} else {
save = append([]byte{'e'}, []byte(err.Error())...)
}
} else if ok {
save = []byte{'s'}
} else {
save = []byte{'f'}
}
if len(save) != 0 {
errSave := cv.ds.Put(key, save)
if errSave != nil {
log.Errorf("error saving result: %+v", errSave)
}
}
return ok, err
} else {
log.Errorf("could not get data from cache: %+v", err)
return execute()
}
}
func (cv *cachingVerifier) VerifySeal(svi proof.SealVerifyInfo) (bool, error) {
return cv.withCache(func() (bool, error) {
return cv.backend.VerifySeal(svi)
}, &svi)
}
func (cv *cachingVerifier) VerifyWinningPoSt(ctx context.Context, info proof.WinningPoStVerifyInfo) (bool, error) {
return cv.backend.VerifyWinningPoSt(ctx, info)
}
func (cv *cachingVerifier) VerifyWindowPoSt(ctx context.Context, info proof.WindowPoStVerifyInfo) (bool, error) {
return cv.withCache(func() (bool, error) {
return cv.backend.VerifyWindowPoSt(ctx, info)
}, &info)
}
func (cv *cachingVerifier) GenerateWinningPoStSectorChallenge(ctx context.Context, proofType abi.RegisteredPoStProof, a abi.ActorID, rnd abi.PoStRandomness, u uint64) ([]uint64, error) {
return cv.backend.GenerateWinningPoStSectorChallenge(ctx, proofType, a, rnd, u)
}
var _ ffiwrapper.Verifier = (*cachingVerifier)(nil)
+43 -165
View File
@@ -16,8 +16,6 @@ import (
"sort"
"time"
"github.com/cockroachdb/pebble"
"github.com/cockroachdb/pebble/bloom"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/store"
@@ -26,16 +24,12 @@ import (
"github.com/filecoin-project/lotus/lib/blockstore"
_ "github.com/filecoin-project/lotus/lib/sigs/bls"
_ "github.com/filecoin-project/lotus/lib/sigs/secp"
"github.com/ipld/go-car"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
bdg "github.com/dgraph-io/badger/v2"
"github.com/ipfs/go-datastore"
badger "github.com/ipfs/go-ds-badger2"
pebbleds "github.com/ipfs/go-ds-pebble"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
)
@@ -66,29 +60,6 @@ var importBenchCmd = &cli.Command{
Name: "repodir",
Usage: "set the repo directory for the lotus bench run (defaults to /tmp)",
},
&cli.StringFlag{
Name: "syscall-cache",
Usage: "read and write syscall results from datastore",
},
&cli.BoolFlag{
Name: "export-traces",
Usage: "should we export execution traces",
Value: true,
},
&cli.BoolFlag{
Name: "no-import",
Usage: "should we import the chain? if set to true chain has to be previously imported",
},
&cli.BoolFlag{
Name: "global-profile",
Value: true,
},
&cli.Int64Flag{
Name: "start-at",
},
&cli.BoolFlag{
Name: "only-import",
},
},
Action: func(cctx *cli.Context) error {
vm.BatchSealVerifyParallelism = cctx.Int("batch-seal-verify-threads")
@@ -103,10 +74,6 @@ var importBenchCmd = &cli.Command{
}
defer cfi.Close() //nolint:errcheck // read only file
go func() {
http.ListenAndServe("localhost:6060", nil) //nolint:errcheck
}()
var tdir string
if rdir := cctx.String("repodir"); rdir != "" {
tdir = rdir
@@ -118,105 +85,33 @@ var importBenchCmd = &cli.Command{
tdir = tmp
}
bdgOpt := badger.DefaultOptions
bdgOpt.GcInterval = 0
bdgOpt.Options = bdg.DefaultOptions("")
bdgOpt.Options.SyncWrites = false
bdgOpt.Options.Truncate = true
bdgOpt.Options.DetectConflicts = false
var bds datastore.Batching
if false {
cache := 512
bds, err = pebbleds.NewDatastore(tdir, &pebble.Options{
// Pebble has a single combined cache area and the write
// buffers are taken from this too. Assign all available
// memory allowance for cache.
Cache: pebble.NewCache(int64(cache * 1024 * 1024)),
// The size of memory table(as well as the write buffer).
// Note, there may have more than two memory tables in the system.
// MemTableStopWritesThreshold can be configured to avoid the memory abuse.
MemTableSize: cache * 1024 * 1024 / 4,
// The default compaction concurrency(1 thread),
// Here use all available CPUs for faster compaction.
MaxConcurrentCompactions: runtime.NumCPU(),
// Per-level options. Options for at least one level must be specified. The
// options for the last level are used for all subsequent levels.
Levels: []pebble.LevelOptions{
{TargetFileSize: 16 * 1024 * 1024, FilterPolicy: bloom.FilterPolicy(10), Compression: pebble.NoCompression},
},
Logger: log,
})
} else {
bds, err = badger.NewDatastore(tdir, &bdgOpt)
}
bds, err := badger.NewDatastore(tdir, nil)
if err != nil {
return err
}
defer bds.Close() //nolint:errcheck
bs := blockstore.NewBlockstore(bds)
cacheOpts := blockstore.DefaultCacheOpts()
cacheOpts.HasBloomFilterSize = 0
cbs, err := blockstore.CachedBlockstore(context.TODO(), bs, cacheOpts)
cbs, err := blockstore.CachedBlockstore(context.TODO(), bs, blockstore.DefaultCacheOpts())
if err != nil {
return err
}
bs = cbs
ds := datastore.NewMapDatastore()
var verifier ffiwrapper.Verifier = ffiwrapper.ProofVerifier
if cctx.IsSet("syscall-cache") {
scds, err := badger.NewDatastore(cctx.String("syscall-cache"), &bdgOpt)
if err != nil {
return xerrors.Errorf("opening syscall-cache datastore: %w", err)
}
defer scds.Close() //nolint:errcheck
verifier = &cachingVerifier{
ds: scds,
backend: verifier,
}
}
if cctx.Bool("only-gc") {
return nil
}
cs := store.NewChainStore(bs, ds, vm.Syscalls(verifier))
cs := store.NewChainStore(bs, ds, vm.Syscalls(ffiwrapper.ProofVerifier))
stm := stmgr.NewStateManager(cs)
if cctx.Bool("global-profile") {
prof, err := os.Create("import-bench.prof")
if err != nil {
return err
}
defer prof.Close() //nolint:errcheck
prof, err := os.Create("import-bench.prof")
if err != nil {
return err
}
defer prof.Close() //nolint:errcheck
if err := pprof.StartCPUProfile(prof); err != nil {
return err
}
if err := pprof.StartCPUProfile(prof); err != nil {
return err
}
var head *types.TipSet
if !cctx.Bool("no-import") {
head, err = cs.Import(cfi)
if err != nil {
return err
}
} else {
cr, err := car.NewCarReader(cfi)
if err != nil {
return err
}
head, err = cs.LoadTipSet(types.NewTipSetKey(cr.Header.Roots...))
if err != nil {
return err
}
}
if cctx.Bool("only-import") {
return nil
head, err := cs.Import(cfi)
if err != nil {
return err
}
gb, err := cs.GetTipsetByHeight(context.TODO(), 0, head, true)
@@ -229,20 +124,6 @@ var importBenchCmd = &cli.Command{
return err
}
startEpoch := abi.ChainEpoch(1)
if cctx.IsSet("start-at") {
startEpoch = abi.ChainEpoch(cctx.Int64("start-at"))
start, err := cs.GetTipsetByHeight(context.TODO(), abi.ChainEpoch(cctx.Int64("start-at")), head, true)
if err != nil {
return err
}
err = cs.SetHead(start)
if err != nil {
return err
}
}
if h := cctx.Int64("height"); h != 0 {
tsh, err := cs.GetTipsetByHeight(context.TODO(), abi.ChainEpoch(h), head, true)
if err != nil {
@@ -253,7 +134,7 @@ var importBenchCmd = &cli.Command{
ts := head
tschain := []*types.TipSet{ts}
for ts.Height() > startEpoch {
for ts.Height() != 0 {
next, err := cs.LoadTipSet(ts.Parents())
if err != nil {
return err
@@ -263,48 +144,45 @@ var importBenchCmd = &cli.Command{
ts = next
}
var enc *json.Encoder
if cctx.Bool("export-traces") {
ibj, err := os.Create("import-bench.json")
if err != nil {
return err
}
defer ibj.Close() //nolint:errcheck
enc = json.NewEncoder(ibj)
ibj, err := os.Create("import-bench.json")
if err != nil {
return err
}
defer ibj.Close() //nolint:errcheck
for i := len(tschain) - 1; i >= 1; i-- {
enc := json.NewEncoder(ibj)
var lastTse *TipSetExec
lastState := tschain[len(tschain)-1].ParentState()
for i := len(tschain) - 2; i >= 0; i-- {
cur := tschain[i]
start := time.Now()
log.Infof("computing state (height: %d, ts=%s)", cur.Height(), cur.Cids())
st, trace, err := stm.ExecutionTrace(context.TODO(), cur)
if err != nil {
return err
}
tse := &TipSetExec{
TipSet: cur.Key(),
Trace: trace,
Duration: time.Since(start),
}
if enc != nil {
stripCallers(tse.Trace)
if err := enc.Encode(tse); err != nil {
return xerrors.Errorf("failed to write out tipsetexec: %w", err)
}
}
if tschain[i-1].ParentState() != st {
stripCallers(tse.Trace)
lastTrace := tse.Trace
if cur.ParentState() != lastState {
lastTrace := lastTse.Trace
d, err := json.MarshalIndent(lastTrace, "", " ")
if err != nil {
panic(err)
}
fmt.Println("TRACE")
fmt.Println(string(d))
//fmt.Println(statediff.Diff(context.Background(), bs, tschain[i-1].ParentState(), st, statediff.ExpandActors))
return xerrors.Errorf("tipset chain had state mismatch at height %d (%s != %s)", cur.Height(), cur.ParentState(), st)
return xerrors.Errorf("tipset chain had state mismatch at height %d (%s != %s)", cur.Height(), cur.ParentState(), lastState)
}
start := time.Now()
st, trace, err := stm.ExecutionTrace(context.TODO(), cur)
if err != nil {
return err
}
stripCallers(trace)
lastTse = &TipSetExec{
TipSet: cur.Key(),
Trace: trace,
Duration: time.Since(start),
}
lastState = st
if err := enc.Encode(lastTse); err != nil {
return xerrors.Errorf("failed to write out tipsetexec: %w", err)
}
}
+1 -1
View File
@@ -86,5 +86,5 @@ func (a *GatewayAPI) MpoolPush(ctx context.Context, sm *types.SignedMessage) (ci
// TODO: additional anti-spam checks
return a.api.MpoolPushUntrusted(ctx, sm)
return a.api.MpoolPush(ctx, sm)
}
+130 -798
View File
File diff suppressed because it is too large Load Diff
-7
View File
@@ -109,11 +109,6 @@ var runCmd = &cli.Command{
Name: "no-local-storage",
Usage: "don't use storageminer repo for sector storage",
},
&cli.BoolFlag{
Name: "no-swap",
Usage: "don't use swap",
Value: false,
},
&cli.BoolFlag{
Name: "addpiece",
Usage: "enable addpiece",
@@ -351,7 +346,6 @@ var runCmd = &cli.Command{
LocalWorker: sectorstorage.NewLocalWorker(sectorstorage.WorkerConfig{
SealProof: spt,
TaskTypes: taskTypes,
NoSwap: cctx.Bool("no-swap"),
}, remote, localStore, nodeApi),
localStore: localStore,
ls: lr,
@@ -471,7 +465,6 @@ func watchMinerConn(ctx context.Context, cctx *cli.Context, nodeApi api.StorageM
"run",
fmt.Sprintf("--listen=%s", cctx.String("listen")),
fmt.Sprintf("--no-local-storage=%t", cctx.Bool("no-local-storage")),
fmt.Sprintf("--no-swap=%t", cctx.Bool("no-swap")),
fmt.Sprintf("--addpiece=%t", cctx.Bool("addpiece")),
fmt.Sprintf("--precommit1=%t", cctx.Bool("precommit1")),
fmt.Sprintf("--unseal=%t", cctx.Bool("unseal")),
+25 -59
View File
@@ -7,7 +7,6 @@ import (
"github.com/docker/go-units"
"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/actors/builtin/power"
"github.com/filecoin-project/lotus/chain/actors/builtin/reward"
@@ -34,19 +33,16 @@ import (
)
type accountInfo struct {
Address address.Address
Balance types.FIL
Type string
Power abi.StoragePower
Worker address.Address
Owner address.Address
InitialPledge types.FIL
PreCommits types.FIL
LockedFunds types.FIL
Sectors uint64
VestingStart abi.ChainEpoch
VestingDuration abi.ChainEpoch
VestingAmount types.FIL
Address address.Address
Balance types.FIL
Type string
Power abi.StoragePower
Worker address.Address
Owner address.Address
InitialPledge types.FIL
PreCommits types.FIL
LockedFunds types.FIL
Sectors uint64
}
var auditsCmd = &cli.Command{
@@ -119,8 +115,10 @@ var chainBalanceCmd = &cli.Command{
infos = append(infos, ai)
}
printAccountInfos(infos, false)
fmt.Printf("Address,Balance,Type,Power,Worker,Owner\n")
for _, acc := range infos {
fmt.Printf("%s,%s,%s,%s,%s,%s\n", acc.Address, acc.Balance, acc.Type, acc.Power, acc.Worker, acc.Owner)
}
return nil
},
}
@@ -198,7 +196,6 @@ var chainBalanceStateCmd = &cli.Command{
LockedFunds: types.FIL(big.NewInt(0)),
InitialPledge: types.FIL(big.NewInt(0)),
PreCommits: types.FIL(big.NewInt(0)),
VestingAmount: types.FIL(big.NewInt(0)),
}
if minerInfo && builtin.IsStorageMinerActor(act.Code) {
@@ -237,32 +234,6 @@ var chainBalanceStateCmd = &cli.Command{
ai.Worker = minfo.Worker
ai.Owner = minfo.Owner
}
if builtin.IsMultisigActor(act.Code) {
mst, err := multisig.Load(store, act)
if err != nil {
return err
}
ai.VestingStart, err = mst.StartEpoch()
if err != nil {
return err
}
ib, err := mst.InitialBalance()
if err != nil {
return err
}
ai.VestingAmount = types.FIL(ib)
ai.VestingDuration, err = mst.UnlockDuration()
if err != nil {
return err
}
}
infos = append(infos, ai)
return nil
})
@@ -270,27 +241,22 @@ var chainBalanceStateCmd = &cli.Command{
return xerrors.Errorf("failed to loop over actors: %w", err)
}
printAccountInfos(infos, minerInfo)
if minerInfo {
fmt.Printf("Address,Balance,Type,Sectors,Worker,Owner,InitialPledge,Locked,PreCommits\n")
for _, acc := range infos {
fmt.Printf("%s,%s,%s,%d,%s,%s,%s,%s,%s\n", acc.Address, acc.Balance, acc.Type, acc.Sectors, acc.Worker, acc.Owner, acc.InitialPledge, acc.LockedFunds, acc.PreCommits)
}
} else {
fmt.Printf("Address,Balance,Type\n")
for _, acc := range infos {
fmt.Printf("%s,%s,%s\n", acc.Address, acc.Balance, acc.Type)
}
}
return nil
},
}
func printAccountInfos(infos []accountInfo, minerInfo bool) {
if minerInfo {
fmt.Printf("Address,Balance,Type,Sectors,Worker,Owner,InitialPledge,Locked,PreCommits,VestingStart,VestingDuration,VestingAmount\n")
for _, acc := range infos {
fmt.Printf("%s,%s,%s,%d,%s,%s,%s,%s,%s,%d,%d,%s\n", acc.Address, acc.Balance.Unitless(), acc.Type, acc.Sectors, acc.Worker, acc.Owner, acc.InitialPledge.Unitless(), acc.LockedFunds.Unitless(), acc.PreCommits.Unitless(), acc.VestingStart, acc.VestingDuration, acc.VestingAmount.Unitless())
}
} else {
fmt.Printf("Address,Balance,Type\n")
for _, acc := range infos {
fmt.Printf("%s,%s,%s\n", acc.Address, acc.Balance.Unitless(), acc.Type)
}
}
}
var chainPledgeCmd = &cli.Command{
Name: "stateroot-pledge",
Description: "Calculate sector pledge numbers",
+1 -1
View File
@@ -111,7 +111,7 @@ var consensusCheckCmd = &cli.Command{
if err != nil {
return err
}
ainfo := lcli.APIInfo{Addr: apima.String()}
ainfo := lcli.APIInfo{Addr: apima}
addr, err := ainfo.DialArgs()
if err != nil {
return err
-290
View File
@@ -1,290 +0,0 @@
package main
import (
"encoding/json"
"fmt"
"os"
"strings"
"github.com/docker/go-units"
"github.com/ipfs/go-datastore"
dsq "github.com/ipfs/go-datastore/query"
logging "github.com/ipfs/go-log"
"github.com/polydawn/refmt/cbor"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/lib/backupds"
"github.com/filecoin-project/lotus/node/repo"
)
var datastoreCmd = &cli.Command{
Name: "datastore",
Description: "access node datastores directly",
Subcommands: []*cli.Command{
datastoreBackupCmd,
datastoreListCmd,
datastoreGetCmd,
},
}
var datastoreListCmd = &cli.Command{
Name: "list",
Description: "list datastore keys",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "repo-type",
Usage: "node type (1 - full, 2 - storage, 3 - worker)",
Value: 1,
},
&cli.BoolFlag{
Name: "top-level",
Usage: "only print top-level keys",
},
&cli.StringFlag{
Name: "get-enc",
Usage: "print values [esc/hex/cbor]",
},
},
ArgsUsage: "[namespace prefix]",
Action: func(cctx *cli.Context) error {
logging.SetLogLevel("badger", "ERROR") // nolint:errcheck
r, err := repo.NewFS(cctx.String("repo"))
if err != nil {
return xerrors.Errorf("opening fs repo: %w", err)
}
exists, err := r.Exists()
if err != nil {
return err
}
if !exists {
return xerrors.Errorf("lotus repo doesn't exist")
}
lr, err := r.Lock(repo.RepoType(cctx.Int("repo-type")))
if err != nil {
return err
}
defer lr.Close() //nolint:errcheck
ds, err := lr.Datastore(datastore.NewKey(cctx.Args().First()).String())
if err != nil {
return err
}
genc := cctx.String("get-enc")
q, err := ds.Query(dsq.Query{
Prefix: datastore.NewKey(cctx.Args().Get(1)).String(),
KeysOnly: genc == "",
})
if err != nil {
return xerrors.Errorf("datastore query: %w", err)
}
defer q.Close() //nolint:errcheck
printKv := kvPrinter(cctx.Bool("top-level"), genc)
for res := range q.Next() {
if err := printKv(res.Key, res.Value); err != nil {
return err
}
}
return nil
},
}
var datastoreGetCmd = &cli.Command{
Name: "get",
Description: "list datastore keys",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "repo-type",
Usage: "node type (1 - full, 2 - storage, 3 - worker)",
Value: 1,
},
&cli.StringFlag{
Name: "enc",
Usage: "encoding (esc/hex/cbor)",
Value: "esc",
},
},
ArgsUsage: "[namespace key]",
Action: func(cctx *cli.Context) error {
logging.SetLogLevel("badger", "ERROR") // nolint:errchec
r, err := repo.NewFS(cctx.String("repo"))
if err != nil {
return xerrors.Errorf("opening fs repo: %w", err)
}
exists, err := r.Exists()
if err != nil {
return err
}
if !exists {
return xerrors.Errorf("lotus repo doesn't exist")
}
lr, err := r.Lock(repo.RepoType(cctx.Int("repo-type")))
if err != nil {
return err
}
defer lr.Close() //nolint:errcheck
ds, err := lr.Datastore(datastore.NewKey(cctx.Args().First()).String())
if err != nil {
return err
}
val, err := ds.Get(datastore.NewKey(cctx.Args().Get(1)))
if err != nil {
return xerrors.Errorf("get: %w", err)
}
return printVal(cctx.String("enc"), val)
},
}
var datastoreBackupCmd = &cli.Command{
Name: "backup",
Description: "manage datastore backups",
Subcommands: []*cli.Command{
datastoreBackupStatCmd,
datastoreBackupListCmd,
},
}
var datastoreBackupStatCmd = &cli.Command{
Name: "stat",
Description: "validate and print info about datastore backup",
ArgsUsage: "[file]",
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() != 1 {
return xerrors.Errorf("expected 1 argument")
}
f, err := os.Open(cctx.Args().First())
if err != nil {
return xerrors.Errorf("opening backup file: %w", err)
}
defer f.Close() // nolint:errcheck
var keys, kbytes, vbytes uint64
err = backupds.ReadBackup(f, func(key datastore.Key, value []byte) error {
keys++
kbytes += uint64(len(key.String()))
vbytes += uint64(len(value))
return nil
})
if err != nil {
return err
}
fmt.Println("Keys: ", keys)
fmt.Println("Key bytes: ", units.BytesSize(float64(kbytes)))
fmt.Println("Value bytes: ", units.BytesSize(float64(vbytes)))
return err
},
}
var datastoreBackupListCmd = &cli.Command{
Name: "list",
Description: "list data in a backup",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "top-level",
Usage: "only print top-level keys",
},
&cli.StringFlag{
Name: "get-enc",
Usage: "print values [esc/hex/cbor]",
},
},
ArgsUsage: "[file]",
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() != 1 {
return xerrors.Errorf("expected 1 argument")
}
f, err := os.Open(cctx.Args().First())
if err != nil {
return xerrors.Errorf("opening backup file: %w", err)
}
defer f.Close() // nolint:errcheck
printKv := kvPrinter(cctx.Bool("top-level"), cctx.String("get-enc"))
err = backupds.ReadBackup(f, func(key datastore.Key, value []byte) error {
return printKv(key.String(), value)
})
if err != nil {
return err
}
return err
},
}
func kvPrinter(toplevel bool, genc string) func(sk string, value []byte) error {
seen := map[string]struct{}{}
return func(s string, value []byte) error {
if toplevel {
k := datastore.NewKey(datastore.NewKey(s).List()[0])
if k.Type() != "" {
s = k.Type()
} else {
s = k.String()
}
_, has := seen[s]
if has {
return nil
}
seen[s] = struct{}{}
}
s = fmt.Sprintf("%q", s)
s = strings.Trim(s, "\"")
fmt.Println(s)
if genc != "" {
fmt.Print("\t")
if err := printVal(genc, value); err != nil {
return err
}
}
return nil
}
}
func printVal(enc string, val []byte) error {
switch enc {
case "esc":
s := fmt.Sprintf("%q", string(val))
s = strings.Trim(s, "\"")
fmt.Println(s)
case "hex":
fmt.Printf("%x\n", val)
case "cbor":
var out interface{}
if err := cbor.Unmarshal(cbor.DecodeOptions{}, val, &out); err != nil {
return xerrors.Errorf("unmarshaling cbor: %w", err)
}
s, err := json.Marshal(&out)
if err != nil {
return xerrors.Errorf("remarshaling as json: %w", err)
}
fmt.Println(string(s))
default:
return xerrors.New("unknown encoding")
}
return nil
}
+104 -164
View File
@@ -5,10 +5,10 @@ import (
"encoding/json"
"net"
"net/http"
"sync"
"os"
"strings"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/api"
lcli "github.com/filecoin-project/lotus/cli"
"github.com/ipfs/go-cid"
@@ -19,54 +19,58 @@ type dealStatsServer struct {
api api.FullNode
}
// Requested by @jbenet
// How many epochs back to look at for dealstats
var epochLookback = abi.ChainEpoch(10)
// these lists grow continuously with the network
// TODO: need to switch this to an LRU of sorts, to ensure refreshes
var knownFiltered = new(sync.Map)
var resolvedWallets = new(sync.Map)
var filteredClients map[address.Address]bool
func init() {
for _, a := range []string{
"t0100", // client for genesis miner
"t0101", // client for genesis miner
"t0102", // client for genesis miner
"t0112", // client for genesis miner
"t0113", // client for genesis miner
"t0114", // client for genesis miner
"t1nslxql4pck5pq7hddlzym3orxlx35wkepzjkm3i", // SR1 dealbot wallet
"t1stghxhdp2w53dym2nz2jtbpk6ccd4l2lxgmezlq", // SR1 dealbot wallet
"t1mcr5xkgv4jdl3rnz77outn6xbmygb55vdejgbfi", // SR1 dealbot wallet
"t1qiqdbbmrdalbntnuapriirduvxu5ltsc5mhy7si", // SR1 dealbot wallet
} {
a, err := address.NewFromString(a)
fc := []string{"t0112", "t0113", "t0114", "t010089"}
filtered, set := os.LookupEnv("FILTERED_CLIENTS")
if set {
fc = strings.Split(filtered, ":")
}
filteredClients = make(map[address.Address]bool)
for _, a := range fc {
addr, err := address.NewFromString(a)
if err != nil {
panic(err)
}
knownFiltered.Store(a, true)
filteredClients[addr] = true
}
}
type dealCountResp struct {
Epoch int64 `json:"epoch"`
Endpoint string `json:"endpoint"`
Payload int64 `json:"payload"`
Total int64 `json:"total"`
Epoch int64 `json:"epoch"`
}
func (dss *dealStatsServer) handleStorageDealCount(w http.ResponseWriter, r *http.Request) {
ctx := context.Background()
epoch, deals := dss.filteredDealList()
if epoch == 0 {
head, err := dss.api.ChainHead(ctx)
if err != nil {
log.Warnf("failed to get chain head: %s", err)
w.WriteHeader(500)
return
}
deals, err := dss.api.StateMarketDeals(ctx, head.Key())
if err != nil {
log.Warnf("failed to get market deals: %s", err)
w.WriteHeader(500)
return
}
var count int64
for _, d := range deals {
if !filteredClients[d.Proposal.Client] {
count++
}
}
if err := json.NewEncoder(w).Encode(&dealCountResp{
Endpoint: "COUNT_DEALS",
Payload: int64(len(deals)),
Epoch: epoch,
Total: count,
Epoch: int64(head.Height()),
}); err != nil {
log.Warnf("failed to write back deal count response: %s", err)
return
@@ -74,28 +78,39 @@ func (dss *dealStatsServer) handleStorageDealCount(w http.ResponseWriter, r *htt
}
type dealAverageResp struct {
Epoch int64 `json:"epoch"`
Endpoint string `json:"endpoint"`
Payload int64 `json:"payload"`
AverageSize int64 `json:"average_size"`
Epoch int64 `json:"epoch"`
}
func (dss *dealStatsServer) handleStorageDealAverageSize(w http.ResponseWriter, r *http.Request) {
ctx := context.Background()
epoch, deals := dss.filteredDealList()
if epoch == 0 {
head, err := dss.api.ChainHead(ctx)
if err != nil {
log.Warnf("failed to get chain head: %s", err)
w.WriteHeader(500)
return
}
deals, err := dss.api.StateMarketDeals(ctx, head.Key())
if err != nil {
log.Warnf("failed to get market deals: %s", err)
w.WriteHeader(500)
return
}
var count int64
var totalBytes int64
for _, d := range deals {
totalBytes += int64(d.deal.Proposal.PieceSize.Unpadded())
if !filteredClients[d.Proposal.Client] {
count++
totalBytes += int64(d.Proposal.PieceSize.Unpadded())
}
}
if err := json.NewEncoder(w).Encode(&dealAverageResp{
Endpoint: "AVERAGE_DEAL_SIZE",
Payload: totalBytes / int64(len(deals)),
Epoch: epoch,
AverageSize: totalBytes / count,
Epoch: int64(head.Height()),
}); err != nil {
log.Warnf("failed to write back deal average response: %s", err)
return
@@ -103,27 +118,37 @@ func (dss *dealStatsServer) handleStorageDealAverageSize(w http.ResponseWriter,
}
type dealTotalResp struct {
Epoch int64 `json:"epoch"`
Endpoint string `json:"endpoint"`
Payload int64 `json:"payload"`
TotalBytes int64 `json:"total_size"`
Epoch int64 `json:"epoch"`
}
func (dss *dealStatsServer) handleStorageDealTotalReal(w http.ResponseWriter, r *http.Request) {
epoch, deals := dss.filteredDealList()
if epoch == 0 {
ctx := context.Background()
head, err := dss.api.ChainHead(ctx)
if err != nil {
log.Warnf("failed to get chain head: %s", err)
w.WriteHeader(500)
return
}
deals, err := dss.api.StateMarketDeals(ctx, head.Key())
if err != nil {
log.Warnf("failed to get market deals: %s", err)
w.WriteHeader(500)
return
}
var totalBytes int64
for _, d := range deals {
totalBytes += int64(d.deal.Proposal.PieceSize.Unpadded())
if !filteredClients[d.Proposal.Client] {
totalBytes += int64(d.Proposal.PieceSize.Unpadded())
}
}
if err := json.NewEncoder(w).Encode(&dealTotalResp{
Endpoint: "DEAL_BYTES",
Payload: totalBytes,
Epoch: epoch,
TotalBytes: totalBytes,
Epoch: int64(head.Height()),
}); err != nil {
log.Warnf("failed to write back deal average response: %s", err)
return
@@ -132,12 +157,6 @@ func (dss *dealStatsServer) handleStorageDealTotalReal(w http.ResponseWriter, r
}
type clientStatsOutput struct {
Epoch int64 `json:"epoch"`
Endpoint string `json:"endpoint"`
Payload []*clientStats `json:"payload"`
}
type clientStats struct {
Client address.Address `json:"client"`
DataSize int64 `json:"data_size"`
NumCids int `json:"num_cids"`
@@ -149,41 +168,51 @@ type clientStats struct {
}
func (dss *dealStatsServer) handleStorageClientStats(w http.ResponseWriter, r *http.Request) {
epoch, deals := dss.filteredDealList()
if epoch == 0 {
ctx := context.Background()
head, err := dss.api.ChainHead(ctx)
if err != nil {
log.Warnf("failed to get chain head: %s", err)
w.WriteHeader(500)
return
}
stats := make(map[address.Address]*clientStats)
deals, err := dss.api.StateMarketDeals(ctx, head.Key())
if err != nil {
log.Warnf("failed to get market deals: %s", err)
w.WriteHeader(500)
return
}
stats := make(map[address.Address]*clientStatsOutput)
for _, d := range deals {
if filteredClients[d.Proposal.Client] {
continue
}
st, ok := stats[d.deal.Proposal.Client]
st, ok := stats[d.Proposal.Client]
if !ok {
st = &clientStats{
Client: d.resolvedWallet,
st = &clientStatsOutput{
Client: d.Proposal.Client,
cids: make(map[cid.Cid]bool),
providers: make(map[address.Address]bool),
}
stats[d.deal.Proposal.Client] = st
stats[d.Proposal.Client] = st
}
st.DataSize += int64(d.deal.Proposal.PieceSize.Unpadded())
st.cids[d.deal.Proposal.PieceCID] = true
st.providers[d.deal.Proposal.Provider] = true
st.DataSize += int64(d.Proposal.PieceSize.Unpadded())
st.cids[d.Proposal.PieceCID] = true
st.providers[d.Proposal.Provider] = true
st.NumDeals++
}
out := clientStatsOutput{
Epoch: epoch,
Endpoint: "CLIENT_DEAL_STATS",
Payload: make([]*clientStats, 0, len(stats)),
}
for _, cs := range stats {
cs.NumCids = len(cs.cids)
cs.NumMiners = len(cs.providers)
out.Payload = append(out.Payload, cs)
out := make([]*clientStatsOutput, 0, len(stats))
for _, cso := range stats {
cso.NumCids = len(cso.cids)
cso.NumMiners = len(cso.providers)
out = append(out, cso)
}
if err := json.NewEncoder(w).Encode(out); err != nil {
@@ -192,93 +221,6 @@ func (dss *dealStatsServer) handleStorageClientStats(w http.ResponseWriter, r *h
}
}
type dealInfo struct {
deal api.MarketDeal
resolvedWallet address.Address
}
// filteredDealList returns the current epoch and a list of filtered deals
// on error returns an epoch of 0
func (dss *dealStatsServer) filteredDealList() (int64, map[string]dealInfo) {
ctx := context.Background()
head, err := dss.api.ChainHead(ctx)
if err != nil {
log.Warnf("failed to get chain head: %s", err)
return 0, nil
}
head, err = dss.api.ChainGetTipSetByHeight(ctx, head.Height()-epochLookback, head.Key())
if err != nil {
log.Warnf("failed to walk back %s epochs: %s", epochLookback, err)
return 0, nil
}
// Disabled as per @pooja's request
//
// // Exclude any address associated with a miner
// miners, err := dss.api.StateListMiners(ctx, head.Key())
// if err != nil {
// log.Warnf("failed to get miner list: %s", err)
// return 0, nil
// }
// for _, m := range miners {
// info, err := dss.api.StateMinerInfo(ctx, m, head.Key())
// if err != nil {
// log.Warnf("failed to get info for known miner '%s': %s", m, err)
// continue
// }
// knownFiltered.Store(info.Owner, true)
// knownFiltered.Store(info.Worker, true)
// for _, a := range info.ControlAddresses {
// knownFiltered.Store(a, true)
// }
// }
deals, err := dss.api.StateMarketDeals(ctx, head.Key())
if err != nil {
log.Warnf("failed to get market deals: %s", err)
return 0, nil
}
ret := make(map[string]dealInfo, len(deals))
for dealKey, d := range deals {
// Counting no-longer-active deals as per Pooja's request
// // https://github.com/filecoin-project/specs-actors/blob/v0.9.9/actors/builtin/market/deal.go#L81-L85
// if d.State.SectorStartEpoch < 0 {
// continue
// }
if _, isFiltered := knownFiltered.Load(d.Proposal.Client); isFiltered {
continue
}
if _, wasSeen := resolvedWallets.Load(d.Proposal.Client); !wasSeen {
w, err := dss.api.StateAccountKey(ctx, d.Proposal.Client, head.Key())
if err != nil {
log.Warnf("failed to resolve id '%s' to wallet address: %s", d.Proposal.Client, err)
continue
} else {
resolvedWallets.Store(d.Proposal.Client, w)
}
}
w, _ := resolvedWallets.Load(d.Proposal.Client)
if _, isFiltered := knownFiltered.Load(w); isFiltered {
continue
}
ret[dealKey] = dealInfo{
deal: d,
resolvedWallet: w.(address.Address),
}
}
return int64(head.Height()), ret
}
var serveDealStatsCmd = &cli.Command{
Name: "serve-deal-stats",
Flags: []cli.Flag{},
@@ -318,8 +260,6 @@ var serveDealStatsCmd = &cli.Command{
panic(err)
}
log.Warnf("deal-stat server listening on %s\n== NOTE: QUERIES ARE EXPENSIVE - YOU MUST FRONT-CACHE THIS SERVICE\n", list.Addr().String())
return s.Serve(list)
},
}
-57
View File
@@ -1,13 +1,10 @@
package main
import (
"encoding/hex"
"fmt"
"io"
"os"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
"github.com/ipld/go-car"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
@@ -84,57 +81,3 @@ var importCarCmd = &cli.Command{
}
},
}
var importObjectCmd = &cli.Command{
Name: "import-obj",
Usage: "import a raw ipld object into your datastore",
Action: func(cctx *cli.Context) error {
r, err := repo.NewFS(cctx.String("repo"))
if err != nil {
return xerrors.Errorf("opening fs repo: %w", err)
}
exists, err := r.Exists()
if err != nil {
return err
}
if !exists {
return xerrors.Errorf("lotus repo doesn't exist")
}
lr, err := r.Lock(repo.FullNode)
if err != nil {
return err
}
defer lr.Close() //nolint:errcheck
ds, err := lr.Datastore("/chain")
if err != nil {
return err
}
bs := blockstore.NewBlockstore(ds)
c, err := cid.Decode(cctx.Args().Get(0))
if err != nil {
return err
}
data, err := hex.DecodeString(cctx.Args().Get(1))
if err != nil {
return err
}
blk, err := block.NewBlockWithCid(data, c)
if err != nil {
return err
}
if err := bs.Put(blk); err != nil {
return err
}
return nil
},
}
-4
View File
@@ -25,7 +25,6 @@ func main() {
staterootCmd,
auditsCmd,
importCarCmd,
importObjectCmd,
commpToCidCmd,
fetchParamCmd,
proofsCmd,
@@ -38,9 +37,6 @@ func main() {
exportChainCmd,
consensusCmd,
serveDealStatsCmd,
syncCmd,
stateTreePruneCmd,
datastoreCmd,
}
app := &cli.App{
+4 -23
View File
@@ -5,8 +5,6 @@ import (
"math"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/urfave/cli/v2"
"github.com/filecoin-project/lotus/chain/types"
@@ -34,10 +32,6 @@ var noncefix = &cli.Command{
&cli.BoolFlag{
Name: "auto",
},
&cli.Int64Flag{
Name: "gas-fee-cap",
Usage: "specify gas fee cap for nonce filling messages",
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := lcli.GetFullNodeAPI(cctx)
@@ -90,25 +84,12 @@ var noncefix = &cli.Command{
}
fmt.Printf("Creating %d filler messages (%d ~ %d)\n", end-start, start, end)
ts, err := api.ChainHead(ctx)
if err != nil {
return err
}
feeCap := big.Mul(ts.Blocks()[0].ParentBaseFee, big.NewInt(2)) // default fee cap to 2 * parent base fee
if fcf := cctx.Int64("gas-fee-cap"); fcf != 0 {
feeCap = abi.NewTokenAmount(fcf)
}
for i := start; i < end; i++ {
msg := &types.Message{
From: addr,
To: addr,
Value: types.NewInt(0),
Nonce: i,
GasLimit: 1000000,
GasFeeCap: feeCap,
GasPremium: abi.NewTokenAmount(5),
From: addr,
To: addr,
Value: types.NewInt(1),
Nonce: i,
}
smsg, err := api.WalletSignMessage(ctx, addr, msg)
if err != nil {
-289
View File
@@ -1,289 +0,0 @@
package main
import (
"context"
"fmt"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/vm"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/lotus/node/repo"
"github.com/ipfs/bbloom"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/query"
dshelp "github.com/ipfs/go-ipfs-ds-help"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
)
type cidSet interface {
Add(cid.Cid)
Has(cid.Cid) bool
HasRaw([]byte) bool
Len() int
}
type bloomSet struct {
bloom *bbloom.Bloom
}
func newBloomSet(size int64) (*bloomSet, error) {
b, err := bbloom.New(float64(size), 3)
if err != nil {
return nil, err
}
return &bloomSet{bloom: b}, nil
}
func (bs *bloomSet) Add(c cid.Cid) {
bs.bloom.Add(c.Hash())
}
func (bs *bloomSet) Has(c cid.Cid) bool {
return bs.bloom.Has(c.Hash())
}
func (bs *bloomSet) HasRaw(b []byte) bool {
return bs.bloom.Has(b)
}
func (bs *bloomSet) Len() int {
return int(bs.bloom.ElementsAdded())
}
type mapSet struct {
m map[string]struct{}
}
func newMapSet() *mapSet {
return &mapSet{m: make(map[string]struct{})}
}
func (bs *mapSet) Add(c cid.Cid) {
bs.m[string(c.Hash())] = struct{}{}
}
func (bs *mapSet) Has(c cid.Cid) bool {
_, ok := bs.m[string(c.Hash())]
return ok
}
func (bs *mapSet) HasRaw(b []byte) bool {
_, ok := bs.m[string(b)]
return ok
}
func (bs *mapSet) Len() int {
return len(bs.m)
}
var stateTreePruneCmd = &cli.Command{
Name: "state-prune",
Description: "Deletes old state root data from local chainstore",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "repo",
Value: "~/.lotus",
},
&cli.Int64Flag{
Name: "keep-from-lookback",
Usage: "keep stateroots at or newer than the current height minus this lookback",
Value: 1800, // 2 x finality
},
&cli.IntFlag{
Name: "delete-up-to",
Usage: "delete up to the given number of objects (used to run a faster 'partial' sync)",
},
&cli.BoolFlag{
Name: "use-bloom-set",
Usage: "use a bloom filter for the 'good' set instead of a map, reduces memory usage but may not clean up as much",
},
&cli.BoolFlag{
Name: "dry-run",
Usage: "only enumerate the good set, don't do any deletions",
},
&cli.BoolFlag{
Name: "only-ds-gc",
Usage: "Only run datastore GC",
},
&cli.IntFlag{
Name: "gc-count",
Usage: "number of times to run gc",
Value: 20,
},
},
Action: func(cctx *cli.Context) error {
ctx := context.TODO()
fsrepo, err := repo.NewFS(cctx.String("repo"))
if err != nil {
return err
}
lkrepo, err := fsrepo.Lock(repo.FullNode)
if err != nil {
return err
}
defer lkrepo.Close() //nolint:errcheck
ds, err := lkrepo.Datastore("/chain")
if err != nil {
return err
}
defer ds.Close() //nolint:errcheck
mds, err := lkrepo.Datastore("/metadata")
if err != nil {
return err
}
defer mds.Close() //nolint:errcheck
if cctx.Bool("only-ds-gc") {
gcds, ok := ds.(datastore.GCDatastore)
if ok {
fmt.Println("running datastore gc....")
for i := 0; i < cctx.Int("gc-count"); i++ {
if err := gcds.CollectGarbage(); err != nil {
return xerrors.Errorf("datastore GC failed: %w", err)
}
}
fmt.Println("gc complete!")
return nil
}
return fmt.Errorf("datastore doesnt support gc")
}
bs := blockstore.NewBlockstore(ds)
cs := store.NewChainStore(bs, mds, vm.Syscalls(ffiwrapper.ProofVerifier))
if err := cs.Load(); err != nil {
return fmt.Errorf("loading chainstore: %w", err)
}
var goodSet cidSet
if cctx.Bool("use-bloom-set") {
bset, err := newBloomSet(10000000)
if err != nil {
return err
}
goodSet = bset
} else {
goodSet = newMapSet()
}
ts := cs.GetHeaviestTipSet()
rrLb := abi.ChainEpoch(cctx.Int64("keep-from-lookback"))
if err := cs.WalkSnapshot(ctx, ts, rrLb, true, func(c cid.Cid) error {
if goodSet.Len()%20 == 0 {
fmt.Printf("\renumerating keep set: %d ", goodSet.Len())
}
goodSet.Add(c)
return nil
}); err != nil {
return fmt.Errorf("snapshot walk failed: %w", err)
}
fmt.Println()
fmt.Printf("Successfully marked keep set! (%d objects)\n", goodSet.Len())
if cctx.Bool("dry-run") {
return nil
}
var b datastore.Batch
var batchCount int
markForRemoval := func(c cid.Cid) error {
if b == nil {
nb, err := ds.Batch()
if err != nil {
return fmt.Errorf("opening batch: %w", err)
}
b = nb
}
batchCount++
if err := b.Delete(dshelp.MultihashToDsKey(c.Hash())); err != nil {
return err
}
if batchCount > 100 {
if err := b.Commit(); err != nil {
return xerrors.Errorf("failed to commit batch deletes: %w", err)
}
b = nil
batchCount = 0
}
return nil
}
res, err := ds.Query(query.Query{KeysOnly: true})
if err != nil {
return xerrors.Errorf("failed to query datastore: %w", err)
}
dupTo := cctx.Int("delete-up-to")
var deleteCount int
var goodHits int
for {
v, ok := res.NextSync()
if !ok {
break
}
bk, err := dshelp.BinaryFromDsKey(datastore.RawKey(v.Key[len("/blocks"):]))
if err != nil {
return xerrors.Errorf("failed to parse key: %w", err)
}
if goodSet.HasRaw(bk) {
goodHits++
continue
}
nc := cid.NewCidV1(cid.Raw, bk)
deleteCount++
if err := markForRemoval(nc); err != nil {
return fmt.Errorf("failed to remove cid %s: %w", nc, err)
}
if deleteCount%20 == 0 {
fmt.Printf("\rdeleting %d objects (good hits: %d)... ", deleteCount, goodHits)
}
if dupTo != 0 && deleteCount > dupTo {
break
}
}
if b != nil {
if err := b.Commit(); err != nil {
return xerrors.Errorf("failed to commit final batch delete: %w", err)
}
}
gcds, ok := ds.(datastore.GCDatastore)
if ok {
fmt.Println("running datastore gc....")
for i := 0; i < cctx.Int("gc-count"); i++ {
if err := gcds.CollectGarbage(); err != nil {
return xerrors.Errorf("datastore GC failed: %w", err)
}
}
fmt.Println("gc complete!")
}
return nil
},
}
-64
View File
@@ -1,64 +0,0 @@
package main
import (
"fmt"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/types"
lcli "github.com/filecoin-project/lotus/cli"
"github.com/urfave/cli/v2"
)
var syncCmd = &cli.Command{
Name: "sync",
Usage: "tools for diagnosing sync issues",
Flags: []cli.Flag{},
Subcommands: []*cli.Command{
syncValidateCmd,
},
}
var syncValidateCmd = &cli.Command{
Name: "validate",
Usage: "checks whether a provided tipset is valid",
Action: func(cctx *cli.Context) error {
api, closer, err := lcli.GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer closer()
ctx := lcli.ReqContext(cctx)
if cctx.Args().Len() < 1 {
fmt.Println("usage: <blockCid1> <blockCid2>...")
fmt.Println("At least one block cid must be provided")
return nil
}
args := cctx.Args().Slice()
var tscids []cid.Cid
for _, s := range args {
c, err := cid.Decode(s)
if err != nil {
return fmt.Errorf("block cid was invalid: %s", err)
}
tscids = append(tscids, c)
}
tsk := types.NewTipSetKey(tscids...)
valid, err := api.SyncValidateTipset(ctx, tsk)
if err != nil {
fmt.Println("Tipset is invalid: ", err)
}
if valid {
fmt.Println("Tipset is valid")
}
return nil
},
}

Some files were not shown because too many files have changed in this diff Show More