cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity

Merge remote-tracking branch 'origin/master' into conformance-tests

Browse Source
This commit is contained in:
Raúl Kripalani 2020-08-26 15:12:02 +01:00
commit d3ac243cea
200 changed files with 9416 additions and 2175 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
.circleci/config.yml
View File

@ -273,6 +273,27 @@ jobs:
- run:
command: "! go fmt ./... 2>&1 | read"
cbor-gen-check:
executor: golang
steps:
- install-deps
- prepare
- run: make deps
- run: go install golang.org/x/tools/cmd/goimports
- run: go install github.com/hannahhoward/cbor-gen-for
- run: go generate ./...
- run: git --no-pager diff
- run: git --no-pager diff --quiet
docs-check:
executor: golang
steps:
- install-deps
- prepare
- run: make docsgen
- run: git --no-pager diff
- run: git --no-pager diff --quiet
lint: &lint
description: |
Run golangci-lint.
@ -309,9 +330,6 @@ jobs:
command: |
$HOME/.local/bin/golangci-lint run -v --timeout 2m \
--concurrency << parameters.concurrency >> << parameters.args >>
lint-changes:
<<: *lint
lint-all:
<<: *lint
@ -340,10 +358,11 @@ workflows:
version: 2.1
ci:
jobs:
- lint-changes:
args: "--new-from-rev origin/next"
- lint-all
- mod-tidy-check
- gofmt
- cbor-gen-check
- docs-check
- test:
codecov-upload: true
test-suite-name: full

21
.golangci.yml
View File

@ -23,6 +23,14 @@ issues:
- "Potential file inclusion via variable"
- "should have( a package)? comment"
- "Error return value of `logging.SetLogLevel` is not checked"
- "comment on exported"
- "(func|method) \\w+ should be \\w+"
- "(type|var|struct field|(method|func) parameter) `\\w+` should be `\\w+`"
- "(G306|G301|G307|G108|G302|G204|G104)"
- "don't use ALL_CAPS in Go names"
- "string .* has .* occurrences, make it a constant"
- "a blank import should be only in a main or test package, or have a comment justifying it"
- "package comment should be of the form"
exclude-use-default: false
exclude-rules:
@ -46,6 +54,19 @@ issues:
linters:
- gosec
- path: chain/vectors/gen/.*
linters:
- gosec
- path: cmd/lotus-bench/.*
linters:
- gosec
- path: api/test/.*
text: "context.Context should be the first parameter"
linters:
- golint
linters-settings:
goconst:
min-occurrences: 6

190
CHANGELOG.md
View File

@ -1,7 +1,189 @@
# lotus changelog
# Lotus changelog
## 0.1.0 / 2019-12-11
# 0.5.4
We are very excited to release **lotus** 0.1.0. This is our testnet release. To install lotus and join the testnet, please visit [docs.lotu.sh](docs.lotu.sh). Please file bug reports as [issues](https://github.com/filecoin-project/lotus/issues).
A patch release, containing a few nice bugfixes and improvements:
A huge thank you to all contributors for this testnet release!
- Fix parsing of peer ID in `lotus-miner actor set-peer-id` (@whyrusleeping)
- Update dependencies, fixing several bugs (@Stebalien)
- Fix remaining linter warnings (@Stebalien)
- Use safe string truncation (@Ingar)
- Allow tweaking of blocksync message window size (@whyrusleeping)
- Add some additional gas stats to metrics (@Kubuxu)
- Fix an edge case bug in message selection, add many tests (@vyzo)
# 0.5.3
Yet another hotfix release.
A lesson for readers, having people who have been awake for 12+ hours review
your hotfix PR is not a good idea. Find someone who has enough slept recently
enough to give you good code review, otherwise you'll end up quickly bumping
versions again.
- Fixed a bug in the mempool that was introduced in v0.5.2
# 0.5.2 / 2020-08-24
This is a hotfix release.
- Fix message selection to not include messages that are invalid for block
inclusion.
- Improve SelectMessage handling of the case where the message pools tipset
differs from our mining base.
# 0.5.1 / 2020-08-24
The Space Race release!
This release contains the genesis car file and bootstrap peers for the space
race network.
Additionally, we included two small fixes to genesis creation:
- Randomize ticket value in genesis generation
- Correctly set t099 (burnt funds actor) to have valid account actor state
# 0.5.0 / 2020-08-20
This version of Lotus will be used for the incentivized testnet Space Race competition,
and can be considered mainnet-ready code. It includes some protocol
changes, upgrades of core dependencies, and various bugfixes and UX/performance improvements.
## Highlights
Among the highlights included in this release are:
- Gas changes: We implemented EIP-1559 and introduced real gas values.
- Deal-making: We now support "Committed Capacity" sectors, "fast-retrieval" deals,
and the packing of multiple deals into a single sector.
- Renamed features: We renamed some of the binaries, environment variables, and default
paths associated with a Lotus node.
### Gas changes
We made some significant changes to the mechanics of gas in this release.
#### Network fee
We implemented something similar to
[Ethereum's EIP-1559](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1559.md).
The `Message` structure had three changes:
- The `GasPrice` field has been removed
- A new `GasFeeCap` field has been added, which controls the maximum cost
the sender incurs for the message
- A new `GasPremium` field has been added, which controls the reward a miner
earns for including the message
A sender will never be charged more than `GasFeeCap * GasLimit`.
A miner will typically earn `GasPremium * GasLimit` as a reward.
The `Blockheader` structure has one new field, called `ParentBaseFee`.
Informally speaking,the `ParentBaseFee`
is increased when blocks are densely packed with messages, and decreased otherwise.
The `ParentBaseFee` is used when calculating how much a sender burns when executing a message. _Burning_ simply refers to sending attoFIL to a dedicated, unreachable account.
A message causes `ParentBaseFee * GasUsed` attoFIL to be burnt.
#### Real gas values
This release also includes our first "real" gas costs for primitive operations.
The costs were designed to account for both the _time_ that message execution takes,
as well as the _space_ a message adds to the state tree.
## Deal-making changes
There are three key changes to the deal-making process.
#### Committed Capacity sectors
Miners can now pledge "Committed Capacity" (CC) sectors, which are explicitly
stated as containing junk data, and must not include any deals. Miners can do this
to increase their storage power, and win block rewards from this pledged storage.
They can mark these sectors as "upgradable" with `lotus-miner sectors mark-for-upgrade`.
If the miner receives and accepts one or more storage deals, the sector that includes
those deals will _replace_ the CC sector. This is intended to maximize the amount of useful
storage on the Filecoin network.
#### Fast-retrieval deals
Clients can now include a `fast-retrieval` flag when proposing deals with storage miners.
If set to true, the miner will include an extra copy of the deal data. This
data can be quickly served in a retrieval deal, since it will not need to be unsealed.
#### Multiple deals per sector
Miners can now pack multiple deals into a single sector, so long as all the deals
fit into the sector capacity. This should increase the packing efficiency of miners.
### Renamed features
To improve the user experience, we updated several names to mainatin
standard prefixing, and to better reflect the meaning of the features being referenced.
In particular, the Lotus miner binary is now called `lotus-miner`, the default
path for miner data is now `~/.lotusminer`, and the environment variable
that sets the path for miner data is now `$LOTUS_MINER_PATH`. A full list of renamed
features can be found [here](https://github.com/filecoin-project/lotus/issues/2304).
## Changelog
#### Downstream upgrades
- Upgrades markets to v0.5.6 (https://github.com/filecoin-project/lotus/pull/3058)
- Upgrades specs-actors to v0.9.3 (https://github.com/filecoin-project/lotus/pull/3151)
#### Core protocol
- Introduces gas values, replacing placeholders (https://github.com/filecoin-project/lotus/pull/2343)
- Implements EIP-1559, introducing a network base fee, message gas fee cap, and message gas fee premium (https://github.com/filecoin-project/lotus/pull/2874)
- Implements Poisson Sortition for elections (https://github.com/filecoin-project/lotus/pull/2084)
#### Deal-making lifecycle
- Introduces "Committed Capacity" sectors (https://github.com/filecoin-project/lotus/pull/2220)
- Introduces "fast-retrieval" flag for deals (https://github.com/filecoin-project/lotus/pull/2323
- Supports packing multiple deals into one sector (https://github.com/filecoin-project/storage-fsm/pull/38)
#### Enhancements
- Optimized message pool selection logic (https://github.com/filecoin-project/lotus/pull/2838)
- Window-based scheduling of sealing tasks (https://github.com/filecoin-project/sector-storage/pull/67)
- Faster window PoSt (https://github.com/filecoin-project/lotus/pull/2209/files)
- Refactors the payment channel manager (https://github.com/filecoin-project/lotus/pull/2640)
- Refactors blocksync (https://github.com/filecoin-project/lotus/pull/2715/files)
#### UX
- Provide status updates for data-transfer (https://github.com/filecoin-project/lotus/pull/3162, https://github.com/filecoin-project/lotus/pull/3191)
- Miners can customise asks (https://github.com/filecoin-project/lotus/pull/2046)
- Miners can toggle auto-acceptance of deals (https://github.com/filecoin-project/lotus/pull/1994)
- Miners can maintain a blocklist of piece CIDs (https://github.com/filecoin-project/lotus/pull/2069)
## Contributors
The following contributors had 10 or more commits go into this release.
We are grateful for every contribution!
| Contributor | Commits | Lines ± |
|--------------------|---------|---------------|
| magik6k | 361 | +13197/-6136 |
| Kubuxu | 227 | +5670/-2587 |
| arajasek | 120 | +2916/-1264 |
| whyrusleeping | 112 | +3979/-1089 |
| vyzo | 99 | +3343/-1305 |
| dirkmc | 68 | +8732/-3621 |
| laser | 45 | +1489/-501 |
| hannahhoward | 43 | +2654/-990 |
| frrist | 37 | +6630/-4338 |
| schomatis | 28 | +3016/-1368 |
| placer14 | 27 | +824/-350 |
| raulk | 25 | +28718/-29849 |
| mrsmkl | 22 | +560/-368 |
| travisperson | 18 | +1354/-314 |
| nonsense | 16 | +2956/-2842 |
| ingar | 13 | +331/-123 |
| daviddias | 11 | +311/-11 |
| Stebalien | 11 | +1204/-980 |
| RobQuistNL | 10 | +69/-74 |
# 0.1.0 / 2019-12-11
We are very excited to release **lotus** 0.1.0. This is our testnet release. To install lotus and join the testnet, please visit [lotu.sh](lotu.sh). Please file bug reports as [issues](https://github.com/filecoin-project/lotus/issues).
A huge thank you to all contributors for this testnet release!

3
Makefile
View File

@ -279,5 +279,8 @@ method-gen:
gen: type-gen method-gen
docsgen:
go run ./api/docgen > documentation/en/api-methods.md
print-%:
@echo $*=$($*)

4
README.md
View File

@ -1,5 +1,5 @@
<p align="center">
<a href="https://docs.lotu.sh/" title="Lotus Docs">
<a href="https://lotu.sh/" title="Lotus Docs">
<img src="documentation/images/lotus_logo_h.png" alt="Project Lotus Logo" width="244" />
</a>
</p>
@ -18,7 +18,7 @@ Lotus is an implementation of the Filecoin Distributed Storage Network. For more
## Building & Documentation
For instructions on how to build lotus from source, please visit [https://docs.lotu.sh](https://docs.lotu.sh) or read the source [here](https://github.com/filecoin-project/lotus/tree/master/documentation).
For instructions on how to build lotus from source, please visit [https://lotu.sh](https://lotu.sh) or read the source [here](https://github.com/filecoin-project/lotus/tree/master/documentation).
## Reporting a Vulnerability

11
api/api_full.go
View File

@ -132,6 +132,9 @@ type FullNode interface {
GasEstimateGasPremium(_ context.Context, nblocksincl uint64,
sender address.Address, gaslimit int64, tsk types.TipSetKey) (types.BigInt, error)
// GasEstimateMessageGas estimates gas values for unset message gas fields
GasEstimateMessageGas(context.Context, *types.Message, *MessageSendSpec, types.TipSetKey) (*types.Message, error)
// MethodGroup: Sync
// The Sync method group contains methods for interacting with and
// observing the lotus sync service.
@ -181,6 +184,9 @@ type FullNode interface {
MpoolGetNonce(context.Context, address.Address) (uint64, error)
MpoolSub(context.Context) (<-chan MpoolUpdate, error)
// MpoolClear clears pending messages from the mpool
MpoolClear(context.Context, bool) error
// MpoolGetConfig returns (a copy of) the current mpool config
MpoolGetConfig(context.Context) (*types.MpoolConfig, error)
// MpoolSetConfig sets the mpool config to (a copy of) the supplied config
@ -256,6 +262,9 @@ type FullNode interface {
ClientGenCar(ctx context.Context, ref FileRef, outpath string) error
// ClientDealSize calculates real deal data size
ClientDealSize(ctx context.Context, root cid.Cid) (DataSize, error)
// ClientListTransfers returns the status of all ongoing transfers of data
ClientListDataTransfers(ctx context.Context) ([]DataTransferChannel, error)
ClientDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error)
// ClientUnimport removes references to the specified file from filestore
//ClientUnimport(path string)
@ -472,6 +481,8 @@ type DealInfo struct {
Duration uint64
DealID abi.DealID
CreationTime time.Time
}
type MsgLookup struct {

21
api/api_storage.go
View File

@ -79,6 +79,8 @@ type StorageMiner interface {
MarketGetAsk(ctx context.Context) (*storagemarket.SignedStorageAsk, error)
MarketSetRetrievalAsk(ctx context.Context, rask *retrievalmarket.Ask) error
MarketGetRetrievalAsk(ctx context.Context) (*retrievalmarket.Ask, error)
MarketListDataTransfers(ctx context.Context) ([]DataTransferChannel, error)
MarketDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error)
DealsImportData(ctx context.Context, dealPropCid cid.Cid, file string) error
DealsList(ctx context.Context) ([]storagemarket.StorageDeal, error)
@ -118,15 +120,16 @@ type SectorLog struct {
}
type SectorInfo struct {
SectorID abi.SectorNumber
State SectorState
CommD *cid.Cid
CommR *cid.Cid
Proof []byte
Deals []abi.DealID
Ticket SealTicket
Seed SealSeed
Retries uint64
SectorID abi.SectorNumber
State SectorState
CommD *cid.Cid
CommR *cid.Cid
Proof []byte
Deals []abi.DealID
Ticket SealTicket
Seed SealSeed
Retries uint64
ToUpgrade bool
LastErr string

79
api/apistruct/struct.go
View File

@ -90,9 +90,10 @@ type FullNodeStruct struct {
BeaconGetEntry func(ctx context.Context, epoch abi.ChainEpoch) (*types.BeaconEntry, error) `perm:"read"`
GasEstimateGasPremium func(context.Context, uint64, address.Address, int64, types.TipSetKey) (types.BigInt, error) `perm:"read"`
GasEstimateGasLimit func(context.Context, *types.Message, types.TipSetKey) (int64, error) `perm:"read"`
GasEstimateFeeCap func(context.Context, *types.Message, int64, types.TipSetKey) (types.BigInt, error) `perm:"read"`
GasEstimateGasPremium func(context.Context, uint64, address.Address, int64, types.TipSetKey) (types.BigInt, error) `perm:"read"`
GasEstimateGasLimit func(context.Context, *types.Message, types.TipSetKey) (int64, error) `perm:"read"`
GasEstimateFeeCap func(context.Context, *types.Message, int64, types.TipSetKey) (types.BigInt, error) `perm:"read"`
GasEstimateMessageGas func(context.Context, *types.Message, *api.MessageSendSpec, types.TipSetKey) (*types.Message, error) `perm:"read"`
SyncState func(context.Context) (*api.SyncState, error) `perm:"read"`
SyncSubmitBlock func(ctx context.Context, blk *types.BlockMsg) error `perm:"write"`
@ -105,7 +106,9 @@ type FullNodeStruct struct {
MpoolSelect func(context.Context, types.TipSetKey, float64) ([]*types.SignedMessage, error) `perm:"read"`
MpoolPending func(context.Context, types.TipSetKey) ([]*types.SignedMessage, error) `perm:"read"`
MpoolPending func(context.Context, types.TipSetKey) ([]*types.SignedMessage, error) `perm:"read"`
MpoolClear func(context.Context, bool) error `perm:"write"`
MpoolPush func(context.Context, *types.SignedMessage) (cid.Cid, error) `perm:"write"`
MpoolPushMessage func(context.Context, *types.Message, *api.MessageSendSpec) (*types.SignedMessage, error) `perm:"sign"`
MpoolGetNonce func(context.Context, address.Address) (uint64, error) `perm:"read"`
@ -127,21 +130,23 @@ type FullNodeStruct struct {
WalletImport func(context.Context, *types.KeyInfo) (address.Address, error) `perm:"admin"`
WalletDelete func(context.Context, address.Address) error `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
ClientDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
StateNetworkName func(context.Context) (dtypes.NetworkName, error) `perm:"read"`
StateMinerSectors func(context.Context, address.Address, *abi.BitField, bool, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
@ -235,6 +240,8 @@ type StorageMinerStruct struct {
MarketGetAsk func(ctx context.Context) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
MarketSetRetrievalAsk func(ctx context.Context, rask *retrievalmarket.Ask) error `perm:"admin"`
MarketGetRetrievalAsk func(ctx context.Context) (*retrievalmarket.Ask, error) `perm:"read"`
MarketListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
MarketDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
PledgeSector func(context.Context) error `perm:"write"`
@ -449,17 +456,27 @@ func (c *FullNodeStruct) ClientDealSize(ctx context.Context, root cid.Cid) (api.
return c.Internal.ClientDealSize(ctx, root)
}
func (c *FullNodeStruct) GasEstimateGasPremium(ctx context.Context, nblocksincl uint64,
sender address.Address, gaslimit int64, tsk types.TipSetKey) (types.BigInt, error) {
func (c *FullNodeStruct) ClientListDataTransfers(ctx context.Context) ([]api.DataTransferChannel, error) {
return c.Internal.ClientListDataTransfers(ctx)
}
func (c *FullNodeStruct) ClientDataTransferUpdates(ctx context.Context) (<-chan api.DataTransferChannel, error) {
return c.Internal.ClientDataTransferUpdates(ctx)
}
func (c *FullNodeStruct) GasEstimateGasPremium(ctx context.Context, nblocksincl uint64, sender address.Address, gaslimit int64, tsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.GasEstimateGasPremium(ctx, nblocksincl, sender, gaslimit, tsk)
}
func (c *FullNodeStruct) GasEstimateFeeCap(ctx context.Context, msg *types.Message,
maxqueueblks int64, tsk types.TipSetKey) (types.BigInt, error) {
func (c *FullNodeStruct) GasEstimateFeeCap(ctx context.Context, msg *types.Message, maxqueueblks int64, tsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.GasEstimateFeeCap(ctx, msg, maxqueueblks, tsk)
}
func (c *FullNodeStruct) GasEstimateGasLimit(ctx context.Context, msg *types.Message,
tsk types.TipSetKey) (int64, error) {
func (c *FullNodeStruct) GasEstimateMessageGas(ctx context.Context, msg *types.Message, spec *api.MessageSendSpec, tsk types.TipSetKey) (*types.Message, error) {
return c.Internal.GasEstimateMessageGas(ctx, msg, spec, tsk)
}
func (c *FullNodeStruct) GasEstimateGasLimit(ctx context.Context, msg *types.Message, tsk types.TipSetKey) (int64, error) {
return c.Internal.GasEstimateGasLimit(ctx, msg, tsk)
}
@ -479,6 +496,10 @@ func (c *FullNodeStruct) MpoolPending(ctx context.Context, tsk types.TipSetKey)
return c.Internal.MpoolPending(ctx, tsk)
}
func (c *FullNodeStruct) MpoolClear(ctx context.Context, local bool) error {
return c.Internal.MpoolClear(ctx, local)
}
func (c *FullNodeStruct) MpoolPush(ctx context.Context, smsg *types.SignedMessage) (cid.Cid, error) {
return c.Internal.MpoolPush(ctx, smsg)
}
@ -1067,6 +1088,14 @@ func (c *StorageMinerStruct) MarketGetRetrievalAsk(ctx context.Context) (*retrie
return c.Internal.MarketGetRetrievalAsk(ctx)
}
func (c *StorageMinerStruct) MarketListDataTransfers(ctx context.Context) ([]api.DataTransferChannel, error) {
return c.Internal.MarketListDataTransfers(ctx)
}
func (c *StorageMinerStruct) MarketDataTransferUpdates(ctx context.Context) (<-chan api.DataTransferChannel, error) {
return c.Internal.MarketDataTransferUpdates(ctx)
}
func (c *StorageMinerStruct) DealsImportData(ctx context.Context, dealPropCid cid.Cid, file string) error {
return c.Internal.DealsImportData(ctx, dealPropCid, file)
}

4
api/cbor_gen.go
View File

@ -6,8 +6,8 @@ import (
"fmt"
"io"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
abi "github.com/filecoin-project/specs-actors/actors/abi"
paych "github.com/filecoin-project/specs-actors/actors/builtin/paych"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
)

17
api/client/client.go
View File

@ -1,6 +1,7 @@
package client
import (
"context"
"net/http"
"net/url"
"path"
@ -14,9 +15,9 @@ import (
)
// NewCommonRPC creates a new http jsonrpc client.
func NewCommonRPC(addr string, requestHeader http.Header) (api.Common, jsonrpc.ClientCloser, error) {
func NewCommonRPC(ctx context.Context, addr string, requestHeader http.Header) (api.Common, jsonrpc.ClientCloser, error) {
var res apistruct.CommonStruct
closer, err := jsonrpc.NewMergeClient(addr, "Filecoin",
closer, err := jsonrpc.NewMergeClient(ctx, addr, "Filecoin",
[]interface{}{
&res.Internal,
},
@ -27,9 +28,9 @@ func NewCommonRPC(addr string, requestHeader http.Header) (api.Common, jsonrpc.C
}
// NewFullNodeRPC creates a new http jsonrpc client.
func NewFullNodeRPC(addr string, requestHeader http.Header) (api.FullNode, jsonrpc.ClientCloser, error) {
func NewFullNodeRPC(ctx context.Context, addr string, requestHeader http.Header) (api.FullNode, jsonrpc.ClientCloser, error) {
var res apistruct.FullNodeStruct
closer, err := jsonrpc.NewMergeClient(addr, "Filecoin",
closer, err := jsonrpc.NewMergeClient(ctx, addr, "Filecoin",
[]interface{}{
&res.CommonStruct.Internal,
&res.Internal,
@ -39,9 +40,9 @@ func NewFullNodeRPC(addr string, requestHeader http.Header) (api.FullNode, jsonr
}
// NewStorageMinerRPC creates a new http jsonrpc client for miner
func NewStorageMinerRPC(addr string, requestHeader http.Header, opts ...jsonrpc.Option) (api.StorageMiner, jsonrpc.ClientCloser, error) {
func NewStorageMinerRPC(ctx context.Context, addr string, requestHeader http.Header, opts ...jsonrpc.Option) (api.StorageMiner, jsonrpc.ClientCloser, error) {
var res apistruct.StorageMinerStruct
closer, err := jsonrpc.NewMergeClient(addr, "Filecoin",
closer, err := jsonrpc.NewMergeClient(ctx, addr, "Filecoin",
[]interface{}{
&res.CommonStruct.Internal,
&res.Internal,
@ -53,7 +54,7 @@ func NewStorageMinerRPC(addr string, requestHeader http.Header, opts ...jsonrpc.
return &res, closer, err
}
func NewWorkerRPC(addr string, requestHeader http.Header) (api.WorkerAPI, jsonrpc.ClientCloser, error) {
func NewWorkerRPC(ctx context.Context, addr string, requestHeader http.Header) (api.WorkerAPI, jsonrpc.ClientCloser, error) {
u, err := url.Parse(addr)
if err != nil {
return nil, nil, err
@ -69,7 +70,7 @@ func NewWorkerRPC(addr string, requestHeader http.Header) (api.WorkerAPI, jsonrp
u.Path = path.Join(u.Path, "../streams/v0/push")
var res apistruct.WorkerStruct
closer, err := jsonrpc.NewMergeClient(addr, "Filecoin",
closer, err := jsonrpc.NewMergeClient(ctx, addr, "Filecoin",
[]interface{}{
&res.Internal,
},

40
api/docgen/docgen.go
View File

@ -12,22 +12,29 @@ import (
"time"
"unicode"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-filestore"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/multiformats/go-multiaddr"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apistruct"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-filestore"
"github.com/libp2p/go-libp2p-core/network"
peer "github.com/libp2p/go-libp2p-peer"
"github.com/multiformats/go-multiaddr"
)
var ExampleValues = map[reflect.Type]interface{}{
@ -66,11 +73,12 @@ func init() {
ExampleValues[reflect.TypeOf(addr)] = addr
pid, err := peer.IDB58Decode("12D3KooWGzxzKZYveHXtpG6AsrUJBcWxHBFS2HsEoGTxrMLvKXtf")
pid, err := peer.Decode("12D3KooWGzxzKZYveHXtpG6AsrUJBcWxHBFS2HsEoGTxrMLvKXtf")
if err != nil {
panic(err)
}
addExample(pid)
addExample(&pid)
addExample(bitfield.NewFromSet([]uint64{5}))
addExample(abi.RegisteredSealProof_StackedDrg32GiBV1)
@ -98,6 +106,12 @@ func init() {
addExample(build.APIVersion)
addExample(api.PCHInbound)
addExample(time.Minute)
addExample(datatransfer.TransferID(3))
addExample(datatransfer.Ongoing)
addExample(multistore.StoreID(50))
addExample(retrievalmarket.ClientEventDealAccepted)
addExample(retrievalmarket.DealStatusNew)
addExample(network.ReachabilityPublic)
addExample(&types.ExecutionTrace{
Msg: exampleValue(reflect.TypeOf(&types.Message{}), nil).(*types.Message),
MsgRct: exampleValue(reflect.TypeOf(&types.MessageReceipt{}), nil).(*types.MessageReceipt),
@ -111,6 +125,14 @@ func init() {
addExample(map[string]api.MarketBalance{
"t026363": exampleValue(reflect.TypeOf(api.MarketBalance{}), nil).(api.MarketBalance),
})
addExample(map[string]*pubsub.TopicScoreSnapshot{
"/blocks": {
TimeInMesh: time.Minute,
FirstMessageDeliveries: 122,
MeshMessageDeliveries: 1234,
InvalidMessageDeliveries: 3,
},
})
maddr, err := multiaddr.NewMultiaddr("/ip4/52.36.61.156/tcp/1347/p2p/12D3KooWFETiESTf1v4PGUvtnxMAcEFMzLZbJGg4tjWfGEimYior")
if err != nil {

56
api/test/blockminer.go Normal file
View File

@ -0,0 +1,56 @@
package test
import (
"context"
"fmt"
"sync/atomic"
"testing"
"time"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/specs-actors/actors/abi"
)
type BlockMiner struct {
ctx context.Context
t *testing.T
miner TestStorageNode
blocktime time.Duration
mine int64
nulls int64
done chan struct{}
}
func NewBlockMiner(ctx context.Context, t *testing.T, miner TestStorageNode, blocktime time.Duration) *BlockMiner {
return &BlockMiner{
ctx: ctx,
t: t,
miner: miner,
blocktime: blocktime,
mine: int64(1),
done: make(chan struct{}),
}
}
func (bm *BlockMiner) MineBlocks() {
time.Sleep(time.Second)
go func() {
defer close(bm.done)
for atomic.LoadInt64(&bm.mine) == 1 {
time.Sleep(bm.blocktime)
nulls := atomic.SwapInt64(&bm.nulls, 0)
if err := bm.miner.MineOne(bm.ctx, miner.MineReq{
InjectNulls: abi.ChainEpoch(nulls),
Done: func(bool, error) {},
}); err != nil {
bm.t.Error(err)
}
}
}()
}
func (bm *BlockMiner) Stop() {
atomic.AddInt64(&bm.mine, -1)
fmt.Println("shutting down mining")
<-bm.done
}

2
api/test/ccupgrade.go
View File

@ -20,7 +20,7 @@ func TestCCUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]

16
api/test/deals.go
View File

@ -47,7 +47,7 @@ func TestDealFlow(t *testing.T, b APIBuilder, blocktime time.Duration, carExport
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@ -84,7 +84,7 @@ func TestDoubleDealFlow(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@ -141,14 +141,14 @@ func makeDeal(t *testing.T, ctx context.Context, rseed int, client *impl.FullNod
info, err := client.ClientGetDealInfo(ctx, *deal)
require.NoError(t, err)
testRetrieval(t, ctx, err, client, fcid, &info.PieceCID, carExport, data)
testRetrieval(t, ctx, client, fcid, &info.PieceCID, carExport, data)
}
func TestFastRetrievalDealFlow(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@ -193,7 +193,7 @@ func TestFastRetrievalDealFlow(t *testing.T, b APIBuilder, blocktime time.Durati
info, err := client.ClientGetDealInfo(ctx, *deal)
require.NoError(t, err)
testRetrieval(t, ctx, err, client, fcid, &info.PieceCID, false, data)
testRetrieval(t, ctx, client, fcid, &info.PieceCID, false, data)
atomic.AddInt64(&mine, -1)
fmt.Println("shutting down mining")
<-done
@ -203,7 +203,7 @@ func TestSenondDealRetrieval(t *testing.T, b APIBuilder, blocktime time.Duration
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@ -267,7 +267,7 @@ func TestSenondDealRetrieval(t *testing.T, b APIBuilder, blocktime time.Duration
rf, _ := miner.SectorsRefs(ctx)
fmt.Printf("refs: %+v\n", rf)
testRetrieval(t, ctx, err, client, fcid2, &info.PieceCID, false, data2)
testRetrieval(t, ctx, client, fcid2, &info.PieceCID, false, data2)
}
atomic.AddInt64(&mine, -1)
@ -373,7 +373,7 @@ func startSealingWaiting(t *testing.T, ctx context.Context, miner TestStorageNod
}
}
func testRetrieval(t *testing.T, ctx context.Context, err error, client *impl.FullNodeAPI, fcid cid.Cid, piece *cid.Cid, carExport bool, data []byte) {
func testRetrieval(t *testing.T, ctx context.Context, client *impl.FullNodeAPI, fcid cid.Cid, piece *cid.Cid, carExport bool, data []byte) {
offers, err := client.ClientFindData(ctx, fcid, piece)
if err != nil {
t.Fatal(err)

11
api/test/mining.go
View File

@ -20,11 +20,12 @@ import (
"github.com/filecoin-project/lotus/node/impl"
)
//nolint:deadcode,varcheck
var log = logging.Logger("apitest")
func (ts *testSuite) testMining(t *testing.T) {
ctx := context.Background()
apis, sn := ts.makeNodes(t, 1, oneMiner)
apis, sn := ts.makeNodes(t, 1, OneMiner)
api := apis[0]
newHeads, err := api.ChainNotify(ctx)
@ -38,7 +39,7 @@ func (ts *testSuite) testMining(t *testing.T) {
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(2), h1.Height())
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, sn[0], nil)
require.NoError(t, err)
<-newHeads
@ -55,7 +56,7 @@ func (ts *testSuite) testMiningReal(t *testing.T) {
}()
ctx := context.Background()
apis, sn := ts.makeNodes(t, 1, oneMiner)
apis, sn := ts.makeNodes(t, 1, OneMiner)
api := apis[0]
newHeads, err := api.ChainNotify(ctx)
@ -69,7 +70,7 @@ func (ts *testSuite) testMiningReal(t *testing.T) {
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(2), h1.Height())
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, sn[0], nil)
require.NoError(t, err)
<-newHeads
@ -78,7 +79,7 @@ func (ts *testSuite) testMiningReal(t *testing.T) {
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(3), h2.Height())
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, sn[0], nil)
require.NoError(t, err)
<-newHeads

86
api/test/paych.go
View File

@ -23,14 +23,13 @@ import (
"github.com/filecoin-project/lotus/chain/events/state"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/lotus/miner"
)
func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 2, oneMiner)
n, sn := b(t, 2, OneMiner)
paymentCreator := n[0]
paymentReceiver := n[1]
@ -51,8 +50,8 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
}
// start mining blocks
bm := newBlockMiner(ctx, t, miner, blocktime)
bm.mineBlocks()
bm := NewBlockMiner(ctx, t, miner, blocktime)
bm.MineBlocks()
// send some funds to register the receiver
receiverAddr, err := paymentReceiver.WalletNew(ctx, wallet.ActSigType("secp256k1"))
@ -60,7 +59,7 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
t.Fatal(err)
}
sendFunds(ctx, t, paymentCreator, receiverAddr, abi.NewTokenAmount(1e18))
SendFunds(ctx, t, paymentCreator, receiverAddr, abi.NewTokenAmount(1e18))
// setup the payment channel
createrAddr, err := paymentCreator.WalletDefaultAddress(ctx)
@ -154,6 +153,9 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
}, int(build.MessageConfidence)+1, build.SealRandomnessLookbackLimit, func(oldTs, newTs *types.TipSet) (bool, events.StateChange, error) {
return preds.OnPaymentChannelActorChanged(channel, preds.OnToSendAmountChanges())(ctx, oldTs.Key(), newTs.Key())
})
if err != nil {
t.Fatal(err)
}
select {
case <-finished:
@ -201,10 +203,10 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
}
// shut down mining
bm.stop()
bm.Stop()
}
func waitForBlocks(ctx context.Context, t *testing.T, bm *blockMiner, paymentReceiver TestNode, receiverAddr address.Address, count int) {
func waitForBlocks(ctx context.Context, t *testing.T, bm *BlockMiner, paymentReceiver TestNode, receiverAddr address.Address, count int) {
// We need to add null blocks in batches, if we add too many the chain can't sync
batchSize := 60
for i := 0; i < count; i += batchSize {
@ -249,73 +251,3 @@ func waitForMessage(ctx context.Context, t *testing.T, paymentCreator TestNode,
fmt.Println("Confirmed", desc)
return res
}
type blockMiner struct {
ctx context.Context
t *testing.T
miner TestStorageNode
blocktime time.Duration
mine int64
nulls int64
done chan struct{}
}
func newBlockMiner(ctx context.Context, t *testing.T, miner TestStorageNode, blocktime time.Duration) *blockMiner {
return &blockMiner{
ctx: ctx,
t: t,
miner: miner,
blocktime: blocktime,
mine: int64(1),
done: make(chan struct{}),
}
}
func (bm *blockMiner) mineBlocks() {
time.Sleep(time.Second)
go func() {
defer close(bm.done)
for atomic.LoadInt64(&bm.mine) == 1 {
time.Sleep(bm.blocktime)
nulls := atomic.SwapInt64(&bm.nulls, 0)
if err := bm.miner.MineOne(bm.ctx, miner.MineReq{
InjectNulls: abi.ChainEpoch(nulls),
Done: func(bool, error) {},
}); err != nil {
bm.t.Error(err)
}
}
}()
}
func (bm *blockMiner) stop() {
atomic.AddInt64(&bm.mine, -1)
fmt.Println("shutting down mining")
<-bm.done
}
func sendFunds(ctx context.Context, t *testing.T, sender TestNode, addr address.Address, amount abi.TokenAmount) {
senderAddr, err := sender.WalletDefaultAddress(ctx)
if err != nil {
t.Fatal(err)
}
msg := &types.Message{
From: senderAddr,
To: addr,
Value: amount,
}
sm, err := sender.MpoolPushMessage(ctx, msg, nil)
if err != nil {
t.Fatal(err)
}
res, err := sender.StateWaitMsg(ctx, sm.Cid(), 1)
if err != nil {
t.Fatal(err)
}
if res.Receipt.ExitCode != 0 {
t.Fatal("did not successfully send money")
}
}

16
api/test/test.go
View File

@ -4,6 +4,8 @@ import (
"context"
"testing"
"github.com/multiformats/go-multiaddr"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@ -14,10 +16,16 @@ import (
type TestNode struct {
api.FullNode
// ListenAddr is the address on which an API server is listening, if an
// API server is created for this Node
ListenAddr multiaddr.Multiaddr
}
type TestStorageNode struct {
api.StorageMiner
// ListenAddr is the address on which an API server is listening, if an
// API server is created for this Node
ListenAddr multiaddr.Multiaddr
MineOne func(context.Context, miner.MineReq) error
}
@ -54,11 +62,11 @@ func TestApis(t *testing.T, b APIBuilder) {
t.Run("testMiningReal", ts.testMiningReal)
}
var oneMiner = []StorageMiner{{Full: 0, Preseal: PresealGenesis}}
var OneMiner = []StorageMiner{{Full: 0, Preseal: PresealGenesis}}
func (ts *testSuite) testVersion(t *testing.T) {
ctx := context.Background()
apis, _ := ts.makeNodes(t, 1, oneMiner)
apis, _ := ts.makeNodes(t, 1, OneMiner)
api := apis[0]
v, err := api.Version(ctx)
@ -70,7 +78,7 @@ func (ts *testSuite) testVersion(t *testing.T) {
func (ts *testSuite) testID(t *testing.T) {
ctx := context.Background()
apis, _ := ts.makeNodes(t, 1, oneMiner)
apis, _ := ts.makeNodes(t, 1, OneMiner)
api := apis[0]
id, err := api.ID(ctx)
@ -82,7 +90,7 @@ func (ts *testSuite) testID(t *testing.T) {
func (ts *testSuite) testConnectTwo(t *testing.T) {
ctx := context.Background()
apis, _ := ts.makeNodes(t, 2, oneMiner)
apis, _ := ts.makeNodes(t, 2, OneMiner)
p, err := apis[0].NetPeers(ctx)
if err != nil {

64
api/test/util.go Normal file
View File

@ -0,0 +1,64 @@
package test
import (
"context"
"testing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/miner"
)
func SendFunds(ctx context.Context, t *testing.T, sender TestNode, addr address.Address, amount abi.TokenAmount) {
senderAddr, err := sender.WalletDefaultAddress(ctx)
if err != nil {
t.Fatal(err)
}
msg := &types.Message{
From: senderAddr,
To: addr,
Value: amount,
}
sm, err := sender.MpoolPushMessage(ctx, msg, nil)
if err != nil {
t.Fatal(err)
}
res, err := sender.StateWaitMsg(ctx, sm.Cid(), 1)
if err != nil {
t.Fatal(err)
}
if res.Receipt.ExitCode != 0 {
t.Fatal("did not successfully send money")
}
}
func MineUntilBlock(ctx context.Context, t *testing.T, sn TestStorageNode, cb func()) {
for i := 0; i < 1000; i++ {
var success bool
var err error
wait := make(chan struct{})
sn.MineOne(ctx, miner.MineReq{
Done: func(win bool, e error) {
success = win
err = e
wait <- struct{}{}
},
})
<-wait
if err != nil {
t.Fatal(err)
}
if success {
if cb != nil {
cb()
}
return
}
t.Log("did not mine block, trying again", i)
}
t.Fatal("failed to mine 1000 times in a row...")
}

15
api/test/window_post.go
View File

@ -24,11 +24,16 @@ import (
"github.com/filecoin-project/lotus/node/impl"
)
func TestPledgeSector(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int) {
os.Setenv("BELLMAN_NO_GPU", "1")
func init() {
err := os.Setenv("BELLMAN_NO_GPU", "1")
if err != nil {
panic(fmt.Sprintf("failed to set BELLMAN_NO_GPU env variable: %s", err))
}
}
func TestPledgeSector(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int) {
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]
@ -110,10 +115,8 @@ func pledgeSectors(t *testing.T, ctx context.Context, miner TestStorageNode, n,
}
func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int) {
os.Setenv("BELLMAN_NO_GPU", "1")
ctx := context.Background()
n, sn := b(t, 1, oneMiner)
n, sn := b(t, 1, OneMiner)
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]

66
api/types.go
View File

@ -2,10 +2,14 @@ package api
import (
"encoding/json"
"fmt"
"github.com/filecoin-project/go-address"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
pubsub "github.com/libp2p/go-libp2p-pubsub"
@ -46,9 +50,10 @@ type PubsubScore struct {
}
type MinerInfo struct {
Owner address.Address // Must be an ID-address.
Worker address.Address // Must be an ID-address.
NewWorker address.Address // Must be an ID-address.
Owner address.Address // Must be an ID-address.
Worker address.Address // Must be an ID-address.
NewWorker address.Address // Must be an ID-address.
ControlAddresses []address.Address // Must be an ID-addresses.
WorkerChangeEpoch abi.ChainEpoch
PeerId *peer.ID
Multiaddrs []abi.Multiaddrs
@ -64,10 +69,13 @@ func NewApiMinerInfo(info *miner.MinerInfo) MinerInfo {
}
mi := MinerInfo{
Owner: info.Owner,
Worker: info.Worker,
NewWorker: address.Undef,
WorkerChangeEpoch: -1,
Owner: info.Owner,
Worker: info.Worker,
ControlAddresses: info.ControlAddresses,
NewWorker: address.Undef,
WorkerChangeEpoch: -1,
PeerId: pid,
Multiaddrs: info.Multiaddrs,
SealProofType: info.SealProofType,
@ -98,3 +106,47 @@ func (ms *MessageSendSpec) Get() MessageSendSpec {
return *ms
}
type DataTransferChannel struct {
TransferID datatransfer.TransferID
Status datatransfer.Status
BaseCID cid.Cid
IsInitiator bool
IsSender bool
Voucher string
Message string
OtherPeer peer.ID
Transferred uint64
}
// NewDataTransferChannel constructs an API DataTransferChannel type from full channel state snapshot and a host id
func NewDataTransferChannel(hostID peer.ID, channelState datatransfer.ChannelState) DataTransferChannel {
channel := DataTransferChannel{
TransferID: channelState.TransferID(),
Status: channelState.Status(),
BaseCID: channelState.BaseCID(),
IsSender: channelState.Sender() == hostID,
Message: channelState.Message(),
}
stringer, ok := channelState.Voucher().(fmt.Stringer)
if ok {
channel.Voucher = stringer.String()
} else {
voucherJSON, err := json.Marshal(channelState.Voucher())
if err != nil {
channel.Voucher = fmt.Errorf("Voucher Serialization: %w", err).Error()
} else {
channel.Voucher = string(voucherJSON)
}
}
if channel.IsSender {
channel.IsInitiator = !channelState.IsPull()
channel.Transferred = channelState.Sent()
channel.OtherPeer = channelState.Recipient()
} else {
channel.IsInitiator = channelState.IsPull()
channel.Transferred = channelState.Received()
channel.OtherPeer = channelState.Sender()
}
return channel
}

18
build/bootstrap/bootstrappers.pi
View File

@ -1,12 +1,6 @@
/dns4/bootstrap-0-sin.fil-test.net/tcp/1347/p2p/12D3KooWPdUquftaQvoQEtEdsRBAhwD6jopbF2oweVTzR59VbHEd
/ip4/86.109.15.57/tcp/1347/p2p/12D3KooWPdUquftaQvoQEtEdsRBAhwD6jopbF2oweVTzR59VbHEd
/dns4/bootstrap-0-dfw.fil-test.net/tcp/1347/p2p/12D3KooWQSCkHCzosEyrh8FgYfLejKgEPM5VB6qWzZE3yDAuXn8d
/ip4/139.178.84.45/tcp/1347/p2p/12D3KooWQSCkHCzosEyrh8FgYfLejKgEPM5VB6qWzZE3yDAuXn8d
/dns4/bootstrap-0-fra.fil-test.net/tcp/1347/p2p/12D3KooWEXN2eQmoyqnNjde9PBAQfQLHN67jcEdWU6JougWrgXJK
/ip4/136.144.49.17/tcp/1347/p2p/12D3KooWEXN2eQmoyqnNjde9PBAQfQLHN67jcEdWU6JougWrgXJK
/dns4/bootstrap-1-sin.fil-test.net/tcp/1347/p2p/12D3KooWLmJkZd33mJhjg5RrpJ6NFep9SNLXWc4uVngV4TXKwzYw
/ip4/86.109.15.123/tcp/1347/p2p/12D3KooWLmJkZd33mJhjg5RrpJ6NFep9SNLXWc4uVngV4TXKwzYw
/dns4/bootstrap-1-dfw.fil-test.net/tcp/1347/p2p/12D3KooWGXLHjiz6pTRu7x2pkgTVCoxcCiVxcNLpMnWcJ3JiNEy5
/ip4/139.178.86.3/tcp/1347/p2p/12D3KooWGXLHjiz6pTRu7x2pkgTVCoxcCiVxcNLpMnWcJ3JiNEy5
/dns4/bootstrap-1-fra.fil-test.net/tcp/1347/p2p/12D3KooW9szZmKttS9A1FafH3Zc2pxKwwmvCWCGKkRP4KmbhhC4R
/ip4/136.144.49.131/tcp/1347/p2p/12D3KooW9szZmKttS9A1FafH3Zc2pxKwwmvCWCGKkRP4KmbhhC4R
/dns4/bootstrap-0.testnet.fildev.network/tcp/1347/p2p/12D3KooWJTUBUjtzWJGWU1XSiY21CwmHaCNLNYn2E7jqHEHyZaP7
/dns4/bootstrap-1.testnet.fildev.network/tcp/1347/p2p/12D3KooW9yeKXha4hdrJKq74zEo99T8DhriQdWNoojWnnQbsgB3v
/dns4/bootstrap-2.testnet.fildev.network/tcp/1347/p2p/12D3KooWCrx8yVG9U9Kf7w8KLN3Edkj5ZKDhgCaeMqQbcQUoB6CT
/dns4/bootstrap-4.testnet.fildev.network/tcp/1347/p2p/12D3KooWPkL9LrKRQgHtq7kn9ecNhGU9QaziG8R5tX8v9v7t3h34
/dns4/bootstrap-3.testnet.fildev.network/tcp/1347/p2p/12D3KooWKYSsbpgZ3HAjax5M1BXCwXLa6gVkUARciz7uN3FNtr7T
/dns4/bootstrap-5.testnet.fildev.network/tcp/1347/p2p/12D3KooWQYzqnLASJAabyMpPb1GcWZvNSe7JDcRuhdRqonFoiK9W

16
build/drand.go
View File

@ -2,7 +2,7 @@ package build
import "github.com/filecoin-project/lotus/node/modules/dtypes"
var DrandNetwork = DrandMainnet
var DrandNetwork = DrandIncentinet
func DrandConfig() dtypes.DrandConfig {
return DrandConfigs[DrandNetwork]
@ -15,6 +15,7 @@ const (
DrandTestnet
DrandDevnet
DrandLocalnet
DrandIncentinet
)
var DrandConfigs = map[DrandEnum]dtypes.DrandConfig{
@ -55,4 +56,17 @@ var DrandConfigs = map[DrandEnum]dtypes.DrandConfig{
},
ChainInfoJSON: `{"public_key":"8cda589f88914aa728fd183f383980b35789ce81b274e5daee1f338b77d02566ef4d3fb0098af1f844f10f9c803c1827","period":25,"genesis_time":1595348225,"hash":"e73b7dc3c4f6a236378220c0dd6aa110eb16eed26c11259606e07ee122838d4f","groupHash":"567d4785122a5a3e75a9bc9911d7ea807dd85ff76b78dc4ff06b075712898607"}`,
},
DrandIncentinet: {
Servers: []string{
"https://pl-eu.incentinet.drand.sh",
"https://pl-us.incentinet.drand.sh",
"https://pl-sin.incentinet.drand.sh",
},
Relays: []string{
"/dnsaddr/pl-eu.incentinet.drand.sh/",
"/dnsaddr/pl-us.incentinet.drand.sh/",
"/dnsaddr/pl-sin.incentinet.drand.sh/",
},
ChainInfoJSON: `{"public_key":"8cad0c72c606ab27d36ee06de1d5b2db1faf92e447025ca37575ab3a8aac2eaae83192f846fc9e158bc738423753d000","period":30,"genesis_time":1595873820,"hash":"80c8b872c714f4c00fdd3daa465d5514049f457f01f85a4caf68cdcd394ba039","groupHash":"d9406aaed487f7af71851b4399448e311f2328923d454e971536c05398ce2d9b"}`,
},
}

BIN
build/genesis/devnet.car
View File

Binary file not shown.

2
build/params_testnet.go
View File

@ -13,7 +13,7 @@ import (
)
func init() {
power.ConsensusMinerMinPower = big.NewInt(1024 << 30)
power.ConsensusMinerMinPower = big.NewInt(10 << 40)
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{
abi.RegisteredSealProof_StackedDrg32GiBV1: {},
abi.RegisteredSealProof_StackedDrg64GiBV1: {},

4
build/version.go
View File

@ -25,7 +25,7 @@ func buildType() string {
}
// BuildVersion is the local build version, set by build system
const BuildVersion = "0.4.6"
const BuildVersion = "0.5.4"
func UserVersion() string {
return BuildVersion + buildType() + CurrentCommit
@ -53,7 +53,7 @@ func (ve Version) EqMajorMinor(v2 Version) bool {
}
// APIVersion is a semver version of the rpc api exposed
var APIVersion Version = newVer(0, 11, 0)
var APIVersion Version = newVer(0, 12, 0)
//nolint:varcheck,deadcode
const (

4
chain/block_receipt_tracker.go
View File

@ -7,8 +7,8 @@ import (
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/hashicorp/golang-lru"
peer "github.com/libp2p/go-libp2p-core/peer"
lru "github.com/hashicorp/golang-lru"
"github.com/libp2p/go-libp2p-core/peer"
)
type blockReceiptTracker struct {

4
chain/blocksync/cbor_gen.go
View File

@ -6,8 +6,8 @@ import (
"fmt"
"io"
"github.com/filecoin-project/lotus/chain/types"
"github.com/ipfs/go-cid"
types "github.com/filecoin-project/lotus/chain/types"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
)

10
chain/blocksync/client.go
View File

@ -172,7 +172,7 @@ func (client *BlockSync) processResponse(
resLength, req.Length)
}
if resLength < int(req.Length) && res.Status != Partial {
return nil, xerrors.Errorf("got less than requested without a proper status: %s", res.Status)
return nil, xerrors.Errorf("got less than requested without a proper status: %d", res.Status)
}
validRes := &validatedResponse{}
@ -205,7 +205,7 @@ func (client *BlockSync) processResponse(
validRes.messages = make([]*CompactedMessages, resLength)
for i := 0; i < resLength; i++ {
if res.Chain[i].Messages == nil {
return nil, xerrors.Errorf("no messages included for tipset at height (head - %d): %w", i)
return nil, xerrors.Errorf("no messages included for tipset at height (head - %d)", i)
}
validRes.messages[i] = res.Chain[i].Messages
}
@ -308,6 +308,12 @@ func (client *BlockSync) GetChainMessages(
length uint64,
) ([]*CompactedMessages, error) {
ctx, span := trace.StartSpan(ctx, "GetChainMessages")
if span.IsRecordingEvents() {
span.AddAttributes(
trace.StringAttribute("tipset", fmt.Sprint(head.Cids())),
trace.Int64Attribute("count", int64(length)),
)
}
defer span.End()
req := &Request{

3
chain/blocksync/protocol.go
View File

@ -1,9 +1,10 @@
package blocksync
import (
"time"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/store"
"time"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log"

1
chain/events/state/diff_adt.go
View File

@ -2,6 +2,7 @@ package state
import (
"bytes"
"github.com/filecoin-project/specs-actors/actors/util/adt"
typegen "github.com/whyrusleeping/cbor-gen"
)

1
chain/events/state/predicates.go
View File

@ -3,6 +3,7 @@ package state
import (
"bytes"
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"

18
chain/events/state/predicates_test.go
View File

@ -114,10 +114,10 @@ func TestMarketPredicates(t *testing.T) {
}
oldBalances := map[address.Address]balance{
tutils.NewIDAddr(t, 1): balance{abi.NewTokenAmount(1000), abi.NewTokenAmount(1000)},
tutils.NewIDAddr(t, 2): balance{abi.NewTokenAmount(2000), abi.NewTokenAmount(500)},
tutils.NewIDAddr(t, 3): balance{abi.NewTokenAmount(3000), abi.NewTokenAmount(2000)},
tutils.NewIDAddr(t, 5): balance{abi.NewTokenAmount(3000), abi.NewTokenAmount(1000)},
tutils.NewIDAddr(t, 1): {abi.NewTokenAmount(1000), abi.NewTokenAmount(1000)},
tutils.NewIDAddr(t, 2): {abi.NewTokenAmount(2000), abi.NewTokenAmount(500)},
tutils.NewIDAddr(t, 3): {abi.NewTokenAmount(3000), abi.NewTokenAmount(2000)},
tutils.NewIDAddr(t, 5): {abi.NewTokenAmount(3000), abi.NewTokenAmount(1000)},
}
oldStateC := createMarketState(ctx, t, store, oldDeals, oldProps, oldBalances)
@ -162,10 +162,10 @@ func TestMarketPredicates(t *testing.T) {
// NB: DealProposals cannot be modified, so don't test that case.
}
newBalances := map[address.Address]balance{
tutils.NewIDAddr(t, 1): balance{abi.NewTokenAmount(3000), abi.NewTokenAmount(0)},
tutils.NewIDAddr(t, 2): balance{abi.NewTokenAmount(2000), abi.NewTokenAmount(500)},
tutils.NewIDAddr(t, 4): balance{abi.NewTokenAmount(5000), abi.NewTokenAmount(0)},
tutils.NewIDAddr(t, 5): balance{abi.NewTokenAmount(1000), abi.NewTokenAmount(3000)},
tutils.NewIDAddr(t, 1): {abi.NewTokenAmount(3000), abi.NewTokenAmount(0)},
tutils.NewIDAddr(t, 2): {abi.NewTokenAmount(2000), abi.NewTokenAmount(500)},
tutils.NewIDAddr(t, 4): {abi.NewTokenAmount(5000), abi.NewTokenAmount(0)},
tutils.NewIDAddr(t, 5): {abi.NewTokenAmount(1000), abi.NewTokenAmount(3000)},
}
newStateC := createMarketState(ctx, t, store, newDeals, newProps, newBalances)
@ -505,6 +505,7 @@ func createBalanceTable(ctx context.Context, t *testing.T, store adt.Store, bala
lockedMapRootCid, err := lockedMapRoot.Root()
require.NoError(t, err)
lockedRoot, err := adt.AsBalanceTable(store, lockedMapRootCid)
require.NoError(t, err)
for addr, balance := range balances {
err := escrowRoot.Add(addr, big.Add(balance.available, balance.locked))
@ -542,6 +543,7 @@ func createEmptyMinerState(ctx context.Context, t *testing.T, store adt.Store, o
emptyVestingFunds := miner.ConstructVestingFunds()
emptyVestingFundsCid, err := store.Put(store.Context(), emptyVestingFunds)
require.NoError(t, err)
emptyDeadlines := miner.ConstructDeadlines(emptyDeadline)
emptyDeadlinesCid, err := store.Put(store.Context(), emptyDeadlines)

18
chain/gen/gen.go
View File

@ -41,10 +41,11 @@ import (
"github.com/filecoin-project/lotus/node/repo"
)
var log = logging.Logger("gen")
const msgsPerBlock = 20
//nolint:deadcode,varcheck
var log = logging.Logger("gen")
var ValidWpostForTesting = []abi.PoStProof{{
ProofBytes: []byte("valid proof"),
}}
@ -92,10 +93,8 @@ func (m mybs) Get(c cid.Cid) (block.Block, error) {
return b, nil
}
var rootkey, _ = address.NewIDAddress(80)
var rootkeyMultisig = genesis.MultisigMeta{
Signers: []address.Address{rootkey},
Signers: []address.Address{remAccTestKey},
Threshold: 1,
VestingDuration: 0,
VestingStart: 0,
@ -108,12 +107,13 @@ var DefaultVerifregRootkeyActor = genesis.Actor{
}
var remAccTestKey, _ = address.NewFromString("t1ceb34gnsc6qk5dt6n7xg6ycwzasjhbxm3iylkiy")
var remAccMeta = genesis.AccountMeta{
Owner: remAccTestKey,
var remAccMeta = genesis.MultisigMeta{
Signers: []address.Address{remAccTestKey},
Threshold: 1,
}
var DefaultRemainderAccountActor = genesis.Actor{
Type: genesis.TAccount,
Type: genesis.TMultisig,
Balance: big.NewInt(0),
Meta: remAccMeta.ActorMeta(),
}
@ -606,7 +606,7 @@ func IsRoundWinner(ctx context.Context, ts *types.TipSet, round abi.ChainEpoch,
buf := new(bytes.Buffer)
if err := miner.MarshalCBOR(buf); err != nil {
return nil, xerrors.Errorf("failed to cbor marshal address: %w")
return nil, xerrors.Errorf("failed to cbor marshal address: %w", err)
}
electionRand, err := store.DrawRandomness(brand.Data, crypto.DomainSeparationTag_ElectionProofProduction, round, buf.Bytes())

216
chain/gen/genesis/genesis.go
View File

@ -2,6 +2,7 @@ package genesis
import (
"context"
"crypto/rand"
"encoding/json"
"fmt"
@ -118,11 +119,6 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, xerrors.Errorf("making new state tree: %w", err)
}
emptyobject, err := cst.Put(context.TODO(), []struct{}{})
if err != nil {
return nil, nil, xerrors.Errorf("failed putting empty object: %w", err)
}
// Create system actor
sysact, err := SetupSystemActor(bs)
@ -135,7 +131,7 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
// Create init actor
initact, keyIDs, err := SetupInitActor(bs, template.NetworkName, template.Accounts, template.VerifregRootKey)
idStart, initact, keyIDs, err := SetupInitActor(bs, template.NetworkName, template.Accounts, template.VerifregRootKey)
if err != nil {
return nil, nil, xerrors.Errorf("setup init actor: %w", err)
}
@ -191,31 +187,47 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, xerrors.Errorf("set market actor: %w", err)
}
burntRoot, err := cst.Put(ctx, &account.State{
Address: builtin.BurntFundsActorAddr,
})
if err != nil {
return nil, nil, xerrors.Errorf("failed to setup burnt funds actor state: %w", err)
}
// Setup burnt-funds
err = state.SetActor(builtin.BurntFundsActorAddr, &types.Actor{
Code: builtin.AccountActorCodeID,
Balance: types.NewInt(0),
Head: emptyobject,
Head: burntRoot,
})
if err != nil {
return nil, nil, xerrors.Errorf("set burnt funds account actor: %w", err)
}
// Create accounts
for id, info := range template.Accounts {
if info.Type != genesis.TAccount && info.Type != genesis.TMultisig {
for _, info := range template.Accounts {
switch info.Type {
case genesis.TAccount:
if err := createAccountActor(ctx, cst, state, info, keyIDs); err != nil {
return nil, nil, xerrors.Errorf("failed to create account actor: %w", err)
}
case genesis.TMultisig:
ida, err := address.NewIDAddress(uint64(idStart))
if err != nil {
return nil, nil, err
}
idStart++
if err := createMultisigAccount(ctx, bs, cst, state, ida, info, keyIDs); err != nil {
return nil, nil, err
}
default:
return nil, nil, xerrors.New("unsupported account type")
}
ida, err := address.NewIDAddress(uint64(AccountStart + id))
if err != nil {
return nil, nil, err
}
if err = createAccount(ctx, bs, cst, state, ida, info); err != nil {
return nil, nil, err
}
}
vregroot, err := address.NewIDAddress(80)
@ -223,8 +235,8 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, err
}
if err = createAccount(ctx, bs, cst, state, vregroot, template.VerifregRootKey); err != nil {
return nil, nil, err
if err = createMultisigAccount(ctx, bs, cst, state, vregroot, template.VerifregRootKey, keyIDs); err != nil {
return nil, nil, xerrors.Errorf("failed to set up verified registry signer: %w", err)
}
// Setup the first verifier as ID-address 81
@ -288,73 +300,104 @@ func MakeInitialStateTree(ctx context.Context, bs bstore.Blockstore, template ge
return nil, nil, err
}
if err := createAccount(ctx, bs, cst, state, remAccKey, template.RemainderAccount); err != nil {
return nil, nil, err
}
err = state.SetActor(remAccKey, &types.Actor{
Code: builtin.AccountActorCodeID,
Balance: remainingFil,
Head: emptyobject,
})
if err != nil {
return nil, nil, xerrors.Errorf("set burnt funds account actor: %w", err)
template.RemainderAccount.Balance = remainingFil
if err := createMultisigAccount(ctx, bs, cst, state, remAccKey, template.RemainderAccount, keyIDs); err != nil {
return nil, nil, xerrors.Errorf("failed to set up remainder account: %w", err)
}
return state, keyIDs, nil
}
func createAccount(ctx context.Context, bs bstore.Blockstore, cst cbor.IpldStore, state *state.StateTree, ida address.Address, info genesis.Actor) error {
if info.Type == genesis.TAccount {
var ainfo genesis.AccountMeta
if err := json.Unmarshal(info.Meta, &ainfo); err != nil {
return xerrors.Errorf("unmarshaling account meta: %w", err)
}
st, err := cst.Put(ctx, &account.State{Address: ainfo.Owner})
if err != nil {
return err
}
err = state.SetActor(ida, &types.Actor{
Code: builtin.AccountActorCodeID,
Balance: info.Balance,
Head: st,
})
if err != nil {
return xerrors.Errorf("setting account from actmap: %w", err)
}
return nil
} else if info.Type == genesis.TMultisig {
var ainfo genesis.MultisigMeta
if err := json.Unmarshal(info.Meta, &ainfo); err != nil {
return xerrors.Errorf("unmarshaling account meta: %w", err)
}
pending, err := adt.MakeEmptyMap(adt.WrapStore(ctx, cst)).Root()
if err != nil {
return xerrors.Errorf("failed to create empty map: %v", err)
}
st, err := cst.Put(ctx, &multisig.State{
Signers: ainfo.Signers,
NumApprovalsThreshold: uint64(ainfo.Threshold),
StartEpoch: abi.ChainEpoch(ainfo.VestingStart),
UnlockDuration: abi.ChainEpoch(ainfo.VestingDuration),
PendingTxns: pending,
InitialBalance: info.Balance,
})
if err != nil {
return err
}
err = state.SetActor(ida, &types.Actor{
Code: builtin.MultisigActorCodeID,
Balance: info.Balance,
Head: st,
})
if err != nil {
return xerrors.Errorf("setting account from actmap: %w", err)
}
return nil
func createAccountActor(ctx context.Context, cst cbor.IpldStore, state *state.StateTree, info genesis.Actor, keyIDs map[address.Address]address.Address) error {
var ainfo genesis.AccountMeta
if err := json.Unmarshal(info.Meta, &ainfo); err != nil {
return xerrors.Errorf("unmarshaling account meta: %w", err)
}
st, err := cst.Put(ctx, &account.State{Address: ainfo.Owner})
if err != nil {
return err
}
return fmt.Errorf("failed to create account")
ida, ok := keyIDs[ainfo.Owner]
if !ok {
return fmt.Errorf("no registered ID for account actor: %s", ainfo.Owner)
}
err = state.SetActor(ida, &types.Actor{
Code: builtin.AccountActorCodeID,
Balance: info.Balance,
Head: st,
})
if err != nil {
return xerrors.Errorf("setting account from actmap: %w", err)
}
return nil
}
func createMultisigAccount(ctx context.Context, bs bstore.Blockstore, cst cbor.IpldStore, state *state.StateTree, ida address.Address, info genesis.Actor, keyIDs map[address.Address]address.Address) error {
if info.Type != genesis.TMultisig {
return fmt.Errorf("can only call createMultisigAccount with multisig Actor info")
}
var ainfo genesis.MultisigMeta
if err := json.Unmarshal(info.Meta, &ainfo); err != nil {
return xerrors.Errorf("unmarshaling account meta: %w", err)
}
pending, err := adt.MakeEmptyMap(adt.WrapStore(ctx, cst)).Root()
if err != nil {
return xerrors.Errorf("failed to create empty map: %v", err)
}
var signers []address.Address
for _, e := range ainfo.Signers {
idAddress, ok := keyIDs[e]
if !ok {
return fmt.Errorf("no registered key ID for signer: %s", e)
}
// Check if actor already exists
_, err := state.GetActor(e)
if err == nil {
signers = append(signers, idAddress)
continue
}
st, err := cst.Put(ctx, &account.State{Address: e})
if err != nil {
return err
}
err = state.SetActor(idAddress, &types.Actor{
Code: builtin.AccountActorCodeID,
Balance: types.NewInt(0),
Head: st,
})
if err != nil {
return xerrors.Errorf("setting account from actmap: %w", err)
}
signers = append(signers, idAddress)
}
st, err := cst.Put(ctx, &multisig.State{
Signers: signers,
NumApprovalsThreshold: uint64(ainfo.Threshold),
StartEpoch: abi.ChainEpoch(ainfo.VestingStart),
UnlockDuration: abi.ChainEpoch(ainfo.VestingDuration),
PendingTxns: pending,
InitialBalance: info.Balance,
})
if err != nil {
return err
}
err = state.SetActor(ida, &types.Actor{
Code: builtin.MultisigActorCodeID,
Balance: info.Balance,
Head: st,
})
if err != nil {
return xerrors.Errorf("setting account from actmap: %w", err)
}
return nil
}
func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot cid.Cid, template genesis.Template, keyIDs map[address.Address]address.Address) (cid.Cid, error) {
@ -375,11 +418,12 @@ func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot ci
return cid.Undef, xerrors.Errorf("failed to create NewVM: %w", err)
}
for _, m := range template.Miners {
for mi, m := range template.Miners {
for si, s := range m.Sectors {
if s.Deal.Provider != m.ID {
return cid.Undef, xerrors.Errorf("Sector %d in miner %d in template had mismatch in provider and miner ID: %s != %s", si, mi, s.Deal.Provider, m.ID)
}
// Add the miner to the market actor's balance table
_, err = doExec(ctx, vm, builtin.StorageMarketActorAddr, m.Owner, builtin.MethodsMarket.AddBalance, mustEnc(adt.Empty))
for _, s := range m.Sectors {
amt := s.Deal.PieceSize
verifNeeds[keyIDs[s.Deal.Client]] += amt
sum += amt
@ -469,8 +513,10 @@ func MakeGenesisBlock(ctx context.Context, bs bstore.Blockstore, sys vm.SyscallB
log.Infof("Empty Genesis root: %s", emptyroot)
tickBuf := make([]byte, 32)
_, _ = rand.Read(tickBuf)
genesisticket := &types.Ticket{
VRFProof: []byte("vrf proof0000000vrf proof0000000"),
VRFProof: tickBuf,
}
filecoinGenesisCid, err := cid.Decode("bafyreiaqpwbbyjo4a42saasj36kkrpv4tsherf2e7bvezkert2a7dhonoi")

18
chain/gen/genesis/miners.go
View File

@ -129,6 +129,9 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
return nil
})
if err != nil {
return cid.Undef, xerrors.Errorf("mutating state: %w", err)
}
}
// Add market funds
@ -137,7 +140,7 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
params := mustEnc(&minerInfos[i].maddr)
_, err := doExecValue(ctx, vm, builtin.StorageMarketActorAddr, m.Worker, m.MarketBalance, builtin.MethodsMarket.AddBalance, params)
if err != nil {
return cid.Undef, xerrors.Errorf("failed to create genesis miner: %w", err)
return cid.Undef, xerrors.Errorf("failed to create genesis miner (add balance): %w", err)
}
}
@ -149,7 +152,7 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
ret, err := doExecValue(ctx, vm, builtin.StorageMarketActorAddr, m.Worker, big.Zero(), builtin.MethodsMarket.PublishStorageDeals, mustEnc(params))
if err != nil {
return xerrors.Errorf("failed to create genesis miner: %w", err)
return xerrors.Errorf("failed to create genesis miner (publish deals): %w", err)
}
var ids market.PublishStorageDealsReturn
if err := ids.UnmarshalCBOR(bytes.NewReader(ret)); err != nil {
@ -217,9 +220,12 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
}
err = vm.MutateState(ctx, builtin.RewardActorAddr, func(sct cbor.IpldStore, st *reward.State) error {
st = reward.ConstructState(qaPow)
*st = *reward.ConstructState(qaPow)
return nil
})
if err != nil {
return cid.Undef, xerrors.Errorf("mutating state: %w", err)
}
}
for i, m := range miners {
@ -244,7 +250,7 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
// we've added fake power for this sector above, remove it now
err = vm.MutateState(ctx, builtin.StoragePowerActorAddr, func(cst cbor.IpldStore, st *power.State) error {
st.TotalQualityAdjPower = types.BigSub(st.TotalQualityAdjPower, sectorWeight)
st.TotalQualityAdjPower = types.BigSub(st.TotalQualityAdjPower, sectorWeight) //nolint:scopelint
st.TotalRawBytePower = types.BigSub(st.TotalRawBytePower, types.NewInt(uint64(m.SectorSize)))
return nil
})
@ -324,13 +330,13 @@ type fakeRand struct{}
func (fr *fakeRand) GetChainRandomness(ctx context.Context, personalization crypto.DomainSeparationTag, randEpoch abi.ChainEpoch, entropy []byte) ([]byte, error) {
out := make([]byte, 32)
_, _ = rand.New(rand.NewSource(int64(randEpoch * 1000))).Read(out)
_, _ = rand.New(rand.NewSource(int64(randEpoch * 1000))).Read(out) //nolint
return out, nil
}
func (fr *fakeRand) GetBeaconRandomness(ctx context.Context, personalization crypto.DomainSeparationTag, randEpoch abi.ChainEpoch, entropy []byte) ([]byte, error) {
out := make([]byte, 32)
_, _ = rand.New(rand.NewSource(int64(randEpoch))).Read(out)
_, _ = rand.New(rand.NewSource(int64(randEpoch))).Read(out) //nolint
return out, nil
}

78
chain/gen/genesis/t01_init.go
View File

@ -20,9 +20,9 @@ import (
bstore "github.com/filecoin-project/lotus/lib/blockstore"
)
func SetupInitActor(bs bstore.Blockstore, netname string, initialActors []genesis.Actor, rootVerifier genesis.Actor) (*types.Actor, map[address.Address]address.Address, error) {
func SetupInitActor(bs bstore.Blockstore, netname string, initialActors []genesis.Actor, rootVerifier genesis.Actor) (int64, *types.Actor, map[address.Address]address.Address, error) {
if len(initialActors) > MaxAccounts {
return nil, nil, xerrors.New("too many initial actors")
return 0, nil, nil, xerrors.New("too many initial actors")
}
var ias init_.State
@ -33,55 +33,105 @@ func SetupInitActor(bs bstore.Blockstore, netname string, initialActors []genesi
amap := adt.MakeEmptyMap(store)
keyToId := map[address.Address]address.Address{}
counter := int64(AccountStart)
for i, a := range initialActors {
for _, a := range initialActors {
if a.Type == genesis.TMultisig {
var ainfo genesis.MultisigMeta
if err := json.Unmarshal(a.Meta, &ainfo); err != nil {
return 0, nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
}
for _, e := range ainfo.Signers {
if _, ok := keyToId[e]; ok {
continue
}
fmt.Printf("init set %s t0%d\n", e, counter)
value := cbg.CborInt(counter)
if err := amap.Put(adt.AddrKey(e), &value); err != nil {
return 0, nil, nil, err
}
counter = counter + 1
var err error
keyToId[e], err = address.NewIDAddress(uint64(value))
if err != nil {
return 0, nil, nil, err
}
}
// Need to add actors for all multisigs too
continue
}
if a.Type != genesis.TAccount {
return nil, nil, xerrors.Errorf("unsupported account type: %s", a.Type) // TODO: Support msig (skip here)
return 0, nil, nil, xerrors.Errorf("unsupported account type: %s", a.Type)
}
var ainfo genesis.AccountMeta
if err := json.Unmarshal(a.Meta, &ainfo); err != nil {
return nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
return 0, nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
}
fmt.Printf("init set %s t0%d\n", ainfo.Owner, AccountStart+int64(i))
fmt.Printf("init set %s t0%d\n", ainfo.Owner, counter)
value := cbg.CborInt(AccountStart + int64(i))
value := cbg.CborInt(counter)
if err := amap.Put(adt.AddrKey(ainfo.Owner), &value); err != nil {
return nil, nil, err
return 0, nil, nil, err
}
counter = counter + 1
var err error
keyToId[ainfo.Owner], err = address.NewIDAddress(uint64(value))
if err != nil {
return nil, nil, err
return 0, nil, nil, err
}
}
if rootVerifier.Type == genesis.TAccount {
var ainfo genesis.AccountMeta
if err := json.Unmarshal(rootVerifier.Meta, &ainfo); err != nil {
return nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
return 0, nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
}
value := cbg.CborInt(80)
if err := amap.Put(adt.AddrKey(ainfo.Owner), &value); err != nil {
return nil, nil, err
return 0, nil, nil, err
}
} else if rootVerifier.Type == genesis.TMultisig {
var ainfo genesis.MultisigMeta
if err := json.Unmarshal(rootVerifier.Meta, &ainfo); err != nil {
return 0, nil, nil, xerrors.Errorf("unmarshaling account meta: %w", err)
}
for _, e := range ainfo.Signers {
if _, ok := keyToId[e]; ok {
continue
}
fmt.Printf("init set %s t0%d\n", e, counter)
value := cbg.CborInt(counter)
if err := amap.Put(adt.AddrKey(e), &value); err != nil {
return 0, nil, nil, err
}
counter = counter + 1
var err error
keyToId[e], err = address.NewIDAddress(uint64(value))
if err != nil {
return 0, nil, nil, err
}
}
}
amapaddr, err := amap.Root()
if err != nil {
return nil, nil, err
return 0, nil, nil, err
}
ias.AddressMap = amapaddr
statecid, err := store.Put(store.Context(), &ias)
if err != nil {
return nil, nil, err
return 0, nil, nil, err
}
act := &types.Actor{
@ -89,5 +139,5 @@ func SetupInitActor(bs bstore.Blockstore, netname string, initialActors []genesi
Head: statecid,
}
return act, keyToId, nil
return counter, act, keyToId, nil
}

6
chain/gen/genesis/util.go
View File

@ -21,14 +21,10 @@ func mustEnc(i cbg.CBORMarshaler) []byte {
return enc
}
func doExec(ctx context.Context, vm *vm.VM, to, from address.Address, method abi.MethodNum, params []byte) ([]byte, error) {
return doExecValue(ctx, vm, to, from, types.NewInt(0), method, params)
}
func doExecValue(ctx context.Context, vm *vm.VM, to, from address.Address, value types.BigInt, method abi.MethodNum, params []byte) ([]byte, error) {
act, err := vm.StateTree().GetActor(from)
if err != nil {
return nil, xerrors.Errorf("doExec failed to get from actor: %w", err)
return nil, xerrors.Errorf("doExec failed to get from actor (%s): %w", from, err)
}
ret, err := vm.ApplyImplicitMessage(ctx, &types.Message{

1
chain/market/fundmgr_test.go
View File

@ -147,6 +147,7 @@ func TestAddFunds(t *testing.T) {
}
for testCase, data := range testCases {
//nolint:scopelint
t.Run(testCase, func(t *testing.T) {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

2
chain/messagepool/block_proba.go
View File

@ -56,7 +56,7 @@ func binomialCoefficient(n, k float64) float64 {
for d := 1.0; d <= k; d++ {
r *= n
r /= d
n -= 1
n--
}
return r
}

203
chain/messagepool/messagepool.go
View File

@ -12,6 +12,7 @@ import (
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/hashicorp/go-multierror"
lru "github.com/hashicorp/golang-lru"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
@ -26,6 +27,7 @@ import (
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
"github.com/filecoin-project/lotus/lib/sigs"
@ -36,7 +38,7 @@ import (
var log = logging.Logger("messagepool")
const futureDebug = false
var futureDebug = false
var rbfNumBig = types.NewInt(uint64((ReplaceByFeeRatioDefault - 1) * RbfDenom))
var rbfDenomBig = types.NewInt(RbfDenom)
@ -188,12 +190,7 @@ func New(api Provider, ds dtypes.MetadataDS, netName dtypes.NetworkName) (*Messa
// enable initial prunes
mp.pruneCooldown <- struct{}{}
if err := mp.loadLocal(); err != nil {
log.Errorf("loading local messages: %+v", err)
}
go mp.runLoop()
// load the current tipset and subscribe to head changes _before_ loading local messages
mp.curTs = api.SubscribeHeadChanges(func(rev, app []*types.TipSet) error {
err := mp.HeadChange(rev, app)
if err != nil {
@ -202,6 +199,12 @@ func New(api Provider, ds dtypes.MetadataDS, netName dtypes.NetworkName) (*Messa
return err
})
if err := mp.loadLocal(); err != nil {
log.Errorf("loading local messages: %+v", err)
}
go mp.runLoop()
return mp, nil
}
@ -316,6 +319,11 @@ func (mp *MessagePool) checkMessage(m *types.SignedMessage) error {
return xerrors.Errorf("mpool message too large (%dB): %w", m.Size(), ErrMessageTooBig)
}
// Perform syntaxtic validation, minGas=0 as we check if correctly in select messages
if err := m.Message.ValidForBlockInclusion(0); err != nil {
return xerrors.Errorf("message not valid for block inclusion: %w", err)
}
if m.Message.To == address.Undef {
return ErrInvalidToAddr
}
@ -498,42 +506,16 @@ func (mp *MessagePool) getNonceLocked(addr address.Address, curTs *types.TipSet)
}
func (mp *MessagePool) getStateNonce(addr address.Address, curTs *types.TipSet) (uint64, error) {
// TODO: this method probably should be cached
act, err := mp.api.StateGetActor(addr, curTs)
act, err := mp.api.GetActorAfter(addr, curTs)
if err != nil {
return 0, err
}
baseNonce := act.Nonce
// TODO: the correct thing to do here is probably to set curTs to chain.head
// but since we have an accurate view of the world until a head change occurs,
// this should be fine
if curTs == nil {
return baseNonce, nil
}
msgs, err := mp.api.MessagesForTipset(curTs)
if err != nil {
return 0, xerrors.Errorf("failed to check messages for tipset: %w", err)
}
for _, m := range msgs {
msg := m.VMMessage()
if msg.From == addr {
if msg.Nonce != baseNonce {
return 0, xerrors.Errorf("tipset %s has bad nonce ordering (%d != %d)", curTs.Cids(), msg.Nonce, baseNonce)
}
baseNonce++
}
}
return baseNonce, nil
return act.Nonce, nil
}
func (mp *MessagePool) getStateBalance(addr address.Address, ts *types.TipSet) (types.BigInt, error) {
act, err := mp.api.StateGetActor(addr, ts)
act, err := mp.api.GetActorAfter(addr, ts)
if err != nil {
return types.EmptyInt, err
}
@ -669,6 +651,10 @@ func (mp *MessagePool) Pending() ([]*types.SignedMessage, *types.TipSet) {
mp.lk.Lock()
defer mp.lk.Unlock()
return mp.allPending()
}
func (mp *MessagePool) allPending() ([]*types.SignedMessage, *types.TipSet) {
out := make([]*types.SignedMessage, 0)
for a := range mp.pending {
out = append(out, mp.pendingFor(a)...)
@ -676,6 +662,7 @@ func (mp *MessagePool) Pending() ([]*types.SignedMessage, *types.TipSet) {
return out, mp.curTs
}
func (mp *MessagePool) PendingFor(a address.Address) ([]*types.SignedMessage, *types.TipSet) {
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
@ -744,30 +731,42 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
}
}
var merr error
for _, ts := range revert {
pts, err := mp.api.LoadTipSet(ts.Parents())
if err != nil {
return err
}
msgs, err := mp.MessagesForBlocks(ts.Blocks())
if err != nil {
return err
log.Errorf("error loading reverted tipset parent: %s", err)
merr = multierror.Append(merr, err)
continue
}
mp.curTs = pts
msgs, err := mp.MessagesForBlocks(ts.Blocks())
if err != nil {
log.Errorf("error retrieving messages for reverted block: %s", err)
merr = multierror.Append(merr, err)
continue
}
for _, msg := range msgs {
add(msg)
}
}
for _, ts := range apply {
mp.curTs = ts
for _, b := range ts.Blocks() {
bmsgs, smsgs, err := mp.api.MessagesForBlock(b)
if err != nil {
return xerrors.Errorf("failed to get messages for apply block %s(height %d) (msgroot = %s): %w", b.Cid(), b.Height, b.Messages, err)
xerr := xerrors.Errorf("failed to get messages for apply block %s(height %d) (msgroot = %s): %w", b.Cid(), b.Height, b.Messages, err)
log.Errorf("error retrieving messages for block: %s", xerr)
merr = multierror.Append(merr, xerr)
continue
}
for _, msg := range smsgs {
rm(msg.Message.From, msg.Message.Nonce)
maybeRepub(msg.Cid())
@ -778,8 +777,6 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
maybeRepub(msg.Cid())
}
}
mp.curTs = ts
}
if repubTrigger {
@ -798,7 +795,9 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
}
if len(revert) > 0 && futureDebug {
msgs, ts := mp.Pending()
mp.lk.Lock()
msgs, ts := mp.allPending()
mp.lk.Unlock()
buckets := map[address.Address]*statBucket{}
@ -815,7 +814,8 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
}
for a, bkt := range buckets {
act, err := mp.api.StateGetActor(a, ts)
// TODO that might not be correct with GatActorAfter but it is only debug code
act, err := mp.api.GetActorAfter(a, ts)
if err != nil {
log.Debugf("%s, err: %s\n", a, err)
continue
@ -862,7 +862,72 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
}
}
return nil
return merr
}
func (mp *MessagePool) runHeadChange(from *types.TipSet, to *types.TipSet, rmsgs map[address.Address]map[uint64]*types.SignedMessage) error {
add := func(m *types.SignedMessage) {
s, ok := rmsgs[m.Message.From]
if !ok {
s = make(map[uint64]*types.SignedMessage)
rmsgs[m.Message.From] = s
}
s[m.Message.Nonce] = m
}
rm := func(from address.Address, nonce uint64) {
s, ok := rmsgs[from]
if !ok {
return
}
if _, ok := s[nonce]; ok {
delete(s, nonce)
return
}
}
revert, apply, err := store.ReorgOps(mp.api.LoadTipSet, from, to)
if err != nil {
return xerrors.Errorf("failed to compute reorg ops for mpool pending messages: %w", err)
}
var merr error
for _, ts := range revert {
msgs, err := mp.MessagesForBlocks(ts.Blocks())
if err != nil {
log.Errorf("error retrieving messages for reverted block: %s", err)
merr = multierror.Append(merr, err)
continue
}
for _, msg := range msgs {
add(msg)
}
}
for _, ts := range apply {
for _, b := range ts.Blocks() {
bmsgs, smsgs, err := mp.api.MessagesForBlock(b)
if err != nil {
xerr := xerrors.Errorf("failed to get messages for apply block %s(height %d) (msgroot = %s): %w", b.Cid(), b.Height, b.Messages, err)
log.Errorf("error retrieving messages for block: %s", xerr)
merr = multierror.Append(merr, xerr)
continue
}
for _, msg := range smsgs {
rm(msg.Message.From, msg.Message.Nonce)
}
for _, msg := range bmsgs {
rm(msg.From, msg.Nonce)
}
}
}
return merr
}
type statBucket struct {
@ -915,6 +980,7 @@ func (mp *MessagePool) Updates(ctx context.Context) (<-chan api.MpoolUpdate, err
go func() {
defer mp.changes.Unsub(sub, localUpdates)
defer close(out)
for {
select {
@ -923,9 +989,13 @@ func (mp *MessagePool) Updates(ctx context.Context) (<-chan api.MpoolUpdate, err
case out <- u.(api.MpoolUpdate):
case <-ctx.Done():
return
case <-mp.closer:
return
}
case <-ctx.Done():
return
case <-mp.closer:
return
}
}
}()
@ -962,3 +1032,40 @@ func (mp *MessagePool) loadLocal() error {
return nil
}
func (mp *MessagePool) Clear(local bool) {
mp.lk.Lock()
defer mp.lk.Unlock()
// remove everything if local is true, including removing local messages from
// the datastore
if local {
for a := range mp.localAddrs {
mset, ok := mp.pending[a]
if !ok {
continue
}
for _, m := range mset.msgs {
err := mp.localMsgs.Delete(datastore.NewKey(string(m.Cid().Bytes())))
if err != nil {
log.Warnf("error deleting local message: %s", err)
}
}
}
mp.pending = make(map[address.Address]*msgSet)
mp.republished = nil
return
}
// remove everything except the local messages
for a := range mp.pending {
_, isLocal := mp.localAddrs[a]
if isLocal {
continue
}
delete(mp.pending, a)
}
}

354
chain/messagepool/messagepool_test.go
View File

@ -3,9 +3,11 @@ package messagepool
import (
"context"
"fmt"
"sort"
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/chain/wallet"
@ -30,14 +32,25 @@ type testMpoolAPI struct {
balance map[address.Address]types.BigInt
tipsets []*types.TipSet
published int
}
func newTestMpoolAPI() *testMpoolAPI {
return &testMpoolAPI{
tma := &testMpoolAPI{
bmsgs: make(map[cid.Cid][]*types.SignedMessage),
statenonce: make(map[address.Address]uint64),
balance: make(map[address.Address]types.BigInt),
}
genesis := mock.MkBlock(nil, 1, 1)
tma.tipsets = append(tma.tipsets, mock.TipSet(genesis))
return tma
}
func (tma *testMpoolAPI) nextBlock() *types.BlockHeader {
newBlk := mock.MkBlock(tma.tipsets[len(tma.tipsets)-1], 1, 1)
tma.tipsets = append(tma.tipsets, mock.TipSet(newBlk))
return newBlk
}
func (tma *testMpoolAPI) applyBlock(t *testing.T, b *types.BlockHeader) {
@ -68,12 +81,11 @@ func (tma *testMpoolAPI) setBalanceRaw(addr address.Address, v types.BigInt) {
func (tma *testMpoolAPI) setBlockMessages(h *types.BlockHeader, msgs ...*types.SignedMessage) {
tma.bmsgs[h.Cid()] = msgs
tma.tipsets = append(tma.tipsets, mock.TipSet(h))
}
func (tma *testMpoolAPI) SubscribeHeadChanges(cb func(rev, app []*types.TipSet) error) *types.TipSet {
tma.cb = cb
return nil
return tma.tipsets[0]
}
func (tma *testMpoolAPI) PutMessage(m types.ChainMsg) (cid.Cid, error) {
@ -81,18 +93,47 @@ func (tma *testMpoolAPI) PutMessage(m types.ChainMsg) (cid.Cid, error) {
}
func (tma *testMpoolAPI) PubSubPublish(string, []byte) error {
tma.published++
return nil
}
func (tma *testMpoolAPI) StateGetActor(addr address.Address, ts *types.TipSet) (*types.Actor, error) {
func (tma *testMpoolAPI) GetActorAfter(addr address.Address, ts *types.TipSet) (*types.Actor, error) {
// regression check for load bug
if ts == nil {
panic("GetActorAfter called with nil tipset")
}
balance, ok := tma.balance[addr]
if !ok {
balance = types.NewInt(1000e6)
tma.balance[addr] = balance
}
msgs := make([]*types.SignedMessage, 0)
for _, b := range ts.Blocks() {
for _, m := range tma.bmsgs[b.Cid()] {
if m.Message.From == addr {
msgs = append(msgs, m)
}
}
}
sort.Slice(msgs, func(i, j int) bool {
return msgs[i].Message.Nonce < msgs[j].Message.Nonce
})
nonce := tma.statenonce[addr]
for _, m := range msgs {
if m.Message.Nonce != nonce {
break
}
nonce++
}
return &types.Actor{
Code: builtin.StorageMarketActorCodeID,
Nonce: tma.statenonce[addr],
Nonce: nonce,
Balance: balance,
}, nil
}
@ -178,7 +219,7 @@ func TestMessagePool(t *testing.T) {
t.Fatal(err)
}
a := mock.MkBlock(nil, 1, 1)
a := tma.nextBlock()
sender, err := w.GenerateKey(crypto.SigTypeBLS)
if err != nil {
@ -204,7 +245,7 @@ func TestMessagePool(t *testing.T) {
assertNonce(t, mp, sender, 2)
}
func TestRevertMessages(t *testing.T) {
func TestMessagePoolMessagesInEachBlock(t *testing.T) {
tma := newTestMpoolAPI()
w, err := wallet.NewWallet(wallet.NewMemKeyStore())
@ -219,8 +260,57 @@ func TestRevertMessages(t *testing.T) {
t.Fatal(err)
}
a := mock.MkBlock(nil, 1, 1)
b := mock.MkBlock(mock.TipSet(a), 1, 1)
a := tma.nextBlock()
sender, err := w.GenerateKey(crypto.SigTypeBLS)
if err != nil {
t.Fatal(err)
}
target := mock.Address(1001)
var msgs []*types.SignedMessage
for i := 0; i < 5; i++ {
m := mock.MkMessage(sender, target, uint64(i), w)
msgs = append(msgs, m)
mustAdd(t, mp, m)
}
tma.setStateNonce(sender, 0)
tma.setBlockMessages(a, msgs[0], msgs[1])
tma.applyBlock(t, a)
tsa := mock.TipSet(a)
_, _ = mp.Pending()
selm, _ := mp.SelectMessages(tsa, 1)
if len(selm) == 0 {
t.Fatal("should have returned the rest of the messages")
}
}
func TestRevertMessages(t *testing.T) {
futureDebug = true
defer func() {
futureDebug = false
}()
tma := newTestMpoolAPI()
w, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
a := tma.nextBlock()
b := tma.nextBlock()
sender, err := w.GenerateKey(crypto.SigTypeBLS)
if err != nil {
@ -254,6 +344,7 @@ func TestRevertMessages(t *testing.T) {
assertNonce(t, mp, sender, 4)
p, _ := mp.Pending()
fmt.Printf("%+v\n", p)
if len(p) != 3 {
t.Fatal("expected three messages in mempool")
}
@ -275,7 +366,7 @@ func TestPruningSimple(t *testing.T) {
t.Fatal(err)
}
a := mock.MkBlock(nil, 1, 1)
a := tma.nextBlock()
tma.applyBlock(t, a)
sender, err := w.GenerateKey(crypto.SigTypeBLS)
@ -308,3 +399,246 @@ func TestPruningSimple(t *testing.T) {
t.Fatal("expected only 5 messages in pool, got: ", len(msgs))
}
}
func TestLoadLocal(t *testing.T) {
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
msgs := make(map[cid.Cid]struct{})
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
cid, err := mp.Push(m)
if err != nil {
t.Fatal(err)
}
msgs[cid] = struct{}{}
}
err = mp.Close()
if err != nil {
t.Fatal(err)
}
mp, err = New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
pmsgs, _ := mp.Pending()
if len(msgs) != len(pmsgs) {
t.Fatalf("expected %d messages, but got %d", len(msgs), len(pmsgs))
}
for _, m := range pmsgs {
cid := m.Cid()
_, ok := msgs[cid]
if !ok {
t.Fatal("unknown message")
}
delete(msgs, cid)
}
if len(msgs) > 0 {
t.Fatalf("not all messages were laoded; missing %d messages", len(msgs))
}
}
func TestClearAll(t *testing.T) {
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
_, err := mp.Push(m)
if err != nil {
t.Fatal(err)
}
}
for i := 0; i < 10; i++ {
m := makeTestMessage(w2, a2, a1, uint64(i), gasLimit, uint64(i+1))
mustAdd(t, mp, m)
}
mp.Clear(true)
pending, _ := mp.Pending()
if len(pending) > 0 {
t.Fatalf("cleared the mpool, but got %d pending messages", len(pending))
}
}
func TestClearNonLocal(t *testing.T) {
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
_, err := mp.Push(m)
if err != nil {
t.Fatal(err)
}
}
for i := 0; i < 10; i++ {
m := makeTestMessage(w2, a2, a1, uint64(i), gasLimit, uint64(i+1))
mustAdd(t, mp, m)
}
mp.Clear(false)
pending, _ := mp.Pending()
if len(pending) != 10 {
t.Fatalf("expected 10 pending messages, but got %d instead", len(pending))
}
for _, m := range pending {
if m.Message.From != a1 {
t.Fatalf("expected message from %s but got one from %s instead", a1, m.Message.From)
}
}
}
func TestUpdates(t *testing.T) {
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.TODO())
defer cancel()
ch, err := mp.Updates(ctx)
if err != nil {
t.Fatal(err)
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
_, err := mp.Push(m)
if err != nil {
t.Fatal(err)
}
_, ok := <-ch
if !ok {
t.Fatal("expected update, but got a closed channel instead")
}
}
err = mp.Close()
if err != nil {
t.Fatal(err)
}
_, ok := <-ch
if ok {
t.Fatal("expected closed channel, but got an update instead")
}
}

13
chain/messagepool/provider.go
View File

@ -16,7 +16,7 @@ type Provider interface {
SubscribeHeadChanges(func(rev, app []*types.TipSet) error) *types.TipSet
PutMessage(m types.ChainMsg) (cid.Cid, error)
PubSubPublish(string, []byte) error
StateGetActor(address.Address, *types.TipSet) (*types.Actor, error)
GetActorAfter(address.Address, *types.TipSet) (*types.Actor, error)
StateAccountKey(context.Context, address.Address, *types.TipSet) (address.Address, error)
MessagesForBlock(*types.BlockHeader) ([]*types.Message, []*types.SignedMessage, error)
MessagesForTipset(*types.TipSet) ([]types.ChainMsg, error)
@ -43,12 +43,17 @@ func (mpp *mpoolProvider) PutMessage(m types.ChainMsg) (cid.Cid, error) {
}
func (mpp *mpoolProvider) PubSubPublish(k string, v []byte) error {
return mpp.ps.Publish(k, v)
return mpp.ps.Publish(k, v) //nolint
}
func (mpp *mpoolProvider) StateGetActor(addr address.Address, ts *types.TipSet) (*types.Actor, error) {
func (mpp *mpoolProvider) GetActorAfter(addr address.Address, ts *types.TipSet) (*types.Actor, error) {
var act types.Actor
return &act, mpp.sm.WithParentState(ts, mpp.sm.WithActor(addr, stmgr.GetActor(&act)))
stcid, _, err := mpp.sm.TipSetState(context.TODO(), ts)
if err != nil {
return nil, xerrors.Errorf("computing tipset state for GetActor: %w", err)
}
return &act, mpp.sm.WithStateTree(stcid, mpp.sm.WithActor(addr, stmgr.GetActor(&act)))
}
func (mpp *mpoolProvider) StateAccountKey(ctx context.Context, addr address.Address, ts *types.TipSet) (address.Address, error) {

66
chain/messagepool/repub_test.go Normal file
View File

@ -0,0 +1,66 @@
package messagepool
import (
"testing"
"time"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-datastore"
)
func TestRepubMessages(t *testing.T) {
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
mp, err := New(tma, ds, "mptest")
if err != nil {
t.Fatal(err)
}
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
tma.setBalance(a1, 1) // in FIL
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
_, err := mp.Push(m)
if err != nil {
t.Fatal(err)
}
}
if tma.published != 10 {
t.Fatalf("expected to have published 10 messages, but got %d instead", tma.published)
}
mp.repubTrigger <- struct{}{}
time.Sleep(100 * time.Millisecond)
if tma.published != 20 {
t.Fatalf("expected to have published 20 messages, but got %d instead", tma.published)
}
}

82
chain/messagepool/selection.go
View File

@ -14,7 +14,6 @@ import (
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
abig "github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/ipfs/go-cid"
)
var bigBlockGasLimit = big.NewInt(build.BlockGasLimit)
@ -528,7 +527,6 @@ func (mp *MessagePool) getPendingMessages(curTs, ts *types.TipSet) (map[address.
start := time.Now()
result := make(map[address.Address]map[uint64]*types.SignedMessage)
haveCids := make(map[cid.Cid]struct{})
defer func() {
if dt := time.Since(start); dt > time.Millisecond {
log.Infow("get pending messages done", "took", dt)
@ -554,10 +552,6 @@ func (mp *MessagePool) getPendingMessages(curTs, ts *types.TipSet) (map[address.
}
result[a] = msetCopy
// mark the messages as seen
for _, m := range mset.msgs {
haveCids[m.Cid()] = struct{}{}
}
}
}
@ -566,72 +560,11 @@ func (mp *MessagePool) getPendingMessages(curTs, ts *types.TipSet) (map[address.
return result, nil
}
// nope, we need to sync the tipsets
for {
if curTs.Height() == ts.Height() {
if curTs.Equals(ts) {
return result, nil
}
// different blocks in tipsets -- we mark them as seen so that they are not included in
// in the message set we return, but *neither me (vyzo) nor why understand why*
// this code is also probably completely untested in production, so I am adding a big fat
// warning to revisit this case and sanity check this decision.
log.Warnf("mpool tipset has same height as target tipset but it's not equal; beware of dragons!")
have, err := mp.MessagesForBlocks(ts.Blocks())
if err != nil {
return nil, xerrors.Errorf("error retrieving messages for tipset: %w", err)
}
for _, m := range have {
haveCids[m.Cid()] = struct{}{}
}
}
msgs, err := mp.MessagesForBlocks(ts.Blocks())
if err != nil {
return nil, xerrors.Errorf("error retrieving messages for tipset: %w", err)
}
for _, m := range msgs {
if _, have := haveCids[m.Cid()]; have {
continue
}
haveCids[m.Cid()] = struct{}{}
mset, ok := result[m.Message.From]
if !ok {
mset = make(map[uint64]*types.SignedMessage)
result[m.Message.From] = mset
}
other, dupNonce := mset[m.Message.Nonce]
if dupNonce {
// duplicate nonce, selfishly keep the message with the highest GasPrice
// if the gas prices are the same, keep the one with the highest GasLimit
switch m.Message.GasPremium.Int.Cmp(other.Message.GasPremium.Int) {
case 0:
if m.Message.GasLimit > other.Message.GasLimit {
mset[m.Message.Nonce] = m
}
case 1:
mset[m.Message.Nonce] = m
}
} else {
mset[m.Message.Nonce] = m
}
}
if curTs.Height() >= ts.Height() {
return result, nil
}
ts, err = mp.api.LoadTipSet(ts.Parents())
if err != nil {
return nil, xerrors.Errorf("error loading parent tipset: %w", err)
}
if err := mp.runHeadChange(curTs, ts, result); err != nil {
return nil, xerrors.Errorf("failed to process difference between mpool head and given head: %w", err)
}
return result, nil
}
func (mp *MessagePool) getGasReward(msg *types.SignedMessage, baseFee types.BigInt, ts *types.TipSet) *big.Int {
@ -671,7 +604,12 @@ func (mp *MessagePool) createMessageChains(actor address.Address, mset map[uint6
// cannot exceed the block limit; drop all messages that exceed the limit
// - the total gasReward cannot exceed the actor's balance; drop all messages that exceed
// the balance
a, _ := mp.api.StateGetActor(actor, ts)
a, err := mp.api.GetActorAfter(actor, ts)
if err != nil {
log.Errorf("failed to load actor state, not building chain for %s: %w", actor, err)
return nil
}
curNonce := a.Nonce
balance := a.Balance.Int
gasLimit := int64(0)

106
chain/messagepool/selection_test.go
View File

@ -79,7 +79,7 @@ func TestMessageChains(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
@ -317,7 +317,7 @@ func TestMessageChainSkipping(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
@ -387,7 +387,7 @@ func TestBasicMessageSelection(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -440,12 +440,12 @@ func TestBasicMessageSelection(t *testing.T) {
}
// now we make a block with all the messages and advance the chain
block2 := mock.MkBlock(ts, 2, 2)
block2 := tma.nextBlock()
tma.setBlockMessages(block2, msgs...)
tma.applyBlock(t, block2)
// we should have no pending messages in the mpool
pend, ts2 := mp.Pending()
pend, _ := mp.Pending()
if len(pend) != 0 {
t.Fatalf("expected no pending messages, but got %d", len(pend))
}
@ -458,13 +458,13 @@ func TestBasicMessageSelection(t *testing.T) {
m = makeTestMessage(w2, a2, a1, uint64(i), gasLimit, uint64(i+1))
msgs = append(msgs, m)
}
block3 := mock.MkBlock(ts2, 3, 3)
block3 := tma.nextBlock()
tma.setBlockMessages(block3, msgs...)
ts3 := mock.TipSet(block3)
// now create another set of messages and add them to the mpool
for i := 20; i < 30; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(2*i+1))
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(2*i+200))
mustAdd(t, mp, m)
m = makeTestMessage(w2, a2, a1, uint64(i), gasLimit, uint64(i+1))
mustAdd(t, mp, m)
@ -480,12 +480,12 @@ func TestBasicMessageSelection(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if len(msgs) != 40 {
t.Fatalf("expected 40 messages, got %d", len(msgs))
if len(msgs) != 20 {
t.Fatalf("expected 20 messages, got %d", len(msgs))
}
nextNonce = 10
for i := 0; i < 20; i++ {
nextNonce = 20
for i := 0; i < 10; i++ {
if msgs[i].Message.From != a1 {
t.Fatalf("expected message from actor a1")
}
@ -495,8 +495,8 @@ func TestBasicMessageSelection(t *testing.T) {
nextNonce++
}
nextNonce = 10
for i := 20; i < 40; i++ {
nextNonce = 20
for i := 10; i < 20; i++ {
if msgs[i].Message.From != a2 {
t.Fatalf("expected message from actor a2")
}
@ -531,7 +531,7 @@ func TestMessageSelectionTrimming(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -594,7 +594,7 @@ func TestPriorityMessageSelection(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -649,6 +649,73 @@ func TestPriorityMessageSelection(t *testing.T) {
}
}
func TestPriorityMessageSelection2(t *testing.T) {
mp, tma := makeTestMpool()
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
mp.cfg.PriorityAddrs = []address.Address{a1}
nMessages := int(2 * build.BlockGasLimit / gasLimit)
for i := 0; i < nMessages; i++ {
bias := (nMessages - i) / 3
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(1+i%3+bias))
mustAdd(t, mp, m)
m = makeTestMessage(w2, a2, a1, uint64(i), gasLimit, uint64(1+i%3+bias))
mustAdd(t, mp, m)
}
msgs, err := mp.SelectMessages(ts, 1.0)
if err != nil {
t.Fatal(err)
}
expectedMsgs := int(build.BlockGasLimit / gasLimit)
if len(msgs) != expectedMsgs {
t.Fatalf("expected %d messages but got %d", expectedMsgs, len(msgs))
}
// all messages must be from a1
nextNonce := uint64(0)
for _, m := range msgs {
if m.Message.From != a1 {
t.Fatal("expected messages from a1 before messages from a2")
}
if m.Message.Nonce != nextNonce {
t.Fatalf("expected nonce %d but got %d", nextNonce, m.Message.Nonce)
}
nextNonce++
}
}
func TestOptimalMessageSelection1(t *testing.T) {
// this test uses just a single actor sending messages with a low tq
// the chain depenent merging algorithm should pick messages from the actor
@ -676,7 +743,7 @@ func TestOptimalMessageSelection1(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -743,7 +810,7 @@ func TestOptimalMessageSelection2(t *testing.T) {
t.Fatal(err)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -821,7 +888,7 @@ func TestOptimalMessageSelection3(t *testing.T) {
wallets = append(wallets, w)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
@ -879,7 +946,6 @@ func testCompetitiveMessageSelection(t *testing.T, rng *rand.Rand, getPremium fu
// actors send with an randomly distributed premium dictated by the getPremium function.
// a number of miners select with varying ticket quality and we compare the
// capacity and rewards of greedy selection -vs- optimal selection
mp, tma := makeTestMpool()
nActors := 300
@ -902,7 +968,7 @@ func testCompetitiveMessageSelection(t *testing.T, rng *rand.Rand, getPremium fu
wallets = append(wallets, w)
}
block := mock.MkBlock(nil, 1, 1)
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)

3
chain/metrics/consensus.go
View File

@ -44,7 +44,7 @@ func SendHeadNotifs(nickname string) func(mctx helpers.MetricsCtx, lc fx.Lifecyc
}
}()
go func() {
sub, err := ps.Subscribe(topic)
sub, err := ps.Subscribe(topic) //nolint
if err != nil {
return
}
@ -116,6 +116,7 @@ func sendHeadNotifs(ctx context.Context, ps *pubsub.PubSub, topic string, chain
return err
}
//nolint
if err := ps.Publish(topic, b); err != nil {
return err
}

4
chain/state/statetree.go
View File

@ -246,7 +246,7 @@ func (st *StateTree) DeleteActor(addr address.Address) error {
}
func (st *StateTree) Flush(ctx context.Context) (cid.Cid, error) {
ctx, span := trace.StartSpan(ctx, "stateTree.Flush")
ctx, span := trace.StartSpan(ctx, "stateTree.Flush") //nolint:staticcheck
defer span.End()
if len(st.snaps.layers) != 1 {
return cid.Undef, xerrors.Errorf("tried to flush state tree with snapshots on the stack")
@ -268,7 +268,7 @@ func (st *StateTree) Flush(ctx context.Context) (cid.Cid, error) {
}
func (st *StateTree) Snapshot(ctx context.Context) error {
ctx, span := trace.StartSpan(ctx, "stateTree.SnapShot")
ctx, span := trace.StartSpan(ctx, "stateTree.SnapShot") //nolint:staticcheck
defer span.End()
st.snaps.addLayer()

2
chain/store/index_test.go
View File

@ -42,7 +42,7 @@ func TestIndexSeeks(t *testing.T) {
if err := cs.PutTipSet(ctx, mock.TipSet(gen)); err != nil {
t.Fatal(err)
}
cs.SetGenesis(gen)
assert.NoError(t, cs.SetGenesis(gen))
// Put 113 blocks from genesis
for i := 0; i < 113; i++ {

13
chain/store/store.go
View File

@ -483,6 +483,10 @@ func (cs *ChainStore) NearestCommonAncestor(a, b *types.TipSet) (*types.TipSet,
}
func (cs *ChainStore) ReorgOps(a, b *types.TipSet) ([]*types.TipSet, []*types.TipSet, error) {
return ReorgOps(cs.LoadTipSet, a, b)
}
func ReorgOps(lts func(types.TipSetKey) (*types.TipSet, error), a, b *types.TipSet) ([]*types.TipSet, []*types.TipSet, error) {
left := a
right := b
@ -490,7 +494,7 @@ func (cs *ChainStore) ReorgOps(a, b *types.TipSet) ([]*types.TipSet, []*types.Ti
for !left.Equals(right) {
if left.Height() > right.Height() {
leftChain = append(leftChain, left)
par, err := cs.LoadTipSet(left.Parents())
par, err := lts(left.Parents())
if err != nil {
return nil, nil, err
}
@ -498,7 +502,7 @@ func (cs *ChainStore) ReorgOps(a, b *types.TipSet) ([]*types.TipSet, []*types.Ti
left = par
} else {
rightChain = append(rightChain, right)
par, err := cs.LoadTipSet(right.Parents())
par, err := lts(right.Parents())
if err != nil {
log.Infof("failed to fetch right.Parents: %s", err)
return nil, nil, err
@ -509,6 +513,7 @@ func (cs *ChainStore) ReorgOps(a, b *types.TipSet) ([]*types.TipSet, []*types.Ti
}
return leftChain, rightChain, nil
}
// GetHeaviestTipSet returns the current heaviest tipset known (i.e. our head).
@ -925,7 +930,7 @@ func (cs *ChainStore) LoadMessagesFromCids(cids []cid.Cid) ([]*types.Message, er
for i, c := range cids {
m, err := cs.GetMessage(c)
if err != nil {
return nil, xerrors.Errorf("failed to get message: (%s):%d: %w", err, c, i)
return nil, xerrors.Errorf("failed to get message: (%s):%d: %w", c, i, err)
}
msgs = append(msgs, m)
@ -939,7 +944,7 @@ func (cs *ChainStore) LoadSignedMessagesFromCids(cids []cid.Cid) ([]*types.Signe
for i, c := range cids {
m, err := cs.GetSignedMessage(c)
if err != nil {
return nil, xerrors.Errorf("failed to get message: (%s):%d: %w", err, c, i)
return nil, xerrors.Errorf("failed to get message: (%s):%d: %w", c, i, err)
}
msgs = append(msgs, m)

11
chain/sub/incoming.go
View File

@ -292,10 +292,9 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
log.Warnf("received block from unknown miner or miner that doesn't meet min power over pubsub; rejecting message")
recordFailure("unknown_miner")
return pubsub.ValidationReject
} else {
log.Warnf("cannot validate block message; unknown miner or miner that doesn't meet min power in unsynced chain")
return pubsub.ValidationIgnore
}
log.Warnf("cannot validate block message; unknown miner or miner that doesn't meet min power in unsynced chain")
return pubsub.ValidationIgnore
}
err = sigs.CheckBlockSignature(ctx, blk.Header, key)
@ -546,7 +545,11 @@ func (mv *MessageValidator) Validate(ctx context.Context, pid peer.ID, msg *pubs
)
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
switch {
case xerrors.Is(err, messagepool.ErrBroadcastAnyway) || xerrors.Is(err, messagepool.ErrRBFTooLowPremium):
case xerrors.Is(err, messagepool.ErrBroadcastAnyway):
fallthrough
case xerrors.Is(err, messagepool.ErrRBFTooLowPremium):
fallthrough
case xerrors.Is(err, messagepool.ErrNonceTooLow):
return pubsub.ValidationIgnore
default:
return pubsub.ValidationReject

16
chain/sync.go
View File

@ -7,6 +7,7 @@ import (
"fmt"
"os"
"sort"
"strconv"
"strings"
"time"
@ -52,6 +53,19 @@ import (
//the theoretical max height based on systime are quickly rejected
const MaxHeightDrift = 5
var defaultMessageFetchWindowSize = 200
func init() {
if s := os.Getenv("LOTUS_BSYNC_MSG_WINDOW"); s != "" {
val, err := strconv.Atoi(s)
if err != nil {
log.Errorf("failed to parse LOTUS_BSYNC_MSG_WINDOW: %s", err)
return
}
defaultMessageFetchWindowSize = val
}
}
var log = logging.Logger("chain")
var LocalIncoming = "incoming"
@ -1399,7 +1413,7 @@ func (syncer *Syncer) iterFullTipsets(ctx context.Context, headers []*types.TipS
span.AddAttributes(trace.Int64Attribute("num_headers", int64(len(headers))))
windowSize := 200
windowSize := defaultMessageFetchWindowSize
for i := len(headers) - 1; i >= 0; {
fts, err := syncer.store.TryFillTipSet(headers[i])
if err != nil {

10
chain/sync_manager.go
View File

@ -343,12 +343,12 @@ func (sm *SyncManager) scheduleProcessResult(res *syncResult) {
sm.syncQueue.buckets = append(sm.syncQueue.buckets, relbucket)
}
return
} else {
// TODO: this is the case where we try to sync a chain, and
// fail, and we have more blocks on top of that chain that
// have come in since. The question is, should we try to
// sync these? or just drop them?
}
// TODO: this is the case where we try to sync a chain, and
// fail, and we have more blocks on top of that chain that
// have come in since. The question is, should we try to
// sync these? or just drop them?
log.Error("failed to sync chain but have new unconnected blocks from chain")
}
if sm.nextSyncTarget == nil && !sm.syncQueue.Empty() {

26
chain/sync_test.go
View File

@ -3,11 +3,12 @@ package chain_test
import (
"context"
"fmt"
"github.com/ipfs/go-cid"
"os"
"testing"
"time"
"github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
logging "github.com/ipfs/go-log/v2"
"github.com/libp2p/go-libp2p-core/peer"
@ -36,7 +37,10 @@ import (
func init() {
build.InsecurePoStValidation = true
os.Setenv("TRUST_PARAMS", "1")
err := os.Setenv("TRUST_PARAMS", "1")
if err != nil {
panic(err)
}
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{
abi.RegisteredSealProof_StackedDrg2KiBV1: {},
}
@ -212,20 +216,6 @@ func (tu *syncTestUtil) mineNewBlock(src int, miners []int) {
tu.g.CurTipset = mts
}
func fblkToBlkMsg(fb *types.FullBlock) *types.BlockMsg {
out := &types.BlockMsg{
Header: fb.Header,
}
for _, msg := range fb.BlsMessages {
out.BlsMessages = append(out.BlsMessages, msg.Cid())
}
for _, msg := range fb.SecpkMessages {
out.SecpkMessages = append(out.SecpkMessages, msg.Cid())
}
return out
}
func (tu *syncTestUtil) addSourceNode(gen int) {
if tu.genesis != nil {
tu.t.Fatal("source node already exists")
@ -454,7 +444,7 @@ func (wpp badWpp) GenerateCandidates(context.Context, abi.PoStRandomness, uint64
func (wpp badWpp) ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error) {
return []abi.PoStProof{
abi.PoStProof{
{
PoStProof: abi.RegisteredPoStProof_StackedDrgWinning2KiBV1,
ProofBytes: []byte("evil"),
},
@ -587,7 +577,7 @@ func TestDuplicateNonce(t *testing.T) {
msgs := make([][]*types.SignedMessage, 2)
// Each miner includes a message from the banker with the same nonce, but to different addresses
for k, _ := range msgs {
for k := range msgs {
msgs[k] = []*types.SignedMessage{makeMsg(tu.g.Miners[k])}
}

46
chain/types/cbor_gen.go
View File

@ -6,10 +6,10 @@ import (
"fmt"
"io"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/ipfs/go-cid"
abi "github.com/filecoin-project/specs-actors/actors/abi"
crypto "github.com/filecoin-project/specs-actors/actors/crypto"
exitcode "github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
)
@ -637,15 +637,10 @@ func (t *Message) MarshalCBOR(w io.Writer) error {
scratch := make([]byte, 9)
// t.Version (int64) (int64)
if t.Version >= 0 {
if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajUnsignedInt, uint64(t.Version)); err != nil {
return err
}
} else {
if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajNegativeInt, uint64(-t.Version-1)); err != nil {
return err
}
// t.Version (uint64) (uint64)
if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajUnsignedInt, uint64(t.Version)); err != nil {
return err
}
// t.To (address.Address) (struct)
@ -729,30 +724,19 @@ func (t *Message) UnmarshalCBOR(r io.Reader) error {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Version (int64) (int64)
// t.Version (uint64) (uint64)
{
maj, extra, err := cbg.CborReadHeaderBuf(br, scratch)
var extraI int64
maj, extra, err = cbg.CborReadHeaderBuf(br, scratch)
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.Version = uint64(extra)
t.Version = int64(extraI)
}
// t.To (address.Address) (struct)

2
chain/types/message.go
View File

@ -25,7 +25,7 @@ type ChainMsg interface {
}
type Message struct {
Version int64
Version uint64
To address.Address
From address.Address

4
chain/types/signedmessage.go
View File

@ -62,8 +62,8 @@ func (sm *SignedMessage) Serialize() ([]byte, error) {
return buf.Bytes(), nil
}
func (m *SignedMessage) ChainLength() int {
ser, err := m.Serialize()
func (sm *SignedMessage) ChainLength() int {
ser, err := sm.Serialize()
if err != nil {
panic(err)
}

4
chain/types/tipset.go
View File

@ -238,3 +238,7 @@ func (ts *TipSet) IsChildOf(parent *TipSet) bool {
// height for their processing logic at the moment to obviate it.
ts.height > parent.height
}
func (ts *TipSet) String() string {
return fmt.Sprintf("%v", ts.cids)
}

1
chain/validation/factories.go
View File

@ -2,6 +2,7 @@ package validation
import (
"context"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/specs-actors/actors/runtime"
cbor "github.com/ipfs/go-ipld-cbor"

5
chain/validation/keymanager.go
View File

@ -2,9 +2,10 @@ package validation
import (
"fmt"
"github.com/minio/blake2b-simd"
"math/rand"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-crypto"
acrypto "github.com/filecoin-project/specs-actors/actors/crypto"
@ -69,7 +70,7 @@ func (k *KeyManager) Sign(addr address.Address, data []byte) (acrypto.Signature,
}
func (k *KeyManager) newSecp256k1Key() *wallet.Key {
randSrc := rand.New(rand.NewSource(k.secpSeed))
randSrc := rand.New(rand.NewSource(k.secpSeed)) // nolint
prv, err := crypto.GenerateKeyFromSeed(randSrc)
if err != nil {
panic(err)

2
chain/vectors/vectors_test.go
View File

@ -18,7 +18,7 @@ func LoadVector(t *testing.T, f string, out interface{}) {
if err != nil {
t.Fatal(err)
}
defer fi.Close()
defer fi.Close() //nolint:errcheck
if err := json.NewDecoder(fi).Decode(out); err != nil {
t.Fatal(err)

12
chain/vm/runtime.go
View File

@ -410,8 +410,10 @@ type shimStateHandle struct {
func (ssh *shimStateHandle) Create(obj vmr.CBORMarshaler) {
c := ssh.rt.Put(obj)
// TODO: handle error below
ssh.rt.stateCommit(EmptyObjectCid, c)
err := ssh.rt.stateCommit(EmptyObjectCid, c)
if err != nil {
panic(fmt.Errorf("failed to commit state after creating object: %w", err))
}
}
func (ssh *shimStateHandle) Readonly(obj vmr.CBORUnmarshaler) {
@ -440,8 +442,10 @@ func (ssh *shimStateHandle) Transaction(obj vmr.CBORer, f func()) {
c := ssh.rt.Put(obj)
// TODO: handle error below
ssh.rt.stateCommit(baseState, c)
err = ssh.rt.stateCommit(baseState, c)
if err != nil {
panic(fmt.Errorf("failed to commit state after transaction: %w", err))
}
}
func (rt *Runtime) GetBalance(a address.Address) (types.BigInt, aerrors.ActorError) {

3
chain/vm/vm.go
View File

@ -254,6 +254,9 @@ func (vm *VM) send(ctx context.Context, msg *types.Message, parent *Runtime,
if aerr := rt.chargeGasSafe(rt.Pricelist().OnMethodInvocation(msg.Value, msg.Method)); aerr != nil {
return nil, aerrors.Wrap(aerr, "not enough gas for method invocation")
}
// not charging any gas, just logging
//nolint:errcheck
defer rt.chargeGasSafe(newGasCharge("OnMethodInvocationDone", 0, 0))
if types.BigCmp(msg.Value, types.NewInt(0)) != 0 {

1
cli/auth.go
View File

@ -2,6 +2,7 @@ package cli
import (
"fmt"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"

4
cli/chain.go
View File

@ -319,7 +319,7 @@ var chainSetHeadCmd = &cli.Command{
ts, err = api.ChainGetTipSetByHeight(ctx, abi.ChainEpoch(cctx.Uint64("epoch")), types.EmptyTSK)
}
if ts == nil {
ts, err = parseTipSet(api, ctx, cctx.Args().Slice())
ts, err = parseTipSet(ctx, api, cctx.Args().Slice())
}
if err != nil {
return err
@ -337,7 +337,7 @@ var chainSetHeadCmd = &cli.Command{
},
}
func parseTipSet(api api.FullNode, ctx context.Context, vals []string) (*types.TipSet, error) {
func parseTipSet(ctx context.Context, api api.FullNode, vals []string) (*types.TipSet, error) {
var headers []*types.BlockHeader
for _, c := range vals {
blkc, err := cid.Decode(c)

272
cli/client.go
View File

@ -3,6 +3,7 @@ package cli
import (
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"sort"
@ -10,8 +11,10 @@ import (
"text/tabwriter"
"time"
tm "github.com/buger/goterm"
"github.com/docker/go-units"
"github.com/fatih/color"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-cidutil/cidenc"
@ -76,6 +79,7 @@ var clientCmd = &cli.Command{
WithCategory("util", clientCommPCmd),
WithCategory("util", clientCarGenCmd),
WithCategory("util", clientInfoCmd),
WithCategory("util", clientListTransfers),
},
}
@ -926,11 +930,11 @@ var clientQueryAskCmd = &cli.Command{
return xerrors.Errorf("failed to get peerID for miner: %w", err)
}
if peer.ID(*mi.PeerId) == peer.ID("SETME") {
if *mi.PeerId == peer.ID("SETME") {
return fmt.Errorf("the miner hasn't initialized yet")
}
pid = peer.ID(*mi.PeerId)
pid = *mi.PeerId
}
ask, err := api.ClientQueryAsk(ctx, pid, maddr)
@ -993,6 +997,10 @@ var clientListDeals = &cli.Command{
return err
}
sort.Slice(localDeals, func(i, j int) bool {
return localDeals[i].CreationTime.Before(localDeals[j].CreationTime)
})
var deals []deal
for _, v := range localDeals {
if v.DealID == 0 {
@ -1021,7 +1029,7 @@ var clientListDeals = &cli.Command{
if cctx.Bool("verbose") {
w := tabwriter.NewWriter(os.Stdout, 2, 4, 2, ' ', 0)
fmt.Fprintf(w, "DealCid\tDealId\tProvider\tState\tOn Chain?\tSlashed?\tPieceCID\tSize\tPrice\tDuration\tMessage\n")
fmt.Fprintf(w, "Created\tDealCid\tDealId\tProvider\tState\tOn Chain?\tSlashed?\tPieceCID\tSize\tPrice\tDuration\tMessage\n")
for _, d := range deals {
onChain := "N"
if d.OnChainDealState.SectorStartEpoch != -1 {
@ -1034,58 +1042,56 @@ var clientListDeals = &cli.Command{
}
price := types.FIL(types.BigMul(d.LocalDeal.PricePerEpoch, types.NewInt(d.LocalDeal.Duration)))
fmt.Fprintf(w, "%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%s\n", d.LocalDeal.ProposalCid, d.LocalDeal.DealID, d.LocalDeal.Provider, dealStateString(color, d.LocalDeal.State), onChain, slashed, d.LocalDeal.PieceCID, types.SizeStr(types.NewInt(d.LocalDeal.Size)), price, d.LocalDeal.Duration, d.LocalDeal.Message)
fmt.Fprintf(w, "%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%s\n", d.LocalDeal.CreationTime.Format(time.Stamp), d.LocalDeal.ProposalCid, d.LocalDeal.DealID, d.LocalDeal.Provider, dealStateString(color, d.LocalDeal.State), onChain, slashed, d.LocalDeal.PieceCID, types.SizeStr(types.NewInt(d.LocalDeal.Size)), price, d.LocalDeal.Duration, d.LocalDeal.Message)
}
return w.Flush()
} else {
w := tablewriter.New(tablewriter.Col("DealCid"),
tablewriter.Col("DealId"),
tablewriter.Col("Provider"),
tablewriter.Col("State"),
tablewriter.Col("On Chain?"),
tablewriter.Col("Slashed?"),
tablewriter.Col("PieceCID"),
tablewriter.Col("Size"),
tablewriter.Col("Price"),
tablewriter.Col("Duration"),
tablewriter.NewLineCol("Message"))
}
for _, d := range deals {
propcid := d.LocalDeal.ProposalCid.String()
propcid = "..." + propcid[len(propcid)-8:]
w := tablewriter.New(tablewriter.Col("DealCid"),
tablewriter.Col("DealId"),
tablewriter.Col("Provider"),
tablewriter.Col("State"),
tablewriter.Col("On Chain?"),
tablewriter.Col("Slashed?"),
tablewriter.Col("PieceCID"),
tablewriter.Col("Size"),
tablewriter.Col("Price"),
tablewriter.Col("Duration"),
tablewriter.NewLineCol("Message"))
onChain := "N"
if d.OnChainDealState.SectorStartEpoch != -1 {
onChain = fmt.Sprintf("Y (epoch %d)", d.OnChainDealState.SectorStartEpoch)
}
for _, d := range deals {
propcid := ellipsis(d.LocalDeal.ProposalCid.String(), 8)
slashed := "N"
if d.OnChainDealState.SlashEpoch != -1 {
slashed = fmt.Sprintf("Y (epoch %d)", d.OnChainDealState.SlashEpoch)
}
piece := d.LocalDeal.PieceCID.String()
piece = "..." + piece[len(piece)-8:]
price := types.FIL(types.BigMul(d.LocalDeal.PricePerEpoch, types.NewInt(d.LocalDeal.Duration)))
w.Write(map[string]interface{}{
"DealCid": propcid,
"DealId": d.LocalDeal.DealID,
"Provider": d.LocalDeal.Provider,
"State": dealStateString(color, d.LocalDeal.State),
"On Chain?": onChain,
"Slashed?": slashed,
"PieceCID": piece,
"Size": types.SizeStr(types.NewInt(d.LocalDeal.Size)),
"Price": price,
"Duration": d.LocalDeal.Duration,
"Message": d.LocalDeal.Message,
})
onChain := "N"
if d.OnChainDealState.SectorStartEpoch != -1 {
onChain = fmt.Sprintf("Y (epoch %d)", d.OnChainDealState.SectorStartEpoch)
}
return w.Flush(os.Stdout)
slashed := "N"
if d.OnChainDealState.SlashEpoch != -1 {
slashed = fmt.Sprintf("Y (epoch %d)", d.OnChainDealState.SlashEpoch)
}
piece := ellipsis(d.LocalDeal.PieceCID.String(), 8)
price := types.FIL(types.BigMul(d.LocalDeal.PricePerEpoch, types.NewInt(d.LocalDeal.Duration)))
w.Write(map[string]interface{}{
"DealCid": propcid,
"DealId": d.LocalDeal.DealID,
"Provider": d.LocalDeal.Provider,
"State": dealStateString(color, d.LocalDeal.State),
"On Chain?": onChain,
"Slashed?": slashed,
"PieceCID": piece,
"Size": types.SizeStr(types.NewInt(d.LocalDeal.Size)),
"Price": price,
"Duration": d.LocalDeal.Duration,
"Message": d.LocalDeal.Message,
})
}
return w.Flush(os.Stdout)
},
}
@ -1203,3 +1209,175 @@ var clientInfoCmd = &cli.Command{
return nil
},
}
var clientListTransfers = &cli.Command{
Name: "list-transfers",
Usage: "List ongoing data transfers for deals",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "color",
Usage: "use color in display output",
Value: true,
},
&cli.BoolFlag{
Name: "completed",
Usage: "show completed data transfers",
},
&cli.BoolFlag{
Name: "watch",
Usage: "watch deal updates in real-time, rather than a one time list",
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer closer()
ctx := ReqContext(cctx)
channels, err := api.ClientListDataTransfers(ctx)
if err != nil {
return err
}
completed := cctx.Bool("completed")
color := cctx.Bool("color")
watch := cctx.Bool("watch")
if watch {
channelUpdates, err := api.ClientDataTransferUpdates(ctx)
if err != nil {
return err
}
for {
tm.Clear() // Clear current screen
tm.MoveCursor(1, 1)
OutputDataTransferChannels(tm.Screen, channels, completed, color)
tm.Flush()
select {
case <-ctx.Done():
return nil
case channelUpdate := <-channelUpdates:
var found bool
for i, existing := range channels {
if existing.TransferID == channelUpdate.TransferID &&
existing.OtherPeer == channelUpdate.OtherPeer &&
existing.IsSender == channelUpdate.IsSender &&
existing.IsInitiator == channelUpdate.IsInitiator {
channels[i] = channelUpdate
found = true
break
}
}
if !found {
channels = append(channels, channelUpdate)
}
}
}
}
OutputDataTransferChannels(os.Stdout, channels, completed, color)
return nil
},
}
// OutputDataTransferChannels generates table output for a list of channels
func OutputDataTransferChannels(out io.Writer, channels []lapi.DataTransferChannel, completed bool, color bool) {
sort.Slice(channels, func(i, j int) bool {
return channels[i].TransferID < channels[j].TransferID
})
var receivingChannels, sendingChannels []lapi.DataTransferChannel
for _, channel := range channels {
if !completed && channel.Status == datatransfer.Completed {
continue
}
if channel.IsSender {
sendingChannels = append(sendingChannels, channel)
} else {
receivingChannels = append(receivingChannels, channel)
}
}
fmt.Fprintf(out, "Sending Channels\n\n")
w := tablewriter.New(tablewriter.Col("ID"),
tablewriter.Col("Status"),
tablewriter.Col("Sending To"),
tablewriter.Col("Root Cid"),
tablewriter.Col("Initiated?"),
tablewriter.Col("Transferred"),
tablewriter.Col("Voucher"),
tablewriter.NewLineCol("Message"))
for _, channel := range sendingChannels {
w.Write(toChannelOutput(color, "Sending To", channel))
}
w.Flush(out) //nolint:errcheck
fmt.Fprintf(out, "\nReceiving Channels\n\n")
w = tablewriter.New(tablewriter.Col("ID"),
tablewriter.Col("Status"),
tablewriter.Col("Receiving From"),
tablewriter.Col("Root Cid"),
tablewriter.Col("Initiated?"),
tablewriter.Col("Transferred"),
tablewriter.Col("Voucher"),
tablewriter.NewLineCol("Message"))
for _, channel := range receivingChannels {
w.Write(toChannelOutput(color, "Receiving From", channel))
}
w.Flush(out) //nolint:errcheck
}
func channelStatusString(useColor bool, status datatransfer.Status) string {
s := datatransfer.Statuses[status]
if !useColor {
return s
}
switch status {
case datatransfer.Failed, datatransfer.Cancelled:
return color.RedString(s)
case datatransfer.Completed:
return color.GreenString(s)
default:
return s
}
}
func toChannelOutput(useColor bool, otherPartyColumn string, channel lapi.DataTransferChannel) map[string]interface{} {
rootCid := ellipsis(channel.BaseCID.String(), 8)
otherParty := ellipsis(channel.OtherPeer.String(), 8)
initiated := "N"
if channel.IsInitiator {
initiated = "Y"
}
voucher := channel.Voucher
if len(voucher) > 40 {
voucher = ellipsis(voucher, 37)
}
return map[string]interface{}{
"ID": channel.TransferID,
"Status": channelStatusString(useColor, channel.Status),
otherPartyColumn: otherParty,
"Root Cid": rootCid,
"Initiated?": initiated,
"Transferred": channel.Transferred,
"Voucher": voucher,
"Message": channel.Message,
}
}
func ellipsis(s string, length int) string {
if length > 0 && len(s) > length {
return "..." + s[len(s)-length:]
}
return s
}

35
cli/cmd.go
View File

@ -12,7 +12,7 @@ import (
logging "github.com/ipfs/go-log/v2"
"github.com/mitchellh/go-homedir"
"github.com/multiformats/go-multiaddr"
manet "github.com/multiformats/go-multiaddr-net"
manet "github.com/multiformats/go-multiaddr/net"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
@ -67,6 +67,19 @@ func (a APIInfo) AuthHeader() http.Header {
return nil
}
// The flag passed on the command line with the listen address of the API
// server (only used by the tests)
func flagForAPI(t repo.RepoType) string {
switch t {
case repo.FullNode:
return "api"
case repo.StorageMiner:
return "miner-api"
default:
panic(fmt.Sprintf("Unknown repo type: %v", t))
}
}
func flagForRepo(t repo.RepoType) string {
switch t {
case repo.FullNode:
@ -102,6 +115,20 @@ func envForRepoDeprecation(t repo.RepoType) string {
}
func GetAPIInfo(ctx *cli.Context, t repo.RepoType) (APIInfo, error) {
// Check if there was a flag passed with the listen address of the API
// server (only used by the tests)
apiFlag := flagForAPI(t)
if ctx.IsSet(apiFlag) {
strma := ctx.String(apiFlag)
strma = strings.TrimSpace(strma)
apima, err := multiaddr.NewMultiaddr(strma)
if err != nil {
return APIInfo{}, err
}
return APIInfo{Addr: apima}, nil
}
envKey := envForRepo(t)
env, ok := os.LookupEnv(envKey)
if !ok {
@ -186,7 +213,7 @@ func GetAPI(ctx *cli.Context) (api.Common, jsonrpc.ClientCloser, error) {
return nil, nil, err
}
return client.NewCommonRPC(addr, headers)
return client.NewCommonRPC(ctx.Context, addr, headers)
}
func GetFullNodeAPI(ctx *cli.Context) (api.FullNode, jsonrpc.ClientCloser, error) {
@ -195,7 +222,7 @@ func GetFullNodeAPI(ctx *cli.Context) (api.FullNode, jsonrpc.ClientCloser, error
return nil, nil, err
}
return client.NewFullNodeRPC(addr, headers)
return client.NewFullNodeRPC(ctx.Context, addr, headers)
}
func GetStorageMinerAPI(ctx *cli.Context, opts ...jsonrpc.Option) (api.StorageMiner, jsonrpc.ClientCloser, error) {
@ -204,7 +231,7 @@ func GetStorageMinerAPI(ctx *cli.Context, opts ...jsonrpc.Option) (api.StorageMi
return nil, nil, err
}
return client.NewStorageMinerRPC(addr, headers, opts...)
return client.NewStorageMinerRPC(ctx.Context, addr, headers, opts...)
}
func DaemonContext(cctx *cli.Context) context.Context {

2
cli/helper.go
View File

@ -39,7 +39,7 @@ func RunApp(app *cli.App) {
}
var phe *PrintHelpErr
if xerrors.As(err, &phe) {
cli.ShowCommandHelp(phe.Ctx, phe.Ctx.Command.Name)
_ = cli.ShowCommandHelp(phe.Ctx, phe.Ctx.Command.Name)
}
os.Exit(1)
}

3
cli/log.go
View File

@ -60,7 +60,8 @@ var logSetLevel = &cli.Command{
Environment Variables:
GOLOG_LOG_LEVEL - Default log level for all log systems
GOLOG_LOG_FMT - Change output log format (json, nocolor)
GOLOG_FILE - Write logs to file in addition to stderr
GOLOG_FILE - Write logs to file
GOLOG_OUTPUT - Specify whether to output to file, stderr, stdout or a combination, i.e. file+stderr
`,
Flags: []cli.Flag{
&cli.StringSliceFlag{

36
cli/mpool.go
View File

@ -20,6 +20,7 @@ var mpoolCmd = &cli.Command{
Usage: "Manage message pool",
Subcommands: []*cli.Command{
mpoolPending,
mpoolClear,
mpoolSub,
mpoolStat,
mpoolReplaceCmd,
@ -83,6 +84,39 @@ var mpoolPending = &cli.Command{
},
}
var mpoolClear = &cli.Command{
Name: "clear",
Usage: "Clear all pending messages from the mpool (USE WITH CARE)",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "local",
Usage: "also clear local messages",
},
&cli.BoolFlag{
Name: "really-do-it",
Usage: "must be specified for the action to take effect",
},
},
Action: func(cctx *cli.Context) error {
api, closer, err := GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer closer()
really := cctx.Bool("really-do-it")
if !really {
//nolint:golint
return fmt.Errorf("--really-do-it must be specified for this action to have an effect; you have been warned")
}
local := cctx.Bool("local")
ctx := ReqContext(cctx)
return api.MpoolClear(ctx, local)
},
}
var mpoolSub = &cli.Command{
Name: "sub",
Usage: "Subscribe to mpool changes",
@ -313,7 +347,7 @@ var mpoolReplaceCmd = &cli.Command{
if err != nil {
return fmt.Errorf("parsing gas-premium: %w", err)
}
// TODO: estiamte fee cap here
// TODO: estimate fee cap here
msg.GasFeeCap, err = types.BigFromString(cctx.String("gas-feecap"))
if err != nil {
return fmt.Errorf("parsing gas-feecap: %w", err)

14
cli/multisig.go
View File

@ -12,6 +12,7 @@ import (
"text/tabwriter"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/go-address"
@ -231,10 +232,10 @@ var msigInspectCmd = &cli.Command{
})
w := tabwriter.NewWriter(os.Stdout, 8, 4, 0, ' ', 0)
fmt.Fprintf(w, "ID\tState\tTo\tValue\tMethod\tParams\n")
fmt.Fprintf(w, "ID\tState\tApprovals\tTo\tValue\tMethod\tParams\n")
for _, txid := range txids {
tx := pending[txid]
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%d\t%x\n", txid, state(tx), tx.To, types.FIL(tx.Value), tx.Method, tx.Params)
fmt.Fprintf(w, "%d\t%s\t%d\t%s\t%s\t%d\t%x\n", txid, state(tx), len(tx.Approved), tx.To, types.FIL(tx.Value), tx.Method, tx.Params)
}
if err := w.Flush(); err != nil {
return xerrors.Errorf("flushing output: %+v", err)
@ -355,6 +356,15 @@ var msigProposeCmd = &cli.Command{
from = defaddr
}
act, err := api.StateGetActor(ctx, msig, types.EmptyTSK)
if err != nil {
return fmt.Errorf("failed to look up multisig %s: %w", msig, err)
}
if act.Code != builtin.MultisigActorCodeID {
return fmt.Errorf("actor %s is not a multisig actor", msig)
}
msgCid, err := api.MsigPropose(ctx, msig, dest, types.BigInt(value), from, method, params)
if err != nil {
return err

2
cli/net.go
View File

@ -180,7 +180,7 @@ var netFindPeer = &cli.Command{
return nil
}
pid, err := peer.IDB58Decode(cctx.Args().First())
pid, err := peer.Decode(cctx.Args().First())
if err != nil {
return err
}

94
cli/paych.go
View File

@ -4,10 +4,13 @@ import (
"bytes"
"encoding/base64"
"fmt"
"io"
"sort"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/paychmgr"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/urfave/cli/v2"
@ -71,7 +74,7 @@ var paychGetCmd = &cli.Command{
return err
}
fmt.Println(chAddr)
fmt.Fprintln(cctx.App.Writer, chAddr)
return nil
},
}
@ -94,7 +97,7 @@ var paychListCmd = &cli.Command{
}
for _, v := range chs {
fmt.Println(v.String())
fmt.Fprintln(cctx.App.Writer, v.String())
}
return nil
},
@ -135,7 +138,7 @@ var paychSettleCmd = &cli.Command{
return fmt.Errorf("settle message execution failed (exit code %d)", mwait.Receipt.ExitCode)
}
fmt.Printf("Settled channel %s\n", ch)
fmt.Fprintf(cctx.App.Writer, "Settled channel %s\n", ch)
return nil
},
}
@ -175,7 +178,7 @@ var paychCloseCmd = &cli.Command{
return fmt.Errorf("collect message execution failed (exit code %d)", mwait.Receipt.ExitCode)
}
fmt.Printf("Collected funds for channel %s\n", ch)
fmt.Fprintf(cctx.App.Writer, "Collected funds for channel %s\n", ch)
return nil
},
}
@ -239,7 +242,7 @@ var paychVoucherCreateCmd = &cli.Command{
return err
}
fmt.Println(enc)
fmt.Fprintln(cctx.App.Writer, enc)
return nil
},
}
@ -275,7 +278,7 @@ var paychVoucherCheckCmd = &cli.Command{
return err
}
fmt.Println("voucher is valid")
fmt.Fprintln(cctx.App.Writer, "voucher is valid")
return nil
},
}
@ -323,7 +326,7 @@ var paychVoucherListCmd = &cli.Command{
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "export",
Usage: "Print export strings",
Usage: "Print voucher as serialized string",
},
},
Action: func(cctx *cli.Context) error {
@ -349,16 +352,11 @@ var paychVoucherListCmd = &cli.Command{
return err
}
for _, v := range vouchers {
if cctx.Bool("export") {
enc, err := EncodedString(v)
if err != nil {
return err
}
fmt.Printf("Lane %d, Nonce %d: %s; %s\n", v.Lane, v.Nonce, v.Amount.String(), enc)
} else {
fmt.Printf("Lane %d, Nonce %d: %s\n", v.Lane, v.Nonce, v.Amount.String())
for _, v := range sortVouchers(vouchers) {
export := cctx.Bool("export")
err := outputVoucher(cctx.App.Writer, v, export)
if err != nil {
return err
}
}
@ -368,8 +366,14 @@ var paychVoucherListCmd = &cli.Command{
var paychVoucherBestSpendableCmd = &cli.Command{
Name: "best-spendable",
Usage: "Print voucher with highest value that is currently spendable",
Usage: "Print vouchers with highest value that is currently spendable for each lane",
ArgsUsage: "[channelAddress]",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "export",
Usage: "Print voucher as serialized string",
},
},
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() != 1 {
return ShowHelp(cctx, fmt.Errorf("must pass payment channel address"))
@ -388,37 +392,53 @@ var paychVoucherBestSpendableCmd = &cli.Command{
ctx := ReqContext(cctx)
vouchers, err := api.PaychVoucherList(ctx, ch)
vouchersByLane, err := paychmgr.BestSpendableByLane(ctx, api, ch)
if err != nil {
return err
}
var best *paych.SignedVoucher
for _, v := range vouchers {
spendable, err := api.PaychVoucherCheckSpendable(ctx, ch, v, nil, nil)
var vouchers []*paych.SignedVoucher
for _, vchr := range vouchersByLane {
vouchers = append(vouchers, vchr)
}
for _, best := range sortVouchers(vouchers) {
export := cctx.Bool("export")
err := outputVoucher(cctx.App.Writer, best, export)
if err != nil {
return err
}
if spendable {
if best == nil || v.Amount.GreaterThan(best.Amount) {
best = v
}
}
}
if best == nil {
return fmt.Errorf("No spendable vouchers for that channel")
}
return nil
},
}
enc, err := EncodedString(best)
func sortVouchers(vouchers []*paych.SignedVoucher) []*paych.SignedVoucher {
sort.Slice(vouchers, func(i, j int) bool {
if vouchers[i].Lane == vouchers[j].Lane {
return vouchers[i].Nonce < vouchers[j].Nonce
}
return vouchers[i].Lane < vouchers[j].Lane
})
return vouchers
}
func outputVoucher(w io.Writer, v *paych.SignedVoucher, export bool) error {
var enc string
if export {
var err error
enc, err = EncodedString(v)
if err != nil {
return err
}
}
fmt.Println(enc)
fmt.Printf("Amount: %s\n", best.Amount)
return nil
},
fmt.Fprintf(w, "Lane %d, Nonce %d: %s", v.Lane, v.Nonce, v.Amount.String())
if export {
fmt.Fprintf(w, "; %s", enc)
}
fmt.Fprintln(w)
return nil
}
var paychVoucherSubmitCmd = &cli.Command{
@ -462,7 +482,7 @@ var paychVoucherSubmitCmd = &cli.Command{
return fmt.Errorf("message execution failed (exit code %d)", mwait.Receipt.ExitCode)
}
fmt.Println("channel updated successfully")
fmt.Fprintln(cctx.App.Writer, "channel updated successfully")
return nil
},

330
cli/paych_test.go Normal file
View File

@ -0,0 +1,330 @@
package cli
import (
"bytes"
"context"
"flag"
"fmt"
"os"
"strconv"
"strings"
"testing"
"time"
"github.com/filecoin-project/specs-actors/actors/abi/big"
saminer "github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/multiformats/go-multiaddr"
"github.com/filecoin-project/lotus/chain/events"
"github.com/filecoin-project/lotus/api/apibstore"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
cbor "github.com/ipfs/go-ipld-cbor"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/api/test"
"github.com/filecoin-project/lotus/chain/wallet"
builder "github.com/filecoin-project/lotus/node/test"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/stretchr/testify/require"
"github.com/urfave/cli/v2"
)
func init() {
power.ConsensusMinerMinPower = big.NewInt(2048)
saminer.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{
abi.RegisteredSealProof_StackedDrg2KiBV1: {},
}
verifreg.MinVerifiedDealSize = big.NewInt(256)
}
// TestPaymentChannels does a basic test to exercise the payment channel CLI
// commands
func TestPaymentChannels(t *testing.T) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
blocktime := 5 * time.Millisecond
ctx := context.Background()
nodes, addrs := startTwoNodesOneMiner(ctx, t, blocktime)
paymentCreator := nodes[0]
paymentReceiver := nodes[0]
creatorAddr := addrs[0]
receiverAddr := addrs[1]
// Create mock CLI
mockCLI := newMockCLI(t)
creatorCLI := mockCLI.client(paymentCreator.ListenAddr)
receiverCLI := mockCLI.client(paymentReceiver.ListenAddr)
// creator: paych get <creator> <receiver> <amount>
channelAmt := "100000"
cmd := []string{creatorAddr.String(), receiverAddr.String(), channelAmt}
chstr := creatorCLI.runCmd(paychGetCmd, cmd)
chAddr, err := address.NewFromString(chstr)
require.NoError(t, err)
// creator: paych voucher create <channel> <amount>
voucherAmt := 100
vamt := strconv.Itoa(voucherAmt)
cmd = []string{chAddr.String(), vamt}
voucher := creatorCLI.runCmd(paychVoucherCreateCmd, cmd)
// receiver: paych voucher add <channel> <voucher>
cmd = []string{chAddr.String(), voucher}
receiverCLI.runCmd(paychVoucherAddCmd, cmd)
// creator: paych settle <channel>
cmd = []string{chAddr.String()}
creatorCLI.runCmd(paychSettleCmd, cmd)
// Wait for the chain to reach the settle height
chState := getPaychState(ctx, t, paymentReceiver, chAddr)
waitForHeight(ctx, t, paymentReceiver, chState.SettlingAt)
// receiver: paych collect <channel>
cmd = []string{chAddr.String()}
receiverCLI.runCmd(paychCloseCmd, cmd)
}
type voucherSpec struct {
serialized string
amt int
lane int
}
// TestPaymentChannelVouchers does a basic test to exercise some payment
// channel voucher commands
func TestPaymentChannelVouchers(t *testing.T) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")
blocktime := 5 * time.Millisecond
ctx := context.Background()
nodes, addrs := startTwoNodesOneMiner(ctx, t, blocktime)
paymentCreator := nodes[0]
creatorAddr := addrs[0]
receiverAddr := addrs[1]
// Create mock CLI
mockCLI := newMockCLI(t)
creatorCLI := mockCLI.client(paymentCreator.ListenAddr)
// creator: paych get <creator> <receiver> <amount>
channelAmt := "100000"
cmd := []string{creatorAddr.String(), receiverAddr.String(), channelAmt}
chstr := creatorCLI.runCmd(paychGetCmd, cmd)
chAddr, err := address.NewFromString(chstr)
require.NoError(t, err)
var vouchers []voucherSpec
// creator: paych voucher create <channel> <amount>
// Note: implied --lane=0
voucherAmt1 := 100
vamt1 := strconv.Itoa(voucherAmt1)
cmd = []string{chAddr.String(), vamt1}
voucher1 := creatorCLI.runCmd(paychVoucherCreateCmd, cmd)
vouchers = append(vouchers, voucherSpec{serialized: voucher1, lane: 0, amt: voucherAmt1})
// creator: paych voucher create <channel> <amount> --lane=5
lane5 := "--lane=5"
voucherAmt2 := 50
vamt2 := strconv.Itoa(voucherAmt2)
cmd = []string{lane5, chAddr.String(), vamt2}
voucher2 := creatorCLI.runCmd(paychVoucherCreateCmd, cmd)
vouchers = append(vouchers, voucherSpec{serialized: voucher2, lane: 5, amt: voucherAmt2})
// creator: paych voucher create <channel> <amount> --lane=5
voucherAmt3 := 70
vamt3 := strconv.Itoa(voucherAmt3)
cmd = []string{lane5, chAddr.String(), vamt3}
voucher3 := creatorCLI.runCmd(paychVoucherCreateCmd, cmd)
vouchers = append(vouchers, voucherSpec{serialized: voucher3, lane: 5, amt: voucherAmt3})
// creator: paych voucher list <channel> --export
cmd = []string{"--export", chAddr.String()}
list := creatorCLI.runCmd(paychVoucherListCmd, cmd)
// Check that voucher list output is correct
checkVoucherOutput(t, list, vouchers)
// creator: paych voucher best-spendable <channel>
cmd = []string{"--export", chAddr.String()}
bestSpendable := creatorCLI.runCmd(paychVoucherBestSpendableCmd, cmd)
// Check that best spendable output is correct
bestVouchers := []voucherSpec{
{serialized: voucher1, lane: 0, amt: voucherAmt1},
{serialized: voucher3, lane: 5, amt: voucherAmt3},
}
checkVoucherOutput(t, bestSpendable, bestVouchers)
}
func checkVoucherOutput(t *testing.T, list string, vouchers []voucherSpec) {
lines := strings.Split(list, "\n")
listVouchers := make(map[string]string)
for _, line := range lines {
parts := strings.Split(line, ";")
serialized := strings.TrimSpace(parts[1])
listVouchers[serialized] = strings.TrimSpace(parts[0])
}
for _, vchr := range vouchers {
res, ok := listVouchers[vchr.serialized]
require.True(t, ok)
require.Regexp(t, fmt.Sprintf("Lane %d", vchr.lane), res)
require.Regexp(t, fmt.Sprintf("%d", vchr.amt), res)
}
}
func startTwoNodesOneMiner(ctx context.Context, t *testing.T, blocktime time.Duration) ([]test.TestNode, []address.Address) {
n, sn := builder.RPCMockSbBuilder(t, 2, test.OneMiner)
paymentCreator := n[0]
paymentReceiver := n[1]
miner := sn[0]
// Get everyone connected
addrs, err := paymentCreator.NetAddrsListen(ctx)
if err != nil {
t.Fatal(err)
}
if err := paymentReceiver.NetConnect(ctx, addrs); err != nil {
t.Fatal(err)
}
if err := miner.NetConnect(ctx, addrs); err != nil {
t.Fatal(err)
}
// Start mining blocks
bm := test.NewBlockMiner(ctx, t, miner, blocktime)
bm.MineBlocks()
// Send some funds to register the receiver
receiverAddr, err := paymentReceiver.WalletNew(ctx, wallet.ActSigType("secp256k1"))
if err != nil {
t.Fatal(err)
}
test.SendFunds(ctx, t, paymentCreator, receiverAddr, abi.NewTokenAmount(1e18))
// Get the creator's address
creatorAddr, err := paymentCreator.WalletDefaultAddress(ctx)
if err != nil {
t.Fatal(err)
}
// Create mock CLI
return n, []address.Address{creatorAddr, receiverAddr}
}
type mockCLI struct {
t *testing.T
cctx *cli.Context
out *bytes.Buffer
}
func newMockCLI(t *testing.T) *mockCLI {
// Create a CLI App with an --api flag so that we can specify which node
// the command should be executed against
app := cli.NewApp()
app.Flags = []cli.Flag{
&cli.StringFlag{
Name: "api",
Hidden: true,
},
}
var out bytes.Buffer
app.Writer = &out
app.Setup()
cctx := cli.NewContext(app, &flag.FlagSet{}, nil)
return &mockCLI{t: t, cctx: cctx, out: &out}
}
func (c *mockCLI) client(addr multiaddr.Multiaddr) *mockCLIClient {
return &mockCLIClient{t: c.t, addr: addr, cctx: c.cctx, out: c.out}
}
// mockCLIClient runs commands against a particular node
type mockCLIClient struct {
t *testing.T
addr multiaddr.Multiaddr
cctx *cli.Context
out *bytes.Buffer
}
func (c *mockCLIClient) runCmd(cmd *cli.Command, input []string) string {
// prepend --api=<node api listener address>
apiFlag := "--api=" + c.addr.String()
input = append([]string{apiFlag}, input...)
fs := c.flagSet(cmd)
err := fs.Parse(input)
require.NoError(c.t, err)
err = cmd.Action(cli.NewContext(c.cctx.App, fs, c.cctx))
require.NoError(c.t, err)
// Get the output
str := strings.TrimSpace(c.out.String())
c.out.Reset()
return str
}
func (c *mockCLIClient) flagSet(cmd *cli.Command) *flag.FlagSet {
// Apply app level flags (so we can process --api flag)
fs := &flag.FlagSet{}
for _, f := range c.cctx.App.Flags {
err := f.Apply(fs)
if err != nil {
c.t.Fatal(err)
}
}
// Apply command level flags
for _, f := range cmd.Flags {
err := f.Apply(fs)
if err != nil {
c.t.Fatal(err)
}
}
return fs
}
// waitForHeight waits for the node to reach the given chain epoch
func waitForHeight(ctx context.Context, t *testing.T, node test.TestNode, height abi.ChainEpoch) {
atHeight := make(chan struct{})
chainEvents := events.NewEvents(ctx, node)
err := chainEvents.ChainAt(func(ctx context.Context, ts *types.TipSet, curH abi.ChainEpoch) error {
close(atHeight)
return nil
}, nil, 1, height)
if err != nil {
t.Fatal(err)
}
select {
case <-atHeight:
case <-ctx.Done():
}
}
// getPaychState gets the state of the payment channel with the given address
func getPaychState(ctx context.Context, t *testing.T, node test.TestNode, chAddr address.Address) paych.State {
act, err := node.StateGetActor(ctx, chAddr, types.EmptyTSK)
require.NoError(t, err)
store := cbor.NewCborStore(apibstore.NewAPIBlockstore(node))
var chState paych.State
err = store.Get(ctx, act.Head, &chState)
require.NoError(t, err)
return chState
}

4
cli/pprof.go
View File

@ -9,7 +9,7 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/node/repo"
manet "github.com/multiformats/go-multiaddr-net"
manet "github.com/multiformats/go-multiaddr/net"
)
var pprofCmd = &cli.Command{
@ -44,7 +44,7 @@ var PprofGoroutines = &cli.Command{
addr = "http://" + addr + "/debug/pprof/goroutine?debug=2"
r, err := http.Get(addr)
r, err := http.Get(addr) //nolint:gosec
if err != nil {
return err
}

39
cli/state.go
View File

@ -105,6 +105,9 @@ var stateMinerInfo = &cli.Command{
fmt.Printf("Owner:\t%s\n", mi.Owner)
fmt.Printf("Worker:\t%s\n", mi.Worker)
for i, controlAddress := range mi.ControlAddresses {
fmt.Printf("Control %d: \t%s\n", i, controlAddress)
}
fmt.Printf("PeerID:\t%s\n", mi.PeerId)
fmt.Printf("SectorSize:\t%s (%d)\n", types.SizeStr(types.NewInt(uint64(mi.SectorSize))), mi.SectorSize)
fmt.Printf("Multiaddrs: \t")
@ -350,6 +353,9 @@ var stateReplaySetCmd = &cli.Command{
}
ts, err = types.NewTipSet(headers)
if err != nil {
return err
}
} else {
var r *api.MsgLookup
r, err = fapi.StateWaitMsg(ctx, mcid, build.MessageConfidence)
@ -362,9 +368,9 @@ var stateReplaySetCmd = &cli.Command{
return xerrors.Errorf("loading tipset: %w", err)
}
ts, err = fapi.ChainGetTipSet(ctx, childTs.Parents())
}
if err != nil {
return err
if err != nil {
return err
}
}
}
@ -562,10 +568,30 @@ var stateGetActorCmd = &cli.Command{
return err
}
var strtype string
switch a.Code {
case builtin.AccountActorCodeID:
strtype = "account"
case builtin.MultisigActorCodeID:
strtype = "multisig"
case builtin.CronActorCodeID:
strtype = "cron"
case builtin.InitActorCodeID:
strtype = "init"
case builtin.StorageMinerActorCodeID:
strtype = "miner"
case builtin.StorageMarketActorCodeID:
strtype = "market"
case builtin.StoragePowerActorCodeID:
strtype = "power"
default:
strtype = "unknown"
}
fmt.Printf("Address:\t%s\n", addr)
fmt.Printf("Balance:\t%s\n", types.FIL(a.Balance))
fmt.Printf("Nonce:\t\t%d\n", a.Nonce)
fmt.Printf("Code:\t\t%s\n", a.Code)
fmt.Printf("Code:\t\t%s (%s)\n", a.Code, strtype)
fmt.Printf("Head:\t\t%s\n", a.Head)
return nil
@ -1476,7 +1502,7 @@ func parseParamsForMethod(act cid.Cid, method uint64, args []string) ([]byte, er
}
p.Elem().Field(i).Set(reflect.ValueOf(val))
case reflect.TypeOf(peer.ID("")):
pid, err := peer.IDB58Decode(args[i])
pid, err := peer.Decode(args[i])
if err != nil {
return nil, fmt.Errorf("failed to parse peer ID: %s", err)
}
@ -1561,6 +1587,9 @@ var stateMarketBalanceCmd = &cli.Command{
}
balance, err := api.StateMarketBalance(ctx, addr, ts.Key())
if err != nil {
return err
}
fmt.Printf("Escrow: %s\n", types.FIL(balance.Escrow))
fmt.Printf("Locked: %s\n", types.FIL(balance.Locked))

5
cli/wallet.go
View File

@ -385,10 +385,9 @@ var walletVerify = &cli.Command{
if api.WalletVerify(ctx, addr, msg, &sig) {
fmt.Println("valid")
return nil
} else {
fmt.Println("invalid")
return NewCliError("CLI Verify called with invalid signature")
}
fmt.Println("invalid")
return NewCliError("CLI Verify called with invalid signature")
},
}

34
cmd/lotus-bench/import.go
View File

@ -214,7 +214,7 @@ func countGasCosts(et *types.ExecutionTrace) (int64, int64) {
}
for _, sub := range et.Subcalls {
c, v := countGasCosts(&sub)
c, v := countGasCosts(&sub) //nolint
cgas += c
vgas += v
}
@ -222,24 +222,6 @@ func countGasCosts(et *types.ExecutionTrace) (int64, int64) {
return cgas, vgas
}
func compStats(vals []float64) (float64, float64) {
var sum float64
for _, v := range vals {
sum += v
}
av := sum / float64(len(vals))
var varsum float64
for _, v := range vals {
delta := av - v
varsum += delta * delta
}
return av, math.Sqrt(varsum / float64(len(vals)))
}
type stats struct {
timeTaken meanVar
gasRatio meanVar
@ -264,20 +246,20 @@ func (cov1 *covar) VarianceX() float64 {
return cov1.m2x / (cov1.n - 1)
}
func (v1 *covar) StddevX() float64 {
return math.Sqrt(v1.VarianceX())
func (cov1 *covar) StddevX() float64 {
return math.Sqrt(cov1.VarianceX())
}
func (cov1 *covar) VarianceY() float64 {
return cov1.m2y / (cov1.n - 1)
}
func (v1 *covar) StddevY() float64 {
return math.Sqrt(v1.VarianceY())
func (cov1 *covar) StddevY() float64 {
return math.Sqrt(cov1.VarianceY())
}
func (cov1 *covar) AddPoint(x, y float64) {
cov1.n += 1
cov1.n++
dx := x - cov1.meanX
cov1.meanX += dx / cov1.n
@ -344,7 +326,7 @@ type meanVar struct {
func (v1 *meanVar) AddPoint(value float64) {
// based on https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm
v1.n += 1
v1.n++
delta := value - v1.mean
v1.mean += delta / v1.n
delta2 := value - v1.mean
@ -481,7 +463,7 @@ var importAnalyzeCmd = &cli.Command{
}
go func() {
http.ListenAndServe("localhost:6060", nil)
http.ListenAndServe("localhost:6060", nil) //nolint:errcheck
}()
fi, err := os.Open(cctx.Args().First())

7
cmd/lotus-chainwatch/processor/miner.go
View File

@ -88,6 +88,8 @@ create table if not exists sector_info
verified_deal_weight text not null,
initial_pledge text not null,
expected_day_reward text not null,
expected_storage_pledge text not null,
constraint sector_info_pk
primary key (miner_id, sector_id, sealed_cid)
@ -307,6 +309,7 @@ func (p *Processor) storeMinerPreCommitInfo(ctx context.Context, miners []minerA
}
stmt, err := tx.Prepare(`copy spi (miner_id, sector_id, sealed_cid, state_root, seal_rand_epoch, expiration_epoch, precommit_deposit, precommit_epoch, deal_weight, verified_deal_weight, is_replace_capacity, replace_sector_deadline, replace_sector_partition, replace_sector_number) from STDIN`)
if err != nil {
return xerrors.Errorf("Failed to prepare miner precommit info statement: %w", err)
}
@ -431,7 +434,7 @@ func (p *Processor) storeMinerSectorInfo(ctx context.Context, miners []minerActo
return xerrors.Errorf("Failed to create temp table for sector_: %w", err)
}
stmt, err := tx.Prepare(`copy si (miner_id, sector_id, sealed_cid, state_root, activation_epoch, expiration_epoch, deal_weight, verified_deal_weight, initial_pledge) from STDIN`)
stmt, err := tx.Prepare(`copy si (miner_id, sector_id, sealed_cid, state_root, activation_epoch, expiration_epoch, deal_weight, verified_deal_weight, initial_pledge, expected_day_reward, expected_storage_pledge) from STDIN`)
if err != nil {
return xerrors.Errorf("Failed to prepare miner sector info statement: %w", err)
}
@ -463,6 +466,8 @@ func (p *Processor) storeMinerSectorInfo(ctx context.Context, miners []minerActo
added.DealWeight.String(),
added.VerifiedDealWeight.String(),
added.InitialPledge.String(),
added.ExpectedDayReward.String(),
added.ExpectedStoragePledge.String(),
); err != nil {
return err
}

135
cmd/lotus-chainwatch/processor/power.go Normal file
View File

@ -0,0 +1,135 @@
package processor
import (
"bytes"
"context"
"time"
"golang.org/x/xerrors"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/util/smoothing"
)
type powerActorInfo struct {
common actorInfo
epochSmoothingEstimate *smoothing.FilterEstimate
}
func (p *Processor) setupPower() error {
tx, err := p.db.Begin()
if err != nil {
return err
}
if _, err := tx.Exec(`
create table if not exists power_smoothing_estimates
(
state_root text not null
constraint power_smoothing_estimates_pk
primary key,
position_estimate text not null,
velocity_estimate text not null
);
`); err != nil {
return err
}
return tx.Commit()
}
func (p *Processor) HandlePowerChanges(ctx context.Context, powerTips ActorTips) error {
powerChanges, err := p.processPowerActors(ctx, powerTips)
if err != nil {
return xerrors.Errorf("Failed to process power actors: %w", err)
}
if err := p.persistPowerActors(ctx, powerChanges); err != nil {
return err
}
return nil
}
func (p *Processor) processPowerActors(ctx context.Context, powerTips ActorTips) ([]powerActorInfo, error) {
start := time.Now()
defer func() {
log.Debugw("Processed Power Actors", "duration", time.Since(start).String())
}()
var out []powerActorInfo
for tipset, powers := range powerTips {
for _, act := range powers {
var pw powerActorInfo
pw.common = act
powerActor, err := p.node.StateGetActor(ctx, builtin.StoragePowerActorAddr, tipset)
if err != nil {
return nil, xerrors.Errorf("get power state (@ %s): %w", pw.common.stateroot.String(), err)
}
powerStateRaw, err := p.node.ChainReadObj(ctx, powerActor.Head)
if err != nil {
return nil, xerrors.Errorf("read state obj (@ %s): %w", pw.common.stateroot.String(), err)
}
var powerActorState power.State
if err := powerActorState.UnmarshalCBOR(bytes.NewReader(powerStateRaw)); err != nil {
return nil, xerrors.Errorf("unmarshal state (@ %s): %w", pw.common.stateroot.String(), err)
}
pw.epochSmoothingEstimate = powerActorState.ThisEpochQAPowerSmoothed
out = append(out, pw)
}
}
return out, nil
}
func (p *Processor) persistPowerActors(ctx context.Context, powers []powerActorInfo) error {
// NB: use errgroup when there is more than a single store operation
return p.storePowerSmoothingEstimates(powers)
}
func (p *Processor) storePowerSmoothingEstimates(powers []powerActorInfo) error {
tx, err := p.db.Begin()
if err != nil {
return xerrors.Errorf("begin power_smoothing_estimates tx: %w", err)
}
if _, err := tx.Exec(`create temp table rse (like power_smoothing_estimates) on commit drop`); err != nil {
return xerrors.Errorf("prep power_smoothing_estimates: %w", err)
}
stmt, err := tx.Prepare(`copy rse (state_root, position_estimate, velocity_estimate) from stdin;`)
if err != nil {
return xerrors.Errorf("prepare tmp power_smoothing_estimates: %w", err)
}
for _, powerState := range powers {
if _, err := stmt.Exec(
powerState.common.stateroot.String(),
powerState.epochSmoothingEstimate.PositionEstimate.String(),
powerState.epochSmoothingEstimate.VelocityEstimate.String(),
); err != nil {
return xerrors.Errorf("failed to store smoothing estimate: %w", err)
}
}
if err := stmt.Close(); err != nil {
return xerrors.Errorf("close prepared power_smoothing_estimates: %w", err)
}
if _, err := tx.Exec(`insert into power_smoothing_estimates select * from rse on conflict do nothing`); err != nil {
return xerrors.Errorf("insert power_smoothing_estimates from tmp: %w", err)
}
if err := tx.Commit(); err != nil {
return xerrors.Errorf("commit power_smoothing_estimates tx: %w", err)
}
return nil
}

72
cmd/lotus-chainwatch/processor/processor.go
View File

@ -8,7 +8,6 @@ import (
"sync"
"time"
"golang.org/x/sync/errgroup"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
@ -88,6 +87,10 @@ func (p *Processor) setupSchemas() error {
return err
}
if err := p.setupPower(); err != nil {
return err
}
return nil
}
@ -140,52 +143,69 @@ func (p *Processor) Start(ctx context.Context) {
"AccountChanges", len(actorChanges[builtin.AccountActorCodeID]),
"nullRounds", len(nullRounds))
grp, ctx := errgroup.WithContext(ctx)
grp := sync.WaitGroup{}
grp.Go(func() error {
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandleMarketChanges(ctx, actorChanges[builtin.StorageMarketActorCodeID]); err != nil {
return xerrors.Errorf("Failed to handle market changes: %w", err)
log.Errorf("Failed to handle market changes: %w", err)
return
}
log.Info("Processed Market Changes")
return nil
})
}()
grp.Go(func() error {
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandleMinerChanges(ctx, actorChanges[builtin.StorageMinerActorCodeID]); err != nil {
return xerrors.Errorf("Failed to handle miner changes: %w", err)
log.Errorf("Failed to handle miner changes: %w", err)
return
}
log.Info("Processed Miner Changes")
return nil
})
}()
grp.Go(func() error {
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandleRewardChanges(ctx, actorChanges[builtin.RewardActorCodeID], nullRounds); err != nil {
return xerrors.Errorf("Failed to handle reward changes: %w", err)
log.Errorf("Failed to handle reward changes: %w", err)
return
}
log.Info("Processed Reward Changes")
return nil
})
}()
grp.Go(func() error {
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandlePowerChanges(ctx, actorChanges[builtin.StoragePowerActorCodeID]); err != nil {
log.Errorf("Failed to handle power actor changes: %w", err)
return
}
log.Info("Processes Power Changes")
}()
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandleMessageChanges(ctx, toProcess); err != nil {
return xerrors.Errorf("Failed to handle message changes: %w", err)
log.Errorf("Failed to handle message changes: %w", err)
return
}
log.Info("Processed Message Changes")
return nil
})
}()
grp.Go(func() error {
grp.Add(1)
go func() {
defer grp.Done()
if err := p.HandleCommonActorsChanges(ctx, actorChanges); err != nil {
return xerrors.Errorf("Failed to handle common actor changes: %w", err)
log.Errorf("Failed to handle common actor changes: %w", err)
return
}
log.Info("Processed CommonActor Changes")
return nil
})
}()
if err := grp.Wait(); err != nil {
log.Errorw("Failed to handle actor changes...retrying", "error", err)
continue
}
grp.Wait()
if err := p.markBlocksProcessed(ctx, toProcess); err != nil {
log.Fatalw("Failed to mark blocks as processed", "error", err)

67
cmd/lotus-chainwatch/processor/reward.go
View File

@ -11,6 +11,7 @@ import (
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
"github.com/filecoin-project/specs-actors/actors/util/smoothing"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
@ -24,6 +25,8 @@ type rewardActorInfo struct {
// base reward in attofil for each block found during this epoch
baseBlockReward big.Int
epochSmoothingEstimate *smoothing.FilterEstimate
}
func (p *Processor) setupRewards() error {
@ -53,6 +56,15 @@ create table if not exists chain_power
primary key,
baseline_power text not null
);
create table if not exists reward_smoothing_estimates
(
state_root text not null
constraint reward_smoothing_estimates_pk
primary key,
position_estimate text not null,
velocity_estimate text not null
);
`); err != nil {
return err
}
@ -63,7 +75,7 @@ create table if not exists chain_power
func (p *Processor) HandleRewardChanges(ctx context.Context, rewardTips ActorTips, nullRounds []types.TipSetKey) error {
rewardChanges, err := p.processRewardActors(ctx, rewardTips, nullRounds)
if err != nil {
log.Fatalw("Failed to process reward actors", "error", err)
return xerrors.Errorf("Failed to process reward actors: %w", err)
}
if err := p.persistRewardActors(ctx, rewardChanges); err != nil {
@ -103,6 +115,7 @@ func (p *Processor) processRewardActors(ctx context.Context, rewardTips ActorTip
rw.baseBlockReward = rewardActorState.ThisEpochReward
rw.baselinePower = rewardActorState.ThisEpochBaselinePower
rw.epochSmoothingEstimate = rewardActorState.ThisEpochRewardSmoothed
out = append(out, rw)
}
}
@ -146,7 +159,7 @@ func (p *Processor) persistRewardActors(ctx context.Context, rewards []rewardAct
log.Debugw("Persisted Reward Actors", "duration", time.Since(start).String())
}()
grp, ctx := errgroup.WithContext(ctx)
grp, ctx := errgroup.WithContext(ctx) //nolint
grp.Go(func() error {
if err := p.storeChainPower(rewards); err != nil {
@ -162,6 +175,13 @@ func (p *Processor) persistRewardActors(ctx context.Context, rewards []rewardAct
return nil
})
grp.Go(func() error {
if err := p.storeRewardSmoothingEstimates(rewards); err != nil {
return err
}
return nil
})
return grp.Wait()
}
@ -243,3 +263,46 @@ func (p *Processor) storeBaseBlockReward(rewards []rewardActorInfo) error {
return nil
}
func (p *Processor) storeRewardSmoothingEstimates(rewards []rewardActorInfo) error {
tx, err := p.db.Begin()
if err != nil {
return xerrors.Errorf("begin reward_smoothing_estimates tx: %w", err)
}
if _, err := tx.Exec(`create temp table rse (like reward_smoothing_estimates) on commit drop`); err != nil {
return xerrors.Errorf("prep reward_smoothing_estimates: %w", err)
}
stmt, err := tx.Prepare(`copy rse (state_root, position_estimate, velocity_estimate) from stdin;`)
if err != nil {
return xerrors.Errorf("prepare tmp reward_smoothing_estimates: %w", err)
}
for _, rewardState := range rewards {
if rewardState.epochSmoothingEstimate == nil {
continue
}
if _, err := stmt.Exec(
rewardState.common.stateroot.String(),
rewardState.epochSmoothingEstimate.PositionEstimate.String(),
rewardState.epochSmoothingEstimate.VelocityEstimate.String(),
); err != nil {
return xerrors.Errorf("failed to store smoothing estimate: %w", err)
}
}
if err := stmt.Close(); err != nil {
return xerrors.Errorf("close prepared reward_smoothing_estimates: %w", err)
}
if _, err := tx.Exec(`insert into reward_smoothing_estimates select * from rse on conflict do nothing`); err != nil {
return xerrors.Errorf("insert reward_smoothing_estimates from tmp: %w", err)
}
if err := tx.Commit(); err != nil {
return xerrors.Errorf("commit reward_smoothing_estimates tx: %w", err)
}
return nil
}

4
cmd/lotus-chainwatch/run.go
View File

@ -24,12 +24,12 @@ var runCmd = &cli.Command{
Flags: []cli.Flag{
&cli.IntFlag{
Name: "max-batch",
Value: 1000,
Value: 50,
},
},
Action: func(cctx *cli.Context) error {
go func() {
http.ListenAndServe(":6060", nil)
http.ListenAndServe(":6060", nil) //nolint:errcheck
}()
ll := cctx.String("log-level")
if err := logging.SetLogLevel("*", ll); err != nil {

4
cmd/lotus-chainwatch/scheduler/refresh_top_miners_by_base_reward.go
View File

@ -49,11 +49,11 @@ func setupTopMinerByBaseRewardSchema(ctx context.Context, db *sql.DB) error {
order by 1 desc
limit 1;
`); err != nil {
return xerrors.Errorf("create top_miner_by_base_reward views", err)
return xerrors.Errorf("create top_miner_by_base_reward views: %w", err)
}
if err := tx.Commit(); err != nil {
return xerrors.Errorf("commiting top_miner_by_base_reward views", err)
return xerrors.Errorf("committing top_miner_by_base_reward views; %w", err)
}
return nil
}

8
cmd/lotus-chainwatch/scheduler/scheduler.go
View File

@ -25,7 +25,7 @@ func PrepareScheduler(db *sql.DB) *Scheduler {
func (s *Scheduler) setupSchema(ctx context.Context) error {
if err := setupTopMinerByBaseRewardSchema(ctx, s.db); err != nil {
return xerrors.Errorf("setup top miners by reward schema", err)
return xerrors.Errorf("setup top miners by reward schema: %w", err)
}
return nil
}
@ -35,14 +35,14 @@ func (s *Scheduler) Start(ctx context.Context) {
log.Debug("Starting Scheduler")
if err := s.setupSchema(ctx); err != nil {
log.Fatalw("applying scheduling schema", err)
log.Fatalw("applying scheduling schema", "error", err)
}
go func() {
// run once on start after schema has initialized
time.Sleep(5 * time.Second)
if err := refreshTopMinerByBaseReward(ctx, s.db); err != nil {
log.Errorf(err.Error())
log.Errorw("failed to refresh top miner", "error", err)
}
refreshTopMinerCh := time.NewTicker(30 * time.Second)
defer refreshTopMinerCh.Stop()
@ -50,7 +50,7 @@ func (s *Scheduler) Start(ctx context.Context) {
select {
case <-refreshTopMinerCh.C:
if err := refreshTopMinerByBaseReward(ctx, s.db); err != nil {
log.Errorf(err.Error())
log.Errorw("failed to refresh top miner", "error", err)
}
case <-ctx.Done():
return

268
cmd/lotus-fountain/main.go
View File

@ -1,77 +1,27 @@
package main
import (
"bytes"
"context"
"fmt"
"html/template"
"io"
"io/ioutil"
"net"
"net/http"
"os"
"sort"
"strconv"
"time"
rice "github.com/GeertJohan/go.rice"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/types"
lcli "github.com/filecoin-project/lotus/cli"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
)
var log = logging.Logger("main")
var supportedSectors struct {
SectorSizes []struct {
Name string
Value uint64
Default bool
}
}
func init() {
for supportedSector, _ := range miner.SupportedProofTypes {
sectorSize, err := supportedSector.SectorSize()
if err != nil {
panic(err)
}
supportedSectors.SectorSizes = append(supportedSectors.SectorSizes, struct {
Name string
Value uint64
Default bool
}{
Name: sectorSize.ShortString(),
Value: uint64(sectorSize),
Default: false,
})
}
sort.Slice(supportedSectors.SectorSizes[:], func(i, j int) bool {
return supportedSectors.SectorSizes[i].Value < supportedSectors.SectorSizes[j].Value
})
supportedSectors.SectorSizes[0].Default = true
}
func main() {
logging.SetLogLevel("*", "INFO")
@ -144,11 +94,6 @@ var runCmd = &cli.Command{
return xerrors.Errorf("parsing source address (provide correct --from flag!): %w", err)
}
defaultMinerPeer, err := peer.Decode("12D3KooWJpBNhwgvoZ15EB1JwRTRpxgM9D2fwq6eEktrJJG74aP6")
if err != nil {
return err
}
h := &handler{
ctx: ctx,
api: nodeApi,
@ -162,23 +107,10 @@ var runCmd = &cli.Command{
WalletRate: 15 * time.Minute,
WalletBurst: 2,
}),
minerLimiter: NewLimiter(LimiterConfig{
TotalRate: 500 * time.Millisecond,
TotalBurst: build.BlockMessageLimit,
IPRate: 10 * time.Minute,
IPBurst: 2,
WalletRate: 1 * time.Hour,
WalletBurst: 2,
}),
defaultMinerPeer: defaultMinerPeer,
}
http.Handle("/", http.FileServer(rice.MustFindBox("site").HTTPBox()))
http.HandleFunc("/miner.html", h.minerhtml)
http.HandleFunc("/send", h.send)
http.HandleFunc("/mkminer", h.mkminer)
http.HandleFunc("/msgwait", h.msgwait)
http.HandleFunc("/msgwaitaddr", h.msgwaitaddr)
fmt.Printf("Open http://%s\n", cctx.String("front"))
@ -198,48 +130,13 @@ type handler struct {
from address.Address
sendPerRequest types.FIL
limiter *Limiter
minerLimiter *Limiter
defaultMinerPeer peer.ID
}
func (h *handler) minerhtml(w http.ResponseWriter, r *http.Request) {
f, err := rice.MustFindBox("site").Open("_miner.html")
if err != nil {
w.WriteHeader(500)
_, _ = w.Write([]byte(err.Error()))
return
}
tmpl, err := ioutil.ReadAll(f)
if err != nil {
w.WriteHeader(500)
_, _ = w.Write([]byte(err.Error()))
return
}
var executedTmpl bytes.Buffer
t, err := template.New("miner.html").Parse(string(tmpl))
if err := t.Execute(&executedTmpl, supportedSectors); err != nil {
w.WriteHeader(500)
_, _ = w.Write([]byte(err.Error()))
return
}
if _, err := io.Copy(w, &executedTmpl); err != nil {
log.Errorf("failed to write template to string %s", err)
}
return
limiter *Limiter
}
func (h *handler) send(w http.ResponseWriter, r *http.Request) {
to, err := address.NewFromString(r.FormValue("address"))
if err != nil {
w.WriteHeader(400)
_, _ = w.Write([]byte(err.Error()))
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
@ -282,168 +179,9 @@ func (h *handler) send(w http.ResponseWriter, r *http.Request) {
To: to,
}, nil)
if err != nil {
w.WriteHeader(400)
_, _ = w.Write([]byte(err.Error()))
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
_, _ = w.Write([]byte(smsg.Cid().String()))
}
func (h *handler) mkminer(w http.ResponseWriter, r *http.Request) {
owner, err := address.NewFromString(r.FormValue("address"))
if err != nil {
w.WriteHeader(400)
_, _ = w.Write([]byte(err.Error()))
return
}
if owner.Protocol() != address.BLS {
w.WriteHeader(400)
_, _ = w.Write([]byte("Miner address must use BLS. A BLS address starts with the prefix 't3'."))
_, _ = w.Write([]byte("Please create a BLS address by running \"lotus wallet new bls\" while connected to a Lotus node."))
return
}
ssize, err := strconv.ParseInt(r.FormValue("sectorSize"), 10, 64)
if err != nil {
return
}
log.Infof("%s: create actor start", owner)
// Limit based on wallet address
limiter := h.minerLimiter.GetWalletLimiter(owner.String())
if !limiter.Allow() {
http.Error(w, http.StatusText(http.StatusTooManyRequests)+": wallet limit", http.StatusTooManyRequests)
return
}
// Limit based on IP
reqIP := r.Header.Get("X-Real-IP")
if reqIP == "" {
h, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
log.Errorf("could not get ip from: %s, err: %s", r.RemoteAddr, err)
}
reqIP = h
}
limiter = h.minerLimiter.GetIPLimiter(reqIP)
if !limiter.Allow() {
http.Error(w, http.StatusText(http.StatusTooManyRequests)+": IP limit", http.StatusTooManyRequests)
return
}
// General limiter owner allow throttling all messages that can make it into the mpool
if !h.minerLimiter.Allow() {
http.Error(w, http.StatusText(http.StatusTooManyRequests)+": global limit", http.StatusTooManyRequests)
return
}
smsg, err := h.api.MpoolPushMessage(h.ctx, &types.Message{
Value: types.BigInt(h.sendPerRequest),
From: h.from,
To: owner,
}, nil)
if err != nil {
w.WriteHeader(400)
w.Write([]byte("pushfunds: " + err.Error()))
return
}
log.Infof("%s: push funds %s", owner, smsg.Cid())
spt, err := ffiwrapper.SealProofTypeFromSectorSize(abi.SectorSize(ssize))
if err != nil {
w.WriteHeader(400)
w.Write([]byte("sealprooftype: " + err.Error()))
return
}
params, err := actors.SerializeParams(&power.CreateMinerParams{
Owner: owner,
Worker: owner,
SealProofType: spt,
Peer: abi.PeerID(h.defaultMinerPeer),
})
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
createStorageMinerMsg := &types.Message{
To: builtin.StoragePowerActorAddr,
From: h.from,
Value: big.Zero(),
Method: builtin.MethodsPower.CreateMiner,
Params: params,
}
signed, err := h.api.MpoolPushMessage(r.Context(), createStorageMinerMsg, nil)
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
log.Infof("%s: create miner msg: %s", owner, signed.Cid())
http.Redirect(w, r, fmt.Sprintf("/wait.html?f=%s&m=%s&o=%s", signed.Cid(), smsg.Cid(), owner), 303)
}
func (h *handler) msgwait(w http.ResponseWriter, r *http.Request) {
c, err := cid.Parse(r.FormValue("cid"))
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
mw, err := h.api.StateWaitMsg(r.Context(), c, build.MessageConfidence)
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
if mw.Receipt.ExitCode != 0 {
w.WriteHeader(400)
w.Write([]byte(xerrors.Errorf("create miner failed: exit code %d", mw.Receipt.ExitCode).Error()))
return
}
w.WriteHeader(200)
}
func (h *handler) msgwaitaddr(w http.ResponseWriter, r *http.Request) {
c, err := cid.Parse(r.FormValue("cid"))
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
mw, err := h.api.StateWaitMsg(r.Context(), c, build.MessageConfidence)
if err != nil {
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
if mw.Receipt.ExitCode != 0 {
w.WriteHeader(400)
w.Write([]byte(xerrors.Errorf("create miner failed: exit code %d", mw.Receipt.ExitCode).Error()))
return
}
w.WriteHeader(200)
var ma power.CreateMinerReturn
if err := ma.UnmarshalCBOR(bytes.NewReader(mw.Receipt.Return)); err != nil {
log.Errorf("%w", err)
w.WriteHeader(400)
w.Write([]byte(err.Error()))
return
}
fmt.Fprintf(w, "{\"addr\": \"%s\"}", ma.IDAddress)
}

51
cmd/lotus-fountain/site/_miner.html
View File

@ -1,51 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<title>Creating Miner - Lotus Fountain</title>
<link rel="stylesheet" type="text/css" href="main.css">
</head>
<body>
<div class="Index">
<div class="Index-nodes">
<div class="Index-node">
[CREATING MINER]
</div>
<div class="Index-node" id="formnd">
<form id="f" action='/mkminer' method='POST'>
<span>Enter owner/worker address:</span>
<input type='text' name='address' style="width: 300px" placeholder="t3...">
<select name="sectorSize">
{{range .SectorSizes}}
<option {{if .Default}}selected{{end}} value="{{ .Value }}">{{ .Name }}</option>
{{end}}
</select>
<button type='submit'>Create Miner</button>
</form>
</div>
<div id="plswait" style="display: none" class="Index-node">
<b>Waiting for transaction on chain..</b>
</div>
<div class="Index-node">
<span>When creating miner, DO NOT REFRESH THE PAGE, wait for it to load. This can take more than 5min.</span>
</div>
<div class="Index-node">
<span>If you don't have an owner/worker address, you can create it by following <a target="_blank" href="https://docs.lotu.sh/en+mining#get-started-22083">these instructions</a>.</span>
</div>
</div>
<div class="Index-footer">
<div>
<a href="index.html">[Back]</a>
<span style="float: right">Not dispensing real Filecoin tokens</span>
</div>
</div>
</div>
<script>
let f = document.getElementById('f')
f.onsubmit = ev => {
document.getElementById('plswait').style.display = 'block'
document.getElementById('formnd').style.display = 'none'
}
</script>
</body>
</html>

3
cmd/lotus-fountain/site/index.html
View File

@ -13,9 +13,6 @@
<div class="Index-node">
<a href="funds.html">[Send Funds]</a>
</div>
<div class="Index-node">
<a href="miner.html">[Create Miner]</a>
</div>
</div>
<div class="Index-footer">
<div>

69
cmd/lotus-fountain/site/wait.html
View File

@ -1,69 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<title>Creating Miner (wait) - Lotus Fountain</title>
<link rel="stylesheet" type="text/css" href="main.css">
</head>
<body>
<div class="Index">
<div class="Index-nodes">
<div class="Index-node">
[CREATING MINER]
</div>
<div class="Index-node">
Gas Funds:&nbsp;&nbsp;&nbsp;<span id="fcid"></span> - <span id="fstate">WAIT</span>
</div>
<div class="Index-node">
Miner Actor:&nbsp;<span id="mcid"></span> - <span id="mstate">WAIT</span>
</div>
<div class="Index-node" style="display: none" id="fwait">
New miners address is: <b id="actaddr">t</b>
</div>
<div class="Index-node" style="display: none" id="mwait">
<div style="padding-bottom: 1em">To initialize the miner run the following command:</div>
<div style="overflow-x: visible; white-space: nowrap; background: #353500">
<code>lotus-miner init --actor=<span id="actaddr2">t</span> --owner=<span id="owner">t3</span></code>
</div>
</div>
</div>
<div class="Index-footer">
<div>
<a href="index.html">[Back]</a>
<span style="float: right">Not dispensing real Filecoin tokens</span>
</div>
</div>
</div>
<script>
const params = new URLSearchParams(window.location.search);
const fcid = document.getElementById('fcid')
const mcid = document.getElementById('mcid')
fcid.innerText = params.get('f')
mcid.innerText = params.get('m')
async function waitFunds() {
await fetch('/msgwait?cid=' + params.get('f'))
document.getElementById('fstate').innerText = "OK"
document.getElementById('fwait').style.display = 'block'
}
async function waitMiner() {
const resp = await fetch('/msgwaitaddr?cid=' + params.get('f'))
document.getElementById('mstate').innerText = "OK"
const addr = (await resp.json()).addr
document.getElementById('actaddr').innerText = addr
document.getElementById('actaddr2').innerText = addr
document.getElementById('owner').innerText = params.get('o')
document.getElementById('mwait').style.display = 'block'
}
waitFunds()
waitMiner()
</script>
</body>
</html>

7
cmd/lotus-keygen/main.go
View File

@ -54,7 +54,12 @@ func main() {
if err != nil {
return err
}
defer fi.Close()
defer func() {
err2 := fi.Close()
if err == nil {
err = err2
}
}()
b, err := json.Marshal(ki)
if err != nil {

24
cmd/lotus-pcr/main.go
View File

@ -142,11 +142,11 @@ var runCmd = &cli.Command{
},
Action: func(cctx *cli.Context) error {
go func() {
http.ListenAndServe(":6060", nil)
http.ListenAndServe(":6060", nil) //nolint:errcheck
}()
ctx := context.Background()
api, closer, err := stats.GetFullNodeAPI(cctx.String("lotus-path"))
api, closer, err := stats.GetFullNodeAPI(cctx.Context, cctx.String("lotus-path"))
if err != nil {
log.Fatal(err)
}
@ -394,7 +394,7 @@ func (r *refunder) ProcessTipset(ctx context.Context, tipset *types.TipSet) (*Mi
}
if r.percentExtra > 0 {
refundValue = types.BigAdd(refundValue, types.BigDiv(types.BigMul(refundValue, types.NewInt(100)), types.NewInt(uint64(r.percentExtra))))
refundValue = types.BigAdd(refundValue, types.BigMul(types.BigDiv(refundValue, types.NewInt(100)), types.NewInt(uint64(r.percentExtra))))
}
log.Debugw("processing message", "method", messageMethod, "cid", msg.Cid, "from", m.From, "to", m.To, "value", m.Value, "gas_fee_cap", m.GasFeeCap, "gas_premium", m.GasPremium, "gas_used", recps[i].GasUsed, "refund", refundValue)
@ -445,7 +445,7 @@ func (r *refunder) Refund(ctx context.Context, tipset *types.TipSet, refunds *Mi
// Calculate the minimum balance as the total refund we need to issue plus 5% to cover fees
minBalance := types.BigAdd(refundSum, types.BigDiv(refundSum, types.NewInt(500)))
if balance.LessThan(minBalance) {
log.Errorw("not sufficent funds to cover refunds", "balance", balance, "refund_sum", refundSum, "minimum_required", minBalance)
log.Errorw("not sufficient funds to cover refunds", "balance", balance, "refund_sum", refundSum, "minimum_required", minBalance)
return xerrors.Errorf("wallet does not have enough balance to cover refund")
}
@ -467,24 +467,24 @@ func (r *refunder) Refund(ctx context.Context, tipset *types.TipSet, refunds *Mi
return nil
}
type repo struct {
type Repo struct {
last abi.ChainEpoch
path string
}
func NewRepo(path string) (*repo, error) {
func NewRepo(path string) (*Repo, error) {
path, err := homedir.Expand(path)
if err != nil {
return nil, err
}
return &repo{
return &Repo{
last: 0,
path: path,
}, nil
}
func (r *repo) exists() (bool, error) {
func (r *Repo) exists() (bool, error) {
_, err := os.Stat(r.path)
notexist := os.IsNotExist(err)
if notexist {
@ -494,7 +494,7 @@ func (r *repo) exists() (bool, error) {
}
func (r *repo) init() error {
func (r *Repo) init() error {
exist, err := r.exists()
if err != nil {
return err
@ -511,7 +511,7 @@ func (r *repo) init() error {
return nil
}
func (r *repo) Open() (err error) {
func (r *Repo) Open() (err error) {
if err = r.init(); err != nil {
return
}
@ -542,11 +542,11 @@ func (r *repo) Open() (err error) {
return
}
func (r *repo) Height() abi.ChainEpoch {
func (r *Repo) Height() abi.ChainEpoch {
return r.last
}
func (r *repo) SetHeight(last abi.ChainEpoch) (err error) {
func (r *Repo) SetHeight(last abi.ChainEpoch) (err error) {
r.last = last
var f *os.File
f, err = os.OpenFile(filepath.Join(r.path, "height"), os.O_RDWR, 0644)

9
cmd/lotus-seal-worker/main.go
View File

@ -185,7 +185,7 @@ var runCmd = &cli.Command{
return err
}
if v.APIVersion != build.APIVersion {
return xerrors.Errorf("lotus-miner API version doesn't match: local: ", api.Version{APIVersion: build.APIVersion})
return xerrors.Errorf("lotus-miner API version doesn't match: local: %s", api.Version{APIVersion: build.APIVersion})
}
log.Infof("Remote version %s", v)
@ -420,10 +420,11 @@ func watchMinerConn(ctx context.Context, cctx *cli.Context, nodeApi api.StorageM
log.Errorf("getting executable for auto-restart: %+v", err)
}
log.Sync()
_ = log.Sync()
// TODO: there are probably cleaner/more graceful ways to restart,
// but this is good enough for now (FSM can recover from the mess this creates)
//nolint:gosec
if err := syscall.Exec(exe, []string{exe,
fmt.Sprintf("--worker-repo=%s", cctx.String("worker-repo")),
fmt.Sprintf("--miner-repo=%s", cctx.String("miner-repo")),
@ -450,7 +451,7 @@ func extractRoutableIP(timeout time.Duration) (string, error) {
env, ok := os.LookupEnv(minerMultiAddrKey)
if !ok {
// TODO remove after deprecation period
env, ok = os.LookupEnv(deprecatedMinerMultiAddrKey)
_, ok = os.LookupEnv(deprecatedMinerMultiAddrKey)
if ok {
log.Warnf("Using a deprecated env(%s) value, please use env(%s) instead.", deprecatedMinerMultiAddrKey, minerMultiAddrKey)
}
@ -461,7 +462,7 @@ func extractRoutableIP(timeout time.Duration) (string, error) {
if err != nil {
return "", err
}
defer conn.Close()
defer conn.Close() //nolint:errcheck
localAddr := conn.LocalAddr().(*net.TCPAddr)

3
cmd/lotus-seed/main.go
View File

@ -114,6 +114,9 @@ var preSealCmd = &cli.Command{
return err
}
kb, err := hex.DecodeString(string(kh))
if err != nil {
return err
}
if err := json.Unmarshal(kb, k); err != nil {
return err
}

1
cmd/lotus-seed/seed/seed.go
View File

@ -114,6 +114,7 @@ func PreSeal(maddr address.Address, spt abi.RegisteredSealProof, offset abi.Sect
}
miner := &genesis.Miner{
ID: maddr,
Owner: minerAddr.Address,
Worker: minerAddr.Address,
MarketBalance: big.Zero(),

160
cmd/lotus-shed/genesis-verify.go Normal file
View File

@ -0,0 +1,160 @@
package main
import (
"context"
"fmt"
"os"
"sort"
"github.com/fatih/color"
"github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
saacc "github.com/filecoin-project/specs-actors/actors/builtin/account"
saminer "github.com/filecoin-project/specs-actors/actors/builtin/miner"
samsig "github.com/filecoin-project/specs-actors/actors/builtin/multisig"
)
type addrInfo struct {
Key address.Address
Balance types.FIL
}
type msigInfo struct {
Signers []address.Address
Balance types.FIL
Threshold uint64
}
type minerInfo struct {
}
var genesisVerifyCmd = &cli.Command{
Name: "verify-genesis",
Description: "verify some basic attributes of a genesis car file",
Action: func(cctx *cli.Context) error {
if !cctx.Args().Present() {
return fmt.Errorf("must pass genesis car file")
}
bs := blockstore.NewBlockstore(datastore.NewMapDatastore())
cs := store.NewChainStore(bs, datastore.NewMapDatastore(), nil)
cf := cctx.Args().Get(0)
f, err := os.Open(cf)
if err != nil {
return xerrors.Errorf("opening the car file: %w", err)
}
ts, err := cs.Import(f)
if err != nil {
return err
}
sm := stmgr.NewStateManager(cs)
total, err := stmgr.CheckTotalFIL(context.TODO(), sm, ts)
if err != nil {
return err
}
fmt.Println("Genesis: ", ts.Key())
expFIL := big.Mul(big.NewInt(int64(build.FilBase)), big.NewInt(int64(build.FilecoinPrecision)))
fmt.Printf("Total FIL: %s", types.FIL(total))
if !expFIL.Equals(total) {
color.Red(" INCORRECT!")
}
fmt.Println()
cst := cbor.NewCborStore(bs)
stree, err := state.LoadStateTree(cst, ts.ParentState())
if err != nil {
return err
}
var accAddrs, msigAddrs []address.Address
kaccounts := make(map[address.Address]addrInfo)
kmultisigs := make(map[address.Address]msigInfo)
kminers := make(map[address.Address]minerInfo)
ctx := context.TODO()
if err := stree.ForEach(func(addr address.Address, act *types.Actor) error {
switch act.Code {
case builtin.StorageMinerActorCodeID:
var st saminer.State
if err := cst.Get(ctx, act.Head, &st); err != nil {
return err
}
kminers[addr] = minerInfo{}
case builtin.MultisigActorCodeID:
var st samsig.State
if err := cst.Get(ctx, act.Head, &st); err != nil {
return xerrors.Errorf("multisig actor: %w", err)
}
kmultisigs[addr] = msigInfo{
Balance: types.FIL(act.Balance),
Signers: st.Signers,
Threshold: st.NumApprovalsThreshold,
}
msigAddrs = append(msigAddrs, addr)
case builtin.AccountActorCodeID:
var st saacc.State
if err := cst.Get(ctx, act.Head, &st); err != nil {
log.Warn(xerrors.Errorf("account actor %s: %w", addr, err))
}
kaccounts[addr] = addrInfo{
Key: st.Address,
Balance: types.FIL(act.Balance.Copy()),
}
accAddrs = append(accAddrs, addr)
}
return nil
}); err != nil {
return err
}
sort.Slice(accAddrs, func(i, j int) bool {
return accAddrs[i].String() < accAddrs[j].String()
})
sort.Slice(msigAddrs, func(i, j int) bool {
return msigAddrs[i].String() < msigAddrs[j].String()
})
fmt.Println("Account Actors:")
for _, acc := range accAddrs {
a := kaccounts[acc]
fmt.Printf("%s\t%s\t%s\n", acc, a.Key, a.Balance)
}
fmt.Println("Multisig Actors:")
for _, acc := range msigAddrs {
m := kmultisigs[acc]
fmt.Printf("%s\t%s\t%d\t[", acc, m.Balance, m.Threshold)
for i, s := range m.Signers {
fmt.Print(s)
if i != len(m.Signers)-1 {
fmt.Print(",")
}
}
fmt.Printf("]\n")
}
return nil
},
}

89
cmd/lotus-shed/jwt.go Normal file
View File

@ -0,0 +1,89 @@
package main
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"os"
"github.com/gbrlsnchs/jwt/v3"
"github.com/urfave/cli/v2"
"github.com/filecoin-project/lotus/api/apistruct"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/modules"
)
var jwtCmd = &cli.Command{
Name: "jwt",
Usage: "work with lotus jwt secrets and tokens",
Description: `The subcommands of jwt provide helpful tools for working with jwt files without
having to run the lotus daemon.`,
Subcommands: []*cli.Command{
jwtNewCmd,
},
}
var jwtNewCmd = &cli.Command{
Name: "new",
Usage: "create a new jwt secret and token for lotus",
ArgsUsage: "<name>",
Description: `Jwt tokens are used to authenticate api requests to the lotus daemon.
The created jwt token have full privileges and should not be shared.`,
Flags: []cli.Flag{},
Action: func(cctx *cli.Context) error {
if !cctx.Args().Present() {
return fmt.Errorf("please specify a name")
}
keyName := cctx.Args().First()
sk, err := ioutil.ReadAll(io.LimitReader(rand.Reader, 32))
if err != nil {
return err
}
keyInfo := types.KeyInfo{
Type: modules.KTJwtHmacSecret,
PrivateKey: sk,
}
p := modules.JwtPayload{
Allow: apistruct.AllPermissions,
}
token, err := jwt.Sign(&p, jwt.NewHS256(keyInfo.PrivateKey))
if err != nil {
return err
}
filename := fmt.Sprintf("jwt-%s.jwts", keyName)
file, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
return err
}
defer func() {
if err := file.Close(); err != nil {
log.Warnf("failed to close output file: %w", err)
}
}()
bytes, err := json.Marshal(keyInfo)
if err != nil {
return err
}
encoded := hex.EncodeToString(bytes)
if _, err := file.Write([]byte(encoded)); err != nil {
return err
}
filenameToken := fmt.Sprintf("jwt-%s.token", keyName)
return ioutil.WriteFile(filenameToken, token, 0600)
},
}

12
cmd/lotus-shed/keyinfo.go
View File

@ -69,7 +69,7 @@ var keyinfoImportCmd = &cli.Command{
if err != nil {
return err
}
defer inputFile.Close()
defer inputFile.Close() //nolint:errcheck
input = bufio.NewReader(inputFile)
}
@ -98,7 +98,7 @@ var keyinfoImportCmd = &cli.Command{
return err
}
defer lkrepo.Close()
defer lkrepo.Close() //nolint:errcheck
keystore, err := lkrepo.KeyStore()
if err != nil {
@ -150,7 +150,7 @@ var keyinfoInfoCmd = &cli.Command{
The 'format' flag takes a golang text/template template as its value.
The following fields can be retrived through this command
The following fields can be retrieved through this command
Type
Address
PublicKey
@ -159,7 +159,7 @@ var keyinfoInfoCmd = &cli.Command{
Examples
Retreive the address of a lotus wallet
Retrieve the address of a lotus wallet
lotus-shed keyinfo info --format '{{ .Address }}' wallet.keyinfo
`,
Flags: []cli.Flag{
@ -181,7 +181,7 @@ var keyinfoInfoCmd = &cli.Command{
if err != nil {
return err
}
defer inputFile.Close()
defer inputFile.Close() //nolint:errcheck
input = bufio.NewReader(inputFile)
}
@ -330,7 +330,7 @@ var keyinfoNewCmd = &cli.Command{
filename = strings.ReplaceAll(filename, "<addr>", keyAddr)
filename = strings.ReplaceAll(filename, "<type>", keyType)
file, err := os.Create(filename)
file, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
return err
}

2
cmd/lotus-shed/main.go
View File

@ -19,6 +19,7 @@ func main() {
base16Cmd,
bitFieldCmd,
keyinfoCmd,
jwtCmd,
noncefix,
bigIntParseCmd,
staterootCmd,
@ -29,6 +30,7 @@ func main() {
verifRegCmd,
miscCmd,
mpoolCmd,
genesisVerifyCmd,
}
app := &cli.App{

2
cmd/lotus-stats/main.go
View File

@ -56,7 +56,7 @@ func main() {
height = h
}
api, closer, err := stats.GetFullNodeAPI(repo)
api, closer, err := stats.GetFullNodeAPI(ctx, repo)
if err != nil {
log.Fatal(err)
}

245
cmd/lotus-storage-miner/actor.go
View File

@ -2,19 +2,26 @@ package main
import (
"fmt"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/libp2p/go-libp2p-core/peer"
"golang.org/x/xerrors"
"os"
"strings"
"github.com/fatih/color"
"github.com/libp2p/go-libp2p-core/peer"
ma "github.com/multiformats/go-multiaddr"
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/types"
lcli "github.com/filecoin-project/lotus/cli"
"github.com/filecoin-project/lotus/lib/tablewriter"
"github.com/filecoin-project/lotus/storage"
)
var actorCmd = &cli.Command{
@ -24,6 +31,7 @@ var actorCmd = &cli.Command{
actorSetAddrsCmd,
actorWithdrawCmd,
actorSetPeeridCmd,
actorControl,
},
}
@ -129,7 +137,7 @@ var actorSetPeeridCmd = &cli.Command{
ctx := lcli.ReqContext(cctx)
pid, err := peer.IDFromString(cctx.Args().Get(0))
pid, err := peer.Decode(cctx.Args().Get(0))
if err != nil {
return fmt.Errorf("failed to parse input as a peerId: %w", err)
}
@ -240,3 +248,232 @@ var actorWithdrawCmd = &cli.Command{
return nil
},
}
var actorControl = &cli.Command{
Name: "control",
Usage: "Manage control addresses",
Subcommands: []*cli.Command{
actorControlList,
actorControlSet,
},
}
var actorControlList = &cli.Command{
Name: "list",
Usage: "Get currently set control addresses",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "verbose",
},
&cli.BoolFlag{
Name: "color",
Value: true,
},
},
Action: func(cctx *cli.Context) error {
color.NoColor = !cctx.Bool("color")
nodeApi, closer, err := lcli.GetStorageMinerAPI(cctx)
if err != nil {
return err
}
defer closer()
api, acloser, err := lcli.GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer acloser()
ctx := lcli.ReqContext(cctx)
maddr, err := nodeApi.ActorAddress(ctx)
if err != nil {
return err
}
mi, err := api.StateMinerInfo(ctx, maddr, types.EmptyTSK)
if err != nil {
return err
}
tw := tablewriter.New(
tablewriter.Col("name"),
tablewriter.Col("ID"),
tablewriter.Col("key"),
tablewriter.Col("use"),
tablewriter.Col("balance"),
)
postAddr, err := storage.AddressFor(ctx, api, mi, storage.PoStAddr, types.FromFil(1))
if err != nil {
return xerrors.Errorf("getting address for post: %w", err)
}
printKey := func(name string, a address.Address) {
b, err := api.WalletBalance(ctx, a)
if err != nil {
fmt.Printf("%s\t%s: error getting balance: %s\n", name, a, err)
return
}
k, err := api.StateAccountKey(ctx, a, types.EmptyTSK)
if err != nil {
fmt.Printf("%s\t%s: error getting account key: %s\n", name, a, err)
return
}
kstr := k.String()
if !cctx.Bool("verbose") {
kstr = kstr[:9] + "..."
}
bstr := types.FIL(b).String()
switch {
case b.LessThan(types.FromFil(10)):
bstr = color.RedString(bstr)
case b.LessThan(types.FromFil(50)):
bstr = color.YellowString(bstr)
default:
bstr = color.GreenString(bstr)
}
var uses []string
if a == mi.Worker {
uses = append(uses, color.YellowString("other"))
}
if a == postAddr {
uses = append(uses, color.GreenString("post"))
}
tw.Write(map[string]interface{}{
"name": name,
"ID": a,
"key": kstr,
"use": strings.Join(uses, " "),
"balance": bstr,
})
}
printKey("owner", mi.Owner)
printKey("worker", mi.Worker)
for i, ca := range mi.ControlAddresses {
printKey(fmt.Sprintf("control-%d", i), ca)
}
return tw.Flush(os.Stdout)
},
}
var actorControlSet = &cli.Command{
Name: "set",
Usage: "Set control address(-es)",
ArgsUsage: "[...address]",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "really-do-it",
Usage: "Actually send transaction performing the action",
Value: false,
},
},
Action: func(cctx *cli.Context) error {
nodeApi, closer, err := lcli.GetStorageMinerAPI(cctx)
if err != nil {
return err
}
defer closer()
api, acloser, err := lcli.GetFullNodeAPI(cctx)
if err != nil {
return err
}
defer acloser()
ctx := lcli.ReqContext(cctx)
maddr, err := nodeApi.ActorAddress(ctx)
if err != nil {
return err
}
mi, err := api.StateMinerInfo(ctx, maddr, types.EmptyTSK)
if err != nil {
return err
}
del := map[address.Address]struct{}{}
existing := map[address.Address]struct{}{}
for _, controlAddress := range mi.ControlAddresses {
ka, err := api.StateAccountKey(ctx, controlAddress, types.EmptyTSK)
if err != nil {
return err
}
del[ka] = struct{}{}
existing[ka] = struct{}{}
}
var toSet []address.Address
for i, as := range cctx.Args().Slice() {
a, err := address.NewFromString(as)
if err != nil {
return xerrors.Errorf("parsing address %d: %w", i, err)
}
ka, err := api.StateAccountKey(ctx, a, types.EmptyTSK)
if err != nil {
return err
}
// make sure the address exists on chain
_, err = api.StateLookupID(ctx, ka, types.EmptyTSK)
if err != nil {
return xerrors.Errorf("looking up %s: %w", ka, err)
}
delete(del, ka)
toSet = append(toSet, ka)
}
for a := range del {
fmt.Println("Remove", a)
}
for _, a := range toSet {
if _, exists := existing[a]; !exists {
fmt.Println("Add", a)
}
}
if !cctx.Bool("really-do-it") {
fmt.Println("Pass --really-do-it to actually execute this action")
return nil
}
cwp := &miner.ChangeWorkerAddressParams{
NewWorker: mi.Worker,
NewControlAddrs: toSet,
}
sp, err := actors.SerializeParams(cwp)
if err != nil {
return xerrors.Errorf("serializing params: %w", err)
}
smsg, err := api.MpoolPushMessage(ctx, &types.Message{
From: mi.Owner,
To: maddr,
Method: builtin.MethodsMiner.ChangeWorkerAddress,
Value: big.Zero(),
Params: sp,
}, nil)
if err != nil {
return xerrors.Errorf("mpool push: %w", err)
}
fmt.Println("Message CID:", smsg.Cid())
return nil
},
}

48
cmd/lotus-storage-miner/info.go
View File

@ -11,14 +11,22 @@ import (
"github.com/urfave/cli/v2"
"golang.org/x/xerrors"
cbor "github.com/ipfs/go-ipld-cbor"
"github.com/filecoin-project/go-fil-markets/storagemarket"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apibstore"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
lcli "github.com/filecoin-project/lotus/cli"
"github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/lotus/lib/bufbstore"
)
var infoCmd = &cli.Command{
@ -47,6 +55,11 @@ func infoCmdAct(cctx *cli.Context) error {
ctx := lcli.ReqContext(cctx)
head, err := api.ChainHead(ctx)
if err != nil {
return xerrors.Errorf("getting chain head: %w", err)
}
maddr, err := getActorAddress(ctx, nodeApi, cctx.String("actor"))
if err != nil {
return err
@ -141,10 +154,43 @@ func infoCmdAct(cctx *cli.Context) error {
fmt.Println()
deals, err := nodeApi.MarketListIncompleteDeals(ctx)
if err != nil {
return err
}
var nactiveDeals, nVerifDeals, ndeals uint64
var activeDealBytes, activeVerifDealBytes, dealBytes abi.PaddedPieceSize
for _, deal := range deals {
ndeals++
dealBytes += deal.Proposal.PieceSize
if deal.State == storagemarket.StorageDealActive {
nactiveDeals++
activeDealBytes += deal.Proposal.PieceSize
if deal.Proposal.VerifiedDeal {
nVerifDeals++
activeVerifDealBytes += deal.Proposal.PieceSize
}
}
}
fmt.Printf("Deals: %d, %s\n", ndeals, types.SizeStr(types.NewInt(uint64(dealBytes))))
fmt.Printf("\tActive: %d, %s (Verified: %d, %s)\n", nactiveDeals, types.SizeStr(types.NewInt(uint64(activeDealBytes))), nVerifDeals, types.SizeStr(types.NewInt(uint64(activeVerifDealBytes))))
fmt.Println()
tbs := bufbstore.NewTieredBstore(apibstore.NewAPIBlockstore(api), blockstore.NewTemporary())
_, err = mas.UnlockVestedFunds(adt.WrapStore(ctx, cbor.NewCborStore(tbs)), head.Height())
if err != nil {
return xerrors.Errorf("calculating vested funds: %w", err)
}
fmt.Printf("Miner Balance: %s\n", color.YellowString("%s", types.FIL(mact.Balance)))
fmt.Printf("\tPreCommit: %s\n", types.FIL(mas.PreCommitDeposits))
fmt.Printf("\tPledge: %s\n", types.FIL(mas.InitialPledgeRequirement))
fmt.Printf("\tLocked: %s\n", types.FIL(mas.LockedFunds))
color.Green("\tAvailable: %s", types.FIL(types.BigSub(mact.Balance, types.BigAdd(mas.LockedFunds, mas.PreCommitDeposits))))
color.Green("\tAvailable: %s", types.FIL(mas.GetAvailableBalance(mact.Balance)))
wb, err := api.WalletBalance(ctx, mi.Worker)
if err != nil {
return xerrors.Errorf("getting worker balance: %w", err)

Some files were not shown because too many files have changed in this diff Show More