cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity

Merge branch 'master' into digitalocean-use-snap

Browse Source
This commit is contained in:
Cory Schwartz 2022-01-18 17:02:41 -08:00
commit a3458213f4
306 changed files with 13432 additions and 3005 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.circleci/config.yml
View File

@ -906,6 +906,11 @@ workflows:
suite: itest-sector_terminate
target: "./itests/sector_terminate_test.go"
- test:
name: test-itest-self_sent_txn
suite: itest-self_sent_txn
target: "./itests/self_sent_txn_test.go"
- test:
name: test-itest-tape
suite: itest-tape
@ -956,7 +961,7 @@ workflows:
codecov-upload: false
suite: conformance-bleeding-edge
target: "./conformance"
vectors-branch: master
vectors-branch: specs-actors-v7
- trigger-testplans:
filters:
branches:

2
.circleci/template.yml
View File

@ -785,7 +785,7 @@ workflows:
codecov-upload: false
suite: conformance-bleeding-edge
target: "./conformance"
vectors-branch: master
vectors-branch: specs-actors-v7
- trigger-testplans:
filters:
branches:

21
.github/pull_request_template.md vendored Normal file
View File

@ -0,0 +1,21 @@
## Related Issues
<!-- link all issues that this PR might resolve/fix. If an issue doesn't exist, include a brief motivation for the change being made.-->
## Proposed Changes
<!-- provide a clear list of the changes being made-->
## Additional Info
<!-- callouts, links to documentation, and etc-->
## Checklist
Before you mark the PR ready for review, please make sure that:
- [ ] All commits have a clear commit message.
- [ ] The PR title is in the form of of `<PR type>: <area>: <change being made>`
- example: ` fix: mempool: Introduce a cache for valid signatures`
- `PR type`: _fix_, _feat_, _INTERFACE BREAKING CHANGE_, _CONSENSUS BREAKING_, _build_, _chore_, _ci_, _docs_, _misc_,_perf_, _refactor_, _revert_, _style_, _test_
- `area`: _api_, _chain_, _state_, _vm_, _data transfer_, _market_, _mempool_, _message_, _block production_, _multisig_, _networking_, _paychan_, _proving_, _sealing_, _wallet_
- [ ] This PR has tests for new functionality or change in behaviour
- [ ] If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in https://lotus.filecoin.io or [Discussion Tutorials.](https://github.com/filecoin-project/lotus/discussions/categories/tutorials)
- [ ] CI is green

102
CHANGELOG.md
View File

@ -1,5 +1,107 @@
# Lotus changelog
# v1.13.2 / 2022-01-09
Lotus v1.13.2 is a *highly recommended* feature release with remarkable retrieval improvements, new features like
worker management, schedule enhancements and so on.
## Highlights
- 🚀🚀🚀Improve retrieval deal experience
- Testing result with MinerX.3 shows the retrieval deal success rate has increased dramatically with faster transfer
speed, you can join or follow along furthur performance testings [here](https://github.com/filecoin-project/lotus/discussions/7874). We recommend application developers to integrate with the new
retrieval APIs to provide a better client experience.
- 🌟🌟🌟 Reduce retrieval Time-To-First-Byte over 100x ([#7693](https://github.com/filecoin-project/lotus/pull/7693))
- This change makes most free, small retrievals sub-second
- 🌟🌟🌟 Partial retrieval ux improvements ([#7610](https://github.com/filecoin-project/lotus/pull/7610))
- New retrieval commands for clients:
- `lotus client ls`: retrieve and list desired object links
- `lotus client cat`: retrieve and print the data from the network
- 🌟🌟 The monolith `ClientRetrieve` method was broken into:
- `ClientRetrieve` which retrieves data into the local repo (or into an IPFS node if ipfs integration is enabled)
- `ClientRetrieveWait` which will wait for the retrieval to complete
- `ClientExport` which will export data from the local node
- Note: this change only applies to v1 API. v0 API remains unchanged.
- 🌟 Support for full ipld selectors was added (for example making it possible to only retrieve list of directories in a deal, without fetching any file data)
- To learn more, see [here](https://github.com/filecoin-project/lotus/blob/0523c946f984b22b3f5de8cc3003cc791389527e/api/types.go#L230-L264)
- 🚀🚀 Sealing scheduler enhancements ([#7703](https://github.com/filecoin-project/lotus/pull/7703),
[#7269](https://github.com/filecoin-project/lotus/pull/7269)), [#7714](https://github.com/filecoin-project/lotus/pull/7714)
- Workers are now aware of cgroup memory limits
- Multiple tasks which use a GPU can be scheduled on a single worker
- Workers can override default resource table through env vars
- Default value list: https://gist.github.com/magik6k/c0e1c7cd73c1241a9acabc30bf469a43
- 🚀🚀 Sector storage groups ([#7453](https://github.com/filecoin-project/lotus/pull/7453))
- Storage groups allow for better control of data flow between workers, for example, it makes it possible to define that data from PC1 on a given worker has to have it's PC2 step executed on the same worker
- To set it up, follow the instructions under the `Sector Storage Group` section [here](https://lotus.filecoin.io/docs/storage-providers/seal-workers/#lotus-worker-co-location)
## New Features
- Add RLE dump code ([#7691](https://github.com/filecoin-project/lotus/pull/7691))
- Shed: Add a util to list miner faults ([#7605](https://github.com/filecoin-project/lotus/pull/7605))
- lotus-shed msg: Decode submessages/msig proposals ([#7639](https://github.com/filecoin-project/lotus/pull/7639))
- CLI: Add a lotus multisig cancel command ([#7645](https://github.com/filecoin-project/lotus/pull/7645))
- shed: simple wallet balancer util ([#7414](https://github.com/filecoin-project/lotus/pull/7414))
- balancing token balance between multiple accounts
## Improvements
- Add verbose mode to `lotus-miner pieces list-cids` ([#7699](https://github.com/filecoin-project/lotus/pull/7699))
- retrieval: Only output matching nodes, MatchPath dagspec ([#7706](https://github.com/filecoin-project/lotus/pull/7706))
- Cleanup partial retrieval codepaths ( zero functional changes ) ([#7688](https://github.com/filecoin-project/lotus/pull/7688))
- storage: Use 1M buffers for Tar transfers ([#7681](https://github.com/filecoin-project/lotus/pull/7681))
- Chore/dm level tests plus merkle proof cars ([#7673](https://github.com/filecoin-project/lotus/pull/7673))
- Shed: Add a util to create miners more easily ([#7595](https://github.com/filecoin-project/lotus/pull/7595))
- add timeout flag to wait-api command ([#7592](https://github.com/filecoin-project/lotus/pull/7592))
- add log for restart windows post scheduler ([#7613](https://github.com/filecoin-project/lotus/pull/7613))
- remove jaeger envvars ([#7631](https://github.com/filecoin-project/lotus/pull/7631))
- remove api and jaeger env from docker file ([#7624](https://github.com/filecoin-project/lotus/pull/7624))
- Wdpost worker: Reduce challenge confidence to 1 epoch ([#7572](https://github.com/filecoin-project/lotus/pull/7572))
- add additional methods to lotus gateway ([#7644](https://github.com/filecoin-project/lotus/pull/7644))
- Add caches to lotus-stats and splitcode ([#7329](https://github.com/filecoin-project/lotus/pull/7329))
- remote store: Remove debug printf ([#7664](https://github.com/filecoin-project/lotus/pull/7664))
- docsgen-cli: Handle commands with no description correctly ([#7659](https://github.com/filecoin-project/lotus/pull/7659))
## Bug Fixes
- fix docker logic error ([#7709](https://github.com/filecoin-project/lotus/pull/7709))
- add missing NodeType tag ([#7559](https://github.com/filecoin-project/lotus/pull/7559))
- checkCommit should return SectorCommitFailed ([#7555](https://github.com/filecoin-project/lotus/pull/7555))
- ffiwrapper: Validate PC2 by calling C1 with random seeds ([#7710](https://github.com/filecoin-project/lotus/pull/7710))
## Dependency Updates
- Update go-graphsync v0.10.6 ([#7708](https://github.com/filecoin-project/lotus/pull/7708))
- update go-libp2p-pubsub to v0.5.6 ([#7581](https://github.com/filecoin-project/lotus/pull/7581))
- Update go-state-types ([#7591](https://github.com/filecoin-project/lotus/pull/7591))
- disable mplex stream muxer ([#7689](https://github.com/filecoin-project/lotus/pull/7689))
- Bump ws from 5.2.2 to 5.2.3 in /lotuspond/front ([#7660](https://github.com/filecoin-project/lotus/pull/7660))
- Bump color-string from 1.5.3 to 1.6.0 in /lotuspond/front ([#7658](https://github.com/filecoin-project/lotus/pull/7658))
- Bump postcss from 7.0.17 to 7.0.39 in /lotuspond/front ([#7657](https://github.com/filecoin-project/lotus/pull/7657))
- Bump path-parse from 1.0.6 to 1.0.7 in /lotuspond/front ([#7656](https://github.com/filecoin-project/lotus/pull/7656))
- Bump tmpl from 1.0.4 to 1.0.5 in /lotuspond/front ([#7655](https://github.com/filecoin-project/lotus/pull/7655))
- Bump url-parse from 1.4.7 to 1.5.3 in /lotuspond/front ([#7654](https://github.com/filecoin-project/lotus/pull/7654))
- github.com/filecoin-project/go-state-types (v0.1.1-0.20210915140513-d354ccf10379 -> v0.1.1):
## Others
- Update archive script ([#7690](https://github.com/filecoin-project/lotus/pull/7690))
## Contributors
| Contributor | Commits | Lines ± | Files Changed |
|-------------|---------|---------|---------------|
| @magik6k | 89 | +5200/-1818 | 232 |
| Travis Person | 5 | +1473/-953 | 38 |
| @arajasek | 6 | +550/-38 | 19 |
| @clinta | 4 | +393/-123 | 26 |
| @ribasushi | 3 | +334/-68 | 7 |
| @jennijuju| 13 | +197/-120 | 67 |
| @Kubuxu | 10 | +153/-30 | 10 |
| @coryschwartz | 6 | +18/-26 | 6 |
| Marten Seemann | 2 | +6/-34 | 5 |
| @vyzo | 1 | +3/-3 | 2 |
| @hannahhoward | 1 | +3/-3 | 2 |
| @zenground0 | 2 | +2/-2 | 2 |
| @yaohcn | 2 | +2/-2 | 2 |
| @jennijuju | 1 | +1/-1 | 1 |
| @hunjixin | 1 | +1/-0 | 1 |
# v1.13.1 / 2021-11-26
This is an optional Lotus v1.13.1 release.

3
api/api_full.go
View File

@ -900,6 +900,7 @@ type QueryOffer struct {
Size uint64
MinPrice types.BigInt
UnsealPrice types.BigInt
PricePerByte abi.TokenAmount
PaymentInterval uint64
PaymentIntervalIncrease uint64
Miner address.Address
@ -1083,7 +1084,7 @@ type CirculatingSupply struct {
type MiningBaseInfo struct {
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []builtin.SectorInfo
Sectors []builtin.ExtendedSectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry

36
api/api_storage.go
View File

@ -14,6 +14,7 @@ import (
"github.com/filecoin-project/go-address"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/go-state-types/abi"
abinetwork "github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/specs-actors/v2/actors/builtin/market"
"github.com/filecoin-project/specs-storage/storage"
@ -99,8 +100,8 @@ type StorageMiner interface {
// Returns null if message wasn't sent
SectorTerminateFlush(ctx context.Context) (*cid.Cid, error) //perm:admin
// SectorTerminatePending returns a list of pending sector terminations to be sent in the next batch message
SectorTerminatePending(ctx context.Context) ([]abi.SectorID, error) //perm:admin
SectorMarkForUpgrade(ctx context.Context, id abi.SectorNumber) error //perm:admin
SectorTerminatePending(ctx context.Context) ([]abi.SectorID, error) //perm:admin
SectorMarkForUpgrade(ctx context.Context, id abi.SectorNumber, snap bool) error //perm:admin
// SectorPreCommitFlush immediately sends a PreCommit message with sectors batched for PreCommit.
// Returns null if message wasn't sent
SectorPreCommitFlush(ctx context.Context) ([]sealiface.PreCommitBatchRes, error) //perm:admin
@ -111,6 +112,7 @@ type StorageMiner interface {
SectorCommitFlush(ctx context.Context) ([]sealiface.CommitBatchRes, error) //perm:admin
// SectorCommitPending returns a list of pending Commit sectors to be sent in the next aggregate message
SectorCommitPending(ctx context.Context) ([]abi.SectorID, error) //perm:admin
SectorMatchPendingPiecesToOpenSectors(ctx context.Context) error //perm:admin
// WorkerConnect tells the node to connect to workers RPC
WorkerConnect(context.Context, string) error //perm:admin retry:true
@ -118,17 +120,21 @@ type StorageMiner interface {
WorkerJobs(context.Context) (map[uuid.UUID][]storiface.WorkerJob, error) //perm:admin
//storiface.WorkerReturn
ReturnAddPiece(ctx context.Context, callID storiface.CallID, pi abi.PieceInfo, err *storiface.CallError) error //perm:admin retry:true
ReturnSealPreCommit1(ctx context.Context, callID storiface.CallID, p1o storage.PreCommit1Out, err *storiface.CallError) error //perm:admin retry:true
ReturnSealPreCommit2(ctx context.Context, callID storiface.CallID, sealed storage.SectorCids, err *storiface.CallError) error //perm:admin retry:true
ReturnSealCommit1(ctx context.Context, callID storiface.CallID, out storage.Commit1Out, err *storiface.CallError) error //perm:admin retry:true
ReturnSealCommit2(ctx context.Context, callID storiface.CallID, proof storage.Proof, err *storiface.CallError) error //perm:admin retry:true
ReturnFinalizeSector(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnReleaseUnsealed(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnMoveStorage(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnUnsealPiece(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnReadPiece(ctx context.Context, callID storiface.CallID, ok bool, err *storiface.CallError) error //perm:admin retry:true
ReturnFetch(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnAddPiece(ctx context.Context, callID storiface.CallID, pi abi.PieceInfo, err *storiface.CallError) error //perm:admin retry:true
ReturnSealPreCommit1(ctx context.Context, callID storiface.CallID, p1o storage.PreCommit1Out, err *storiface.CallError) error //perm:admin retry:true
ReturnSealPreCommit2(ctx context.Context, callID storiface.CallID, sealed storage.SectorCids, err *storiface.CallError) error //perm:admin retry:true
ReturnSealCommit1(ctx context.Context, callID storiface.CallID, out storage.Commit1Out, err *storiface.CallError) error //perm:admin retry:true
ReturnSealCommit2(ctx context.Context, callID storiface.CallID, proof storage.Proof, err *storiface.CallError) error //perm:admin retry:true
ReturnFinalizeSector(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnReplicaUpdate(ctx context.Context, callID storiface.CallID, out storage.ReplicaUpdateOut, err *storiface.CallError) error //perm:admin retry:true
ReturnProveReplicaUpdate1(ctx context.Context, callID storiface.CallID, vanillaProofs storage.ReplicaVanillaProofs, err *storiface.CallError) error //perm:admin retry:true
ReturnProveReplicaUpdate2(ctx context.Context, callID storiface.CallID, proof storage.ReplicaUpdateProof, err *storiface.CallError) error //perm:admin retry:true
ReturnGenerateSectorKeyFromData(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnReleaseUnsealed(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnMoveStorage(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnUnsealPiece(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
ReturnReadPiece(ctx context.Context, callID storiface.CallID, ok bool, err *storiface.CallError) error //perm:admin retry:true
ReturnFetch(ctx context.Context, callID storiface.CallID, err *storiface.CallError) error //perm:admin retry:true
// SealingSchedDiag dumps internal sealing scheduler state
SealingSchedDiag(ctx context.Context, doSched bool) (interface{}, error) //perm:admin
@ -161,6 +167,8 @@ type StorageMiner interface {
MarketGetRetrievalAsk(ctx context.Context) (*retrievalmarket.Ask, error) //perm:read
MarketListDataTransfers(ctx context.Context) ([]DataTransferChannel, error) //perm:write
MarketDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error) //perm:write
// MarketDataTransferDiagnostics generates debugging information about current data transfers over graphsync
MarketDataTransferDiagnostics(ctx context.Context, p peer.ID) (*TransferDiagnostics, error) //perm:write
// MarketRestartDataTransfer attempts to restart a data transfer with the given transfer ID and other peer
MarketRestartDataTransfer(ctx context.Context, transferID datatransfer.TransferID, otherPeer peer.ID, isInitiator bool) error //perm:write
// MarketCancelDataTransfer cancels a data transfer with the given transfer ID and other peer
@ -247,7 +255,7 @@ type StorageMiner interface {
CheckProvable(ctx context.Context, pp abi.RegisteredPoStProof, sectors []storage.SectorRef, expensive bool) (map[abi.SectorNumber]string, error) //perm:admin
ComputeProof(ctx context.Context, ssi []builtin.SectorInfo, rand abi.PoStRandomness) ([]builtin.PoStProof, error) //perm:read
ComputeProof(ctx context.Context, ssi []builtin.ExtendedSectorInfo, rand abi.PoStRandomness, poStEpoch abi.ChainEpoch, nv abinetwork.Version) ([]builtin.PoStProof, error) //perm:read
}
var _ storiface.WorkerReturn = *new(StorageMiner)

4
api/api_worker.go
View File

@ -39,6 +39,10 @@ type Worker interface {
SealCommit1(ctx context.Context, sector storage.SectorRef, ticket abi.SealRandomness, seed abi.InteractiveSealRandomness, pieces []abi.PieceInfo, cids storage.SectorCids) (storiface.CallID, error) //perm:admin
SealCommit2(ctx context.Context, sector storage.SectorRef, c1o storage.Commit1Out) (storiface.CallID, error) //perm:admin
FinalizeSector(ctx context.Context, sector storage.SectorRef, keepUnsealed []storage.Range) (storiface.CallID, error) //perm:admin
ReplicaUpdate(ctx context.Context, sector storage.SectorRef, pieces []abi.PieceInfo) (storiface.CallID, error) //perm:admin
ProveReplicaUpdate1(ctx context.Context, sector storage.SectorRef, sectorKey, newSealed, newUnsealed cid.Cid) (storiface.CallID, error) //perm:admin
ProveReplicaUpdate2(ctx context.Context, sector storage.SectorRef, sectorKey, newSealed, newUnsealed cid.Cid, vanillaProofs storage.ReplicaVanillaProofs) (storiface.CallID, error) //perm:admin
GenerateSectorKeyFromData(ctx context.Context, sector storage.SectorRef, commD cid.Cid) (storiface.CallID, error) //perm:admin
ReleaseUnsealed(ctx context.Context, sector storage.SectorRef, safeToFree []storage.Range) (storiface.CallID, error) //perm:admin
MoveStorage(ctx context.Context, sector storage.SectorRef, types storiface.SectorFileType) (storiface.CallID, error) //perm:admin
UnsealPiece(context.Context, storage.SectorRef, storiface.UnpaddedByteIndex, abi.UnpaddedPieceSize, abi.SealRandomness, cid.Cid) (storiface.CallID, error) //perm:admin

6
api/docgen/docgen.go
View File

@ -1,6 +1,7 @@
package docgen
import (
"encoding/json"
"fmt"
"go/ast"
"go/parser"
@ -15,6 +16,7 @@ import (
"github.com/filecoin-project/go-bitfield"
"github.com/google/uuid"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-graphsync"
"github.com/libp2p/go-libp2p-core/metrics"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
@ -120,6 +122,7 @@ func init() {
addExample(api.FullAPIVersion1)
addExample(api.PCHInbound)
addExample(time.Minute)
addExample(graphsync.RequestID(4))
addExample(datatransfer.TransferID(3))
addExample(datatransfer.Ongoing)
addExample(storeIDExample)
@ -250,6 +253,7 @@ func init() {
addExample(map[abi.SectorNumber]string{
123: "can't acquire read lock",
})
addExample(json.RawMessage(`"json raw message"`))
addExample(map[api.SectorState]int{
api.SectorState(sealing.Proving): 120,
})
@ -346,7 +350,7 @@ func ExampleValue(method string, t, parent reflect.Type) interface{} {
switch t.Kind() {
case reflect.Slice:
out := reflect.New(t).Elem()
reflect.Append(out, reflect.ValueOf(ExampleValue(method, t.Elem(), t)))
out = reflect.Append(out, reflect.ValueOf(ExampleValue(method, t.Elem(), t)))
return out.Interface()
case reflect.Chan:
return ExampleValue(method, t.Elem(), nil)

147
api/proxy_gen.go
View File

@ -17,6 +17,7 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/go-state-types/dline"
abinetwork "github.com/filecoin-project/go-state-types/network"
apitypes "github.com/filecoin-project/lotus/api/types"
"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
@ -620,7 +621,7 @@ type StorageMinerStruct struct {
CheckProvable func(p0 context.Context, p1 abi.RegisteredPoStProof, p2 []storage.SectorRef, p3 bool) (map[abi.SectorNumber]string, error) `perm:"admin"`
ComputeProof func(p0 context.Context, p1 []builtin.SectorInfo, p2 abi.PoStRandomness) ([]builtin.PoStProof, error) `perm:"read"`
ComputeProof func(p0 context.Context, p1 []builtin.ExtendedSectorInfo, p2 abi.PoStRandomness, p3 abi.ChainEpoch, p4 abinetwork.Version) ([]builtin.PoStProof, error) `perm:"read"`
CreateBackup func(p0 context.Context, p1 string) error `perm:"admin"`
@ -668,6 +669,8 @@ type StorageMinerStruct struct {
MarketCancelDataTransfer func(p0 context.Context, p1 datatransfer.TransferID, p2 peer.ID, p3 bool) error `perm:"write"`
MarketDataTransferDiagnostics func(p0 context.Context, p1 peer.ID) (*TransferDiagnostics, error) `perm:"write"`
MarketDataTransferUpdates func(p0 context.Context) (<-chan DataTransferChannel, error) `perm:"write"`
MarketGetAsk func(p0 context.Context) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
@ -716,12 +719,20 @@ type StorageMinerStruct struct {
ReturnFinalizeSector func(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error `perm:"admin"`
ReturnGenerateSectorKeyFromData func(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error `perm:"admin"`
ReturnMoveStorage func(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error `perm:"admin"`
ReturnProveReplicaUpdate1 func(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaVanillaProofs, p3 *storiface.CallError) error `perm:"admin"`
ReturnProveReplicaUpdate2 func(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateProof, p3 *storiface.CallError) error `perm:"admin"`
ReturnReadPiece func(p0 context.Context, p1 storiface.CallID, p2 bool, p3 *storiface.CallError) error `perm:"admin"`
ReturnReleaseUnsealed func(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error `perm:"admin"`
ReturnReplicaUpdate func(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateOut, p3 *storiface.CallError) error `perm:"admin"`
ReturnSealCommit1 func(p0 context.Context, p1 storiface.CallID, p2 storage.Commit1Out, p3 *storiface.CallError) error `perm:"admin"`
ReturnSealCommit2 func(p0 context.Context, p1 storiface.CallID, p2 storage.Proof, p3 *storiface.CallError) error `perm:"admin"`
@ -748,7 +759,9 @@ type StorageMinerStruct struct {
SectorGetSealDelay func(p0 context.Context) (time.Duration, error) `perm:"read"`
SectorMarkForUpgrade func(p0 context.Context, p1 abi.SectorNumber) error `perm:"admin"`
SectorMarkForUpgrade func(p0 context.Context, p1 abi.SectorNumber, p2 bool) error `perm:"admin"`
SectorMatchPendingPiecesToOpenSectors func(p0 context.Context) error `perm:"admin"`
SectorPreCommitFlush func(p0 context.Context) ([]sealiface.PreCommitBatchRes, error) `perm:"admin"`
@ -855,6 +868,8 @@ type WorkerStruct struct {
FinalizeSector func(p0 context.Context, p1 storage.SectorRef, p2 []storage.Range) (storiface.CallID, error) `perm:"admin"`
GenerateSectorKeyFromData func(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid) (storiface.CallID, error) `perm:"admin"`
Info func(p0 context.Context) (storiface.WorkerInfo, error) `perm:"admin"`
MoveStorage func(p0 context.Context, p1 storage.SectorRef, p2 storiface.SectorFileType) (storiface.CallID, error) `perm:"admin"`
@ -863,10 +878,16 @@ type WorkerStruct struct {
ProcessSession func(p0 context.Context) (uuid.UUID, error) `perm:"admin"`
ProveReplicaUpdate1 func(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid) (storiface.CallID, error) `perm:"admin"`
ProveReplicaUpdate2 func(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid, p5 storage.ReplicaVanillaProofs) (storiface.CallID, error) `perm:"admin"`
ReleaseUnsealed func(p0 context.Context, p1 storage.SectorRef, p2 []storage.Range) (storiface.CallID, error) `perm:"admin"`
Remove func(p0 context.Context, p1 abi.SectorID) error `perm:"admin"`
ReplicaUpdate func(p0 context.Context, p1 storage.SectorRef, p2 []abi.PieceInfo) (storiface.CallID, error) `perm:"admin"`
SealCommit1 func(p0 context.Context, p1 storage.SectorRef, p2 abi.SealRandomness, p3 abi.InteractiveSealRandomness, p4 []abi.PieceInfo, p5 storage.SectorCids) (storiface.CallID, error) `perm:"admin"`
SealCommit2 func(p0 context.Context, p1 storage.SectorRef, p2 storage.Commit1Out) (storiface.CallID, error) `perm:"admin"`
@ -3692,14 +3713,14 @@ func (s *StorageMinerStub) CheckProvable(p0 context.Context, p1 abi.RegisteredPo
return *new(map[abi.SectorNumber]string), ErrNotSupported
}
func (s *StorageMinerStruct) ComputeProof(p0 context.Context, p1 []builtin.SectorInfo, p2 abi.PoStRandomness) ([]builtin.PoStProof, error) {
func (s *StorageMinerStruct) ComputeProof(p0 context.Context, p1 []builtin.ExtendedSectorInfo, p2 abi.PoStRandomness, p3 abi.ChainEpoch, p4 abinetwork.Version) ([]builtin.PoStProof, error) {
if s.Internal.ComputeProof == nil {
return *new([]builtin.PoStProof), ErrNotSupported
}
return s.Internal.ComputeProof(p0, p1, p2)
return s.Internal.ComputeProof(p0, p1, p2, p3, p4)
}
func (s *StorageMinerStub) ComputeProof(p0 context.Context, p1 []builtin.SectorInfo, p2 abi.PoStRandomness) ([]builtin.PoStProof, error) {
func (s *StorageMinerStub) ComputeProof(p0 context.Context, p1 []builtin.ExtendedSectorInfo, p2 abi.PoStRandomness, p3 abi.ChainEpoch, p4 abinetwork.Version) ([]builtin.PoStProof, error) {
return *new([]builtin.PoStProof), ErrNotSupported
}
@ -3956,6 +3977,17 @@ func (s *StorageMinerStub) MarketCancelDataTransfer(p0 context.Context, p1 datat
return ErrNotSupported
}
func (s *StorageMinerStruct) MarketDataTransferDiagnostics(p0 context.Context, p1 peer.ID) (*TransferDiagnostics, error) {
if s.Internal.MarketDataTransferDiagnostics == nil {
return nil, ErrNotSupported
}
return s.Internal.MarketDataTransferDiagnostics(p0, p1)
}
func (s *StorageMinerStub) MarketDataTransferDiagnostics(p0 context.Context, p1 peer.ID) (*TransferDiagnostics, error) {
return nil, ErrNotSupported
}
func (s *StorageMinerStruct) MarketDataTransferUpdates(p0 context.Context) (<-chan DataTransferChannel, error) {
if s.Internal.MarketDataTransferUpdates == nil {
return nil, ErrNotSupported
@ -4220,6 +4252,17 @@ func (s *StorageMinerStub) ReturnFinalizeSector(p0 context.Context, p1 storiface
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnGenerateSectorKeyFromData(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error {
if s.Internal.ReturnGenerateSectorKeyFromData == nil {
return ErrNotSupported
}
return s.Internal.ReturnGenerateSectorKeyFromData(p0, p1, p2)
}
func (s *StorageMinerStub) ReturnGenerateSectorKeyFromData(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error {
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnMoveStorage(p0 context.Context, p1 storiface.CallID, p2 *storiface.CallError) error {
if s.Internal.ReturnMoveStorage == nil {
return ErrNotSupported
@ -4231,6 +4274,28 @@ func (s *StorageMinerStub) ReturnMoveStorage(p0 context.Context, p1 storiface.Ca
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnProveReplicaUpdate1(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaVanillaProofs, p3 *storiface.CallError) error {
if s.Internal.ReturnProveReplicaUpdate1 == nil {
return ErrNotSupported
}
return s.Internal.ReturnProveReplicaUpdate1(p0, p1, p2, p3)
}
func (s *StorageMinerStub) ReturnProveReplicaUpdate1(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaVanillaProofs, p3 *storiface.CallError) error {
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnProveReplicaUpdate2(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateProof, p3 *storiface.CallError) error {
if s.Internal.ReturnProveReplicaUpdate2 == nil {
return ErrNotSupported
}
return s.Internal.ReturnProveReplicaUpdate2(p0, p1, p2, p3)
}
func (s *StorageMinerStub) ReturnProveReplicaUpdate2(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateProof, p3 *storiface.CallError) error {
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnReadPiece(p0 context.Context, p1 storiface.CallID, p2 bool, p3 *storiface.CallError) error {
if s.Internal.ReturnReadPiece == nil {
return ErrNotSupported
@ -4253,6 +4318,17 @@ func (s *StorageMinerStub) ReturnReleaseUnsealed(p0 context.Context, p1 storifac
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnReplicaUpdate(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateOut, p3 *storiface.CallError) error {
if s.Internal.ReturnReplicaUpdate == nil {
return ErrNotSupported
}
return s.Internal.ReturnReplicaUpdate(p0, p1, p2, p3)
}
func (s *StorageMinerStub) ReturnReplicaUpdate(p0 context.Context, p1 storiface.CallID, p2 storage.ReplicaUpdateOut, p3 *storiface.CallError) error {
return ErrNotSupported
}
func (s *StorageMinerStruct) ReturnSealCommit1(p0 context.Context, p1 storiface.CallID, p2 storage.Commit1Out, p3 *storiface.CallError) error {
if s.Internal.ReturnSealCommit1 == nil {
return ErrNotSupported
@ -4396,14 +4472,25 @@ func (s *StorageMinerStub) SectorGetSealDelay(p0 context.Context) (time.Duration
return *new(time.Duration), ErrNotSupported
}
func (s *StorageMinerStruct) SectorMarkForUpgrade(p0 context.Context, p1 abi.SectorNumber) error {
func (s *StorageMinerStruct) SectorMarkForUpgrade(p0 context.Context, p1 abi.SectorNumber, p2 bool) error {
if s.Internal.SectorMarkForUpgrade == nil {
return ErrNotSupported
}
return s.Internal.SectorMarkForUpgrade(p0, p1)
return s.Internal.SectorMarkForUpgrade(p0, p1, p2)
}
func (s *StorageMinerStub) SectorMarkForUpgrade(p0 context.Context, p1 abi.SectorNumber) error {
func (s *StorageMinerStub) SectorMarkForUpgrade(p0 context.Context, p1 abi.SectorNumber, p2 bool) error {
return ErrNotSupported
}
func (s *StorageMinerStruct) SectorMatchPendingPiecesToOpenSectors(p0 context.Context) error {
if s.Internal.SectorMatchPendingPiecesToOpenSectors == nil {
return ErrNotSupported
}
return s.Internal.SectorMatchPendingPiecesToOpenSectors(p0)
}
func (s *StorageMinerStub) SectorMatchPendingPiecesToOpenSectors(p0 context.Context) error {
return ErrNotSupported
}
@ -4891,6 +4978,17 @@ func (s *WorkerStub) FinalizeSector(p0 context.Context, p1 storage.SectorRef, p2
return *new(storiface.CallID), ErrNotSupported
}
func (s *WorkerStruct) GenerateSectorKeyFromData(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid) (storiface.CallID, error) {
if s.Internal.GenerateSectorKeyFromData == nil {
return *new(storiface.CallID), ErrNotSupported
}
return s.Internal.GenerateSectorKeyFromData(p0, p1, p2)
}
func (s *WorkerStub) GenerateSectorKeyFromData(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid) (storiface.CallID, error) {
return *new(storiface.CallID), ErrNotSupported
}
func (s *WorkerStruct) Info(p0 context.Context) (storiface.WorkerInfo, error) {
if s.Internal.Info == nil {
return *new(storiface.WorkerInfo), ErrNotSupported
@ -4935,6 +5033,28 @@ func (s *WorkerStub) ProcessSession(p0 context.Context) (uuid.UUID, error) {
return *new(uuid.UUID), ErrNotSupported
}
func (s *WorkerStruct) ProveReplicaUpdate1(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid) (storiface.CallID, error) {
if s.Internal.ProveReplicaUpdate1 == nil {
return *new(storiface.CallID), ErrNotSupported
}
return s.Internal.ProveReplicaUpdate1(p0, p1, p2, p3, p4)
}
func (s *WorkerStub) ProveReplicaUpdate1(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid) (storiface.CallID, error) {
return *new(storiface.CallID), ErrNotSupported
}
func (s *WorkerStruct) ProveReplicaUpdate2(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid, p5 storage.ReplicaVanillaProofs) (storiface.CallID, error) {
if s.Internal.ProveReplicaUpdate2 == nil {
return *new(storiface.CallID), ErrNotSupported
}
return s.Internal.ProveReplicaUpdate2(p0, p1, p2, p3, p4, p5)
}
func (s *WorkerStub) ProveReplicaUpdate2(p0 context.Context, p1 storage.SectorRef, p2 cid.Cid, p3 cid.Cid, p4 cid.Cid, p5 storage.ReplicaVanillaProofs) (storiface.CallID, error) {
return *new(storiface.CallID), ErrNotSupported
}
func (s *WorkerStruct) ReleaseUnsealed(p0 context.Context, p1 storage.SectorRef, p2 []storage.Range) (storiface.CallID, error) {
if s.Internal.ReleaseUnsealed == nil {
return *new(storiface.CallID), ErrNotSupported
@ -4957,6 +5077,17 @@ func (s *WorkerStub) Remove(p0 context.Context, p1 abi.SectorID) error {
return ErrNotSupported
}
func (s *WorkerStruct) ReplicaUpdate(p0 context.Context, p1 storage.SectorRef, p2 []abi.PieceInfo) (storiface.CallID, error) {
if s.Internal.ReplicaUpdate == nil {
return *new(storiface.CallID), ErrNotSupported
}
return s.Internal.ReplicaUpdate(p0, p1, p2)
}
func (s *WorkerStub) ReplicaUpdate(p0 context.Context, p1 storage.SectorRef, p2 []abi.PieceInfo) (storiface.CallID, error) {
return *new(storiface.CallID), ErrNotSupported
}
func (s *WorkerStruct) SealCommit1(p0 context.Context, p1 storage.SectorRef, p2 abi.SealRandomness, p3 abi.InteractiveSealRandomness, p4 []abi.PieceInfo, p5 storage.SectorCids) (storiface.CallID, error) {
if s.Internal.SealCommit1 == nil {
return *new(storiface.CallID), ErrNotSupported

25
api/types.go
View File

@ -10,6 +10,7 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-graphsync"
"github.com/libp2p/go-libp2p-core/peer"
pubsub "github.com/libp2p/go-libp2p-pubsub"
@ -53,6 +54,30 @@ type MessageSendSpec struct {
MaxFee abi.TokenAmount
}
// GraphSyncDataTransfer provides diagnostics on a data transfer happening over graphsync
type GraphSyncDataTransfer struct {
// GraphSync request id for this transfer
RequestID graphsync.RequestID
// Graphsync state for this transfer
RequestState string
// If a channel ID is present, indicates whether this is the current graphsync request for this channel
// (could have changed in a restart)
IsCurrentChannelRequest bool
// Data transfer channel ID for this transfer
ChannelID *datatransfer.ChannelID
// Data transfer state for this transfer
ChannelState *DataTransferChannel
// Diagnostic information about this request -- and unexpected inconsistencies in
// request state
Diagnostics []string
}
// TransferDiagnostics give current information about transfers going over graphsync that may be helpful for debugging
type TransferDiagnostics struct {
ReceivingTransfers []*GraphSyncDataTransfer
SendingTransfers []*GraphSyncDataTransfer
}
type DataTransferChannel struct {
TransferID datatransfer.TransferID
Status datatransfer.Status

4
api/v0api/gateway.go
View File

@ -8,7 +8,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/dline"
"github.com/filecoin-project/go-state-types/network"
abinetwork "github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
@ -57,7 +57,7 @@ type Gateway interface {
StateMinerInfo(ctx context.Context, actor address.Address, tsk types.TipSetKey) (miner.MinerInfo, error)
StateMinerProvingDeadline(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*dline.Info, error)
StateMinerPower(context.Context, address.Address, types.TipSetKey) (*api.MinerPower, error)
StateNetworkVersion(context.Context, types.TipSetKey) (network.Version, error)
StateNetworkVersion(context.Context, types.TipSetKey) (abinetwork.Version, error)
StateSearchMsg(ctx context.Context, msg cid.Cid) (*api.MsgLookup, error)
StateSectorGetInfo(ctx context.Context, maddr address.Address, n abi.SectorNumber, tsk types.TipSetKey) (*miner.SectorOnChainInfo, error)
StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)

12
api/v0api/proxy_gen.go
View File

@ -13,7 +13,7 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/go-state-types/dline"
"github.com/filecoin-project/go-state-types/network"
abinetwork "github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/api"
apitypes "github.com/filecoin-project/lotus/api/types"
"github.com/filecoin-project/lotus/chain/actors/builtin/miner"
@ -451,7 +451,7 @@ type GatewayStruct struct {
StateMinerProvingDeadline func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*dline.Info, error) ``
StateNetworkVersion func(p0 context.Context, p1 types.TipSetKey) (network.Version, error) ``
StateNetworkVersion func(p0 context.Context, p1 types.TipSetKey) (abinetwork.Version, error) ``
StateSearchMsg func(p0 context.Context, p1 cid.Cid) (*api.MsgLookup, error) ``
@ -2703,15 +2703,15 @@ func (s *GatewayStub) StateMinerProvingDeadline(p0 context.Context, p1 address.A
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (network.Version, error) {
func (s *GatewayStruct) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (abinetwork.Version, error) {
if s.Internal.StateNetworkVersion == nil {
return *new(network.Version), ErrNotSupported
return *new(abinetwork.Version), ErrNotSupported
}
return s.Internal.StateNetworkVersion(p0, p1)
}
func (s *GatewayStub) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (network.Version, error) {
return *new(network.Version), ErrNotSupported
func (s *GatewayStub) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (abinetwork.Version, error) {
return *new(abinetwork.Version), ErrNotSupported
}
func (s *GatewayStruct) StateSearchMsg(p0 context.Context, p1 cid.Cid) (*api.MsgLookup, error) {

6
api/version.go
View File

@ -54,10 +54,10 @@ func VersionForType(nodeType NodeType) (Version, error) {
// semver versions of the rpc api exposed
var (
FullAPIVersion0 = newVer(1, 4, 0)
FullAPIVersion1 = newVer(2, 1, 0)
FullAPIVersion0 = newVer(1, 5, 0)
FullAPIVersion1 = newVer(2, 2, 0)
MinerAPIVersion0 = newVer(1, 2, 0)
MinerAPIVersion0 = newVer(1, 3, 0)
WorkerAPIVersion0 = newVer(1, 5, 0)
)

18
blockstore/api.go
View File

@ -25,35 +25,35 @@ func NewAPIBlockstore(cio ChainIO) Blockstore {
return Adapt(bs) // return an adapted blockstore.
}
func (a *apiBlockstore) DeleteBlock(cid.Cid) error {
func (a *apiBlockstore) DeleteBlock(context.Context, cid.Cid) error {
return xerrors.New("not supported")
}
func (a *apiBlockstore) Has(c cid.Cid) (bool, error) {
return a.api.ChainHasObj(context.TODO(), c)
func (a *apiBlockstore) Has(ctx context.Context, c cid.Cid) (bool, error) {
return a.api.ChainHasObj(ctx, c)
}
func (a *apiBlockstore) Get(c cid.Cid) (blocks.Block, error) {
bb, err := a.api.ChainReadObj(context.TODO(), c)
func (a *apiBlockstore) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {
bb, err := a.api.ChainReadObj(ctx, c)
if err != nil {
return nil, err
}
return blocks.NewBlockWithCid(bb, c)
}
func (a *apiBlockstore) GetSize(c cid.Cid) (int, error) {
bb, err := a.api.ChainReadObj(context.TODO(), c)
func (a *apiBlockstore) GetSize(ctx context.Context, c cid.Cid) (int, error) {
bb, err := a.api.ChainReadObj(ctx, c)
if err != nil {
return 0, err
}
return len(bb), nil
}
func (a *apiBlockstore) Put(blocks.Block) error {
func (a *apiBlockstore) Put(context.Context, blocks.Block) error {
return xerrors.New("not supported")
}
func (a *apiBlockstore) PutMany([]blocks.Block) error {
func (a *apiBlockstore) PutMany(context.Context, []blocks.Block) error {
return xerrors.New("not supported")
}

262
blockstore/autobatch.go Normal file
View File

@ -0,0 +1,262 @@
package blockstore
import (
"context"
"sync"
"time"
"golang.org/x/xerrors"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
)
// autolog is a logger for the autobatching blockstore. It is subscoped from the
// blockstore logger.
var autolog = log.Named("auto")
// contains the same set of blocks twice, once as an ordered list for flushing, and as a map for fast access
type blockBatch struct {
blockList []block.Block
blockMap map[cid.Cid]block.Block
}
type AutobatchBlockstore struct {
// TODO: drop if memory consumption is too high
addedCids map[cid.Cid]struct{}
stateLock sync.Mutex
bufferedBatch blockBatch
flushingBatch blockBatch
flushErr error
flushCh chan struct{}
doFlushLock sync.Mutex
flushRetryDelay time.Duration
doneCh chan struct{}
shutdown context.CancelFunc
backingBs Blockstore
bufferCapacity int
bufferSize int
}
func NewAutobatch(ctx context.Context, backingBs Blockstore, bufferCapacity int) *AutobatchBlockstore {
ctx, cancel := context.WithCancel(ctx)
bs := &AutobatchBlockstore{
addedCids: make(map[cid.Cid]struct{}),
backingBs: backingBs,
bufferCapacity: bufferCapacity,
flushCh: make(chan struct{}, 1),
doneCh: make(chan struct{}),
// could be made configable
flushRetryDelay: time.Millisecond * 100,
shutdown: cancel,
}
bs.bufferedBatch.blockMap = make(map[cid.Cid]block.Block)
go bs.flushWorker(ctx)
return bs
}
func (bs *AutobatchBlockstore) Put(ctx context.Context, blk block.Block) error {
bs.stateLock.Lock()
defer bs.stateLock.Unlock()
_, ok := bs.addedCids[blk.Cid()]
if !ok {
bs.addedCids[blk.Cid()] = struct{}{}
bs.bufferedBatch.blockList = append(bs.bufferedBatch.blockList, blk)
bs.bufferedBatch.blockMap[blk.Cid()] = blk
bs.bufferSize += len(blk.RawData())
if bs.bufferSize >= bs.bufferCapacity {
// signal that a flush is appropriate, may be ignored
select {
case bs.flushCh <- struct{}{}:
default:
// do nothing
}
}
}
return nil
}
func (bs *AutobatchBlockstore) flushWorker(ctx context.Context) {
defer close(bs.doneCh)
for {
select {
case <-bs.flushCh:
// TODO: check if we _should_ actually flush. We could get a spurious wakeup
// here.
putErr := bs.doFlush(ctx, false)
for putErr != nil {
select {
case <-ctx.Done():
return
case <-time.After(bs.flushRetryDelay):
autolog.Errorf("FLUSH ERRORED: %w, retrying after %v", putErr, bs.flushRetryDelay)
putErr = bs.doFlush(ctx, true)
}
}
case <-ctx.Done():
// Do one last flush.
_ = bs.doFlush(ctx, false)
return
}
}
}
// caller must NOT hold stateLock
// set retryOnly to true to only retry a failed flush and not flush anything new.
func (bs *AutobatchBlockstore) doFlush(ctx context.Context, retryOnly bool) error {
bs.doFlushLock.Lock()
defer bs.doFlushLock.Unlock()
// If we failed to flush last time, try flushing again.
if bs.flushErr != nil {
bs.flushErr = bs.backingBs.PutMany(ctx, bs.flushingBatch.blockList)
}
// If we failed, or we're _only_ retrying, bail.
if retryOnly || bs.flushErr != nil {
return bs.flushErr
}
// Then take the current batch...
bs.stateLock.Lock()
// We do NOT clear addedCids here, because its purpose is to expedite Puts
bs.flushingBatch = bs.bufferedBatch
bs.bufferedBatch.blockList = make([]block.Block, 0, len(bs.flushingBatch.blockList))
bs.bufferedBatch.blockMap = make(map[cid.Cid]block.Block, len(bs.flushingBatch.blockMap))
bs.stateLock.Unlock()
// And try to flush it.
bs.flushErr = bs.backingBs.PutMany(ctx, bs.flushingBatch.blockList)
// If we succeeded, reset the batch. Otherwise, we'll try again next time.
if bs.flushErr == nil {
bs.stateLock.Lock()
bs.flushingBatch = blockBatch{}
bs.stateLock.Unlock()
}
return bs.flushErr
}
// caller must NOT hold stateLock
func (bs *AutobatchBlockstore) Flush(ctx context.Context) error {
return bs.doFlush(ctx, false)
}
func (bs *AutobatchBlockstore) Shutdown(ctx context.Context) error {
// TODO: Prevent puts after we call this to avoid losing data.
bs.shutdown()
select {
case <-bs.doneCh:
case <-ctx.Done():
return ctx.Err()
}
bs.doFlushLock.Lock()
defer bs.doFlushLock.Unlock()
return bs.flushErr
}
func (bs *AutobatchBlockstore) Get(ctx context.Context, c cid.Cid) (block.Block, error) {
// may seem backward to check the backingBs first, but that is the likeliest case
blk, err := bs.backingBs.Get(ctx, c)
if err == nil {
return blk, nil
}
if err != ErrNotFound {
return blk, err
}
bs.stateLock.Lock()
defer bs.stateLock.Unlock()
v, ok := bs.flushingBatch.blockMap[c]
if ok {
return v, nil
}
v, ok = bs.bufferedBatch.blockMap[c]
if ok {
return v, nil
}
return bs.Get(ctx, c)
}
func (bs *AutobatchBlockstore) DeleteBlock(context.Context, cid.Cid) error {
// if we wanted to support this, we would have to:
// - flush
// - delete from the backingBs (if present)
// - remove from addedCids (if present)
// - if present in addedCids, also walk the ordered lists and remove if present
return xerrors.New("deletion is unsupported")
}
func (bs *AutobatchBlockstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
// see note in DeleteBlock()
return xerrors.New("deletion is unsupported")
}
func (bs *AutobatchBlockstore) Has(ctx context.Context, c cid.Cid) (bool, error) {
_, err := bs.Get(ctx, c)
if err == nil {
return true, nil
}
if err == ErrNotFound {
return false, nil
}
return false, err
}
func (bs *AutobatchBlockstore) GetSize(ctx context.Context, c cid.Cid) (int, error) {
blk, err := bs.Get(ctx, c)
if err != nil {
return 0, err
}
return len(blk.RawData()), nil
}
func (bs *AutobatchBlockstore) PutMany(ctx context.Context, blks []block.Block) error {
for _, blk := range blks {
if err := bs.Put(ctx, blk); err != nil {
return err
}
}
return nil
}
func (bs *AutobatchBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {
if err := bs.Flush(ctx); err != nil {
return nil, err
}
return bs.backingBs.AllKeysChan(ctx)
}
func (bs *AutobatchBlockstore) HashOnRead(enabled bool) {
bs.backingBs.HashOnRead(enabled)
}
func (bs *AutobatchBlockstore) View(ctx context.Context, cid cid.Cid, callback func([]byte) error) error {
blk, err := bs.Get(ctx, cid)
if err != nil {
return err
}
return callback(blk.RawData())
}

34
blockstore/autobatch_test.go Normal file
View File

@ -0,0 +1,34 @@
package blockstore
import (
"context"
"testing"
"github.com/stretchr/testify/require"
)
func TestAutobatchBlockstore(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
ab := NewAutobatch(ctx, NewMemory(), len(b0.RawData())+len(b1.RawData())-1)
require.NoError(t, ab.Put(ctx, b0))
require.NoError(t, ab.Put(ctx, b1))
require.NoError(t, ab.Put(ctx, b2))
v0, err := ab.Get(ctx, b0.Cid())
require.NoError(t, err)
require.Equal(t, b0.RawData(), v0.RawData())
v1, err := ab.Get(ctx, b1.Cid())
require.NoError(t, err)
require.Equal(t, b1.RawData(), v1.RawData())
v2, err := ab.Get(ctx, b2.Cid())
require.NoError(t, err)
require.Equal(t, b2.RawData(), v2.RawData())
require.NoError(t, ab.Flush(ctx))
require.NoError(t, ab.Shutdown(ctx))
}

16
blockstore/badger/blockstore.go
View File

@ -525,7 +525,7 @@ func (b *Blockstore) Size() (int64, error) {
// View implements blockstore.Viewer, which leverages zero-copy read-only
// access to values.
func (b *Blockstore) View(cid cid.Cid, fn func([]byte) error) error {
func (b *Blockstore) View(ctx context.Context, cid cid.Cid, fn func([]byte) error) error {
if err := b.access(); err != nil {
return err
}
@ -552,7 +552,7 @@ func (b *Blockstore) View(cid cid.Cid, fn func([]byte) error) error {
}
// Has implements Blockstore.Has.
func (b *Blockstore) Has(cid cid.Cid) (bool, error) {
func (b *Blockstore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
if err := b.access(); err != nil {
return false, err
}
@ -582,7 +582,7 @@ func (b *Blockstore) Has(cid cid.Cid) (bool, error) {
}
// Get implements Blockstore.Get.
func (b *Blockstore) Get(cid cid.Cid) (blocks.Block, error) {
func (b *Blockstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error) {
if !cid.Defined() {
return nil, blockstore.ErrNotFound
}
@ -619,7 +619,7 @@ func (b *Blockstore) Get(cid cid.Cid) (blocks.Block, error) {
}
// GetSize implements Blockstore.GetSize.
func (b *Blockstore) GetSize(cid cid.Cid) (int, error) {
func (b *Blockstore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
if err := b.access(); err != nil {
return 0, err
}
@ -652,7 +652,7 @@ func (b *Blockstore) GetSize(cid cid.Cid) (int, error) {
}
// Put implements Blockstore.Put.
func (b *Blockstore) Put(block blocks.Block) error {
func (b *Blockstore) Put(ctx context.Context, block blocks.Block) error {
if err := b.access(); err != nil {
return err
}
@ -691,7 +691,7 @@ func (b *Blockstore) Put(block blocks.Block) error {
}
// PutMany implements Blockstore.PutMany.
func (b *Blockstore) PutMany(blocks []blocks.Block) error {
func (b *Blockstore) PutMany(ctx context.Context, blocks []blocks.Block) error {
if err := b.access(); err != nil {
return err
}
@ -755,7 +755,7 @@ func (b *Blockstore) PutMany(blocks []blocks.Block) error {
}
// DeleteBlock implements Blockstore.DeleteBlock.
func (b *Blockstore) DeleteBlock(cid cid.Cid) error {
func (b *Blockstore) DeleteBlock(ctx context.Context, cid cid.Cid) error {
if err := b.access(); err != nil {
return err
}
@ -774,7 +774,7 @@ func (b *Blockstore) DeleteBlock(cid cid.Cid) error {
})
}
func (b *Blockstore) DeleteMany(cids []cid.Cid) error {
func (b *Blockstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
if err := b.access(); err != nil {
return err
}

14
blockstore/badger/blockstore_test.go
View File

@ -2,6 +2,7 @@ package badgerbs
import (
"bytes"
"context"
"fmt"
"io/ioutil"
"os"
@ -98,6 +99,7 @@ func openBlockstore(optsSupplier func(path string) Options) func(tb testing.TB,
}
func testMove(t *testing.T, optsF func(string) Options) {
ctx := context.Background()
basePath, err := ioutil.TempDir("", "")
if err != nil {
t.Fatal(err)
@ -122,7 +124,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
// add some blocks
for i := 0; i < 10; i++ {
blk := blocks.NewBlock([]byte(fmt.Sprintf("some data %d", i)))
err := db.Put(blk)
err := db.Put(ctx, blk)
if err != nil {
t.Fatal(err)
}
@ -132,7 +134,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
// delete some of them
for i := 5; i < 10; i++ {
c := have[i].Cid()
err := db.DeleteBlock(c)
err := db.DeleteBlock(ctx, c)
if err != nil {
t.Fatal(err)
}
@ -145,7 +147,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
g.Go(func() error {
for i := 10; i < 1000; i++ {
blk := blocks.NewBlock([]byte(fmt.Sprintf("some data %d", i)))
err := db.Put(blk)
err := db.Put(ctx, blk)
if err != nil {
return err
}
@ -165,7 +167,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
// now check that we have all the blocks in have and none in the deleted lists
checkBlocks := func() {
for _, blk := range have {
has, err := db.Has(blk.Cid())
has, err := db.Has(ctx, blk.Cid())
if err != nil {
t.Fatal(err)
}
@ -174,7 +176,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
t.Fatal("missing block")
}
blk2, err := db.Get(blk.Cid())
blk2, err := db.Get(ctx, blk.Cid())
if err != nil {
t.Fatal(err)
}
@ -185,7 +187,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
}
for _, c := range deleted {
has, err := db.Has(c)
has, err := db.Has(ctx, c)
if err != nil {
t.Fatal(err)
}

58
blockstore/badger/blockstore_test_suite.go
View File

@ -44,28 +44,31 @@ func (s *Suite) RunTests(t *testing.T, prefix string) {
}
func (s *Suite) TestGetWhenKeyNotPresent(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
}
c := cid.NewCidV0(u.Hash([]byte("stuff")))
bl, err := bs.Get(c)
bl, err := bs.Get(ctx, c)
require.Nil(t, bl)
require.Equal(t, blockstore.ErrNotFound, err)
}
func (s *Suite) TestGetWhenKeyIsNil(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
}
_, err := bs.Get(cid.Undef)
_, err := bs.Get(ctx, cid.Undef)
require.Equal(t, blockstore.ErrNotFound, err)
}
func (s *Suite) TestPutThenGetBlock(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -73,15 +76,16 @@ func (s *Suite) TestPutThenGetBlock(t *testing.T) {
orig := blocks.NewBlock([]byte("some data"))
err := bs.Put(orig)
err := bs.Put(ctx, orig)
require.NoError(t, err)
fetched, err := bs.Get(orig.Cid())
fetched, err := bs.Get(ctx, orig.Cid())
require.NoError(t, err)
require.Equal(t, orig.RawData(), fetched.RawData())
}
func (s *Suite) TestHas(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -89,19 +93,20 @@ func (s *Suite) TestHas(t *testing.T) {
orig := blocks.NewBlock([]byte("some data"))
err := bs.Put(orig)
err := bs.Put(ctx, orig)
require.NoError(t, err)
ok, err := bs.Has(orig.Cid())
ok, err := bs.Has(ctx, orig.Cid())
require.NoError(t, err)
require.True(t, ok)
ok, err = bs.Has(blocks.NewBlock([]byte("another thing")).Cid())
ok, err = bs.Has(ctx, blocks.NewBlock([]byte("another thing")).Cid())
require.NoError(t, err)
require.False(t, ok)
}
func (s *Suite) TestCidv0v1(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -109,15 +114,17 @@ func (s *Suite) TestCidv0v1(t *testing.T) {
orig := blocks.NewBlock([]byte("some data"))
err := bs.Put(orig)
err := bs.Put(ctx, orig)
require.NoError(t, err)
fetched, err := bs.Get(cid.NewCidV1(cid.DagProtobuf, orig.Cid().Hash()))
fetched, err := bs.Get(ctx, cid.NewCidV1(cid.DagProtobuf, orig.Cid().Hash()))
require.NoError(t, err)
require.Equal(t, orig.RawData(), fetched.RawData())
}
func (s *Suite) TestPutThenGetSizeBlock(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -127,21 +134,21 @@ func (s *Suite) TestPutThenGetSizeBlock(t *testing.T) {
missingBlock := blocks.NewBlock([]byte("missingBlock"))
emptyBlock := blocks.NewBlock([]byte{})
err := bs.Put(block)
err := bs.Put(ctx, block)
require.NoError(t, err)
blockSize, err := bs.GetSize(block.Cid())
blockSize, err := bs.GetSize(ctx, block.Cid())
require.NoError(t, err)
require.Len(t, block.RawData(), blockSize)
err = bs.Put(emptyBlock)
err = bs.Put(ctx, emptyBlock)
require.NoError(t, err)
emptySize, err := bs.GetSize(emptyBlock.Cid())
emptySize, err := bs.GetSize(ctx, emptyBlock.Cid())
require.NoError(t, err)
require.Zero(t, emptySize)
missingSize, err := bs.GetSize(missingBlock.Cid())
missingSize, err := bs.GetSize(ctx, missingBlock.Cid())
require.Equal(t, blockstore.ErrNotFound, err)
require.Equal(t, -1, missingSize)
}
@ -203,6 +210,7 @@ func (s *Suite) TestDoubleClose(t *testing.T) {
}
func (s *Suite) TestReopenPutGet(t *testing.T) {
ctx := context.Background()
bs, path := s.NewBlockstore(t)
c, ok := bs.(io.Closer)
if !ok {
@ -210,7 +218,7 @@ func (s *Suite) TestReopenPutGet(t *testing.T) {
}
orig := blocks.NewBlock([]byte("some data"))
err := bs.Put(orig)
err := bs.Put(ctx, orig)
require.NoError(t, err)
err = c.Close()
@ -219,7 +227,7 @@ func (s *Suite) TestReopenPutGet(t *testing.T) {
bs, err = s.OpenBlockstore(t, path)
require.NoError(t, err)
fetched, err := bs.Get(orig.Cid())
fetched, err := bs.Get(ctx, orig.Cid())
require.NoError(t, err)
require.Equal(t, orig.RawData(), fetched.RawData())
@ -228,6 +236,7 @@ func (s *Suite) TestReopenPutGet(t *testing.T) {
}
func (s *Suite) TestPutMany(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -238,15 +247,15 @@ func (s *Suite) TestPutMany(t *testing.T) {
blocks.NewBlock([]byte("foo2")),
blocks.NewBlock([]byte("foo3")),
}
err := bs.PutMany(blks)
err := bs.PutMany(ctx, blks)
require.NoError(t, err)
for _, blk := range blks {
fetched, err := bs.Get(blk.Cid())
fetched, err := bs.Get(ctx, blk.Cid())
require.NoError(t, err)
require.Equal(t, blk.RawData(), fetched.RawData())
ok, err := bs.Has(blk.Cid())
ok, err := bs.Has(ctx, blk.Cid())
require.NoError(t, err)
require.True(t, ok)
}
@ -259,6 +268,7 @@ func (s *Suite) TestPutMany(t *testing.T) {
}
func (s *Suite) TestDelete(t *testing.T) {
ctx := context.Background()
bs, _ := s.NewBlockstore(t)
if c, ok := bs.(io.Closer); ok {
defer func() { require.NoError(t, c.Close()) }()
@ -269,10 +279,10 @@ func (s *Suite) TestDelete(t *testing.T) {
blocks.NewBlock([]byte("foo2")),
blocks.NewBlock([]byte("foo3")),
}
err := bs.PutMany(blks)
err := bs.PutMany(ctx, blks)
require.NoError(t, err)
err = bs.DeleteBlock(blks[1].Cid())
err = bs.DeleteBlock(ctx, blks[1].Cid())
require.NoError(t, err)
ch, err := bs.AllKeysChan(context.Background())
@ -285,17 +295,17 @@ func (s *Suite) TestDelete(t *testing.T) {
cid.NewCidV1(cid.Raw, blks[2].Cid().Hash()),
})
has, err := bs.Has(blks[1].Cid())
has, err := bs.Has(ctx, blks[1].Cid())
require.NoError(t, err)
require.False(t, has)
}
func insertBlocks(t *testing.T, bs blockstore.BasicBlockstore, count int) []cid.Cid {
ctx := context.Background()
keys := make([]cid.Cid, count)
for i := 0; i < count; i++ {
block := blocks.NewBlock([]byte(fmt.Sprintf("some data %d", i)))
err := bs.Put(block)
err := bs.Put(ctx, block)
require.NoError(t, err)
// NewBlock assigns a CIDv0; we convert it to CIDv1 because that's what
// the store returns.

12
blockstore/blockstore.go
View File

@ -1,6 +1,8 @@
package blockstore
import (
"context"
cid "github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
logging "github.com/ipfs/go-log/v2"
@ -27,7 +29,7 @@ type BasicBlockstore = blockstore.Blockstore
type Viewer = blockstore.Viewer
type BatchDeleter interface {
DeleteMany(cids []cid.Cid) error
DeleteMany(ctx context.Context, cids []cid.Cid) error
}
// BlockstoreIterator is a trait for efficient iteration
@ -93,17 +95,17 @@ type adaptedBlockstore struct {
var _ Blockstore = (*adaptedBlockstore)(nil)
func (a *adaptedBlockstore) View(cid cid.Cid, callback func([]byte) error) error {
blk, err := a.Get(cid)
func (a *adaptedBlockstore) View(ctx context.Context, cid cid.Cid, callback func([]byte) error) error {
blk, err := a.Get(ctx, cid)
if err != nil {
return err
}
return callback(blk.RawData())
}
func (a *adaptedBlockstore) DeleteMany(cids []cid.Cid) error {
func (a *adaptedBlockstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
for _, cid := range cids {
err := a.DeleteBlock(cid)
err := a.DeleteBlock(ctx, cid)
if err != nil {
return err
}

46
blockstore/buffered.go
View File

@ -88,34 +88,34 @@ func (bs *BufferedBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid,
return out, nil
}
func (bs *BufferedBlockstore) DeleteBlock(c cid.Cid) error {
if err := bs.read.DeleteBlock(c); err != nil {
func (bs *BufferedBlockstore) DeleteBlock(ctx context.Context, c cid.Cid) error {
if err := bs.read.DeleteBlock(ctx, c); err != nil {
return err
}
return bs.write.DeleteBlock(c)
return bs.write.DeleteBlock(ctx, c)
}
func (bs *BufferedBlockstore) DeleteMany(cids []cid.Cid) error {
if err := bs.read.DeleteMany(cids); err != nil {
func (bs *BufferedBlockstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
if err := bs.read.DeleteMany(ctx, cids); err != nil {
return err
}
return bs.write.DeleteMany(cids)
return bs.write.DeleteMany(ctx, cids)
}
func (bs *BufferedBlockstore) View(c cid.Cid, callback func([]byte) error) error {
func (bs *BufferedBlockstore) View(ctx context.Context, c cid.Cid, callback func([]byte) error) error {
// both stores are viewable.
if err := bs.write.View(c, callback); err == ErrNotFound {
if err := bs.write.View(ctx, c, callback); err == ErrNotFound {
// not found in write blockstore; fall through.
} else {
return err // propagate errors, or nil, i.e. found.
}
return bs.read.View(c, callback)
return bs.read.View(ctx, c, callback)
}
func (bs *BufferedBlockstore) Get(c cid.Cid) (block.Block, error) {
if out, err := bs.write.Get(c); err != nil {
func (bs *BufferedBlockstore) Get(ctx context.Context, c cid.Cid) (block.Block, error) {
if out, err := bs.write.Get(ctx, c); err != nil {
if err != ErrNotFound {
return nil, err
}
@ -123,20 +123,20 @@ func (bs *BufferedBlockstore) Get(c cid.Cid) (block.Block, error) {
return out, nil
}
return bs.read.Get(c)
return bs.read.Get(ctx, c)
}
func (bs *BufferedBlockstore) GetSize(c cid.Cid) (int, error) {
s, err := bs.read.GetSize(c)
func (bs *BufferedBlockstore) GetSize(ctx context.Context, c cid.Cid) (int, error) {
s, err := bs.read.GetSize(ctx, c)
if err == ErrNotFound || s == 0 {
return bs.write.GetSize(c)
return bs.write.GetSize(ctx, c)
}
return s, err
}
func (bs *BufferedBlockstore) Put(blk block.Block) error {
has, err := bs.read.Has(blk.Cid()) // TODO: consider dropping this check
func (bs *BufferedBlockstore) Put(ctx context.Context, blk block.Block) error {
has, err := bs.read.Has(ctx, blk.Cid()) // TODO: consider dropping this check
if err != nil {
return err
}
@ -145,11 +145,11 @@ func (bs *BufferedBlockstore) Put(blk block.Block) error {
return nil
}
return bs.write.Put(blk)
return bs.write.Put(ctx, blk)
}
func (bs *BufferedBlockstore) Has(c cid.Cid) (bool, error) {
has, err := bs.write.Has(c)
func (bs *BufferedBlockstore) Has(ctx context.Context, c cid.Cid) (bool, error) {
has, err := bs.write.Has(ctx, c)
if err != nil {
return false, err
}
@ -157,7 +157,7 @@ func (bs *BufferedBlockstore) Has(c cid.Cid) (bool, error) {
return true, nil
}
return bs.read.Has(c)
return bs.read.Has(ctx, c)
}
func (bs *BufferedBlockstore) HashOnRead(hor bool) {
@ -165,8 +165,8 @@ func (bs *BufferedBlockstore) HashOnRead(hor bool) {
bs.write.HashOnRead(hor)
}
func (bs *BufferedBlockstore) PutMany(blks []block.Block) error {
return bs.write.PutMany(blks)
func (bs *BufferedBlockstore) PutMany(ctx context.Context, blks []block.Block) error {
return bs.write.PutMany(ctx, blks)
}
func (bs *BufferedBlockstore) Read() Blockstore {

24
blockstore/discard.go
View File

@ -18,39 +18,39 @@ func NewDiscardStore(bs Blockstore) Blockstore {
return &discardstore{bs: bs}
}
func (b *discardstore) Has(cid cid.Cid) (bool, error) {
return b.bs.Has(cid)
func (b *discardstore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
return b.bs.Has(ctx, cid)
}
func (b *discardstore) HashOnRead(hor bool) {
b.bs.HashOnRead(hor)
}
func (b *discardstore) Get(cid cid.Cid) (blocks.Block, error) {
return b.bs.Get(cid)
func (b *discardstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error) {
return b.bs.Get(ctx, cid)
}
func (b *discardstore) GetSize(cid cid.Cid) (int, error) {
return b.bs.GetSize(cid)
func (b *discardstore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
return b.bs.GetSize(ctx, cid)
}
func (b *discardstore) View(cid cid.Cid, f func([]byte) error) error {
return b.bs.View(cid, f)
func (b *discardstore) View(ctx context.Context, cid cid.Cid, f func([]byte) error) error {
return b.bs.View(ctx, cid, f)
}
func (b *discardstore) Put(blk blocks.Block) error {
func (b *discardstore) Put(ctx context.Context, blk blocks.Block) error {
return nil
}
func (b *discardstore) PutMany(blks []blocks.Block) error {
func (b *discardstore) PutMany(ctx context.Context, blks []blocks.Block) error {
return nil
}
func (b *discardstore) DeleteBlock(cid cid.Cid) error {
func (b *discardstore) DeleteBlock(ctx context.Context, cid cid.Cid) error {
return nil
}
func (b *discardstore) DeleteMany(cids []cid.Cid) error {
func (b *discardstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
return nil
}

10
blockstore/fallback.go
View File

@ -71,14 +71,14 @@ func (fbs *FallbackStore) getFallback(c cid.Cid) (blocks.Block, error) {
// chain bitswap puts blocks in temp blockstore which is cleaned up
// every few min (to drop any messages we fetched but don't want)
// in this case we want to keep this block around
if err := fbs.Put(b); err != nil {
if err := fbs.Put(ctx, b); err != nil {
return nil, xerrors.Errorf("persisting fallback-fetched block: %w", err)
}
return b, nil
}
func (fbs *FallbackStore) Get(c cid.Cid) (blocks.Block, error) {
b, err := fbs.Blockstore.Get(c)
func (fbs *FallbackStore) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {
b, err := fbs.Blockstore.Get(ctx, c)
switch err {
case nil:
return b, nil
@ -89,8 +89,8 @@ func (fbs *FallbackStore) Get(c cid.Cid) (blocks.Block, error) {
}
}
func (fbs *FallbackStore) GetSize(c cid.Cid) (int, error) {
sz, err := fbs.Blockstore.GetSize(c)
func (fbs *FallbackStore) GetSize(ctx context.Context, c cid.Cid) (int, error) {
sz, err := fbs.Blockstore.GetSize(ctx, c)
switch err {
case nil:
return sz, nil

32
blockstore/idstore.go
View File

@ -38,7 +38,7 @@ func decodeCid(cid cid.Cid) (inline bool, data []byte, err error) {
return false, nil, err
}
func (b *idstore) Has(cid cid.Cid) (bool, error) {
func (b *idstore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
inline, _, err := decodeCid(cid)
if err != nil {
return false, xerrors.Errorf("error decoding Cid: %w", err)
@ -48,10 +48,10 @@ func (b *idstore) Has(cid cid.Cid) (bool, error) {
return true, nil
}
return b.bs.Has(cid)
return b.bs.Has(ctx, cid)
}
func (b *idstore) Get(cid cid.Cid) (blocks.Block, error) {
func (b *idstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error) {
inline, data, err := decodeCid(cid)
if err != nil {
return nil, xerrors.Errorf("error decoding Cid: %w", err)
@ -61,10 +61,10 @@ func (b *idstore) Get(cid cid.Cid) (blocks.Block, error) {
return blocks.NewBlockWithCid(data, cid)
}
return b.bs.Get(cid)
return b.bs.Get(ctx, cid)
}
func (b *idstore) GetSize(cid cid.Cid) (int, error) {
func (b *idstore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
inline, data, err := decodeCid(cid)
if err != nil {
return 0, xerrors.Errorf("error decoding Cid: %w", err)
@ -74,10 +74,10 @@ func (b *idstore) GetSize(cid cid.Cid) (int, error) {
return len(data), err
}
return b.bs.GetSize(cid)
return b.bs.GetSize(ctx, cid)
}
func (b *idstore) View(cid cid.Cid, cb func([]byte) error) error {
func (b *idstore) View(ctx context.Context, cid cid.Cid, cb func([]byte) error) error {
inline, data, err := decodeCid(cid)
if err != nil {
return xerrors.Errorf("error decoding Cid: %w", err)
@ -87,10 +87,10 @@ func (b *idstore) View(cid cid.Cid, cb func([]byte) error) error {
return cb(data)
}
return b.bs.View(cid, cb)
return b.bs.View(ctx, cid, cb)
}
func (b *idstore) Put(blk blocks.Block) error {
func (b *idstore) Put(ctx context.Context, blk blocks.Block) error {
inline, _, err := decodeCid(blk.Cid())
if err != nil {
return xerrors.Errorf("error decoding Cid: %w", err)
@ -100,10 +100,10 @@ func (b *idstore) Put(blk blocks.Block) error {
return nil
}
return b.bs.Put(blk)
return b.bs.Put(ctx, blk)
}
func (b *idstore) PutMany(blks []blocks.Block) error {
func (b *idstore) PutMany(ctx context.Context, blks []blocks.Block) error {
toPut := make([]blocks.Block, 0, len(blks))
for _, blk := range blks {
inline, _, err := decodeCid(blk.Cid())
@ -118,13 +118,13 @@ func (b *idstore) PutMany(blks []blocks.Block) error {
}
if len(toPut) > 0 {
return b.bs.PutMany(toPut)
return b.bs.PutMany(ctx, toPut)
}
return nil
}
func (b *idstore) DeleteBlock(cid cid.Cid) error {
func (b *idstore) DeleteBlock(ctx context.Context, cid cid.Cid) error {
inline, _, err := decodeCid(cid)
if err != nil {
return xerrors.Errorf("error decoding Cid: %w", err)
@ -134,10 +134,10 @@ func (b *idstore) DeleteBlock(cid cid.Cid) error {
return nil
}
return b.bs.DeleteBlock(cid)
return b.bs.DeleteBlock(ctx, cid)
}
func (b *idstore) DeleteMany(cids []cid.Cid) error {
func (b *idstore) DeleteMany(ctx context.Context, cids []cid.Cid) error {
toDelete := make([]cid.Cid, 0, len(cids))
for _, cid := range cids {
inline, _, err := decodeCid(cid)
@ -152,7 +152,7 @@ func (b *idstore) DeleteMany(cids []cid.Cid) error {
}
if len(toDelete) > 0 {
return b.bs.DeleteMany(toDelete)
return b.bs.DeleteMany(ctx, toDelete)
}
return nil

22
blockstore/ipfs.go
View File

@ -79,12 +79,12 @@ func NewRemoteIPFSBlockstore(ctx context.Context, maddr multiaddr.Multiaddr, onl
return Adapt(bs), nil
}
func (i *IPFSBlockstore) DeleteBlock(cid cid.Cid) error {
func (i *IPFSBlockstore) DeleteBlock(ctx context.Context, cid cid.Cid) error {
return xerrors.Errorf("not supported")
}
func (i *IPFSBlockstore) Has(cid cid.Cid) (bool, error) {
_, err := i.offlineAPI.Block().Stat(i.ctx, path.IpldPath(cid))
func (i *IPFSBlockstore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
_, err := i.offlineAPI.Block().Stat(ctx, path.IpldPath(cid))
if err != nil {
// The underlying client is running in Offline mode.
// Stat() will fail with an err if the block isn't in the
@ -99,8 +99,8 @@ func (i *IPFSBlockstore) Has(cid cid.Cid) (bool, error) {
return true, nil
}
func (i *IPFSBlockstore) Get(cid cid.Cid) (blocks.Block, error) {
rd, err := i.api.Block().Get(i.ctx, path.IpldPath(cid))
func (i *IPFSBlockstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error) {
rd, err := i.api.Block().Get(ctx, path.IpldPath(cid))
if err != nil {
return nil, xerrors.Errorf("getting ipfs block: %w", err)
}
@ -113,8 +113,8 @@ func (i *IPFSBlockstore) Get(cid cid.Cid) (blocks.Block, error) {
return blocks.NewBlockWithCid(data, cid)
}
func (i *IPFSBlockstore) GetSize(cid cid.Cid) (int, error) {
st, err := i.api.Block().Stat(i.ctx, path.IpldPath(cid))
func (i *IPFSBlockstore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
st, err := i.api.Block().Stat(ctx, path.IpldPath(cid))
if err != nil {
return 0, xerrors.Errorf("getting ipfs block: %w", err)
}
@ -122,23 +122,23 @@ func (i *IPFSBlockstore) GetSize(cid cid.Cid) (int, error) {
return st.Size(), nil
}
func (i *IPFSBlockstore) Put(block blocks.Block) error {
func (i *IPFSBlockstore) Put(ctx context.Context, block blocks.Block) error {
mhd, err := multihash.Decode(block.Cid().Hash())
if err != nil {
return err
}
_, err = i.api.Block().Put(i.ctx, bytes.NewReader(block.RawData()),
_, err = i.api.Block().Put(ctx, bytes.NewReader(block.RawData()),
options.Block.Hash(mhd.Code, mhd.Length),
options.Block.Format(cid.CodecToStr[block.Cid().Type()]))
return err
}
func (i *IPFSBlockstore) PutMany(blocks []blocks.Block) error {
func (i *IPFSBlockstore) PutMany(ctx context.Context, blocks []blocks.Block) error {
// TODO: could be done in parallel
for _, block := range blocks {
if err := i.Put(block); err != nil {
if err := i.Put(ctx, block); err != nil {
return err
}
}

18
blockstore/mem.go
View File

@ -15,24 +15,24 @@ func NewMemory() MemBlockstore {
// MemBlockstore is a terminal blockstore that keeps blocks in memory.
type MemBlockstore map[cid.Cid]blocks.Block
func (m MemBlockstore) DeleteBlock(k cid.Cid) error {
func (m MemBlockstore) DeleteBlock(ctx context.Context, k cid.Cid) error {
delete(m, k)
return nil
}
func (m MemBlockstore) DeleteMany(ks []cid.Cid) error {
func (m MemBlockstore) DeleteMany(ctx context.Context, ks []cid.Cid) error {
for _, k := range ks {
delete(m, k)
}
return nil
}
func (m MemBlockstore) Has(k cid.Cid) (bool, error) {
func (m MemBlockstore) Has(ctx context.Context, k cid.Cid) (bool, error) {
_, ok := m[k]
return ok, nil
}
func (m MemBlockstore) View(k cid.Cid, callback func([]byte) error) error {
func (m MemBlockstore) View(ctx context.Context, k cid.Cid, callback func([]byte) error) error {
b, ok := m[k]
if !ok {
return ErrNotFound
@ -40,7 +40,7 @@ func (m MemBlockstore) View(k cid.Cid, callback func([]byte) error) error {
return callback(b.RawData())
}
func (m MemBlockstore) Get(k cid.Cid) (blocks.Block, error) {
func (m MemBlockstore) Get(ctx context.Context, k cid.Cid) (blocks.Block, error) {
b, ok := m[k]
if !ok {
return nil, ErrNotFound
@ -49,7 +49,7 @@ func (m MemBlockstore) Get(k cid.Cid) (blocks.Block, error) {
}
// GetSize returns the CIDs mapped BlockSize
func (m MemBlockstore) GetSize(k cid.Cid) (int, error) {
func (m MemBlockstore) GetSize(ctx context.Context, k cid.Cid) (int, error) {
b, ok := m[k]
if !ok {
return 0, ErrNotFound
@ -58,7 +58,7 @@ func (m MemBlockstore) GetSize(k cid.Cid) (int, error) {
}
// Put puts a given block to the underlying datastore
func (m MemBlockstore) Put(b blocks.Block) error {
func (m MemBlockstore) Put(ctx context.Context, b blocks.Block) error {
// Convert to a basic block for safety, but try to reuse the existing
// block if it's already a basic block.
k := b.Cid()
@ -76,9 +76,9 @@ func (m MemBlockstore) Put(b blocks.Block) error {
// PutMany puts a slice of blocks at the same time using batching
// capabilities of the underlying datastore whenever possible.
func (m MemBlockstore) PutMany(bs []blocks.Block) error {
func (m MemBlockstore) PutMany(ctx context.Context, bs []blocks.Block) error {
for _, b := range bs {
_ = m.Put(b) // can't fail
_ = m.Put(ctx, b) // can't fail
}
return nil
}

46
blockstore/splitstore/splitstore.go
View File

@ -216,17 +216,17 @@ func Open(path string, ds dstore.Datastore, hot, cold bstore.Blockstore, cfg *Co
}
// Blockstore interface
func (s *SplitStore) DeleteBlock(_ cid.Cid) error {
func (s *SplitStore) DeleteBlock(_ context.Context, _ cid.Cid) error {
// afaict we don't seem to be using this method, so it's not implemented
return errors.New("DeleteBlock not implemented on SplitStore; don't do this Luke!") //nolint
}
func (s *SplitStore) DeleteMany(_ []cid.Cid) error {
func (s *SplitStore) DeleteMany(_ context.Context, _ []cid.Cid) error {
// afaict we don't seem to be using this method, so it's not implemented
return errors.New("DeleteMany not implemented on SplitStore; don't do this Luke!") //nolint
}
func (s *SplitStore) Has(cid cid.Cid) (bool, error) {
func (s *SplitStore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
if isIdentiyCid(cid) {
return true, nil
}
@ -234,7 +234,7 @@ func (s *SplitStore) Has(cid cid.Cid) (bool, error) {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
has, err := s.hot.Has(cid)
has, err := s.hot.Has(ctx, cid)
if err != nil {
return has, err
@ -245,10 +245,10 @@ func (s *SplitStore) Has(cid cid.Cid) (bool, error) {
return true, nil
}
return s.cold.Has(cid)
return s.cold.Has(ctx, cid)
}
func (s *SplitStore) Get(cid cid.Cid) (blocks.Block, error) {
func (s *SplitStore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error) {
if isIdentiyCid(cid) {
data, err := decodeIdentityCid(cid)
if err != nil {
@ -261,7 +261,7 @@ func (s *SplitStore) Get(cid cid.Cid) (blocks.Block, error) {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
blk, err := s.hot.Get(cid)
blk, err := s.hot.Get(ctx, cid)
switch err {
case nil:
@ -273,7 +273,7 @@ func (s *SplitStore) Get(cid cid.Cid) (blocks.Block, error) {
s.debug.LogReadMiss(cid)
}
blk, err = s.cold.Get(cid)
blk, err = s.cold.Get(ctx, cid)
if err == nil {
stats.Record(s.ctx, metrics.SplitstoreMiss.M(1))
@ -285,7 +285,7 @@ func (s *SplitStore) Get(cid cid.Cid) (blocks.Block, error) {
}
}
func (s *SplitStore) GetSize(cid cid.Cid) (int, error) {
func (s *SplitStore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
if isIdentiyCid(cid) {
data, err := decodeIdentityCid(cid)
if err != nil {
@ -298,7 +298,7 @@ func (s *SplitStore) GetSize(cid cid.Cid) (int, error) {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
size, err := s.hot.GetSize(cid)
size, err := s.hot.GetSize(ctx, cid)
switch err {
case nil:
@ -310,7 +310,7 @@ func (s *SplitStore) GetSize(cid cid.Cid) (int, error) {
s.debug.LogReadMiss(cid)
}
size, err = s.cold.GetSize(cid)
size, err = s.cold.GetSize(ctx, cid)
if err == nil {
stats.Record(s.ctx, metrics.SplitstoreMiss.M(1))
}
@ -321,7 +321,7 @@ func (s *SplitStore) GetSize(cid cid.Cid) (int, error) {
}
}
func (s *SplitStore) Put(blk blocks.Block) error {
func (s *SplitStore) Put(ctx context.Context, blk blocks.Block) error {
if isIdentiyCid(blk.Cid()) {
return nil
}
@ -329,7 +329,7 @@ func (s *SplitStore) Put(blk blocks.Block) error {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
err := s.hot.Put(blk)
err := s.hot.Put(ctx, blk)
if err != nil {
return err
}
@ -340,7 +340,7 @@ func (s *SplitStore) Put(blk blocks.Block) error {
return nil
}
func (s *SplitStore) PutMany(blks []blocks.Block) error {
func (s *SplitStore) PutMany(ctx context.Context, blks []blocks.Block) error {
// filter identites
idcids := 0
for _, blk := range blks {
@ -374,7 +374,7 @@ func (s *SplitStore) PutMany(blks []blocks.Block) error {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
err := s.hot.PutMany(blks)
err := s.hot.PutMany(ctx, blks)
if err != nil {
return err
}
@ -430,7 +430,7 @@ func (s *SplitStore) HashOnRead(enabled bool) {
s.cold.HashOnRead(enabled)
}
func (s *SplitStore) View(cid cid.Cid, cb func([]byte) error) error {
func (s *SplitStore) View(ctx context.Context, cid cid.Cid, cb func([]byte) error) error {
if isIdentiyCid(cid) {
data, err := decodeIdentityCid(cid)
if err != nil {
@ -451,14 +451,14 @@ func (s *SplitStore) View(cid cid.Cid, cb func([]byte) error) error {
s.protectView(cid)
defer s.viewDone()
err := s.hot.View(cid, cb)
err := s.hot.View(ctx, cid, cb)
switch err {
case bstore.ErrNotFound:
if s.isWarm() {
s.debug.LogReadMiss(cid)
}
err = s.cold.View(cid, cb)
err = s.cold.View(ctx, cid, cb)
if err == nil {
stats.Record(s.ctx, metrics.SplitstoreMiss.M(1))
}
@ -502,7 +502,7 @@ func (s *SplitStore) Start(chain ChainAccessor, us stmgr.UpgradeSchedule) error
// load base epoch from metadata ds
// if none, then use current epoch because it's a fresh start
bs, err := s.ds.Get(baseEpochKey)
bs, err := s.ds.Get(s.ctx, baseEpochKey)
switch err {
case nil:
s.baseEpoch = bytesToEpoch(bs)
@ -523,7 +523,7 @@ func (s *SplitStore) Start(chain ChainAccessor, us stmgr.UpgradeSchedule) error
}
// load warmup epoch from metadata ds
bs, err = s.ds.Get(warmupEpochKey)
bs, err = s.ds.Get(s.ctx, warmupEpochKey)
switch err {
case nil:
s.warmupEpoch = bytesToEpoch(bs)
@ -536,7 +536,7 @@ func (s *SplitStore) Start(chain ChainAccessor, us stmgr.UpgradeSchedule) error
}
// load markSetSize from metadata ds to provide a size hint for marksets
bs, err = s.ds.Get(markSetSizeKey)
bs, err = s.ds.Get(s.ctx, markSetSizeKey)
switch err {
case nil:
s.markSetSize = bytesToInt64(bs)
@ -547,7 +547,7 @@ func (s *SplitStore) Start(chain ChainAccessor, us stmgr.UpgradeSchedule) error
}
// load compactionIndex from metadata ds to provide a hint as to when to perform moving gc
bs, err = s.ds.Get(compactionIndexKey)
bs, err = s.ds.Get(s.ctx, compactionIndexKey)
switch err {
case nil:
s.compactionIndex = bytesToInt64(bs)
@ -609,5 +609,5 @@ func (s *SplitStore) checkClosing() error {
func (s *SplitStore) setBaseEpoch(epoch abi.ChainEpoch) error {
s.baseEpoch = epoch
return s.ds.Put(baseEpochKey, epochToBytes(epoch))
return s.ds.Put(s.ctx, baseEpochKey, epochToBytes(epoch))
}

4
blockstore/splitstore/splitstore_check.go
View File

@ -96,7 +96,7 @@ func (s *SplitStore) doCheck(curTs *types.TipSet) error {
return errStopWalk
}
has, err := s.hot.Has(c)
has, err := s.hot.Has(s.ctx, c)
if err != nil {
return xerrors.Errorf("error checking hotstore: %w", err)
}
@ -105,7 +105,7 @@ func (s *SplitStore) doCheck(curTs *types.TipSet) error {
return nil
}
has, err = s.cold.Has(c)
has, err = s.cold.Has(s.ctx, c)
if err != nil {
return xerrors.Errorf("error checking coldstore: %w", err)
}

20
blockstore/splitstore/splitstore_compact.go
View File

@ -577,13 +577,13 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
return xerrors.Errorf("error saving base epoch: %w", err)
}
err = s.ds.Put(markSetSizeKey, int64ToBytes(s.markSetSize))
err = s.ds.Put(s.ctx, markSetSizeKey, int64ToBytes(s.markSetSize))
if err != nil {
return xerrors.Errorf("error saving mark set size: %w", err)
}
s.compactionIndex++
err = s.ds.Put(compactionIndexKey, int64ToBytes(s.compactionIndex))
err = s.ds.Put(s.ctx, compactionIndexKey, int64ToBytes(s.compactionIndex))
if err != nil {
return xerrors.Errorf("error saving compaction index: %w", err)
}
@ -835,10 +835,10 @@ func (s *SplitStore) view(c cid.Cid, cb func([]byte) error) error {
return cb(data)
}
err := s.hot.View(c, cb)
err := s.hot.View(s.ctx, c, cb)
switch err {
case bstore.ErrNotFound:
return s.cold.View(c, cb)
return s.cold.View(s.ctx, c, cb)
default:
return err
@ -850,13 +850,13 @@ func (s *SplitStore) has(c cid.Cid) (bool, error) {
return true, nil
}
has, err := s.hot.Has(c)
has, err := s.hot.Has(s.ctx, c)
if has || err != nil {
return has, err
}
return s.cold.Has(c)
return s.cold.Has(s.ctx, c)
}
func (s *SplitStore) moveColdBlocks(cold []cid.Cid) error {
@ -867,7 +867,7 @@ func (s *SplitStore) moveColdBlocks(cold []cid.Cid) error {
return err
}
blk, err := s.hot.Get(c)
blk, err := s.hot.Get(s.ctx, c)
if err != nil {
if err == bstore.ErrNotFound {
log.Warnf("hotstore missing block %s", c)
@ -879,7 +879,7 @@ func (s *SplitStore) moveColdBlocks(cold []cid.Cid) error {
batch = append(batch, blk)
if len(batch) == batchSize {
err = s.cold.PutMany(batch)
err = s.cold.PutMany(s.ctx, batch)
if err != nil {
return xerrors.Errorf("error putting batch to coldstore: %w", err)
}
@ -888,7 +888,7 @@ func (s *SplitStore) moveColdBlocks(cold []cid.Cid) error {
}
if len(batch) > 0 {
err := s.cold.PutMany(batch)
err := s.cold.PutMany(s.ctx, batch)
if err != nil {
return xerrors.Errorf("error putting batch to coldstore: %w", err)
}
@ -1058,7 +1058,7 @@ func (s *SplitStore) purge(cids []cid.Cid, markSet MarkSetVisitor) error {
deadCids = append(deadCids, c)
}
err := s.hot.DeleteMany(deadCids)
err := s.hot.DeleteMany(s.ctx, deadCids)
if err != nil {
return xerrors.Errorf("error purging cold objects: %w", err)
}

36
blockstore/splitstore/splitstore_expose.go
View File

@ -20,28 +20,28 @@ func (s *SplitStore) Expose() bstore.Blockstore {
return &exposedSplitStore{s: s}
}
func (es *exposedSplitStore) DeleteBlock(_ cid.Cid) error {
func (es *exposedSplitStore) DeleteBlock(_ context.Context, _ cid.Cid) error {
return errors.New("DeleteBlock: operation not supported")
}
func (es *exposedSplitStore) DeleteMany(_ []cid.Cid) error {
func (es *exposedSplitStore) DeleteMany(_ context.Context, _ []cid.Cid) error {
return errors.New("DeleteMany: operation not supported")
}
func (es *exposedSplitStore) Has(c cid.Cid) (bool, error) {
func (es *exposedSplitStore) Has(ctx context.Context, c cid.Cid) (bool, error) {
if isIdentiyCid(c) {
return true, nil
}
has, err := es.s.hot.Has(c)
has, err := es.s.hot.Has(ctx, c)
if has || err != nil {
return has, err
}
return es.s.cold.Has(c)
return es.s.cold.Has(ctx, c)
}
func (es *exposedSplitStore) Get(c cid.Cid) (blocks.Block, error) {
func (es *exposedSplitStore) Get(ctx context.Context, c cid.Cid) (blocks.Block, error) {
if isIdentiyCid(c) {
data, err := decodeIdentityCid(c)
if err != nil {
@ -51,16 +51,16 @@ func (es *exposedSplitStore) Get(c cid.Cid) (blocks.Block, error) {
return blocks.NewBlockWithCid(data, c)
}
blk, err := es.s.hot.Get(c)
blk, err := es.s.hot.Get(ctx, c)
switch err {
case bstore.ErrNotFound:
return es.s.cold.Get(c)
return es.s.cold.Get(ctx, c)
default:
return blk, err
}
}
func (es *exposedSplitStore) GetSize(c cid.Cid) (int, error) {
func (es *exposedSplitStore) GetSize(ctx context.Context, c cid.Cid) (int, error) {
if isIdentiyCid(c) {
data, err := decodeIdentityCid(c)
if err != nil {
@ -70,21 +70,21 @@ func (es *exposedSplitStore) GetSize(c cid.Cid) (int, error) {
return len(data), nil
}
size, err := es.s.hot.GetSize(c)
size, err := es.s.hot.GetSize(ctx, c)
switch err {
case bstore.ErrNotFound:
return es.s.cold.GetSize(c)
return es.s.cold.GetSize(ctx, c)
default:
return size, err
}
}
func (es *exposedSplitStore) Put(blk blocks.Block) error {
return es.s.Put(blk)
func (es *exposedSplitStore) Put(ctx context.Context, blk blocks.Block) error {
return es.s.Put(ctx, blk)
}
func (es *exposedSplitStore) PutMany(blks []blocks.Block) error {
return es.s.PutMany(blks)
func (es *exposedSplitStore) PutMany(ctx context.Context, blks []blocks.Block) error {
return es.s.PutMany(ctx, blks)
}
func (es *exposedSplitStore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {
@ -93,7 +93,7 @@ func (es *exposedSplitStore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, e
func (es *exposedSplitStore) HashOnRead(enabled bool) {}
func (es *exposedSplitStore) View(c cid.Cid, f func([]byte) error) error {
func (es *exposedSplitStore) View(ctx context.Context, c cid.Cid, f func([]byte) error) error {
if isIdentiyCid(c) {
data, err := decodeIdentityCid(c)
if err != nil {
@ -103,10 +103,10 @@ func (es *exposedSplitStore) View(c cid.Cid, f func([]byte) error) error {
return f(data)
}
err := es.s.hot.View(c, f)
err := es.s.hot.View(ctx, c, f)
switch err {
case bstore.ErrNotFound:
return es.s.cold.View(c, f)
return es.s.cold.View(ctx, c, f)
default:
return err

48
blockstore/splitstore/splitstore_test.go
View File

@ -30,6 +30,7 @@ func init() {
}
func testSplitStore(t *testing.T, cfg *Config) {
ctx := context.Background()
chain := &mockChain{t: t}
// the myriads of stores
@ -39,7 +40,7 @@ func testSplitStore(t *testing.T, cfg *Config) {
// this is necessary to avoid the garbage mock puts in the blocks
garbage := blocks.NewBlock([]byte{1, 2, 3})
err := cold.Put(garbage)
err := cold.Put(ctx, garbage)
if err != nil {
t.Fatal(err)
}
@ -60,21 +61,21 @@ func testSplitStore(t *testing.T, cfg *Config) {
t.Fatal(err)
}
err = cold.Put(blk)
err = cold.Put(ctx, blk)
if err != nil {
t.Fatal(err)
}
// create a garbage block that is protected with a rgistered protector
protected := blocks.NewBlock([]byte("protected!"))
err = hot.Put(protected)
err = hot.Put(ctx, protected)
if err != nil {
t.Fatal(err)
}
// and another one that is not protected
unprotected := blocks.NewBlock([]byte("unprotected!"))
err = hot.Put(unprotected)
err = hot.Put(ctx, unprotected)
if err != nil {
t.Fatal(err)
}
@ -109,11 +110,11 @@ func testSplitStore(t *testing.T, cfg *Config) {
if err != nil {
t.Fatal(err)
}
err = ss.Put(stateRoot)
err = ss.Put(ctx, stateRoot)
if err != nil {
t.Fatal(err)
}
err = ss.Put(sblk)
err = ss.Put(ctx, sblk)
if err != nil {
t.Fatal(err)
}
@ -176,7 +177,7 @@ func testSplitStore(t *testing.T, cfg *Config) {
}
// ensure our protected block is still there
has, err := hot.Has(protected.Cid())
has, err := hot.Has(ctx, protected.Cid())
if err != nil {
t.Fatal(err)
}
@ -186,7 +187,7 @@ func testSplitStore(t *testing.T, cfg *Config) {
}
// ensure our unprotected block is in the coldstore now
has, err = hot.Has(unprotected.Cid())
has, err = hot.Has(ctx, unprotected.Cid())
if err != nil {
t.Fatal(err)
}
@ -195,7 +196,7 @@ func testSplitStore(t *testing.T, cfg *Config) {
t.Fatal("unprotected block is still in hotstore")
}
has, err = cold.Has(unprotected.Cid())
has, err = cold.Has(ctx, unprotected.Cid())
if err != nil {
t.Fatal(err)
}
@ -222,6 +223,7 @@ func TestSplitStoreCompactionWithBadger(t *testing.T) {
}
func TestSplitStoreSuppressCompactionNearUpgrade(t *testing.T) {
ctx := context.Background()
chain := &mockChain{t: t}
// the myriads of stores
@ -231,7 +233,7 @@ func TestSplitStoreSuppressCompactionNearUpgrade(t *testing.T) {
// this is necessary to avoid the garbage mock puts in the blocks
garbage := blocks.NewBlock([]byte{1, 2, 3})
err := cold.Put(garbage)
err := cold.Put(ctx, garbage)
if err != nil {
t.Fatal(err)
}
@ -252,7 +254,7 @@ func TestSplitStoreSuppressCompactionNearUpgrade(t *testing.T) {
t.Fatal(err)
}
err = cold.Put(blk)
err = cold.Put(ctx, blk)
if err != nil {
t.Fatal(err)
}
@ -288,11 +290,11 @@ func TestSplitStoreSuppressCompactionNearUpgrade(t *testing.T) {
if err != nil {
t.Fatal(err)
}
err = ss.Put(stateRoot)
err = ss.Put(ctx, stateRoot)
if err != nil {
t.Fatal(err)
}
err = ss.Put(sblk)
err = ss.Put(ctx, sblk)
if err != nil {
t.Fatal(err)
}
@ -431,7 +433,7 @@ func newMockStore() *mockStore {
return &mockStore{set: make(map[cid.Cid]blocks.Block)}
}
func (b *mockStore) Has(cid cid.Cid) (bool, error) {
func (b *mockStore) Has(_ context.Context, cid cid.Cid) (bool, error) {
b.mx.Lock()
defer b.mx.Unlock()
_, ok := b.set[cid]
@ -440,7 +442,7 @@ func (b *mockStore) Has(cid cid.Cid) (bool, error) {
func (b *mockStore) HashOnRead(hor bool) {}
func (b *mockStore) Get(cid cid.Cid) (blocks.Block, error) {
func (b *mockStore) Get(_ context.Context, cid cid.Cid) (blocks.Block, error) {
b.mx.Lock()
defer b.mx.Unlock()
@ -451,8 +453,8 @@ func (b *mockStore) Get(cid cid.Cid) (blocks.Block, error) {
return blk, nil
}
func (b *mockStore) GetSize(cid cid.Cid) (int, error) {
blk, err := b.Get(cid)
func (b *mockStore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
blk, err := b.Get(ctx, cid)
if err != nil {
return 0, err
}
@ -460,15 +462,15 @@ func (b *mockStore) GetSize(cid cid.Cid) (int, error) {
return len(blk.RawData()), nil
}
func (b *mockStore) View(cid cid.Cid, f func([]byte) error) error {
blk, err := b.Get(cid)
func (b *mockStore) View(ctx context.Context, cid cid.Cid, f func([]byte) error) error {
blk, err := b.Get(ctx, cid)
if err != nil {
return err
}
return f(blk.RawData())
}
func (b *mockStore) Put(blk blocks.Block) error {
func (b *mockStore) Put(_ context.Context, blk blocks.Block) error {
b.mx.Lock()
defer b.mx.Unlock()
@ -476,7 +478,7 @@ func (b *mockStore) Put(blk blocks.Block) error {
return nil
}
func (b *mockStore) PutMany(blks []blocks.Block) error {
func (b *mockStore) PutMany(_ context.Context, blks []blocks.Block) error {
b.mx.Lock()
defer b.mx.Unlock()
@ -486,7 +488,7 @@ func (b *mockStore) PutMany(blks []blocks.Block) error {
return nil
}
func (b *mockStore) DeleteBlock(cid cid.Cid) error {
func (b *mockStore) DeleteBlock(_ context.Context, cid cid.Cid) error {
b.mx.Lock()
defer b.mx.Unlock()
@ -494,7 +496,7 @@ func (b *mockStore) DeleteBlock(cid cid.Cid) error {
return nil
}
func (b *mockStore) DeleteMany(cids []cid.Cid) error {
func (b *mockStore) DeleteMany(_ context.Context, cids []cid.Cid) error {
b.mx.Lock()
defer b.mx.Unlock()

14
blockstore/splitstore/splitstore_warmup.go
View File

@ -75,7 +75,7 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
count++
has, err := s.hot.Has(c)
has, err := s.hot.Has(s.ctx, c)
if err != nil {
return err
}
@ -84,7 +84,7 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
return nil
}
blk, err := s.cold.Get(c)
blk, err := s.cold.Get(s.ctx, c)
if err != nil {
if err == bstore.ErrNotFound {
missing++
@ -97,7 +97,7 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
batchHot = append(batchHot, blk)
if len(batchHot) == batchSize {
err = s.hot.PutMany(batchHot)
err = s.hot.PutMany(s.ctx, batchHot)
if err != nil {
return err
}
@ -112,7 +112,7 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
}
if len(batchHot) > 0 {
err = s.hot.PutMany(batchHot)
err = s.hot.PutMany(s.ctx, batchHot)
if err != nil {
return err
}
@ -121,13 +121,13 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
log.Infow("warmup stats", "visited", count, "warm", xcount, "missing", missing)
s.markSetSize = count + count>>2 // overestimate a bit
err = s.ds.Put(markSetSizeKey, int64ToBytes(s.markSetSize))
err = s.ds.Put(s.ctx, markSetSizeKey, int64ToBytes(s.markSetSize))
if err != nil {
log.Warnf("error saving mark set size: %s", err)
}
// save the warmup epoch
err = s.ds.Put(warmupEpochKey, epochToBytes(epoch))
err = s.ds.Put(s.ctx, warmupEpochKey, epochToBytes(epoch))
if err != nil {
return xerrors.Errorf("error saving warm up epoch: %w", err)
}
@ -136,7 +136,7 @@ func (s *SplitStore) doWarmup(curTs *types.TipSet) error {
s.mx.Unlock()
// also save the compactionIndex, as this is used as an indicator of warmup for upgraded nodes
err = s.ds.Put(compactionIndexKey, int64ToBytes(s.compactionIndex))
err = s.ds.Put(s.ctx, compactionIndexKey, int64ToBytes(s.compactionIndex))
if err != nil {
return xerrors.Errorf("error saving compaction index: %w", err)
}

32
blockstore/sync.go
View File

@ -20,53 +20,53 @@ type SyncBlockstore struct {
bs MemBlockstore // specifically use a memStore to save indirection overhead.
}
func (m *SyncBlockstore) DeleteBlock(k cid.Cid) error {
func (m *SyncBlockstore) DeleteBlock(ctx context.Context, k cid.Cid) error {
m.mu.Lock()
defer m.mu.Unlock()
return m.bs.DeleteBlock(k)
return m.bs.DeleteBlock(ctx, k)
}
func (m *SyncBlockstore) DeleteMany(ks []cid.Cid) error {
func (m *SyncBlockstore) DeleteMany(ctx context.Context, ks []cid.Cid) error {
m.mu.Lock()
defer m.mu.Unlock()
return m.bs.DeleteMany(ks)
return m.bs.DeleteMany(ctx, ks)
}
func (m *SyncBlockstore) Has(k cid.Cid) (bool, error) {
func (m *SyncBlockstore) Has(ctx context.Context, k cid.Cid) (bool, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return m.bs.Has(k)
return m.bs.Has(ctx, k)
}
func (m *SyncBlockstore) View(k cid.Cid, callback func([]byte) error) error {
func (m *SyncBlockstore) View(ctx context.Context, k cid.Cid, callback func([]byte) error) error {
m.mu.RLock()
defer m.mu.RUnlock()
return m.bs.View(k, callback)
return m.bs.View(ctx, k, callback)
}
func (m *SyncBlockstore) Get(k cid.Cid) (blocks.Block, error) {
func (m *SyncBlockstore) Get(ctx context.Context, k cid.Cid) (blocks.Block, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return m.bs.Get(k)
return m.bs.Get(ctx, k)
}
func (m *SyncBlockstore) GetSize(k cid.Cid) (int, error) {
func (m *SyncBlockstore) GetSize(ctx context.Context, k cid.Cid) (int, error) {
m.mu.RLock()
defer m.mu.RUnlock()
return m.bs.GetSize(k)
return m.bs.GetSize(ctx, k)
}
func (m *SyncBlockstore) Put(b blocks.Block) error {
func (m *SyncBlockstore) Put(ctx context.Context, b blocks.Block) error {
m.mu.Lock()
defer m.mu.Unlock()
return m.bs.Put(b)
return m.bs.Put(ctx, b)
}
func (m *SyncBlockstore) PutMany(bs []blocks.Block) error {
func (m *SyncBlockstore) PutMany(ctx context.Context, bs []blocks.Block) error {
m.mu.Lock()
defer m.mu.Unlock()
return m.bs.PutMany(bs)
return m.bs.PutMany(ctx, bs)
}
func (m *SyncBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {

40
blockstore/timed.go
View File

@ -92,28 +92,28 @@ func (t *TimedCacheBlockstore) rotate() {
t.mu.Unlock()
}
func (t *TimedCacheBlockstore) Put(b blocks.Block) error {
func (t *TimedCacheBlockstore) Put(ctx context.Context, b blocks.Block) error {
// Don't check the inactive set here. We want to keep this block for at
// least one interval.
t.mu.Lock()
defer t.mu.Unlock()
return t.active.Put(b)
return t.active.Put(ctx, b)
}
func (t *TimedCacheBlockstore) PutMany(bs []blocks.Block) error {
func (t *TimedCacheBlockstore) PutMany(ctx context.Context, bs []blocks.Block) error {
t.mu.Lock()
defer t.mu.Unlock()
return t.active.PutMany(bs)
return t.active.PutMany(ctx, bs)
}
func (t *TimedCacheBlockstore) View(k cid.Cid, callback func([]byte) error) error {
func (t *TimedCacheBlockstore) View(ctx context.Context, k cid.Cid, callback func([]byte) error) error {
// The underlying blockstore is always a "mem" blockstore so there's no difference,
// from a performance perspective, between view & get. So we call Get to avoid
// calling an arbitrary callback while holding a lock.
t.mu.RLock()
block, err := t.active.Get(k)
block, err := t.active.Get(ctx, k)
if err == ErrNotFound {
block, err = t.inactive.Get(k)
block, err = t.inactive.Get(ctx, k)
}
t.mu.RUnlock()
@ -123,51 +123,51 @@ func (t *TimedCacheBlockstore) View(k cid.Cid, callback func([]byte) error) erro
return callback(block.RawData())
}
func (t *TimedCacheBlockstore) Get(k cid.Cid) (blocks.Block, error) {
func (t *TimedCacheBlockstore) Get(ctx context.Context, k cid.Cid) (blocks.Block, error) {
t.mu.RLock()
defer t.mu.RUnlock()
b, err := t.active.Get(k)
b, err := t.active.Get(ctx, k)
if err == ErrNotFound {
b, err = t.inactive.Get(k)
b, err = t.inactive.Get(ctx, k)
}
return b, err
}
func (t *TimedCacheBlockstore) GetSize(k cid.Cid) (int, error) {
func (t *TimedCacheBlockstore) GetSize(ctx context.Context, k cid.Cid) (int, error) {
t.mu.RLock()
defer t.mu.RUnlock()
size, err := t.active.GetSize(k)
size, err := t.active.GetSize(ctx, k)
if err == ErrNotFound {
size, err = t.inactive.GetSize(k)
size, err = t.inactive.GetSize(ctx, k)
}
return size, err
}
func (t *TimedCacheBlockstore) Has(k cid.Cid) (bool, error) {
func (t *TimedCacheBlockstore) Has(ctx context.Context, k cid.Cid) (bool, error) {
t.mu.RLock()
defer t.mu.RUnlock()
if has, err := t.active.Has(k); err != nil {
if has, err := t.active.Has(ctx, k); err != nil {
return false, err
} else if has {
return true, nil
}
return t.inactive.Has(k)
return t.inactive.Has(ctx, k)
}
func (t *TimedCacheBlockstore) HashOnRead(_ bool) {
// no-op
}
func (t *TimedCacheBlockstore) DeleteBlock(k cid.Cid) error {
func (t *TimedCacheBlockstore) DeleteBlock(ctx context.Context, k cid.Cid) error {
t.mu.Lock()
defer t.mu.Unlock()
return multierr.Combine(t.active.DeleteBlock(k), t.inactive.DeleteBlock(k))
return multierr.Combine(t.active.DeleteBlock(ctx, k), t.inactive.DeleteBlock(ctx, k))
}
func (t *TimedCacheBlockstore) DeleteMany(ks []cid.Cid) error {
func (t *TimedCacheBlockstore) DeleteMany(ctx context.Context, ks []cid.Cid) error {
t.mu.Lock()
defer t.mu.Unlock()
return multierr.Combine(t.active.DeleteMany(ks), t.inactive.DeleteMany(ks))
return multierr.Combine(t.active.DeleteMany(ctx, ks), t.inactive.DeleteMany(ctx, ks))
}
func (t *TimedCacheBlockstore) AllKeysChan(_ context.Context) (<-chan cid.Cid, error) {

24
blockstore/timed_test.go
View File

@ -19,6 +19,8 @@ func TestTimedCacheBlockstoreSimple(t *testing.T) {
tc.clock = mClock
tc.doneRotatingCh = make(chan struct{})
ctx := context.Background()
_ = tc.Start(context.Background())
mClock.Add(1) // IDK why it is needed but it makes it work
@ -27,18 +29,18 @@ func TestTimedCacheBlockstoreSimple(t *testing.T) {
}()
b1 := blocks.NewBlock([]byte("foo"))
require.NoError(t, tc.Put(b1))
require.NoError(t, tc.Put(ctx, b1))
b2 := blocks.NewBlock([]byte("bar"))
require.NoError(t, tc.Put(b2))
require.NoError(t, tc.Put(ctx, b2))
b3 := blocks.NewBlock([]byte("baz"))
b1out, err := tc.Get(b1.Cid())
b1out, err := tc.Get(ctx, b1.Cid())
require.NoError(t, err)
require.Equal(t, b1.RawData(), b1out.RawData())
has, err := tc.Has(b1.Cid())
has, err := tc.Has(ctx, b1.Cid())
require.NoError(t, err)
require.True(t, has)
@ -46,17 +48,17 @@ func TestTimedCacheBlockstoreSimple(t *testing.T) {
<-tc.doneRotatingCh
// We should still have everything.
has, err = tc.Has(b1.Cid())
has, err = tc.Has(ctx, b1.Cid())
require.NoError(t, err)
require.True(t, has)
has, err = tc.Has(b2.Cid())
has, err = tc.Has(ctx, b2.Cid())
require.NoError(t, err)
require.True(t, has)
// extend b2, add b3.
require.NoError(t, tc.Put(b2))
require.NoError(t, tc.Put(b3))
require.NoError(t, tc.Put(ctx, b2))
require.NoError(t, tc.Put(ctx, b3))
// all keys once.
allKeys, err := tc.AllKeysChan(context.Background())
@ -71,15 +73,15 @@ func TestTimedCacheBlockstoreSimple(t *testing.T) {
<-tc.doneRotatingCh
// should still have b2, and b3, but not b1
has, err = tc.Has(b1.Cid())
has, err = tc.Has(ctx, b1.Cid())
require.NoError(t, err)
require.False(t, has)
has, err = tc.Has(b2.Cid())
has, err = tc.Has(ctx, b2.Cid())
require.NoError(t, err)
require.True(t, has)
has, err = tc.Has(b3.Cid())
has, err = tc.Has(ctx, b3.Cid())
require.NoError(t, err)
require.True(t, has)
}

32
blockstore/union.go
View File

@ -19,72 +19,72 @@ func Union(stores ...Blockstore) Blockstore {
return unionBlockstore(stores)
}
func (m unionBlockstore) Has(cid cid.Cid) (has bool, err error) {
func (m unionBlockstore) Has(ctx context.Context, cid cid.Cid) (has bool, err error) {
for _, bs := range m {
if has, err = bs.Has(cid); has || err != nil {
if has, err = bs.Has(ctx, cid); has || err != nil {
break
}
}
return has, err
}
func (m unionBlockstore) Get(cid cid.Cid) (blk blocks.Block, err error) {
func (m unionBlockstore) Get(ctx context.Context, cid cid.Cid) (blk blocks.Block, err error) {
for _, bs := range m {
if blk, err = bs.Get(cid); err == nil || err != ErrNotFound {
if blk, err = bs.Get(ctx, cid); err == nil || err != ErrNotFound {
break
}
}
return blk, err
}
func (m unionBlockstore) View(cid cid.Cid, callback func([]byte) error) (err error) {
func (m unionBlockstore) View(ctx context.Context, cid cid.Cid, callback func([]byte) error) (err error) {
for _, bs := range m {
if err = bs.View(cid, callback); err == nil || err != ErrNotFound {
if err = bs.View(ctx, cid, callback); err == nil || err != ErrNotFound {
break
}
}
return err
}
func (m unionBlockstore) GetSize(cid cid.Cid) (size int, err error) {
func (m unionBlockstore) GetSize(ctx context.Context, cid cid.Cid) (size int, err error) {
for _, bs := range m {
if size, err = bs.GetSize(cid); err == nil || err != ErrNotFound {
if size, err = bs.GetSize(ctx, cid); err == nil || err != ErrNotFound {
break
}
}
return size, err
}
func (m unionBlockstore) Put(block blocks.Block) (err error) {
func (m unionBlockstore) Put(ctx context.Context, block blocks.Block) (err error) {
for _, bs := range m {
if err = bs.Put(block); err != nil {
if err = bs.Put(ctx, block); err != nil {
break
}
}
return err
}
func (m unionBlockstore) PutMany(blks []blocks.Block) (err error) {
func (m unionBlockstore) PutMany(ctx context.Context, blks []blocks.Block) (err error) {
for _, bs := range m {
if err = bs.PutMany(blks); err != nil {
if err = bs.PutMany(ctx, blks); err != nil {
break
}
}
return err
}
func (m unionBlockstore) DeleteBlock(cid cid.Cid) (err error) {
func (m unionBlockstore) DeleteBlock(ctx context.Context, cid cid.Cid) (err error) {
for _, bs := range m {
if err = bs.DeleteBlock(cid); err != nil {
if err = bs.DeleteBlock(ctx, cid); err != nil {
break
}
}
return err
}
func (m unionBlockstore) DeleteMany(cids []cid.Cid) (err error) {
func (m unionBlockstore) DeleteMany(ctx context.Context, cids []cid.Cid) (err error) {
for _, bs := range m {
if err = bs.DeleteMany(cids); err != nil {
if err = bs.DeleteMany(ctx, cids); err != nil {
break
}
}

40
blockstore/union_test.go
View File

@ -15,79 +15,81 @@ var (
)
func TestUnionBlockstore_Get(t *testing.T) {
ctx := context.Background()
m1 := NewMemory()
m2 := NewMemory()
_ = m1.Put(b1)
_ = m2.Put(b2)
_ = m1.Put(ctx, b1)
_ = m2.Put(ctx, b2)
u := Union(m1, m2)
v1, err := u.Get(b1.Cid())
v1, err := u.Get(ctx, b1.Cid())
require.NoError(t, err)
require.Equal(t, b1.RawData(), v1.RawData())
v2, err := u.Get(b2.Cid())
v2, err := u.Get(ctx, b2.Cid())
require.NoError(t, err)
require.Equal(t, b2.RawData(), v2.RawData())
}
func TestUnionBlockstore_Put_PutMany_Delete_AllKeysChan(t *testing.T) {
ctx := context.Background()
m1 := NewMemory()
m2 := NewMemory()
u := Union(m1, m2)
err := u.Put(b0)
err := u.Put(ctx, b0)
require.NoError(t, err)
var has bool
// write was broadcasted to all stores.
has, _ = m1.Has(b0.Cid())
has, _ = m1.Has(ctx, b0.Cid())
require.True(t, has)
has, _ = m2.Has(b0.Cid())
has, _ = m2.Has(ctx, b0.Cid())
require.True(t, has)
has, _ = u.Has(b0.Cid())
has, _ = u.Has(ctx, b0.Cid())
require.True(t, has)
// put many.
err = u.PutMany([]blocks.Block{b1, b2})
err = u.PutMany(ctx, []blocks.Block{b1, b2})
require.NoError(t, err)
// write was broadcasted to all stores.
has, _ = m1.Has(b1.Cid())
has, _ = m1.Has(ctx, b1.Cid())
require.True(t, has)
has, _ = m1.Has(b2.Cid())
has, _ = m1.Has(ctx, b2.Cid())
require.True(t, has)
has, _ = m2.Has(b1.Cid())
has, _ = m2.Has(ctx, b1.Cid())
require.True(t, has)
has, _ = m2.Has(b2.Cid())
has, _ = m2.Has(ctx, b2.Cid())
require.True(t, has)
// also in the union store.
has, _ = u.Has(b1.Cid())
has, _ = u.Has(ctx, b1.Cid())
require.True(t, has)
has, _ = u.Has(b2.Cid())
has, _ = u.Has(ctx, b2.Cid())
require.True(t, has)
// deleted from all stores.
err = u.DeleteBlock(b1.Cid())
err = u.DeleteBlock(ctx, b1.Cid())
require.NoError(t, err)
has, _ = u.Has(b1.Cid())
has, _ = u.Has(ctx, b1.Cid())
require.False(t, has)
has, _ = m1.Has(b1.Cid())
has, _ = m1.Has(ctx, b1.Cid())
require.False(t, has)
has, _ = m2.Has(b1.Cid())
has, _ = m2.Has(ctx, b1.Cid())
require.False(t, has)
// check that AllKeysChan returns b0 and b2, twice (once per backing store)

4
build/bootstrap/butterflynet.pi
View File

@ -1,2 +1,2 @@
/dns4/bootstrap-0.butterfly.fildev.network/tcp/1347/p2p/12D3KooWBzv5sf4eTyo8cjJGfGnpxo6QkEPkRShG9GqjE2A5QaW5
/dns4/bootstrap-1.butterfly.fildev.network/tcp/1347/p2p/12D3KooWBo9TSD4XXRFtu6snv6QNYvXgRaSaVb116YiYEsDWgKtq
/dns4/bootstrap-0.butterfly.fildev.network/tcp/1347/p2p/12D3KooWBdRCBLUeKvoy22u5DcXs61adFn31v8WWCZgmBjDCjbsC
/dns4/bootstrap-1.butterfly.fildev.network/tcp/1347/p2p/12D3KooWDUQJBA18njjXnG9RtLxoN3muvdU7PEy55QorUEsdAqdy

BIN
build/genesis/butterflynet.car
View File

Binary file not shown.

BIN
build/openrpc/full.json.gz
View File

Binary file not shown.

BIN
build/openrpc/miner.json.gz
View File

Binary file not shown.

BIN
build/openrpc/worker.json.gz
View File

Binary file not shown.

4
build/params_2k.go
View File

@ -17,7 +17,7 @@ import (
const BootstrappersFile = ""
const GenesisFile = ""
const GenesisNetworkVersion = network.Version14
const GenesisNetworkVersion = network.Version15
var UpgradeBreezeHeight = abi.ChainEpoch(-1)
@ -47,6 +47,8 @@ var UpgradeHyperdriveHeight = abi.ChainEpoch(-16)
var UpgradeChocolateHeight = abi.ChainEpoch(-17)
var UpgradeOhSnapHeight = abi.ChainEpoch(-18)
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}

9
build/params_butterfly.go
View File

@ -16,7 +16,7 @@ var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}
const GenesisNetworkVersion = network.Version13
const GenesisNetworkVersion = network.Version14
const BootstrappersFile = "butterflynet.pi"
const GenesisFile = "butterflynet.car"
@ -40,12 +40,17 @@ const UpgradeTrustHeight = -13
const UpgradeNorwegianHeight = -14
const UpgradeTurboHeight = -15
const UpgradeHyperdriveHeight = -16
const UpgradeChocolateHeight = 6360
const UpgradeChocolateHeight = -17
// 2022-01-17T19:00:00Z
const UpgradeOhSnapHeight = 30262
func init() {
policy.SetConsensusMinerMinPower(abi.NewStoragePower(2 << 30))
policy.SetSupportedProofTypes(
abi.RegisteredSealProof_StackedDrg512MiBV1,
abi.RegisteredSealProof_StackedDrg32GiBV1,
abi.RegisteredSealProof_StackedDrg64GiBV1,
)
SetAddressNetwork(address.Testnet)

2
build/params_calibnet.go
View File

@ -54,6 +54,8 @@ const UpgradeHyperdriveHeight = 420
const UpgradeChocolateHeight = 312746
const UpgradeOhSnapHeight = 99999999
func init() {
policy.SetConsensusMinerMinPower(abi.NewStoragePower(32 << 30))
policy.SetSupportedProofTypes(

1
build/params_interop.go
View File

@ -47,6 +47,7 @@ var UpgradeTurboHeight = abi.ChainEpoch(-15)
var UpgradeHyperdriveHeight = abi.ChainEpoch(-16)
var UpgradeChocolateHeight = abi.ChainEpoch(-17)
var UpgradeOhSnapHeight = abi.ChainEpoch(-18)
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,

10
build/params_mainnet.go
View File

@ -62,18 +62,20 @@ const UpgradeNorwegianHeight = 665280
const UpgradeTurboHeight = 712320
// 2021-06-30T22:00:00Z
var UpgradeHyperdriveHeight = abi.ChainEpoch(892800)
const UpgradeHyperdriveHeight = 892800
// 2021-10-26T13:30:00Z
var UpgradeChocolateHeight = abi.ChainEpoch(1231620)
const UpgradeChocolateHeight = 1231620
var UpgradeOhSnapHeight = abi.ChainEpoch(999999999999)
func init() {
if os.Getenv("LOTUS_USE_TEST_ADDRESSES") != "1" {
SetAddressNetwork(address.Mainnet)
}
if os.Getenv("LOTUS_DISABLE_CHOCOLATE") == "1" {
UpgradeChocolateHeight = math.MaxInt64
if os.Getenv("LOTUS_DISABLE_SNAPDEALS") == "1" {
UpgradeOhSnapHeight = math.MaxInt64
}
Devnet = false

2
build/params_shared_vals.go
View File

@ -34,7 +34,7 @@ const NewestNetworkVersion = network.Version{{.latestNetworkVersion}}
/* inline-gen start */
const NewestNetworkVersion = network.Version14
const NewestNetworkVersion = network.Version15
/* inline-gen end */

5
build/params_testground.go
View File

@ -99,6 +99,7 @@ var (
UpgradeTurboHeight abi.ChainEpoch = -14
UpgradeHyperdriveHeight abi.ChainEpoch = -15
UpgradeChocolateHeight abi.ChainEpoch = -16
UpgradeOhSnapHeight abi.ChainEpoch = -17
DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
@ -106,8 +107,8 @@ var (
GenesisNetworkVersion = network.Version0
NewestNetworkVersion = network.Version14
ActorUpgradeNetworkVersion = network.Version4
NewestNetworkVersion = network.Version15
ActorUpgradeNetworkVersion = network.Version15
Devnet = true
ZeroAddress = MustParseAddress("f3yaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaby2smx7a")

2
build/version.go
View File

@ -37,7 +37,7 @@ func BuildTypeString() string {
}
// BuildVersion is the local build version
const BuildVersion = "1.13.3-dev"
const BuildVersion = "1.15.0-dev"
func UserVersion() string {
if os.Getenv("LOTUS_VERSION_IGNORE_COMMIT") == "1" {

15
chain/actors/builtin/account/account.go
View File

@ -23,6 +23,8 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
func init() {
@ -50,6 +52,10 @@ func init() {
builtin.RegisterActorState(builtin6.AccountActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.AccountActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var Methods = builtin4.MethodsAccount
@ -75,6 +81,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.AccountActorCodeID:
return load6(store, act.Head)
case builtin7.AccountActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -100,6 +109,9 @@ func MakeState(store adt.Store, av actors.Version, addr address.Address) (State,
case actors.Version6:
return make6(store, addr)
case actors.Version7:
return make7(store, addr)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -125,6 +137,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.AccountActorCodeID, nil
case actors.Version7:
return builtin7.AccountActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

40
chain/actors/builtin/account/v7.go Normal file
View File

@ -0,0 +1,40 @@
package account
import (
"github.com/filecoin-project/go-address"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/actors/adt"
account7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/account"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store, addr address.Address) (State, error) {
out := state7{store: store}
out.State = account7.State{Address: addr}
return &out, nil
}
type state7 struct {
account7.State
store adt.Store
}
func (s *state7) PubkeyAddress() (address.Address, error) {
return s.Address, nil
}
func (s *state7) GetState() interface{} {
return &s.State
}

61
chain/actors/builtin/builtin.go
View File

@ -23,46 +23,50 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
smoothing6 "github.com/filecoin-project/specs-actors/v6/actors/util/smoothing"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
smoothing7 "github.com/filecoin-project/specs-actors/v7/actors/util/smoothing"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/cbor"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/types"
miner6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/miner"
proof6 "github.com/filecoin-project/specs-actors/v6/actors/runtime/proof"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
proof7 "github.com/filecoin-project/specs-actors/v7/actors/runtime/proof"
)
var SystemActorAddr = builtin6.SystemActorAddr
var BurntFundsActorAddr = builtin6.BurntFundsActorAddr
var CronActorAddr = builtin6.CronActorAddr
var SystemActorAddr = builtin7.SystemActorAddr
var BurntFundsActorAddr = builtin7.BurntFundsActorAddr
var CronActorAddr = builtin7.CronActorAddr
var SaftAddress = makeAddress("t0122")
var ReserveAddress = makeAddress("t090")
var RootVerifierAddress = makeAddress("t080")
var (
ExpectedLeadersPerEpoch = builtin6.ExpectedLeadersPerEpoch
ExpectedLeadersPerEpoch = builtin7.ExpectedLeadersPerEpoch
)
const (
EpochDurationSeconds = builtin6.EpochDurationSeconds
EpochsInDay = builtin6.EpochsInDay
SecondsInDay = builtin6.SecondsInDay
EpochDurationSeconds = builtin7.EpochDurationSeconds
EpochsInDay = builtin7.EpochsInDay
SecondsInDay = builtin7.SecondsInDay
)
const (
MethodSend = builtin6.MethodSend
MethodConstructor = builtin6.MethodConstructor
MethodSend = builtin7.MethodSend
MethodConstructor = builtin7.MethodConstructor
)
// These are all just type aliases across actor versions. In the future, that might change
// and we might need to do something fancier.
type SectorInfo = proof6.SectorInfo
type PoStProof = proof6.PoStProof
type SectorInfo = proof7.SectorInfo
type ExtendedSectorInfo = proof7.ExtendedSectorInfo
type PoStProof = proof7.PoStProof
type FilterEstimate = smoothing0.FilterEstimate
func QAPowerForWeight(size abi.SectorSize, duration abi.ChainEpoch, dealWeight, verifiedWeight abi.DealWeight) abi.StoragePower {
return miner6.QAPowerForWeight(size, duration, dealWeight, verifiedWeight)
return miner7.QAPowerForWeight(size, duration, dealWeight, verifiedWeight)
}
func FromV0FilterEstimate(v0 smoothing0.FilterEstimate) FilterEstimate {
@ -101,6 +105,12 @@ func FromV6FilterEstimate(v6 smoothing6.FilterEstimate) FilterEstimate {
}
func FromV7FilterEstimate(v7 smoothing7.FilterEstimate) FilterEstimate {
return (FilterEstimate)(v7)
}
type ActorStateLoader func(store adt.Store, root cid.Cid) (cbor.Marshaler, error)
var ActorStateLoaders = make(map[cid.Cid]ActorStateLoader)
@ -138,6 +148,9 @@ func ActorNameByCode(c cid.Cid) string {
case builtin6.IsBuiltinActor(c):
return builtin6.ActorNameByCode(c)
case builtin7.IsBuiltinActor(c):
return builtin7.ActorNameByCode(c)
default:
return "<unknown>"
}
@ -169,6 +182,10 @@ func IsBuiltinActor(c cid.Cid) bool {
return true
}
if builtin7.IsBuiltinActor(c) {
return true
}
return false
}
@ -198,6 +215,10 @@ func IsAccountActor(c cid.Cid) bool {
return true
}
if c == builtin7.AccountActorCodeID {
return true
}
return false
}
@ -227,6 +248,10 @@ func IsStorageMinerActor(c cid.Cid) bool {
return true
}
if c == builtin7.StorageMinerActorCodeID {
return true
}
return false
}
@ -256,6 +281,10 @@ func IsMultisigActor(c cid.Cid) bool {
return true
}
if c == builtin7.MultisigActorCodeID {
return true
}
return false
}
@ -285,6 +314,10 @@ func IsPaymentChannelActor(c cid.Cid) bool {
return true
}
if c == builtin7.PaymentChannelActorCodeID {
return true
}
return false
}

1
chain/actors/builtin/builtin.go.template
View File

@ -45,6 +45,7 @@ const (
// These are all just type aliases across actor versions. In the future, that might change
// and we might need to do something fancier.
type SectorInfo = proof{{.latestVersion}}.SectorInfo
type ExtendedSectorInfo = proof{{.latestVersion}}.ExtendedSectorInfo
type PoStProof = proof{{.latestVersion}}.PoStProof
type FilterEstimate = smoothing0.FilterEstimate

12
chain/actors/builtin/cron/cron.go
View File

@ -17,6 +17,8 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
func MakeState(store adt.Store, av actors.Version) (State, error) {
@ -40,6 +42,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -65,14 +70,17 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.CronActorCodeID, nil
case actors.Version7:
return builtin7.CronActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)
}
var (
Address = builtin6.CronActorAddr
Methods = builtin6.MethodsCron
Address = builtin7.CronActorAddr
Methods = builtin7.MethodsCron
)
type State interface {

35
chain/actors/builtin/cron/v7.go Normal file
View File

@ -0,0 +1,35 @@
package cron
import (
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/actors/adt"
cron7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/cron"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
out.State = *cron7.ConstructState(cron7.BuiltInEntries())
return &out, nil
}
type state7 struct {
cron7.State
store adt.Store
}
func (s *state7) GetState() interface{} {
return &s.State
}

19
chain/actors/builtin/init/init.go
View File

@ -25,6 +25,8 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
func init() {
@ -52,11 +54,15 @@ func init() {
builtin.RegisterActorState(builtin6.InitActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.InitActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var (
Address = builtin6.InitActorAddr
Methods = builtin6.MethodsInit
Address = builtin7.InitActorAddr
Methods = builtin7.MethodsInit
)
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -80,6 +86,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.InitActorCodeID:
return load6(store, act.Head)
case builtin7.InitActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -105,6 +114,9 @@ func MakeState(store adt.Store, av actors.Version, networkName string) (State, e
case actors.Version6:
return make6(store, networkName)
case actors.Version7:
return make7(store, networkName)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -130,6 +142,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.InitActorCodeID, nil
case actors.Version7:
return builtin7.InitActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

114
chain/actors/builtin/init/v7.go Normal file
View File

@ -0,0 +1,114 @@
package init
import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/node/modules/dtypes"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
init7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/init"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store, networkName string) (State, error) {
out := state7{store: store}
s, err := init7.ConstructState(store, networkName)
if err != nil {
return nil, err
}
out.State = *s
return &out, nil
}
type state7 struct {
init7.State
store adt.Store
}
func (s *state7) ResolveAddress(address address.Address) (address.Address, bool, error) {
return s.State.ResolveAddress(s.store, address)
}
func (s *state7) MapAddressToNewID(address address.Address) (address.Address, error) {
return s.State.MapAddressToNewID(s.store, address)
}
func (s *state7) ForEachActor(cb func(id abi.ActorID, address address.Address) error) error {
addrs, err := adt7.AsMap(s.store, s.State.AddressMap, builtin7.DefaultHamtBitwidth)
if err != nil {
return err
}
var actorID cbg.CborInt
return addrs.ForEach(&actorID, func(key string) error {
addr, err := address.NewFromBytes([]byte(key))
if err != nil {
return err
}
return cb(abi.ActorID(actorID), addr)
})
}
func (s *state7) NetworkName() (dtypes.NetworkName, error) {
return dtypes.NetworkName(s.State.NetworkName), nil
}
func (s *state7) SetNetworkName(name string) error {
s.State.NetworkName = name
return nil
}
func (s *state7) SetNextID(id abi.ActorID) error {
s.State.NextID = id
return nil
}
func (s *state7) Remove(addrs ...address.Address) (err error) {
m, err := adt7.AsMap(s.store, s.State.AddressMap, builtin7.DefaultHamtBitwidth)
if err != nil {
return err
}
for _, addr := range addrs {
if err = m.Delete(abi.AddrKey(addr)); err != nil {
return xerrors.Errorf("failed to delete entry for address: %s; err: %w", addr, err)
}
}
amr, err := m.Root()
if err != nil {
return xerrors.Errorf("failed to get address map root: %w", err)
}
s.State.AddressMap = amr
return nil
}
func (s *state7) SetAddressMap(mcid cid.Cid) error {
s.State.AddressMap = mcid
return nil
}
func (s *state7) AddressMap() (adt.Map, error) {
return adt7.AsMap(s.store, s.State.AddressMap, builtin7.DefaultHamtBitwidth)
}
func (s *state7) GetState() interface{} {
return &s.State
}

22
chain/actors/builtin/market/market.go
View File

@ -25,6 +25,8 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
@ -56,11 +58,15 @@ func init() {
builtin.RegisterActorState(builtin6.StorageMarketActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.StorageMarketActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var (
Address = builtin6.StorageMarketActorAddr
Methods = builtin6.MethodsMarket
Address = builtin7.StorageMarketActorAddr
Methods = builtin7.MethodsMarket
)
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -84,6 +90,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.StorageMarketActorCodeID:
return load6(store, act.Head)
case builtin7.StorageMarketActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -109,6 +118,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -134,6 +146,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.StorageMarketActorCodeID, nil
case actors.Version7:
return builtin7.StorageMarketActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)
@ -211,6 +226,9 @@ func DecodePublishStorageDealsReturn(b []byte, nv network.Version) (PublishStora
case actors.Version6:
return decodePublishStorageDealsReturn6(b)
case actors.Version7:
return decodePublishStorageDealsReturn7(b)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}

252
chain/actors/builtin/market/v7.go Normal file
View File

@ -0,0 +1,252 @@
package market
import (
"bytes"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/types"
market7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/market"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
s, err := market7.ConstructState(store)
if err != nil {
return nil, err
}
out.State = *s
return &out, nil
}
type state7 struct {
market7.State
store adt.Store
}
func (s *state7) TotalLocked() (abi.TokenAmount, error) {
fml := types.BigAdd(s.TotalClientLockedCollateral, s.TotalProviderLockedCollateral)
fml = types.BigAdd(fml, s.TotalClientStorageFee)
return fml, nil
}
func (s *state7) BalancesChanged(otherState State) (bool, error) {
otherState7, ok := otherState.(*state7)
if !ok {
// there's no way to compare different versions of the state, so let's
// just say that means the state of balances has changed
return true, nil
}
return !s.State.EscrowTable.Equals(otherState7.State.EscrowTable) || !s.State.LockedTable.Equals(otherState7.State.LockedTable), nil
}
func (s *state7) StatesChanged(otherState State) (bool, error) {
otherState7, ok := otherState.(*state7)
if !ok {
// there's no way to compare different versions of the state, so let's
// just say that means the state of balances has changed
return true, nil
}
return !s.State.States.Equals(otherState7.State.States), nil
}
func (s *state7) States() (DealStates, error) {
stateArray, err := adt7.AsArray(s.store, s.State.States, market7.StatesAmtBitwidth)
if err != nil {
return nil, err
}
return &dealStates7{stateArray}, nil
}
func (s *state7) ProposalsChanged(otherState State) (bool, error) {
otherState7, ok := otherState.(*state7)
if !ok {
// there's no way to compare different versions of the state, so let's
// just say that means the state of balances has changed
return true, nil
}
return !s.State.Proposals.Equals(otherState7.State.Proposals), nil
}
func (s *state7) Proposals() (DealProposals, error) {
proposalArray, err := adt7.AsArray(s.store, s.State.Proposals, market7.ProposalsAmtBitwidth)
if err != nil {
return nil, err
}
return &dealProposals7{proposalArray}, nil
}
func (s *state7) EscrowTable() (BalanceTable, error) {
bt, err := adt7.AsBalanceTable(s.store, s.State.EscrowTable)
if err != nil {
return nil, err
}
return &balanceTable7{bt}, nil
}
func (s *state7) LockedTable() (BalanceTable, error) {
bt, err := adt7.AsBalanceTable(s.store, s.State.LockedTable)
if err != nil {
return nil, err
}
return &balanceTable7{bt}, nil
}
func (s *state7) VerifyDealsForActivation(
minerAddr address.Address, deals []abi.DealID, currEpoch, sectorExpiry abi.ChainEpoch,
) (weight, verifiedWeight abi.DealWeight, err error) {
w, vw, _, err := market7.ValidateDealsForActivation(&s.State, s.store, deals, minerAddr, sectorExpiry, currEpoch)
return w, vw, err
}
func (s *state7) NextID() (abi.DealID, error) {
return s.State.NextID, nil
}
type balanceTable7 struct {
*adt7.BalanceTable
}
func (bt *balanceTable7) ForEach(cb func(address.Address, abi.TokenAmount) error) error {
asMap := (*adt7.Map)(bt.BalanceTable)
var ta abi.TokenAmount
return asMap.ForEach(&ta, func(key string) error {
a, err := address.NewFromBytes([]byte(key))
if err != nil {
return err
}
return cb(a, ta)
})
}
type dealStates7 struct {
adt.Array
}
func (s *dealStates7) Get(dealID abi.DealID) (*DealState, bool, error) {
var deal7 market7.DealState
found, err := s.Array.Get(uint64(dealID), &deal7)
if err != nil {
return nil, false, err
}
if !found {
return nil, false, nil
}
deal := fromV7DealState(deal7)
return &deal, true, nil
}
func (s *dealStates7) ForEach(cb func(dealID abi.DealID, ds DealState) error) error {
var ds7 market7.DealState
return s.Array.ForEach(&ds7, func(idx int64) error {
return cb(abi.DealID(idx), fromV7DealState(ds7))
})
}
func (s *dealStates7) decode(val *cbg.Deferred) (*DealState, error) {
var ds7 market7.DealState
if err := ds7.UnmarshalCBOR(bytes.NewReader(val.Raw)); err != nil {
return nil, err
}
ds := fromV7DealState(ds7)
return &ds, nil
}
func (s *dealStates7) array() adt.Array {
return s.Array
}
func fromV7DealState(v7 market7.DealState) DealState {
return (DealState)(v7)
}
type dealProposals7 struct {
adt.Array
}
func (s *dealProposals7) Get(dealID abi.DealID) (*DealProposal, bool, error) {
var proposal7 market7.DealProposal
found, err := s.Array.Get(uint64(dealID), &proposal7)
if err != nil {
return nil, false, err
}
if !found {
return nil, false, nil
}
proposal := fromV7DealProposal(proposal7)
return &proposal, true, nil
}
func (s *dealProposals7) ForEach(cb func(dealID abi.DealID, dp DealProposal) error) error {
var dp7 market7.DealProposal
return s.Array.ForEach(&dp7, func(idx int64) error {
return cb(abi.DealID(idx), fromV7DealProposal(dp7))
})
}
func (s *dealProposals7) decode(val *cbg.Deferred) (*DealProposal, error) {
var dp7 market7.DealProposal
if err := dp7.UnmarshalCBOR(bytes.NewReader(val.Raw)); err != nil {
return nil, err
}
dp := fromV7DealProposal(dp7)
return &dp, nil
}
func (s *dealProposals7) array() adt.Array {
return s.Array
}
func fromV7DealProposal(v7 market7.DealProposal) DealProposal {
return (DealProposal)(v7)
}
func (s *state7) GetState() interface{} {
return &s.State
}
var _ PublishStorageDealsReturn = (*publishStorageDealsReturn7)(nil)
func decodePublishStorageDealsReturn7(b []byte) (PublishStorageDealsReturn, error) {
var retval market7.PublishStorageDealsReturn
if err := retval.UnmarshalCBOR(bytes.NewReader(b)); err != nil {
return nil, xerrors.Errorf("failed to unmarshal PublishStorageDealsReturn: %w", err)
}
return &publishStorageDealsReturn7{retval}, nil
}
type publishStorageDealsReturn7 struct {
market7.PublishStorageDealsReturn
}
func (r *publishStorageDealsReturn7) IsDealValid(index uint64) (bool, error) {
return r.ValidDeals.IsSet(index)
}
func (r *publishStorageDealsReturn7) DealIDs() ([]abi.DealID, error) {
return r.IDs, nil
}

4
chain/actors/builtin/miner/actor.go.template
View File

@ -23,6 +23,7 @@ import (
miner2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/miner"
miner3 "github.com/filecoin-project/specs-actors/v3/actors/builtin/miner"
miner5 "github.com/filecoin-project/specs-actors/v5/actors/builtin/miner"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
{{range .versions}}
builtin{{.}} "github.com/filecoin-project/specs-actors{{import .}}actors/builtin"
{{end}}
@ -177,6 +178,7 @@ type SectorOnChainInfo struct {
InitialPledge abi.TokenAmount
ExpectedDayReward abi.TokenAmount
ExpectedStoragePledge abi.TokenAmount
SectorKeyCID *cid.Cid
}
type SectorPreCommitInfo = miner0.SectorPreCommitInfo
@ -192,6 +194,7 @@ type SectorPreCommitOnChainInfo struct {
type PoStPartition = miner0.PoStPartition
type RecoveryDeclaration = miner0.RecoveryDeclaration
type FaultDeclaration = miner0.FaultDeclaration
type ReplicaUpdate = miner7.ReplicaUpdate
// Params
type DeclareFaultsParams = miner0.DeclareFaultsParams
@ -200,6 +203,7 @@ type SubmitWindowedPoStParams = miner0.SubmitWindowedPoStParams
type ProveCommitSectorParams = miner0.ProveCommitSectorParams
type DisputeWindowedPoStParams = miner3.DisputeWindowedPoStParams
type ProveCommitAggregateParams = miner5.ProveCommitAggregateParams
type ProveReplicaUpdatesParams = miner7.ProveReplicaUpdatesParams
func PreferredSealProofTypeFromWindowPoStType(nver network.Version, proof abi.RegisteredPoStProof) (abi.RegisteredSealProof, error) {
// We added support for the new proofs in network version 7, and removed support for the old

21
chain/actors/builtin/miner/miner.go
View File

@ -23,6 +23,7 @@ import (
miner2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/miner"
miner3 "github.com/filecoin-project/specs-actors/v3/actors/builtin/miner"
miner5 "github.com/filecoin-project/specs-actors/v5/actors/builtin/miner"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
@ -35,6 +36,8 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
func init() {
@ -63,9 +66,13 @@ func init() {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.StorageMinerActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var Methods = builtin6.MethodsMiner
var Methods = builtin7.MethodsMiner
// Unchanged between v0, v2, v3, v4, and v5 actors
var WPoStProvingPeriod = miner0.WPoStProvingPeriod
@ -102,6 +109,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.StorageMinerActorCodeID:
return load6(store, act.Head)
case builtin7.StorageMinerActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -127,6 +137,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -152,6 +165,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.StorageMinerActorCodeID, nil
case actors.Version7:
return builtin7.StorageMinerActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)
@ -251,6 +267,7 @@ type SectorOnChainInfo struct {
InitialPledge abi.TokenAmount
ExpectedDayReward abi.TokenAmount
ExpectedStoragePledge abi.TokenAmount
SectorKeyCID *cid.Cid
}
type SectorPreCommitInfo = miner0.SectorPreCommitInfo
@ -266,6 +283,7 @@ type SectorPreCommitOnChainInfo struct {
type PoStPartition = miner0.PoStPartition
type RecoveryDeclaration = miner0.RecoveryDeclaration
type FaultDeclaration = miner0.FaultDeclaration
type ReplicaUpdate = miner7.ReplicaUpdate
// Params
type DeclareFaultsParams = miner0.DeclareFaultsParams
@ -274,6 +292,7 @@ type SubmitWindowedPoStParams = miner0.SubmitWindowedPoStParams
type ProveCommitSectorParams = miner0.ProveCommitSectorParams
type DisputeWindowedPoStParams = miner3.DisputeWindowedPoStParams
type ProveCommitAggregateParams = miner5.ProveCommitAggregateParams
type ProveReplicaUpdatesParams = miner7.ProveReplicaUpdatesParams
func PreferredSealProofTypeFromWindowPoStType(nver network.Version, proof abi.RegisteredPoStProof) (abi.RegisteredSealProof, error) {
// We added support for the new proofs in network version 7, and removed support for the old

23
chain/actors/builtin/miner/state.go.template
View File

@ -138,11 +138,22 @@ func (s *state{{.v}}) GetSectorExpiration(num abi.SectorNumber) (*SectorExpirati
return nil, err
}
// NOTE: this can be optimized significantly.
// 1. If the sector is non-faulty, it will either expire on-time (can be
{{if (ge .v 7) -}}
// 1. If the sector is non-faulty, it will expire on-time (can be
// learned from the sector info).
{{- else -}}
// 1. If the sector is non-faulty, it will either expire on-time (can be
// learned from the sector info), or in the next quantized expiration
// epoch (i.e., the first element in the partition's expiration queue.
{{- end}}
{{if (ge .v 6) -}}
// 2. If it's faulty, it will expire early within the first 42 entries
// of the expiration queue.
{{- else -}}
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
{{- end}}
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner{{.v}}.Deadline) error {
@ -554,8 +565,7 @@ func (p *partition{{.v}}) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV{{.v}}SectorOnChainInfo(v{{.v}} miner{{.v}}.SectorOnChainInfo) SectorOnChainInfo {
{{if (ge .v 2)}}
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v{{.v}}.SectorNumber,
SealProof: v{{.v}}.SealProof,
SealedCID: v{{.v}}.SealedCID,
@ -567,10 +577,11 @@ func fromV{{.v}}SectorOnChainInfo(v{{.v}} miner{{.v}}.SectorOnChainInfo) SectorO
InitialPledge: v{{.v}}.InitialPledge,
ExpectedDayReward: v{{.v}}.ExpectedDayReward,
ExpectedStoragePledge: v{{.v}}.ExpectedStoragePledge,
{{if (ge .v 7)}}
SectorKeyCID: v{{.v}}.SectorKeyCID,
{{end}}
}
{{else}}
return (SectorOnChainInfo)(v0)
{{end}}
return info
}
func fromV{{.v}}SectorPreCommitOnChainInfo(v{{.v}} miner{{.v}}.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

18
chain/actors/builtin/miner/v0.go
View File

@ -140,6 +140,7 @@ func (s *state0) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner0.Deadline) error {
@ -505,9 +506,20 @@ func (p *partition0) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV0SectorOnChainInfo(v0 miner0.SectorOnChainInfo) SectorOnChainInfo {
return (SectorOnChainInfo)(v0)
info := SectorOnChainInfo{
SectorNumber: v0.SectorNumber,
SealProof: v0.SealProof,
SealedCID: v0.SealedCID,
DealIDs: v0.DealIDs,
Activation: v0.Activation,
Expiration: v0.Expiration,
DealWeight: v0.DealWeight,
VerifiedDealWeight: v0.VerifiedDealWeight,
InitialPledge: v0.InitialPledge,
ExpectedDayReward: v0.ExpectedDayReward,
ExpectedStoragePledge: v0.ExpectedStoragePledge,
}
return info
}
func fromV0SectorPreCommitOnChainInfo(v0 miner0.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

6
chain/actors/builtin/miner/v2.go
View File

@ -138,6 +138,7 @@ func (s *state2) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner2.Deadline) error {
@ -535,8 +536,7 @@ func (p *partition2) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV2SectorOnChainInfo(v2 miner2.SectorOnChainInfo) SectorOnChainInfo {
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v2.SectorNumber,
SealProof: v2.SealProof,
SealedCID: v2.SealedCID,
@ -549,7 +549,7 @@ func fromV2SectorOnChainInfo(v2 miner2.SectorOnChainInfo) SectorOnChainInfo {
ExpectedDayReward: v2.ExpectedDayReward,
ExpectedStoragePledge: v2.ExpectedStoragePledge,
}
return info
}
func fromV2SectorPreCommitOnChainInfo(v2 miner2.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

6
chain/actors/builtin/miner/v3.go
View File

@ -140,6 +140,7 @@ func (s *state3) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner3.Deadline) error {
@ -536,8 +537,7 @@ func (p *partition3) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV3SectorOnChainInfo(v3 miner3.SectorOnChainInfo) SectorOnChainInfo {
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v3.SectorNumber,
SealProof: v3.SealProof,
SealedCID: v3.SealedCID,
@ -550,7 +550,7 @@ func fromV3SectorOnChainInfo(v3 miner3.SectorOnChainInfo) SectorOnChainInfo {
ExpectedDayReward: v3.ExpectedDayReward,
ExpectedStoragePledge: v3.ExpectedStoragePledge,
}
return info
}
func fromV3SectorPreCommitOnChainInfo(v3 miner3.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

6
chain/actors/builtin/miner/v4.go
View File

@ -140,6 +140,7 @@ func (s *state4) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner4.Deadline) error {
@ -536,8 +537,7 @@ func (p *partition4) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV4SectorOnChainInfo(v4 miner4.SectorOnChainInfo) SectorOnChainInfo {
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v4.SectorNumber,
SealProof: v4.SealProof,
SealedCID: v4.SealedCID,
@ -550,7 +550,7 @@ func fromV4SectorOnChainInfo(v4 miner4.SectorOnChainInfo) SectorOnChainInfo {
ExpectedDayReward: v4.ExpectedDayReward,
ExpectedStoragePledge: v4.ExpectedStoragePledge,
}
return info
}
func fromV4SectorPreCommitOnChainInfo(v4 miner4.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

6
chain/actors/builtin/miner/v5.go
View File

@ -140,6 +140,7 @@ func (s *state5) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner5.Deadline) error {
@ -536,8 +537,7 @@ func (p *partition5) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV5SectorOnChainInfo(v5 miner5.SectorOnChainInfo) SectorOnChainInfo {
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v5.SectorNumber,
SealProof: v5.SealProof,
SealedCID: v5.SealedCID,
@ -550,7 +550,7 @@ func fromV5SectorOnChainInfo(v5 miner5.SectorOnChainInfo) SectorOnChainInfo {
ExpectedDayReward: v5.ExpectedDayReward,
ExpectedStoragePledge: v5.ExpectedStoragePledge,
}
return info
}
func fromV5SectorPreCommitOnChainInfo(v5 miner5.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

8
chain/actors/builtin/miner/v6.go
View File

@ -138,8 +138,9 @@ func (s *state6) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, e
// 1. If the sector is non-faulty, it will either expire on-time (can be
// learned from the sector info), or in the next quantized expiration
// epoch (i.e., the first element in the partition's expiration queue.
// 2. If it's faulty, it will expire early within the first 14 entries
// 2. If it's faulty, it will expire early within the first 42 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner6.Deadline) error {
@ -536,8 +537,7 @@ func (p *partition6) UnprovenSectors() (bitfield.BitField, error) {
}
func fromV6SectorOnChainInfo(v6 miner6.SectorOnChainInfo) SectorOnChainInfo {
return SectorOnChainInfo{
info := SectorOnChainInfo{
SectorNumber: v6.SectorNumber,
SealProof: v6.SealProof,
SealedCID: v6.SealedCID,
@ -550,7 +550,7 @@ func fromV6SectorOnChainInfo(v6 miner6.SectorOnChainInfo) SectorOnChainInfo {
ExpectedDayReward: v6.ExpectedDayReward,
ExpectedStoragePledge: v6.ExpectedStoragePledge,
}
return info
}
func fromV6SectorPreCommitOnChainInfo(v6 miner6.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {

571
chain/actors/builtin/miner/v7.go Normal file
View File

@ -0,0 +1,571 @@
package miner
import (
"bytes"
"errors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
rle "github.com/filecoin-project/go-bitfield/rle"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/dline"
"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors/adt"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
out.State = miner7.State{}
return &out, nil
}
type state7 struct {
miner7.State
store adt.Store
}
type deadline7 struct {
miner7.Deadline
store adt.Store
}
type partition7 struct {
miner7.Partition
store adt.Store
}
func (s *state7) AvailableBalance(bal abi.TokenAmount) (available abi.TokenAmount, err error) {
defer func() {
if r := recover(); r != nil {
err = xerrors.Errorf("failed to get available balance: %w", r)
available = abi.NewTokenAmount(0)
}
}()
// this panics if the miner doesnt have enough funds to cover their locked pledge
available, err = s.GetAvailableBalance(bal)
return available, err
}
func (s *state7) VestedFunds(epoch abi.ChainEpoch) (abi.TokenAmount, error) {
return s.CheckVestedFunds(s.store, epoch)
}
func (s *state7) LockedFunds() (LockedFunds, error) {
return LockedFunds{
VestingFunds: s.State.LockedFunds,
InitialPledgeRequirement: s.State.InitialPledge,
PreCommitDeposits: s.State.PreCommitDeposits,
}, nil
}
func (s *state7) FeeDebt() (abi.TokenAmount, error) {
return s.State.FeeDebt, nil
}
func (s *state7) InitialPledge() (abi.TokenAmount, error) {
return s.State.InitialPledge, nil
}
func (s *state7) PreCommitDeposits() (abi.TokenAmount, error) {
return s.State.PreCommitDeposits, nil
}
func (s *state7) GetSector(num abi.SectorNumber) (*SectorOnChainInfo, error) {
info, ok, err := s.State.GetSector(s.store, num)
if !ok || err != nil {
return nil, err
}
ret := fromV7SectorOnChainInfo(*info)
return &ret, nil
}
func (s *state7) FindSector(num abi.SectorNumber) (*SectorLocation, error) {
dlIdx, partIdx, err := s.State.FindSector(s.store, num)
if err != nil {
return nil, err
}
return &SectorLocation{
Deadline: dlIdx,
Partition: partIdx,
}, nil
}
func (s *state7) NumLiveSectors() (uint64, error) {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
return 0, err
}
var total uint64
if err := dls.ForEach(s.store, func(dlIdx uint64, dl *miner7.Deadline) error {
total += dl.LiveSectors
return nil
}); err != nil {
return 0, err
}
return total, nil
}
// GetSectorExpiration returns the effective expiration of the given sector.
//
// If the sector does not expire early, the Early expiration field is 0.
func (s *state7) GetSectorExpiration(num abi.SectorNumber) (*SectorExpiration, error) {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
return nil, err
}
// NOTE: this can be optimized significantly.
// 1. If the sector is non-faulty, it will expire on-time (can be
// learned from the sector info).
// 2. If it's faulty, it will expire early within the first 42 entries
// of the expiration queue.
stopErr := errors.New("stop")
out := SectorExpiration{}
err = dls.ForEach(s.store, func(dlIdx uint64, dl *miner7.Deadline) error {
partitions, err := dl.PartitionsArray(s.store)
if err != nil {
return err
}
quant := s.State.QuantSpecForDeadline(dlIdx)
var part miner7.Partition
return partitions.ForEach(&part, func(partIdx int64) error {
if found, err := part.Sectors.IsSet(uint64(num)); err != nil {
return err
} else if !found {
return nil
}
if found, err := part.Terminated.IsSet(uint64(num)); err != nil {
return err
} else if found {
// already terminated
return stopErr
}
q, err := miner7.LoadExpirationQueue(s.store, part.ExpirationsEpochs, quant, miner7.PartitionExpirationAmtBitwidth)
if err != nil {
return err
}
var exp miner7.ExpirationSet
return q.ForEach(&exp, func(epoch int64) error {
if early, err := exp.EarlySectors.IsSet(uint64(num)); err != nil {
return err
} else if early {
out.Early = abi.ChainEpoch(epoch)
return nil
}
if onTime, err := exp.OnTimeSectors.IsSet(uint64(num)); err != nil {
return err
} else if onTime {
out.OnTime = abi.ChainEpoch(epoch)
return stopErr
}
return nil
})
})
})
if err == stopErr {
err = nil
}
if err != nil {
return nil, err
}
if out.Early == 0 && out.OnTime == 0 {
return nil, xerrors.Errorf("failed to find sector %d", num)
}
return &out, nil
}
func (s *state7) GetPrecommittedSector(num abi.SectorNumber) (*SectorPreCommitOnChainInfo, error) {
info, ok, err := s.State.GetPrecommittedSector(s.store, num)
if !ok || err != nil {
return nil, err
}
ret := fromV7SectorPreCommitOnChainInfo(*info)
return &ret, nil
}
func (s *state7) ForEachPrecommittedSector(cb func(SectorPreCommitOnChainInfo) error) error {
precommitted, err := adt7.AsMap(s.store, s.State.PreCommittedSectors, builtin7.DefaultHamtBitwidth)
if err != nil {
return err
}
var info miner7.SectorPreCommitOnChainInfo
if err := precommitted.ForEach(&info, func(_ string) error {
return cb(fromV7SectorPreCommitOnChainInfo(info))
}); err != nil {
return err
}
return nil
}
func (s *state7) LoadSectors(snos *bitfield.BitField) ([]*SectorOnChainInfo, error) {
sectors, err := miner7.LoadSectors(s.store, s.State.Sectors)
if err != nil {
return nil, err
}
// If no sector numbers are specified, load all.
if snos == nil {
infos := make([]*SectorOnChainInfo, 0, sectors.Length())
var info7 miner7.SectorOnChainInfo
if err := sectors.ForEach(&info7, func(_ int64) error {
info := fromV7SectorOnChainInfo(info7)
infos = append(infos, &info)
return nil
}); err != nil {
return nil, err
}
return infos, nil
}
// Otherwise, load selected.
infos7, err := sectors.Load(*snos)
if err != nil {
return nil, err
}
infos := make([]*SectorOnChainInfo, len(infos7))
for i, info7 := range infos7 {
info := fromV7SectorOnChainInfo(*info7)
infos[i] = &info
}
return infos, nil
}
func (s *state7) loadAllocatedSectorNumbers() (bitfield.BitField, error) {
var allocatedSectors bitfield.BitField
err := s.store.Get(s.store.Context(), s.State.AllocatedSectors, &allocatedSectors)
return allocatedSectors, err
}
func (s *state7) IsAllocated(num abi.SectorNumber) (bool, error) {
allocatedSectors, err := s.loadAllocatedSectorNumbers()
if err != nil {
return false, err
}
return allocatedSectors.IsSet(uint64(num))
}
func (s *state7) GetProvingPeriodStart() (abi.ChainEpoch, error) {
return s.State.ProvingPeriodStart, nil
}
func (s *state7) UnallocatedSectorNumbers(count int) ([]abi.SectorNumber, error) {
allocatedSectors, err := s.loadAllocatedSectorNumbers()
if err != nil {
return nil, err
}
allocatedRuns, err := allocatedSectors.RunIterator()
if err != nil {
return nil, err
}
unallocatedRuns, err := rle.Subtract(
&rle.RunSliceIterator{Runs: []rle.Run{{Val: true, Len: abi.MaxSectorNumber}}},
allocatedRuns,
)
if err != nil {
return nil, err
}
iter, err := rle.BitsFromRuns(unallocatedRuns)
if err != nil {
return nil, err
}
sectors := make([]abi.SectorNumber, 0, count)
for iter.HasNext() && len(sectors) < count {
nextNo, err := iter.Next()
if err != nil {
return nil, err
}
sectors = append(sectors, abi.SectorNumber(nextNo))
}
return sectors, nil
}
func (s *state7) GetAllocatedSectors() (*bitfield.BitField, error) {
var allocatedSectors bitfield.BitField
if err := s.store.Get(s.store.Context(), s.State.AllocatedSectors, &allocatedSectors); err != nil {
return nil, err
}
return &allocatedSectors, nil
}
func (s *state7) LoadDeadline(idx uint64) (Deadline, error) {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
return nil, err
}
dl, err := dls.LoadDeadline(s.store, idx)
if err != nil {
return nil, err
}
return &deadline7{*dl, s.store}, nil
}
func (s *state7) ForEachDeadline(cb func(uint64, Deadline) error) error {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
return err
}
return dls.ForEach(s.store, func(i uint64, dl *miner7.Deadline) error {
return cb(i, &deadline7{*dl, s.store})
})
}
func (s *state7) NumDeadlines() (uint64, error) {
return miner7.WPoStPeriodDeadlines, nil
}
func (s *state7) DeadlinesChanged(other State) (bool, error) {
other7, ok := other.(*state7)
if !ok {
// treat an upgrade as a change, always
return true, nil
}
return !s.State.Deadlines.Equals(other7.Deadlines), nil
}
func (s *state7) MinerInfoChanged(other State) (bool, error) {
other0, ok := other.(*state7)
if !ok {
// treat an upgrade as a change, always
return true, nil
}
return !s.State.Info.Equals(other0.State.Info), nil
}
func (s *state7) Info() (MinerInfo, error) {
info, err := s.State.GetInfo(s.store)
if err != nil {
return MinerInfo{}, err
}
var pid *peer.ID
if peerID, err := peer.IDFromBytes(info.PeerId); err == nil {
pid = &peerID
}
mi := MinerInfo{
Owner: info.Owner,
Worker: info.Worker,
ControlAddresses: info.ControlAddresses,
NewWorker: address.Undef,
WorkerChangeEpoch: -1,
PeerId: pid,
Multiaddrs: info.Multiaddrs,
WindowPoStProofType: info.WindowPoStProofType,
SectorSize: info.SectorSize,
WindowPoStPartitionSectors: info.WindowPoStPartitionSectors,
ConsensusFaultElapsed: info.ConsensusFaultElapsed,
}
if info.PendingWorkerKey != nil {
mi.NewWorker = info.PendingWorkerKey.NewWorker
mi.WorkerChangeEpoch = info.PendingWorkerKey.EffectiveAt
}
return mi, nil
}
func (s *state7) DeadlineInfo(epoch abi.ChainEpoch) (*dline.Info, error) {
return s.State.RecordedDeadlineInfo(epoch), nil
}
func (s *state7) DeadlineCronActive() (bool, error) {
return s.State.DeadlineCronActive, nil
}
func (s *state7) sectors() (adt.Array, error) {
return adt7.AsArray(s.store, s.Sectors, miner7.SectorsAmtBitwidth)
}
func (s *state7) decodeSectorOnChainInfo(val *cbg.Deferred) (SectorOnChainInfo, error) {
var si miner7.SectorOnChainInfo
err := si.UnmarshalCBOR(bytes.NewReader(val.Raw))
if err != nil {
return SectorOnChainInfo{}, err
}
return fromV7SectorOnChainInfo(si), nil
}
func (s *state7) precommits() (adt.Map, error) {
return adt7.AsMap(s.store, s.PreCommittedSectors, builtin7.DefaultHamtBitwidth)
}
func (s *state7) decodeSectorPreCommitOnChainInfo(val *cbg.Deferred) (SectorPreCommitOnChainInfo, error) {
var sp miner7.SectorPreCommitOnChainInfo
err := sp.UnmarshalCBOR(bytes.NewReader(val.Raw))
if err != nil {
return SectorPreCommitOnChainInfo{}, err
}
return fromV7SectorPreCommitOnChainInfo(sp), nil
}
func (s *state7) EraseAllUnproven() error {
dls, err := s.State.LoadDeadlines(s.store)
if err != nil {
return err
}
err = dls.ForEach(s.store, func(dindx uint64, dl *miner7.Deadline) error {
ps, err := dl.PartitionsArray(s.store)
if err != nil {
return err
}
var part miner7.Partition
err = ps.ForEach(&part, func(pindx int64) error {
_ = part.ActivateUnproven()
err = ps.Set(uint64(pindx), &part)
return nil
})
if err != nil {
return err
}
dl.Partitions, err = ps.Root()
if err != nil {
return err
}
return dls.UpdateDeadline(s.store, dindx, dl)
})
if err != nil {
return err
}
return s.State.SaveDeadlines(s.store, dls)
}
func (d *deadline7) LoadPartition(idx uint64) (Partition, error) {
p, err := d.Deadline.LoadPartition(d.store, idx)
if err != nil {
return nil, err
}
return &partition7{*p, d.store}, nil
}
func (d *deadline7) ForEachPartition(cb func(uint64, Partition) error) error {
ps, err := d.Deadline.PartitionsArray(d.store)
if err != nil {
return err
}
var part miner7.Partition
return ps.ForEach(&part, func(i int64) error {
return cb(uint64(i), &partition7{part, d.store})
})
}
func (d *deadline7) PartitionsChanged(other Deadline) (bool, error) {
other7, ok := other.(*deadline7)
if !ok {
// treat an upgrade as a change, always
return true, nil
}
return !d.Deadline.Partitions.Equals(other7.Deadline.Partitions), nil
}
func (d *deadline7) PartitionsPoSted() (bitfield.BitField, error) {
return d.Deadline.PartitionsPoSted, nil
}
func (d *deadline7) DisputableProofCount() (uint64, error) {
ops, err := d.OptimisticProofsSnapshotArray(d.store)
if err != nil {
return 0, err
}
return ops.Length(), nil
}
func (p *partition7) AllSectors() (bitfield.BitField, error) {
return p.Partition.Sectors, nil
}
func (p *partition7) FaultySectors() (bitfield.BitField, error) {
return p.Partition.Faults, nil
}
func (p *partition7) RecoveringSectors() (bitfield.BitField, error) {
return p.Partition.Recoveries, nil
}
func (p *partition7) UnprovenSectors() (bitfield.BitField, error) {
return p.Partition.Unproven, nil
}
func fromV7SectorOnChainInfo(v7 miner7.SectorOnChainInfo) SectorOnChainInfo {
info := SectorOnChainInfo{
SectorNumber: v7.SectorNumber,
SealProof: v7.SealProof,
SealedCID: v7.SealedCID,
DealIDs: v7.DealIDs,
Activation: v7.Activation,
Expiration: v7.Expiration,
DealWeight: v7.DealWeight,
VerifiedDealWeight: v7.VerifiedDealWeight,
InitialPledge: v7.InitialPledge,
ExpectedDayReward: v7.ExpectedDayReward,
ExpectedStoragePledge: v7.ExpectedStoragePledge,
SectorKeyCID: v7.SectorKeyCID,
}
return info
}
func fromV7SectorPreCommitOnChainInfo(v7 miner7.SectorPreCommitOnChainInfo) SectorPreCommitOnChainInfo {
return SectorPreCommitOnChainInfo{
Info: (SectorPreCommitInfo)(v7.Info),
PreCommitDeposit: v7.PreCommitDeposit,
PreCommitEpoch: v7.PreCommitEpoch,
DealWeight: v7.DealWeight,
VerifiedDealWeight: v7.VerifiedDealWeight,
}
}
func (s *state7) GetState() interface{} {
return &s.State
}

71
chain/actors/builtin/multisig/message7.go Normal file
View File

@ -0,0 +1,71 @@
package multisig
import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
init7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/init"
multisig7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/multisig"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message7 struct{ message0 }
func (m message7) Create(
signers []address.Address, threshold uint64,
unlockStart, unlockDuration abi.ChainEpoch,
initialAmount abi.TokenAmount,
) (*types.Message, error) {
lenAddrs := uint64(len(signers))
if lenAddrs < threshold {
return nil, xerrors.Errorf("cannot require signing of more addresses than provided for multisig")
}
if threshold == 0 {
threshold = lenAddrs
}
if m.from == address.Undef {
return nil, xerrors.Errorf("must provide source address")
}
// Set up constructor parameters for multisig
msigParams := &multisig7.ConstructorParams{
Signers: signers,
NumApprovalsThreshold: threshold,
UnlockDuration: unlockDuration,
StartEpoch: unlockStart,
}
enc, actErr := actors.SerializeParams(msigParams)
if actErr != nil {
return nil, actErr
}
// new actors are created by invoking 'exec' on the init actor with the constructor params
execParams := &init7.ExecParams{
CodeCID: builtin7.MultisigActorCodeID,
ConstructorParams: enc,
}
enc, actErr = actors.SerializeParams(execParams)
if actErr != nil {
return nil, actErr
}
return &types.Message{
To: init_.Address,
From: m.from,
Method: builtin7.MethodsInit.Exec,
Params: enc,
Value: initialAmount,
}, nil
}

32
chain/actors/builtin/multisig/multisig.go
View File

@ -13,7 +13,7 @@ import (
"github.com/ipfs/go-cid"
msig0 "github.com/filecoin-project/specs-actors/actors/builtin/multisig"
msig6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/multisig"
msig7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/multisig"
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
@ -27,6 +27,8 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
@ -58,6 +60,10 @@ func init() {
builtin.RegisterActorState(builtin6.MultisigActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.MultisigActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -81,6 +87,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.MultisigActorCodeID:
return load6(store, act.Head)
case builtin7.MultisigActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -106,6 +115,9 @@ func MakeState(store adt.Store, av actors.Version, signers []address.Address, th
case actors.Version6:
return make6(store, signers, threshold, startEpoch, unlockDuration, initialBalance)
case actors.Version7:
return make7(store, signers, threshold, startEpoch, unlockDuration, initialBalance)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -131,6 +143,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.MultisigActorCodeID, nil
case actors.Version7:
return builtin7.MultisigActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)
@ -156,7 +171,7 @@ type State interface {
type Transaction = msig0.Transaction
var Methods = builtin6.MethodsMultisig
var Methods = builtin7.MethodsMultisig
func Message(version actors.Version, from address.Address) MessageBuilder {
switch version {
@ -178,6 +193,9 @@ func Message(version actors.Version, from address.Address) MessageBuilder {
case actors.Version6:
return message6{message0{from}}
case actors.Version7:
return message7{message0{from}}
default:
panic(fmt.Sprintf("unsupported actors version: %d", version))
}
@ -201,13 +219,13 @@ type MessageBuilder interface {
}
// this type is the same between v0 and v2
type ProposalHashData = msig6.ProposalHashData
type ProposeReturn = msig6.ProposeReturn
type ProposeParams = msig6.ProposeParams
type ApproveReturn = msig6.ApproveReturn
type ProposalHashData = msig7.ProposalHashData
type ProposeReturn = msig7.ProposeReturn
type ProposeParams = msig7.ProposeParams
type ApproveReturn = msig7.ApproveReturn
func txnParams(id uint64, data *ProposalHashData) ([]byte, error) {
params := msig6.TxnIDParams{ID: msig6.TxnID(id)}
params := msig7.TxnIDParams{ID: msig7.TxnID(id)}
if data != nil {
if data.Requester.Protocol() != address.ID {
return nil, xerrors.Errorf("proposer address must be an ID address, was %s", data.Requester)

119
chain/actors/builtin/multisig/v7.go Normal file
View File

@ -0,0 +1,119 @@
package multisig
import (
"bytes"
"encoding/binary"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors/adt"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
msig7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/multisig"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store, signers []address.Address, threshold uint64, startEpoch abi.ChainEpoch, unlockDuration abi.ChainEpoch, initialBalance abi.TokenAmount) (State, error) {
out := state7{store: store}
out.State = msig7.State{}
out.State.Signers = signers
out.State.NumApprovalsThreshold = threshold
out.State.StartEpoch = startEpoch
out.State.UnlockDuration = unlockDuration
out.State.InitialBalance = initialBalance
em, err := adt7.StoreEmptyMap(store, builtin7.DefaultHamtBitwidth)
if err != nil {
return nil, err
}
out.State.PendingTxns = em
return &out, nil
}
type state7 struct {
msig7.State
store adt.Store
}
func (s *state7) LockedBalance(currEpoch abi.ChainEpoch) (abi.TokenAmount, error) {
return s.State.AmountLocked(currEpoch - s.State.StartEpoch), nil
}
func (s *state7) StartEpoch() (abi.ChainEpoch, error) {
return s.State.StartEpoch, nil
}
func (s *state7) UnlockDuration() (abi.ChainEpoch, error) {
return s.State.UnlockDuration, nil
}
func (s *state7) InitialBalance() (abi.TokenAmount, error) {
return s.State.InitialBalance, nil
}
func (s *state7) Threshold() (uint64, error) {
return s.State.NumApprovalsThreshold, nil
}
func (s *state7) Signers() ([]address.Address, error) {
return s.State.Signers, nil
}
func (s *state7) ForEachPendingTxn(cb func(id int64, txn Transaction) error) error {
arr, err := adt7.AsMap(s.store, s.State.PendingTxns, builtin7.DefaultHamtBitwidth)
if err != nil {
return err
}
var out msig7.Transaction
return arr.ForEach(&out, func(key string) error {
txid, n := binary.Varint([]byte(key))
if n <= 0 {
return xerrors.Errorf("invalid pending transaction key: %v", key)
}
return cb(txid, (Transaction)(out)) //nolint:unconvert
})
}
func (s *state7) PendingTxnChanged(other State) (bool, error) {
other7, ok := other.(*state7)
if !ok {
// treat an upgrade as a change, always
return true, nil
}
return !s.State.PendingTxns.Equals(other7.PendingTxns), nil
}
func (s *state7) transactions() (adt.Map, error) {
return adt7.AsMap(s.store, s.PendingTxns, builtin7.DefaultHamtBitwidth)
}
func (s *state7) decodeTransaction(val *cbg.Deferred) (Transaction, error) {
var tx msig7.Transaction
if err := tx.UnmarshalCBOR(bytes.NewReader(val.Raw)); err != nil {
return Transaction{}, err
}
return tx, nil
}
func (s *state7) GetState() interface{} {
return &s.State
}

4
chain/actors/builtin/paych/message.go.template
View File

@ -39,7 +39,11 @@ func (m message{{.v}}) Create(to address.Address, initialAmount abi.TokenAmount)
func (m message{{.v}}) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych{{.v}}.UpdateChannelStateParams{
{{if (ge .v 7)}}
Sv: toV{{.v}}SignedVoucher(*sv),
{{else}}
Sv: *sv,
{{end}}
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message0.go
View File

@ -39,7 +39,9 @@ func (m message0) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message0) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych0.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message2.go
View File

@ -39,7 +39,9 @@ func (m message2) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message2) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych2.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message3.go
View File

@ -39,7 +39,9 @@ func (m message3) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message3) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych3.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message4.go
View File

@ -39,7 +39,9 @@ func (m message4) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message4) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych4.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message5.go
View File

@ -39,7 +39,9 @@ func (m message5) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message5) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych5.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

4
chain/actors/builtin/paych/message6.go
View File

@ -39,7 +39,9 @@ func (m message6) Create(to address.Address, initialAmount abi.TokenAmount) (*ty
func (m message6) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych6.UpdateChannelStateParams{
Sv: *sv,
Sv: *sv,
Secret: secret,
})
if aerr != nil {

76
chain/actors/builtin/paych/message7.go Normal file
View File

@ -0,0 +1,76 @@
package paych
import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
init7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/init"
paych7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/paych"
"github.com/filecoin-project/lotus/chain/actors"
init_ "github.com/filecoin-project/lotus/chain/actors/builtin/init"
"github.com/filecoin-project/lotus/chain/types"
)
type message7 struct{ from address.Address }
func (m message7) Create(to address.Address, initialAmount abi.TokenAmount) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych7.ConstructorParams{From: m.from, To: to})
if aerr != nil {
return nil, aerr
}
enc, aerr := actors.SerializeParams(&init7.ExecParams{
CodeCID: builtin7.PaymentChannelActorCodeID,
ConstructorParams: params,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: init_.Address,
From: m.from,
Value: initialAmount,
Method: builtin7.MethodsInit.Exec,
Params: enc,
}, nil
}
func (m message7) Update(paych address.Address, sv *SignedVoucher, secret []byte) (*types.Message, error) {
params, aerr := actors.SerializeParams(&paych7.UpdateChannelStateParams{
Sv: toV7SignedVoucher(*sv),
Secret: secret,
})
if aerr != nil {
return nil, aerr
}
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin7.MethodsPaych.UpdateChannelState,
Params: params,
}, nil
}
func (m message7) Settle(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin7.MethodsPaych.Settle,
}, nil
}
func (m message7) Collect(paych address.Address) (*types.Message, error) {
return &types.Message{
To: paych,
From: m.from,
Value: abi.NewTokenAmount(0),
Method: builtin7.MethodsPaych.Collect,
}, nil
}

20
chain/actors/builtin/paych/paych.go
View File

@ -27,6 +27,8 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
@ -58,6 +60,10 @@ func init() {
builtin.RegisterActorState(builtin6.PaymentChannelActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.PaymentChannelActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
// Load returns an abstract copy of payment channel state, irregardless of actor version
@ -82,6 +88,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.PaymentChannelActorCodeID:
return load6(store, act.Head)
case builtin7.PaymentChannelActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -107,6 +116,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -132,6 +144,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.PaymentChannelActorCodeID, nil
case actors.Version7:
return builtin7.PaymentChannelActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)
@ -185,7 +200,7 @@ func DecodeSignedVoucher(s string) (*SignedVoucher, error) {
return &sv, nil
}
var Methods = builtin6.MethodsPaych
var Methods = builtin7.MethodsPaych
func Message(version actors.Version, from address.Address) MessageBuilder {
switch version {
@ -208,6 +223,9 @@ func Message(version actors.Version, from address.Address) MessageBuilder {
case actors.Version6:
return message6{from}
case actors.Version7:
return message7{from}
default:
panic(fmt.Sprintf("unsupported actors version: %d", version))
}

18
chain/actors/builtin/paych/state.go.template
View File

@ -112,3 +112,21 @@ func (ls *laneState{{.v}}) Redeemed() (big.Int, error) {
func (ls *laneState{{.v}}) Nonce() (uint64, error) {
return ls.LaneState.Nonce, nil
}
{{if (ge .v 7)}}
func toV{{.v}}SignedVoucher(sv SignedVoucher) paych{{.v}}.SignedVoucher {
return paych{{.v}}.SignedVoucher{
ChannelAddr: sv.ChannelAddr,
TimeLockMin: sv.TimeLockMin,
TimeLockMax: sv.TimeLockMax,
SecretHash: sv.SecretPreimage,
Extra: sv.Extra,
Lane: sv.Lane,
Nonce: sv.Nonce,
Amount: sv.Amount,
MinSettleHeight: sv.MinSettleHeight,
Merges: sv.Merges,
Signature: sv.Signature,
}
}
{{end}}

130
chain/actors/builtin/paych/v7.go Normal file
View File

@ -0,0 +1,130 @@
package paych
import (
"github.com/ipfs/go-cid"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/chain/actors/adt"
paych7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/paych"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
out.State = paych7.State{}
return &out, nil
}
type state7 struct {
paych7.State
store adt.Store
lsAmt *adt7.Array
}
// Channel owner, who has funded the actor
func (s *state7) From() (address.Address, error) {
return s.State.From, nil
}
// Recipient of payouts from channel
func (s *state7) To() (address.Address, error) {
return s.State.To, nil
}
// Height at which the channel can be `Collected`
func (s *state7) SettlingAt() (abi.ChainEpoch, error) {
return s.State.SettlingAt, nil
}
// Amount successfully redeemed through the payment channel, paid out on `Collect()`
func (s *state7) ToSend() (abi.TokenAmount, error) {
return s.State.ToSend, nil
}
func (s *state7) getOrLoadLsAmt() (*adt7.Array, error) {
if s.lsAmt != nil {
return s.lsAmt, nil
}
// Get the lane state from the chain
lsamt, err := adt7.AsArray(s.store, s.State.LaneStates, paych7.LaneStatesAmtBitwidth)
if err != nil {
return nil, err
}
s.lsAmt = lsamt
return lsamt, nil
}
// Get total number of lanes
func (s *state7) LaneCount() (uint64, error) {
lsamt, err := s.getOrLoadLsAmt()
if err != nil {
return 0, err
}
return lsamt.Length(), nil
}
func (s *state7) GetState() interface{} {
return &s.State
}
// Iterate lane states
func (s *state7) ForEachLaneState(cb func(idx uint64, dl LaneState) error) error {
// Get the lane state from the chain
lsamt, err := s.getOrLoadLsAmt()
if err != nil {
return err
}
// Note: we use a map instead of an array to store laneStates because the
// client sets the lane ID (the index) and potentially they could use a
// very large index.
var ls paych7.LaneState
return lsamt.ForEach(&ls, func(i int64) error {
return cb(uint64(i), &laneState7{ls})
})
}
type laneState7 struct {
paych7.LaneState
}
func (ls *laneState7) Redeemed() (big.Int, error) {
return ls.LaneState.Redeemed, nil
}
func (ls *laneState7) Nonce() (uint64, error) {
return ls.LaneState.Nonce, nil
}
func toV7SignedVoucher(sv SignedVoucher) paych7.SignedVoucher {
return paych7.SignedVoucher{
ChannelAddr: sv.ChannelAddr,
TimeLockMin: sv.TimeLockMin,
TimeLockMax: sv.TimeLockMax,
SecretHash: sv.SecretPreimage,
Extra: sv.Extra,
Lane: sv.Lane,
Nonce: sv.Nonce,
Amount: sv.Amount,
MinSettleHeight: sv.MinSettleHeight,
Merges: sv.Merges,
Signature: sv.Signature,
}
}

19
chain/actors/builtin/power/power.go
View File

@ -26,6 +26,8 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
func init() {
@ -53,11 +55,15 @@ func init() {
builtin.RegisterActorState(builtin6.StoragePowerActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.StoragePowerActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var (
Address = builtin6.StoragePowerActorAddr
Methods = builtin6.MethodsPower
Address = builtin7.StoragePowerActorAddr
Methods = builtin7.MethodsPower
)
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -81,6 +87,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.StoragePowerActorCodeID:
return load6(store, act.Head)
case builtin7.StoragePowerActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -106,6 +115,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -131,6 +143,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.StoragePowerActorCodeID, nil
case actors.Version7:
return builtin7.StoragePowerActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

187
chain/actors/builtin/power/v7.go Normal file
View File

@ -0,0 +1,187 @@
package power
import (
"bytes"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
power7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/power"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
s, err := power7.ConstructState(store)
if err != nil {
return nil, err
}
out.State = *s
return &out, nil
}
type state7 struct {
power7.State
store adt.Store
}
func (s *state7) TotalLocked() (abi.TokenAmount, error) {
return s.TotalPledgeCollateral, nil
}
func (s *state7) TotalPower() (Claim, error) {
return Claim{
RawBytePower: s.TotalRawBytePower,
QualityAdjPower: s.TotalQualityAdjPower,
}, nil
}
// Committed power to the network. Includes miners below the minimum threshold.
func (s *state7) TotalCommitted() (Claim, error) {
return Claim{
RawBytePower: s.TotalBytesCommitted,
QualityAdjPower: s.TotalQABytesCommitted,
}, nil
}
func (s *state7) MinerPower(addr address.Address) (Claim, bool, error) {
claims, err := s.claims()
if err != nil {
return Claim{}, false, err
}
var claim power7.Claim
ok, err := claims.Get(abi.AddrKey(addr), &claim)
if err != nil {
return Claim{}, false, err
}
return Claim{
RawBytePower: claim.RawBytePower,
QualityAdjPower: claim.QualityAdjPower,
}, ok, nil
}
func (s *state7) MinerNominalPowerMeetsConsensusMinimum(a address.Address) (bool, error) {
return s.State.MinerNominalPowerMeetsConsensusMinimum(s.store, a)
}
func (s *state7) TotalPowerSmoothed() (builtin.FilterEstimate, error) {
return builtin.FromV7FilterEstimate(s.State.ThisEpochQAPowerSmoothed), nil
}
func (s *state7) MinerCounts() (uint64, uint64, error) {
return uint64(s.State.MinerAboveMinPowerCount), uint64(s.State.MinerCount), nil
}
func (s *state7) ListAllMiners() ([]address.Address, error) {
claims, err := s.claims()
if err != nil {
return nil, err
}
var miners []address.Address
err = claims.ForEach(nil, func(k string) error {
a, err := address.NewFromBytes([]byte(k))
if err != nil {
return err
}
miners = append(miners, a)
return nil
})
if err != nil {
return nil, err
}
return miners, nil
}
func (s *state7) ForEachClaim(cb func(miner address.Address, claim Claim) error) error {
claims, err := s.claims()
if err != nil {
return err
}
var claim power7.Claim
return claims.ForEach(&claim, func(k string) error {
a, err := address.NewFromBytes([]byte(k))
if err != nil {
return err
}
return cb(a, Claim{
RawBytePower: claim.RawBytePower,
QualityAdjPower: claim.QualityAdjPower,
})
})
}
func (s *state7) ClaimsChanged(other State) (bool, error) {
other7, ok := other.(*state7)
if !ok {
// treat an upgrade as a change, always
return true, nil
}
return !s.State.Claims.Equals(other7.State.Claims), nil
}
func (s *state7) SetTotalQualityAdjPower(p abi.StoragePower) error {
s.State.TotalQualityAdjPower = p
return nil
}
func (s *state7) SetTotalRawBytePower(p abi.StoragePower) error {
s.State.TotalRawBytePower = p
return nil
}
func (s *state7) SetThisEpochQualityAdjPower(p abi.StoragePower) error {
s.State.ThisEpochQualityAdjPower = p
return nil
}
func (s *state7) SetThisEpochRawBytePower(p abi.StoragePower) error {
s.State.ThisEpochRawBytePower = p
return nil
}
func (s *state7) GetState() interface{} {
return &s.State
}
func (s *state7) claims() (adt.Map, error) {
return adt7.AsMap(s.store, s.Claims, builtin7.DefaultHamtBitwidth)
}
func (s *state7) decodeClaim(val *cbg.Deferred) (Claim, error) {
var ci power7.Claim
if err := ci.UnmarshalCBOR(bytes.NewReader(val.Raw)); err != nil {
return Claim{}, err
}
return fromV7Claim(ci), nil
}
func fromV7Claim(v7 power7.Claim) Claim {
return Claim{
RawBytePower: v7.RawBytePower,
QualityAdjPower: v7.QualityAdjPower,
}
}

19
chain/actors/builtin/reward/reward.go
View File

@ -21,6 +21,8 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
"github.com/filecoin-project/lotus/chain/types"
@ -51,11 +53,15 @@ func init() {
builtin.RegisterActorState(builtin6.RewardActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.RewardActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var (
Address = builtin6.RewardActorAddr
Methods = builtin6.MethodsReward
Address = builtin7.RewardActorAddr
Methods = builtin7.MethodsReward
)
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -79,6 +85,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.RewardActorCodeID:
return load6(store, act.Head)
case builtin7.RewardActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -104,6 +113,9 @@ func MakeState(store adt.Store, av actors.Version, currRealizedPower abi.Storage
case actors.Version6:
return make6(store, currRealizedPower)
case actors.Version7:
return make7(store, currRealizedPower)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -129,6 +141,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.RewardActorCodeID, nil
case actors.Version7:
return builtin7.RewardActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

98
chain/actors/builtin/reward/v7.go Normal file
View File

@ -0,0 +1,98 @@
package reward
import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
reward7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/reward"
smoothing7 "github.com/filecoin-project/specs-actors/v7/actors/util/smoothing"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store, currRealizedPower abi.StoragePower) (State, error) {
out := state7{store: store}
out.State = *reward7.ConstructState(currRealizedPower)
return &out, nil
}
type state7 struct {
reward7.State
store adt.Store
}
func (s *state7) ThisEpochReward() (abi.TokenAmount, error) {
return s.State.ThisEpochReward, nil
}
func (s *state7) ThisEpochRewardSmoothed() (builtin.FilterEstimate, error) {
return builtin.FilterEstimate{
PositionEstimate: s.State.ThisEpochRewardSmoothed.PositionEstimate,
VelocityEstimate: s.State.ThisEpochRewardSmoothed.VelocityEstimate,
}, nil
}
func (s *state7) ThisEpochBaselinePower() (abi.StoragePower, error) {
return s.State.ThisEpochBaselinePower, nil
}
func (s *state7) TotalStoragePowerReward() (abi.TokenAmount, error) {
return s.State.TotalStoragePowerReward, nil
}
func (s *state7) EffectiveBaselinePower() (abi.StoragePower, error) {
return s.State.EffectiveBaselinePower, nil
}
func (s *state7) EffectiveNetworkTime() (abi.ChainEpoch, error) {
return s.State.EffectiveNetworkTime, nil
}
func (s *state7) CumsumBaseline() (reward7.Spacetime, error) {
return s.State.CumsumBaseline, nil
}
func (s *state7) CumsumRealized() (reward7.Spacetime, error) {
return s.State.CumsumRealized, nil
}
func (s *state7) InitialPledgeForPower(qaPower abi.StoragePower, networkTotalPledge abi.TokenAmount, networkQAPower *builtin.FilterEstimate, circSupply abi.TokenAmount) (abi.TokenAmount, error) {
return miner7.InitialPledgeForPower(
qaPower,
s.State.ThisEpochBaselinePower,
s.State.ThisEpochRewardSmoothed,
smoothing7.FilterEstimate{
PositionEstimate: networkQAPower.PositionEstimate,
VelocityEstimate: networkQAPower.VelocityEstimate,
},
circSupply,
), nil
}
func (s *state7) PreCommitDepositForPower(networkQAPower builtin.FilterEstimate, sectorWeight abi.StoragePower) (abi.TokenAmount, error) {
return miner7.PreCommitDepositForPower(s.State.ThisEpochRewardSmoothed,
smoothing7.FilterEstimate{
PositionEstimate: networkQAPower.PositionEstimate,
VelocityEstimate: networkQAPower.VelocityEstimate,
},
sectorWeight), nil
}
func (s *state7) GetState() interface{} {
return &s.State
}

10
chain/actors/builtin/system/system.go
View File

@ -17,10 +17,12 @@ import (
builtin5 "github.com/filecoin-project/specs-actors/v5/actors/builtin"
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
)
var (
Address = builtin6.SystemActorAddr
Address = builtin7.SystemActorAddr
)
func MakeState(store adt.Store, av actors.Version) (State, error) {
@ -44,6 +46,9 @@ func MakeState(store adt.Store, av actors.Version) (State, error) {
case actors.Version6:
return make6(store)
case actors.Version7:
return make7(store)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -69,6 +74,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.SystemActorCodeID, nil
case actors.Version7:
return builtin7.SystemActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

35
chain/actors/builtin/system/v7.go Normal file
View File

@ -0,0 +1,35 @@
package system
import (
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/actors/adt"
system7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/system"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store) (State, error) {
out := state7{store: store}
out.State = system7.State{}
return &out, nil
}
type state7 struct {
system7.State
store adt.Store
}
func (s *state7) GetState() interface{} {
return &s.State
}

75
chain/actors/builtin/verifreg/v7.go Normal file
View File

@ -0,0 +1,75 @@
package verifreg
import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
verifreg7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/verifreg"
adt7 "github.com/filecoin-project/specs-actors/v7/actors/util/adt"
)
var _ State = (*state7)(nil)
func load7(store adt.Store, root cid.Cid) (State, error) {
out := state7{store: store}
err := store.Get(store.Context(), root, &out)
if err != nil {
return nil, err
}
return &out, nil
}
func make7(store adt.Store, rootKeyAddress address.Address) (State, error) {
out := state7{store: store}
s, err := verifreg7.ConstructState(store, rootKeyAddress)
if err != nil {
return nil, err
}
out.State = *s
return &out, nil
}
type state7 struct {
verifreg7.State
store adt.Store
}
func (s *state7) RootKey() (address.Address, error) {
return s.State.RootKey, nil
}
func (s *state7) VerifiedClientDataCap(addr address.Address) (bool, abi.StoragePower, error) {
return getDataCap(s.store, actors.Version7, s.verifiedClients, addr)
}
func (s *state7) VerifierDataCap(addr address.Address) (bool, abi.StoragePower, error) {
return getDataCap(s.store, actors.Version7, s.verifiers, addr)
}
func (s *state7) ForEachVerifier(cb func(addr address.Address, dcap abi.StoragePower) error) error {
return forEachCap(s.store, actors.Version7, s.verifiers, cb)
}
func (s *state7) ForEachClient(cb func(addr address.Address, dcap abi.StoragePower) error) error {
return forEachCap(s.store, actors.Version7, s.verifiedClients, cb)
}
func (s *state7) verifiedClients() (adt.Map, error) {
return adt7.AsMap(s.store, s.VerifiedClients, builtin7.DefaultHamtBitwidth)
}
func (s *state7) verifiers() (adt.Map, error) {
return adt7.AsMap(s.store, s.Verifiers, builtin7.DefaultHamtBitwidth)
}
func (s *state7) GetState() interface{} {
return &s.State
}

19
chain/actors/builtin/verifreg/verifreg.go
View File

@ -21,6 +21,8 @@ import (
builtin6 "github.com/filecoin-project/specs-actors/v6/actors/builtin"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
"github.com/filecoin-project/lotus/chain/actors"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin"
@ -53,11 +55,15 @@ func init() {
return load6(store, root)
})
builtin.RegisterActorState(builtin7.VerifiedRegistryActorCodeID, func(store adt.Store, root cid.Cid) (cbor.Marshaler, error) {
return load7(store, root)
})
}
var (
Address = builtin6.VerifiedRegistryActorAddr
Methods = builtin6.MethodsVerifiedRegistry
Address = builtin7.VerifiedRegistryActorAddr
Methods = builtin7.MethodsVerifiedRegistry
)
func Load(store adt.Store, act *types.Actor) (State, error) {
@ -81,6 +87,9 @@ func Load(store adt.Store, act *types.Actor) (State, error) {
case builtin6.VerifiedRegistryActorCodeID:
return load6(store, act.Head)
case builtin7.VerifiedRegistryActorCodeID:
return load7(store, act.Head)
}
return nil, xerrors.Errorf("unknown actor code %s", act.Code)
}
@ -106,6 +115,9 @@ func MakeState(store adt.Store, av actors.Version, rootKeyAddress address.Addres
case actors.Version6:
return make6(store, rootKeyAddress)
case actors.Version7:
return make7(store, rootKeyAddress)
}
return nil, xerrors.Errorf("unknown actor version %d", av)
}
@ -131,6 +143,9 @@ func GetActorCodeID(av actors.Version) (cid.Cid, error) {
case actors.Version6:
return builtin6.VerifiedRegistryActorCodeID, nil
case actors.Version7:
return builtin7.VerifiedRegistryActorCodeID, nil
}
return cid.Undef, xerrors.Errorf("unknown actor version %d", av)

84
chain/actors/policy/policy.go
View File

@ -40,14 +40,19 @@ import (
miner6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/miner"
verifreg6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/verifreg"
paych6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/paych"
builtin7 "github.com/filecoin-project/specs-actors/v7/actors/builtin"
market7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/market"
miner7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/miner"
verifreg7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/verifreg"
paych7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/paych"
)
const (
ChainFinality = miner6.ChainFinality
ChainFinality = miner7.ChainFinality
SealRandomnessLookback = ChainFinality
PaychSettleDelay = paych6.SettleDelay
MaxPreCommitRandomnessLookback = builtin6.EpochsInDay + SealRandomnessLookback
PaychSettleDelay = paych7.SettleDelay
MaxPreCommitRandomnessLookback = builtin7.EpochsInDay + SealRandomnessLookback
)
// SetSupportedProofTypes sets supported proof types, across all actor versions.
@ -72,6 +77,8 @@ func SetSupportedProofTypes(types ...abi.RegisteredSealProof) {
miner6.PreCommitSealProofTypesV8 = make(map[abi.RegisteredSealProof]struct{}, len(types))
miner7.PreCommitSealProofTypesV8 = make(map[abi.RegisteredSealProof]struct{}, len(types))
AddSupportedProofTypes(types...)
}
@ -119,6 +126,15 @@ func AddSupportedProofTypes(types ...abi.RegisteredSealProof) {
miner6.WindowPoStProofTypes[wpp] = struct{}{}
miner7.PreCommitSealProofTypesV8[t+abi.RegisteredSealProof_StackedDrg2KiBV1_1] = struct{}{}
wpp, err = t.RegisteredWindowPoStProof()
if err != nil {
// Fine to panic, this is a test-only method
panic(err)
}
miner7.WindowPoStProofTypes[wpp] = struct{}{}
}
}
@ -139,11 +155,13 @@ func SetPreCommitChallengeDelay(delay abi.ChainEpoch) {
miner6.PreCommitChallengeDelay = delay
miner7.PreCommitChallengeDelay = delay
}
// TODO: this function shouldn't really exist. Instead, the API should expose the precommit delay.
func GetPreCommitChallengeDelay() abi.ChainEpoch {
return miner6.PreCommitChallengeDelay
return miner7.PreCommitChallengeDelay
}
// SetConsensusMinerMinPower sets the minimum power of an individual miner must
@ -173,6 +191,10 @@ func SetConsensusMinerMinPower(p abi.StoragePower) {
policy.ConsensusMinerMinPower = p
}
for _, policy := range builtin7.PoStProofPolicies {
policy.ConsensusMinerMinPower = p
}
}
// SetMinVerifiedDealSize sets the minimum size of a verified deal. This should
@ -191,6 +213,8 @@ func SetMinVerifiedDealSize(size abi.StoragePower) {
verifreg6.MinVerifiedDealSize = size
verifreg7.MinVerifiedDealSize = size
}
func GetMaxProveCommitDuration(ver actors.Version, t abi.RegisteredSealProof) (abi.ChainEpoch, error) {
@ -220,6 +244,10 @@ func GetMaxProveCommitDuration(ver actors.Version, t abi.RegisteredSealProof) (a
return miner6.MaxProveCommitDuration[t], nil
case actors.Version7:
return miner7.MaxProveCommitDuration[t], nil
default:
return 0, xerrors.Errorf("unsupported actors version")
}
@ -255,6 +283,11 @@ func SetProviderCollateralSupplyTarget(num, denom big.Int) {
Denominator: denom,
}
market7.ProviderCollateralSupplyTarget = builtin7.BigFrac{
Numerator: num,
Denominator: denom,
}
}
func DealProviderCollateralBounds(
@ -298,13 +331,18 @@ func DealProviderCollateralBounds(
min, max := market6.DealProviderCollateralBounds(size, verified, rawBytePower, qaPower, baselinePower, circulatingFil)
return min, max, nil
case actors.Version7:
min, max := market7.DealProviderCollateralBounds(size, verified, rawBytePower, qaPower, baselinePower, circulatingFil)
return min, max, nil
default:
return big.Zero(), big.Zero(), xerrors.Errorf("unsupported actors version")
}
}
func DealDurationBounds(pieceSize abi.PaddedPieceSize) (min, max abi.ChainEpoch) {
return market6.DealDurationBounds(pieceSize)
return market7.DealDurationBounds(pieceSize)
}
// Sets the challenge window and scales the proving period to match (such that
@ -345,6 +383,13 @@ func SetWPoStChallengeWindow(period abi.ChainEpoch) {
// scale it if we're scaling the challenge period.
miner6.WPoStDisputeWindow = period * 30
miner7.WPoStChallengeWindow = period
miner7.WPoStProvingPeriod = period * abi.ChainEpoch(miner7.WPoStPeriodDeadlines)
// by default, this is 2x finality which is 30 periods.
// scale it if we're scaling the challenge period.
miner7.WPoStDisputeWindow = period * 30
}
func GetWinningPoStSectorSetLookback(nwVer network.Version) abi.ChainEpoch {
@ -357,15 +402,15 @@ func GetWinningPoStSectorSetLookback(nwVer network.Version) abi.ChainEpoch {
}
func GetMaxSectorExpirationExtension() abi.ChainEpoch {
return miner6.MaxSectorExpirationExtension
return miner7.MaxSectorExpirationExtension
}
func GetMinSectorExpiration() abi.ChainEpoch {
return miner6.MinSectorExpiration
return miner7.MinSectorExpiration
}
func GetMaxPoStPartitions(nv network.Version, p abi.RegisteredPoStProof) (int, error) {
sectorsPerPart, err := builtin6.PoStProofWindowPoStPartitionSectors(p)
sectorsPerPart, err := builtin7.PoStProofWindowPoStPartitionSectors(p)
if err != nil {
return 0, err
}
@ -378,8 +423,8 @@ func GetMaxPoStPartitions(nv network.Version, p abi.RegisteredPoStProof) (int, e
func GetDefaultSectorSize() abi.SectorSize {
// supported sector sizes are the same across versions.
szs := make([]abi.SectorSize, 0, len(miner6.PreCommitSealProofTypesV8))
for spt := range miner6.PreCommitSealProofTypesV8 {
szs := make([]abi.SectorSize, 0, len(miner7.PreCommitSealProofTypesV8))
for spt := range miner7.PreCommitSealProofTypesV8 {
ss, err := spt.SectorSize()
if err != nil {
panic(err)
@ -404,7 +449,7 @@ func GetSectorMaxLifetime(proof abi.RegisteredSealProof, nwVer network.Version)
return builtin4.SealProofPoliciesV0[proof].SectorMaxLifetime
}
return builtin6.SealProofPoliciesV11[proof].SectorMaxLifetime
return builtin7.SealProofPoliciesV11[proof].SectorMaxLifetime
}
func GetAddressedSectorsMax(nwVer network.Version) (int, error) {
@ -432,6 +477,9 @@ func GetAddressedSectorsMax(nwVer network.Version) (int, error) {
case actors.Version6:
return miner6.AddressedSectorsMax, nil
case actors.Version7:
return miner7.AddressedSectorsMax, nil
default:
return 0, xerrors.Errorf("unsupported network version")
}
@ -469,6 +517,10 @@ func GetDeclarationsMax(nwVer network.Version) (int, error) {
return miner6.DeclarationsMax, nil
case actors.Version7:
return miner7.DeclarationsMax, nil
default:
return 0, xerrors.Errorf("unsupported network version")
}
@ -505,6 +557,10 @@ func AggregateProveCommitNetworkFee(nwVer network.Version, aggregateSize int, ba
return miner6.AggregateProveCommitNetworkFee(aggregateSize, baseFee), nil
case actors.Version7:
return miner7.AggregateProveCommitNetworkFee(aggregateSize, baseFee), nil
default:
return big.Zero(), xerrors.Errorf("unsupported network version")
}
@ -541,6 +597,10 @@ func AggregatePreCommitNetworkFee(nwVer network.Version, aggregateSize int, base
return miner6.AggregatePreCommitNetworkFee(aggregateSize, baseFee), nil
case actors.Version7:
return miner7.AggregatePreCommitNetworkFee(aggregateSize, baseFee), nil
default:
return big.Zero(), xerrors.Errorf("unsupported network version")
}

7
chain/actors/version.go
View File

@ -20,9 +20,9 @@ const ({{range .actorVersions}}
/* inline-gen start */
var LatestVersion = 6
var LatestVersion = 7
var Versions = []int{0, 2, 3, 4, 5, 6}
var Versions = []int{0, 2, 3, 4, 5, 6, 7}
const (
Version0 Version = 0
@ -31,6 +31,7 @@ const (
Version4 Version = 4
Version5 Version = 5
Version6 Version = 6
Version7 Version = 7
)
/* inline-gen end */
@ -50,6 +51,8 @@ func VersionForNetwork(version network.Version) (Version, error) {
return Version5, nil
case network.Version14:
return Version6, nil
case network.Version15:
return Version7, nil
default:
return -1, fmt.Errorf("unsupported network version %d", version)
}

8
chain/checkpoint.go
View File

@ -13,7 +13,7 @@ func (syncer *Syncer) SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) e
return xerrors.Errorf("called with empty tsk")
}
ts, err := syncer.ChainStore().LoadTipSet(tsk)
ts, err := syncer.ChainStore().LoadTipSet(ctx, tsk)
if err != nil {
tss, err := syncer.Exchange.GetBlocks(ctx, tsk, 1)
if err != nil {
@ -28,7 +28,7 @@ func (syncer *Syncer) SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) e
return xerrors.Errorf("failed to switch chain when syncing checkpoint: %w", err)
}
if err := syncer.ChainStore().SetCheckpoint(ts); err != nil {
if err := syncer.ChainStore().SetCheckpoint(ctx, ts); err != nil {
return xerrors.Errorf("failed to set the chain checkpoint: %w", err)
}
@ -41,7 +41,7 @@ func (syncer *Syncer) switchChain(ctx context.Context, ts *types.TipSet) error {
return nil
}
if anc, err := syncer.store.IsAncestorOf(ts, hts); err == nil && anc {
if anc, err := syncer.store.IsAncestorOf(ctx, ts, hts); err == nil && anc {
return nil
}
@ -50,7 +50,7 @@ func (syncer *Syncer) switchChain(ctx context.Context, ts *types.TipSet) error {
return xerrors.Errorf("failed to collect chain for checkpoint: %w", err)
}
if err := syncer.ChainStore().SetHead(ts); err != nil {
if err := syncer.ChainStore().SetHead(ctx, ts); err != nil {
return xerrors.Errorf("failed to set the chain head: %w", err)
}
return nil

63
chain/consensus/filcns/compute_state.go
View File

@ -28,6 +28,7 @@ import (
exported4 "github.com/filecoin-project/specs-actors/v4/actors/builtin/exported"
exported5 "github.com/filecoin-project/specs-actors/v5/actors/builtin/exported"
exported6 "github.com/filecoin-project/specs-actors/v6/actors/builtin/exported"
exported7 "github.com/filecoin-project/specs-actors/v7/actors/builtin/exported"
/* inline-gen end */
@ -59,6 +60,7 @@ func NewActorRegistry() *vm.ActorRegistry {
inv.Register(vm.ActorsVersionPredicate(actors.Version4), exported4.BuiltinActors()...)
inv.Register(vm.ActorsVersionPredicate(actors.Version5), exported5.BuiltinActors()...)
inv.Register(vm.ActorsVersionPredicate(actors.Version6), exported6.BuiltinActors()...)
inv.Register(vm.ActorsVersionPredicate(actors.Version7), exported7.BuiltinActors()...)
/* inline-gen end */
@ -90,16 +92,16 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context, sm *stmgr.StateManager
partDone()
}()
makeVmWithBaseState := func(base cid.Cid) (*vm.VM, error) {
makeVmWithBaseStateAndEpoch := func(base cid.Cid, e abi.ChainEpoch) (*vm.VM, error) {
vmopt := &vm.VMOpts{
StateBase: base,
Epoch: epoch,
Epoch: e,
Rand: r,
Bstore: sm.ChainStore().StateBlockstore(),
Actors: NewActorRegistry(),
Syscalls: sm.Syscalls,
CircSupplyCalc: sm.GetVMCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
NetworkVersion: sm.GetNetworkVersion(ctx, e),
BaseFee: baseFee,
LookbackState: stmgr.LookbackStateGetterForTipset(sm, ts),
}
@ -107,12 +109,7 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context, sm *stmgr.StateManager
return sm.VMConstructor()(ctx, vmopt)
}
vmi, err := makeVmWithBaseState(pstate)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("making vm: %w", err)
}
runCron := func(epoch abi.ChainEpoch) error {
runCron := func(vmCron *vm.VM, epoch abi.ChainEpoch) error {
cronMsg := &types.Message{
To: cron.Address,
From: builtin.SystemActorAddr,
@ -124,56 +121,58 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context, sm *stmgr.StateManager
Method: cron.Methods.EpochTick,
Params: nil,
}
ret, err := vmi.ApplyImplicitMessage(ctx, cronMsg)
ret, err := vmCron.ApplyImplicitMessage(ctx, cronMsg)
if err != nil {
return err
return xerrors.Errorf("running cron: %w", err)
}
if em != nil {
if err := em.MessageApplied(ctx, ts, cronMsg.Cid(), cronMsg, ret, true); err != nil {
return xerrors.Errorf("callback failed on cron message: %w", err)
}
}
if ret.ExitCode != 0 {
return xerrors.Errorf("CheckProofSubmissions exit was non-zero: %d", ret.ExitCode)
return xerrors.Errorf("cron exit was non-zero: %d", ret.ExitCode)
}
return nil
}
for i := parentEpoch; i < epoch; i++ {
var err error
if i > parentEpoch {
// run cron for null rounds if any
if err := runCron(i); err != nil {
return cid.Undef, cid.Undef, err
vmCron, err := makeVmWithBaseStateAndEpoch(pstate, i)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("making cron vm: %w", err)
}
pstate, err = vmi.Flush(ctx)
// run cron for null rounds if any
if err = runCron(vmCron, i); err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("running cron: %w", err)
}
pstate, err = vmCron.Flush(ctx)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("flushing vm: %w", err)
return cid.Undef, cid.Undef, xerrors.Errorf("flushing cron vm: %w", err)
}
}
// handle state forks
// XXX: The state tree
newState, err := sm.HandleStateForks(ctx, pstate, i, em, ts)
pstate, err = sm.HandleStateForks(ctx, pstate, i, em, ts)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("error handling state forks: %w", err)
}
if pstate != newState {
vmi, err = makeVmWithBaseState(newState)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("making vm: %w", err)
}
}
vmi.SetBlockHeight(i + 1)
pstate = newState
}
partDone()
partDone = metrics.Timer(ctx, metrics.VMApplyMessages)
vmi, err := makeVmWithBaseStateAndEpoch(pstate, epoch)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("making vm: %w", err)
}
var receipts []cbg.CBORMarshaler
processedMsgs := make(map[cid.Cid]struct{})
for _, b := range bms {
@ -241,7 +240,7 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context, sm *stmgr.StateManager
partDone()
partDone = metrics.Timer(ctx, metrics.VMApplyCron)
if err := runCron(epoch); err != nil {
if err := runCron(vmi, epoch); err != nil {
return cid.Cid{}, cid.Cid{}, err
}
@ -289,7 +288,7 @@ func (t *TipSetExecutor) ExecuteTipSet(ctx context.Context, sm *stmgr.StateManag
var parentEpoch abi.ChainEpoch
pstate := blks[0].ParentStateRoot
if blks[0].Height > 0 {
parent, err := sm.ChainStore().GetBlock(blks[0].Parents[0])
parent, err := sm.ChainStore().GetBlock(ctx, blks[0].Parents[0])
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("getting parent block: %w", err)
}
@ -297,9 +296,9 @@ func (t *TipSetExecutor) ExecuteTipSet(ctx context.Context, sm *stmgr.StateManag
parentEpoch = parent.Height
}
r := rand.NewStateRand(sm.ChainStore(), ts.Cids(), sm.Beacon())
r := rand.NewStateRand(sm.ChainStore(), ts.Cids(), sm.Beacon(), sm.GetNetworkVersion)
blkmsgs, err := sm.ChainStore().BlockMsgsForTipset(ts)
blkmsgs, err := sm.ChainStore().BlockMsgsForTipset(ctx, ts)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("getting block messages for tipset: %w", err)
}

53
chain/consensus/filcns/filecoin.go
View File

@ -26,7 +26,7 @@ import (
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/go-state-types/network"
blockadt "github.com/filecoin-project/specs-actors/actors/util/adt"
proof2 "github.com/filecoin-project/specs-actors/v2/actors/runtime/proof"
"github.com/filecoin-project/specs-actors/v7/actors/runtime/proof"
bstore "github.com/filecoin-project/lotus/blockstore"
"github.com/filecoin-project/lotus/build"
@ -90,19 +90,19 @@ func (filec *FilecoinEC) ValidateBlock(ctx context.Context, b *types.FullBlock)
h := b.Header
baseTs, err := filec.store.LoadTipSet(types.NewTipSetKey(h.Parents...))
baseTs, err := filec.store.LoadTipSet(ctx, types.NewTipSetKey(h.Parents...))
if err != nil {
return xerrors.Errorf("load parent tipset failed (%s): %w", h.Parents, err)
}
winPoStNv := filec.sm.GetNtwkVersion(ctx, baseTs.Height())
winPoStNv := filec.sm.GetNetworkVersion(ctx, baseTs.Height())
lbts, lbst, err := stmgr.GetLookbackTipSetForRound(ctx, filec.sm, baseTs, h.Height)
if err != nil {
return xerrors.Errorf("failed to get lookback tipset for block: %w", err)
}
prevBeacon, err := filec.store.GetLatestBeaconEntry(baseTs)
prevBeacon, err := filec.store.GetLatestBeaconEntry(ctx, baseTs)
if err != nil {
return xerrors.Errorf("failed to get latest beacon entry: %w", err)
}
@ -171,7 +171,7 @@ func (filec *FilecoinEC) ValidateBlock(ctx context.Context, b *types.FullBlock)
}
if stateroot != h.ParentStateRoot {
msgs, err := filec.store.MessagesForTipset(baseTs)
msgs, err := filec.store.MessagesForTipset(ctx, baseTs)
if err != nil {
log.Error("failed to load messages for tipset during tipset state mismatch error: ", err)
} else {
@ -400,12 +400,21 @@ func (filec *FilecoinEC) VerifyWinningPoStProof(ctx context.Context, nv network.
return xerrors.Errorf("failed to get ID from miner address %s: %w", h.Miner, err)
}
sectors, err := stmgr.GetSectorsForWinningPoSt(ctx, nv, filec.verifier, filec.sm, lbst, h.Miner, rand)
xsectors, err := stmgr.GetSectorsForWinningPoSt(ctx, nv, filec.verifier, filec.sm, lbst, h.Miner, rand)
if err != nil {
return xerrors.Errorf("getting winning post sector set: %w", err)
}
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, proof2.WinningPoStVerifyInfo{
sectors := make([]proof.SectorInfo, len(xsectors))
for i, xsi := range xsectors {
sectors[i] = proof.SectorInfo{
SealProof: xsi.SealProof,
SectorNumber: xsi.SectorNumber,
SealedCID: xsi.SealedCID,
}
}
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, proof.WinningPoStVerifyInfo{
Randomness: rand,
Proofs: h.WinPoStProof,
ChallengedSectors: sectors,
@ -449,7 +458,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
stateroot, _, err := filec.sm.TipSetState(ctx, baseTs)
if err != nil {
return err
return xerrors.Errorf("failed to compute tipsettate for %s: %w", baseTs.Key(), err)
}
st, err := state.LoadStateTree(filec.store.ActorStore(ctx), stateroot)
@ -457,7 +466,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
return xerrors.Errorf("failed to load base state tree: %w", err)
}
nv := filec.sm.GetNtwkVersion(ctx, b.Header.Height)
nv := filec.sm.GetNetworkVersion(ctx, b.Header.Height)
pl := vm.PricelistByEpoch(baseTs.Height())
var sumGasLimit int64
checkMsg := func(msg types.ChainMsg) error {
@ -466,7 +475,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
// Phase 1: syntactic validation, as defined in the spec
minGas := pl.OnChainMessage(msg.ChainLength())
if err := m.ValidForBlockInclusion(minGas.Total(), nv); err != nil {
return err
return xerrors.Errorf("msg %s invalid for block inclusion: %w", m.Cid(), err)
}
// ValidForBlockInclusion checks if any single message does not exceed BlockGasLimit
@ -479,10 +488,10 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
// Phase 2: (Partial) semantic validation:
// the sender exists and is an account actor, and the nonces make sense
var sender address.Address
if filec.sm.GetNtwkVersion(ctx, b.Header.Height) >= network.Version13 {
if filec.sm.GetNetworkVersion(ctx, b.Header.Height) >= network.Version13 {
sender, err = st.LookupID(m.From)
if err != nil {
return err
return xerrors.Errorf("failed to lookup sender %s: %w", m.From, err)
}
} else {
sender = m.From
@ -519,7 +528,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
return xerrors.Errorf("block had invalid bls message at index %d: %w", i, err)
}
c, err := store.PutMessage(tmpbs, m)
c, err := store.PutMessage(ctx, tmpbs, m)
if err != nil {
return xerrors.Errorf("failed to store message %s: %w", m.Cid(), err)
}
@ -532,7 +541,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
smArr := blockadt.MakeEmptyArray(tmpstore)
for i, m := range b.SecpkMessages {
if filec.sm.GetNtwkVersion(ctx, b.Header.Height) >= network.Version14 {
if filec.sm.GetNetworkVersion(ctx, b.Header.Height) >= network.Version14 {
if m.Signature.Type != crypto.SigTypeSecp256k1 {
return xerrors.Errorf("block had invalid secpk message at index %d: %w", i, err)
}
@ -553,7 +562,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
return xerrors.Errorf("secpk message %s has invalid signature: %w", m.Cid(), err)
}
c, err := store.PutMessage(tmpbs, m)
c, err := store.PutMessage(ctx, tmpbs, m)
if err != nil {
return xerrors.Errorf("failed to store message %s: %w", m.Cid(), err)
}
@ -565,12 +574,13 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
bmroot, err := bmArr.Root()
if err != nil {
return err
return xerrors.Errorf("failed to root bls msgs: %w", err)
}
smroot, err := smArr.Root()
if err != nil {
return err
return xerrors.Errorf("failed to root secp msgs: %w", err)
}
mrcid, err := tmpstore.Put(ctx, &types.MsgMeta{
@ -578,7 +588,7 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
SecpkMessages: smroot,
})
if err != nil {
return err
return xerrors.Errorf("failed to put msg meta: %w", err)
}
if b.Header.Messages != mrcid {
@ -586,7 +596,12 @@ func (filec *FilecoinEC) checkBlockMessages(ctx context.Context, b *types.FullBl
}
// Finally, flush.
return vm.Copy(ctx, tmpbs, filec.store.ChainBlockstore(), mrcid)
err = vm.Copy(ctx, tmpbs, filec.store.ChainBlockstore(), mrcid)
if err != nil {
return xerrors.Errorf("failed to flush:%w", err)
}
return nil
}
func (filec *FilecoinEC) IsEpochBeyondCurrMax(epoch abi.ChainEpoch) bool {

10
chain/consensus/filcns/mine.go
View File

@ -15,7 +15,7 @@ import (
)
func (filec *FilecoinEC) CreateBlock(ctx context.Context, w api.Wallet, bt *api.BlockTemplate) (*types.FullBlock, error) {
pts, err := filec.sm.ChainStore().LoadTipSet(bt.Parents)
pts, err := filec.sm.ChainStore().LoadTipSet(ctx, bt.Parents)
if err != nil {
return nil, xerrors.Errorf("failed to load parent tipset: %w", err)
}
@ -59,14 +59,14 @@ func (filec *FilecoinEC) CreateBlock(ctx context.Context, w api.Wallet, bt *api.
blsSigs = append(blsSigs, msg.Signature)
blsMessages = append(blsMessages, &msg.Message)
c, err := filec.sm.ChainStore().PutMessage(&msg.Message)
c, err := filec.sm.ChainStore().PutMessage(ctx, &msg.Message)
if err != nil {
return nil, err
}
blsMsgCids = append(blsMsgCids, c)
} else {
c, err := filec.sm.ChainStore().PutMessage(msg)
} else if msg.Signature.Type == crypto.SigTypeSecp256k1 {
c, err := filec.sm.ChainStore().PutMessage(ctx, msg)
if err != nil {
return nil, err
}
@ -74,6 +74,8 @@ func (filec *FilecoinEC) CreateBlock(ctx context.Context, w api.Wallet, bt *api.
secpkMsgCids = append(secpkMsgCids, c)
secpkMessages = append(secpkMessages, msg)
} else {
return nil, xerrors.Errorf("unknown sig type: %d", msg.Signature.Type)
}
}

116
chain/consensus/filcns/upgrades.go
View File

@ -5,7 +5,10 @@ import (
"runtime"
"time"
"github.com/docker/go-units"
"github.com/filecoin-project/specs-actors/v6/actors/migration/nv14"
"github.com/filecoin-project/specs-actors/v7/actors/migration/nv15"
"github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
@ -156,6 +159,22 @@ func DefaultUpgradeSchedule() stmgr.UpgradeSchedule {
StopWithin: 5,
}},
Expensive: true,
}, {
Height: build.UpgradeOhSnapHeight,
Network: network.Version15,
Migration: UpgradeActorsV7,
PreMigrations: []stmgr.PreMigration{{
PreMigration: PreUpgradeActorsV7,
StartWithin: 120,
DontStartWithin: 60,
StopWithin: 35,
}, {
PreMigration: PreUpgradeActorsV7,
StartWithin: 30,
DontStartWithin: 15,
StopWithin: 5,
}},
Expensive: true,
},
}
@ -625,7 +644,7 @@ func splitGenesisMultisig0(ctx context.Context, em stmgr.ExecMonitor, addr addre
// TODO: After the Liftoff epoch, refactor this to use resetMultisigVesting
func resetGenesisMsigs0(ctx context.Context, sm *stmgr.StateManager, store adt0.Store, tree *state.StateTree, startEpoch abi.ChainEpoch) error {
gb, err := sm.ChainStore().GetGenesis()
gb, err := sm.ChainStore().GetGenesis(ctx)
if err != nil {
return xerrors.Errorf("getting genesis block: %w", err)
}
@ -1170,7 +1189,7 @@ func upgradeActorsV6Common(
// Perform the migration
newHamtRoot, err := nv14.MigrateStateTree(ctx, store, stateRoot.Actors, epoch, config, migrationLogger{}, cache)
if err != nil {
return cid.Undef, xerrors.Errorf("upgrading to actors v5: %w", err)
return cid.Undef, xerrors.Errorf("upgrading to actors v6: %w", err)
}
// Persist the result.
@ -1197,6 +1216,99 @@ func upgradeActorsV6Common(
return newRoot, nil
}
func UpgradeActorsV7(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, cb stmgr.ExecMonitor, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
// Use all the CPUs except 3.
workerCount := runtime.NumCPU() - 3
if workerCount <= 0 {
workerCount = 1
}
config := nv15.Config{
MaxWorkers: uint(workerCount),
JobQueueSize: 1000,
ResultQueueSize: 100,
ProgressLogPeriod: 10 * time.Second,
}
newRoot, err := upgradeActorsV7Common(ctx, sm, cache, root, epoch, ts, config)
if err != nil {
return cid.Undef, xerrors.Errorf("migrating actors v6 state: %w", err)
}
return newRoot, nil
}
func PreUpgradeActorsV7(ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache, root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet) error {
// Use half the CPUs for pre-migration, but leave at least 3.
workerCount := runtime.NumCPU()
if workerCount <= 4 {
workerCount = 1
} else {
workerCount /= 2
}
lbts, lbRoot, err := stmgr.GetLookbackTipSetForRound(ctx, sm, ts, epoch)
if err != nil {
return xerrors.Errorf("error getting lookback ts for premigration: %w", err)
}
config := nv15.Config{MaxWorkers: uint(workerCount),
ProgressLogPeriod: time.Minute * 5}
_, err = upgradeActorsV7Common(ctx, sm, cache, lbRoot, epoch, lbts, config)
return err
}
func upgradeActorsV7Common(
ctx context.Context, sm *stmgr.StateManager, cache stmgr.MigrationCache,
root cid.Cid, epoch abi.ChainEpoch, ts *types.TipSet,
config nv15.Config,
) (cid.Cid, error) {
writeStore := blockstore.NewAutobatch(ctx, sm.ChainStore().StateBlockstore(), units.GiB)
// TODO: pretty sure we'd achieve nothing by doing this, confirm in review
//buf := blockstore.NewTieredBstore(sm.ChainStore().StateBlockstore(), writeStore)
store := store.ActorStore(ctx, writeStore)
// Load the state root.
var stateRoot types.StateRoot
if err := store.Get(ctx, root, &stateRoot); err != nil {
return cid.Undef, xerrors.Errorf("failed to decode state root: %w", err)
}
if stateRoot.Version != types.StateTreeVersion4 {
return cid.Undef, xerrors.Errorf(
"expected state root version 4 for actors v7 upgrade, got %d",
stateRoot.Version,
)
}
// Perform the migration
newHamtRoot, err := nv15.MigrateStateTree(ctx, store, stateRoot.Actors, epoch, config, migrationLogger{}, cache)
if err != nil {
return cid.Undef, xerrors.Errorf("upgrading to actors v7: %w", err)
}
// Persist the result.
newRoot, err := store.Put(ctx, &types.StateRoot{
Version: types.StateTreeVersion4,
Actors: newHamtRoot,
Info: stateRoot.Info,
})
if err != nil {
return cid.Undef, xerrors.Errorf("failed to persist new state root: %w", err)
}
// Persists the new tree and shuts down the flush worker
if err := writeStore.Flush(ctx); err != nil {
return cid.Undef, xerrors.Errorf("writeStore flush failed: %w", err)
}
if err := writeStore.Shutdown(ctx); err != nil {
return cid.Undef, xerrors.Errorf("writeStore shutdown failed: %w", err)
}
return newRoot, nil
}
type migrationLogger struct{}
func (ml migrationLogger) Log(level rt.LogLevel, msg string, args ...interface{}) {

2
chain/events/events_test.go
View File

@ -87,7 +87,7 @@ func (fcs *fakeCS) ChainGetPath(ctx context.Context, from, to types.TipSetKey) (
}
// copied from the chainstore
revert, apply, err := store.ReorgOps(func(tsk types.TipSetKey) (*types.TipSet, error) {
revert, apply, err := store.ReorgOps(ctx, func(ctx context.Context, tsk types.TipSetKey) (*types.TipSet, error) {
return fcs.ChainGetTipSet(ctx, tsk)
}, fromTs, toTs)
if err != nil {

Some files were not shown because too many files have changed in this diff Show More